Skip to content

Conversation

@wgtmac
Copy link
Member

@wgtmac wgtmac commented Dec 18, 2025

Rationale for this change

ColumnSizeCommand is a great tool to investigate column size. It would be good if we can sort it by column size to help spot the dominating columns.

What changes are included in this PR?

  • Support sorting by column sizes in the descending order.
  • Support printing ratio as percentage.

Are these changes tested?

Before this change

~/Projects/parquet-testing/data (master*) » parquet-cli column-size alltypes_plain.snappy.parquet

float_col-> Size In Bytes: 51 Size In Ratio: 0.08947369
int_col-> Size In Bytes: 51 Size In Ratio: 0.08947369
string_col-> Size In Bytes: 53 Size In Ratio: 0.092982456
bool_col-> Size In Bytes: 26 Size In Ratio: 0.045614034
date_string_col-> Size In Bytes: 55 Size In Ratio: 0.096491225
smallint_col-> Size In Bytes: 51 Size In Ratio: 0.08947369
timestamp_col-> Size In Bytes: 63 Size In Ratio: 0.110526316
tinyint_col-> Size In Bytes: 51 Size In Ratio: 0.08947369
bigint_col-> Size In Bytes: 59 Size In Ratio: 0.10350877
id-> Size In Bytes: 51 Size In Ratio: 0.08947369
double_col-> Size In Bytes: 59 Size In Ratio: 0.10350877

After this change

~/Projects/parquet-testing/data (master*) » parquet-cli column-size alltypes_plain.snappy.parquet -s -p

timestamp_col-> Size In Bytes: 63 Size In Ratio: 11.0526%
bigint_col-> Size In Bytes: 59 Size In Ratio: 10.3509%
double_col-> Size In Bytes: 59 Size In Ratio: 10.3509%
date_string_col-> Size In Bytes: 55 Size In Ratio: 9.6491%
string_col-> Size In Bytes: 53 Size In Ratio: 9.2982%
float_col-> Size In Bytes: 51 Size In Ratio: 8.9474%
int_col-> Size In Bytes: 51 Size In Ratio: 8.9474%
smallint_col-> Size In Bytes: 51 Size In Ratio: 8.9474%
tinyint_col-> Size In Bytes: 51 Size In Ratio: 8.9474%
id-> Size In Bytes: 51 Size In Ratio: 8.9474%
bool_col-> Size In Bytes: 26 Size In Ratio: 4.5614%

Are there any user-facing changes?

Yes. Two flags are added but are by default off.

Closes #3372

- Support sorting by column sizes in the descending order.
- Support printing ratio as percentage.
@wgtmac wgtmac requested a review from Fokko December 18, 2025 15:46
@wgtmac wgtmac changed the title Enhance ColumnSizeCommand GH-3372: Enhance ColumnSizeCommand to sort by size Dec 19, 2025
@wgtmac
Copy link
Member Author

wgtmac commented Jan 20, 2026

@gszadovszky @Fokko Do you think this is useful for the community? At least this is useful for me to figure out dominating column in an easy way.

Copy link
Contributor

@gszadovszky gszadovszky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wgtmac, any change that improves usability of parquet-cli are very welcomed.

@wgtmac wgtmac merged commit 7190ab6 into apache:master Jan 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve ColumnSizeCommand to sort by column size

2 participants