Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-44101: [C++][Parquet] Tools: Debug Print for Json should be valid JSON #44532

Merged
merged 4 commits into from
Oct 31, 2024

Conversation

mapleFU
Copy link
Member

@mapleFU mapleFU commented Oct 25, 2024

Rationale for this change

The printJson is not a valid json now. This is ok for human-read, but when I want to analysis it with json tools or ai, it will prevent from using it.

What changes are included in this PR?

Change the output to be a valid json.

Style:

previously, the \" trailing would be added in start of object, but this patch put it to end of object

Before:

stream << "\", \"number\":\""  << number;
stream << "\"...";

After:

stream << ", \"number\":\""  << number << "\"";

Are these changes tested?

Yes

Are there any user-facing changes?

Minor format change

@mapleFU mapleFU requested a review from wgtmac as a code owner October 25, 2024 05:52
@mapleFU
Copy link
Member Author

mapleFU commented Oct 25, 2024

@pitrou @wgtmac this is a minor change for tool with testing. Would you mind take a look?

@@ -1172,6 +1180,30 @@ TEST_F(TestJSONWithLocalFile, JSONOutputFLBA) {
EXPECT_THAT(json_content, testing::HasSubstr(json_contains));
}

// GH-44101: Test that JSON output is valid JSON
TEST_F(TestJSONWithLocalFile, ValidJsonOutput) {
auto checkJsonValid = [](std::string_view json_string) -> ::arrow::Status {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: check_valid_json?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Oct 25, 2024
@mapleFU
Copy link
Member Author

mapleFU commented Oct 25, 2024

https://github.com/apache/arrow/actions/runs/11514744258/job/32053979673?pr=44532

Emmm seems this is orc related, and unrelated to this patch? 🤔

@pitrou
Copy link
Member

pitrou commented Oct 25, 2024

@mapleFU The error message seems related, perhaps we need to make RapidJSON mandatory when Parquet tests are enabled?

D:/a/arrow/arrow/cpp/src/parquet/reader_test.cc(32): fatal error C1083: Cannot open include file: 'rapidjson/document.h': No such file or directory

@mapleFU
Copy link
Member Author

mapleFU commented Oct 25, 2024

Aha, my bad, will fix this

@@ -320,7 +320,7 @@ if(ARROW_TESTING)
# "link" our dependencies so that include paths are configured
# correctly
target_link_libraries(parquet_testing PUBLIC ${ARROW_GTEST_GMOCK})
list(APPEND PARQUET_TEST_LINK_LIBS parquet_testing)
list(APPEND PARQUET_TEST_LINK_LIBS parquet_testing RapidJSON)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add rapidjson to PARQUET_TEST_LINK_LIBS, hope will not affect too much

@kou kou changed the title GH-44101: [C++][Parquet] Tools: Debug Print for Json should be valid Json GH-44101: [C++][Parquet] Tools: Debug Print for Json should be valid JSON Oct 28, 2024
@mapleFU
Copy link
Member Author

mapleFU commented Oct 31, 2024

Would merge this firstly

@mapleFU mapleFU merged commit f5691d4 into apache:main Oct 31, 2024
37 checks passed
@mapleFU mapleFU removed the awaiting committer review Awaiting committer review label Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants