[java] Fix avro logical-types conversions for BQ storage #33422
Conversation
Most of the avro logical-type to BQ are broken. Add support for both joda and java time to ensure compatibility with older avro versions
| @@ -115,34 +120,90 @@ static String convertUUID(Object value) { | |||
| } | |||
|
|
|||
| static Long convertTimestamp(Object value, boolean micros) { | |||
| if (value instanceof ReadableInstant) { | |||
| return ((ReadableInstant) value).getMillis() * (micros ? 1000 : 1); | |||
There was a problem hiding this comment.
This was wrong. BQ always expects epoch microseconds. Conversion should be applied on the raw type depending if it represents millis or micros
| .setScale(type.getScale(), RoundingMode.DOWN) | ||
| .round(new MathContext(type.getPrecision(), RoundingMode.DOWN)); |
There was a problem hiding this comment.
Does this seems legit to round ? We might also fail if the BigDecimal precision and scale do not match with expected logical type
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
|
assign set of reviewers |
|
Assigning reviewers. If you would like to opt out of this review, comment R: @kennknowles for label java. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
Reminder, please take a look at this pr: @kennknowles @chamikaramj |
Most of the avro logical-type to BQ are broken.
Add support for both joda and java time to ensure compatibility with older avro versions