[Improve][transforms-v2][SQL] Support can use column.a, column['a'], and column[0] to get the data

[Zhouwen-CN]

#5245

Purpose of this pull request

Check list

• Code changed are covered with tests, or it does not need tests for reason:
• If any new Jar binary package adding in your PR, please add License Notice according
  New License Guide
• If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
• If you are contributing the connector code, please check that the following files are updated:
  1. Update change log that in connector document. For more details you can refer to connector-v2
  2. Update plugin-mapping.properties and add new connector information in it
  3. Update the pom file of seatunnel-dist
• Update the release-note.

[EricJoy2048]

Good features, Please wait CI complete.

[EricJoy2048]

LGTM

[EricJoy2048]

@rewerma PATL

[Zhouwen-CN]

What is this mistake? Did I cause it

[EricJoy2048]

What is this mistake? Did I cause it

No, it's not you problem. Can you rebase the dev branch and push again ?

[Zhouwen-CN]

What is this mistake? Did I cause it

No, it's not you problem. Can you rebase the dev branch and push again ?

Ok, tonight

[Zhouwen-CN]

What is this mistake? Did I cause it

No, it's not you problem. Can you rebase the dev branch and push again ?

Is this how it is done?

[EricJoy2048]

What is this mistake? Did I cause it

No, it's not you problem. Can you rebase the dev branch and push again ?

Is this how it is done?

It seems some thing is wrong. Some modifications that should not be submitted in this PR.

[EricJoy2048]

How to rebase?

git checkout chen0623-bak:sql
git fetch ${the apache remote}
git rebase ${the apache remote}/dev

[Zhouwen-CN]

sync fork

git fetch origin (at local)
git checkout sql (at local)
git rebase dev (at local)
git push sql (Prompt me whether to merge or rebase, I chose rebase)

like this

[EricJoy2048]

sync fork git fetch origin (at local) git checkout sql (at local) git rebase dev (at local) git push sql (Prompt me whether to merge or rebase, I chose rebase)

like this

For you current state, I think you can do it like this:

git fetch --all
git checkout dev
git pull
git checkout sql
git reset --hard bb1371a04599b0f2c421dc97504e6bb43b3ef8f8
git rebase dev (in this step, you need fix the conflict, and use git rebase --continue at the appropriate time)
git push -f

[Zhouwen-CN]

@EricJoy2048
Is this how it is done?
Thank, I get it

[EricJoy2048]

@wu-a-ge PTAL. I have found that you have extensive experience in this field. Can you help review this PR？

[EricJoy2048]

@rewerma PTAL

[Zhouwen-CN]

@EricJoy2048
I git it, I'm back from Internet

transform-v2-it-part-2 (8, ubuntu-latest) :
	when map size lt 5
	scala use Map$Map1 Map$Map2 Map$Map3 Map$Map4
	when map size gte 5
	scala use Map$HashTrieMap
	So we can avoid this error by using Map
		Modify SeaTunnelRowConverter 207 line
		Modify SeaTunnelRowConverter 173 line
        Or modify config file map size gte 5

FakeSource does not specify the rows parameter for testing without reporting errors because map.size default=5

I changed the map length of the test file to 5
The local test has passed

[wu-a-ge]

The map and array values in the test case are only strings, can we consider other types? For example,null,int,date,timestamp

[Zhouwen-CN]

Are you saying that the key of the map cannot be certain to be a string
This is indeed a problem
I think Visitor should be used to access it

Because it is JSON serialization, the key will be parsed as a string, even if it is an int
The type is taken from upstream, I think there should be no problem

[wu-a-ge]

What I am talking about is the value, and it is necessary to consider that after these values are taken out, the downstream sink can write correctly

[Zhouwen-CN]

The generics of array can only be basic types
If the value type of a map is date, time, or timestamp, the underlying type is actually a Java8 time type
Jackson will report an error during serialization and needs to register the JavaTimeModule module
Specific dependencies need to be added as well

[Zhouwen-CN]

I added a Visitor to obtain the key or index
The key of a map may be int, double, or string, it may not necessarily be a string
Although it will be converted to a string during JSON serialization, But we may have other serializers
I don't know if this is suitable or not
and When ArrayIndexOutOfBoundsException return null, Ensure the robustness of the program

I think the parameter Object[] inputFields should also be passed in when parsing types
Because if I don't pass in parameters, I need to make a lot of non null judgments
And it cannot parse the subscript calculation of column[c1+1] as a reference column

[EricJoy2048]

Hi, Way are you close this pr?

[Zhouwen-CN]

Hi, Way are you close this pr?

I think the visitor mode extension is better
I will improve it in the new PR

Referring to select *, its implementation is simpler and more decoupled

[sxiongzhang]

I will improve it in the new PR

hello ，has the pr been submitted?