-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python/DSVW repro #17635
base: main
Are you sure you want to change the base?
Python/DSVW repro #17635
Commits on Jun 25, 2024
-
python: Start modelling using MaD
- empty models for now - `summaryModel` of `codeql/python-all` will be added to shortly.
Configuration menu - View commit details
-
Copy full SHA for df406b4 - Browse repository at this point
Copy the full SHA df406b4View commit details -
python: add modelling for
urlib.parse
- `quote` together with `re.compile` recover regex injection alerts on haiwen/seahub - `quote_plus` recovers the URL redirection alert on DemocracyClub/EveryElection - `unquote` recovers path injection alerts on `cloudera/hue` - it was tedious finding justifications for the rest..
Configuration menu - View commit details
-
Copy full SHA for 281ac05 - Browse repository at this point
Copy the full SHA 281ac05View commit details -
python: move model to
Stdlib.yml
There is already a model there so we add to that one. We did observe that this existing model was blocked by the external MaD model. This is concerning and needs to be cleared up.
Configuration menu - View commit details
-
Copy full SHA for c004ffa - Browse repository at this point
Copy the full SHA c004ffaView commit details -
Configuration menu - View commit details
-
Copy full SHA for d410136 - Browse repository at this point
Copy the full SHA d410136View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1e97600 - Browse repository at this point
Copy the full SHA 1e97600View commit details -
Configuration menu - View commit details
-
Copy full SHA for b80a711 - Browse repository at this point
Copy the full SHA b80a711View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2118f23 - Browse repository at this point
Copy the full SHA 2118f23View commit details -
Configuration menu - View commit details
-
Copy full SHA for 501cda4 - Browse repository at this point
Copy the full SHA 501cda4View commit details -
Configuration menu - View commit details
-
Copy full SHA for bc55117 - Browse repository at this point
Copy the full SHA bc55117View commit details -
Two of the generated summaries have been excluded: - ["re", "Member[split]", "Argument[0,pattern:]", "ReturnValue", "taint"] From the documentation, it is not clear why pattern should figure in the return value, as that is the part denoting split point and thus all those instances are filtered out. From the implementation Spit function: https://github.com/python/cpython/blob/3.12/Lib/re/__init__.py#L199 _compile function being called by split: https://github.com/python/cpython/blob/3.12/Lib/re/__init__.py#L280 We see that in case the pattern is already a compiled `Pattern`, it is returned directly from _compile and could thus be part of the return value from split. This is probably not possible to arrange for an attacker, and so an FP in practice. - ["urllib2", "Member[unquote]", "Argument[0,string:]", "ReturnValue", "taint"] urllib2 seems to be only in Python2 (e.g. https://docs.python.org/2.7/library/urllib2.html) and I cannot locate the function unquote.
Configuration menu - View commit details
-
Copy full SHA for bdc4808 - Browse repository at this point
Copy the full SHA bdc4808View commit details -
Configuration menu - View commit details
-
Copy full SHA for eb32cbe - Browse repository at this point
Copy the full SHA eb32cbeView commit details -
Configuration menu - View commit details
-
Copy full SHA for 571be8b - Browse repository at this point
Copy the full SHA 571be8bView commit details
Commits on Jun 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for b261145 - Browse repository at this point
Copy the full SHA b261145View commit details -
Configuration menu - View commit details
-
Copy full SHA for a3076f4 - Browse repository at this point
Copy the full SHA a3076f4View commit details
Commits on Jun 28, 2024
-
Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for bbc3ff2 - Browse repository at this point
Copy the full SHA bbc3ff2View commit details -
It is not clear from the code how this could happen and I do not remember the path I saw, perhaps it was unreasonable.
Configuration menu - View commit details
-
Copy full SHA for 59f9532 - Browse repository at this point
Copy the full SHA 59f9532View commit details -
Python: Add value steps for sequence elements
It would be nice to simplify to a single sequence content type..
Configuration menu - View commit details
-
Copy full SHA for 5ddfe75 - Browse repository at this point
Copy the full SHA 5ddfe75View commit details -
Configuration menu - View commit details
-
Copy full SHA for 77a0087 - Browse repository at this point
Copy the full SHA 77a0087View commit details -
Configuration menu - View commit details
-
Copy full SHA for e40ae2e - Browse repository at this point
Copy the full SHA e40ae2eView commit details
Commits on Jul 22, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e30f725 - Browse repository at this point
Copy the full SHA e30f725View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3434c38 - Browse repository at this point
Copy the full SHA 3434c38View commit details
Commits on Sep 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f95926e - Browse repository at this point
Copy the full SHA f95926eView commit details -
Configuration menu - View commit details
-
Copy full SHA for e7f9b5b - Browse repository at this point
Copy the full SHA e7f9b5bView commit details
Commits on Sep 25, 2024
-
python: capture flow through comprehensions
- add comprehension functions as `DataFlowCallable`s - add comprehension call as `DataFlowCall` - create capture argument node for comprehension calls
Configuration menu - View commit details
-
Copy full SHA for fc2dc28 - Browse repository at this point
Copy the full SHA fc2dc28View commit details
Commits on Sep 27, 2024
-
Python: use comprehension function argument
For a comprehension `[x for x in l] - `l` is now a legal argument (in DataFlowPublic) - `l` is the argument of the comprehension function (in DataFlowDispatch) - the parameter of the comprehension function is being read rather than `l` (in IterableUnpacking) Thus the read that used to cross callable boundaries is now split into a arg-param edge and a read from that param.
Configuration menu - View commit details
-
Copy full SHA for 294092b - Browse repository at this point
Copy the full SHA 294092bView commit details -
Python: use synthetic node for comprehension capture argument
We used to use the CfgNode for the comprehension itself. In cases where that is also an argument, say ```python ",".join([x for x in l]) ``` that would be an argument to two different calls causing a dataflow consistency violation.
Configuration menu - View commit details
-
Copy full SHA for 72530a8 - Browse repository at this point
Copy the full SHA 72530a8View commit details
Commits on Sep 30, 2024
-
- add yield as a dataflow return - replace comprehension store step with a store step to the yield
Configuration menu - View commit details
-
Copy full SHA for d4ea62e - Browse repository at this point
Copy the full SHA d4ea62eView commit details -
Python: fix dataflow inconsistencies
- adjust scope of argument, the argument is outside the called function - add missing post-update nodes for the new arguments
Configuration menu - View commit details
-
Copy full SHA for 310819d - Browse repository at this point
Copy the full SHA 310819dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3ef05a6 - Browse repository at this point
Copy the full SHA 3ef05a6View commit details -
Python: update test expectations
We now have a new callable, yielding new enclosing callables
Configuration menu - View commit details
-
Copy full SHA for f9f46f0 - Browse repository at this point
Copy the full SHA f9f46f0View commit details -
Configuration menu - View commit details
-
Copy full SHA for ded3974 - Browse repository at this point
Copy the full SHA ded3974View commit details -
Configuration menu - View commit details
-
Copy full SHA for fb07a56 - Browse repository at this point
Copy the full SHA fb07a56View commit details -
Python: use yield step also for taint
Using the comprehension store step meant that all comprehensions would receive taint. This because comprehension flow now goes via a callable, meaning they share the return node.
Configuration menu - View commit details
-
Copy full SHA for 7392d18 - Browse repository at this point
Copy the full SHA 7392d18View commit details -
- also adjust test expectations in experimental
Configuration menu - View commit details
-
Copy full SHA for a22ea6c - Browse repository at this point
Copy the full SHA a22ea6cView commit details -
More doc is needed, but this should turn the tests green
Configuration menu - View commit details
-
Copy full SHA for 438e664 - Browse repository at this point
Copy the full SHA 438e664View commit details -
Configuration menu - View commit details
-
Copy full SHA for dacc0ab - Browse repository at this point
Copy the full SHA dacc0abView commit details
Commits on Oct 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e0a3c8a - Browse repository at this point
Copy the full SHA e0a3c8aView commit details -
Update python/ql/lib/change-notes/2024-09-24-std-lib-models.md
Co-authored-by: Rasmus Wriedt Larsen <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 2eac11e - Browse repository at this point
Copy the full SHA 2eac11eView commit details -
Update python/ql/lib/semmle/python/dataflow/new/internal/DataFlowDisp…
…atch.qll Co-authored-by: Taus <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 2b6aab1 - Browse repository at this point
Copy the full SHA 2b6aab1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 64890a1 - Browse repository at this point
Copy the full SHA 64890a1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7816f34 - Browse repository at this point
Copy the full SHA 7816f34View commit details -
Configuration menu - View commit details
-
Copy full SHA for cef8744 - Browse repository at this point
Copy the full SHA cef8744View commit details -
Configuration menu - View commit details
-
Copy full SHA for 05910de - Browse repository at this point
Copy the full SHA 05910deView commit details -
Python: use imprecise content in cp
We had accidentally used precise content leadingto blowup
Configuration menu - View commit details
-
Copy full SHA for f39dc41 - Browse repository at this point
Copy the full SHA f39dc41View commit details -
Configuration menu - View commit details
-
Copy full SHA for 38b1eb7 - Browse repository at this point
Copy the full SHA 38b1eb7View commit details -
Merge branch 'python/add-comprehension-capture-flow' of https://githu…
…b.com/yoff/codeql into python/DSVW-repro
Configuration menu - View commit details
-
Copy full SHA for 02d4da2 - Browse repository at this point
Copy the full SHA 02d4da2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4040195 - Browse repository at this point
Copy the full SHA 4040195View commit details -
Merge branch 'stdlib-optparse' of https://github.com/yoff/codeql into…
… python/DSVW-repro
Configuration menu - View commit details
-
Copy full SHA for 495bf71 - Browse repository at this point
Copy the full SHA 495bf71View commit details -
Configuration menu - View commit details
-
Copy full SHA for dff02cf - Browse repository at this point
Copy the full SHA dff02cfView commit details
Commits on Oct 2, 2024
-
Python: missing steps for repro
- API graph subscript operator to understand comprehensions - captureJumpStep to not require definig value to exist - stdlib modelling: finditer returns list of match objects - adjust taint output of finditer - adjust `ReMatchMethodsSummary.getACall`
Configuration menu - View commit details
-
Copy full SHA for 6df1f5a - Browse repository at this point
Copy the full SHA 6df1f5aView commit details