Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow translating from javascript files #15612

Open
wants to merge 69 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
05f5601
add translations route for js translations
cofiem Mar 17, 2024
ece9fb5
use js translations for timeago
cofiem Mar 17, 2024
d70e76e
add babel config for js translations
cofiem Mar 17, 2024
9003fa5
add js util to get translations from backend
cofiem Mar 17, 2024
d2273a4
add messages extracted from js
cofiem Mar 17, 2024
291189b
use the default pybabel js function name
cofiem Mar 17, 2024
64f4d9a
typo
cofiem Mar 17, 2024
a10b3fb
Merge branch 'main' into feature/translation-js
cofiem Mar 17, 2024
ec3b0be
extract singular and plural translations from js
cofiem Mar 19, 2024
5920aeb
Merge branch 'pypi:main' into feature/translation-js
cofiem Mar 19, 2024
11db677
Merge remote-tracking branch 'origin/feature/translation-js' into fea…
cofiem Mar 19, 2024
f705788
fix lint issues and remove logging
cofiem Mar 19, 2024
4e95739
add translation to more js strings
cofiem Mar 20, 2024
266c1e5
add translation to more js strings
cofiem Mar 20, 2024
8ff40e4
add translation to more python strings
cofiem Mar 20, 2024
7dc6ea4
update and add tests for js translations
cofiem Mar 20, 2024
c593101
update translations
cofiem Mar 20, 2024
fa2721a
Merge branch 'pypi:main' into feature/translation-js
cofiem Mar 24, 2024
c1a7811
fix route test
cofiem Mar 24, 2024
25f2c28
avoid redirect
cofiem Mar 24, 2024
1a6b4e0
fix lint error
cofiem Mar 24, 2024
d73ee32
update translation url in tests
cofiem Mar 24, 2024
b44655f
update translations
cofiem Mar 24, 2024
9a43147
don't mix translated and untranslated text
cofiem Mar 31, 2024
d3ccf58
Merge branch 'pypi:main' into feature/translation-js
cofiem Mar 31, 2024
18eb472
remove unused import
cofiem Mar 31, 2024
7303e14
update translations
cofiem Mar 31, 2024
42ed35d
update translations
cofiem Mar 31, 2024
07341f3
add tests for translation action
cofiem Mar 31, 2024
0d94a86
Merge branch 'main' into feature/translation-js
di Apr 1, 2024
11d929c
Merge branch 'refs/heads/main' into feature/translation-js
cofiem Apr 5, 2024
8fb15dd
draft of alternate attempt that generates js-only translations and us…
cofiem Apr 6, 2024
bacbd4d
Merge branch 'pypi:main' into feature/translation-js
cofiem Apr 14, 2024
67b771f
check point
cofiem Apr 18, 2024
6139ea6
Merge branch 'refs/heads/main' into feature/translation-js
cofiem Apr 18, 2024
4e1667d
check point
cofiem Apr 19, 2024
11f093a
implemented webpack localize plugin
cofiem Apr 20, 2024
8a8b6f9
Merge branch 'refs/heads/main' into feature/translation-js
cofiem Apr 20, 2024
1bcda98
embed plural forms using webpack and determine translation string
cofiem Apr 20, 2024
ac16bc5
add tests for messages-access.js
cofiem Apr 21, 2024
6946b68
fix lint issues
cofiem Apr 21, 2024
3b08211
Merge branch 'refs/heads/main' into feature/translation-js
cofiem Jun 29, 2024
791c41c
add removed dev dependency for js translations
cofiem Jun 29, 2024
93b9447
update translations
cofiem Jun 29, 2024
6535a1c
Merge branch 'refs/heads/main' into feature/translation-js
cofiem Jul 13, 2024
448918a
Merge branch 'main' into feature/translation-js
di Aug 12, 2024
cf57600
Merge branch 'main' into feature/translation-js
di Aug 15, 2024
d18b04a
Merge branch 'main' into feature/translation-js
cofiem Aug 19, 2024
88e027e
build js translation data as part of static_pipeline webpack build
cofiem Aug 19, 2024
fd7a88e
Merge branch 'main' into feature/translation-js
di Aug 19, 2024
0f7196d
address feedback: don't translate the admin UI
cofiem Aug 21, 2024
010fdd2
address feedback: don't translate the admin UI
cofiem Aug 21, 2024
43d6be1
address feedback: fix make dependency order
cofiem Aug 21, 2024
d32b447
address feedback: improve docs for placeholder js variables
cofiem Aug 21, 2024
7d1e8cb
address feedback: add missing webpack js files to eslint paths
cofiem Aug 21, 2024
34ed5e6
address feedback: fix eslint issues
cofiem Aug 21, 2024
0f13ec1
address feedback: extract shared value to constant
cofiem Aug 21, 2024
b6fc1e6
address feedback: use jest each for tests
cofiem Aug 21, 2024
6b094a2
address feedback: remove old todo
cofiem Aug 21, 2024
70f2c33
address feedback: fix eslint issues
cofiem Aug 21, 2024
8e2e3f5
address feedback: improve plural forms pattern and check earlier
cofiem Aug 21, 2024
fd0dd80
address feedback: make sure msgid and msgstr are really strings
cofiem Aug 21, 2024
7d35f82
Merge remote-tracking branch 'origin/feature/translation-js' into fea…
cofiem Aug 21, 2024
50261ef
Merge branch 'pypi:main' into feature/translation-js
cofiem Aug 21, 2024
7cd21a5
Merge remote-tracking branch 'origin/feature/translation-js' into fea…
cofiem Aug 21, 2024
6fa0444
Revert change to `make statuc_pipelines`
cofiem Aug 22, 2024
b6edde8
encode known problematic characters
cofiem Aug 22, 2024
3c91638
Merge branch 'main' into feature/translation-js
cofiem Aug 31, 2024
ef820c2
fix lint space issue
cofiem Aug 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ FROM static-deps AS static
# small amount of copying when only `webpack.config.js` is modified.
COPY warehouse/static/ /opt/warehouse/src/warehouse/static/
COPY warehouse/admin/static/ /opt/warehouse/src/warehouse/admin/static/
COPY warehouse/locale/ /opt/warehouse/src/warehouse/locale/
COPY webpack.config.js /opt/warehouse/src/
COPY webpack.plugin.localize.js /opt/warehouse/src/

RUN NODE_ENV=production npm run build

Expand Down
3 changes: 3 additions & 0 deletions babel.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,6 @@
encoding = utf-8
extensions=warehouse.utils.html:ClientSideIncludeExtension,warehouse.i18n.extensions.TrimmedTranslatableTagsExtension
silent=False
[javascript: **.js]
encoding=utf-8
silent=False
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,7 @@ services:
volumes:
- ./warehouse:/opt/warehouse/src/warehouse:z
- ./webpack.config.js:/opt/warehouse/src/webpack.config.js:z
- ./webpack.plugin.localize.js:/opt/warehouse/src/webpack.plugin.localize.js:z
- ./.babelrc:/opt/warehouse/src/.babelrc:z
- ./.stylelintrc.json:/opt/warehouse/src/.stylelintrc.json:z
- ./tests/frontend:/opt/warehouse/src/tests/frontend:z
Expand Down
22 changes: 22 additions & 0 deletions docs/dev/development/frontend.rst
Original file line number Diff line number Diff line change
Expand Up @@ -151,3 +151,25 @@ One of these blocks provides code syntax highlighting, which can be tested with
reference project provided at `<http://localhost/project/pypi-code-highlighting-demo/>`_
when using development database. Source reStructuredText file is available
`here <https://github.com/evemorgen/pypi-code-highlighting-demo>`_.


Javascript localization support
-------------------------------

Strings in JS can be translated, see the see the :doc:`../translations` docs.

As part of the webpack build,
the translation data for each locale in ``KNOWN_LOCALES``
is placed in |warehouse/static/js/warehouse/utils/messages-access.js|_.

A separate js bundle is generated for each locale,
named like this: ``warehouse.[locale].[contenthash].js``.

The JS bundle to include is selected in |warehouse/templates/base.html|_
using the current :code:`request.localizer.locale_name`.

.. |warehouse/static/js/warehouse/utils/messages-access.js| replace:: ``warehouse/static/js/warehouse/utils/messages-access.js``
.. _warehouse/static/js/warehouse/utils/messages-access.js: https://github.com/pypi/warehouse/blob/main/warehouse/static/js/warehouse/utils/messages-access.js

.. |warehouse/templates/base.html| replace:: ``warehouse/templates/base.html``
.. _warehouse/templates/base.html: https://github.com/pypi/warehouse/blob/main/warehouse/templates/base.html
29 changes: 27 additions & 2 deletions docs/dev/translations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ To add a new known locale:
1. Check for `outstanding Weblate pull requests
<https://github.com/pypi/warehouse/pulls/weblate>`_ and merge them if so.
2. In a new branch for |pypi/warehouse|_, add the new language identifier to
``KNOWN_LOCALES`` in |warehouse/i18n/__init__.py|_.
``KNOWN_LOCALES`` in |warehouse/i18n/__init__.py|_ and |webpack.plugin.localize.js|_.
The value is the locale code, and corresponds to a directory in
``warehouse/locale``.
3. Commit these changes and make a new pull request to |pypi/warehouse|_.
Expand All @@ -45,6 +45,8 @@ To add a new known locale:
.. _pypi/warehouse: https://github.com/pypi/warehouse
.. |warehouse/i18n/__init__.py| replace:: ``warehouse/i18n/__init__.py``
.. _warehouse/i18n/__init__.py: https://github.com/pypi/warehouse/blob/main/warehouse/i18n/__init__.py
.. |webpack.plugin.localize.js| replace:: ``webpack.plugin.localize.js``
.. _webpack.plugin.localize.js: https://github.com/pypi/warehouse/blob/main/webpack.plugin.localize.js
.. |pypi/infra| replace:: ``pypi/infra``
.. _pypi/infra: https://github.com/pypi/infra

Expand All @@ -62,11 +64,23 @@ In Python, given a request context, call :code:`request._(message)` to mark
from warehouse.i18n import localize as _
message = _("Your message here.")

In javascript, use :code:`gettext("singular", ...placeholder_values)` and
:code:`ngettext("singular", "plural", count, ...placeholder_values)`.
The function names are important because they need to be recognised by pybabel.

.. code-block:: javascript

import { gettext, ngettext } from "../utils/messages-access";
gettext("Get some fruit");
// -> (en) "Get some fruit"
ngettext("Yesterday", "In the past", numDays);
// -> (en) numDays is 1: "Yesterday"; numDays is 3: "In the past"


Passing non-translatable values to translated strings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To pass values you don't want to be translated into
In html, to pass values you don't want to be translated into
translated strings, define them inside the :code:`{% trans %}` tag.
For example, to pass a non-translatable link
:code:`request.route_path('classifiers')` into a string, instead of
Expand All @@ -86,6 +100,15 @@ Instead, define it inside the :code:`{% trans %}` tag:
Filter by <a href="{{ href }}">classifier</a>
{% endtrans %}

In javascript, use :code:`%1`, :code:`%2`, etc as
placeholders and provide the placeholder values:

.. code-block:: javascript

import { ngettext } from "../utils/messages-access";
ngettext("Yesterday", "About %1 days ago", numDays, numDays);
// -> (en) numDays is 1: "Yesterday"; numDays is 3: "About 3 days ago"


Marking new strings for pluralization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand All @@ -105,6 +128,8 @@ variants of a string, for example:

This is not yet directly possible in Python for Warehouse.

In javascript, use :code:`ngettext()` as described above.

Marking views as translatable
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
130 changes: 130 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
"scripts": {
"build": "webpack",
"watch": "webpack --watch",
"lint": "eslint 'warehouse/static/js/**' 'warehouse/admin/static/js/**' 'tests/frontend/**' --ignore-pattern 'warehouse/static/js/vendor/**'",
"lint:fix": "eslint 'warehouse/static/js/**' 'warehouse/admin/static/js/**' 'tests/frontend/**' --ignore-pattern 'warehouse/static/js/vendor/**' --fix",
"lint": "eslint 'warehouse/static/js/**' 'warehouse/admin/static/js/**' 'tests/frontend/**' 'webpack.*.js' --ignore-pattern 'warehouse/static/js/vendor/**'",
"lint:fix": "eslint 'warehouse/static/js/**' 'warehouse/admin/static/js/**' 'tests/frontend/**' 'webpack.*.js' --ignore-pattern 'warehouse/static/js/vendor/**' --fix",
"stylelint": "stylelint '**/*.scss' --cache",
"stylelint:fix": "stylelint '**/*.scss' --cache --fix",
"test": "jest --coverage"
Expand Down Expand Up @@ -46,6 +46,7 @@
"css-loader": "^7.1.2",
"css-minimizer-webpack-plugin": "^7.0.0",
"eslint": "^8.36.0",
"gettext-parser": "^7.0.1",
"glob": "^10.2.2",
"image-minimizer-webpack-plugin": "^4.0.2",
"jest": "^29.5.0",
Expand Down
Loading