SynchronousOnlyOperation when generating regex expression for urls from database entries

[mateokurti]

I am facing an issue now that I'm trying to integrate channels into my Django application. On one of my urls, I generate a regex expression which have a dependency from a database transaction.

class NewsTypeManager(Manager):

    def names_regex(self):
        names = self.get_queryset().values_list('name', flat=True)
        regex = r'(?P<newstype>'
        try:
            for n in names:
                regex += r'|{}'.format(n)
        except ProgrammingError:
            regex += r'|{}'.format('')
        regex += r')'
        return regex

router = DefaultRouter()
router.register(r'news/' + NewsType.objects.names_regex(), NewsModelViewSet,  basename='news')

I'm configuring the router like this:

application = ProtocolTypeRouter(
    {
        "http": get_asgi_application(),
        "websocket": AuthMiddlewareStack(
            URLRouter(apps.news.routing.websocket_urlpatterns)
        ),
    }
)

The error I'm having considers the line for n in names: on the names_regex method. I'm attaching the logs below:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/asgiref/sync.py", line 486, in thread_handler
    raise exc_info[1]
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 42, in inner
    response = await get_response(request)
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 235, in _get_response_async
    callback, callback_args, callback_kwargs = self.resolve_request(request)
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 313, in resolve_request
    resolver_match = resolver.resolve(request.path_info)
  File "/usr/local/lib/python3.10/site-packages/django/urls/resolvers.py", line 638, in resolve
    for pattern in self.url_patterns:
  File "/usr/local/lib/python3.10/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/local/lib/python3.10/site-packages/django/urls/resolvers.py", line 690, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/usr/local/lib/python3.10/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/local/lib/python3.10/site-packages/django/urls/resolvers.py", line 683, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
    File "/srv/my_app/./apps/config/urls.py", line 28, in <module>
    from ..news.api.urls import router as news_router
  File "/srv/my_app/./apps/news/api/urls.py", line 7, in <module>
    news_type_names_regex = NewsType.objects.names_regex()
  File "/srv/my_app/./apps/news/managers.py", line 95, in names_regex
    for n in names:
  File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 394, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 1866, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 281, in __iter__
    for row in compiler.results_iter(
  File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1346, in results_iter
    results = self.execute_sql(
  File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1393, in execute_sql
    cursor = self.connection.cursor()
  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 24, in inner
    raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.

I get it that names_regex is a synchronous method and the error shows that it is called on an async context, but I'm not sure how to resolve it. Any suggestion?

[carltongibson]

The usual approach would be to match any name ([a-z]+, or whatever) and then do the lookup in the view, returning a 404, if the name didn't match (asdfggh, say)

[mateokurti]

That makes sense, however since it is something that almost never change, I've done the lookup on the urls. Moving it in the view, would do the extra query on every request. Is this the only approach?

[carltongibson]

You can move calculating the regex somewhere (like an AppConfig.ready()) where it's done after django.setup() but not in an async request context.