DjangoUnicodeDecodeError when trying to write binary field

[ya-pekatoros]

Hi there!

I've faced interesting issue, when using Silk in Django projects, using pickle (https://docs.python.org/3/library/pickle.html) and also trying to save request.body as binary data in DB (MySQL) - BinaryField in Django

During tests I've face these:

Traceback (most recent call last):
  File "/venv/lib/python3.10/site-packages/django/utils/encoding.py", line 70, in force_str
    s = str(s, encoding, errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  --- PROJECT MODULES AND FUNCTIONS ---
    resp = self.client.post(
  File "/venv/lib/python3.10/site-packages/django/test/client.py", line 852, in post
    response = super().post(
  File "/venv/lib/python3.10/site-packages/django/test/client.py", line 441, in post
    return self.generic(
  File "/venv/lib/python3.10/site-packages/django/test/client.py", line 541, in generic
    return self.request(**r)
  File "/venv/lib/python3.10/site-packages/django/test/client.py", line 810, in request
    self.check_exception(response)
  File "/venv/lib/python3.10/site-packages/django/test/client.py", line 663, in check_exception
    raise exc_value
  File "/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 103, in view
    return self.dispatch(request, *args, **kwargs)
  File "/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "venv/lib/python3.10/site-packages/rest_framework/decorators.py", line 50, in handler
    return func(*args, **kwargs)
  File "/django/response.py", line 289, in wrapper
    return func(request, *args, **kwargs)
 --- PROJECT MODULES AND FUNCTIONS ---
  File "/venv/lib/python3.10/site-packages/django/db/models/base.py", line 812, in save
    self.save_base(
  File "/venv/lib/python3.10/site-packages/django/db/models/base.py", line 863, in save_base
    updated = self._save_table(
  File "/venv/lib/python3.10/site-packages/django/db/models/base.py", line 976, in _save_table
    updated = self._do_update(
  File "/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1040, in _do_update
    return filtered._update(values) > 0
  File "/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1215, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql
    cursor = super().execute_sql(result_type)
  File "/venv/lib/python3.10/site-packages/silk/sql.py", line 80, in execute_sql
    sql_query = q % tuple(force_str(param) for param in params)
  File "/venv/lib/python3.10/site-packages/silk/sql.py", line 80, in <genexpr>
    sql_query = q % tuple(force_str(param) for param in params)
  File "/venv/lib/python3.10/site-packages/django/utils/encoding.py", line 74, in force_str
    raise DjangoUnicodeDecodeError(s, *e.args)
django.utils.encoding.DjangoUnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte. You passed in b'\x80\x04\x95\x98\x01\x00\x00\x00\x00\x00\x00\x8c\x10requests.cookies\x94\x8c\x11RequestsCookieJar\x94\x93\x94)\x81\x94}\x94(\x8c\x07_policy\x94\x8c\x0ehttp.cookiejar\x94\x8c\x13DefaultCookiePolicy\x94\x93\x94)\x81\x94}\x94(\x8c\x08netscape\x94\x88\x8c\x07rfc2965\x94\x89\x8c\x13rfc2109_as_netscape\x94N\x8c\x0chide_cookie2\x94\x89\x8c\rstrict_domain\x94\x89\x8c\x1bstrict_rfc2965_unverifiable\x94\x88\x8c\x16strict_ns_unverifiable\x94\x89\x8c\x10strict_ns_domain\x94K\x00\x8c\x1cstrict_ns_set_initial_dollar\x94\x89\x8c\x12strict_ns_set_path\x94\x89\x8c\x10secure_protocols\x94\x8c\x05https\x94\x8c\x03wss\x94\x86\x94\x8c\x10_blocked_domains\x94)\x8c\x10_allowed_domains\x94Nub\x8c\x08_cookies\x94}\x94ub.' (<class 'bytes'>)

So Silk is trying to decode any content in SQL queries, how to switch off these for binary data?

I know about SilkyIgnorePaths, but its not very reliable to find all paths with binary data recording. I think it will be better to somehow wrap around bad decoding tries.

[pauliusbaulius]

Encountered the same issue, all binary data gets encoded again by django-silk.