scrachy.settings.defaults.storage.SCRACHY_CACHE_DEFAULT_ENCODING

scrachy.settings.defaults.storage.SCRACHY_CACHE_DEFAULT_ENCODING: str = 'utf-8'

Sometimes it is not possible to determine the encoding of a page because it was not set properly at the source. But this also seems to happen for compressed pages which have an encoding based on the compression algorithm (e.g., gzip). However, Scrapy will raise an exception when constructing a scrapy.http.TextResponse if it can’t determine the encoding. To avoid these issues you can specify a default encoding to use when Scrapy fails to automatically identify a compatible one.