Additional data for the ORDS dataset from the Open Repair Alliance https://openrepair.org/
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-10-21 09:41:27 +01:00
dat Adds dat/ords directory 2026-08-22 06:07:48 +01:00
tests Refactors for venv 2026-07-29 07:11:16 +01:00
.gitattributes Initial commit 2026-04-06 20:39:49 +01:00
.gitignore Adds .gitignore 2026-07-29 07:13:53 +01:00
canonical_brand_map.csv Adds data/code to "correct" some ORDS brand values 2026-10-21 09:41:27 +01:00
LICENSE Initial commit 2026-04-06 21:33:59 +02:00
ords_extra_helper.py Adds data/code to "correct" some ORDS brand values 2026-10-21 09:41:27 +01:00
ords_problem_en_deepl.csv Initial commit 2026-04-06 20:39:49 +01:00
ords_problem_en_gemma.csv Replaces gemma 4bQ4 translations with gemma 12bQ4 translations 2026-04-19 10:50:52 +01:00
ords_problem_en_opusmt.csv Initial commit 2026-04-06 20:39:49 +01:00
ords_product_en.csv Updates data and script for translated product strings 2026-10-07 07:28:48 +01:00
ords_unu_keys.csv Updates unu_key mapping 2026-10-12 14:42:56 +01:00
README.md Adds data/code to "correct" some ORDS brand values 2026-10-21 09:41:27 +01:00
requirements.txt Refactors for venv 2026-07-29 07:11:16 +01:00
tableschema.json Updates tableschema json 2026-07-10 15:14:12 +01:00
unu_keys.csv Initial commit 2026-04-06 20:39:49 +01:00

ORDS Extra

Additional data for the 2026-07 ORDS dataset from the Open Repair Alliance.

Licence

Made available under the CC BY-NC-ND 4.0 licence © Monique Szpak (https://zenlan.codeberg.page).

Open Repair Data

The data files in this repo are designed to join to the 2026-07 Open Repair Data aggregate from the Open Repair Alliance

You can get the ORDS data in a zipfile or from Github

Files

  • ords_problem_en_deepl.csv ISO language code and english translation of problem text using the DeepL API.
  • ords_problem_en_gemma.csv ISO language code and english translation of problem text using "gemmatranslate" locally, a model based on Gemma3.
  • ords_problem_en_opusmt.csv ISO language code and english translation of problem text using Opus-MT models from the Helsinki-NLP project.
  • ords_product_en.csv English translations of partner_product_category 'product' values via TranslateGemma, DeepL, Google and human.
  • unu_keys.csv United Nations University (Unitar) categories, imported from the EU Recycler Information Center.
  • ords_unu_keys.csv UNU Key mapping of ORDS categories.
  • canonical_brand_map.csv A mapping of the most common brand names to ORDS brand values that are malformed/misspelled.
  • ords_extra_helper.py A Python script that will join the above data to the ORDS dataset.

The problem translation files contain rows for non-English problem strings only. Currently for 2025 "eee" records only.

The product translation file contains rows for all 2026 product strings and some of the source English strings have been corrected. There are still a few that failed to translate.

The dat/misc folder holds assorted peripheral data files that are potentially useful in repair data analysis.

Translations

Repair data arrives once a year from all over the world and is hardly grammatically well-formed, often scribbled in the midst of a busy, noisy, messy community repair event. It can contain spelling mistakes, abbreviations, colloquialisms, acronyms, jargon, emojis, brand names, markup, markdown, html and non-printing characters. A lot of data cleaning is carried out prior to publication.

Language detection

The datasets arrive with no indication of the language used in each record. At last count there are at least 12 languages represented. The majority of records use one of English, German, Dutch, or French, but there is also some Italian, Spanish, Danish, Norwegian, Chinese, Korean and Welsh to be found.

Over time, I have tried a variety of tools and methods to determine the language and translate the problem text. Google detect is very poor with short strings and often gets Danish/Dutch/German confused, DeepL somewhat better but not free.

I trained a small language detection model on repair data and it is fairly successful. Manual tweaking is nonetheless required and the language map will still contain some erroneous detections.

The product value is not always in the same language as the problem text and therefore requires it's own language detection. TranslateGemma and DeepL scored marginally better at detection and translation for these short strings than some other translators, although there are still some wild mistranslations and the detected language required manual correction. Note that it is not uncommon to find a "brand name" used as a product type, e.g. "Hoover", "Karcher", "Nespresso". These may be flagged as English as they do not require translation, e.g. "Dolce Gusto" is identifiable as a type of coffee machine although it literally translates to "sweet taste".

Translation Tools

Problem Text

Note that English translation of the problem field in the 2026 dataset is a work in progress.

  • DeepL API is probably the best of the three. Am relying on the free monthly allowance, so it takes some time to complete DeepL translations.

  • TranslateGemma, an open translation model built on Gemma 3 has a stab at summarising what it thinks the text might say. Sometimes it produces the opposite meaning.

  • Helsinki-NLP Opus-MT can produce some very weird results and likes to inject the ocassional profanity. Does not have a Norwegian-English model. The Opus MT models are trained using MarianNMT, the same engine used by Mozilla for the Firefox translator.

Product values

Product strings are derived from the partner_product_category values, usually being the part after the tilde delimiter, or, if no tilde, the entire field.

By using a variety of translation APIs/models along with a mixture of NLP tools and human adjudication, a fair set of English product string translations has evolved.

Brand values

The brand column contains many spelling mistakes and other problems. An effort is made to weed out a lot of common issues when the ORDS dataset is compiled, e.g. strings such as "...", "?", "N/A". Nonetheless there will be some irrelevent/nonsense values that have slipped through.

The canonical_brand_map.csv file is a stab at providing improved brand values by using the Python difflib.SequenceMatcher to score the current db values against known good strings. Simpler than fuzzy/regex matching and apparently quite effective. Experimentation settled on a ratio score of 0.7 for pretty decent results albeit a few false positives are returned. e.g. "Sharp" => "Shark" and "Shark" => "Sharp". Recommend human review.

  • The map of canonical brand names is currently for eee products only.
  • The map is not a definitive list of canonical brand names.
  • Not all ORDS brand values have a canonical mapping.
  • The brand_name values have been gleaned from brand websites and Wikipedia.
  • The map includes ORDS brand values that are already canonical.
  • There may be discrepencies regarding internationalisation.

UNU Keys

ORDS data contains a column that lists the category and/or type of product as provided by the contributor - partner_product_category. Most commonly in the format <partner_category> ~ <partner_product> although sometimes simply <partner_product>. Not all of the partner networks use a standardised set of categories or product types to describe the item brought in for repair, so the <partner_category> can be free text or empty, and the <partner_product> is always free-text. Even when a list of options is present at data-entry, miscategorisation is common. The ORDS data maps these product types and categories to it's own standard category set as best it can. An alternative and widely recognised category set is provided by UNITAR - United Nations Institute for Training and Research..

These keys are commonly found in academic papers dealing with subjects around e-waste and trickle out to wider reports and news stories, e.g. OF 16 BILLION MOBILE PHONES POSSESSED WORLDWIDE, 5.3 BILLION WILL BECOME WASTE IN 2022.

The file ords_unu_keys.csv provides a mapping of the ORDS records to the "UNU Key" categories produced by UNITAR - United Nations Institute for Training and Research. Note that only "eee" records are mapped.

Mapping categories to keys

The product is split out from the ORDS partner_product_category value and searched for regex patterns that map to 'products' that can map to UNU keys.

The result is an improvement on the coarse one-to-one mapping of ORDS product_category to UNU key, bearing in mind that:

  • Sometimes the categorisation at source was not optimal and/or could have been mapped to the wrong ORDS product_category, this can affect the assignment of the UNU key.

  • Sometimes a mapping is not obvious from the <partner_product> value, which could be empty, it could be that the problem text enabled the categorisation, e.g. <partner_product>="steamer" could be anything (e.g. kitchen appliance, steam iron...), but the problem text may have indicated a "steam mop".

  • The matching process is not perfect, it has to work across at least 10 European languages and a couple of Asian languages, international characters, spelling mistakes, abbreviations, acronyms, jargon and brand names.

  • Manual revision is required to fine-tune the mappings but, given the volume, it is inevitable that some slip through the net

  • Given that English translations of the <partner_product> strings have evolved some uniformity and "good enough" accuracy, it should now be possible to improve the UNU key mappings. See file ords_product_en.csv.

  • When a <partner_product> does not find a regex match, it is assigned a default UNU key for its ORDS product_category.