- Python 100%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
| dat | ||
| tests | ||
| .gitattributes | ||
| .gitignore | ||
| canonical_brand_map.csv | ||
| LICENSE | ||
| ords_extra_helper.py | ||
| ords_problem_en_deepl.csv | ||
| ords_problem_en_gemma.csv | ||
| ords_problem_en_opusmt.csv | ||
| ords_product_en.csv | ||
| ords_unu_keys.csv | ||
| README.md | ||
| requirements.txt | ||
| tableschema.json | ||
| unu_keys.csv | ||
ORDS Extra
Additional data for the 2026-07 ORDS dataset from the Open Repair Alliance.
Licence
Made available under the CC BY-NC-ND 4.0 licence © Monique Szpak (https://zenlan.codeberg.page).
Open Repair Data
The data files in this repo are designed to join to the 2026-07 Open Repair Data aggregate from the Open Repair Alliance
You can get the ORDS data in a zipfile or from Github
Files
ords_problem_en_deepl.csvISO language code and english translation ofproblemtext using the DeepL API.ords_problem_en_gemma.csvISO language code and english translation ofproblemtext using "gemmatranslate" locally, a model based on Gemma3.ords_problem_en_opusmt.csvISO language code and english translation ofproblemtext using Opus-MT models from the Helsinki-NLP project.ords_product_en.csvEnglish translations ofpartner_product_category'product' values via TranslateGemma, DeepL, Google and human.unu_keys.csvUnited Nations University (Unitar) categories, imported from the EU Recycler Information Center.ords_unu_keys.csvUNU Key mapping of ORDS categories.canonical_brand_map.csvA mapping of the most common brand names to ORDSbrandvalues that are malformed/misspelled.ords_extra_helper.pyA Python script that will join the above data to the ORDS dataset.
The problem translation files contain rows for non-English problem strings only. Currently for 2025 "eee" records only.
The product translation file contains rows for all 2026 product strings and some of the source English strings have been corrected. There are still a few that failed to translate.
The dat/misc folder holds assorted peripheral data files that are potentially useful in repair data analysis.
Translations
Repair data arrives once a year from all over the world and is hardly grammatically well-formed, often scribbled in the midst of a busy, noisy, messy community repair event. It can contain spelling mistakes, abbreviations, colloquialisms, acronyms, jargon, emojis, brand names, markup, markdown, html and non-printing characters. A lot of data cleaning is carried out prior to publication.
Language detection
The datasets arrive with no indication of the language used in each record. At last count there are at least 12 languages represented. The majority of records use one of English, German, Dutch, or French, but there is also some Italian, Spanish, Danish, Norwegian, Chinese, Korean and Welsh to be found.
Over time, I have tried a variety of tools and methods to determine the language and translate the problem text. Google detect is very poor with short strings and often gets Danish/Dutch/German confused, DeepL somewhat better but not free.
I trained a small language detection model on repair data and it is fairly successful. Manual tweaking is nonetheless required and the language map will still contain some erroneous detections.
The product value is not always in the same language as the problem text and therefore requires it's own language detection. TranslateGemma and DeepL scored marginally better at detection and translation for these short strings than some other translators, although there are still some wild mistranslations and the detected language required manual correction. Note that it is not uncommon to find a "brand name" used as a product type, e.g. "Hoover", "Karcher", "Nespresso". These may be flagged as English as they do not require translation, e.g. "Dolce Gusto" is identifiable as a type of coffee machine although it literally translates to "sweet taste".
Translation Tools
Problem Text
Note that English translation of the problem field in the 2026 dataset is a work in progress.
-
DeepL API is probably the best of the three. Am relying on the free monthly allowance, so it takes some time to complete DeepL translations.
-
TranslateGemma, an open translation model built on Gemma 3 has a stab at summarising what it thinks the text might say. Sometimes it produces the opposite meaning.
-
Helsinki-NLP Opus-MT can produce some very weird results and likes to inject the ocassional profanity. Does not have a Norwegian-English model. The Opus MT models are trained using MarianNMT, the same engine used by Mozilla for the Firefox translator.
Product values
Product strings are derived from the partner_product_category values, usually being the part after the tilde delimiter, or, if no tilde, the entire field.
By using a variety of translation APIs/models along with a mixture of NLP tools and human adjudication, a fair set of English product string translations has evolved.
Brand values
The brand column contains many spelling mistakes and other problems. An effort is made to weed out a lot of common issues when the ORDS dataset is compiled, e.g. strings such as "...", "?", "N/A". Nonetheless there will be some irrelevent/nonsense values that have slipped through.
The canonical_brand_map.csv file is a stab at providing improved brand values by using the Python difflib.SequenceMatcher to score the current db values against known good strings. Simpler than fuzzy/regex matching and apparently quite effective. Experimentation settled on a ratio score of 0.7 for pretty decent results albeit a few false positives are returned. e.g. "Sharp" => "Shark" and "Shark" => "Sharp". Recommend human review.
- The map of canonical brand names is currently for eee products only.
- The map is not a definitive list of canonical brand names.
- Not all ORDS
brandvalues have a canonical mapping. - The
brand_namevalues have been gleaned from brand websites and Wikipedia. - The map includes ORDS
brandvalues that are already canonical. - There may be discrepencies regarding internationalisation.
UNU Keys
ORDS data contains a column that lists the category and/or type of product as provided by the contributor - partner_product_category. Most commonly in the format <partner_category> ~ <partner_product> although sometimes simply <partner_product>. Not all of the partner networks use a standardised set of categories or product types to describe the item brought in for repair, so the <partner_category> can be free text or empty, and the <partner_product> is always free-text. Even when a list of options is present at data-entry, miscategorisation is common. The ORDS data maps these product types and categories to it's own standard category set as best it can. An alternative and widely recognised category set is provided by UNITAR - United Nations Institute for Training and Research..
These keys are commonly found in academic papers dealing with subjects around e-waste and trickle out to wider reports and news stories, e.g. OF 16 BILLION MOBILE PHONES POSSESSED WORLDWIDE, 5.3 BILLION WILL BECOME WASTE IN 2022.
The file ords_unu_keys.csv provides a mapping of the ORDS records to the "UNU Key" categories produced by UNITAR - United Nations Institute for Training and Research. Note that only "eee" records are mapped.
Mapping categories to keys
The product is split out from the ORDS partner_product_category value and searched for regex patterns that map to 'products' that can map to UNU keys.
The result is an improvement on the coarse one-to-one mapping of ORDS product_category to UNU key, bearing in mind that:
-
Sometimes the categorisation at source was not optimal and/or could have been mapped to the wrong ORDS
product_category, this can affect the assignment of the UNU key. -
Sometimes a mapping is not obvious from the
<partner_product>value, which could be empty, it could be that theproblemtext enabled the categorisation, e.g.<partner_product>="steamer" could be anything (e.g. kitchen appliance, steam iron...), but theproblemtext may have indicated a "steam mop". -
The matching process is not perfect, it has to work across at least 10 European languages and a couple of Asian languages, international characters, spelling mistakes, abbreviations, acronyms, jargon and brand names.
-
Manual revision is required to fine-tune the mappings but, given the volume, it is inevitable that some slip through the net
-
Given that English translations of the
<partner_product>strings have evolved some uniformity and "good enough" accuracy, it should now be possible to improve the UNU key mappings. See fileords_product_en.csv. -
When a
<partner_product>does not find a regex match, it is assigned a default UNU key for its ORDSproduct_category.