OpenRefine
Tool Box
Practical tools and services for research data management
©
Lachlan Donald | Unsplash
Manual maintenance and preparation of data is often a tedious and time-consuming routine task. For many tasks, however, there are auxiliary programmes that can significantly reduce the effort and increase the data quality. In order to help you find your way through the multitude of programmes, we have put together a small selection of freely available tools to help you get started.
Notes
The LUIS offers licences for numerous software tools or provides support in obtaining them. If you need special software, please check first whether it can be obtained from the LUIS.
On the page www.forschungsdaten.info/praxis-kompakt/tools you will find a detailed list of programmes and online services that support you in various data processing and project planning tasks.
LUH, TIB and L3S jointly operate the Data Science Lab as a platform on which research activities in the field of data science are to be bundled. There you will find an overview of technical infrastructure and can also take advantage of training and consulting services on this topic.
Tools for data preparation
-
Clean up tables
Are you familiar with these situations? You want to evaluate tabular data, but you notice that the structure and content are inconsistent and incorrect. Typical problems are e.g:
- text in columns for numerical values
- several values in the same cell
- special characters have not been coded correctly
With the programme "OpenRefine", such and other inconsistencies can be corrected relatively quickly and semi-automatically.
-
Rename files
Hmmmm, what is in the file "APV20864XgbT.txt" again? Manually renaming many individual files is time-consuming and rarely succeeds without errors. Fortunately, there are tools like RenameMaster for such cases. With them, hundreds of file names can be changed and standardised in one fell swoop.
-
Edit images automatically
Do you want to create thumbnails of all the images in a folder? Or standardise the format? Or change the names? These and other work steps can be conveniently carried out by batch processing with IrfanView. The software is free for private use and for use in educational institutions (including universities).
-
Making scanned/photographed texts searchable
If you work a lot with analogue texts, you will often be faced with the task of having to digitise these texts in whole or in part for further evaluation. The PDF24 Creator bundles numerous tools for creating and editing PDF documents. This also includes automatic text recognition (OCR) in raster images (e.g. jpg or png).
Tools for project development and management
-
Develop ideas and projects
Paper cards on pinboards are still popular tools for brainstorming. However, the more ideas that come together and need to be structured, the more annoying it becomes to rearrange them. With the software Freemind, mind maps can be created very easily and intuitively on the computer. This way, they can be expanded and restructured again and again as needed.
-
Plan and manage projects
Research projects usually have to be completed by a certain date. It almost always gets very tight towards the end. Solid time and task planning right at the beginning can quickly pay off, especially for larger projects with many participants. ProjectLibre is an open source tool that supports you in planning. Alternatively, LUH members can use the Only Office project management tool as part of the personal cloud services.
Tools for data security
-
Encrypt folders and hard disks
Anyone who works with sensitive data should encrypt it to protect it from unauthorised access. A proven open source programme with which you can encrypt partitions or even entire hard drives relatively easily is VeraCrypt. For the highest security requirements, the programme also offers the option of nesting several encryption algorithms or hiding encrypted hard disk areas. Individual folders can also be encrypted with a packing programme such as WinRAR or 7zip.
