Abstract: Input data of a data mining algorithm must conform to a very specific tabular format. Data scientists arrange data into that format by creating long and complex scripts, where different low-level operations are performed, and which can be a time-consuming and error-prone process. To alleviate this situation, we present Lavoisier, a declarative language for data selection and formatting in a data mining context. Using Lavoisier, script size for data preparation can be reduced by 40% on average, and by up to 80% in some cases. Additionally, accidental complexity present in state-of-the-art technologies is considerably mitigated.

Authorship

Download reference