[AI Alliance] Workshop: Preparing High Quality Datasets with Data Prep Kit

Overview When building machine learning and data applications, a significant portion of your time will be dedicated to data wrangling - from content extraction and filtering out problematic and low quality data. In this hands-on session we will explore Data Prep Kit - an open source toolkit, designed to streamline these essential tasks. Attendees will learn first hand how to use the Data Prep Kit to improve overall data quality such as removing spam and low quality documents, removing HAP (Hate Abuse Profanity) speech, removing PII (Personally Identifiable Information) data, thus leading to higher quality dataset.

Description Join us for an interactive, hands-on session where you will learn to clean up data and prepare high quality datasets.

In this workshop we will do the following:

• Extract content from various documents (PDFs, HTML)
• cleanup and remove markups
• Detect and remove SPAM content
• Score and remove low-quality documents
• Identify and remove PII data
• Detect and remove HAP (Hate Abuse Profanity) speech from documents

More about Data Prep Kit : https://github.com/IBM/data-prep-kit

What do you need to participate in this workshop?

• Comfortable in python programming language
• We will run the workshop code using Google Collab (free) - no other setup is needed!

Session Type Workshop (hands-on)

Audience LLM app developers, data scientists, data engineers

Technical Level Beginner - Intermediate

Prerequisites

• Comfortable in python programming language
• We will run the workshop using Google Collab (free to use) - no other setup is needed!

Duration 60 mins

Industry Cross industry

Speaker Bio https://sujee.dev/bio

About the AI Alliance The AI Alliance is an international community of researchers, developers and organizational leaders committed to support and enhance open innovation across the AI technology landscape to accelerate progress, improve safety, security and trust in AI, and maximize benefits to people and society everywhere. Members of the AI Alliance believe that open innovation is essential to develop and achieve safe and responsible AI that benefit society rather than benefit a select few big players.