Audience
Companies wanting a solution to search, quantify and edit data for LLMs
About Lilac
Lilac is an open source tool that enables data and AI practitioners to improve their products by improving their data. Understand your data with powerful search and filtering. Collaborate with your team on a single, centralized dataset. Apply best practices for data curation, like removing duplicates and PII to reduce dataset size and lower training cost and time. See how your pipeline impacts your data using our diff viewer. Clustering is a technique that automatically assigns categories to each document by analyzing the text content and putting similar documents in the same category. This reveals the overarching structure of your dataset. Lilac uses state-of-the-art algorithms and LLMs to cluster the dataset and assign informative, descriptive titles. Before we do advanced searching, like concept or semantic search, we can immediately use keyword search by typing a keyword in the search box.