You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> **OmniParse is a platform that ingests/parses any unstructured data into structured, actionable data optimized for GenAI (LLM) applcaitons. Whether working with documents, tables, images, videos, audio files, or web pages, OmniParse prepares your data to be clean, structured and ready for AI applications, such as RAG , fine-tuning and more.**
25
+
26
+
### Try it out
27
+
28
+
[](https://colab.research.google.com/github/adithya-s-k/omniparse/blob/main/examples/OmniParse\_GoogleColab.ipynb) Gradio and NextJS interactive demo coming soon
29
+
30
+
### Features
31
+
32
+
✅ Completely local, no external APIs\
33
+
✅ Fits in a T4 GPU\
34
+
✅ Supports 10+ file types\
35
+
✅ Convert documents, multimedia, and web pages to high-quality structured markdown\
36
+
✅ Table extraction, image extraction/captioning, audio/video transcription, web page crawling\
37
+
✅ Easily deployable using Docker and Skypilot\
38
+
✅ Colab friendly
39
+
40
+
#### Problem Statement
41
+
42
+
It's challenging to process data as it comes in different shapes and sizes. OmniParse aims to be an ingestion/parsing platform where you can ingest any type of data, such as documents, images, audio, video, and web content, and get the most structured and actionable output that is GenAI (LLM) friendly.
✅ Convert Documents, Multimedia, Web pages to high-quality structured markdown\
33
-
✅ Table Extraction, Image Extraction/Captioning, Audio/Video Transcription, Web page Crawling\
34
-
✅ Easily Deployable using Docker and Skypilot\
35
-
✅ CPU/GPU compatible\
36
-
✅ Batch processing for handling multiple files at once\
37
-
✅ Comprehensive logging and error handling for robust performance 
50
+
> Note: The server only works on Linux-based systems. This is due to certain dependencies and system-specific configurations that are not compatible with Windows or macOS. To install OmniParse, you can use `pip`:
Client library compatible with Langchain, llamaindex, and haystack integrations coming soon.
138
+
139
+
* API Endpoints
140
+
* Document Parsing
141
+
* Parse Any Document
142
+
* Parse PDF
143
+
* Parse PowerPoint
144
+
* Parse Word Document
145
+
* Media Parsing
146
+
* Parse Any Media
147
+
* Parse Image
148
+
* Process Image
149
+
* Parse Video
150
+
* Parse Audio
151
+
* Website Parsing
152
+
* Parse Website
153
+
154
+
#### Document Parsing
155
+
156
+
**Parse Any Document**
157
+
158
+
Endpoint: `/parse_document` Method: POST
159
+
160
+
Parses PDF, PowerPoint, or Word documents.
161
+
162
+
Curl command:
163
+
164
+
```
165
+
curl -X POST -F "file=@/path/to/document" http://localhost:8000/parse_document
166
+
```
167
+
168
+
**Parse PDF**
169
+
170
+
Endpoint: `/parse_document/pdf` Method: POST
171
+
172
+
Parses PDF documents.
173
+
174
+
Curl command:
175
+
176
+
```
177
+
curl -X POST -F "file=@/path/to/document.pdf" http://localhost:8000/parse_document/pdf
178
+
```
179
+
180
+
**Parse PowerPoint**
181
+
182
+
Endpoint: `/parse_document/ppt` Method: POST
183
+
184
+
Parses PowerPoint presentations.
185
+
186
+
Curl command:
187
+
188
+
```
189
+
curl -X POST -F "file=@/path/to/presentation.ppt" http://localhost:8000/parse_document/ppt
190
+
```
191
+
192
+
**Parse Word Document**
193
+
194
+
Endpoint: `/parse_document/docs` Method: POST
195
+
196
+
Parses Word documents.
197
+
198
+
Curl command:
199
+
200
+
```
201
+
curl -X POST -F "file=@/path/to/document.docx" http://localhost:8000/parse_document/docs
202
+
```
203
+
204
+
#### Media Parsing
205
+
206
+
**Parse Image**
207
+
208
+
Endpoint: `/parse_media/image` Method: POST
209
+
210
+
Parses image files (PNG, JPEG, JPG, TIFF, WEBP).
211
+
212
+
Curl command:
213
+
214
+
```
215
+
curl -X POST -F "file=@/path/to/image.jpg" http://localhost:8000/parse_media/image
216
+
```
217
+
218
+
**Process Image**
219
+
220
+
Endpoint: `/parse_media/process_image` Method: POST
221
+
222
+
Processes an image with a specific task.
223
+
224
+
Possible task inputs: `OCR | OCR with Region | Caption | Detailed Caption | More Detailed Caption | Object Detection | Dense Region Caption | Region Proposal`
*`task`: The processing task (e.g., Caption, Object Detection)
236
+
*`prompt`: Optional prompt for certain tasks
237
+
238
+
**Parse Video**
239
+
240
+
Endpoint: `/parse_media/video` Method: POST
241
+
242
+
Parses video files (MP4, AVI, MOV, MKV).
243
+
244
+
Curl command:
245
+
246
+
```
247
+
curl -X POST -F "file=@/path/to/video.mp4" http://localhost:8000/parse_media/video
248
+
```
249
+
250
+
**Parse Audio**
251
+
252
+
Endpoint: `/parse_media/audio` Method: POST
253
+
254
+
Parses audio files (MP3, WAV, FLAC).
255
+
256
+
Curl command:
257
+
258
+
```
259
+
curl -X POST -F "file=@/path/to/audio.mp3" http://localhost:8000/parse_media/audio
260
+
```
261
+
262
+
#### Website Parsing
263
+
264
+
**Parse Website**
265
+
266
+
Endpoint: `/parse_website` Method: POST
267
+
268
+
Parses a website given its URL.
269
+
270
+
Curl command:
271
+
272
+
```
273
+
curl -X POST -H "Content-Type: application/json" -d '{"url": "https://example.com"}' http://localhost:8000/parse_website
274
+
```
275
+
276
+
Arguments:
277
+
278
+
*`url`: The URL of the website to parse
279
+
280
+
</details>
281
+
282
+
### Coming Soon/ RoadMap
283
+
284
+
🦙 LlamaIndex | Langchain | Haystack integrations coming soon 📚 Batch processing data ⭐ Dynamic chunking and structured data extraction based on specified Schema\
285
+
🛠️ One magic API: just feed in your file prompt what you want, and we will take care of the rest\
286
+
🔧 Dynamic model selection and support for external APIs\
287
+
📄 Batch processing for handling multiple files at once\
288
+
📦 New open-source model to replace Surya OCR and Marker
289
+
290
+
**Final goal**: replace all the different models currently being used with a single MultiModel Model to parse any type of data and get the data you need.
291
+
292
+
### License
293
+
294
+
OmniParse is licensed under the GPL-3.0 license. See `LICENSE` for more information.
295
+
296
+
### Acknowledgements
297
+
298
+
This project builds upon the remarkable [Marker](https://github.com/VikParuchuri/marker) project created by [Vik Paruchuri](https://twitter.com/VikParuchuri). We express our gratitude for the inspiration and foundation provided by this project. Special thanks to [Surya-OCR](https://github.com/VikParuchuri/surya) and [Texify](https://github.com/VikParuchuri/texify) for the OCR models extensively used in this project, and to [Crawl4AI](https://github.com/unclecode/crawl4ai) for their contributions.
299
+
300
+
Models being used:
301
+
302
+
* Surya OCR, Detect, Layout, Order, and Texify
303
+
* Florence-2 base
304
+
* Whisper Small
305
+
306
+
Thank you to the authors for their contributions to these models.
307
+
308
+
***
309
+
310
+
### Contact
311
+
312
+
[](https://adithyask.com)
49
313
314
+
For any inquiries, please contact us at adithyaskolavi@gmail.com
0 commit comments