Custom pre-chunk

[duynt88]

Hi team,

I believe that the main problem of RAG is not a model but it is the chunking method.
There are some fairly good approaches, but they are not perfect for different data domains.

I see that the current LightRAG only support very basic chunking method.
I see some closed threads discussed about using insert_custom_chunks function. However, I only see any API or option on WebUI to use it.

It will helpful if we can add some more chunking method like paragraphs or semantic (use for uploading file directly to WebUI).
And the best option is a way to process the pre-chunking outside before calling API to upload to LightRAG. we can use a Placeholder or use a new API to upload multiple separate chucks with the same filename. Then LightRag will merge into a same file after generating vectors and grab.

So, please consider adding it or guide me if it already available.

Thanks