Better File Chunk | 更加强大的文件分块 #3550
Replies: 8 comments 14 replies
-
|
excel可以说是知识库的刚需了，转成html然后借用html分块。 |
Beta Was this translation helpful? Give feedback.
-
|
然而市面上并没有免费的 excel 分块方案。excel 转成 html 我也没找到现成方案。 要么就是用 Unstructed.io ，自己填这个 key。 如果要自己做的话比较麻烦，后续考虑成付费项吧。 |
Beta Was this translation helpful? Give feedback.
-
|
可能引入下SpreadJS？不过这个包似乎太重了。 |
Beta Was this translation helpful? Give feedback.
-
|
SpreadJS 是纯前端组件，而 excel 分块是在服务端做的。另外我看了下 SpreadJS 收费可不便宜： https://developer.mescius.com/spreadjs/pricing ，你确定个人能付得起 $999 就为了这个功能么… 现阶段可行的方案应该就只有 Unstructed.io 了 |
Beta Was this translation helpful? Give feedback.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
-
|
计划支持一波 Lrc/Lrcx，这样就可以做歌词文件的分析了 |
Beta Was this translation helpful? Give feedback.
-
|
可以支持一下typst么？比较新的标记语言，和latex算是竞品。 |
Beta Was this translation helpful? Give feedback.
-
|
能否给出一个示例？以及给出一下你期望的分块的模式？ |
Beta Was this translation helpful? Give feedback.
-
|
示例： == 函数极限与连续
=== 极限定义
$forall xi > 0$, $exists delta > 0$ ,当 $|x - x_(0)| < delta $时, $|f(x) - A| <= 0$ ,则 $limits(lim)_(x -> x_(0)) f(x) = A$
当$x -> 0$ 时, $ln(x + sqrt(1 + x^(2))) ~ x$, $ln(x + sqrt(1 + x^(2)))' = (1)/(sqrt(1 + x^(2)))$
=== 常见的大小关系比较
#block(fill:rgb("#f0f0f0"), inset: 10pt, radius:5pt, width: 100%)[
当 $x -> oo$ 时:
$ln^alpha x << x^beta << a^x$
$ln^alpha n << n^beta << a^n << n! << n^n$
]
#mblock[
$2/pi < sin x < x < tan x < 4/pi$
]
=== 泰勒公式
#block(fill:rgb("#f0f0f0"), inset: 10pt, radius:5pt, width: 100%)[
$sin(x) = x - (x^3)/3! + o(x^3)$
$cos x = 1 - x^2/2! + x^4/4! + o(x^(4))$
$arcsin x = x + x^3/3! + o(x^3)$
$tan x = x + x^3/3 + o(x^3)$
$arctan x = x - x^3/3 + o(x^(3))$
$ln(1 + x) = x - x^2/2 + x^3/3 + o(x^3)$
$e^x = 1 + x + x^2/2! + x^3/3! + o(x^3)$
$(1 + x)^alpha = 1 + alpha x + (alpha (alpha - 1) x^2)/2! + o(x^2)$
$ln(x + sqrt(1 + x^(2))) = x + (1)/(6) x^(3) + o(x^(3))$
]至于分块模式我不是很了解，不知道怎样分比较好。 |
Beta Was this translation helpful? Give feedback.
-
|
java和python有现有的分块方案吗,请求加上 |
Beta Was this translation helpful? Give feedback.
-
|
最简单的就是当成一整个文件来切分，这样啥也不用做就直接能支持。但这样的问题是如果是超长代码块，可能会有太长的问题 |
Beta Was this translation helpful? Give feedback.
-
|
请问Unstructed.io如何配置，项目里有集成 吗？ |
Beta Was this translation helpful? Give feedback.
-
|
Please add ePub support as well to the list of files supported? Most of the books are in ePub or PDF format, also seems like LLM's are better at reading ePub (due to less clutter than PDF's formatting). You also could add Mobi support, but Mobi support is dying since AMZN (it was only AMZN who were supporting it) stopped supporting it and even they have moved onto ePub now. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
I will check it. Thanks for your advice~ |
Beta Was this translation helpful? Give feedback.
-
|
Unstructed.io 怎么使用？ |
Beta Was this translation helpful? Give feedback.
-
背景
在 RAG 中，只有将文件合理分块后，才能做好检索与查询，但是市面上文件类型是非常多的，目前一期只做了一部分的分块支持。
目前支持的分块类型：
纯文本类：
代码类：
富文本类：
表格类：
音频类：
视频类：
如果有对文件类型的分块诉求，请在下面留言，并说明对此类文件的分块设想
Beta Was this translation helpful? Give feedback.
All reactions