You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Building prefix dict from the default dictionary ...
Dumping model to file cache C:\Users\AppData\Local\Temp\jieba.cache
Loading model cost 1.309 seconds.
Prefix dict has been built succesfully.
性能问题——加载jieba分词的model需要1s左右
性能指标:在初次打开阶段时间较长,后续逐渐变好,所以这是为啥呢?
——已经定位原因,首次加载jieba分词时loading了1.309s导致的
解决:
jieba 采用延迟加载,import jieba和 jieba.Tokenizer()不会立即触发词典的加载, 一旦有必要才开始加载词典构建前缀字典。如果你想手工初始 jieba,也可以手动初始化。 #!/usr/bin/python # -*- coding: UTF-8 -*- import jieba jieba.initialize()
【Me】https://github.com/Valuebai/
【参考】
1、在分布式环境Spark中关闭jieba延时加载等优化方法 (3):https://blog.csdn.net/macanv/article/details/87860691
2、jieba延迟加载问题解决:https://blog.csdn.net/yjs17125/article/details/81739382
The text was updated successfully, but these errors were encountered: