Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【seg_features】的长度问题 #131

Open
ztang07 opened this issue Jan 12, 2021 · 0 comments
Open

【seg_features】的长度问题 #131

ztang07 opened this issue Jan 12, 2021 · 0 comments

Comments

@ztang07
Copy link

ztang07 commented Jan 12, 2021

image

如图所示,segs的长度会按照句子的字长度进行padding,但是segs基于原句子进行分词,其长度索引并不能直接对应原句子的字。
比如,segs[0]可能对应原句子中的chars[0] + chars[1] + chars[2],这样经过模型的建模之后对应位置的logits好像有影响,decode时,该位置索引的logits从直观上看起来并不直接对应原来这个位置的某个字。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant