Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeline Anomaly #9940

Closed
MomentQYC opened this issue Feb 15, 2023 · 15 comments
Closed

Timeline Anomaly #9940

MomentQYC opened this issue Feb 15, 2023 · 15 comments
Labels
⚠️bug? This might be a bug

Comments

@MomentQYC
Copy link
Contributor

💡 Summary

Unable to use "Home" and "Social" timeline.
api/notes/timeline always shows 500 status code.
And also the following error is reported:

	"QueryFailedError: canceling statement due to statement timeout
    at PostgresQueryRunner.query (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async SelectQueryBuilder.loadRawResults (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder…[email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:681:29)
    at async SelectQueryBuilder.getMany (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:747:25)
    at async file:///home/misskey/misskey/packages/backend/built/server/api/endpoints/notes/timeline.js:123:30
    at async ApiCallService.call (file:///home/misskey/misskey/packages/backend/built/server/api/ApiCallService.js:273:16)"', $7 = NULL, $8 = 'user1id', $9 = 'f', $10 = 'public', $11 = '{}', $12 = '{}', $13 = '{}', $14 = '{useri2d}', $15 = '[]', $16 = '{}', $17 = '{}', $18 = 'f', $19 = NULL, $20 = NULL, $21 = 'user2id', $22 = NULL, $23 = NULL, $24 = NULL

🥰 Expected Behavior

Normal use

🤬 Actual Behavior

Unable to use

📝 Steps to Reproduce

Unknown. But there is some circumstantial evidence pointing to a possible relationship with the person being followed.

📌 Environment

Misskey version:13.6.1
Your OS:All
Your browser:Chromium-based browsers

Additional Information:

  1. The problem occurs only after Ver 13.0.0 and is relatively rare.
  2. Users experiencing problems unfollowing one or more people find that the timeline returns to normal (Everyone's situation is different).
  3. The VACUUM FULL command was executed, but it did not work.
  4. There is more than one instance of this problem and it can be considered as a common problem.
@MomentQYC MomentQYC added the ⚠️bug? This might be a bug label Feb 15, 2023
@MomentQYC
Copy link
Contributor Author

MomentQYC commented Feb 15, 2023

This has been proven to be independent of the statement_timeout, as setting a large value will still cause the problem.
I knew Misskey itself would set that value, so I modified it.
System occupancy is normal.

@MomentQYC
Copy link
Contributor Author

BTW,api/notes/hybrid-timeline can also be abnormal

@yitsushi
Copy link
Contributor

yitsushi commented Mar 9, 2023

Are there any updates on this issue? My timeline shows me this error since 13.9. Usually this one: https://bin.efertone.me/?4aa8eeee3018263e#4mMn5XYwy9w4dJY6FAtbUNX3PTdMd3mTWW5yufAfAUD7

That query is pretty sus. Before 13.9 I did not see this issue.

@catb00mer
Copy link

Looking for updates, also. This is still a problem for me 😢

On my end, all of the timeline endpoints respond with 500.

Unfollowing a few accounts fixes the issue for a little while. But there's no clear correlation between what accounts I unfollow, and whether it continues working 🤕

@yitsushi
Copy link
Contributor

For me the 13.10 update looks like it's working, at least I did not see 500s for a while now.

@yitsushi
Copy link
Contributor

I would like to raise this issue, on the home timeline it's resolved for me, but the search functionality is totally broken. Even after I increased the statement_timeout to 30000 (30s), after 30s it hits timeout.

Apr 16 11:12:04 slippy misskey[3487118]: {
Apr 16 11:12:04 slippy misskey[3487118]:   ep: 'notes/search',
Apr 16 11:12:04 slippy misskey[3487118]:   ps: {
Apr 16 11:12:04 slippy misskey[3487118]:     query: 'no waste',
Apr 16 11:12:04 slippy misskey[3487118]:     limit: 10,
Apr 16 11:12:04 slippy misskey[3487118]:     i: 'xxxxxxxxxx',
Apr 16 11:12:04 slippy misskey[3487118]:     offset: 0,
Apr 16 11:12:04 slippy misskey[3487118]:     userId: null,
Apr 16 11:12:04 slippy misskey[3487118]:     channelId: null
Apr 16 11:12:04 slippy misskey[3487118]:   },
Apr 16 11:12:04 slippy misskey[3487118]:   e: {
Apr 16 11:12:04 slippy misskey[3487118]:     message: 'canceling statement due to statement timeout',
Apr 16 11:12:04 slippy misskey[3487118]:     code: 'QueryFailedError',
Apr 16 11:12:04 slippy misskey[3487118]:     stack: 'QueryFailedError: canceling statement due to statement timeout\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at PostgresQueryRunner.query (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typ
eorm/driver/postgres/PostgresQueryRunner.js:211:19)\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at async SelectQueryBuilder.loadRawResults (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/no
de_modules/typeorm/query-builder/SelectQueryBuilder.js:2162:25)\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at async SelectQueryBuilder.getRawMany (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_m
odules/typeorm/query-builder/SelectQueryBuilder.js:644:29)\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at async SelectQueryBuilder.executeEntitiesAndRawResults (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected].
[email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:1958:26)\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at async SelectQueryBuilder.getRawAndEntities (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]
/node_modules/typeorm/query-builder/SelectQueryBuilder.js:682:29)\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at async SelectQueryBuilder.getMany (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modu
les/typeorm/query-builder/SelectQueryBuilder.js:748:25)\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at async file:///home/misskey/misskey/packages/backend/built/server/api/endpoints/notes/search.js:129:27\n' +
Apr 16 11:12:04 slippy misskey[3487118]:       '    at async ApiCallService.call (file:///home/misskey/misskey/packages/backend/built/server/api/ApiCallService.js:275:16)',
Apr 16 11:12:04 slippy misskey[3487118]:     id: '58f07687-4d49-4a6f-ac42-9a3a87261554'
Apr 16 11:12:04 slippy misskey[3487118]:   }
Apr 16 11:12:04 slippy misskey[3487118]: }
Apr 16 11:12:04 slippy misskey[3487118]: QueryFailedError: canceling statement due to statement timeout
Apr 16 11:12:04 slippy misskey[3487118]:     at PostgresQueryRunner.query (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/dr
iver/postgres/PostgresQueryRunner.js:211:19)
Apr 16 11:12:04 slippy misskey[3487118]:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Apr 16 11:12:04 slippy misskey[3487118]:     at async SelectQueryBuilder.loadRawResults (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modu
les/typeorm/query-builder/SelectQueryBuilder.js:2162:25)
Apr 16 11:12:04 slippy misskey[3487118]:     at async SelectQueryBuilder.getRawMany (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/
typeorm/query-builder/SelectQueryBuilder.js:644:29)
Apr 16 11:12:04 slippy misskey[3487118]:     at async SelectQueryBuilder.executeEntitiesAndRawResults (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected]_pg@8.
10.0/node_modules/typeorm/query-builder/SelectQueryBuilder.js:1958:26)
Apr 16 11:12:04 slippy misskey[3487118]:     at async SelectQueryBuilder.getRawAndEntities (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_m
odules/typeorm/query-builder/SelectQueryBuilder.js:682:29)
Apr 16 11:12:04 slippy misskey[3487118]:     at async SelectQueryBuilder.getMany (/home/misskey/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typ
eorm/query-builder/SelectQueryBuilder.js:748:25)
Apr 16 11:12:04 slippy misskey[3487118]:     at async file:///home/misskey/misskey/packages/backend/built/server/api/endpoints/notes/search.js:129:27
Apr 16 11:12:04 slippy misskey[3487118]:     at async ApiCallService.call (file:///home/misskey/misskey/packages/backend/built/server/api/ApiCallService.js:275:16) {
Apr 16 11:12:04 slippy misskey[3487118]:   query: `SELECT DISTINCT "distinctAlias"."note_id" AS "ids_note_id", "distinctAlias"."note_id" FROM (SELECT "note"."id" AS "note_id", "no
te"."createdAt" AS "note_createdAt", "note"."replyId" AS "note_replyId", "note"."renoteId" AS "note_renoteId", "note"."threadId" AS "note_threadId", "note"."text" AS "note_text",
"note"."name" AS "note_name", "note"."cw" AS "note_cw", "note"."userId" AS "note_userId", "note"."localOnly" AS "note_localOnly", "note"."reactionAcceptance" AS "note_reactionAcce
ptance", "note"."renoteCount" AS "note_renoteCount", "note"."repliesCount" AS "note_repliesCount", "note"."reactions" AS "note_reactions", "note"."visibility" AS "note_visibility"
, "note"."uri" AS "note_uri", "note"."url" AS "note_url", "note"."fileIds" AS "note_fileIds", "note"."attachedFileTypes" AS "note_attachedFileTypes", "note"."visibleUserIds" AS "n
ote_visibleUserIds", "note"."mentions" AS "note_mentions", "note"."mentionedRemoteUsers" AS "note_mentionedRemoteUsers", "note"."emojis" AS "note_emojis", "note"."tags" AS "note_t
ags", "note"."hasPoll" AS "note_hasPoll", "note"."channelId" AS "note_channelId", "note"."userHost" AS "note_userHost", "note"."replyUserId" AS "note_replyUserId", "note"."replyUs
erHost" AS "note_replyUserHost", "note"."renoteUserId" AS "note_renoteUserId", "note"."renoteUserHost" AS "note_renoteUserHost", "user"."id" AS "user_id", "user"."createdAt" AS "u
ser_createdAt", "user"."updatedAt" AS "user_updatedAt", "user"."lastFetchedAt" AS "user_lastFetchedAt", "user"."lastActiveDate" AS "user_lastActiveDate", "user"."hideOnlineStatus"
 AS "user_hideOnlineStatus", "user"."username" AS "user_username", "user"."name" AS "user_name", "user"."followersCount" AS "user_followersCount", "user"."followingCount" AS "user
_followingCount", "user"."movedToUri" AS "user_movedToUri", "user"."alsoKnownAs" AS "user_alsoKnownAs", "user"."notesCount" AS "user_notesCount", "user"."avatarId" AS "user_avatar
Id", "user"."bannerId" AS "user_bannerId", "user"."avatarUrl" AS "user_avatarUrl", "user"."bannerUrl" AS "user_bannerUrl", "user"."avatarBlurhash" AS "user_avatarBlurhash", "user"
."bannerBlurhash" AS "user_bannerBlurhash", "user"."tags" AS "user_tags", "user"."isSuspended" AS "user_isSuspended", "user"."isLocked" AS "user_isLocked", "user"."isBot" AS "user
_isBot", "user"."isCat" AS "user_isCat", "user"."isRoot" AS "user_isRoot", "user"."isExplorable" AS "user_isExplorable", "user"."isDeleted" AS "user_isDeleted", "user"."emojis" AS
 "user_emojis", "user"."host" AS "user_host", "user"."inbox" AS "user_inbox", "user"."sharedInbox" AS "user_sharedInbox", "user"."featured" AS "user_featured", "user"."uri" AS "us
er_uri", "user"."followersUri" AS "user_followersUri", "user"."showTimelineReplies" AS "user_showTimelineReplies", "user"."token" AS "user_token", "reply"."id" AS "reply_id", "rep
ly"."createdAt" AS "reply_createdAt", "reply"."replyId" AS "reply_replyId", "reply"."renoteId" AS "reply_renoteId", "reply"."threadId" AS "reply_threadId", "reply"."text" AS "repl
y_text", "reply"."name" AS "reply_name", "reply"."cw" AS "reply_cw", "reply"."userId" AS "reply_userId", "reply"."localOnly" AS "reply_localOnly", "reply"."reactionAcceptance" AS
"reply_reactionAcceptance", "reply"."renoteCount" AS "reply_renoteCount", "reply"."repliesCount" AS "reply_repliesCount", "reply"."reactions" AS "reply_reactions", "reply"."visibi
lity" AS "reply_visibility", "reply"."uri" AS "reply_uri", "reply"."url" AS "reply_url", "reply"."fileIds" AS "reply_fileIds", "reply"."attachedFileTypes" AS "reply_attachedFileTy
pes", "reply"."visibleUserIds" AS "reply_visibleUserIds", "reply"."mentions" AS "reply_mentions", "reply"."mentionedRemoteUsers" AS "reply_mentionedRemoteUsers", "reply"."emojis"
AS "reply_emojis", "reply"."tags" AS "reply_tags", "reply"."hasPoll" AS "reply_hasPoll", "reply"."channelId" AS "reply_channelId", "reply"."userHost" AS "reply_userHost", "reply".
"replyUserId" AS "reply_replyUserId", "reply"."replyUserHost" AS "reply_replyUserHost", "reply"."renoteUserId" AS "reply_renoteUserId", "reply"."renoteUserHost" AS "reply_renoteUs
erHost", "renote"."id" AS "renote_id", "renote"."createdAt" AS "renote_createdAt", "renote"."replyId" AS "renote_replyId", "renote"."renoteId" AS "renote_renoteId", "renote"."thre
adId" AS "renote_threadId", "renote"."text" AS "renote_text", "renote"."name" AS "renote_name", "renote"."cw" AS "renote_cw", "renote"."userId" AS "renote_userId", "renote"."local
Only" AS "renote_localOnly", "renote"."reactionAcceptance" AS "renote_reactionAcceptance", "renote"."renoteCount" AS "renote_renoteCount", "renote"."repliesCount" AS "renote_repli
esCount", "renote"."reactions" AS "renote_reactions", "renote"."visibility" AS "renote_visibility", "renote"."uri" AS "renote_uri", "renote"."url" AS "renote_url", "renote"."fileI
ds" AS "renote_fileIds", "renote"."attachedFileTypes" AS "renote_attachedFileTypes", "renote"."visibleUserIds" AS "renote_visibleUserIds", "renote"."mentions" AS "renote_mentions"
, "renote"."mentionedRemoteUsers" AS "renote_mentionedRemoteUsers", "renote"."emojis" AS "renote_emojis", "renote"."tags" AS "renote_tags", "renote"."hasPoll" AS "renote_hasPoll",
 "renote"."channelId" AS "renote_channelId", "renote"."userHost" AS "renote_userHost", "renote"."replyUserId" AS "renote_replyUserId", "renote"."replyUserHost" AS "renote_replyUse
rHost", "renote"."renoteUserId" AS "renote_renoteUserId", "renote"."renoteUserHost" AS "renote_renoteUserHost", "replyUser"."id" AS "replyUser_id", "replyUser"."createdAt" AS "rep
lyUser_createdAt", "replyUser"."updatedAt" AS "replyUser_updatedAt", "replyUser"."lastFetchedAt" AS "replyUser_lastFetchedAt", "replyUser"."lastActiveDate" AS "replyUser_lastActiv
eDate", "replyUser"."hideOnlineStatus" AS "replyUser_hideOnlineStatus", "replyUser"."username" AS "replyUser_username", "replyUser"."name" AS "replyUser_name", "replyUser"."follow
ersCount" AS "replyUser_followersCount", "replyUser"."followingCount" AS "replyUser_followingCount", "replyUser"."movedToUri" AS "replyUser_movedToUri", "replyUser"."alsoKnownAs"
AS "replyUser_alsoKnownAs", "replyUser"."notesCount" AS "replyUser_notesCount", "replyUser"."avatarId" AS "replyUser_avatarId", "replyUser"."bannerId" AS "replyUser_bannerId", "re
plyUser"."avatarUrl" AS "replyUser_avatarUrl", "replyUser"."bannerUrl" AS "replyUser_bannerUrl", "replyUser"."avatarBlurhash" AS "replyUser_avatarBlurhash", "replyUser"."bannerBlu
rhash" AS "replyUser_bannerBlurhash", "replyUser"."tags" AS "replyUser_tags", "replyUser"."isSuspended" AS "replyUser_isSuspended", "replyUser"."isLocked" AS "replyUser_isLocked",
 "replyUser"."isBot" AS "replyUser_isBot", "replyUser"."isCat" AS "replyUser_isCat", "replyUser"."isRoot" AS "replyUser_isRoot", "replyUser"."isExplorable" AS "replyUser_isExplora
ble", "replyUser"."isDeleted" AS "replyUser_isDeleted", "replyUser"."emojis" AS "replyUser_emojis", "replyUser"."host" AS "replyUser_host", "replyUser"."inbox" AS "replyUser_inbox
", "replyUser"."sharedInbox" AS "replyUser_sharedInbox", "replyUser"."featured" AS "replyUser_featured", "replyUser"."uri" AS "replyUser_uri", "replyUser"."followersUri" AS "reply
User_followersUri", "replyUser"."showTimelineReplies" AS "replyUser_showTimelineReplies", "replyUser"."token" AS "replyUser_token", "renoteUser"."id" AS "renoteUser_id", "renoteUs
er"."createdAt" AS "renoteUser_createdAt", "renoteUser"."updatedAt" AS "renoteUser_updatedAt", "renoteUser"."lastFetchedAt" AS "renoteUser_lastFetchedAt", "renoteUser"."lastActive
Date" AS "renoteUser_lastActiveDate", "renoteUser"."hideOnlineStatus" AS "renoteUser_hideOnlineStatus", "renoteUser"."username" AS "renoteUser_username", "renoteUser"."name" AS "r
enoteUser_name", "renoteUser"."followersCount" AS "renoteUser_followersCount", "renoteUser"."followingCount" AS "renoteUser_followingCount", "renoteUser"."movedToUri" AS "renoteUs
er_movedToUri", "renoteUser"."alsoKnownAs" AS "renoteUser_alsoKnownAs", "renoteUser"."notesCount" AS "renoteUser_notesCount", "renoteUser"."avatarId" AS "renoteUser_avatarId", "re
noteUser"."bannerId" AS "renoteUser_bannerId", "renoteUser"."avatarUrl" AS "renoteUser_avatarUrl", "renoteUser"."bannerUrl" AS "renoteUser_bannerUrl", "renoteUser"."avatarBlurhash
" AS "renoteUser_avatarBlurhash", "renoteUser"."bannerBlurhash" AS "renoteUser_bannerBlurhash", "renoteUser"."tags" AS "renoteUser_tags", "renoteUser"."isSuspended" AS "renoteUser
_isSuspended", "renoteUser"."isLocked" AS "renoteUser_isLocked", "renoteUser"."isBot" AS "renoteUser_isBot", "renoteUser"."isCat" AS "renoteUser_isCat", "renoteUser"."isRoot" AS "
renoteUser_isRoot", "renoteUser"."isExplorable" AS "renoteUser_isExplorable", "renoteUser"."isDeleted" AS "renoteUser_isDeleted", "renoteUser"."emojis" AS "renoteUser_emojis", "re
noteUser"."host" AS "renoteUser_host", "renoteUser"."inbox" AS "renoteUser_inbox", "renoteUser"."sharedInbox" AS "renoteUser_sharedInbox", "renoteUser"."featured" AS "renoteUser_f
eatured", "renoteUser"."uri" AS "renoteUser_uri", "renoteUser"."followersUri" AS "renoteUser_followersUri", "renoteUser"."showTimelineReplies" AS "renoteUser_showTimelineReplies",
 "renoteUser"."token" AS "renoteUser_token" FROM "note" "note" INNER JOIN "user" "user" ON "user"."id"="note"."userId"  LEFT JOIN "note" "reply" ON "reply"."id"="note"."replyId"
LEFT JOIN "note" "renote" ON "renote"."id"="note"."renoteId"  LEFT JOIN "user" "replyUser" ON "replyUser"."id"="reply"."userId"  LEFT JOIN "user" "renoteUser" ON "renoteUser"."id"
="renote"."userId" WHERE "note"."text" ILIKE $1 AND (("note"."visibility" = 'public' OR "note"."visibility" = 'home') OR "note"."userId" = $2 OR $3 = ANY("note"."visibleUserIds")
OR $4 = ANY("note"."mentions") OR ("note"."visibility" = 'followers' AND ("note"."userId" IN (SELECT "following"."followeeId" AS "following_followeeId" FROM "following" "following
" WHERE "following"."followerId" = $5) OR "note"."replyUserId" = $6))) AND "note"."userId" NOT IN (SELECT "muting"."muteeId" AS "muting_muteeId" FROM "muting" "muting" WHERE "muti
ng"."muterId" = $7) AND ("n`... 1585 more characters,
Apr 16 11:12:04 slippy misskey[3487118]:   parameters: [
Apr 16 11:12:04 slippy misskey[3487118]:     '%no waste%', '83sv4lyx22',
Apr 16 11:12:04 slippy misskey[3487118]:     '83sv4lyx22', '83sv4lyx22',
Apr 16 11:12:04 slippy misskey[3487118]:     '83sv4lyx22', '83sv4lyx22',
Apr 16 11:12:04 slippy misskey[3487118]:     '83sv4lyx22', '83sv4lyx22',
Apr 16 11:12:04 slippy misskey[3487118]:     '83sv4lyx22', '83sv4lyx22',
Apr 16 11:12:04 slippy misskey[3487118]:     '83sv4lyx22', '83sv4lyx22',
Apr 16 11:12:04 slippy misskey[3487118]:     '83sv4lyx22', '83sv4lyx22',
Apr 16 11:12:04 slippy misskey[3487118]:     '83sv4lyx22'
Apr 16 11:12:04 slippy misskey[3487118]:   ],
Apr 16 11:12:04 slippy misskey[3487118]:   driverError: error: canceling statement due to statement timeout
Apr 16 11:12:04 slippy misskey[3487118]:       at Parser.parseErrorMessage (/home/misskey/misskey/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:287:
98)
Apr 16 11:12:04 slippy misskey[3487118]:       at Parser.handlePacket (/home/misskey/misskey/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:126:29)
Apr 16 11:12:04 slippy misskey[3487118]:       at Parser.parse (/home/misskey/misskey/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:39:38)
Apr 16 11:12:04 slippy misskey[3487118]:       at Socket.<anonymous> (/home/misskey/misskey/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/index.js:11:42)
Apr 16 11:12:04 slippy misskey[3487118]:       at Socket.emit (node:events:512:28)
Apr 16 11:12:04 slippy misskey[3487118]:       at addChunk (node:internal/streams/readable:324:12)
Apr 16 11:12:04 slippy misskey[3487118]:       at readableAddChunk (node:internal/streams/readable:297:9)
Apr 16 11:12:04 slippy misskey[3487118]:       at Readable.push (node:internal/streams/readable:234:10)
Apr 16 11:12:04 slippy misskey[3487118]:       at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
Apr 16 11:12:04 slippy misskey[3487118]:     length: 109,
Apr 16 11:12:04 slippy misskey[3487118]:     severity: 'ERROR',
Apr 16 11:12:04 slippy misskey[3487118]:     code: '57014',
Apr 16 11:12:04 slippy misskey[3487118]:     detail: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     hint: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     position: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     internalPosition: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     internalQuery: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     where: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     schema: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     table: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     column: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     dataType: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     constraint: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:     file: 'postgres.c',
Apr 16 11:12:04 slippy misskey[3487118]:     line: '3312',
Apr 16 11:12:04 slippy misskey[3487118]:     routine: 'ProcessInterrupts'
Apr 16 11:12:04 slippy misskey[3487118]:   },
Apr 16 11:12:04 slippy misskey[3487118]:   length: 109,
Apr 16 11:12:04 slippy misskey[3487118]:   severity: 'ERROR',
Apr 16 11:12:04 slippy misskey[3487118]:   code: '57014',
Apr 16 11:12:04 slippy misskey[3487118]:   detail: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   hint: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   position: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   internalPosition: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   internalQuery: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   where: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   schema: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   table: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   column: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   dataType: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   constraint: undefined,
Apr 16 11:12:04 slippy misskey[3487118]:   file: 'postgres.c',
Apr 16 11:12:04 slippy misskey[3487118]:   line: '3312',
Apr 16 11:12:04 slippy misskey[3487118]:   routine: 'ProcessInterrupts'
Apr 16 11:12:04 slippy misskey[3487118]: } 58f07687-4d49-4a6f-ac42-9a3a87261554

From an older release:

Elasticsearch support has been removed
Instead, we envision a mechanism that allows you to set up an arbitrary search provider in the future. With this mechanism, Elasticsearch will still be available.

I couldn't figure out if any had been implemented and how to configure it.

@yitsushi
Copy link
Contributor

yitsushi commented Apr 16, 2023

Manually added an index (with the trgm extension) and now it's blazingly fast (from 15s to 0.227s on pure like query)

CREATE INDEX "IDX_NOTE_TEXT" ON "note" USING gin ("text" gin_trgm_ops);

Full walkthrough: https://slippy.xyz/notes/9dmwuyvuue

@creamlike1024
Copy link

Similar issue, but unable to open the mention notification

misskey log:

misskey-web-1  | ERR  * [api]   Internal error occurred in notes/mentions: canceling statement due to statement timeout
misskey-web-1  | {
misskey-web-1  |   ep: 'notes/mentions',
misskey-web-1  |   ps: { limit: 10, i: 'ZQ11t1erfoT09ZjF', following: false },
misskey-web-1  |   e: {
misskey-web-1  |     message: 'canceling statement due to statement timeout',
misskey-web-1  |     code: 'QueryFailedError',
misskey-web-1  |     stack: 'QueryFailedError: canceling statement due to statement timeout\n' +
misskey-web-1  |       '    at PostgresQueryRunner.query (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19)\n' +
misskey-web-1  |       '    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' +
misskey-web-1  |       '    at async SelectQueryBuilder.loadRawResults (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:2183:25)\n' +
misskey-web-1  |       '    at async SelectQueryBuilder.getRawMany (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:646:29)\n' +
misskey-web-1  |       '    at async SelectQueryBuilder.executeEntitiesAndRawResults (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:1973:26)\n' +
misskey-web-1  |       '    at async SelectQueryBuilder.getRawAndEntities (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:684:29)\n' +
misskey-web-1  |       '    at async SelectQueryBuilder.getMany (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:750:25)\n' +
misskey-web-1  |       '    at async file:///misskey/packages/backend/built/server/api/endpoints/notes/mentions.js:103:30\n' +
misskey-web-1  |       '    at async ApiCallService.call (file:///misskey/packages/backend/built/server/api/ApiCallService.js:285:16)',
misskey-web-1  |     id: 'ea787c66-18dd-4bdd-850b-daa6bfe8f0b5'
misskey-web-1  |   }
misskey-web-1  | }
misskey-web-1  | QueryFailedError: canceling statement due to statement timeout
misskey-web-1  |     at PostgresQueryRunner.query (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19)
misskey-web-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
misskey-web-1  |     at async SelectQueryBuilder.loadRawResults (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:2183:25)
misskey-web-1  |     at async SelectQueryBuilder.getRawMany (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:646:29)
misskey-web-1  |     at async SelectQueryBuilder.executeEntitiesAndRawResults (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:1973:26)
misskey-web-1  |     at async SelectQueryBuilder.getRawAndEntities (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:684:29)
misskey-web-1  |     at async SelectQueryBuilder.getMany (/misskey/node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/typeorm/query-builder/SelectQueryBuilder.js:750:25)
misskey-web-1  |     at async file:///misskey/packages/backend/built/server/api/endpoints/notes/mentions.js:103:30
misskey-web-1  |     at async ApiCallService.call (file:///misskey/packages/backend/built/server/api/ApiCallService.js:285:16) {
misskey-web-1  |   query: `SELECT DISTINCT "distinctAlias"."note_id" AS "ids_note_id", "distinctAlias"."note_id" FROM (SELECT "note"."id" AS "note_id", "note"."createdAt" AS "note_createdAt", "note"."replyId" AS "note_replyId", "note"."renoteId" AS "note_renoteId", "note"."threadId" AS "note_threadId", "note"."text" AS "note_text", "note"."name" AS "note_name", "note"."cw" AS "note_cw", "note"."userId" AS "note_userId", "note"."localOnly" AS "note_localOnly", "note"."reactionAcceptance" AS "note_reactionAcceptance", "note"."renoteCount" AS "note_renoteCount", "note"."repliesCount" AS "note_repliesCount", "note"."reactions" AS "note_reactions", "note"."visibility" AS "note_visibility", "note"."uri" AS "note_uri", "note"."url" AS "note_url", "note"."fileIds" AS "note_fileIds", "note"."attachedFileTypes" AS "note_attachedFileTypes", "note"."visibleUserIds" AS "note_visibleUserIds", "note"."mentions" AS "note_mentions", "note"."mentionedRemoteUsers" AS "note_mentionedRemoteUsers", "note"."emojis" AS "note_emojis", "note"."tags" AS "note_tags", "note"."hasPoll" AS "note_hasPoll", "note"."channelId" AS "note_channelId", "note"."userHost" AS "note_userHost", "note"."replyUserId" AS "note_replyUserId", "note"."replyUserHost" AS "note_replyUserHost", "note"."renoteUserId" AS "note_renoteUserId", "note"."renoteUserHost" AS "note_renoteUserHost", "user"."id" AS "user_id", "user"."createdAt" AS "user_createdAt", "user"."updatedAt" AS "user_updatedAt", "user"."lastFetchedAt" AS "user_lastFetchedAt", "user"."lastActiveDate" AS "user_lastActiveDate", "user"."hideOnlineStatus" AS "user_hideOnlineStatus", "user"."username" AS "user_username", "user"."name" AS "user_name", "user"."followersCount" AS "user_followersCount", "user"."followingCount" AS "user_followingCount", "user"."movedToUri" AS "user_movedToUri", "user"."movedAt" AS "user_movedAt", "user"."alsoKnownAs" AS "user_alsoKnownAs", "user"."notesCount" AS "user_notesCount", "user"."avatarId" AS "user_avatarId", "user"."bannerId" AS "user_bannerId", "user"."avatarUrl" AS "user_avatarUrl", "user"."bannerUrl" AS "user_bannerUrl", "user"."avatarBlurhash" AS "user_avatarBlurhash", "user"."bannerBlurhash" AS "user_bannerBlurhash", "user"."tags" AS "user_tags", "user"."isSuspended" AS "user_isSuspended", "user"."isLocked" AS "user_isLocked", "user"."isBot" AS "user_isBot", "user"."isCat" AS "user_isCat", "user"."isRoot" AS "user_isRoot", "user"."isExplorable" AS "user_isExplorable", "user"."isDeleted" AS "user_isDeleted", "user"."emojis" AS "user_emojis", "user"."host" AS "user_host", "user"."inbox" AS "user_inbox", "user"."sharedInbox" AS "user_sharedInbox", "user"."featured" AS "user_featured", "user"."uri" AS "user_uri", "user"."followersUri" AS "user_followersUri", "user"."showTimelineReplies" AS "user_showTimelineReplies", "user"."token" AS "user_token", "reply"."id" AS "reply_id", "reply"."createdAt" AS "reply_createdAt", "reply"."replyId" AS "reply_replyId", "reply"."renoteId" AS "reply_renoteId", "reply"."threadId" AS "reply_threadId", "reply"."text" AS "reply_text", "reply"."name" AS "reply_name", "reply"."cw" AS "reply_cw", "reply"."userId" AS "reply_userId", "reply"."localOnly" AS "reply_localOnly", "reply"."reactionAcceptance" AS "reply_reactionAcceptance", "reply"."renoteCount" AS "reply_renoteCount", "reply"."repliesCount" AS "reply_repliesCount", "reply"."reactions" AS "reply_reactions", "reply"."visibility" AS "reply_visibility", "reply"."uri" AS "reply_uri", "reply"."url" AS "reply_url", "reply"."fileIds" AS "reply_fileIds", "reply"."attachedFileTypes" AS "reply_attachedFileTypes", "reply"."visibleUserIds" AS "reply_visibleUserIds", "reply"."mentions" AS "reply_mentions", "reply"."mentionedRemoteUsers" AS "reply_mentionedRemoteUsers", "reply"."emojis" AS "reply_emojis", "reply"."tags" AS "reply_tags", "reply"."hasPoll" AS "reply_hasPoll", "reply"."channelId" AS "reply_channelId", "reply"."userHost" AS "reply_userHost", "reply"."replyUserId" AS "reply_replyUserId", "reply"."replyUserHost" AS "reply_replyUserHost", "reply"."renoteUserId" AS "reply_renoteUserId", "reply"."renoteUserHost" AS "reply_renoteUserHost", "renote"."id" AS "renote_id", "renote"."createdAt" AS "renote_createdAt", "renote"."replyId" AS "renote_replyId", "renote"."renoteId" AS "renote_renoteId", "renote"."threadId" AS "renote_threadId", "renote"."text" AS "renote_text", "renote"."name" AS "renote_name", "renote"."cw" AS "renote_cw", "renote"."userId" AS "renote_userId", "renote"."localOnly" AS "renote_localOnly", "renote"."reactionAcceptance" AS "renote_reactionAcceptance", "renote"."renoteCount" AS "renote_renoteCount", "renote"."repliesCount" AS "renote_repliesCount", "renote"."reactions" AS "renote_reactions", "renote"."visibility" AS "renote_visibility", "renote"."uri" AS "renote_uri", "renote"."url" AS "renote_url", "renote"."fileIds" AS "renote_fileIds", "renote"."attachedFileTypes" AS "renote_attachedFileTypes", "renote"."visibleUserIds" AS "renote_visibleUserIds", "renote"."mentions" AS "renote_mentions", "renote"."mentionedRemoteUsers" AS "renote_mentionedRemoteUsers", "renote"."emojis" AS "renote_emojis", "renote"."tags" AS "renote_tags", "renote"."hasPoll" AS "renote_hasPoll", "renote"."channelId" AS "renote_channelId", "renote"."userHost" AS "renote_userHost", "renote"."replyUserId" AS "renote_replyUserId", "renote"."replyUserHost" AS "renote_replyUserHost", "renote"."renoteUserId" AS "renote_renoteUserId", "renote"."renoteUserHost" AS "renote_renoteUserHost", "replyUser"."id" AS "replyUser_id", "replyUser"."createdAt" AS "replyUser_createdAt", "replyUser"."updatedAt" AS "replyUser_updatedAt", "replyUser"."lastFetchedAt" AS "replyUser_lastFetchedAt", "replyUser"."lastActiveDate" AS "replyUser_lastActiveDate", "replyUser"."hideOnlineStatus" AS "replyUser_hideOnlineStatus", "replyUser"."username" AS "replyUser_username", "replyUser"."name" AS "replyUser_name", "replyUser"."followersCount" AS "replyUser_followersCount", "replyUser"."followingCount" AS "replyUser_followingCount", "replyUser"."movedToUri" AS "replyUser_movedToUri", "replyUser"."movedAt" AS "replyUser_movedAt", "replyUser"."alsoKnownAs" AS "replyUser_alsoKnownAs", "replyUser"."notesCount" AS "replyUser_notesCount", "replyUser"."avatarId" AS "replyUser_avatarId", "replyUser"."bannerId" AS "replyUser_bannerId", "replyUser"."avatarUrl" AS "replyUser_avatarUrl", "replyUser"."bannerUrl" AS "replyUser_bannerUrl", "replyUser"."avatarBlurhash" AS "replyUser_avatarBlurhash", "replyUser"."bannerBlurhash" AS "replyUser_bannerBlurhash", "replyUser"."tags" AS "replyUser_tags", "replyUser"."isSuspended" AS "replyUser_isSuspended", "replyUser"."isLocked" AS "replyUser_isLocked", "replyUser"."isBot" AS "replyUser_isBot", "replyUser"."isCat" AS "replyUser_isCat", "replyUser"."isRoot" AS "replyUser_isRoot", "replyUser"."isExplorable" AS "replyUser_isExplorable", "replyUser"."isDeleted" AS "replyUser_isDeleted", "replyUser"."emojis" AS "replyUser_emojis", "replyUser"."host" AS "replyUser_host", "replyUser"."inbox" AS "replyUser_inbox", "replyUser"."sharedInbox" AS "replyUser_sharedInbox", "replyUser"."featured" AS "replyUser_featured", "replyUser"."uri" AS "replyUser_uri", "replyUser"."followersUri" AS "replyUser_followersUri", "replyUser"."showTimelineReplies" AS "replyUser_showTimelineReplies", "replyUser"."token" AS "replyUser_token", "renoteUser"."id" AS "renoteUser_id", "renoteUser"."createdAt" AS "renoteUser_createdAt", "renoteUser"."updatedAt" AS "renoteUser_updatedAt", "renoteUser"."lastFetchedAt" AS "renoteUser_lastFetchedAt", "renoteUser"."lastActiveDate" AS "renoteUser_lastActiveDate", "renoteUser"."hideOnlineStatus" AS "renoteUser_hideOnlineStatus", "renoteUser"."username" AS "renoteUser_username", "renoteUser"."name" AS "renoteUser_name", "renoteUser"."followersCount" AS "renoteUser_followersCount", "renoteUser"."followingCount" AS "renoteUser_followingCount", "renoteUser"."movedToUri" AS "renoteUser_movedToUri", "renoteUser"."movedAt" AS "renoteUser_movedAt", "renoteUser"."alsoKnownAs" AS "renoteUser_alsoKnownAs", "renoteUser"."notesCount" AS "renoteUser_notesCount", "renoteUser"."avatarId" AS "renoteUser_avatarId", "renoteUser"."bannerId" AS "renoteUser_bannerId", "renoteUser"."avatarUrl" AS "renoteUser_avatarUrl", "renoteUser"."bannerUrl" AS "renoteUser_bannerUrl", "renoteUser"."avatarBlurhash" AS "renoteUser_avatarBlurhash", "renoteUser"."bannerBlurhash" AS "renoteUser_bannerBlurhash", "renoteUser"."tags" AS "renoteUser_tags", "renoteUser"."isSuspended" AS "renoteUser_isSuspended", "renoteUser"."isLocked" AS "renoteUser_isLocked", "renoteUser"."isBot" AS "renoteUser_isBot", "renoteUser"."isCat" AS "renoteUser_isCat", "renoteUser"."isRoot" AS "renoteUser_isRoot", "renoteUser"."isExplorable" AS "renoteUser_isExplorable", "renoteUser"."isDeleted" AS "renoteUser_isDeleted", "renoteUser"."emojis" AS "renoteUser_emojis", "renoteUser"."host" AS "renoteUser_host", "renoteUser"."inbox" AS "renoteUser_inbox", "renoteUser"."sharedInbox" AS "renoteUser_sharedInbox", "renoteUser"."featured" AS "renoteUser_featured", "renoteUser"."uri" AS "renoteUser_uri", "renoteUser"."followersUri" AS "renoteUser_followersUri", "renoteUser"."showTimelineReplies" AS "renoteUser_showTimelineReplies", "renoteUser"."token" AS "renoteUser_token" FROM "note" "note" INNER JOIN "user" "user" ON "user"."id"="note"."userId"  LEFT JOIN "note" "reply" ON "reply"."id"="note"."replyId"  LEFT JOIN "note" "renote" ON "renote"."id"="note"."renoteId"  LEFT JOIN "user" "replyUser" ON "replyUser"."id"="reply"."userId"  LEFT JOIN "user" "renoteUser" ON "renoteUser"."id"="renote"."userId" WHERE ('{"8vxhsbo8yc"}' <@ "note"."mentions" OR '{"8vxhsbo8yc"}' <@ "note"."visibleUserIds") AND (("note"."visibility" = 'public' OR "note"."visibility" = 'home') OR "note"."userId" = $1 OR $2 = ANY("note"."visibleUserIds") OR $3 = ANY("note"."mentions") OR ("note"."visibility" = 'followers' AND ("note"."userId" IN (SELECT "following"."followeeId" AS "following_followeeId" FROM "following" "following" WHERE "fol`... 2131 more characters,
misskey-web-1  |   parameters: [
misskey-web-1  |     '8vxhsbo8yc', '8vxhsbo8yc',
misskey-web-1  |     '8vxhsbo8yc', '8vxhsbo8yc',
misskey-web-1  |     '8vxhsbo8yc', '8vxhsbo8yc',
misskey-web-1  |     '8vxhsbo8yc', '8vxhsbo8yc',
misskey-web-1  |     '8vxhsbo8yc', '8vxhsbo8yc',
misskey-web-1  |     '8vxhsbo8yc', '8vxhsbo8yc',
misskey-web-1  |     '8vxhsbo8yc', '8vxhsbo8yc',
misskey-web-1  |     '8vxhsbo8yc', '8vxhsbo8yc'
misskey-web-1  |   ],
misskey-web-1  |   driverError: error: canceling statement due to statement timeout
misskey-web-1  |       at Parser.parseErrorMessage (/misskey/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:287:98)
misskey-web-1  |       at Parser.handlePacket (/misskey/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:126:29)
misskey-web-1  |       at Parser.parse (/misskey/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/parser.js:39:38)
misskey-web-1  |       at Socket.<anonymous> (/misskey/node_modules/.pnpm/[email protected]/node_modules/pg-protocol/dist/index.js:11:42)
misskey-web-1  |       at Socket.emit (node:events:513:28)
misskey-web-1  |       at addChunk (node:internal/streams/readable:324:12)
misskey-web-1  |       at readableAddChunk (node:internal/streams/readable:297:9)
misskey-web-1  |       at Readable.push (node:internal/streams/readable:234:10)
misskey-web-1  |       at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
misskey-web-1  |     length: 109,
misskey-web-1  |     severity: 'ERROR',
misskey-web-1  |     code: '57014',
misskey-web-1  |     detail: undefined,
misskey-web-1  |     hint: undefined,
misskey-web-1  |     position: undefined,
misskey-web-1  |     internalPosition: undefined,
misskey-web-1  |     internalQuery: undefined,
misskey-web-1  |     where: undefined,
misskey-web-1  |     schema: undefined,
misskey-web-1  |     table: undefined,
misskey-web-1  |     column: undefined,
misskey-web-1  |     dataType: undefined,
misskey-web-1  |     constraint: undefined,
misskey-web-1  |     file: 'postgres.c',
misskey-web-1  |     line: '3326',
misskey-web-1  |     routine: 'ProcessInterrupts'
misskey-web-1  |   },
misskey-web-1  |   length: 109,
misskey-web-1  |   severity: 'ERROR',
misskey-web-1  |   code: '57014',
misskey-web-1  |   detail: undefined,
misskey-web-1  |   hint: undefined,
misskey-web-1  |   position: undefined,
misskey-web-1  |   internalPosition: undefined,
misskey-web-1  |   internalQuery: undefined,
misskey-web-1  |   where: undefined,
misskey-web-1  |   schema: undefined,
misskey-web-1  |   table: undefined,
misskey-web-1  |   column: undefined,
misskey-web-1  |   dataType: undefined,
misskey-web-1  |   constraint: undefined,
misskey-web-1  |   file: 'postgres.c',
misskey-web-1  |   line: '3326',
misskey-web-1  |   routine: 'ProcessInterrupts'
misskey-web-1  | } ea787c66-18dd-4bdd-850b-daa6bfe8f0b5

Get the query statement from postgresql log, fill the argument:

SELECT DISTINCT "distinctAlias"."note_id" AS "ids_note_id", "distinctAlias"."note_id" FROM (SELECT "note"."id" AS "note_id", "note"."createdAt" AS "note_createdAt", "note"."replyId" AS "note_replyId", "note"."renoteId" AS "note_renoteId", "note"."threadId" AS "note_threadId", "note"."text" AS "note_text", "note"."name" AS "note_name", "note"."cw" AS "note_cw", "note"."userId" AS "note_userId", "note"."localOnly" AS "note_localOnly", "note"."reactionAcceptance" AS "note_reactionAcceptance", "note"."renoteCount" AS "note_renoteCount", "note"."repliesCount" AS "note_repliesCount", "note"."reactions" AS "note_reactions", "note"."visibility" AS "note_visibility", "note"."uri" AS "note_uri", "note"."url" AS "note_url", "note"."fileIds" AS "note_fileIds", "note"."attachedFileTypes" AS "note_attachedFileTypes", "note"."visibleUserIds" AS "note_visibleUserIds", "note"."mentions" AS "note_mentions", "note"."mentionedRemoteUsers" AS "note_mentionedRemoteUsers", "note"."emojis" AS "note_emojis", "note"."tags" AS "note_tags", "note"."hasPoll" AS "note_hasPoll", "note"."channelId" AS "note_channelId", "note"."userHost" AS "note_userHost", "note"."replyUserId" AS "note_replyUserId", "note"."replyUserHost" AS "note_replyUserHost", "note"."renoteUserId" AS "note_renoteUserId", "note"."renoteUserHost" AS "note_renoteUserHost", "user"."id" AS "user_id", "user"."createdAt" AS "user_createdAt", "user"."updatedAt" AS "user_updatedAt", "user"."lastFetchedAt" AS "user_lastFetchedAt", "user"."lastActiveDate" AS "user_lastActiveDate", "user"."hideOnlineStatus" AS "user_hideOnlineStatus", "user"."username" AS "user_username", "user"."name" AS "user_name", "user"."followersCount" AS "user_followersCount", "user"."followingCount" AS "user_followingCount", "user"."movedToUri" AS "user_movedToUri", "user"."movedAt" AS "user_movedAt", "user"."alsoKnownAs" AS "user_alsoKnownAs", "user"."notesCount" AS "user_notesCount", "user"."avatarId" AS "user_avatarId", "user"."bannerId" AS "user_bannerId", "user"."avatarUrl" AS "user_avatarUrl", "user"."bannerUrl" AS "user_bannerUrl", "user"."avatarBlurhash" AS "user_avatarBlurhash", "user"."bannerBlurhash" AS "user_bannerBlurhash", "user"."tags" AS "user_tags", "user"."isSuspended" AS "user_isSuspended", "user"."isLocked" AS "user_isLocked", "user"."isBot" AS "user_isBot", "user"."isCat" AS "user_isCat", "user"."isRoot" AS "user_isRoot", "user"."isExplorable" AS "user_isExplorable", "user"."isDeleted" AS "user_isDeleted", "user"."emojis" AS "user_emojis", "user"."host" AS "user_host", "user"."inbox" AS "user_inbox", "user"."sharedInbox" AS "user_sharedInbox", "user"."featured" AS "user_featured", "user"."uri" AS "user_uri", "user"."followersUri" AS "user_followersUri", "user"."showTimelineReplies" AS "user_showTimelineReplies", "user"."token" AS "user_token", "reply"."id" AS "reply_id", "reply"."createdAt" AS "reply_createdAt", "reply"."replyId" AS "reply_replyId", "reply"."renoteId" AS "reply_renoteId", "reply"."threadId" AS "reply_threadId", "reply"."text" AS "reply_text", "reply"."name" AS "reply_name", "reply"."cw" AS "reply_cw", "reply"."userId" AS "reply_userId", "reply"."localOnly" AS "reply_localOnly", "reply"."reactionAcceptance" AS "reply_reactionAcceptance", "reply"."renoteCount" AS "reply_renoteCount", "reply"."repliesCount" AS "reply_repliesCount", "reply"."reactions" AS "reply_reactions", "reply"."visibility" AS "reply_visibility", "reply"."uri" AS "reply_uri", "reply"."url" AS "reply_url", "reply"."fileIds" AS "reply_fileIds", "reply"."attachedFileTypes" AS "reply_attachedFileTypes", "reply"."visibleUserIds" AS "reply_visibleUserIds", "reply"."mentions" AS "reply_mentions", "reply"."mentionedRemoteUsers" AS "reply_mentionedRemoteUsers", "reply"."emojis" AS "reply_emojis", "reply"."tags" AS "reply_tags", "reply"."hasPoll" AS "reply_hasPoll", "reply"."channelId" AS "reply_channelId", "reply"."userHost" AS "reply_userHost", "reply"."replyUserId" AS "reply_replyUserId", "reply"."replyUserHost" AS "reply_replyUserHost", "reply"."renoteUserId" AS "reply_renoteUserId", "reply"."renoteUserHost" AS "reply_renoteUserHost", "renote"."id" AS "renote_id", "renote"."createdAt" AS "renote_createdAt", "renote"."replyId" AS "renote_replyId", "renote"."renoteId" AS "renote_renoteId", "renote"."threadId" AS "renote_threadId", "renote"."text" AS "renote_text", "renote"."name" AS "renote_name", "renote"."cw" AS "renote_cw", "renote"."userId" AS "renote_userId", "renote"."localOnly" AS "renote_localOnly", "renote"."reactionAcceptance" AS "renote_reactionAcceptance", "renote"."renoteCount" AS "renote_renoteCount", "renote"."repliesCount" AS "renote_repliesCount", "renote"."reactions" AS "renote_reactions", "renote"."visibility" AS "renote_visibility", "renote"."uri" AS "renote_uri", "renote"."url" AS "renote_url", "renote"."fileIds" AS "renote_fileIds", "renote"."attachedFileTypes" AS "renote_attachedFileTypes", "renote"."visibleUserIds" AS "renote_visibleUserIds", "renote"."mentions" AS "renote_mentions", "renote"."mentionedRemoteUsers" AS "renote_mentionedRemoteUsers", "renote"."emojis" AS "renote_emojis", "renote"."tags" AS "renote_tags", "renote"."hasPoll" AS "renote_hasPoll", "renote"."channelId" AS "renote_channelId", "renote"."userHost" AS "renote_userHost", "renote"."replyUserId" AS "renote_replyUserId", "renote"."replyUserHost" AS "renote_replyUserHost", "renote"."renoteUserId" AS "renote_renoteUserId", "renote"."renoteUserHost" AS "renote_renoteUserHost", "replyUser"."id" AS "replyUser_id", "replyUser"."createdAt" AS "replyUser_createdAt", "replyUser"."updatedAt" AS "replyUser_updatedAt", "replyUser"."lastFetchedAt" AS "replyUser_lastFetchedAt", "replyUser"."lastActiveDate" AS "replyUser_lastActiveDate", "replyUser"."hideOnlineStatus" AS "replyUser_hideOnlineStatus", "replyUser"."username" AS "replyUser_username", "replyUser"."name" AS "replyUser_name", "replyUser"."followersCount" AS "replyUser_followersCount", "replyUser"."followingCount" AS "replyUser_followingCount", "replyUser"."movedToUri" AS "replyUser_movedToUri", "replyUser"."movedAt" AS "replyUser_movedAt", "replyUser"."alsoKnownAs" AS "replyUser_alsoKnownAs", "replyUser"."notesCount" AS "replyUser_notesCount", "replyUser"."avatarId" AS "replyUser_avatarId", "replyUser"."bannerId" AS "replyUser_bannerId", "replyUser"."avatarUrl" AS "replyUser_avatarUrl", "replyUser"."bannerUrl" AS "replyUser_bannerUrl", "replyUser"."avatarBlurhash" AS "replyUser_avatarBlurhash", "replyUser"."bannerBlurhash" AS "replyUser_bannerBlurhash", "replyUser"."tags" AS "replyUser_tags", "replyUser"."isSuspended" AS "replyUser_isSuspended", "replyUser"."isLocked" AS "replyUser_isLocked", "replyUser"."isBot" AS "replyUser_isBot", "replyUser"."isCat" AS "replyUser_isCat", "replyUser"."isRoot" AS "replyUser_isRoot", "replyUser"."isExplorable" AS "replyUser_isExplorable", "replyUser"."isDeleted" AS "replyUser_isDeleted", "replyUser"."emojis" AS "replyUser_emojis", "replyUser"."host" AS "replyUser_host", "replyUser"."inbox" AS "replyUser_inbox", "replyUser"."sharedInbox" AS "replyUser_sharedInbox", "replyUser"."featured" AS "replyUser_featured", "replyUser"."uri" AS "replyUser_uri", "replyUser"."followersUri" AS "replyUser_followersUri", "replyUser"."showTimelineReplies" AS "replyUser_showTimelineReplies", "replyUser"."token" AS "replyUser_token", "renoteUser"."id" AS "renoteUser_id", "renoteUser"."createdAt" AS "renoteUser_createdAt", "renoteUser"."updatedAt" AS "renoteUser_updatedAt", "renoteUser"."lastFetchedAt" AS "renoteUser_lastFetchedAt", "renoteUser"."lastActiveDate" AS "renoteUser_lastActiveDate", "renoteUser"."hideOnlineStatus" AS "renoteUser_hideOnlineStatus", "renoteUser"."username" AS "renoteUser_username", "renoteUser"."name" AS "renoteUser_name", "renoteUser"."followersCount" AS "renoteUser_followersCount", "renoteUser"."followingCount" AS "renoteUser_followingCount", "renoteUser"."movedToUri" AS "renoteUser_movedToUri", "renoteUser"."movedAt" AS "renoteUser_movedAt", "renoteUser"."alsoKnownAs" AS "renoteUser_alsoKnownAs", "renoteUser"."notesCount" AS "renoteUser_notesCount", "renoteUser"."avatarId" AS "renoteUser_avatarId", "renoteUser"."bannerId" AS "renoteUser_bannerId", "renoteUser"."avatarUrl" AS "renoteUser_avatarUrl", "renoteUser"."bannerUrl" AS "renoteUser_bannerUrl", "renoteUser"."avatarBlurhash" AS "renoteUser_avatarBlurhash", "renoteUser"."bannerBlurhash" AS "renoteUser_bannerBlurhash", "renoteUser"."tags" AS "renoteUser_tags", "renoteUser"."isSuspended" AS "renoteUser_isSuspended", "renoteUser"."isLocked" AS "renoteUser_isLocked", "renoteUser"."isBot" AS "renoteUser_isBot", "renoteUser"."isCat" AS "renoteUser_isCat", "renoteUser"."isRoot" AS "renoteUser_isRoot", "renoteUser"."isExplorable" AS "renoteUser_isExplorable", "renoteUser"."isDeleted" AS "renoteUser_isDeleted", "renoteUser"."emojis" AS "renoteUser_emojis", "renoteUser"."host" AS "renoteUser_host", "renoteUser"."inbox" AS "renoteUser_inbox", "renoteUser"."sharedInbox" AS "renoteUser_sharedInbox", "renoteUser"."featured" AS "renoteUser_featured", "renoteUser"."uri" AS "renoteUser_uri", "renoteUser"."followersUri" AS "renoteUser_followersUri", "renoteUser"."showTimelineReplies" AS "renoteUser_showTimelineReplies", "renoteUser"."token" AS "renoteUser_token" FROM "note" "note" INNER JOIN "user" "user" ON "user"."id"="note"."userId"  LEFT JOIN "note" "reply" ON "reply"."id"="note"."replyId"  LEFT JOIN "note" "renote" ON "renote"."id"="note"."renoteId"  LEFT JOIN "user" "replyUser" ON "replyUser"."id"="reply"."userId"  LEFT JOIN "user" "renoteUser" ON "renoteUser"."id"="renote"."userId" WHERE ('{"8vxhsbo8yc"}' <@ "note"."mentions" OR '{"8vxhsbo8yc"}' <@ "note"."visibleUserIds") AND (("note"."visibility" = 'public' OR "note"."visibility" = 'home') OR "note"."userId" = '8vxhsbo8yc' OR '8vxhsbo8yc' = ANY("note"."visibleUserIds") OR '8vxhsbo8yc' = ANY("note"."mentions") OR ("note"."visibility" = 'followers' AND ("note"."userId" IN (SELECT "following"."followeeId" AS "following_followeeId" FROM "following" "following" WHERE "following"."followerId" = '8vxhsbo8yc') OR "note"."replyUserId" = '8vxhsbo8yc'))) AND "note"."userId" NOT IN (SELECT "muting"."muteeId" AS "muting_muteeId" FROM "muting" "muting" WHERE "muting"."muterId" = '8vxhsbo8yc') AND ("note"."replyUserId" IS NULL OR "note"."replyUserId" NOT IN (SELECT "muting"."muteeId" AS "muting_muteeId" FROM "muting" "muting" WHERE "muting"."muterId" = '8vxhsbo8yc')) AND ("note"."renoteUserId" IS NULL OR "note"."renoteUserId" NOT IN (SELECT "muting"."muteeId" AS "muting_muteeId" FROM "muting" "muting" WHERE "muting"."muterId" = '8vxhsbo8yc')) AND ("note"."userHost" IS NULL OR NOT ((SELECT "user_profile"."mutedInstances" AS "user_profile_mutedInstances" FROM "user_profile" "user_profile" WHERE "user_profile"."userId" = '8vxhsbo8yc')::jsonb ? "note"."userHost")) AND ("note"."replyUserHost" IS NULL OR NOT ((SELECT "user_profile"."mutedInstances" AS "user_profile_mutedInstances" FROM "user_profile" "user_profile" WHERE "user_profile"."userId" = '8vxhsbo8yc')::jsonb ? "note"."replyUserHost")) AND ("note"."renoteUserHost" IS NULL OR NOT ((SELECT "user_profile"."mutedInstances" AS "user_profile_mutedInstances" FROM "user_profile" "user_profile" WHERE "user_profile"."userId" = '8vxhsbo8yc')::jsonb ? "note"."renoteUserHost")) AND "note"."id" NOT IN (SELECT "threadMuted"."threadId" AS "threadMuted_threadId" FROM "note_thread_muting" "threadMuted" WHERE "threadMuted"."userId" = '8vxhsbo8yc') AND ("note"."threadId" IS NULL OR "note"."threadId" NOT IN (SELECT "threadMuted"."threadId" AS "threadMuted_threadId" FROM "note_thread_muting" "threadMuted" WHERE "threadMuted"."userId" = '8vxhsbo8yc')) AND "note"."userId" NOT IN (SELECT "blocking"."blockerId" AS "blocking_blockerId" FROM "blocking" "blocking" WHERE "blocking"."blockeeId" = '8vxhsbo8yc') AND ("note"."replyUserId" IS NULL OR "note"."replyUserId" NOT IN (SELECT "blocking"."blockerId" AS "blocking_blockerId" FROM "blocking" "blocking" WHERE "blocking"."blockeeId" = '8vxhsbo8yc')) AND ("note"."renoteUserId" IS NULL OR "note"."renoteUserId" NOT IN (SELECT "blocking"."blockerId" AS "blocking_blockerId" FROM "blocking" "blocking" WHERE "blocking"."blockeeId" = '8vxhsbo8yc'))) "distinctAlias" ORDER BY "distinctAlias"."note_id" DESC, "note_id" ASC LIMIT 10;

Appending EXPLAIN ANALYSE before the sql statement then execuate in psql:

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         QUERY PLAN                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=1019.66..3928.97 rows=10 width=22) (actual time=11338.860..11441.954 rows=10 loops=1)
   InitPlan 5 (returns $4)
     ->  Index Scan using "UQ_51cb79b5555effaf7d69ba1cff9" on user_profile  (cost=0.42..2.64 rows=1 width=5) (actual time=0.132..0.133 rows=1 loops=1)
           Index Cond: (("userId")::text = '8vxhsbo8yc'::text)
   InitPlan 6 (returns $5)
     ->  Index Scan using "UQ_51cb79b5555effaf7d69ba1cff9" on user_profile user_profile_1  (cost=0.42..2.64 rows=1 width=5) (actual time=0.025..0.025 rows=1 loops=1)
           Index Cond: (("userId")::text = '8vxhsbo8yc'::text)
   InitPlan 7 (returns $6)
     ->  Index Scan using "UQ_51cb79b5555effaf7d69ba1cff9" on user_profile user_profile_2  (cost=0.42..2.64 rows=1 width=5) (actual time=0.017..0.021 rows=1 loops=1)
           Index Cond: (("userId")::text = '8vxhsbo8yc'::text)
   ->  Unique  (cost=1011.75..575600.16 rows=1975 width=22) (actual time=11338.855..11441.927 rows=10 loops=1)
         ->  Nested Loop  (cost=1011.75..575595.23 rows=1975 width=22) (actual time=11338.851..11441.911 rows=10 loops=1)
               ->  Gather Merge  (cost=1011.34..574201.50 rows=1975 width=44) (actual time=11338.606..11441.382 rows=10 loops=1)
                     Workers Planned: 2
                     Params Evaluated: $4, $5, $6
                     Workers Launched: 2
                     ->  Parallel Index Scan Backward using "PK_96d0c172a4fba276b1bbed43058" on note  (cost=11.31..572973.51 rows=823 width=44) (actual time=78.202..7550.477 rows=59 loops=3)
                           Filter: ((NOT (hashed SubPlan 2)) AND (("replyUserId" IS NULL) OR (NOT (hashed SubPlan 3))) AND (("renoteUserId" IS NULL) OR (NOT (hashed SubPlan 4))) AND (("userHost" IS NULL) OR (NOT ($4 ? ("userHost")::text))) AND (("replyUserHost" IS NULL) OR (NOT ($5 ? ("replyUserHost")::text))) AND (("renoteUserHost" IS NULL) OR (NOT ($6 ? ("renoteUserHost")::text))) AND (NOT (hashed SubPlan 8)) AND (("threadId" IS NULL) OR (NOT (hashed SubPlan 9))) AND (NOT (hashed SubPlan 10)) AND (("replyUserId" IS NULL) OR (NOT (hashed SubPlan 11))) AND (("renoteUserId" IS NULL) OR (NOT (hashed SubPlan 12))) AND (('{8vxhsbo8yc}'::character varying[] <@ mentions) OR ('{8vxhsbo8yc}'::character varying[] <@ "visibleUserIds")) AND ((visibility = 'public'::note_visibility_enum) OR (visibility = 'home'::note_visibility_enum) OR (("userId")::text = '8vxhsbo8yc'::text) OR ('8vxhsbo8yc'::text = ANY (("visibleUserIds")::text[])) OR ('8vxhsbo8yc'::text = ANY ((mentions)::text[])) OR ((visibility = 'followers'::note_visibility_enum) AND ((hashed SubPlan 1) OR (("replyUserId")::text = '8vxhsbo8yc'::text)))))
                           Rows Removed by Filter: 1907223
                           SubPlan 2
                             ->  Seq Scan on muting  (cost=0.00..0.00 rows=1 width=82) (actual time=0.012..0.015 rows=0 loops=3)
                                   Filter: (("muterId")::text = '8vxhsbo8yc'::text)
                           SubPlan 3
                             ->  Seq Scan on muting muting_1  (cost=0.00..0.00 rows=1 width=82) (actual time=0.003..0.006 rows=0 loops=3)
                                   Filter: (("muterId")::text = '8vxhsbo8yc'::text)
                           SubPlan 4
                             ->  Seq Scan on muting muting_2  (cost=0.00..0.00 rows=1 width=82) (actual time=0.002..0.004 rows=0 loops=3)
                                   Filter: (("muterId")::text = '8vxhsbo8yc'::text)
                           SubPlan 8
                             ->  Seq Scan on note_thread_muting "threadMuted"  (cost=0.00..0.00 rows=1 width=516) (actual time=0.008..0.010 rows=0 loops=3)
                                   Filter: (("userId")::text = '8vxhsbo8yc'::text)
                           SubPlan 9
                             ->  Seq Scan on note_thread_muting "threadMuted_1"  (cost=0.00..0.00 rows=1 width=516) (actual time=0.001..0.002 rows=0 loops=3)
                                   Filter: (("userId")::text = '8vxhsbo8yc'::text)
                           SubPlan 10
                             ->  Seq Scan on blocking  (cost=0.00..1.15 rows=5 width=11) (actual time=0.043..0.046 rows=5 loops=3)
                                   Filter: (("blockeeId")::text = '8vxhsbo8yc'::text)
                                   Rows Removed by Filter: 7
                           SubPlan 11
                             ->  Seq Scan on blocking blocking_1  (cost=0.00..1.15 rows=5 width=11) (actual time=0.009..0.011 rows=5 loops=3)
                                   Filter: (("blockeeId")::text = '8vxhsbo8yc'::text)
                                   Rows Removed by Filter: 7
                           SubPlan 12
                             ->  Seq Scan on blocking blocking_2  (cost=0.00..1.15 rows=5 width=11) (actual time=0.004..0.006 rows=5 loops=3)
                                   Filter: (("blockeeId")::text = '8vxhsbo8yc'::text)
                                   Rows Removed by Filter: 7
                           SubPlan 1
                             ->  Index Scan using "IDX_6516c5a6f3c015b4eed39978be" on following  (cost=0.14..7.24 rows=54 width=11) (never executed)
                                   Index Cond: (("followerId")::text = '8vxhsbo8yc'::text)
               ->  Index Only Scan using "PK_cace4a159ff9f2512dd42373760" on "user"  (cost=0.42..0.71 rows=1 width=11) (actual time=0.050..0.050 rows=1 loops=10)
                     Index Cond: (id = (note."userId")::text)
                     Heap Fetches: 2
 Planning Time: 26.921 ms
 Execution Time: 11443.043 ms
(54 rows)

It takes a long time to get the result. And If incrase the statement_timeout it can be displayed after a long query time.
But I tested on several other sites and did not find this issue. It was basically completed within one second. So I'm curious about the reason and how to solve it?

Environment:
Misskey 13.12.2 (docker)
Postgresql 15-alpine (docker)

@creamlike1024
Copy link

Manually added an index (with the trgm extension) and now it's blazingly fast (from 15s to 0.227s on pure like query)

CREATE INDEX "IDX_NOTE_TEXT" ON "note" USING gin ("text" gin_trgm_ops);

Full walkthrough: https://slippy.xyz/notes/9dmwuyvuue

Thanks for your resolution but I tried it and not work for notes/mention

@creamlike1024
Copy link

Maybe related to #7632 #8875

@zekexy
Copy link
Contributor

zekexy commented Sep 10, 2023

Manually added an index (with the trgm extension) and now it's blazingly fast (from 15s to 0.227s on pure like query)

CREATE INDEX "IDX_NOTE_TEXT" ON "note" USING gin ("text" gin_trgm_ops);

Full walkthrough: https://slippy.xyz/notes/9dmwuyvuue

Thanks for your resolution but I tried it and not work for notes/mention

Looking forward to knowing whether this #11799 can solve your problem

@creamlike1024
Copy link

Manually added an index (with the trgm extension) and now it's blazingly fast (from 15s to 0.227s on pure like query)

CREATE INDEX "IDX_NOTE_TEXT" ON "note" USING gin ("text" gin_trgm_ops);

Full walkthrough: https://slippy.xyz/notes/9dmwuyvuue

Thanks for your resolution but I tried it and not work for notes/mention

Looking forward to knowing whether this #11799 can solve your problem

Great work. You solved this problem that has been around for over a year. Thank you!!!

@KisaragiEffective
Copy link
Collaborator

KisaragiEffective commented Jul 18, 2024

triage: closing per #9940 (comment)

@KisaragiEffective
Copy link
Collaborator

@MomentQYC do they happen on recent release?

@MomentQYC
Copy link
Contributor Author

do they happen on recent release?

This has not been found in recent releases
Close this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⚠️bug? This might be a bug
Projects
None yet
Development

No branches or pull requests

6 participants