When using MySQL as the metadata engine, an issue arises with the 'jfs_node' node type. #4225

sinmem · 2023-12-01T06:17:29Z

sinmem
Dec 1, 2023

While the system automatically initializes the file system, the 'uid' and 'gid' within the 'node' are initially set as 'int'. However, file uploads via the gateway require 'bigint', resulting in upload failures that necessitate manual modification. Is this a software bug or an operational mistake?
version: juicefs-1.1.1-windows-amd64.tar.gz

2023/12/01 13:13:49.526515 juicefs[42588] <ERROR>: error: Error 1264 (22003): Out of range value for column 'uid' at row 1
goroutine 291 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x65
github.com/juicedata/juicefs/pkg/meta.errno({0x3e43be0, 0xc0024fa108})
        /go/src/github.com/juicedata/juicefs/pkg/meta/utils.go:122 +0xc5
github.com/juicedata/juicefs/pkg/meta.(*dbMeta).doMknod(0xc000e54960, {0x3e78638, 0xc001db2200}, 0x1, {0xc002234871, 0x4}, 0x2, 0x1ff, 0x12, 0x0, ...)
        /go/src/github.com/juicedata/juicefs/pkg/meta/sql.go:1226 +0x2a5
github.com/juicedata/juicefs/pkg/meta.(*baseMeta).Mknod(0xc00028aa00, {0x3e78638, 0xc001db2200}, 0x1, {0xc002234871, 0x4}, 0x8?, 0x0?, 0x0?, 0x0, ...)
        /go/src/github.com/juicedata/juicefs/pkg/meta/base.go:1352 +0x2d5
github.com/juicedata/juicefs/pkg/meta.(*baseMeta).Mkdir(0xc00028aa00, {0x3e78638?, 0xc001db2200?}, 0x1, {0xc002234871?, 0x3e78638?}, 0x2200?, 0x1db?, 0xc0?, 0xc0025ca538, ...)
        /go/src/github.com/juicedata/juicefs/pkg/meta/base.go:1376 +0x78
github.com/juicedata/juicefs/pkg/fs.(*FileSystem).Mkdir(0xc001dba000, {0x3e78638?, 0xc001db2200}, {0xc002234870, 0x5}, 0x1ff, 0x4a?)
        /go/src/github.com/juicedata/juicefs/pkg/fs/fs.go:441 +0x376
github.com/juicedata/juicefs/pkg/gateway.(*jfsObjects).mkdirAll(0xc001d94420, {0x3e642c8, 0xc000c46100}, {0xc002234870, 0x5})
        /go/src/github.com/juicedata/juicefs/pkg/gateway/gateway.go:638 +0xf3
github.com/juicedata/juicefs/pkg/gateway.(*jfsObjects).mkdirAll(0xc001d94420, {0x3e642c8, 0xc000c46100}, {0xc002234870, 0x9})
        /go/src/github.com/juicedata/juicefs/pkg/gateway/gateway.go:640 +0x14d
github.com/juicedata/juicefs/pkg/gateway.(*jfsObjects).putObject(0xc001d94420, {0x3e642c8, 0xc000c46100}, {0xc0021a5872, 0x5}, {0xc00227f448, 0x8}, _, {{0x0, 0x0}, ...})
        /go/src/github.com/juicedata/juicefs/pkg/gateway/gateway.go:658 +0x1d1
github.com/juicedata/juicefs/pkg/gateway.(*jfsObjects).PutObject(_, {_, _}, {_, _}, {_, _}, _, {{0x0, 0x0}, ...})
        /go/src/github.com/juicedata/juicefs/pkg/gateway/gateway.go:731 +0x3ae
github.com/minio/minio/cmd.(*webAPIHandlers).Upload(0xc00027e220, {0x3e5aa90, 0xc00207a5a0}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/web-handlers.go:1321 +0x16e3
net/http.HandlerFunc.ServeHTTP(...)
        /usr/local/go/src/net/http/server.go:2084
github.com/minio/minio/cmd.httpTraceHdrs.func1({0x3e5aa90?, 0xc00207a5a0?}, 0x8?)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/handler-utils.go:371 +0x53
net/http.HandlerFunc.ServeHTTP(0xc00208b300?, {0x3e5aa90?, 0xc00207a5a0?}, 0xa0?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setRedirectHandler.func1({0x3e5aa90, 0xc00207a5a0}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:135 +0x147
net/http.HandlerFunc.ServeHTTP(0x3e5aa60?, {0x3e5aa90?, 0xc00207a5a0?}, 0x10?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.addCustomHeaders.func1({0x3e5aa60, 0xc0010aff08}, 0xc00109cc60?)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:660 +0x136
net/http.HandlerFunc.ServeHTTP(0x343e720?, {0x3e5aa60?, 0xc0010aff08?}, 0x17?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.addSecurityHeaders.func1({0x3e5aa60, 0xc0010aff08}, 0x0?)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:669 +0x184
net/http.HandlerFunc.ServeHTTP(0x0?, {0x3e5aa60?, 0xc0010aff08?}, 0x0?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setBucketForwardingHandler.func1({0x3e5aa60, 0xc0010aff08}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:543 +0x237
net/http.HandlerFunc.ServeHTTP(0xc0021a5864?, {0x3e5aa60?, 0xc0010aff08?}, 0x47a2f3?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setRequestValidityHandler.func1({0x3e5aa60, 0xc0010aff08}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:531 +0x394
net/http.HandlerFunc.ServeHTTP(0xc0025b4660?, {0x3e5aa60?, 0xc0010aff08?}, 0x90108?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setHTTPStatsHandler.func1({0x3e623b0?, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:465 +0x11f
net/http.HandlerFunc.ServeHTTP(0x2eb61b0?, {0x3e623b0?, 0xc00025eb60?}, 0x0?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setRequestSizeLimitHandler.func1({0x3e623b0?, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:49 +0x102
net/http.HandlerFunc.ServeHTTP(0xc0010b8a70?, {0x3e623b0?, 0xc00025eb60?}, 0xc0025b4900?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setRequestHeaderSizeLimitHandler.func1({0x3e623b0, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:68 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc0021ecf00?, {0x3e623b0?, 0xc00025eb60?}, 0xc0010afed8?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.crossDomainPolicy.ServeHTTP({{0x3e4e460?, 0xc0010afde8?}}, {0x3e623b0?, 0xc00025eb60?}, 0x1b?)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/crossdomain-xml-handler.go:51 +0xef
github.com/minio/minio/cmd.setBrowserRedirectHandler.func1({0x3e623b0, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:161 +0xb0
net/http.HandlerFunc.ServeHTTP(0xc00208b300?, {0x3e623b0?, 0xc00025eb60?}, 0x3e30dec?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setReservedBucketHandler.func1({0x3e623b0, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:293 +0x2c2
net/http.HandlerFunc.ServeHTTP(0xc0021ed0b8?, {0x3e623b0?, 0xc00025eb60?}, 0xc0025b4660?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setBrowserCacheControlHandler.func1({0x3e623b0, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:262 +0x2b7
net/http.HandlerFunc.ServeHTTP(0xc00208b300?, {0x3e623b0?, 0xc00025eb60?}, 0x30?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setTimeValidityHandler.func1({0x3e623b0, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:357 +0x3a3
net/http.HandlerFunc.ServeHTTP(0xc00208b300?, {0x3e623b0?, 0xc00025eb60?}, 0x0?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setIgnoreResourcesHandler.func1({0x3e623b0, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:452 +0x337
net/http.HandlerFunc.ServeHTTP(0xc00208b300?, {0x3e623b0?, 0xc00025eb60?}, 0xc0021ed4e8?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setAuthHandler.func1({0x3e623b0, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/auth-handler.go:504 +0x2a9
net/http.HandlerFunc.ServeHTTP(0x35a3000?, {0x3e623b0?, 0xc00025eb60?}, 0x39?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.setSSETLSHandler.func1({0x3e623b0?, 0xc00025eb60?}, 0xc00208b300?)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:703 +0x3ca
net/http.HandlerFunc.ServeHTTP(0x90?, {0x3e623b0?, 0xc00025eb60?}, 0xc0010afea8?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.filterReservedMetadata.func1({0x3e623b0, 0xc00025eb60}, 0xc00208b300)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:108 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc00208b200?, {0x3e623b0?, 0xc00025eb60?}, 0xc0025aa880?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/gorilla/mux.(*Router).ServeHTTP(0xc0009675c0, {0x3e623b0, 0xc00025eb60}, 0xc0024b4600)
        /go/pkg/mod/github.com/gorilla/[email protected]/mux.go:210 +0x1cf
github.com/rs/cors.(*Cors).Handler.func1({0x3e623b0, 0xc00025eb60}, 0xc0024b4600)
        /go/pkg/mod/github.com/rs/[email protected]/cors.go:219 +0x1bd
net/http.HandlerFunc.ServeHTTP(0xc000c47040?, {0x3e623b0?, 0xc00025eb60?}, 0x5463bc0?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
github.com/minio/minio/cmd.criticalErrorHandler.ServeHTTP({{0x3e4e460?, 0xc000e544e0?}}, {0x3e623b0?, 0xc00025eb60?}, 0x0?)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/generic-handlers.go:688 +0x85
github.com/minio/minio/cmd/http.(*Server).Start.func1({0x3e623b0?, 0xc00025eb60?}, 0xc0025aa810?)
        /go/pkg/mod/github.com/juicedata/[email protected]/cmd/http/server.go:101 +0x1ba
net/http.HandlerFunc.ServeHTTP(0x0?, {0x3e623b0?, 0xc00025eb60?}, 0x40ef65?)
        /usr/local/go/src/net/http/server.go:2084 +0x2f
net/http.serverHandler.ServeHTTP({0x3e563f0?}, {0x3e623b0, 0xc00025eb60}, 0xc0024b4600)
        /usr/local/go/src/net/http/server.go:2916 +0x43b
net/http.(*conn).serve(0xc000e4be00, {0x3e64370, 0xc000df2150})
        /usr/local/go/src/net/http/server.go:1966 +0x5d7
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:3071 +0x4db [utils.go:122]
2023/12/01 13:13:49.528884 juicefs[42588] <ERROR>: create /.sys/tmp/1b9c312e-acd0-4f8d-bb6f-902e203f4994: The system cannot find the file specified. [gateway.go:662]

API: SYSTEM(bucket=5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5, object=5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5)
Time: 13:13:49 CST 12/01/2023
DeploymentID: ad15981e-8e29-4016-ad02-e3c2cb3aec4a
RemoteHost: 5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5
Error: syscall.Errno
       3: \go\pkg\mod\github.com\juicedata\m[email protected]\cmd\web-handlers.go:2428:cmd.toWebAPIError()
       2: \go\pkg\mod\github.com\juicedata\m[email protected]\cmd\web-handlers.go:2438:cmd.writeWebErrorResponse()
       1: \go\pkg\mod\github.com\juicedata\m[email protected]\cmd\web-handlers.go:1323:cmd.(*webAPIHandlers).Upload()

API: SYSTEM(bucket=5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5, object=5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5)
Time: 13:13:49 CST 12/01/2023
DeploymentID: ad15981e-8e29-4016-ad02-e3c2cb3aec4a
RemoteHost: 5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5
Error: syscall.Errno
       5: \go\pkg\mod\github.com\juicedata\m[email protected]\cmd\api-errors.go:2006:cmd.toAPIErrorCode()
       4: \go\pkg\mod\github.com\juicedata\m[email protected]\cmd\api-errors.go:2031:cmd.toAPIError()
       3: \go\pkg\mod\github.com\juicedata\m[email protected]\cmd\web-handlers.go:2429:cmd.toWebAPIError()
       2: \go\pkg\mod\github.com\juicedata\m[email protected]\cmd\web-handlers.go:2438:cmd.writeWebErrorResponse()
       1: \go\pkg\mod\github.com\juicedata\m[email protected]\cmd\web-handlers.go:1323:cmd.(*webAPIHandlers).Upload()

zhijian-pro · 2023-12-04T06:07:26Z

zhijian-pro
Dec 4, 2023
Collaborator

Currently, the uid and gid of more than int32 are not supported.
Is the uid of such a large size specially set by yourself ?

It can be made to work by manually repairing the table structure, which is a workaround to the solution.

ALTER TABLE jfs_node
MODIFY uid bigint;
ALTER TABLE jfs_node
MODIFY gid bigint;

0 replies

sinmem · 2023-12-07T02:02:44Z

sinmem
Dec 7, 2023
Author

No, this happens with the gateway that's started using the 'juicefs gateway' command from 'juicefs-1.1.1-windows-amd64.tar.gz'. Whether using the web interface or making API calls through the gateway, I keep encountering this issue when uploading data. Seems like an internal software incompatibility, right? Any suitable fix for this?

FileInputStream input = new FileInputStream("F:/Administrator/Downloads/202311/juicefs-1.1.1-windows-amd64.tar.gz");
ObjectMetadata meta = new ObjectMetadata();
meta.setContentLength(input.available());
s3client.putObject("myifs","20234797", input, meta);

set MINIO_ROOT_USER=admin
set MINIO_ROOT_PASSWORD=123456789
juicefs gateway postgres://***:***@***:5432/pgdata ***:9080
pause

1 reply

zhijian-pro Dec 13, 2023
Collaborator

The root of the problem is that our database uid gid field type is set too small. Manually changing the data type of the jfs_node field in the pg database to a larger one should solve the problem.

Or make sure that the uid of the user who starts juicefs gateway is less than 65530.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

When using MySQL as the metadata engine, an issue arises with the 'jfs_node' node type. #4225

{{title}}

Replies: 2 comments 1 reply

{{title}}

{{title}}

{{title}}

Select a reply

When using MySQL as the metadata engine, an issue arises with the 'jfs_node' node type. #4225

sinmem Dec 1, 2023

Replies: 2 comments · 1 reply

zhijian-pro Dec 4, 2023 Collaborator

sinmem Dec 7, 2023 Author

zhijian-pro Dec 13, 2023 Collaborator

sinmem
Dec 1, 2023

Replies: 2 comments 1 reply

zhijian-pro
Dec 4, 2023
Collaborator

sinmem
Dec 7, 2023
Author

zhijian-pro Dec 13, 2023
Collaborator