Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mounting more than 64 shares with 1 C# process not possible. #364

Open
4 of 5 tasks
JFra-AME opened this issue Nov 29, 2024 · 5 comments
Open
4 of 5 tasks

Mounting more than 64 shares with 1 C# process not possible. #364

JFra-AME opened this issue Nov 29, 2024 · 5 comments

Comments

@JFra-AME
Copy link

Environment

  • Windows version: Windows 10
  • Processor architecture: x64 Intel(R) Core(TM) i7-6700HQ
  • Dokany version: 2.2.0.1000
  • Library type (Dokany/FUSE): Dokany DotNet wrapper

Check List

  • I checked my issue doesn't exist yet
  • My issue is valid with mirror default sample and not specific to my user-mode driver implementation
  • I can always reproduce the issue with the provided description below.
  • I have updated Dokany to the latest version and have reboot my computer after.
  • I tested one of the last snapshot from appveyor CI

Description

Background

I am using the Dokany .NET wrapper to provide users within my organization with their own virtual drives, each equipped with custom access control. This setup requires the creation of over 64 individual shares, as each user is assigned a unique share.

Problem

When attempting to create multiple shares for users, I encountered a hard limit of 64 shares per C# process. This restriction causes issues when mounting more than 64 shares in a directory such as C:\shares.
Specifically:

  • Shares exceeding the 64-share "limit" become inaccessible. When trying to access shares 65 and beyond, the system displays an error message: "The parameter is incorrect."
    image

  • Discrepancy in displayed size: Successfully mounted shares correctly reflect the size of the C: disk. However, shares beyond the limit (those that are "corrupt") show a default or incorrect size:
    image

This behaviour prevents me from scaling the virtual drive solution to meet the needs of the organization.

Replication

To replicate this I provide an altered code of the Mirror .NET sample.
In the class Program of the sample DokanNetMirror
Once this code is running it will take 64 seconds till it will make the "corrupt" shares.

private static async Task Main(string[] args)
{
    try
    {
        var arguments = args
           .Select(x => x.Split(new char[] { '=' }, 2, StringSplitOptions.RemoveEmptyEntries))
           .ToDictionary(x => x[0], x => x.Length > 1 ? x[1] as object : true, StringComparer.OrdinalIgnoreCase);

        var mirrorPath = arguments.ContainsKey(MirrorKey)
           ? arguments[MirrorKey] as string
           : @"C:\productdata";

        var mountPath = arguments.ContainsKey(MountKey)
           ? arguments[MountKey] as string
           : @"C:\Share\";

        var unsafeReadWrite = arguments.ContainsKey(UseUnsafeKey);

        using (var mirrorLogger = new ConsoleLogger("[Mirror] "))
        using (var dokanLogger = new ConsoleLogger("[Dokan] "))
        using (var dokan = new Dokan(dokanLogger))
        {
            var mirror = unsafeReadWrite
                ? new UnsafeMirror(mirrorLogger, mirrorPath)
                : new Mirror(mirrorLogger, mirrorPath);

            // Creation of shares 1 till 128
            for (int i = 1; i <= 128; i++)
            {
                // Delay for a more smooth creation of the shares
                await Task.Delay(1000);

                // Creation of the correct mount path so each share gets its own mount
                var actualMountPath = mountPath + i + @"\";
                dokanLogger.Warn($"Started mounting on path {actualMountPath}");

                // If the mountPath does not exist create it.
                if (!Directory.Exists(actualMountPath))
                    Directory.CreateDirectory(actualMountPath);

                var dokanBuilder = new DokanInstanceBuilder(dokan)
                    .ConfigureLogger(() => dokanLogger)
                    .ConfigureOptions(options =>
                    {
                        options.Options = DokanOptions.DebugMode | DokanOptions.StderrOutput;
                        options.MountPoint = actualMountPath;
                    });

                // Run async so that the loop continues and the share does not close down .
                Task.Run(async () =>
                {
                    using (var dokanInstance = dokanBuilder.Build(mirror))
                    using (var notify = new Notify(mirrorPath, actualMountPath, dokanInstance))
                    {
                        // Debug message
                        dokanLogger.Warn($"Is dokany running for:{actualMountPath}. Answer: {dokanInstance.IsFileSystemRunning()}");

                        await dokanInstance.WaitForFileSystemClosedAsync(uint.MaxValue);
                    }
                });
            }
        }
        Console.WriteLine(@"Success");
    }
    catch (DokanException ex)
    {
        Console.WriteLine(@"Error: " + ex.Message);
    }
}

More details

image

Multible processes

I have also tested running multiple processes, which appears to work, but still remains limited to only 64 shares per process. While this approach offers some benefits, my goal is to enable the creation of more than 64 shares within a single process.

Logs

DebugView kernel logs.
image
DokanyDebugLogs1.txt

@tsdworks
Copy link

I have the same problem.

@Liryna
Copy link
Member

Liryna commented Dec 1, 2024

Hi @JFra-AME , Thanks for reporting this! From your logs all the mount succeeded. Could you add a line that tries to list the root folder after the mount ? This should output some debug logs and see which request is failing.

@JFra-AME
Copy link
Author

JFra-AME commented Dec 3, 2024

Hi @Liryna,

Thank you for the quick response!

I updated the code as shown below and observed the following results:

// Run async so that the loop keeps looping but the share does not close down.
_ = Task.Run(async () =>
{
    using (var dokanInstance = dokanBuilder.Build(mirror))
    using (var notify = new Notify(mirrorPath, actualMountPath, dokanInstance))
    {
        // Debug message
        dokanLogger.Warn($"Is dokany running for:{actualMountPath}. Answer: {dokanInstance.IsFileSystemRunning()}");

        // Put in a delay to make sure the drive is mounted before asking for its contents
        await Task.Delay(500);

        try
        {
            // Attempt to look into the mounted share and provide the root folder
            var files = Directory.EnumerateFileSystemEntries(actualMountPath);
            dokanLogger.Warn($"Contents of {actualMountPath}: {string.Join(", ", files)}");
        }
        catch (Exception ex)
        {
            dokanLogger.Error($"Failed to list contents of {actualMountPath}: {ex.Message}");
        }

        await dokanInstance.WaitForFileSystemClosedAsync(uint.MaxValue);
    }
});

Folder structure

For this test I used the following folder structure:
C:\Test\Test\Test.docx
Which will be mapped like this:
C:\Share{Number}\Test\Test.docx

DebugView

DebugView

Dokany logs

(Only the last ones should be interesting)
DokanyLogs_03-12-2024.txt

Findings

The same error returned by File Explorer, "Parameter is incorrect," is thrown when attempting to open the directory in the code.

@JFra-AME
Copy link
Author

JFra-AME commented Dec 5, 2024

I've done a bit more investigation, and it turns out this bug doesn't occur when enabling single-thread.

var dokanBuilder = new DokanInstanceBuilder(dokan)
    .ConfigureLogger(() => dokanLogger)
    .ConfigureOptions(options =>
    {
        options.Options = DokanOptions.DebugMode | DokanOptions.StderrOutput;
        options.MountPoint = actualMountPath;
        options.SingleThread = true;
    });

image

However, this is not a viable solution, but it is something interesting I discovered. Dokany is noticeably slower, and occasional exceptions can be observed.

@Liryna
Copy link
Member

Liryna commented Dec 10, 2024

I believe the limit is on the NTFS side which has a limit of number of reparse point (which is used by dokan to mount in the folder).
https://learn.microsoft.com/en-us/windows/win32/fileio/reparse-points

There is a limit of 63 reparse points on any given path.

NOTE: The limit can be reduced depending on the length of the reparse point. For example, if your reparse point targets a fully qualified path, the limit becomes 31.

To confirm this you could in the same instance of the process use a different NTFS drive to mount the shares folders after number 60 for example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants