From 7653e438a192dce1478e3d1ccfeb20355b7173ef Mon Sep 17 00:00:00 2001 From: Andrew Innes Date: Thu, 29 Jun 2023 14:36:31 +0800 Subject: [PATCH] Squashed commit of the following: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit commit 9cde9c07739f76a37d729d3a323f49f5d4bc100f Author: Andrew Innes Date: Wed Jun 28 19:27:10 2023 +0800 Revert various glitches Signed-off-by: Andrew Innes commit d0c8c0fb05088bb016bc208d5f8cb709195cff87 Author: Andrew Innes Date: Thu Jun 29 08:24:13 2023 +0800 Windows: znode: expose zfs_get_zplprop to libzpool Signed-off-by: Andrew Innes commit 3d747f29b2864b661223d09bc8375d34e2105825 Author: Richard Yao Date: Sun Dec 4 17:42:43 2022 -0500 Fix TOCTOU race in zpool_do_labelclear() Coverity reported a TOCTOU race in `zpool_do_labelclear()`. This is not believed to be a real security issue, but fixing it reduces the number of syscalls we do and will prevent other static analyzers from complaining about this. The code is expected to be equivalent. However, under rare circumstances, such as ELOOP, ENAMETOOLONG, ENOMEM, ENOTDIR and EOVERFLOW, we will display the error message that we currently display for the `open()` syscall rather than the one that we currently display for the `stat()` syscall. This is considered to be an improvement. Reported-by: Coverity (CID-1524188) Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 1e255365c9bf0e7858561d527c0ebdf8f90bc925 Author: Alexander Motin Date: Tue Jun 27 20:03:37 2023 -0400 ZIL: Fix another use-after-free. lwb->lwb_issued_txg can not be accessed after lwb_state is set to LWB_STATE_FLUSH_DONE and zl_lock is dropped, since the lwb may be freed by zil_sync(). We must save the txg number before that. This is similar to the 55b1842f92, but as I see the bug is not new. It existed for quite a while, just was not triggered due to smaller race window. Reviewed-by: Allan Jude Reviewed-by: Brian Atkinson Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14988 Closes #14999 commit 233893e7cb7a98895061100ef8363f0ac30204b5 Author: Alexander Motin Date: Tue Jun 27 20:00:30 2023 -0400 Use big transactions for small recordsize writes. When ZFS appends files in chunks bigger than recordsize, it borrows buffer from ARC and fills it before opening transaction. This supposed to help in case of page faults to not hold transaction open indefinitely. The problem appears when recordsize is set lower than default 128KB. Since each block is committed in separate transaction, per-transaction overhead becomes significant, and what is even worse, active use of of per-dataset and per-pool locks to protect space use accounting for each transaction badly hurts the code SMP scalability. The same transaction size limitation applies in case of file rewrite, but without even excuse of buffer borrowing. To address the issue, disable the borrowing mechanism if recordsize is smaller than default and the write request is 4x bigger than it. In such case writes up to 32MB are executed in single transaction, that dramatically reduces overhead and lock contention. Since the borrowing mechanism is not used for file rewrites, and it was never used by zvols, which seem to work fine, I don't think this change should create significant problems, partially because in addition to the borrowing mechanism there are also used pre-faults. My tests with 4/8 threads writing several files same time on datasets with 32KB recordsize in 1MB requests show reduction of CPU usage by the user threads by 25-35%. I would measure it in GB/s, but at that block size we are now limited by the lock contention of single write issue taskqueue, which is a separate problem we are going to work on. Reviewed-by: Brian Atkinson Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14964 commit aea27422747921798a9b9e1b8e0f6230d5672ba5 Author: Laevos <5572812+Laevos@users.noreply.github.com> Date: Tue Jun 27 16:58:32 2023 -0700 Remove unnecessary commas in zpool-create.8 Reviewed-by: Brian Atkinson Reviewed-by: Brian Behlendorf Signed-off-by: Laevos <5572812+Laevos@users.noreply.github.com> Closes #15011 commit 38a821c0d8f6bb51a866354e76078abf6a6ba1fc Author: Alexander Motin Date: Tue Jun 27 12:09:48 2023 -0400 Another set of vdev queue optimizations. Switch FIFO queues (SYNC/TRIM) and active queue of vdev queue from time-sorted AVL-trees to simple lists. AVL-trees are too expensive for such a simple task. To change I/O priority without searching through the trees, add io_queue_state field to struct zio. To not check number of queued I/Os for each priority add vq_cqueued bitmap to struct vdev_queue. Update it when adding/removing I/Os. Make vq_cactive a separate array instead of struct vdev_queue_class member. Together those allow to avoid lots of cache misses when looking for work in vdev_queue_class_to_issue(). Introduce deadline of ~0.5s for LBA-sorted queues. Before this I saw some I/Os waiting in a queue for up to 8 seconds and possibly more due to starvation. With this change I no longer see it. I had to slightly more complicate the comparison function, but since it uses all the same cache lines the difference is minimal. For a sequential I/Os the new code in vdev_queue_io_to_issue() actually often uses more simple avl_first(), falling back to avl_find() and avl_nearest() only when needed. Arrange members in struct zio to access only one cache line when searching through vdev queues. While there, remove io_alloc_node, reusing the io_queue_node instead. Those two are never used same time. Remove zfs_vdev_aggregate_trim parameter. It was disabled for 4 years since implemented, while still wasted time maintaining the offset-sorted tree of TRIM requests. Just remove the tree. Remove locking from txg_all_lists_empty(). It is racy by design, while 2 pair of locks/unlocks take noticeable time under the vdev queue lock. With these changes in my tests with volblocksize=4KB I measure vdev queue lock spin time reduction by 50% on read and 75% on write. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14925 commit 1737e75ab4e09a2d20e7cc64fa83dae047a302e9 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Mon Jun 26 16:57:12 2023 -0400 Add a delay to tearing down threads. It's been observed that in certain workloads (zvol-related being a big one), ZFS will end up spending a large amount of time spinning up taskqs only to tear them down again almost immediately, then spin them up again... I noticed this when I looked at what my mostly-idle system was doing and wondered how on earth taskq creation/destroy was a bunch of time... So I added a configurable delay to avoid it tearing down tasks the first time it notices them idle, and the total number of threads at steady state went up, but the amount of time being burned just tearing down/turning up new ones almost vanished. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #14938 commit 68b8e2ffab23cba6ae87f18c59b044c833934f2f Author: Alexander Motin Date: Sat Jun 17 22:51:37 2023 -0400 Fix memory leak in zil_parse(). 482da24e2 missed arc_buf_destroy() calls on log parse errors, possibly leaking up to 128KB of memory per dataset during ZIL replay. Reviewed-by: Brian Behlendorf Reviewed-by: Paul Dagnelie Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14987 commit ea0d03a8bd040e438bcaa43b8e449cbf717e14f3 Author: George Amanakis Date: Thu Jun 15 21:45:36 2023 +0200 Shorten arcstat_quiescence sleep time With the latest L2ARC fixes, 2 seconds is too long to wait for quiescence of arcstats like l2_size. Shorten this interval to avoid having the persistent L2ARC tests in ZTS prematurely terminated. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14981 commit 3fa141285b8105b3cc11c1296b77ad6d24250f2c Author: Alexander Motin Date: Thu Jun 15 13:49:03 2023 -0400 Remove ARC/ZIO physdone callbacks. Those callbacks were introduced many years ago as part of a bigger patch to smoothen the write throttling within a txg. They allow to account completion of individual physical writes within a logical one, improving cases when some of physical writes complete much sooner than others, gradually opening the write throttle. Few years after that ZFS got allocation throttling, working on a level of logical writes and limiting number of writes queued to vdevs at any point, and so limiting latency distribution between the physical writes and especially writes of multiple copies. The addition of scheduling deadline I proposed in #14925 should further reduce the latency distribution. Grown memory sizes over the past 10 years should also reduce importance of the smoothing. While the use of physdone callback may still in theory provide some smoother throttling, there are cases where we simply can not afford it. Since dirty data accounting is protected by pool-wide lock, in case of 6-wide RAIDZ, for example, it requires us to take it 8 times per logical block write, creating huge lock contention. My tests of this patch show radical reduction of the lock spinning time on workloads when smaller blocks are written to RAIDZ pools, when each of the disks receives 8-16KB chunks, but the total rate reaching 100K+ blocks per second. Same time attempts to measure any write time fluctuations didn't show anything noticeable. While there, remove also io_child_count/io_parent_count counters. They are used only for couple assertions that can be avoided. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14948 commit 9efc735904d194987f06870f355e08d94e39ab81 Author: Brian Behlendorf Date: Wed Jun 14 10:04:05 2023 -0500 ZTS: Skip send_raw_ashift on FreeBSD On FreeBSD 14 this test runs slowly in the CI environment and is killed by the 10 minute timeout. Skip the test on FreeBSD until the slow down is resolved. Signed-off-by: Brian Behlendorf Issue #14961 commit 9c54894bfc77f585806984f44c70a839543e6715 Author: Alexander Motin Date: Wed Jun 14 11:02:27 2023 -0400 Switch refcount tracking from lists to AVL-trees. With large number of tracked references list searches under the lock become too expensive, creating enormous lock contention. On my tests with ZFS_DEBUG enabled this increases write throughput with 32KB blocks from ~1.2GB/s to ~7.5GB/s. Reviewed-by: Brian Atkinson Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14970 commit 4e62540827a6ed15e08b2a627896d24bc661fa38 Author: George Amanakis Date: Wed Jun 14 17:01:17 2023 +0200 Store the L2ARC device ashift in the vdev label If this is not done, and the pool has an ashift other than the default (at the moment 9) then the following happens: 1) vdev_alloc() assigns the ashift of the pool to L2ARC device, but upon export it is not stored anywhere 2) at the first import, vdev_open() sees an vdev_ashift() of 0 and assigns the logical_ashift, which is 9 3) reading the contents of L2ARC, including the header fails 4) L2ARC buffers are not restored in ARC. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14313 Closes #14963 commit adaa3e64ea46f21cc5f544228c48363977b7733e Author: George Amanakis Date: Sat Jun 10 02:05:47 2023 +0200 Fix the L2ARC write size calculating logic (2) While commit bcd5321 adjusts the write size based on the size of the log block, this happens after comparing the unadjusted write size to the evicted (target) size. In this case l2ad_hand will exceed l2ad_evict and violate an assertion at the end of l2arc_write_buffers(). Fix this by adding the max log block size to the allocated size of the buffer to be committed before comparing the result to the target size. Also reset the l2arc_trim_ahead ZFS module variable when the adjusted write size exceeds the size of the L2ARC device. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14936 Closes #14954 commit 67118a7d6e74a6e818127096162478017610d13e Author: Andrew Innes Date: Wed Jun 28 12:31:10 2023 +0800 Windows: Finally drop long disabled vdev cache. Signed-off-by: Andrew Innes commit 5d80c98c28c931339138753a4e4c1156dbf951f4 Author: Alexander Motin Date: Fri Jun 9 15:40:55 2023 -0400 Finally drop long disabled vdev cache. It was a vdev level read cache, designed to aggregate many small reads by speculatively issuing bigger reads instead and caching the result. But since it has almost no idea about what is going on with exception of ZIO_FLAG_DONT_CACHE flag set by higher layers, it was found to make more harm than good, for which reason it was disabled for the past 12 years. These days we have much better instruments to enlarge the I/Os, such as speculative and prescient prefetches, I/O scheduler, I/O aggregation etc. Besides just the dead code removal this removes one extra mutex lock/unlock per write inside vdev_cache_write(), not otherwise disabled and trying to do some work. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14953 commit 1f1ab33781b5736654b988e2e618ea79788fa1f7 Author: Brian Behlendorf Date: Fri Jun 9 11:10:01 2023 -0700 ZTS: Skip checkpoint_discard_busy Until the ASSERT which is occasionally hit while running checkpoint_discard_busy is resolved skip this test case. Signed-off-by: Brian Behlendorf Issue #12053 Closes #14952 commit b94049c2cbedbbe2af8e629bf974a6ed93f11acb Author: Alexander Motin Date: Fri Jun 9 13:14:05 2023 -0400 Improve l2arc reporting in arc_summary. - Do not report L2ARC as FAULTED in presence of in-flight writes. - Report read and write I/Os, bytes and errors. - Remove few numbers not important to average user. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #12304 Closes #14946 commit 31044b5cfb6f91d376034c4d6374f61baaf03232 Author: Andrew Innes Date: Wed Jun 28 12:00:39 2023 +0800 Windows: Use list_remove_head() where possible. Signed-off-by: Andrew Innes commit 32eda54d0d75a94b6aa71dc80aa958095feb8011 Author: Alexander Motin Date: Fri Jun 9 13:12:52 2023 -0400 Use list_remove_head() where possible. ... instead of list_head() + list_remove(). On FreeBSD the list functions are not inlined, so in addition to more compact code this also saves another function call. Reviewed-by: Brian Atkinson Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14955 commit fe7693a3f87229d1ae93b5ce2bb84d8bb86a9f5c Author: Alexander Motin Date: Fri Jun 9 13:08:05 2023 -0400 ZIL: Fix race introduced by f63811f0721. We are not allowed to access lwb after setting LWB_STATE_FLUSH_DONE state and dropping zl_lock, since it may be freed by zil_sync(). To free itxs and waiters after dropping the lock we need to move lwb_itxs and lwb_waiters lists elements to local storage. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14957 Closes #14959 commit 44c5a0c92f98e8c21221bd7051729d1947a10736 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Wed Jun 7 14:14:05 2023 -0400 Revert "systemd: Use non-absolute paths in Exec* lines" This reverts commit 79b20949b25c8db4d379f6486b0835a6613b480c since it doesn't work with the systemd version shipped with RHEL7-based systems. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #14943 Closes #14945 commit ba5af00257eb4eb3363f297819a21c4da811392f Author: Brian Behlendorf Date: Wed Jun 7 10:43:43 2023 -0700 Linux: Never sleep in kmem_cache_alloc(..., KM_NOSLEEP) (#14926) When a kmem cache is exhausted and needs to be expanded a new slab is allocated. KM_SLEEP callers can block and wait for the allocation, but KM_NOSLEEP callers were incorrectly allowed to block as well. Resolve this by attempting an emergency allocation as a best effort. This may fail but that's fine since any KM_NOSLEEP consumer is required to handle an allocation failure. Signed-off-by: Brian Behlendorf Reviewed-by: Adam Moss Reviewed-by: Brian Atkinson Reviewed-by: Richard Yao Reviewed-by: Tony Hutter commit d4ecd4efde1692641d1d0b89851e7a15e90632f8 Author: George Amanakis Date: Tue Jun 6 21:32:37 2023 +0200 Fix the L2ARC write size calculating logic l2arc_write_size() should return the write size after adjusting for trim and overhead of the L2ARC log blocks. Also take into account the allocated size of log blocks when deciding when to stop writing buffers to L2ARC. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14939 commit 8692ab174e18faf444681d67d7ea4418600553cc Author: Rob Norris Date: Wed Mar 15 18:18:10 2023 +1100 zdb: add -B option to generate backup stream This is more-or-less like `zfs send`, but specifying the snapshot by its objset id for situations where it can't be referenced any other way. Sponsored-By: Klara, Inc. Reviewed-by: Tino Reichardt Reviewed-by: WHR Signed-off-by: Rob Norris Closes #14642 commit df84ca3f3bf9f265ebc76de17394df529fd07af6 Author: Andrew Innes Date: Wed Jun 28 11:05:55 2023 +0800 Windows: znode: expose zfs_get_zplprop to libzpool Signed-off-by: Andrew Innes commit 944c58247a13a92c9e4ffb2c0a9e6b6293dca37e Author: Rob Norris Date: Sun Jun 4 11:14:20 2023 +1000 znode: expose zfs_get_zplprop to libzpool There's no particular reason this function should be kernel-only, and I want to use it (indirectly) from zdb. I've moved it to zfs_znode.c because libzpool does not compile in zfs_vfsops.c, and this at least matches the header its imported from. Sponsored-By: Klara, Inc. Reviewed-by: Tino Reichardt Reviewed-by: WHR Signed-off-by: Rob Norris Closes #14642 commit 429f58cdbb195c8d50ed95c7309ee54d37526b70 Author: Alexander Motin Date: Mon Jun 5 14:51:44 2023 -0400 Introduce zfs_refcount_(add|remove)_few(). There are two places where we need to add/remove several references with semantics of zfs_refcount_(add|remove). But when debug/tracing is disabled, it is a crime to run multiple atomic_inc() in a loop, especially under congested pool-wide allocator lock. Introduced new functions implement the same semantics as the loop, but without overhead in production builds. Reviewed-by: Rich Ercolani Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14934 commit 077c2f359feb69a13bee37ac4220d271d1c7bf27 Author: Brian Behlendorf Date: Mon Jun 5 11:08:24 2023 -0700 Linux 6.3 compat: META (#14930) Update the META file to reflect compatibility with the 6.3 kernel. Signed-off-by: Brian Behlendorf Reviewed-by: Tony Hutter commit c2fcd6e484107fc7435087771757e88ba84f6093 Author: Graham Perrin Date: Fri Jun 2 19:25:13 2023 +0100 zfs-create(8): ZFS for swap: caution, clarity Make the section heading more generic (the section relates to ZFS files as well as ZFS volumes). Swapping to a ZFS volume is prone to deadlock. Remove the related instruction, direct readers to OpenZFS FAQ. Related, but not linked from within the manual page: (Using a zvol for a swap device on Linux). Reviewed-by: Brian Behlendorf Signed-off-by: Graham Perrin Issue #7734 Closes #14756 commit 251dbe83e14085a26100aa894d79772cbb69dcda Author: Alexander Motin Date: Fri Jun 2 14:01:58 2023 -0400 ZIL: Allow to replay blocks of any size. There seems to be no reason for ZIL blocks to be limited by 128KB other than replay code is written in such a way. This change does not increase the limit yet, just removes the artificial limitation. Avoided extra memcpy() may save us a second during replay. Reviewed-by: Brian Behlendorf Reviewed-by: Prakash Surya Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14910 commit 76170249d538965655dbd3206cd59566b1d3944b Author: Val Packett Date: Thu May 11 18:16:57 2023 -0300 PAM: enable testing on FreeBSD Reviewed-by: Brian Behlendorf Reviewed-by: Felix Dörre Signed-off-by: Val Packett Closes #14834 commit d1b68a45441cae8c399a8a3ed60b29726ed031ff Author: Val Packett Date: Fri May 5 22:17:12 2023 -0300 PAM: support password changes even when not mounted There's usually no requirement that a user be logged in for changing their password, so let's not be surprising here. We need to use the fetch_lazy mechanism for the old password to avoid a double prompt for it, so that mechanism is now generalized a bit. Reviewed-by: Brian Behlendorf Reviewed-by: Felix Dörre Signed-off-by: Val Packett Closes #14834 commit 7424feff72f1e17ea27bcfe0d36cabce7c732eea Author: Val Packett Date: Fri May 5 22:34:58 2023 -0300 PAM: add 'uid_min' and 'uid_max' options for changing the uid range Instead of a fixed >=1000 check, allow the configuration to override the minimum UID and add a maximum one as well. While here, add the uid range check to the authenticate method as well, and fix the return in the chauthtok method (seems very wrong to report success when we've done absolutely nothing). Reviewed-by: Brian Behlendorf Reviewed-by: Felix Dörre Signed-off-by: Val Packett Closes #14834 commit fc9e012f5fc7e7997acee2b6d8d759622b319f0e Author: Val Packett Date: Fri May 5 22:02:13 2023 -0300 PAM: add 'forceunmount' flag Probably not always a good idea, but it's nice to have the option. It is a workaround for FreeBSD calling the PAM session end earier than the last process is actually done touching the mount, for example. Reviewed-by: Brian Behlendorf Reviewed-by: Felix Dörre Signed-off-by: Val Packett Closes #14834 commit a39ed83bd31cc0c8c98dc3c4cc3d11b03d9af620 Author: Val Packett Date: Fri May 5 19:35:57 2023 -0300 PAM: add 'recursive_homes' flag to use with 'prop_mountpoint' It's not always desirable to have a fixed flat homes directory. With the 'recursive_homes' flag, 'prop_mountpoint' search would traverse the whole tree starting at 'homes' (which can now be '*' to mean all pools) to find a dataset with a mountpoint matching the home directory. Reviewed-by: Brian Behlendorf Reviewed-by: Felix Dörre Signed-off-by: Val Packett Closes #14834 commit 7f8d5ef815b7559fcc671ff2add33ba9c2a74867 Author: Val Packett Date: Fri May 5 21:56:39 2023 -0300 PAM: use boolean_t for config flags Since we already use boolean_t in the file, we can use it here. Reviewed-by: Brian Behlendorf Reviewed-by: Felix Dörre Signed-off-by: Val Packett Closes #14834 commit e2872932c85189f06a68f0ad10bd8eb6895d79c2 Author: Val Packett Date: Fri May 5 20:00:48 2023 -0300 PAM: do not fail to mount if the key's already loaded If we're expecting a working home directory on login, it would be rather frustrating to not have it mounted just because it e.g. failed to unmount once on logout. Reviewed-by: Brian Behlendorf Reviewed-by: Felix Dörre Signed-off-by: Val Packett Closes #14834 commit b897137e2044c3ef6120820f753d940b7dfb58be Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Wed May 31 19:58:41 2023 -0400 Revert "initramfs: use `mount.zfs` instead of `mount`" This broke mounting of snapshots on / for users. See https://github.com/openzfs/zfs/issues/9461#issuecomment-1376162949 for more context. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #14908 commit 10cde4f8f60d4d55887d7122a5742e6e4f90280c Author: Luís Henriques <73643340+lumigch@users.noreply.github.com> Date: Tue May 30 23:15:24 2023 +0100 Fix NULL pointer dereference when doing concurrent 'send' operations A NULL pointer will occur when doing a 'zfs send -S' on a dataset that is still being received. The problem is that the new 'send' will rightfully fail to own the datasets (i.e. dsl_dataset_own_force() will fail), but then dmu_send() will still do the dsl_dataset_disown(). Reviewed-by: Brian Behlendorf Signed-off-by: Luís Henriques Closes #14903 Closes #14890 commit 12452d79a3fd29af1dc0b95f3e367e3ce339702b Author: Brian Behlendorf Date: Mon May 29 12:55:35 2023 -0700 ZTS: zvol_misc_trim disable blk mq Disable the zvol_misc_fua.ksh and zvol_misc_trim.ksh test cases on impacted kernels. This issue is being actively worked in #14872 and as part of that fix this commit will be reverted. VERIFY(zh->zh_claim_txg == 0) failed PANIC at zil.c:904:zil_create() Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Issue #14872 Closes #14870 commit 803c04f233e60a2d23f0463f299eba96c0968602 Author: Richard Yao Date: Fri May 26 18:47:52 2023 -0400 Use __attribute__((malloc)) on memory allocation functions This informs the C compiler that pointers returned from these functions do not alias other functions, which allows it to do better code optimization and should make the compiled code smaller. References: https://stackoverflow.com/a/53654773 https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-malloc-function-attribute https://clang.llvm.org/docs/AttributeReference.html#malloc Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14827 commit 64d8bbe15f77876ae9639b9971a743776a41bf9a Author: Brian Behlendorf Date: Fri May 26 15:39:23 2023 -0700 ZTS: Add zpool_resilver_concurrent exception The zpool_resilver_concurrent test case requires the ZED which is not used on FreeBSD. Add this test to the known list of skipped tested for FreeBSD. Signed-off-by: Brian Behlendorf Closes #14904 commit e396d30d29ed131194605222e6ba1fec1ef8b2ca Author: Mike Swanson Date: Fri May 26 15:37:15 2023 -0700 Add compatibility symlinks for FreeBSD 12.{3,4} and 13.{0,1,2} Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Mike Swanson Closes #14902 commit f6dd0b8c1cc41707d299b7123f80912f43d03340 Author: Colm Date: Fri May 26 10:04:19 2023 -0700 Adding new read-only compatible zpool features to compatibility.d/grub2 GRUB2 is compatible with all "read-only compatible" features, so it is safe to add new features of this type to the grub2 compatibility list. We generally want to include all compatible features, to minimize the differences between grub2-compatible pools and no-compatibility pools. Adding new properties `livelist` and `zpool_checkpoint` accordingly. Also adding them to the man page which references this file as an example, for consistency. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Colm Buckley Closes #14893 commit 013d3a1e0e00d83dabe70837b23dab48c1bac592 Author: Richard Yao Date: Fri May 26 13:03:12 2023 -0400 btree: Implement faster binary search algorithm This implements a binary search algorithm for B-Trees that reduces branching to the absolute minimum necessary for a binary search algorithm. It also enables the compiler to inline the comparator to ensure that the only slowdown when doing binary search is from waiting for memory accesses. Additionally, it instructs the compiler to unroll the loop, which gives an additional 40% improve with Clang and 8% improvement with GCC. Consumers must opt into using the faster algorithm. At present, only B-Trees used inside kernel code have been modified to use the faster algorithm. Micro-benchmarks suggest that this can improve binary search performance by up to 3.5 times when compiling with Clang 16 and up to 1.9 times when compiling with GCC 12.2. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14866 commit 1854df330aa57cda39f076e8ab11e17ca3697bb8 Author: George Amanakis Date: Fri May 26 18:53:00 2023 +0200 Fix inconsistent definition of zfs_scrub_error_blocks_per_txg Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14894 commit 8735e6ac03742fcf43adde3ce127af698a32c53a Author: Damiano Albani Date: Fri May 26 01:10:54 2023 +0200 Add missing files to Debian DKMS package Reviewed-by: Tino Reichardt Reviewed-by: Umer Saleem Reviewed-by: Brian Behlendorf Signed-off-by: Damiano Albani Closes #14887 Closes #14889 commit d439021bd05a5cc0bb271a5470abb67af2f7bcda Author: Brian Behlendorf Date: Thu May 25 13:53:08 2023 -0700 Update compatibility.d files Add an openzfs-2.2 compatibility file for the next release. Edon-R support has been enabled for FreeBSD removing the need for different FreeBSD and Linux files. Symlinks for the -linux and -freebsd names are created for any scripts expecting that convention. Additionally, a symlink for ubunutu-22.04 was added. Signed-off-by: Brian Behlendorf Closes #14833 commit da54d5f3f9576b958e3eadf4f4d8f68c91b3d6e4 Author: Alexander Motin Date: Thu May 25 16:51:53 2023 -0400 zil: Add some more statistics. In addition to a number of actual log bytes written, account also a total written bytes including padding and total allocated bytes (bytes <= write <= alloc). It should allow to monitor zil traffic and space efficiency. Add dtrace probe for zil block size selection. Make zilstat report more information and fit it into less width. Reviewed-by: Ameer Hamza Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14863 commit faa4955023d089668bd6c564c195a933d1eac455 Author: Alexander Motin Date: Thu May 25 12:48:43 2023 -0400 ZIL: Reduce scope of per-dataset zl_issuer_lock. Before this change ZIL copied all log data while holding the lock. It caused huge lock contention on workloads with many big parallel writes. This change splits the process into two parts: first, zil_lwb_assign() estimates the log space needed for all transactions, and zil_lwb_write_close() allocates blocks and zios while holding the lock, then, after the lock in dropped, zil_lwb_commit() copies the data, and zil_lwb_write_issue() issues the I/Os. Also while there slightly reduce scope of zl_lock. Reviewed-by: Paul Dagnelie Reviewed-by: Prakash Surya Reviewed-by: Richard Yao Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14841 commit f77b9f7ae83834ade1da21cfc16b8a273df3acfc Author: Dimitri John Ledkov <19779+xnox@users.noreply.github.com> Date: Wed May 24 20:31:28 2023 +0100 systemd: Use non-absolute paths in Exec* lines Since systemd v239, Exec* binaries are resolved from PATH when they are not-absolute. Switch to this by default for ease of downstream maintenance. Many downstream distributions move individual binaries to locations that existing compile-time configurations cannot accommodate. Reviewed-by: Brian Behlendorf Signed-off-by: Dimitri John Ledkov Closes #14880 commit 4bfb9d28cffd4dfeb4b91359b497d100f668bb34 Author: Akash B Date: Thu May 25 00:58:09 2023 +0530 Fix concurrent resilvers initiated at same time For draid vdevs it was possible to initiate both the sequential and healing resilver at same time. This fixes the following two scenarios. 1) There's a window where a sequential rebuild can be started via ZED even if a healing resilver has been scheduled. - This is fixed by adding additional check in spa_vdev_attach() for any scheduled resilver and return appropriate error code when a resilver is already in progress. 2) It was possible for zpool clear to start a healing resilver when it wasn't needed at all. This occurs because during a vdev_open() the device is presumed to be healthy not until the device is validated by vdev_validate() and it's set unavailable. However, by this point an async resilver will have already been requested if the DTL isn't empty. - This is fixed by cancelling the SPA_ASYNC_RESILVER request immediately at the end of vdev_reopen() when a resilver is unneeded. Finally, added a testcase in ZTS for verification. Reviewed-by: Brian Behlendorf Reviewed-by: Dipak Ghosh Signed-off-by: Akash B Closes #14881 Closes #14892 commit c9bb406d177a00aa1f0058d29aeb29e478223273 Author: youzhongyang Date: Wed May 24 15:23:42 2023 -0400 Linux 6.4 compat: reclaimed_slab renamed to reclaimed Reviewed-by: Richard Yao Reviewed-by: Brian Atkinson Reviewed-by: Brian Behlendorf Signed-off-by: Youzhong Yang Closes #14891 commit 79e61a873b136f13fcf140beb925ceddc1f94767 Author: Brian Atkinson Date: Fri May 19 16:05:53 2023 -0400 Hold db_mtx when updating db_state Commit 555ef90 did some general code refactoring for dmu_buf_will_not_fill() and dmu_buf_will_fill(). However, the db_mtx was not held when update db->db_state in those code block. The rest of the dbuf code always holds the db_mtx when updating db_state. This is important because cv_wait() db_changed is used to check for db_state changes. Updating dmu_buf_will_not_fill() and dmu_buf_will_fill() to hold the db_mtx when updating db_state. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Brian Atkinson Closes #14875 commit d7be0cdf93a568b6c9b4a4e15a88a5d88ebbb764 Author: Brian Behlendorf Date: Fri May 19 13:05:09 2023 -0700 Probe vdevs before marking removed Before allowing the ZED to mark a vdev as REMOVED due to a hotplug event confirm that it is non-responsive with probe. Any device which can be successfully probed should be left ONLINE to prevent a healthy pool from being incorrectly SUSPENDED. This may occur for at least the following two scenarios. 1) Drive expansion (zpool online -e) in VMware environments. If, during the partition resize operation, a partition is removed and re-created then udev will send a removed event. 2) Re-scanning the namespaces of an NVMe device (nvme ns-rescan) may result in a udev remove and add event being delivered. Finally, update the ZED to only kick in a spare when the removal was successful. Reviewed-by: Ameer Hamza Reviewed-by: Tony Hutter Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Issue #14859 Closes #14861 commit 054bb22686045ea1499065a4456568f0c21d939b Author: Andrew Innes Date: Tue Jun 27 09:20:56 2023 +0800 Windows: Teach zpool scrub to scrub only blocks in error log Signed-off-by: Andrew Innes commit b61e89a3e68ae19819493183ff3d1fe7bf4ffe2b Author: George Amanakis Date: Fri Dec 17 21:35:28 2021 +0100 Teach zpool scrub to scrub only blocks in error log Added a flag '-e' in zpool scrub to scrub only blocks in error log. A user can pause, resume and cancel the error scrub by passing additional command line arguments -p -s just like a regular scrub. This involves adding a new flag, creating new libzfs interfaces, a new ioctl, and the actual iteration and read-issuing logic. Error scrubbing is executed in multiple txg to make sure pool performance is not affected. Reviewed-by: Brian Behlendorf Reviewed-by: Tony Hutter Co-authored-by: TulsiJain tulsi.jain@delphix.com Signed-off-by: George Amanakis Closes #8995 Closes #12355 commit 61bfb3cb5dd792ec7ca0fbfca59b165f3ddbe1f5 Author: Brian Behlendorf Date: Thu May 18 10:02:20 2023 -0700 Add the ability to uninitialize zpool initialize functions well for touching every free byte...once. But if we want to do it again, we're currently out of luck. So let's add zpool initialize -u to clear it. Co-authored-by: Rich Ercolani Signed-off-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #12451 Closes #14873 commit 855b62942d4ca5dab3d65b7000f9d284fd1560bb Author: Antonio Russo Date: Mon May 15 17:11:33 2023 -0600 test-runner: pass kmemleak and kmsg to Cmd.run test-runner.py orchestrates all of the ZTS executions. The `Cmd` object manages these process, and its `run` method specifically invokes these possibly long-running processes, possibly retrying in the event of a timeout. Since its inception, memory leak detection using the kmemleak infrastructure [1], and kernel logging [2] have been added to this run mechanism. However, the callback to cull a process beyond its timeout threshold, `kill_cmd`, has evaded modernization by both of these changes. As a result, this function fails to properly invoke `run`, leading to an untrapped exception and unreported test failure. This patch extends `kill_cmd` to receive these kernel devices through the `options` parameter, and regularizes all the `.run` calls from `Cmd`, and its subclasses, to accept that parameter. [1] Commit a69765ea5b563e0cd4d15fac4b1ac08c6ccf12d1 [2] Commit fc2c0256c55a2859d1988671b0896d22b75c8aba Reviewed-by: John Wren Kennedy Signed-off-by: Antonio Russo Closes #14849 commit 537939565123fd2afa097e9a56ee3efd28779e5f Author: Richard Yao Date: Fri May 12 17:10:14 2023 -0400 Fix undefined behavior in spa_sync_props() 8eae2d214cfa53862833eeeda9a5c1e9d5ded47d caused Coverity to begin complaining about "Improper use of negative value" in two places in spa_sync_props() because Coverity correctly inferred from `prop == ZPOOL_PROP_INVAL` that prop could be -1 while both zpool_prop_to_name() and zpool_prop_get_type() use it an array index, which is undefined behavior. Assuming that the system does not panic from an attempt to read invalid memory, the case statement for ZPOOL_PROP_INVAL will ensure that only user properties will reach this code when prop is ZPOOL_PROP_INVAL, such that execution will continue safely. However, if we are unlucky enough to read invalid memory, then the system will panic. This issue predates the patch that caused coverity to begin complaining. Thankfully, our userland tools do not pass nonsense to us, so this bug should not be triggered unless a future userland tool attempts to set a property that we do not understand. Reported-by: Coverity (CID-1561129) Reported-by: Coverity (CID-1561130) Reviewed-by: Brian Behlendorf Reviewed-by: George Amanakis Signed-off-by: Richard Yao Closes #14860 commit 02351b380f0430980bfb92e83d0800df104bd06a Author: Richard Yao Date: Fri May 12 16:47:56 2023 -0400 Fix use after free regression in spa_remove_healed_errors() 6839ec6f1098c28ff7b772f1b31b832d05e6b567 placed code in spa_remove_healed_errors() that uses a pointer after the kmem_free() call that frees it. Reported-by: Coverity (CID-1562375) Reviewed-by: Brian Behlendorf Reviewed-by: George Amanakis Signed-off-by: Richard Yao Closes #14860 commit e9b315ffb79ff6419694a2713fcd5fd448317904 Author: Andrew Innes Date: Mon May 15 13:52:35 2023 +0800 Use python3 on windows commit 3346a5b78c2db15801ce54a70a323952fdf67fa5 Author: Jorgen Lundman Date: Thu Jun 22 08:56:38 2023 +0900 zfs_write() ignores errors If files were advanced by zfs_freesp() we ignored any errors returned by it. Signed-off-by: Jorgen Lundman commit cce49c08316bc6a5dff287f4fa15856e26d5b18a Author: Jorgen Lundman Date: Thu Jun 22 08:55:55 2023 +0900 Correct Stream event path The Stream path events used the incorrect name "stream", now uses "file.txt:stream" as per ntfs. Signed-off-by: Jorgen Lundman commit 0f83d31e288d789fb4e10a7e4b12e27887820498 Author: Jorgen Lundman Date: Wed Jun 21 14:30:13 2023 +0900 Add stub for file_hard_link_information() Signed-off-by: Jorgen Lundman commit 8d6db9490364e4d281546445571d2ca9d5abda22 Author: Jorgen Lundman Date: Wed Jun 21 14:29:43 2023 +0900 Return correct FileID in dirlist Signed-off-by: Jorgen Lundman commit 4c011397229e3c38259d6956458a4fd287dca72d Author: Andrew Innes Date: Wed Jun 21 10:17:30 2023 +0800 Fix logic (#232) Signed-off-by: Andrew Innes commit 467436b676ad897025b7ed90d8f033969da441cc Author: Andrew Innes Date: Wed Jun 21 09:47:38 2023 +0800 Run winbtrfs tests by default (#231) Signed-off-by: Andrew Innes commit 56eca2a5d116c66b10579f9cf6d5f271991c7e2e Author: Jorgen Lundman Date: Wed Jun 21 09:54:00 2023 +0900 SetFilePositionInformation SetFileValidDataLengthInformation Signed-off-by: Jorgen Lundman commit b4fbbda470f27aee565dfa9bc0d68217b969339c Author: Andrew Innes Date: Tue Jun 20 16:33:12 2023 +0800 Add sleep to tests (#230) Signed-off-by: Andrew Innes commit 94f1f52807d1f8c0c2931e9e52b91f0ce5e488f4 Author: Jorgen Lundman Date: Tue Jun 20 16:53:50 2023 +0900 CreateFile of newfile:newstream should create both In addition, many more stream fixes, illegal chars, and names Signed-off-by: Jorgen Lundman commit 894d512880d39ecf40e841c6d7b73157dfe397e0 Author: Jorgen Lundman Date: Tue Jun 20 08:41:37 2023 +0900 Windows streams should return parent file ID When asked for File ID of a stream, it should return the FileID of the parent file, which is two levels up. Signed-off-by: Jorgen Lundman commit 0cc45d2154a2866b2f494c3790a57555c29e60c3 Author: Jorgen Lundman Date: Tue Jun 20 08:32:44 2023 +0900 Support FILE_STANDARD_INFORMATION_EX Signed-off-by: Jorgen Lundman commit a6edd02999d581db56f4a53567f4c5db11778f64 Author: Jorgen Lundman Date: Mon Jun 19 10:36:13 2023 +0900 Add xattr compat code from upstream and adjust calls to new API calls. This adds xattr=sa support to Windows. Signed-off-by: Jorgen Lundman commit 0e1476a3942990385d32c02403ebe2c815d567db Author: Jorgen Lundman Date: Wed Jun 14 11:56:09 2023 +0900 Set EA can panic Signed-off-by: Jorgen Lundman commit 4a1adef6b8c2851195d692a42d5718c9a1b03490 Author: Jorgen Lundman Date: Wed Jun 14 09:49:57 2023 +0900 Incorrect MAXPATH used in delete entry Signed-off-by: Jorgen Lundman commit 2c0d119e37cb3eed1acac90efa9fe0f8c173e0f0 Author: Jorgen Lundman Date: Tue Jun 13 16:19:42 2023 +0900 Large changes fixing FS notify events Some incorrect behavior still, query name of a stream is wrong. Signed-off-by: Jorgen Lundman commit 5b2b2b0550a493497a0b460206079fd57c639543 Author: Jorgen Lundman Date: Tue May 16 14:42:52 2023 +0900 file name and file full information buffer overrun When a buffer is not big enough, we would still null terminate on the full string, beyond the supplied buffer. Signed-off-by: Jorgen Lundman commit 94bfb92951a5ccdef7b2a1fb818fafdafbc4fff0 Author: Jorgen Lundman Date: Tue May 16 11:48:12 2023 +0900 Correct Query EA and Query Streams Which includes: * NextEntryOffset is not offset from Buffer, but from one struct to the next struct. * Pack only complete EAs, and return Overflow if does not fit * query file EA information would return from Information=size * Call cleareaszie on VP when EAs have changed Signed-off-by: Jorgen Lundman commit 9c7a4071fcfc99c3308620fc1943355f9ade34b3 Author: Alexander Motin Date: Fri May 12 12:49:26 2023 -0400 zil: Free lwb_buf after write completion. There is no sense to keep that memory allocated during the flush. Reviewed-by: Brian Behlendorf Reviewed-by: Prakash Surya Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14855 commit 7e91b3222ddaadc10c92d1065529886dd3806acc Author: Alexander Motin Date: Fri May 12 12:14:29 2023 -0400 zil: Some micro-optimizations. Should not cause functional changes. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14854 commit 6b62c3b0e10de782c3aef0e1206aa48875519c4e Author: Don Brady Date: Fri May 12 10:12:28 2023 -0600 Refine special_small_blocks property validation When the special_small_blocks property is being set during a pool create it enforces a limit of 128KiB even if the pool's record size is larger. If the recordsize property is being set during a pool create, then use that value instead of the default SPA_OLD_MAXBLOCKSIZE value. Reviewed-by: Brian Behlendorf Signed-off-by: Don Brady Closes #13815 Closes #14811 commit d0ab2dddde618c394fa7fe88211276786ba8ca12 Author: Brian Behlendorf Date: Fri May 12 09:07:58 2023 -0700 ZTS: Add auto_replace_001_pos to exceptions The auto_replace_001_pos test case does not reliably pass on Fedora 37 and newer. Until the test case can be updated to make it reliable add it to the list of "maybe" exceptions on Linux. Signed-off-by: Brian Behlendorf Issue #14851 Closes #14852 commit 1e3e7a103a5026e9a2005acec7017e4024d95115 Author: Pawel Jakub Dawidek Date: Tue May 9 22:32:30 2023 -0700 Make sure we are not trying to clone a spill block. Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14825 commit a22891c3272d8527d4c8cb7ff52a25ef396e7add Author: Pawel Jakub Dawidek Date: Thu May 4 16:14:19 2023 -0700 Correct comment. Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14825 commit 9b016166dd5875db87963b5deeca8eeda094b571 Author: Pawel Jakub Dawidek Date: Wed May 3 23:25:22 2023 -0700 Remove badly placed comment. Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14825 commit 6bcd48e213a279781ecd6df22799532cbec353d6 Author: Pawel Jakub Dawidek Date: Wed May 3 00:24:47 2023 -0700 Don't call zfs_exit_two() before zfs_enter_two(). Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14825 commit 0919c985e294a89169adacd5ed4a240945e5fbee Author: Pawel Jakub Dawidek Date: Tue May 2 15:46:14 2023 -0700 Don't use dmu_buf_is_dirty() for unassigned transaction. The dmu_buf_is_dirty() call doesn't make sense here for two reasons: 1. txg is 0 for unassigned tx, so it was a no-op. 2. It is equivalent of checking if we have dirty records and we are doing this few lines earlier. Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14825 commit 7f88494ac91c61aeffad810e7d167badb875166e Author: Pawel Jakub Dawidek Date: Tue May 2 14:24:43 2023 -0700 Deny block cloning is dbuf size doesn't match BP size. I don't know an easy way to shrink down dbuf size, so just deny block cloning into dbufs that don't match our BP's size. This fixes the following situation: 1. Create a small file, eg. 1kB of random bytes. Its dbuf will be 1kB. 2. Create a larger file, eg. 2kB of random bytes. Its dbuf will be 2kB. 3. Truncate the large file to 0. Its dbuf will remain 2kB. 4. Clone the small file into the large file. Small file's BP lsize is 1kB, but the large file's dbuf is 2kB. Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14825 commit 49657002f9cb57b9b4675100aaf58e1e93984bbf Author: Pawel Jakub Dawidek Date: Sun Apr 30 02:47:09 2023 -0700 Additional block cloning fixes. Reimplement some of the block cloning vs dbuf logic, mostly to fix situation where we clone a block and in the same transaction group we want to partially overwrite the clone. Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14825 commit 4d31369d3055bf0cf1d4f3e1e7d43d745f2fd05f Author: Alexander Motin Date: Thu May 11 17:27:12 2023 -0400 zil: Don't expect zio_shrink() to succeed. At least for RAIDZ zio_shrink() does not reduce zio size, but reduced wsz in that case likely results in writing uninitialized memory. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14853 commit 663dc5f616e6d0427207ffcf7a83dd02fe06a707 Author: Ameer Hamza Date: Wed May 10 05:56:35 2023 +0500 Prevent panic during concurrent snapshot rollback and zvol read Protect zvol_cdev_read with zv_suspend_lock to prevent concurrent release of the dnode, avoiding panic when a snapshot is rolled back in parallel during ongoing zvol read operation. Reviewed-by: Chunwei Chen Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Signed-off-by: Ameer Hamza Closes #14839 commit 7375f4f61ca587f893435184f398a767ae52fbea Author: Tony Hutter Date: Tue May 9 17:55:19 2023 -0700 pam: Fix "buffer overflow" in pam ZTS tests on F38 The pam ZTS tests were reporting a buffer overflow on F38, possibly due to F38 now setting _FORTIFY_SOURCE=3 by default. gdb and valgrind narrowed this down to a snprintf() buffer overflow in zfs_key_config_modify_session_counter(). I'm not clear why this particular snprintf() was being flagged as an overflow, but when I replaced it with an asprintf(), the test passed reliably. Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Closes #14802 Closes #14842 commit 9d3ed831f309e28a9cad56c8b1520292dbad0d7b Author: Brian Behlendorf Date: Tue May 9 09:03:10 2023 -0700 Add dmu_tx_hold_append() interface Provides an interface which callers can use to declare a write when the exact starting offset in not yet known. Since the full range being updated is not available only the first L0 block at the provided offset will be prefetched. Reviewed-by: Olaf Faaland Signed-off-by: Brian Behlendorf Closes #14819 commit 2b6033d71da38015c885297d1ee6577871099744 Author: Brian Behlendorf Date: Tue May 9 08:57:02 2023 -0700 Debug auto_replace_001_pos failures Reduced the timeout to 60 seconds which should be more than sufficient and allow the test to be marked as FAILED rather than KILLED. Also dump the pool status on cleanup. Reviewed-by: Brian Atkinson Signed-off-by: Brian Behlendorf Closes #14829 commit f4adc2882fb162c82e9738c5d2d30e3ba8a66367 Author: George Amanakis Date: Tue May 9 17:54:41 2023 +0200 Remove duplicate code in l2arc_evict() l2arc_evict() performs the adjustment of the size of buffers to be written on L2ARC unnecessarily. l2arc_write_size() is called right before l2arc_evict() and performs those adjustments. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: George Amanakis Closes #14828 commit 9b2c182d291bbb3ece9ceb1c72800d238d19b2e7 Author: Alexander Motin Date: Tue May 9 11:54:01 2023 -0400 Remove single parent assertion from zio_nowait(). We only need to know if ZIO has any parent there. We do not care if it has more than one, but use of zio_unique_parent() == NULL asserts that. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14823 commit 4def61804c052a1235179e3a7c98305d8075e0e9 Author: George Amanakis Date: Tue May 9 17:53:27 2023 +0200 Enable the head_errlog feature to remove errors In case check_filesystem() does not error out and does not report an error, remove that error block from error lists and logs without requiring a scrub. This can happen when the original file and all snapshots/clones referencing it have been removed. Otherwise zpool status will still report that "Permanent errors have been detected..." without actually reporting any of them. To implement this change the functions introduced in corrective receive were modified to take into account the head_errlog feature. Before this change: ============================= pool: test state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 /home/user/vdev_a ONLINE 0 0 2 errors: Permanent errors have been detected in the following files: ============================= After this change: ============================= pool: test state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 /home/user/vdev_a ONLINE 0 0 2 errors: No known data errors ============================= Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: George Amanakis Closes #14813 commit 3f2f9533ca8512ef515a73ac5661598a65b896b6 Author: George Amanakis Date: Mon May 8 22:35:03 2023 +0200 Fixes in head_errlog feature with encryption For the head_errlog feature use dsl_dataset_hold_obj_flags() instead of dsl_dataset_hold_obj() in order to enable access to the encryption keys (if loaded). This enables reporting of errors in encrypted filesystems which are not mounted but have their keys loaded. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14837 commit 288ea63effae3ba24fcb6dc412a3125b9f3e1da9 Author: Matthew Ahrens Date: Mon May 8 11:20:23 2023 -0700 Verify block pointers before writing them out If a block pointer is corrupted (but the block containing it checksums correctly, e.g. due to a bug that overwrites random memory), we can often detect it before the block is read, with the `zfs_blkptr_verify()` function, which is used in `arc_read()`, `zio_free()`, etc. However, such corruption is not typically recoverable. To recover from it we would need to detect the memory error before the block pointer is written to disk. This PR verifies BP's that are contained in indirect blocks and dnodes before they are written to disk, in `dbuf_write_ready()`. This way, we'll get a panic before the on-disk data is corrupted. This will help us to diagnose what's causing the corruption, as well as being much easier to recover from. To minimize performance impact, only checks that can be done without holding the spa_config_lock are performed. Additionally, when corruption is detected, the raw words of the block pointer are logged. (Note that `dprintf_bp()` is a no-op by default, but if enabled it is not safe to use with invalid block pointers.) Reviewed-by: Rich Ercolani Reviewed-by: Brian Behlendorf Reviewed-by: Paul Zuchowski Reviewed-by: Alexander Motin Signed-off-by: Matthew Ahrens Closes #14817 commit 23132688b9d54ef11413925f88c02d83d607ec2b Author: Brian Behlendorf Date: Mon May 8 11:17:41 2023 -0700 zdb: consistent xattr output When using zdb to output the value of an xattr only interpret it as printable characters if the entire byte array is printable. Additionally, if the --parseable option is set always output the buffer contents as octal for easy parsing. Reviewed-by: Olaf Faaland Signed-off-by: Brian Behlendorf Closes #14830 commit 6deb342248e10af92e2d3fbb4e4b1221812188ff Author: Brian Behlendorf Date: Mon May 8 10:09:30 2023 -0700 ZTS: add snapshot/snapshot_002_pos exception Add snapshot_002_pos to the known list of occasional failures for FreeBSD until it can be made entirely reliable. Reviewed-by: Tino Reichardt Signed-off-by: Brian Behlendorf Issue #14831 Closes #14832 commit a0a125bab291fe005d29be5375a5bb2a1c8261c7 Author: Alexander Motin Date: Fri May 5 12:17:55 2023 -0400 Fix two abd_gang_add_gang() issues. - There is no reason to assert that added gang is not empty. It may be weird to add an empty gang, but it is legal. - When moving chain list from the added gang clear its size, or it will trigger assertion in abd_verify() when that gang is freed. Reviewed-by: Brian Atkinson Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14816 commit aefb80389458dcccdcb9659914714264248b8e52 Author: Pawel Jakub Dawidek Date: Sat May 6 01:09:12 2023 +0900 Simplify and optimize random_int_between(). Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14805 commit cf53b4376d902baecc04e450038d49c84c848e56 Author: Pawel Jakub Dawidek Date: Sat May 6 00:51:41 2023 +0900 Plug memory leak in zfsdev_state. On kernel module unload, free all zfsdev state structures, except for zfsdev_state_listhead, which is statically allocated. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Closes #14824 commit 409f6b6fa0caba14be1995bbe28ca70e55ab7666 Author: Ameer Hamza Date: Thu May 4 03:10:32 2023 +0500 zpool import -m also removing spare and cache when log device is missing spa_import() relies on a pool config fetched by spa_try_import() for spare/cache devices. Import flags are not passed to spa_tryimport(), which makes it return early due to a missing log device and missing retrieving the cache device and spare eventually. Passing ZFS_IMPORT_MISSING_LOG to spa_tryimport() makes it fetch the correct configuration regardless of the missing log device. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Ameer Hamza Closes #14794 commit 2b517ae76321b7295d4508ca222a7a6aa7588497 Author: buzzingwires <131118055+buzzingwires@users.noreply.github.com> Date: Wed May 3 12:03:57 2023 -0400 Allow zhack label repair to restore detached devices. This commit expands on the zhack label repair command in d04b5c9 by adding the -u option to undetach a device by regenerating uberblocks, in addition to the existing functionality of fixing checksums, now represented by -c. Previous behavior is retained in the case of no options. The changes are heavily inspired by Jeff Bonwick's labelfix utility, as archived at: https://gist.github.com/jjwhitney/baaa63144da89726e482 Additionally, it is now capable of properly determining the size of block devices and other media, as well as handling sizes which are not divisible by 2^18. This should make it viable for use on physical devices and partitions, in addition to files. These changes should make it possible to import zpools that have had their uberblocks erased, such as in the case of pools rendered inaccessible by erroneous detach commands. Reviewed-by: Brian Behlendorf Signed-off-by: buzzingwires Closes #14773 commit abf7fed50defb5953dc4f3abf366f05737870c81 Author: George Amanakis Date: Wed May 3 18:00:14 2023 +0200 Optimize check_filesystem() and process_error_log() Integrate check_clones() into check_filesystem() and implement a list instead of iterating recursively over the clones, thus eliminating the risk of a stack overflow. Also use kmem_zalloc() to allocate large structures in process_error_log() reducing its stack size from ~700 to ~128 bytes. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14744 commit 3fafb734603fe60eee05b46701516b345a4c86f0 Author: Pawel Jakub Dawidek Date: Wed May 3 01:24:26 2023 +0900 Use correct block pointer in block cloning case. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Pawel Jakub Dawidek Closes #14806 commit bdfce893777435bacce193232fe6a78b78365143 Author: Brian Behlendorf Date: Tue May 2 09:21:47 2023 -0700 Wrap clang specific pragma Clang specific pragmas need to be wrapped to prevent a build warning when compiling with gcc. Reviewed-by: Tino Reichardt Signed-off-by: Brian Behlendorf Closes #14814 commit efe80865b3b10d4c5a89ea26dcf2ae67cad5146d Author: Mateusz Guzik Date: Tue May 2 02:21:27 2023 +0200 blake3: fix up bogus checksums in face of cpu migration This is a temporary measure until a better fix is sorted out. Reviewed-by: Richard Yao Reviewed-by: Alexander Motin Reviewed-by: Rich Ercolani Signed-off-by: Mateusz Guzik Sponsored by: Rubicon Communications, LLC ("Netgate") Closes #14785 Closes #14808 commit d85ede28317354f7bd6275940e6897c9ed49e33e Author: Serapheim Dimitropoulos Date: Mon May 1 17:18:42 2023 -0700 Correct ABD size for split block ZIOs Currently when layering the ABD buffer of each split block on top of an indirect vdev's ZIO ABD we don't specify the split block's ABD. This results in those ABDs being incorrectly sized by inheriting the size of their parent ABD which is larger than what each split block needs. The above behavior isn't causing any bugs currently but can lead to unexpected ABD sizes for people analyzing and/or working on the ZIO codepath. This patch fixes this behavior by properly setting the ABD size for split block ZIOs. Reviewed-by: Matthew Ahrens Reviewed-by: Igor Kozhukhov Reviewed-by: Brian Behlendorf Reviewed-by: Mark Maybee Reviewed-by: Brian Atkinson Signed-off-by: Serapheim Dimitropoulos Closes #14804 commit 91eab2f77462bbb4967cfc2d8f8dede1a7d21538 Author: Justin Hibbits Date: Thu Apr 27 15:49:21 2023 -0400 powerpc64: Support ELFv2 asm on Big Endian FreeBSD/powerpc64 is all ELFv2 since FreeBSD 13, even big endian. The existing sha256 and sha512 asm code assumes that BE is all ELFv1, and LE is ELFv2. Minor changes to add ELFv2 in the BE side gets this working correctly on FreeBSD with latest OpenZFS import. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Justin Hibbits Closes #14779 commit 7d15ebcaa5f3a283a78ec7ecc3ca91333d113601 Author: Alexander Motin Date: Thu Apr 27 15:32:58 2023 -0400 Mark TX_COMMIT transaction with TXG_NOTHROTTLE. TX_COMMIT has no on-disk representation and does not produce any more dirty data. It should not wait for anything, and even just skipping the checks if not waiting gives improvement noticeable in profiler. Reviewed-by: Brian Behlendorf Reviewed-by: Prakash Surya Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14798 commit e907d6938cdd372fbcda48e46ecd4673bbac5cf1 Author: Val Packett Date: Thu Apr 27 13:49:03 2023 -0300 PAM: support the authentication facility Implement the pam_sm_authenticate method, using the noop argument of lzc_load_key to do a passphrase check without actually loading the key. This allows using ZFS as the source of truth for user passwords, without storing any password hashes in /etc or using other PAM modules. Reviewed-by: Brian Behlendorf Reviewed-by: Felix Dörre Signed-off-by: Val Packett Closes #14789 commit 87cab6b92ccd53d4721c8c50c058a1a8473b3816 Author: Tino Reichardt Date: Wed Apr 26 21:40:26 2023 +0200 Fix BLAKE3 aarch64 assembly for FreeBSD and macOS The x18 register isn't useable within FreeBSD kernel space, so we have to fix the BLAKE3 aarch64 assembly for not using it. The source files are here: https://github.com/mcmilk/BLAKE3-tests Reviewed-by: Kyle Evans Signed-off-by: Tino Reichardt Closes #14728 commit 129da105b125068d8004c58a65248a9cd236ff4c Author: Brian Behlendorf Date: Wed Apr 26 11:49:16 2023 -0700 Fix checkstyle warning Resolve a missed checkstyle warning. Reviewed-by: Alexander Motin Reviewed-by: Mateusz Guzik Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #14799 commit 1f898b8c0867b0bb2b8f9fe71eeaaae1fd354d39 Author: Alexander Motin Date: Wed Apr 26 12:20:43 2023 -0400 Fix positive ABD size assertion in abd_verify(). Gang ABDs without childred are legal, and they do have zero size. For other ABD types zero size doesn't have much sense and likely not working correctly now. Reviewed-by: Igor Kozhukhov Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14795 commit 2a97901e5cc014fdd5950377787ad5c788605758 Author: Mateusz Guzik Date: Thu Apr 20 09:00:03 2023 +0000 FreeBSD: fix up EINVAL from getdirentries on .zfs Without the change: /.zfs /.zfs/snapshot find: /.zfs: Invalid argument Signed-off-by: Mateusz Guzik Closes #14774 commit 5ca32dd496d0da9d5c49271061bdc6373d5d4440 Author: Mateusz Guzik Date: Thu Apr 20 08:59:38 2023 +0000 FreeBSD: add missing vn state transition for .zfs Signed-off-by: Mateusz Guzik Closes #14774 commit 6569bb6fc05113c7995eb02d194c884421ff7f8b Author: Rob N Date: Thu Apr 27 01:50:44 2023 +1000 tests/zdb_encrypted: parse numbers a little more robustly On FreeBSD, `wc` prints some leading spaces, while on Linux it does not. So we tell ksh to expect an integer, and it does the rest. Reviewed-by: Brian Behlendorf Signed-off-by: Rob Norris Closes #14791 Closes #14797 commit 6fc16b48c4e7381d9b9c203ed611de136611ff34 Author: Brian Behlendorf Date: Wed Apr 26 08:43:39 2023 -0700 zdb: Fix minor memory leak Commit 6b6aaf6dc2e65c63c74fbd7840c14627e9a91ce2 introduced a small memory leak in zdb. This was detected by the LeakSanitizer and was causing all ztest runs to fail. Reviewed-by: Igor Kozhukhov Reviewed-by: Rich Ercolani Signed-off-by: Brian Behlendorf Closes #14796 commit 702c144fa41774e567cb2b516e6d9a2e4218c64f Author: Brian Behlendorf Date: Tue Apr 25 16:40:55 2023 -0700 Revert "Fix data race between zil_commit() and zil_suspend()" This reverts commit 4c856fb333ac57d9b4a6ddd44407fd022a702f00 to resolve a newly introduced deadlock which in practice in more disruptive that the issue this commit intended to address. Reviewed-by: Richard Yao Reviewed-by: Mark Maybee Signed-off-by: Brian Behlendorf Issue #14775 Closes #14790 commit bd9e3d065f794d5ed0f258a5201972d12544e29d Author: Han Gao Date: Wed Apr 26 07:05:45 2023 +0800 Add loongarch64 support Add loongarch64 definitions & lua module setjmp asm LoongArch is a new RISC ISA, which is a bit like MIPS or RISC-V. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Han Gao Signed-off-by: WANG Xuerui Closes #13422 commit 303578a010895a3ec3c83a75596e8628dc58eb79 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Mon Apr 24 19:55:07 2023 -0400 Taught zdb -bb to print metadata totals People often want estimates of how much of their pool is occupied by metadata, but they end up using lots of text processing on zdb's output to get it. So let's just...provide it for them. Now, zdb -bbbs will output something like: Blocks LSIZE PSIZE ASIZE avg comp %Total Type [...] 68 1.06M 272K 544K 8K 4.00 0.00 L6 Total 1.71K 212M 6.85M 13.7M 8K 30.91 0.00 L5 Total 1.71K 212M 6.85M 13.7M 8K 30.91 0.00 L4 Total 1.73K 214M 6.92M 13.8M 8K 30.89 0.00 L3 Total 18.7K 2.29G 111M 221M 11.8K 21.19 0.00 L2 Total 3.56M 454G 28.4G 56.9G 16.0K 15.97 0.19 L1 Total 308M 36.8T 28.2T 28.6T 95.1K 1.30 99.80 L0 Total 311M 37.3T 28.3T 28.6T 94.2K 1.32 100.00 Total 50.4M 774G 113G 291G 5.77K 6.85 0.99 Metadata Total Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #14746 commit 8c7736bb3f51e502b9d1858e8111caf9db505a28 Author: Mateusz Guzik Date: Tue Apr 25 01:15:42 2023 +0200 FreeBSD: add missing vop_fplookup assignments It became illegal to not have them as of 5f6df177758b9dff88e4b6069aeb2359e8b0c493 ("vfs: validate that vop vectors provide all or none fplookup vops") upstream. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Mateusz Guzik Closes #14788 commit 318228d9d68cd48e7904fb43ed9a5fa41e8ffff9 Author: Mateusz Guzik Date: Wed Apr 5 21:28:52 2023 +0000 FreeBSD: try to fallback early if can't do optimized copy Not complete, but already shaves on some locking. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Mateusz Guzik Sponsored by: Rubicon Communications, LLC ("Netgate") Closes #14723 commit e48a9e8f77c32e7ec7df16140d6a960eea33840a Author: Mateusz Guzik Date: Wed Apr 5 21:12:17 2023 +0000 FreeBSD: fix up EXDEV handling for clone_range API contract requires VOPs to handle EXDEV internally, worst case by falling back to the generic copy routine. This broke with the recent changes. While here whack custom loop to lock 2 vnodes with vn_lock_pair, which provides the same functionality internally. write start/finish around it plays no role so got eliminated. One difference is that vn_lock_pair always takes an exclusive lock on both vnodes. I did not patch around it because current code takes an exclusive lock on the target vnode. zfs supports shared-locking for writes, so this serializes different calls to the routine as is, despite range locking inside. At the same time you may notice the source vnode can get some traffic if only shared-locked, thus once more this goes the safer route of exclusive-locking. Note this should be patched to use shared-locking for both once the feature is considered stable. Technically the switch to vn_lock_pair should be a separate change, but it would only introduce churn immediately whacked by the rest of the patch. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Mateusz Guzik Sponsored by: Rubicon Communications, LLC ("Netgate") Closes #14723 commit dde33905824824a8b60f9afadca0de1e23050ef7 Author: Dimitry Andric Date: Fri Apr 21 19:22:52 2023 +0200 FreeBSD: make zfs_vfs_held() definition consistent with declaration Noticed while attempting to change FreeBSD's boolean_t into an actual bool: in include/sys/zfs_ioctl_impl.h, zfs_vfs_held() is declared to return a boolean_t, but in module/os/freebsd/zfs/zfs_ioctl_os.c it is defined to return an int. Make the definition match the declaration. Reviewed-by: Alexander Motin Reviewed-by: Brian Atkinson Signed-off-by: Dimitry Andric Closes #14776 commit 6added3d174e91088fadabc3b601123b3df449bc Author: Allan Jude Date: Fri Apr 21 13:20:36 2023 -0400 Add support for zpool user properties Usage: zpool set org.freebsd:comment="this is my pool" poolname Tests are based on zfs_set's user property tests. Also stop truncating property values at MAXNAMELEN, use ZFS_MAXPROPLEN. Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude Signed-off-by: Mateusz Piotrowski Sponsored-by: Beckhoff Automation GmbH & Co. KG. Sponsored-by: Klara Inc. Closes #11680 commit 9c6ebe42d54a1c5d108b73f634171fe78d8a4e1c Author: Richard Yao Date: Tue Apr 11 17:56:16 2023 +0000 Linux: Suppress -Wordered-compare-function-pointers in tracepoint code Clang points out that there is a comparison against -1, but we cannot fix it because that is from the kernel headers, which we must support. We can workaround this by using a pragma. Sponsored-By: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf Reviewed-by: Youzhong Yang Signed-off-by: Richard Yao Closes #14738 commit fd758cbb285f962f67d7907695a61458dbeed636 Author: Richard Yao Date: Tue Apr 11 17:50:43 2023 +0000 Linux: zfs_zaccess_trivial() should always call generic_permission() Building with Clang on Linux generates a warning that err could be uninitialized if mnt_ns is a NULL pointer. However, mnt_ns should never be NULL, so there is no need to put this behind an if statement. Taking it outside of the if statement means that the possibility of err being uninitialized goes from being always zero in a way that the compiler could not realize to a way that is always zero in a way that the compiler can realize. Sponsored-By: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf Reviewed-by: Youzhong Yang Signed-off-by: Richard Yao Closes #14738 commit ff4ac2820d4930aada280e79904df54ceef28420 Author: Brian Behlendorf Date: Thu Apr 20 10:25:16 2023 -0700 ZTS: zvol_misc_trim retry busy export Retry the export if the pool is busy due to an open zvol. Observed in the CI on Fedora 37. cannot export 'testpool': pool is busy ERROR: zpool export testpool exited 1 Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #14769 commit 335d47f2c14efe776720f7558c3f9ab9f5fc8c6b Author: rob-wing <98866084+rob-wing@users.noreply.github.com> Date: Thu Apr 20 09:07:56 2023 -0800 Create zap for root vdev And add it to the AVZ, this is not backwards compatible with older pools due to an assertion in spa_sync() that verifies the number of ZAPs of all vdevs matches the number of ZAPs in the AVZ. Granted, the assertion only applies to #DEBUG builds - still, a feature flag is introduced to avoid the assertion, com.klarasystems:vdev_zaps_v2 Notably, this allows to get/set properties on the root vdev: % zpool set user:prop=value root-0 Before this commit, it was already possible to get/set properties on top-level vdevs with the syntax - (e.g. mirror-0): % zpool set user:prop=value mirror-0 This syntax also applies to the root vdev as it is is of type 'root' with a vdev_id of 0, root-0. The keyword 'root' as an alias for 'root-0'. The following tests have been added: - zpool get all properties from root vdev - zpool set a property on root vdev - verify root vdev ZAP is created Reviewed-by: Brian Behlendorf Signed-off-by: Rob Wing Sponsored-by: Seagate Technology Submitted-by: Klara, Inc. Closes #14405 commit b45b267c120766a5fbdde275d8952d4ebe4f2abb Author: Herb Wartens Date: Wed Apr 19 13:22:59 2023 -0700 Allow MMP to bypass waiting for other threads At our site we have seen cases when multi-modifier protection is enabled (multihost=on) on our pool and the pool gets suspended due to a single disk that is failing and responding very slowly. Our pools have 90 disks in them and we expect disks to fail. The current version of MMP requires that we wait for other writers before moving on. When a disk is responding very slowly, we observed that waiting here was bad enough to cause the pool to suspend. This change allows the MMP thread to bypass waiting for other threads and reduces the chances the pool gets suspended. Reviewed-by: Brian Behlendorf Signed-off-by: Herb Wartens Closes #14659 commit d9a9e6cf05e134468487849f540e222af5c6271d Author: Paul Dagnelie Date: Wed Apr 19 13:20:02 2023 -0700 ZTS: send-c_volume is flaky We use block_device_wait to wait for the zvol block device to actually appear, and we log the result of the dd calls by using an intermediate file. Reviewed-by: George Melikov Reviewed-by: John Wren Kennedy Reviewed-by: Brian Behlendorf Signed-off-by: Paul Dagnelie Closes #14767 commit 2c98d658cc652cee61fb511033ac689a2eeb9339 Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Wed Apr 19 21:04:32 2023 +0500 Fix "Detach spare vdev in case if resilvering does not happen" Spare vdev should detach from the pool when a disk is reinserted. However, spare detachment depends on the completion of resilvering, and if resilver does not schedule, the spare vdev keeps attached to the pool until the next resilvering. When a zfs pool contains several disks (25+ mirror), resilvering does not always happen when a disk is reinserted. In this patch, spare vdev is manually detached from the pool when resilvering does not occur and it has been tested on both Linux and FreeBSD. Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Signed-off-by: Ameer Hamza Closes #14722 commit f079812f822fa5d856e9a7e2a257c80227955dd0 Author: наб Date: Wed Apr 19 18:03:42 2023 +0200 zfsprops.7: update mandlock https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=f7e33bdbd6d1bdf9c3df8bba5abcf3399f957ac3 https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/commit/?id=7e59106e9c34458540f7d382d5b49071d1b7104f Fixes: commit fb9baa9b2045a193a3caf0a46b5cac5ef7a84b61 ("zfsprops.8: remove nbmand-not-used-on-Linux and pointer to mount(8)") Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #14765 commit 2abb239e8ae3fe152c88112076309b921575306c Author: youzhongyang Date: Tue Apr 18 21:10:40 2023 -0400 Silence clang warning of flexible array not at end Reviewed-by: Brian Behlendorf Reviewed-by: Jorgen Lundman Signed-off-by: Youzhong Yang Closes #14764 commit 52a70a9f02821958548e116eee055a29c2b4efb7 Author: Low-power Date: Wed Apr 19 02:34:41 2023 +0800 Values printed by zpool-iostat(8) should be right-aligned This inappropriate left-alignment was introduced in 7bb7b1f. Reviewed-by: Tony Hutter Reviewed-by: Matthew Ahrens Reviewed-by: Tino Reichardt Signed-off-by: WHR Closes #14751 commit 2a9a97c47416ba7ac17b6f9fb62a582476bacafe Author: Tony Hutter Date: Tue Apr 18 08:41:52 2023 -0700 Revert "ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()" This reverts commit 4b3133e671b958fa2c915a4faf57812820124a7b. Users identified this commit as a possible source of data corruption: https://github.com/openzfs/zfs/issues/14753 Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Issue #14753 Closes #14761 commit 3e3e19927787ada7d3e0ed42c35a2c190ea7350b Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Mon Apr 17 20:38:09 2023 -0400 Work around Raspberry Pi kernel packaging oddities On Debian and Ubuntu and friends, you get something like "linux-image-$(uname -r)" and "linux-headers-$(uname -r)" you can put a Depends on. On Raspberry Pi OS, you get "raspberrypi-kernel" and "raspberrypi-kernel-headers", with version numbers like 20230411. There is not, as far as I can tell, a reasonable way to map that to a kernel version short of reaching out and digging around in the changelogs or Makefile, so just special-case it so the packages don't fail to install at install time. They still might not build if the versions don't match, but I don't see a way to do anything about that... Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #14745 Closes #14747 commit c81931300b8f5efb1d41a505eaff1f610c97da3d Author: Pawel Jakub Dawidek Date: Tue Apr 18 08:42:09 2023 +0900 Fix VERIFY(!zil_replaying(zilog, tx)) panic The zfs_log_clone_range() function is never called from the zfs_clone_range_replay() function, so I assumed it is safe to assert that zil_replaying() is never TRUE here. It turns out zil_replaying() also returns TRUE when the sync property is set to disabled. Fix the problem by just returning if zil_replaying() returns TRUE. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reported by: Florian Smeets Signed-off-by: Pawel Jakub Dawidek Closes #14758 commit 5d4543bcd4a0f7edf76122e5a4018ecdcc83cb09 Author: dodexahedron Date: Thu Apr 13 09:15:34 2023 -0700 Minor improvements to zpoolconcepts.7 * Fixed one typo (effects -> affects) * Re-worded raidz description to make it clearer that it is not quite the same as RAID5, though similar * Clarified that data is not necessarily written in a static stripe width * Minor grammar consistency improvement * Noted that "volumes" means zvols * Fixed a couple of split infinitives * Clarified that hot spares come from the same pool they were assigned to * "we" -> ZFS * Fixed warnings thrown by mandoc, and removed unnecessary wordiness in one fixed line. Reviewed-by: Brian Behlendorf Signed-off-by: Brandon Thetford Closes #14726 commit 3d2d0da5b11995d4d43a0ca0bcbe041e5de3b0df Author: youzhongyang Date: Thu Apr 13 12:12:03 2023 -0400 Linux 6.3 compat: Fix memcpy "detected field-spanning write" error Add a new union member of flexible array to dnode_phys_t and use it in the macro so we can silence the memcpy() fortify error. Reviewed-by: Brian Behlendorf Signed-off-by: Youzhong Yang Closes #14737 commit 56405ae88cc6dc3166145e97ecabb912ea17821b Author: Pawel Jakub Dawidek Date: Thu Apr 13 08:15:05 2023 +0900 Fix data corruption when cloning embedded blocks Don't overwrite blk_phys_birth, as for embedded blocks it is part of the payload. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Pawel Jakub Dawidek Issue #13392 Closes #14739 commit 8594995d99ad2a6302b8b61450ab9a9b1a78f9ca Author: наб Date: Wed Apr 12 19:08:49 2023 +0200 initramfs: source user scripts from /e/z/initramfs-tools-load-key{,.d/*} By dropping in a file in a directory (for packages) or by making a file (for local administrators), custom key loading methods may be provided for the rootfs and necessities. Reviewed-by: Brian Behlendorf Signed-off-by: Nicholas Morris Signed-off-by: Ahelenia Ziemiańska Co-authored-by: Nicholas Morris Supersedes: #14704 Closes: #13757 Closes #14733 commit e7dcfe2d0a926f585430631cdbf09a215e0a1da4 Author: George Amanakis Date: Wed Apr 12 17:53:53 2023 +0200 Fix in check_filesystem() Fix the code in case of missing snapshots. Previously the check was in a conditional that would be executed if the filesystem had snapshots. Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Signed-off-by: George Amanakis Closes #14735 commit 3797f1dd78c284b4b4b5ff98b8a753bbc6e888e8 Author: Jorgen Lundman Date: Fri Apr 21 16:23:10 2023 +0900 Clean up warnings - part 1 Signed-off-by: Jorgen Lundman commit 356c478ce4fe0504abc1fcfee8090bd1c8480eca Author: Jorgen Lundman Date: Thu Apr 20 15:29:28 2023 +0900 Plumb file block-clone feature Tested with reflink.exe which allowed files to be cloned on block level. Including crossing datasets if within the same pool. Clean up the two set disposition Signed-off-by: Jorgen Lundman commit 1a8b8c5466c51ae779c1c0443a39b8863bbf924f Author: Jorgen Lundman Date: Wed Apr 19 10:05:37 2023 +0900 Add WRITEONLY to tunables for zfs version Even if "zfs version" exists in the registry, we should always write it out, to update new kernel driver version Signed-off-by: Jorgen Lundman commit e7d14944aabc9364e7fd451415742b9c869ae80b Author: Jorgen Lundman Date: Sat Apr 15 18:34:04 2023 +0900 Correct directory query search pattern Earlier dirlist cleanup broke the use of search patterns. Additionally, query device name should return failure on non-root files (if not used on volumes) Signed-off-by: Jorgen Lundman commit 0873fb51a23e6a77658137846a94098fcc7a3902 Author: Alan Somers Date: Mon Apr 10 15:24:27 2023 -0600 Trim needless zeroes from checksum events The ereport.fs.zfs.checksum event contains histograms of the bits that were wrongly set or cleared according to their bit position in a 64-bit word. So the maximum value that any histogram bucket could have would be 64. But ZFS currently uses a uint32_t to hold each bucket. As a result, the event report is full of needless zeroes. Change the bucket size to uint8_t, stripping 768 needless zeros from each event. Original event format: ``` class=ereport.fs.zfs.checksum ena=639460469834258433 pool=testpool.1933 pool_guid=4979719877084416563 pool_state=0 pool_context=0 pool_failmode=wait vdev_guid=4136721804819128578 vdev_type=file vdev_path=/tmp/kyua.1TxP3A/2/work/file1.1933 vdev_ashift=9 vdev_complete_ts=609837019678 vdev_delta_ts=33450 vdev_read_errors=0 vdev_write_errors=0 vdev_cksum_errors=20 vdev_delays=0 parent_guid=2751977006639883417 parent_type=raidz vdev_spare_guids= zio_err=0 zio_flags=1048752 zio_stage=4194304 zio_pipeline=65011712 zio_delay=0 zio_timestamp=0 zio_delta=0 zio_priority=4 zio_offset=702976 zio_size=1024 zio_objset=24 zio_object=0 zio_level=3 zio_blkid=0 bad_ranges=0000000000000400 bad_ranges_min_gap=8 bad_range_sets=0000079e bad_range_clears=00000854 bad_set_histogram=000000210000001a000000150000001d000000240000001b000000220000001b000000210000002100000018000000260000002300000025000000210000001e000000250000001b0000001d0000001e0000001600000025000000180000001b000000240000001b000000240000001b0000001c000000210000001b0000001e000000210000001a0000001e000000220000001d0000001b000000200000001f0000001a000000250000001f0000001d0000001b0000001d000000240000001d0000001b0000001b0000001f00000024000000190000001a0000001f0000001e000000240000001e0000002400000021000000200000001d0000001d00000021 bad_cleared_histogram=000000220000002700000021000000210000001b0000001a000000250000001f0000001c0000001e0000002400000022000000220000002400000022000000240000002200000021000000220000001b0000002100000021000000190000001b000000240000002400000020000000290000002a00000028000000250000002400000020000000270000002500000016000000270000001c000000210000001f000000240000001c0000002100000022000000240000002100000023000000210000002700000022000000240000001b00000022000000210000001c00000023000000150000002600000020000000270000001e0000001d0000002400000026 time=00000016806457270000000323406839 eid=458 ``` New format: ``` class=ereport.fs.zfs.checksum ena=96599319807790081 pool=testpool.1933 pool_guid=1236902063710799041 pool_state=0 pool_context=0 pool_failmode=wait vdev_guid=2774253874431514999 vdev_type=file vdev_path=/tmp/kyua.6Temlq/2/work/file1.1933 vdev_ashift=9 vdev_complete_ts=92124283803 vdev_delta_ts=46670 vdev_read_errors=0 vdev_write_errors=0 vdev_cksum_errors=20 vdev_delays=0 parent_guid=8090931855087882905 parent_type=raidz vdev_spare_guids= zio_err=0 zio_flags=1048752 zio_stage=4194304 zio_pipeline=65011712 zio_delay=0 zio_timestamp=0 zio_delta=0 zio_priority=4 zio_offset=1028608 zio_size=512 zio_objset=0 zio_object=0 zio_level=0 zio_blkid=4 bad_ranges=0000000000000200 bad_ranges_min_gap=8 bad_range_sets=0000061f bad_range_clears=000001f4 bad_set_histogram=1719161c1c1c101618171a151a1a19161e1c171d1816161c191f1a18192117191c131d171b1613151a171419161a1b1319101b14171b18151e191a1b141a1c17 bad_cleared_histogram=06090a0808070a0b020609060506090a01090a050a0a0509070609080d050d0607080d060507080c04070807070a0608020c080c080908040808090a05090a07 time=00000016806477050000000604157480 eid=62 ``` Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Signed-off-by: Alan Somers Sponsored-by: Axcient Closes #14716 commit 5183c82c32d2682bcb7adf11ca0d582088347e7a Author: Andrew Innes Date: Wed Apr 12 15:32:39 2023 +0800 Build fix after rebase. zuserns_t -> zidmap_t Signed-off-by: Andrew Innes commit ee354731b63838bd98c65be0eba2891630f1cf5d Author: youzhongyang Date: Mon Apr 10 17:15:36 2023 -0400 Linux 6.3 compat: idmapped mount API changes Linux kernel 6.3 changed a bunch of APIs to use the dedicated idmap type for mounts (struct mnt_idmap), we need to detect these changes and make zfs work with the new APIs. Reviewed-by: Brian Behlendorf Signed-off-by: Youzhong Yang Closes #14682 commit c669f4bc499243a75bc4fcdec15c32a815c9eb06 Author: Kyle Evans Date: Fri Apr 7 00:40:34 2023 +0000 module: freebsd: fix aarch64 fpu handling Just like x86, aarch64 needs to use the fpu_kern(9) API around FPU usage, otherwise we panic promptly at boot as soon as ZFS attempts to do checksum benchmarking. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Signed-off-by: Kyle Evans Closes #14715 commit 03954f5348927040227799363919b3633b3f67e8 Author: Kyle Evans Date: Fri Apr 7 00:40:34 2023 +0000 module: resync part of Makefile.bsd sha256-armv8.S and sha512-armv8.S need the same treatment as the sse bits; removal of -mgeneral-regs-only from flags. This fixes errors about requiring NEON, which is a difference in clang vs. gcc treatment of -mgeneral-regs-only being specified on asm files. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Signed-off-by: Kyle Evans Closes #14715 commit adf519a25a63e3922d45f6dcba8a7c623cb65936 Author: Rob N Date: Tue Apr 11 04:53:02 2023 +1000 libzfs: add v2 iterator interfaces f6a0dac84 modified the zfs_iter_* functions to take a new "flags" parameter, and introduced a variety of flags to ask the kernel to limit the results in various ways, reducing the amount of work the caller needed to do to filter out things they didn't need. Unfortunately this change broke the ABI for existing clients (read: older versions of the `zfs` program), and was reverted 399b98198. dc95911d2 reintroduced the original patch, with the understanding that a backwards-compatible fix would be made before the 2.2 release branch was tagged. This commit is that fix. This introduces zfs_iter_*_v2 functions that have the new flags argument, and reverts the existing functions to not have the flags parameter, as they were before. The old functions are now reimplemented in terms of the new, with flags set to 0. Reviewed-by: Brian Behlendorf Reviewed-by: George Wilson Original-patch-by: George Wilson Signed-off-by: Rob Norris Sponsored-by: Klara, Inc. Closes #14597 commit b93f72334073e0bc95faf466c57dc745dee6f903 Author: Rob N Date: Fri Apr 7 03:52:50 2023 +1000 vdev: expose zfs_vdev_max_ms_shift as a module parameter Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Reviewed-by: Alexander Motin Signed-off-by: Rob Norris Sponsored-by: Klara, Inc. Sponsored-by: Seagate Technology LLC Closes #14719 commit 4a6ac1d855e86926775bed69d1cffe297d68f0eb Author: George Amanakis Date: Thu Apr 6 19:46:18 2023 +0200 Fix typo in check_clones() Run kmem_free() after zap_cursor_fini(). Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Reviewed-by: Adam Moss Signed-off-by: George Amanakis Closes #14702 commit 98878efb42d0e46861af9baee632335f9112faca Author: Damian Szuberski Date: Fri Apr 7 03:43:24 2023 +1000 ZTS: add existing tests to runfiles Some test cases were committed to the repository but never added to runfiles. Move `zfs_unshare_008_pos` to the Linux-only runfile. Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #14701 commit 51e4051c7bf4f8feda7cf66182bf4246aba913a2 Author: Andrew Innes Date: Fri Apr 7 01:40:23 2023 +0800 ZTS: Use inbuilt monotonic time Make the test runner try to use the included python monotonic time function instead of calling librt. This makes the test runner work on macos where librt wasn't available. Reviewed-by: Tino Reichardt Signed-off-by: Andrew Innes Closes #14700 commit 57cd6982cd1f26a9abe0f7ef98a9dcb2b75c401e Author: Martin Matuška Date: Thu Apr 6 19:35:02 2023 +0200 Miscellaneous FreBSD compilation bugfixes Add missing machine/md_var.h to spl/sys/simd_aarch64.h and spl/sys/simd_arm.h In spl/sys/simd_x86.h, PCB_FPUNOSAVE exists only on amd64, use PCB_NPXNOSAVE on i386 In FreeBSD sys/elf_common.h redefines AT_UID and AT_GID on FreeBSD, we need a hack in vnode.h similar to Linux. sys/simd.h needs to be included early. In zfs_freebsd_copy_file_range() we pass a (size_t *)lenp to zfs_clone_range() that expects a (uint64_t *) Allow compiling armv6 world by limiting ARM macros in sha256_impl.c and sha512_impl.c to __ARM_ARCH > 6 Reviewed-by: Alexander Motin Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Reviewed-by: Pawel Jakub Dawidek Reviewed-by: Signed-off-by: WHR Signed-off-by: Martin Matuska Closes #14674 commit 74abbc06617514368bfcf25edb9e1c0c02c2d9e6 Author: Rob N Date: Fri Apr 7 03:31:19 2023 +1000 vdev: expose zfs_vdev_def_queue_depth as a module parameter It was previously available only to FreeBSD. Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Reviewed-by: Alexander Motin Signed-off-by: Rob Norris Sponsored-by: Klara, Inc. Sponsored-by: Seagate Technology LLC Closes #14718 commit 9a04c9990e0f165f8fdda45549a7579ab09a1927 Author: Paul Dagnelie Date: Thu Apr 6 10:29:27 2023 -0700 Storage device expansion "silently" fails on degraded vdev When a vdev is degraded or faulted, we refuse to expand it when doing online -e. However, we also don't actually cause the online command to fail, even though the disk didn't expand. This is confusing and misleading, and can result in violated expectations. Reviewed-by: Brian Behlendorf Reviewed-by: Matthew Ahrens Reviewed-by: Mark Maybee Signed-off-by: Paul Dagnelie Closes 14145 commit 2f768d52958bec8b2f505abb444823f90e3de0f4 Author: Alexander Motin Date: Thu Apr 6 02:42:22 2023 +0900 Fix some signedness issues in arc_evict() It may happen that "wanted total ARC size" (wt) is negative, that was expected. But multiplication product of it and unsigned fractions result in unsigned value, incorrectly shifted right with a sing loss. Reviewed-by: Brian Behlendorf Reviewed-by: Prakash Surya Reviewed-by: Paul Dagnelie Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14692 commit 2a5ffd5ff6b1c4a1bb1c40b37a2078507bd79db4 Author: youzhongyang Date: Wed Apr 5 13:01:38 2023 -0400 Linux 6.3 compat: writepage_t first arg struct folio* The type def of writepage_t in kernel 6.3 is changed to take struct folio* as the first argument. We need to detect this change and pass correct function to write_cache_pages(). Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Youzhong Yang Closes #14699 commit 3008934501b7ed11b51582f6b0d0ce5fb5088727 Author: Tino Reichardt Date: Wed Apr 5 18:57:01 2023 +0200 Fix "Add colored output to zfs list" Running `zfs list -o avail rpool` resulted in a core dump. This commit will fix this. Run the needed overhead only, when `use_color()` is true. Reviewed-by: Brian Behlendorf Reviewed-by: George Wilson Signed-off-by: Tino Reichardt Closes #14712 commit a754a25cf6f0e117354910cca30ecfe239d90443 Author: Jorgen Lundman Date: Wed Apr 5 14:38:12 2023 +0900 Don't STATUS_REPARSE if the request was to this mount If we are already looking up entries for this mount it will cause a "recursion" of sorts to return STATUS_REPARSE - as it requests the lookup in the same mount. Signed-off-by: Jorgen Lundman commit 5db33d13d5058b306fb3a4feec7e30bc7ff0da4d Merge: fdc4c1e18c f6e8c6ea67 Author: Jorgen Lundman Date: Tue Apr 4 15:38:31 2023 +0900 Merge branch 'development' of github.com:openzfsonwindows/openzfs into development commit fdc4c1e18c3f60e5397764a4fed82fd0ea62c31b Author: Jorgen Lundman Date: Tue Apr 4 15:36:07 2023 +0900 Windows zfsctl_is_node() takes znode The rename handler was passing the wrong structure cause all renames to fail. Signed-off-by: Jorgen Lundman commit f6e8c6ea67d565ed533beb3b0048d2a2263ff545 Author: наб Date: Fri Mar 31 18:47:48 2023 +0200 contrib: dracut: fix race with root=zfs:dset when necessities required This had always worked in my testing, but a user on hardware reported this to happen 100%, and I reproduced it once with cold VM host caches. dracut-zfs-generator runs as a systemd generator, i.e. at Some Relatively Early Time; if root= is a fixed dataset, it tries to "solve [necessities] statically at generation time". If by that point zfs-import.target hasn't popped (because the import is taking a non-negligible amount of time for whatever reason), it'll see no children for the root datase, and as such generate no mounts. This has never had any right to work. No-one caught this earlier because it's just that much more convenient to have root=zfs:AUTO, which orders itself properly. To fix this, always run zfs-nonroot-necessities.service; this additionally simplifies the implementation by: * making BOOTFS from zfs-env-bootfs.service be the real, canonical, root dataset name, not just "whatever the first bootfs is", and only set it if we're ZFS-booting * zfs-{rollback,snapshot}-bootfs.service can use this instead of re-implementing it * having zfs-env-bootfs.service also set BOOTFSFLAGS * this means the sysroot.mount drop-in can be fixed text * zfs-nonroot-necessities.service can also be constant and always enabled, because it's conditioned on BOOTFS being set There is no longer any code generated at run-time (the sysroot.mount drop-in is an unavoidable gratuitous cp). The flow of BOOTFS{,FLAGS} from zfs-env-bootfs.service to sysroot.mount is not noted explicitly in dracut.zfs(7), because (a) at some point it's just visual noise and (b) it's already ordered via d-p-m.s from z-i.t. Reviewed-by: Brian Behlendorf Signed-off-by: Ahelenia Ziemiańska Closes #14690 commit faf608e60fb9e054dc71d2d63b736be819d778f9 Author: youzhongyang Date: Fri Mar 31 12:46:22 2023 -0400 linux 6.3 compat: needs REQ_PREFLUSH | REQ_OP_WRITE Modify bio_set_flush() so if kernel version is >= 4.10, flags REQ_PREFLUSH and REQ_OP_WRITE are set together. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Signed-off-by: Youzhong Yang Closes #14695 commit bc1793894294e2a132a15c2c84c0d6d9078e2029 Author: Brian Behlendorf Date: Fri Mar 31 09:43:54 2023 -0700 Use vmem_zalloc to silence allocation warning The kmem allocation in zfs_prune_aliases() will trigger a large allocation warning on systems with 64K pages. Resolve this by switching to vmem_alloc() which internally uses kvmalloc() so the right allocator will be used based on the allocation size. Reviewed-by: Richard Yao Reviewed-by: Tino Reichardt Reviewed-by: Brian Atkinson Signed-off-by: Brian Behlendorf Closes #8491 Closes #14694 commit ed898b35d9599be7b01201c087dcd3399ce22b3b Author: Andrew Innes Date: Thu Mar 30 15:04:50 2023 +0800 Fix Linux build Signed-off-by: Andrew Innes commit 0bf87288a0c156a04a8c048bfdfbcc9c7ba41400 Author: Tony Hutter Date: Wed Mar 29 15:39:36 2023 -0700 Linux 6.2 compat: META Update the META file to reflect compatibility with the 6.2 kernel. Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Closes #14689 commit 300a525a7624b5142de4b76b3ec082c03b206d4d Author: George Amanakis Date: Wed Mar 29 01:51:58 2023 +0200 Fixes in persistent error log Address the following bugs in persistent error log: 1) Check nested clones, eg "fs->snap->clone->snap2->clone2". 2) When deleting files containing error blocks in those clones (from "clone" the example above), do not break the check chain. 3) When deleting files in the originating fs before syncing the errlog to disk, do not break the check chain. This happens because at the time of introducing the error block in the error list, we do not have its birth txg and the head filesystem. If the original file is deleted before the error list is synced to the error log (which is when we actually lookup the birth txg and the head filesystem), then we do not have access to this info anymore and break the check chain. The most prominent change is related to achieving (3). We expand the spa_error_entry_t structure to accommodate the newly introduced zbookmark_err_phys_t structure (containing the birth txg of the error block).Due to compatibility reasons we cannot remove the zbookmark_phys_t structure and we also need to place the new structure after se_avl, so it is not accounted for in avl_find(). Then we modify spa_log_error() to also provide the birth txg of the error block. With these changes in place we simplify the previously introduced function get_head_and_birth_txg() (now named get_head_ds()). We chose not to follow the same approach for the head filesystem (thus completely removing get_head_ds()) to avoid introducing new lock contentions. The stack sizes of nested functions (as measured by checkstack.pl in the linux kernel) are: check_filesystem [zfs]: 272 (was 912) check_clones [zfs]: 64 We also introduced two new tests covering the above changes. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14633 commit bc581f1d8ddcba386dde22a2a58b55d07542e137 Author: Kevin Jin <33590050+jxdking@users.noreply.github.com> Date: Tue Mar 28 11:43:41 2023 -0400 Fix short-lived txg caused by autotrim Current autotrim causes short-lived txg through: 1. calling txg_wait_synced() in metaslab_enable() 2. calling txg_wait_open() with should_quiesce = true This patch addresses all the issues mentioned above. A new cv, vdev_autotrim_kick_cv is added to kick autotrim activity. It will be signaled once a txg is synced so that it does not change the original autotrim pace. Also because it is a cv, the wait is interruptible which speeds up the vdev_autotrim_stop_wait() call. Finally, combining big zfs_txg_timeout, txg_wait_open() also causes delay when exporting a pool. Reviewed-by: Brian Behlendorf Signed-off-by: jxdking Issue #8993 Closes #12194 commit e6baa412c832bec4c271b227e0b9071aceb80620 Author: Brian Behlendorf Date: Tue Mar 28 08:19:03 2023 -0700 Additional limits on hole reporting Holding the zp->z_rangelock as a RL_READER over the range 0-UINT64_MAX is sufficient to prevent the dnode from being re-dirtied by concurrent writers. To avoid potentially looping multiple times for external caller which do not take the rangelock holes are not reported after the first sync. While not optimal this is always functionally correct. This change adds the missing rangelock calls on FreeBSD to zvol_cdev_ioctl(). Reviewed-by: Brian Atkinson Signed-off-by: Brian Behlendorf Closes #14512 Closes #14641 commit 70ab86087cfa3500d53c2a7c14fe55a3c75afc8e Author: George Wilson Date: Tue Mar 28 11:13:32 2023 -0400 Revert "Do not hold spa_config in ZIL while blocked on IO" This reverts commit 7d638df09be7482935bcf6ec8e4ea2ac8a8be1a8. Reviewed-by: Prakash Surya Reviewed-by: Brian Behlendorf Signed-off-by: George Wilson Closes #14678 commit 971fc9a3ca1b4b520e5915e325e0500825632df6 Author: Rob N Date: Tue Mar 28 05:55:54 2023 +1100 config: don't link libudev on FreeBSD FreeBSD has a libudev shim in libudev-devd. If present, configure would detect it and produce binaries linked against it, even though nothing used it. That is surprising and unnecessary, so lets remove it. Reviewed-by: Brian Behlendorf Signed-off-by: Rob Norris Closes #14669 commit 98f2c742f65c03d3cfa29edbdc1ffeb152188709 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Mon Mar 27 14:29:19 2023 -0400 linux 6.3 compat: add another bdev_io_acct case Linux 6.3+, and backports from it (6.2.8+), changed the signatures on bdev_io_{start,end}_acct. Add a case for it. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #14658 Closes #14668 commit 86eb0adb3409625e1a240ebc53bcfc237f5bf894 Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Fri Mar 24 22:30:38 2023 +0500 Update vdev state for spare vdev zfsd fetches new pool configuration through ZFS_IOC_POOL_STATS but it does not get updated nvlist configuration for spare vdev since the configuration is read by spa_spares->sav_config. In this commit, updating the vdev state for spare vdev that is consumed by zfsd on spare disk hotplug. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Ameer Hamza Closes #14653 commit 7511e6b661dc64dcc155dee8352c0924c8a83d00 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Fri Mar 24 13:29:19 2023 -0400 Drop lying to the compiler in the fletcher4 code This is probably the uncontroversial part of #13631, which fixes a real problem people are having. There's still things to improve in our code after this is merged, but it should stop the breakage that people have reported, where we lie about a type always being aligned and then pass in stack objects with no alignment requirement and hope for the best. Of course, our SIMD code was written with unaligned accesses, so it doesn't care if we drop this...but some auto-vectorized code that gcc emits sure does, since we told it it can assume they're aligned. Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Signed-off-by: Rich Ercolani Closes #14649 commit 05c0d32e18e8d07bda059427b3bebe3afb3489b1 Author: George Wilson Date: Fri Mar 24 13:27:07 2023 -0400 panic loop when removing slog device There is a window in the slog removal code where a panic loop could ensue if the system crashes during that operation. The original design of slog removal did not persisted any state because the removal happened synchronously. This was changed by a later commit which persisted the vdev_removing flag and exposed this bug. If a slog removal is in progress and happens to crash after persisting the vdev_removing flag to the label but before the vdev is removed from the spa config, then the pool will continue to panic on import. Here's a sample of the panic: [ 134.387411] VERIFY0(0 == dmu_buf_hold_array(os, object, offset, size, FALSE, FTAG, &numbufs, &dbp)) failed (0 == 22) [ 134.393865] PANIC at dmu.c:1135:dmu_write() [ 134.396035] Kernel panic - not syncing: VERIFY0(0 == dmu_buf_hold_array(os, object, offset, size, FALSE, FTAG, &numbufs, &dbp)) failed (0 == 22) [ 134.397857] CPU: 2 PID: 5914 Comm: txg_sync Kdump: loaded Tainted: P OE 5.4.0-1100-dx2023020205-b3751f8c2-azure #106 [ 134.407938] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018 [ 134.407938] Call Trace: [ 134.407938] dump_stack+0x57/0x6d [ 134.407938] panic+0xfb/0x2d7 [ 134.407938] spl_panic+0xcf/0x102 [spl] [ 134.407938] ? traverse_impl+0x1ca/0x420 [zfs] [ 134.407938] ? dmu_object_alloc_impl+0x3b4/0x3c0 [zfs] [ 134.407938] ? dnode_hold+0x1b/0x20 [zfs] [ 134.407938] dmu_write+0xc3/0xd0 [zfs] [ 134.407938] ? space_map_alloc+0x55/0x80 [zfs] [ 134.407938] metaslab_sync+0x61a/0x830 [zfs] [ 134.407938] ? queued_spin_unlock+0x9/0x10 [zfs] [ 134.407938] vdev_sync+0x72/0x190 [zfs] [ 134.407938] spa_sync_iterate_to_convergence+0x160/0x250 [zfs] [ 134.407938] spa_sync+0x2f7/0x670 [zfs] [ 134.407938] txg_sync_thread+0x22d/0x2d0 [zfs] [ 134.407938] ? txg_dispatch_callbacks+0xf0/0xf0 [zfs] [ 134.407938] thread_generic_wrapper+0x83/0xa0 [spl] [ 134.407938] kthread+0x104/0x140 [ 134.407938] ? kasan_check_write.constprop.0+0x10/0x10 [spl] [ 134.407938] ? kthread_park+0x90/0x90 [ 134.457802] ret_from_fork+0x1f/0x40 This change no longer persists the vdev_removing flag when removing slog devices and also cleans up some code that was added which is not used. Reviewed-by: Brian Behlendorf Reviewed-by: Matthew Ahrens Reviewed-by: Mark Maybee Signed-off-by: George Wilson Closes #14652 commit 9bd87cd33c88acb49bcb765a964353c176c2bf20 Author: Tino Reichardt Date: Mon Mar 13 23:31:44 2023 +0100 Add colored output to zfs list Use a bold header row and colorize the AVAIL column based on the used space percentage of volume. We define these colors: - when > 80%, use yellow - when > 90%, use red Reviewed-by: WHR Reviewed-by: Brian Behlendorf Signed-off-by: Ethan Coe-Renner Signed-off-by: Tino Reichardt Closes #14621 Closes #14350 commit 1c689ef6a2e304047c13a4c0f35110b25eef44fa Author: Tino Reichardt Date: Mon Mar 13 23:30:09 2023 +0100 Colorize zpool iostat output Use a bold header and colorize the space suffixes in iostat by order of magnitude like this: - K is green - M is yellow - G is red - T is lightblue - P is magenta - E is cyan - 0 space is colored gray Reviewed-by: WHR Reviewed-by: Brian Behlendorf Signed-off-by: Ethan Coe-Renner Signed-off-by: Tino Reichardt Closes #14621 Closes #14459 commit ac9b65b129e4718973b3d9b2a8eb8430e0e7c702 Author: Tino Reichardt Date: Mon Mar 13 23:23:04 2023 +0100 Add more ANSI colors to libzfs Reviewed-by: WHR Reviewed-by: Brian Behlendorf Signed-off-by: Ethan Coe-Renner Signed-off-by: Tino Reichardt Closes #14621 commit 666ccc653174e0216c4f3c1719017a5339484714 Author: Matthew Ahrens Date: Fri Mar 24 10:20:07 2023 -0700 Fix prefetching of indirect blocks while destroying When traversing a tree of block pointers (e.g. for `zfs destroy ` or `zfs send`), we prefetch the indirect blocks that will be needed, in `traverse_prefetch_metadata()`. In the case of `zfs destroy `, we do a little traversing each txg, and resume the traversal the next txg. So the indirect blocks that will be needed, and thus are candidates for prefetching, does not include blocks that are before the resume point. The problem is that the logic for determining if the indirect blocks are before the resume point is incorrect, causing the (up to 1024) L1 indirect blocks that are inside the first L2 to not be prefetched. In practice, if we are able to read many more than 1024 blocks per txg, then this will be inconsequential. But if i/o latency is more than a few milliseconds, almost no L1's will be prefetched, so they will be read serially, and thus the destroying will be very slow. This can be observed as `zpool get freeing` decreasing very slowly. Specifically: When we first examine the L2 that contains the block we'll be resuming from, we have not yet resumed, so `td_resume` is nonzero. At this point, all calls to `traverse_prefetch_metadata()` will fail, even if the L1 in question is after the resume point. It isn't until the callback is issued for the resume point that we zero out `td_resume`, but by this point we've already attempted and failed to prefetch everything under this L2 indirect block. This commit addresses the issue by reusing the existing `resume_skip_check()` to determine if the L1's bookmark is before or after the resume point. To do so, this function is made non-mutating (the caller now zeros `td_resume`). Note, this bug likely predates (was not introduced by) #11803. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Matthew Ahrens Closes #14603 commit a796f372460d97067f47d8b83b32a85d81f04944 Author: Pawel Jakub Dawidek Date: Fri Mar 24 18:18:35 2023 +0100 Fix cloning into already dirty dbufs. Undirty the dbuf and destroy its buffer when cloning into it. Coverity ID: CID-1535375 Reported-by: Richard Yao Reported-by: Benjamin Coddington Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Signed-off-by: Pawel Jakub Dawidek Closes #14655 commit 4338431d5bbc6f5467cea9880b68644e6791ff80 Author: Rob N Date: Sat Mar 25 04:14:39 2023 +1100 man: add ZIO_STAGE_BRT_FREE to zpool-events And bump all the values after it, matching the header update in 67a1b037. Reviewed-by: Alexander Motin Reviewed-by: Brian Atkinson Signed-off-by: Rob Norris Closes #14665 commit 8ca57cc9a9f2c0f7d2769f1cea8c55cb36f87abd Author: Andrew Innes Date: Wed Mar 22 18:36:09 2023 +0800 Handle both types of slash Signed-off-by: Andrew Innes commit bce09f3a3ed78098aa7b1d2167ffce5aded945db Author: Pawel Jakub Dawidek Date: Wed Mar 22 17:24:41 2023 +0100 Fix build on FreeBSD Constify some variables after d1807f168edd09ca26a5a0c6b570686b982808ad. Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Signed-off-by: Pawel Jakub Dawidek Closes #14656 commit 6add2a1b0fdb9dbaeb42f6b2d0082974500b7ca9 Author: Timothy Day Date: Wed Mar 22 12:22:52 2023 -0400 Fix kmodtool for packaging mainline Linux kmodtool currently incorrectly identifies official RHEL kernels, as opposed to custom kernels. This can cause the openZFS kmod RPM build to break. The issue can be reproduced by building a set of mainline Linux RPMs, installing them, and then attempting to build the openZFS kmod package against them. Reviewed-by: Brian Behlendorf Signed-off-by: Timothy Day Closes #14617 commit b4da22fdf3aee557605a68270307c286a1d0a34e Author: Tino Reichardt Date: Wed Mar 22 16:39:48 2023 +0100 Remove unused constant EdonR256_BLOCK_BITSIZE Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Reviewed-by: George Melikov Signed-off-by: Tino Reichardt Closes #14650 commit 1428f38216d39b07aab2df0fb86e1bcf51d1e729 Author: Alexander Motin Date: Fri Mar 17 20:31:08 2023 -0400 FreeBSD: Remove extra arc_reduce_target_size() call Remove arc_reduce_target_size() call from arc_prune_task(). The idea of arc_prune_task() is to remove external references on ARC metadata, such as vnodes. Since arc_prune_async() is called only from ARC itself, it makes no sense to create a parasitic loop between ARC eviction and the pruning, treatening to drop ARC to its minimum. I can't guess why it was added as part of FreeBSD to OpenZFS integration. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14639 commit 75422560f9006dae466638cb12ed439fd6d485dc Author: Andrew Innes Date: Sat Mar 18 12:22:54 2023 +0800 Windows: Remove build for wsl There's no reason to use this anymore. Just run zfs-linux.yml Signed-off-by: Andrew Innes commit d3cb6642075c16263ae313663a3c6c1e3d2ec05d Author: Richard Yao Date: Thu Mar 16 17:27:49 2023 -0400 Fix possible bad bit shift in dnode_next_offset_level() 031d7c2fe6afaa78943bd0a563b91fc84ace42d7 did not handle reverse iteration, such that the original issue theoretically could still occur. Note that contrary to the claim in the ZFS disk format specification that a maximum of 6 levels are possible, 9 levels are possible with recordsize=512 and and indirect block size of 16KB. In this unusual configuration, span will be 65. The maximum size of span at 70 can be reached at recordsize=16K and an indirect blocksize of 16KB. When we are at this indirection level and are traversing backward, the minimum value is start, but we cannot calculate that with 64-bit arithmetic, so we avoid the calculation and instead rely on the earlier statement that did `*offset = start;`. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Reported-by: Coverity (CID-1466214) Closes #14618 commit 4b8f585e16e2d4e22668aba7cfcad6e003a9d54c Author: naivekun Date: Fri Mar 17 02:54:10 2023 +0800 QAT: Fix uninitialized seed in QAT compression CpaDcRqResults have to be initialized with checksum=1 for adler32. Otherwise when error CPA_DC_OVERFLOW occurred, the next compress operation will continue on previously part-compressed data, and write invalid checksum data. When zfs decompress the compressed data, a invalid checksum will occurred and lead to #14463 Reviewed-by: Tino Reichardt Reviewed-by: Weigang Li Reviewed-by: Chengfei Zhu Signed-off-by: naivekun Closes #14632 Closes #14463 commit 8079523cbc0c3e8db560f61d643f42fc43115be8 Author: Tino Reichardt Date: Thu Mar 16 18:00:14 2023 +0100 Refine some details for the github actions update Set the retention-days variable to 14 days for these artifacts: - the zloop error logs - the zloop vdev files - the compiled modules Add the abality to re-run some part of the functional testings. Fix some comments and remove the deleting of the modules artifact. Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #14637 commit 6526b763e2567c4c0d3172a41a33f4ac7412a00b Author: datacore-rm Date: Fri Feb 24 15:16:15 2023 +0530 use NonPagedPoolNx during memory alloc to pass HLK test commit fbe4b24d00d32cbd53f45fcb65914a59145edd97 Author: Andrew Innes Date: Fri Mar 17 19:56:31 2023 +0800 Windows: Registry key buffer min size Requesting a pool of at least 1 byte stops the driver verifier from complaining Closes #210 Signed-off-by: Andrew Innes commit 4fbeb2bc2322b8738c72cefa6b60811d9d7bfec4 Author: Tino Reichardt Date: Wed Mar 15 23:37:56 2023 +0100 Add git repo checkout to testing workflow Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #14634 commit 034d792cd0c507c7a74f4bc79ff6e6b0a17ab146 Author: Attila Fülöp Date: Wed Mar 15 19:13:25 2023 +0100 spl: cmn_err_once() should be usable in brace-less if else statements Commit 11913870 (#14567) added cmn_err_once() by #define'ing a compound statement but failed to consider usage in a single statement brace-less if else. Fix the problem by using the common "do {} while (0)" construct. Reviewed-by: Brian Behlendorf Signed-off-by: Attila Fülöp Closes #14629 commit f570cc68f9fde2f4a7fe524ce1ee4d900bd24701 Author: Low-power Date: Thu Mar 16 01:44:28 2023 +0800 Fix building for powerpc* targets with some compilers Under some configurations, GCC didn't predefined macro 'powerpc' for such a target. Use the guaranteed macro '__powerpc__' instead. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: WHR Closes #14631 commit c8aec3d1a8cc5f7da192df17233c72ff05570a97 Author: Tino Reichardt Date: Wed Mar 15 18:41:05 2023 +0100 Split functional testings via github action matrix This commit changes the workflow of the github actions. We split the workflow into different parts: 1) build zfs modules for Ubuntu 20.04 and 22.04 (~25m) 2) 2x zloop test (~10m) + 2x sanity test (~25m) 3) functional testings in parts 1..5 (each ~1h) - these could be triggered, when sanity tests are ok - currently I just start them all in the same time 4) cleanup and create summary When everything is fine, the full run with all testings should be done in around 2 hours. The codeql.yml and checkstyle.yml are not part in this circle. The testings are also modified a bit: - report info about CPU and checksum benchmarks - reset the debugging logs for each test - when some error occurred, we call dmesg with -c to get only the log output for the last failed test - we empty also the dbgsys Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #14078 commit ace46c2cb212ee02de781ab0f9ec1c018eec2046 Author: Alek P Date: Wed Mar 15 10:34:10 2023 -0700 Improve tests and update man page for healing recv Fix the manpage. The "SYNOPSIS" section is incorrectly formatted for receive -c. I also took this opportunity to reword some parts and fix a run-on sentence in the manpage. Add large block testing for corrective recv. This adds a new test that makes sure blocks generated using zfs send -L/--large-block large-block send flag are able to be used for healing. Since with unloaded key and errlog feature enabled corruption is not shown in zpool status #13675 is fixed the zfs_receive_corrective.ksh test no longer sets -o feature@head_errlog=disabled on pool creation so that it can also test for regressions related to head_errlog feature. Note that the zfs_receive_compressed_corrective.ksh and zfs_receive_large_block_corrective.ksh tests are still creating pools with -o feature@head_errlog=disabled. Reviewed-by: Brian Behlendorf Signed-off-by: Alek Pinchuk Closes #14615 commit 167368de987b72ec35482ca5855d3daeda876a18 Author: WHR Date: Tue Mar 14 21:25:25 2023 +0800 Refactor CONFIG_SPE check on Linux/powerpc Commit 5401472 adds a check to call enable_kernel_spe and disable_kernel_spe only if CONFIG_SPE is defined. Refactor this check in a way similar to what CONFIG_ALTIVEC and CONFIG_VSX are checked, in order to remove redundant kfpu_begin() and kfpu_end() implementations. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: WHR Closes #14623 commit e11d028e3452d9f29b5ebfe7d2a1d6ca1bc60fb2 Author: WHR Date: Tue Mar 14 21:24:30 2023 +0800 Fix missing semicolons in commit 1f196e3 Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: WHR Closes #14623 commit 8d8f50c411af8cf97c1a2d1fa90adbaa44ab08ca Author: ofthesun9 Date: Wed Mar 15 00:40:55 2023 +0100 Fix for mountpoint=legacy We need to clear mountpoint only after checking it. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: ofthesun9 Closes #14599 Closes #14604 commit 4b29e2a80349cb4bc6ea60393d3fb9245fa830f4 Author: Jorgen Lundman Date: Fri Mar 17 10:27:43 2023 +0900 Remove edonr_byteorder.h Signed-off-by: Jorgen Lundman commit e9b56c5860092dd150deb086e52256a690981025 Author: Tino Reichardt Date: Tue Mar 14 23:59:58 2023 +0100 Remove unused Edon-R variants This commit removes the edonr_byteorder.h file and all unused variants of Edon-R. Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13618 commit 220c1905639c3b5806504c1aad362d0834a7af81 Author: Richard Yao Date: Sat Mar 11 13:39:24 2023 -0500 nvpair: Use flexible array member for nvpair name strings Coverity reported possible out-of-bounds reads from doing `((char *)(nvp) + sizeof (nvpair_t))` to get the nvpair name string. These were initially marked as false positives, but since we are now using C99 flexible array members elsewhere, we could use them here too as cleanup to make the code easier to understand. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Reported-by: Coverity (CID-977165) Reported-by: Coverity (CID-1524109) Reported-by: Coverity (CID-1524642) Closes #14612 commit ecc09ed98c164999c4ff7aedcc5379ef1dd80b48 Author: Richard Yao Date: Sat Mar 11 13:39:24 2023 -0500 nvpair: Constify string functions After addressing coverity complaints involving `nvpair_name()`, the compiler started complaining about dropping const. This lead to a rabbit hole where not only `nvpair_name()` needed to be constified, but also `nvpair_value_string()`, `fnvpair_value_string()` and a few other static functions, plus variable pointers throughout the code. The result became a fairly big change, so it has been split out into its own patch. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14612 commit d17fe710abb1331a3eaec4822d194ee0d1b60d3f Author: Richard Yao Date: Sun Mar 12 12:30:21 2023 -0400 discover_cached_paths() should not corrupt nvlist string value discover_cached_paths() will write a NULL into a string from a nvlist to use it as a substring, but does not restore it before return. This corrupts the nvlist. It should be harmless unless the string is needed again later, but we should not do this, so let us fix it. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14612 commit 46c7ad716ee062cdbd14219cd94eef8b3983e6f6 Author: Richard Yao Date: Sat Mar 11 15:25:04 2023 -0500 zpool_valid_proplist() should not corrupt nvpair name string on error The strings returned from parsing nvlists should be immutable, but to simplify the code when we want a substring from it, we sometimes will write a NULL into it and then restore the value afterward. Provided there is no concurrent access, this is okay, unless we forget to restore the value afterward. This was caught when constifying string functions related to nvlists. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14612 commit 44864cd235f3ae50b7262b81582cbf066b0320d7 Author: Richard Yao Date: Sat Mar 11 13:21:04 2023 -0500 Fix possible NULL pointer dereference in nvlist_lookup_nvpair_ei_sep() Clang's static analyzer complains about a possible NULL pointer dereference in nvlist_lookup_nvpair_ei_sep() because it unconditionally dereferences a pointer initialized by `nvpair_value_nvlist_array()` under the assumption that `nvpair_value_nvlist_array()` will always initialize the pointer without checking to see if an error was returned to indicate otherwise. This itself is improper error handling, so we fix it. However, fixing it to properly respond to errors is not enough to avoid a NULL pointer dereference, since we can receive NULL when the array is empty, so we also add a NULL check. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14612 commit 120b333113ab340a8d02baa70819e3ba41c4073b Author: Richard Yao Date: Sat Mar 11 13:17:59 2023 -0500 Silence clang static analyzer warnings about stored stack addresses Clang's static analyzer complains that nvs_xdr() and nvs_native() functions return pointers to stack memory. That is technically true, but the pointers are stored in stack memory from the caller's stack frame, are not read by the caller and are deallocated when the caller returns, so this is harmless. We set the pointers to NULL to silence the warnings. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14612 commit 8aa4b0ff02125f3a206ced50025db3ad4aab6611 Author: Richard Yao Date: Tue Mar 14 18:00:54 2023 -0400 Fix possible NULL pointer dereference in dbuf_verify() Coverity reported a dereference after a NULL check in dbuf_verify(). If `dn` is `NULL`, we can just assume that !dn->dn_free_txg, so we change `!dn->dn_free_txg` to `(dn == NULL || !dn->dn_free_txg)`. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Reported-by: Coverity (CID-992298) Closes #14619 commit ec7f858984aaa5cfedc4bfb88a655c2be72035c7 Author: Tino Reichardt Date: Tue Mar 14 22:44:01 2023 +0100 Replace dead opensolaris.org license links The commit replaces all findings of the link: http://www.opensolaris.org/os/licensing with this one: https://opensource.org/licenses/CDDL-1.0 Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Reviewed-by: WHR Signed-off-by: Tino Reichardt Closes #14625 commit 07c3e4c785f89fe4587b07c1d499968d1852e0e7 Author: Richard Yao Date: Fri Mar 10 17:42:09 2023 -0500 Zero zio_prop_t in flush_write_batch_impl() After 67a1b0379159c46bcd60a462a2790248046c8804 was merged, coverity started complaining about an uninitialized scalar variable in flush_write_batch_impl() due to the new field zp.zp_brtwrite. Upon inspection, it appears that uninitialized memory was being copied for non-raw streams, so this is a pre-existing issue. The addition of zp_brtwrite by the block cloning commit caused Coverity to begin to notice it. Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Reviewed-by: Rob Norris Signed-off-by: Richard Yao Reported-by: Coverity (CID-1535378) Closes #14607 commit d366dde767e301620b681c44862886dab5816601 Author: Richard Yao Date: Fri Mar 10 17:47:56 2023 -0500 Fix uninitialized scalar value read regression in dmu_recv_begin() da19d919a853ad05ef300fe000e6c96c4db84bcf changed this in a way that permits execution to reach `if (err == 0)` without initializing err. This could randomly cause the sync task to not execute. We fix that by initializing err to zero. Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Reviewed-by: Rob Norris Signed-off-by: Richard Yao Reported-by: Coverity (CID-1535377) Closes #14607 commit 04de0983faa011fc0e8426e5be46dc962d5d1e9e Author: Matthew Ahrens Date: Tue Mar 14 14:30:29 2023 -0700 ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced() `lseek(SEEK_DATA | SEEK_HOLE)` are only accurate when the on-disk blocks reflect all writes, i.e. when there are no dirty data blocks. To ensure this, if the target dnode is dirty, they wait for the open txg to be synced, so we can call them "stabilizing operations". If they cause txg_wait_synced often, it can be detrimental to performance. Typically, a group of files are all modified, and then SEEK_DATA/HOLE are performed on them. In this case, the first SEEK does a txg_wait_synced(), and subsequent SEEKs don't need to wait, so performance is good. However, if a workload involves an interleaved metadata modification, the subsequent SEEK may do a txg_wait_synced() unnecessarily. For example, if we do a `read()` syscall to each file before we do its SEEK. This applies even with `relatime=on`, when the `read()` is the first read after the last write. The txg_wait_synced() is unnecessary because the SEEK operations only care that the structure of the tree of indirect and data blocks is up to date on disk. They don't care about metadata like the contents of the bonus or spill blocks. (They also don't care if an existing data block is modified, but this would be more involved to filter out.) This commit changes the behavior of SEEK_DATA/HOLE operations such that they do not call txg_wait_synced() if there is only a pending change to the bonus or spill block. Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Signed-off-by: Matthew Ahrens Closes #13368 Issue #14594 Issue #14512 Issue #14009 commit 9e01527776b34f882f99b26d650c42fe26c1f78b Author: Jorgen Lundman Date: Fri Mar 17 10:13:28 2023 +0900 Ensure kfpu_begin()/end() are in same scope. For the variabe defined inside kfpu_begin(). Use kfpu_begin_ctx() for the split init/fini functions. Signed-off-by: Jorgen Lundman commit 27b335c2104d89c412b62a522499776ef4c4f857 Author: Attila Fülöp Date: Tue Mar 14 17:45:28 2023 +0100 zcommon: Refactor FPU state handling in fletcher4 Currently calls to kfpu_begin() and kfpu_end() are split between the init() and fini() functions of the particular SIMD implementation. This was done in #14247 as an optimization measure for the ABD adapter. Unfortunately the split complicates FPU handling on platforms that use a local FPU state buffer, like Windows and macOS. To ease porting, we introduce a boolean struct member in fletcher_4_ops_t, indicating use of the FPU, and move the FPU state handling from the SIMD implementations to the call sites. Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Reviewed-by: Jorgen Lundman Signed-off-by: Attila Fülöp Closes #14600 commit f0fbd0d9b84aea86a2953a25ba3fa33de3afe3e4 Author: Jorgen Lundman Date: Thu Mar 16 13:42:31 2023 +0900 Revert "Upstream: add kfpu_begin_ctx(X) kfpu_end_ctx(X)" This reverts commit 4abbdf2216424e2b81c08c1b961436fb9f53aa7f. commit 7e08b66fdc46f4896837575ef06efc6a60a37214 Author: Jorgen Lundman Date: Sat Mar 11 15:37:15 2023 +0900 Windows libzfs.abi changes Signed-off-by: Jorgen Lundman commit 02481eb23043d5265679917e3cc32099af522a76 Author: George Melikov Date: Sat Mar 11 03:23:01 2023 +0300 ABI files: sync with new Ubuntu release in CI We may try to build ZFS inside container too, but let's just sync them for now. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: George Melikov Closes #14605 commit 62f5d52dffa441dce825f699abfe18bb63c86375 Author: Pawel Jakub Dawidek Date: Fri Mar 10 20:59:53 2023 +0100 Implementation of block cloning for ZFS Block Cloning allows to manually clone a file (or a subset of its blocks) into another (or the same) file by just creating additional references to the data blocks without copying the data itself. Those references are kept in the Block Reference Tables (BRTs). The whole design of block cloning is documented in module/zfs/brt.c. Reviewed-by: Alexander Motin Reviewed-by: Christian Schwarz Reviewed-by: Brian Behlendorf Reviewed-by: Rich Ercolani Signed-off-by: Pawel Jakub Dawidek Closes #13392 commit 5223cbdd897cc49558754a8d61eb3f73d7ced1f2 Author: Paul Dagnelie Date: Fri Mar 10 09:52:44 2023 -0800 Fix incremental receive silently failing for recursive sends The problem occurs because dmu_recv_begin pulls in the payload and next header from the input stream in order to use the contents of the begin record's nvlist. However, the change to do that before the other checks in dmu_recv_begin occur caused a regression where an empty send stream in a recursive send could have its END record consumed by this, which broke the logic of recv_skip. A test is also included to protect against this case in the future. Reviewed-by: Brian Behlendorf Signed-off-by: Paul Dagnelie Closes #12661 Closes #14568 commit adca605934093c8312ff8750e09e866aea8c9c50 Author: Low-power Date: Sat Mar 11 01:35:00 2023 +0800 Workaround for Linux PowerPC GPL-only cpu_has_feature() Linux since 4.7 makes interface 'cpu_has_feature' to use jump labels on powerpc if CONFIG_JUMP_LABEL_FEATURE_CHECKS is enabled, in this case however the inline function references GPL-only symbol 'cpu_feature_keys'. ZFS currently uses 'cpu_has_feature' either directly or indirectly from several places; while it is unknown how this issue didn't break ZFS on 64-bit little-endian powerpc, it is known to break ZFS with many Linux versions on both 32-bit and 64-bit big-endian powerpc. Until this issue is fixed in Linux, we have to workaround it by overriding affected inline functions without depending on 'cpu_feature_keys'. Reviewed-by: Brian Behlendorf Signed-off-by: WHR Closes #14590 commit 4686d4d9218a39bc0bea6b5eeba491127ab1d697 Author: Richard Yao Date: Fri Mar 10 12:34:00 2023 -0500 txg_sync should handle write errors in ZIL The txg_sync thread will see certain buffers in a DR_IN_DMU_SYNC state when ZIL is writing them out. Then it waits until the state changes, but has an assertion to check that they were not DR_NOT_OVERRIDDEN. If the data write failed with an error, ZIL will put it into the DR_NOT_OVERRIDDEN state. It looks like the code will handle that state without an issue, so we can just delete the assertion. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Sponsored-By: Wasabi Technology, Inc. Closes #14283 commit cde2e06ef5a7c35e920a8fc64a83cccdb987343f Author: Richard Yao Date: Mon Mar 6 13:29:36 2023 -0500 Suppress clang static analyzer warning in vdev_stat_update() 63652e154643cfe596fe077c13de0e7be34dd863 added unnecessary branches in `vdev_stat_update()` to suppress an ASAN false positive the breaks ztest. This had the downside of causing false positive reports in both Coverity and Clang's static analyzer. vd is never NULL, so we add a preprocessor check to only apply the workaround when compiling with ASAN support. Reported-by: Coverity (CID-1524583) Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 5e8f2e8b833306c2b00a8ec822fb64ab2ef7c3e9 Author: Richard Yao Date: Mon Mar 6 09:48:42 2023 -0500 Refactor loop in dump_histogram() The current loop triggers a complaint that we are using an array offset prior to a range check from cpp/offset-use-before-range-check when we are actually calculating maximum and minimum values. I was about to file a false positive report with CodeQL, but after looking at how the code is structured, I really cannot blame CodeQL for mistaking this for a range check. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit d2f6a943c0e47f13b6f106f75b02a1f7595c30b0 Author: Richard Yao Date: Mon Mar 6 11:15:37 2023 -0500 Suppress static analyzer warning in dmu_objset_create_impl_dnstats() ae7e7006500ca37c471dd625cd5cbfc590b49885 added an assertion to suppress a complaint from Clang's static analyzer. Unfortunately, it missed another way for Clang to complain about this function. This adds another assertion to handle that. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 2e198f74c242bddd1f4377842b270a06568ad232 Author: Richard Yao Date: Sun Mar 5 02:01:58 2023 -0500 Linux: Fix octal detection in define_ddi_strtox() Clang Tidy reported this as a misc-redundant-expression because writing `8` instead of `'8'` meant that the condition could never be true. The only place where we have a chance of this being a bug would be in nvlist_lookup_nvpair_ei_sep(). I am not sure if we ever pass an octal to that, but if we ever do, it should work properly now instead of failing. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit f333906b064dfbfdda5ec164633881b960c4fe25 Author: Richard Yao Date: Sun Mar 5 01:32:03 2023 -0500 Linux: Suppress clang static analyzer warning in zfs_remove() Clang's static analyzer points out that if we fail to find an extended attribute directory, but somehow find it when calculating delete_now and delete_now is true, we will have a NULL pointer dereference when we try to unlink the extended attribute directory. I am not sure if this is possible, but if it is, I do not see a sane way of handling this other than rolling back the transaction and retrying. For now, let us do an VERIFY_IMPLY(). If this trips, it will stop the transaction from committing, which will prevent an attribute directory leak. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 1f66e74debc4aa756797f04adcddf3cfbfde0852 Author: Richard Yao Date: Sat Mar 4 23:48:06 2023 -0500 Linux: Silence static analyzer warning in crypto_create_ctx_template() A CodeChecker report from Clang's CTU analysis indicated that we were assigning uninitialized values in crypto_create_ctx_template() when we call it from zio_crypt_key_init(). This occurs because the ->cm_param and ->cm_param_len fields are uninitialized. Thankfully, the uninitialized values are only used in the skein via KCF_PROV_CREATE_CTX_TEMPLATE() -> skein_create_ctx_template() -> skein_mac_ctx_build() -> skein_get_digest_bitlen(), but that should not be called from here. We fix this to avoid a possible trap should this code change in the future. The FreeBSD version of zio_crypt_key_init() is unaffected. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 42b66848b6e8ed94f758e387d17219171592f50b Author: Richard Yao Date: Sat Mar 4 22:31:33 2023 -0500 Suppress Clang Static Analyzer warning in bpobj_enqueue() scan-build does not do cross translation unit analysis to realize that `dmu_buf_hold()` will always set `bpo->bpo_cached_dbuf` to a non-NULL pointer, so we add an assertion to make it realize this. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 8a6e0550a40a9f16bcaa684b97066291b59cc304 Author: Richard Yao Date: Sat Mar 4 22:23:48 2023 -0500 Suppress Clang Static Analyzer warning in dsl_dir_rename_sync() Clang's static analyzer reports that if we try to rename a root dataset in `dsl_dir_rename_sync()`, we will have a NULL pointer passed to strlcpy(). This is impossible because `dsl_dir_rename_check()` will prevent us from doing this. We add an assertion to silence this warning. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 202b695ad7fb994dab7c0d46ddbc3852de962702 Author: Richard Yao Date: Sat Mar 4 17:57:01 2023 -0500 Cleanup: Remove constant comparisons reported by CodeQL CodeQL's cpp/constant-comparison query from its security-and-extended query set reported 4 instances where we have comparions that always evaluate the same way. In `draid_config_by_type()`, we have an early `if (nparity == 0)` check that returns `EINVAL`, making a later `if (nparity == 0 || nparity > VDEV_DRAID_MAXPARITY)` partially redundant. The later check prints an error message when parity is 0, but the early check does not. This is not useful feedback, so we move the later check to the place where the early check runs to replace the early check. In `perform_thread_merge()`, we return when `num_threads == 0`. After that block, we do `if (num_threads > 0) {`, which will always be true. We remove the `if` statement. In `sa_modify_attrs()`, we have a loop condition that is `k != 2`, but at the end of the loop, we have `if (k == 0 && hdl->sa_spill)` followed by an else that does a break. The result is that k != 2 will never be evaluated when it is false. We drop the comparison. In `zap_leaf_array_read()`, we have a for loop condition that is `i < ZAP_LEAF_ARRAY_BYTES && len > 0`. However, that loop itself is in a loop that is `while (len > 0)` and while the value of len is decremented inside the loop, when `len == 0`, it will return, such that `len > 0` inside the loop condition will always be true. We drop that part of the condition. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 3572deab5a0ffba20345ed2b03d8cd3f10f341e1 Author: Richard Yao Date: Sat Mar 4 16:48:29 2023 -0500 Linux cleanup: zvol_discard() should only call blk_queue_io_stat() once Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 8a33085f1e4a2f616b68aa2638e60ee72f4a4c94 Author: Richard Yao Date: Sat Mar 4 16:11:49 2023 -0500 Suppress Clang Static Analyzer warning in dbuf_dnode_findbp() Clang's static analyzer reports that if a `blkid == DMU_SPILL_BLKID` is passed, then we can have a NULL pointer dereference when either ->dn_have_spill or `DNODE_FLAG_SPILL_BLKPTR` is not set. This should not happen. We add an `ASSERT()` to suppress reports about NULL pointer dereferences. Originally, I wanted to use one or two IMPLY statements on pre-conditions before the call to `dbuf_findbp()`, but Clang's static analyzer did not understand it. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 203f37d7d1e3d647098fa95d9a5fbf2cfa9ab25f Author: Richard Yao Date: Sat Mar 4 15:38:06 2023 -0500 Suppress Clang Static Analyzer warning in vdev_split() Clang's static analyzer pointed out that we can have a NULL pointer dereference if we ever attempt to split a vdev that has only 1 child. If that happens, we are left with zero children, but then try to access a non-existent child. Calling vdev_split() on a vdev with only 1 child should be impossible due to how the code is structured. If this ever happens, it would be best to stop execution immediately even in a production environment to allow for the best possible chance of recovery by an expert, so we use `VERIFY3U()` instead of `ASSERT3U()`. Unfortunately, while that defensive assertion will prevent execution from ever reaching the NULL pointer dereference, Clang's static analyzer does not realize that, so we add an `ASSERT()` to inform it of this. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 55944357bcdc6af861aa1bbf6e0fd83e719584cc Author: Richard Yao Date: Fri Mar 3 21:15:28 2023 -0500 Suppress Clang Static Analyzer warning about SNPRINTF_BLKPTR() Clang's static analyzer pointed out that if we can pass a -1 array index to copyname[copies] if there are no valid DVAs. This is an absurd situation, but it suggests that we are missing an assertion, so we add it. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 494f08ea0d64afee76972038e0bcf4d1b207644f Author: Richard Yao Date: Sun Feb 12 16:38:27 2023 -0500 Suppress Clang Static Analyzer false positive in the AVL tree code. This has been filed as llvm/llvm-project#60694. Switching from a copy through a C pointer dereference to an explicit memcpy() is a workaround that prevents a false positive. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 3f544f7f33ccff5219af33fec93690eda490324b Author: Richard Yao Date: Tue Feb 7 18:47:40 2023 -0500 Suppress Clang Static Analyzer defect report in abd_get_size() Clang's static analyzer reports a possible NULL pointer dereference in abd_get_size() when called from vdev_draid_map_alloc_write() called from vdev_draid_map_alloc_row() and vdc->vdc_nparity == 0. This should be impossible, so we add an assertion to silence the defect report. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14575 commit 25c3a26f7ea0a5488964bbc913c678b250859678 Author: Alexander Motin Date: Wed Mar 8 14:17:23 2023 -0500 More adaptive ARC eviction Traditionally ARC adaptation was limited to MRU/MFU distribution. But for years people with metadata-centric workload demanded mechanisms to also manage data/metadata distribution, that in original ZFS was just a FIFO. As result ZFS effectively got separate states for data and metadata, minimum and maximum metadata limits etc, but it all required manual tuning, was not adaptive and in its heart remained a bad FIFO. This change removes most of existing eviction logic, rewriting it from scratch. This makes MRU/MFU adaptation individual for data and meta- data, same as the distribution between data and metadata themselves. Since most of required states separation was already done, it only required to make arcs_size state field specific per data/metadata. The adaptation logic is still based on previous concept of ghost hits, just now it balances ARC capacity between 4 states: MRU data, MRU metadata, MFU data and MFU metadata. To simplify arc_c changes instead of arc_p measured in bytes, this code uses 3 variable arc_meta, arc_pd and arc_pm, representing ARC balance between metadata and data, MRU and MFU for data, and MRU and MFU for metadata respectively as 32-bit fixed point fractions. Since we care about the math result only when need to evict, this moves all the logic from arc_adapt() to arc_evict(), that reduces per-block overhead, since per-block operations are limited to stats collection, now moved from arc_adapt() to arc_access() and using cheaper wmsums. This also allows to remove ugly ARC_HDR_DO_ADAPT flag from many places. This change also removes number of metadata specific tunables, part of which were actually not functioning correctly, since not all metadata are equal and some (like L2ARC headers) are not really evictable. Instead it introduced single opaque knob zfs_arc_meta_balance, tuning ARC's reaction on ghost hits, allowing administrator give more or less preference to metadata without setting strict limits. Some of old code parts like arc_evict_meta() are just removed, because since introduction of ABD ARC they really make no sense: only headers referenced by small number of buffers are not evictable, and they are really not evictable no matter what this code do. Instead just call arc_prune_async() if too much metadata appear not evictable. Reviewed-by: Brian Behlendorf Reviewed-by: Allan Jude Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14359 commit d24567bc438c0df9cc3ce3280e217fa8ced271c2 Author: Attila Fülöp Date: Wed Mar 8 20:12:15 2023 +0100 ICP: AES-GCM: Unify gcm_init_ctx() and gmac_init_ctx() gmac_init_ctx() duplicates most of the code in gcm_int_ctx() while it just needs to set its own IV length and AAD tag length. Introduce gcm_init_ctx_impl() which handles the GCM and GMAC differences while reusing the duplicated code. While here, fix a flaw where the AVX implementation would accept a context using a byte swapped key schedule which it could not handle. Also constify the IV and AAD pointers passed to gcm_init{,_avx}(). Reviewed-by: Brian Behlendorf Reviewed-by: Rob Norris Signed-off-by: Attila Fülöp Closes #14529 commit 9d2d88b35de741fc804fbb2a4edce0953e6ada1a Author: Richard Yao Date: Tue Mar 7 19:12:28 2023 -0500 Do not hold spa_config in ZIL while blocked on IO Otherwise, we can get a deadlock that looks like this: 1. fsync() grabs spa_config_enter(zilog->zl_spa, SCL_STATE, lwb, RW_READER) as part of zil_lwb_write_issue() . It then blocks on the txg_sync when a flush fails from a drive power cycling. 2. The txg_sync then blocks on the pool suspending due to the loss of too many disks. 3. zpool clear then blocks on spa_config_enter(spa, SCL_STATE | SCL_L2ARC | SCL_ZIO, spa, RW_WRITER) because it is a writer. The disks cannot be brought online due to fsync() holding that lock and the user gets upset since fsync() is uninterruptibly blocked inside the kernel. We need to grab the lock for vdev_lookup_top(), but we do not need to hold it while there is outstanding IO. This fixes a regression introduced by 1ce23dcaff6c3d777cb0d9a4a2cf02b43f777d78. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Sponsored-By: Wasabi Technology, Inc. Closes #14519 commit 174fc344a4878bb3d998ef392da05cde06b733e6 Author: Low-power Date: Wed Mar 8 06:06:52 2023 +0800 Fix build for Linux/powerpc without CONFIG_ALTIVEC or CONFIG_VSX This fixes building ZFS for Linux 4.7+ powerpc* architecture, where Linux was configured without CONFIG_ALTIVEC or CONFIG_VSX. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: WHR Closes #14591 commit 61286f86c9becff0e7e8b34157d5ef78eabbb804 Author: Rob N Date: Wed Mar 8 09:05:14 2023 +1100 Better handling for future crypto parameters The intent is that this is like ENOTSUP, but specifically for when something can't be done because we have no support for the requested crypto parameters; eg unlocking a dataset or receiving a stream encrypted with a suite we don't support. Its not intended to be recoverable without upgrading ZFS itself. If the request could be made to work by enabling a feature or modifying some other configuration item, then some other code should be used. load-key: In the future we might have more crypto suites (ie new values for the `encryption` property. Right now trying to load a key on such a future crypto suite will look up suite parameters off the end of the crypto table, resulting in misbehaviour and/or crashes (or, with debug enabled, trip the assertion in `zio_crypt_key_unwrap`). Instead, lets check the value we got from the dataset, and if we can't handle it, abort early. recv: When receiving a raw stream encrypted with an unknown crypto suite, `zfs recv` would report a generic `invalid backup stream` (EINVAL). While technically correct, its not super helpful, so lets ship a more specific error code and message. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Rob Norris Closes #14577 commit 1573da17cb84bd432eb342f57f281d7be7e172b5 Author: George Amanakis Date: Tue Mar 7 22:50:44 2023 +0100 Fix a typo in ac2038a Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: George Amanakis Closes #14585 Closes #14592 commit 2fec8506381d703b56ca2d19791bfadac7465d2e Author: Andriy Gapon Date: Tue Mar 7 23:48:43 2023 +0200 [FreeBSD] fix false assert in cache_vop_rmdir when replaying ZIL The assert is enabled when DEBUG_VFS_LOCKS kernel option is set. The exact panic is: panic: condition seqc_in_modify(_vp->v_seqc) not met It happens because seqc protocol is not followed for ZIL replay. But we actually do not need to make any namecache calls at that stage, because the namecache use is not enabled until after the replay is completed. Reviewed-by: Alexander Motin Signed-off-by: Andriy Gapon Closes #14566 commit cd2bfa704ef35efba1d2bcb9823c86be1dae633c Author: Attila Fülöp Date: Tue Mar 7 22:44:11 2023 +0100 spl: Add cmn_err_once() to log a message only on the first call Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Attila Fülöp Closes #14567 commit e4765079633a96bddc7c9180a107f342c690d5ab Author: q66 Date: Tue Mar 7 02:07:01 2023 +0100 initramfs: fix zpool get argument order When using the zfs initramfs scripts on my system, I get various errors at initramfs stage, such as: cannot open '-o': name must begin with a letter My zfs binaries are compiled with musl libc, which may be why this happens. In any case, fix the argument order to make the zpool binary happy, and to match its --help output. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Daniel Kolesa Closes #14572 commit d1fd97aa218a02097fc631c2dc40108cd7aee1ce Author: Tino Reichardt Date: Tue Mar 7 02:01:01 2023 +0100 Fix detection of IBM Power8 machines (ISA 2.07) An IBM POWER7 system with Power ISA 2.06 tried to execute zfs_sha256_power8() - which should only be run on ISA 2.07 machines. The detection is implemented via the zfs_isa207_available() call, but this check was not used. This pull request will fix this. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Tino Reichardt Signed-off-by: Low-power Closes #14576 commit d5e444be86350752d8fc890031d8e4f52529bbbd Author: Andriy Gapon Date: Tue Mar 7 02:30:54 2023 +0200 [FreeBSD] zfs_znode_alloc: lock the vnode earlier This is needed because of a possible error path where zfs_vnode_forget() is called. That function calls vgone() and vput(), the former requires the vnode to be exclusively locked and the latter expects it to be locked. It should be safe to lock the vnode as early as possible because it is not yet visible, so there is no interaction with other locks. While here, remove a tautological assignment to 'vp'. Reviewed-by: Alexander Motin Reviewed-by: Richard Yao Signed-off-by: Andriy Gapon Closes #14565 commit 27d1553a935c73c44d8d48fd5fa290539b8a7b85 Author: George Amanakis Date: Tue Mar 7 01:13:05 2023 +0100 Optimize the is_l2cacheable functions by placing the most common use case (no special vdevs) first and avoid allocating new variables. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14494 Closes #14563 commit 59b4e95bccc1ca4a7d67133dfc92108ea1072a5e Author: Richard Yao Date: Mon Mar 6 18:30:29 2023 -0500 Fix memory leak in ztest This is tripping LeakSanitizer, which causes zloop test failures on pull requests. Reviewed-by: Tino Reichardt Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14583 commit 53b83e8508a8a9398c6fc108055305fac55b5de6 Author: Richard Yao Date: Sat Mar 4 18:42:01 2023 -0500 Add missing increment to dsl_deadlist_move_bpobj() dc5c8006f684b1df3f2d4b6b8c121447d2db0017 was recently merged to prefetch up to 128 deadlists. Unfortunately, a loop was missing an increment, such that it will prefetch all deadlists. The performance properties of that patch probably should be re-evaluated. This was caught by CodeQL's cpp/constant-comparison check in an experimental branch where I am testing the security-and-extended queries. It complained about the `i < 128` part of the loop condition always evaluating to the same thing. The standard CodeQL configuration we use missed this because it does not include that check. Reviewed-by: Tino Reichardt Reviewed-by: Alexander Motin Signed-off-by: Richard Yao Closes #14573 commit fb50ca6e7b7ce66eda7efa753f9e96d2e1efab57 Author: Richard Yao Date: Sat Mar 4 15:53:58 2023 -0500 SHA2Init() should use signed assertions when checking an enum The recent 4c5fec01a48acc184614ab8735e6954961990235 commit caused Coverity to report that ASSERT3U(algotype, >=, SHA256_MECH_INFO_TYPE); is always true. That is because the signed algotype and signed SHA256_MECH_INFO_TYPE values were cast to unsigned types. To fix this, we switch the assertions to use ASSERT3S(), which retains the signedness of the original values for the comparison. Reviewed-by: Tino Reichardt Reviewed-by: Alexander Motin Signed-off-by: Richard Yao Reported-by: Coverity (CID-1535300) Closes #14573 commit e31b6bdc767efeb91a092fa086d54428686b36af Author: Jorgen Lundman Date: Sat Mar 11 15:01:10 2023 +0900 Compile fixes for latest rebase Signed-off-by: Jorgen Lundman commit 23b95c59854b3d67f2820694e1350e9b467dd2c8 Author: Jorgen Lundman Date: Sat Mar 11 07:51:17 2023 +0900 zfs unmount can handle snapshots Signed-off-by: Jorgen Lundman commit c40b7de5b7cb80b4fce60ff5fe6243cec5b80120 Author: Jorgen Lundman Date: Fri Mar 10 17:36:27 2023 +0900 zfs-tests: allow snapshot mounting Signed-off-by: Jorgen Lundman commit 551dd50aa416c82c64600b21dfe59d671cba61cc Author: Jorgen Lundman Date: Fri Mar 10 14:14:10 2023 +0900 zfs_prop can not be #ifdef platform Signed-off-by: Jorgen Lundman commit ab9b1b171a6ea2f78cb7c9412ac48ee730f077e8 Author: Jorgen Lundman Date: Fri Mar 10 12:21:48 2023 +0900 sanitize string for system() Signed-off-by: Jorgen Lundman commit 9b76e5f94aac50dd06f8c2e54e6dcb215db0d963 Author: Jorgen Lundman Date: Fri Mar 10 12:05:15 2023 +0900 Widen kmem long variables Since long is 32bit on Windows, some places we do need to use uint64_t instead. Signed-off-by: Jorgen Lundman commit 1ca674501b3ab172a77798ad6645b76729353fb4 Author: Jorgen Lundman Date: Fri Mar 10 11:46:16 2023 +0900 Change sysconf() to be uin64_t and fix warnings The spirit of sysconf() is that it is 64bit in posix worlds and luckily we define it internally in libspl for Windows. Signed-off-by: Jorgen Lundman commit 7d4fa7307f1161539f884e2795b8f01f334b6dfd Author: Jorgen Lundman Date: Fri Mar 10 11:09:00 2023 +0900 Windows: Fix multiplication converted to larger type This fixes the instances of the "Multiplication result converted to larger type" alert that codeQL scanning found. Signed-off-by: Jorgen Lundman commit cdb54ed69c6233e4be1fa0e176630148c2e5c71d Author: Jorgen Lundman Date: Fri Mar 10 09:38:42 2023 +0900 Make zfs_rollback_os() match prototype commit 8c5c82b0af50a12509f8a037b7cb3de9c7fc3556 Author: Jorgen Lundman Date: Fri Mar 10 09:21:46 2023 +0900 zfs_rollback_os should live in libzfs.h commit 3ac5cefdce6eadb7b1715aa3f6d307802984341d Author: Jorgen Lundman Date: Fri Mar 10 09:08:37 2023 +0900 Update libzfs.abi file commit 79f00a3a3464958ca6d72df311ea166a4f473162 Author: Jorgen Lundman Date: Fri Mar 10 08:53:32 2023 +0900 Match zfs_snapshot_mount() commit e549176959d8b770f8da120c756b22b45065c711 Author: Jorgen Lundman Date: Fri Mar 10 08:37:36 2023 +0900 zfs_delete_blocks might overflow when deleting files Since it was defined as a "long" (32bit on Windows) it could trivially overflow and defer tiny deletes until later. commit a01f33b80be62b424628519a567e9819e4c73afe Author: Jorgen Lundman Date: Thu Mar 9 19:57:51 2023 +0900 Windows self review fixes commit 45f59264193b13c88a3f0515e85755a662fd77d4 Author: Jorgen Lundman Date: Thu Mar 9 14:11:07 2023 +0900 Don't modify FreeBSD files without reason commit 02ab9cb94172024d34279b7325538a68c450eeed Author: Jorgen Lundman Date: Thu Mar 9 14:09:13 2023 +0900 Change sprintfs to snprintf It appears CodeQL does not know the windows variant commit acf95dc5255ee13dd9f90869c679245aa6b0d987 Author: Jorgen Lundman Date: Tue Jan 24 14:54:22 2023 +0900 Windows: rename vmem_alloc() and vmem_free() Sadly, Linux defines these as 2 argument funtcions, even though illumos has them as 3 arguments. On Linux they are used as pairs to kmem_alloc() for larger allocations. So let's rename out internal-to-SPL functions to vmem_alloc_impl() and vmem_free_impl(). This allows us to avoid the whole #define mess. Signed-off-by: Jorgen Lundman commit 30729e7d965b4167c416653795524c4b9cd212e3 Author: Jorgen Lundman Date: Thu Mar 9 09:02:10 2023 +0900 Add cmn_err_once and update file commit ac3a05b6174cc4d2a9d30dc502b221afd7cf6119 Author: Jorgen Lundman Date: Wed Mar 8 14:34:59 2023 +0900 CodeQL fixes commit f3fb60c1988f4c0228b680a98058d37f377449bf Author: Jorgen Lundman Date: Wed Mar 8 13:49:20 2023 +0900 cstyle on parsedump.py commit 3a80bb9b3ab92dcda5a95e5d7c8c397cd2f82045 Author: Andrew Innes Date: Wed Mar 8 06:04:04 2023 +0800 Wrap lines commit 8edcc7ede17e900b4ab2025d9dc859a8381c9767 Author: Andrew Innes Date: Tue Mar 7 18:10:25 2023 +0800 Wrap URL Signed-off-by: Andrew Innes commit 723352ddb31a940da21d63a94e80101f9127c304 Author: Jorgen Lundman Date: Wed Jan 18 04:09:19 2023 +0900 Refresh: Unify Assembler files between Linux and Windows Add new macro ASMABI used by Windows to change calling API to "sysv_abi". Reviewed-by: Attila Fülöp Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Jorgen Lundman Closes #14228 commit 4abbdf2216424e2b81c08c1b961436fb9f53aa7f Author: Jorgen Lundman Date: Tue Mar 7 14:21:28 2023 +0900 Upstream: add kfpu_begin_ctx(X) kfpu_end_ctx(X) To allow a little more flexibility and pass the xsave state around. Linux and FreeBSD these macros call the original kfpu_begin() and kfpu_end() macros. Signed-off-by: Joergen Lundman commit 92be030d71d81076cccc370a8337039ab42a348a Author: Richard Yao Date: Mon Dec 5 14:00:34 2022 -0500 Micro-optimize fletcher4 calculations When processing abds, we execute 1 `kfpu_begin()`/`kfpu_end()` pair on every page in the abd. This is wasteful and slows down checksum performance versus what the benchmark claimed. We correct this by moving those calls to the init and fini functions. Also, we always check the buffer length against 0 before calling the non-scalar checksum functions. This means that we do not need to execute the loop condition for the first loop iteration. That allows us to micro-optimize the checksum calculations by switching to do-while loops. Note that we do not apply that micro-optimization to the scalar implementation because there is no check in `fletcher_4_incremental_native()`/`fletcher_4_incremental_byteswap()` against 0 sized buffers being passed. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14247 commit b8b9a3a00337f5bce094b0c752c75c75e3712897 Author: Jorgen Lundman Date: Sat Mar 4 17:00:35 2023 +0900 Retire kfpu-vars The kfpu-begin() calls are no longer nested. commit f11d25feb5eeeec0e0a714872ee22dc77067c8cd Author: Jorgen Lundman Date: Sat Mar 4 16:23:44 2023 +0900 Windows: Use correct SHA-NI bit on leaf7 commit 07f568d7b231cbea1cfdd04b0de05de991b0ac30 Author: Jorgen Lundman Date: Sat Mar 4 16:23:22 2023 +0900 Make sure all SHA2 transform function has wrappers For ASMABI to work, it is required the calling convention is consistent. commit 75dc92b3ab09c0a5b5bf0a13308a6ad7628e168d Author: Jorgen Lundman Date: Fri Mar 3 19:03:29 2023 +0900 Windows: All compiles fixes from rebase commit 2934c88f5cdefae8d16bc350e800c4e4d4567fda Author: Jorgen Lundman Date: Fri Mar 3 18:58:47 2023 +0900 Restore ASMABI and other Unify work commit b0f718b71ae430747d6cc7ae20ad696c9fd3d53f Author: Tino Reichardt Date: Wed Mar 1 20:44:49 2023 +0100 Use SECTION_STATIC macro for sha2 x86_64 assembly - instead of ".section .rodata" we should use SECTION_STATIC Tested-by: Rich Ercolani Tested-by: Sebastian Gottschall Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13741 commit 3dce364123b5cd452a32df713acfcd29e744b21a Author: Tino Reichardt Date: Mon Feb 27 16:14:37 2023 +0100 Update BLAKE3 for using the new impl handling This commit changes the BLAKE3 implementation handling and also the calls to it from the ztest command. Tested-by: Rich Ercolani Tested-by: Sebastian Gottschall Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13741 commit 383a18e10a54e7e25abe197de22cd7c0754559f5 Author: Tino Reichardt Date: Wed Mar 1 09:40:28 2023 +0100 Add generic implementation handling and SHA2 impl The skeleton file module/icp/include/generic_impl.c can be used for iterating over different implementations of algorithms. It is used by SHA256, SHA512 and BLAKE3 currently. The Solaris SHA2 implementation got replaced with a version which is based on public domain code of cppcrypto v0.10. These assembly files are taken from current openssl master: - sha256-x86_64.S: x64, SSSE3, AVX, AVX2, SHA-NI (x86_64) - sha512-x86_64.S: x64, AVX, AVX2 (x86_64) - sha256-armv7.S: ARMv7, NEON, ARMv8-CE (arm) - sha512-armv7.S: ARMv7, NEON (arm) - sha256-armv8.S: ARMv7, NEON, ARMv8-CE (aarch64) - sha512-armv8.S: ARMv7, ARMv8-CE (aarch64) - sha256-ppc.S: Generic PPC64 LE/BE (ppc64) - sha512-ppc.S: Generic PPC64 LE/BE (ppc64) - sha256-p8.S: Power8 ISA Version 2.07 LE/BE (ppc64) - sha512-p8.S: Power8 ISA Version 2.07 LE/BE (ppc64) Tested-by: Rich Ercolani Tested-by: Sebastian Gottschall Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13741 commit 2d9f39a7480f35506cfc00b994d88d6b180fac9b Author: Tino Reichardt Date: Wed Sep 28 10:54:35 2022 +0200 Add SHA2 SIMD feature tests for libspl These are added via HWCAP interface: - zfs_neon_available() for arm and aarch64 - zfs_sha256_available() for arm and aarch64 - zfs_sha512_available() for aarch64 This one via cpuid() call: - zfs_shani_available() for x86_64 Tested-by: Rich Ercolani Tested-by: Sebastian Gottschall Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13741 commit 160896204e07c58776bb25a3933b03538dd6036d Author: Tino Reichardt Date: Wed Sep 28 10:54:02 2022 +0200 Add SHA2 SIMD feature tests for Linux These are added: - zfs_neon_available() for arm and aarch64 - zfs_sha256_available() for arm and aarch64 - zfs_sha512_available() for aarch64 - zfs_shani_available() for x86_64 Tested-by: Rich Ercolani Tested-by: Sebastian Gottschall Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Co-Authored-By: Sebastian Gottschall Closes #13741 commit bef092a65fbed79cb0e43644509f11f67f995bc4 Author: Tino Reichardt Date: Wed Sep 28 10:53:18 2022 +0200 Add SHA2 SIMD feature tests for FreeBSD These are added: - zfs_neon_available() for arm and aarch64 - zfs_sha256_available() for arm and aarch64 - zfs_sha512_available() for aarch64 - zfs_shani_available() for x86_64 Changes: - simd_powerpc.h: change license from CDDL to BSD Tested-by: Rich Ercolani Tested-by: Sebastian Gottschall Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13741 commit 4139f2c3386e5f217c60bedeb83196e66038e792 Author: Tino Reichardt Date: Wed Sep 28 10:55:04 2022 +0200 Add ARM architecture to OpenZFS buildsystem Tested-by: Rich Ercolani Tested-by: Sebastian Gottschall Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13741 commit 234a839daeb89b82a79d8570f76a0956ca0757ad Author: Tino Reichardt Date: Mon Feb 27 16:11:51 2023 +0100 Remove old or redundant SHA2 files We had three sha2.h headers in different places. The FreeBSD version, the Linux version and the generic solaris version. The only assembly used for acceleration was some old x86-64 openssl implementation for sha256 within the icp module. For FreeBSD the whole SHA2 files of FreeBSD were copied into OpenZFS, these files got removed also. Tested-by: Rich Ercolani Tested-by: Sebastian Gottschall Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #13741 commit 619458fdd51fc01f675a5ce9ba4fa562cccc1454 Author: Rob N Date: Fri Mar 3 08:39:09 2023 +1100 zdb: add decryption support The approach is straightforward: for dataset ops, if a key was offered, find the encryption root and the various encryption parameters, derive a wrapping key if necessary, and then unlock the encryption root. After that all the regular dataset ops will return unencrypted data, and that's kinda the whole thing. Reviewed-by: Brian Behlendorf Reviewed-by: Jorgen Lundman Signed-off-by: Rob Norris Closes #11551 Closes #12707 Closes #14503 commit 95a9c0be0b73ecab9157688abc71061de819afb2 Author: Alexander Motin Date: Wed Mar 1 18:27:40 2023 -0500 System-wide speculative prefetch limit. With some pathological access patterns it is possible to make ZFS accumulate almost unlimited amount of speculative prefetch ZIOs. Combined with linear ABD allocations in RAIDZ code, it appears to be possible to exhaust system KVA, triggering kernel panic. Address this by introducing a system-wide counter of active prefetch requests and blocking prefetch distance doubling per stream hits if the number of active requests is higher that ~6% of ARC size. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14516 commit edb8e93adcaeba26c76e73e610d6eda91a82633f Author: Brian Behlendorf Date: Wed Mar 1 14:40:45 2023 -0800 Accommodate debug-kernel stack frame size The blk_queue_discard() and blk_queue_sector_erase() functions slightly exceed the allowed 4096 maximum stack frame size when building with the RedHat debug kernel which causes their configure checks to fail. Add an exception for these two tests so the interfaces are correctly detected. Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Closes #14540 commit 9d9fcb2573e4954e3a4a0eacf85de2da38d57ff4 Author: Richard Yao Date: Wed Mar 1 16:23:09 2023 -0500 Fix data race between zil_commit() and zil_suspend() openzfsonwindows/openzfs#206 found that it is possible to trip `VERIFY(list_is_empty(&lwb->lwb_itxs))` when a `zil_commit()` is delayed by the scheduler long enough for a parallel `zil_suspend()` operation to exit `zil_commit_impl()`. This is a data race. To prevent this, we introduce a `zilog->zl_suspend_lock` rwlock to ensure that all outstanding `zil_commit()` operations finish before `zil_suspend()` begins and that subsequent operations fallback to `txg_wait_synced()` after `zil_suspend()` has begun. On `PREEMPT_RT` Linux kernels, the `rw_enter()` implementation suffers from writer starvation. This means that a ZIL intensive system can delay `zil_suspend()` indefinitely. This is a pre-existing problem that affects everything that uses rw locks, so it needs to be addressed in the SPL. However, builds against `PREEMPT_RT` Linux kernels are currently broken due to a GPL symbol issue (#11097), so we can safely disregard that issue for now. Reported-by: Arun KV Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14514 commit b25c1e18cdaa22a5030be37599a2790eb234c3d2 Author: Richard Yao Date: Wed Mar 1 16:20:53 2023 -0500 Remove bad kmem_free() oversight from previous zfsdev_state_list patch I forgot to remove the corresponding kmem_free() from zfs_kmod_fini() in 9a14ce43c3d6a9939804215bbbe66de5115ace42. Clang's static analyzer did not complain, but the Coverity scan that was run after the patch was merged did. Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Signed-off-by: Richard Yao Reported-by: Coverity (CID-1535275) Closes #14556 commit f602d91dc1de9ed9db2f4ebd00bc22e7fd81d583 Author: Richard Yao Date: Wed Mar 1 16:19:47 2023 -0500 Linux: zfs_fillpage() should handle partial pages from end of file After 89cd2197b94986d315b9b1be707b645baf59af4f was merged, Clang's static analyzer began complaining about a dead assignment in `zfs_fillpage()`. Upon inspection, I noticed that the dead assignment was because we are not using the calculated io_len that we should use to avoid asking the DMU to read past the end of a file. This should result in `dmu_buf_hold_array_by_dnode()` calling `zfs_panic_recover()`. This issue predates 89cd2197b94986d315b9b1be707b645baf59af4f, but its simplification of zfs_fillpage() eliminated the only use of the assignment to io_len, which made Clang's static analyzer complain about the issue. Also, as a precaution, we add an assertion that io_offset < i_size. If this ever fails, bad things will happen. Otherwise, we are blindly trusting the kernel not to give us invalid offsets. We continue to blindly trust it on non-debug kernels. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14534 commit 5c42b81926d9f3109eac061eaea67c36d1a083f8 Author: Tino Reichardt Date: Wed Mar 1 21:25:09 2023 +0100 Update reclaim disk space script The script uses systemd-run, which does the job in background. We should take the the time and wait for the job to finish. Maybe some functional tests suffer from not really freed disk space and fail because of this. We also add some trimming in the end of the script. Reviewed-by: George Melikov Reviewed-by: Brian Behlendorf Signed-off-by: Tino Reichardt Closes #14554 commit 7f960d16807d50cf803aff3e668f307524399c35 Author: Richard Yao Date: Wed Mar 1 12:39:41 2023 -0500 Handle unexpected errors in zil_lwb_commit() without ASSERT() We tripped `ASSERT(error == ENOENT || error == EEXIST || error == EALREADY)` in `zil_lwb_commit()` at Klara when doing robustness testing of ZIL against drive power cycles. That assertion presumably exists because when this code was written, the only errors expected from here were EIO, ENOENT, EEXIST and EALREADY, with EIO having its own handling before the assertion. However, upon doing a manual depth first search traversal of the source tree, it turns out that a large number of unexpected errors are possible here. In theory, EINVAL and ENOSPC can come from dnode_hold_impl(). However, most unexpected errors originate in the block layer and come to us from zio_wait() in various ways. One way is ->zl_get_data() -> dmu_buf_hold() -> dbuf_read() -> zio_wait(). From vdev_disk.c on Linux alone, zio_wait() can return the unexpected errors ENXIO, ENOTSUP, EOPNOTSUPP, ETIMEDOUT, ENOSPC, ENOLINK, EREMOTEIO, EBADE, ENODATA, EILSEQ and ENOMEM This was only observed after what have been likely over 1000 test iterations, so we do not expect to reproduce this again to find out what the error code was. However, circumstantial evidence suggests that the error was ENXIO. When ENXIO or any other unexpected error occurs, the `fsync()` or equivalent operation that called zil_commit() will return success, when in fact, dirty data has not been committed to stable storage. This is a violation of the Single UNIX Specification. The code should be able to handle this and any other unknown error by calling `txg_wait_synced()`. In addition to changing the code to call txg_wait_synced() on unexpected errors instead of returning, we modify it to print information about unexpected errors to dmesg. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Sponsored-By: Wasabi Technology, Inc. Closes #14532 commit 0063b7188838f4207877f9d03570570d96deef0e Author: Rob N Date: Thu Mar 2 04:38:31 2023 +1100 mancheck: exclude dotfiles in man dir Its not uncommon for an editor to drop a hidden swap file in the dir while editing a file there. mancheck would find it and run mandoc on it, which would complain about its distinctly not-manpage format. A more correct solution might be to reconfigure the editor to not put swap files in the same dir, but its the default a lot of the time, and this is a very small change that gives a very nice quality-of-life improvement. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Rob Norris Closes #14549 commit a118698b30dba1a27cec39627c33ad853203fae4 Author: Rob N Date: Wed Mar 1 12:35:33 2023 +1100 man: note that zdb operates directly on pool storage A frequent misunderstanding is that zdb accesses the pool through the kernel or filesystem, leading to confusion particularly when it can't access something that it seems like it should be able to. I've seen this confusion recently when zdb couldn't access a pool because the user didn't have permission to read directly from the block devices, and when it couldn't show attributes of encrypted files even though the dataset was unlocked. The manpage already speaks to another symptom of this, namely that zdb may "behave erratically" on an active pool; here I'm trying to make that a little more explicit. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: George Melikov Signed-off-by: Rob Norris Closes #14539 commit fa48e5c0e863fcf0211f96d0dd8677b9bfbdd106 Author: Allan Jude Date: Tue Feb 28 20:33:59 2023 -0500 Makefile.bsd: cleanup and sync with FreeBSD xxhash.c was not being compiled, so when FreeBSD's kernel switched to a newer version of ZSTD a few weeks ago, out-of-tree ZFS failed to build Sync module/Makefile.bsd with FreeBSD's sys/modules/zfs/Makefile And restore the alphabetical sort in a number of places Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Allan Jude Sponsored-by: Klara, Inc. Closes #14508 commit 85c48c2153a93cecef5f78a8ebbe67992340dd50 Author: Richard Yao Date: Tue Feb 7 05:43:05 2023 -0500 Suppress static analyzer warning in dmu_objset_create_impl_dnstats() Clang's static analyzer claims that dereferencing ds in dmu_objset_create_impl_dnstats() could cause a NULL pointer dereference when a previous NULL check confirms that it is NULL. It is only NULL on the MOS, for which dmu_objset_userused_enabled(os) should always return false, so ds will never be dereferenced when it is NULL. We add an assertion to suppress this warning. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14470 commit f307821e4bc6edb7ec5808bfa5f5151575f3101c Author: Richard Yao Date: Tue Feb 7 04:51:56 2023 -0500 Suppress static analyzer warning in dbuf_hold_copy() Clang's static analyzer claims that dbuf_hold_copy() will have a NULL pointer dereference in data->b_data when called by dbuf_hold_impl(). This is impossible because data is dr->dt.dl.dr_data, which is non-NULL whenever db->db_level == 0, which is always the case whenever dbuf_hold_impl() calls dbuf_hold_copy(). We add an assertion to suppress the complaint. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14470 commit 70b955f119f6159b93fbe26cea01fcac940b3140 Author: Richard Yao Date: Tue Feb 7 03:23:45 2023 -0500 Statically allocate first node of zfsdev_state_list This avoids a call to kmem_alloc() during module load. It also suppresses a defect report from Clang's static analyzer that claims that we will have a NULL pointer dereference in zfsdev_state_init() because it does not understand that this has already been allocated in zfs_kmod_init(). Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14470 commit 0bab95c48be6718caac52bfefc717fd817495e69 Author: Richard Yao Date: Tue Feb 7 03:02:10 2023 -0500 Suppress static analyzer warning in sa_attr_iter() Clang's static analyzer points out that when IS_SA_BONUSTYPE(type) is true and .sa_length is 0 for an attribute, we have a NULL pointer dereference. We suppress this with an IMPLY() statement. This was also identified by Coverity. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Reported-by: Coverity (CID-1017954) Closes #14470 commit dcb26c3dc064b5d52c41ff500b819fa2657ba293 Author: Richard Yao Date: Tue Feb 7 02:38:45 2023 -0500 Suppress static analyzer warnings in zio_checksum_error_impl() Clang's static analyzer informs us of multiple NULL pointer dereferences involving zio_checksum_error_impl(). The first is a NULL pointer dereference if bp is NULL and ci->ci_flags & ZCHECKSUM_FLAG_EMBEDDED is false, but bp is NULL implies that ci->ci_flags & ZCHECKSUM_FLAG_EMBEDDED is true, so we add an IMPLY() statement to suppress the report. The second and third are identical, and are duplicated because while the NULL pointer dereference occurs in zio_checksum_gang_verifier(), it is called by zio_checksum_error_impl() and there is a report for each of the two functions. The reports state that when bp is NULL, ci->ci_flags & ZCHECKSUM_FLAG_EMBEDDED is true and checksum is not ZIO_CHECKSUM_LABEL, we also have a NULL pointer dereference. bp is NULL should imply that checksum == ZIO_CHECKSUM_LABEL, so we add an IMPLY() statement to suppress the second report. The two reports are functionally identical. A fourth variation of this was also reported by Coverity. It occurs when checksum == ZIO_CHECKSUM_ZILOG2. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Reported-by: Coverity (CID-1524672) Closes #14470 commit dbd5d014bf868f7449445166653d1cf785d82fb1 Author: Richard Yao Date: Tue Feb 28 20:28:50 2023 -0500 icp: Prevent compilers from optimizing away memset() in gcm_clear_ctx() The recently merged f58e513f7408f353bf0151fdaf235d4e062e8950 was intended to zero sensitive data before exit from encryption functions to harden the code against theoretical information leaks. Unfortunately, the method by which it did that is optimized away by the compiler, so some information still leaks. This was confirmed by counting function calls in disassembly. After studying how the OpenBSD, FreeBSD and Linux kernels handle this, and looking at our disassembly, I decided on a two-factor approach to protect us from compiler dead store elimination passes. The first factor is to stop trying to inline gcm_clear_ctx(). GCC does not actually inline it in the first place, and testing suggests that dead store elimination passes appear to become more powerful in a bad way when inlining is forced, so we recognize that and move gcm_clear_ctx() to a C file. The second factor is to implement an explicit_memset() function based on the technique used by `secure_zero_memory()` in FreeBSD's blake2 implementation, which coincidentally is functionally identical to the one used by Linux. The source for this appears to be a LLVM bug: https://llvm.org/bugs/show_bug.cgi?id=15495 Unlike both FreeBSD and Linux, we explicitly avoid the inline keyword, based on my observations that GCC's dead store elimination pass becomes more powerful when inlining is forced, under the assumption that it will be equally powerful when the compiler does decide to inline function calls. Disassembly of GCC's output confirms that all 6 memset() calls are executed with this patch applied. Reviewed-by: Attila Fülöp Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14544 commit ec6f8d6832ab799912faf0e4c812a7c0fdc1c4d5 Author: Richard Yao Date: Tue Feb 28 20:27:20 2023 -0500 Linux: Assert mutex is held in mutex_exit() A spurious mutex_exit() in a development branch caused weird issues until I identified it. An assertion prior to mutex_exit() would have caught it. Rather than adding assertions before invocations of mutex_exit() in the code, let us simply add an assertion to mutex_exit(). It is cheap and will likely improve developer productivity. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Sponsored-By: Wasabi Technology, Inc. Closes #14541 commit 46a63d44c98f990a4a309c9f0268bcab27f6a116 Author: George Amanakis Date: Tue Feb 28 23:03:52 2023 +0100 Revert zfeature_active() to static Commit 34ce4c4 made zfeature_active() non-static. This is not required. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: George Amanakis Closes #14546 commit 13dda9cb56f6410883fbc76de694c670f90d26ab Author: Tino Reichardt Date: Tue Feb 28 23:02:48 2023 +0100 Fix github build failures because of vdevprops.7 This small fix adds the manpage vdevprops.7 to the file contrib/debian/openzfs-zfsutils.install and the github actions will work again. Reviewed-by: Brian Behlendorf Reviewed-by: George Melikov Reviewed-by: Brian Atkinson Signed-off-by: Tino Reichardt Closes #14553 commit 57c28153312fb5861e10a0a52a6d27b8b646ab23 Author: John Poduska Date: Mon Feb 27 19:49:34 2023 -0500 Prevent incorrect datasets being mounted During a mount, zpl_mount_impl(), uses sget() with the callback zpl_test_super() to find a super_block with a matching objset, stored in z_os. It does so without taking the teardown lock on the zfsvfs. The problem is that operations like rollback will replace the z_os. And, there is a window where the objset in the rollback is freed, but z_os still points to it. Then, a mount like operation, for instance a clone, can reallocate that exact same pointer and zpl_test_super() will then match the super_block associated with the rollback as opposed to the clone. This fix tests for a match and if so, takes the teardown lock before doing the final match test. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: John Poduska Closes #14518 commit d0ae5d7eae5a1ad793b36ad6b86fc24ab8a77505 Author: D. Ebdrup Date: Tue Feb 28 01:38:09 2023 +0100 Add vdevprops.7 to the Makefile Adding vdevprops.7 to the Makefile ensures that it gets installed properly on FreeBSD. Reviewed-by: Richard Yao Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Reviewed-by: George Melikov Signed-off-by: Daniel Ebdrup Jensen Closes #14527 commit 7d17af34f4b7cdb4c769c60919505726bceaf8ec Author: Richard Yao Date: Mon Feb 27 17:41:02 2023 -0500 Skip memory allocation when compressing holes Hole detection in the zio compression code allows us to opportunistically skip compression on holes. We can go a step further by not doing memory allocations on holes either. Reviewed-by: Brian Atkinson Reviewed-by: Alexander Motin Signed-off-by: Richard Yao Sponsored-by: Wasabi Technology, Inc. Closes #14500 commit b5039ce125e053208dd3162c96c7adcc5843aac3 Author: Attila Fülöp Date: Mon Feb 27 23:38:12 2023 +0100 ICP: AES-GCM: Refactor gcm_clear_ctx() Currently the temporary buffer in which decryption takes place isn't cleared on context destruction. Further in some routines we fail to call gcm_clear_ctx() on error exit. Both flaws may result in leaking sensitive data. We follow best practices and zero out the plaintext buffer before freeing the memory holding it. Also move all cleanup into gcm_clear_ctx() and call it on any context destruction. The performance impact should be negligible. Reviewed-by: Brian Behlendorf Reviewed-by: Rob Norris Signed-off-by: Attila Fülöp Closes #14528 commit da1a92439a1a72e0423fc067f821f6a8dbce775f Author: Mariusz Zaborski Date: Mon Feb 27 23:27:58 2023 +0100 Move zap_attribute_t to the heap in dsl_deadlist_merge In the case of a regular compilation, the compiler raises a warning for a dsl_deadlist_merge function, that the stack size is to large. In debug build this can generate an error. Move large structures to heap. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Mariusz Zaborski Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Closes #14524 commit 2c99fee48840b8ccb098f9160661d885c4e79ebe Author: Brian Behlendorf Date: Mon Feb 27 09:19:25 2023 -0800 Workaround GitHub Action failure Ubuntu 20.04 and 22.04 workflows are failing due to an error which is hit when running `apt-get update`. Until the problematic package is fixed apply the suggested workaround described here: https://github.com/orgs/community/discussions/47863 Reviewed-by: Matthew Ahrens Reviewed-by: George Melikov Reviewed-by: Richard Yao Reviewed-by: Tino Reichardt Signed-off-by: Brian Behlendorf Closes #14530 commit 14f99f0c4283b8009cc0ddca4d280b087493ced6 Author: Dimitry Andric Date: Sat Feb 25 01:45:48 2023 +0100 Use .section .rodata instead of .rodata on FreeBSD In commit 0a5b942d4 the FreeBSD SECTION_STATIC macro was set to ".rodata". This assembler directive is supported by LLVM (as a convenience alias for ".section .rodata") by not by GNU as. This caused the FreeBSD builds that are done with gcc to fail. Therefore, use ".section .rodata" instead, similar to the other asm_linkage.h headers. Reviewed-by: Mateusz Guzik Reviewed-by: Attila Fülöp Reviewed-by: Jorgen Lundman Signed-off-by: Dimitry Andric Closes #14526 commit 6db68fa3743b1e81834c4ec5719bfd32736cde01 Author: George Amanakis Date: Fri Feb 24 02:14:52 2023 +0100 Move dmu_buf_rele() after dsl_dataset_sync_done() Otherwise the dataset may be freed after the last dmu_buf_rele() leading to a panic. Reviewed-by: Mark Maybee Reviewed-by: Matthew Ahrens Signed-off-by: George Amanakis Closes #14522 Closes #14523 commit e977ccd91239a89504bde9122bbb450ce2328029 Author: Brian Behlendorf Date: Thu Feb 23 17:10:46 2023 -0800 ZTS: Minor fixes - The migration_012_pos.ksh test case was failing because of a missing space after `log_must`. - None of the tests listed in the runfiles should include the .ksh suffix. Reviewed-by: Richard Yao Reviewed-by: Brian Atkinson Reviewed-by: George Melikov Reviewed-by: Ryan Moeller Signed-off-by: Brian Behlendorf Closes #14515 commit cfc6db8c682fa47f810c5cdbe153c88c8238bd6f Author: Brian Behlendorf Date: Thu Feb 23 10:57:24 2023 -0800 Fix buffered/direct/mmap I/O race When a page is faulted in for memory mapped I/O the page lock may be dropped before it has been read and marked up to date. If a buffered read encounters such a page in mappedread() it must wait until the page has been updated. Failure to do so will result in a panic on debug builds and incorrect data on production builds. The critical part of this change is in mappedread() where pages which are not up to date are now handled. Additionally, it includes the following simplifications. - zfs_getpage() and zfs_fillpage() could be passed an array of pages. This could be more efficient if it was used but in practice only a single page was ever provided. These interfaces were simplified to acknowledge that. - update_pages() was modified to correctly set the PG_error bit on a page when it cannot be read by dmu_read(). - Setting PG_error and PG_uptodate was moved to zfs_fillpage() from zpl_readpage_common(). This is consistent with the handling in update_pages() and mappedread(). - Minor additional refactoring to comments and variable declarations to improve readability. - Add a test case to exercise concurrent buffered, direct, and mmap IO to the same file. - Reduce the mmap_sync test case default run time. Reviewed-by: Richard Yao Reviewed-by: Brian Atkinson Signed-off-by: Brian Behlendorf Closes #13608 Closes #14498 commit 61e23230adc7cf5b7bf09e2faa3586cc934e014c Author: Richard Yao Date: Thu Feb 23 13:19:08 2023 -0500 Fix NULL pointer dereference in zio_ready() Clang's static analyzer correctly identified a NULL pointer dereference in zio_ready() when ZIO_FLAG_NODATA has been set on a zio that is missing a block pointer. The NULL pointer dereference occurs because we have logic intended to disable ZIO_FLAG_NODATA when it has been set on a gang block. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14469 commit ff142f0f3e7f7b25d5b5d5546051ffa3a3ccdc2b Author: Richard Yao Date: Wed Feb 22 19:33:23 2023 -0500 Use rw_tryupgrade() in dmu_bonus_hold_by_dnode() When dn->dn_bonus == NULL, dmu_bonus_hold_by_dnode() will unlock its read lock on dn->dn_struct_rwlock and grab a write lock. This can be micro-optimized by calling rw_tryupgrade(). Linux will not benefit from this since it does not support rwlock upgrades, but FreeBSD will. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14517 commit c6279b594e3ff6b8af9132d6a173677d0e77daac Author: Paul Dagnelie Date: Tue Feb 21 17:30:05 2023 -0800 Improve error message of zfs redact We improve the error message of zfs redact by checking if the target snapshot exists, and if all the redaction snapshots exist. As a future improvement we could iterate over every snapshot provided and use that to determine which one specifically doesn't exist. Reviewed-by: Brian Behlendorf Signed-off-by: Paul Dagnelie Closes #11426 Closes #14496 commit 52156d9b3884e692e83cebac5796c4bbdb1a02a3 Author: rob-wing <98866084+rob-wing@users.noreply.github.com> Date: Tue Feb 21 16:26:33 2023 -0900 FreeBSD: don't verify recycled vnode for zfs control directory Under certain loads, the following panic is hit: panic: page fault KDB: stack backtrace: #0 0xffffffff805db025 at kdb_backtrace+0x65 #1 0xffffffff8058e86f at vpanic+0x17f #2 0xffffffff8058e6e3 at panic+0x43 #3 0xffffffff808adc15 at trap_fatal+0x385 #4 0xffffffff808adc6f at trap_pfault+0x4f #5 0xffffffff80886da8 at calltrap+0x8 #6 0xffffffff80669186 at vgonel+0x186 #7 0xffffffff80669841 at vgone+0x31 #8 0xffffffff8065806d at vfs_hash_insert+0x26d #9 0xffffffff81a39069 at sfs_vgetx+0x149 #10 0xffffffff81a39c54 at zfsctl_snapdir_lookup+0x1e4 #11 0xffffffff8065a28c at lookup+0x45c #12 0xffffffff806594b9 at namei+0x259 #13 0xffffffff80676a33 at kern_statat+0xf3 #14 0xffffffff8067712f at sys_fstatat+0x2f #15 0xffffffff808ae50c at amd64_syscall+0x10c #16 0xffffffff808876bb at fast_syscall_common+0xf8 The page fault occurs because vgonel() will call VOP_CLOSE() for active vnodes. For this reason, define vop_close for zfsctl_ops_snapshot. While here, define vop_open for consistency. After adding the necessary vop, the bug progresses to the following panic: panic: VERIFY3(vrecycle(vp) == 1) failed (0 == 1) cpuid = 17 KDB: stack backtrace: #0 0xffffffff805e29c5 at kdb_backtrace+0x65 #1 0xffffffff8059620f at vpanic+0x17f #2 0xffffffff81a27f4a at spl_panic+0x3a #3 0xffffffff81a3a4d0 at zfsctl_snapshot_inactive+0x40 #4 0xffffffff8066fdee at vinactivef+0xde #5 0xffffffff80670b8a at vgonel+0x1ea #6 0xffffffff806711e1 at vgone+0x31 #7 0xffffffff8065fa0d at vfs_hash_insert+0x26d #8 0xffffffff81a39069 at sfs_vgetx+0x149 #9 0xffffffff81a39c54 at zfsctl_snapdir_lookup+0x1e4 #10 0xffffffff80661c2c at lookup+0x45c #11 0xffffffff80660e59 at namei+0x259 #12 0xffffffff8067e3d3 at kern_statat+0xf3 #13 0xffffffff8067eacf at sys_fstatat+0x2f #14 0xffffffff808b5ecc at amd64_syscall+0x10c #15 0xffffffff8088f07b at fast_syscall_common+0xf8 This is caused by a race condition that can occur when allocating a new vnode and adding that vnode to the vfs hash. If the newly created vnode loses the race when being inserted into the vfs hash, it will not be recycled as its usecount is greater than zero, hitting the above assertion. Fix this by dropping the assertion. FreeBSD-issue: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252700 Reviewed-by: Andriy Gapon Reviewed-by: Mateusz Guzik Reviewed-by: Alek Pinchuk Reviewed-by: Ryan Moeller Signed-off-by: Rob Wing Co-authored-by: Rob Wing Submitted-by: Klara, Inc. Sponsored-by: rsync.net Closes #14501 commit 020b8bf5d33bda671a96e7ca5ff962e1652bb19f Author: Allan Jude Date: Tue Feb 21 20:23:01 2023 -0500 Fix per-jail zfs.mount_snapshot setting When jail.conf set the nopersist flag during startup, it was incorrectly destroying the per-jail ZFS settings. Reported-by: Martin Matuska Reviewed-by: Ryan Moeller Signed-off-by: Allan Jude Sponsored-by: Modirum MDPay Sponsored-by: Klara, Inc. Closes #14509 commit d5395a5d02ca10a74e26e6cc1200bf97403a24f7 Author: George Amanakis Date: Tue Feb 21 18:36:22 2023 +0100 Partially revert eee9362a7 With commit 34ce4c42f applied, there is no need for eee9362a7. Revert that aside from the test. All tests introduced in those commits pass. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14502 commit 543963776ab78708eb797eed042858f27e125487 Author: Richard Yao Date: Thu Feb 16 17:10:52 2023 -0500 Sync thread should avoid holding the spa config write lock when possible spa_sync() currently grabs the write lock due to an old hack that is documented by a comment: We need the write lock here because, for aux vdevs, calling vdev_config_dirty() modifies sav_config. This is ugly and will become unnecessary when we eliminate the aux vdev wart by integrating all vdevs into the root vdev tree. This has lead to deadlocks in rare edge cases from holding the write lock. We can reduce incidence of these deadlocks by not grabbing the write lock on pools without auxillary vdevs. Sponsored-By: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14282 commit e3daf6cfd3debd102ef5d0a9738eb389b1290356 Author: Paul Dagnelie Date: Thu Feb 16 09:23:39 2023 -0800 zfs redact fails when dnodesize=auto Add handling to dmu_object_next for the case where *objectp == 0. Reviewed-by: Brian Behlendorf Signed-off-by: Paul Dagnelie Closes #14479 commit f9ee9226e48de06945e944e5a69f8b2226ef38c5 Author: Brian Behlendorf Date: Wed Feb 15 09:06:29 2023 -0800 zdb: zero-pad checksum output follow up Apply zero padding for checksums consistently. The SNPRINTF_BLKPTR macro was not updated in commit ac7648179c8 which results in the `cli_root/zdb/zdb_checksum.ksh` test case reliably failing. Reviewed-by: Igor Kozhukhov Reviewed-by: Akash B Reviewed-by: Brian Atkinson Signed-off-by: Brian Behlendorf Closes #14497 commit 2172b401320a68bf60985e328a9cf18a5a39ac8d Author: Richard Yao Date: Tue Feb 14 14:05:41 2023 -0500 Suppress Clang static analyzer complaint in zfs_replay_create() Clang's static analyzer incorrectly complains about an undefined value here when lr->lr_common.lrc_txtype == TX_SYMLINK and txtype == TX_CREATE. This is impossible, because of this line: txtype = (lr->lr_common.lrc_txtype & ~TX_CI((uint64_t)0x1 << 63)); Changing the code to compare against txtype suppresses the report. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14472 commit 752ced1608af28e2d38c4837ba0a7afb97fc4327 Author: Brian Behlendorf Date: Tue Feb 14 11:04:34 2023 -0800 Linux: use filemap_range_has_page() As of the 4.13 kernel filemap_range_has_page() can be used to check if there is a page mapped in a given file range. When available this interface should be used which eliminates the need for the zp->z_is_mapped boolean. Reviewed-by: Brian Atkinson Signed-off-by: Brian Behlendorf Closes #14493 commit 8108ab55afb219ed44800659e5fa35e6f1862eb6 Author: Richard Yao Date: Tue Feb 14 14:03:42 2023 -0500 Give strlcat() full buffer lengths rather than smaller buffer lengths strlcat() is supposed to be given the length of the destination buffer, including the existing contents. Unfortunately, I had been overzealous when I wrote a51288aabbbc176a8a73a8b3cd56f79607db32cf, since I gave it the length of the destination buffer, minus the existing contents. This likely caused a regression on large strings. On the topic of being overzealous, the use of strlcat() in dmu_send_estimate_fast() was unnecessary because recv_clone_name is a fixed length string. We continue using strlcat() mostly as defensive programming, in case the string length is ever changed, even though it is unnecessary. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14476 commit 7d17bdf95a3e60d0e2f4130f62ae67440422a558 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Mon Feb 13 19:40:13 2023 -0500 quick fix for lingering snapdir unmount problems Unfortunately, even after e79b6807, I still, much more rarely, tripped asserts when playing with many ctldir mounts at once. Since this appears to happen if we dispatched twice too fast, just ignore it. We don't actually need to do anything if someone already started doing it for us. Reviewed-by: Brian Behlendorf Signed-off-by: Rich Ercolani Closes #14462 commit 0dddba325f65d57489f17847713d168071f925e3 Author: George Amanakis Date: Tue Feb 14 01:37:46 2023 +0100 Fix a race condition in dsl_dataset_sync() when activating features The zio returned from arc_write() in dmu_objset_sync() uses zio_nowait(). However we may reach the end of dsl_dataset_sync() which checks if we need to activate features in the filesystem without knowing if that zio has even run through the ZIO pipeline yet. In that case we will flag features to be activated in dsl_dataset_block_born() but dsl_dataset_sync() has already completed its run and those features will not actually be activated. Mitigate this by moving the feature activation code in dsl_dataset_sync_done(). Also add new ASSERTs in dsl_scan_visitbp() checking if a block contradicts any filesystem flags. Reviewed-by: Matthew Ahrens Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Reviewed-by: Brian Atkinson Signed-off-by: George Amanakis Closes #13816 commit 2df1f43c284b8eb739fcc710831f1057d81269ac Author: Prakash Surya Date: Mon Feb 13 16:35:59 2023 -0800 Reduce need for contiguous memory for ioctls We've had cases where we trigger an OOM despite having memory freely available on the system. For example, here, we had about 21GB free: kernel: Node 0 Normal: 2418758*4kB (UME) 1549533*8kB (UE) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 22071296kB The problem being, all the memory is in 4K and 8K contiguous regions, but the allocation request was for a 16K contiguous region: kernel: SafeExecutors-4 invoked oom-killer: gfp_mask=0x42dc0(GFP_KERNEL|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO), order=2, oom_score_adj=0 The offending allocation came from this call trace: kernel: Call Trace: kernel: dump_stack+0x57/0x7a kernel: dump_header+0x4f/0x1e1 kernel: oom_kill_process.cold.33+0xb/0x10 kernel: out_of_memory+0x1ad/0x490 kernel: __alloc_pages_slowpath+0xd55/0xe40 kernel: __alloc_pages_nodemask+0x2df/0x330 kernel: kmalloc_large_node+0x42/0x90 kernel: __kmalloc_node+0x25a/0x320 kernel: ? spl_kmem_free_impl+0x21/0x30 [spl] kernel: spl_kmem_alloc_impl+0xa5/0x100 [spl] kernel: spl_kmem_zalloc+0x19/0x20 [spl] kernel: zfsdev_ioctl+0x2b/0xe0 [zfs] kernel: do_vfs_ioctl+0xa9/0x640 kernel: ? __audit_syscall_entry+0xdd/0x130 kernel: ksys_ioctl+0x67/0x90 kernel: __x64_sys_ioctl+0x1a/0x20 kernel: do_syscall_64+0x5e/0x200 kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 kernel: RIP: 0033:0x7fdca3674317 The problem is, for each ioctl that ZFS makes, it has to allocate a zfs_cmd_t structure, which is 13744 bytes in size (on my system): sdb> sizeof zfs_cmd (size_t)13744 This size, coupled with the fact that we currently allocate it with kmem_zalloc, means we need a 16K contiguous region of memory to satisfy the request. The solution taken by this change, is to use "vmem" instead of "kmem" to do the allocation, such that we don't necessarily need a contiguous 16K memory region to satisfy the allocation. Arguably, a better solution would be not to require such a large allocation to begin with (e.g. reduce the size of the zfs_cmd_t structure), but that'd be a much larger change than this "one liner". Thus, I've opted for this approach for now; we can always circle back and attempt to reduce the size of the structure in the future. Reviewed-by: Matthew Ahrens Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: Mark Maybee Reviewed-by: Don Brady Signed-off-by: Prakash Surya Closes #14474 commit dd450dbb7524908bdbc767dd8bbca21d2f408863 Author: Alexander Motin Date: Mon Feb 13 16:21:53 2023 -0500 Improve arc_read() error reporting Debugging reported NULL de-reference panic in dnode_hold_impl() I found that for certain types of errors arc_read() may only return error code, but not properly report it via done and pio arguments. Lack of done calls may result in reference and/or memory leaks in higher level code. Lack of error reporting via pio may result in unnoticed errors there. For example, dbuf_read(), where dbuf_read_impl() ignores arc_read() return, relies completely on the pio mechanism and missed the errors. This patch makes arc_read() to always call done callback and always propagate errors to parent zio, if either is provided. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14454 commit f69cdff7d96c4dfb8093e19792ef3c8100c55305 Author: Andreas Vögele Date: Thu Feb 9 20:57:50 2023 +0100 rpm: Use libtirpc-devel and /usr/lib on SUSE SUSE Linux distributions require libtirpc-devel. The dracut and udev directories are /usr/lib/dracut and /usr/lib/udev. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Reviewed-by: Richard Yao Signed-off-by: Andreas Vögele Closes #14467 Closes #14468 commit f379a9ff8070a42890671d7bcaa29c6bbc40e457 Author: Rob N ★ Date: Wed Feb 8 08:48:22 2023 +1100 zdb: zero-pad checksum output The leading zeroes are part of the checksum so we should show them. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Rob Norris Closes #14464 commit 086d7d26030b96cc7cffaa23b6ff6bd26b4f1493 Author: Ryan Moeller Date: Mon Feb 6 14:16:01 2023 -0500 initramfs: Make mountpoint=none work In initramfs, mount.zfs fails to mount a dataset with mountpoint=none, but mount.zfs -o zfsutil works. Use -o zfsutil when mountpoint=none. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Ryan Moeller Closes #14455 commit 75d58bac3f1c0196e711902895e0d9bf397dffe9 Author: Richard Yao Date: Thu Feb 2 17:30:39 2023 -0500 Add assertion and make variables unsigned in abd_alloc_chunks() Clang's static analyzer pointed out that if alloc_pages >= nr_pages before the loop, the value of page will be undefined and will be used anyway. This should not be possible, but as cleanup, we add an assertion. We also recognize that the local variables should be unsigned in the first place, so we make them unsigned. This is not enough to avoid the need for the assertion, since there is still the case that alloc_pages == nr_pages and nr_pages == 0, which the assertion implicitly checks. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14456 commit f6ee4657fa83faab0a453644bc00ac01538b790f Author: Richard Yao Date: Mon Jan 23 15:03:21 2023 -0500 Cleanup: spa vdev processing should check NULL pointers The PVS Studio 2016 FreeBSD kernel report stated: \contrib\opensolaris\uts\common\fs\zfs\spa.c (1341): error V595: The 'spa->spa_spares.sav_vdevs' pointer was utilized before it was verified against nullptr. Check lines: 1341, 1342. \sys\cddl\contrib\opensolaris\uts\common\fs\zfs\spa.c (1355): error V595: The 'spa->spa_l2cache.sav_vdevs' pointer was utilized before it was verified against nullptr. Check lines: 1355, 1357. \sys\cddl\contrib\opensolaris\uts\common\fs\zfs\spa.c (1398): error V595: The 'spa->spa_spares.sav_vdevs' pointer was utilized before it was verified against nullptr. Check lines: 1398, 1408. \sys\cddl\contrib\opensolaris\uts\common\fs\zfs\spa.c (1583): error V595: The 'oldvdevs' pointer was utilized before it was verified against nullptr. Check lines: 1583, 1595. In practice, all of these uses were safe because a NULL pointer implied a 0 vdev count, which kept us from iterating over vdevs. However, rearranging the code to check the pointer first is not a terrible micro-optimization and makes it more readable, so let us do that. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14456 commit 9d312d1264c7c548b2e5bead492ad2ae99e49bee Author: Richard Yao Date: Fri Dec 23 00:00:38 2022 -0500 zfs_get_temporary_prop() should not pass NULL to strcpy() `dsl_dir_activity_in_progress()` can call `zfs_get_temporary_prop()` with the forth value set to NULL, which will pass NULL to `strcpy()` when there is a match Clang's static analyzer caught this with the help of CodeChecker for Cross Translation Unit analysis. Reviewed-by: Brian Behlendorf Reviewed-by: Brian Atkinson Signed-off-by: Richard Yao Closes #14456 commit a5d9422838de158c91041be0696bc39de22b08e4 Author: Matthew Ahrens Date: Mon Feb 6 09:37:06 2023 -0800 EIO caused by encryption + recursive gang Encrypted blocks can not have 3 DVAs, because they use the space of the 3rd DVA for the IV+salt. zio_write_gang_block() takes this into account, setting `gbh_copies` to no more than 2 in this case. Gang members BP's do not have the X (encrypted) bit set (nor do they have the DMU level and type fields set), because encryption is not handled at this level. The gang block is reassembled, and then encryption (and compression) are handled. To check if this gang block is encrypted, the code in zio_write_gang_block() checks `pio->io_bp`. This is normally fine, because the block that's being ganged is typically the encrypted BP. The problem is that if there is "recursive ganging", where a gang member is itself a gang block, then when zio_write_gang_block() is called to create a gang block for a gang member, `pio->io_bp` is the gang member's BP, which doesn't have the X bit set, so the number of DVA's is not restricted to 2. It should instead be looking at the the "gang leader", i.e. the top-level gang block, to determine how many DVA's can be used, to avoid a "NDVA's inversion" (where a child has more DVA's than its parent). gang leader BP: X (encrypted) bit set, 2 DVA's, IV+salt in 3rd DVA's space: ``` DVA[0]=<1:...:100400> DVA[1]=<0:...:100400> salt=... iv=... [L0 ZFS plain file] fletcher4 uncompressed encrypted LE gang unique double size=100000L/100000P birth=... fill=1 cksum=... ``` leader's GBH contains a BP with gang bit set and 3 DVA's: ``` DVA[0]=<1:...:55600> DVA[1]=<0:...:55600> [L0 unallocated] fletcher4 uncompressed unencrypted LE contiguous unique double size=55600L/55600P birth=... fill=0 cksum=... DVA[0]=<1:...:55600> DVA[1]=<0:...:55600> [L0 unallocated] fletcher4 uncompressed unencrypted LE contiguous unique double size=55600L/55600P birth=... fill=0 cksum=... DVA[0]=<1:...:55600> DVA[1]=<0:...:55600> DVA[2]=<1:...:200> [L0 unallocated] fletcher4 uncompressed unencrypted LE gang unique double size=55400L/55400P birth=... fill=0 cksum=... ``` On nondebug bits, having the 3rd DVA in the gang block works for the most part, because it's true that all 3 DVA's are available in the gang member BP (in the GBH). However, for accounting purposes, gang block DVA's ASIZE include all the space allocated below them, i.e. the 512-byte gang block header (GBH) as well as the gang members below that. We see that above where the gang leader BP is 1MB logical (and after compression: 0x`100000P`), but the ASIZE of each DVA is 2 sectors (1KB) more than 1MB (0x`100400`). Since thre are 3 copies of a block below it, we increment the ATIME of the 3rd DVA of the gang leader by the space used by the 3rd DVA of the child (1 sector, in this case). But there isn't really a 3rd DVA of the parent; the salt is stored in place of the 3rd DVA's ASIZE. So when zio_write_gang_member_ready() increments the parent's BP's `DVA[2]`'s ASIZE, it's actually incrementing the parent's salt. When we later try to read the encrypted recursively-ganged block, the salt doesn't match what we used to write it, so MAC verification fails and we get an EIO. ``` zio_encrypt(): encrypted 515/2/0/403 salt: 25 25 bb 9d ad d6 cd 89 zio_decrypt(): decrypting 515/2/0/403 salt: 26 25 bb 9d ad d6 cd 89 ``` This commit addresses the problem by not increasing the number of copies of the GBH beyond 2 (even for non-encrypted blocks). This simplifies the logic while maintaining the ability to traverse all metadata (including gang blocks) even if one copy is lost. (Note that 3 copies of the GBH will still be created if requested, e.g. for `copies=3` or MOS blocks.) Additionally, the code that increments the parent's DVA's ASIZE is made to check the parent DVA's NDVAS even on nondebug bits. So if there's a similar bug in the future, it will cause a panic when trying to write, rather than corrupting the parent BP and causing an error when reading. Reviewed-by: Brian Behlendorf Co-authored-by: Brian Behlendorf Signed-off-by: Matthew Ahrens Caused-by: #14356 Closes #14440 Closes #14413 commit 77a0de73ae0cbbf4e7bd08289a61c1764a52feab Author: Jorgen Lundman Date: Tue Feb 7 02:34:59 2023 +0900 Restore FreeBSD to use .rodata In https://github.com/openzfs/zfs/pull/14228 the FreeBSD SECTION_STATIC was set to ".data" instead of ".rodata". This commit just restores it back to .rodata. Reviewed-by: Attila Fülöp Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Jorgen Lundman Closes #14460 commit 0395a4c78b3073833b319267e87c0384e1f98d8d Author: Jorgen Lundman Date: Tue Feb 7 02:27:55 2023 +0900 Unify assembly files with macOS The remaining changes needed to make the assembly files work with macOS. Reviewed-by: Attila Fülöp Reviewed-by: Brian Behlendorf Signed-off-by: Jorgen Lundman Closes #14451 commit 2fca7626cd14cb1f6a16d7fc7bfc891f7f71f894 Author: Reno Reckling Date: Fri Feb 3 00:22:12 2023 +0100 Fix variable shadowing in libzfs_mount We accidentally reused variable name "i" for inner and outer loops. Reviewed-by: Rich Ercolani Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Signed-off-by: Reno Reckling Closes #14452 Closes #14445 commit 9bbe52ee09a3af1b06fcb2ffd10176a55ede4ce2 Author: Paul Dagnelie Date: Thu Feb 2 15:19:26 2023 -0800 Prevent error messages when running tests with no timeout Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Paul Dagnelie Closes #14450 commit 42411f79fc8bd076719552bec76d48fb1faba612 Author: George Amanakis Date: Fri Feb 3 00:17:37 2023 +0100 Teach zdb about DMU_OT_ERROR_LOG objects With the persistent error log feature we need to account for spa_errlog_{scrub, last} containing mappings to other error log objects, which need to be marked as in-use as well. Reviewed-by: Matthew Ahrens Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14442 Closes #14434 commit fc1e3027272e3c02f7e200d49853b74f341feac0 Author: rob-wing <98866084+rob-wing@users.noreply.github.com> Date: Thu Feb 2 14:16:40 2023 -0900 zfs_main.c: fix unused variable error with GCC zfs_setproctitle_init() is stubbed out on FreeBSD. Reviewed-by: Ryan Moeller Reviewed-by: Ameer Hamza Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Rob Wing Closes #14441 commit c8a2526f559263cdf9cf740b5e5c52720c4ec859 Author: Allan Jude Date: Thu Feb 2 18:12:51 2023 -0500 Resolve WS-2021-0184 vulnerability in zstd Pull in d40f55cd950919d7eac951b122668e55e33e5202 from upstream Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Allan Jude Closes #14439 commit 13d7dee4941a64afa8f9b06b035cc43d3e461cb4 Author: George Wilson Date: Thu Feb 2 18:11:35 2023 -0500 rootdelay on zfs should be adaptive The 'rootdelay' boot option currently pauses the boot for a specified amount of time. The original intent was to ensure that slower configurations would have ample time to enumerate the devices to make importing the root pool successful. This, however, causes unnecessary boot delay for environments like Azure which set this parameter by default. This commit changes the initramfs logic to pause until it can successfully load the 'zfs' module. The timeout specified by 'rootdelay' now becomes the maximum amount of time that initramfs will wait before failing the boot. Reviewed-by: Brian Behlendorf Reviewed-by: Prakash Surya Signed-off-by: George Wilson Closes #14430 commit c836dcab9ca0cece49ea571b4fce1008968be5dd Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Fri Feb 3 04:09:57 2023 +0500 Fix console progress reporting for recursive send After commit 19d3961, progress reporting (-v) with replication flag enabled does not report the progress on the console. This commit fixes the issue by updating the logic to check for pa->progress instead of pa_verbosity in send_progress_thread(). Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Ameer Hamza Closes #14448 commit b8b239bc1cf35e963f646fa918cca62be66194b4 Author: Brian Behlendorf Date: Tue Jan 24 15:23:32 2023 -0800 Increase default zfs_rebuild_vdev_limit to 64MB When testing distributed rebuild performance with more capable hardware it was observed than increasing the zfs_rebuild_vdev_limit to 64M reduced the rebuild time by 17%. Beyond 64MB there was some improvement (~2%) but it was not significant when weighed against the increased memory usage. Memory usage is capped at 1/4 of arc_c_max. Additionally, vr_bytes_inflight_max has been moved so it's updated per-metaslab to allow the size to be adjust while a rebuild is running. Reviewed-by: Akash B Reviewed-by: Tony Nguyen Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #14428 commit e84523dd20dd882ffb4ff2987acc317ed0e0269a Author: Brian Behlendorf Date: Tue Jan 24 14:05:45 2023 -0800 Increase default zfs_scan_vdev_limit to 16MB For HDD based pools the default zfs_scan_vdev_limit of 4M per-vdev can significantly limit the maximum scrub performance. Increasing the default to 16M can double the scrub speed from 80 MB/s per disk to 160 MB/s per disk. This does increase the memory footprint during scrub/resilver but given the performance win this is a reasonable trade off. Memory usage is capped at 1/4 of arc_c_max. Note that number of outstanding I/Os has not changed and is still limited by zfs_vdev_scrub_max_active. Reviewed-by: Akash B Reviewed-by: Tony Nguyen Reviewed-by: Alexander Motin Signed-off-by: Brian Behlendorf Closes #14428 commit 99f46ad643bee8611b347f9cacf966d2871ea2ba Author: Alexander Motin Date: Wed Jan 25 14:30:24 2023 -0500 Prefetch on deadlists merge During snapshot deletion ZFS may issue several reads for each deadlist to merge them into next snapshot's or pool's bpobj. Number of the dead lists increases with number of snapshots. On HDD pools it may take significant time during which sync thread is blocked. This patch introduces prescient prefetch of required blocks for up to 128 deadlists ahead. Tests show reduction of time required to delete dataset with 720 snapshots with randomly overwritten file on wide HDD pool from 75-85 to 22-28 seconds. Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Issue #14276 Closes #14402 commit 1236608e662bdc7cc7a45db9076f206072d1df6b Author: Brian Behlendorf Date: Wed Jan 25 11:28:54 2023 -0800 Improve resilver ETAs When resilvering the estimated time remaining is calculated using the average issue rate over the current pass. Where the current pass starts when a scan was started, or restarted, if the pool was exported/imported. For dRAID pools in particular this can result in wildly optimistic estimates since the issue rate will be very high while scanning when non-degraded regions of the pool are scanned. Once repair I/O starts being issued performance drops to a realistic number but the estimated performance is still significantly skewed. To address this we redefine a pass such that it starts after a scanning phase completes so the issue rate is more reflective of recent performance. Additionally, the zfs_scan_report_txgs module option can be set to reset the pass statistics more often. Reviewed-by: Akash B Reviewed-by: Tony Hutter Signed-off-by: Brian Behlendorf Closes #14410 commit 0ee7ccdafa847ddfc05b2f420852964974bd4be7 Author: Coleman Kane Date: Tue Jan 24 14:20:50 2023 -0500 linux 6.2 compat: zpl_set_acl arg2 is now struct dentry Linux 6.2 changes the second argument of the set_acl operation to be a "struct dentry *" rather than a "struct inode *". The inode* parameter is still available as dentry->d_inode, so adjust the call to the _impl function call to dereference and pass that pointer to it. Also document that the get_acl -> get_inode_acl member name change from commit 884a693 was an API change also introduced in Linux 6.2. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Coleman Kane Closes #14415 commit f7650a0b34281c3b1f8f48b44bced0ef80cee5f3 Author: Alexander Motin Date: Tue Jan 24 12:20:32 2023 -0500 Introduce minimal ZIL block commit delay Despite all optimizations, tests on actual hardware show that FreeBSD kernel can't sleep for less then ~2us. Similar tests on Linux show ~50us delay at least from nanosleep() (haven't tested inside kernel). It means that on very fast log device ZIL may not be able to satisfy zfs_commit_timeout_pct block commit timeout, increasing log latency more than desired. Handle that by introduction of zil_min_commit_timeout parameter, specifying minimal timeout value where additional delays to aggregate writes may be skipped. Also skip delays if the LWB is more than 7/8 full, that often happens if I/O sizes are constant and match one of LWB sizes. Both things are applied only if there were no already outstanding log blocks, that may indicate single-threaded workload, that by definition can not benefit from the commit delays. While there, add short time moving average to zl_last_lwb_latency to make it more stable. Tests of single-threaded 4KB writes to NVDIMM SLOG on FreeBSD show IOPS increase by 9% instead of expected 5%. For zfs_commit_timeout_pct of 1 there IOPS increase by 5.5% instead of expected 1%. Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Reviewed-by: Prakash Surya Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14418 commit 24078bc5835956a2278743503c559b3f22d9359a Author: Attila Fülöp Date: Mon Jan 23 20:25:21 2023 +0100 x86 asm: Replace .align with .balign The .align directive used to align storage locations is ambiguous. On some platforms and assemblers it takes a byte count, on others the argument is interpreted as a shift value. The current usage expects the first interpretation. Replace it with the unambiguous .balign directive which always expects a byte count, regardless of platform and assembler. Reviewed-by: Jorgen Lundman Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Signed-off-by: Attila Fülöp Closes #14422 commit b31db49e4ecb8f41ff0e0c8a223ae36af5f52df7 Author: Attila Fülöp Date: Mon Jan 23 19:48:39 2023 +0100 IPC: blake3 x86 asm: fix placement of .size directives The .size directive used by the SET_SIZE C macro uses the special dot symbol to calculate the size of a function. The dot symbol refers to the current address, so for the calculation to be meaningful the SET_SIZE macro must be placed immediately after the end of the function the size is calculated for. Reviewed-by: Jorgen Lundman Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Signed-off-by: Attila Fülöp Closes #14422 commit 95d64f563b0ab97108f0984bcca17cae1a3e3604 Author: David Hedberg Date: Mon Jan 23 22:19:43 2023 +0100 Wait for txg sync if the last DRR_FREEOBJECTS might result in a hole If we receive a DRR_FREEOBJECTS as the first entry in an object range, this might end up producing a hole if the freed objects were the only existing objects in the block. If the txg starts syncing before we've processed any following DRR_OBJECT records, this leads to a possible race where the backing arc_buf_t gets its psize set to 0 in the arc_write_ready() callback while still being referenced from a dirty record in the open txg. To prevent this, we insert a txg_wait_synced call if the first record in the range was a DRR_FREEOBJECTS that actually resulted in one or more freed objects. Reviewed-by: Brian Behlendorf Signed-off-by: David Hedberg Sponsored by: Findity AB Closes #11893 Closes #14358 commit 4b878c2555cecd10d057abbdb252b62c0374ae73 Author: Richard Yao Date: Mon Jan 23 16:16:22 2023 -0500 Reject streams that set ->drr_payloadlen to unreasonably large values In the zstream code, Coverity reported: "The argument could be controlled by an attacker, who could invoke the function with arbitrary values (for example, a very high or negative buffer size)." It did not report this in the kernel. This is likely because the userspace code stored this in an int before passing it into the allocator, while the kernel code stored it in a uint32_t. However, this did reveal a potentially real problem. On 32-bit systems and systems with only 4GB of physical memory or less in general, it is possible to pass a large enough value that the system will hang. Even worse, on Linux systems, the kernel memory allocator is not able to support allocations up to the maximum 4GB allocation size that this allows. This had already been limited in userspace to 64MB by `ZFS_SENDRECV_MAX_NVLIST`, but we need a hard limit in the kernel to protect systems. After some discussion, we settle on 256MB as a hard upper limit. Attempting to receive a stream that requires more memory than that will result in E2BIG being returned to user space. Reported-by: Coverity (CID-1529836) Reported-by: Coverity (CID-1529837) Reported-by: Coverity (CID-1529838) Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14285 commit d7b993f2c3c1c1d8187978cea0e7faab2bac229a Author: rob-wing <98866084+rob-wing@users.noreply.github.com> Date: Mon Jan 23 12:14:25 2023 -0900 Configure zed's diagnosis engine with vdev properties Introduce four new vdev properties: checksum_n checksum_t io_n io_t These properties can be used for configuring the thresholds of zed's diagnosis engine and are interpeted as events in T . When this property is set to a non-default value on a top-level vdev, those thresholds will also apply to its leaf vdevs. This behavior can be overridden by explicitly setting the property on the leaf vdev. Note that, these properties do not persist across vdev replacement. For this reason, it is advisable to set the property on the top-level vdev instead of the leaf vdev. The default values for zed's diagnosis engine (10 events, 600 seconds) remains unchanged. Reviewed-by: Tony Hutter Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Reviewed-by: Allan Jude Signed-off-by: Rob Wing Sponsored-by: Seagate Technology LLC Closes #13805 commit 6e9bbe65a5bac5826345fe03c5a1765f38fd9732 Author: Richard Yao Date: Mon Jan 23 16:12:37 2023 -0500 free_blocks(): Fix reports from 2016 PVS Studio FreeBSD report In 2016, the authors of PVS Studio ran it on the FreeBSD kernel, which identified a number of bugs / cleanup opportunities in the FreeBSD ZFS kernel code. A few of them persist to the present day: https://reviews.freebsd.org/D5245 Note that the scan was done against freebsd/freebsd-src@46763fd4ca8a37f836c9bf2333f9d687509278f3. In particular, we have the following in free_blocks(): \sys\cddl\contrib\opensolaris\uts\common\fs\zfs\dnode_sync.c (174): error V547: Expression '__left >= __right' is always true. Unsigned type value is always >= 0. \sys\cddl\contrib\opensolaris\uts\common\fs\zfs\dnode_sync.c (171): error V634: The priority of the '*' operation is higher than that of the '<<' operation. It's possible that parentheses should be used in the expression. \sys\cddl\contrib\opensolaris\uts\common\fs\zfs\dnode_sync.c (175): error V547: Expression '__left >= __right' is always true. Unsigned type value is always >= 0. A couple of assertions accidentally typecast the arguments they check to unsigned in such a way that the result is always true. Also, parentheses are missing around `1<db_blkid * 1<db_blkid << epbs)`. A few of the function local variables probably never should have been 32-bit in the first place, so we make them 64-bit. We also replace the existing assertions with additional assertions to ensure that 64-bit unsigned arithmetic is safe. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14407 commit 70b27d2144bceda26cdb73b5c71e3ca03e69002f Author: Chunwei Chen Date: Fri Jan 20 11:49:56 2023 -0800 Fix reading uninitialized variable in receive_read When zfs_file_read returns error, resid may be uninitialized. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Chunwei Chen Closes #14404 commit 1c08ee0ae2195401b5a565fbdc7cfafe3b0feece Author: Allan Jude Date: Fri Jan 20 14:11:54 2023 -0500 zfs_receive_one: Check for the more likely error first If zfs_receive_one() gets back EINVAL, check for the more likely case, embedded block pointers + encryption and return that error, before falling back to the less likely case, a resumable stream when the kernel has not been upgraded to support resume. Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude Sponsored-by: rsync.net Sponsored-by: Klara Inc. Closes #14379 commit 41731639c99303b7011563053f489e2ac6601cf1 Author: Richard Yao Date: Fri Jan 20 14:10:15 2023 -0500 Cleanup ->dd_space_towrite should be unsigned This is only ever used with unsigned data, so the type itself should be unsigned. Also, PVS Studio's 2016 FreeBSD kernel report correctly identified the following assertion as always being true, so we can drop it: ASSERT3U(dd->dd_space_towrite[i & TXG_MASK], >=, 0); The reason it was always true is because it would do casts to give us unsigned comparisons. This could have been fixed by switching to `ASSERT3S()`, but upon inspection, it turned out that this variable never should have been allowed to be signed in the first place. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14408 commit 5e630eb54015efeb4c765a0b4e68cb70c4454e40 Author: Mark Johnston Date: Mon Jan 16 07:53:16 2023 -0500 Micro-optimize dsl_prop_get_dd() Use the saved property index instead of looking it up once per DSL directory when traversing up towards the root. Reviewed-by: Brian Behlendorf Reviewed-by: Igor Kozhukhov Reviewed-by: Richard Yao Reviewed-by: Allan Jude Reviewed-by: Akash B Signed-off-by: Mark Johnston Sponsored-by: The FreeBSD Foundation Closes #14397 commit 8bb7c14f673a9d1115f1ae4b72e8f2f1e6e86e29 Author: Mark Johnston Date: Sun Jan 15 17:05:19 2023 -0500 Avoid passing an uninitialized index to dsl_prop_known_index Reported-by: KMSAN Reviewed-by: Brian Behlendorf Reviewed-by: Igor Kozhukhov Reviewed-by: Richard Yao Reviewed-by: Allan Jude Reviewed-by: Akash B Signed-off-by: Mark Johnston Sponsored-by: The FreeBSD Foundation Closes #14397 commit b2345c0133520bea878eefb9727444b22a916004 Author: Chunwei Chen Date: Thu Jan 19 16:59:05 2023 -0800 Fix unprotected zfs_znode_dmu_fini In original code, zfs_znode_dmu_fini is called in zfs_rmnode without zfs_znode_hold_enter. It seems to assume it's ok to do so when the znode is unlinked. However this assumption is not correct, as zfs_zget can be called by NFS through zpl_fh_to_dentry as pointed out by Christian in https://github.com/openzfs/zfs/pull/12767, which could result in a use-after-free bug. Reviewed-by: Brian Behlendorf Co-authored-by: Ryan Moeller Signed-off-by: Chunwei Chen Signed-off-by: Ryan Moeller Closes #12767 Closes #14364 commit a8b93b3812e1c0f8109239097ccbc67f88d762ae Author: George Melikov Date: Wed Jan 18 00:12:29 2023 +0300 Man: fix defaults for zfs_dirty_data_max_max It was changed in e99932f7dec6efeb006e225e0bf0901c30345cac, but without docs update. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: George Melikov Closes #14400 commit 079ff7bb4d5c5545b8e3d813d42cd4de8c7b3b4a Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Tue Jan 17 23:17:35 2023 +0500 Use setproctitle to report progress of zfs send This allows parsing of zfs send progress by checking the process title. Doing so requires some changes to the send code in libzfs_sendrecv.c; primarily these changes move some of the accounting around, to allow for the code to be verbose as normal, or set the process title. Unlike BSD, setproctitle() isn't standard in Linux; thus, borrowed it from libbsd with slight modifications. Authored-by: Sean Eric Fagan Co-authored-by: Ryan Moeller Co-authored-by: Ameer Hamza Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Ameer Hamza Closes #14376 commit 03318b51125414f4e72e936f0dff83fb61700b8f Author: Richard Yao Date: Tue Jan 17 12:57:12 2023 -0500 Cleanup of dead code suggested by Clang Static Analyzer (#14380) I recently gained the ability to run Clang's static analyzer on the linux kernel modules via a few hacks. This extended coverage to code that was previously missed since Clang's static analyzer only looked at code that we built in userspace. Running it against the Linux kernel modules built from my local branch produced a total of 72 reports against my local branch. Of those, 50 were reports of logic errors and 22 were reports of dead code. Since we already had cleaned up all of the previous dead code reports, I felt it would be a good next step to clean up these dead code reports. Clang did a further breakdown of the dead code reports into: Dead assignment 15 Dead increment 2 Dead nested assignment 5 The benefit of cleaning these up, especially in the case of dead nested assignment, is that they can expose places where our error handling is incorrect. A number of them were fairly straight forward. However several were not: In vdev_disk_physio_completion(), not only were we not using the return value from the static function vdev_disk_dio_put(), but nothing used it, so I changed it to return void and removed the existing (void) cast in the other area where we call it in addition to no longer storing it to a stack value. In FSE_createDTable(), the function is dead code. Its helper function FSE_freeDTable() is also dead code, as are the CPP definitions in `module/zstd/include/zstd_compat_wrapper.h`. We just delete it all. In zfs_zevent_wait(), we have an optimization opportunity. cv_wait_sig() returns 0 if there are waiting signals and 1 if there are none. The Linux SPL version literally returns `signal_pending(current) ? 0 : 1)` and FreeBSD implements the same semantics, we can just do `!cv_wait_sig()` in place of `signal_pending(current)` to avoid unnecessarily calling it again. zfs_setattr() on FreeBSD version did not have error handling issue because the code was removed entirely from FreeBSD version. The error is from updating the attribute directory's files. After some thought, I decided to propapage errors on it to userspace. In zfs_secpolicy_tmp_snapshot(), we ignore a lack of permission from the first check in favor of checking three other permissions. I assume this is intentional. In zfs_create_fs(), the return value of zap_update() was not checked despite setting an important version number. I see no backward compatibility reason to permit failures, so we add an assertion to catch failures. Interestingly, Linux is still using ASSERT(error == 0) from OpenSolaris while FreeBSD has switched to the improved ASSERT0(error) from illumos, although illumos has yet to adopt it here. ASSERT(error == 0) was used on Linux while ASSERT0(error) was used on FreeBSD since the entire file needs conversion and that should be the subject of another patch. dnode_move()'s issue was caused by us not having implemented POINTER_IS_VALID() on Linux. We have a stub in `include/os/linux/spl/sys/kmem_cache.h` for it, when it really should be in `include/os/linux/spl/sys/kmem.h` to be consistent with Illumos/OpenSolaris. FreeBSD put both `POINTER_IS_VALID()` and `POINTER_INVALIDATE()` in `include/os/freebsd/spl/sys/kmem.h`, so we copy what it did. Whenever a report was in platform-specific code, I checked the FreeBSD version to see if it also applied to FreeBSD, but it was only relevant a few times. Lastly, the patch that enabled Clang's static analyzer to be run on the Linux kernel modules needs more work before it can be put into a PR. I plan to do that in the future as part of the on-going static analysis work that I am doing. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14380 commit ba17e92a1a27e48a87be48828087d083f167c62a Author: Brian Behlendorf Date: Tue Jan 17 09:53:53 2023 -0800 ZTS: Annotate additonal flaky test cases Update several flaky test cases in zts-report.py.in until they can be made entirely reliable. Reviewed-by: George Melikov Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Closes #14392 commit 5bbbd6eb33ec75659f362fd1abe117ea8f2bf0d6 Author: Brian Behlendorf Date: Tue Jan 17 09:52:27 2023 -0800 CI: Reclaim space after package operations Rather than reclaiming space before updating the packages do it afterwards. This avoids issues with apt returning an error due to missing files on the system. This commit includes a revert for 6320b9e6. Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #14387 commit ee219092bc5174b0f6bada2aa98d834a44c4be64 Author: Rob Wing Date: Tue Dec 20 21:52:26 2022 -0900 zpool-set: print error message when pool or vdev is not valid Reviewed-by: Tony Hutter Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Signed-off-by: Rob Wing Sponsored-by: Seagate Technology Submitted-by: Klara, Inc. Closes #14310 commit 86ccc499ea5344c90b9501b8deb3610c50e6ef96 Author: Rob Wing Date: Tue Dec 20 21:33:11 2022 -0900 zpool-set: update usage text Reviewed-by: Tony Hutter Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Signed-off-by: Rob Wing Sponsored-by: Seagate Technology Submitted-by: Klara, Inc. Closes #14310 commit 8ff4466f744cf5cb283c4d53a4f79b4525f785d1 Author: Richard Yao Date: Fri Jan 13 13:58:58 2023 -0500 Linux ppc64le ieee128 compat: Do not redefine __asm on external headers There is an external assembly declaration extension in GNU C that glibc uses when building with ieee128 floating point support on ppc64le. Marking that as volatile makes no sense, so the build breaks. It does not make sense to only mark this as volatile on Linux, since if do not want the compiler reordering things on Linux, we do not want the compiler reordering things on any other platform, so we stop treating Linux specially and just manually inline the CPP macro so that we can eliminate it. This should fix the build on ppc64le. Tested-by: @gyakovlev Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14308 Closes #14384 commit edca24fdf49f8c728109206571691d333973f7a9 Author: Richard Yao Date: Tue Jan 10 16:58:21 2023 -0500 Cleanup: Use NULL when doing NULL pointer comparisons The Linux 5.16.14 kernel's coccicheck caught this. The semantic patch that caught it was: ./scripts/coccinelle/null/badzero.cocci Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit 12f05666d289e965c181b3e888c8f5a32d86d3a2 Author: Richard Yao Date: Tue Jan 10 16:54:23 2023 -0500 Cleanup: Remove unneeded semicolons The Linux 5.16.14 kernel's coccicheck caught this. The semantic patch that caught it was: ./scripts/coccinelle/misc/semicolon.cocci Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit 05f91413af3f89213cafdf0ee3be6c3951a1fce5 Author: Richard Yao Date: Tue Jan 10 16:41:26 2023 -0500 Cleanup: Use MIN() macro The Linux 5.16.14 kernel's coccicheck caught this. The semantic patch that caught it was: ./scripts/coccinelle/misc/minmax.cocci There was a third opportunity to use `MIN()`, but that was in `FSE_minTableLog()` in `module/zstd/lib/compress/fse_compress.c`. Upstream zstd has yet to make this change and I did not want to change header includes just for MIN, or do a one off, so I left it alone. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit 2a153149f9c11c1c02205e51a443ed81a43bb19b Author: Richard Yao Date: Tue Jan 10 16:34:29 2023 -0500 Cleanup: !A || A && B is equivalent to !A || B In zfs_zaccess_dataset_check(), we have the following subexpression: (!IS_DEVVP(ZTOV(zp)) || (IS_DEVVP(ZTOV(zp)) && (v4_mode & WRITE_MASK_ATTRS))) When !IS_DEVVP(ZTOV(zp)) is false, IS_DEVVP(ZTOV(zp)) is true under the law of the excluded middle since we are not doing pseudoboolean alegbra. Therefore doing: (IS_DEVVP(ZTOV(zp)) && (v4_mode & WRITE_MASK_ATTRS)) Is unnecessary and we can just do: (v4_mode & WRITE_MASK_ATTRS) The Linux 5.16.14 kernel's coccicheck caught this. The semantic patch that caught it was: ./scripts/coccinelle/misc/excluded_middle.cocci Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit 3af7262810194fecb58ebfc0b235020321d54d4c Author: Richard Yao Date: Tue Jan 10 15:44:35 2023 -0500 Cleanup: Replace oldstyle struct hack with C99 flexible array members The Linux 5.16.14 kernel's coccicheck caught this. The semantic patch that caught it was: ./scripts/coccinelle/misc/flexible_array.cocci However, unlike the cases where the GNU zero length array extension had been used, coccicheck would not suggest patches for the older style single member arrays. That was good because blindly changing them would break size calculations in most cases. Therefore, this required care to make sure that we did not break size calculations. In the case of `indirect_split_t`, we use `offsetof(indirect_split_t, is_child[is->is_children])` to calculate size. This might be subtly wrong according to an old mailing list thread: https://inbox.sourceware.org/gcc-prs/20021226123454.27019.qmail@sources.redhat.com/T/ That is because the C99 specification should consider the flexible array members to start at the end of a structure, but compilers prefer to put padding at the end. A suggestion was made to allow compilers to allocate padding after the VLA like compilers already did: http://std.dkuug.dk/JTC1/SC22/WG14/www/docs/n983.htm However, upon thinking about it, whether or not we allocate end of structure padding does not matter, so using offsetof() to calculate the size of the structure is fine, so long as we do not mix it with sizeof() on structures with no array members. In the case that we mix them and padding causes offsetof(struct_t, vla_member[0]) to differ from sizeof(struct_t), we would be doing unsafe operations if we underallocate via `offsetof()` and then overcopy via sizeof(). Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit 8351aadced818dcaabd4efbd221ac2261f769a6f Author: Richard Yao Date: Tue Jan 10 15:18:57 2023 -0500 Cleanup: Fix indentation in zfs_dbgmsg_t fdc2d303710416868a05084e984fd8f231e948bd accidentally broke the indentation. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit f5442e65e439b2fe675d7eb690f5581b22abcfd8 Author: Richard Yao Date: Tue Jan 10 15:14:59 2023 -0500 Cleanup: Use C99 flexible array members instead of zero length arrays The Linux 5.16.14 kernel's coccicheck caught this. The semantic patch that caught it was: ./scripts/coccinelle/misc/flexible_array.cocci The Linux kernel's documentation makes a good case for why we should not use these: https://www.kernel.org/doc/html/latest/process/deprecated.html#zero-length-and-one-element-arrays Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit a20cf0b2a8f81cb6ac44a18c0ab505d358908be6 Author: Richard Yao Date: Tue Jan 10 14:06:53 2023 -0500 Cleanup: Use kmem_zalloc() instead of memset() to zero memory The Linux 5.16.14 kernel's coccicheck caught this. The semantic patch that caught it was: ./scripts/coccinelle/api/alloc/zalloc-simple.cocci Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit ee0ec1f0f0976c1cc1dd0a31699624d6bda836cf Author: Richard Yao Date: Tue Jan 10 14:03:46 2023 -0500 Cleanup: Remove unnecessary explicit casts of pointers from allocators The Linux 5.16.14 kernel's coccicheck caught these. The semantic patch that caught them was: ./scripts/coccinelle/api/alloc/alloc_cast.cocci Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14372 commit 1b08839ec3c173f6ad950be22a6cd26f2f852284 Author: Gian-Carlo DeFazio Date: Thu Jan 12 10:14:04 2023 -0800 change how d_alias is replaced by du.d_alias d_alias may need to be converted to du.d_alias depending on the kernel version. d_alias is currently in only one place in the code which changes "hlist_for_each_entry(dentry, &inode->i_dentry, d_alias)" to "hlist_for_each_entry(dentry, &inode->i_dentry, d_u.d_alias)" as neccesary. This effectively results in a double macro expansion for code that uses the zfs headers but already has its own macro for just d_alias (lustre in this case). Remove the conditional code for hlist_for_each_entry and have a macro for "d_alias -> du.d_alias" instead. Reviewed-by: Brian Behlendorf Signed-off-by: Gian-Carlo DeFazio Closes #14377 commit 9081c75ae219d2f27dfd815065242a57dd0e9660 Author: George Amanakis Date: Thu Jan 12 03:00:39 2023 +0100 Activate filesystem features only in syncing context When activating filesystem features after receiving a snapshot, do so only in syncing context. Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #14304 Closes #14252 commit 5ccb8feb0e2ac2f143fc73f90e1a759f8ffbcc0c Author: Brian Behlendorf Date: Wed Jan 11 15:18:51 2023 -0800 CI: remove unused packages/snaps Removing portions of packages/snaps directly with rm can result in unexpected errors when running `apt update`. Free up the additional space by removing (some) packages with the proper tools. This change frees up slightly less space than before, but it is expected to still be sufficient. Reviewed-by: George Melikov Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Closes #14374 commit 2ed7805a16b92da4bc294bae7ea7a81587af2570 Author: rob-wing <98866084+rob-wing@users.noreply.github.com> Date: Wed Jan 11 14:14:35 2023 -0900 zpool: do guid-based comparison in is_vdev_cb() is_vdev_cb() uses string comparison to find a matching vdev and will fallback to comparing the guid via a string. These changes drop the string comparison and compare the guids instead. Reviewed-by: Tony Hutter Reviewed-by: Richard Yao Reviewed-by: Allan Jude Signed-off-by: Rob Wing Co-authored-by: Rob Wing Sponsored-by: Seagate Technology Submitted-by: Klara, Inc. Closes #14311 commit ebfa3b704f93b08dd3a8fafd30401f2ac3193a06 Author: Mateusz Piotrowski Date: Wed Jan 11 18:38:20 2023 +0100 Turn default_bs and default_ibs into ZFS_MODULE_PARAMs The default_bs and default_ibs tunables control the default block size and indirect block size. So far, default_bs and default_ibs were tunable only on FreeBSD, e.g., sysctl vfs.zfs.default_ibs Remove the FreeBSD-specific sysctl code and expose default_bs and default_ibs as tunables on both Linux and FreeBSD using ZFS_MODULE_PARAM. One of the use cases for changing the values of those tunables is to lower the indirect block size, which may improve performance of large directories (as discussed during the OpenZFS Leadership Meeting on 2022-08-16). Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Mateusz Piotrowski Sponsored-by: Wasabi Technology, Inc. Closes #14293 commit 410b3d9366eb415cf899517ce713810c253cd6f1 Author: Tony Hutter Date: Tue Jan 10 15:53:33 2023 -0800 Update META to 6.1 kernel ZFS successfully builds against the 6.1.4 kernel. Reviewed-by: Brian Behlendorf Signed-off-by: Tony Hutter Closes #14371 commit c9084224450f64e28182610a50529622243611f4 Author: Mateusz Piotrowski <0mp@FreeBSD.org> Date: Tue Jan 10 22:41:54 2023 +0100 Add tunable to allow changing micro ZAP's max size This change turns `MZAP_MAX_BLKSZ` into a `ZFS_MODULE_PARAM()` called `zap_micro_max_size`. As a result, we can experiment with different micro ZAP sizes to improve directory size scaling. Reviewed-by: Brian Behlendorf Co-authored-by: Mateusz Piotrowski Co-authored-by: Toomas Soome Signed-off-by: Mateusz Piotrowski Sponsored-by: Wasabi Technology, Inc. Closes #14292 commit a32e25eea5e8b0882a85dfa1e33cb6022eda6a18 Author: Alyssa Ross Date: Tue Jan 10 21:40:31 2023 +0000 etc/systemd/zfs-mount-generator: avoid strndupa The non-standard strndupa function is not implemented by musl libc, and can be dangerous due to its potential to blow the stack. (musl _does_ implement strdupa, used elsewhere in this function.) With a similar amount of code, we can use a heap allocation to construct the pool name, which is musl-friendly and doesn't have potential stack problems. (Why care about musl when systemd only supports glibc? Some distros patch systemd with portability fixes, and it would be nice to be able to use ZFS on those distros.) Reviewed-by: Brian Behlendorf Signed-off-by: Alyssa Ross Closes #14327 commit 048c805364cc69cc0801b4d26aa38ccf600bbd2c Author: Matthew Ahrens Date: Tue Jan 10 13:39:22 2023 -0800 Batch enqueue/dequeue for bqueue The Blocking Queue (bqueue) code is used by zfs send/receive to send messages between the various threads. It uses a shared linked list, which is locked whenever we enqueue or dequeue. For workloads which process many blocks per second, the locking on the shared list can be quite expensive. This commit changes the bqueue logic to have 3 linked lists: 1. An enquing list, which is used only by the (single) enquing thread, and thus needs no locks. 2. A shared list, with an associated lock. 3. A dequing list, which is used only by the (single) dequing thread, and thus needs no locks. The entire enquing list can be moved to the shared list in constant time, and the entire shared list can be moved to the dequing list in constant time. These operations only happen when the `fill_fraction` is reached, or on an explicit flush request. Therefore, the lock only needs to be acquired infrequently. The API already allows for dequing to block until an explicit flush, so callers don't need to be changed. Reviewed-by: Brian Behlendorf Reviewed-by: George Wilson Reviewed-by: Richard Yao Signed-off-by: Matthew Ahrens Closes #14121 commit b3a6736e3fa610b814fa59fd484b5ff3912233b7 Author: Brian Behlendorf Date: Wed Jan 4 16:05:36 2023 -0800 ztest: update ztest_dmu_snapshot_create_destroy() ECHRNG is returned when the channel program encounters a runtime error. For example, this can happen when a snapshot doesn't exist. We handle this error the same way as the existing EEXIST and ENOENT error checks. Additionally, improve the internal debug message to include the error describing why a pool couldn't be opened. Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Closes #14351 commit ea5e7922ac0e463b91a9d653206228874acde4cb Author: Brian Behlendorf Date: Wed Jan 4 16:23:28 2023 -0800 ztest: ztest_dsl_prop_set_uint64() ENOSPC consistency It is possible for ztest_dsl_prop_set_uint64() to fail with ENOSPC and this needs to be handled consistently. Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Closes #14351 commit fca65966bf24aa04930f65d91d8fc719e31ae043 Author: Brian Behlendorf Date: Wed Jan 4 16:04:02 2023 -0800 ztest: reduce `zpool split` frequency There's no need to so aggressively test splitting a pool. Reduce the occurence of this test to once every 10 seconds. Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Closes #14351 commit 08dbaa5211a15cf9a67cb4d1f5cb2a6ace392a75 Author: Brian Behlendorf Date: Wed Jan 4 15:59:50 2023 -0800 ztest: update expectation for sparing a special device Commit c23738c70eb86a7f04f93292caef2ed977047608 modified the expected behavior of attach to prevent hot spares from being used as special vdev replacements. We update ztest's expectations accordingly to prevent it from failing when testing the updated behavior. Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Closes #14351 commit 55f7780219afe69472c8c4678c186a3ce0c2fc2a Author: Matthew Ahrens Date: Mon Jan 9 16:43:45 2023 -0800 ztest fails assertion in zio_write_gang_member_ready() Encrypted blocks can have up to 2 DVA's, as the third DVA is reserved for the salt+IV. However, dmu_write_policy() allows non-encrypted blocks (e.g. DMU_OT_OBJSET) inside encrypted datasets to request and allocate 3 DVA's, since they don't need a salt+IV (they are merely authenicated). However, if such a block becomes a gang block, the gang code incorrectly limits the gang block header to 2 DVA's. This leads to a "NDVAs inversion", where a parent block (the gang block header) has less DVA's than its children (the gang members), causing an assertion failure in zio_write_gang_member_ready(). This commit addresses the problem by only restricting the gang block header to 2 DVA's if the block is actually encrypted (and thus its gang block members can have at most 2 DVA's). Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Matthew Ahrens Closes #14250 Closes #14356 commit 6dccf3eb49f5d4aa336814c8a3599c28c063b5d7 Author: Charles Suh Date: Mon Jan 9 12:49:35 2023 -0800 libzpool: fix ddi_strtoull to update nptr Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Charles Suh Closes #14360 commit 115670de15bc24c16cc2bbae16788810f541be7d Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Tue Jan 10 01:43:03 2023 +0500 zed: add hotplug support for spare vdevs This commit supports for spare vdev hotplug. The spare vdev associated with all the pools will be marked as "Removed" when the drive is physically detached and will become "Available" when the drive is reattached. Currently, the spare vdev status does not change on the drive removal and the same is the case with reattachment. Reviewed-by: Tony Hutter Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Ameer Hamza Closes #14295 commit f9c3ef6f7881783fcf2a2e7e585d806774da856f Author: Alexander Motin Date: Mon Jan 9 13:45:17 2023 -0500 Remove some dead ARC code. (#14340) Every ARC buffer holds a reference on the header. It means headers with buffers are never evictable. When we are evicting a header, there can be no more buffers to free. Just assert that. b_evict_lock seems not protecting anything now. Remove it. Buffers checksum should also be freed with the last uncompressed buffer, so it should not be there also when we are evicting the header. Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. commit 76204c5a556f3b3cd671ced78f073b0a60b09b85 Author: Coleman Kane Date: Mon Dec 26 22:59:13 2022 -0500 linux 6.2 compat: bio->bi_rw was renamed bio->bi_opf The bi_rw member of struct bio was renamed to bi_opf in Linux 6.2. As well, Linux's implementation of bio_set_op_attrs(...) has been removed. The HAVE_BIO_BI_OPF macro already appears to be defined, but the removal of the bio_set_op_attrs(...) implementation makes the build fall back on the locally-defined implementation, which isn't updated for the bio->bi_opf change. This commit adds that update. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: Tony Hutter Signed-off-by: Coleman Kane Closes #14324 Closes #14331 commit 3257519c65278ae83c02068a0092f90e08dcc26f Author: Coleman Kane Date: Mon Dec 26 22:04:34 2022 -0500 linux 6.2 compat: get_acl() got moved to get_inode_acl() in 6.2 Linux 6.2 renamed the get_acl() operation to get_inode_acl() in the inode_operations struct. This should fix Issue #14323. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: Tony Hutter Signed-off-by: Coleman Kane Closes #14323 Closes #14331 commit 539ae40ac2653cb0a6539277ade5c7b3c8e69c2d Author: Antonio Russo Date: Wed Jan 4 21:41:16 2023 -0700 Introduce ZFS_LINUX_REQUIRE_API autoconf macro Currently, if API tests fail, we either ignore the failures, or unconditionally halt the kernel build. This leads to situations where incompatibilities with existing APIs may develop, but not trip the configure compatibility checks. This introduces a new mechanism to require APIs for kernels above a particular version. While not perfect, this at least guarantees mainline kernels do not break existing APIs without at least providing some warning. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Antonio Russo Closes #14343 commit 66ad5d62434ce26caee660762de5352a10aa8db0 Author: Antonio Russo Date: Sat Dec 31 07:51:32 2022 -0700 Linux 6.1 compat: open inside tmpfile() Linux 863f144 modified the .tmpfile interface to pass a struct file, rather than a struct dentry, and expect the tmpfile implementation to open inside of tmpfile(). This patch implements a configuration test that checks for this new API and appropriately sets a HAVE_TMPFILE_DENTRY flag that tracks this old API. Contingent on this flag, the appropriate API is implemented. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Antonio Russo Closes #14301 Closes #14343 commit 70807879ec3d553425fd7889442e3c10bdc72b0a Author: Antonio Russo Date: Fri Jan 6 11:52:08 2023 -0700 ZTS: close in mmapwrite.c mmapwrite is used during the ZTS to identify issues with mmap-ed files. This helper program exercises this pathway by continuously writing to a file. ee6bf97c7 modified the writing threads to terminate after a set amount of total data is written. This change allows standard program execution to reach the end of a writer thread without closing the file descriptor, introducing a resource "leak." This patch appeases resource leak analyses by close()-ing the file at the end of the thread. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Antonio Russo Closes #14353 commit 039f07a79d3fe1fb54b8980702ac9b5d717c1e60 Author: Antonio Russo Date: Thu Jan 5 13:50:30 2023 -0700 ZTS: limit mmapwrite file size mmapwrite spawns several threads, all of which perform writes on a file for the purpose of testing the behavior of mmap(2)-ed files. One thread performs an mmap and a write to the beginning of that region, while the others perform regular writes after lseek(2)-ing the end of the file. Because these regular writes are set in a while (1) loop, they will write an unbounded amount of data to disk. The mmap_write_001_pos test script SIGKILLs them after 30 seconds, but on fast testbeds, this may be enough time to exhaust the available space in the filesystem, leading to spurious test failures. Instead, limit the total file size by checking that the lseek return value is no greater than 250 * 1024*1024 bytes, which is less than the default minimum vdev size defined in includes/default.cfg . Reviewed-by: Brian Behlendorf Signed-off-by: Antonio Russo Closes #14277 Closes #14345 commit 754f6405e52e51540be85d2e921d4105d13e7814 Author: Clemens Lang Date: Thu Jan 5 20:07:43 2023 +0000 contrib: dracut: Do not timeout waiting for pw systemd-ask-password has a default timeout of 90 seconds, which means that dracut will fall back to the rescue shell 4.5 minutes after boot if no password is entered. This is undesirable when combined with, for example, unlocking remotely using dracut-sshd and systemd-tty-ask-password-agent. See also https://github.com/gsauthof/dracut-sshd#timeout and https://bugzilla.redhat.com/show_bug.cgi?id=868421. Reviewed-by: Brian Behlendorf Signed-off-by: Clemens Lang Closes #14341 commit 8eec19d723bef815c149f705a6527ec72f0415c8 Author: Richard Yao Date: Thu Jan 5 14:16:21 2023 -0500 Illumos #15286: do_composition() needs sign awareness Authored by: Dan McDonald Reviewed by: Patrick Mooney Reviewed by: Richard Lowe Approved by: Joshua M. Clulow Ported-by: Richard Yao Illumos-issue: https://www.illumos.org/issues/15286 Illumos-commit: https://github.com/illumos/illumos-gate/commit/f137b22e734e85642da3e56e8b94da3f5f027c73 Porting Notes: The patch in illumos did not have much of a commit message, and did not provide attribution to the reporter, while original patch proposed to OpenZFS did, so I am listing the reporter (myself) and original patch author (also myself) below while including the original commit message with some minor corrections as part of the porting notes: In do_composition(), we have: size = u8_number_of_bytes[*p]; if (size <= 1 || (p + size) > oslast) break; There, we have type promotion from int8_t to size_t, which is unsigned. C will sign extend the value as part of the widening before treating the value as unsigned and the negative values we can counter are error values from U8_ILLEGAL_CHAR and U8_OUT_OF_RANGE_CHAR, which are -1 and -2 respectively. The unsigned versions of these under two's complement are SIZE_MAX and SIZE_MAX-1 respectively. The bounds check is written under the assumption that `size <= 1` does a signed comparison. This is followed by a pointer comparison to see if the string has the correct length, which is fine. A little further down we have: for (i = 0; i < size; i++) tc[i] = *p++; When an error condition is encountered, this will attempt to iterate at least SIZE_MAX-1 times, which will massively overflow the buffer, which is not fine. The kernel will kill the loop as soon as it hits the kernel stack guard on Linux systems built with CONFIG_VMAP_STACK=y, which should be just about all of them. That prevents arbitrary code execution and just about any other bad thing that a black hat attacker might attempt with knowledge of this buffer overflow. Other systems' kernels have mitigations for unbounded in-kernel buffer overflows that will catch this too. Also, the patch in illumos-gate made an effort to fix C style issues that had been fixed in the OpenZFS/ZFSOnLinux repository. Those issues had been mentioned in the email that I originally sent them about this issue. One of the fixes had not been already done, so it is included. Another to collect_a_seq()'s arguments was handled differently in OpenZFS. For the sake of avoiding unnecessary differences, it has been adopted. This has the interesting effect that if you correct the paths in the illumos-gate patch to match the current OpenZFS repository, you can reverse apply it cleanly. Original-patch-by: Richard Yao Reported-by: Richard Yao Co-authored-by: Dan McDonald Closes #14318 Closes #14342 commit 2a7dcc5eb1357962dce1836c98f3824a297fd921 Author: Matthew Ahrens Date: Thu Jan 5 11:04:24 2023 -0800 removal of LegacyVersion broke ax_python_dev.m4 The 22.0 release of the python `packaging` package removed the `LegacyVersion` trait, causing ZFS to no longer compile. This commit replaces the sections of `ax_python_dev.m4` that rely on `LegacyVersion` with updated implementations from the upstream `autoconf-archive`. Reviewed-by: Brian Behlendorf Signed-off-by: Matthew Ahrens Closes #14297 commit 3ffdd8254d94cd987a6a95b50d19e1e04c11253d Author: Mateusz Guzik Date: Thu Jan 5 19:56:40 2023 +0100 FreeBSD: catch up to 1400077 Reviewed-by: Richard Yao Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Signed-off-by: Mateusz Guzik Closes #14328 commit e33d4c72e515b7659dcc7fb14ef03724edb05e41 Author: Martin Rüegg Date: Thu Dec 29 10:59:12 2022 +0100 Fix shebang for helper script of deb-utils Shebang was missing the `!` between `#` and the actual path. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Martin Rüegg Closes #14339 commit 1855dbef67b2b59f237b7f01897528d6e347c522 Author: Martin Rüegg Date: Thu Dec 29 10:56:39 2022 +0100 Add quotation marks around `$PATH` for deb-utils Fix #14338, failing to build deb-utils if existing `$PATH` variable would include a whitespace. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Martin Rüegg Closes #14339 commit e64a4fcc97a0c4502e1e5bc1281edfaf6cc33197 Author: Alexander Motin Date: Thu Jan 5 12:31:55 2023 -0500 Pack zrlock_t by 8 bytes On FreeBSD this reduces this structure size from 64 to 56 bytes. dnode_handle_t respectively reduces from 72 to 64 bytes. It sounds like a waste to need 72 bytes to be able to relocate 808 bytes of dnode_t, which relocation on FreeBSD is not even supported. Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14317 commit 81e9f473b146c849eb8a60adbf42198be1e943f6 Author: Alexander Motin Date: Thu Jan 5 12:29:13 2023 -0500 Update arc_summary and arcstat outputs Recent ARC commits added new statistic counters, such as iohits, uncached state, etc. Represent those. Also some of previously reported numbers were confusing or even made no sense. Cleanup and restructure existing reports. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Issue #14115 Issue #14123 Issue #14243 Closes #14320 commit 67d51285b875d1d650547a299e4d487678d012cc Author: Alexander Motin Date: Thu Jan 5 12:15:31 2023 -0500 Hide b_freeze_* under ZFS_DEBUG This saves 40 bytes per full ARC header, reducing it on FreeBSD from 240 to 200 bytes on production bits. Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Reviewed-by: Matthew Ahrens Signed-off-by: Alexander Motin Closes #14315 commit 393cf4bf59caf56f727e0367de58dd0b0f1c910d Author: Alexander Motin Date: Wed Jan 4 19:29:54 2023 -0500 Implement uncached prefetch Previously the primarycache property was handled only in the dbuf layer. Since the speculative prefetcher is implemented in the ARC, it had to be disabled for uncacheable buffers. This change gives the ARC knowledge about uncacheable buffers via arc_read() and arc_write(). So when remove_reference() drops the last reference on the ARC header, it can either immediately destroy it, or if it is marked as prefetch, put it into a new arc_uncached state. That state is scanned every second, evicting stale buffers that were not demand read. This change also tracks dbufs that were read from the beginning, but not to the end. It is assumed that such buffers may receive further reads, and so they are stored in dbuf cache. If a following reads reaches the end of the buffer, it is immediately evicted. Otherwise it will follow regular dbuf cache eviction. Since the dbuf layer does not know actual file sizes, this logic is not applied to the final buffer of a dnode. Since uncacheable buffers should no longer stay in the ARC for long, this patch also tries to optimize I/O by allocating ARC physical buffers as linear to allow buffer sharing. Reviewed-by: Brian Behlendorf Reviewed-by: George Wilson Reviewed-by: Ryan Moeller Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14243 commit a09705911d6fe95499511e566c41b37c062d167b Author: Alexander Motin Date: Thu Dec 22 15:10:24 2022 -0500 arc_read()/arc_access() refactoring and cleanup ARC code was many times significantly modified over the years, that created significant amount of tangled and potentially broken code. This should make arc_access()/arc_read() code some more readable. - Decouple prefetch status tracking from b_refcnt. It made sense originally, but became highly cryptic over the years. Move all the logic into arc_access(). While there, clean up and comment state transitions in arc_access(). Some transitions were weird IMO. - Unify arc_access() calls to arc_read() instead of sometimes calling it from arc_read_done(). To avoid extra state changes and checks add one more b_refcnt for ARC_FLAG_IO_IN_PROGRESS. - Reimplement ARC_FLAG_WAIT in case of ARC_FLAG_IO_IN_PROGRESS with the same callback mechanism to not falsely account them as hits. Count those as "iohits", an intermediate between "hits" and "misses". While there, call read callbacks in original request order, that should be good for fairness and random speculations/allocations/aggregations. - Introduce additional statistic counters for prefetch, accounting predictive vs prescient and hits vs iohits vs misses. - Remove hash_lock argument from functions not needing it. - Remove ARC_FLAG_PREDICTIVE_PREFETCH, since it should be opposite to ARC_FLAG_PRESCIENT_PREFETCH if ARC_FLAG_PREFETCH is set. We may wish to add ARC_FLAG_PRESCIENT_PREFETCH to few more places. - Fix few false positive tests found in the process. Reviewed-by: George Wilson Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: Ryan Moeller Signed-off-by: Alexander Motin Sponsored by: iXsystems, Inc. Closes #14123 commit bd3ec1aa45ca67a675c7096dbd14d4a368fce216 Author: Ryan Moeller Date: Thu Dec 22 14:50:09 2022 -0500 FreeBSD: Fix potential boot panic with bad label vdev_geom_read_pool_label() can leave NULL in configs. Check for it and skip consistently when generating rootconf. Reviewed-by: Alexander Motin Signed-off-by: Ryan Moeller Closes #14291 commit 6a17af2085ea0fe26382f43348c885303c4cef0e Author: Matthew Ahrens Date: Thu Dec 22 11:48:49 2022 -0800 deadlock between spa_errlog_lock and dp_config_rwlock There is a lock order inversion deadlock between `spa_errlog_lock` and `dp_config_rwlock`: A thread in `spa_delete_dataset_errlog()` is running from a sync task. It is holding the `dp_config_rwlock` for writer (see `dsl_sync_task_sync()`), and waiting for the `spa_errlog_lock`. A thread in `dsl_pool_config_enter()` is holding the `spa_errlog_lock` (see `spa_get_errlog_size()`) and waiting for the `dp_config_rwlock` (as reader). Note that this was introduced by #12812. This commit address this by defining the lock ordering to be dp_config_rwlock first, then spa_errlog_lock / spa_errlist_lock. spa_get_errlog() and spa_get_errlog_size() can acquire the locks in this order, and then process_error_block() and get_head_and_birth_txg() can verify that the dp_config_rwlock is already held. Additionally, a buffer overrun in `spa_get_errlog()` is corrected. Many code paths didn't check if `*count` got to zero, instead continuing to overwrite past the beginning of the userspace buffer at `uaddr`. Tested by having some errors in the pool (via `zinject -t data /path/to/file`), one thread running `zpool iostat 0.001`, and another thread runs `zfs destroy` (in a loop, although it hits the first time). This reproduces the problem easily without the fix, and works with the fix. Reviewed-by: Mark Maybee Reviewed-by: George Amanakis Reviewed-by: George Wilson Reviewed-by: Brian Behlendorf Signed-off-by: Matthew Ahrens Closes #14239 Closes #14289 commit bfee270af1ae62170df3454d848ab7329b1f5238 Author: Brian Behlendorf Date: Thu Dec 22 11:34:28 2022 -0800 Documentation corrections - Update the link to the OpenZFS Code of Conduct. - Remove extra "the" from contrib/initramfs/scripts/zfs Reviewed-by: George Melikov Signed-off-by: Brian Behlendorf Closes #14298 Closes #14307 commit f05aa26c4fd59a8e75930f64a98a8526e260513d Author: Brian Behlendorf Date: Wed Dec 21 09:17:00 2022 -0800 Revert "zdb: zdb_ddt_leak_init() reads uninitialized memory..." This reverts commit d30db519af44b905fc52b8c8ba34f6378aa03470. With this change applied zloop.sh fails reliably with the following ASSERT. zio_wait(zio_claim(NULL, zcb->zcb_spa, refcnt ? 0 : spa_min_claim_txg( zcb->zcb_spa), bp, NULL, NULL, ZIO_FLAG_CANFAIL)) == 0 (0x2 == 0x0) ASSERT at cmd/zdb/zdb.c:5452:zdb_count_block() Reviewed-by: George Melikov Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Closes #14306 commit 932949ab91143ded45e3ec061f0a317ae026f9e6 Author: George Melikov Date: Tue Dec 20 03:08:15 2022 +0300 systemd: set restart=always for zfs-zed.service Reviewed-by: Richard Laager Reviewed-by: Brian Behlendorf Signed-off-by: George Melikov Co-authored-by: Attila Fülöp Closes #14294 commit 72cb93cdde1e9e0a3dc551f273b4af8f2d1760fd Author: Jorgen Lundman Date: Fri Mar 3 11:47:04 2023 +0900 Code review fixes commit daba516eb62753a0c51408d2130ec5af6e086342 Author: datacore-rm Date: Fri Mar 3 11:22:20 2023 +0900 During zpool import, do not search zpool labels in volumes commit caee21a1d9743352df8fd34a5ff231c1a30ae601 Author: Andrew Innes Date: Sun Feb 12 12:24:16 2023 +0800 Add instructions for using the script Signed-off-by: Andrew Innes commit 0e71e3bb5d4e743f5065736cfc3ada5bd70f7008 Author: Andrew Innes Date: Sun Feb 12 11:04:21 2023 +0800 Script for extracting crash logs Signed-off-by: Andrew Innes commit 8d8f2891b96f565090e7dd729f98f734c10550a1 Author: Andrew Innes Date: Sun Feb 12 11:07:51 2023 +0800 Add script to installer Signed-off-by: Andrew Innes commit b08fe110caf752118d92489104be8711b37c4999 Author: Jorgen Lundman Date: Wed Feb 22 16:11:55 2023 +0900 Take out call to IoDetachDevice() It can panic, when otherwise unmount succeeds without it. commit b8520cb6338e73814f81846b57362e0391eee334 Author: Jorgen Lundman Date: Wed Feb 22 16:10:55 2023 +0900 Add skip unlinked drain tunable commit 8891ad103ce4b3cd453a25424829df9f0d5ba8b4 Author: Jorgen Lundman Date: Wed Feb 22 16:08:38 2023 +0900 Disable auto-remount on mountpoint property Changing the mountpoint property would automatically trigger an unmount, then mount of the datasets. Currently unmount is a bit dicey on Windows commit eed7d32827dcfd277191f654cd339235a043d2e4 Author: Jorgen Lundman Date: Wed Feb 22 16:07:35 2023 +0900 Reopen of vdev_file would panic Attempting to spa_strfree() of a NULL name. When we skip the open, we do not have the name assigned. commit 0d8d86466188539fdeb65578e74a7ba691b722cd Author: Jorgen Lundman Date: Fri Feb 10 11:53:14 2023 +0900 Move IoDetachDevice inside the sanity check commit dbd33f6a7286a305d55df2fdf9ef5236b644cb8e Author: Jorgen Lundman Date: Fri Feb 10 11:46:00 2023 +0900 Replace mutex_destroy guard with spinlock Using atomic counter/busy-wait was a bit of a hack, so model on Linux ZOL mutex using a spinlock. Unfortunately, all the Windows fundamentals like spinlock() keep futzing with the IRQ level, so it is unknown if this will have some unintended consequence. commit 137a66199b1ebef61e53969e65f8d217a121d633 Author: Jorgen Lundman Date: Mon Jan 30 14:42:53 2023 +0900 Return vfs_mount to stage1 It is needed for all the probing Windows does to issue the stage2 mount request. Without zfsvfs it would just reply with errors, and mount would not happen. Instead, add vfs_changeowner to move vnodes over from dcb to vcb. commit 7c6d4b0c45030f2f31a9d5b74bb0a0d4c9b05fec Author: Jorgen Lundman Date: Tue Jan 24 16:52:39 2023 +0900 Attempt to run scrub with large stacks commit 1833fc84651c06dcbbe351b5aa20f5078be75d65 Author: Jorgen Lundman Date: Fri Jan 27 11:34:10 2023 +0900 Cleanup unmount to release driverletter It is not yet working 10% but we are closer. You can export/import a pool and get the same driverletter back. commit cae83dd3cea7703ba38b8023040b5775f7aeb163 Author: Jorgen Lundman Date: Fri Jan 20 12:16:00 2023 +0900 Add ioctl to fetch mountpoint for snapshots The mounted directories is still not correct, they do not get registered with moutmgr, so userland can not look up where they are mounted. Until this can be understood and fixed, there is now a new ioctl to call to retrieve the mountpoint. userland calls it in getmntany() to gather all filesystems. this should let "export -a" to work properly, as it must unmount snapshots before the dataset (as they are nested) which is done by sorting the mountpoints, longest first. There are still weird things we need to solve, in addition the includes headers are just weird. commit 877cc332afaa9c5dad7fc6bd713fb8eaa2226265 Author: Jorgen Lundman Date: Fri Jan 13 12:35:15 2023 +0900 zfs_vfs_mount() was called with dcb So the mount process creates the dcb, calls mount on it then waits for device matching, when matching creates vcb. This means rootvp belongs to dcb, and all over vnodes vcb. Since vcb and dcb are "practically" the same, it have appeared to make no difference, but it is unattractive to mix the two types, as well as, replying a different device name for rootvp only. Now delay calling zfs_vfs_mount() until the match event. It means error handling is more complex, and not complete yet. commit 64d1db430f6554a4dad2d4c9eefdf89db63594bd Author: Jorgen Lundman Date: Sun Dec 18 14:15:55 2022 +0900 Windows: allow mounting of snapshots Break readdir emit function up into smaller, so that zfsctl readdir can also call the same emit. If a dataset is mount on E: E:/.zfs/snapshots/helloworld/root ^-real ^-fake ^-real ^-fake ^-fake For a mounted dataset (here, "E:") the root is a real "zp", but ".zfs", "snapshots" and any snapshot names like "helloworld" are fake "zp" nodes - exist only in memory, not on disk. We redirect requests like readdir, and set-reparse-point (mount) to zfsctl instead, and tag the memory nodes only. Kernel work to handle mounting and unmounting of snapshots, and initial support for readonly (which doesn't exist as a mount option) by putting checks for rdonly in many places. * NOTE * Currently, export -a will do the wrong thing, and not unmount mounted snapshots first, so it will *hang* due to this. To fix: Mounting something inside a dataset, should hold an additional rootvp count, so the filesystem is "busy". and userland libzfs needs to know about snapshots, and include them in the sorted filesystems to unmount. This most likely does not work as getmntany() does not return the mountpoint "path" for snapshots, just that they exist. commit 841d9162d15027e80bdb361e279baa86b54365f5 Author: Jorgen Lundman Date: Wed May 20 10:04:11 2020 +0900 Upstream: Allow cmd/zfs mount unmount of snapshots "zfs mount dataset@snapshot" as mounting of snapshot has to be done manually from userland in macOS. There simply is no way to handle it within the kernel. In a future PR we will add events to `zed` to run snapshot scripts that issue `zfs mount snapshot`, so that kernel can "trigger" mounting events, and simulate ".zfs" on macOS. In addition, as it is the only other change in this source file; Add zfs_rollback_os() call to the rollback logic, so platforms can do specific requirements. In our case, we need to kick Finder into refreashing after rollback, or it will disable the old content. Signed-off-by: Jorgen Lundman commit 927d330067e9d9e153d79d7c494fa2b5eda09c92 Author: Jorgen Lundman Date: Fri Jan 27 11:50:11 2023 +0900 Minor cstyle fixes commit fd9924dfce3bd2dafd82f3114366b75c97c92c42 Author: Jorgen Lundman Date: Wed Jan 11 19:19:27 2023 +0900 Tighten ptr use in case sensitive fix vol BSOD as well commit c01313d7f79d01284a1df5f0dcb6a66a7ecaa20c Author: Charles Suh Date: Sun Jan 1 02:27:42 2023 -0800 Windows: fix incorrect offset and length when using a full disk with zero partitions Signed-off-by: Charles Suh commit f13f233d2d92d03100b17b8d85e4b3047c38b5fd Author: Jorgen Lundman Date: Mon Jan 2 14:01:39 2023 +0900 Accidentally marked a C function as ASMABI causing hilarious crashes commit a2e1896ddfacbd7d33fafa4dadb746ad5d76f41d Author: Andrew Innes Date: Mon Jan 2 07:35:14 2023 +0800 Test uninstall zfs Signed-off-by: Andrew Innes commit 3f43fd7c2e1cc40c3c48ce093bb450b112baabe0 Author: Ethan Coe-Renner Date: Mon Dec 12 15:30:51 2022 -0800 Add color output to zfs diff. This adds support to color zfs diff (in the style of git diff) conditional on the ZFS_COLOR environment variable. Signed-off-by: Ethan Coe-Renner commit 3aa9a9bb64c021a0ed17b94aee37641a3d8fc70f Author: Doug Rabson Date: Wed Dec 14 01:35:07 2022 +0000 FreeBSD: Remove stray debug printf Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Reviewed-by: Richard Yao Signed-off-by: Doug Rabson Closes #14286 Closes #14287 commit d11a65032112b6aacb92ccae8521439ab7c9d59c Author: Andrew Innes Date: Sun Dec 18 07:47:06 2022 +0800 Windows: Add native-deb* targets to build native Debian packages Signed-off-by: Andrew Innes commit aaad0d643542a483fedcc5d7d19896727dd020a4 Author: Umer Saleem Date: Wed Dec 14 06:33:05 2022 +0500 Add native-deb* targets to build native Debian packages In continuation of previous #13451, this commits adds native-deb* targets for make to build native debian packages. Github workflows are updated to build and test native Debian packages. Native packages only build with pre-configured paths (see the dh_auto_configure section in contrib/debian/rules.in). While building native packages, paths should not be configured. Initial config flags e.g. '--enable-debug' are replaced in contrib/debian/rules.in. Additional packages on top of existing zfs packages required to build native packages include debhelper-compat, dh-python, dkms, po-debconf, python3-all-dev, python3-sphinx. Reviewed-by: George Melikov Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Umer Saleem Closes #14265 commit 77e9174095d7f75f883b882ac0431065f928bdf4 Author: Richard Yao Date: Tue Dec 13 20:31:47 2022 -0500 Zero end of embedded block buffer in dump_write_embedded() This fixes a kernel stack leak. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Tested-by: Nicholas Sherlock Signed-off-by: Richard Yao Closes #13778 Closes #14255 commit 9ec174016f2545c125dcbb0bdd0fad5913af6aad Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Wed Dec 14 06:30:46 2022 +0500 Allow receiver to override encryption properties in case of replication Currently, the receiver fails to override the encryption property for the plain replicated dataset with the error: "cannot receive incremental stream: encryption property 'encryption' cannot be set for incremental streams.". The problem is resolved by allowing the receiver to override the encryption property for plain replicated send. Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Ameer Hamza Closes #14253 Closes #13533 commit 9b67b5f474c51d09c956ffa9fa8e2d395135e62e Author: Richard Yao Date: Tue Dec 13 20:29:21 2022 -0500 Cache dbuf_hash() calculation We currently compute a 64-bit hash three times, which consumes 0.8% CPU time on ARC eviction heavy workloads. Caching the 64-bit value in the dbuf allows us to avoid that overhead. Sponsored-By: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf Reviewed-by: Matthew Ahrens Signed-off-by: Richard Yao Closes #14251 commit 153ca0d7733a67b4af376ad593a8f36553ff7e10 Author: Allan Jude Date: Tue Dec 13 20:27:54 2022 -0500 zfs list: Allow more fields in ZFS_ITER_SIMPLE mode If the fields to be listed and sorted by are constrained to those populated by dsl_dataset_fast_stat(), then zfs list is much faster, as it does not need to open each objset and reads its properties. A previous optimization by Pawel Dawidek (0cee24064a79f9c01fc4521543c37acea538405f) took advantage of this to make listing snapshot names sorted only by name much faster. However, it was limited to `-o name -s name`, this work extends this optimization to work with: - name - guid - createtxg - numclones - inconsistent - redacted - origin and could be further extended to any other properties supported by dsl_dataset_fast_stat() or similar, that do not require extra locking or reading from disk. This was committed before (9a9e2e343dfa2af28bf7910de77ae73aa006de62), but was reverted due to a regression when used with an older kernel. If the kernel does not populate zc->zc_objset_stats, we now fallback to getting the properties via the slower interface, to avoid problems with newer userland and older kernels. Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude Closes #14110 commit 956085fe04f93b538afe6d6b4341a511b65a7b2d Author: Marcel Menzel Date: Wed Dec 14 02:26:10 2022 +0100 Change ZEVENT_POOL_GUID to ZEVENT_POOL to display pool names Outgoing mails for ZFS pool events include the pool GUID, but not the actual pool name. Let's change this for better readability, as it is already done in the mails for finished pool resilvers. Reviewed-by: Brian Behlendorf Reviewed-by: Tony Hutter Reviewed-by: George Melikov Reviewed-by Richard Yao Signed-off-by: Marcel Menzel Closes #14272 commit 31adabc66af7a7bab7c18c9782fbb348cb3caef9 Author: Richard Yao Date: Mon Dec 12 13:40:05 2022 -0500 Address theoretical uninitialized variable usage in zstream Coverity has long complained about the checksum being uninitialized if an END record is processed before its BEGIN record. This should not happen, but there was no code to check for it. I had left this unfixed since it was a low priority issue, but then 9f4ede63d23be4f43ba8dd0ca42c6a773a8eaa8d added another instance of this. I am making an effort to "hold the line" to keep new coverity defect reports from going unaddressed, so I find myself forced to fix this much earlier than I had originally planned to address it. The solution is to maintain a counter and a flag. Then use VERIFY statements to verify the following runtime constraints: * Every record either has a corresponding BEGIN record, is a BEGIN record or is the end of stream END record for replication streams. * BEGIN records cannot be nested. i.e. There must be an END record before another BEGIN record may be seen. Failure to meet these constraints will cause the program to exit. This is sufficient to ensure that the checksum is never accessed when uninitialized. Reported-by: Coverity (CID 1524578) Reported-by: Coverity (CID 1524633) Reported-by: Coverity (CID 1527295) Reviewed-by: Brian Behlendorf Reviewed-by: Damian Szuberski Signed-off-by: Richard Yao Closes #14176 commit c5f9930a658485b8207f1c387d016b00674ecc18 Author: Ryan Moeller Date: Mon Dec 12 13:23:06 2022 -0500 initramfs: Fix legacy mountpoint rootfs Legacy mountpoint datasets should not pass `-o zfsutil` to `mount.zfs`. Fix the logic in `mount_fs()` to not forget we have a legacy mountpoint when checking for an `org.zol:mountpoint` userprop. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #14274 commit c4c2b38ddb9496fb2c1c14adc1dee3632ee60110 Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Mon Dec 12 23:21:37 2022 +0500 Skip permission checks for extended attributes zfs_zaccess_trivial() calls the generic_permission() to read xattr attributes. This causes deadlock if called from zpl_xattr_set_dir() context as xattr and the dent locks are already held in this scenario. This commit skips the permissions checks for extended attributes since the Linux VFS stack already checks it before passing us the control. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Reviewed-by: Youzhong Yang Signed-off-by: Ameer Hamza Closes #14220 commit 3c0fddff2eb53a09dde42c077a99a3efbf6d6d51 Author: Allan Jude Date: Fri Dec 9 14:04:29 2022 -0500 Restrict visibility of per-dataset kstats inside FreeBSD jails When inside a jail, visibility on datasets not "jailed" to the jail is restricted. However, it was possible to enumerate all datasets in the pool by looking at the kstats sysctl MIB. Only the kstats corresponding to datasets that the user has visibility on are accessible now. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Allan Jude Closes #14254 commit f759c4872917432f1d817a38467424c563fba425 Author: Richard Yao Date: Fri Dec 9 13:51:23 2022 -0500 Linux PPC: Fix build failures on kernels built without CONFIG_SPE We do a simple ifdef to avoid calling enable_kernel_spe()/ disable_kernel_spe() on PowerPC. Reported-by: Rich Ercolani Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Tested-by: Georgy Yakovlev Signed-off-by: Richard Yao Closes #14233 Closes #14244 commit bc35b108bea97eb16b615c23ad62edf8fa14f355 Author: Serapheim Dimitropoulos Date: Fri Dec 9 10:48:33 2022 -0800 Bypass metaslab throttle for removal allocations Context: We recently had a scenario where a customer with 2x10TB disks at 95+% fragmentation and capacity, wanted to migrate their disks to a 2x20TB setup. So they added the 2 new disks and submitted the removal of the first 10TB disk. The removal took a lot more than expected (order of more than a week to 2 weeks vs a couple of days) and once it was done it generated a huge indirect mappign table in RAM (~16GB vs expected ~1GB). Root-Cause: The removal code calls `metaslab_alloc_dva()` to allocate a new block for each evacuating block in the removing device and it tries to batch them into 16MB segments. If it can't find such a segment it tries for 8MBs, 4MBs, all the way down to 512 bytes. In our scenario what would happen is that `metaslab_alloc_dva()` from the removal thread pick the new devices initially but wouldn't allocate from them because of throttling in their metaslab allocation queue's depth (see `metaslab_group_allocatable()`) as these devices are new and favored for most types of allocations because of their free space. So then the removal thread would look at the old fragmented disk for allocations and wouldn't find any contiguous space and finally retry with a smaller allocation size until it would to the low KB range. This caused a lot of small mappings to be generated blowing up the size of the indirect table. It also wasted a lot of CPU while the removal was active making everything slow. This patch: Make all allocations coming from the device removal thread bypass the throttle checks. These allocations are not even counted in the metaslab allocation queues anyway so why check them? Side-Fix: Allocations with METASLAB_DONT_THROTTLE in their flags would not be accounted at the throttle queues but they'd still abide by the throttling rules which seems wrong. This patch fixes this by checking for that flag in `metaslab_group_allocatable()`. I did a quick check to see where else this flag is used and it doesn't seem like this change would cause issues. Reviewed-by: Matthew Ahrens Reviewed-by: Mark Maybee Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Signed-off-by: Serapheim Dimitropoulos Closes #14159 commit bd099a15e52a697060550cbad157961d30dd9959 Author: Richard Yao Date: Sun Dec 4 17:03:14 2022 -0500 Do not pass -1 to strerror() from zfs_send_cb_impl() `zfs_send_cb_impl()` calls `dump_filesystems()`, which calls `dump_filesystem()`, which will return `-1` as an error when `zfs_open()` returns `NULL`. This will be passed to `zfs_standard_error()`, which passes it to `zfs_standard_error_fmt()`, which passes it to `strerror()`. To fix this, we modify zfs_open() to set `errno` whenever it returns NULL. Most of the cases already have `errno` set (since they pass it to `zfs_standard_error_fmt()`, which makes this easy. Then we modify `dump_filesystem()` to pass `errno` instead of `-1`. Reported-by: Coverity (CID-1524598) Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14264 commit 07dd0159c8047a42b7df6d9a23996ea1ced2a60f Author: Richard Yao Date: Sun Dec 4 16:31:28 2022 -0500 Fix dereference after null check in enqueue_range If the bp is NULL, we have a hole. However, when we build with assertions, we will dereference bp when `blkid == DMU_SPILL_BLKID`. When this happens on a hole, we will have a NULL pointer dereference. Reported-by: Coverity (CID-1524670) Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14264 commit 71cc92b416d5369c7f512866e2592bfd23013bb2 Author: Richard Yao Date: Sun Dec 4 15:41:24 2022 -0500 zdb: Handle theoretical buffer overflow when printing float CodeQL pointed out that for extreme floating point values, `sprintf()` will overwrite a 32 character buffer. It cited 1e304 as an example, which causes `sprintf()` to print 308 characters. In practice, the numbers should never exceed 100, so this should not happen. To silence the warning and also handle unexpected situations, we change the code to use `snprintf()`. This was missed during my audit of our use of `sprintf()`, since I did not think to consider extreme floating point representations. It also really should not happen, so this change is purely defensive programming. This was found by CodeQL's cpp/overrunning-write-with-float check. Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14264 commit ac58cf8df8609afdaefeec40c827cbd9ee4d41b3 Author: Richard Yao Date: Sat Dec 3 15:09:48 2022 -0500 zdb: zdb_ddt_leak_init() reads uninitialized memory when birth == 0 This was written by Jeff Bonick and was committed to OpenSolaris on November 1, 2009. It appears that Jeff meant to continue the outer loop iteration when `ddp->ddp_phys_birth == 0`, but put his check inside the inner loop. This causes a pointer to uninitialized memory to be passed to ddt_lookup() inside a VERIFY() statement whenever that condition is true. Reported-by: Coverity (CID 1524462) Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14264 commit 7790e4612f3a1b4087c66146a716609429b76f47 Author: Richard Yao Date: Sat Dec 3 18:08:55 2022 -0500 ztest: comparisons against errno should not assign to it 888914486e26d81c8dbfd7a9d8091c05f9fc98ba introduced this regression. I used cscope to verify that there are no other instances of this in the codebase. This is the one of the few bugs that are extremely easy to identify using cscope. Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14264 commit e6b2f1c709dce63fc21d10927479a3eb0918669c Author: Richard Yao Date: Sat Dec 3 21:43:33 2022 -0500 Fix potential buffer overflow in zpool command The ZPOOL_SCRIPTS_PATH environment variable can be passed here. This allows for arbitrarily long strings to be passed to sprintf(), which can overflow the buffer. I missed this in my earlier audit of the codebase. CodeQL's cpp/unbounded-write check caught this. Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14264 commit b67e6132ed6961613df2684ba95a3a13ad0b398e Author: Richard Yao Date: Sun Dec 4 17:58:46 2022 -0500 zdb: Fix big parameter passed by value This is not in performance critical code, but static analyzers will complain about it, so lets switch to pass by pointer here. Reported-by: Coverity (CID-1524384) Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14263 commit d6bbb07550f7fc3976f2ce504a04174546f77c24 Author: Richard Yao Date: Sun Dec 4 16:46:44 2022 -0500 Linux: Cleanup unnecessary NULL check in __vdev_disk_physio() zio is never NULL when given to the vdev. Coverity complained saying: "Either the check against null is unnecessary, or there may be a null pointer dereference." Reported-by: Coverity (CID-1466174) Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14263 commit 48228925a078f85f4516cceb81e548429f069da8 Author: Richard Yao Date: Sat Dec 3 22:09:58 2022 -0500 Remove duplicate statically allocated variable dsl_dataset_snapshot_sync_impl() declares `static zil_header_t zero_zil __maybe_unused;`, but this is also declared globally. This wastes memory. CodeQL's cpp/local-variable-hides-global-variable check caught this. Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14263 commit 178f0267c6c39e81ae9ea3caaee640cb74f25824 Author: Richard Yao Date: Sat Dec 3 21:09:42 2022 -0500 Cleanup: zhack should not declare function prototypes in main() Instead, it should include the proper header. CodeQL caught this. Reviewed-by: Damian Szuberski Reviewed-by: Alexander Motin Reviewed-by: Ryan Moeller Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14263 commit f1827ad1535847fbab0411a342cb306a793ebb3e Author: Brian Behlendorf Date: Wed Dec 7 17:26:33 2022 -0800 ZTS: Add missing tests to Makefile.am The send-c_zstream_recompress.ksh test case was being skipped because it was not added to the Makefile.am, and was thus left out of the package. As for the renameat2 tests these were being skipped because when the patch was rebased it was not updated to use the new Makefile layout for the tests directory. Correct this. Add missing pre/post sections to sanity.run so the pyzfs tests will run. Reviewed-by: George Melikov Reviewed-by: Damian Szuberski Signed-off-by: Brian Behlendorf Closes #14266 commit acea77f02bf5fe25e7be61095894b4c1865599a2 Author: Richard Yao Date: Mon Dec 5 13:16:50 2022 -0500 FreeBSD: zfs_register_callbacks() must implement error check correctly I read the following article and noticed a couple of ZFS bugs mentioned: https://pvs-studio.com/en/blog/posts/cpp/0377/ I decided to search for them in the modern OpenZFS codebase and then found one that matched the description of the first one: V593 Consider reviewing the expression of the 'A = B != C' kind. The expression is calculated as following: 'A = (B != C)'. zfs_vfsops.c 498 The consequence of this is that the error value is replaced with `1` when there is an error. When there is no error, 0 is correctly passed. This is a very minor issue that is unlikely to cause any real problems. The incorrect error code would either be returned to the mount command on a failure or any of `zfs receive`, `zfs recv`, `zfs rollback` or `zfs upgrade`. The second one has already been fixed. Reviewed-by: Alexander Motin Reviewed-by: Damian Szuberski Signed-off-by: Richard Yao Closes #14261 commit 0230754c7e2b089a0a72716437d9079f65103058 Author: George Wilson Date: Fri Dec 2 19:46:29 2022 -0600 zio can deadlock during device removal When doing a device removal on a pool with gang blocks, the zio pipeline can deadlock when trying to free blocks from a device which is being removed with a stack similar to this: 0xffff8ab9a13a1740 UNINTERRUPTIBLE 4 __schedule+0x2e5 __schedule+0x2e5 schedule+0x33 schedule_preempt_disabled+0xe __mutex_lock.isra.12+0x2a7 __mutex_lock.isra.12+0x2a7 __mutex_lock_slowpath+0x13 mutex_lock+0x2c free_from_removing_vdev+0x61 metaslab_free_impl+0xd6 metaslab_free_dva+0x5e metaslab_free+0x196 zio_free_sync+0xe4 zio_free_gang+0x38 zio_gang_tree_issue+0x42 zio_gang_tree_issue+0xa2 zio_gang_issue+0x6d zio_execute+0x94 zio_execute+0x94 taskq_thread+0x23b kthread+0x120 ret_from_fork+0x1f Since there are gang blocks we have to read the gang members as part of the free. This can be seen with a zio dependency tree that looks like this: sdb> echo 0xffff900c24f8a700 | zio -rc | zio ADDRESS TYPE STAGE WAITER 0xffff900c24f8a700 NULL CHECKSUM_VERIFY 0xffff900ddfd31740 0xffff900c24f8c920 FREE GANG_ASSEMBLE - 0xffff900d93d435a0 READ DONE In the illustration above we are processing frees but because of gang block we have to read the constituents blocks. Once we finish the READ in the zio pipeline we will execute the parent. In this case the parent is a FREE but the zio taskq is a READ and we continue to process the pipeline leading to the stack above. In the stack above, we are blocked waiting for the svr_lock so as a result a READ interrupt taskq thread is now consumed. Eventually, all of the READ taskq threads end up blocked and we're unable to complete any read requests. In zio_notify_parent there is an optimization to continue to use the taskq thread to exectue the parent's pipeline. To resolve the deadlock above, we only allow this optimization if the parent's zio type matches the child which just completed. Reviewed-by: Brian Behlendorf Reviewed-by: Matthew Ahrens Signed-off-by: George Wilson External-issue: DLPX-80130 Closes #14236 commit 34fc9919e3fe7fe144c1e7624b2f201f9f880e52 Author: George Wilson Date: Fri Dec 2 19:45:33 2022 -0600 nopwrites on dmu_sync-ed blocks can result in a panic After a device has been removed, any nopwrites for blocks on that indirect vdev should be ignored and a new block should be allocated. The original code attempted to handle this but used the wrong block pointer when checking for indirect vdevs and failed to check all DVAs. This change corrects both of these issues and modifies the test case to ensure that it properly tests nopwrites with device removal. Reviewed-by: Prakash Surya Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: George Wilson Closes #14235 commit 662b19243cd828e1f515db455a3ea95c2bbcd8b8 Author: Rob Wing Date: Mon Nov 14 07:57:53 2022 +0000 ZTS: test reported checksum errors for ZED Test checksum error reporting to ZED via the call paths vdev_raidz_io_done_unrecoverable() and zio_checksum_verify(). Sponsored-by: Seagate Technology LLC Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Rob Wing Closes #14190 commit 8e456223a3bdcb6ea2ffb31049428b3d2ac1d7ba Author: Rob Wing Date: Mon Nov 14 07:40:38 2022 +0000 Bump checksum error counter before reporting to ZED The checksum error counter is incremented after reporting to ZED. This leads ZED to receiving a checksum error report with 0 checksum errors. To avoid this, bump the checksum error counter before reporting to ZED. Sponsored-by: Seagate Technology LLC Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Rob Wing Closes #14190 commit fff4612b4212dc26685aafa070abe5221a73e775 Author: Xinliang Liu Date: Sat Dec 3 09:39:48 2022 +0800 autoconf: add support for openEuler Add config support for openEuler, so that it set the right sysconfig dir for openEuler. And DEFAULT_INIT_SCRIPT is no longer needed since commit "2a34db1bd Base init scripts for SYSV systems". Reviewed-by: George Melikov Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Xinliang Liu Closes #14241 commit ccbc5f196ce862fb03c3f80e1c43046b39aa78d7 Author: szubersk Date: Wed Nov 30 20:27:28 2022 +1000 Fix Clang 15 compilation errors - Clang 15 doesn't support `-fno-ipa-sra` anymore. Do a separate check for `-fno-ipa-sra` support by $KERNEL_CC. - Don't enable `-mgeneral-regs-only` for certain module files. Fix #13260 - Scope `GCC diagnostic ignored` statements to GCC only. Clang doesn't need them to compile the code. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #13260 Closes #14150 commit a4f51d3489dfca56d8cc1c772f0f69da7a928055 Author: szubersk Date: Wed Nov 30 20:27:28 2022 +1000 Fix GCC 12 compilation errors Squelch false positives reported by GCC 12 with UBSan. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #14150 commit cbbd2037a898de405205d30e850253e0b98c77af Author: Brian Behlendorf Date: Wed Nov 30 10:25:57 2022 -0800 Retire Ubuntu 18.04 CI builder The GitHub-hosted Ubuntu 18.04 has been deprecated and will be entirely unsupported as of April 2023. Leading up to this there will be scheduled "brownouts" to encourage users to update their workflows. This commit retires our use of the GitHub-hosted Ubuntu 18.04 runners in advance of their removal. Reviewed-by: Richard Yao Reviewed-by: George Melikov Reviewed-by: Damian Szuberski Signed-off-by: Brian Behlendorf Closes #14238 commit af8e4760296a227dad7ad39e974e1adcfe4a0eb0 Author: Yann Collet Date: Thu Mar 18 11:28:22 2021 -0700 zstd: Refactor prefetching for the decoding loop Following facebook/zstd#2545, I noticed that one field in `seq_t` is optional, and only used in combination with prefetching. (This may have contributed to static analyzer failure to detect correct initialization). I then wondered if it would be possible to rewrite the code so that this optional part is handled directly by the prefetching code rather than delegated as an option into `ZSTD_decodeSequence()`. This resulted into this refactoring exercise where the prefetching responsibility is better isolated into its own function and `ZSTD_decodeSequence()` is streamlined to contain strictly Sequence decoding operations. Incidently, due to better code locality, it reduces the need to send information around, leading to simplified interface, and smaller state structures. Port of facebook/zstd@f5434663ea2a79b287ab4cd299179342f64a23a7 Reported-by: Coverity (CID 1462271) Reviewed-by: Damian Szuberski Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Ported-by: Richard Yao Closes #14212 commit b88212dfdcfa53a77e9338d9d84836bd675e44a7 Author: Nick Terrell Date: Mon Jun 22 10:21:38 2020 -0700 zstd: [superblock] Add defensive assert and bounds check The bound check condition should always be met because we selected `set_basic` as our encoding type. But that code is very far away, so assert it is true so if it is ever false we can catch it, and add a bounds check. Port of facebook/zstd@1047097dadd5f8dfd47e25af0738eeb4bd82f6ec Reported-by: Coverity (CID 1524446) Reviewed-by: Damian Szuberski Reviewed-by: Brian Behlendorf Reviewed-by: Tino Reichardt Ported-by: Richard Yao Closes #14212 commit 9bb722c576ea68d1539a49cb2fb784646675f9f4 Author: Richard Yao Date: Wed Oct 19 16:28:52 2022 -0400 Coverity Model Update When reviewing Clang Static Analyzer reports against a branch that had experimental header changes based on the Coverity model file to inform it that KM_SLEEP allocations cannot return NULL, I found a report saying that a KM_PUSHPAGE allocation returned NULL. The actual implementation does not return NULL unless KM_NOSLEEP has been passed, so we backport the correction from the experimental header changes to the Coverity model. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Richard Yao Closes #14210 commit 7f62a3450e8652dae044d33d0fead0dfe19c35ae Author: Richard Yao Date: Sun Nov 20 18:04:08 2022 -0500 Fix NULL pointer dereference in dbuf_prefetch_indirect_done() When ZFS is built with assertions, a prefetch is done on a redacted blkptr and `dpa->dpa_dnode` is NULL, we will have a NULL pointer dereference in `dbuf_prefetch_indirect_done()`. Both Coverity and Clang's Static Analyzer caught this. Reported-by: Coverity (CID 1524671) Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Richard Yao Closes #14210 commit 8c5cedaa528e5a6a42597b9501e67bb02e163f6e Author: Richard Yao Date: Wed Nov 16 16:23:53 2022 -0500 zdb: Silence Coverity complaint about verify_livelist_allocs() svb is declared on the stack. We then set parts of svb.svb_dva with DVA_SET_VDEV(), DVA_SET_OFFSET() and DVA_SET_ASIZE(). However, the DVA contains other fields for pad, GRID and G. When setting the fields we use, we technically read uninitialized bits from the fields we do not use. This makes Coverity and Clang's Static Analyzer complain. Presumably, other static analyzers might complain too. There is no real bug here, but we are still technically reading undefined data and unless we stop doing that, static analyzers will complain about it in perpetuum and this could obscure real issues. We silence the static analyzer complaints by using a 0 struct initializer. Reported by: Coverity (CID 1524627) Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Richard Yao Closes #14210 commit 3580c5e765f4b0ee678ff4431883b565f3dddd5b Author: Richard Yao Date: Wed Nov 16 16:38:19 2022 -0500 Cleanup: Delete dead code from send_merge_thread() range is always deferenced before it reaches this check, such that the kmem_zalloc() call is never executed. A previously version of this had erronously also pruned the `range->eos_marker = B_TRUE` line, but it must be set whenever we encounter an error or are cancelled early. Coverity incorrectly complained about a potential NULL pointer dereference because of this. Reported-by: Coverity (CID 1524550) Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Signed-off-by: Richard Yao Closes #14210 commit 4aae10fd9c021a1d0f7d4182b547539fe1c47bab Author: Andrew Innes Date: Sat Dec 17 06:57:22 2022 +0800 Windows: Fix the last two CFI callback prototype mismatches Signed-off-by: Andrew Innes commit 73035af427449e5a1ddae9c85d36827db92ed93c Author: Alexander Date: Tue Nov 29 18:56:16 2022 +0100 Fix the last two CFI callback prototype mismatches There was the series from me a year ago which fixed most of the callback vs implementation prototype mismatches. It was based on running the CFI-enabled kernel (in permissive mode -- warning instead of panic) and performing a full ZTS cycle, and then fixing all of the problems caught by CFI. Now, Clang 16-dev has new warning flag, -Wcast-function-type-strict, which detect such mismatches at compile-time. It allows to find the remaining issues missed by the first series. There are only two of them left: one for the secpolicy_vnode_setattr() callback and one for taskq_dispatch(). The fix is easy, since they are not used anywhere else. Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Lobakin Closes #14207 commit 1d4465c8b3e5a648796d642c4bdaa8661dc4eca5 Author: Richard Yao Date: Tue Nov 29 12:53:33 2022 -0500 Lua: Fix bad bitshift in lua_strx2number() The port of lua to OpenZFS modified lua to use int64_t for numbers instead of double. As part of this, a function for calculating exponentiation was replaced with a bit shift. Unfortunately, it did not handle negative values. Also, it only supported exponents numbers with 7 digits before before overflow. This supports exponents up to 15 digits before overflow. Clang's static analyzer reported this as "Result of operation is garbage or undefined" because the exponent was negative. Reviewed-by: Damian Szuberski Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14204 commit 119e51e19bd5a76500093850cf70b4b8eacca841 Author: Brooks Davis Date: Tue Nov 29 09:51:35 2022 -0800 Don't leak packed recieved proprties When local properties (e.g., from -o and -x) are provided, don't leak the packed representation of the received properties due to variable reuse. Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14197 commit fac66e7ba7cf153d577b86d3252476b69978f60f Author: Alexander Motin Date: Tue Nov 29 12:49:02 2022 -0500 Remove few pointer dereferences in dbuf_read() Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: Ryan Moeller Signed-off-by: Alexander Motin Closes #14199 commit 0644823c8291f22d25d708118ff1bca1cbd44b27 Author: Mateusz Guzik Date: Tue Nov 29 18:35:25 2022 +0100 FreeBSD: stop using buffer cache-only routines on sync Both vop_fsync and vfs_stdsync are effectively just costly no-ops as they only act on ->v_bufobj.bo_dirty et al, which are unused by zfs. Reviewed-by: Ryan Moeller Signed-off-by: Mateusz Guzik Closes #14157 commit d7b0c99d95fd421e7135b58285648d1e34c024b2 Author: Alexander Motin Date: Tue Nov 29 12:33:45 2022 -0500 Switch dnode stats to wmsums I've noticed that some of those counters are used in hot paths like dnode_hold_impl(), and results of this change is visible in profiler. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: Ryan Moeller Signed-off-by: Alexander Motin Closes #14198 commit 08945f109482acc4d5ec96bfa5d5c18af17f37fa Author: Xinliang Liu Date: Wed Nov 30 01:27:22 2022 +0800 rpm: add support for openEuler OpenEuler uses the same package manager DNF as RHEL/Fedora. And it is similar to RHEL/Fedora. OpenEuler Linux is becoming the mainstream Linux distro in China. So adding support for it makes sense for the users. For more details about it see: https://www.openeuler.org/en/. Reviewed-by: Richard Yao Reviewed-by: George Melikov Reviewed-by: Damian Szuberski Signed-off-by: Xinliang Liu Closes #14222 commit 679f776e93a3bcd43ca54fb23c73b3622fb81eb6 Author: Alexander Motin Date: Tue Nov 29 12:26:03 2022 -0500 Micro-optimize zrl_remove() atomic_dec_32() should be a bit lighter than atomic_dec_32_nv(). Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Signed-off-by: Ryan Moeller Signed-off-by: Alexander Motin Closes #14200 commit 91d0b90fa4aa5e11e181c058dfbceeda53b9c29d Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Tue Nov 29 22:24:10 2022 +0500 zed: unclean disk attachment faults the vdev If the attached disk already contains a vdev GUID, it means the disk is not clean. In such a scenario, the physical path would be a match that makes the disk faulted when trying to online it. So, we would only want to proceed if either GUID matches with the last attached disk or the disk is in a clean state. Reviewed-by: Brian Behlendorf Reviewed-by: Alexander Motin Reviewed-by: Ryan Moeller Reviewed-by: Tony Hutter Signed-off-by: Ameer Hamza Closes #14181 commit 4c1481deb9447f6b763167bb73bba81dcb9ad87c Author: Richard Yao Date: Mon Nov 28 16:49:58 2022 -0500 Convert some sprintf() calls to kmem_scnprintf() These `sprintf()` calls are used repeatedly to write to a buffer. There is no protection against overflow other than reviewers explicitly checking to see if the buffers are big enough. However, such issues are easily missed during review and when they are missed, we would rather stop printing rather than have a buffer overflow, so we convert these functions to use `kmem_scnprintf()`. The Linux kernel provides an entire page for module parameters, so we are safe to write up to PAGE_SIZE. Removing `sprintf()` from these functions removes the last instances of `sprintf()` usage in our platform-independent kernel code. This improves XNU kernel compatibility because the XNU kernel does not support (removed support for?) `sprintf()`. Reviewed-by: Brian Behlendorf Reviewed-by: Jorgen Lundman Signed-off-by: Richard Yao Closes #14209 commit f43bb3ccc749fc8242f324eba03274b1e9bb3ce5 Author: Allan Jude Date: Mon Nov 28 16:40:49 2022 -0500 Avoid a null pointer dereference in zfs_mount() on FreeBSD When mounting the root filesystem, vfs_t->mnt_vnodecovered is null This will cause zfsctl_is_node() to dereference a null pointer when mounting, or updating the mount flags, on the root filesystem, both of which happen during the boot process. Reported-by: Martin Matuska Reviewed-by: Richard Yao Reviewed-by: Alexander Motin Reviewed-by: Richard Yao Signed-off-by: Allan Jude Closes #14218 commit 50dff605fbdef333ec420ac6e0f1637939872261 Author: наб Date: Mon Nov 28 22:37:07 2022 +0100 cmd: zfs: fix missing mention of zfs diff -h Fixes: 344bbc82e7054f61d5e7b3610b119820285fd2cb Reviewed-by: Damian Szuberski Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Ahelenia Ziemiańska Closes #14224 commit eafa752da77b7ba98a083157113571d415e7e2ef Author: Richard Yao Date: Mon Nov 28 16:29:55 2022 -0500 Set multiple make jobs on CodeQL github workflows github supports 2 processors right now, although we use nproc instead of hard coding -j2, so if that ever increases, we will take advantage of it right away. Reviewed-by: George Melikov Reviewed-by: Damian Szuberski Reviewed-by: Tino Reichardt Signed-off-by: Richard Yao Closes #14217 commit 8f1f9b8f82d749f83a53cc816b14f56ae87fcc16 Author: Minsoo Choo Date: Mon Nov 28 16:24:17 2022 -0500 Add header According to the UNIX standard, does not include some PTHREAD_* values which are included in . OpenZFS uses some of these values in its code, and this might cause build failure on systems that do not have these PTHREAD_* values in Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: Damian Szuberski Signed-off-by: Minsoo Choo Closes #14225 commit 5183711c4db8a9fb1f2f50d4274c90cf73b9ff7f Author: Allan Jude Date: Mon Nov 28 14:41:14 2022 -0500 zfs.4: Correct the default for zfs_vdev_async_write_max_active Reviewed-by: Alexander Motin Reviewed-by: Richard Yao Reviewed-by: George Melikov Signed-off-by: Allan Jude Closes #14179 commit a1fb704b43ef8d1fcbfbf0e3c690b403691a1642 Author: Damian Szuberski Date: Tue Nov 29 05:39:41 2022 +1000 Python3: replace `distutils` with `sysconfig` - `distutils` module is long time deprecated and already deleted from the CPython mainline. - To remain compatible with Debian/Ubuntu Python3 packaging style, try `distutils.sysconfig.get_python_path(0,0)` first with fallback on `sysconfig.get_path('purelib')` - pyzfs_unittest suite is run unconditionally as a part of ZTS. - Add pyzfs_unittest suite to sanity tests. Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #12833 Closes #13280 Closes #14177 commit 91e5815f1008e4cce88ce18278a40ca81ddf1f91 Author: Andrew Innes Date: Sat Dec 17 13:45:17 2022 +0800 Windows: Remove atomics from zh_refcount Signed-off-by: Andrew Innes commit 3c8a86caf6447590550407d64ff4ec602daa717e Author: Alexander Motin Date: Mon Nov 28 14:36:53 2022 -0500 Remove atomics from zh_refcount It is protected by z_hold_locks, so we do not need more serialization, simple integer math should be fine. Reviewed-by: Brian Behlendorf Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Signed-off-by: Alexander Motin Closes #14196 commit 64f6df7aebc4fa5f240e023c3bce04cfc80774bd Author: Jorgen Lundman Date: Thu Dec 8 09:34:06 2022 +0900 Upstream: add kfpu-vars variable Empty for Linux and FreeBSD. Signed-off-by: Jorgen Lundman commit ebb805e480a488bcbc2f9a5b0e650d5005d5136a Author: Jorgen Lundman Date: Tue Dec 6 20:17:00 2022 +0900 Add kfpu_vars to allow variables to be defined commit b8ceee2008725e2f6baa8fde620772d27ad0bfef Author: Jorgen Lundman Date: Tue Dec 6 20:04:13 2022 +0900 Windows: re-enable gcm after aligning struct commit 5e1fcca082a3b2cc0b513c5fa4cbb6f144d2ee09 Author: Jorgen Lundman Date: Sat Dec 3 16:24:55 2022 +0900 Unify Assembler files between Linux and Windows Add new macro ASMABI used by Windows to change calling API to "sysv_abi". Signed-off-by: Jorgen Lundman commit 4b34d77e3319eef1c97f964724d56bb1530c32b9 Author: Jorgen Lundman Date: Mon Dec 5 15:09:14 2022 +0900 Windows: update installer readme commit bb5938f92e5a3ddf52f2f1212aee087f4f8dce38 Author: Jorgen Lundman Date: Mon Dec 5 10:38:37 2022 +0900 Disable gcm ASM until it can be debugged commit 688f1ba7e621274647ef48767e5b09f2b7fc9129 Author: Jorgen Lundman Date: Mon Dec 5 09:22:08 2022 +0900 Restore some changes to gcm assembler This will disappear when Unify PR is merged. commit f2244c728ee6f0c5e3a23a3cffab1256de6579be Author: Jorgen Lundman Date: Sun Nov 27 16:42:29 2022 +0900 Remove the Windows versions of assembler files commit fb87a9533bb4983e600cd5c96489987cc6ed1dbd Author: Jorgen Lundman Date: Sun Nov 27 16:13:36 2022 +0900 Unify setjmp for x86 64 commit 9bac6ce739f81da2cb35fcfef4bde8e7892df213 Author: Jorgen Lundman Date: Sat Nov 26 15:43:22 2022 +0900 Unify sha, aes, gcm aes-256-gcm WILL panic immediately. commit 4cfa1b79593ff36a176c5d5194dd21c10d5ddd81 Author: Jorgen Lundman Date: Fri Nov 25 17:29:02 2022 +0900 Unify x86_64/modes/ commit 6a777dd2aa805541daa93a0570d2cff3c2f8b145 Author: Andrew Innes Date: Fri Nov 25 14:01:37 2022 +0800 Revert "Windows: Add winfstest" This reverts commit 0f3159d67eeb4c460691b13b4677941e67187735. commit a06f8deb64b1d201d9018fae8869b6417dfcaad8 Author: Andrew Innes Date: Fri Nov 25 14:01:37 2022 +0800 Revert "Migrate to python3" This reverts commit 037f5d8acd19212bfaa04ac9cd5ad292e8030807. commit c7504e23f311d2036ae99f3179cb23c793c664e2 Author: Jorgen Lundman Date: Fri Nov 25 14:12:01 2022 +0900 Remove stack.h and trap.h and nothing breaks? commit 037f5d8acd19212bfaa04ac9cd5ad292e8030807 Author: Andrew Innes Date: Tue Oct 11 19:12:54 2022 +0800 Migrate to python3 Signed-off-by: Andrew Innes commit 0f3159d67eeb4c460691b13b4677941e67187735 Author: Andrew Innes Date: Tue Oct 11 18:50:19 2022 +0800 Windows: Add winfstest From https://github.com/billziss-gh/secfs.test/tree/master/winfstest Signed-off-by: Andrew Innes commit fdc1b2c08724e5a821ce82e4db25df0b2a09460b Author: Andrew Innes Date: Thu Nov 17 12:01:26 2022 +0800 Don't cstyle zlib Signed-off-by: Andrew Innes commit 40e7a486570af58cebbce9fda903a0c807fffbbc Author: Andrew Innes Date: Wed Nov 16 15:42:53 2022 +0800 rename to 1.2.13 commit cb89e57ffbddb924b531347a586acc33c1e4a0dd Author: Andrew Innes Date: Wed Nov 16 15:35:00 2022 +0800 remove stuff commit 2243718a59843e6370ccf7bf740ef8cd3cb8e586 Author: Andrew Innes Date: Wed Nov 16 15:20:12 2022 +0800 zconf.in.h -> zconf.h.in commit 58c388d41eb744896fa772e3b9b778b082cc0d31 Author: Andrew Innes Date: Wed Nov 16 06:26:27 2022 +0800 Remove asm Signed-off-by: Andrew Innes commit 55ed5113dca3b14e02186ccdb863422beb8caf62 Author: Andrew Innes Date: Tue Nov 15 16:12:33 2022 +0800 Hacky fixes Signed-off-by: Andrew Innes commit 9a5afdc9b4d65758774018454fae2dc2396749a4 Author: Andrew Innes Date: Tue Nov 15 16:08:47 2022 +0800 Breaking commit Signed-off-by: Andrew Innes commit 7630d97d60184cff22192632cbee9f84409b2e49 Author: Andrew Innes Date: Tue Nov 15 14:55:10 2022 +0800 Smallest I could get the diff before breaking Signed-off-by: Andrew Innes commit 3005ac498841ef9ef576bea6959d81786f240592 Author: Andrew Innes Date: Tue Nov 15 14:15:57 2022 +0800 Update zlib now that gzio.c has been replaced Signed-off-by: Andrew Innes commit 11ee032a79e864421fc0ada643e927a2c246f661 Author: Andrew Innes Date: Tue Nov 15 14:01:12 2022 +0800 gzio.c might have been expanded Signed-off-by: Andrew Innes commit d184d7395007e0ca3db195a3fb3c82158648f8dc Author: Andrew Innes Date: Mon Nov 14 19:59:44 2022 +0800 Update zlib to 1.2.13 https://github.com/madler/zlib Signed-off-by: Andrew Innes commit e2ec3c954eabc22b11e2dd308e4c4bfcf2ec1bdf Author: Andrew Innes Date: Mon Nov 14 18:25:22 2022 +0800 Replace zlib with near stock gnuwin zlib 1.2.3 By replacing most of the reformatted code with original code, we can take advantage of cleaner diffs while upgrading https://gnuwin32.sourceforge.net/packages/zlib.htm Signed-off-by: Andrew Innes commit 89b977077c2a6893961b3961581344feb264e0d1 Author: Jorgen Lundman Date: Fri Nov 25 12:20:38 2022 +0900 Unify aes_amd64.S Also call save registers (kfpu_begin()) on windows This appears to fix blake3 + aes used together. commit 246cbc72ba31b177063357cd4a04a6602594fca1 Author: Jorgen Lundman Date: Thu Nov 24 13:21:14 2022 +0900 Autoconf would pick up on .in file and attempt to expand it, when it is only used in Windows CMakelists.txt. Rename it to .win to avoid and make it more clear it is only for Windows. Signed-off-by: Jorgen Lundman commit 7a9d51f803b410dd5d4a0190542d724df4e621a8 Author: Jorgen Lundman Date: Mon Nov 21 16:09:43 2022 +0900 Attempt to unify blake3_sse2.S Since we can ask clang to call assembler functions with sysv_abi on Windows, we can now unify those files into one copy, instead of a separate copy for each platform. This PR is created to explore the idea of unifying the assembler files. Is that "desirable"? Initially only blake3_sse2.S has been updated. Chosen by a kabal of 7 members as least offensive (criteria unknown). This is a continuation of (or perhaps return to) work that illumos started. All assembler files should include . This file will now include the arch specific header, for example . This file has been moved into platform specific location. Ie; include/os/linux/spl/sys/ia32/asm_linkage.h include/os/macos/spl/sys/ia32/asm_linkage.h include/os/windows/spl/sys/ia32/asm_linkage.h (and so on, imagine the same for aarch64 etc). and the same in libspl: lib/libspl/include/os/linux/sys/ia32/asm_linkage.h So some header duplication between kernel and userland, like we already have. In these headers files, macros are defined: ENTRY_NP() SET_SIZE() to be used by the assembler files. Importantly for Windows, we also define a new ASMABI which is set to "__attributes__((sysv_abi))". In C, when we extern prototype the assembler functions, we add ASMABI to it so the compiler will know to juggle the parameters and registers used. (The Unix rdi, rsi, rcx, rdx, r8, r9 versus Windows rdx, rdc, r8, r9, stack, stack ordering problem.) For example: extern void ASMABI zfs_blake3_compress_in_place_sse2(uint32_t cv[8], const uint8_t block[BLAKE3_BLOCK_LEN], uint8_t block_len, uint64_t counter, uint8_t flags); This does mean we need to include spl headers when compiling module/icp and libicp. But it isn't to strange that the illumos-crypto-psomething uses SolarisPlatformLsomething. Right now, macOS and Windows uses a separate copy of all assembler files, with Linux bits commented out. So it is "more" work to complete this PR. It is no work to do nothing. But it is generally desirable to remove code duplication? It now also sets sysv_abi for Linux. I do not know if this is going to be an issue. It felt "more correct" to do so since the assembler files definitely are written that way, and calling them any other way will not work (well). I think Linux developers should decide here. Do we hate it? Is it worth continuing? I do not care one bit about what the macros are named (okok, let's not test that statement). So if there are better names, speak. Signed-off-by: Jorgen Lundman commit bb09c41e665aabf549422be2fc73a752ae336fcd Author: Jorgen Lundman Date: Tue Nov 22 10:16:39 2022 +0900 cstyle corrections commit 5871feb92ab4018a8d7b8e341c4890be64d50873 Author: Jorgen Lundman Date: Mon Oct 24 09:00:57 2022 +0900 Update assembler files for Windows Add blake3 assembly and tunables Change the impl tunables to use the Linux style. Change call to blake3 assembly to use sysv abi. update tunables to handle linux module param call. Eventually, we can get rid of the Windows copies of the assembler files by using the sysv_abi. Signed-off-by: Jorgen Lundman commit 27bc3309cd2e6498424c96d32f51a5b2ae27bf38 Author: Andrew Innes Date: Mon Nov 14 08:21:09 2022 +0800 ignore Wint-conversion Visual Studio 2022 update 2022114 started breaking builds with errors Signed-off-by: Andrew Innes commit 167ea9632a7788f775adad39b718f28c09e1fe2b Author: John Wren Kennedy Date: Fri Nov 18 12:43:18 2022 -0700 ZTS: zts-report silently ignores perf test results The regex used to extract test result information from a test run only matches the functional tests. Update the regex so it matches both. Reviewed-by: Richard Yao Reviewed-by: George Melikov Reviewed-by: Tony Nguyen Signed-off-by: John Wren Kennedy Closes #14185 commit 5929f0aedda6b23b2b28ca9c67f1da2caf796293 Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Sat Nov 19 00:39:59 2022 +0500 zed: post a udev change event from spa_vdev_attach() In order for zed to process the removal event correctly, udev change event needs to be posted to sync the blkid information. spa_create() and spa_config_update() posts the event already through spa_write_cachefile(). Doing the same for spa_vdev_attach() that handles the case for vdev attachment and replacement. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: Alexander Motin Reviewed-by: Ryan Moeller Signed-off-by: Ameer Hamza Closes #14172 commit 1594956b682962eaa07788012fc4c9783d1e3102 Author: George Amanakis Date: Fri Nov 18 20:38:37 2022 +0100 Fix setting the large_block feature after receiving a snapshot We are not allowed to dirty a filesystem when done receiving a snapshot. In this case the flag SPA_FEATURE_LARGE_BLOCKS will not be set on that filesystem since the filesystem is not on dp_dirty_datasets, and a subsequent encrypted raw send will fail. Fix this by checking in dsl_dataset_snapshot_sync_impl() if the feature needs to be activated and do so if appropriate. Reviewed-by: Brian Behlendorf Signed-off-by: George Amanakis Closes #13699 Closes #13782 commit b9c24317d58ff90e72a90d27efd06a2181997230 Author: Laura Hild Date: Fri Nov 18 14:36:19 2022 -0500 Correct multipathd.target to .service https://github.com/openzfs/zfs/pull/9863 says it "orders zfs-import-cache.service and zfs-import-scan.service after multipathd.service" but the commit (79add96) actually ordered them after .target. Reviewed-by: Brian Behlendorf Signed-off-by: Laura Hild Closes #12709 Closes #14171 commit 78d97530662ddcab4a6a1d4fe4bb2e99b5ea417d Author: Richard Yao Date: Thu Nov 3 13:53:17 2022 -0400 FreeBSD: do_mount() passes wrong string length to helper It should pass `MNT_LINE_MAX`, but passes `sizeof (mntpt)`. This is harmless because the strlen is not actually used by the helper, but FreeBSD's Coverity scans complained about it. This was missed in my audit of various string functions since it is not actually passed to a string function. Upon review, it was noticed that the helper function does not need to be a separate function, so I have inlined it as cleanup. Reported-by: Coverity (CID 1432079) Reviewed-by: Brian Behlendorf Reviewed-by: szubersk Reviewed-by: Alexander Motin Signed-off-by: Richard Yao Closes #14136 commit c85c755c85e28eb5c3b94cfd67db787d74219454 Author: Richard Yao Date: Thu Nov 3 13:58:38 2022 -0400 FreeBSD: get_zfs_ioctl_version() should be cast to (void) FreeBSD's Coverity scans complain that we ignore the return value. There is no need to check the return value so we cast it to (void) to suppress further complaints by static analyzers. Reported-by: Coverity (CID 1018175) Reviewed-by: Brian Behlendorf Reviewed-by: szubersk Reviewed-by: Alexander Motin Signed-off-by: Richard Yao Closes #14136 commit 57280a0331ae2c1ad811e2494ff1204c192ee4d1 Author: szubersk Date: Sat Nov 12 22:48:32 2022 +1000 Ubuntu 22.04 integration: GitHub workflows - GitHub workflows are run on Ubuntu 22.04 - Extract the `checkstyle` workflow dependencies to a separate file. - Refresh the `build-dependencies.txt` list. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: George Melikov Signed-off-by: szubersk Closes #14148 commit 2c15d99efd9ebe0c45926f1b9b5a9cd630e77f97 Author: szubersk Date: Sat Nov 12 22:30:57 2022 +1000 Ubuntu 22.04 integration: ZTS Add `detect_odr_violation=1` to ASAN_OPTIONS to allow both libzfs and libzpool expose ``` zfeature_info_t spa_feature_table[SPA_FEATURES] ``` from module/zcommon/zfeature_common.c in public ABI. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: George Melikov Signed-off-by: szubersk Closes #14148 commit c27948c0b1369b6bdc03f46d832187b0759ca637 Author: szubersk Date: Sat Nov 12 22:29:29 2022 +1000 Ubuntu 22.04 integration: Cppcheck Suppress a false positive found by new Cppcheck version. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: George Melikov Signed-off-by: szubersk Closes #14148 commit 9c32a45bb46c8a6051513e2b9a243fd16dc40cd7 Author: szubersk Date: Sat Nov 12 22:23:30 2022 +1000 Ubuntu 22.04 integration: mancheck Correct new mandoc errors. ``` STYLE: input text line longer than 80 bytes STYLE: no blank before trailing delimiter ``` Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: George Melikov Signed-off-by: szubersk Closes #14148 commit b491a28d3f28cf56a2986d42d231d12850471050 Author: szubersk Date: Sat Nov 12 22:22:49 2022 +1000 Ubuntu 22.04 integration: ShellCheck - Add new SC2312 global exclude. ``` Consider invoking this command separately to avoid masking its return value (or use '|| true' to ignore). [SC2312] ``` - Correct errors detected by new ShellCheck version. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Reviewed-by: George Melikov Signed-off-by: szubersk Closes #14148 commit a58118ea83a36924f4755bda65d45dff74bee379 Author: Andrew Innes Date: Mon Nov 14 08:24:30 2022 +0800 Use correct flags Signed-off-by: Andrew Innes commit d8d5ed27c51a627e16b6846c23c57e1bd31c6a3c Author: Damian Szuberski Date: Thu Nov 17 03:27:53 2022 +1000 Make autodetection disable pyzfs for kernel/srpm configurations Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: szubersk Closes #13394 Closes #14178 commit 4547c6d0819b3039cb2e2b51f07e4ffa56fda5e2 Author: Rich Ercolani <214141+rincebrain@users.noreply.github.com> Date: Tue Nov 15 17:44:12 2022 -0500 Handle and detect #13709's unlock regression (#14161) In #13709, as in #11294 before it, it turns out that 63a26454 still had the same failure mode as when it was first landed as d1d47691, and fails to unlock certain datasets that formerly worked. Rather than reverting it again, let's add handling to just throw out the accounting metadata that failed to unlock when that happens, as well as a test with a pre-broken pool image to ensure that we never get bitten by this again. Fixes: #13709 Signed-off-by: Rich Ercolani Reviewed-by: Brian Behlendorf Reviewed-by: Tony Hutter commit 57bb6632331da401243d9668966331f010d15f06 Author: Andrew Innes Date: Mon Nov 14 13:47:41 2022 +0800 use HTOI macro when casting handles Signed-off-by: Andrew Innes commit 1dcd8c8e047e9561aef5296e0d8bc6d776ffba33 Author: Andrew Innes Date: Mon Nov 14 15:02:36 2022 +0800 Pointer cast fixes Signed-off-by: Andrew Innes commit c59f28b9f5e9a5549b732241353924014a8b5a5e Author: Andrew Innes Date: Mon Nov 14 10:46:06 2022 +0800 cast alloc to correct pointer type Signed-off-by: Andrew Innes commit 3aaac9b94c55617c6e52eead009fe102d89ac292 Author: shodanshok Date: Fri Nov 11 19:41:36 2022 +0100 Fix arc_p aggressive increase The original ARC paper called for an initial 50/50 MRU/MFU split and this is accounted in various places where arc_p = arc_c >> 1, with further adjustment based on ghost lists size/hit. However, in current code both arc_adapt() and arc_get_data_impl() aggressively grow arc_p until arc_c is reached, causing unneeded pressure on MFU and greatly reducing its scan-resistance until ghost list adjustments kick in. This patch restores the original behavior of initially having arc_p as 1/2 of total ARC, without preventing MRU to use up to 100% total ARC when MFU is empty. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Gionatan Danti Closes #14137 Closes #14120 commit 518857484e620e5163f95dcd859e30040b31003a Author: Andrew Innes Date: Mon Nov 14 10:56:01 2022 +0800 cast alloc to correct pointer type Signed-off-by: Andrew Innes commit ebca313adfacf9b9990dede8e92d461fdd6c08c3 Author: Andrew Innes Date: Mon Nov 14 10:42:39 2022 +0800 cast user_addr_t to void pointer Signed-off-by: Andrew Innes commit f423a22d92dfffd4ddfa7082ba68ecef7d0d7f48 Author: Andrew Innes Date: Mon Nov 14 10:35:21 2022 +0800 cast pc_t to pointer typedef uintptr_t pc_t; Signed-off-by: Andrew Innes commit ed152632015c9857cf650bc97316d4a87483ad30 Author: Andrew Innes Date: Mon Nov 14 11:23:13 2022 +0800 Windows: getpid return type match linux Signed-off-by: Andrew Innes commit b29cffda4db3e3b8bd0f2f3535c45e7cddd56f21 Author: Andrew Innes Date: Fri Nov 11 08:50:34 2022 +0800 Windows: Add ability to recompress send streams with new compression algorithm Signed-off-by: Andrew Innes commit c0c3df31fc76d5090c51544748ac5cf72f9fb5d8 Author: Paul Dagnelie Date: Thu Nov 10 15:23:46 2022 -0800 Add ability to recompress send streams with new compression algorithm As new compression algorithms are added to ZFS, it could be useful for people to recompress data with new algorithms. There is currently no mechanism to do this aside from copying the data manually into a new filesystem with the new algorithm enabled. This tool allows the transformation to happen through zfs send, allowing it to be done efficiently to remote systems and in an incremental fashion. A new zstream command is added that decompresses WRITE records and then recompresses them with a provided algorithm, and then re-emits the modified send stream. It may also be possible to re-compress embedded block pointers, but that was not attempted for the initial version. Reviewed-by: Brian Behlendorf Reviewed-by: Matthew Ahrens Signed-off-by: Paul Dagnelie Closes #14106 commit adc04582e4534e3f2ee6aa7f6799035de8aced5e Author: Brooks Davis Date: Tue Nov 1 20:43:32 2022 +0000 Make 1-bit bitfields unsigned This fixes -Wsingle-bit-bitfield-constant-conversion warning from clang-16 like: lib/libzfs/libzfs_dataset.c:4529:19: error: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Werror,-Wsingle-bit-bitfield-constant-conversion] flags.nounmount = B_TRUE; ^ ~~~~~~ Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Signed-off-by: Brooks Davis Closes #14125 commit fb27b6e850653a3acfd3f15afbdec887c18e0ed5 Author: Andrew Innes Date: Fri Nov 11 09:38:18 2022 +0800 Windows: acl: use uintptr_t for ace walker cookies Signed-off-by: Andrew Innes commit 299424e92cee55639d08a2eb9d1f58cbba45a700 Author: Brooks Davis Date: Thu Oct 27 23:39:06 2022 +0100 lua: cast through uintptr_t when return a pointer Don't assume size_t can carry pointer provenance and use uintptr_t (identialy on all current platforms) instead. Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Signed-off-by: Brooks Davis Closes #14131 commit 49f282c42b2f893fbaef0e24133ca571887989d1 Author: Brooks Davis Date: Thu Oct 27 23:28:03 2022 +0100 Use intptr_t when storing an integer in a pointer Cast the integer type to (u)intptr_t before casting to "void *". In CHERI C/C++ we warn on bare casts from integers to pointers to catch attempts to create pointers our of thin air. We allow the warning to be supressed with a suitable cast through (u)intptr_t. Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Signed-off-by: Brooks Davis Closes #14131 commit a6d50d3e10a88d55e0c7ebf8e9dc0321e7c3fe7e Author: Brooks Davis Date: Thu Oct 27 23:25:42 2022 +0100 recvd_props_mode: use a uintptr_t to stash nvlists Avoid assuming than a uint64_t can hold a pointer. Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Signed-off-by: Brooks Davis Closes #14131 commit 7f3a2739e80e3a5899aa60b73c1e6f07a85aaad6 Author: Brooks Davis Date: Thu Oct 27 23:20:05 2022 +0100 zfs_onexit_add_cb: make action_handle point to a uintptr_t Avoid assuming than a uint64_t can hold a pointer and reduce the number of casts in the process. Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Signed-off-by: Brooks Davis Closes #14131 commit 7729ebf90ea12acca389c1dfb4808ff8aabe17bb Author: Brooks Davis Date: Thu Oct 27 23:04:17 2022 +0100 acl: use uintptr_t for ace walker cookies Avoid assuming that a pointer can fit in a uint64_t and use uintptr_t instead. Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Signed-off-by: Brooks Davis Closes #14131 commit b290429704b924cd70753e403431502bc06ece78 Author: Richard Yao Date: Tue Nov 1 16:58:17 2022 -0400 FreeBSD: Fix regression from kmem_scnprintf() in libzfs kmem_scnprintf() is only available in libzpool. Recent buildbot issues with showing FreeBSD results kept us from seeing this before 97143b9d314d54409244f3995576d8cc8c1ebf0a was merged. The code has been changed to sanitize the output from `kmem_scnprintf()`. Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14111 commit 921f4bca810eafad9fe6cd0d25f9398bb0f06e45 Author: Alexander Lobakin Date: Sun Oct 16 23:41:39 2022 +0200 icp: fix all !ENDBR objtool warnings in x86 Asm code Currently, only Blake3 x86 Asm code has signs of being ENDBR-aware. At least, under certain conditions it includes some header file and uses some custom macro from there. Linux has its own NOENDBR since several releases ago. It's defined in the same , so currently already is provided with it. Let's unify those two into one %ENDBR macro. At first, check if it's present already. If so -- use Linux kernel version. Otherwise, try to go that second way and use %_CET_ENDBR from if available. If no, fall back to just empty definition. This fixes a couple more 'relocations to !ENDBR' across the module. And now that we always have the latest/actual ENDBR definition, use it at the entrance of the few corresponding functions that objtool still complains about. This matches the way how it's used in the upstream x86 core Asm code. Reviewed-by: Attila Fülöp Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Lobakin Closes #14035 commit 5b41dfb44888819ee69ae6760d0fb54774489ceb Author: Alexander Lobakin Date: Sun Oct 16 23:23:44 2022 +0200 icp: fix rodata being marked as text in x86 Asm code objtool properly complains that it can't decode some of the instructions from ICP x86 Asm code. As mentioned in the Makefile, where those object files were excluded from objtool check (but they can still be visible under IBT and LTO), those are just constants, not code. In that case, they must be placed in .rodata, so they won't be marked as "allocatable, executable" (ax) in EFL headers and this effectively prevents objtool from trying to decode this data. That reveals a whole bunch of other issues in ICP Asm code, as previously objtool was bailing out after that warning message. Reviewed-by: Attila Fülöp Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Lobakin Closes #14035 commit c401c9c0c785d4fae6c27a5f2ad89154e1396223 Author: Alexander Lobakin Date: Sun Oct 16 16:53:22 2022 +0200 icp: properly fix all RETs in x86_64 Asm code Commit 43569ee37420 ("Fix objtool: missing int3 after ret warning") addressed replacing all `ret`s in x86 asm code to a macro in the Linux kernel in order to enable SLS. That was done by copying the upstream macro definitions and fixed objtool complaints. Since then, several more mitigations were introduced, including Rethunk. It requires to have a jump to one of the thunks in order to work, so the RET macro was changed again. And, as ZFS code didn't use the mainline defition, but copied it, this is currently missing. Objtool reminds about it time to time (Clang 16, CONFIG_RETHUNK=y): fs/zfs/lua/zlua.o: warning: objtool: setjmp+0x25: 'naked' return found in RETHUNK build fs/zfs/lua/zlua.o: warning: objtool: longjmp+0x27: 'naked' return found in RETHUNK build Do it the following way: * if we're building under Linux, unconditionally include in the related files. It is available in x86 sources since even pre-2.6 times, so doesn't need any conftests; * then, if RET macro is available, it will be used directly, so that we will always have the version actual to the kernel we build; * if there's no such macro, we define it as a simple `ret`, as it was on pre-SLS times. This ensures we always have the up-to-date definition with no need to update it manually, and at the same time is safe for the whole variety of kernels ZFS module supports. Then, there's a couple more "naked" rets left in the code, they're just defined as: .byte 0xf3,0xc3 In fact, this is just: rep ret `rep ret` instead of just `ret` seems to mitigate performance issues on some old AMD processors and most likely makes no sense as of today. Anyways, address those rets, so that they will be protected with Rethunk and SLS. Include here which now always has RET definition and replace those constructs with just RET. This wipes the last couple of places with unpatched rets objtool's been complaining about. Reviewed-by: Attila Fülöp Reviewed-by: Tino Reichardt Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Alexander Lobakin Closes #14035 commit 939460fe25baed697c34f489daf15f57b80f0bce Author: Andrew Innes Date: Thu Nov 10 13:49:21 2022 +0800 Windows: Introduce kmem_scnprintf() Signed-off-by: Andrew Innes commit 34e484ba0f2309c647ec08bb88d991245b54edd0 Author: Richard Yao Date: Thu Oct 27 14:16:04 2022 -0400 Introduce kmem_scnprintf() `snprintf()` is meant to protect against buffer overflows, but operating on the buffer using its return value, possibly by calling it again, can cause a buffer overflow, because it will return how many characters it would have written if it had enough space even when it did not. In a number of places, we repeatedly call snprintf() by successively incrementing a buffer offset and decrementing a buffer length, by its return value. This is a potentially unsafe usage of `snprintf()` whenever the buffer length is reached. CodeQL complained about this. To fix this, we introduce `kmem_scnprintf()`, which will return 0 when the buffer is zero or the number of written characters, minus 1 to exclude the NULL character, when the buffer was too small. In all other cases, it behaves like snprintf(). The name is inspired by the Linux and XNU kernels' `scnprintf()`. The implementation was written before I thought to look at `scnprintf()` and had a good name for it, but it turned out to have identical semantics to the Linux kernel version. That lead to the name, `kmem_scnprintf()`. CodeQL only catches this issue in loops, so repeated use of snprintf() outside of a loop was not caught. As a result, a thorough audit of the codebase was done to examine all instances of `snprintf()` usage for potential problems and a few were caught. Fixes for them are included in this patch. Unfortunately, ZED is one of the places where `snprintf()` is potentially used incorrectly. Since using `kmem_scnprintf()` in it would require changing how it is linked, we modify its usage to make it safe, no matter what buffer length is used. In addition, there was a bug in the use of the return value where the NULL format character was not being written by pwrite(). That has been fixed. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14098 commit 605eafaae69f3d675c393f85b31bbe22969ea3e5 Author: John Wren Kennedy Date: Thu Nov 10 15:00:04 2022 -0700 ZTS: random_readwrite test doesn't run correctly This test uses fio's bssplit mechanism to choose io sizes for the test, leaving the PERF_IOSIZES variable empty. Because that variable is empty, the innermost loop in do_fio_run_impl is never executed, and as a result, this test does the setup but collects no data. Setting the variable to "bssplit" allows performance data to be gathered. Reviewed-by: Tony Nguyen Signed-off-by: John Wren Kennedy Closes #14163 commit 5a2a131ccbbfad3c216ae1f6d058beeab19cea59 Author: Richard Yao Date: Thu Nov 10 09:09:35 2022 -0500 Cleanup: Suppress Coverity dereference before/after NULL check reports f224eddf922a33ca4b86d83148e9e6fa155fc290 began dereferencing a NULL checked pointer in zpl_vap_init(), which made Coverity complain because either the dereference is unsafe or the NULL check is unnecessary. Upon inspection, this pointer is guaranteed to never be NULL because it is from the Linux kernel VFS. The calls into ZFS simply would not make sense if this pointer were NULL, so the NULL check is unnecessary. Reported-by: Coverity (CID 1527260) Reported-by: Coverity (CID 1527262) Reviewed-by: Mariusz Zaborski Reviewed-by: Youzhong Yang Signed-off-by: Richard Yao Closes #14170 commit 34644ac9cb33e51a7268271eef71f77d9298ada7 Author: Richard Yao Date: Thu Nov 10 09:01:58 2022 -0500 Fix potential NULL pointer dereference regression 945b407486a0072ec7dd117a0bde2f72d52eb445 neglected to `NULL` check `tx->tx_objset`, which is already done in the function. This upset Coverity, which complained about a "dereference after null check". Upon inspection, it was found that whenever `dmu_tx_create_dd()` is called followed by `dmu_tx_assign()`, such as in `dsl_sync_task_common()`, `tx->tx_objset` will be `NULL`. Reported-by: Coverity (CID 1527261) Reviewed-by: Mariusz Zaborski Reviewed-by: Youzhong Yang Signed-off-by: Richard Yao Closes #14170 commit 36a9d26837ed70940e42a06ceb2d0dc53dc51c1f Author: Mariusz Zaborski Date: Thu Nov 10 22:37:12 2022 +0100 Allow to control failfast Linux defaults to setting "failfast" on BIOs, so that the OS will not retry IOs that fail, and instead report the error to ZFS. In some cases, such as errors reported by the HBA driver, not the device itself, we would wish to retry rather than generating vdev errors in ZFS. This new property allows that. This introduces a per vdev option to disable the failfast option. This also introduces a global module parameter to define the failfast mask value. Reviewed-by: Brian Behlendorf Co-authored-by: Allan Jude Signed-off-by: Allan Jude Signed-off-by: Mariusz Zaborski Sponsored-by: Seagate Technology LLC Submitted-by: Klara, Inc. Closes #14056 commit a48f7531143e1e15243367be9d5c7937440fa92b Author: Andrew Innes Date: Thu Nov 10 19:44:07 2022 +0800 Windows: Optionally skip zil_close during zvol_create_minor_impl Signed-off-by: Andrew Innes commit 5dbbcdfa7de694a24bb1a41eb8714cfa1a16db6e Author: Alan Somers Date: Tue Nov 8 13:38:08 2022 -0700 Optionally skip zil_close during zvol_create_minor_impl If there were no zil entries to replay, skip zil_close. zil_close waits for a transaction to sync. That can take several seconds, for example during pool import of a resilvering pool. Skipping zil_close can cut the time for "zpool import" from 2 hours to 45 seconds on a resilvering pool with a thousand zvols. Reviewed-by: Richard Yao Reviewed-by: Alexander Motin Reviewed-by: Ryan Moeller Sponsored-by: Axcient Closes #13999 Closes #14015 commit 7639af9e34096af0158ba880fd831dadd79154ca Author: Mariusz Zaborski Date: Tue Nov 8 21:40:22 2022 +0100 quota: disable quota check for ZVOL The quota for ZVOLs is set to the size of the volume. When the quota reaches the maximum, there isn't an excellent way to check if the new writers are overwriting the data or if they are inserting a new one. Because of that, when we reach the maximum quota, we wait till txg is flushed. This is causing a significant fluctuation in bandwidth. In the case of ZVOL, the quota is enforced by the volsize, so we can omit it. This commit adds a sysctl thats allow to control if the quota mechanism should be enforced or not. Reviewed-by: Brian Behlendorf Signed-off-by: Mariusz Zaborski Sponsored-by: Zededa Inc. Sponsored-by: Klara Inc. Closes #13838 commit 159afdc9d0280ec93264e4b11aa736b2548c3435 Author: youzhongyang Date: Tue Nov 8 13:28:56 2022 -0500 Support idmapped mount in user namespace Linux 5.17 commit torvalds/linux@5dfbfe71e enables "the idmapping infrastructure to support idmapped mounts of filesystems mounted with an idmapping". Update the OpenZFS accordingly to improve the idmapped mount support. This pull request contains the following changes: - xattr setter functions are fixed to take mnt_ns argument. Without this, cp -p would fail for an idmapped mount in a user namespace. - idmap_util is enhanced/fixed for its use in a user ns context. - One test case added to test idmapped mount in a user ns. Reviewed-by: Christian Brauner Reviewed-by: Brian Behlendorf Signed-off-by: Youzhong Yang Closes #14097 commit 617710d7bf8bb379ccf912eac032415081356569 Author: Damian Szuberski Date: Wed Nov 9 04:16:01 2022 +1000 dsl_prop_known_index(): check for invalid prop Resolve UBSAN array-index-out-of-bounds error in zprop_desc_t. Reviewed-by: Brian Behlendorf Signed-off-by: szubersk Closes #14142 Closes #14147 commit 40780644c7892964f89ff25d8d00563d63a73a1e Author: Mohamed Tawfik Date: Tue Nov 8 20:08:21 2022 +0200 Adds the `-p` option to `zfs holds` This allows for printing a machine-readable, accurate to the second, hold creation time in the form of a unix epoch timestamp. Additionally, updates relevant documentation and man pages accordingly. Reviewed-by: Allan Jude Reviewed-by: Brian Behlendorf Signed-off-by: Mohamed Tawfik Closes #13690 Closes #14152 commit a317af40ed9fe34f11e95a540d50496decc4ba1d Author: Brooks Davis Date: Fri Oct 28 00:55:45 2022 +0100 freebsd: simplify MD isa_defs.h Most of this file was a pile of defines, apparently from Solaris that controlled nothing in the source tree. A few things controlled the definition of unused types or macros which I have removed. Considerable further cleanup is possible including removal of architectures FreeBSD never supported. This file should likely converge with the Linux version to the extent possible. Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14127 commit f1db32a58aa14a879694b2976a9a5cc12b84a507 Author: Brooks Davis Date: Fri Oct 28 00:41:53 2022 +0100 freebsd: trim dkio.h to the minimum Only DKIOCFLUSHWRITECACHE is required. Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14127 commit 7c4b19b6332bfaddfcfe03d0e9b562c6d5e7ae94 Author: Brooks Davis Date: Thu Oct 27 22:45:44 2022 +0100 freebsd: add ifdefs around legacy ioctl support Require that ZFS_LEGACY_SUPPORT be defined for legacy ioctl support to be built. For now, define it in zfs_ioctl_compat.h so support is always built. This will allow systems that need never support pre-openzfs tools a mechanism to remove support at build time. This code should be removed once the need for tool compatability is gone. No functional change at this time. Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14127 commit 91ba107c7f56880200ecb5b5e9c894bbc7448a1a Author: Brooks Davis Date: Thu Oct 27 22:28:55 2022 +0100 freebsd: remove no-op vn_renamepath() vn_renamepath() is a Solaris-ism that was defined away in the FreeBSD port. Now that the only use is in the FreeBSD zfs_vnops_os.c, drop it entierly. Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14127 commit ea2b9a3ccde96658d3c8df8629610f80b5ca9734 Author: Brooks Davis Date: Thu Oct 27 22:24:42 2022 +0100 freebsd: remove unused vn_rename() Reviewed-by: Ryan Moeller Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14127 commit 6b36b8f19ba44dd25e15be4e41c6c56f0b6f98c5 Author: Ameer Hamza <106930537+ixhamza@users.noreply.github.com> Date: Fri Nov 4 23:33:47 2022 +0500 zed: Prevent special vdev to be replaced by hot spare Special vdevs should not be replaced by a hot spare. Log vdevs already support this, extending the functionality for special vdevs. Reviewed-by: Ryan Moeller Reviewed-by: Tony Hutter Reviewed-by: Richard Yao Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Ameer Hamza Closes #14129 commit f97bb622811bc9548f36f841a12a7ae56b835c8e Author: Richard Yao Date: Fri Nov 4 14:06:14 2022 -0400 FreeBSD: Fix out of bounds read in zfs_ioctl_ozfs_to_legacy() There is an off by 1 error in the check. Fortunately, this function does not appear to be used in kernel space, despite being compiled as part of the kernel module. However, it is used in userspace. Callers of lzc_ioctl_fd() likely will crash if they attempt to use the unimplemented request number. This was reported by FreeBSD's coverity scan. Reported-by: Coverity (CID 1432059) Reviewed-by: Ryan Moeller Reviewed-by: Damian Szuberski Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14135 commit 0531a99bc5ce7460371229c262d66333a32fbee3 Author: Serapheim Dimitropoulos Date: Thu Nov 3 15:02:46 2022 -0700 Expose zfs_vdev_open_timeout_ms as a tunable Some of our customers have been occasionally hitting zfs import failures in Linux because udevd doesn't create the by-id symbolic links in time for zpool import to use them. The main issue is that the systemd-udev-settle.service that zfs-import-cache.service and other services depend on is racy. There is also an openzfs issue filed (see https://github.com/openzfs/zfs/issues/10891) outlining the problem and potential solutions. With the proper solutions being significant in terms of complexity and the priority of the issue being low for the time being, this patch exposes `zfs_vdev_open_timeout_ms` as a tunable so people that are experiencing this issue often can increase it as a workaround. Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Reviewed-by: Alexander Motin Reviewed-by: Don Brady Reviewed-by: Brian Behlendorf Signed-off-by: Serapheim Dimitropoulos Closes #14133 commit 7c8495698223407df7fca5db3fb6b72f7b9a34d7 Author: Allan Jude Date: Thu Nov 3 14:53:24 2022 -0400 Allow mounting snapshots in .zfs/snapshot as a regular user Rather than doing a terrible credential swapping hack, we just check that the thing being mounted is a snapshot, and the mountpoint is the zfsctl directory, then we allow it. If the mount attempt is from inside a jail, on an unjailed dataset (mounted from the host, not by the jail), the ability to mount the snapshot is controlled by a new per-jail parameter: zfs.mount_snapshot Reviewed-by: Brian Behlendorf Co-authored-by: Ryan Moeller Signed-off-by: Ryan Moeller Signed-off-by: Allan Jude Sponsored-by: Modirum MDPay Sponsored-by: Klara Inc. Closes #13758 commit 138d92e96cad8c0f5fa931268c6c24da8feeaade Author: Richard Yao Date: Thu Nov 3 13:47:48 2022 -0400 Cleanup: Remove branches that always evaluate the same way Coverity reported that the ASSERT in taskq_create() is always true and the `*offp > MAXOFFSET_T` check in zfs_file_seek() is always false. We delete them as cleanup. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14130 commit b742953aeb70e3c14a917e373a45d3ff7820ad6a Author: Brooks Davis Date: Tue Nov 1 20:45:36 2022 +0000 Remove an unused variable Clang-16 detects this set-but-unused variable which is assigned and incremented, but never referenced otherwise. Reviewed-by: Matthew Ahrens Reviewed-by: Richard Yao Signed-off-by: Brooks Davis Closes #14125 commit 830b39f6b3faa5d5b627b299c70e945cd10de376 Author: Richard Yao Date: Thu Nov 3 12:58:14 2022 -0400 Address warnings about possible division by zero from clangsa * The complaint in ztest_replay_write() is only possible if something went horribly wrong. An assertion will silence this and if it goes off, we will know that something is wrong. * The complaint in spa_estimate_metaslabs_to_flush() is not impossible, but seems very unlikely. We resolve this by passing the value from the `MIN()` that does not go to infinity when the variable is zero. There was a third report from Clang's scan-build, but that was a definite false positive and disappeared when checked again through Clang's static analyzer with Z3 refution via CodeChecker. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14124 commit e955f102c713c171d409e0401797a23938f81172 Author: Brooks Davis Date: Thu Nov 3 09:57:05 2022 -0700 libuutil: deobfuscate internal pointers uu_avl and uu_list stored internal next/prev pointers and parent pointers (unused) obfuscated (byte swapped) to hide them from a long forgotten leak checker (No one at the 2022 OpenZFS developers meeting could recall the history.) This would break on CHERI systems and adds no obvious value. Rename the members, use proper types rather than uintptr_t, and eliminate the related macros. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14126 commit c8fa84c09d9c2ba797210c5166d63e446a0dad15 Author: Attila Fülöp Date: Thu Nov 3 17:55:13 2022 +0100 Deny receiving into encrypted datasets if the keys are not loaded Commit 68ddc06b611854560fefa377437eb3c9480e084b introduced support for receiving unencrypted datasets as children of encrypted ones but unfortunately got the logic upside down. This resulted in failing to deny receives of incremental sends into encrypted datasets without their keys loaded. If receiving a filesystem, the receive was done into a newly created unencrypted child dataset of the target. In case of volumes the receive made the target volume undeletable since a dataset was created below it, which we obviously can't handle. Incremental streams with embedded blocks are affected as well. We fix the broken logic to properly deny receives in such cases. Reviewed-by: Brian Behlendorf Signed-off-by: Attila Fülöp Closes #13598 Closes #14055 Closes #14119 commit 4211949b388d60d402d4e4800b94225a2ffab118 Author: Andrew Innes Date: Thu Nov 10 10:57:33 2022 +0800 Windows: Remove zpl_revalidate: fix snapshot rollback We were calling a function and storing the result in a struct The struct member was removed upstream, and it doesn't look like we were actually reading the value anywhere anyway so now we just call the function and don't store the result like in freebsd Signed-off-by: Andrew Innes commit 0efb59a8a3635622e2c06a8827b5686bd2586903 Author: Pavel Snajdr Date: Thu Dec 5 01:52:27 2019 +0100 Remove zpl_revalidate: fix snapshot rollback Open files, which aren't present in the snapshot, which is being roll-backed to, need to disappear from the visible VFS image of the dataset. Kernel provides d_drop function to drop invalid entry from the dcache, but inode can be referenced by dentry multiple dentries. The introduced zpl_d_drop_aliases function walks and invalidates all aliases of an inode. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Pavel Snajdr Closes #9600 Closes #14070 commit 0b097326a526bc06207b0c23a3c1b4f2427e5093 Author: Brooks Davis Date: Fri Oct 28 17:36:43 2022 +0100 linux isa_defs.h: Don't define _ALIGNMENT_REQUIRED Nothing consumes this definition so stop defining it. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14128 commit fc37dba459ee45d3cd61134224b48cc5cd583574 Author: Brooks Davis Date: Fri Oct 28 00:58:41 2022 +0100 Improve RISC-V support Check __riscv_xlen == 64 rather than _LP64 and define _LP64 if missing. Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Brooks Davis Closes #14128 commit 09528b0b63d63a38157e875ab8b957dfd3beb1b8 Author: Vince van Oosten Date: Sun Oct 23 11:11:58 2022 +0200 include overrides for zfs snapshot/rollback bootfs.service Reviewed-by: Brian Behlendorf Signed-off-by: Vince van Oosten Closes #14075 Closes #14076 commit 48240cdc886de692008de045d6aedbd646a3946c Author: Vince van Oosten Date: Sun Oct 23 11:11:18 2022 +0200 include overrides for zfs-import.target Reviewed-by: Brian Behlendorf Signed-off-by: Vince van Oosten Closes #14075 Closes #14076 commit 54f7a95b641c7bd9f46df05625f9c095e7bf95c4 Author: Vince van Oosten Date: Sun Oct 23 10:55:46 2022 +0200 include systemd overrides to zfs-dracut module If a user that uses systemd and dracut wants to overide certain settings, they typically use `systemctl edit [unit]` or place a file in `/etc/systemd/system/[unit].d/override.conf` directly. The zfs-dracut module did not include those overrides however, so this did not have any effect at boot time. For zfs-import-scan.service and zfs-import-cache.service, overrides are now included in the dracut initramfs image. Reviewed-by: Brian Behlendorf Signed-off-by: Vince van Oosten Closes #14075 Closes #14076 commit fdd80e0bf0b899b1ef3722833e3872900117a769 Author: Ryan Moeller Date: Tue Nov 1 15:19:32 2022 -0400 zil: Relax assertion in zil_parse Rather than panic debug builds when we fail to parse a whole ZIL, let's instead improve the logging of errors and continue like in a release build. Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Ryan Moeller Closes #14116 commit 7a866bc5f019d37c4ccc551dda0de7459bc7087f Author: youzhongyang Date: Tue Nov 1 15:08:37 2022 -0400 ZTS: rsend_009_pos.ksh is destructive on zfs-on-root system Reviewed-by: Brian Behlendorf Reviewed-by: Allan Jude Reviewed-by: George Melikov Signed-off-by: Youzhong Yang Closes #14113 commit a53ab8f36156c551122da6c098df46a5c4b9bfaa Author: Richard Yao Date: Mon Oct 31 13:01:04 2022 -0400 Fix oversights from 4170ae4e 4170ae4ea600fea6ac9daa8b145960c9de3915fc was intended to tackle TOCTOU race conditions reported by CodeQL, but as an oversight, a file descriptor was not closed and some comments were not updated. Interestingly, CodeQL did not complain about the file descriptor leak, so there is room for improvement in how we configure it to try to detect this issue so that we get early warning about this. In addition, an optimization opportunity was missed by mistake in lib/libshare/os/linux/smb.c, which prevented us from truly closing the TOCTOU race. This was also caught by Coverity. Reported-by: Coverity (CID 1524424) Reported-by: Coverity (CID 1526804) Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14109 commit f19800796bdd0a0c75d4e4c05a74c323b20359a3 Author: Allan Jude Date: Sat Oct 29 16:08:54 2022 -0400 Avoid null pointer dereference in dsl_fs_ss_limit_check() Check for cr == NULL before dereferencing it in dsl_enforce_ds_ss_limits() to lookup the zone/jail ID. Reported-by: Coverity (CID 1210459) Reviewed-by: Richard Yao Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude Closes #14103 commit 93b738d5da31fea112029ab505397054fd0465a6 Author: Andrew Innes Date: Thu Nov 10 11:30:44 2022 +0800 Windows: zfs_rename: support RENAME_* flags Signed-off-by: Andrew Innes commit 13c127a7697e6aa32a0716a48871c628be4eb04f Author: Aleksa Sarai Date: Sat Jun 22 10:35:11 2019 +1000 zfs_rename: support RENAME_* flags Implement support for Linux's RENAME_* flags (for renameat2). Aside from being quite useful for userspace (providing race-free ways to exchange paths and implement mv --no-clobber), they are used by overlayfs and are thus required in order to use overlayfs-on-ZFS. In order for us to represent the new renameat2(2) flags in the ZIL, we create two new transaction types for the two flags which need transactional-level support (RENAME_EXCHANGE and RENAME_WHITEOUT). RENAME_NOREPLACE does not need any ZIL support because we know that if the operation succeeded before creating the ZIL entry, there was no file to be clobbered and thus it can be treated as a regular TX_RENAME. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Reviewed-by: Pavel Snajdr Signed-off-by: Aleksa Sarai Closes #12209 Closes #14070 commit 085d34952bad280bae51ee48c763f5c0ace01370 Author: Aleksa Sarai Date: Fri Apr 26 23:23:07 2019 +1000 zfs_rename: restructure to have cleaner fallbacks This is in preparation for RENAME_EXCHANGE and RENAME_WHITEOUT support for ZoL, but the changes here allow for far nicer fallbacks than the previous implementation (the source and target are re-linked in case of the final link failing). In addition, a small cleanup was done for the "target exists but is a different type" codepath so that it's more understandable. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Aleksa Sarai Closes #12209 Closes #14070 commit 4ce810e411dd2135772c5cdec32d91c8f422efd7 Author: Aleksa Sarai Date: Wed May 18 20:29:33 2022 +1000 debug: add VERIFY_{IMPLY,EQUIV} variants This allows for much cleaner VERIFY-level assertions. Reviewed-by: Ryan Moeller Reviewed-by: Alexander Motin Reviewed-by: Brian Behlendorf Signed-off-by: Aleksa Sarai Closes #14070 commit af415b7ae1cc83df6532f6111e2fe0c34e135a85 Author: Richard Yao Date: Thu Oct 27 15:41:39 2022 -0400 Cleanup dump_bookmarks() Assertions are meant to check assumptions, but the way that this assertion is written does not check an assumption, since it is provably always true. Removing the assertion will cause a compiler warning (made into an error by -Werror) about printing up to 512 bytes to a 256-byte buffer, so instead, we change the assertion to verify the assumption that we never do a snprintf() that is truncated to avoid overrunning the 256-byte buffer. This was caught by an audit of the codebase to look for misuse of `snprintf()` after CodeQL reported that we had misused `snprintf()`. An explanation of how snprintf() can be misused is here: https://www.redhat.com/en/blog/trouble-snprintf This particular instance did not misuse `snprintf()`, but it was caught by the audit anyway. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14098 commit df87bac43da6829f289c4b175e3f08bde43cfdf7 Author: Richard Yao Date: Thu Oct 27 12:45:26 2022 -0400 Fix too few arguments to formatting function CodeQL reported that when the VERIFY3U condition is false, we do not pass enough arguments to `spl_panic()`. This is because the format string from `snprintf()` was concatenated into the format string for `spl_panic()`, which causes us to have an unexpected format specifier. A CodeQL developer suggested fixing the macro to have a `%s` format string that takes a stringified RIGHT argument, which would fix this. However, upon inspection, the VERIFY3U check was never necessary in the first place, so we remove it in favor of just calling `snprintf()`. Lastly, it is interesting that every other static analyzer run on the codebase did not catch this, including some that made an effort to catch such things. Presumably, all of them relied on header annotations, which we have not yet done on `spl_panic()`. CodeQL apparently is able to track the flow of arguments on their way to annotated functions, which llowed it to catch this when others did not. A future patch that I have in development should annotate `spl_panic()`, so the others will catch this too. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14098 commit 24d978c6caca8d21062a04843c231a01df9014ba Author: Richard Yao Date: Thu Oct 27 11:03:48 2022 -0400 Fix TOCTOU race conditions reported by CodeQL and Coverity CodeQL and Coverity both complained about: * lib/libshare/os/linux/smb.c * tests/zfs-tests/cmd/mmapwrite.c * twice * tests/zfs-tests/tests/functional/tmpfile/tmpfile_002_pos.c * tests/zfs-tests/tests/functional/tmpfile/tmpfile_stat_mode.c * coverity had a second complaint that CodeQL did not have * tests/zfs-tests/cmd/suid_write_to_file.c * Coverity had two complaints and CodeQL had one complaint, both differed. The CodeQL complaint is about the main point of the test, so it is not fixable without a hack involving `fork()`. The issues reported by CodeQL are fixed, with the exception of the last one, which is deemed to be a false positive that is too much trouble to wrokaround. The issues reported by Coverity were only fixed if CodeQL complained about them. There were issues reported by Coverity in a number of other files that were not reported by CodeQL, but fixing the CodeQL complaints is considered a priority since we want to integrate it into a github workflow, so the remaining Coverity complaints are left for future work. Reviewed-by: Brian Behlendorf Signed-off-by: Richard Yao Closes #14098 commit c188781a38bf27a936e5bdade378cec821b1e848 Author: Brian Behlendorf Date: Fri Oct 28 13:25:37 2022 -0700 Revert "Cleanup: Delete dead code from send_merge_thread()" This reverts commit fb823de9f due to a regression. It is in fact possible for the range->eos_marker to be false on error. Reviewed-by: Richard Yao Signed-off-by: Brian Behlendorf Issue #14042 Closes #14104 commit fe349dbd6ba7dbcfebdcf5b60e648962afb056d6 Author: Rob N ★ Date: Sat Oct 29 05:46:44 2022 +1100 debug: fix output from VERIFY0 assertion The previous version reported all the right info, but the VERIFY3 name made a little more confusing when looking for the matching location in the source code. Reviewed-by: Brian Behlendorf Reviewed-by: Richard Yao Signed-off-by: Rob N ★ Closes #14099 commit 7aa87bead97edd3f282de6d7c014784c4af58745 Author: Mariusz Zaborski Date: Fri Oct 28 20:44:18 2022 +0200 quota: extend quota for dataset This patch relax the quota limitation for dataset by around 3%. What this means is that user can write more data then the quota is set to. However thanks to that we can get more stable bandwidth, in case when we are overwriting data in-place, and not consuming any additional space. Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude Signed-off-by: Mariusz Zaborski Sponsored-by: Zededa Inc. Sponsored-by: Klara Inc. Closes #13839 commit d9edd7b208ebb024ea31f90f6736fae5c44a7ee7 Author: shodanshok Date: Fri Oct 28 19:21:54 2022 +0200 Fix ARC target collapse when zfs_arc_meta_limit_percent=100 Reclaim metadata when arc_available_memory < 0 even if meta_used is not bigger than arc_meta_limit. As described in https://github.com/openzfs/zfs/issues/14054 if zfs_arc_meta_limit_percent=100 then ARC target can collapse to arc_min due to arc_purge not freeing any metadata. This patch lets arc_prune to do its work when arc_available_memory is negative even if meta_used is not bigger than arc_meta_limit, avoiding ARC target collapse. Reviewed-by: Brian Behlendorf Signed-off-by: Gionatan Danti Closes #14054 Closes #14093 commit ce32656ea92df58b7a26ca0f1488b1707560c021 Author: vaclavskala <33496485+vaclavskala@users.noreply.github.com> Date: Fri Oct 28 19:16:31 2022 +0200 Propagate extent_bytes change to autotrim thread The autotrim thread only reads zfs_trim_extent_bytes_min and zfs_trim_extent_bytes_max variable only on thread start. We should check for parameter changes during thread execution to allow parameter changes take effect without needing to disable then restart the autotrim. Reviewed-by: Brian Behlendorf Signed-off-by: Václav Skála Closes #14077 commit 24fdf4484c02330fe5fd59b219e4adeb2bb03b04 Author: Andrew Innes Date: Thu Oct 27 12:38:49 2022 +0800 Windows: use umem_free_aligned Also remove our posix_memalign_free Also include umem.h Signed-off-by: Andrew Innes commit e5169410dfe0d4555bdd49e6fd375f9c5441165f Author: Andrew Innes Date: Tue Nov 8 18:07:38 2022 +0800 clean up windows-build-test.yml Add test for alternate data streams Change it so that only passing tests run by default use the newer github env setting method update to suisei-cn/actions-download-file@v1.3.0 Signed-off-by: Andrew Innes commit 650f78e2575eb42c0e65db8ce5a88352af5cd360 Author: Jorgen Lundman Date: Tue Nov 8 08:33:15 2022 +0900 Promote to beta. Update ReadMe. Signed-off-by: Jorgen Lundman commit e0343504032d111f1549ff90eeb031871f379e0b Author: Andrew Innes Date: Thu Oct 27 14:17:45 2022 +0800 cstyle fixes Signed-off-by: Andrew Innes commit b422ce6c64e97d46dd656896a85ed5b6ad118499 Author: Andrew Innes Date: Wed Oct 26 12:24:32 2022 +0800 Windows: Add CodeQL workflow Signed-off-by: Andrew Innes commit 9ec6ff14dc5daf11ca2f45e04c687be2ce35139a Author: Andrew Innes Date: Thu Oct 20 18:23:10 2022 +0800 Windows: Support idmapped mount Signed-off-by: Andrew Innes commit c1e2bc3b820b18cacf246aa3ba61cea150d1dfe3 Author: Jorgen Lundman Date: Fri Oct 21 16:44:38 2022 +0900 Add Windows assembler: gcm_pclmulqdq commit 805afe31b0daf23c6a91817a75b81da1217945cf Author: Andrew Innes Date: Wed Oct 19 09:30:35 2022 +0800 Clean up workflow Signed-off-by: Andrew Innes commit a209216221e6b81d38deff57016f237c289ada18 Author: Jorgen Lundman Date: Fri Oct 21 16:11:41 2022 +0900 Add Windows assembler: aes_amd64 commit 023c0507a90e83c1938d27657dba30cad7fe089d Author: Jorgen Lundman Date: Fri Oct 21 15:18:23 2022 +0900 Add Windows assembler: sha256, sha512, aesNI commit 0aa54ef65dc1fb4ff4b1cc762d64b1f1cfba8833 Author: Jorgen Lundman Date: Fri Oct 21 15:10:03 2022 +0900 Code cleanup for PR. Remove possible existing dprintf We defined dprintf in spl, so a simpler version can be used there, but when ZFS headers are loaded, the proper dprintf is used. Simply remove any existing dprintf before changing to avoid warnings. Fixed changed to Linux that were unintended Move Windows specific things from common headers into os/ commit efd88f5b7e90b7e5901862b59ed5856e72534e11 Author: Andrew Innes Date: Wed Oct 12 15:22:09 2022 +0800 Add winfstest to windows-build-test.yml Signed-off-by: Andrew Innes commit d5d61bc43a41947c928f5c63b3fbf2864abd28c3 Author: Andrew Innes Date: Wed Oct 19 15:05:15 2022 +0800 Windows: Fix declarations of non-global variables Signed-off-by: Andrew Innes commit 4ffe415f6ebba5056dd99deb0fe5ddd334f57cb3 Author: Jorgen Lundman Date: Tue Oct 18 15:57:22 2022 +0900 Minor cstyle correction commit 8ab43b4233cd388a35e9fdf7502ee493cb82a95c Author: Jorgen Lundman Date: Tue Oct 18 15:10:01 2022 +0900 Review corrections, cleanup. We had two versions of redact does not call bqueue destroy Left over module tunables Undo slash correction for Linux Signed-off-by: Jorgen Lundman commit dcf2bebd9ffede0c65ffe4c39c7ac6553a65d8f4 Author: Jorgen Lundman Date: Mon Oct 17 14:06:47 2022 +0900 Update ABI changes Signed-off-by: Jorgen Lundman commit 9e436612f3c44ae7b5c6ceed6481df3a1292c867 Author: Andrew Innes Date: Mon Oct 17 16:36:46 2022 +0800 add copyright Signed-off-by: Andrew Innes commit 877fab9ba5af3e27ac9f532fb297e4fa2802e6ef Author: Jorgen Lundman Date: Mon Oct 17 11:38:36 2022 +0900 cstyle corrections for Windows Signed-off-by: Jorgen Lundman commit f5ebabe2d03855d5c83917a18c3cd53c6b3a497c Author: Jorgen Lundman Date: Fri Jul 31 10:32:26 2020 +0900 Add Windows support to OpenZFS Signed-off-by: Jorgen Lundman --- .github/workflows/codeql-windows.yml | 69 + .github/workflows/windows-build-test.yml | 1937 ++++ .gitignore | 3 +- AUTHORS | 1 + CMakeLists.txt | 97 + CMakeSettings.json | 28 + Makefile.am | 1 + README.md | 7 + cmd/CMakeLists.txt | 13 + cmd/os/windows/kstat/CMakeLists.txt | 13 + cmd/os/windows/kstat/kstat.c | 1723 +++ cmd/os/windows/kstat/kstat.h | 259 + cmd/os/windows/kstat/resource.h | 15 + cmd/os/windows/kstat/resource.rc | 110 + cmd/os/windows/zfsinstaller/CMakeLists.txt | 15 + cmd/os/windows/zfsinstaller/resource.h | 15 + cmd/os/windows/zfsinstaller/resource.rc | 109 + cmd/os/windows/zfsinstaller/zfsinstaller.cpp | 904 ++ cmd/os/windows/zfsinstaller/zfsinstaller.h | 39 + cmd/raidz_test/CMakeLists.txt | 22 + cmd/raidz_test/os/windows/resource.h | 15 + cmd/raidz_test/os/windows/resource.rc | 111 + cmd/raidz_test/raidz_test.c | 3 + cmd/zdb/CMakeLists.txt | 22 + cmd/zdb/os/windows/resource.h | 15 + cmd/zdb/os/windows/resource.rc | 110 + cmd/zdb/zdb.c | 10 +- cmd/zfs/CMakeLists.txt | 23 + cmd/zfs/os/windows/resource.h | 15 + cmd/zfs/os/windows/resource.rc | 110 + cmd/zfs/zfs_main.c | 34 +- cmd/zpool/CMakeLists.txt | 28 + cmd/zpool/os/windows/resource.h | 15 + cmd/zpool/os/windows/resource.rc | 109 + cmd/zpool/os/windows/zpool_vdev_os.c | 171 + cmd/zpool/zpool_util.c | 2 + cmd/zpool/zpool_vdev.c | 33 +- cmd/zstream/CMakeLists.txt | 26 + cmd/zstream/os/windows/resource.h | 15 + cmd/zstream/os/windows/resource.rc | 110 + cmd/zstream/zstream_redup.c | 2 + contrib/windows/Inno.Setup/HowToDebug.txt | 140 + .../Inno.Setup/OPENSOLARIS.LICENSE.txt | 384 + .../windows/Inno.Setup/ZFSInstall-debug.iss | 121 + contrib/windows/Inno.Setup/environment.iss | 45 + contrib/windows/Inno.Setup/openzfs-large.bmp | Bin 0 -> 154542 bytes contrib/windows/Inno.Setup/openzfs-small.bmp | Bin 0 -> 8965254 bytes contrib/windows/Inno.Setup/openzfs.ico | Bin 0 -> 318431 bytes contrib/windows/Inno.Setup/readme.txt | 14 + contrib/windows/OpenZFS/OpenZFS.sln | 51 + .../windows/OpenZFS/OpenZFS/OpenZFS.vcxproj | 233 + contrib/windows/TestCert/README.md | 15 + .../TestCert/create_test_sign_cert.ps1 | 28 + .../TestCert/test_sign_cert_nopass.pfx | Bin 0 -> 2259 bytes .../windows/TestCert/test_sign_cert_pass.pfx | Bin 0 -> 2646 bytes contrib/windows/cmake/FindOpenSSL.cmake | 675 ++ .../cmake/FindPackageHandleStandardArgs.cmake | 605 ++ .../windows/cmake/FindPackageMessage.cmake | 48 + contrib/windows/cmake/FindWdk.cmake | 218 + .../cmake/GetGitRevisionDescription.cmake | 284 + .../cmake/GetGitRevisionDescription.cmake.in | 43 + .../cmake/SelectLibraryConfigurations.cmake | 80 + contrib/windows/parsedump/README.md | 16 + contrib/windows/parsedump/parsedump.bat | 34 + contrib/windows/parsedump/parsedump.py | 47 + include/libzfs.h | 3 + include/libzutil.h | 8 + include/os/windows/Trace.h | 89 + include/os/windows/spl/rpc/types.h | 29 + include/os/windows/spl/rpc/xdr.h | 171 + include/os/windows/spl/spl-debug.h | 61 + include/os/windows/spl/sys/acl.h | 128 + include/os/windows/spl/sys/acl_impl.h | 5 + include/os/windows/spl/sys/atomic.h | 212 + include/os/windows/spl/sys/attr.h | 25 + include/os/windows/spl/sys/avl_impl.h | 144 + include/os/windows/spl/sys/bootconf.h | 25 + include/os/windows/spl/sys/bootprops.h | 25 + include/os/windows/spl/sys/buf.h | 25 + include/os/windows/spl/sys/byteorder.h | 71 + include/os/windows/spl/sys/callb.h | 67 + include/os/windows/spl/sys/cmn_err.h | 60 + include/os/windows/spl/sys/compress.h | 25 + include/os/windows/spl/sys/condvar.h | 125 + include/os/windows/spl/sys/conf.h | 25 + include/os/windows/spl/sys/console.h | 30 + include/os/windows/spl/sys/cpupart.h | 25 + include/os/windows/spl/sys/cpuvar.h | 25 + include/os/windows/spl/sys/crc32.h | 25 + include/os/windows/spl/sys/cred.h | 59 + include/os/windows/spl/sys/ctype.h | 27 + include/os/windows/spl/sys/ddi.h | 25 + include/os/windows/spl/sys/debug.h | 251 + include/os/windows/spl/sys/dirent.h | 57 + include/os/windows/spl/sys/disp.h | 25 + include/os/windows/spl/sys/dkio.h | 15 + include/os/windows/spl/sys/dklabel.h | 25 + include/os/windows/spl/sys/dnlc.h | 38 + include/os/windows/spl/sys/dumphdr.h | 25 + include/os/windows/spl/sys/errno.h | 160 + include/os/windows/spl/sys/extdirent.h | 7 + include/os/windows/spl/sys/fcntl.h | 61 + include/os/windows/spl/sys/file.h | 54 + include/os/windows/spl/sys/ia32/asm_linkage.h | 181 + include/os/windows/spl/sys/idmap.h | 27 + include/os/windows/spl/sys/int_limits.h | 25 + include/os/windows/spl/sys/int_types.h | 27 + include/os/windows/spl/sys/inttypes.h | 25 + include/os/windows/spl/sys/ioctl.h | 30 + include/os/windows/spl/sys/isa_defs.h | 689 ++ include/os/windows/spl/sys/kidmap.h | 27 + include/os/windows/spl/sys/kmem.h | 160 + include/os/windows/spl/sys/kmem_cache.h | 25 + include/os/windows/spl/sys/kmem_impl.h | 501 + include/os/windows/spl/sys/kstat.h | 370 + include/os/windows/spl/sys/linker_set.h | 163 + include/os/windows/spl/sys/list.h | 143 + include/os/windows/spl/sys/lookasidelist.h | 51 + include/os/windows/spl/sys/md5.h | 70 + include/os/windows/spl/sys/md5_consts.h | 133 + include/os/windows/spl/sys/misc.h | 22 + include/os/windows/spl/sys/mntent.h | 25 + include/os/windows/spl/sys/mod_os.h | 344 + include/os/windows/spl/sys/mount.h | 141 + include/os/windows/spl/sys/mutex.h | 108 + include/os/windows/spl/sys/note.h | 25 + include/os/windows/spl/sys/param.h | 34 + include/os/windows/spl/sys/policy.h | 88 + include/os/windows/spl/sys/priv.h | 525 + include/os/windows/spl/sys/proc.h | 46 + include/os/windows/spl/sys/processor.h | 29 + include/os/windows/spl/sys/procfs_list.h | 66 + include/os/windows/spl/sys/random.h | 49 + include/os/windows/spl/sys/resource.h | 25 + include/os/windows/spl/sys/rwlock.h | 71 + include/os/windows/spl/sys/sched.h | 27 + include/os/windows/spl/sys/seg_kmem.h | 92 + include/os/windows/spl/sys/sid.h | 103 + include/os/windows/spl/sys/signal.h | 83 + include/os/windows/spl/sys/simd.h | 757 ++ include/os/windows/spl/sys/stat.h | 84 + include/os/windows/spl/sys/string.h | 29 + include/os/windows/spl/sys/stropts.h | 168 + include/os/windows/spl/sys/sunddi.h | 194 + include/os/windows/spl/sys/sunldi.h | 32 + include/os/windows/spl/sys/sysevent.h | 36 + .../os/windows/spl/sys/sysevent/eventdefs.h | 140 + include/os/windows/spl/sys/sysmacros.h | 253 + include/os/windows/spl/sys/systeminfo.h | 33 + include/os/windows/spl/sys/systm.h | 125 + include/os/windows/spl/sys/taskq.h | 121 + include/os/windows/spl/sys/taskq_impl.h | 181 + include/os/windows/spl/sys/thread.h | 103 + include/os/windows/spl/sys/time.h | 116 + include/os/windows/spl/sys/timer.h | 77 + include/os/windows/spl/sys/trace.h | 25 + include/os/windows/spl/sys/trace_zfs.h | 1 + include/os/windows/spl/sys/tsd.h | 57 + include/os/windows/spl/sys/types.h | 188 + include/os/windows/spl/sys/types32.h | 13 + include/os/windows/spl/sys/uio.h | 194 + include/os/windows/spl/sys/unistd.h | 25 + include/os/windows/spl/sys/utfconv.h | 26 + include/os/windows/spl/sys/utsname.h | 47 + include/os/windows/spl/sys/varargs.h | 27 + include/os/windows/spl/sys/vfs.h | 82 + include/os/windows/spl/sys/vfs_opreg.h | 25 + include/os/windows/spl/sys/vmem.h | 178 + include/os/windows/spl/sys/vmem_impl.h | 155 + include/os/windows/spl/sys/vmsystm.h | 29 + include/os/windows/spl/sys/vnode.h | 552 + include/os/windows/spl/sys/wmsum.h | 72 + include/os/windows/spl/sys/zmod.h | 121 + include/os/windows/spl/sys/zone.h | 34 + include/os/windows/spl/unistd.h | 25 + include/os/windows/spl_config.h | 102 + include/os/windows/zfs/sys/fs/zfsdi.h | 146 + include/os/windows/zfs/sys/policy.h | 25 + include/os/windows/zfs/sys/vdev_disk_os.h | 38 + include/os/windows/zfs/sys/wzvol.h | 493 + include/os/windows/zfs/sys/wzvolwmi.h | 4884 +++++++++ include/os/windows/zfs/sys/zconf.h | 119 + include/os/windows/zfs/sys/zfs_bootenv_os.h | 25 + include/os/windows/zfs/sys/zfs_context_os.h | 219 + include/os/windows/zfs/sys/zfs_ctldir.h | 195 + include/os/windows/zfs/sys/zfs_dir.h | 82 + include/os/windows/zfs/sys/zfs_ioctl_compat.h | 217 + include/os/windows/zfs/sys/zfs_mount.h | 73 + include/os/windows/zfs/sys/zfs_vfsops_os.h | 306 + include/os/windows/zfs/sys/zfs_vnops_os.h | 168 + include/os/windows/zfs/sys/zfs_windows.h | 221 + include/os/windows/zfs/sys/zfs_znode_impl.h | 217 + include/os/windows/zfs/sys/zlib.h | 1356 +++ include/os/windows/zfs/sys/zpl.h | 27 + include/os/windows/zfs/sys/zvol_os.h | 83 + include/os/windows/zfs/zfs_config.h | 152 + include/sys/abd.h | 4 +- include/sys/abd_impl.h | 3 + include/sys/fm/fs/zfs.h | 2 + include/sys/fs/zfs.h | 26 +- include/sys/mntent.h | 3 + include/sys/vdev_file.h | 4 + include/sys/zfs_bootenv.h | 2 + include/sys/zfs_debug.h | 1 + include/sys/zfs_file.h | 2 + include/sys/zfs_ioctl_impl.h | 4 + include/sys/zio.h | 4 + include/sys/zio_checksum.h | 4 + include/sys/zvol_impl.h | 4 + include/zfs_gitrev.h.win | 3 + lib/CMakeLists.txt | 23 + lib/libavl/CMakeLists.txt | 8 + lib/libefi/CMakeLists.txt | 7 + lib/libefi/rdwr_efi_windows.c | 1511 +++ lib/libicp/CMakeLists.txt | 64 + lib/libnvpair/CMakeLists.txt | 14 + lib/libshare/CMakeLists.txt | 12 + lib/libshare/os/windows/nfs.c | 414 + lib/libshare/os/windows/smb.c | 94 + lib/libspl/CMakeLists.txt | 35 + lib/libspl/atomic.c | 2 +- lib/libspl/include/os/windows/aio.h | 71 + lib/libspl/include/os/windows/dirent.h | 563 + lib/libspl/include/os/windows/dlfcn.h | 26 + lib/libspl/include/os/windows/err.h | 48 + lib/libspl/include/os/windows/fcntl.h | 48 + lib/libspl/include/os/windows/getopt.h | 45 + lib/libspl/include/os/windows/grp.h | 38 + lib/libspl/include/os/windows/langinfo.h | 29 + lib/libspl/include/os/windows/libgen.h | 36 + lib/libspl/include/os/windows/libintl.h | 35 + lib/libspl/include/os/windows/mntent.h | 159 + lib/libspl/include/os/windows/poll.h | 26 + lib/libspl/include/os/windows/pwd.h | 45 + lib/libspl/include/os/windows/regex.h | 145 + lib/libspl/include/os/windows/rpc/types.h | 38 + lib/libspl/include/os/windows/rpc/xdr.h | 300 + lib/libspl/include/os/windows/sched.h | 40 + lib/libspl/include/os/windows/search.h | 55 + lib/libspl/include/os/windows/signal.h | 93 + lib/libspl/include/os/windows/stdio.h | 34 + lib/libspl/include/os/windows/string.h | 76 + lib/libspl/include/os/windows/sys/byteorder.h | 218 + lib/libspl/include/os/windows/sys/errno.h | 30 + lib/libspl/include/os/windows/sys/file.h | 48 + .../include/os/windows/sys/ia32/asm_linkage.h | 183 + lib/libspl/include/os/windows/sys/ioctl.h | 28 + lib/libspl/include/os/windows/sys/kstat.h | 852 ++ lib/libspl/include/os/windows/sys/mman.h | 38 + lib/libspl/include/os/windows/sys/mnttab.h | 87 + lib/libspl/include/os/windows/sys/mount.h | 122 + lib/libspl/include/os/windows/sys/param.h | 66 + lib/libspl/include/os/windows/sys/resource.h | 26 + lib/libspl/include/os/windows/sys/socket.h | 31 + lib/libspl/include/os/windows/sys/stat.h | 99 + lib/libspl/include/os/windows/sys/sysmacros.h | 119 + lib/libspl/include/os/windows/sys/time.h | 104 + lib/libspl/include/os/windows/sys/timer.h | 26 + lib/libspl/include/os/windows/sys/types.h | 225 + lib/libspl/include/os/windows/sys/types32.h | 109 + lib/libspl/include/os/windows/sys/uio.h | 52 + lib/libspl/include/os/windows/sys/utsname.h | 41 + lib/libspl/include/os/windows/sys/vfs.h | 26 + lib/libspl/include/os/windows/sys/wait.h | 26 + .../include/os/windows/sys/zfs_context_os.h | 52 + lib/libspl/include/os/windows/sys/zfs_mount.h | 74 + lib/libspl/include/os/windows/termios.h | 50 + lib/libspl/include/os/windows/unistd.h | 92 + lib/libspl/include/os/windows/uuid/uuid.h | 108 + lib/libspl/include/os/windows/wosix.h | 168 + lib/libspl/include/rpc/xdr.h | 2 + lib/libspl/include/sys/debug.h | 21 + lib/libspl/include/sys/uio.h | 5 +- lib/libspl/os/windows/gethostid.c | 33 + lib/libspl/os/windows/getmntany.c | 640 ++ lib/libspl/os/windows/getopt.c | 314 + lib/libspl/os/windows/getoptl.c | 152 + lib/libspl/os/windows/page.c | 41 + lib/libspl/os/windows/posix.c | 1825 ++++ lib/libspl/os/windows/signal.c | 92 + lib/libspl/os/windows/uio.c | 113 + lib/libspl/os/windows/weakpragma.c | 43 + lib/libspl/os/windows/xdr.c | 729 ++ lib/libspl/os/windows/xdr_array.c | 145 + lib/libspl/os/windows/xdr_float.c | 319 + lib/libspl/os/windows/xdr_mem.c | 267 + lib/libspl/os/windows/zone.c | 28 + lib/libtpool/CMakeLists.txt | 8 + lib/libunicode/CMakeLists.txt | 9 + lib/libuutil/CMakeLists.txt | 13 + lib/libzfs/CMakeLists.txt | 38 + lib/libzfs/libzfs.abi | 31 +- lib/libzfs/libzfs_crypto.c | 10 + lib/libzfs/libzfs_dataset.c | 19 +- lib/libzfs/libzfs_iter.c | 16 +- lib/libzfs/libzfs_mount.c | 3 +- lib/libzfs/libzfs_pool.c | 5 +- lib/libzfs/libzfs_util.c | 3 +- lib/libzfs/os/freebsd/libzfs_compat.c | 7 + lib/libzfs/os/freebsd/libzfs_zmount.c | 16 + lib/libzfs/os/linux/libzfs_mount_os.c | 16 + lib/libzfs/os/linux/libzfs_util_os.c | 7 + lib/libzfs/os/windows/libzfs_mount_os.c | 467 + lib/libzfs/os/windows/libzfs_pool_os.c | 343 + lib/libzfs/os/windows/libzfs_util_os.c | 297 + lib/libzfs_core/CMakeLists.txt | 8 + lib/libzfs_core/libzfs_core.c | 12 + .../os/windows/libzfs_core_ioctl.c | 118 + lib/libzfsbootenv/CMakeLists.txt | 12 + lib/libzpool/CMakeLists.txt | 191 + lib/libzpool/kernel.c | 44 + lib/libzpool/util.c | 4 +- lib/libzstd/CMakeLists.txt | 31 + lib/libzutil/CMakeLists.txt | 27 + lib/libzutil/os/windows/zutil_compat.c | 27 + .../os/windows/zutil_device_path_os.c | 223 + lib/libzutil/os/windows/zutil_import_os.c | 1170 ++ lib/libzutil/zutil_device_path.c | 12 +- lib/libzutil/zutil_import.h | 2 + lib/os/windows/CMakeLists.txt | 5 + lib/os/windows/libkstat/CMakeLists.txt | 8 + lib/os/windows/libkstat/gmatch.c | 167 + lib/os/windows/libkstat/kstat.c | 386 + lib/os/windows/libkstat/kstat.h | 60 + lib/os/windows/libpthread/CMakeLists.txt | 7 + lib/os/windows/libpthread/pthread.c | 51 + lib/os/windows/libpthread/pthread.h | 1653 +++ lib/os/windows/libregex/CMakeLists.txt | 15 + lib/os/windows/libregex/COPYRIGHT | 54 + lib/os/windows/libregex/Makefile.inc | 14 + lib/os/windows/libregex/WHATSNEW | 95 + lib/os/windows/libregex/cname.h | 142 + lib/os/windows/libregex/engine.c | 1278 +++ lib/os/windows/libregex/re_format.7 | 494 + lib/os/windows/libregex/regcomp.c | 2457 +++++ lib/os/windows/libregex/regerror.c | 180 + lib/os/windows/libregex/regex.3 | 861 ++ lib/os/windows/libregex/regex2.h | 207 + lib/os/windows/libregex/regexec.c | 248 + lib/os/windows/libregex/regfree.c | 97 + lib/os/windows/libregex/regsub.c | 160 + lib/os/windows/libregex/utils.h | 84 + lib/os/windows/libuuid/CMakeLists.txt | 21 + lib/os/windows/libuuid/COPYING | 5 + lib/os/windows/libuuid/all-io.h | 83 + lib/os/windows/libuuid/c.h | 328 + lib/os/windows/libuuid/clear.c | 43 + lib/os/windows/libuuid/compare.c | 55 + lib/os/windows/libuuid/copy.c | 46 + lib/os/windows/libuuid/gen_uuid.c | 584 + lib/os/windows/libuuid/isnull.c | 48 + lib/os/windows/libuuid/pack.c | 69 + lib/os/windows/libuuid/parse.c | 80 + lib/os/windows/libuuid/randutils.c | 126 + lib/os/windows/libuuid/randutils.h | 13 + lib/os/windows/libuuid/test_uuid.c | 181 + lib/os/windows/libuuid/unpack.c | 63 + lib/os/windows/libuuid/unparse.c | 80 + lib/os/windows/libuuid/uuid.h | 108 + lib/os/windows/libuuid/uuidP.h | 61 + lib/os/windows/libuuid/uuid_time.c | 175 + lib/os/windows/libuuid/uuidd.h | 54 + lib/os/windows/zlib-1.2.13/CMakeLists.txt | 34 + lib/os/windows/zlib-1.2.13/ChangeLog | 1590 +++ lib/os/windows/zlib-1.2.13/FAQ | 368 + lib/os/windows/zlib-1.2.13/INDEX | 68 + lib/os/windows/zlib-1.2.13/README | 118 + lib/os/windows/zlib-1.2.13/adler32.c | 186 + lib/os/windows/zlib-1.2.13/compress.c | 86 + .../zlib-1.2.13/contrib/minizip/crypt.h | 132 + .../zlib-1.2.13/contrib/minizip/ioapi.c | 257 + .../zlib-1.2.13/contrib/minizip/ioapi.h | 210 + .../zlib-1.2.13/contrib/minizip/unzip.c | 2130 ++++ .../zlib-1.2.13/contrib/minizip/unzip.h | 437 + .../windows/zlib-1.2.13/contrib/minizip/zip.c | 2002 ++++ .../windows/zlib-1.2.13/contrib/minizip/zip.h | 367 + lib/os/windows/zlib-1.2.13/crc32.c | 1125 ++ lib/os/windows/zlib-1.2.13/crc32.h | 9446 +++++++++++++++++ lib/os/windows/zlib-1.2.13/deflate.c | 2217 ++++ lib/os/windows/zlib-1.2.13/deflate.h | 346 + lib/os/windows/zlib-1.2.13/doc/algorithm.txt | 209 + lib/os/windows/zlib-1.2.13/gzclose.c | 25 + lib/os/windows/zlib-1.2.13/gzguts.h | 219 + lib/os/windows/zlib-1.2.13/gzlib.c | 639 ++ lib/os/windows/zlib-1.2.13/gzread.c | 650 ++ lib/os/windows/zlib-1.2.13/gzwrite.c | 677 ++ lib/os/windows/zlib-1.2.13/infback.c | 644 ++ lib/os/windows/zlib-1.2.13/inffast.c | 323 + lib/os/windows/zlib-1.2.13/inffast.h | 11 + lib/os/windows/zlib-1.2.13/inffixed.h | 94 + lib/os/windows/zlib-1.2.13/inflate.c | 1595 +++ lib/os/windows/zlib-1.2.13/inflate.h | 126 + lib/os/windows/zlib-1.2.13/inftrees.c | 304 + lib/os/windows/zlib-1.2.13/inftrees.h | 62 + lib/os/windows/zlib-1.2.13/test/minigzip.c | 651 ++ lib/os/windows/zlib-1.2.13/trees.c | 1181 +++ lib/os/windows/zlib-1.2.13/trees.h | 128 + lib/os/windows/zlib-1.2.13/uncompr.c | 93 + lib/os/windows/zlib-1.2.13/zconf.h | 552 + lib/os/windows/zlib-1.2.13/zconf.h.in | 547 + lib/os/windows/zlib-1.2.13/zlib-dll-res.rc | 134 + lib/os/windows/zlib-1.2.13/zlib.3 | 149 + lib/os/windows/zlib-1.2.13/zlib.h | 1935 ++++ lib/os/windows/zlib-1.2.13/zutil.c | 356 + lib/os/windows/zlib-1.2.13/zutil.h | 275 + module/CMakeLists.txt | 135 + module/icp/CMakeLists.txt | 59 + module/icp/algs/aes/aes_impl.c | 30 +- module/icp/algs/blake3/blake3_impl.c | 27 +- module/icp/algs/modes/gcm.c | 59 +- module/icp/algs/sha2/sha256_impl.c | 26 +- module/icp/algs/sha2/sha512_impl.c | 27 +- module/icp/asm-x86_64/sha2/sha256-x86_64.S | 22 +- module/icp/asm-x86_64/sha2/sha512-x86_64.S | 16 +- module/icp/illumos-crypto.c | 2 + module/icp/include/aes/aes_impl.h | 9 + module/lua/CMakeLists.txt | 55 + module/lua/ldo.c | 13 +- module/nvpair/CMakeLists.txt | 12 + module/os/windows/.gitignore | 3 + module/os/windows/CMakeLists.txt | 152 + module/os/windows/CMakeSettings.json | 27 + module/os/windows/OpenZFS.inf | 118 + module/os/windows/OpenZFS.man | 1120 ++ module/os/windows/PORTING_NOTES.txt | 89 + module/os/windows/README.md | 454 + module/os/windows/Wpp.c | 35 + module/os/windows/debug.c | 186 + module/os/windows/driver.c | 174 + module/os/windows/resource.h | 15 + module/os/windows/resource.rc | 89 + module/os/windows/spl/CMakeLists.txt | 97 + module/os/windows/spl/spl-atomic.c | 46 + module/os/windows/spl/spl-condvar.c | 292 + module/os/windows/spl/spl-cred.c | 161 + module/os/windows/spl/spl-ddi.c | 622 ++ module/os/windows/spl/spl-debug.c | 39 + module/os/windows/spl/spl-err.c | 77 + module/os/windows/spl/spl-kmem.c | 6630 ++++++++++++ module/os/windows/spl/spl-kstat.c | 2409 +++++ module/os/windows/spl/spl-list.c | 197 + module/os/windows/spl/spl-lookasidelist.c | 192 + module/os/windows/spl/spl-md5.c | 663 ++ module/os/windows/spl/spl-mount.c | 128 + module/os/windows/spl/spl-mutex.c | 205 + module/os/windows/spl/spl-policy.c | 188 + module/os/windows/spl/spl-proc.c | 36 + module/os/windows/spl/spl-proc_list.c | 168 + module/os/windows/spl/spl-processor.c | 127 + module/os/windows/spl/spl-rwlock.c | 246 + module/os/windows/spl/spl-seg_kmem.c | 248 + module/os/windows/spl/spl-taskq.c | 2868 +++++ module/os/windows/spl/spl-thread.c | 142 + module/os/windows/spl/spl-time.c | 150 + module/os/windows/spl/spl-tsd.c | 396 + module/os/windows/spl/spl-uio.c | 117 + module/os/windows/spl/spl-vmem.c | 3960 +++++++ module/os/windows/spl/spl-vnode.c | 2101 ++++ module/os/windows/spl/spl-windows.c | 773 ++ module/os/windows/spl/spl-wmsum.c | 1 + module/os/windows/spl/spl-xdr.c | 527 + module/os/windows/spl/spl-zlib.c | 198 + module/os/windows/zfs/CMakeLists.txt | 102 + module/os/windows/zfs/abd_os.c | 490 + module/os/windows/zfs/arc_os.c | 828 ++ module/os/windows/zfs/qat.c | 105 + module/os/windows/zfs/qat_compress.c | 569 + module/os/windows/zfs/qat_crypt.c | 630 ++ module/os/windows/zfs/spa_misc_os.c | 77 + module/os/windows/zfs/sysctl_os.c | 755 ++ module/os/windows/zfs/vdev_disk.c | 904 ++ module/os/windows/zfs/vdev_file.c | 502 + module/os/windows/zfs/zfs_acl.c | 3007 ++++++ module/os/windows/zfs/zfs_ctldir.c | 1425 +++ module/os/windows/zfs/zfs_debug.c | 288 + module/os/windows/zfs/zfs_dir.c | 1208 +++ module/os/windows/zfs/zfs_file_os.c | 431 + module/os/windows/zfs/zfs_fuid_os.c | 52 + module/os/windows/zfs/zfs_ioctl_os.c | 1232 +++ module/os/windows/zfs/zfs_racct.c | 32 + module/os/windows/zfs/zfs_vfsops.c | 2308 ++++ module/os/windows/zfs/zfs_vnops_os.c | 3408 ++++++ module/os/windows/zfs/zfs_vnops_windows.c | 6627 ++++++++++++ module/os/windows/zfs/zfs_vnops_windows_lib.c | 5287 +++++++++ .../os/windows/zfs/zfs_vnops_windows_mount.c | 1593 +++ .../os/windows/zfs/zfs_vnops_windows_xattr.c | 1058 ++ module/os/windows/zfs/zfs_windows_zvol.c | 1095 ++ module/os/windows/zfs/zfs_windows_zvol_scsi.c | 1124 ++ module/os/windows/zfs/zfs_windows_zvol_wmi.c | 1172 ++ module/os/windows/zfs/zfs_znode.c | 2413 +++++ module/os/windows/zfs/zio_crypt.c | 2049 ++++ module/os/windows/zfs/zvol_os.c | 1085 ++ module/unicode/CMakeLists.txt | 9 + module/zcommon/CMakeLists.txt | 21 + module/zcommon/zfs_fletcher.c | 67 +- module/zcommon/zfs_prop.c | 15 + module/zfs/CMakeLists.txt | 198 + module/zfs/dsl_scan.c | 35 + module/zfs/sa.c | 4 +- module/zfs/spa.c | 3 + module/zfs/vdev.c | 18 + module/zfs/vdev_raidz_math.c | 29 +- module/zfs/zap_micro.c | 3 + module/zfs/zfs_fuid.c | 18 + module/zfs/zfs_ioctl.c | 17 +- module/zfs/zfs_log.c | 18 + module/zfs/zvol.c | 8 +- module/zstd/CMakeLists.txt | 30 + module/zstd/include/limits.h | 2 + module/zstd/include/stddef.h | 2 + module/zstd/include/stdint.h | 2 + module/zstd/include/string.h | 2 + module/zstd/lib/common/bitstream.h | 2 +- module/zstd/lib/common/compiler.h | 4 +- module/zstd/lib/common/cpu.h | 4 +- module/zstd/lib/common/mem.h | 6 +- module/zstd/lib/common/xxhash.c | 4 +- module/zstd/lib/common/zstd_internal.h | 4 +- .../lib/compress/zstd_compress_internal.h | 8 +- .../cli_root/zfs_mount/zfs_mount_011_neg.ksh | 6 +- .../zfs_unmount/zfs_unmount_008_neg.ksh | 3 +- 521 files changed, 157878 insertions(+), 100 deletions(-) create mode 100644 .github/workflows/codeql-windows.yml create mode 100644 .github/workflows/windows-build-test.yml create mode 100644 CMakeLists.txt create mode 100644 CMakeSettings.json create mode 100644 cmd/CMakeLists.txt create mode 100644 cmd/os/windows/kstat/CMakeLists.txt create mode 100644 cmd/os/windows/kstat/kstat.c create mode 100644 cmd/os/windows/kstat/kstat.h create mode 100644 cmd/os/windows/kstat/resource.h create mode 100644 cmd/os/windows/kstat/resource.rc create mode 100644 cmd/os/windows/zfsinstaller/CMakeLists.txt create mode 100644 cmd/os/windows/zfsinstaller/resource.h create mode 100644 cmd/os/windows/zfsinstaller/resource.rc create mode 100644 cmd/os/windows/zfsinstaller/zfsinstaller.cpp create mode 100644 cmd/os/windows/zfsinstaller/zfsinstaller.h create mode 100644 cmd/raidz_test/CMakeLists.txt create mode 100644 cmd/raidz_test/os/windows/resource.h create mode 100644 cmd/raidz_test/os/windows/resource.rc create mode 100644 cmd/zdb/CMakeLists.txt create mode 100644 cmd/zdb/os/windows/resource.h create mode 100644 cmd/zdb/os/windows/resource.rc create mode 100644 cmd/zfs/CMakeLists.txt create mode 100644 cmd/zfs/os/windows/resource.h create mode 100644 cmd/zfs/os/windows/resource.rc create mode 100644 cmd/zpool/CMakeLists.txt create mode 100644 cmd/zpool/os/windows/resource.h create mode 100644 cmd/zpool/os/windows/resource.rc create mode 100644 cmd/zpool/os/windows/zpool_vdev_os.c create mode 100644 cmd/zstream/CMakeLists.txt create mode 100644 cmd/zstream/os/windows/resource.h create mode 100644 cmd/zstream/os/windows/resource.rc create mode 100644 contrib/windows/Inno.Setup/HowToDebug.txt create mode 100644 contrib/windows/Inno.Setup/OPENSOLARIS.LICENSE.txt create mode 100644 contrib/windows/Inno.Setup/ZFSInstall-debug.iss create mode 100644 contrib/windows/Inno.Setup/environment.iss create mode 100644 contrib/windows/Inno.Setup/openzfs-large.bmp create mode 100644 contrib/windows/Inno.Setup/openzfs-small.bmp create mode 100644 contrib/windows/Inno.Setup/openzfs.ico create mode 100644 contrib/windows/Inno.Setup/readme.txt create mode 100644 contrib/windows/OpenZFS/OpenZFS.sln create mode 100644 contrib/windows/OpenZFS/OpenZFS/OpenZFS.vcxproj create mode 100644 contrib/windows/TestCert/README.md create mode 100644 contrib/windows/TestCert/create_test_sign_cert.ps1 create mode 100644 contrib/windows/TestCert/test_sign_cert_nopass.pfx create mode 100644 contrib/windows/TestCert/test_sign_cert_pass.pfx create mode 100644 contrib/windows/cmake/FindOpenSSL.cmake create mode 100644 contrib/windows/cmake/FindPackageHandleStandardArgs.cmake create mode 100644 contrib/windows/cmake/FindPackageMessage.cmake create mode 100644 contrib/windows/cmake/FindWdk.cmake create mode 100644 contrib/windows/cmake/GetGitRevisionDescription.cmake create mode 100644 contrib/windows/cmake/GetGitRevisionDescription.cmake.in create mode 100644 contrib/windows/cmake/SelectLibraryConfigurations.cmake create mode 100644 contrib/windows/parsedump/README.md create mode 100644 contrib/windows/parsedump/parsedump.bat create mode 100644 contrib/windows/parsedump/parsedump.py create mode 100644 include/os/windows/Trace.h create mode 100644 include/os/windows/spl/rpc/types.h create mode 100644 include/os/windows/spl/rpc/xdr.h create mode 100644 include/os/windows/spl/spl-debug.h create mode 100644 include/os/windows/spl/sys/acl.h create mode 100644 include/os/windows/spl/sys/acl_impl.h create mode 100644 include/os/windows/spl/sys/atomic.h create mode 100644 include/os/windows/spl/sys/attr.h create mode 100644 include/os/windows/spl/sys/avl_impl.h create mode 100644 include/os/windows/spl/sys/bootconf.h create mode 100644 include/os/windows/spl/sys/bootprops.h create mode 100644 include/os/windows/spl/sys/buf.h create mode 100644 include/os/windows/spl/sys/byteorder.h create mode 100644 include/os/windows/spl/sys/callb.h create mode 100644 include/os/windows/spl/sys/cmn_err.h create mode 100644 include/os/windows/spl/sys/compress.h create mode 100644 include/os/windows/spl/sys/condvar.h create mode 100644 include/os/windows/spl/sys/conf.h create mode 100644 include/os/windows/spl/sys/console.h create mode 100644 include/os/windows/spl/sys/cpupart.h create mode 100644 include/os/windows/spl/sys/cpuvar.h create mode 100644 include/os/windows/spl/sys/crc32.h create mode 100644 include/os/windows/spl/sys/cred.h create mode 100644 include/os/windows/spl/sys/ctype.h create mode 100644 include/os/windows/spl/sys/ddi.h create mode 100644 include/os/windows/spl/sys/debug.h create mode 100644 include/os/windows/spl/sys/dirent.h create mode 100644 include/os/windows/spl/sys/disp.h create mode 100644 include/os/windows/spl/sys/dkio.h create mode 100644 include/os/windows/spl/sys/dklabel.h create mode 100644 include/os/windows/spl/sys/dnlc.h create mode 100644 include/os/windows/spl/sys/dumphdr.h create mode 100644 include/os/windows/spl/sys/errno.h create mode 100644 include/os/windows/spl/sys/extdirent.h create mode 100644 include/os/windows/spl/sys/fcntl.h create mode 100644 include/os/windows/spl/sys/file.h create mode 100644 include/os/windows/spl/sys/ia32/asm_linkage.h create mode 100644 include/os/windows/spl/sys/idmap.h create mode 100644 include/os/windows/spl/sys/int_limits.h create mode 100644 include/os/windows/spl/sys/int_types.h create mode 100644 include/os/windows/spl/sys/inttypes.h create mode 100644 include/os/windows/spl/sys/ioctl.h create mode 100644 include/os/windows/spl/sys/isa_defs.h create mode 100644 include/os/windows/spl/sys/kidmap.h create mode 100644 include/os/windows/spl/sys/kmem.h create mode 100644 include/os/windows/spl/sys/kmem_cache.h create mode 100644 include/os/windows/spl/sys/kmem_impl.h create mode 100644 include/os/windows/spl/sys/kstat.h create mode 100644 include/os/windows/spl/sys/linker_set.h create mode 100644 include/os/windows/spl/sys/list.h create mode 100644 include/os/windows/spl/sys/lookasidelist.h create mode 100644 include/os/windows/spl/sys/md5.h create mode 100644 include/os/windows/spl/sys/md5_consts.h create mode 100644 include/os/windows/spl/sys/misc.h create mode 100644 include/os/windows/spl/sys/mntent.h create mode 100644 include/os/windows/spl/sys/mod_os.h create mode 100644 include/os/windows/spl/sys/mount.h create mode 100644 include/os/windows/spl/sys/mutex.h create mode 100644 include/os/windows/spl/sys/note.h create mode 100644 include/os/windows/spl/sys/param.h create mode 100644 include/os/windows/spl/sys/policy.h create mode 100644 include/os/windows/spl/sys/priv.h create mode 100644 include/os/windows/spl/sys/proc.h create mode 100644 include/os/windows/spl/sys/processor.h create mode 100644 include/os/windows/spl/sys/procfs_list.h create mode 100644 include/os/windows/spl/sys/random.h create mode 100644 include/os/windows/spl/sys/resource.h create mode 100644 include/os/windows/spl/sys/rwlock.h create mode 100644 include/os/windows/spl/sys/sched.h create mode 100644 include/os/windows/spl/sys/seg_kmem.h create mode 100644 include/os/windows/spl/sys/sid.h create mode 100644 include/os/windows/spl/sys/signal.h create mode 100644 include/os/windows/spl/sys/simd.h create mode 100644 include/os/windows/spl/sys/stat.h create mode 100644 include/os/windows/spl/sys/string.h create mode 100644 include/os/windows/spl/sys/stropts.h create mode 100644 include/os/windows/spl/sys/sunddi.h create mode 100644 include/os/windows/spl/sys/sunldi.h create mode 100644 include/os/windows/spl/sys/sysevent.h create mode 100644 include/os/windows/spl/sys/sysevent/eventdefs.h create mode 100644 include/os/windows/spl/sys/sysmacros.h create mode 100644 include/os/windows/spl/sys/systeminfo.h create mode 100644 include/os/windows/spl/sys/systm.h create mode 100644 include/os/windows/spl/sys/taskq.h create mode 100644 include/os/windows/spl/sys/taskq_impl.h create mode 100644 include/os/windows/spl/sys/thread.h create mode 100644 include/os/windows/spl/sys/time.h create mode 100644 include/os/windows/spl/sys/timer.h create mode 100644 include/os/windows/spl/sys/trace.h create mode 100644 include/os/windows/spl/sys/trace_zfs.h create mode 100644 include/os/windows/spl/sys/tsd.h create mode 100644 include/os/windows/spl/sys/types.h create mode 100644 include/os/windows/spl/sys/types32.h create mode 100644 include/os/windows/spl/sys/uio.h create mode 100644 include/os/windows/spl/sys/unistd.h create mode 100644 include/os/windows/spl/sys/utfconv.h create mode 100644 include/os/windows/spl/sys/utsname.h create mode 100644 include/os/windows/spl/sys/varargs.h create mode 100644 include/os/windows/spl/sys/vfs.h create mode 100644 include/os/windows/spl/sys/vfs_opreg.h create mode 100644 include/os/windows/spl/sys/vmem.h create mode 100644 include/os/windows/spl/sys/vmem_impl.h create mode 100644 include/os/windows/spl/sys/vmsystm.h create mode 100644 include/os/windows/spl/sys/vnode.h create mode 100644 include/os/windows/spl/sys/wmsum.h create mode 100644 include/os/windows/spl/sys/zmod.h create mode 100644 include/os/windows/spl/sys/zone.h create mode 100644 include/os/windows/spl/unistd.h create mode 100644 include/os/windows/spl_config.h create mode 100644 include/os/windows/zfs/sys/fs/zfsdi.h create mode 100644 include/os/windows/zfs/sys/policy.h create mode 100644 include/os/windows/zfs/sys/vdev_disk_os.h create mode 100644 include/os/windows/zfs/sys/wzvol.h create mode 100644 include/os/windows/zfs/sys/wzvolwmi.h create mode 100644 include/os/windows/zfs/sys/zconf.h create mode 100644 include/os/windows/zfs/sys/zfs_bootenv_os.h create mode 100644 include/os/windows/zfs/sys/zfs_context_os.h create mode 100644 include/os/windows/zfs/sys/zfs_ctldir.h create mode 100644 include/os/windows/zfs/sys/zfs_dir.h create mode 100644 include/os/windows/zfs/sys/zfs_ioctl_compat.h create mode 100644 include/os/windows/zfs/sys/zfs_mount.h create mode 100644 include/os/windows/zfs/sys/zfs_vfsops_os.h create mode 100644 include/os/windows/zfs/sys/zfs_vnops_os.h create mode 100644 include/os/windows/zfs/sys/zfs_windows.h create mode 100644 include/os/windows/zfs/sys/zfs_znode_impl.h create mode 100644 include/os/windows/zfs/sys/zlib.h create mode 100644 include/os/windows/zfs/sys/zpl.h create mode 100644 include/os/windows/zfs/sys/zvol_os.h create mode 100644 include/os/windows/zfs/zfs_config.h create mode 100644 include/zfs_gitrev.h.win create mode 100644 lib/CMakeLists.txt create mode 100644 lib/libavl/CMakeLists.txt create mode 100644 lib/libefi/CMakeLists.txt create mode 100644 lib/libefi/rdwr_efi_windows.c create mode 100644 lib/libicp/CMakeLists.txt create mode 100644 lib/libnvpair/CMakeLists.txt create mode 100644 lib/libshare/CMakeLists.txt create mode 100644 lib/libshare/os/windows/nfs.c create mode 100644 lib/libshare/os/windows/smb.c create mode 100644 lib/libspl/CMakeLists.txt create mode 100644 lib/libspl/include/os/windows/aio.h create mode 100644 lib/libspl/include/os/windows/dirent.h create mode 100644 lib/libspl/include/os/windows/dlfcn.h create mode 100644 lib/libspl/include/os/windows/err.h create mode 100644 lib/libspl/include/os/windows/fcntl.h create mode 100644 lib/libspl/include/os/windows/getopt.h create mode 100644 lib/libspl/include/os/windows/grp.h create mode 100644 lib/libspl/include/os/windows/langinfo.h create mode 100644 lib/libspl/include/os/windows/libgen.h create mode 100644 lib/libspl/include/os/windows/libintl.h create mode 100644 lib/libspl/include/os/windows/mntent.h create mode 100644 lib/libspl/include/os/windows/poll.h create mode 100644 lib/libspl/include/os/windows/pwd.h create mode 100644 lib/libspl/include/os/windows/regex.h create mode 100644 lib/libspl/include/os/windows/rpc/types.h create mode 100644 lib/libspl/include/os/windows/rpc/xdr.h create mode 100644 lib/libspl/include/os/windows/sched.h create mode 100644 lib/libspl/include/os/windows/search.h create mode 100644 lib/libspl/include/os/windows/signal.h create mode 100644 lib/libspl/include/os/windows/stdio.h create mode 100644 lib/libspl/include/os/windows/string.h create mode 100644 lib/libspl/include/os/windows/sys/byteorder.h create mode 100644 lib/libspl/include/os/windows/sys/errno.h create mode 100644 lib/libspl/include/os/windows/sys/file.h create mode 100644 lib/libspl/include/os/windows/sys/ia32/asm_linkage.h create mode 100644 lib/libspl/include/os/windows/sys/ioctl.h create mode 100644 lib/libspl/include/os/windows/sys/kstat.h create mode 100644 lib/libspl/include/os/windows/sys/mman.h create mode 100644 lib/libspl/include/os/windows/sys/mnttab.h create mode 100644 lib/libspl/include/os/windows/sys/mount.h create mode 100644 lib/libspl/include/os/windows/sys/param.h create mode 100644 lib/libspl/include/os/windows/sys/resource.h create mode 100644 lib/libspl/include/os/windows/sys/socket.h create mode 100644 lib/libspl/include/os/windows/sys/stat.h create mode 100644 lib/libspl/include/os/windows/sys/sysmacros.h create mode 100644 lib/libspl/include/os/windows/sys/time.h create mode 100644 lib/libspl/include/os/windows/sys/timer.h create mode 100644 lib/libspl/include/os/windows/sys/types.h create mode 100644 lib/libspl/include/os/windows/sys/types32.h create mode 100644 lib/libspl/include/os/windows/sys/uio.h create mode 100644 lib/libspl/include/os/windows/sys/utsname.h create mode 100644 lib/libspl/include/os/windows/sys/vfs.h create mode 100644 lib/libspl/include/os/windows/sys/wait.h create mode 100644 lib/libspl/include/os/windows/sys/zfs_context_os.h create mode 100644 lib/libspl/include/os/windows/sys/zfs_mount.h create mode 100644 lib/libspl/include/os/windows/termios.h create mode 100644 lib/libspl/include/os/windows/unistd.h create mode 100644 lib/libspl/include/os/windows/uuid/uuid.h create mode 100644 lib/libspl/include/os/windows/wosix.h create mode 100644 lib/libspl/os/windows/gethostid.c create mode 100644 lib/libspl/os/windows/getmntany.c create mode 100644 lib/libspl/os/windows/getopt.c create mode 100644 lib/libspl/os/windows/getoptl.c create mode 100644 lib/libspl/os/windows/page.c create mode 100644 lib/libspl/os/windows/posix.c create mode 100644 lib/libspl/os/windows/signal.c create mode 100644 lib/libspl/os/windows/uio.c create mode 100644 lib/libspl/os/windows/weakpragma.c create mode 100644 lib/libspl/os/windows/xdr.c create mode 100644 lib/libspl/os/windows/xdr_array.c create mode 100644 lib/libspl/os/windows/xdr_float.c create mode 100644 lib/libspl/os/windows/xdr_mem.c create mode 100644 lib/libspl/os/windows/zone.c create mode 100644 lib/libtpool/CMakeLists.txt create mode 100644 lib/libunicode/CMakeLists.txt create mode 100644 lib/libuutil/CMakeLists.txt create mode 100644 lib/libzfs/CMakeLists.txt create mode 100644 lib/libzfs/os/windows/libzfs_mount_os.c create mode 100644 lib/libzfs/os/windows/libzfs_pool_os.c create mode 100644 lib/libzfs/os/windows/libzfs_util_os.c create mode 100644 lib/libzfs_core/CMakeLists.txt create mode 100644 lib/libzfs_core/os/windows/libzfs_core_ioctl.c create mode 100644 lib/libzfsbootenv/CMakeLists.txt create mode 100644 lib/libzpool/CMakeLists.txt create mode 100644 lib/libzstd/CMakeLists.txt create mode 100644 lib/libzutil/CMakeLists.txt create mode 100644 lib/libzutil/os/windows/zutil_compat.c create mode 100644 lib/libzutil/os/windows/zutil_device_path_os.c create mode 100644 lib/libzutil/os/windows/zutil_import_os.c create mode 100644 lib/os/windows/CMakeLists.txt create mode 100644 lib/os/windows/libkstat/CMakeLists.txt create mode 100644 lib/os/windows/libkstat/gmatch.c create mode 100644 lib/os/windows/libkstat/kstat.c create mode 100644 lib/os/windows/libkstat/kstat.h create mode 100644 lib/os/windows/libpthread/CMakeLists.txt create mode 100644 lib/os/windows/libpthread/pthread.c create mode 100644 lib/os/windows/libpthread/pthread.h create mode 100644 lib/os/windows/libregex/CMakeLists.txt create mode 100644 lib/os/windows/libregex/COPYRIGHT create mode 100644 lib/os/windows/libregex/Makefile.inc create mode 100644 lib/os/windows/libregex/WHATSNEW create mode 100644 lib/os/windows/libregex/cname.h create mode 100644 lib/os/windows/libregex/engine.c create mode 100644 lib/os/windows/libregex/re_format.7 create mode 100644 lib/os/windows/libregex/regcomp.c create mode 100644 lib/os/windows/libregex/regerror.c create mode 100644 lib/os/windows/libregex/regex.3 create mode 100644 lib/os/windows/libregex/regex2.h create mode 100644 lib/os/windows/libregex/regexec.c create mode 100644 lib/os/windows/libregex/regfree.c create mode 100644 lib/os/windows/libregex/regsub.c create mode 100644 lib/os/windows/libregex/utils.h create mode 100644 lib/os/windows/libuuid/CMakeLists.txt create mode 100644 lib/os/windows/libuuid/COPYING create mode 100644 lib/os/windows/libuuid/all-io.h create mode 100644 lib/os/windows/libuuid/c.h create mode 100644 lib/os/windows/libuuid/clear.c create mode 100644 lib/os/windows/libuuid/compare.c create mode 100644 lib/os/windows/libuuid/copy.c create mode 100644 lib/os/windows/libuuid/gen_uuid.c create mode 100644 lib/os/windows/libuuid/isnull.c create mode 100644 lib/os/windows/libuuid/pack.c create mode 100644 lib/os/windows/libuuid/parse.c create mode 100644 lib/os/windows/libuuid/randutils.c create mode 100644 lib/os/windows/libuuid/randutils.h create mode 100644 lib/os/windows/libuuid/test_uuid.c create mode 100644 lib/os/windows/libuuid/unpack.c create mode 100644 lib/os/windows/libuuid/unparse.c create mode 100644 lib/os/windows/libuuid/uuid.h create mode 100644 lib/os/windows/libuuid/uuidP.h create mode 100644 lib/os/windows/libuuid/uuid_time.c create mode 100644 lib/os/windows/libuuid/uuidd.h create mode 100644 lib/os/windows/zlib-1.2.13/CMakeLists.txt create mode 100644 lib/os/windows/zlib-1.2.13/ChangeLog create mode 100644 lib/os/windows/zlib-1.2.13/FAQ create mode 100644 lib/os/windows/zlib-1.2.13/INDEX create mode 100644 lib/os/windows/zlib-1.2.13/README create mode 100644 lib/os/windows/zlib-1.2.13/adler32.c create mode 100644 lib/os/windows/zlib-1.2.13/compress.c create mode 100644 lib/os/windows/zlib-1.2.13/contrib/minizip/crypt.h create mode 100644 lib/os/windows/zlib-1.2.13/contrib/minizip/ioapi.c create mode 100644 lib/os/windows/zlib-1.2.13/contrib/minizip/ioapi.h create mode 100644 lib/os/windows/zlib-1.2.13/contrib/minizip/unzip.c create mode 100644 lib/os/windows/zlib-1.2.13/contrib/minizip/unzip.h create mode 100644 lib/os/windows/zlib-1.2.13/contrib/minizip/zip.c create mode 100644 lib/os/windows/zlib-1.2.13/contrib/minizip/zip.h create mode 100644 lib/os/windows/zlib-1.2.13/crc32.c create mode 100644 lib/os/windows/zlib-1.2.13/crc32.h create mode 100644 lib/os/windows/zlib-1.2.13/deflate.c create mode 100644 lib/os/windows/zlib-1.2.13/deflate.h create mode 100644 lib/os/windows/zlib-1.2.13/doc/algorithm.txt create mode 100644 lib/os/windows/zlib-1.2.13/gzclose.c create mode 100644 lib/os/windows/zlib-1.2.13/gzguts.h create mode 100644 lib/os/windows/zlib-1.2.13/gzlib.c create mode 100644 lib/os/windows/zlib-1.2.13/gzread.c create mode 100644 lib/os/windows/zlib-1.2.13/gzwrite.c create mode 100644 lib/os/windows/zlib-1.2.13/infback.c create mode 100644 lib/os/windows/zlib-1.2.13/inffast.c create mode 100644 lib/os/windows/zlib-1.2.13/inffast.h create mode 100644 lib/os/windows/zlib-1.2.13/inffixed.h create mode 100644 lib/os/windows/zlib-1.2.13/inflate.c create mode 100644 lib/os/windows/zlib-1.2.13/inflate.h create mode 100644 lib/os/windows/zlib-1.2.13/inftrees.c create mode 100644 lib/os/windows/zlib-1.2.13/inftrees.h create mode 100644 lib/os/windows/zlib-1.2.13/test/minigzip.c create mode 100644 lib/os/windows/zlib-1.2.13/trees.c create mode 100644 lib/os/windows/zlib-1.2.13/trees.h create mode 100644 lib/os/windows/zlib-1.2.13/uncompr.c create mode 100644 lib/os/windows/zlib-1.2.13/zconf.h create mode 100644 lib/os/windows/zlib-1.2.13/zconf.h.in create mode 100644 lib/os/windows/zlib-1.2.13/zlib-dll-res.rc create mode 100644 lib/os/windows/zlib-1.2.13/zlib.3 create mode 100644 lib/os/windows/zlib-1.2.13/zlib.h create mode 100644 lib/os/windows/zlib-1.2.13/zutil.c create mode 100644 lib/os/windows/zlib-1.2.13/zutil.h create mode 100644 module/CMakeLists.txt create mode 100644 module/icp/CMakeLists.txt create mode 100644 module/lua/CMakeLists.txt create mode 100644 module/nvpair/CMakeLists.txt create mode 100644 module/os/windows/.gitignore create mode 100644 module/os/windows/CMakeLists.txt create mode 100644 module/os/windows/CMakeSettings.json create mode 100644 module/os/windows/OpenZFS.inf create mode 100644 module/os/windows/OpenZFS.man create mode 100644 module/os/windows/PORTING_NOTES.txt create mode 100644 module/os/windows/README.md create mode 100644 module/os/windows/Wpp.c create mode 100644 module/os/windows/debug.c create mode 100644 module/os/windows/driver.c create mode 100644 module/os/windows/resource.h create mode 100644 module/os/windows/resource.rc create mode 100644 module/os/windows/spl/CMakeLists.txt create mode 100644 module/os/windows/spl/spl-atomic.c create mode 100644 module/os/windows/spl/spl-condvar.c create mode 100644 module/os/windows/spl/spl-cred.c create mode 100644 module/os/windows/spl/spl-ddi.c create mode 100644 module/os/windows/spl/spl-debug.c create mode 100644 module/os/windows/spl/spl-err.c create mode 100644 module/os/windows/spl/spl-kmem.c create mode 100644 module/os/windows/spl/spl-kstat.c create mode 100644 module/os/windows/spl/spl-list.c create mode 100644 module/os/windows/spl/spl-lookasidelist.c create mode 100644 module/os/windows/spl/spl-md5.c create mode 100644 module/os/windows/spl/spl-mount.c create mode 100644 module/os/windows/spl/spl-mutex.c create mode 100644 module/os/windows/spl/spl-policy.c create mode 100644 module/os/windows/spl/spl-proc.c create mode 100644 module/os/windows/spl/spl-proc_list.c create mode 100644 module/os/windows/spl/spl-processor.c create mode 100644 module/os/windows/spl/spl-rwlock.c create mode 100644 module/os/windows/spl/spl-seg_kmem.c create mode 100644 module/os/windows/spl/spl-taskq.c create mode 100644 module/os/windows/spl/spl-thread.c create mode 100644 module/os/windows/spl/spl-time.c create mode 100644 module/os/windows/spl/spl-tsd.c create mode 100644 module/os/windows/spl/spl-uio.c create mode 100644 module/os/windows/spl/spl-vmem.c create mode 100644 module/os/windows/spl/spl-vnode.c create mode 100644 module/os/windows/spl/spl-windows.c create mode 100644 module/os/windows/spl/spl-wmsum.c create mode 100644 module/os/windows/spl/spl-xdr.c create mode 100644 module/os/windows/spl/spl-zlib.c create mode 100644 module/os/windows/zfs/CMakeLists.txt create mode 100644 module/os/windows/zfs/abd_os.c create mode 100644 module/os/windows/zfs/arc_os.c create mode 100644 module/os/windows/zfs/qat.c create mode 100644 module/os/windows/zfs/qat_compress.c create mode 100644 module/os/windows/zfs/qat_crypt.c create mode 100644 module/os/windows/zfs/spa_misc_os.c create mode 100644 module/os/windows/zfs/sysctl_os.c create mode 100644 module/os/windows/zfs/vdev_disk.c create mode 100644 module/os/windows/zfs/vdev_file.c create mode 100644 module/os/windows/zfs/zfs_acl.c create mode 100644 module/os/windows/zfs/zfs_ctldir.c create mode 100644 module/os/windows/zfs/zfs_debug.c create mode 100644 module/os/windows/zfs/zfs_dir.c create mode 100644 module/os/windows/zfs/zfs_file_os.c create mode 100644 module/os/windows/zfs/zfs_fuid_os.c create mode 100644 module/os/windows/zfs/zfs_ioctl_os.c create mode 100644 module/os/windows/zfs/zfs_racct.c create mode 100644 module/os/windows/zfs/zfs_vfsops.c create mode 100644 module/os/windows/zfs/zfs_vnops_os.c create mode 100644 module/os/windows/zfs/zfs_vnops_windows.c create mode 100644 module/os/windows/zfs/zfs_vnops_windows_lib.c create mode 100644 module/os/windows/zfs/zfs_vnops_windows_mount.c create mode 100644 module/os/windows/zfs/zfs_vnops_windows_xattr.c create mode 100644 module/os/windows/zfs/zfs_windows_zvol.c create mode 100644 module/os/windows/zfs/zfs_windows_zvol_scsi.c create mode 100644 module/os/windows/zfs/zfs_windows_zvol_wmi.c create mode 100644 module/os/windows/zfs/zfs_znode.c create mode 100644 module/os/windows/zfs/zio_crypt.c create mode 100644 module/os/windows/zfs/zvol_os.c create mode 100644 module/unicode/CMakeLists.txt create mode 100644 module/zcommon/CMakeLists.txt create mode 100644 module/zfs/CMakeLists.txt create mode 100644 module/zstd/CMakeLists.txt diff --git a/.github/workflows/codeql-windows.yml b/.github/workflows/codeql-windows.yml new file mode 100644 index 000000000000..94cd039fbb19 --- /dev/null +++ b/.github/workflows/codeql-windows.yml @@ -0,0 +1,69 @@ +###################################################################################### +# # +# If you're looking for instructions on how to build this under windows go to # +#https://github.com/openzfsonwindows/openzfs/blob/windows/module/os/windows/README.md# +# # +###################################################################################### + +name: "CodeQL windows" + +on: + push: + pull_request: + +env: + # Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.) + BUILD_TYPE: Debug + +jobs: + analyze: + name: Analyze + timeout-minutes: 120 + # The CMake configure and build commands are platform agnostic and should work equally well on Windows or Mac. + # You can convert this to a matrix build if you need cross-platform coverage. + # See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix + runs-on: windows-latest + permissions: + actions: read + contents: read + security-events: write + + steps: + - uses: ilammy/msvc-dev-cmd@v1 + + - uses: actions/checkout@v3 + with: + #repository: openzfsonwindows/openzfs + fetch-depth: 0 + + - name: Import signing certificate + run: | + $plaintextpwd = 'password1234' + $pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\CurrentUser\My -Password $pwd -Exportable + + - name: Checkout openssl + uses: actions/checkout@v3 + with: + repository: andrewc12/openssl # optional, default is ${{ github.repository }} + path: openssl # optional + + # Initializes the CodeQL tools for scanning. + - name: Initialize CodeQL + uses: github/codeql-action/init@v2 + # Override language selection by uncommenting this and choosing your languages + with: + languages: cpp + + - name: Configure CMake + # Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make. + # See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type + run: cmake -G "Ninja" -B ${{github.workspace}}/out/build/x64-Debug ${{github.workspace}} -DOPENSSL_ROOT_DIR=${{github.workspace}}/openssl/ -DCRYPTO_STATIC_TEST=${{github.workspace}}/openssl/lib/VC/static/libcrypto64MTd.lib -DLIB_EAY_DEBUG=${{github.workspace}}/openssl/lib/VC/static/libcrypto64MTd.lib -DLIB_EAY_RELEASE=${{github.workspace}}/openssl/lib/VC/static/libcrypto64MT.lib -DOPENSSL_INCLUDE_DIR=${{github.workspace}}/openssl/include -DSSL_EAY_DEBUG=${{github.workspace}}/openssl/lib/VC/static/libssl64MTd.lib -DSSL_EAY_RELEASE=${{github.workspace}}/openssl/lib/VC/static/libssl64MT.lib + + - name: Build + working-directory: ${{github.workspace}}/out/build/x64-Debug + # Build your program with the given configuration + run: cmake --build ${{github.workspace}}/out/build/x64-Debug + + - name: Perform CodeQL Analysis + uses: github/codeql-action/analyze@v2 diff --git a/.github/workflows/windows-build-test.yml b/.github/workflows/windows-build-test.yml new file mode 100644 index 000000000000..23c6a0ece528 --- /dev/null +++ b/.github/workflows/windows-build-test.yml @@ -0,0 +1,1937 @@ +###################################################################################### +# # +# If you're looking for instructions on how to build this under windows go to # +#https://github.com/openzfsonwindows/openzfs/blob/windows/module/os/windows/README.md# +# # +###################################################################################### + +name: windows-build-test + +on: + push: + pull_request: + workflow_dispatch: + inputs: + run-failing: + type: boolean + default: true + description: Run tests that fail + + + +env: + # Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.) + BUILD_TYPE: Debug + +jobs: + build_windows: + timeout-minutes: 30 + # The CMake configure and build commands are platform agnostic and should work equally well on Windows or Mac. + # You can convert this to a matrix build if you need cross-platform coverage. + # See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix + runs-on: windows-latest + + steps: + - uses: ilammy/msvc-dev-cmd@v1 + + - uses: actions/checkout@v3 + with: + #repository: openzfsonwindows/openzfs + fetch-depth: 0 + + - name: debug - git status + run: git status + + #https://stackoverflow.com/a/60883893 + # "C:\Program Files\Git\bin\git.exe" -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +90acd8f20e6d0c14b83241cc7dcd17df27d5a95b:refs/remotes/origin/andrew_workflows-7 + #- name: checkout again + # run: git fetch --prune --unshallow + + + - name: Import signing certificate + run: | + $plaintextpwd = 'password1234' + $pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\CurrentUser\My -Password $pwd -Exportable + + #- name: Remove provided ssl + # run: Remove-Item -Path "C:\Strawberry\" -Recurse -Force + + #- name: Install openssl + # run: git clone https://github.com/andrewc12/openssl.git + + - name: Checkout openssl + uses: actions/checkout@v3 + with: + repository: andrewc12/openssl # optional, default is ${{ github.repository }} + path: openssl # optional + + - name: Configure CMake + # Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make. + # See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type + run: cmake -G "Ninja" -B ${{github.workspace}}/out/build/x64-Debug ${{github.workspace}} -DOPENSSL_ROOT_DIR=${{github.workspace}}/openssl/ -DCRYPTO_STATIC_TEST=${{github.workspace}}/openssl/lib/VC/static/libcrypto64MTd.lib -DLIB_EAY_DEBUG=${{github.workspace}}/openssl/lib/VC/static/libcrypto64MTd.lib -DLIB_EAY_RELEASE=${{github.workspace}}/openssl/lib/VC/static/libcrypto64MT.lib -DOPENSSL_INCLUDE_DIR=${{github.workspace}}/openssl/include -DSSL_EAY_DEBUG=${{github.workspace}}/openssl/lib/VC/static/libssl64MTd.lib -DSSL_EAY_RELEASE=${{github.workspace}}/openssl/lib/VC/static/libssl64MT.lib + + - name: Build + working-directory: ${{github.workspace}}/out/build/x64-Debug + # Build your program with the given configuration + run: cmake --build ${{github.workspace}}/out/build/x64-Debug + + - name: debug - get git rev + run: git describe --always --long --dirty + + - name: Download Inno Setup istaller + uses: suisei-cn/actions-download-file@v1.3.0 + id: innoinstaller + with: + url: 'https://jrsoftware.org/download.php/is.exe' + target: ${{github.workspace}}/ + + - name: debug - echo filename + run: echo ${{ steps.innoinstaller.outputs.filename }} + + - name: debug - list + run: ls ${{github.workspace}}\ + + - name: debug - list + run: ls ${{github.workspace}}\${{ steps.innoinstaller.outputs.filename }} + + - name: install inno + run: 'Start-Process -FilePath "${{github.workspace}}\${{ steps.innoinstaller.outputs.filename }}" -Wait -ArgumentList "/NORESTART /ALLUSERS /VERYSILENT /LOG=`"${{github.workspace}}\InnoSetup-Install.log`""' + + #- name: Wait for install to finish + # run: Start-Sleep -Seconds 30 + # uses: iFaxity/wait-on-action@v1 + # with: + # resource: 'C:\Program Files\OpenZFS On Windows\zpool.exe' + + - name: debug - print log + run: cat "${{github.workspace}}\InnoSetup-Install.log" + + - name: run ISCC.exe to construct OpenZFS installer + run: '&"C:\Program Files (x86)\Inno Setup 6\ISCC.exe" .\ZFSInstall-debug.iss "/Ssigntoolc=C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x64\signtool.exe sign /v /fd sha256 /n `$qOpenZFS Test Signing Certificate`$q /t http://timestamp.digicert.com `$f" | Tee-Object -FilePath "${{github.workspace}}\iscc.log"' + working-directory: ${{github.workspace}}\contrib\windows\Inno.Setup + + #- name: debug - get exe name + # https://stackoverflow.com/questions/4426442/unix-tail-equivalent-command-in-windows-powershell + # run: Get-Content "${{github.workspace}}\iscc.log" -Tail 1 + + - name: get installer exe name + id: innoout + # https://stackoverflow.com/questions/4426442/unix-tail-equivalent-command-in-windows-powershell + #https://www.jamescroft.co.uk/setting-github-actions-environment-variables-in-powershell/ + run: | + $p = Get-Content "${{github.workspace}}\iscc.log" -Tail 1 + echo $p + $f = (Get-Item $p ).Name + echo $f + echo installername=$f + #echo "installername=$f" >> $GITHUB_ENV + #echo "installername=$f" >> $GITHUB_OUTPUT + echo "installername=$f" | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append + #shell: pwsh + + + - name: debug - echo + run: echo "${{ steps.innoout.outputs.installername }}" + + - name: Upload a Build Artifact + if: ${{ failure() }} + uses: actions/upload-artifact@v3.1.0 + with: + name: build_output + path: ${{github.workspace}}/out + + - name: Collect dev build + if: ${{ failure() }} + run: | + mkdir ${{github.workspace}}/zfs_collect + copy ${{github.workspace}}/out/build/x64-Debug/module/os/windows/driver/* ${{github.workspace}}/zfs_collect + copy ${{github.workspace}}/out/build/x64-Debug/cmd/os/windows/kstat/kstat.exe ${{github.workspace}}/zfs_collect/kstat.exe + copy ${{github.workspace}}/out/build/x64-Debug/cmd/os/windows/zfsinstaller/zfsinstaller.exe ${{github.workspace}}/zfs_collect/zfsinstaller.exe + copy ${{github.workspace}}/out/build/x64-Debug/cmd/zdb/zdb.exe ${{github.workspace}}/zfs_collect/zdb.exe + copy ${{github.workspace}}/out/build/x64-Debug/cmd/zfs/zfs.exe ${{github.workspace}}/zfs_collect/zfs.exe + copy ${{github.workspace}}/out/build/x64-Debug/cmd/zpool/zpool.exe ${{github.workspace}}/zfs_collect/zpool.exe + copy ${{github.workspace}}/out/build/x64-Debug/cmd/zstream/zstreamdump.exe ${{github.workspace}}/zfs_collect/zstreamdump.exe + + - name: Upload dev build + if: ${{ failure() }} + uses: actions/upload-artifact@v3.1.0 + with: + name: dev_build + path: ${{github.workspace}}/zfs_collect/* + + - name: Upload dev build with Inno Setup + uses: actions/upload-artifact@v3.1.0 + with: + name: dev_build_inno + path: ${{github.workspace}}\contrib\windows\${{ steps.innoout.outputs.name }} + + + + + + # https://github.com/MicrosoftDocs/windows-powershell-docs/issues/266 + - name: Import root certificate + run: | + $plaintextpwd = 'password1234' + $pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\Root -Password $pwd + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\TrustedPublisher -Password $pwd + + - name: install zfs + run: 'Start-Process -FilePath "${{github.workspace}}\contrib\windows\${{ steps.innoout.outputs.installername }}" -Wait -ArgumentList "/NORESTART /ALLUSERS /VERYSILENT /LOG=`"${{github.workspace}}\InnoSetup-Install.log`""' + + #- name: Wait for install to finish + # run: Start-Sleep -Seconds 30 + # uses: iFaxity/wait-on-action@v1 + # with: + # resource: 'C:\Program Files\OpenZFS On Windows\zpool.exe' + + - name: debug - print log + run: cat "${{github.workspace}}\InnoSetup-Install.log" + + + + + + + + + + + + + + + + + + + +# This is a basic workflow to help you get started with Actions +# https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/ + + test1_tests: + needs: [build_windows] #, build_wsl] + timeout-minutes: 30 + runs-on: windows-latest + steps: + + - uses: actions/checkout@v3 + + + - uses: actions/download-artifact@v3 + with: + name: dev_build_inno + +# - uses: actions/download-artifact@v3 +# with: +# name: build result + +# - name: get files +# run: Get-ChildItem -Recurse + + - name: get diskdrive + run: wmic diskdrive list + +# - name: Download +# uses: suisei-cn/actions-download-file@v1.3.0 +# #id: innoinstaller +# with: +# url: 'https://github.com/andrewc12/zfsfiledump/raw/main/testdrives.zip' +# target: ${{github.workspace}}/ +# +# +# - name: make disk +# run: | +# Expand-Archive -LiteralPath ${{github.workspace}}/testdrives.zip -DestinationPath D:\ +# +# - name: Download +# uses: suisei-cn/actions-download-file@v1.3.0 +# #id: innoinstaller +# with: +# url: 'https://github.com/andrewc12/zfsfiledump/raw/main/scriptname.txt' +# target: ${{github.workspace}}/ +# +# - name: make disk +# run: | +# diskpart /s scriptname.txt +# +# +# +# - name: get diskdrive +# run: wmic diskdrive list +# +# +# - name: get files +# run: Get-ChildItem -Recurse + + + - name: get zfsexename + id: zfsinstaller + run: | + $p = Get-ChildItem | Where-Object {$_.Name -like 'OpenZFSOnWindows-*.exe'} | Select-Object -first 1 + echo $p + $f = (Get-Item $p ).Name + echo $f + echo zfsinstallerfilename=$f + echo "zfsinstallerfilename=$f" | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append + + # https://github.com/MicrosoftDocs/windows-powershell-docs/issues/266 + - name: Import root certificate + run: | + $plaintextpwd = 'password1234' + $pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\Root -Password $pwd + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\TrustedPublisher -Password $pwd + + - name: debug - echo filename + run: echo "${{ steps.zfsinstaller.outputs.zfsinstallerfilename }}" + +# - name: debug - list +# run: ls ${{github.workspace}}\ + + - name: debug - list + run: ls ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} + + #- name: install zfs + # run: ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} /NORESTART /ALLUSERS /VERYSILENT /LOG="${{github.workspace}}\InnoSetup-Install.log" + + - name: install zfs + run: 'Start-Process -FilePath "${{github.workspace}}\${{ steps.zfsinstaller.outputs.zfsinstallerfilename }}" -Wait -ArgumentList "/NORESTART /ALLUSERS /VERYSILENT /LOG=`"${{github.workspace}}\InnoSetup-Install.log`""' + + #- name: Wait for install to finish + # run: Start-Sleep -Seconds 30 + # uses: iFaxity/wait-on-action@v1 + # with: + # resource: 'C:\Program Files\OpenZFS On Windows\zpool.exe' + +# - name: debug - print log +# run: cat "${{github.workspace}}\InnoSetup-Install.log" +# +# - name: debug - list +# run: ls "C:\Program Files" +# +# - name: debug - list +# run: ls "C:\Program Files\OpenZFS On Windows" + + - name: debug - get status + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' + + + + + + + + - name: Create backing files + run: | + $f = new-object System.IO.FileStream ${{github.workspace}}\test01.dat, Create, ReadWrite + $f.SetLength(1GB) + $f.Close() + $f = new-object System.IO.FileStream ${{github.workspace}}\test02.dat, Create, ReadWrite + $f.SetLength(1GB) + $f.Close() + $f = new-object System.IO.FileStream ${{github.workspace}}\test03.dat, Create, ReadWrite + $f.SetLength(1GB) + $f.Close() + +# - run: choco install gsudo + + - name: tests/functional/cli_root/zpool_create/zpool_create_001_pos + timeout-minutes: 1 + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test01 \\?\${{github.workspace}}\test01.dat' + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test01' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test02 \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test02' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test03 \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat \\?\${{github.workspace}}\test03.dat' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test03' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test04 mirror \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test04' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test05 mirror \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat \\?\${{github.workspace}}\test03.dat' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test05' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test06 raidz \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat \\?\${{github.workspace}}\test03.dat' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test06' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test07 raidz1 \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat \\?\${{github.workspace}}\test03.dat' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test07' + timeout-minutes: 1 + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + + + + - name: uninstall zfs + run: 'Start-Process -FilePath "C:\Program Files\OpenZFS On Windows\unins000.exe" -Wait -ArgumentList "/NORESTART /VERYSILENT /LOG=`"${{github.workspace}}\InnoSetup-Uninstall.log`""' + + - name: debug - print log + run: cat "${{github.workspace}}\InnoSetup-Uninstall.log" + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +# This is a basic workflow to help you get started with Actions +# https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/ + + test2_winbtrfs: + needs: [build_windows] #, build_wsl] + timeout-minutes: 30 + runs-on: windows-latest +# if: ${{ inputs.run-failing }} + steps: + + - uses: actions/checkout@v3 + + + - uses: actions/download-artifact@v3 + with: + name: dev_build_inno + +# - uses: actions/download-artifact@v3 +# with: +# name: build result + +# - name: get files +# run: Get-ChildItem -Recurse + + - name: get diskdrive + run: wmic diskdrive list + +# - name: Download +# uses: suisei-cn/actions-download-file@v1.3.0 +# #id: innoinstaller +# with: +# url: 'https://github.com/andrewc12/zfsfiledump/raw/main/testdrives.zip' +# target: ${{github.workspace}}/ +# +# +# - name: make disk +# run: | +# Expand-Archive -LiteralPath ${{github.workspace}}/testdrives.zip -DestinationPath D:\ +# +# - name: Download +# uses: suisei-cn/actions-download-file@v1.3.0 +# #id: innoinstaller +# with: +# url: 'https://github.com/andrewc12/zfsfiledump/raw/main/scriptname.txt' +# target: ${{github.workspace}}/ +# +# - name: make disk +# run: | +# diskpart /s scriptname.txt +# +# +# +# - name: get diskdrive +# run: wmic diskdrive list +# +# +# - name: get files +# run: Get-ChildItem -Recurse + + + - name: get zfsexename + id: zfsinstaller + run: | + $p = Get-ChildItem | Where-Object {$_.Name -like 'OpenZFSOnWindows-*.exe'} | Select-Object -first 1 + echo $p + $f = (Get-Item $p ).Name + echo $f + echo "filename=$f" | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append + + # https://github.com/MicrosoftDocs/windows-powershell-docs/issues/266 + - name: Import root certificate + run: | + $plaintextpwd = 'password1234' + $pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\Root -Password $pwd + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\TrustedPublisher -Password $pwd + + - name: debug - echo filename + run: echo ${{ steps.zfsinstaller.outputs.filename }} + +# - name: debug - list +# run: ls ${{github.workspace}}\ + + - name: debug - list + run: ls ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} + + #- name: install zfs + # run: ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} /NORESTART /ALLUSERS /VERYSILENT /LOG="${{github.workspace}}\InnoSetup-Install.log" + + - name: install zfs + run: 'Start-Process -FilePath "${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }}" -Wait -ArgumentList "/NORESTART /ALLUSERS /VERYSILENT /LOG=`"${{github.workspace}}\InnoSetup-Install.log`""' + + #- name: Wait for install to finish + # run: Start-Sleep -Seconds 30 + # uses: iFaxity/wait-on-action@v1 + # with: + # resource: 'C:\Program Files\OpenZFS On Windows\zpool.exe' + +# - name: debug - print log +# run: cat "${{github.workspace}}\InnoSetup-Install.log" +# +# - name: debug - list +# run: ls "C:\Program Files" +# +# - name: debug - list +# run: ls "C:\Program Files\OpenZFS On Windows" + + - name: debug - get status + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' + + + + + + + + - name: Create backing files + run: | + $f = new-object System.IO.FileStream ${{github.workspace}}\test01.dat, Create, ReadWrite + $f.SetLength(1GB) + $f.Close() + + - name: create pool + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create tank \\?\${{github.workspace}}\test01.dat' + + - name: get pool status + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' + + - name: get pool mount + run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount' + + - name: get pool mount + id: drive # Remember to give an ID if you need the output filename + run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount | Select-String -Pattern "^tank +([A-Za-z]):[/\\]" | % {"drive=$($_.matches.groups[1].value):\"} | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append' + + + - name: echo + run: echo ${{ steps.drive.outputs.drive }} + + + - name: Download winbtrfs + uses: suisei-cn/actions-download-file@v1.3.0 + id: winbtrs # Remember to give an ID if you need the output filename + with: + url: "https://github.com/andrewc12/btrfs/releases/download/v1.8/x64-Debug.zip" + target: ${{github.workspace}}/ + + - name: extract + run: Expand-Archive -LiteralPath '${{ steps.winbtrs.outputs.filename }}' -DestinationPath ${{github.workspace}}\winbtrfs + + - name: debug - list + run: ls ${{github.workspace}}\ + + - name: debug - list + run: ls ${{github.workspace}}\winbtrfs + + + #- name: run winbtrfs tests + # run: '& "${{github.workspace}}\winbtrfs\test.exe" ${{ steps.drive.outputs.drive }}' + + - name: test dummy status step + id: dummy + run: exit 0 + + - name: debug - dummy status + run: echo ${{ steps.dummy.conclusion }} + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + #https://github.com/maharmstone/btrfs/blob/master/src/tests/test.cpp#L453 + - name: run winbtrfs create tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" create ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs supersede tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" supersede ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs overwrite tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" overwrite ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs open_id tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" open_id ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs io tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" io ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs mmap tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" mmap ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs rename tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" rename ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs rename_ex tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" rename_ex ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs delete tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" delete ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs delete_ex tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" delete_ex ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs links tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" links ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs links_ex tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" links_ex ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs oplock_i tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_i ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs oplock_ii tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_ii ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs oplock_batch tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_batch ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs oplock_filter tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_filter ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs oplock_r tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_r ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs oplock_rw tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_rw ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs oplock_rh tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_rh ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs oplock_rwh tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_rwh ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs cs tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" cs ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs reparse tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" reparse ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs streams tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" streams ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + #- name: run winbtrfs ea tests + # if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + # run: '& "${{github.workspace}}\winbtrfs\test.exe" ea ${{ steps.drive.outputs.drive }}' + + - name: run winbtrfs fileinfo tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" fileinfo ${{ steps.drive.outputs.drive }}' + + - run: Start-Sleep -Seconds 10 + timeout-minutes: 1 + + - name: run winbtrfs ea tests + if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} + timeout-minutes: 1 + run: '& "${{github.workspace}}\winbtrfs\test.exe" ea ${{ steps.drive.outputs.drive }}' + + #- name: run export zpool + # if: ${{ always() }} + # timeout-minutes: 1 + # run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" export tank' + + + + + + + + + + + + + + + + + + + + + + + + + + + +# This is a basic workflow to help you get started with Actions +# https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/ + +# test3: +# needs: [build_windows, build_wsl] +# runs-on: windows-latest +# steps: +# +# - uses: actions/checkout@v3 +# +# +# - uses: actions/download-artifact@v3 +# with: +# name: dev_build_inno +# +## - uses: actions/download-artifact@v3 +## with: +## name: build result +# +## - name: get files +## run: Get-ChildItem -Recurse +# +# - name: get diskdrive +# run: wmic diskdrive list +# +## - name: Download +## uses: suisei-cn/actions-download-file@v1.3.0 +## #id: innoinstaller +## with: +## url: 'https://github.com/andrewc12/zfsfiledump/raw/main/testdrives.zip' +## target: ${{github.workspace}}/ +## +## +## - name: make disk +## run: | +## Expand-Archive -LiteralPath ${{github.workspace}}/testdrives.zip -DestinationPath D:\ +## +## - name: Download +## uses: suisei-cn/actions-download-file@v1.3.0 +## #id: innoinstaller +## with: +## url: 'https://github.com/andrewc12/zfsfiledump/raw/main/scriptname.txt' +## target: ${{github.workspace}}/ +## +## - name: make disk +## run: | +## diskpart /s scriptname.txt +## +## +## +## - name: get diskdrive +## run: wmic diskdrive list +## +## +## - name: get files +## run: Get-ChildItem -Recurse +# +# +# - name: get zfsexename +# id: zfsinstaller +# run: | +# $p = Get-ChildItem | Where-Object {$_.Name -like 'OpenZFSOnWindows-*.exe'} | Select-Object -first 1 +# echo $p +# $f = (Get-Item $p ).Name +# echo $f +# echo "::set-output name=filename::$($f)" +# +# # https://github.com/MicrosoftDocs/windows-powershell-docs/issues/266 +# - name: Import root certificate +# run: | +# $plaintextpwd = 'password1234' +# $pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText +# Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\Root -Password $pwd +# Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\TrustedPublisher -Password $pwd +# +# - name: debug - echo filename +# run: echo ${{ steps.zfsinstaller.outputs.filename }} +# +## - name: debug - list +## run: ls ${{github.workspace}}\ +# +# - name: debug - list +# run: ls ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} +# +# #- name: install zfs +# # run: ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} /NORESTART /ALLUSERS /VERYSILENT /LOG="${{github.workspace}}\InnoSetup-Install.log" +# +# - name: install zfs +# run: 'Start-Process -FilePath "${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }}" -Wait -ArgumentList "/NORESTART /ALLUSERS /VERYSILENT /LOG=`"${{github.workspace}}\InnoSetup-Install.log`""' +# +# #- name: Wait for install to finish +# # run: Start-Sleep -Seconds 30 +# # uses: iFaxity/wait-on-action@v1 +# # with: +# # resource: 'C:\Program Files\OpenZFS On Windows\zpool.exe' +# +## - name: debug - print log +## run: cat "${{github.workspace}}\InnoSetup-Install.log" +## +## - name: debug - list +## run: ls "C:\Program Files" +## +## - name: debug - list +## run: ls "C:\Program Files\OpenZFS On Windows" +# +# - name: debug - get status +# run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' +# +# +# +# +# +# +# +# - name: Create backing files +# run: | +# $f = new-object System.IO.FileStream ${{github.workspace}}\test01.dat, Create, ReadWrite +# $f.SetLength(1GB) +# $f.Close() +# $f = new-object System.IO.FileStream ${{github.workspace}}\test02.dat, Create, ReadWrite +# $f.SetLength(1GB) +# $f.Close() +# $f = new-object System.IO.FileStream ${{github.workspace}}\test03.dat, Create, ReadWrite +# $f.SetLength(1GB) +# $f.Close() +# +## - run: choco install gsudo +# +# - name: tests/functional/cli_root/zpool_create/zpool_create_001_pos +# run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test01 \\?\${{github.workspace}}\test01.dat' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test01' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test02 \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test02' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test03 \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat \\?\${{github.workspace}}\test03.dat' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test03' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test04 mirror \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test04' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test05 mirror \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat \\?\${{github.workspace}}\test03.dat' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test05' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test06 raidz \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat \\?\${{github.workspace}}\test03.dat' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test06' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create -f test07 raidz1 \\?\${{github.workspace}}\test01.dat \\?\${{github.workspace}}\test02.dat \\?\${{github.workspace}}\test03.dat' +# - run: Start-Sleep -Seconds 10 +# +# - run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" destroy -f test07' +# - run: Start-Sleep -Seconds 10 +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +# +## - name: Setup WSL +## id: mywslid +## uses: Vampire/setup-wsl@v1.3.1 +## with: +## wsl-shell-user: root +## +## - shell: wsl-bash {0} +## run: | +## id +## +## - shell: wsl-bash {0} +## run: | +## apt update -y +## +## - shell: wsl-bash {0} +## run: | +## apt install libasan5 ksh wget sudo passwd adduser python3 rsync -y +## +## - shell: wsl-bash {0} +## run: | +## #dpkg -i libzfs5_2.1.99-1_amd64.deb +## #dpkg -i libnvpair3_2.1.99-1_amd64.deb +## #dpkg -i libuutil3_2.1.99-1_amd64.deb +## #dpkg -i zfs_2.1.99-1_amd64.deb +## dpkg -i zfs-test_2.1.99-1_amd64.deb +## #dpkg -i python3-pyzfs_2.1.99-1_amd64.deb +## +## +## - shell: wsl-bash {0} +## run: | +## useradd -m -p 4qBD5NWD3IkbU test +## +## - shell: wsl-bash {0} +## run: | +## adduser test sudo +## +## - shell: wsl-bash {0} +## run: | +## ls -la /etc +## +## +## - shell: wsl-bash {0} +## run: | +## chmod ug+rw /etc/sudoers +## - shell: wsl-bash {0} +## run: | +## cat /etc/sudoers +## exit 0 +## +## - shell: wsl-bash {0} +## run: | +## echo "test ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers +## exit 0 +## +## - shell: wsl-bash {0} +## run: | +## chmod ug-w /etc/sudoers +## +## +## - shell: wsl-bash {0} +## run: | +## cat /usr/share/zfs/common.sh +## +## +## - shell: wsl-bash {0} +## run: | +## sed -i -e 's#BIN_DIR=/usr/bin#BIN_DIR="/mnt/c/zfs"#' /usr/share/zfs/common.sh +## +## +## +## - shell: wsl-bash {0} +## run: | +## sed -i -e 's#SBIN_DIR=/sbin#SBIN_DIR="/mnt/c/zfs"#' /usr/share/zfs/common.sh +## +## +## +## - shell: wsl-bash {0} +## run: | +## cat /usr/share/zfs/common.sh +## +## +## - shell: wsl-bash {0} +## run: | +## cp /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zpool.exe /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zpool +## cp /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zfs.exe /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zfs +## cp /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zdb.exe /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zdb +## cp /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zstreamdump.exe /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zstreamdump +## mkdir /mnt/c/zfs +## cp /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zpool.exe /mnt/c/zfs/zpool +## cp /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zfs.exe /mnt/c/zfs/zfs +## cp /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zdb.exe /mnt/c/zfs/zdb +## cp /mnt/c/Program\ Files/OpenZFS\ On\ Windows/zstreamdump.exe /mnt/c/zfs/zstreamdump +## +## - name: debug clean +## shell: cmd +## run: DEL /F "${{ steps.mywslid.outputs.wsl-shell-wrapper-path }}" +## +## - uses: Vampire/setup-wsl@v1 +## with: +## wsl-shell-user: test +## +## - shell: wsl-bash {0} +## run: id +## +## +## +## +## +## - shell: wsl-bash {0} +## run: id +## +## +## +## +## +## - shell: wsl-bash {0} +## run: | +## export DISKS='physicaldrive2 physicaldrive3 physicaldrive4' +## export RAID_TEST_DISKS='physicaldrive5 physicaldrive6 physicaldrive7 physicaldrive8 physicaldrive9' +## #openzfs_folder + "scripts/zfs-tests.sh"+" -r "+ openzfs_folder +"tests/runfiles/windows-All.run" +## #bash -x /usr/share/zfs/zfs-tests.sh -v || exit 0 +## cat /usr/share/zfs/common.sh +## bash -x /usr/share/zfs/zfs-tests.sh -v -r windows-All.run || exit 0 +## +## +## - shell: wsl-bash {0} +## run: | +## sudo rsync -avx --no-links /var/tmp/test_results /mnt/d/test_results +## sudo chown -R test:test /mnt/d/test_results +## sudo chmod -R a+r /mnt/d/test_results +## exit 0 +## +## +## +## - name: Upload dev build with Inno Setup +## uses: actions/upload-artifact@v3.1.0 +## with: +## name: testres +## path: d:\test_results\* +## +## - name: create empty backing file +## run: | +## $f = new-object System.IO.FileStream ${{github.workspace}}\test.dat, Create, ReadWrite +## $f.SetLength(1GB) +## $f.Close() +## +## - name: create pool +## run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create tank \\?\${{github.workspace}}\test.dat' +## +## - name: get pool status +## run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' +## +## - name: get pool mount +## run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount' +## +## - name: get pool mount +## id: drive # Remember to give an ID if you need the output filename +## run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount | Select-String -Pattern "^tank +([A-Za-z]:\\)" | % {"::set-output name=drive::$($_.matches.groups[1].value)"}' +## +## - name: echo +## run: echo ${{ steps.drive.outputs.drive }} +## +## - name: get +## run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" get all' +## +## - name: get bcd +## run: 'bcdedit' +## +## - name: Download winbtrfs +## uses: suisei-cn/actions-download-file@v1.3.0 +## id: winbtrs # Remember to give an ID if you need the output filename +## with: +## url: "https://github.com/andrewc12/btrfs/releases/download/v1.8/x64-Debug.zip" +## target: ${{github.workspace}}/ +## +## - name: extract +## run: Expand-Archive -LiteralPath '${{ steps.winbtrs.outputs.filename }}' -DestinationPath ${{github.workspace}}\winbtrfs +## +## - name: debug - list +## run: ls ${{github.workspace}}\ +## +## - name: debug - list +## run: ls ${{github.workspace}}\winbtrfs +## +## +## #- name: run winbtrfs tests +## # run: '& "${{github.workspace}}\winbtrfs\test.exe" ${{ steps.drive.outputs.drive }}' +## +## - name: test dummy status step +## id: dummy +## run: exit 0 +## +## - name: debug - dummy status +## run: echo ${{ steps.dummy.conclusion }} +## +## +## #https://github.com/maharmstone/btrfs/blob/master/src/tests/test.cpp#L453 +## - name: run winbtrfs create tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" create ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs supersede tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" supersede ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs overwrite tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" overwrite ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs open_id tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" open_id ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs io tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" io ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs mmap tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" mmap ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs rename tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" rename ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs rename_ex tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" rename_ex ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs delete tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" delete ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs delete_ex tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" delete_ex ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs links tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" links ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs links_ex tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" links_ex ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs oplock_i tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_i ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs oplock_ii tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_ii ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs oplock_batch tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_batch ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs oplock_filter tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_filter ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs oplock_r tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_r ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs oplock_rw tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_rw ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs oplock_rh tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_rh ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs oplock_rwh tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" oplock_rwh ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs cs tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" cs ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs reparse tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" reparse ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs streams tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" streams ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs ea tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: | +## echo these crash +## exit 1 +## #- name: run winbtrfs ea tests +## # if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## # run: '& "${{github.workspace}}\winbtrfs\test.exe" ea ${{ steps.drive.outputs.drive }}' +## +## - name: run winbtrfs fileinfo tests +## if: ${{ ( success() || failure() ) && steps.dummy.conclusion == 'success' }} +## run: '& "${{github.workspace}}\winbtrfs\test.exe" fileinfo ${{ steps.drive.outputs.drive }}' +## +## +## #- name: run export zpool +## # if: ${{ always() }} +## # timeout-minutes: 1 +## # run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" export tank' +## +## +## +## +## +## +## #- name: small test +## # run: winsat disk -drive e +## +## +## + + + + + + + + + + + + +# rebase_info: +# runs-on: ubuntu-latest +# +# steps: +# +# - uses: actions/checkout@v3 +# with: +# #repository: openzfsonwindows/openzfs +# fetch-depth: 0 +# +# - run: | +# git config --global user.email "you@example.com" +# git config --global user.name "Your Name" +# +# - run: git remote add openzfs https://github.com/openzfs/zfs.git +# +# - run: git fetch --all +# +# - run: git rebase openzfs/master +# + + + + + + + + + + + + + + + + + + + + + + +# This is a basic workflow to help you get started with Actions +# https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/ + + test3_winfstest: + needs: [build_windows] #, build_wsl] + timeout-minutes: 30 + runs-on: windows-latest + if: ${{ inputs.run-failing }} + steps: + + - uses: actions/checkout@v3 + + + - uses: actions/download-artifact@v3 + with: + name: dev_build_inno + +# - uses: actions/download-artifact@v3 +# with: +# name: build result + +# - name: get files +# run: Get-ChildItem -Recurse + + - name: get diskdrive + run: wmic diskdrive list + +# - name: Download +# uses: suisei-cn/actions-download-file@v1.3.0 +# #id: innoinstaller +# with: +# url: 'https://github.com/andrewc12/zfsfiledump/raw/main/testdrives.zip' +# target: ${{github.workspace}}/ +# +# +# - name: make disk +# run: | +# Expand-Archive -LiteralPath ${{github.workspace}}/testdrives.zip -DestinationPath D:\ +# +# - name: Download +# uses: suisei-cn/actions-download-file@v1.3.0 +# #id: innoinstaller +# with: +# url: 'https://github.com/andrewc12/zfsfiledump/raw/main/scriptname.txt' +# target: ${{github.workspace}}/ +# +# - name: make disk +# run: | +# diskpart /s scriptname.txt +# +# +# +# - name: get diskdrive +# run: wmic diskdrive list +# +# +# - name: get files +# run: Get-ChildItem -Recurse + + + - name: get zfsexename + id: zfsinstaller + run: | + $p = Get-ChildItem | Where-Object {$_.Name -like 'OpenZFSOnWindows-*.exe'} | Select-Object -first 1 + echo $p + $f = (Get-Item $p ).Name + echo $f + echo "filename=$f" | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append + + # https://github.com/MicrosoftDocs/windows-powershell-docs/issues/266 + - name: Import root certificate + run: | + $plaintextpwd = 'password1234' + $pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\Root -Password $pwd + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\TrustedPublisher -Password $pwd + + - name: debug - echo filename + run: echo ${{ steps.zfsinstaller.outputs.filename }} + +# - name: debug - list +# run: ls ${{github.workspace}}\ + + - name: debug - list + run: ls ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} + + #- name: install zfs + # run: ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} /NORESTART /ALLUSERS /VERYSILENT /LOG="${{github.workspace}}\InnoSetup-Install.log" + + - name: install zfs + run: 'Start-Process -FilePath "${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }}" -Wait -ArgumentList "/NORESTART /ALLUSERS /VERYSILENT /LOG=`"${{github.workspace}}\InnoSetup-Install.log`""' + + #- name: Wait for install to finish + # run: Start-Sleep -Seconds 30 + # uses: iFaxity/wait-on-action@v1 + # with: + # resource: 'C:\Program Files\OpenZFS On Windows\zpool.exe' + +# - name: debug - print log +# run: cat "${{github.workspace}}\InnoSetup-Install.log" +# +# - name: debug - list +# run: ls "C:\Program Files" +# +# - name: debug - list +# run: ls "C:\Program Files\OpenZFS On Windows" + + - name: debug - get status + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' + + + + - run: choco install gsudo + + + + - name: Create backing files + run: | + $f = new-object System.IO.FileStream ${{github.workspace}}\test01.dat, Create, ReadWrite + $f.SetLength(1GB) + $f.Close() + + - name: create pool + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create tank \\?\${{github.workspace}}\test01.dat' + + - name: get pool status + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' + + - name: get pool mount + run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount' + + - name: get pool mount + id: drive # Remember to give an ID if you need the output filename + run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount | Select-String -Pattern "^tank +([A-Za-z]):[/\\]" | % {"drive=$($_.matches.groups[1].value):\"} | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append' + + - name: get pool mount + id: drivenoslash # Remember to give an ID if you need the output filename + run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount | Select-String -Pattern "^tank +([A-Za-z]):[/\\]" | % {"drive=$($_.matches.groups[1].value):"} | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append' + + - name: echo + run: echo ${{ steps.drive.outputs.drive }} + + + - name: Download winbtrfs + uses: suisei-cn/actions-download-file@v1.3.0 + id: winbtrs # Remember to give an ID if you need the output filename + with: + url: "https://github.com/andrewc12/btrfs/releases/download/v1.8/x64-Debug.zip" + target: ${{github.workspace}}/ + + - name: extract + run: Expand-Archive -LiteralPath '${{ steps.winbtrs.outputs.filename }}' -DestinationPath ${{github.workspace}}\winbtrfs + + - name: debug - list + run: ls ${{github.workspace}}\ + + - name: debug - list + run: ls ${{github.workspace}}\winbtrfs + + + #- name: run winbtrfs tests + # run: '& "${{github.workspace}}\winbtrfs\test.exe" ${{ steps.drive.outputs.drive }}' + + - name: test dummy status step + id: dummy + run: exit 0 + + - name: debug - dummy status + run: echo ${{ steps.dummy.conclusion }} + + + + +# - name: install winfstest +# run: | +# Copy-Item -Path "${{github.workspace}}\cmd\os\windows\winfstest" -Destination "${{ steps.drive.outputs.drive }}\winfstest" -Recurse +# Copy-Item "${{github.workspace}}\out\build\x64-Debug\cmd\os\windows\winfstest\winfstest.exe" -Destination "${{ steps.drive.outputs.drive }}\winfstest" + + - name: Download + uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/winfstest.zip' + target: ${{github.workspace}}/ + + + - name: extract winfstest + run: | + new-item ${{ steps.drive.outputs.drive }}\winfstest -itemtype directory + Expand-Archive -LiteralPath ${{github.workspace}}/winfstest.zip -DestinationPath ${{ steps.drive.outputs.drive }}\winfstest + + + #- name: run winbtrfs tests + # run: '& "${{github.workspace}}\winbtrfs\test.exe" ${{ steps.drive.outputs.drive }}' + + #https://github.com/maharmstone/btrfs/blob/master/src/tests/test.cpp#L453 + - name: run winfstests + timeout-minutes: 1 + run: | + ${{ steps.drivenoslash.outputs.drive }} + cd ${{ steps.drive.outputs.drive }}\winfstest + set PYTHONPATH=${{ steps.drive.outputs.drive }}\winfstest + python ${{ steps.drive.outputs.drive }}\winfstest\simpletap.py --run + shell: cmd + + + + + + + + test4_adstest: + needs: [build_windows] #, build_wsl] + timeout-minutes: 30 + # The CMake configure and build commands are platform agnostic and should work equally well on Windows or Mac. + # You can convert this to a matrix build if you need cross-platform coverage. + # See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix + runs-on: windows-latest + if: ${{ inputs.run-failing }} + + steps: + + - uses: actions/checkout@v3 + + + - uses: actions/download-artifact@v3 + with: + name: dev_build_inno + +# - uses: actions/download-artifact@v3 +# with: +# name: build result + +# - name: get files +# run: Get-ChildItem -Recurse + + - name: get diskdrive + run: wmic diskdrive list + + + #- uses: suisei-cn/actions-download-file@v1.3.0 + # #id: innoinstaller + # with: + # url: 'https://github.com/andrewc12/zfsfiledump/raw/main/OpenZFSOnWindowsaa' + # target: ${{github.workspace}}/ + + + + #- uses: suisei-cn/actions-download-file@v1.3.0 + # #id: innoinstaller + # with: + # url: 'https://github.com/andrewc12/zfsfiledump/raw/main/OpenZFSOnWindowsab' + # target: ${{github.workspace}}/ + + + + #- uses: suisei-cn/actions-download-file@v1.3.0 + # #id: innoinstaller + # with: + # url: 'https://github.com/andrewc12/zfsfiledump/raw/main/OpenZFSOnWindowsac' + # target: ${{github.workspace}}/ + + + + #- uses: suisei-cn/actions-download-file@v1.3.0 + # #id: innoinstaller + # with: + # url: 'https://github.com/andrewc12/zfsfiledump/raw/main/OpenZFSOnWindowsad' + # target: ${{github.workspace}}/ + + + + + + + + +# - name: Download +# uses: suisei-cn/actions-download-file@v1.3.0 +# #id: innoinstaller +# with: +# url: 'https://github.com/andrewc12/zfsfiledump/raw/main/testdrives.zip' +# target: ${{github.workspace}}/ +# +# +# - name: make disk +# run: | +# Expand-Archive -LiteralPath ${{github.workspace}}/testdrives.zip -DestinationPath D:\ +# +# - name: Download +# uses: suisei-cn/actions-download-file@v1.3.0 +# #id: innoinstaller +# with: +# url: 'https://github.com/andrewc12/zfsfiledump/raw/main/scriptname.txt' +# target: ${{github.workspace}}/ +# +# - name: make disk +# run: | +# diskpart /s scriptname.txt +# +# +# +# - name: get diskdrive +# run: wmic diskdrive list +# +# +# - name: get files +# run: Get-ChildItem -Recurse + + #- run: copy /b OpenZFSOnWindowsaa+OpenZFSOnWindowsab+OpenZFSOnWindowsac+OpenZFSOnWindowsad OpenZFSOnWindows-a.exe + # shell: cmd + + + + - name: get zfsexename + id: zfsinstaller + run: | + $p = Get-ChildItem | Where-Object {$_.Name -like 'OpenZFSOnWindows-*.exe'} | Select-Object -first 1 + echo $p + $f = (Get-Item $p ).Name + echo $f + echo "filename=$f" | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append + + + + + - uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/ads_split.vhd.zst.001' + target: ${{github.workspace}}/ + + + + - uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/ads_split.vhd.zst.002' + target: ${{github.workspace}}/ + + + + - uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/ads_split.vhd.zst.003' + target: ${{github.workspace}}/ + + + + - uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/ads_split.vhd.zst.004' + target: ${{github.workspace}}/ + + + + - uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/ads_split.vhd.zst.005' + target: ${{github.workspace}}/ + + + + + - uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/ads_split.vhd.zst.006' + target: ${{github.workspace}}/ + + + + - uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/ads_split.vhd.zst.007' + target: ${{github.workspace}}/ + + + + + - uses: suisei-cn/actions-download-file@v1.3.0 + #id: innoinstaller + with: + url: 'https://github.com/andrewc12/zfsfiledump/raw/main/ads_split.vhd.zst.008' + target: ${{github.workspace}}/ + + + + + - run: copy /b ads_split.vhd.zst.001+ads_split.vhd.zst.002+ads_split.vhd.zst.003+ads_split.vhd.zst.004+ads_split.vhd.zst.005+ads_split.vhd.zst.006+ads_split.vhd.zst.007+ads_split.vhd.zst.008 ads_split.vhd.zst + shell: cmd + + + + + # https://github.com/MicrosoftDocs/windows-powershell-docs/issues/266 + - name: Import root certificate + run: | + $plaintextpwd = 'password1234' + $pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\Root -Password $pwd + Import-PfxCertificate -FilePath ${{github.workspace}}/contrib/windows/TestCert/test_sign_cert_pass.pfx -CertStoreLocation Cert:\LocalMachine\TrustedPublisher -Password $pwd + + - name: debug - echo filename + run: echo ${{ steps.zfsinstaller.outputs.filename }} + +# - name: debug - list +# run: ls ${{github.workspace}}\ + + - name: debug - list + run: ls ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} + + #- name: install zfs + # run: ${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }} /NORESTART /ALLUSERS /VERYSILENT /LOG="${{github.workspace}}\InnoSetup-Install.log" + + - name: install zfs + run: 'Start-Process -FilePath "${{github.workspace}}\${{ steps.zfsinstaller.outputs.filename }}" -Wait -ArgumentList "/NORESTART /ALLUSERS /VERYSILENT /LOG=`"${{github.workspace}}\InnoSetup-Install.log`""' + + #- name: Wait for install to finish + # run: Start-Sleep -Seconds 30 + # uses: iFaxity/wait-on-action@v1 + # with: + # resource: 'C:\Program Files\OpenZFS On Windows\zpool.exe' + +# - name: debug - print log +# run: cat "${{github.workspace}}\InnoSetup-Install.log" +# +# - name: debug - list +# run: ls "C:\Program Files" +# +# - name: debug - list +# run: ls "C:\Program Files\OpenZFS On Windows" + + - name: debug - get status + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' + + + + + + + + - name: Create backing files + run: | + $f = new-object System.IO.FileStream ${{github.workspace}}\test01.dat, Create, ReadWrite + $f.SetLength(1GB) + $f.Close() + + - name: create pool + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" create tank \\?\${{github.workspace}}\test01.dat' + + - name: get pool status + run: '& "C:\Program Files\OpenZFS On Windows\zpool.exe" status' + + - name: get pool mount + run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount' + + - name: get pool mount + id: drive # Remember to give an ID if you need the output filename + run: '& "C:\Program Files\OpenZFS On Windows\zfs.exe" mount | Select-String -Pattern "^tank +([A-Za-z]):[/\\]" | % {"drive=$($_.matches.groups[1].value):\"} | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append' + + - name: echo + run: echo ${{ steps.drive.outputs.drive }} + + - name: extract + run: zstd -d ${{github.workspace}}\ads_split.vhd.zst + + + - name: get diskdrive + run: wmic diskdrive list + + - name: get diskdrive + run: (Get-PSDrive).Name -match '^[a-z]$' + + + - name: Create backing files + id: drive2 # Remember to give an ID if you need the output filename + run: | + $mountResult = Mount-DiskImage ${{github.workspace}}\ads_split.vhd -PassThru + $mountResult | Get-Volume + echo "drive=F:"| Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append + #$mountResult | Get-Volume | % {"drive=$($_.matches.groups[1].value)"} | Out-File -FilePath $Env:GITHUB_OUTPUT -Encoding utf8 -Append + + + - name: get diskdrive + run: wmic diskdrive list + + + - name: get diskdrive + run: (Get-PSDrive).Name -match '^[a-z]$' + + - name: echo + run: echo ${{ steps.drive2.outputs.drive }} + + + - name: run tests + run: robocopy ${{ steps.drive2.outputs.drive }}\curtin ${{ steps.drive.outputs.drive }}curtin /E /b /R:0 /W:0 diff --git a/.gitignore b/.gitignore index 8d91dd9466c5..a0c090630029 100644 --- a/.gitignore +++ b/.gitignore @@ -59,7 +59,8 @@ !RELEASES.md !TEST !zfs.release.in - +!CMakeLists.txt +!CMakeSettings.json # # Normal rules diff --git a/AUTHORS b/AUTHORS index c2af58d75085..3d5d7973e764 100644 --- a/AUTHORS +++ b/AUTHORS @@ -38,6 +38,7 @@ CONTRIBUTORS: Andreas Dilger Andrew Barnes Andrew Hamilton + Andrew Innes Andrew Reid Andrew Stormont Andrew Tselischev diff --git a/CMakeLists.txt b/CMakeLists.txt new file mode 100644 index 000000000000..5e95afd8daee --- /dev/null +++ b/CMakeLists.txt @@ -0,0 +1,97 @@ +cmake_minimum_required(VERSION 3.13) +project(OpenZFS LANGUAGES C CXX ASM) +SET (CMAKE_ASM_COMPILER_ID Clang) +SET (CMAKE_ASM_COMPILER clang-cl.exe) + +include(GNUInstallDirs) + +list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/contrib/windows/cmake") +find_package(WDK REQUIRED) + +option(BUILD_SHARED_LIBS "Build Shared Libraries" OFF) + +set(OPENSSL_USE_STATIC_LIBS TRUE) +set(OPENSSL_MSVC_STATIC_RT TRUE) +find_package(OpenSSL REQUIRED) +# message(STATUS "OpenSSL libs: " ${OPENSSL_LIBRARIES}) +# Seriously, why does it pick MD versions and not MT like requested? +# string(REGEX REPLACE "64MD" "64MT" INTERNAL_OPENSSL_LIBRARIES ${OPENSSL_LIBRARIES}) +#message(STATUS "OpenSSL libs: " ${OPENSSL_CRYPTO_LIBRARY}) +#cmake_path(GET ${OPENSSL_LIBRARIES} ROOT_PATH rootPath) +#get_filename_component(rootPath "${OPENSSL_CRYPTO_LIBRARY}" DIRECTORY) +#message(STATUS "OpenSSL root: " ${rootPath}) + + +# Attempt to simulate scripts/make-gitrev.h and zfs_gitrev.h +include(GetGitRevisionDescription) +include(GetGitRevisionDescription) +git_describe_working_tree(GIT_GITREV) +configure_file("${CMAKE_CURRENT_SOURCE_DIR}/include/zfs_gitrev.h.win" "${CMAKE_CURRENT_SOURCE_DIR}/include/zfs_gitrev.h" @ONLY) +# + +function(use_clang) +# It seems "-include file" is "/FI file" on Windows +add_compile_options(/FI ${CMAKE_SOURCE_DIR}/include/os/windows/zfs/zfs_config.h) +set(CMAKE_C_COMPILER clang-cl.exe PARENT_SCOPE) +set(CMAKE_CXX_COMPILER clang-cl.exe PARENT_SCOPE) +if(CMAKE_SIZEOF_VOID_P EQUAL 4) + add_compile_options(-m32) +else() + add_compile_options(-m64) +endif() + +add_compile_options( + # These are all MS headers + -Wno-nonportable-include-path + -Wno-unknown-pragmas + -Wno-ignored-pragma-intrinsic + -Wno-pragma-pack + -Wno-microsoft-enum-forward-reference + -Wno-visibility + -Wno-microsoft-anon-tag + # -Wno-ignored-attributes + -Wno-unused-command-line-argument + -Wno-unused-local-typedef + -Wno-int-to-void-pointer-cast + # ZFS related + -Wno-misleading-indentation + -Wno-dangling-else + -Wno-missing-field-initializers + #-fms-extensions + -Wno-unused-function + -Wno-unused-label + #Visual Studio 2022 update 2022114 started breaking builds with errors + -Wno-int-conversion +) +endfunction() + +if(CMAKE_SIZEOF_VOID_P EQUAL 4) + add_definitions(-D__x86_32__ -D__i386 -D__i386__ -D_LP32 -DWIN32) +elseif(CMAKE_SIZEOF_VOID_P EQUAL 8) + add_definitions(-D__x86_64__ -D_LP64 -D__LP64__ -D__x86_64 -D_AMD64_ -D_WIN64 -DAMD64 -DWIN64) +else() + message(FATAL_ERROR "Unsupported architecture") +endif() + +if(CMAKE_BUILD_TYPE STREQUAL "Debug") + add_definitions(-DDBG -DZFS_DEBUG) +endif() + +# Avoid dependency add_compile_definitions vcruntime140.dll +set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} /MT") +set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /MTd") +set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} /MT") +set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MTd") + +# Let's try to emulate gitrev +#find_path(BASH_DIR NAMES bash.exe git-bash.exe PATHS "/c/Program Files (x86)/Git/") +#execute_process(COMMAND $BASH_DIR -c "scripts/make_gitrev.h include/zfs_gitrev.h" ) + + +add_subdirectory(module) +#add_subdirectory(module/zfs) +#add_subdirectory(zfsinstaller) + +add_subdirectory(lib) +add_subdirectory(cmd) + diff --git a/CMakeSettings.json b/CMakeSettings.json new file mode 100644 index 000000000000..282a2d48f8dc --- /dev/null +++ b/CMakeSettings.json @@ -0,0 +1,28 @@ +{ + "configurations": [ + { + "name": "x64-Debug", + "generator": "Ninja", + "configurationType": "Debug", + "inheritEnvironments": [ "msvc_x64" ], + "buildRoot": "${projectDir}\\out\\build\\${name}", + "installRoot": "${projectDir}\\out\\install\\${name}", + "cmakeCommandArgs": "", + "buildCommandArgs": "", + "ctestCommandArgs": "", + "variables": [] + }, + { + "name": "x64-Release", + "generator": "Ninja", + "configurationType": "RelWithDebInfo", + "buildRoot": "${projectDir}\\out\\build\\${name}", + "installRoot": "${projectDir}\\out\\install\\${name}", + "cmakeCommandArgs": "", + "buildCommandArgs": "", + "ctestCommandArgs": "", + "inheritEnvironments": [ "msvc_x64_x64" ], + "variables": [] + } + ] +} \ No newline at end of file diff --git a/Makefile.am b/Makefile.am index 11e45dae8255..1401d2600420 100644 --- a/Makefile.am +++ b/Makefile.am @@ -128,6 +128,7 @@ cstyle: ! -path './include/sys/lua/*' \ ! -path './module/lua/l*.[ch]' \ ! -path './module/zfs/lz4.c' \ + ! -path './lib/os/windows/zlib-1.2.13/*' \ $(cstyle_line) filter_executable = -exec test -x '{}' \; -print diff --git a/README.md b/README.md index 331889560950..4599b2f16d2c 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,16 @@ + ![img](https://openzfs.github.io/openzfs-docs/_static/img/logo/480px-Open-ZFS-Secondary-Logo-Colour-halfsize.png) OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD. +The Windows port is considered 'Beta' quality, your ZFS data will be safe +but you might come across system crashes if you discover a bug. Please +file any bugs on GitHub. + +Developers for Windows, please see [Windows ReadMe](https://github.com/openzfsonwindows/openzfs/tree/windows/module/os/windows). + [![codecov](https://codecov.io/gh/openzfs/zfs/branch/master/graph/badge.svg)](https://codecov.io/gh/openzfs/zfs) [![coverity](https://scan.coverity.com/projects/1973/badge.svg)](https://scan.coverity.com/projects/openzfs-zfs) diff --git a/cmd/CMakeLists.txt b/cmd/CMakeLists.txt new file mode 100644 index 000000000000..21baf36ad801 --- /dev/null +++ b/cmd/CMakeLists.txt @@ -0,0 +1,13 @@ + +include_directories(PRIVATE "${CMAKE_SOURCE_DIR}/lib/libspl/include/os/windows/" "${CMAKE_SOURCE_DIR}/lib/libspl/include" "${CMAKE_SOURCE_DIR}/include" + "${CMAKE_SOURCE_DIR}/lib/os/windows/libpthread" "${OPENSSL_INCLUDE_DIR}") + +add_definitions(-D_CRT_SECURE_NO_WARNINGS) + +add_subdirectory(zpool) +add_subdirectory(zfs) +add_subdirectory(zdb) +add_subdirectory(zstream) +add_subdirectory(os/windows/kstat) +add_subdirectory(os/windows/zfsinstaller) +add_subdirectory(raidz_test) diff --git a/cmd/os/windows/kstat/CMakeLists.txt b/cmd/os/windows/kstat/CMakeLists.txt new file mode 100644 index 000000000000..4383fae5dc7f --- /dev/null +++ b/cmd/os/windows/kstat/CMakeLists.txt @@ -0,0 +1,13 @@ + +use_clang() + +um_add_executable(kstat kstat.c resource.rc) + +include_directories(PRIVATE "${CMAKE_SOURCE_DIR}/lib/os/windows") + +target_link_libraries(kstat PRIVATE libkstat) +install(TARGETS kstat RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}") +install(FILES $ + DESTINATION "${CMAKE_INSTALL_BINDIR}" + OPTIONAL +) diff --git a/cmd/os/windows/kstat/kstat.c b/cmd/os/windows/kstat/kstat.c new file mode 100644 index 000000000000..e8e73aafe429 --- /dev/null +++ b/cmd/os/windows/kstat/kstat.c @@ -0,0 +1,1723 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2013 David Hoeppner. All rights reserved. + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. + * Copyright 2016 Joyent, Inc. + * Copyright 2018 Jorgen Lundman . All rights reserved. + */ + +/* + * Display kernel statistics + * + * This is a reimplementation of the perl kstat command originally found + * under usr/src/cmd/kstat/kstat.pl + * + * Incompatibilities: + * - perl regular expressions replaced with extended REs bracketed by '/' + * + * Flags added: + * -C similar to the -p option but value is separated by a colon + * -h display help + * -j json format + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include // ick + +#include "kstat.h" +#include "statcommon.h" + +#pragma comment(lib, "Ws2_32.lib") + +extern hrtime_t gethrtime(void); + +char *cmdname = "kstat"; /* Name of this command */ +int caught_cont = 0; /* Have caught a SIGCONT */ + +static uint_t g_timestamp_fmt = NODATE; + +/* Helper flag - header was printed already? */ +static boolean_t g_headerflg; + +/* Saved command line options */ +static boolean_t g_cflg = B_FALSE; +static boolean_t g_jflg = B_FALSE; +static boolean_t g_lflg = B_FALSE; +static boolean_t g_pflg = B_FALSE; +static boolean_t g_qflg = B_FALSE; +static boolean_t g_wflg = B_FALSE; +static ks_pattern_t g_ks_class = {"*", 0}; + +static boolean_t g_matched = B_FALSE; + +/* Sorted list of kstat instances */ +static list_t instances_list; +static list_t selector_list; + +int +main(int argc, char **argv) +{ + ks_selector_t *nselector; + ks_selector_t *uselector; + kstat_ctl_t *kc; + hrtime_t start_n; + hrtime_t period_n; + boolean_t errflg = B_FALSE; + boolean_t nselflg = B_FALSE; + boolean_t uselflg = B_FALSE; + char *q; + int count = 1; + int infinite_cycles = 0; + int interval = 0; + int n = 0; + int c, m, tmp; + + (void) setlocale(LC_ALL, ""); +#if !defined(TEXT_DOMAIN) /* Should be defined by cc -D */ +#define TEXT_DOMAIN "SYS_TEST" /* Use this only if it wasn't */ +#endif + // (void) textdomain(TEXT_DOMAIN); + + /* + * Create the selector list and a dummy default selector to match + * everything. While we process the cmdline options we will add + * selectors to this list. + */ + list_create(&selector_list, sizeof (ks_selector_t), + offsetof(ks_selector_t, ks_next)); + + nselector = new_selector(); + + /* + * Parse named command line arguments. + */ + while ((c = getopt(argc, argv, "h?CqjlpT:m:i:n:s:c:w")) != EOF) + switch (c) { + case 'h': + case '?': + usage(); + exit(0); + break; + case 'C': + g_pflg = g_cflg = B_TRUE; + break; + case 'q': + g_qflg = B_TRUE; + break; + case 'j': + /* + * If we're printing JSON, we're going to force numeric + * representation to be in the C locale to assure that + * the decimal point is compliant with RFC 7159 (i.e., + * ASCII 0x2e). + */ +// (void) setlocale(LC_NUMERIC, "C"); + g_jflg = B_TRUE; + break; + case 'l': + g_pflg = g_lflg = B_TRUE; + break; + case 'p': + g_pflg = B_TRUE; + break; + case 'T': + switch (*optarg) { + case 'd': + g_timestamp_fmt = DDATE; + break; + case 'u': + g_timestamp_fmt = UDATE; + break; + default: + errflg = B_TRUE; + } + break; + case 'm': + nselflg = B_TRUE; + nselector->ks_module.pstr = + (char *)ks_safe_strdup(optarg); + break; + case 'i': + nselflg = B_TRUE; + nselector->ks_instance.pstr = + (char *)ks_safe_strdup(optarg); + break; + case 'n': + nselflg = B_TRUE; + nselector->ks_name.pstr = + (char *)ks_safe_strdup(optarg); + break; + case 's': + nselflg = B_TRUE; + nselector->ks_statistic.pstr = + (char *)ks_safe_strdup(optarg); + break; + case 'c': + g_ks_class.pstr = + (char *)ks_safe_strdup(optarg); + break; + case 'w': + g_wflg = B_TRUE; + break; + + default: + errflg = B_TRUE; + break; + } + + if (g_qflg && (g_jflg || g_pflg)) { + (void) fprintf(stderr, gettext( + "-q and -lpj are mutually exclusive\n")); + errflg = B_TRUE; + } + + if (errflg) { + usage(); + exit(2); + } + + argc -= optind; + argv += optind; + + if (g_wflg) { + /* + * kstat_write mode: consume commandline arguments: + * kstat -w module:instance:name:statistic_name=value + */ + n = write_mode(argc, argv); + exit(n); + } + + /* + * Consume the rest of the command line. Parsing the + * unnamed command line arguments. + */ + while (argc--) { + errno = 0; + tmp = strtoul(*argv, &q, 10); + if (tmp == ULONG_MAX && errno == ERANGE) { + if (n == 0) { + (void) fprintf(stderr, gettext( + "Interval is too large\n")); + } else if (n == 1) { + (void) fprintf(stderr, gettext( + "Count is too large\n")); + } + usage(); + exit(2); + } + + if (errno != 0 || *q != '\0') { + m = 0; + uselector = new_selector(); + while ((q = (char *)strsep(argv, ":")) != NULL) { + m++; + if (m > 4) { + free(uselector); + usage(); + exit(2); + } + + if (*q != '\0') { + switch (m) { + case 1: + uselector->ks_module.pstr = + (char *)ks_safe_strdup(q); + break; + case 2: + uselector->ks_instance.pstr = + (char *)ks_safe_strdup(q); + break; + case 3: + uselector->ks_name.pstr = + (char *)ks_safe_strdup(q); + break; + case 4: + uselector->ks_statistic.pstr = + (char *)ks_safe_strdup(q); + break; + default: + assert(B_FALSE); + } + } + } + + uselflg = B_TRUE; + list_insert_tail(&selector_list, uselector); + } else { + if (tmp < 1) { + if (n == 0) { + (void) fprintf(stderr, gettext( + "Interval must be an " + "integer >= 1")); + } else if (n == 1) { + (void) fprintf(stderr, gettext( + "Count must be an integer >= 1")); + } + usage(); + exit(2); + } else { + if (n == 0) { + interval = tmp; + count = -1; + } else if (n == 1) { + count = tmp; + } else { + usage(); + exit(2); + } + } + n++; + } + argv++; + } + + /* + * Check if we founded a named selector on the cmdline. + */ + if (uselflg) { + if (nselflg) { + (void) fprintf(stderr, gettext( + "[module[:instance[:name[:statistic]]]] and " + "-m -i -n -s are mutually exclusive")); + usage(); + exit(2); + } else { + free(nselector); + } + } else { + list_insert_tail(&selector_list, nselector); + } + + assert(!list_is_empty(&selector_list)); + + list_create(&instances_list, sizeof (ks_instance_t), + offsetof(ks_instance_t, ks_next)); + + while ((kc = kstat_open()) == NULL) { + if (errno == EAGAIN) { + (void) usleep(200); + } else { + perror("kstat_open"); + exit(3); + } + } + + if (count > 1) { +#ifndef WIN32 + if (signal(SIGCONT, cont_handler) == SIG_ERR) { + (void) fprintf(stderr, gettext( + "signal failed")); + exit(3); + } +#endif + } + + period_n = (hrtime_t)interval * NANOSEC; + start_n = gethrtime(); + + while (count == -1 || count-- > 0) { + ks_instances_read(kc); + ks_instances_print(); + + if (interval && count) { + ks_sleep_until(&start_n, period_n, infinite_cycles, + &caught_cont); + (void) kstat_chain_update(kc); + (void) putchar('\n'); + } + } + + (void) kstat_close(kc); + + /* + * Return a non-zero exit code if we didn't match anything. + */ + return (g_matched ? 0 : 1); +} + +/* + * Print usage. + */ +static void +usage(void) +{ + (void) fprintf(stderr, gettext( + "Usage:\n" + "kstat [ -Cjlpq ] [ -T d|u ] [ -c class ]\n" + " [ -m module ] [ -i instance ] [ -n name ] [ -s statistic ]\n" + " [ interval [ count ] ]\n" + "kstat [ -Cjlpq ] [ -T d|u ] [ -c class ]\n" + " [ module[:instance[:name[:statistic]]] ... ]\n" + " [ interval [ count ] ]\n" + "kstat -w module:instance:name:statistic=value [ ... ] \n")); +} + +/* + * Sort compare function. + */ +static int +compare_instances(ks_instance_t *l_arg, ks_instance_t *r_arg) +{ + int rval; + + rval = strcasecmp(l_arg->ks_module, r_arg->ks_module); + if (rval == 0) { + if (l_arg->ks_instance == r_arg->ks_instance) { + return (strcasecmp(l_arg->ks_name, r_arg->ks_name)); + } else if (l_arg->ks_instance < r_arg->ks_instance) { + return (-1); + } else { + return (1); + } + } else { + return (rval); + } +} + +static char * +ks_safe_strdup(char *str) +{ + char *ret; + + if (str == NULL) { + return (NULL); + } + + while ((ret = _strdup(str)) == NULL) { + if (errno == EAGAIN) { + (void) usleep(200); + } else { + perror("strdup"); + exit(3); + } + } + + return (ret); +} + +static void +ks_sleep_until(hrtime_t *wakeup, hrtime_t interval, int forever, + int *caught_cont) +{ + hrtime_t now, pause, pause_left; + struct timespec pause_tv; + int status; + + now = gethrtime(); + pause = *wakeup + interval - now; + + if (pause <= 0 || pause < (interval / 4)) { + if (forever || *caught_cont) { + *wakeup = now + interval; + pause = interval; + } else { + pause = interval / 2; + *wakeup += interval; + } + } else { + *wakeup += interval; + } + + if (pause < 1000) { + return; + } + + pause_left = pause; + do { + pause_tv.tv_sec = pause_left / NANOSEC; + pause_tv.tv_nsec = pause_left % NANOSEC; + status = nanosleep(&pause_tv, (struct timespec *)NULL); + if (status < 0) { + if (errno == EINTR) { + now = gethrtime(); + pause_left = *wakeup - now; + if (pause_left < 1000) { + return; + } + } else { + perror("nanosleep"); + exit(3); + } + } + } while (status != 0); +} + +/* + * Inserts an instance in the per selector list. + */ +static void +nvpair_insert(ks_instance_t *ksi, char *name, ks_value_t *value, + uchar_t data_type) +{ + ks_nvpair_t *instance; + ks_nvpair_t *tmp; + + instance = (ks_nvpair_t *)malloc(sizeof (ks_nvpair_t)); + if (instance == NULL) { + perror("malloc"); + exit(3); + } + + (void) strlcpy(instance->name, name, KSTAT_STRLEN); + (void) memcpy(&instance->value, value, sizeof (ks_value_t)); + instance->data_type = data_type; + + tmp = list_head(&ksi->ks_nvlist); + while (tmp != NULL && strcasecmp(instance->name, tmp->name) > 0) + tmp = list_next(&ksi->ks_nvlist, tmp); + + list_insert_before(&ksi->ks_nvlist, tmp, instance); +} + +/* + * Allocates a new all-matching selector. + */ +static ks_selector_t * +new_selector(void) +{ + ks_selector_t *selector; + + selector = (ks_selector_t *)malloc(sizeof (ks_selector_t)); + if (selector == NULL) { + perror("malloc"); + exit(3); + } + + list_link_init(&selector->ks_next); + + selector->ks_module.pstr = "*"; + selector->ks_instance.pstr = "*"; + selector->ks_name.pstr = "*"; + selector->ks_statistic.pstr = "*"; + + return (selector); +} + +/* + * This function was taken from the perl kstat module code - please + * see for further comments there. + */ +static kstat_raw_reader_t +lookup_raw_kstat_fn(char *module, char *name) +{ + char key[KSTAT_STRLEN * 2]; + register char *f, *t; + int n = 0; + + for (f = module, t = key; *f != '\0'; f++, t++) { + while (*f != '\0' && isdigit(*f)) + f++; + *t = *f; + } + *t++ = ':'; + + for (f = name; *f != '\0'; f++, t++) { + while (*f != '\0' && isdigit(*f)) + f++; + *t = *f; + } + *t = '\0'; + + while (ks_raw_lookup[n].fn != NULL) { + if (strncmp(ks_raw_lookup[n].name, key, strlen(key)) == 0) + return (ks_raw_lookup[n].fn); + n++; + } + + return (0); +} + +/* + * Match a string against a shell glob or extended regular expression. + */ +static boolean_t +ks_match(const char *str, ks_pattern_t *pattern) +{ +#ifndef WIN32 + int regcode; + char *regstr; + char *errbuf; + size_t bufsz; + + if (pattern->pstr != NULL && gmatch(pattern->pstr, "/*/") != 0) { + /* All regex patterns are strdup'd copies */ + regstr = pattern->pstr + 1; + *(strrchr(regstr, '/')) = '\0'; + + regcode = regcomp(&pattern->preg, regstr, + REG_EXTENDED | REG_NOSUB); + if (regcode != 0) { + bufsz = regerror(regcode, NULL, NULL, 0); + if (bufsz != 0) { + errbuf = malloc(bufsz); + if (errbuf == NULL) { + perror("malloc"); + exit(3); + } + (void) regerror(regcode, NULL, errbuf, bufsz); + (void) fprintf(stderr, "kstat: %s\n", errbuf); + } + usage(); + exit(2); + } + pattern->pstr = NULL; + } +#endif + +#ifndef WIN32 + if (pattern->pstr == NULL) { + return (regexec(&pattern->preg, str, 0, NULL, 0) == 0); + } + +#endif + return ((gmatch(str, pattern->pstr) != 0)); +} + +/* + * Iterate over all kernel statistics and save matches. + */ +static void +ks_instances_read(kstat_ctl_t *kc) +{ + kstat_raw_reader_t save_raw = NULL; + kid_t id; + ks_selector_t *selector; + ks_instance_t *ksi; + ks_instance_t *tmp; + kstat_t *kp; + boolean_t skip; + + for (kp = kc->kc_chain; kp != NULL; kp = kp->ks_next) { + /* Don't bother storing the kstat headers */ + if (strncmp(kp->ks_name, "kstat_", 6) == 0) { + continue; + } + + /* Don't bother storing raw stats we don't understand */ + if (kp->ks_type == KSTAT_TYPE_RAW) { + save_raw = lookup_raw_kstat_fn(kp->ks_module, + kp->ks_name); + if (save_raw == NULL) { +#ifdef REPORT_UNKNOWN + (void) fprintf(stderr, + "Unknown kstat type %s:%d:%s - " + "%d of size %d\n", kp->ks_module, + kp->ks_instance, kp->ks_name, + kp->ks_ndata, kp->ks_data_size); +#endif + continue; + } + } + + /* + * Iterate over the list of selectors and skip + * instances we dont want. We filter for statistics + * later, as we dont know them yet. + */ + skip = B_TRUE; + selector = list_head(&selector_list); + while (selector != NULL) { + if (ks_match(kp->ks_module, &selector->ks_module) && + ks_match(kp->ks_name, &selector->ks_name)) { + skip = B_FALSE; + break; + } + selector = list_next(&selector_list, selector); + } + + if (skip) { + continue; + } + + /* + * Allocate a new instance and fill in the values + * we know so far. + */ + ksi = (ks_instance_t *)malloc(sizeof (ks_instance_t)); + if (ksi == NULL) { + perror("malloc"); + exit(3); + } + + list_link_init(&ksi->ks_next); + + (void) strlcpy(ksi->ks_module, kp->ks_module, KSTAT_STRLEN); + (void) strlcpy(ksi->ks_name, kp->ks_name, KSTAT_STRLEN); + (void) strlcpy(ksi->ks_class, kp->ks_class, KSTAT_STRLEN); + + ksi->ks_instance = kp->ks_instance; + ksi->ks_snaptime = kp->ks_snaptime; + ksi->ks_type = kp->ks_type; + + list_create(&ksi->ks_nvlist, sizeof (ks_nvpair_t), + offsetof(ks_nvpair_t, nv_next)); + + SAVE_HRTIME_X(ksi, "crtime", kp->ks_crtime); + if (g_pflg) { + SAVE_STRING_X(ksi, "class", kp->ks_class); + } + + /* Insert this instance into a sorted list */ + tmp = list_head(&instances_list); + while (tmp != NULL && compare_instances(ksi, tmp) > 0) + tmp = list_next(&instances_list, tmp); + + list_insert_before(&instances_list, tmp, ksi); + + /* Read the actual statistics */ + id = kstat_read(kc, kp, NULL); + if (id == -1) { +#ifdef REPORT_UNKNOWN + perror("kstat_read"); +#endif + continue; + } + + SAVE_HRTIME_X(ksi, "snaptime", kp->ks_snaptime); + + switch (kp->ks_type) { + case KSTAT_TYPE_RAW: + save_raw(kp, ksi); + break; + case KSTAT_TYPE_NAMED: + save_named(kp, ksi); + break; + case KSTAT_TYPE_INTR: + save_intr(kp, ksi); + break; + case KSTAT_TYPE_IO: + save_io(kp, ksi); + break; + case KSTAT_TYPE_TIMER: + save_timer(kp, ksi); + break; + default: + assert(B_FALSE); /* Invalid type */ + break; + } + } +} + +/* + * Print the value of a name-value pair. + */ +static void +ks_value_print(ks_nvpair_t *nvpair) +{ + switch (nvpair->data_type) { + case KSTAT_DATA_CHAR: + (void) fprintf(stdout, "%s", nvpair->value.c); + break; + case KSTAT_DATA_INT32: + (void) fprintf(stdout, "%d", nvpair->value.i32); + break; + case KSTAT_DATA_UINT32: + (void) fprintf(stdout, "%u", nvpair->value.ui32); + break; + case KSTAT_DATA_INT64: + (void) fprintf(stdout, "%lld", nvpair->value.i64); + break; + case KSTAT_DATA_UINT64: + (void) fprintf(stdout, "%llu", nvpair->value.ui64); + break; + case KSTAT_DATA_STRING: + (void) fprintf(stdout, "%s", KSTAT_NAMED_STR_PTR(nvpair)); + break; + case KSTAT_DATA_HRTIME: + if (nvpair->value.ui64 == 0) + (void) fprintf(stdout, "0"); + else + (void) fprintf(stdout, "%.9f", + nvpair->value.ui64 / 1000000000.0); + break; + default: + assert(B_FALSE); + } +} + +/* + * Print a single instance. + */ + +static void +ks_instance_print(ks_instance_t *ksi, ks_nvpair_t *nvpair, boolean_t last) +{ + if (g_headerflg) { + if (!g_pflg) { + (void) fprintf(stdout, DFLT_FMT, + ksi->ks_module, ksi->ks_instance, + ksi->ks_name, ksi->ks_class); + } + g_headerflg = B_FALSE; + } + + if (g_pflg) { + (void) fprintf(stdout, KS_PFMT, + ksi->ks_module, ksi->ks_instance, + ksi->ks_name, nvpair->name); + if (!g_lflg) { + (void) putchar(g_cflg ? ':': '\t'); + ks_value_print(nvpair); + } + } else { + (void) fprintf(stdout, KS_DFMT, nvpair->name); + ks_value_print(nvpair); + } + + (void) putchar('\n'); +} + +/* + * Print a C string as a JSON string. + */ +static void +ks_print_json_string(const char *str) +{ + char c; + + (void) putchar('"'); + + while ((c = *str++) != '\0') { + /* + * For readability, we use the allowed alternate escape + * sequence for quote, question mark, reverse solidus (look + * it up!), newline and tab -- and use the universal escape + * sequence for all other control characters. + */ + switch (c) { + case '"': + case '?': + case '\\': + (void) fprintf(stdout, "\\%c", c); + break; + + case '\n': + (void) fprintf(stdout, "\\n"); + break; + + case '\t': + (void) fprintf(stdout, "\\t"); + break; + + default: + /* + * By escaping those characters for which isprint(3C) + * is false, we escape both the RFC 7159 mandated + * escaped range of 0x01 through 0x1f as well as DEL + * (0x7f -- the control character that RFC 7159 forgot) + * and then everything else that's unprintable for + * good measure. + */ + if (!isprint(c)) { + (void) fprintf(stdout, "\\u%04hhx", (uint8_t)c); + break; + } + + (void) putchar(c); + break; + } + } + + (void) putchar('"'); +} + +/* + * Print a single instance in JSON format. + */ +static void +ks_instance_print_json(ks_instance_t *ksi, ks_nvpair_t *nvpair, boolean_t last) +{ + static int headers; + + if (g_headerflg) { + if (headers++ > 0) + (void) fprintf(stdout, ", "); + + (void) fprintf(stdout, "{\n\t\"module\": "); + ks_print_json_string(ksi->ks_module); + + (void) fprintf(stdout, + ",\n\t\"instance\": %d,\n\t\"name\": ", ksi->ks_instance); + ks_print_json_string(ksi->ks_name); + + (void) fprintf(stdout, ",\n\t\"class\": "); + ks_print_json_string(ksi->ks_class); + + (void) fprintf(stdout, ",\n\t\"type\": %d,\n", ksi->ks_type); + + if (ksi->ks_snaptime == 0) + (void) fprintf(stdout, "\t\"snaptime\": 0,\n"); + else + (void) fprintf(stdout, "\t\"snaptime\": %.9f,\n", + ksi->ks_snaptime / 1000000000.0); + + (void) fprintf(stdout, "\t\"data\": {\n"); + + g_headerflg = B_FALSE; + } + + (void) fprintf(stdout, "\t\t"); + ks_print_json_string(nvpair->name); + (void) fprintf(stdout, ": "); + + switch (nvpair->data_type) { + case KSTAT_DATA_CHAR: + ks_print_json_string(nvpair->value.c); + break; + + case KSTAT_DATA_STRING: + ks_print_json_string(KSTAT_NAMED_STR_PTR(nvpair)); + break; + + default: + ks_value_print(nvpair); + break; + } + + if (!last) + (void) putchar(','); + + (void) putchar('\n'); +} + +/* + * Print all instances. + */ +static void +ks_instances_print(void) +{ + ks_selector_t *selector; + ks_instance_t *ksi, *ktmp; + ks_nvpair_t *nvpair, *ntmp, *next; + void (*ks_print_fn)(ks_instance_t *, ks_nvpair_t *, boolean_t); + char *ks_number; + + if (g_timestamp_fmt != NODATE) + print_timestamp(g_timestamp_fmt); + + if (g_jflg) { + ks_print_fn = &ks_instance_print_json; + (void) putchar('['); + } else { + ks_print_fn = &ks_instance_print; + } + + /* Iterate over each selector */ + selector = list_head(&selector_list); + while (selector != NULL) { + + /* Iterate over each instance */ + for (ksi = list_head(&instances_list); ksi != NULL; + ksi = list_next(&instances_list, ksi)) { + + (void) asprintf(&ks_number, "%d", ksi->ks_instance); + if (!(ks_match(ksi->ks_module, &selector->ks_module) && + ks_match(ksi->ks_name, &selector->ks_name) && + ks_match(ks_number, &selector->ks_instance) && + ks_match(ksi->ks_class, &g_ks_class))) { + free(ks_number); + continue; + } + + free(ks_number); + + g_headerflg = B_TRUE; + + /* + * Find our first statistic to print. + */ + for (nvpair = list_head(&ksi->ks_nvlist); + nvpair != NULL; + nvpair = list_next(&ksi->ks_nvlist, nvpair)) { + if (ks_match(nvpair->name, + &selector->ks_statistic)) + break; + } + + while (nvpair != NULL) { + boolean_t last; + + /* + * Find the next statistic to print so we can + * indicate to the print function if this + * statistic is the last to be printed for + * this instance. + */ + for (next = list_next(&ksi->ks_nvlist, nvpair); + next != NULL; + next = list_next(&ksi->ks_nvlist, next)) { + if (ks_match(next->name, + &selector->ks_statistic)) + break; + } + + g_matched = B_TRUE; + last = next == NULL ? B_TRUE : B_FALSE; + + if (!g_qflg) + (*ks_print_fn)(ksi, nvpair, last); + + nvpair = next; + } + + if (!g_headerflg) { + if (g_jflg) { + (void) fprintf(stdout, "\t}\n}"); + } else if (!g_pflg) { + (void) putchar('\n'); + } + } + } + + selector = list_next(&selector_list, selector); + } + + if (g_jflg) + (void) fprintf(stdout, "]\n"); + + (void) fflush(stdout); + + /* Free the instances list */ + ksi = list_head(&instances_list); + while (ksi != NULL) { + nvpair = list_head(&ksi->ks_nvlist); + while (nvpair != NULL) { + ntmp = nvpair; + nvpair = list_next(&ksi->ks_nvlist, nvpair); + list_remove(&ksi->ks_nvlist, ntmp); + if (ntmp->data_type == KSTAT_DATA_STRING) + free(ntmp->value.str.addr.ptr); + free(ntmp); + } + + ktmp = ksi; + ksi = list_next(&instances_list, ksi); + list_remove(&instances_list, ktmp); + list_destroy(&ktmp->ks_nvlist); + free(ktmp); + } +} + +/* + * kstat -w module:instance:name:statistic=value [ ... ] + * e.g. "kstat -w zfs:0:tunable:zfs_arc_mac=1234567890 + * + */ +int +write_mode(int argc, char **argv) +{ + char *arg; + int instance, rc = 0; + int failure = 0; + uint64_t value, before_value; + kstat_ctl_t *kc; + + if (argc == 0) { + usage(); + (void) fprintf(stderr, "-w takes at least one argument\n"); + (void) fprintf(stderr, + "\te.g. kstat -w zfs:0:tunable:zfs_arc_max=1200000\n"); + return (-1); + } + + while ((kc = kstat_open()) == NULL) { + if (errno == EAGAIN) { + (void) usleep(200); + } else { + perror("kstat_open"); + exit(3); + } + } + + while (argc--) { + char mod[KSTAT_STRLEN + 1], name[KSTAT_STRLEN + 1], + stat[KSTAT_STRLEN + 1]; + + arg = *argv; + + // TODO: make this more flexible. Spaces, and other types than + // uint64. + // Call C11 sscanf_s which takes string-width following ptr. + if ((rc = sscanf_s(arg, + "%[^:]:%d:%[^:]:%[^=]=%llu", + mod, KSTAT_STRLEN, + &instance, + name, KSTAT_STRLEN, + stat, KSTAT_STRLEN, + &value)) != 5) { + (void) fprintf(stderr, "Unable to parse '%s'\n input " + "not in 'module:instance:name:statisticname=value' " + "format. %d\n", arg, rc); + failure++; + } else { + kstat_t *ks; + ks = kstat_lookup(kc, mod, instance, name); + if (ks == NULL) { + (void) fprintf(stderr, + "Unable to lookup '%s:%d:%s': %d\n", + mod, instance, name, errno); + failure++; + } else { + if (kstat_read(kc, ks, NULL) == -1) { + (void) fprintf(stderr, + "Unable to read '%s:%d:%s': %d\n", + mod, instance, name, errno); + failure++; + } else { + kstat_named_t *kn = + kstat_data_lookup(ks, stat); + if (kn == NULL) { + (void) fprintf(stderr, + "Unable to find '%s' in " + "'%s:%d:%s': %d\n", + stat, mod, instance, + name, errno); + failure++; + } else { + before_value = kn->value.ui64; + kn->value.ui64 = value; + + /* Update kernel */ + rc = kstat_write(kc, ks, NULL); + + if (rc == -1) { + (void) fprintf(stderr, + "Unable to write " + "'%s:%d:%s:%s': " + "%d\n", + mod, instance, + name, + stat, errno); + failure++; + } else { + (void) fprintf(stderr, + "%s:%d:%s:%s: %llu " + "-> %llu\n", + mod, instance, + name, stat, + before_value, + value); + } // rc + } // kstat_data_lookup + } // kstat_read + } // kstat_lookup + } // sscanf + argv++; + } + + kstat_close(kc); + return (failure); +} + + +#ifndef WIN32 +static void +save_cpu_stat(kstat_t *kp, ks_instance_t *ksi) +{ + cpu_stat_t *stat; + cpu_sysinfo_t *sysinfo; + cpu_syswait_t *syswait; + cpu_vminfo_t *vminfo; + + stat = (cpu_stat_t *)(kp->ks_data); + sysinfo = &stat->cpu_sysinfo; + syswait = &stat->cpu_syswait; + vminfo = &stat->cpu_vminfo; + + SAVE_UINT32_X(ksi, "idle", sysinfo->cpu[CPU_IDLE]); + SAVE_UINT32_X(ksi, "user", sysinfo->cpu[CPU_USER]); + SAVE_UINT32_X(ksi, "kernel", sysinfo->cpu[CPU_KERNEL]); + SAVE_UINT32_X(ksi, "wait", sysinfo->cpu[CPU_WAIT]); + SAVE_UINT32_X(ksi, "wait_io", sysinfo->wait[W_IO]); + SAVE_UINT32_X(ksi, "wait_swap", sysinfo->wait[W_SWAP]); + SAVE_UINT32_X(ksi, "wait_pio", sysinfo->wait[W_PIO]); + SAVE_UINT32(ksi, sysinfo, bread); + SAVE_UINT32(ksi, sysinfo, bwrite); + SAVE_UINT32(ksi, sysinfo, lread); + SAVE_UINT32(ksi, sysinfo, lwrite); + SAVE_UINT32(ksi, sysinfo, phread); + SAVE_UINT32(ksi, sysinfo, phwrite); + SAVE_UINT32(ksi, sysinfo, pswitch); + SAVE_UINT32(ksi, sysinfo, trap); + SAVE_UINT32(ksi, sysinfo, intr); + SAVE_UINT32(ksi, sysinfo, syscall); + SAVE_UINT32(ksi, sysinfo, sysread); + SAVE_UINT32(ksi, sysinfo, syswrite); + SAVE_UINT32(ksi, sysinfo, sysfork); + SAVE_UINT32(ksi, sysinfo, sysvfork); + SAVE_UINT32(ksi, sysinfo, sysexec); + SAVE_UINT32(ksi, sysinfo, readch); + SAVE_UINT32(ksi, sysinfo, writech); + SAVE_UINT32(ksi, sysinfo, rcvint); + SAVE_UINT32(ksi, sysinfo, xmtint); + SAVE_UINT32(ksi, sysinfo, mdmint); + SAVE_UINT32(ksi, sysinfo, rawch); + SAVE_UINT32(ksi, sysinfo, canch); + SAVE_UINT32(ksi, sysinfo, outch); + SAVE_UINT32(ksi, sysinfo, msg); + SAVE_UINT32(ksi, sysinfo, sema); + SAVE_UINT32(ksi, sysinfo, namei); + SAVE_UINT32(ksi, sysinfo, ufsiget); + SAVE_UINT32(ksi, sysinfo, ufsdirblk); + SAVE_UINT32(ksi, sysinfo, ufsipage); + SAVE_UINT32(ksi, sysinfo, ufsinopage); + SAVE_UINT32(ksi, sysinfo, inodeovf); + SAVE_UINT32(ksi, sysinfo, fileovf); + SAVE_UINT32(ksi, sysinfo, procovf); + SAVE_UINT32(ksi, sysinfo, intrthread); + SAVE_UINT32(ksi, sysinfo, intrblk); + SAVE_UINT32(ksi, sysinfo, idlethread); + SAVE_UINT32(ksi, sysinfo, inv_swtch); + SAVE_UINT32(ksi, sysinfo, nthreads); + SAVE_UINT32(ksi, sysinfo, cpumigrate); + SAVE_UINT32(ksi, sysinfo, xcalls); + SAVE_UINT32(ksi, sysinfo, mutex_adenters); + SAVE_UINT32(ksi, sysinfo, rw_rdfails); + SAVE_UINT32(ksi, sysinfo, rw_wrfails); + SAVE_UINT32(ksi, sysinfo, modload); + SAVE_UINT32(ksi, sysinfo, modunload); + SAVE_UINT32(ksi, sysinfo, bawrite); +#ifdef STATISTICS /* see header file */ + SAVE_UINT32(ksi, sysinfo, rw_enters); + SAVE_UINT32(ksi, sysinfo, win_uo_cnt); + SAVE_UINT32(ksi, sysinfo, win_uu_cnt); + SAVE_UINT32(ksi, sysinfo, win_so_cnt); + SAVE_UINT32(ksi, sysinfo, win_su_cnt); + SAVE_UINT32(ksi, sysinfo, win_suo_cnt); +#endif + + SAVE_INT32(ksi, syswait, iowait); + SAVE_INT32(ksi, syswait, swap); + SAVE_INT32(ksi, syswait, physio); + + SAVE_UINT32(ksi, vminfo, pgrec); + SAVE_UINT32(ksi, vminfo, pgfrec); + SAVE_UINT32(ksi, vminfo, pgin); + SAVE_UINT32(ksi, vminfo, pgpgin); + SAVE_UINT32(ksi, vminfo, pgout); + SAVE_UINT32(ksi, vminfo, pgpgout); + SAVE_UINT32(ksi, vminfo, swapin); + SAVE_UINT32(ksi, vminfo, pgswapin); + SAVE_UINT32(ksi, vminfo, swapout); + SAVE_UINT32(ksi, vminfo, pgswapout); + SAVE_UINT32(ksi, vminfo, zfod); + SAVE_UINT32(ksi, vminfo, dfree); + SAVE_UINT32(ksi, vminfo, scan); + SAVE_UINT32(ksi, vminfo, rev); + SAVE_UINT32(ksi, vminfo, hat_fault); + SAVE_UINT32(ksi, vminfo, as_fault); + SAVE_UINT32(ksi, vminfo, maj_fault); + SAVE_UINT32(ksi, vminfo, cow_fault); + SAVE_UINT32(ksi, vminfo, prot_fault); + SAVE_UINT32(ksi, vminfo, softlock); + SAVE_UINT32(ksi, vminfo, kernel_asflt); + SAVE_UINT32(ksi, vminfo, pgrrun); + SAVE_UINT32(ksi, vminfo, execpgin); + SAVE_UINT32(ksi, vminfo, execpgout); + SAVE_UINT32(ksi, vminfo, execfree); + SAVE_UINT32(ksi, vminfo, anonpgin); + SAVE_UINT32(ksi, vminfo, anonpgout); + SAVE_UINT32(ksi, vminfo, anonfree); + SAVE_UINT32(ksi, vminfo, fspgin); + SAVE_UINT32(ksi, vminfo, fspgout); + SAVE_UINT32(ksi, vminfo, fsfree); +} + +static void +save_var(kstat_t *kp, ks_instance_t *ksi) +{ + struct var *var = (struct var *)(kp->ks_data); + + assert(kp->ks_data_size == sizeof (struct var)); + + SAVE_INT32(ksi, var, v_buf); + SAVE_INT32(ksi, var, v_call); + SAVE_INT32(ksi, var, v_proc); + SAVE_INT32(ksi, var, v_maxupttl); + SAVE_INT32(ksi, var, v_nglobpris); + SAVE_INT32(ksi, var, v_maxsyspri); + SAVE_INT32(ksi, var, v_clist); + SAVE_INT32(ksi, var, v_maxup); + SAVE_INT32(ksi, var, v_hbuf); + SAVE_INT32(ksi, var, v_hmask); + SAVE_INT32(ksi, var, v_pbuf); + SAVE_INT32(ksi, var, v_sptmap); + SAVE_INT32(ksi, var, v_maxpmem); + SAVE_INT32(ksi, var, v_autoup); + SAVE_INT32(ksi, var, v_bufhwm); +} + +static void +save_ncstats(kstat_t *kp, ks_instance_t *ksi) +{ + struct ncstats *ncstats = (struct ncstats *)(kp->ks_data); + + assert(kp->ks_data_size == sizeof (struct ncstats)); + + SAVE_INT32(ksi, ncstats, hits); + SAVE_INT32(ksi, ncstats, misses); + SAVE_INT32(ksi, ncstats, enters); + SAVE_INT32(ksi, ncstats, dbl_enters); + SAVE_INT32(ksi, ncstats, long_enter); + SAVE_INT32(ksi, ncstats, long_look); + SAVE_INT32(ksi, ncstats, move_to_front); + SAVE_INT32(ksi, ncstats, purges); +} + +static void +save_sysinfo(kstat_t *kp, ks_instance_t *ksi) +{ + sysinfo_t *sysinfo = (sysinfo_t *)(kp->ks_data); + + assert(kp->ks_data_size == sizeof (sysinfo_t)); + + SAVE_UINT32(ksi, sysinfo, updates); + SAVE_UINT32(ksi, sysinfo, runque); + SAVE_UINT32(ksi, sysinfo, runocc); + SAVE_UINT32(ksi, sysinfo, swpque); + SAVE_UINT32(ksi, sysinfo, swpocc); + SAVE_UINT32(ksi, sysinfo, waiting); +} + +static void +save_vminfo(kstat_t *kp, ks_instance_t *ksi) +{ + vminfo_t *vminfo = (vminfo_t *)(kp->ks_data); + + assert(kp->ks_data_size == sizeof (vminfo_t)); + + SAVE_UINT64(ksi, vminfo, freemem); + SAVE_UINT64(ksi, vminfo, swap_resv); + SAVE_UINT64(ksi, vminfo, swap_alloc); + SAVE_UINT64(ksi, vminfo, swap_avail); + SAVE_UINT64(ksi, vminfo, swap_free); + SAVE_UINT64(ksi, vminfo, updates); +} + +static void +save_nfs(kstat_t *kp, ks_instance_t *ksi) +{ + struct mntinfo_kstat *mntinfo = (struct mntinfo_kstat *)(kp->ks_data); + + assert(kp->ks_data_size == sizeof (struct mntinfo_kstat)); + + SAVE_STRING(ksi, mntinfo, mik_proto); + SAVE_UINT32(ksi, mntinfo, mik_vers); + SAVE_UINT32(ksi, mntinfo, mik_flags); + SAVE_UINT32(ksi, mntinfo, mik_secmod); + SAVE_UINT32(ksi, mntinfo, mik_curread); + SAVE_UINT32(ksi, mntinfo, mik_curwrite); + SAVE_INT32(ksi, mntinfo, mik_timeo); + SAVE_INT32(ksi, mntinfo, mik_retrans); + SAVE_UINT32(ksi, mntinfo, mik_acregmin); + SAVE_UINT32(ksi, mntinfo, mik_acregmax); + SAVE_UINT32(ksi, mntinfo, mik_acdirmin); + SAVE_UINT32(ksi, mntinfo, mik_acdirmax); + SAVE_UINT32_X(ksi, "lookup_srtt", mntinfo->mik_timers[0].srtt); + SAVE_UINT32_X(ksi, "lookup_deviate", mntinfo->mik_timers[0].deviate); + SAVE_UINT32_X(ksi, "lookup_rtxcur", mntinfo->mik_timers[0].rtxcur); + SAVE_UINT32_X(ksi, "read_srtt", mntinfo->mik_timers[1].srtt); + SAVE_UINT32_X(ksi, "read_deviate", mntinfo->mik_timers[1].deviate); + SAVE_UINT32_X(ksi, "read_rtxcur", mntinfo->mik_timers[1].rtxcur); + SAVE_UINT32_X(ksi, "write_srtt", mntinfo->mik_timers[2].srtt); + SAVE_UINT32_X(ksi, "write_deviate", mntinfo->mik_timers[2].deviate); + SAVE_UINT32_X(ksi, "write_rtxcur", mntinfo->mik_timers[2].rtxcur); + SAVE_UINT32(ksi, mntinfo, mik_noresponse); + SAVE_UINT32(ksi, mntinfo, mik_failover); + SAVE_UINT32(ksi, mntinfo, mik_remap); + SAVE_STRING(ksi, mntinfo, mik_curserver); +} + +#ifdef __sparc +static void +save_sfmmu_global_stat(kstat_t *kp, ks_instance_t *ksi) +{ + struct sfmmu_global_stat *sfmmug = + (struct sfmmu_global_stat *)(kp->ks_data); + + assert(kp->ks_data_size == sizeof (struct sfmmu_global_stat)); + + SAVE_INT32(ksi, sfmmug, sf_tsb_exceptions); + SAVE_INT32(ksi, sfmmug, sf_tsb_raise_exception); + SAVE_INT32(ksi, sfmmug, sf_pagefaults); + SAVE_INT32(ksi, sfmmug, sf_uhash_searches); + SAVE_INT32(ksi, sfmmug, sf_uhash_links); + SAVE_INT32(ksi, sfmmug, sf_khash_searches); + SAVE_INT32(ksi, sfmmug, sf_khash_links); + SAVE_INT32(ksi, sfmmug, sf_swapout); + SAVE_INT32(ksi, sfmmug, sf_tsb_alloc); + SAVE_INT32(ksi, sfmmug, sf_tsb_allocfail); + SAVE_INT32(ksi, sfmmug, sf_tsb_sectsb_create); + SAVE_INT32(ksi, sfmmug, sf_scd_1sttsb_alloc); + SAVE_INT32(ksi, sfmmug, sf_scd_2ndtsb_alloc); + SAVE_INT32(ksi, sfmmug, sf_scd_1sttsb_allocfail); + SAVE_INT32(ksi, sfmmug, sf_scd_2ndtsb_allocfail); + SAVE_INT32(ksi, sfmmug, sf_tteload8k); + SAVE_INT32(ksi, sfmmug, sf_tteload64k); + SAVE_INT32(ksi, sfmmug, sf_tteload512k); + SAVE_INT32(ksi, sfmmug, sf_tteload4m); + SAVE_INT32(ksi, sfmmug, sf_tteload32m); + SAVE_INT32(ksi, sfmmug, sf_tteload256m); + SAVE_INT32(ksi, sfmmug, sf_tsb_load8k); + SAVE_INT32(ksi, sfmmug, sf_tsb_load4m); + SAVE_INT32(ksi, sfmmug, sf_hblk_hit); + SAVE_INT32(ksi, sfmmug, sf_hblk8_ncreate); + SAVE_INT32(ksi, sfmmug, sf_hblk8_nalloc); + SAVE_INT32(ksi, sfmmug, sf_hblk1_ncreate); + SAVE_INT32(ksi, sfmmug, sf_hblk1_nalloc); + SAVE_INT32(ksi, sfmmug, sf_hblk_slab_cnt); + SAVE_INT32(ksi, sfmmug, sf_hblk_reserve_cnt); + SAVE_INT32(ksi, sfmmug, sf_hblk_recurse_cnt); + SAVE_INT32(ksi, sfmmug, sf_hblk_reserve_hit); + SAVE_INT32(ksi, sfmmug, sf_get_free_success); + SAVE_INT32(ksi, sfmmug, sf_get_free_throttle); + SAVE_INT32(ksi, sfmmug, sf_get_free_fail); + SAVE_INT32(ksi, sfmmug, sf_put_free_success); + SAVE_INT32(ksi, sfmmug, sf_put_free_fail); + SAVE_INT32(ksi, sfmmug, sf_pgcolor_conflict); + SAVE_INT32(ksi, sfmmug, sf_uncache_conflict); + SAVE_INT32(ksi, sfmmug, sf_unload_conflict); + SAVE_INT32(ksi, sfmmug, sf_ism_uncache); + SAVE_INT32(ksi, sfmmug, sf_ism_recache); + SAVE_INT32(ksi, sfmmug, sf_recache); + SAVE_INT32(ksi, sfmmug, sf_steal_count); + SAVE_INT32(ksi, sfmmug, sf_pagesync); + SAVE_INT32(ksi, sfmmug, sf_clrwrt); + SAVE_INT32(ksi, sfmmug, sf_pagesync_invalid); + SAVE_INT32(ksi, sfmmug, sf_kernel_xcalls); + SAVE_INT32(ksi, sfmmug, sf_user_xcalls); + SAVE_INT32(ksi, sfmmug, sf_tsb_grow); + SAVE_INT32(ksi, sfmmug, sf_tsb_shrink); + SAVE_INT32(ksi, sfmmug, sf_tsb_resize_failures); + SAVE_INT32(ksi, sfmmug, sf_tsb_reloc); + SAVE_INT32(ksi, sfmmug, sf_user_vtop); + SAVE_INT32(ksi, sfmmug, sf_ctx_inv); + SAVE_INT32(ksi, sfmmug, sf_tlb_reprog_pgsz); + SAVE_INT32(ksi, sfmmug, sf_region_remap_demap); + SAVE_INT32(ksi, sfmmug, sf_create_scd); + SAVE_INT32(ksi, sfmmug, sf_join_scd); + SAVE_INT32(ksi, sfmmug, sf_leave_scd); + SAVE_INT32(ksi, sfmmug, sf_destroy_scd); +} +#endif + +#ifdef __sparc +static void +save_sfmmu_tsbsize_stat(kstat_t *kp, ks_instance_t *ksi) +{ + struct sfmmu_tsbsize_stat *sfmmut; + + assert(kp->ks_data_size == sizeof (struct sfmmu_tsbsize_stat)); + sfmmut = (struct sfmmu_tsbsize_stat *)(kp->ks_data); + + SAVE_INT32(ksi, sfmmut, sf_tsbsz_8k); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_16k); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_32k); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_64k); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_128k); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_256k); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_512k); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_1m); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_2m); + SAVE_INT32(ksi, sfmmut, sf_tsbsz_4m); +} +#endif + +#ifdef __sparc +static void +save_simmstat(kstat_t *kp, ks_instance_t *ksi) +{ + uchar_t *simmstat; + char *simm_buf; + char *list = NULL; + int i; + + assert(kp->ks_data_size == sizeof (uchar_t) * SIMM_COUNT); + + for (i = 0, simmstat = (uchar_t *)(kp->ks_data); i < SIMM_COUNT - 1; + i++, simmstat++) { + if (list == NULL) { + (void) asprintf(&simm_buf, "%d,", *simmstat); + } else { + (void) asprintf(&simm_buf, "%s%d,", list, *simmstat); + free(list); + } + list = simm_buf; + } + + (void) asprintf(&simm_buf, "%s%d", list, *simmstat); + SAVE_STRING_X(ksi, "status", simm_buf); + free(list); + free(simm_buf); +} +#endif + +#ifdef __sparc +/* + * Helper function for save_temperature(). + */ +static char * +short_array_to_string(short *shortp, int len) +{ + char *list = NULL; + char *list_buf; + + for (; len > 1; len--, shortp++) { + if (list == NULL) { + (void) asprintf(&list_buf, "%hd,", *shortp); + } else { + (void) asprintf(&list_buf, "%s%hd,", list, *shortp); + free(list); + } + list = list_buf; + } + + (void) asprintf(&list_buf, "%s%hd", list, *shortp); + free(list); + return (list_buf); +} + +static void +save_temperature(kstat_t *kp, ks_instance_t *ksi) +{ + struct temp_stats *temps = (struct temp_stats *)(kp->ks_data); + char *buf; + + assert(kp->ks_data_size == sizeof (struct temp_stats)); + + SAVE_UINT32(ksi, temps, index); + + buf = short_array_to_string(temps->l1, L1_SZ); + SAVE_STRING_X(ksi, "l1", buf); + free(buf); + + buf = short_array_to_string(temps->l2, L2_SZ); + SAVE_STRING_X(ksi, "l2", buf); + free(buf); + + buf = short_array_to_string(temps->l3, L3_SZ); + SAVE_STRING_X(ksi, "l3", buf); + free(buf); + + buf = short_array_to_string(temps->l4, L4_SZ); + SAVE_STRING_X(ksi, "l4", buf); + free(buf); + + buf = short_array_to_string(temps->l5, L5_SZ); + SAVE_STRING_X(ksi, "l5", buf); + free(buf); + + SAVE_INT32(ksi, temps, max); + SAVE_INT32(ksi, temps, min); + SAVE_INT32(ksi, temps, state); + SAVE_INT32(ksi, temps, temp_cnt); + SAVE_INT32(ksi, temps, shutdown_cnt); + SAVE_INT32(ksi, temps, version); + SAVE_INT32(ksi, temps, trend); + SAVE_INT32(ksi, temps, override); +} +#endif + +#ifdef __sparc +static void +save_temp_over(kstat_t *kp, ks_instance_t *ksi) +{ + short *sh = (short *)(kp->ks_data); + char *value; + + assert(kp->ks_data_size == sizeof (short)); + + (void) asprintf(&value, "%hu", *sh); + SAVE_STRING_X(ksi, "override", value); + free(value); +} +#endif + +#ifdef __sparc +static void +save_ps_shadow(kstat_t *kp, ks_instance_t *ksi) +{ + uchar_t *uchar = (uchar_t *)(kp->ks_data); + + assert(kp->ks_data_size == SYS_PS_COUNT); + + SAVE_CHAR_X(ksi, "core_0", *uchar++); + SAVE_CHAR_X(ksi, "core_1", *uchar++); + SAVE_CHAR_X(ksi, "core_2", *uchar++); + SAVE_CHAR_X(ksi, "core_3", *uchar++); + SAVE_CHAR_X(ksi, "core_4", *uchar++); + SAVE_CHAR_X(ksi, "core_5", *uchar++); + SAVE_CHAR_X(ksi, "core_6", *uchar++); + SAVE_CHAR_X(ksi, "core_7", *uchar++); + SAVE_CHAR_X(ksi, "pps_0", *uchar++); + SAVE_CHAR_X(ksi, "clk_33", *uchar++); + SAVE_CHAR_X(ksi, "clk_50", *uchar++); + SAVE_CHAR_X(ksi, "v5_p", *uchar++); + SAVE_CHAR_X(ksi, "v12_p", *uchar++); + SAVE_CHAR_X(ksi, "v5_aux", *uchar++); + SAVE_CHAR_X(ksi, "v5_p_pch", *uchar++); + SAVE_CHAR_X(ksi, "v12_p_pch", *uchar++); + SAVE_CHAR_X(ksi, "v3_pch", *uchar++); + SAVE_CHAR_X(ksi, "v5_pch", *uchar++); + SAVE_CHAR_X(ksi, "p_fan", *uchar++); +} +#endif + +#ifdef __sparc +static void +save_fault_list(kstat_t *kp, ks_instance_t *ksi) +{ + struct ft_list *fault; + char name[KSTAT_STRLEN + 7]; + int i; + + for (i = 1, fault = (struct ft_list *)(kp->ks_data); + i <= 999999 && i <= kp->ks_data_size / sizeof (struct ft_list); + i++, fault++) { + (void) snprintf(name, sizeof (name), "unit_%d", i); + SAVE_INT32_X(ksi, name, fault->unit); + (void) snprintf(name, sizeof (name), "type_%d", i); + SAVE_INT32_X(ksi, name, fault->type); + (void) snprintf(name, sizeof (name), "fclass_%d", i); + SAVE_INT32_X(ksi, name, fault->fclass); + (void) snprintf(name, sizeof (name), "create_time_%d", i); + SAVE_HRTIME_X(ksi, name, fault->create_time); + (void) snprintf(name, sizeof (name), "msg_%d", i); + SAVE_STRING_X(ksi, name, fault->msg); + } +} +#endif +#endif + +static void +save_named(kstat_t *kp, ks_instance_t *ksi) +{ + kstat_named_t *knp; + int n; + + for (n = kp->ks_ndata, knp = KSTAT_NAMED_PTR(kp); n > 0; n--, knp++) { + /* + * Annoyingly, some drivers have kstats with uninitialized + * members (which kstat_install(9F) is sadly powerless to + * prevent, and kstat_read(3KSTAT) unfortunately does nothing + * to stop). To prevent these from confusing us to be + * KSTAT_DATA_CHAR statistics, we skip over them. + */ + if (knp->name[0] == '\0') + continue; + + switch (knp->data_type) { + case KSTAT_DATA_CHAR: + nvpair_insert(ksi, knp->name, + (ks_value_t *)&knp->value, KSTAT_DATA_CHAR); + break; + case KSTAT_DATA_INT32: + nvpair_insert(ksi, knp->name, + (ks_value_t *)&knp->value, KSTAT_DATA_INT32); + break; + case KSTAT_DATA_UINT32: + nvpair_insert(ksi, knp->name, + (ks_value_t *)&knp->value, KSTAT_DATA_UINT32); + break; + case KSTAT_DATA_INT64: + nvpair_insert(ksi, knp->name, + (ks_value_t *)&knp->value, KSTAT_DATA_INT64); + break; + case KSTAT_DATA_UINT64: + nvpair_insert(ksi, knp->name, + (ks_value_t *)&knp->value, KSTAT_DATA_UINT64); + break; + case KSTAT_DATA_STRING: + SAVE_STRING_X(ksi, knp->name, KSTAT_NAMED_STR_PTR(knp)); + break; + default: + assert(B_FALSE); /* Invalid data type */ + break; + } + } +} + +static void +save_intr(kstat_t *kp, ks_instance_t *ksi) +{ + kstat_intr_t *intr = KSTAT_INTR_PTR(kp); + char *intr_names[] = {"hard", "soft", "watchdog", "spurious", + "multiple_service"}; + int n; + + for (n = 0; n < KSTAT_NUM_INTRS; n++) + SAVE_UINT32_X(ksi, intr_names[n], intr->intrs[n]); +} + +static void +save_io(kstat_t *kp, ks_instance_t *ksi) +{ + kstat_io_t *ksio = KSTAT_IO_PTR(kp); + + SAVE_UINT64(ksi, ksio, nread); + SAVE_UINT64(ksi, ksio, nwritten); + SAVE_UINT32(ksi, ksio, reads); + SAVE_UINT32(ksi, ksio, writes); + SAVE_HRTIME(ksi, ksio, wtime); + SAVE_HRTIME(ksi, ksio, wlentime); + SAVE_HRTIME(ksi, ksio, wlastupdate); + SAVE_HRTIME(ksi, ksio, rtime); + SAVE_HRTIME(ksi, ksio, rlentime); + SAVE_HRTIME(ksi, ksio, rlastupdate); + SAVE_UINT32(ksi, ksio, wcnt); + SAVE_UINT32(ksi, ksio, rcnt); +} + +static void +save_timer(kstat_t *kp, ks_instance_t *ksi) +{ + kstat_timer_t *ktimer = KSTAT_TIMER_PTR(kp); + + SAVE_STRING(ksi, ktimer, name); + SAVE_UINT64(ksi, ktimer, num_events); + SAVE_HRTIME(ksi, ktimer, elapsed_time); + SAVE_HRTIME(ksi, ktimer, min_time); + SAVE_HRTIME(ksi, ktimer, max_time); + SAVE_HRTIME(ksi, ktimer, start_time); + SAVE_HRTIME(ksi, ktimer, stop_time); +} diff --git a/cmd/os/windows/kstat/kstat.h b/cmd/os/windows/kstat/kstat.h new file mode 100644 index 000000000000..90e1d72fb810 --- /dev/null +++ b/cmd/os/windows/kstat/kstat.h @@ -0,0 +1,259 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Copyright 2013 David Hoeppner. All rights reserved. + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. + */ + +#ifndef _STAT_KSTAT_H +#define _STAT_KSTAT_H + +/* + * Structures needed by the kstat reader functions. + */ +#include +#include + +#ifdef __sparc +#include +#include +#include +#include +#endif + +#define KSTAT_DATA_HRTIME (KSTAT_DATA_STRING + 1) + +typedef union ks_value { + char c[16]; + int32_t i32; + uint32_t ui32; + struct { + union { + char *ptr; + char __pad[8]; + } addr; + uint32_t len; + } str; + + int64_t i64; + uint64_t ui64; +} ks_value_t; + +#define SAVE_HRTIME(I, S, N) \ +{ \ + ks_value_t v; \ + v.ui64 = S->N; \ + nvpair_insert(I, #N, &v, KSTAT_DATA_UINT64); \ +} + +#define SAVE_INT32(I, S, N) \ +{ \ + ks_value_t v; \ + v.i32 = S->N; \ + nvpair_insert(I, #N, &v, KSTAT_DATA_INT32); \ +} + +#define SAVE_UINT32(I, S, N) \ +{ \ + ks_value_t v; \ + v.ui32 = S->N; \ + nvpair_insert(I, #N, &v, KSTAT_DATA_UINT32); \ +} + +#define SAVE_INT64(I, S, N) \ +{ \ + ks_value_t v; \ + v.i64 = S->N; \ + nvpair_insert(I, #N, &v, KSTAT_DATA_INT64); \ +} + +#define SAVE_UINT64(I, S, N) \ +{ \ + ks_value_t v; \ + v.ui64 = S->N; \ + nvpair_insert(I, #N, &v, KSTAT_DATA_UINT64); \ +} + +/* + * We dont want const "strings" because we free + * the instances later. + */ +#define SAVE_STRING(I, S, N) \ +{ \ + ks_value_t v; \ + v.str.addr.ptr = _strdup(S->N); \ + v.str.len = (uint32_t)strlen(S->N); \ + nvpair_insert(I, #N, &v, KSTAT_DATA_STRING); \ +} + +#define SAVE_HRTIME_X(I, N, V) \ +{ \ + ks_value_t v; \ + v.ui64 = V; \ + nvpair_insert(I, N, &v, KSTAT_DATA_HRTIME); \ +} + +#define SAVE_INT32_X(I, N, V) \ +{ \ + ks_value_t v; \ + v.i32 = V; \ + nvpair_insert(I, N, &v, KSTAT_DATA_INT32); \ +} + +#define SAVE_UINT32_X(I, N, V) \ +{ \ + ks_value_t v; \ + v.ui32 = V; \ + nvpair_insert(I, N, &v, KSTAT_DATA_UINT32); \ +} + +#define SAVE_UINT64_X(I, N, V) \ +{ \ + ks_value_t v; \ + v.ui64 = V; \ + nvpair_insert(I, N, &v, KSTAT_DATA_UINT64); \ +} + +#define SAVE_STRING_X(I, N, V) \ +{ \ + ks_value_t v; \ + v.str.addr.ptr = _strdup(V); \ + v.str.len = (uint32_t)((V) ? strlen(V) : 0); \ + nvpair_insert(I, N, &v, KSTAT_DATA_STRING); \ +} + +#define SAVE_CHAR_X(I, N, V) \ +{ \ + ks_value_t v; \ + (void) asprintf(&v.str.addr.ptr, "%c", V); \ + v.str.len = 1; \ + nvpair_insert(I, N, &v, KSTAT_DATA_STRING); \ +} + +#define DFLT_FMT \ + "module: %-30.30s instance: %-6d\n" \ + "name: %-30.30s class: %-.30s\n" + +#define KS_DFMT "\t%-30s " +#define KS_PFMT "%s:%d:%s:%s" + +typedef struct ks_instance { + list_node_t ks_next; + char ks_name[KSTAT_STRLEN]; + char ks_module[KSTAT_STRLEN]; + char ks_class[KSTAT_STRLEN]; + int ks_instance; + uchar_t ks_type; + hrtime_t ks_snaptime; + list_t ks_nvlist; +} ks_instance_t; + +typedef struct ks_nvpair { + list_node_t nv_next; + char name[KSTAT_STRLEN]; + uchar_t data_type; + ks_value_t value; +} ks_nvpair_t; + +typedef struct ks_pattern { + char *pstr; + regex_t preg; +} ks_pattern_t; + +typedef struct ks_selector { + list_node_t ks_next; + ks_pattern_t ks_module; + ks_pattern_t ks_instance; + ks_pattern_t ks_name; + ks_pattern_t ks_statistic; +} ks_selector_t; + +static void usage(void); +static int compare_instances(ks_instance_t *, ks_instance_t *); +static void nvpair_insert(ks_instance_t *, char *, ks_value_t *, uchar_t); +static boolean_t ks_match(const char *, ks_pattern_t *); +static ks_selector_t *new_selector(void); +static void ks_instances_read(kstat_ctl_t *); +static void ks_value_print(ks_nvpair_t *); +static void ks_instances_print(void); +static char *ks_safe_strdup(char *); +static void ks_sleep_until(hrtime_t *, hrtime_t, int, int *); +static int write_mode(int argc, char **argv); + +/* Raw kstat readers */ +#ifndef WIN32 +static void save_cpu_stat(kstat_t *, ks_instance_t *); +static void save_var(kstat_t *, ks_instance_t *); +static void save_ncstats(kstat_t *, ks_instance_t *); +static void save_sysinfo(kstat_t *, ks_instance_t *); +static void save_vminfo(kstat_t *, ks_instance_t *); +static void save_nfs(kstat_t *, ks_instance_t *); +#ifdef __sparc +static void save_sfmmu_global_stat(kstat_t *, ks_instance_t *); +static void save_sfmmu_tsbsize_stat(kstat_t *, ks_instance_t *); +static void save_simmstat(kstat_t *, ks_instance_t *); +/* Helper function for save_temperature() */ +static char *short_array_to_string(short *, int); +static void save_temperature(kstat_t *, ks_instance_t *); +static void save_temp_over(kstat_t *, ks_instance_t *); +static void save_ps_shadow(kstat_t *, ks_instance_t *); +static void save_fault_list(kstat_t *, ks_instance_t *); +#endif +#endif + +/* Named kstat readers */ +static void save_named(kstat_t *, ks_instance_t *); +static void save_intr(kstat_t *, ks_instance_t *); +static void save_io(kstat_t *, ks_instance_t *); +static void save_timer(kstat_t *, ks_instance_t *); + +/* Typedef for raw kstat reader functions */ +typedef void (*kstat_raw_reader_t)(kstat_t *, ks_instance_t *); + +static struct { + kstat_raw_reader_t fn; + char *name; +} ks_raw_lookup[] = { + /* Function name kstat name */ +#ifndef WIN32 + {save_cpu_stat, "cpu_stat:cpu_stat"}, + {save_var, "unix:var"}, + {save_ncstats, "unix:ncstats"}, + {save_sysinfo, "unix:sysinfo"}, + {save_vminfo, "unix:vminfo"}, + {save_nfs, "nfs:mntinfo"}, +#ifdef __sparc + {save_sfmmu_global_stat, "unix:sfmmu_global_stat"}, + {save_sfmmu_tsbsize_stat, "unix:sfmmu_tsbsize_stat"}, + {save_simmstat, "unix:simm-status"}, + {save_temperature, "unix:temperature"}, + {save_temp_over, "unix:temperature override"}, + {save_ps_shadow, "unix:ps_shadow"}, + {save_fault_list, "unix:fault_list"}, +#endif +#endif + {NULL, NULL}, +}; + +static kstat_raw_reader_t lookup_raw_kstat_fn(char *, char *); + +#endif /* _STAT_KSTAT_H */ diff --git a/cmd/os/windows/kstat/resource.h b/cmd/os/windows/kstat/resource.h new file mode 100644 index 000000000000..8a09cc650089 --- /dev/null +++ b/cmd/os/windows/kstat/resource.h @@ -0,0 +1,15 @@ +// {{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by resource.rc +// + +// Next default values for new objects +// +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1000 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif diff --git a/cmd/os/windows/kstat/resource.rc b/cmd/os/windows/kstat/resource.rc new file mode 100644 index 000000000000..5e3d52066d6e --- /dev/null +++ b/cmd/os/windows/kstat/resource.rc @@ -0,0 +1,110 @@ +// Microsoft Visual C++ generated resource script. +// +#include +#include "resource.h" + +#define APSTUDIO_READONLY_SYMBOLS +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 2 resource. +// +#include "winres.h" + +///////////////////////////////////////////////////////////////////////////// +#undef APSTUDIO_READONLY_SYMBOLS + +///////////////////////////////////////////////////////////////////////////// +// English (United States) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US +#pragma code_page(1252) + +#ifdef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// TEXTINCLUDE +// + +1 TEXTINCLUDE +BEGIN + "resource.h\0" +END + +2 TEXTINCLUDE +BEGIN + "#include ""winres.h""\r\n" + "\0" +END + +3 TEXTINCLUDE +BEGIN + "\r\n" + "\0" +END + +#endif // APSTUDIO_INVOKED + +#endif // English (United States) resources +///////////////////////////////////////////////////////////////////////////// + + +///////////////////////////////////////////////////////////////////////////// +// English (India) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENN) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_INDIA +#pragma code_page(1252) + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// + +VS_VERSION_INFO VERSIONINFO + FILEVERSION 1,0,0,1 + PRODUCTVERSION 1,0,0,1 + FILEFLAGSMASK 0x3fL +#ifdef _DEBUG + FILEFLAGS 0x1L +#else + FILEFLAGS 0x0L +#endif + FILEOS 0x40004L + FILETYPE 0x1L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "400904b0" + BEGIN + VALUE "CompanyName", "TODO: " + VALUE "FileDescription", "kstat.exe" + VALUE "FileVersion", "1.0.0.1" + VALUE "LegalCopyright", "Copyright (C) 2021" + VALUE "OriginalFilename", "resource.rc" + VALUE "ProductName", "kstat.exe" + VALUE "ProductVersion", "1.0.0.1" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x4009, 1200 + END +END + +#endif // English (India) resources +///////////////////////////////////////////////////////////////////////////// + + + +#ifndef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 3 resource. +// + + +///////////////////////////////////////////////////////////////////////////// +#endif // not APSTUDIO_INVOKED + diff --git a/cmd/os/windows/zfsinstaller/CMakeLists.txt b/cmd/os/windows/zfsinstaller/CMakeLists.txt new file mode 100644 index 000000000000..63331025a366 --- /dev/null +++ b/cmd/os/windows/zfsinstaller/CMakeLists.txt @@ -0,0 +1,15 @@ + +use_clang() + +um_add_executable(zfsinstaller + zfsinstaller.cpp + zfsinstaller.h + resource.rc +) +target_compile_definitions(zfsinstaller PRIVATE UNICODE _UNICODE) +target_link_libraries(zfsinstaller setupapi libspl) +install(TARGETS zfsinstaller RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}/driver") +install(FILES $ + DESTINATION "${CMAKE_INSTALL_BINDIR}/driver" + OPTIONAL +) diff --git a/cmd/os/windows/zfsinstaller/resource.h b/cmd/os/windows/zfsinstaller/resource.h new file mode 100644 index 000000000000..8a09cc650089 --- /dev/null +++ b/cmd/os/windows/zfsinstaller/resource.h @@ -0,0 +1,15 @@ +// {{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by resource.rc +// + +// Next default values for new objects +// +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1000 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif diff --git a/cmd/os/windows/zfsinstaller/resource.rc b/cmd/os/windows/zfsinstaller/resource.rc new file mode 100644 index 000000000000..a77bf60b0358 --- /dev/null +++ b/cmd/os/windows/zfsinstaller/resource.rc @@ -0,0 +1,109 @@ +// Microsoft Visual C++ generated resource script. +// +#include "resource.h" + +#define APSTUDIO_READONLY_SYMBOLS +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 2 resource. +// +#include "winres.h" + +///////////////////////////////////////////////////////////////////////////// +#undef APSTUDIO_READONLY_SYMBOLS + +///////////////////////////////////////////////////////////////////////////// +// English (United States) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US +#pragma code_page(1252) + +#ifdef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// TEXTINCLUDE +// + +1 TEXTINCLUDE +BEGIN + "resource.h\0" +END + +2 TEXTINCLUDE +BEGIN + "#include ""winres.h""\r\n" + "\0" +END + +3 TEXTINCLUDE +BEGIN + "\r\n" + "\0" +END + +#endif // APSTUDIO_INVOKED + +#endif // English (United States) resources +///////////////////////////////////////////////////////////////////////////// + + +///////////////////////////////////////////////////////////////////////////// +// English (India) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENN) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_INDIA +#pragma code_page(1252) + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// + +VS_VERSION_INFO VERSIONINFO + FILEVERSION 1,0,0,1 + PRODUCTVERSION 1,0,0,1 + FILEFLAGSMASK 0x3fL +#ifdef _DEBUG + FILEFLAGS 0x1L +#else + FILEFLAGS 0x0L +#endif + FILEOS 0x40004L + FILETYPE 0x1L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "400904b0" + BEGIN + VALUE "CompanyName", "TODO: " + VALUE "FileDescription", "zfsinstaller.exe" + VALUE "FileVersion", "1.0.0.1" + VALUE "LegalCopyright", "Copyright (C) 2021" + VALUE "OriginalFilename", "resource.rc" + VALUE "ProductName", "zfsinstaller.exe" + VALUE "ProductVersion", "1.0.0.1" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x4009, 1200 + END +END + +#endif // English (India) resources +///////////////////////////////////////////////////////////////////////////// + + + +#ifndef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 3 resource. +// + + +///////////////////////////////////////////////////////////////////////////// +#endif // not APSTUDIO_INVOKED + diff --git a/cmd/os/windows/zfsinstaller/zfsinstaller.cpp b/cmd/os/windows/zfsinstaller/zfsinstaller.cpp new file mode 100644 index 000000000000..e8a75b4f86c5 --- /dev/null +++ b/cmd/os/windows/zfsinstaller/zfsinstaller.cpp @@ -0,0 +1,904 @@ + +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License(the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2018 Julian Heuking + */ + +extern "C" { +#include +#include +#include +extern char *optarg; +extern int optind; + +#include +// kernel header' +// #include +#define ZFSIOCTL_BASE 0x800 +} + +#include "zfsinstaller.h" + +#include +#include +#define _SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING 1 +#include +namespace fs = std::experimental::filesystem; + +#define MAX_PATH_LEN 1024 + +// Usage: +// zfsinstaller install [inf] [installFolder] +// defaults to something like %ProgramFiles%\ZFS) +// zfsinstaller uninstall [inf] (could default to something +// like %ProgramFiles%\ZFS\ZFSin.inf) +// zfsinstaller trace [-f Flags] [-l Levels] [-s SizeOfETLInMB] +// [-p AbsolutePathOfETL] +// zfsinstaller trace -d + +const unsigned char OPEN_ZFS_GUID[] = "c20c603c-afd4-467d-bf76-c0a4c10553df"; +const unsigned char LOGGER_SESSION[] = "autosession\\OpenZFS_trace"; +const std::string ETL_FILE("\\OpenZFS.etl"); +const std::string MANIFEST_FILE("\\OpenZFS.man"); + +enum manifest_install_types +{ + MAN_INSTALL, + MAN_UNINSTALL, +}; + +int +session_exists(void) +{ + char command[MAX_PATH_LEN]; + + sprintf_s(command, "logman query %s > nul", LOGGER_SESSION); + + return (system(command)); // returns 0 if Session exists + // else non-zero if Session does not exist +} + +int +zfs_log_session_delete(void) +{ + char command[MAX_PATH_LEN]; + + int ret = session_exists(); + + if (ret == 0) { // Session exists + sprintf_s(command, "logman delete %s > nul", + LOGGER_SESSION); + ret = system(command); + if (ret == 0) + fprintf(stderr, "Logman session %s deleted " + "successfully\n", LOGGER_SESSION); + else + fprintf(stderr, "Error while deleting session %s\n", + LOGGER_SESSION); + return (ret); + } else + return (0); // Session does not exist ; We will pass success +} + +int +validate_flag_level(const char *str, size_t len) +{ + if ((str[0] == '0') && (str[1] == 'x' || str[1] == 'X')) str += 2; + + while (*str == '0') ++str; + + size_t length = strlen(str); + if (length > len) + return (1); + if (*str == '-') + return (2); + if (str[strspn(str, "0123456789abcdefABCDEF")] == 0) + return (0); // Vaild hex string; + else + return (3); // Invalid hex string; +} + +int +validate_args(const char *flags, const char *levels, + int size_in_mb, const char *etl_file) { + if (validate_flag_level(flags, 8)) { + fprintf(stderr, "Valid input for flags should be in " + "interval [0x0, 0xffffffff]\n"); + return (1); + } + + if (validate_flag_level(levels, 2)) { + fprintf(stderr, "Valid input for levels should be in " + "interval [0x0, 0xff]\n"); + return (2); + } + + if (etl_file) { + if (!strstr(etl_file, ".etl")) { + fprintf(stderr, "Etl file path/name %s is incorrect\n", + etl_file); + return (3); + } + } else { + fprintf(stderr, "Etl file path/name is incorrect\n"); + return (4); + } + + if (size_in_mb <= 0) { + fprintf(stderr, "Size of etl should be greater than 0\n"); + return (5); + } + return (0); +} + +int +move_file(const char *etl_file) +{ + char move_etl[MAX_PATH_LEN]; + time_t rawtime; + struct tm timeinfo; + char buffer[25]; + + time(&rawtime); + localtime_s(&timeinfo, &rawtime); + strftime(buffer, sizeof (buffer), "_%Y%m%d%H%M%S", &timeinfo); + + strcpy_s(move_etl, etl_file); + char *etl = strstr(move_etl, ".etl"); + if (etl) + etl[0] = 0; + + strcat_s(move_etl, buffer); + strcat_s(move_etl, ".etl"); + + if (0 == rename(etl_file, move_etl)) { + fprintf(stderr, "%s already exists\n", etl_file); + fprintf(stderr, "%s has been renamed to %s\n", + etl_file, move_etl); + return (0); + } else { + fprintf(stderr, "Error while renaming the file %s\n", etl_file); + return (1); + } +} + +void +hex_modify(std::string& hex) +{ + if (hex[0] == '0' && (hex[1] == 'x' || hex[1] == 'X')) + return; + hex = std::string("0x") + hex; +} + +std::string get_cwd() { + CHAR cwd_path[MAX_PATH_LEN] = { 0 }; + DWORD len = GetCurrentDirectoryA(MAX_PATH_LEN, cwd_path); + return (std::string(cwd_path)); +} + +int +arg_parser(int argc, char **argv, std::string &flags, + std::string &levels, int &size_in_mb, std::string &etl_file) +{ + int option_index = 0; + while ((option_index = getopt(argc, argv, "l:f:s:p:d")) != -1) { + switch (option_index) { + case 'p': + etl_file = std::string(optarg); + break; + case 'l': + levels = std::string(optarg); + hex_modify(levels); + break; + case 'f': + flags = std::string(optarg); + hex_modify(flags); + break; + case 's': + size_in_mb = atoi(optarg); + break; + case 'd': + fprintf(stderr, "-d cannot used with other " + "parameters\n"); + return (1); + default: + fprintf(stderr, "Incorrect argument provided\n"); + return (ERROR_BAD_ARGUMENTS); + } + } + int index = optind; + while (index < argc) { + fprintf(stderr, "Non-option argument %s\n", argv[index]); + index++; + } + if (optind < argc) + return (ERROR_BAD_ARGUMENTS); + + if (0 == flags.size()) flags = std::string("0xffffffff"); + if (0 == levels.size()) levels = std::string("0x4"); + if (-1 == size_in_mb) size_in_mb = 250; + if (0 == etl_file.size()) { + TCHAR CurrentPath[MAX_PATH_LEN + 1] = L""; + DWORD len = GetCurrentDirectory(MAX_PATH_LEN, CurrentPath); + int size_needed = WideCharToMultiByte(CP_UTF8, 0, + &CurrentPath[0], MAX_PATH_LEN, NULL, 0, NULL, NULL); + std::string CwdPath(size_needed, 0); + WideCharToMultiByte(CP_UTF8, 0, &CurrentPath[0], + MAX_PATH_LEN, &CwdPath[0], size_needed, NULL, NULL); + CwdPath.erase(len); + etl_file = get_cwd() + ETL_FILE; + } + return (0); +} + +void +sanitize(char *s) +{ + static char ok_chars[] = "abcdefghijklmnopqrstuvwxyz" + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "1234567890_-.@"; + char *cp = s; + const char *end = s + strlen(s); + for (cp += strspn(cp, ok_chars); cp != end; cp += strspn(cp, ok_chars)) { + *cp = '_'; + } +} + +int +zfs_log_session_create(int argc, char **argv) +{ + std::string flags; + std::string levels; + int size_in_mb = -1; + std::string etl_file; + char command[MAX_PATH_LEN]; + int ret; + + ret = arg_parser(argc, argv, flags, levels, size_in_mb, etl_file); + if (ret) { + printUsage(); + return (ret); + } + + if (validate_args(flags.c_str(), levels.c_str(), size_in_mb, + etl_file.c_str())) { + fprintf(stderr, "Please check the provided values for " + "the arguments\n"); + printUsage(); + return (1); + } + + if (0 != session_exists()) { // If Session does not exist + if (GetFileAttributesA(etl_file.c_str()) != + INVALID_FILE_ATTRIBUTES) { // ETL EXISTS + ret = move_file(etl_file.c_str()); + if (ret) + return (ret); + } + + snprintf(command, MAX_PATH_LEN, + "logman create trace %s -p {%s} %s %s" + " -nb 1 1 -bs 1 -mode Circular -max %d -o \"%s\" ", + LOGGER_SESSION, OPEN_ZFS_GUID, flags.c_str(), + levels.c_str(), size_in_mb, etl_file.c_str()); + + /* sanitize */ + sanitize(command); + + ret = system(command); + if (ret != 0) + fprintf(stderr, "There is an issue creating the " + "session %s\n", LOGGER_SESSION); + else + fprintf(stderr, "Logman Session %s successfully " + "created\n", LOGGER_SESSION); + + return (ret); + } else + fprintf(stderr, "Logman Session %s already exists\n", + LOGGER_SESSION); + + return (0); +} + +int perf_counters(char *inf_path, int type) { + int error = 0; + fs::path path = std::string(inf_path); + std::string final_path; + + char driver_path[MAX_PATH_LEN] = { 0 }; + strncpy_s(driver_path, inf_path, MAX_PATH_LEN); + char *slash = strrchr(driver_path, '\\'); + *slash = '\0'; + + if (path.is_absolute()) + final_path = std::string(driver_path) + MANIFEST_FILE; + else if (path.is_relative()) + final_path = get_cwd() + std::string("\\") + std::string( + driver_path) + MANIFEST_FILE; + + char command[MAX_PATH_LEN] = { 0 }; + switch (type) + { + case MAN_INSTALL: + sprintf_s(command, "lodctr /m:\"%s\"\n", final_path.c_str()); + break; + case MAN_UNINSTALL: + sprintf_s(command, "unlodctr /m:\"%s\"\n", final_path.c_str()); + break; + default: + break; + } + + fprintf(stderr, "Executing %s\n", command); + return (system(command)); +} + +int perf_counters_install(char *inf_path) { + return (perf_counters(inf_path, MAN_INSTALL)); +} + + +int perf_counters_uninstall(char *inf_path) { + return (perf_counters(inf_path, MAN_UNINSTALL)); +} + +int +main(int argc, char *argv[]) +{ + if (argc < 2) { + fprintf(stderr, "too few arguments \n"); + printUsage(); + return (ERROR_BAD_ARGUMENTS); + } + if (argc > 10) { + fprintf(stderr, "too many arguments \n"); + printUsage(); + return (ERROR_BAD_ARGUMENTS); + } + + if (strcmp(argv[1], "install") == 0) { + if (argc == 3) { + zfs_install(argv[2]); + fprintf(stderr, "Installation done."); + } else { + fprintf(stderr, "Incorrect argument usage\n"); + printUsage(); + return (ERROR_BAD_ARGUMENTS); + } + } else if (strcmp(argv[1], "uninstall") == 0) { + if (argc == 3) { + int ret = zfs_uninstall(argv[2]); + if (0 == ret) + return (zfs_log_session_delete()); + return (ret); + } else { + fprintf(stderr, "Incorrect argument usage\n"); + printUsage(); + return (ERROR_BAD_ARGUMENTS); + } + } else if (strcmp(argv[1], "trace") == 0) { + if (argc == 3 && strcmp(argv[2], "-d") == 0) + return (zfs_log_session_delete()); + else + return (zfs_log_session_create(argc - 1, &argv[1])); + } else { + fprintf(stderr, "unknown argument %s\n", argv[1]); + printUsage(); + return (ERROR_BAD_ARGUMENTS); + } + return (0); +} + +void +printUsage() { + fprintf(stderr, "\nUsage:\n\n"); + fprintf(stderr, "Install driver per INF DefaultInstall section:\n"); + fprintf(stderr, "zfsinstaller install inf_path\n"); + fprintf(stderr, "\n"); + fprintf(stderr, "Uninstall driver per INF DefaultUninstall section:\n"); + fprintf(stderr, "zfsinstaller uninstall inf_path\n"); + fprintf(stderr, "\n"); + fprintf(stderr, "zfsinstaller trace [-f Flags] | [-l Levels]" + " | [-s SizeOfETLInMB] | [-p AbsolutePathOfETL]\n"); + fprintf(stderr, "Valid inputs for above arguments are as follows:\n"); + fprintf(stderr, "Flags (in hex) " + "Should be in interval [0x0, 0xffffffff] " + "Default (0xffffffff)\n"); + fprintf(stderr, "Levels (in hex) " + "Should be in interval [0x0, 0xff] " + "Default (0x4)\n"); + fprintf(stderr, "SizeOfETLInMB (in decimal) " + "Should be greater than 0 " + "Default (250)\n"); + fprintf(stderr, "AbsolutePathOfETL " + "Absolute Path including the Etl file name " + "Default ($CWD%s)\n", ETL_FILE.c_str()); + fprintf(stderr, "\n"); + fprintf(stderr, "zfsinstaller trace -d\n"); + fprintf(stderr, "-d To delete the logman session\n"); +} + +DWORD zfs_install(char *inf_path) { + + DWORD error = 0; + // 128+4 If a reboot of the computer is necessary, + // ask the user for permission before rebooting. + + if (_access(inf_path, F_OK) != 0) { + char cwd[1024]; + _getcwd(cwd, sizeof (cwd)); + fprintf(stderr, "Unable to locate '%s' we are at '%s'\r\n", + inf_path, cwd); + return (-1); + } + + error = executeInfSection("OpenZFS_Install 128 ", inf_path); + + // Start driver service if not already running + char serviceName[] = "OpenZFS"; + if (!error) + error = startService(serviceName); + else + fprintf(stderr, "Installation failed, skip " + "starting the service\r\n"); + + if (!error) + error = installRootDevice(inf_path); + + if (!error) + perf_counters_install(inf_path); + + return (error); +} + +DWORD +zfs_uninstall(char *inf_path) +{ + DWORD ret = 0; + + ret = send_zfs_ioc_unregister_fs(); + + Sleep(2000); + + // 128+2 Always ask the users if they want to reboot. + if (ret == 0) + ret = executeInfSection("DefaultUninstall 128 ", inf_path); + + if (ret == 0) { + ret = uninstallRootDevice(inf_path); + perf_counters_uninstall(inf_path); + } + + return (ret); +} + + +DWORD +executeInfSection(const char *cmd, char *inf_path) { + +#ifdef _DEBUG + system("sc query ZFSin"); + fprintf(stderr, "\n\n"); +#endif + + DWORD error = 0; + + size_t len = strlen(cmd) + strlen(inf_path) + 1; + size_t sz = 0; + char buf[MAX_PATH]; + wchar_t wc_buf[MAX_PATH]; + + sprintf_s(buf, "%s%s", cmd, inf_path); + fprintf(stderr, "%s\n", buf); + + mbstowcs_s(&sz, wc_buf, len, buf, MAX_PATH); + + InstallHinfSection( + NULL, + NULL, + wc_buf, + 0 + ); + + +#ifdef _DEBUG + system("sc query ZFSin"); +#endif + + return (error); + // if we want to have some more control on installation, we need to get + // a bit deeper into the setupapi, something like the following... + + /* + * HINF inf = SetupOpenInfFile( + * L"C:\\master_test\\ZFSin\\ZFSin.inf", //PCWSTR FileName, + * NULL, //PCWSTR InfClass, + * INF_STYLE_WIN4, //DWORD InfStyle, + * 0//PUINT ErrorLine + * ); + * + * if (!inf) { + * std::cout << "SetupOpenInfFile failed, err " + * << GetLastError() << "\n"; + * return (-1); + * } + * + * + * int ret = SetupInstallFromInfSection( + * NULL, //owner + * inf, //inf handle + * L"DefaultInstall", + * SPINST_ALL, //flags + * NULL, //RelativeKeyRoot + * NULL, //SourceRootPath + * SP_COPY_NEWER_OR_SAME | SP_COPY_IN_USE_NEEDS_REBOOT, //CopyFlags + * NULL, //MsgHandler + * NULL, //Context + * NULL, //DeviceInfoSet + * NULL //DeviceInfoData + * ); + * + * if (!ret) { + * std::cout << "SetupInstallFromInfSection failed, err " + * << GetLastError() << "\n"; + * return (-1); + * } + * + * SetupCloseInfFile(inf); + */ +} + +DWORD +startService(char *serviceName) +{ + DWORD error = 0; + SC_HANDLE servMgrHdl; + SC_HANDLE zfsServHdl; + + servMgrHdl = OpenSCManager(NULL, NULL, GENERIC_READ | GENERIC_EXECUTE); + + if (!servMgrHdl) { + fprintf(stderr, "OpenSCManager failed, error %d\n", + GetLastError()); + error = GetLastError(); + goto End; + } + + zfsServHdl = OpenServiceA(servMgrHdl, serviceName, + GENERIC_READ | GENERIC_EXECUTE); + + if (!zfsServHdl) { + fprintf(stderr, "OpenServiceA failed, error %d\n", + GetLastError()); + error = GetLastError(); + goto CloseMgr; + } + + if (!StartServiceA(zfsServHdl, NULL, NULL)) { + if (GetLastError() == ERROR_SERVICE_ALREADY_RUNNING) { + fprintf(stderr, "Service is already running\n"); + } else { + fprintf(stderr, "StartServiceA failed, error %d\n", + GetLastError()); + // error = GetLastError(); + goto CloseServ; + } + } + +CloseServ: + CloseServiceHandle(zfsServHdl); +CloseMgr: + CloseServiceHandle(servMgrHdl); +End: + return (error); +} + +#define ZFSIOCTL_TYPE 40000 + +DWORD +send_zfs_ioc_unregister_fs(void) +{ + HANDLE g_fd = CreateFile(L"\\\\.\\ZFS", GENERIC_READ | GENERIC_WRITE, + 0, NULL, OPEN_EXISTING, 0, NULL); + + DWORD bytesReturned; + + if (g_fd == INVALID_HANDLE_VALUE) { + printf("Unable to open ZFS devnode, already uninstalled?\n"); + return (0); + } + + // We use bytesReturned to hold "zfs_module_busy". + BOOL ret = DeviceIoControl( + g_fd, + CTL_CODE(ZFSIOCTL_TYPE, ZFSIOCTL_BASE + ZFS_IOC_UNREGISTER_FS, + METHOD_NEITHER, FILE_ANY_ACCESS), + NULL, + 0, + NULL, + 0, + &bytesReturned, + NULL); + + CloseHandle(g_fd); + + if (!ret) + return (1); + + if (bytesReturned != 0) { + fprintf(stderr, "ZFS: Unable to uninstall until all pools are " + "exported: %lu pool(s)\r\n", bytesReturned); + return (2); + } + + return (0); +} + +#include +#include +#include + +#define ZFS_ROOTDEV "Root\\OpenZFS" +// DevCon uses LoadLib() - but lets just static link +#pragma comment(lib, "Newdev.lib") + +HDEVINFO +openDeviceInfo(char *inf, GUID *ClassGUID, char *ClassName, int namemax) +{ + HDEVINFO DeviceInfoSet = INVALID_HANDLE_VALUE; + char InfPath[MAX_PATH]; + + // Inf must be a full pathname + if (GetFullPathNameA(inf, MAX_PATH, InfPath, NULL) >= MAX_PATH) { + // inf pathname too long + goto final; + } + + // Use the INF File to extract the Class GUID. + if (!SetupDiGetINFClassA(InfPath, ClassGUID, ClassName, + sizeof (ClassName) / sizeof (ClassName[0]), 0)) { + goto final; + } + + // Create the container for the to-be-created + // Device Information Element. + DeviceInfoSet = SetupDiCreateDeviceInfoList(ClassGUID, 0); + if (DeviceInfoSet == INVALID_HANDLE_VALUE) { + goto final; + } + + return (DeviceInfoSet); + +final: + return (NULL); +} + + + +DWORD +installRootDevice(char *inf) +{ + HDEVINFO DeviceInfoSet = INVALID_HANDLE_VALUE; + SP_DEVINFO_DATA DeviceInfoData; + char hwIdList[LINE_LEN + 4]; + GUID ClassGUID; + char ClassName[MAX_CLASS_NAME_LEN]; + int failcode = 12; + + DWORD flags = INSTALLFLAG_FORCE; + BOOL reboot = FALSE; + + DeviceInfoSet = openDeviceInfo(inf, &ClassGUID, ClassName, + MAX_CLASS_NAME_LEN); + + ZeroMemory(hwIdList, sizeof (hwIdList)); + if (FAILED(StringCchCopyA(hwIdList, LINE_LEN, ZFS_ROOTDEV))) { + goto final; + } + + // Now create the element. + // Use the Class GUID and Name from the INF file. + DeviceInfoData.cbSize = sizeof (SP_DEVINFO_DATA); + if (!SetupDiCreateDeviceInfoA(DeviceInfoSet, + ClassName, + &ClassGUID, + NULL, + 0, + DICD_GENERATE_ID, + &DeviceInfoData)) { + goto final; + } + + // Add the HardwareID to the Device's HardwareID property. + if (!SetupDiSetDeviceRegistryPropertyA(DeviceInfoSet, + &DeviceInfoData, + SPDRP_HARDWAREID, + (LPBYTE)hwIdList, + (DWORD) (strlen(hwIdList) + 1 + 1) * sizeof (char))) { + goto final; + } + + // Transform the registry element into an actual devnode + // in the PnP HW tree. + if (!SetupDiCallClassInstaller(DIF_REGISTERDEVICE, + DeviceInfoSet, + &DeviceInfoData)) { + goto final; + } + + failcode = 0; + + // According to devcon we also have to Update now as well. + UpdateDriverForPlugAndPlayDevicesA(NULL, ZFS_ROOTDEV, + inf, flags, &reboot); + + if (reboot) printf("Windows indicated a Reboot is required.\n"); + +final: + + if (DeviceInfoSet != INVALID_HANDLE_VALUE) { + SetupDiDestroyDeviceInfoList(DeviceInfoSet); + } + printf("%s: exit %d:0x%x\n", __func__, failcode, failcode); + + return (failcode); +} + +DWORD +uninstallRootDevice(char *inf) +{ + int failcode = 13; + HDEVINFO DeviceInfoSet = INVALID_HANDLE_VALUE; + SP_DEVINFO_DATA DeviceInfoData; + DWORD DataT; + char *p, *buffer = NULL; + DWORD buffersize = 0; + + printf("%s: \n", __func__); + + DeviceInfoSet = SetupDiGetClassDevs(NULL, // All Classes + 0, 0, DIGCF_ALLCLASSES | DIGCF_PRESENT); + // All devices present on system + if (DeviceInfoSet == INVALID_HANDLE_VALUE) + goto final; + + printf("%s: looking for device rootnode to remove...\n", __func__); + + DeviceInfoData.cbSize = sizeof (SP_DEVINFO_DATA); + for (int i = 0; SetupDiEnumDeviceInfo(DeviceInfoSet, i, + &DeviceInfoData); i++) { + // Call once to get buffersize + while (!SetupDiGetDeviceRegistryPropertyA( + DeviceInfoSet, + &DeviceInfoData, + SPDRP_HARDWAREID, + &DataT, + (PBYTE)buffer, + buffersize, + &buffersize)) { + + if (GetLastError() == ERROR_INVALID_DATA) { + // May be a Legacy Device with no HardwareID. Continue. + break; + } else if (GetLastError() == + ERROR_INSUFFICIENT_BUFFER) { + // We need to change the buffer size. + if (buffer) + free(buffer); + buffer = (char *)malloc(buffersize); + if (buffer) ZeroMemory(buffer, buffersize); + } else { + // Unknown Failure. + goto final; + } + } + + if (GetLastError() == ERROR_INVALID_DATA) + continue; + + // Compare each entry in the buffer multi-sz list + // with our HardwareID. + for (p = buffer; *p && (p < &buffer[buffersize]); + p += strlen(p) + sizeof (char)) { + // printf("%s: comparing '%s' with '%s'\n", + // __func__, "ROOT\\ZFSin", p); + if (!_stricmp(ZFS_ROOTDEV, p)) { + + printf("%s: device found, removing ... \n", + __func__); + + // Worker function to remove device. + if (SetupDiCallClassInstaller(DIF_REMOVE, + DeviceInfoSet, &DeviceInfoData)) { + failcode = 0; + } + break; + } + } + + if (buffer) free(buffer); + buffer = NULL; + buffersize = 0; + } + +final: + + if (DeviceInfoSet != INVALID_HANDLE_VALUE) { + SetupDiDestroyDeviceInfoList(DeviceInfoSet); + } + printf("%s: exit %d:0x%x\n", __func__, failcode, failcode); + + return (failcode); +} + +#if 0 + + + + + ZeroMemory(hwIdList, sizeof (hwIdList)); + if (FAILED(StringCchCopyA(hwIdList, LINE_LEN, "ROOT\\ZFSin"))) { + goto final; + } + + printf("%s: CchCopy\n", __func__); + + // Now create the element. + // Use the Class GUID and Name from the INF file. + DeviceInfoData.cbSize = sizeof (SP_DEVINFO_DATA); + if (!SetupDiCreateDeviceInfoA(DeviceInfoSet, + ClassName, + &ClassGUID, + NULL, + 0, + DICD_GENERATE_ID, + &DeviceInfoData)) { + goto final; + } + + printf("%s: SetupDiCreateDeviceInfoA\n", __func__); + + rmdParams.ClassInstallHeader.cbSize = sizeof (SP_CLASSINSTALL_HEADER); + rmdParams.ClassInstallHeader.InstallFunction = DIF_REMOVE; + rmdParams.Scope = DI_REMOVEDEVICE_GLOBAL; + rmdParams.HwProfile = 0; + if (!SetupDiSetClassInstallParamsA(DeviceInfoSet, &DeviceInfoData, + &rmdParams.ClassInstallHeader, sizeof (rmdParams)) || + !SetupDiCallClassInstaller(DIF_REMOVE, DeviceInfoSet, + &DeviceInfoData)) { + + // failed to invoke DIF_REMOVE + failcode = 14; + goto final; + } + + failcode = 0; + +final: + if (DeviceInfoSet != INVALID_HANDLE_VALUE) { + SetupDiDestroyDeviceInfoList(DeviceInfoSet); + } + printf("%s: exit %d:0x%x\n", __func__, failcode, failcode); + return (failcode); +} +#endif diff --git a/cmd/os/windows/zfsinstaller/zfsinstaller.h b/cmd/os/windows/zfsinstaller/zfsinstaller.h new file mode 100644 index 000000000000..a7ed59791d85 --- /dev/null +++ b/cmd/os/windows/zfsinstaller/zfsinstaller.h @@ -0,0 +1,39 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2018 Julian Heuking + */ + +#pragma once + +#include +#include +#include +#include + +DWORD zfs_install(char *); +DWORD zfs_uninstall(char *); +DWORD executeInfSection(const char *, char *); +DWORD startService(char *); +void printUsage(); +DWORD send_zfs_ioc_unregister_fs(); +DWORD installRootDevice(char *inf_path); +DWORD uninstallRootDevice(char *inf_path); diff --git a/cmd/raidz_test/CMakeLists.txt b/cmd/raidz_test/CMakeLists.txt new file mode 100644 index 000000000000..d28593b443d0 --- /dev/null +++ b/cmd/raidz_test/CMakeLists.txt @@ -0,0 +1,22 @@ + +use_clang() + +um_add_executable(raidz_test + raidz_test.c + raidz_test.h + raidz_bench.c + os/windows/resource.rc +) +# https://github.com/openzfsonwindows/openzfs/issues/31 +target_compile_definitions(raidz_test PRIVATE ZFS_DEBUG) + +target_link_libraries(raidz_test PRIVATE + libzfs + libzfs_core + libzpool +) +install(TARGETS raidz_test RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}") +install(FILES $ + DESTINATION "${CMAKE_INSTALL_BINDIR}" + OPTIONAL +) diff --git a/cmd/raidz_test/os/windows/resource.h b/cmd/raidz_test/os/windows/resource.h new file mode 100644 index 000000000000..7c01e17c2ac3 --- /dev/null +++ b/cmd/raidz_test/os/windows/resource.h @@ -0,0 +1,15 @@ +// {{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by resource.rc +// + +// Next default values for new objects + +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1000 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif diff --git a/cmd/raidz_test/os/windows/resource.rc b/cmd/raidz_test/os/windows/resource.rc new file mode 100644 index 000000000000..e6d2e19bf3e5 --- /dev/null +++ b/cmd/raidz_test/os/windows/resource.rc @@ -0,0 +1,111 @@ +// Microsoft Visual C++ generated resource script. +// +#include +#include "resource.h" + +#define APSTUDIO_READONLY_SYMBOLS +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 2 resource. +// +#include "winres.h" + +///////////////////////////////////////////////////////////////////////////// +#undef APSTUDIO_READONLY_SYMBOLS + +///////////////////////////////////////////////////////////////////////////// +// English (United States) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US +#pragma code_page(1252) + +#ifdef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// TEXTINCLUDE +// + +1 TEXTINCLUDE +BEGIN + "resource.h\0" +END + +2 TEXTINCLUDE +BEGIN + "#include ""winres.h""\r\n" + "\0" +END + +3 TEXTINCLUDE +3 TEXTINCLUDE +BEGIN + "\r\n" + "\0" +END + +#endif // APSTUDIO_INVOKED + +#endif // English (United States) resources +///////////////////////////////////////////////////////////////////////////// + + +///////////////////////////////////////////////////////////////////////////// +// English (India) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENN) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_INDIA +#pragma code_page(1252) + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// + +VS_VERSION_INFO VERSIONINFO + FILEVERSION 1,0,0,1 + PRODUCTVERSION 1,0,0,1 + FILEFLAGSMASK 0x3fL +#ifdef _DEBUG + FILEFLAGS 0x1L +#else + FILEFLAGS 0x0L +#endif + FILEOS 0x40004L + FILETYPE 0x1L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "400904b0" + BEGIN + VALUE "CompanyName", "TODO: " + VALUE "FileDescription", "raidz_test.exe" + VALUE "FileVersion", "1.0.0.1" + VALUE "LegalCopyright", "Copyright (C) 2021" + VALUE "OriginalFilename", "resource.rc" + VALUE "ProductName", "raidz_test.exe" + VALUE "ProductVersion", "1.0.0.1" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x4009, 1200 + END +END + +#endif // English (India) resources +///////////////////////////////////////////////////////////////////////////// + + + +#ifndef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 3 resource. +// + + +///////////////////////////////////////////////////////////////////////////// +#endif // not APSTUDIO_INVOKED + diff --git a/cmd/raidz_test/raidz_test.c b/cmd/raidz_test/raidz_test.c index 195026d3a7ab..06ba4ca12f8d 100644 --- a/cmd/raidz_test/raidz_test.c +++ b/cmd/raidz_test/raidz_test.c @@ -984,6 +984,7 @@ main(int argc, char **argv) /* init gdb pid string early */ (void) sprintf(pid_s, "%d", getpid()); +#ifndef WIN32 action.sa_handler = sig_handler; sigemptyset(&action.sa_mask); action.sa_flags = 0; @@ -994,6 +995,7 @@ main(int argc, char **argv) } (void) setvbuf(stdout, NULL, _IOLBF, 0); +#endif dprintf_setup(&argc, argv); @@ -1020,5 +1022,6 @@ main(int argc, char **argv) umem_free(rand_data, SPA_MAXBLOCKSIZE); kernel_fini(); + printf("Application exit code = %d\n", err); return (err); } diff --git a/cmd/zdb/CMakeLists.txt b/cmd/zdb/CMakeLists.txt new file mode 100644 index 000000000000..99dd921d3d86 --- /dev/null +++ b/cmd/zdb/CMakeLists.txt @@ -0,0 +1,22 @@ + +use_clang() + +um_add_executable(zdb + zdb.c + zdb.h + zdb_il.c + os/windows/resource.rc +) +target_compile_definitions(zdb PRIVATE ZFS_DEBUG) +target_link_libraries(zdb PRIVATE + libnvpair + libuutil + libzfs + libzfs_core + libzpool +) +install(TARGETS zdb RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}") +install(FILES $ + DESTINATION "${CMAKE_INSTALL_BINDIR}" + OPTIONAL +) diff --git a/cmd/zdb/os/windows/resource.h b/cmd/zdb/os/windows/resource.h new file mode 100644 index 000000000000..8a09cc650089 --- /dev/null +++ b/cmd/zdb/os/windows/resource.h @@ -0,0 +1,15 @@ +// {{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by resource.rc +// + +// Next default values for new objects +// +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1000 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif diff --git a/cmd/zdb/os/windows/resource.rc b/cmd/zdb/os/windows/resource.rc new file mode 100644 index 000000000000..81fb21062278 --- /dev/null +++ b/cmd/zdb/os/windows/resource.rc @@ -0,0 +1,110 @@ +// Microsoft Visual C++ generated resource script. +// +#include +#include "resource.h" + +#define APSTUDIO_READONLY_SYMBOLS +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 2 resource. +// +#include "winres.h" + +///////////////////////////////////////////////////////////////////////////// +#undef APSTUDIO_READONLY_SYMBOLS + +///////////////////////////////////////////////////////////////////////////// +// English (United States) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US +#pragma code_page(1252) + +#ifdef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// TEXTINCLUDE +// + +1 TEXTINCLUDE +BEGIN + "resource.h\0" +END + +2 TEXTINCLUDE +BEGIN + "#include ""winres.h""\r\n" + "\0" +END + +3 TEXTINCLUDE +BEGIN + "\r\n" + "\0" +END + +#endif // APSTUDIO_INVOKED + +#endif // English (United States) resources +///////////////////////////////////////////////////////////////////////////// + + +///////////////////////////////////////////////////////////////////////////// +// English (India) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENN) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_INDIA +#pragma code_page(1252) + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// + +VS_VERSION_INFO VERSIONINFO + FILEVERSION 1,0,0,1 + PRODUCTVERSION 1,0,0,1 + FILEFLAGSMASK 0x3fL +#ifdef _DEBUG + FILEFLAGS 0x1L +#else + FILEFLAGS 0x0L +#endif + FILEOS 0x40004L + FILETYPE 0x1L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "400904b0" + BEGIN + VALUE "CompanyName", "TODO: " + VALUE "FileDescription", "zdb.exe" + VALUE "FileVersion", "1.0.0.1" + VALUE "LegalCopyright", "Copyright (C) 2021" + VALUE "OriginalFilename", "resource.rc" + VALUE "ProductName", "zdb.exe" + VALUE "ProductVersion", "1.0.0.1" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x4009, 1200 + END +END + +#endif // English (India) resources +///////////////////////////////////////////////////////////////////////////// + + + +#ifndef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 3 resource. +// + + +///////////////////////////////////////////////////////////////////////////// +#endif // not APSTUDIO_INVOKED + diff --git a/cmd/zdb/zdb.c b/cmd/zdb/zdb.c index 04a10c4eedd7..4521c2c38497 100644 --- a/cmd/zdb/zdb.c +++ b/cmd/zdb/zdb.c @@ -118,6 +118,7 @@ zdb_ot_name(dmu_object_type_t type) extern int reference_tracking_enable; extern int zfs_recover; +extern uint64_t zfs_arc_meta_min, zfs_arc_meta_limit; extern uint_t zfs_vdev_async_read_max_active; extern boolean_t spa_load_verify_dryrun; extern boolean_t spa_mode_readable_spacemaps; @@ -4180,6 +4181,11 @@ dump_cachefile(const char *cachefile) exit(1); } + if (statbuf.st_size == 0) { + (void) close(fd); + return; + } + if ((buf = malloc(statbuf.st_size)) == NULL) { (void) fprintf(stderr, "failed to allocate %llu bytes\n", (u_longlong_t)statbuf.st_size); @@ -8186,7 +8192,7 @@ zdb_dump_block(char *label, void *buf, uint64_t size, int flags) int do_bswap = !!(flags & ZDB_FLAG_BSWAP); unsigned i, j; const char *hdr; - char *c; + unsigned char *c; if (do_bswap) @@ -8206,7 +8212,7 @@ zdb_dump_block(char *label, void *buf, uint64_t size, int flags) (u_longlong_t)(do_bswap ? BSWAP_64(d[i]) : d[i]), (u_longlong_t)(do_bswap ? BSWAP_64(d[i + 1]) : d[i + 1])); - c = (char *)&d[i]; + c = (unsigned char *)&d[i]; for (j = 0; j < 2 * sizeof (uint64_t); j++) (void) printf("%c", isprint(c[j]) ? c[j] : '.'); (void) printf("\n"); diff --git a/cmd/zfs/CMakeLists.txt b/cmd/zfs/CMakeLists.txt new file mode 100644 index 000000000000..993b1f7e3184 --- /dev/null +++ b/cmd/zfs/CMakeLists.txt @@ -0,0 +1,23 @@ + +use_clang() + +um_add_executable(zfs + zfs_iter.c + zfs_main.c + zfs_iter.h + zfs_util.h + zfs_project.c + os/windows/resource.rc +) +target_link_libraries(zfs PRIVATE + libnvpair + libuutil + libzfs + libzfs_core + libzpool +) +install(TARGETS zfs RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}") +install(FILES $ + DESTINATION "${CMAKE_INSTALL_BINDIR}" + OPTIONAL +) diff --git a/cmd/zfs/os/windows/resource.h b/cmd/zfs/os/windows/resource.h new file mode 100644 index 000000000000..8a09cc650089 --- /dev/null +++ b/cmd/zfs/os/windows/resource.h @@ -0,0 +1,15 @@ +// {{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by resource.rc +// + +// Next default values for new objects +// +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1000 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif diff --git a/cmd/zfs/os/windows/resource.rc b/cmd/zfs/os/windows/resource.rc new file mode 100644 index 000000000000..aac3a29d725d --- /dev/null +++ b/cmd/zfs/os/windows/resource.rc @@ -0,0 +1,110 @@ +// Microsoft Visual C++ generated resource script. +// +#include +#include "resource.h" + +#define APSTUDIO_READONLY_SYMBOLS +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 2 resource. +// +#include "winres.h" + +///////////////////////////////////////////////////////////////////////////// +#undef APSTUDIO_READONLY_SYMBOLS + +///////////////////////////////////////////////////////////////////////////// +// English (United States) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US +#pragma code_page(1252) + +#ifdef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// TEXTINCLUDE +// + +1 TEXTINCLUDE +BEGIN + "resource.h\0" +END + +2 TEXTINCLUDE +BEGIN + "#include ""winres.h""\r\n" + "\0" +END + +3 TEXTINCLUDE +BEGIN + "\r\n" + "\0" +END + +#endif // APSTUDIO_INVOKED + +#endif // English (United States) resources +///////////////////////////////////////////////////////////////////////////// + + +///////////////////////////////////////////////////////////////////////////// +// English (India) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENN) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_INDIA +#pragma code_page(1252) + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// + +VS_VERSION_INFO VERSIONINFO + FILEVERSION 1,0,0,1 + PRODUCTVERSION 1,0,0,1 + FILEFLAGSMASK 0x3fL +#ifdef _DEBUG + FILEFLAGS 0x1L +#else + FILEFLAGS 0x0L +#endif + FILEOS 0x40004L + FILETYPE 0x1L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "400904b0" + BEGIN + VALUE "CompanyName", "TODO: " + VALUE "FileDescription", "zfs.exe" + VALUE "FileVersion", "1.0.0.1" + VALUE "LegalCopyright", "Copyright (C) 2021" + VALUE "OriginalFilename", "resource.rc" + VALUE "ProductName", "zfs.exe" + VALUE "ProductVersion", "1.0.0.1" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x4009, 1200 + END +END + +#endif // English (India) resources +///////////////////////////////////////////////////////////////////////////// + + + +#ifndef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 3 resource. +// + + +///////////////////////////////////////////////////////////////////////////// +#endif // not APSTUDIO_INVOKED + diff --git a/cmd/zfs/zfs_main.c b/cmd/zfs/zfs_main.c index e28f1d04f350..b5f46e982b5e 100644 --- a/cmd/zfs/zfs_main.c +++ b/cmd/zfs/zfs_main.c @@ -1454,6 +1454,7 @@ destroy_callback(zfs_handle_t *zhp, void *data) if (zfs_get_type(zhp) == ZFS_TYPE_SNAPSHOT) { cb->cb_snap_count++; fnvlist_add_boolean(cb->cb_batchedsnaps, name); + zfs_snapshot_unmount(zhp, cb->cb_force ? MS_FORCE : 0); if (cb->cb_snap_count % 10 == 0 && cb->cb_defer_destroy) { error = destroy_batched(cb); if (error != 0) { @@ -4178,6 +4179,9 @@ zfs_do_rollback(int argc, char **argv) */ ret = zfs_rollback(zhp, snap, force); + if (ret == 0) + zfs_rollback_os(zhp); + out: zfs_close(snap); zfs_close(zhp); @@ -7254,9 +7258,16 @@ share_mount(int op, int argc, char **argv) } while (getmntent(mnttab, &entry) == 0) { + +#ifdef _WIN32 + /* No df/mount command on Windows, show snapshots too */ + if (strcmp(entry.mnt_fstype, MNTTYPE_ZFS) != 0) + continue; +#else if (strcmp(entry.mnt_fstype, MNTTYPE_ZFS) != 0 || strchr(entry.mnt_special, '@') != NULL) continue; +#endif (void) printf("%-30s %s\n", entry.mnt_special, entry.mnt_mountp); @@ -7273,12 +7284,19 @@ share_mount(int op, int argc, char **argv) } if ((zhp = zfs_open(g_zfs, argv[0], - ZFS_TYPE_FILESYSTEM)) == NULL) { + ZFS_TYPE_FILESYSTEM | ZFS_TYPE_SNAPSHOT)) == NULL) { ret = 1; } else { - ret = share_mount_one(zhp, op, flags, SA_NO_PROTOCOL, - B_TRUE, options); - zfs_commit_shares(NULL); + + if (zfs_get_type(zhp) & ZFS_TYPE_SNAPSHOT) { + ret = zfs_snapshot_mount(zhp, options, + flags); + } else { + ret = share_mount_one(zhp, op, flags, + SA_NO_PROTOCOL, B_TRUE, options); + zfs_commit_shares(NULL); + } + zfs_close(zhp); } } @@ -7653,9 +7671,15 @@ unshare_unmount(int op, int argc, char **argv) flags, B_FALSE)); if ((zhp = zfs_open(g_zfs, argv[0], - ZFS_TYPE_FILESYSTEM)) == NULL) + ZFS_TYPE_FILESYSTEM | ZFS_TYPE_SNAPSHOT)) == NULL) return (1); + if (zfs_get_type(zhp) & ZFS_TYPE_SNAPSHOT) { + ret = zfs_snapshot_unmount(zhp, flags); + zfs_close(zhp); + return (ret); + } + verify(zfs_prop_get(zhp, op == OP_SHARE ? ZFS_PROP_SHARENFS : ZFS_PROP_MOUNTPOINT, nfs_mnt_prop, sizeof (nfs_mnt_prop), NULL, diff --git a/cmd/zpool/CMakeLists.txt b/cmd/zpool/CMakeLists.txt new file mode 100644 index 000000000000..7aa1d0dd0ba7 --- /dev/null +++ b/cmd/zpool/CMakeLists.txt @@ -0,0 +1,28 @@ + +use_clang() + +include_directories(PRIVATE "${CMAKE_SOURCE_DIR}/cmd/zpool") + +um_add_executable(zpool + zpool_iter.c + zpool_main.c + zpool_util.c + zpool_vdev.c + zpool_util.h + os/windows/zpool_vdev_os.c + os/windows/resource.rc +) +target_link_libraries(zpool PRIVATE + libnvpair + libuutil + libuuid + libzfs + libzfs_core + libzpool +) + +install(TARGETS zpool RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}") +install(FILES $ + DESTINATION "${CMAKE_INSTALL_BINDIR}" + OPTIONAL +) diff --git a/cmd/zpool/os/windows/resource.h b/cmd/zpool/os/windows/resource.h new file mode 100644 index 000000000000..8a09cc650089 --- /dev/null +++ b/cmd/zpool/os/windows/resource.h @@ -0,0 +1,15 @@ +// {{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by resource.rc +// + +// Next default values for new objects +// +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1000 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif diff --git a/cmd/zpool/os/windows/resource.rc b/cmd/zpool/os/windows/resource.rc new file mode 100644 index 000000000000..645b0d5f1890 --- /dev/null +++ b/cmd/zpool/os/windows/resource.rc @@ -0,0 +1,109 @@ +// Microsoft Visual C++ generated resource script. +// +#include "resource.h" + +#define APSTUDIO_READONLY_SYMBOLS +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 2 resource. +// +#include "winres.h" + +///////////////////////////////////////////////////////////////////////////// +#undef APSTUDIO_READONLY_SYMBOLS + +///////////////////////////////////////////////////////////////////////////// +// English (United States) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US +#pragma code_page(1252) + +#ifdef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// TEXTINCLUDE +// + +1 TEXTINCLUDE +BEGIN + "resource.h\0" +END + +2 TEXTINCLUDE +BEGIN + "#include ""winres.h""\r\n" + "\0" +END + +3 TEXTINCLUDE +BEGIN + "\r\n" + "\0" +END + +#endif // APSTUDIO_INVOKED + +#endif // English (United States) resources +///////////////////////////////////////////////////////////////////////////// + + +///////////////////////////////////////////////////////////////////////////// +// English (India) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENN) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_INDIA +#pragma code_page(1252) + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// + +VS_VERSION_INFO VERSIONINFO + FILEVERSION 1,0,0,1 + PRODUCTVERSION 1,0,0,1 + FILEFLAGSMASK 0x3fL +#ifdef _DEBUG + FILEFLAGS 0x1L +#else + FILEFLAGS 0x0L +#endif + FILEOS 0x40004L + FILETYPE 0x1L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "400904b0" + BEGIN + VALUE "CompanyName", "TODO: " + VALUE "FileDescription", "zpool.exe" + VALUE "FileVersion", "1.0.0.1" + VALUE "LegalCopyright", "Copyright (C) 2021" + VALUE "OriginalFilename", "resource.rc" + VALUE "ProductName", "zpool.exe" + VALUE "ProductVersion", "1.0.0.1" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x4009, 1200 + END +END + +#endif // English (India) resources +///////////////////////////////////////////////////////////////////////////// + + + +#ifndef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 3 resource. +// + + +///////////////////////////////////////////////////////////////////////////// +#endif // not APSTUDIO_INVOKED + diff --git a/cmd/zpool/os/windows/zpool_vdev_os.c b/cmd/zpool/os/windows/zpool_vdev_os.c new file mode 100644 index 000000000000..64554c225dd4 --- /dev/null +++ b/cmd/zpool/os/windows/zpool_vdev_os.c @@ -0,0 +1,171 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "zpool_util.h" +#include + +#include +#include +#include +#include + +boolean_t +check_sector_size_database(char *path, int *sector_size) +{ + return (B_FALSE); +} + +static void +check_error(int err) +{ + /* + * ENXIO/ENODEV is a valid error message if the device doesn't live in + * /dev/dsk. Don't bother printing an error message in this case. + */ + if (err == ENXIO || err == ENODEV) + return; + + (void) fprintf(stderr, gettext("warning: device in use checking " + "failed: %s\n"), strerror(err)); +} + +static int +check_slice(const char *path, int force, boolean_t isspare) +{ + return (0); +} + +/* + * Validate that a disk including all partitions are safe to use. + * + * For EFI labeled disks this can done relatively easily with the libefi + * library. The partition numbers are extracted from the label and used + * to generate the expected /dev/ paths. Each partition can then be + * checked for conflicts. + * + * For non-EFI labeled disks (MBR/EBR/etc) the same process is possible + * but due to the lack of a readily available libraries this scanning is + * not implemented. Instead only the device path as given is checked. + */ +static int +check_disk(const char *path, int force, + boolean_t isspare, boolean_t iswholedisk) +{ + struct dk_gpt *vtoc; + char slice_path[MAXPATHLEN]; + int err = 0; + int fd, i; + int flags = O_RDONLY|O_DIRECT; + + if (!iswholedisk) + return (check_slice(path, force, isspare)); + + /* only spares can be shared, other devices require exclusive access */ + if (!isspare) + flags |= O_EXCL; + + if ((fd = open(path, flags)) < 0) { + return (-1); + } + + /* + * Expected to fail for non-EFI labeled disks. Just check the device + * as given and do not attempt to detect and scan partitions. + */ + err = efi_alloc_and_read(fd, &vtoc); + if (err) { + (void) close(fd); + return (check_slice(path, force, isspare)); + } + + /* + * The primary efi partition label is damaged however the secondary + * label at the end of the device is intact. Rather than use this + * label we should play it safe and treat this as a non efi device. + */ + if (vtoc->efi_flags & EFI_GPT_PRIMARY_CORRUPT) { + efi_free(vtoc); + (void) close(fd); + + if (force) { + /* Partitions will now be created using the backup */ + return (0); + } else { + vdev_error(gettext("%s contains a corrupt primary " + "EFI label.\n"), path); + return (-1); + } + } + + for (i = 0; i < vtoc->efi_nparts; i++) { + if (vtoc->efi_parts[i].p_tag == V_UNASSIGNED || + uuid_is_null((uchar_t *)&vtoc->efi_parts[i].p_guid)) + continue; + (void) snprintf(slice_path, sizeof (slice_path), + "%ss%d", path, i+1); + + err = check_slice(slice_path, force, isspare); + if (err) + break; + } + + efi_free(vtoc); + (void) close(fd); + + return (err); +} + +int +check_device(const char *path, boolean_t force, + boolean_t isspare, boolean_t iswholedisk) +{ + int error; + + error = check_disk(path, force, isspare, iswholedisk); + if (error != 0) + return (error); + + return (check_file(path, force, isspare)); +} + +int +check_file(const char *file, boolean_t force, boolean_t isspare) +{ + + return (check_file_generic(file, force, isspare)); +} + +void +after_zpool_upgrade(zpool_handle_t *zhp) +{ +} diff --git a/cmd/zpool/zpool_util.c b/cmd/zpool/zpool_util.c index e7ff739e5b49..a0d191c6049a 100644 --- a/cmd/zpool/zpool_util.c +++ b/cmd/zpool/zpool_util.c @@ -118,6 +118,7 @@ array64_max(uint64_t array[], unsigned int len) * Find highest one bit set. * Returns bit number + 1 of highest bit that is set, otherwise returns 0. */ +#ifndef _WIN32 int highbit64(uint64_t i) { @@ -139,3 +140,4 @@ lowbit64(uint64_t i) return (__builtin_ffsll(i)); } +#endif diff --git a/cmd/zpool/zpool_vdev.c b/cmd/zpool/zpool_vdev.c index 99a521aa2a28..11a469af983d 100644 --- a/cmd/zpool/zpool_vdev.c +++ b/cmd/zpool/zpool_vdev.c @@ -284,7 +284,11 @@ make_leaf_vdev(nvlist_t *props, const char *arg, boolean_t is_primary) * 'path'. We detect whether this is a device of file afterwards by * checking the st_mode of the file. */ +#ifdef _WIN32 + if (arg[0] == '/' || arg[0] == '\\') { +#else if (arg[0] == '/') { +#endif /* * Complete device or file path. Exact type is determined by * examining the file descriptor afterwards. Symbolic links @@ -920,7 +924,10 @@ zero_label(const char *path) err = write(fd, buf, size); (void) fdatasync(fd); (void) close(fd); - +#ifdef _WIN32 + /* Not allowed to write to boot sector */ + err = size; +#endif if (err == -1) { (void) fprintf(stderr, gettext("cannot zero first %d bytes " "of '%s': %s\n"), size, path, strerror(errno)); @@ -983,9 +990,12 @@ make_disks(zpool_handle_t *zhp, nvlist_t *nv) /* * Update device id string for mpath nodes (Linux only) */ - if (is_mpath_whole_disk(path)) + if (is_mpath_whole_disk(path)) { update_vdev_config_dev_strs(nv); - + /* update might change path, so fetch again */ + verify(!nvlist_lookup_string(nv, + ZPOOL_CONFIG_PATH, &path)); + } if (!is_spare(NULL, path)) (void) zero_label(path); return (0); @@ -1031,7 +1041,13 @@ make_disks(zpool_handle_t *zhp, nvlist_t *nv) * and then block until udev creates the new link. */ if (!is_exclusive && !is_spare(NULL, udevpath)) { - char *devnode = strrchr(devpath, '/') + 1; + char *devnode = strrchr(devpath, '/'); +#ifdef _WIN32 + char *backslash = strrchr(devpath, '\\'); + if (devnode == NULL || backslash > devnode) + devnode = backslash; +#endif + devnode = &devnode[1]; ret = strncmp(udevpath, UDISK_ROOT, strlen(UDISK_ROOT)); if (ret == 0) { @@ -1060,9 +1076,18 @@ make_disks(zpool_handle_t *zhp, nvlist_t *nv) return (ret); } +#ifndef _WIN32 ret = zero_label(udevpath); if (ret) return (ret); +#else + /* + * Append_partition will only work once label_disk has + * been called (To find offset+length). + * Call it again to get correct path. + */ + (void) zfs_append_partition(udevpath, MAXPATHLEN); +#endif } /* diff --git a/cmd/zstream/CMakeLists.txt b/cmd/zstream/CMakeLists.txt new file mode 100644 index 000000000000..11d7cb62e332 --- /dev/null +++ b/cmd/zstream/CMakeLists.txt @@ -0,0 +1,26 @@ + +use_clang() + +um_add_executable(zstreamdump + zstream.c + zstream_decompress.c + zstream_dump.c + zstream_recompress.c + zstream_redup.c + zstream_token.c + os/windows/resource.rc +) + +target_link_libraries(zstreamdump PRIVATE + libnvpair + libuutil + libuuid + libzfs + libzfs_core + libzpool +) +install(TARGETS zstreamdump RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}") +install(FILES $ + DESTINATION "${CMAKE_INSTALL_BINDIR}" + OPTIONAL +) diff --git a/cmd/zstream/os/windows/resource.h b/cmd/zstream/os/windows/resource.h new file mode 100644 index 000000000000..8a09cc650089 --- /dev/null +++ b/cmd/zstream/os/windows/resource.h @@ -0,0 +1,15 @@ +// {{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by resource.rc +// + +// Next default values for new objects +// +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1000 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif diff --git a/cmd/zstream/os/windows/resource.rc b/cmd/zstream/os/windows/resource.rc new file mode 100644 index 000000000000..41c5ec8bd181 --- /dev/null +++ b/cmd/zstream/os/windows/resource.rc @@ -0,0 +1,110 @@ +// Microsoft Visual C++ generated resource script. +// +#include +#include "resource.h" + +#define APSTUDIO_READONLY_SYMBOLS +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 2 resource. +// +#include "winres.h" + +///////////////////////////////////////////////////////////////////////////// +#undef APSTUDIO_READONLY_SYMBOLS + +///////////////////////////////////////////////////////////////////////////// +// English (United States) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US +#pragma code_page(1252) + +#ifdef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// TEXTINCLUDE +// + +1 TEXTINCLUDE +BEGIN + "resource.h\0" +END + +2 TEXTINCLUDE +BEGIN + "#include ""winres.h""\r\n" + "\0" +END + +3 TEXTINCLUDE +BEGIN + "\r\n" + "\0" +END + +#endif // APSTUDIO_INVOKED + +#endif // English (United States) resources +///////////////////////////////////////////////////////////////////////////// + + +///////////////////////////////////////////////////////////////////////////// +// English (India) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENN) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_INDIA +#pragma code_page(1252) + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// + +VS_VERSION_INFO VERSIONINFO + FILEVERSION 1,0,0,1 + PRODUCTVERSION 1,0,0,1 + FILEFLAGSMASK 0x3fL +#ifdef _DEBUG + FILEFLAGS 0x1L +#else + FILEFLAGS 0x0L +#endif + FILEOS 0x40004L + FILETYPE 0x1L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "400904b0" + BEGIN + VALUE "CompanyName", "TODO: " + VALUE "FileDescription", "zstreamdump.exe" + VALUE "FileVersion", "1.0.0.1" + VALUE "LegalCopyright", "Copyright (C) 2021" + VALUE "OriginalFilename", "resource.rc" + VALUE "ProductName", "zstreamdump.exe" + VALUE "ProductVersion", "1.0.0.1" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x4009, 1200 + END +END + +#endif // English (India) resources +///////////////////////////////////////////////////////////////////////////// + + + +#ifndef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 3 resource. +// + + +///////////////////////////////////////////////////////////////////////////// +#endif // not APSTUDIO_INVOKED + diff --git a/cmd/zstream/zstream_redup.c b/cmd/zstream/zstream_redup.c index c56a09cee75d..305018e194ae 100644 --- a/cmd/zstream/zstream_redup.c +++ b/cmd/zstream/zstream_redup.c @@ -56,6 +56,7 @@ typedef struct redup_table { int numhashbits; } redup_table_t; +#ifndef _WIN32 int highbit64(uint64_t i) { @@ -64,6 +65,7 @@ highbit64(uint64_t i) return (NBBY * sizeof (uint64_t) - __builtin_clzll(i)); } +#endif void * safe_calloc(size_t n) diff --git a/contrib/windows/Inno.Setup/HowToDebug.txt b/contrib/windows/Inno.Setup/HowToDebug.txt new file mode 100644 index 000000000000..0d3f9ead4b85 --- /dev/null +++ b/contrib/windows/Inno.Setup/HowToDebug.txt @@ -0,0 +1,140 @@ + +Install / Uninstall Open ZFS on Windows + +zfsinstaller install ZFSin.inf +zfsinstaller uninstall ZFSin.inf + + +* Run DbgView.exe as Administrator +* Untick Menu / Capture / Capture Win32 (it is noisy) +* Tick Menu / Capture / Capture Kernel (available if run as Administrator) +* Right click ZFSin.INF and click Install +* View kernel messages: + +SPL: start +SPL: Warning: High-overhead kmem debugging features enabled (kmem_flags = 0x6). Performance degradation and large memory overhead possible. See the Solaris Tunable Parameters Reference Manual. +SPL: Loaded module v1.6.0-7-gc1b4a00 (DEBUG mode), (ncpu 2, memsize 1073197056, pages 262011) +SPL: Notice: failed to create fm/misc kstat +ZFS: created kernel device node: FFFFB0080659D7E0: name DiskFileSystemDevice: 0x0 \ZFS created +ZFS: Loaded module v1.6.0-1, ZFS pool version 5000, ZFS filesystem version 5 +ZFS filesystem version: 5 + +Or something similar.. + + + +Mount messages should look something like: + +zfs_windows_mount: 'tank' '\??\?:' +zfs_vfs_uuid_gen UUIDgen: [tank](4) -> [413CEE64-9F5B-3F10-848F-B387FDEDFCC9] +zfs_windows_mount: new devstring '\Device\Volume{413cee64-9f5b-3f10-848f-b387fdedfcc9}' +New device FFFFE78C0641B780 has extension FFFFE78C0641B8D0 +zfs_windows_mount: new symlink '\DosDevices\Global\Volume{413cee64-9f5b-3f10-848f-b387fdedfcc9}' +zfs_windows_mount: new fsname '\Device\ZFS{413cee64-9f5b-3f10-848f-b387fdedfcc9}' +zfs_vfs_mount +zfs_vfs_mount cmdflags 0 rdonly 0 +zfs_vfs_mount: fspec 'tank' : mflag 0000 : optptr 0000000000000000 : optlen 0 : options (null) +zfs_domount +zfs_windows_mount: zfs_vfs_mount() returns 0 +zfs_windows_mount: driveletter 1 '\??\?:' +Verify Volume +dispatcher: enter: major 13: minor 1: IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_MOUNT_VOLUME): type 0x6 + fsDispatcher: enter: major 13: minor 1: IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_MOUNT_VOLUME) fsDeviceObject +IRP_MN_MOUNT_VOLUME fs +*** mount request for FFFFE78C0641B780 : minor +zfs_vnop_mount: mounting 'tank' +=> SendVolumeArrivalNotification +dispatcher: enter: major 0: minor 0: IRP_MJ_CREATE: type 0x6 + diskDispatcher: enter: major 0: minor 0: IRP_MJ_CREATE diskDeviceObject +IRP_MJ_CREATE: FileObject FFFFE78C06003D60 related 0000000000000000 name '(null)' flags 0x0 +dispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP: type 0x6 + diskDispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP diskDeviceObject +dispatcher: enter: major 0: minor 0: IRP_MJ_CREATE: type 0x6 + diskDispatcher: enter: major 0: minor 0: IRP_MJ_CREATE diskDeviceObject +IRP_MJ_CREATE: FileObject FFFFE78C05C01AF0 related 0000000000000000 name '(null)' flags 0x0 +dispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP: type 0x6 + diskDispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP diskDeviceObject +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_QUERY_DEVICE_NAME +replying with '\Device\Volume{413cee64-9f5b-3f10-848f-b387fdedfcc9}' +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_QUERY_UNIQUE_ID +ioctl_query_unique_id: +replying with 'tank' +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_QUERY_STABLE_GUID +dispatcher: exit: 0xc0000002 NotImplemented Information 0x0 : IRP_MJ_DEVICE_CONTROL +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME +ioctl_mountdev_query_suggested_link_name: +dispatcher: exit: 0xc0000225 <*****> Information 0x0 : IRP_MJ_DEVICE_CONTROL +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +dispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE: type 0x6 + diskDispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE diskDeviceObject +dispatcher: enter: major 0: minor 0: IRP_MJ_CREATE: type 0x6 + diskDispatcher: enter: major 0: minor 0: IRP_MJ_CREATE diskDeviceObject +IRP_MJ_CREATE: FileObject FFFFE78C05C01AF0 related 0000000000000000 name '(null)' flags 0x0 +dispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP: type 0x6 + diskDispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP diskDeviceObject +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_LINK_CREATED +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_LINK_CREATED v2 +dispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE: type 0x6 + diskDispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE diskDeviceObject +dispatcher: enter: major 0: minor 0: IRP_MJ_CREATE: type 0x6 + diskDispatcher: enter: major 0: minor 0: IRP_MJ_CREATE diskDeviceObject +IRP_MJ_CREATE: FileObject FFFFE78C05C01AF0 related 0000000000000000 name '(null)' flags 0x0 +dispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP: type 0x6 + diskDispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP diskDeviceObject +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_LINK_CREATED +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_LINK_CREATED v2 +dispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE: type 0x6 + diskDispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE diskDeviceObject +dispatcher: enter: major 0: minor 0: IRP_MJ_CREATE: type 0x6 + diskDispatcher: enter: major 0: minor 0: IRP_MJ_CREATE diskDeviceObject +IRP_MJ_CREATE: FileObject FFFFE78C05C01AF0 related 0000000000000000 name '(null)' flags 0x0 +dispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP: type 0x6 + diskDispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP diskDeviceObject +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_VOLUME_ONLINE +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_VOLUME_POST_ONLINE +dispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE: type 0x6 + diskDispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE diskDeviceObject +dispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE: type 0x6 + diskDispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE diskDeviceObject + IoCallDriver success + IoCallDriver success +<= SendVolumeArrivalNotification +IOCTL_MOUNTMGR_QUERY_POINTS return 0 - looking for '\Device\Volume{413cee64-9f5b-3f10-848f-b387fdedfcc9}' + point 0: '\Device\HarddiskVolume1' '\??\Volume{8af23efd-0000-0000-0000-100000000000}' + point 1: '\Device\HarddiskVolume2' '\DosDevices\C:' + point 2: '\Device\HarddiskVolume2' '\??\Volume{8af23efd-0000-0000-0000-602200000000}' + point 3: '\Device\Volume{413cee64-9f5b-3f10-848f-b387fdedfcc9}' '\??\Volume{7e0eead9-7a66-11e8-b4bd-02150b128b95}' + point 4: '\Device\Volume{413cee64-9f5b-3f10-848f-b387fdedfcc9}' '\DosDevices\E:' + point 5: '\Device\CdRom0' '\DosDevices\D:' + point 6: '\Device\CdRom0' '\??\Volume{6f696e57-0aa2-11e8-b4b3-806e6f6e6963}' +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_VOLUME_GET_GPT_ATTRIBUTES +dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL: type 0x6 + diskDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL diskDeviceObject +IOCTL_MOUNTDEV_QUERY_STABLE_GUID +dispatcher: exit: 0xc0000002 NotImplemented Information 0x0 : IRP_MJ_DEVICE_CONTROL +ZFS: ioctl out: 0 (0x0) +ioctl out result 0 + diff --git a/contrib/windows/Inno.Setup/OPENSOLARIS.LICENSE.txt b/contrib/windows/Inno.Setup/OPENSOLARIS.LICENSE.txt new file mode 100644 index 000000000000..da23621dc843 --- /dev/null +++ b/contrib/windows/Inno.Setup/OPENSOLARIS.LICENSE.txt @@ -0,0 +1,384 @@ +Unless otherwise noted, all files in this distribution are released +under the Common Development and Distribution License (CDDL). +Exceptions are noted within the associated source files. + +-------------------------------------------------------------------- + + +COMMON DEVELOPMENT AND DISTRIBUTION LICENSE Version 1.0 + +1. Definitions. + + 1.1. "Contributor" means each individual or entity that creates + or contributes to the creation of Modifications. + + 1.2. "Contributor Version" means the combination of the Original + Software, prior Modifications used by a Contributor (if any), + and the Modifications made by that particular Contributor. + + 1.3. "Covered Software" means (a) the Original Software, or (b) + Modifications, or (c) the combination of files containing + Original Software with files containing Modifications, in + each case including portions thereof. + + 1.4. "Executable" means the Covered Software in any form other + than Source Code. + + 1.5. "Initial Developer" means the individual or entity that first + makes Original Software available under this License. + + 1.6. "Larger Work" means a work which combines Covered Software or + portions thereof with code not governed by the terms of this + License. + + 1.7. "License" means this document. + + 1.8. "Licensable" means having the right to grant, to the maximum + extent possible, whether at the time of the initial grant or + subsequently acquired, any and all of the rights conveyed + herein. + + 1.9. "Modifications" means the Source Code and Executable form of + any of the following: + + A. Any file that results from an addition to, deletion from or + modification of the contents of a file containing Original + Software or previous Modifications; + + B. Any new file that contains any part of the Original + Software or previous Modifications; or + + C. Any new file that is contributed or otherwise made + available under the terms of this License. + + 1.10. "Original Software" means the Source Code and Executable + form of computer software code that is originally released + under this License. + + 1.11. "Patent Claims" means any patent claim(s), now owned or + hereafter acquired, including without limitation, method, + process, and apparatus claims, in any patent Licensable by + grantor. + + 1.12. "Source Code" means (a) the common form of computer software + code in which modifications are made and (b) associated + documentation included in or with such code. + + 1.13. "You" (or "Your") means an individual or a legal entity + exercising rights under, and complying with all of the terms + of, this License. For legal entities, "You" includes any + entity which controls, is controlled by, or is under common + control with You. For purposes of this definition, + "control" means (a) the power, direct or indirect, to cause + the direction or management of such entity, whether by + contract or otherwise, or (b) ownership of more than fifty + percent (50%) of the outstanding shares or beneficial + ownership of such entity. + +2. License Grants. + + 2.1. The Initial Developer Grant. + + Conditioned upon Your compliance with Section 3.1 below and + subject to third party intellectual property claims, the Initial + Developer hereby grants You a world-wide, royalty-free, + non-exclusive license: + + (a) under intellectual property rights (other than patent or + trademark) Licensable by Initial Developer, to use, + reproduce, modify, display, perform, sublicense and + distribute the Original Software (or portions thereof), + with or without Modifications, and/or as part of a Larger + Work; and + + (b) under Patent Claims infringed by the making, using or + selling of Original Software, to make, have made, use, + practice, sell, and offer for sale, and/or otherwise + dispose of the Original Software (or portions thereof). + + (c) The licenses granted in Sections 2.1(a) and (b) are + effective on the date Initial Developer first distributes + or otherwise makes the Original Software available to a + third party under the terms of this License. + + (d) Notwithstanding Section 2.1(b) above, no patent license is + granted: (1) for code that You delete from the Original + Software, or (2) for infringements caused by: (i) the + modification of the Original Software, or (ii) the + combination of the Original Software with other software + or devices. + + 2.2. Contributor Grant. + + Conditioned upon Your compliance with Section 3.1 below and + subject to third party intellectual property claims, each + Contributor hereby grants You a world-wide, royalty-free, + non-exclusive license: + + (a) under intellectual property rights (other than patent or + trademark) Licensable by Contributor to use, reproduce, + modify, display, perform, sublicense and distribute the + Modifications created by such Contributor (or portions + thereof), either on an unmodified basis, with other + Modifications, as Covered Software and/or as part of a + Larger Work; and + + (b) under Patent Claims infringed by the making, using, or + selling of Modifications made by that Contributor either + alone and/or in combination with its Contributor Version + (or portions of such combination), to make, use, sell, + offer for sale, have made, and/or otherwise dispose of: + (1) Modifications made by that Contributor (or portions + thereof); and (2) the combination of Modifications made by + that Contributor with its Contributor Version (or portions + of such combination). + + (c) The licenses granted in Sections 2.2(a) and 2.2(b) are + effective on the date Contributor first distributes or + otherwise makes the Modifications available to a third + party. + + (d) Notwithstanding Section 2.2(b) above, no patent license is + granted: (1) for any code that Contributor has deleted + from the Contributor Version; (2) for infringements caused + by: (i) third party modifications of Contributor Version, + or (ii) the combination of Modifications made by that + Contributor with other software (except as part of the + Contributor Version) or other devices; or (3) under Patent + Claims infringed by Covered Software in the absence of + Modifications made by that Contributor. + +3. Distribution Obligations. + + 3.1. Availability of Source Code. + + Any Covered Software that You distribute or otherwise make + available in Executable form must also be made available in Source + Code form and that Source Code form must be distributed only under + the terms of this License. You must include a copy of this + License with every copy of the Source Code form of the Covered + Software You distribute or otherwise make available. You must + inform recipients of any such Covered Software in Executable form + as to how they can obtain such Covered Software in Source Code + form in a reasonable manner on or through a medium customarily + used for software exchange. + + 3.2. Modifications. + + The Modifications that You create or to which You contribute are + governed by the terms of this License. You represent that You + believe Your Modifications are Your original creation(s) and/or + You have sufficient rights to grant the rights conveyed by this + License. + + 3.3. Required Notices. + + You must include a notice in each of Your Modifications that + identifies You as the Contributor of the Modification. You may + not remove or alter any copyright, patent or trademark notices + contained within the Covered Software, or any notices of licensing + or any descriptive text giving attribution to any Contributor or + the Initial Developer. + + 3.4. Application of Additional Terms. + + You may not offer or impose any terms on any Covered Software in + Source Code form that alters or restricts the applicable version + of this License or the recipients' rights hereunder. You may + choose to offer, and to charge a fee for, warranty, support, + indemnity or liability obligations to one or more recipients of + Covered Software. However, you may do so only on Your own behalf, + and not on behalf of the Initial Developer or any Contributor. + You must make it absolutely clear that any such warranty, support, + indemnity or liability obligation is offered by You alone, and You + hereby agree to indemnify the Initial Developer and every + Contributor for any liability incurred by the Initial Developer or + such Contributor as a result of warranty, support, indemnity or + liability terms You offer. + + 3.5. Distribution of Executable Versions. + + You may distribute the Executable form of the Covered Software + under the terms of this License or under the terms of a license of + Your choice, which may contain terms different from this License, + provided that You are in compliance with the terms of this License + and that the license for the Executable form does not attempt to + limit or alter the recipient's rights in the Source Code form from + the rights set forth in this License. If You distribute the + Covered Software in Executable form under a different license, You + must make it absolutely clear that any terms which differ from + this License are offered by You alone, not by the Initial + Developer or Contributor. You hereby agree to indemnify the + Initial Developer and every Contributor for any liability incurred + by the Initial Developer or such Contributor as a result of any + such terms You offer. + + 3.6. Larger Works. + + You may create a Larger Work by combining Covered Software with + other code not governed by the terms of this License and + distribute the Larger Work as a single product. In such a case, + You must make sure the requirements of this License are fulfilled + for the Covered Software. + +4. Versions of the License. + + 4.1. New Versions. + + Sun Microsystems, Inc. is the initial license steward and may + publish revised and/or new versions of this License from time to + time. Each version will be given a distinguishing version number. + Except as provided in Section 4.3, no one other than the license + steward has the right to modify this License. + + 4.2. Effect of New Versions. + + You may always continue to use, distribute or otherwise make the + Covered Software available under the terms of the version of the + License under which You originally received the Covered Software. + If the Initial Developer includes a notice in the Original + Software prohibiting it from being distributed or otherwise made + available under any subsequent version of the License, You must + distribute and make the Covered Software available under the terms + of the version of the License under which You originally received + the Covered Software. Otherwise, You may also choose to use, + distribute or otherwise make the Covered Software available under + the terms of any subsequent version of the License published by + the license steward. + + 4.3. Modified Versions. + + When You are an Initial Developer and You want to create a new + license for Your Original Software, You may create and use a + modified version of this License if You: (a) rename the license + and remove any references to the name of the license steward + (except to note that the license differs from this License); and + (b) otherwise make it clear that the license contains terms which + differ from this License. + +5. DISCLAIMER OF WARRANTY. + + COVERED SOFTWARE IS PROVIDED UNDER THIS LICENSE ON AN "AS IS" + BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, + INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT THE COVERED + SOFTWARE IS FREE OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR + PURPOSE OR NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND + PERFORMANCE OF THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY + COVERED SOFTWARE PROVE DEFECTIVE IN ANY RESPECT, YOU (NOT THE + INITIAL DEVELOPER OR ANY OTHER CONTRIBUTOR) ASSUME THE COST OF ANY + NECESSARY SERVICING, REPAIR OR CORRECTION. THIS DISCLAIMER OF + WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS LICENSE. NO USE OF + ANY COVERED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS + DISCLAIMER. + +6. TERMINATION. + + 6.1. This License and the rights granted hereunder will terminate + automatically if You fail to comply with terms herein and fail to + cure such breach within 30 days of becoming aware of the breach. + Provisions which, by their nature, must remain in effect beyond + the termination of this License shall survive. + + 6.2. If You assert a patent infringement claim (excluding + declaratory judgment actions) against Initial Developer or a + Contributor (the Initial Developer or Contributor against whom You + assert such claim is referred to as "Participant") alleging that + the Participant Software (meaning the Contributor Version where + the Participant is a Contributor or the Original Software where + the Participant is the Initial Developer) directly or indirectly + infringes any patent, then any and all rights granted directly or + indirectly to You by such Participant, the Initial Developer (if + the Initial Developer is not the Participant) and all Contributors + under Sections 2.1 and/or 2.2 of this License shall, upon 60 days + notice from Participant terminate prospectively and automatically + at the expiration of such 60 day notice period, unless if within + such 60 day period You withdraw Your claim with respect to the + Participant Software against such Participant either unilaterally + or pursuant to a written agreement with Participant. + + 6.3. In the event of termination under Sections 6.1 or 6.2 above, + all end user licenses that have been validly granted by You or any + distributor hereunder prior to termination (excluding licenses + granted to You by any distributor) shall survive termination. + +7. LIMITATION OF LIABILITY. + + UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER TORT + (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL YOU, THE + INITIAL DEVELOPER, ANY OTHER CONTRIBUTOR, OR ANY DISTRIBUTOR OF + COVERED SOFTWARE, OR ANY SUPPLIER OF ANY OF SUCH PARTIES, BE + LIABLE TO ANY PERSON FOR ANY INDIRECT, SPECIAL, INCIDENTAL, OR + CONSEQUENTIAL DAMAGES OF ANY CHARACTER INCLUDING, WITHOUT + LIMITATION, DAMAGES FOR LOST PROFITS, LOSS OF GOODWILL, WORK + STOPPAGE, COMPUTER FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER + COMMERCIAL DAMAGES OR LOSSES, EVEN IF SUCH PARTY SHALL HAVE BEEN + INFORMED OF THE POSSIBILITY OF SUCH DAMAGES. THIS LIMITATION OF + LIABILITY SHALL NOT APPLY TO LIABILITY FOR DEATH OR PERSONAL + INJURY RESULTING FROM SUCH PARTY'S NEGLIGENCE TO THE EXTENT + APPLICABLE LAW PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO + NOT ALLOW THE EXCLUSION OR LIMITATION OF INCIDENTAL OR + CONSEQUENTIAL DAMAGES, SO THIS EXCLUSION AND LIMITATION MAY NOT + APPLY TO YOU. + +8. U.S. GOVERNMENT END USERS. + + The Covered Software is a "commercial item," as that term is + defined in 48 C.F.R. 2.101 (Oct. 1995), consisting of "commercial + computer software" (as that term is defined at 48 + C.F.R. 252.227-7014(a)(1)) and "commercial computer software + documentation" as such terms are used in 48 C.F.R. 12.212 + (Sept. 1995). Consistent with 48 C.F.R. 12.212 and 48 + C.F.R. 227.7202-1 through 227.7202-4 (June 1995), all + U.S. Government End Users acquire Covered Software with only those + rights set forth herein. This U.S. Government Rights clause is in + lieu of, and supersedes, any other FAR, DFAR, or other clause or + provision that addresses Government rights in computer software + under this License. + +9. MISCELLANEOUS. + + This License represents the complete agreement concerning subject + matter hereof. If any provision of this License is held to be + unenforceable, such provision shall be reformed only to the extent + necessary to make it enforceable. This License shall be governed + by the law of the jurisdiction specified in a notice contained + within the Original Software (except to the extent applicable law, + if any, provides otherwise), excluding such jurisdiction's + conflict-of-law provisions. Any litigation relating to this + License shall be subject to the jurisdiction of the courts located + in the jurisdiction and venue specified in a notice contained + within the Original Software, with the losing party responsible + for costs, including, without limitation, court costs and + reasonable attorneys' fees and expenses. The application of the + United Nations Convention on Contracts for the International Sale + of Goods is expressly excluded. Any law or regulation which + provides that the language of a contract shall be construed + against the drafter shall not apply to this License. You agree + that You alone are responsible for compliance with the United + States export administration regulations (and the export control + laws and regulation of any other countries) when You use, + distribute or otherwise make available any Covered Software. + +10. RESPONSIBILITY FOR CLAIMS. + + As between Initial Developer and the Contributors, each party is + responsible for claims and damages arising, directly or + indirectly, out of its utilization of rights under this License + and You agree to work with Initial Developer and Contributors to + distribute such responsibility on an equitable basis. Nothing + herein is intended or shall be deemed to constitute any admission + of liability. + +-------------------------------------------------------------------- + +NOTICE PURSUANT TO SECTION 9 OF THE COMMON DEVELOPMENT AND +DISTRIBUTION LICENSE (CDDL) + +For Covered Software in this distribution, this License shall +be governed by the laws of the State of California (excluding +conflict-of-law provisions). + +Any litigation relating to this License shall be subject to the +jurisdiction of the Federal Courts of the Northern District of +California and the state courts of the State of California, with +venue lying in Santa Clara County, California. diff --git a/contrib/windows/Inno.Setup/ZFSInstall-debug.iss b/contrib/windows/Inno.Setup/ZFSInstall-debug.iss new file mode 100644 index 000000000000..5a0ba762340d --- /dev/null +++ b/contrib/windows/Inno.Setup/ZFSInstall-debug.iss @@ -0,0 +1,121 @@ +; Script generated by the Inno Setup Script Wizard. +; SEE THE DOCUMENTATION FOR DETAILS ON CREATING INNO SETUP SCRIPT FILES! +#define Root SourcePath+"..\..\..\" + +#define MyAppName "OpenZFS On Windows" +#define MyAppPublisher "OpenZFS" +#define MyAppURL "http://www.openzfsonwindows.org/" + +; Attempt to read in the gitrev version from the header file. +#define VerFile FileOpen(Root + "include\zfs_gitrev.h") +#define VerStr FileRead(VerFile) ; "// This file is only used on Windows and CMake, other platforms" +#define VerStr FileRead(VerFile) ; "// see scripts/make-gitrev.sh" +#define VerStr FileRead(VerFile) ; "#define ZFS_META_GITREV "zfs-2.0.0-7-g689dbcfbf-dirty"" +; #pragma message VerStr + " readline" +#expr FileClose(VerFile) +#undef VerFile +; Parse out 2.0.0-7-g689dbcfbf-dirty and delete the double-quote +#define MyAppVersion StringChange(Copy(VerStr, 30), '"','') + +#pragma message MyAppVersion + " is version" + +#if MyAppVersion == "" +#error Failed to read version from include/zfs_gitrev.h +#endif + +#include "environment.iss" + +[Setup] +; NOTE: The value of AppId uniquely identifies this application. +; Do not use the same AppId value in installers for other applications. +; (To generate a new GUID, click Tools | Generate GUID inside the IDE.) +AppId={{61A38ED1-4D3F-4869-A8D5-87A58A301D56} +AppName={#MyAppName}-debug +AppVersion={#MyAppVersion} +AppVerName={#MyAppName}-{#MyAppVersion} +AppPublisher={#MyAppPublisher} +AppPublisherURL={#MyAppURL} +AppSupportURL={#MyAppURL} +AppUpdatesURL={#MyAppURL} +DefaultDirName={autopf64}\{#MyAppName} +DefaultGroupName={#MyAppName} +AllowNoIcons=yes +LicenseFile={#SourcePath}\OPENSOLARIS.LICENSE.txt +InfoBeforeFile={#SourcePath}\readme.txt +OutputBaseFilename=OpenZFSOnWindows-debug-{#MyAppVersion} +SetupIconFile={#SourcePath}\openzfs.ico +Compression=lzma +SolidCompression=yes +PrivilegesRequired=admin +OutputDir={#SourcePath}\.. +ChangesEnvironment=true +WizardSmallImageFile="{#SourcePath}\openzfs-small.bmp" +WizardImageFile="{#SourcePath}\openzfs-large.bmp" +; Tools/Configure Sign Tools -> Add -> +; "signtoola" = C:\Program Files (x86)\Windows Kits\10\bin\x64\signtool.exe sign /sha1 09A7835D2496C08389CF1CC1031EF73BAE897A08 /n $qWest Coast Enterprises Limited$q /t http://ts.digicert.com /as /fd sha256 /td sha256 /d $qOpenZFS on Windows$q $f +; "signtoolb" = "C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\signtool.exe sign /sha1 ab8e4f6b94cecfa4638847122b511e507e147c50 /as /n $qJoergen Lundman$q /tr http://timestamp.digicert.com /td sha256 /fd sha256 /d $qOpenZFS on Windows$q $f" +; "signtoolc" = "C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x64\signtool.exe sign /v /fd sha256 /n $qOpenZFS Test Signing Certificate$q /t http://timestamp.digicert.com $f" +; SignTool=signtoola +; SignTool=signtoolb +SignTool=signtoolc + + +[Languages] +Name: "english"; MessagesFile: "compiler:Default.isl" + +[Code] +procedure CurStepChanged(CurStep: TSetupStep); +begin + if (CurStep = ssPostInstall) and WizardIsTaskSelected('envPath') + then EnvAddPath(ExpandConstant('{app}')); +end; + +procedure CurUninstallStepChanged(CurUninstallStep: TUninstallStep); +begin + if CurUninstallStep = usPostUninstall + then EnvRemovePath(ExpandConstant('{app}')); +end; + +[Tasks] +Name: envPath; Description: "Add OpenZFS to PATH variable" + +[Files] +Source: "{#Root}\README.md"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\CODE_OF_CONDUCT.md"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\os\windows/kstat/kstat.exe"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\os\windows/zfsinstaller/zfsinstaller.exe"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\zpool\zpool.exe"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\zfs\zfs.exe"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\zdb\zdb.exe"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\zstream\zstreamdump.exe"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\raidz_test\raidz_test.exe"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\module\os\windows\driver\OpenZFS.sys"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\module\os\windows\driver\OpenZFS.cat"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\module\os\windows\driver\OpenZFS.inf"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\module\os\windows\driver\OpenZFS.man"; DestDir: "{app}"; Flags: ignoreversion +;Source: "{#Root}\x64\Debug\ZFSin.cer"; DestDir: "{app}"; Flags: ignoreversion +;Source: "{#Root}\zfs\cmd\arcstat\arcstat.pl"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\module\os\windows\driver\*.pdb"; DestDir: "{app}\symbols"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\zstream\*.pdb"; DestDir: "{app}\symbols"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\zpool\*.pdb"; DestDir: "{app}\symbols"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\zfs\*.pdb"; DestDir: "{app}\symbols"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\zdb\*.pdb"; DestDir: "{app}\symbols"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\raidz_test\*.pdb"; DestDir: "{app}\symbols"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\os\windows\zfsinstaller\*.pdb"; DestDir: "{app}\symbols"; Flags: ignoreversion +Source: "{#Root}\out\build\x64-Debug\cmd\os\windows\kstat\*.pdb"; DestDir: "{app}\symbols"; Flags: ignoreversion +Source: "{#SourcePath}\HowToDebug.txt"; DestDir: "{app}"; Flags: ignoreversion +Source: "{#Root}\contrib\windows\parsedump\*.*"; DestDir: "{app}"; Flags: ignoreversion + + +; NOTE: Don't use "Flags: ignoreversion" on any shared system files + +[Icons] +Name: "{group}\{cm:ProgramOnTheWeb,{#MyAppName}}"; Filename: "{#MyAppURL}" +Name: "{group}\{cm:UninstallProgram,{#MyAppName}}"; Filename: "{uninstallexe}" + +[Run] +Filename: "{app}\ZFSInstaller.exe"; Parameters: "install .\OpenZFS.inf"; StatusMsg: "Installing Driver..."; Flags: runascurrentuser; + +[UninstallRun] +Filename: "{app}\ZFSInstaller.exe"; Parameters: "uninstall .\OpenZFS.inf"; RunOnceId: "driver"; Flags: runascurrentuser; + diff --git a/contrib/windows/Inno.Setup/environment.iss b/contrib/windows/Inno.Setup/environment.iss new file mode 100644 index 000000000000..b9faa5154f4b --- /dev/null +++ b/contrib/windows/Inno.Setup/environment.iss @@ -0,0 +1,45 @@ +; environment.iss - stackoverflow.com Wojciech Mleczek +[Code] +const EnvironmentKey = 'SYSTEM\CurrentControlSet\Control\Session Manager\Environment'; + +procedure EnvAddPath(Path: string); +var + Paths: string; +begin + { Retrieve current path (use empty string if entry not exists) } + if not RegQueryStringValue(HKEY_LOCAL_MACHINE, EnvironmentKey, 'Path', Paths) + then Paths := ''; + + { Skip if string already found in path } + if Pos(';' + Uppercase(Path) + ';', ';' + Uppercase(Paths) + ';') > 0 then exit; + + { App string to the end of the path variable } + Paths := Paths + ';'+ Path +';' + + { Overwrite (or create if missing) path environment variable } + if RegWriteStringValue(HKEY_LOCAL_MACHINE, EnvironmentKey, 'Path', Paths) + then Log(Format('The [%s] added to PATH: [%s]', [Path, Paths])) + else Log(Format('Error while adding the [%s] to PATH: [%s]', [Path, Paths])); +end; + +procedure EnvRemovePath(Path: string); +var + Paths: string; + P: Integer; +begin + { Skip if registry entry not exists } + if not RegQueryStringValue(HKEY_LOCAL_MACHINE, EnvironmentKey, 'Path', Paths) then + exit; + + { Skip if string not found in path } + P := Pos(';' + Uppercase(Path) + ';', ';' + Uppercase(Paths) + ';'); + if P = 0 then exit; + + { Update path variable } + Delete(Paths, P - 1, Length(Path) + 1); + + { Overwrite path environment variable } + if RegWriteStringValue(HKEY_LOCAL_MACHINE, EnvironmentKey, 'Path', Paths) + then Log(Format('The [%s] removed from PATH: [%s]', [Path, Paths])) + else Log(Format('Error while removing the [%s] from PATH: [%s]', [Path, Paths])); +end; diff --git a/contrib/windows/Inno.Setup/openzfs-large.bmp b/contrib/windows/Inno.Setup/openzfs-large.bmp new file mode 100644 index 0000000000000000000000000000000000000000..b6e7786df0b0defb7d800e061d2195e756ad976b GIT binary patch literal 154542 zcmeFacXVCVnXsSlo3_@>n%^I@l9l9RLV#d!ldRq?ODlW*YkSHC_k*U2|<;>E9y z%XRV%9DnY`ak);uf&QnT8<*?k8|ZoR+_+pP-$3W-v*U7|d;`aZPL9iU@(ncg501-q z@(t8=^^VJR@(onAb&kt*@(q+VwT#Pk@(moWtsj@`Hax%XRV%EPiCgxLhaS!2L^>j>~oO4a~j!?oXcF zd+*#)gimzuy?0-2nSWXS;=Q(i_uY5z<&(#kF1?_vT(<1O^6D>trM z<&;gUR=H*M>P_Bc^Xk=xtXZ?!n`~LLMr7^UE$h~Km96X6A^b=;jN|(CTM^#W4cBqo z`t@$vuwk30Y~Qe9`^JsNapT4nOP1K=u_a5~vUKTVBZ&JBS+?x)5yX9ms5^n1!6u&f z#&WqC9J2b6Ra6^7Vu3D?W@k3EKL;OOlR+3@(|Zt-@9 z$i1P(+a1C^W4}Yztl6}7?Iw;cG4BfAG27BuyLJ@ehS5JA>CUm;8#d6qylS95?SOa6 zR}Hk_zM`Np0xq)FhDlbxsb5-8u(ii>16a+4)ht+9UmRtO|D~$)1u3K5hAyMg-gKEa zFPA!Bn6fur#s!S7&K|lvDrG}+EgB79;~sFSzxi?`H5b)(_ysFyjKABzn2zRZHp~y@ zXc&G?d$)@`^3Cw;#`iN)V~ihjgkS$Cm*@3&1;naV`l!vm1PtmnbKg|v4fgos8!sfA zR<3jlJ+E4|ZQZ(0ffzTiEuOKyT)TGbH7IM=Y<1nnZabh5YrYaE;9MSEp(6@uj8Rnh z!TrFmbKHciTxoQof|E~2+YHFvU(tm=o&J?Eetb?IY`Amw zZ0kAJw(ftyJ@+K--i2^}{I;zZ?EjuS?>w+`N0RD)=h*sB+`S7)9((^gw#)Uz?H*JA zci%DBMa3fP_tiM9%%O8C3 zkwuI6LysX3J+yMk64Za~Bg?q}ESR>{TUM{$v2Gn7WcS96dp2#_yJZVs^8W4HxaPF@ zy${WwFS1}hS2%CZoV(`EEl5d53e!^4_wSoAW5)e=-zE3Se8f0YHROI?Fn8|E*x2lZ zgZZ4FngaC5$VhG&?g8Wc1?pGSFLM4}bLPyOJ2yW$DK9yxAT@=*?(ScoY5wATxt{r| z|Ez?Ad0bLz3ecn6{p+4lF1dLAoY{YFauSm7?tdOX(x1_FGQT3{8~qBx z`d`HLn;%wxWY6nAl$59?S2c1H6S58-;Jf1P;tKa{-psd+a|D5yXvPuBe(EFU6uWA) z)%zOOhDTTMD_d8u-oAD%w-8zbEQM)1-zowuv=2$!7ti0yKo0EZo5@a0-~!y2#CMyL z3=TeiL0W2IS{hQ6o?e_Ga{pcPv@E>)u9-7tz&bA}5#jt)#bWO4+5E_T+FkP&s$b#F zJ7&$CJ#%K}f&F}H2p-t6Jt8vlo;&VXc$fQp_b<|ajP#$&wIwF-^``FK18ZCV*7NC~ ze|#uD3+M0OPX7_Af8&}tnTv5<&pdtobk4^IH{Z;d`rp4#t|uu`^{=k~Zg>CsPpG5u z{p9csJ4C*4^_}xonD>Cb66?v_6^MREH#kRrh5J+eA|DeGcB(_UkrECHQze-P4u=a(eZlKTNB9+!_2~WIHE*6*)>fcnYC~K>r#F3{ z=x+5<-xB)B$45{o#N7ru?Itc~l}Jv&txgH>2_+qj&#jdie}pU5=d@Q`4q zUhD5FG%$b@j5A&5hXw`l%{W=}U4{7j2TxPo>XSYDcd{P6e;aF~f3s)T^@Il-*Y9L~ zf&Lw=N4$RJ<1Pzpt%)4zP3$dnltsHmvmfB@gg6Md&l z@tq=PB0du*hQK;1$_wkLD6Ypxou*IbPxwr9_itnE?w?=bPw4(9x%;=X_S*lXiRSg& zSnK}P^{CF(N8qJ-{Z<<;dTs5Wla=^b`oFxa1-DW|8fIIwh7sW|F))a^{a_sv5iut` z{EncYIiVra2o0eFT9Nd#8OhBgGep^I1Cfx3? zHS3Vz;EA{1s?Ri!p`oE(^*TB_ntQ``{*(z5%>JFMz4U+Ut!Do=))(yG$@&8Qd+N2< z{`u7A^xMOA7btI%mV2#Sy*zsH8;o&2)4how1 zm0Kopd$v7uKWp?__;}{++BX{qw=~XE(3^DzL`Sk`=*36UNMm zj=p2YjCnI-=SD~StY3d~ckk36{>1;czYqB1`vLF1@Au|!Cw=qVw-z6sJ%9fE88hyT zjg5_nz^ao+MUfdeLma&Z^tH`8Oy5hNX4Wo$ZPjZunmi%8jD`Xld<;MDnHl@#UAu07 z{w4pn-wpm?*eXGP|8UytZ{E_>a_@o#_up{`39C6XW`LX76S624)@j$SUWZ*@thF)6 zD52COQL)CTGkHnaJuqvg-=mLDc=BApJMU@Ap&txKd@wxkgW>5e?tX9lW_#b%l{*)s-EGb6?M&Yd;$o;h@^`;Kd*`k-ix)3_VBvjt&BHQ~j|K5KYj-?w z4ftz`F_b+gK}tp)>n)~x&Yb(^&Ru%fynshm-1f}#L4WxG&`Zt_uQ)%v>iqB;b)>!Y z{P1jr+yD3PPN}Tkv3fPdR1Ys&#IMhrHIwldsx5k*o<6SEwlSB);U80M25#Idvs5z( zlIP4>aOa&5-gD2x_uO;So;_23@FSEOtlQ2H?>Rqw@ceM1I_^C`y!HI>%Jaim^`O6= z|H{zG9dU6a_BX6rwRrw~TmWWV8}M-(PJZCyWbGN7Y1X?oZj6bHoPWn1*kPvUmM&Ou zL(U=JSAG@p!EoI9;f(Xch3ALM&JS0pBW>Q10qjvbFY^bmT@ z1kgC^Ys!Y}I2_glW=#Lhn?3tprVH=BYZ;T;4?g&xW#u^Q@DGM_&JS0fA3pZsaQ}zH zr`3@*??C*17weSb;)e3Fw1fl-w&{ck&~et+6l8qz;Sore!J4E>QAWmi9R4xJX6Ut>wI_7ON>q_8y5}BPuUx!% z!z0V*4h%Bv0{8y&!(aPQi2vY2iO{0dJuv;<_m4jR!pXL_oRk!5N=d2^fn&FIf^?j< zk%RsC#@ALG?uKr$=TxAfo+P)|E?c_kp~cHHGNWIRG$}E;7SIR1^QYzC`Ce&rOIc<{ z+?FlOZsYFBvTEzC4R~=zMsa55xL$wkdTj(|TCbB5O_MJ3l1Yga7LxvA`4wuu<>6&p zmo8bkdCQ{ojD>|oi;f(5Jil8CEY_#`+-2O|{Wh zKAH+rT|_EMJ+9Xub1nsc?a3NN7WA$+jeCF+Xl5sqlXQ9Ep_CMeAIZ%v%F0UJzmMDC z;I5tA4{GAlt6 zn&F5D3Ol4$!E-)Z*DBDzy=GYV9~?Z%^L&}VcJ=R``=Dlc4E<}9NC+x5!x!v7Hr8sx zo&B5h*YfSC>&LO`a)OU|6lZN=4OT#N*@(1t1(9y6myE>_)K-3^?gx&%gSpmNb4T--l1wOz~A3#!}$@u zVBQ^fSXsOKcd$15H(Br7zI`nHi?dTR{@Cy__3vVh<8$_Jvewtn0<^1r z5n_p1_Ecm{z3B4TKp)A;F>N>?f`WtjELPSOS)uIcsp+Q8MsP5EFWI*}ut}AMugr&Y2?v z;mhQ*-FkgN))^U2)&*&48nlN}d$ZnvwzB4f1Ox@$IcJW^IwgKD0SQC{ys-`n;?@Wz z__46wrf&d$Z>-J!t*njy{r$ymc#7G-oAtIa^>4CH-n(Z^{cG0AN$9on`c2m6^s|MicL1v*=%qF@GCe( zD{BqN-`Cfv*Sh~ZX3w^=cK7cI8t!KQR@P(b-^zMS{kvIjb@p$vHn0CGvev6CRU~Gz zRwHFhH>LjCfM38(yFiz?Z1|+xZsWW4^Y>?IBvBK%iT$v#xLAmyrzT>v!41H#;1DzS z?~~XJ8@p{AZ}FY@`S{pv=7b4m|5n!S{++B{{Y#vz1{d_NZ-7zyXS{4>J=XqJthjI6 zR%ibvYkmFne>GXFWJnrH?bd4-YcHPKX}v9=t*rh1{eAuX_$*e|xG3=X`TClO%?39B z_XZBp%9?lhPJF$wHu{f^m6%g~w=B1J_wQhB_HVL|-?nWm{c~f0n?WcGpvKhyzIe{( zQ!|t3>|e8Qtv*oBdl^OaGkDz2RVO_HSi9mj11*$JD=twX=VdwQ>D1F_OK!60w;V zf9*X2H?6lbB=Z<#DWx_+yY+g)1iss;K0Y&JVyvt&$RHIKo=G!a%GhTX{#m z6R)gY{aaZRAshWWSo25e4To!CJ*NJxtjE;9g|+kgP1ane{(e4o)=YPfWWe35J*~Hg zUc=7EXUdc*tW3AE#vt>R3p)pEdE3V)JS4fdC&b&IoqgY~Vqn%94&SX=UADgyTwgqzOV7Je&F?FLOo)S=WRhYpz# zr%s(ZapJ_7s3yowO%zO#NF}JNq|T>+5GK=!&wo46e1amLS|rm!-z<5zvS=BWhD>r}b8#t*jN# z349hSYYZtsf$C(7`Oc4tvC5AHWX+gM-FdIP%DV#8s2>#eu4 zr-_^Oo;VCLK!NIH&G{1;HT(N(*6bOD4V)ln%T3^9ZT4?v?e5>fnoe~8R@P(b-^zMS z{hO>eZ+7-?vNo?@Si6=>f5d4`$&rqkwPC%LQoC7uQ);L62DFtmly14@mZ

D{BlF zU$|G+oX@?%DA>ySR?dbt2H(L>_s>*{n|0i7qyG`k*ZphO-1B?3Y#B@cZq{S$U$JJM zj)hdt{!P~A^Zt*kHDe?&-#mGzkVx3IpMPwRR8CTo5D zj61F>YvrsN=@{1gqLkX@rk$+$lsDaYBjZ^sYq^<-#MS&79#Dr2eN&NjF(<+H~WWzGS<8H|v{k z{*vnd>gYAI9{eF=z-3Wt?@^hz%!!V@O#^Oa%{TkS8*X4HC@X6;8fC{=J6WRue0Cda z-T?=W-082;Qu=4SY_g_a-RR%j`MQ72nqMI{i`%?;^!~q~uHVX<{%_#=N9lj!L|8Ky zLu|%QanAnTtkw1N{ajww1YK?$ZkfF1BO6tqo(9~?+G@Q?l)1xiZ%S=t&Hem^FMcs3 zFu-KZ_!Q+uS-|9l^-VY3aKjC-ud#T`}bt+?B8U~_w)HLev#|H(yU34G8S`}c6+imoVC?}+mw0)&}A+g&L|0e{QMWb z5bW=7WsUPi8Ts11u*MF2@rz_U+^jck;vK#&?u{v)?D)nokGg*=Yoq@gZ*}zsXu(KmA``)*j<)rPrDJWhaCSvL35a=jNi(h?O<^__@!2 zp3kCL&)4o`{iPdj;ND;;ov|-o$7c9DPUh?4-f*%u`**Y6y-N(^7rr30+R*eL5q@%FDXanxjXh0)h z{M_e|FMRHE|MD;Y@(=&`kAziO`}ZL^2li8w^Y8!u@Bj2q|MYMF`mg#l>+xUz^mlv!6Bl&)gS}WV1MB%Vy5x z7wG>B|M4F-`B$!o{y+QK#GO01ow8K_{6)?mssI10uAh4YA^)&<^X7m1w}0bTv;X+LeB>=FSN`Ka{-f^yi>~YW{D1rh^11)`kAMD`fBC=v@DGHkX>88FcTeh` zxXUH!@p}(!-^vB^8ME0{SSL<(%tor*wIYCRCrOivxEAH9rFl}5W=MKE%k=az0r%<; zZ5-4wD=keVH8m|MDLEnGP-bR#OAFG|($d~gpPrnYl9>}+i0 zZ| zYrp*E>ok7xi(mZgXIg&wv!A{Gt6%-vo4oPrt8cva>YGYledQG`ul?c|Z@%`LcX|Ew zw~$}|nxE=FCx7u*zv7+Wy!qyD-gv`9-hA`7N`9w~{Gm5qdkuN>_1Ay*)>|TPzUd)v zz4i86Z~gwQxBl?k-_oE@E%f>O-~N_25gLa4?stDs^3LymhtSr4iz7Y1^Y+_r`Tg(T zb;%$9@P|LW^A18o|GM?4yRsiuS7s0H#vlIhu1kLY(;w^hA63tEZU`^+pHmI_DKFm@ zdGFnK$0+ar@sBq7+h6~BiSjq^|Nj2BzaeA!r_UVI@$Z-I-v@vH`z6Tvk^g;o{ybv) zr{5gq`0LkS{qy_p$qD>ZJ$M}DUC;ORJ0I;hJpS*he*@eYSN(5}|H13)8+iTIS1(Fl zd-c^zlUK*`?^nNi^=gU=RLuBcq{gp)`AfU};upX26n@wCj`QU!KmYkJJ%7u2&M#ki z1+l*Wi&tJj_(lCAef-%^e)7{F|M+61v$b^;>1b)ONjvemr?fRUdy%H5Hk&jywt332 z#zvd8HZ(Y-zO}(FE%kL?rMbS&t2EWsdY9VTCWkcE)Yznj%dul!7O~x69j&cavE5GX&COP^-$k~Y%JI^6N4Z}%G#nd2oHv+$r=`B$t9ZS~ z314yYi-g{Q)3hRoyNWA?M|z!sO^0&Lv(~2G zN!)zLD|+6v)AvMg<8Fg)Cw?VaI|2qm2xq`xx8&ojt=z9DYa=*xcI(*ssR|OVf1i(?_!Hyg#BVcldEzSk6mZSk6l?k+~x{@9XRwMMk<4yx#!& z#u(u?uyUt;8PS{G{Icl#IBOg9an`ng%^8_rX1(^z@QmyAr&g~$L6g=SXMHWPHnO;v z$6tTCSresuM&{O+)+pU{)-*m=))$wezqr_JoVCiceI(YNb1`~q=2By`an|mslZ%VP zN7rl1?AtZY+BP3Iib*$r#+oE%YZA)*DPO*JxY~Nt9?^`ClsEHcz_IFHZTL8At9Cnc zq_)Z#{n7^7>94=`U;p*h7hgnZh`jXDOV2<5{L@dr@XRyMojdo;Q%`9*bMoZL!NI}4 zzJZ<|1kl_Dh90?Uka#Jhp_UeI744d>Gb3J|^*C#%F}G6I?aRe_jFps>xl%^{m%sew z&+6YFe*gQo-hA`dufP8KYp?zIhd=zEZ-4tcFTea>FTC*dlTSW%`tzW&ZmKuZ;Y zD=wlfw}CPmR_>nyy;kW^?_(~T<93pDzVorO9^);cPy6d%|N8!W@3BIH3RUVJzx(Z% zzxlPVJ$?4<6DLphcXa_;rj}F;tZx8Jgp;`GKD;!p34iTT202nb-0}W}Z;4B!PnFt* z24WWl@gtths)hI8fB&67{DHz?tiiXw{DbO8E?S(6@6RsE5UT+n=!+U@F6CUB`KmDnue`26t>povE7JwMQ zrubIY)_f^9j45M^$;VlHMRFD2WbKyd{wuG1@8y@DfBM`>(6fkAMSu{rf~NiHV{LWu zR(o#s<6fe*Ys`jU-2L+5hr@sW``_Mw@7=e5_u4D3eE&P&`RWTVXpawH@%RLL-bznR z)uv(9Zrb+A5m(%3l%{y9xic}&J0IDGk7V1e@7wLXmAGz?#@*tzZKJS@`~I)m*(&b; z+xzeT`Q3N_^oKvl;@RK)hIP2C1%BlxKmH$L{^y=Mt0Fz&u2kO6{bKF;sN8YlHuNJw zAN_ro*S3;%wl9vg$=!>xTeuUsT{U_a_lFGD@4f%-yMKK9ZPr%*_SdZ4epOb8v)t+D zKmWx~e)6*){_qFi{q8rv`qk&2dg>HI|ISWRb7_pE-5#oK<7<8CspD6dwUxe`yIZ@> z3UklKCz!PwxxN4HJMX^zHY?~gYnJl^8r25$kALukAARq8|NGtVV&1>|t#5tnt6zQL z>8H=0Iz_67h+Q+)V>*-jM=|M_7N3u<=3am5P50fzwKun}8#47;H#~@cBJ}!$56=JP zFMoaSz4zXI2Lt{#doL)xHdy~yv;O`M|L1?c|DA7R(!c%nufP1YuYK#k|N9&N^&z(L!)Yqr;Ik;mDeVp~hS=Xy`w#oV(VU4H$&Fin@tO?S7^|PP7!gAFA{ofz| z;QOM~-+mdThBctSPL7v+?+Y({_4((&`rLCblIDHtsV7gLVi0Ex;a$PJhVEg{`H|Iz z({`9~rzICBuGR#3)zFXrrl*d(wfoZa`X@hD|9y(Fxkf1b$g zrDvXb;pubFKlKz@eTYAE_AId+n$8rw8P3^Rt2@DX)5hI_-s_6L3%+@=^vA;5{S#|!?pZ=UX^MC*MN8kVc55Dsq z(d%!1^X0F9jUesoFTI2T*Q`-$SVQ?)KtK5;8QODCJaO*K8EpHRp`qiwy{u%xk854V z!XtL{Hr$?SenG649mpMCO4gY}s+%*#A=>eQ3S$&;u# z(L4EhX5|DlWNo;wXVxZkSEH}1%R^jCNU)Pw2DtMz%1T?3U8K0%+IXKSFzx{+Oke)y zH%W=V`NkUlp}GnEQ^y(|G{sbKvoV^QL%gfYDjwXq4P5AT zL)EIa%YrXzPSg#+JwTf;nJg%(hcEcakA6gA{JlT^ks$p1U;nCcOKkSW8*GgBYc;sm zBWng+Km6YJj3E3Q-yjGl0vFaVD7}96Swe9uYi+m@}TW=v^*~!6b&uN>WX_$!3#${v;Q?KHtubd{PV4%Aj z7U(_-`?DYY2-8gvuEy7|G3IiywhXvT)+z|chQH)wtqu5TG2nFWW^I5TI5nU|UJd}A zdzfUqupXP5Uo2KPO-Dw~<42z^*0$BYyo5f)v8arU_`>!BghYa44#+;!Ay&SgLx1Uis2j-H{KjkO1_ z(P@tU(@#5VP2W8hP>VWsx?2Z+r-z1^|6(-#oo{@DM2e2hzJb5idTp>)11{ol9fUhr zYIyoapbz4T(3`W?YAGHj_TKmlu5xjDfW^J|D%NGc^0yZMajf zZLCe`x&dxIl}WIN%42UCM&X@_!1335d@Y0PbLUKdEdwqWYi+>YdhPbtupYD`J~hyP zl6_wX#H8!vV0X7MyE-!6`{GTw}xv{)(vh3Y`#v&c(`)y*2he9?Dv2N zcMEhY@zHQw;*ZhLu=_rsNAZMEmRZ3g7JaXG_sb3|r|&(*lh z+|9$j-Il4{re0fFdq7vA9j2aNK@AD*psdWeI|%ND za1+Q`wEJRBx|6T{xQQmlqw!wqH)-L6U*|h^;yS5r3E1?|%(e;Jh?MmWa9y(2+_kbZSfe0^!IxHBjhLer zt-@Z9u+|k3GQ?B3Ax>04=MW#ocRR^C-x+6px%Ar5ao4IqskKnkZMbrixNg`e!&kqf zl^V)cfi*4G)s{LX&s)NiwNA6RBQ%{{MQcp4a`zza$=b}f=*B4M zHF5ffIB+#_6St;ms<75*edI^_r}`x+`X^4E;?o(LtBGI7tYsqX8e^?*1MkE=X)30u zuxPPG^h-zChVoC*ea^DxUb8o zG4`g6M&`4x(`fXxi+XZ&ir4pf6Ro!hd-534(bCl3)X>&ck8~YtMF(A${Ftfc9y;!s zO10_uC>CB5=hHE&(!SWjRP!~-T3^yW{ghEbA;j?rbRr6FD|YgR{)*dhMqw5+F2ik$ zC8uq@F{JG(EjAh!drmgVtJw6{`Ld&>(e|e2E#$W3rUBQD_U49;)@B!ONs%Eo)<*d4 z%And`jvBwAm5kB2Hs(+9#mXIv+PXB_Y$(K{<8ILU-q62=Q;-n-LbxxDbwSe7f<(9E zCoWa8M2-m}c?nCDEX_?=;+BIC9THiRd*Go%N^%Y?&N=XqIxf!HFOBT|i*xoZ%HFqF z$)fCd8jG^xwLF*=kC^R6neh)I8GDrM<@kVZr|)?neb0mGaSvp~J-{(_*8+~|aSPMb z%QR_7J8k#ER8w}T_OAO=b}dZVxsYS}-erdpR~KjPt}e;$Xl+uI1-Fg0?d3(Wwr1YA zx2b_gRb6Up^3@H+-ThPREg_Ta{4V$B=(sP9bNAzjPuQp^UaH!mbcqCcb9MOZr?*na(DV1LU#Geai?$2&Z#*xc1+FQ zJ~eB*PxkgHS;)4jS=%@&**Yb2+mwv0Q!}9a>ka)>073zbKE>RZS$mbWYd(i zO_OM+BAX_qY@D33X=3U|WOB;JiOC!{OiW%sIeGo0r1cY%Hr$@HZc@^^iHYkbCat?I zQOnwi32Sc?Su^p#n%fiB`y_1+OWQdwH}TQxlB~|w=B_rG-yB6Qfwf7T26hjV4m*Kv z_-hwd1={i>tJcdH98D)>H;O&KG}g`8;Vn6#%{gJsS>a7tVItWyLXk!#4OyWL8KE5O zGehe#L+Ua@>(V)f)TYr0txXHAO{E=DlNM5&N;{}JMWiMrs46+AIyta1C8#nfs46Mw zXj0(O#Gne*s7MG<5?FCCK#70(K_sC3fPWdX->>vQK*<6B(gQU7O7{C7+3#1f&+o`S ze>uiaI~+glNW9;Xy?({SE z?efjpF*Re$)RYakXY8C^mK)dE(g<#C%=OX6{gPRm#Od(lz<`aZerbDCnQ?=-j@3Dm ztm7KID)^@JS%DU9eYis~+m=qiltDv0PT2=6Kg?<|Py%n$F#kLb(~>&y%9$PH`DL&Dn*Az^KY zLfdjekL856BH3Xr*`Xp?VJ%r4L!0H+3Tey?ZpsX4;+PdgTO>WGAw95OjsbOPK@Dj^ z^{IiiX@PaANKh^9)WDjQfSTk0j@8Kl)ky(WNdeV~`siPkMh=!n9xsU)ESb(dGf)ySa5%F6a72G`B+|znQXJ7!9MM-8 z-dh;aQyAV|6y9BcgmrPNz%j=s;i*&6&Z?`j`>YoEg-d5!9FwjBw9zpERV)?NOH+P@NJ`mFQmz@jd?8JA%q`Hg_It zF;)BW>b0reI{QuR3Rz>Vs50yj7l&?Gc()lh^;(E4(e~U*{L)xIQ#bou{Y>QPx|vVc z#y(XO^Hgo@leIBVN;~$+nwTf6XPm8uzT6DR6Gx+;klP^o%u#Lwj?t$pW+11Ka&87B z>I64KS@emrs3An|1i2YF4wOa=l&CwQB%;40qVEXSBeGB365I^>mgp^#`@wZfM06Ko zF2b-FNOwNRkk0(j&fL(BywJ|Pu#Q90XwMC8%jFoMKx69^t2z-vAPNU5=^=#>pY^`S|CohMClZ@>bh2g zwGpe^S!>F~XN(DQratzWx|pZyk=S#! zF;CS&e1<^h2=|)kv(+(JgR|8%qOkx^R89x_>7&yTh;uwyG5vJ;^ix7ySpcCQ3G@>f zfifC!k34=PVh}kTjuGfT5 z66x&RyN!@(j+EDY;r8Q^jk;LT z^je66dnU^M6#8C0^Q=e=+AYQ(O|Og<%a5S)rkcZDskt006!fyF6W}hLK2#Qcyd?5O z$@HO;>Bo;m(FXd!5jplBiKH=59El4MH7| zUc{b1@q`M|wNfkAuK5wI*CuFF&B-Uo+_X2`V_-e&*#;!`*+!1&cq|Ie5#oTBBgAWB z&nXcLUk&bbT?RXy{`DrpOJa0~kAGZit|^D|1vm1?8oQKsa@RJ8l_(U?<5#fghx%q6IUwRrH8rOJ<^)F?P|WV)~x)s7O4mScE7 zBLa#4CA0%E(pwycMr+nY{(#oLTLL_Z>%zo+OXLq|VJ*1Dq>BjXL!l5qMg#XPCSA0< zDMJD_hJJ;6f;*aOuO@4)<9J@Tf~zq#=(WIFG}^^oQ^s)t*8 zQ0=(UZX9;3ha%#$2(E!JaF%QqI4OlMG~JMFN3eRY$H^Y@f!9Vr6x`je@z2>&d^>4^~E%B+-k^1oF>tl zm~_mz;KrEe32}+fO{E4ppcQM;Yl+N5RfHx1TdR!G7_bLbCk2-7^Q%0x-p$%$GDRj< zt{iJ^&tb%$(1~=RY-O#7UD|*vrPi#8(-^Z8oHOHQ;=Ukj^!f#GH<-l1ts-+V5(qOG4w1Hx2h`&~Pt!=ol)}matSnqrxPMjv$8 zY&b`W%!$dHGXoiblZ`1m;Bz#4Ww&eOH4=0ial2}dFf(G)YwJs^j_WbFPOB2FLEM^D zG4)!5M!Nycy+Lt{4Yz5}y;}9>n`fii%97)_4b>LCR-hGbD9?PRHWra+O<8h`xu`ay zFmc&2PcRnKF1w0Q+=$85Kx_t%`xK)vC|5+Ckc5mHg(+t(LogMjX#*}U`>=#+hF(iH z1{*HXcpuuW3^-<79Cu-;MB{)K;u4-Q{=#K<5uVY2uME0z*~*OP2<~#wc2DLsd_~=33^7O@`*CRZu z{c5t-xH0KO+6HcwR1wyww##}8YuDlw%sAzah-kMhbM6Y!wX(mcD2okmn)RGgZ4~@j zNv1K%cKK^UGKtERI@iXKGL;0HSZ{22^plc66ZB}ew&Z{|q#{PMRx$bXQ)T2)$)1r^ z0XIsma;Gw^R@pQ78sjX%EeSMYGu|xZrDW1D;{sX?xHjVqv?VsfaZ4yJ!)={DRjREL zDLR)fNmRzy+4?9h8{#btyE4)1;EDvl($xFA+FCSg+bCOJUQ)DX*J!<-=t6oetX)-S z48TRTPf5lMG2OI>QY+BJROJ|T&(Mo+r@};OVxK-+mV_DRxQkgx`*)s{qOG%R|r|d$BWkzkg z3!tfkF%l_EnrIQ$5a*m;l{J;@na-L@%G4Nqsl=(umom6UuOG@8n&5Rj`lT(NG+ z3?eo=dT?4v>irU%DZRc{STmk#Y~+#%!@#ZGv<}JKtX&$dS$o0lVEtkfE?fLHN3q_D zwIs$HW|9v_XQ2r%BCEmD8OKYb87|2HnaQtm$%He=f;eg}gKK5O)hL^p5k}c6AeU(o z@zkfvqV-G}tYt!^M3N`yGRaQ@Ep$@?D(17q&q_8-McT?->pMYb&tNUMm8jSZxN77j z>MThW8F?9*QejOFO+s-pDr!zd(kb$i*sM8A`RjweC8-Y>2HdhKha39GqSr3%*7+8L zwTit7zXi8iTBxnJ$$F??6x^-dHr7qEFyk*a%|yFJqg7G`%1<|l(PkJ0;^Sp8y+>lZ zie_{aOmEANYR`}CC=fu*aT7>0l9DM8Kr6i#)mD_zYa>05c9T6Ti#{#mYy7nc`4gNn z)|jzUTW&;iRzyR3cx^^#4V4e6LG@`ESW!S`y)n)bhM~*KhReuH`D+Q!M2N;iBor^u z!!GU_Aua=Lj%q?gvZEqcZ%LENESPfKO~huT&JOw$hnsb4CUmpmhHmCjFNr?XycqTi zS#>=e1EM%*t##HX|jD zk0v>GrV3jfAkPTDm1zv;#h8Y9h4~-Nt#od@o;hS()@!DCTlyr)OXH0_3VgwB)3X{l3H5V@KY{>6zZ|v`CA7C$#y3*#F!!;%8DLdxwTQezR zE45w}djz*0LgOUui3$s1!5za8TcU9_#6BY_6!F)LtjT+m4>@)Sa~)WdyR5Y~|5$x# zcNnie!Gk3(T4vl;4w&V@eDEd-(R+C0bQ}8jfcRqnOV+Q7|@ogSktmr`T8)e;`5 z<<{lgy}eX!%Wt_Vmy}-5#*&NmmMM=|CMy{NcNImqm2Vp8Y%=}6HW6x>Kv!E+b=uw~ zyB?dGw9zMj&$Q}P3}bkoGUn(nh)-8dSCb&5#A9&Wk|t%ItDGsV=_Eff1sH^;(%n7n z&7H0FQcUiWnu;Pq;>wbo_>FTnFT3^N+R545rcz;u6K8;o!uF`tD0(f#an|NUwoZvl zvQ*~0lxiz8E(0!!#?`c@5uRbmWfnUV)_!G4vB&C4OxC(#I%`|Qi|^J>d<-6ZtSJ|V z2iKWZw|1)yT8YiHr`CWB(`^9_aqIvCw|2cYDJV<2G^MJoM6(VSZ_7ZCPy0FtP0Yq6Q-3V&*^BY{-m(h+RZJe&A0|+ zXtY|aAqzAVaRIG%FyN!+haLl^#uRIpQ%OeJAgu0&;wxhPL^WyP7;S6T9X z($E^MS@XGA5QlXFx7hF&&6?Tj*wd9WdymBQG-eu3!4dLl?Z#qg-21mLS+i)u{}YN*1fAH9@^nknHWIO&R|N9unZe6u^CaB60zPYD#MZ+$x_!02sS*Is7w#Er6fs8 zQ_@MAO)F2E-`gg8q-e20=cUbz^U{kYzw$QRV~$Kse1ltSA2>8(OukZ@G@BZvb9J*Q2J0$aG}PBU3Tum+_cc_Ota&J4#|qYN_)zLkW$)l&(k#(b z8mnkrQfGK@nd~T?-g|hu)cTh1=v5kRrL5al*7|}E?q2=KyfoIAIx>9D^2^wCM& z)E;CzLodi zyl>6af<1oq86hkQAqJCZT%}WJV9&AY14pJ~Y+Fm#4EA*Lize>TS@(8zE{OSmYaX~| z|LVyFyQh&|VYq}lmNFy(tqiy>C{?rAYI0L)v*=n{+CbFe2Mbf0kY!`7j**R7i z)#Pt95t|JsW&Mu%)!7;!7i%4=ac#%@`wgpz^+ui5^Hxj;#N}a9UTmH7V$1C3nr9Q3 zovE7LTd{bkxAR1Q?}>idywMPIS2rWhCt8!3O|#BU-tpLdx9nN%lN%=kVV3+bup&HT zhD+Q5CO82gVq|D=nwq%xv{D<3;+2oqtO2dB*O2+S&$ zpXg)nK0)b~HI#MI)nr{-nEuGUH|<&Fle5!TSj)&OTBS@S6em^A;99Yc!gY4mrf`Q? zaogJHyVZvGb$2~<*W||*+_-NQtbJJz*qjp#LD#g1RGn$Z4Qo|lsEUVCXYMOWkjg~1 z#x1J@q&fw?RyI5%h@@HR!N}%{3=^^0_=K@mxFKtVWZpnaY}PV|A@=i4vne!!b$7)h zC;R*0IMml`vOYOD$U7!w9sBVn(0RYKApPNcZj4(kYjTBkzOdFA6of*#u1OOO*LKuq9M<{idsp21<-Kcs^LP6) z?82^!4X=nI4%bPQQ&gH^!;7ONr`?!G=4}+*RzFU=v#s^PJHN8({+kZ0otnSLm-&}t zS)#dWIe=6w5T4<<8DDqjhe&~nn0J}}KxBw5$#AjfQcx;ExKxRw*UVfq;HpXTD^HGX zZ>-Sqs`cnVZ{7@Jy$781^>=Q+2gYF4wVk)}wYII%8t-%anV|JG@E7TUABhdeUpLKS z`QGU&Z201#o_3yTg1;WZYpbNX^=J||S?}6#|H}Js+P}_+sS08<7VSzLE=xaTrKc_w zXUx@iB%0~zu7+&pYixGg`qIX_sjl+=nK!IkbW6fI(kZ?y>ux1enx_U_`7-Ox#D^*+ zmHKc3a+k{%b!M>-lqLL@thj&{;tcw-LWFfja8pOwp{NqYXHxtynO1{X@4VZJJtgFpw4c z)O@l~SKPD|Dp3DVz1a-j89W><7PT@Cr5@@ZZKEwW`rh8+LrD+K{gR~4Hcl-$;72K- zI0Bi`4v_`mYMzUHDWRJ9Ydz*Nr@9VF@q_$Pl{b^(VP(dZUdxKzss#V?l)HM`T0Tvz z1@|#yQMnZ~qcc*Yx_p$9;ImaTakbqwyY(cvi!a38tOvT~NeahR{Dwnst*=~m*Nr>v}4v9c9Xc+TiH}B}|-!HRasNn2Q_64RquU)|dp@_~@+1 z@4I>L%8BfXP@WjT(tGf+)l$1UY=xP zt0v~GT64|HEH+r^Dt+*HcZU>mD=>+ z&U_YOQQ)o0PN*+Z1){13SaxaXESgRhy1TJpjI4`t5+A*9;-)1Nk~T~(Vm2)~2=hzO z1ze&6wN_VVHYIO{n-=JjMG=u{Ye}TAC|g2t$(t!>t-N*%aX5=U(n5|Vk~F*D1GKsO zv4i-<+IADVwPEG1+hd`xGccOC^Jw+m9&LOyj{w%iifUSu`XVMfPE^EDX@IdEEQ{_g zLbXYq^{a^wDMr=A1+?g++le>Vl&zTm<=v07(Tz_j20W9+u*@t+B3*@1aPN>685DW4 zr3Ra9aEu}oQkQk-U~iWS&v4vs-_69U7OnE^OSWy*9cv!=O5Dnc+1q@|4g@gWjKf8# zC5s|!0dxTpIW%btZWV;f$Xcq-NV&*pOJ&cr%a-h@EaD>>o*hu19#U~oD#JgutX-RU z=t(a0y7|!b))sV+d>0p7(XNZ&((%b@}Z~ zVHd>vQL)fO0d{Ug%c01|tT1-VsZI&0Ob)F|38%!UK3$3p2&>o~MW(CDc3LRQW5=1* z0yMXij?EZWAK0;U)xxjreDwCzO+LkYnIaEli6wWAWXu$9U6ClDwbzybm%=T?)%aR1 z#74VSX^KjuNSajEX|lkF)OtliP))(+>w-0lI-8n`;@M4zY9d`i7D|OlM}8!Q2Tl3+ z^fVO?cDD_7wUhMLHe8O%clUN2+r2h++ar?_*H6veJ*_-J)TAjZoNbY)NjtL7uOQBk zJ+Crl!z&+7DBed!SwJ;KgXzI_8DW*lA+42(Y^B}Td91&?ll?$kYA!KbUsp#`-0}@e zrtElBijmlQt13BwQbT4vu-=k6k*JIcLyD19cqUci46ZRCXm>c1o+MiYw{q6T_*&0; zaO8%}2_!*Un-W-_6y8v>&$g?c7xWH$erX;YV}4a_xGUR+Mgy8+q?YQuf}|zPTo7Cp z?+-ni9Ndt}K1E7B^X9i#?mymj%=PFZd4SPSZ%@_X)B~Gl#;@^B+3dr1XJr%tvgcK5 z@X@4znnMqDHWc=?H})K>Z>l_WtoHEXjP-Gk-+pl2#H{TB_;l`yniR?ugDcZ#HJ0q| z>u6Cixnayv?YfGBgl!A9t?=2qYI5r4DTQ&<*xQ7dnJFqpRT7Lz3PnoC)do8eB+%4Y zTP^H@uL{LcaKdkiz;WF0Rc@NWmtOrLi?4+>b&^pe%|6|%@!;5S7Gm>k7fLb+jGL+o z_N)##ux?83ZfflU>(awoaw1xDqZ)JXq^_W;WM^|lN>h1CRl)B3#D~&$#H4PXc4(*n z;rM{cTI9?CEN$E#0*;WY>z@Qa1bM@A4@_!IRjf zDzqwNR%Q0$+M;dsr3tkq$rX9K4<#&5+!7nV)+ccT``}H(bh9y9Lpq=(^b$p8`xIHk zr%F>W-6G718_rq{u_bTj1}(E)s^mtYrohHpzx1Nrqdui# zESiq1vc_q;Jtj@psdY!z#faHPSR%@DT^*_so&lO?bun(j5ksf0Ja7A>0eet{STSeoN zH>0*t1?0?j>1~l}lKsk4X16z#kwf+n$6l#kxL=Z0ceA!E5B!L%B`m)XYf)_iGsRlx zy`gioID?t!eXA$u?)I-traCmNi=FF=XHZB+@Yb9iM$pxm$*zXstvTUT{IV#ZH8;F5 zJG!er(`D!7nO0`zTzPptKq^_JAAkwp>h zmINBA&Ga{8uEvaz>O}wY)OnrFH6EbhennYZC>x7F&GiMYhL=&8(rf2v%XnuDtig?H zyC*m#``t%SwEm&5ByRLMyg#rpE4-^Ha)51%WNFWgfzoL5oV~IsnGBZMos4uS#r$n$ z8&3}OjWj<3ZY{*{yVuWOz3`^^)l+kJ_~3<`vlsyZg%tr2+&nGWvMxY&w_s@odsq=@ zQH`J!nf+2^!DgY}J`vTXY*;Pd(?_M$-7@Z$l$x^P5}t{_R#RPo#yF^CIAZ~+3~x*i zsZQ{#NT1);+Hhgk4lN#Ap3RxZ226cFz!De={~jlC{{flZJ@e)ezMdm6sqKQ=rO_j4X<(I#brWgNYAi!%=D} z70#8Mo1W7okwR3-vg_J{4c9g6!NDVWDJ$kvXEHf|PatD$a@x#Jk~3%P483GHG~W66UgzT}ilXj|1^y zpvw)QiOarr?5iMKrhu3Y3?()bBd;bt1ULR#XHjHlW0~-B8F0OEiUetDz@;k9ve>DT zW!J&w3Be7e@z*))qla@IyZ^S`k59^rqpCKPV2rXc_RWx8ab&X;+3<3TU-!ir&#q zEWFrxy`P5OOasv3tX--dqDzuw?Ve;xMWbc`}HubPZf7$*>Si6ZcM<)B5iPz?+c0(c_1G)reQjP*^l}3}mEEv5Om#yM4STpM( zfthmF%2NYc4Y*|Ma@i6`5~PXKkJS}lB5Rxb=H;ldy3KccL~D-V+&SXDQvP~yu&glS zvHO@^np_a)j}2!<9yXlqt)7-mko5Mm5}pa`qM64#>o0{u~6cH*zD87IzN5S>IWz6TRWwg*er`+iQ+)n zRvPQgZb-62PA!Y_6zjsM-o~u4vmWg3+Ov7_>V*^HRZ%GE6vol`UwpI*&)5w~wkK5^ zYY1gSsWIR@m_XulIl2a1vV2b_K}4_h_BeX-TGnZ*5(OD*X&c_28>qFLS~UjO1mP^j zEl;_lyNwM49jwiO+-zK?$Q<3bDL60fI_^+&0(GaRn*%Ou!^_~yGtFX)!znx2zwM!o zOK(fw=u?^y*qR%`(o0rc%HBtewit7<-;vr|n4&V7r>@#@vcK1wG$WEX9XAcDzizHQ zx@JiTi#QV3PbrS~V~I^?E@qh+O!;fs{3?Piu=HaEaksTG* zfJUj6k4Bf#ZrQz1Vlx?V$!w-F<5FoR8)&edf>fHZ)s?VLy}P%Q!s+XXwXxBPYahJ& z%ERlH`0aXZQr1rY%2eW58DI1C2PqU}+eull6Gek`3M;>OtV4UrvVrzmod!3LBe70D zu<`NxzOoBvy_Ge#RR6Nii5O>5Y6G;2)5MHN4itq+gr>AxJhg0(qla2bon?qEsxA3& z(QCD3icXNK95^d*)iN%uHx()3uXU9I6CYHNR#8EkejjT^uUppSz7lsoyl(NW2^*%e zX$jLGWWsrB1GveNqTur6ml9THu_03Q;F0O%NKuyV`V2Me!HlO&dfN2bRPDCL+6@nf zZ(25y_0%L%$bqxciV#cohc7{$^}g}urtxe-OEgY+hSp2cOl-J}sRcK)nG%FE)DqUB z$a?NWraxqGEka!}<186m>+v-Tt`jNQLG@G>Qhk*108bLUG}iW1n={{X`9ihJXv3{~ zT~U<2dAZ-7Ra0_y__7C1Q#P#mVrO&~&S=jk9U9e|JDswVW7GiTMRdqyqHKlHls|u< zz1~!68jPcDs!eV!e#^qOize({IVpR`)QZHw=4{D?^lDEnu^FCPaI0apxNMy+BbNeY zj<|K%gjT0Ul}1ZordUh$k%VV@SWQ8ivff5jA)73T^;Sisp|ZrDWn3gk6Q)%idXxnw z-R&c+0XLmB4TryWzGI&YyS7<3RUg^%i2t6IQV~k^FID|uZOx5jR*`BGVrDA7sM;+( z$UaD6)ECuJRzyuvPDDLBJsx^ssHbD7r;{R1)N`z^yS28dGOs9YebTl& z;@3_~+UU#5J{D+`uxh26NQY$Taw!XfyX_gq(T9XUw-p+Q8BA(BI zwZ0tfHZaxS)6QSOq`UTT&`Skv12N|)%sJ+AZjmaByB%U1g%ux&DcBcAwF#S*L781^ zFw0EkHx(^E(c6Az=s5eQnd36N@L)Fur9BmeNn0P88oyc$KDeutcGsteRj0=uO}n!+ zbwP3BJ-K^lX6_85)FgK&<%@niW064?6%H)qWTcGUKF(&OE;p@MtEV`rr}?1T5|tTa zYU9xWszgEIR{3y=$9N0`)?09gb>&Jj9M)pJjad&Oa9ILi#No2lm8vvL3aCh#(a~5+ z&1!#lm#$nL1-BR0o@6iDeSb`>Ycs+c(xu>pdcrn&JYrPKp|Iw{hu9kj%7_uXnQO7t znoVY`3etCOS~fL)4bMoWge;IJY*M(=oQ?aY&M@?7B6c!}`==sN9c6k;sn$JDNPJtF?NptS?0Nlft_12oKPl&OVcJEFnD-)p{sm zpuOTW8m+c67Ntfktfegq-nD(z>`hB=Ke&Es;U1|e;wGUgsl9+IwMaT)2TQ6=C}ff) z_IWZ~!t=_+8EPrhEtVWJPA)|fDRLC;=2>jbQhh{4%3YKz!d_=tP@-j^i9(o_<`JtL z<;g7a04AljGP}X%rIMeK4W;$7*hInABp1)|5v7(moJ1K7ewW!y!OiGQGT{obDpZi9 znJQut>#ZVisS`%Tj2kO7$)$(^m+`gUIz#OlP9i1IuPk|1Luo=yNg7pfn)QgF$2R7& zsrI$$uSsq*G5uuitTWX!hsvWFGN0@_2IW)72i4Kzxe`|GW)O`ffBXTzP0M8I2VNUr z-%ajC*pYi7-y%!)nADI|j6APLwy0IlSmg0N^7IFMv@z5Y8!kJ>i!nb!x{M^eKv$xk z^kGbIiE%)kaoj3-LTMRGei&TipoKVTQX@9gGhAwo4^;|`uXRm$;Qz zxOz?#H-LJ4k35BKNKa*Ay>(!Qsb>=N2-$DcS!$37%)}kYtcTh*T*lYvwK!{8Vh?EL zvLz`>YzAmuJF1+un#xqAsoFT`Nz zDWz!aNXg1;^`N~O`T-v(wR)tBdQPr-n5JknPf13RG3FA1Gmj}bQe$pQk|z?hOBx(O zqZwSw$XW^uHEYR>>LOtoR!cl4`Eb!|l?+!i*OZU86M+kBc`mk89~}&8EZwUDGcVTL ziThK-TArFJPuAnHm~3PhHPDi!SrdnwpshJ`BP@5}E-gq~emAv(KC&@R3TlO3v({do z#)VR|sR9OEo{TJdt?oj*s1%A6C&)7vMV)nECh0P%F_Q>f zivQKbhYG?a4wraL2GeRvJO}6eZ&lxJfsQMVJ(#zpIp{nJvbQ{FHYRk zgFn+KPr{Y*gX&p)?Y*@Ugg~w_^^eZ~a{L+=V`9VEc}#}J!kPzshz)m9 zW)4)I*2Irwvr2SavsUU%1u3GZv~>PNYqSiqBrj?_I8~O_OC_Na^jfD-7-Xr@wCp=7 zD+oxQXhqgrnHf9Rsr_(dHd8InOo(c)FJ?igj=;Sho!YJ68Bxc_7PHxnx!2+)Pm?~z zXl*R4+3JdF6N$~LV)_2N${#&5LWlXGBJrt(vz119O2#(x=MjWnO0;m+6=ES z)FM3MK}OU1k+^J^Aww-ZHKj*9-J>$Wk5y@1&8%OL^}k+0kD~IQXx2~FsYjqvE_gJS zn5v^NcBs2SrA8$?DtWV!t{N5dG_ zTkExJEGA=Wl`ez0j=dR!6MO5SmeOd^Zn4|qrkV9ngKM2DBa0%tT8YudP?IE)trDb` zjH!(fR%(+RYZM7?N+TExY)A`4qj>^yO;SMVexJh$vwPbbuS3?SWP8|{lL#AVl*V+h z-+n_9PY5zaCsn*0hlX-fcRse@%X+@MDkZR;f+98}Eg|-fWD$@&hf6(F@kCh!8cp~u zkLAKa>mvn(Dmw~mG+JWsBgRy(LJ~;D`?1QS;-D|j7buFGQgC2ie^=Xe#ad5`s2nM2Ny(A+ zu`y0TM04KkGeZN4wLGcQ3uTp3W0Lxj`4hH0JTYm*R95M-LJ9@uehvQil?tgf1K(UBl(cXZfJ$|Hcsf1yxhrGC;jLnmB_b(Xi={8yG zhV6J6Z2J}P;6!C!r^fU!?Sd0z%u_dIN3o^~Rh^0;!~s`4YILtSz-X| z;CNgM+nO<*!5~W~Qi#VyBpy?Nw@Q#=y}^y{tEo&Cn+a>3_g0HBWd;+aR!LDYg4wb4(47M|_b=FyLofM#X&a;PS?{~iq z6Qj-J{Z;+#TieV3vw3fBS849C>O5vfUF<}=phfT_ zI^#8WuqM^=@EbFZ*H#ctPHo*~Mr!lds$klD!afDwMBSJAK4o;OHLt3Ll|BmZX=G}X zHk8>&RlECI_YmT6;dD;H;@Ge?AGsFZUn1{L>92oQte!U|cA3wS<)>poL1?_(w z2O@+wj4EQD3B6=z+gFaYzBAPH&WW0peTSmQs~BljKqDv(hwRXy$pj{v3kGWh6Qi?* z+-AvD+Y;9D)WPL}w|Q#o0~gS8)~KQbH$^rXex^s^`zl(_lCf=+lJB^Gk!*(& zqm@mRi`CS{JkKckCkj+riNK{>*F8wPDf z$J#%N?Jx?kb{31^sVsJiha5hYYWVHE(9eT=vTaYK_Ixcab2*L^Po#M&9d#aFC)d8` zFg@h8FNXSBc7DFBYG?N0iX07QKO(zI_6@9T%}_x~fwV|Uvh2*&_!Jek%z9f4rh@g! zIO*H1w--BUU9@aV7Kh<7)b8?O?XAP-5^G7~ zrgTLC3)xfS!BwH=a0kx74Q59Zk`bXP$x@}mrEi2XwV=p=Taj9IYU@7{qe;`=2IU|y zi#363j+wXp3v1bMwsj~7x8&NIPqdAioQu=7uAOdAi8a&?Ar2QQu};Xgl0VMN@zBV# zn9KJBnkRDnrQG-8n4WSIXo3e1fCg^En)}yZt}ObJG0r)G#ElfjcpQKhzB-hL0~7r$ zR72lRzqJfY372Y@=;6pF$f1h2RoyV5OdUtqZIN2~33uGIk=mO@)kEmpt3_(SRc0+- zOF|LWEYwD*mQksrNZ)i5MMpc<#IVLwbD1%>dkFCYi)3VzpZr>|{>~6C9&0z_yEp#T zN>>q-&r4k#k?HklO`L|y^dbsz&a-CVzs{+DSG7w)?AbIx|JihJx8c_3ZzU}iJE(5R zSu6ZjF-t{i>mHO=W}PQ$6AjuDB$QcSnGATWf=O+f{u33Z4-<-0pj!?i zgS9Rkw<2&Dt)?>Ib{}nus3LIHsR3HMIjoYUzCo2Q?J`EIzerud$2HwZmzw-t02jmx zyk)~R;Es;~O?k^onN&0B4%I=o=;0R!E7%8W27HbUPat(l51!28LKRy1x#+oVyx^Oq^*Syte%sN2%CpaYzoTJaeE+NT?-#-VHrlu4@*PvoWr zHw_6Ed+Vq*Wz$NlH5@}*Zq;jKftjQ;wUDuogHl()U~S~V%}dF=j^7gQ>jb7VBlnfU z`uWU^Uk-Xr$=a_!d!ufboo8)5z$DbrRvn?VABfa4-7s1qw~3-e=}M?Ai8k?s@-&U# zf3>Z7jd<-%YfVmYpw%~MlM^)3ikTP;KS7Wm zsf%1Y(-9wmvrYz@Cka?D255a}9=#==-9I3&Q)nqEIlI8ac~sVTtZ!evu=~qM#n!Bj z{n^9yoKwFNiA+63xMOqH^3;}1lU6&OEy+?+CdHczl#SHbZJBO4YZfB47!Aq_yxH|5 zPwjQ;m^3P(Or&;|%o-AizzNChcs1L2s~%A$ToKbFtPakK`*J zYsKWMk0f3xUI4>-VZ;}}Iw98%KPD#KVYm6*>^n8|^PrhBLJJ>0pPqKOA5Y$^-2MK6 zjhxqzMHdG<;!&s=91T?!dV#V<!PTkst)pr=Vuo6FafY)Wg|%ULbzGhquY z8!q6f8cUGNh`vlQ19Z=(O*sil8zzx~vPdnswXs^^w@7WWiH4P#^%mBO$CPTR#$iCK z6+!x#;xS3zuAc~KNNw9NWy6WUHHy$qVp6NWpJaVGn3J&l`q^HY?n$O>*`TfO;@=J{<>h_1)m~M@yV;#)8VK)cUT6AQ2?!|$!7g(nPI@wVwV)(T$Sr+LD64l= ziYWRRMq9(mV6?YSgv>Gle2p6g4rUkS$ z2?IJUXX^IXhK>szh3XpZT+J!%F>IGl=wQ?!+XMaz6avr)Q-i_ zJhc-iT3gVD({wb{i2B4iO3b)m;2Ty{G8fhx=30$Xu2~_OaQE#S>J?N}CX`k13i=Sl zYx!sk#*oF>h!@Nr^unc*Ec4gUS-UxuXDR&iJj)9lN~rCUgrcb1fbQiGJt|x<+7P#c zS{P8CCjRo=B6c^5R6$! z<|E63{kCM*AdU@>?e);yU`p-7wUO)N-7m17J7X=z@_H0%qz z?=U`3RRK>V_=3D!5)J{ej!-e$<83Utw*D!- zQu{%mnP}UNHVAlz6~tpEhho4Lk7=lp9JEK>EW+dSWVie1S|lXXq%uvz;Dv} zvn$$FU2wX2`9M|f!+WD{UyQ#FAr2P_5QiUsQCKUY_PR5XnAJ7- zsA6Sc!(Hgst_|_p@@R=;7Lr-(FTA$i1K-3k@D^sAFwE$z0MCLkc+G_tN}gp^OQl-Y zkf8H1P5M^sEp-%vtCFQUbr6yn%F<6nZD!MvUxTt;IvlfyGoF}k#F1Vr*!ppO{78tk zALTq3?(fDL#1r5Sf%Aoa8mF%KI^jFv$6plIf)(+Tat2a~RjgFnFdr>{4Wli;R#C$( zfOcc42Z_>#dIc55DV6rphVHP~_gZS<9fN9aF*AGakhBM{eQGlk#grs1zz5!_8Hy#nqPDqkwevKin?vZRQrEkTSX64RwTi+c~F5^M^J z!7Z#iIlDZ1T1V-9j4l*lox+TN2e0FEa9n^B+_Cm;K_1H`5P$LLC(}GJPQz=@_1EFY z#9ZvqQu-P@=Qoq6bXo4fqDLNZ0#nsDL=?zrL&$`*2~v&d>#3e42z?NK+o zbg|@WE|Nqhh`FPM+Qxae*pM2mrF)Ccw|W^MAB5k*{8%*!ThwjVn~eZ=IHqRZh}6JM z!OQm1v*mW6D*HY|kEbThl21CRC7IfQmpG2k0o@Vg9*=eSl>6x=BC>w8p(jmdYd$=6PI8# zM{V}lAXz>d%EV?&EfcTFyxI!rEFEZgS6KTc$E|BIyawoB0@kT*bwZXMetek}%%W@4 zJcX>(Z%eNfcwoWx<$EQ}x_RQJEdPI%Z3OnR2e~LSfytoKrk5bTAEj#qp|$ z#L^htex~QbSUaQtb*w}F&bL?$uYWwOoxifllrMcBhvtuY(yiHpf$279?Zrx5Gv-<$ zx9BWz1+*>H?Csk(7mb8|%K}<%TK#5piNk2l4bhR-@VZ^|%u#)>o#{5L8M^qwQ@bUPf&Ri@JH+8VbkpG? zbkpG?*0_3G2BOPN;wR$AzU8kTT$yRi!a>CW2|Cq=_V|Xjt2rZPJkhD(`F~ zswiz(Pmx7p%B_4Df>cm%6jTqe#4!mblE_G}WVE>($|`VJ2CZ%c7TUan=q;&4sg|2= zO@ubnZKY~Zb~6sd1~&&}a}^SYYf=gWQZ(i$I$R$)d#p%Q4b1qQ2rWcB270o#miQ!u zI9&WnvF0p>>$HOO3JB_qnkJ_NN;~qzZrc=ZLboa!ya~#x%uT~(t*xzyv(l{V;4g65 zH&A$0DwHpkG0(ZR@4Z{-$Qc?-`dW)oV~a z{1Ue@-6~K9QVEQ9aMfp`CKKXq(ks(UEFOc)K|E%RR{)q7GOmo5UgW30QT(Gfw z3erMr>_J+Hc>MFY^$YVZL1uCL!V+xtd>?;a8{T^`!j{aT+Wb>ZI?RZTnT*Gg3+^{t z&3-*)&~CadgVwqBN~z?j1-F_5z3-PlVzQV_w`tvyyMtiNdWuXLa|aE$3e1GEVXeR{ zfK}*QnZqo(VsC}qSZ|Un_t&tPqZ0Y(3t>&##mxVX!V7J3*@HFNmq!~ytmC8W5a{`N z{hP)5c+=YYT^VirHR6WNdW>e}x5{W}KvXg%d#=4#EW~JaARvW_`%0hfFb*qIs+;KE zag-A}mlS~6^nTfG!fe3})));hN$aRnS#PigXk)aSC`6X2^C&jK)uy%l~NvY?EQ?vk?>+@Q?Be**GOs7;oQSX$6u zOSj>TSa>!k+zP23XSd_O@SElJ^CwScrXDnvtts7<4&00ZKSN_-4>Q;WqWNwDFy+zo zl#1HYx&^U1Ok4U`yoC|gX^m4;GdcK`;c5D#HzjpRWh;>S8ziL>2Gds(5W~gV+ zy7a2;nLVXh%&@xDvm9KApK-*VJ}Wg7qgAfbLp6e04GVchm)mVZZfv+ICswq&b!@q! zvSTaw+e{yOTYlMcEA$~FmB?c6Nz%j8z9tpu|Ca?|deJs7iiOm12z zE6?)6w0-2v9c5c_)OJ#1s}afyER}}KQ8(ZM8tcu`w@jDqREmuazu3XN%IL|4$Y8^} zkM7-OFo?5T0-9Yf#$u-$fFJh)%rWL+a_x(R6nnnFJK_0{9fyA>)|m0hyEltBt|(`m zLUGn$ZQePK#_Q3XK(odK+bfB&?`!I>y2aF9Vn7EDn#5UwH%41nvzA{rI@|HJ7IIsX zMSzXD_V62&f!hwn16;S>-h?0nPLY}nOXGrFC=2bj7KMtsxywf@+j5l6YlC>fLk1q`Nh-{)hLylgR+dalrdfv+jggrTX1WaWgwUkiP>QdGTjnPq=pG@2_}li+&jA# z>dqR?#4{}J1~eH~3|uWHaNLlZs|&0d2+z!}!hx!s^CwC}td(TF+PntnXEQTBjl1@K zl~%Q#lc^bQ7CGG%h1wUonBvQ15{#|zk~LOwO1OI^+S2D09*J=W-`mGI8BJJQdSy|! z9JI9+sSklm{#vq$XVd!2$*}ZZX?p*W0)yV{wxwDsZ3LcLj5bnBC=;VC$+COFM5mlJ z1ZU!SU2@BVhM4r^Pfx)c(ucgm0OdVHv(c>S`GK?Yve!8WmLHnDN^5F>f+C%8pvY z*HzH4vq+3qY^J-YZ8a=-EmB*OWm+h)xA|ycjpN2>%T4Q!<1R2>Gh&)rcH~sk@`1W{ z)HFWVW&a@ldSUmCH&5`?{xR8KCqGHisbVdr3)R0CQ5FIH;hy61%}Z-{GM+bs$&BfB8RAMzRi_m^J8~*FSdhYPFwtX2LCCnFPpjjSTN?-Lm7X~YD zUF<(MSlwFkPOWY7I$TD;mOtLi0p-Zl5H?Un%tzaP9P13^H83K}bjM^d8oY(_O*v>Q zNplC&nA~lMOX4;+tpoC6rHw%PHmt>QGvEes7b;5Ol}?3cm%1S`D2vxR3xGw3LUE1* zKp1CA(ok*Ac<-(t^HTC9m%V6S2Lek1tSLVxgyla5uM_yb5Y0ntKbP-Awaiz~1IiOs zk)h@fpHDyj=E;+XcW;;=xnm-1;isBnx7A!E(;c{Jy#T@0ZN?mYiM8eVbgVwf6Ot+$LYba7nzv@<~pbE*T zYsf(ZJyEw_0IHVaH8m{LPbgNp(Cu6f)NfgD%eEZu&H@>5G;q0TsBK*C(3U@b1Bb)#l&@(YY`cs zIiVPK!(Xx6`0HF2sSB*Sp+m_PffFGpk6v-rN*j_xg|CtViwM+qwj_EdZLG zAm>{1#+stXDznZFRQ=dkFKRwRs(>vyHcJ?R7=8>EtETIt56)WU{pa?iFEFsB_pT8YZIY|)nxCy$QNS?LvPFJwsW4j3(hOBNH0 zWuk?$-C2my_8g<_3#YR?XjyOMG8u3K+OQVfg_prs{@N~_wfr@zI2lA9h1&i@k%^Id zYa!M)d}Oh(_L01Z6U|;!LsB~E#D|NS#qqCU9fIzQWW*;%!znOg! zz#WG*vnkep_*%)vr7e3i`>S&)D-e(yHmv^I^yED;dPeq~2^&|=jcobk-zzpRYb(y?OdNvS^Za71h%61$ z_QoQNNF)(|kxDSy=BX)3GpSbfBwhn9qH`{GEW?e4A(_ZHAO)jW6r5ql3BV{Wfsb2*d1tN zw7AT}So$pS)6^22^{RlF31<#v!}y5ja3&y#=?h z#!aJ#gE&>}v(5SZe^CLscb$HuSOsK38=h>{m8Qkj$mUxon)F2vSxS~%Qh4yB**p49 zQc#!tB*Ch4rd5AUE*f~>!R$*T2cHY&SpjWVN6m&kYyVWSHLYRSvhK3{zC#5ks`C3u z(`sp3itw8}(}HJIs;hRj9_7x`x^1V%jU$C-w~4VuXQ8ZyG1E-+5h4p^<29a|OIek` zxue@TcV?Nk)fvdCpi}6cdYN;ZQq~8 z0Uuo@nLQQxhs$z@JHEPmb@cIrTQ@HBx0b$Nxn)`P&P)!v>Nt?wxj$29VC~CjElzLQ zn}KqEqzn&^oy(x{__X09Q-{57nX=@#PnYlqe2F^8KW|GX*^LM`@I zdS&{j8EVJZAePZ)LA>p$w1AG90SA3u%C_30cxvfp?a?#MQO4X2wS4&K{@nm;BWX(V zF7R+ZF2_IrRq*<7?Wg5imY|U~?oRJK5a}t+?Jm#jD$VcMpVNLI(!M{^aUi4XKu*X0 z^wFMe5AR-i^6>ucE2r8kKdRWgq+vHx@6wMSLV3n!5<7{l*QK1Bo3{26fh(-3YZ$T~ ze*4|5x0`NRZ|Lk!8LYdb3d1_3k6Ehaj8-Zw#4VM^XkT_uZ8qG}YT0mro@iGmlAWFz z8EIQHM>qSe+wcUt9sd!FJ$9mL&lhi0Ze7;6JEOZSr>_bnZ8^=Q{Z+Y$;Xrw$JnQzv z2~WDG0!eMAx%|_jb+6a#%;-3fJy4y?$(76_5@a4@b3a@lBCDs!%Fs~Tc4`FHTL#>l zOe_Y&WrJg=g_;eAzcT4ctYowmfd|w!b`zxGKIE?JT+07^SUj{;V+z&da~p5 z(16SNE~t?%z;6FhJeT9wKRvJGL0sheAJ3ltF34k(+_Q3+POdb*um`Kx?DQy z$B1njIiV?X<>VoayfY>3$)hP22x| zrl~&Tk%J&yfZ7xD&RTboUI|vajJY!1lDnPMh$NCqWWa@~5I5x{_z2K^p`DTk5{e~U zZ3XArI5@R%usUbFZ@12E_iJkEU&MO8mmwZ^U#CDnE?`U6&%-B+jMf&3Z~WIh1{)jb z+_w7lifx%3NlCDcO+MF3Cr=^c&Q8`xC$Eo&l3F0M81(J6UvFRihpHX;?##j3T#l&) z1%+D)p4O~%M5bLs(3Q}8_5u~iy)&h0_C@KmPa1tr-M(%%veoH}(OmF8r_NnRv*JDTdls ziMDK7=31N;wI!0^>!pAJbPQ`tqV-X5W5#8xWv(&l9(xOJ9-4RNiFh5@aK<9#({eV} z938C5zc$_zVjU{seyDf&F~uUO6>UC7&u80H@_GS?!yhGMo$^U?Y8Fr73HwS)E%cYmOS9KVLjB_wDptM4sJ_t-dgVGGOS#KSGd06s@d1UR^U`@BGXH~2fg9&AO?rVb>I+I7+JD|T7tQ~X| z725uZ!v!&GBQ>d(c->(FX-nRzMue2YTN8t?F4i`V1D|rVtLw8>tI*0|Xm`_I#)cD~ zwH0R`KbSLGpLe>2{Irk~*hpRO!`tV9`0@Rlp(S_sy=z?oCxcXNU)HfdgDq4i>ls>$ z3XMSk1**xG6bD{dLu|}?2Tk41VwEY$poKNKn+OKnfhOjpHgj!VWahPnwMl12YKPbY zGf61;Yq#MDDXj=8(V@C`?p+(VbABIkht5vn@8>bCVNQd7mLLW zSy~jY4Qu{;>Z_|i`{a{kTH!Yk`@**5$U7U{1lV%f%BHV?tR+^$S00(~nDrK#0m?kbH7g`jdSwM` z^<Kl`^@`x6wtDKT?-7-2B0#`?uqTVe?jVy-M*NQ<2HYT9S7m%(RjlB zC+2minwAz9f3bQsTt*UU*qx5`1~TIC?uxhjTh`TmonE^mtzkEPXgtqBF6%6b_SSva zTfeGlZ$b6;W%c~2UCUbb63#7aDM~+D`}d=Dt9W5*wl8hoBaP))WzKM|>SE+#kPhq= zh9Pvj*6l!BJO;008FT@%C*Ks%3b`%Pw)DzFZqnyVpiCEv9JhioIIfUP=xgCNoc?eG zQW}s(TK_&h_23o9ny$j)-MiPVS#z{93Wbl9Wp|fmAFu!5$<(c|OVI`KG1SJJMQP>1Tt$PLcNS%y1aTM6eFDvRc8!oad7@M%3F(KP!VLbTC(d+d9O}K0^m{Hi828FL# zZ~e%)tVIB{*e!vyMFQz?b@oX6r_h$;+I3A$&BU%JdMJ}V-}M{dH4dHGXi)p^&6~To zZwG7KE1R$TYx0g)zx90P5xUB^&z?QJe+xg3%yRkcz@Bv(B?u;8qo8IStBed(=M7dz z25MR4_EqLw8mUsPWR?_cCT8KER2BXAp3nbOy<=JD0R(epaHMYrw}siV*j(xfwDcJh zt$fPDY?sI|-QEmrj#wGA?d&Ct~r@8rd`#nHTq)SFdj0xbdscKjU}8i4Pz?dHv+K z&z}Qy`~`gNHCv0mTy}6<8V9}{JCwsWPa&x_68S_=w!A-0H%qMg8N+(^(fvCcKcc-M zt$9xd?*pM4%8aT}0i!X|W}Iz!QNUtl(4<$&sxa4JEu-!1wuR_yA0>Aiwe=ktaG7pY z6rQsMfREhIA&@pl`?c@mz$=Kg11(L+56pBoOP=x`emPnRR210OmQe&b6xaFo**HCps_!{))yJgPVWC?9=tK;mm*I(D2;ZM|W}1GY{_x@!0hb z+3+2^)-I{ovb1eq7G)8VZVdRThCFQex!z6Q!7?*tS}W{+`tHAO7EcUoFAQ zcb8-h)a3AnVENB9LvV@J)=nS<0au|{m+~ez(J>n?4{i|aZu+O$YH^t-UgnVRR&R_V zG>s6)S@Sxv;{&Y68Vd&MR^Gib3f6vw4E>3CO?H+lqp7C*gwHRP#qnFd1(C+wu0yGp5OIFR`FL$tGB0-cJ)?dvC(y~CTFNN zPqA4==G3ioaNi(S+Y&%?uRXAann>wJ3f$DbaoO3p&Y_wJ|JGPzE{GGE<5CH$jNpXd zg3k=NC@(K;To$an1InX$QV7$vkh2DH=&Y})u%*8Wu}+{nwa1(C#~Shn>;5`R0$ASj3t1@Wn`2yo3#39M<27{tG?L<{rO5c5Zm)nn^sVL#+Gz`VdmU zy0;>7xGeJs{Rt%*mrvGx``m%n{qFS(2R7yJTfeko)0-sV9s47PE22j$5m@pWzrfX1 zn%P}m_-y8pK`fLxsIj|Y`_51QP_iMdY8wZpXLgrH)$vju?IE)($1>#5??kkQ2R~0P zW~hO+crCacN8SMm-7b2YM|OQo-vLmetniz)3N%7opgY;UBPDmVE_bN$y+`+N{)|{B zI~Oy0UM)>~Gx z;mwN8Z&q)UqS9KN)wVCvbMQa955Cv9C#P=LvW8vhEk#+)MH$U|mR%ZczH?>t=EQK# zuH_}`-#D}_15t&oA`QEjb(Fr{eenI311suxW;E>1Y%4;P%kHmfHfZz;P#L41lNDncm`1&lkNqv5-M=w@_uA=a(+_V>oNOxjpZ#CHQMqLa z^RilsGFtcQ%cy6~d(v8^r13DVqip5Hk%k91FU(FqoV;^&sHdcO-RlRpq|xkjxRe?w z_MBWyaLZ+DSB?ZvkTrG({yJRCs>_lasm)x2GVh7LAgpD~;W$_$n$hMuTo)N`T{ZjY zVFK16&@4jqlNTYl35#DQ*1QkNK=Z`0uCJF&t%w{g z%VxV=cWH!o1FQaY8jd_bZy{XoZP~YT%^xZ^r?u?K=q;!F#d0yE4>R3jE6f$Oxn$1` zdU2Up&OiG5VA-^G=Lv2RU(Q-$xM58kex|8lxF&m~!earh9diOs6Y%N^_geg6EP&z|ku@($I>N`md;Oad^POwcw*>hp$c z^VttYuLAgbz-D?HlG?L#qkY>}zj0vwlDeH~Xs1*{Y4||^M^YnxyLPI{U&(d;!- zTljb@yq0Korguk(bqMP#g|&++rTg}MVgCAmzxjsi^kCQKk6)|UlG(gBqqkD($0<%N zY>HBnKGVJWpUxReMZ&xG=QV@Ia){1l{?Boezo~`Ci?8H zzzFEY;DX~M<6RAVx377la`O_Rdb$=*HspdeS(3C(YY`+!qent|xHF3DG)^?Bn>YRz8ssya#f&K;a`iGmpbpJlvJ&JbjB8FnM z&CBB(K0+`d&?2#-U?JU!)=*NUiGf`K$}XSAQg1@;;>~~A`_(eTI{QR*E`kg$m=1x_ zroW<-PTsy40v%#~w5^=Rwz7>&aPG${X>Z6q)0EFHQ4B9Q;%$x80Bx6o zAb~Y~q@+b$%;G-T^4_;kXQB4govSD(lGL6*2fMpBFK_=ebKi!H>TOHmH6$Ib&ppW> zsng^NJo@y5n^rHIb;Kbxzxle7FLr&tq@aE+nvL7$2=I9IEkG TUA=hmtF>#_u37WNnl=9)q(i|! literal 0 HcmV?d00001 diff --git a/contrib/windows/Inno.Setup/openzfs-small.bmp b/contrib/windows/Inno.Setup/openzfs-small.bmp new file mode 100644 index 0000000000000000000000000000000000000000..f80d9e7956e30b874a54c8430b15172ca91fd13a GIT binary patch literal 8965254 zcmeF)zpHG=x&L{8gPCythJi^22Mz}ZH<}xWuVKK6z=0EULEyk(!F6&HfxyPe*A~u% z@P&(D499#EV=S3Tj0AFW?C4lvgFwhgfdb)cjMF`Sjv%h%%-~RD;Cn10U0tg_000IagfB*srAb-Ayoqb1Q0*~ z0R#|0009ILKmY**5I_I{1Q0*~fdT}?4GKV%1p){lfB*srAb+0da!@5M_Y?0tg_000IagfB*srAbZZL=;RRaM85I_I{1Q0*~0R#|0009ILKmY**5I_Kd z0tCbj3P6+v0tg_000IagfB*srAb0s#aNKmY**5I_I{1Q0*~0R#|0009ILKmdV3 z2#6aDVo23M009ILKmY**5I_I{1Q0*~0R#|0009ILK%f8taf1R7Wq|+!2q1s}0tg_0 z00IagfB*srAb1~H^+Abch_XNc0R#|0009ILKmY**5I_I{1Q0*~0R#|0U=RZ027?$4K00IagfB*srAbl0|5jOKmY**5I_I{1Q0*~0R#|0009ILKmdUP1jG#rK$Han2q1s}0tg_000Iag zfB*srAb2q1s}0tg_000IagfB*srAbkwczr9=j{^N}w-|hbW;MIS>d-T1RuKN4Q zk6wL{SLC}~Ctf84N)Qk?I8O0TfBeIj(FrYwl(H_prz_^8VzLKR2wQOaJPZKfmfX$+f>_4xPse^$4(UJ+_halXo89 z{pXMW%^wDQ;EWIafB*A;)NPD~bDyzY_hzHQCYK!o2MK)e{(E=v_w>ozchN?czhAuh z%2oY-oVVl=ukkw~^8fh5rw6a^Lj(}$lR#)3p>>4j7aHKApS0Fnm+Qwfd6X&2w&;>{ znLd53MX47*YW12S1{U#wrF}$7iN>&7$xD=&T79|nBjj1Pnk z-&2~gUH4|Ap8QE!^Ah;`-~RGUw-5b3?&*(TfB9KFpj996!|#6e+4Z9wOMF3GA+P@^ z83LhK?CL06Xot0*)fZhR>b1laSQ>#{0=0ecT>dR|!X-`+@wU+K&y{h@@1s1SqlShm z{OoubgiwcW*YT+q=uX zcLFhwU|Cz}_3ytp?3}RI9=-5vjzs((iMWV_00QL*#JJsMyk+Qcr?iO~zY8YDIG6Lc zo^opK?=|BFp`V6dkEESOeIf=TA+Spze4?lM1SR^NC8iebTv|_nuDqUAk4Lg^U8{*3 zh#TCZ)6R5mZ<_Q>wpr@hn~m!B?sD&qz$|fsrM;edjWv#9-6iu8dh=rCZA~D?&_|pg z#uxXpE<*_&rBoiv8WF+Je1z8CT=)8=eCFDGYQzn~Z%1$UQNPfKtYuWUePqski4}|# z7rV9}S=RRwQ#sn9TK-}PzcH_Hy|L9GVp3!!_*{E*sF83Y?%rKwe z*1jm_TEuS^`!4t3(aP9@K$s9}gBjul zhwKpTvn0p59w+x(2t*$oy5Q(>u|s}-$!9TUK-~MVr_q%Ua~b7R_N{A4af5qZ_*QMh z`D3%}O_QF{AA7S=-QHd9SqVg}AoeGS{ex$0r?=EO)-;OWE#@I)^={?biog;pu=dbb zHd}isSc@TS<>B?ObBPvLsAFj#k7Cn!adbkgWc6kfee!Y3T zl1Cg!yx_2(U)tY9Y%%5_%%14VV|xJm*0ry=L0KK+RvpCodF|1gCOxAs^=6~Gy}R67 z5?Epdu}=StZE?#wFLMxrxrD%o1VRUkaf6p?1&%MHxt!x@OlP01?wnm~e}ApMok1I^ z&F0qpW33izi}l$qXI*paR-a#Ue(Z%1{l~2F?hJirpRKZwQnPPeJBb_Q)v34CU0mzD zAM~b4&uG)$Y*e>*mwO8W%Nj&8w!hYOj&+}6PKPy&T6n%|mL{;wEwBd3HroFh&E?!N zVXK%AGOH$2x((FM@0>kre}8G*;7@=2gSkfkdRc!kvqoCG{glo>+Jm`tm-+WCeN@D| zqfePB(Ur%xHv87KueiZ^y|}d&;`%vk)SD(fqu=yqqq@Dj+_xeSQH54LInu2YggXwLEM6XODJ(FB~ko&K`VdKuGcymQHxQ9iGIyKVjclDI(zJ!ES> zrLGt4@e_^FvD6TRxZF`yzfY%e^W?KYFSy`0h3W?0Xx(b7zYiMqLk&^LP1q-V5WZ#FvDKl?s6fzYL6eEW=Tt(Q8Re~`NkDcwDRcy1RjxJ>WpK_8k) zlezciKK^U%KWnsXa}3J$3DLir3zVyEy62Vi?_2w`8QMqG@m5Z^o}a4)j#V1=t!qzl zgNNF1FItFlZg02VH0c?Aq&FMY?cL>Gi@*o(zjqhDHe(y>vCgr6QT*1i|3R%5A>UgF zM7$uha?EQat+_M z247jwieM#{oXT|_`yg%sV@Bz5HHB5nUv67&gCI~kf+U_vsLZyPsI&lpN!hq zpE>`~m!$0p#C#8H66L@5MNB95ryecQl}EGa!}_rALms8yi5v8*fy}52x2$t-nsiH6 zt*+ghjY{|D@+(hZ*}EV#=T?60x@GUvu@6P;Uo9cfg21wW!OWVSx_{-_920A%v}l3% z*KF;F$KnR&A8n(n#vZ)0tm}U7_3i$Pc^>8%Y~$Od`nE^kdugaPLRW6r{>$?Lm&y|ezqT$> zJKrwZenyykKvyo`#xv!}zICl3ZZL;7(u;RIQ|@g&`hj3GKOYSQhzy?SLkcjO*z-9%6)qJV`#0|s@a`X`+uIFB^ zwOpQ)b%}ELb;|D zOCZLZ$C~%2e9L#^-z&oh%sp7!Y8-ctK&;CU&)6lO+2u8!4zq95@R@k_mFg4NXTSCP zh#QC*W` zTlQK0@6AS+^wo|nM^o;iF%|_Sv+|Im~fIW{N_G>ft6Km`I`uAVF`|Q(?u5~r%O++A`n~fWu zOE0-bW4ZR6n0pXwg-c}#Tx$>K&C%Ay4lZkNVg_@xi~asD;-9_TpXkb$wdvlDX5YFN z6E~ottkqCvKdv`TdPaNoW}~{jyWDpPg#H`;F=M+>=j-qDKm6`j?>_$U>8>mJAAtu1 zVjXAWhUe5v_Vtt6?_>QBsT_gY_L6`9{esxRy7ne+-oLNk(qHuwJD@ACYtOwN#lCed zCvI@PouY@F+*-*k zI?J`^TDPcN2Xw6soHtjy7dtrJ=G+J8Y7fWyzaC--bmh}+x%Y3`x2^@n4d^K~G?o$b z>`jv%(FW)0)SHd!_U>{w$H4vVT%U2t_hJY7l!ZXd$v3`|R&q=~sr`AZTNL|mOSciI zZKs~{fAqbV?jj}?y36txbE(45cK>28=jHo&4b3>t9evJKu#lc7Fk{Rh`rq*RHGgqz zPyN8HWz8Nth(7DqI^KHz^R37yr3kQZDUyft5jUW#T%w!Id~9!;^o+LZ%|?fNWuKNH z@bt;scd^FIjO{?3>)qGI4obAnzVomyk$Azax% zDw!h2u*4QZSM7<-xw4AYJ@k}OV+Nsphwnx^iNCOWiS_%J_}gy#7%{F8{Ic$q9QHid z&TD`FuoZb%k^uXbr1^Yap{>vu>S+wvj>EP$O?pP(>&-^b_sBlEmB8Qs_Ln>H@NxRx zjQyC{!L6;d&-1NIBwkQkPq-%k(6>r!zkRmQ%S(L?@7~T?;sl{pw(^tEOGD4;eU4UU zJ@j(#zwM0JYo)igaIC!Hmk|SrSXwK+JK9JvjX6KzXUEFg%jdzgRey8Jv+2sG==A>` z`_{GL2k*ZpZtxO~C8IWztxUaX(%JZJQJ&sxRJV7ZFZbBL_Vw?-=$_`DR^1204(94# zLQCjf^Xf%!P|nNaGodq;7!NyFtG?@An>*H4VhnTWC#^KlQ+^a>in8@O&N{0edWw1J z?;-|u%=dcvJoNd96J*r3!(P#!_uNjx4l^c>~9JZ=!o z!bfAwsKk1^9d#g9UN4*j+_JA^hhYwG!a7<02S#|n=5R4|XW-P7-otnoFv z@-Z(P<8$_{Z%5-ZX)V|4CNrJanV3%93|3{k%tt9KX={1&!em44to@_8< zj4phv$9W>X#nVo;ndslM$J(M#${N3Ai*cppIc4~5mzhuRWnX$Ma*(z!^^d_~q*UUHLUZXW^ z6@HsV3q57e&}>Ifp~3gikH*;h%wyea`^mBo!YO~P`CY{AvnRT8b_?BBKK8wh-`caS z^+1QYRxg?5yxuhFS?sb~&)#gbwtsfqJ%P_Y{m486dl{r$n=1C%koqPtW6U75<D{uf8!0}p+mH%)p*`}Jm{ZF=`{ zt`hetbK@TNOEb3f%yqU;a=-n}*3a3;^uaAY5B)sG$0V8t;^P%Cf%A>i|hAhqf7hf*^iI)iNp=ur&{zaYp?m*qZjOT&|yb4L(CxR zIP)3~$J}z37=qZr^B%D1zL{TVSO@2M%RK){#5lu;v)XFJ_4GrJ&&oT>74slwjeS4g zLXLkDntshWBR$2<=*q|aWQ1?ncRMS_IJvdY#*U-i%;p2VY0|UVC+gdqjkfFE$GOte zCvUq?S*Iwq4?KST<#8I^1T`5sW)NdBW4tJB<-8*YqtIwGYm>KXcro@=x+ice&zk4Q zTo`e_dt1eSnPUf`%Z`>Z_k}%a)mi)^`jgW0c6z&~(3P`zW1VuaZym13a{d4Q=l@9k zJU*iqs?IrLG`(rkGx|+$HagZ1`}{P4u=9-mtSf)4 zEWWT?ju>Y zZ(|Zl)%}=nuK9e8jU00w0rtI)+qGxKIuJ3>e$8{SWBSoB*y?s$^`=SBX!G7|bZtL9 z`+4>`xV=A{v47aI&as|W{GO8X5(w>K%`;w4$Aq3nZ!G6?E%uk(!r#OWKKGF=SYLV# zF@tsOcdK>VFR@>bI_e?Xp#^8%`$f#)ki{8SJ~gwX6d{ zli5;V>E&9zY0|xLEm_XqY_xsvKF$?mJa=2pC5JBJLdGQ)FeDc_r2hbzUgL_T5$m$&1Lv4gN< zl&jYi_mgb*M{T37H!b(IwRZ0{W)KuJ+`sEubMD;Pw~hT{e*Np;e~~6*|4BaZg%9+m zNpGc#Wx7spHfqzmk8{Ud?0aqSsjO{+w*<&PAK* zF+~ifrk`Jvx3cSTjN>RT(NU~P)YgBFU40p&d5JBv9UJkq zsK;z6gB>3K?5DF~uTtL(deN>w^{97e+iTkX?Nyq})R|`JC#E()<;t^fWxy-44=pak zv$rzPhFbbSZ<=&VR;{kxn~hd%?2){pAy+&k!?VP$rgX{w z)+DOv7zMMAb!K{=&oZM%cS$>}aqNuD3pTk2Sw-CI*!n6pY=+u#s6Nn}COxVhPSvY78#U|H z2fVPkaqc~*`i2bOnRC#l;|V=?OTDAywL({%aU6IXPbqWN49~6C`e3a0GE=&iQ>pXp zQBv01vT5Hv%{j<4*I@R2SIV{ux$P3jaDUz{kN?XOVBfMlug&erJCDT;>_0iR5A>!< zXVA&EE<p_}xi zIn}mfZ~2G*-s8RpQRY$Bv0mddYujV39&3GZm#Cx{!Au;Rl!B37I! zX4gYsp6Nb0Q~&eObG_Sw09|*7{hB z>x&ztald8-i80+Rd~cL%Moc`~(V@Rl>e#X#<{WI-HN=?j*1F)O*N?R#tV^_=k8E+J zOq$$Ue~f4E8PZxFwT=@vsN-cBw4=pmqWx!1+2SlS&3DS|7o(OfL*Fne+YI$$-wal` zFN5)%_i=+*6C+|N5&{_swA990U0d8BqbF`#Ch_h2d(?hAjF>>^)-g{m=1;}_{Mz_; zjGK@1BcI4G+TQ;4ygzIUa}MhF386h?8gsg6NT*({%~0PjT3sR5oa)UVdl)But36M( z$1T1SH@MY;htJQnK3%l?%qi|WTlo5#Ys?a_%-H~hP_NBKtgf|vm+NQVt0vm{QqGpI zQ4R@#+63szwF$S)pMBf%KXmmN&l%$uXVfJR)j2dO34v?`4*BZnpNSh}^TKxJnQ=d! zxGzVX;MzF^HLMl)M{7qtj2}ciMXE<2!*j!0{YA{6o*!h)IbsJ{bh= zW5f;WdRLaVmUYekOl$dR3w7Iq+&?Ff;rAh1)FZ&Y_1NB$bLbMIYmjT}7aD-|if-{H z=hh&w)_+GjPTZh|C-s?UtOYZ}c=xDdj9<^b_D$R)V;+Jr2cO3ff`wF`KqU)==qbzvR#sM@=Q$^EG+VO6z)7ra1>0*X@t}bguE+(&uE2tJL*Qxwjxd zS8hRlyKA!VcC0VGlJ$!2#}LNYRtbSj1V)MntgVZpy;s$lwfB0>dwZXFO zUlVL<`}f-XwS&6tK<@Vh=*ssjM*NR`b6Yi@eM>$|_ncQ~R1yMN2=v^Rt}Qz)E{pf& zDaUB{s95tS(>g@gS~}(*#Mpxw=7b#U&$7=Exz@I=&dcz;utp~lGic=@xvtw&>>$P^ zkC@i9*P6%BOY3`8mit2V` zbttc~3Ey73SaYCvU8Yp|L;sKv$UvZ!Pxq9MxIqTb%U6b8*VG$1Ua;#W5f7Ph?_|Fh zcD>-X|Ff*qeCQcs`Ap21y=GW5uajb1JQd--7YaJ(U zu#Gpx-rlurcuBrl*89q`M$u^f$u{2XDz^}zE8jvd|GDg&-$qfw7%v!FM^-&?>vDbY z{(F5GPAP{au(i);xSqJdufYaTL{e5-j=$G8{FbCQ%v@r@0|Z-*fG&wTdqFkcY=-88>-KXEdhrq0PKIPM`;Wj_2}G(y4a4 z#dqQc+j&w3tvq}xgUvIw)0|Fx|IN3`Bs9t&u8D<_Sr98W?kmm=lW~Ui*b_@ z0<8#K>$g3hCvMQn+xoa}XwnhW&E(_Zv(~D4=s&?@rumEFMGx)2_FWHayU<3~TGMe0 zfeg>Rr!@1>95e0jx^TPZ^;A8#_%4Hnd=0lA&&d*#Ykf}$pWDK>&s}4bKBULCFmstR z++SzrU%T$?Tbpp%{AU|0cus$M`sD4hfaQk3N&@G6bHwk(4OTkG#D`^>7Z>ph_lM(- z80+_p*hd|z6F)fa2dBQt@O*Ge8?oo(DeL*}6oKq(Z~f#)ubwJ?H{YFVzgv7KZm^xJ zWLc}I^?f13eX})#?Yz#FvJuGO|Jj&~R37&2&ze1~BYo)Ey~bxTX3|)~{(N|Z64&tO z?0JeCjNrNT>(FvuoIMQI+c(;C_CBXX`NAKi%Lv3+#SCjL9Mcv@TL=C!>p1#W0zJ&7 z8A0pS26M)Uaw*6!Q4iuYR)pet`hy3KXiw+;7O zui8?VDe)X%G7nnjt={DPYY3F|;a>96)~@lC`OayzUeV*%U$!RE1@0E(@O$ZdZYigJ z>jJ0v;?NlHyp^_h&2{L3m)q1C4+(@1UE_OOoioCi;jLI5yN-V1q0iTT_xNW&P0^-W z+WnX>Zuxn{MI{7U5TGl!puXKT*>^kErLQ#0Siv=VL9DqTA+RNZHGVnTG4!`BJ*@9* z#k{z!*3geNiDG>u>2dwGj&VjhndjHBi*TDKj z>+R^s^8~W5R}}u#}d( zR-WaVe~k|N`uAV#+n7q<6@fKAIodJe23@&mio!y=@9uIM$RJt)Dtn zAA20EbC%fGU@L9sn(NR3*V)lg>j`8SBYdhY^z8Ge>b1ppJ+oWy!MTpiu;+NIcw2`3 z9pWA?A<&8dUAYzcZLZ6{+im!2fFrgf?C3 zJ{9>GJHD3uy5maOCUc*t?^@>zndchB^NNH(4FVaSg^%gzJ@0dHtWH~fE^bhxwVci~ zOItc!$~xahykaZgK6IT{@sc_;<=P{G4DG*1ZoQP5eS5Lg@p6VHv(-A%nRS7sY&50g zzT%s51Qvgr>6P|Y&VOdiDf;GXH0otNqF9?xDo-Hx2O7N}3m=kB6Noii4rvh?J_|j> zUXQ2k=iBQEWM8i+){XBmweCB_4Yv21VBXs9as91rpW*%)aeoPcmIUa^E$P?4Hv86R zyu}qlj~V^Be{KC@O^k@8NC?yyA2HXiD#wFqQr2es-Z-@7~Z9mfk| z4CkS-o2{SKTU|fKsY(dcBGAgOTV7Y(pq4ku7lF0}qHVYATei5i-vq~fEXHPTt?8}3 z9xdd!-F#D)K=|5Pzufw`t#!~6WVdFte?4)75>~uCj|_VwZ@n)_>phVp?k^$GngCt7 zwFT;4pMC4{-8%PK)*D!(F~nLI)+^fD1LwKk8Xp|(7;%Gn{98@uB5=*V6TS6sF(*?x zOd!Mf(0$D>-p4bFgus>rvd%dOT`=Ya^^^{^=`BAKH)!K68MLMFvn^BW`FdN&Nw#F` zTKy2n(DwVG)n7^3w?CVmE3x&8zWmQ|lXT8=-s1af3G}9!l_;Y(qw^+qks@|~E#TK^mAIB|oC9bVOkCJ^_@ zk@~7bb;>o&%ARRnejWVmd(?9-3#Vx68t&(5=b793*zCJBL( z1ZHbTQD3@SNe_}20(l6;p24H{Y4KY?dPpFH-f>9tG1uUsoxIzYK;}6ITWO&sh#zX( zTYlCPs}j7kis8Dw*y``OxU2CiRNQSlkvigB?%|#%?eRD42+t!wSx8=R{)n?f1aIG$DPsFXg zZ|v)>^}CUd6E_&!r`2v=0<**o{{FYW%xmNWHP5&oa&K#~9Kh%9D&wtA?__wJp$}ok8OsW;~Cj{Mfdw8 zhc+T1P=dfn@qo2;5jQB|Ir2atAAwoo27UwF`(CVvyp?`;$#r7vri4HS0$KKV-ETip zt{&2Uxwn2VZqUw4Ld)LT-uu^!y?eGut+h09gDtG=8fyr&ZVS-|Nc9M?Z#}kIa-KyG zSfhQu_UMH>;|;TTX5Zyl<8PxKBW}>Q&ni6v#|X^wJ3-vw-UDL2lYO0~^?Umo+}p>0 z?FnR^bMT#Sf1^E|6(wlhpNJc*FjA#MGQ6E~={@3$);s%{Mt70Q?hJd)i89)Pm9U6UWU2p&T^p61@ zt@A2njITwziM?mDr~UT5^?PxHb{>+U-E6hyowz|eKXc_=1nA1S>`}Tj>|2^>&irDG zee4a~vu0Vc>`$M(ozqZC)eV7?{@6=i;s)Kgr-CAI5rJ987MO4Fxm#piqo@Zh^rD94 z;O+^8Pj0Puuen|i>v^rIN6llz4caxo4Ej>|))uLz53k9apKoDh*H}X!L%Uy7PRCqF zfPJsy*5g?pfB00~;HA&^I*2_VtXI^d7w0K+FIs=ea*7+|@joR&pd^7=;s#Mq>FzGW zZ~uKgWUDnL*dKvQ3A7$hyx&&h{Z?zYU&?Ct_(SF1@-uqst~Iv(Kg&I4i&}5#(}%9b z&$qC$YpfxVq1~@3r(>=oz`oaU>-DTzw1G8RDGjt&FV0c!H9j`lG1^QH-%|<%N)w2- zJz9Tts6OHbpLsWbX%R+d%$#2+)=D*rG;B*tZ7N+q5;3fhy^j}d&af7?NKmG9!&2^4_{bQ?jsBdZ{PM(@T##meSwTkxJ`PT2n z4cax?40?L_TiowzrVRD0SwDFra0vmr@+GwTJC=R>v)QdB{_N9_?qZ#uS#*L^^$ZTN zev^d2r36m-)u`Wz8(iuojz=Irfm!xMj(%C(Aml{0&73rzVi$d3j%VkG*Fz zr~UG7{a)Olop)r=)5G6t`#|e@zWDWjX=h|tMqrOXhWq4Rk-RNPfPD+X>?o@z?>rVa z_+qbv*e_Z_;8Fsk#s*H+OWfd6FL68q`3cOlexbO*-Q9P-{f%_<4eW=Uzn_@Bgc;}Q zWY-M0juY%(ueN49dz&~_|JHi?;tREXpmja%3v|j0eYZ6My7Jb~Fu>)E{CeQ4nET{DKQNI&6=)*Ua5`nb@{{FYWOf~;-_IrKk^NXI6=L+CI>_PY^fA;O)w=5TGkBB*uJY-!bjFwpKH( zYkW*cSU!L4(F=DGOOX&*LtycnnO?;W)_8_v5a@?M-1lbc!)6~J@Pk+XJzd=3uYdio zej1FDPEEjkoy*L>Xt%9iYiq`r(U|M#EN!K>546_N;+`%ckcB|&_7L|PsU89Lt;e>P zoM&3w_?$lSi#K2C#cayin7}z-8u5E^gT_wh@(2_pFyp;{_PD{fzWKFOaRX}<6>QM5 z+Jr#HwTd>W_!Spv6)#w>TU#@}0?AaTEOu!5TH6O&>u7O(34tsGTDOPjYovMv*tZ_r zUUH5x_OUnch&tyrb$a^b?Osf$oDB(F<6FI-BW}>p;anPlf&@PP@adHI{F&nhS;l|v zYDhGXf{j{Mn-IwG+`QX{;{V#(=(S+?%enRYtX!_;+1q(r+xyV$(EmG)ec)SiEG!^jO%+_-&bgD|1Er= zwT_mx?f31uN`EbZ*6ktMyi|_>`_^OITh24CS#V6}rH}UJz1x@fnD31IxpuL=pSdCe zB?-(NGl;%<=9J<0{;rPn;@AIWW^<@}#{_o$IK%&0_eO2aM%bQ7l=e9-#}4tb1Nkw|C!qqYPBJ3`yV; zUmE3Daf2aUS*0c+Fni1(`r(;VwsqNm@}pO0Hix=*Od#9-;#p&Gt@Xp&nsLW2T)pVQ zzP0`1`n7!^gN~Mgvoh3)6`bN4b4atL4DBHslab29z9U)l_PTub=|^|5PR}Sh=PmUJ z4zYfdguo#JxA<1C=ZYH~@&unDke`5?=%{YTEW-Wub!S*_Gi#Fz(ZSDNdZ+@Ms3!p2fGDq>kuy1js-DmjZ zoyXz^{}`7T+HUL@Eg^7-K(E^WE#;nFcRS<o1EPd=Wc{eIFzQ_6gjgzxR5sxWT>`c#lAC0nj>#wUJ0=ODC;7_TWIuuGt{rqx4!;s(1O;C}@25{NYi zqrLXfuWVJunfENpwjO(wM?#=y0j=4qR=(BqeD>?S_UMJXh^0sfECjOqY>o1X8!XoMia=fhF|JhShWlc>UT^#>v+wvezNLntKP=DOv%mi0%~x(g>D<8x%-9~p4F+>!)ftLF z=;E;_&y0Q3nmWfEj2TnLwb^=r*S$06$nJUz?ww&_vu6ZZp;~+HD%FncJ+m9nXV~r@X!ZqxA~d#&(N>W%)f`afBfN7;}^f!BRST7 zuwGFQUR%G6hqS5DKZ_V*Xq5iyGMIH-bp2L4))k(-^H{v#i+l6AzIW#Nf*F42?>=MO z6M3wDXy)2vTmLKD-pIS{zQzCI2JKpBmita^0^$Y_HQN^LeD-T*+S?{WyRY5mXU?B} zXSV+Ox?8X4ek`Lg*X#XevW>+Oj=t0#_E?`(S`XBu}J*4}ORE&dlbXxHYl+;?gd5H}DvXwkRrU$eHKugyP0 zyI)&Y$6ZclxNTxYsD&ed~=`-9Wv+AV9ObZx2|MWg%{$o7p80bKJ}=xUG%F zyvbQpHf=MWJ0t`~Adul1dRITnHkW6&Y%Ts5H)z+>e8zY7Z@Z+cm{2c732-OMiIf z4EMx`CGxH$0ro9PbL_lgoMWuhLnHk1GY&KB5Mz7#y4q<6x9NEJ9p@)*u&xb`epK9J zVtpO=-Q#U#O+OO+W=^4<>^@(%{-2gNvp0@a_l$doWFoAu$z+Xf%aS2(u+{vV)_(5# zVhG!|?&DXZD<5Yw!Z++Yf)%%`LyUP`^PE1zF|pRWG%tY}d|VxG(<`s#CvGrrzw-3S z+hPSP_bgc3z8=5+^1Kf9kD6!G6zqvSwL1<~`HXubN13vu)_so{WjjxpDQ=LVuC3de z>$i^qu8>(?iq!Ph>okA56|ufD%!y~60et$uA{2EkpwlC{;B zFW0gLk$DC6+S|-~8)Vcq%eNvqBG8UNHoYv9R^FOTCKl~Hw9S=U+di%z8rK#{+@P&y zp262zmthM(Uvmw*@)}D09>c!<+OAc}L+7w=!o9}v!TayEGKK3lBtTccZv$AAWg%|R zuwid-=~#a#_Q(-~FWYX8^+WNSW!4nBS?lN7-Rp0)Zp5tiR`2ZzWLpR9SHJxE_7IL- zakuTa_+Q-Mh+#JPxP`sj3V0Q+7; z%{if!AA3fh?eo_jy>K_PE-`Csn9a}CcN>i}IEx#M)vK6OuwC2P@26&8qbM}QO!|Si z!B~Bp+T7h`8*99kR=URat@|EvgEdwve@vEm^;T^mgO=91t+{@?I7O!4RT*`%!0~I&uzb}b71!2e72S=N!-)4?6j{PEL zP8ojV@9GIr9tnYw2>j$nucnI4{ovJqPlI>2?YH<}+@OssWZ5r#EBj}OleB1KuGubL zu&@_5cxZz*z1|#G&G1Yy2kG%@8Dl7EBKLa8?E1ub9)7w;$8;*6-?s@Yu45r?aJubf z{MMX;>wLXt9}(**&YrSqn&Jktd*pca&+r_*=tuFINkiH#SBw9*(v8RS>f36RrA=&Q ze{q9{x^0WLKJzs*_4ygv{%yUNvriFV-&1sacaA`esjT_j-&5Y9QCP3&oX3pweNVor ztoby}Q#KJd=%b%l_9+lQ&)L4TqkLaLPLMzWdfUzt&7c zU;Hd|{4G<ABY>6Z?IKc5I5M0^w`&laec8)&)R4Dk&g=wv3`X# zHi40SSl!y{qv8f_`;NGuMf}(LLv7o^E&Y(W2A{W~MMKH*+O}o8Is$bFWNBYZIkM>G zOLvHD$*T4Q(^O!;S+t&Mow-{%>+pcnY!+fP`5jP0m%aZ8ISu9bf9PC?% z>x{WRdFQdX!Aqa(ETK0^V-vVV({gUVdgK<9^$+g158I8n!9yP2AyA7zOTBPkGl}={Y@Z<=D%+Nywa|_;aBo?L40|Gn zkF`iGHMQU_wI{H(4`dsEy>% z+~6zE`aW@kZQIxg> zKZ}<6kdk)@+)kkNv-EB|i~qB}sCiXbE z#9QkeyY;;w)&u6iAD5IBNU#7*j zUd0be`+&H?_3bj&A`-Knt4+GU9&Rrgp5Jygoh(}BVOl;#;Cceu+F^FP?Z3tU;s%H9 zQtQ)LW50!+_pjOVUei)nv&TU#&t9MJzAx5(-_q9)QOR&GJX9{9l_5~upU>s5k1J!x ztht2#Ra#T)HNRL_TN;NzuYRc9`88E>gUkDgh!>||KK7x!*K*!{>y76D`J_t%HS{le-nW&RS%@2~ z?-Ild=590g4?5mfGCaTSK3n*`f818S>6<{dwkB?1e}k=CRO_|@Nf6o!)d)jB8esmY>2+XVlE_DwM zv3`@(KY_)jUgy>=YuI1h;C%lO`xjV`XzsQb_P74u@wSrT`EB>P!ted#w(?Ei1hTa? zaRYIKty@%-#{f{l8BU-kQOowsop}zPp}4i@4an zeOt~YyPljgFC`FTLTl)2J?0r&htxlT9(_-l`_Ng%4UYE@5r>XFaQSi{zFf;rV(`b? z$ns5w=eOP8{81hWfe{I0Yir^L;s#r{vzA)vCf;h zYuZQj3-&ZP+(vdk&F~w4_qjRBBOx#%foyF}+(6u5%XXGw{fgyY6Jugqr`BsWS*vJ! z4~Vs~79ZT|_3_Vsx_w2@TrtDF@JtbXeIb zsBG2iFoDp@OKS+d=J(p87p%>E*iE|lv{%1V?)>^IUG%;Sh&cv)HownqWh46_-?xo( z|7Cc7TYl@utGK~Ai}}7+0@>PH7Ts{S{kQl(!qosG*CKY7ky&fMkG?@FPoVXF?_t+W{d;-Bay&*LG_kEThV0j&fgJOa{GVs{E#>Q1 zYZW)RdlplT{*fQ|>&K;S=YDzFeKEuH+pfM7<&h8=kwCV#CT{Tge1qtJGOpz@QX2@q zyZwC2|1;fVTI+0LX4|&1t@roH(#P*p&Tucx)DP_Q>Ggevem>)V9=+PLr*g-7MX@(< zX-%Qm{GL8}yC?H0>lXxi^*80tufxtBH~9F&r|uE|Si@giy=M^nBgJnAX+41q&u_aL zl4rT~wsT}(1hTcYESl18`)~Pw#`Qe<^6=xO-C8d_a|Fw8^&)vo&TC7o&bsefq_d>CQGX9@sExK%R;WBjPvX+4L zite?c*z;5Bi$L#MobundzicdKjT_j@Ag@ohY9nF>*WVX2J->x!Lc_S;md?ygAX{4# zH+UE~i2f(W#T$QJwjG6^wa~*4T{G(%7OnRZ&8jb#^~83$1>ZydSo5>Y>%3>W7m62` zX&0^IpG#ZPmzA++&$)zN-b#FGFV?Y+wK-z_CaDhsrL-~m&`bAyV79n{n1MF1 z-v(m6D(N-?*`D7rKkME`+S#)c$nsph=qFj)+ESiOuXNNhtu@oW2H{^>=aRG-kG|jD ztkG4*=l637pKCFnB(puU{qBsmAa=0tx$EC&S)(Y+9Ln|N99fG%=mYzj!L09Nzi6os z0<-$4de_%@XNnt$8PxZup4#kdk6sWzFLVoRu~*-|Ua|W=>pBvJaLZ<50-4rpi8d#0 z@ccSOyM2+^!EKE-%eobdO|#oG%D0rY#VgJ2Hh({Mb_^BP6LsRffYZA$Fmm=~Y^ zJc~WE{H8fg%ePw*c>MayVh3N$JD5o`+{(XdT&E{3PFefXW@qUG#0>iK%i8@&{0{j0 z-~LjgyPh)7Z0ktSE~a$DDJm^)aM;Xi^qf}5WY?Mx6K(%#j6cmb*C6|TnHlCBJpS2F z+dI!dB?z0gm}`(-Kh2^uAG)uI9bDD`Tg|6AWV?8tkq}r2gjNvi^wicRdddHT_upI4 zcuh*67hh7&{U^_+w7{mkpz^kBo?GPbg3_Pt@V^$+`fe)JdF2LH2cdGuTWS`XsPu5m{cv4NmWGuy3bcwC9$sBXp$Do%h+bexH5(^{&k_ zXsS^L>3RY!_0%l=`YC!De0*)6BzADzl9}wN&pK|W*}sV~_A&l&(G_NYefs3>*NI1%ZXYR_$l) zb+YJ7Ys;JAxERYDnxJ%;K&y3GPWeV}<50uDTC9n`&yZev*vHp=+DcFD#n)T$+4u3a z?`hJ{dwqtvQM>lDzS{FP^2t2`+S0wQ6nj2cujt-4GX2+!MyH(j?Wq?QBgPHHg!}Wu z((NVgXEEPI+K#~PZ|@BM)8)3ay(?`)pw*fcr`of)!DaIe7M3xdzNhi)+j#ihU8Z%e z7hA<^kL#3Y(N$@2m$#nqw_X3+^8a39Jz4zw65B*f#&6U6_7E}jOZe41);#5VyZG-o zfzZuLYZkrc7h^M}d<1&+C*{ts-;NkJ(5L73yIOri#0$dyV&L1~$7=qbzb5Z2dgk_) zaK%vxgil}77HXc8g>}t3*2+7^gVR*+Jt%9;pjSO9^vNiDE&pAcFF(Kc@iqTNpU`Tq zNKd}rW85IxOfc5(-`zd_*-xAIO}l-V`5CO@xOljucWPFpC0job~mpWN45JKy!HR$2JL*omFp15sH@hg>E*e$?r*NW ze&~a}jPJeND(lZ^sg<6yUFciArk3N@ce#oktn;AOv7Vm#qF&<$OS|Za^*Zlee^kc) zUMy$5{boNh)<2GMiX}CSUh@hLv3`@3hd{5sq}=)S+mYf1`tkh!R=Q7E<`>w*p#A+S z(|qH2E;Dwhy-z4XSpwN~l1%GhFIr5muQT^o5#wz&zWSWMM7erjS0u~5ImTJ{&a!Ne z{xG*-({exXp5__^xor2tbA8F7?_)2Q+1fd(CMiJ+YR*Y?W^bolPG1zKz8_i&5J7 z2k*ZpW-vj_AY$6~JKpNfbJ=yWtyP*y)4YtNqY*evAlq7CnPPK?w16Hz3te><4I_KZ zV1&5_OHYzzy`pH7!F3j$bm4#MHG`JgQ5Q! z6>B3~ujpP=3@uJ-Pk_#S?~_=RWid+JAjY5bjk3OxiD&Fx;C|HpUUaGip#$vy7H|2! z$LHWve0_&N%QY+xwdG9u_M!6i{8_9m^T{WF9@6PUqsg*%&M6OV^G zy(W*CUzB0ZmLB_>(c%Wt7mYgiAp9%a@0X?TA7M_}HGPkB$`CMa^ErA(*@jrk3Z*8HwdKU!D z5qqe$kI+{{+mE$~#H_n$f)5J0|9iak`v-;d27!(VgtpeYJ?&pV(^~xdW!vig7Go$w z!w!wH=M?_ilg`&W?_<7_X}zMQjfSQe<(ef$TgWo^BiQ#Ex9O=rS$zGH*RW00u~+*< zd(Lhn^O9%jb?eqM^om$Vc?1onL>-0Wsb&;s(n; z$De)r5#R2ZK(;YO#u0VwOp5-1K=|?{n!^^yJ}6-0H(S*A&^1CAiFFPReOLOkh#$n- zE6NA z)HcG;w}`3V%G)!-CB2tpPZ9X|!>8f~U+l9P>qQv*dCFVre%G7kU!uI?26cUEgxt+F z=s#xg`1P0X#HB}Qji1+H%V()I&kNR19>=$4uSrXdc|CZat`2$UpXEbF~4MaM1aS(oKiLWl1y5B_$UkIZ?iu=fb=;0X=$AcdGBwK@T933nftohWnzy*Y_I}pW70os1D`v373amra zQ!Bf?tgW8omgj)|*Lb#)5Ey|#wsG6p^ws^gzt#I)?ICoe7GqGCx-Kz`s$YD3J$S;le8^?Ep43TTH*%V z`dDvQGuI%0+(4|Lw^ns~d1Eih<#&0O*V}6)2L##@2rV;9ds@m7dO@bNlxyqPeFhnH zq=iA~IiZh)tzsU)En`!|2SdA!d?LRQ;sgt)<+bOv-uL}*?fZ=P?8P4YuS;C)7C&Cg z<=W$dXZCT|%i3z!)-lU@(GTtW$QJMS?|s?&{OFsbjHTnpk>3&{?lD%d>l>N-kSy&> zIj$%0Y?)u9O^=eIziPocuDhQy z_xZc=*`uf?wJY1b8hQ)w153<!4-Uk+xvd(=}#WpXg9u7aCztHrp?2w8bU<6zM0cREx_YpQ`;? ztNY@y-;|-b@B5|qUrEIzVjv^6f!b|M{?`+*p7z7>rXRfj-t{n@c@crEdST6Si5pzx zB2yhKhWoHDuGtQc?_nQ^V+`W+D1Y=v zBc)bxgYcDODZcC7S5Em+@89X`k9n@oYY+&n{Fsh1+vl-P2+gR5mpz~7Y`zus6*qX^ z4}5a>tk0RTJ+!WK>}~UlH($AnxgHV%rwFuqmOJ)5b81ZRse1Shfh+{FtrL)OO_yWs zzvky_IEWqOjvd7PO1dY|`@Dmi?Y-B$M~oSS$F};dc`9xY<mGdJzl|HGxz;(?RFXK55@iN;}4(q=qk$Gh(N}<+Ko`S zECM$Zh&lDy^r~a)G~8U-8!~9YXszZ?PSz3)~(_OYZ|0J zzm6|o+ecWRXpPr8rUro+V;_1-ZOx{a{Ea`Y;bjl=?8O&KmQ&o|p$~ZXIRR@F)r}i0 z>kobQ=|@}+fvX5))Hz3eZobMuPU?z4#yH%$@s=$BU#lnFa^BjwLGYj><e_^$J@^2(%&)<1b5Uq&?>q>sMH>sIDjW z>>t;bUEH9qPxY94^fTA`lk<+p87H4K7{xGRD#9^V_zgjPoPc z^}voAiGcNr?sX^{Z(YxxQS;)53<4B*6 zxI~jB<_H*57L3kGICn_4(Vx z4`yo%<$8HJ)h6)doyXz^FMY0O#dub!Jb^Vj`)J3A8-g6wy$7|Ln`r^1ZnOo4i z*Y{ig&}lL~k6$}ZtX~T#dG7==j`6*AjD2leY~9zCTKg>4I?C2pZq2gx^=L@tT3^Hs z`Rg{?=OyhTe&1y6o6C9M!<>4I8$=DP{Z+U9=I^K8w;^$}Q0AT*#$bfYbfEv1#O zWpK%HThxE|8rQ6IeXKsQhFWMHyZQUSjJ8BWI_7oPf6iaa_-3g#KI>e{7IA}X_OvJ{;V zST|2XAPa#O&vfUW!?UijlEr4qk%vIWJwh_?F?g=+m;S!={H4&5?0HaDN6PH4d3u`b zOA_~Za}Ub;>?QqFXrwWAeb&_CH)g50!D2Ige!2EDTe8?lV6 zX^Z;rUL!QC8YvjlVsF=yYT2iT&3n!><{o66Te#a+NBp0E-`%x{8`M|}%GzK0`=0Ls z%6>h8SfgiG)0z4I(6gj61ZMOjaf33xGfFNIi=43?9IJEeUtnI>DBeA8Jz`yxY|rd9 zp8LkNpqjNH5I&l%-9&jZx1Ftd)zs~1vDfQJanC6|Cb&f((Azab`^Yt#%pr){J8XYWwr8`A2)mJf5MJzsPqC0cs_l#N--f`)A3hZ~_+tOg(6}O&BHc#dkbXG&XW|C8`OHXX>r+2pqbOq1 z_AnU9v&XMXj6uxy%wBRo)A22+hAjwWT=ygU-kV$ZGh4(B!uQNQIIkmxo@eix7CqJB zYlhEe95b+WA5rRh!7`Xi4+(4?H`wa8qdxz9pD%{?uq}AkHv#Jv-LJ(>|GnLpM~@L4 zsvp0(-ET(8!QL@nzKeT`gg}o3GLDz^4Bn$B%~9rz`+|i2mA%cB(&*Oav-P-N>%bh= zl0wVN?wfP;T+i1s>|*Ve!+yJ_p9$aIDn9wV_40o*YTRHUug`zJ?O2Cy@eZ$}5eVJ$ zkRCeYXV%rY&AVsx8F7Q#d}gGxpS<&U>b^AUFCow?fsEr0<9Sm;AR~c{?d3b){zgVr z56ZMghbwu^gNtwUW?2U&V=T5L+ckNGPtsI3a=aTa6L!ithiZ+VwKyhx+x(gHPPcX3 zpvCXTrO$8ShuN;#g8z9hbOZur2*i34u};s7n(uy{gF~z}B3(ydpXUtk#SO0Wl@ZUH zb^exqb3|{RuTB})Y7gCQ>+!<#Sze7VB@jNDQ4`L-Mp1CyQsZbHM_II_jJ{Ia4l&-; z9yOQtEIE#k`3DgfozZXCvclGU9 zn_Ie$z%IvG{uej6&R0e}>lbgnGF|*t%wR-Mp0iG&ajkh~-ujr9&wz7!f?A&=5c>Sq z?O@IIdf0n#O+88-lcn!i;s>Fnm#SOVe8daR^&b6vAATEs-3)$vu04egOc%Y*cejii zgq<>^rO%%sexO}s@Hl1In!w}NU!G3$I;L$~qx(AVJH~V5&&3U{^OX_L3SSu6*ALY# z_H>XC7==K_HI75GkPyg3Aft}fOFKEI&z1T<^qSCavZaV+#5{;n^~{n_+&i+wX|maT zgz`j8GW6e!eP7Ks5OWK{uQT=~z4+CZaf7&4rZjTwE#d_|j8(|w56UtQfxrLlFV-u1 z>3*MNdf4T@eMnav{WEcc%l%~3GhTc2!c=hs@zqg1dER=(^Ks2*@N4t7KK8smnXTrr zp7phUT=TqaddV@n)nu@ncXl4Jo<)r1?m31KWsf>UU8H#kj2=UX7#P1U=g(Wk4YJ0H z7m_2io2AbeM_a7wb!Gygm5;9TuB~5afDucP))QFEcBJFP4c7a|=tqv^*XQcO???CM zdFz*PUUiK5lMu*2AnU%Ny=b?)+J3G7L-*?`^%O%0{|_C|IMbdytX^5ecVoOkj6Lo( z1{QNDqWtC+)bW$8;s!nSNj=37mRLbAzcK1~v0M>2Okmc%JrC(*v0k~fp1>i7qkl#} zUGE>GA33_O@7B+JtkJ!B-ulJ!b1gmZmVC`QnAaz>)I5Cf79Xy4ZclNwS`3zagGJ9{ zA(t3J^urlqRG}G#?icfHBm~Y8h_)BL5ne~W8I9~tG4wkJ?CZV+1gh~wb*+e?Xi zYaQ=uT_>bj#yHL+&YsK2h>7^z;#Rqct{-{&4wQ zG}&Og%vA_YX!+Y`9^B$wTIw3oj;Tc;+V0}R@w)G)p$jiQyu9uo6YrNiX6*}V*{4Ll zhwOUDhL?O6dCnHs+-)mM9C=~3w2kF8+RgIa^1Af-+DHknmIne430SY_evf3@@I&7? z_HJaY^jO{a#xZZ0>+{b({b-t4>s*#q+ZhRj2DPR3x%OHyhEPJF1%dFrwSKzgaiIxF z2q1s}0tg_000Iag(1t*0qQ~^o**<^wtvA{jp`)wL<`Yq0af6N=v|>?b*;D@U>n~TN zC~E|A5QuSHHP*r?p{Mq=CQA-$D#fh?vd%S#_GIkft=4ru0tg_000IagfB*v56NoXi zu};s7dhmXogF~#}BpoNP&uoVG;s(e4VWw|pi5rNY&g9i|*EPm5m(cl3=V8vl+>6=;N z22oE5ff)$IIOfutf2sV;Iher%bJQ{OT!UESSwa8-1Q0*~0R#|00D<-dVr`0jy>!<1 zvG0R)n81uaA#QNk7iRlZ++a3~tM3d1Vt>t2T7SuWVhyF)_QadP-?G&)^Lkye|Dl8c z0tg_000IagfB*tH2>ktTf3aTC7jq6m8#m_muwRNB9OmNF@d?ZlHyGcNYKXu+fy{Fb ztkH6BC;!zU5I$Bi-c>TMcYgEhI+d0y0tg_000IagfB*srTt#3OU2Ki^{o11!?jn{V zJtVM(-Dt;{Z}89uW_>qH|2V6C)q7q7w9|QQKXc7PmoKUFm(0r^A2WNVx+8!90tg_0 z00IbPCJ<}%jHdfuTc2ON`ATL}H7VP*EPFmr+@Oi8rMqbCA>XqfuPwWHX*$flV+cJeWGYT_V<|7F;yD@1Q0*~0R#|0AU%Q5jAC!#wHnjt$31=W zc6ys_S-{cleW-ro23vYq?rRG-d5(>_ z1~Yr6x+8!90tg_000Ic~K;YvKpNboNvDd*5zx!3|7410fXd3C7`iL9sIKbrpuJM=N zj}bSR+{q`ce&#h>A_ift_emYILbbE|`hUFf;|jHwB?1T_fB*srAb8ZzW1!&HPLIz&;IxJpZ&jU{HM2bqVJawn2|u{IS1A&n$aW6*E#$8MCKZlx49e< zKmY**5I_I{1Q4i2AoSASG_P9aefO<5YGLjw-&%Y#=PPc|m5Wv_>J|af8{t{=WV*#8M+RA|Ws7)zq;A`^C=cnYY(Fe5d5T8>RD# z`92Z?2q1s}0tg_000Ic~Mc~OhkHrnX*z16vb~mLjh#U0PXVon2EbF&L%veHTMgpIF z^5?r)6R5N%S~9Q1 z@u4yAv#;TCh{@=mi5n#8h!PYd)+-y`-=lukWlP{}iqMQe?8{nG=P#XC#5E$mAtA7a zK=wTiqJ6|X9|-{j5I_I{1Q0*~0R;La@b|y{#d<|w%sB{6?SuE<>(h`0NZtN*OwGukoY2KoD^tx6cW z?`ZygtUfgE>%1!V!HIckr}of}xJ;}8DCH%PeJ%LVZjI5->pxjamVLjD@Z*@@BO!nQ z0tg_000IagfI$BQW?9?&n2xsl`7hplrGJB7QG;Cuv-~e^aD}4`e9A0)#m~Mb+odiX zvGwRHF6kG;m&6Y)CB5D8u}^F1vDKyX6Fb<>A9}k|_Vu=+EyTPY^JaSUEagQ20R#|0 z009ILK;STe7=LHDg555ygSf%K4t+({F4cxF^|vg?ho)_R z=#^F=ucbsRv^%1d8YiXUoMt+Zf_S3D=xL$GsJpLm2E)GQG3)!#)n?OrX0)}q!8rX_iJHtb?`qb0Rcrht?u8{|NO3=jJf&O& ze(>tQcM&_Fzvl9rmZj+>W)S->NC+T+00IagfB*srATS<*nRK&rI{9px@;M$Oem|Ry z$Lq1m)@Zi9UZTFzJ%N~m5%H87eVy3BJ?AX{ee%hl?_%H98uJ87=P7oO#cSG?qnDUL z`2L^%_=k3&Tp0lb5I_I{1Q0*~0R*ZNC^^ouhW@f;p6|Z(Mpf3CGT)MGmTQR{Wb(iq zWy#{3Ta+W#rkibE)gdpd886ti<7{ER`2$(WV|&11W)^|d+r zuYdioyKjB-Yu#wGVh3&fq{UTxiW$V3U$OR=ga85vAb1T&D zwfHPFrqR=}x{v(1xWVXNeR=(6-5-7iEqkl=h|cwy5wEa5)a7ln<{7{L-EX^z8^jp# z7}p)~4rxpRVcVYO!GurRgJVozR4W7!KmY**5I_I{1db61U8B`_%yYU+i|O$auz%8)>GVIzCLYDjk9UYyFOd{ z|7a62E`Nj+>+MB-YBReg|Av1ieSZAu`-Ua9SoZat7zMu5kOT<|3-gMj0gJZlm?Y391yL)E4p7t`>jX{|6(y!ks zIRXeEfB*srAbK!xx|Mbb*C9tN;V@8{b8>Dl?GzE(JMKiXE-8u(@ z7_*-}eh}r0a`t$iGUp&olmBNyV*LKDF51igG44C!9#UTf!e%|rbBT5lYkNruAbS$+9I+$6@vXWHW>?n$wibC0ot z-9FZwgJUhQ?dQGiNxK`{ux?RX-?`-KJ+4c%_*%p|Bm@vZ009ILKmY**5I|rdKu5dR z(MHspceSip{vXkfOMCTt8v(J>`~GgXKZrR9aWB|9h7jZOW3E@|?4!*E*_!2TEn|+e zW6hsl$AE{f8)dhCOkQ98|BAJ|%x%f*UrL4m0tg_000IagfIuq( zp|kF4NHhN*V_io|Gul$zU?dMNUl;w|;n=}$UlHT|L(7l;u$2BCF`HO-DB=@ggP7yB zd(F}Q*AJC%yCrh$b+3bEJb1jun)&uUDB%rd@(5eR+CIBB>hXUugEIag7X%PM009IL zKmY**Mk5eoK1bJZ)@oA0A@r?L(%O2DbR4~I6z{E5kI=41^7k$45^Zbw3;qAt-{aR` z{^xNs-&;z>T=GS+z2gR;!di+1uR+11v009IL zKmY**5V(~E~BMp#>B(SKv@qR_IX zx&(r8uwJ>1`~Pe2TJ-ttGGe5Yx^AV<&wU9EH85oa^FQqjU9wO9PtBt-d^s1vET7% zb5WMqQp60*Nx9r7oPhuW2q1s}0tg^*o`ARkjpdjwGV`K>z^+5I_I{ z1P~}kK-_@7l3kmrQNAC3_p5iGefm+khO}W$HF(HVKTy&KTE-2cDZc*w7sU?jbx_?0 z;y%**x_2Aix|&=nVh1B@#*20w>oPt5*-ym~{^4a2D+m@Nu2r?zdwDg_ph>slq6i>> z00IagfWTA)#0}_$CH2Cd@@lPZUfa?1#8_G9>|a{9t=RvZ@6G9ZTiZQ;LrJ*_#QH6v zuP$2a=&zxd#~AyW)~%T-RU&EoH&oKUc(Uh0hm}?Ph`p5YDk>UtZm)PUs zCqH_1iuojqJ(pK8gFgBlr9=P$1Q0*~0R%=MKwF`$l-5yt%CASwrzgJ3I%ofqA#M=s z$JirzPXF7w-!bQ)k6**1?{~0h%BNmKZ;pMHV?M&~fA`xy4EK*xMp>iG5d)cN?#Llq zM*M4}7*ID(sGtZSfB*srAb`N61jG&KDqCwT*{;`{=97(|^5{eMeIP^JAa3cg|M`B| zyyvf_eW2(4a%TJK8?l4Xo)77x6q|yt)aMflwpYxM7d_Q%b|LQ zt-^;T1Q0*~0R#|0009KHA|P%+Ln*DH^q5~ydnEVBSDEMPXR^c%;y(KL^_RsC?2%mE z2Y&J9D_i|+bDaVNVtt!Kx~$K>@_@T*`YW>BE<%K`xe5I_I{1Q0;rA_C$DbLn~X+r#$mN%LXD!#2bv50^}FgIKR9 z#)I>ZJpM7NEqaPSU-Uz9(1Sy}E}`ej<6*1;kMcYs#v&nr00IagfB*srAkczxA?u8@tU!>E}gr)&W&j~ zrV-zg5I_I{1Q0*~0R#|eML^tuuGm6X%y`Xr-+H4}GrZ`!8QCe5eq`4Nvc(Oefkpe? zEu;VWYz-eU=is8-S&D;W&7Ilz+NJZC)A>uW44;g+mV^KT2q1s}0tg_0KuZGR2DFnJ z+DVUjzV_&amQ8W>wR>c%%yacK8RG`gz+yZ&U!2Pqx3+gy-(P)$ne@QelQ;I9Ev5O& zC!Y@AlTE@GTkK6eY4cZY1Q0*~0R#|00D%((#0_XDTk9&BulMnXPfxU#9A9PTr)>I@ z@PVw_Y3z0I`uATHJFsR^RUe3Tv7|l;*z~KFK-#W|a1v+5fZO-!hc%_rLqCb&LL0 z{9x<&fpv*8w0>nk009ILKmY**5ZEUmZa_a7MXM=MkH@dSyx&C9y)RKud1&`FeIUDT zYM*5OU(^31Z*vaP-8wHSP>lC}^u3p?TO@w4i4Rzp=%VK7U<43A009ILKmdUa35Xle zPipHYJ>?(ckh7w%$Rt?{u)Atv<_Yi~5ey__=KI;~lcTjuY zLHL+;i88cyWk3J{1Q0*~0R#{@L_pkNZf#@HIK1BbLH0ck*y`R^Smb2UL);+lhhhgg z{ipYK9r5Sv_mM-t2Yq&!fOU(+4r;^>VqKzle)H?YPUlku5I_I{1Q0*~0R)l|5I3Nk z^sdpAE`Rp*B}&&xv@4od$>b5=tV|5t2-{~oC z5RKh^aBP3vs=xWgo3Et4cPQNL1Y+*Nx4!u`@dNuFT;&7dJLVYN-VWt}00IagfB*sr zAg~<)dI`N`D@~=B>qMK#oOl%YZhty zm$nsq8I1Ic__Dgh+C@M4(W_zy<{q5%fmq+lnnahiEk`4O00IagfB*srCaRY5)bRVx>zh2`8ac34g(Eiu94|^Hp`7PVBB(ZkUcfS3NliG3Saq!AkXU;^d zNff>zA%Fk^2q1s}0tg_`GXZe}8p&FXWR~N^4YY|_e7#mZdyX5#{ruw(pWekd@>+c4 z+kYRE{XWt2@6uk&{`hAD zd&A#2&5K|E7yBO!xQ3BTg7a+iJgtvAd)m|)$aSf46lMG^uf35Xx`8#`Fy1z|%80R#|0009ILKmdVR2#6cd zAV<|MudSE3fi`ii?>O&s-x)b>5Wfw~Js7u-2;cqS{r6`1382V;7i9cpI&YcY`OUAN>DBg8-aeMF#0tU&)*_m>4XQZ; z2q1s}0tg_0z(WG!27PKBG^%=b6E~>Gmfz07cBZ(&az~A|iypuJviLzKdmo4wES~E% zD*-Wt`muxESV2}lP%Z=zKmY**5I_I{1X>ahH<&}mqt~vtv$(-}R{RR9Lq0HL++ewn z$J+Q&|3h4S*1o;O+z|U0EZ*w17Xo4k5=D5LfUk`2iwMQ?AA0(K6@WbzZ^)CALm=hu)&^v)Y{qYZXkAL>l zyC1yz?|0TSIv-C6M!_nW1v?1=1Q0*~0R#|0009Kn6A(A(Pm`mg*0q_qL0!K5cYMC$ z2KV>ir%&EC|DaLqApG#jJCE;;;XechBoO;5|LT`Nf3`2ex4!wcyZ`>L=e6{aj|m2` zr*SX|HWC5|Ab z+<+hG3vTlPaf5aD?pVVp^ltukxxX#*3WDL^|Mr)4z#WAE0?!eMwR}Q{k9B=Qmyf*w zLZjbu?}WHkTr;j6Wr(swnXEJPoVk2}00IagfB*srAb`N81jG$4*DGi|V>1&s7~8(u zZ+-iU8(el958XS)%8MbK_RZJ7|KhW_t~H1*^DjpuP=Y|Lf3%GAUtX8-|L0zpc6Z%6ze-l2q1s}0tg_000IagfB*srAg~<)af7mY8U3+;p5g}m+fX}fYeR8^a`*5U zTmSA`Z#;`5#CZFWeLMUn`d@1gmGd7tA%Fk^2q1s}0tg_000Iaga4i9GgSPtG$Qn?u zbrUzxCVKVVEaetAxb}{G{=8Vz=!5s)d#0}k=U(?$j2q1s} z0tg_000IagfWX!S#0_YqS+vG18?;>K zD6SuWQJ(mV{Trf;)*IT^H(U(?1Q0*~0R#|0009ILKmdVj2`u9lmuGvgpKBaA%*!F# zMCQad=k?8AY#$o2l!w3)R|vc8`_K|&Sakk#ulwi6`{*;5zdW^85(E%H009ILKmY** z5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0 z009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{ z1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009IL zKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~ z0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY** z5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0 z009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{ z1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009IL zKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~ z0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY** z5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0 z009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{ z1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009IL zKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~ z0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY** z5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0 z009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{ z1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009IL zKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~ z0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY** z5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0009ILKmY**5I_I{1Q0*~0R#|0 z0D=FXy}Q?qB2E7Wd_Hpoa|Rp%N3hbyLrOw~G|6g7Eus~frAZ2#JIM zO)#3&FF*L#ivf3cRdszlA6H2uGj_M#ebuj?>ifCtg8%^n1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+ z009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBly zK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF z5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk z1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+ z009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBly zK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF z5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk z1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+ z009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBly zK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF z5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk z1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+ z009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBly zK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF z5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk z1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+ z009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBly zK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF z5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk z1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+ z009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBly zK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF z5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk z1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+ z009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBly zK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF z5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk z1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N z0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+ z009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBly zK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF z5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk z1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs z0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZ zfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&U zAV7cs0RjXF5FkK+009C72oNAZfB*pk1PBlyK!5-N0t5&UAV7cs0RjXF5FkK+009C7 z2oNAZfB*pk1PBly(5JxPfBkv$_RVYmF^~ZnkO3Kx0U3}18IXa6GSFvb;aw9TK!5-N z0t5&UAW)0Ir;qQy)PMi`@4s&H-+%u7kDK3r`}OA0KY#Hb0~wG38IS=PkO3Kx0U201 z15X~_`|=q)pT%Fjc>3kDc|N0C2oNAZfB*pk1PBlyu$4g09T@rpdINeRG9UvoAOkWW z12P~3GEi;?a!gM?yMK6hy;Zs3?+_qBfB*pk1PBly(1Ac|41I!fYe99obt3~ZAOkWW z12P~3GB73sIVbq?;=z}55}c3FK_TZY5+Fc;009C72oP9UAonE5InEc4?>n!2<@3tN z)Oa3aE~A4JfB*pk1PBlyK!Cs@fz%B-25IS< z#P3t*%76^WfDFih49I{C$UwOnc>e6s&GpsGLuG>R6Cgl<009C72oNC9nLv&&%5{ix zKQ*;nxwV{iW8KJr49I{C$bbyUfDEjifz$*!mdHYY009C72oNAZfIy!DL%$%`IbFN{ z@cY%cG9UvoAOkWW12P~3GEh$j^bh*15xr{y1PBlyK!5;&eg*Uk>Z#Y%*4C8_$bbyU zfDFih49LKq8OSk4IXBrtfB*pk1PBlyK!8BI0=Z_X>k+9v_pCkrZTXV{8IS=PkO3Kx zf%-C#8shDn*X z?hNGGMXz2wt-DC_>Io1aK!5-N0t5(jDv)ywE*{_4GjNW949I{C$bbyUfDFih49LLZ z8OZs}&OPX~*z~pu5FkK+009C7suak5400ThW4Y=+mdpCH{$xM~WIzUFKn7$$21?FA zY7X~3sIp}Aq6rWnK!5-N0t7k|$nieUpFPqqa6Fj|$bbyUfDFih49I{C$iQY9$hA=| z1PBlyK!5-N0t5(@C*Zn7jzuf^STwIqK4m}#WIzUFKn7%>?F_hXQF%3@S44mS0RjXF z5FpUEz=wC&u1n;2vdxYsQ}ef7^IN~xuMEh549I{C$iUnTfI9{K!5-N z0t5(@A)sfVuA5ued7kXZfDFih49I{C$bbwioq^o*AT^4G009C72oNAZfB=Dg1g`$L zbe@6p44h{m12P~3G9UvoAOkWW14U&(?_i($&|f7$fB*pk1PBlykenZ8IS=P zkO3KxfweQ>nnl~xi2e!z0t5&UAV7dX-vV#nymp>}^U!5L24p}6WIzUFKn7$$2CB$_ zYZmogzk2rs2oNAZfB*pk>kGICvg5(lJ|4{PmoXWT0U3}18IS=Pkb$K$pm(r-t>`Zh zAV7cs0RjXF^eLccpnhAre)IceN(N*=24p}6WIzUFV5r%$0U3}18IS=Pkb$}~klulX009C72oNAZ zfB=DI1THTg=oz?vtqjP349I{C$bbyUfDFh$^%?lzfB&^ik>s}!AV7cs0RjXF5a?Lo z)r+Uq*9MLQtLr#0>&yC*0U3}18IS=PkO3K3GXvLGFFUUAymbNu2oNAZfB*pk=LzJV z25Wv^_q)}DG9UvoAOkWW12P~3G9Uw6XCU`U{`B$vdBu`v6Cgl<009C72oUH{z&#Dr zZ(G-I{+`UqfDFih49I{C$bbxNl>zrU=&;E3mI)9bK!5-N0tC($a8Cm@+g3H3zawih zAOkWW12P~3G9UwcWWc=+&Mq7M0s;gG5FkK+0D+DK-oAP5`b6$|Cj&Ad12P~3G9Uvo zAOkYcd%3tatisb}!da_befq;(?$G9UvoAOkWW12P~3`()t5yX*g7H~9wv z0t5&UAV7csft~~|FCOR{=o`p@49I{C$bbyUfDFih49LI_8E}8(o+@MSl>h+(1PBly zKwuE~`>#LM20zpWj^*0tST63|Wk3dGKn7$$24p}6O3Og5=V~E9fB*pk1PBly(3?Q+ z^;OzubFW9eC<8Je12P~3G9UvoAOkW`N(Qd4UiMZddcOn+5FkK+009D>3;gHb|IjyZ zzJUzLfDFih49I{C$bbyUfD9bS!1HI1IbR;_-cb1APM-kO3Kx0U3}18IS=PkO3Juk%7yL2R)RC z-Xj451PBlyK!8B!0`6_F|1nFB8?z2%Kn7$$24p}6WIzUFppFc<$3f=>w6{-y009C7 z2oUH*;O(2&jww2EOpzL{jvCH-vYuo>24p}6WIzUFKnAwRfO{NtQY?F$1PBlyK!5;& zjs@J?Ky9@}ZRKysnGDE)49I{C$bbyUKr0!zzIxel1?jC5AV7cs0RjXF^dRv3*(2v0 zINv}9WIzUFKn7$$24p}6WIzT+W+2C2SqKmyK!5-N0t5)OEbyOy|3iH+vOZ|#_$cqg zv3@cj12P~3G9UvoAOkWmKLhTK+;WBNy%8WlfB*pk1PDYR_w`aU%&!^Pv6BHAkO3Kx z0U3}18IXa#GLU1hECdJ;AV7cs0RjYC75MwFKlKcpYajzMAOkWW12P~3G9UvoAOjOK z&}xn7eGwo)fB*pk1PBm#`{uR!U}AmXShBv3C9}S*ZyAsQ8IS=PkO3Kxfx9yB>En9^ zB>@5i2oNAZfB=Ce1@sNnKzG$Zz9$zlAOkWW12P~3G9Uvoa6AL4r7Q#p5FkK+009C7 zS`_%-fB)rt1LqsafDFih49I{C$bbyUfDFjM#0=;gv{)f~PXq`MAV7cs0Rj=wH#mL_ zkz=g*$bbyUfDFih49I{C$bbyol>vPNH8lYO1PBlyK!8B20{RAOpu1`y-;)a&kO3Kx z0U3}18IS=PIG%yjQWgRP2oNAZfB*pkEehxx9Ipk`N_=EM24p}6WIzUFKn7$$2JXs$ zzCnu>viC%Q009C72oN9;f&cvbAFg9M@j9k=)eOF;mXrY*kO3Kx0U3}18IXa#GN5mu zrY1mu009C72oPvhK;NLR+D0vFeanCh$bbyUfDFih49LJT8PGRqwMzEB2oNAZfB*pk z1oRD-sTtH>9A!WTWIzUFKn7$$24tYG4CouErwI@sK!5-N0t8wW&^PF-wo%Jk-!dQr zG9UvoAOkWW12V8o2J{VDt&+Vj0t5&UAV7cs0eypIY6i6zM;VX-8IS=PkO3Kx0U78k z1NsK)X#xZY5FkK+0D)Eo^bPu|ZPc>Xw+zUD49I{C$bbyUfD9~?0eyp3t7Pws009C7 z2oNAZK;K}QnnCTwQ3hl{24p}6WIzUFKnD8CfWCoxng9U;1PBlyK%i9teS^Mg8?~(U zEdw$j12P~3G9UvoAOp)}K;NL%D%txYK!5-N0t5&U&^K77W>9-^lmQu#0U3}18IS=P zkb%B3pl_g_CP07y0RjXF5NK6E-=MGBMlEZ7%YY2XfDFih49I{C$iOlg&^KtcO7^}8 z5FkK+009C7^bMA&8Pr}JWk3dGKn7$$24p}6WT3AM=o_e~2@oJafB*pk1X>l)H|VRj zQOjE2G9UvoAOkWW12P~3GO$br^bK0AlD#hi1PBlyK!5-NeS>9c2DKMQ8IS=PkO3Kx z0U3}18R#nm`UdK00t5&UAV7csfmQ|d4f?8W)UwvM49I{C$bbyUfDFih3@noYeS=o3 zWbcar0RjXF5FkK6-(Z=VLG8s+24p}6WIzUFKn7$$2Kvf?zJYq0009C72oNAZpj82V zgT87TwXF3m12P~3G9UvoAOkWW1IuJU-=Nhh+4~|ufB*pk1PBn&H&~`-PpuV5FkK+009C7 zS{2YY=&QC-%Ua(uAOkWW12P~3G9UvouuKN@4O*>|y)Oa;2oNAZfB*q~gJo(4wHHSj zkO3Kx0U3}18IS=P=qm&I2I^@71PBlyK!5;&Rt5A8`l@Zzvevf@$bbyUfDFih49I{C zERz9!gI23#?~4Ed0t5&UAV5IhV40dh?Nx{4@4x+eb9wRLCN#Kb z4g`73_0`KS&w2jr(an>G_in83I@b3d>*?b0{V!Qb#*&>6@2-#L>(j^gUmo}B#nUg( zGbYQ{vE0rX!;-<~Hv4-x-`bhGid`u`c`6jC*LIo1$b81B`P(U*aprpSJJUxnKhpO$ls;1C3^Ff!ss}YC zW0{9bFTosd-}%b;oG~m}SaJ+YR+Cv{n0#d)w#FEiye79kXUbUXpELChrhK~bSONqH z5FkK+0D+nW)N|^&7VEj>I`jHV&f^~2t~39ZxxE%Gy{DhYh|IHPzIDlY+++DlpDOcj z&UN{DU9_6d^oz32w;aQg=~g|xukW{=VGPT@Yl~$V!}LMFUk`HGhKIhv7Ndy2L4W`O z0t5&UAh3skzCjx`i`w(+dfaj@_pa?Vb0d!5ZFif>y#1DQxp!qO^L)nbb~Bd#W6m|% zYe}CXb70P+YBOWQxdmJGqwdOa@@))jGhe-a7S%VnYfJY%0t5&UAV7csfffYRZfds{ z>b%U=XU@KqWte+w!Pa~D<;!vYIe(?3#oSw4V|V%mC5~bG2W{jm>v8X%=a1uO&e!r8 zw2gfA{y9P4;K!}o{}UiUfB*pk1PHVupk`CE)m-o8ng@Fxn|o>;$ULCy7uCIuWjK-ghr1{ ztM_V~3mLZi%e$|Dodd|d5A>(X&sdJFu4z8^_88{egYxs0eR}EV7z~;E@v9zt z9r>#5bK}~tx7N=LKW=~ip8x>@1PBlyK!CuH0_HK*XlvI~OaA^^=R$68cR9v*NfW*NhpHeb>e+H(wdU(m$+u$@by;e&uFt=`pc| z-=}X-^N8cM6Cgl<009C72pkj8H&C0^QnQtIU)p2a)v2F3F1)<$E_3a*T$kunzVdU& zg_oDHmgbgnzJzmI%F0;QM~&yP92(DzVP)m3md}cNyv~-!_CwpF?-L+EfB*pk1PJsg zpl_fq+oQ&s@;5UV(o)Z07-`JAm9y=oXHfew;6tu%f6ZLRlx@vp*UMy!W0-SW*2~M( zZL!rcEZfag3y&`+m%XeDeS_PdJp7vg0RjXF5FkLHB>{Z{^;OArU2Dfr{y28%9qhmT zwL0H6WX-(W{uyhrXE5X|{kCjFCGSUCe{IIF{kQSfGv@sr&zQc!ut)Hf009C72oNAZ zpfv$~gX6V;T4`UO+ON|*X1vfl*e_$P&$kWvbKb#z8Ed&`Fyt$}gWT)c{&M@A^)iO( ze{P>G+w*vHeS=|d;41+F1PBlyK!8AN0{RBy>jE{@wpKmNw+-V-dI$2gZN7S$ZyPe^ zTH)K~E8BBVb4mK2+vh9i;zK;Ai2{l2Y-%x(6%IG*xb4%XZzdbs}8 zUE5HOO-|2bO2G$?PUcGqwBYhYD@0wWaXUq0G-B{nC zi{ZzcBtU=w0RjXFbSa>3FugudQ>|y#+jSRy9CX}cbiHg1+imaleR?K?g|Ex!t=Dr| zZjWU?S1g~a+uLf6x$k*#Udy`qx}7!u9%tSA=Zc=kBk!F60RjXF5FpUHfck!1O`vw# z#;Moiz;BN;=0w)ZT5tC>xSgxp|8`9HdKv5WIPlx~O8?V&E$d_~_n+?dT$bAzGls2` zFQ0Yix#}C--a7r8009C72oNAZpd|r)gLyTA+G=^XWYO|fz?{hP`N}r?_RVV%`z&By zcb$A0!@g}^?$@$T#`M;|Wh^}upT*Y68P{p9`Uc;&OaCH3fB*pk1PBmlOF-XXT5X_~ zTF)!jUblP|$h@vOk!7>i<9%w6ZBOQPm(5r94L#nc_E^SzzF0P6`cmJgEq!30$(GF+ z&v~Bu2H$U+{*M3w0t5&UAV8oU0d@Vn8bNKftecwa=P|`~=$Fe_@Au94IbC1BWBgw( zWBO7*r_8+Wa{1Di`Z-_uPVV*K^VqUE<2l{aXV|;9Pv0d#fB*pk1PBnQU%aD}$h-=B5pRa7A=5!Bd%s$}!jG5Ct zoG-np^YfK4%ys+@XH0KunT+{7JKxdg*u&eWpAaBGfB*pk1PIhEV9scMouI~A#?GAX z@iD}G4$jM&Io;#=ay_i`@?}ol>Wj zMm>T60RjXF5FkLHP62&`pFjVriMBB@A2NQ-$aN~@a(XT^mt;9D;NGIsvz6^J_jI?6 z6Ue-pF>HFy%twyPm;UvftaPxIs-dv+-e7R`Q@4@ zmT3aHCW^d`&)YOzkB)%-!T5ZcCz`&^IHzTNzB1P5_<74T0q3+#%b3p;%XrxzOxsR9 zng9U;1PBlyK%gE0^F+(k3~H}wj*bPNK1Sr4C~`I~XBpQl(*>N`)4jl_VJzsf- zF>HLs<_Y)xEMwUCeAyP4v(h){d))Ev2@oJafB*pkEeq%yELS(E!NyxUj&k0pB4gt+ z<~msO^5r@*3z=2SGG&}I-M^@%Npq$%-dc)ivR%v1PBly zK%jg9eS>9d2esHZ&+@m7GGC$XbzHvG@bmI(&TU-2GS0gv^1O^0!^UOIJ(1_-%eAn^ z<;!Q)Wu1KHn76%p76AeT2oNAZfI#^IYWQVq2esH~Pxlj@KQ7o_PvXH_IWJ#%mVBMg7o%m3Y_IdTSI;CsfB*pk1PBl)TfjWzvbBR+ z?6hYxXPGBp&h2!*vJJY1%)ET%S;nx_8PofmpD$zB>3n4yG!HpHW5%#?8RN8^lkIi> z_Uf4g2oNAZfB*pkWeeyVELS(E!NytY8_XXQ%(;!rn7+aMj2XvHXUrHjKVQbM)A`Ca z`0B;e`N*9=^Y`C=)&KmQA!GS=2Kok_4?5mH0RjXF5FkLHVF7)E?dl3O+*fOJZu18e zb8h3dSAB!|88c3u&X_T5e!kp$`*gmt4I0DdXUyl?aTzn#FXyChFn@dXOacT55FkK+ z0D-av)K|;Z4QjA)mgd~%j|ut)<1%K>ZGOg#W2ZA_44a>?Y@0H6I%CGL`5E)M_H@2% zgUcG}8_eHcJ(BHK*BoUe)mPGMw`?de6?-_n#h%Wx9aR9OL$-ak5Mo zFouoG*Vu8+vQ8h~UDvUJ6!t3Ra9j?} zxy=i%d)AK2SGGgf!kU+{oNs0vJDs!ih%M6vuCHFo*T{3GY=@_}!?8yC2Gh4y&mcg6 z009C72oR`4K;K}wx?ya6;c@2N=8O)mAv1nkHRm=bVXh%FWo&jWta%xmV%+;<=Lz@y ztjyo`o*VCf{AY!wtn>}~9(TNZ0t5&UAV7dX%L4ia+t(LM)hRXq9yQkVQRV87OYe5B=V_V#_FY3}e7>@OcV6D~j9p$lu%Dc;pRD(}o7 z?$ffBM;pVYjbYAdnVzxVfBRM6K;K~V4Cot7-&Q??009C72oNAZpbi0jgK2gA)^k%W z|Ghf4kaAw7W5K6wtLpe^*>x=Vw0vc3c5mcq8S{B#TE=qhyk)#V&d+H1Gn4l=ZTs2M z%jb>p+o?wqAV7cs0RjXF)Fq&fFSicavgX<2Z#f=({8*7|?(DISYW|zJtffB*pk1PBlyP?vzdfjVoi z`pfmf$Bz}}L*{L#jt3u~v0Qg&ud$%kzw3DL@%c)hxYqT&=j+yAAD=PjVS?!P@Q zW9bv`wXdo9@9M9Q%U62DHLqu{ef}6+f}PJA{V?B@G zOTBk}gK^ubM-d=EfB*pk1PIh8pl_hoTB>fE^Lt$9P;QbSAOa`YxBo2+u8Yc`Ua=BQI8-%fB*pk1PBnQQ$XL~ z{CYvnb)KPX&>tQ<%;}z&t!$g-br0t&KQX6!e#Z2sj%UoA?s*w=-qi7Y8N<%Ym(Q^0 z`{^4T-$wnE009C72oNAZpk4uegY)YJHP?BDuHSrk{BZ4s^RhK;n|f1+GiFZr{EX>M z9nP0I-ShL6aZGROaK?O&T_$70{?ga!p85udw@*JIK!5-N0t5&Us9Qk2HN8GiQ!Q_1 zUia>C#GLLjS<{=kJ74B>m&uoT-MjOZ@A{0fOvd!4?#@{HDL#`elQE9-9rX?F-adVo z009C72oNAZpnd^0{royXjkTPedEKAKlGLFxwp_*>uls$*at$nfspaxzUibTa<^S}h zmdjW61=m+Ee@5>7I~R}d+fRPZhR+K78t5BzJ`j2P1PBlyK!5;&h6VHu_N_J4df)ub z>wX((a{h&T)qcyy65EM+-EaBIzx1V+%~vv-d2-8F0rzHEK40nc#ntjvz;*nV&)5?C z+8%$0zJXDR009C72oNAZpj82V19ev^HC~Pzw0srF`4^?EmzG|KdEKx1N?)p_^;*h% zF|Ye=1lN~Z=d*?R$*&pHms%%ZrHrMuy$;*ux3S;92oNAZfB*pk1lkhNH&A<(RG+E6 zZoXWLqNH`w(rd{%9PXvzn)7RB+BMg1Za%!b)|XoG+A;C%eYw1NAY&_LY|SyU#oz6- zYwshH_fLQT0RjXF5NKCG-$37>v>LAG>lL{^(fZqC@7F8J{6=Zpd~2_#=j+hBZ^rs# zSkBGs`Fcg}o3VbzS{swsen0vK?bgZO836(W2oNAZpmza%1NGP1_0-hg-}ALTY>(S) zkGB&vq9zA|~CVM`w#c|-%^D<`}>-D^roNM9UKigz%PA+?WPOqOSdat3qe*y#u z5FkK+Ko0`uDb-&k)_IwG>TPix__o_(Z^wi?mu1_0^>|FU`_yimvEJ_2l5;GwU6uHm z#cSF&`=xp78}#;x-TNg#fB*pk1PJsapl{H3ty5aEbFQNX$;#ZV?Fgh^$hmO*zM0{{(bj4&^PF(ZuU+I5FkK+009EM3FsT#T?43% z%5cf?tF15Q+xE*^54{6DgZ=WA{YVe}PxnFIFJnE7VfhS@?a1e{{c^_sxV^qX>z~xU zKLP{@5FkK+K)(X|2FGgwwbK4REnka4&tU(2wLI^pXRv?1TJIgW-u(U>ml=Qtz0WS$AG&Qcv<;M&RV)xvc6||`AWvJURqf6J^3|EKri81QoQ_2l8bs_)0`81QnlHSAY1hGjj~y5zixY$wC}@wNQS^}dd3|J+dP zSmAXOAV7cs0RjXFtS_K%pdPEO-pib5jmK2x7;tkTb!=}L7i;WKW$vQ3?YY<2TjM?7 zjbU|+U)iS8W6%1kZ83(`ld)dM?z`T%zCmpxjn_+n009C72oNAJRzTlCjdoWJRKxdj z+rpMv)Xnh&0;j`ivud$Zx zY;0}laRdkuAV7cs0RjXD0d<->ZHt;}t-q0Zk&=(w%{613V^H@tmNBx_`;EG0Ox@WU z#>LDzmD&R}hSi_3?1M@k!}O%;&)6`Q_4-xcV5lm6B|v}x0RjXF5NJ(6-$1?ALY=mlsOvPR$JRv?#o@uwTp&xSkgCZVf*zSe)-a0%otWm|G*g5*7%ieEa&Kz(mzPf z)AKimwU@IV$8&vy)~aLghX4Tr1PBlyKwuCsx2a}psm2@n2Yb!EW!}&AV%pxO_MCf5 z-y;1Bb8YQrEbDg9^A9qH`CQO$zOuj1`rWI4py%0kwp#i;Ipw|R8w^#YuLKAXAV7cs z0RpWFm>*EXO{tA)d2Ht1GQXN@Y^=THy3DB=TfQDqdLbDr@;-BnXy%*_{g=n&zdXKW zlf1WBtbGnb=Gj`G%gy$hykwhAcCy_b%2$4t$Ke9m^0 z?PsZNGV`99>&?7mi+zN=r@Xhk$0g=?dlpFVXV2o~-0O zCOdh*$1;}3JIT^;VWan7EdKtr79#6ct^F9M- z8?g`|K!5-N0t5&UXi=c$c`9|#d36xKKKwHGxb{5Tp|LY_QO2OQbC&tX%q#A(qz9Jy zoR;Uxjnlt;>7VR1hUK{W)_YVk*7g`}{hq&m^$qse!~2^A2oNAZfB*pkTL_r5JHK8~ zb1|$f13C6Pb?#nsZ9k4}c^&57+S-n?Z^?1+r7Y=P=pVF|F=JR;>$A4~sMjlhLk9E> zO4-wU9RvsvAV7cs0Rl@4=o_eehPubsKC+PO;gr%Z_;EbT`3Egu`=f_#EIp~c&M){e zPj~%)=6CWMdsr8(zQ>Zsu#BxfSM{+kZFT$dzNY6_-{7vjt?v;aK!5-N0t5)OAfUdT zULUBbnAMnp%*SV5zJ?|9&gORO+s-m~viAKkmdJ1Bb~E=}<96KhYtH#1wTxjo*T(rR z^^IXYZ(rW~{JiQLEU{KUGJ*GR~&YQ`;Dp*Q-BOPsVE5pL?Cc%Hz=?EP!n1wOJyMQxg}j=e$LpLdVDDlzi0VvEBlbz=5uf7F!#09pISZ_OKocz z!%DvH)$J@D`uD3BPmN)pdp}EM!tY^OS_bqD4vl`kPk;ac0t5&UAW*x2zJdCywED~I zStgUYMn>jDTUZT-nLqqD@otOeYwlaSb7GnjA3~nc|S{UgMMFmxwTIETPLmT z3tRZS=sgf1K!5-N0t5&wE1+-CUwxydZGWBT`01LD0l#aMGw)VbZfidV{H~1Y9h8%? zjA5nTzx}T4jQL*1Fz2?ElQEx#tc#^Gpl>i{1oRjJ1PBlyK!5;&8U^$X)J;p(O@2=q zS!%IoFpRBw2W4ce#h$^CvAj29*}gezc??TGSMOloeA&MFFP#B>gJBQqD**xo2oNAZ zfIw>k`UdK!rRyiZZ@)~nJl{5qy?O`x@kTKUR+CF2s z{(o!NR~z#5u>bEF7*w1 z8o9k!0t5&UAV7dXUjq6Deb+i_-8R<$+c&TK8o2)N-KJKT;q^0|b6I*^a_sw)hxcS` zgWf?;=d#@1mh?Z@AHVD)7_XfHeS_Phntu}@K!5-N0t5)OB%p7go?5$}^844#Snu^} z=CoyOy^QsEF3atGe0p;7wO+nr+vB+`w=?FvmUZ)G`{lcK273P-c6&VYZvq4e5FkK+ z0D-my^bOQgYu8hL|9Tng^*Hd`qp<6Qua~cEyVqAQZ*zC~Z|AkFm$9Di%W`PrabC-M z`SSUL?G_o(H#ju9`91*x1PBlyK!8B)0{RAOsx4|Ne`B4T^>$B#L*wnM7f&4%zQVO+ zavZeftAO)b))~Wky~f&)+m-WL*2$O89$dG`fWE0**C+ZhW3E@Ue8znCV7f&Hd>;F8 zH1q!i2oNAZfB*pktqABFsHwK7sr-#)bC!Fx=GqpPZvy6Zm(7@YkZ;)<{*`%dW7#q} zOJB+uHYDroSJzxyCR?^qj$3EI{_^{=%>NM}K!5-N0t5)OBcN}fzS_FJ^7qcq+|?hK z?F>Q-z7zAh=VvS1XZli>p9M0nYy4U!W9Hd@&R4z@7a3b7V;nhdl>y_~-Q$_>5+Fc; z009C72-Gj2Z=kN)s;=^PmdTnq-NWOqdEI3)mit9m?iSFSIzMC1HMl!l`EJg^u&+En zV+`;3 ze_77`k+JC+b5HkiTSc7YYkIzzG20^p`Ud01HIE`dfB*pk1PBnQPe9*5jkQOO8$$(e#X^*J2qb;Qq+d^W}2|v%NB)Z!m6L^C$uY2oNAZ zfB=E|1oREmS$ox4{_eEwsY|DguQ_&J#-?Q~bCs5H0`8+REnC?Z(`&Je7jUhj=@~QT zGutBrKA(*r*F2H{0RjXF5FkLHE&+W5HP#+AmcKbYcbTWOj2B29FK6R(<{Cxgvo$F|=Cs^77JT}c>iXc*GUizD=^1lQ%lM4Bx54y$<_V{7dxcrqrl9c%h z&AE-smpPC5`Lz!nmofLi=*Ny>bM=G?~R%bdsj{Mv_(%h-J3zMuK`Uw^s|*5`Agd{$skat7=}`yQvg zdjbRq5FkK+K+6K=Jk(+(*J57V*L=1-2Cezt^bN+1Z{|Gar!aQLsna=|FYKA}bvj>+ z810z>`%vRF0RjXF5FkK+K&t}!25PN6Yb}3!T>e@ef7X6q`Ud0jWzJ)M6756BWo*8% zXWCDW%NQq4du6~r)HqFm009C72oNC9s(?8Ub=F>WmcKhbd##Q?YrijjgK-%%=P^Ht z_Mzi4mh0_UrVALu#^uZB2~K-vz&>>P*yb4o2oNAZfB*pkbqJXAP;2d3Yx&#b@>j=b zQr@dH=Qb{1<~-(S*FJPy#>P*yb4o2oNAZfB*pkbqJXA zP;2d3Yx&#b^5)009C72oNC9s(`-1_<0X=kgWE~!0*5P+UgLr z+WT@W__XcSyvOt?x(?Q~j5()eTE;$od~d$E@cH7sJ`?!6)5e~)z3e}yjcp!HfB*pk z1PBlyP>+DVL0PrO+O>_}U+?<<@b0>vfu(>~nmS(I$LHNSEz?7rex8ht&)D_V%jw7+ zKO^_Gk+1RjV#R8&4CotlJWzY<1PBlyK!5;&#s%~Z)LDDgS^n;{?B$v$mhl3{wrTmw z_GlR|kUpL=vCX3i5FkK+009C7>Jc!vHf`R+ zJQ1(GGT{2)F|=KS2fee=5B0cV3(?l|4~*~)gQH+4Lu=5)`?S2CtI zbv$GGQs-yPXAFj0Wp?etgZa5^YfN%)^fLiIoM9nM#N;@%nOXUt~}hFfMp-{AO|=BES*5FkK+009E^3g{cC zt+uSK{Hh65yyRILzOvapZaCgSs(|wtI`K-Zl%M9on+&!N8 zE&&1r2oNAZfI$5M`UYyNEo&=(Yni;6*Zq0yb*{lO*&4P{=N$Z;F@34!GUl9vpYxS5 z%(({3Wo#JBd}X?J2J{Vn9?yJ-009C72oNAZpbY_i1NGF}^_1VgPR7jZem@4gufei8 zGq3x7zH+~9=Nc@VF>`I-XG~ve*?jrz!E>t&7|XsN%lsb!0t5&UAV7dXI|BL!>Z+~k zDt~9$tYuy|$ER9uZgSlw`C2w#@iZ6mZR@(ccpzh+GqL1$mOf1SF_xPz4olklUCU;b zC(mOUu&*?V6Cgl<009C72(&7oZ=e=BRtx!A+4)ebejS-F9^YSdK2z?$=05YfAKqQJ zy89dbzVxNm8wb;mvE1DFEVf?8jOT3E&VbKiMsWfJ2oNAZfB=D31=OnQskQ4Vzkl6~ z^?tpg)ax>~UdGas>3MPAjPuO;X67{k`j7-Pn3 zW}xSNWYa~mH%EW~0RjXF5Fp^(_=T_6yk=eEcdL63Wgv5#y)Bt<%Y5S@6W?EUd%61K zvbTtRxxa%PEt{i5o;k0jxBh3|Z|+lb$i(-z$sT9U=Vn0Ppu3L;-Y@|I1PBlyK%g4| z$5N`7&aIa`zl;n$fA*-GaVzs}?rE@X?s_{WJbe;l+qN0&ssCvV+csOaW!`IMK;NL7 zkDuNs0RjXF5FkLHI{|$IwbYuml;6E=zIv!#&9`lvsoUFdFTDfzLEbN8z4SlxK5uVZ z{=Hx3*t0)313i59>aIZch6xZLK!5-N0tCJY=o_e)&aIa`zpM%9X#gZ(q+bBlGbWCrvN6x9R>5FkK+009E63g{cCotCVf{H`+c)M5>*XHZ7A zvW>UeJIH;u^WKbO`)94y-hrOM{@Jp9TL(*LpvBKzt=7HX7XbnU2oNAZfWR~XeFOE= z()E+yS4O5!nn!4_od5v>1PBlyK;W2wzJZ!)iJHmp zC^tu$bIrUNXmnHr?V_2>CZAo8E-?RLTm6&_4 zsh-^E8`Ss-(`zO`fB*pk1PBlqBcN|kPhFyhw64y{KHCzr`m zyt;hVpRcUL+%KTCF)Xjg7}iF`Um<2ZDs2A7%}t@_B{VUzo6ZGC2Q#)?0xS8V_4f6yFI4)w{?A22J{U|t#Z9C z0t5&UAV7csfpZ1)4b(t))j+=2S1vO5mTPNlHNTr{#iY(|{eGc2_Bdl~j&slZz3adH zJpQe3b1~TJ8dt{6)?Hg4m%P8eKI^Qt zzBBe%2oNAZfB*pk1PHV!(8IjST6KcotKMue0~r(2L&)6zl1r}Pk=nY|dA7W-%+Y4P zdahmOJTtc*r&fCj=@kt{y?^+x%UT7Hsh3a(CW6BZ718%($_H> z#;{hm%@+4(-jmO1TVz1rV98I5eis1(1PBlyK!89=0{RAOsx4|Nf1}QvWsW!Vu&IeN z*EnXrGV`~2TzU_AUdv-l<`36APh&W4zU8^_q2H3Mq-Qdw=aPI4{gsyIzi;eEF#F~@_s!N@S?euhsD%Il0t5&UAV7dXivlgr>8Y#A zs;jII>!XDXWG**z1gGX6m)>6tJ%pC$cU#EE67M1V<@D*cUedEkA7+X7;CC4BWk3dc z$$-AW)_W^|j{pGz1PBlyKwv8YeFJq}FLj;wDeLECAoKN^8<=~2m?g*b%qhE8N_%l*u3zM{ zLcKY&KeFy*VBHMp8D0zf$cI-^O%0<^MvmcAV7cs z0RjXF5Qu=j!FF|p8jiILl%0Xhxwo+7z6Y{aUe);W%zlKw4D6c$eS=|d;41+F1PBlyK!8AN z0{RB~)*5O({xYy%23qeK4Erm+gMBiV`v^|xG>ZN-fe7<6e(T<~C$` zn=GeK)7z4BD{@^X8QUgfoH@%t85z(w=xuN3{SqKRfB*pk1o{!sHz=d#Q2SXAGO$es zavXRs*N(YsPiD+tKW9D9A?2D_#6xT=B5UhpEyvsUzPx(zRK7O6CYEh^oo$&b*RdJU zH|Tw@=lv5PK!5-N0tDI>&^J&QjjfA3PF8BqfZG1+zAU|y+Sg^Num97>_w9aQnDEYg zwy?h_bsP1%WOL~Z_-rv@=is3P2oNAZfB*pkRSBr=)lW;;Pkx_Fm6(CvuTS*jj>*`+ z{B~;Y=v&ItKPhn>@S2v-JyWK=XTaG1ZS40i0t5&UAV7csfwlz99jW(vulKzFKDL3Y zKQ7xEyw7|$=5?3N{ga3Hy1sA5U2)7@n(=knoblu-1Lb8v-{7u2fbS6?K!5-N0t5)O zAfRthUj3mav`%DT*$iZyv-~VzUU#{i=}Y~bFZa?|E?@QuEM=hF4CovDJpTI*0RjXF z5FkK+KpO)32IbZvYDDWs2A0Xd_0`KZ2JD63g?Zg&@|AN9EO!g&O)Zl#`vQ(KP;LhF z4elQQeU|_M0t5&UAV8pg0eypV>ku`fbt41kXTY59@%_~EXOCP<=AY-~%((`KGp09n zUbgH5_{l)|8PGR4ybtgb0t5&UAV7csfw~3s4a%=a)Qr}V49w4fIo;#?DEB#-pRH_{ z`clX9m7hB2;Jl0(|M|&4*%{C`IK3b62m%BM5FkK+0D(FM^bN|cMbwJcj||Mu!0*5P zTIayL@vFYPc;K9andcmsFFGAoy{Y+QnDL*T43wV%`+?K@0goU+fB*pk1PBnQQ$XLK z{CY&qXdTJGybR5Dec|7sjr*9w3nDvqYeS?m7Zr(Zp0t5&UAV8pT0eu5?T`zT=_t{t0U888) z&dc%O)3=Wg@2;mIcm8PCDw>`#`z>ZNP<{sV4LaYsdHVzi5FkK+0D*=D^bN|dN7Rhg zkqk`FK<2?M;|1Kic3Q^Vb9;QYhDT;@%~&=qXS{gHK=~QaHyB2FUkMN(K!5-N0t8wU z&^IW*9#Jz|M=~%y1MXKleQ%X(JIUI(tX*C_m=4_eGp?^*%9nd1%YY0tmjQi)&UbF! zJ^=y*2oNAZpkV=hgK=|2=F&LHKs_07Ecm>gmU|qG&$xM_=^1m6gYo&YzhNZ<^<=>Q zVEX>RGYAkMK!5-N0tD(1&^M^3E>S~TS28dz0~yyW(*-j3A!Fk*_V&%|>A;?Glb)W8 zjmsD(PBKtO2J{W=+z1dLK!5-N0t8wW&^M@~CQ&j5f8bdJ2oNAZfB*pkvXJl}ibtMDiGN6W^x9`b$8ZtI6WAlVP>-n=sGWNOeWIJOZ z1MOtMXAb)|0t5&UAV7csfmQ|d4ce(&)S%X-41CQ%<|ZxE1>CcC{221#-Su=}&$#iK zV|>Pp=d5I)jtuA<*troPK!5-N0t5)ODxik1qb5;1T2C@CJ_8xoEYk%t_aSHFa^_lC z)AN;Q$k(`hapEKc^<+TbVBY?~vj`9%K!5-N0tCtz&^M^3E>S~TS28dz1Hb?FYx#q3 zfv?atWX9zy+o5@)=>h)xuRrw-T%$+^WT3eW*dI*aA9w}<0t5&UAV7dX9Rm6WdNSZTGSl}#PafWDZa!zNZPqz0)AQvzSoMsbYi&1vuMD)90eyq%`vcD)K!5-N z0t5&Us6#;CKn>So4d*?{^{W?8>llO!d6l0&zSlFDcpPvctFD@tk-0Tvv$0tQWT3SS z=o?hC8}dR45FkK+009C7CJN{qsNY(v-@G50P6jS79!wk!`+oS=VN2FlNXzQMG8fkzV{K!5-N0t5)uBcN|kem$aQw2ow8dIsF%VA_5t zeHJ;JmNRoj<1^-bmTB3tE%K6qdNN>NFn(X)kpu`3AV7cs0RnXi=o{2im#87FD;b!U zfm~1Ba$3OsYNzFD*bZ|)2+KHuTsLfR^L1X{xN(z#GBcoWFm6BKQ3MDOAV7cs0Rr_2 z=o^$-gQyLy7a2G|1J9p5s&7!<;T5|^(fq8rR?)b?x<=9bjM*2klYz1`pl>j4Kj2XW z2oNAZfB*pk^$F-3lwFIc6|El`n4f`Mt0>3PTaF93M$!C?nJ+q?t^CwAist9b_|HxT z%FlrP!0G*fM-U)DfB*pk1PIhApl?uqJ)&l`j$~kd26BBA%V7a?y7RM@?NV>*aJGh@ z^z&49kaFxvY{fB*pk1PBly(3*h0LAiB^8qvCuf%7xq8uZ8aTIO`m z&zL!*!x?k$jPvtlKfq81%Fcj&z~Oy>pAaBGfB*pk1PIhEpl?uiEuvPmeq`YM4CMO# zmY)TjYjA$HvVH1J{hY1gJDE>2hAopbjvQs6%naxo3?sg;1PBlyK!5-N0<8(?88FdM1 zL_yfB*pk1PBly(1w7%!SeNknvAIol$QbXy5H}!jQz`JH1p(^uL5u1 zyk4v)V?13xt4x{7z;YSTH?Zd;K!5-N0t5&UXjQ=6(Qtr?Oaae9{e70C8U$$GWGEiOy^bPFw2oNAZfB*pk1X>kP+m}~=s0pnT8CWj^ zz274__d<5f`StRZduiml{yi_5LoK!5-N0t5*3A)s$iM$Mu2vmRt%+YIz}4_o&%*fw9;o_o62L3#$xW!XMs z_8ZJ)VE+v08}v~#dzSkvz4D6eMUOr3oSSNeW1PBlyK!5;&-Uajx_N_J4di-Tz-wd>VJ(OTpp1+SS@YHUkb&(ppl{IRM<{QK009C72oNAZ;In|f!S?lqnvS^) zl$U|l?swpR24!WewcbJQXK;D(z_p1!-^apu&OiqG&Vaswl9~Vk0t5&UAV8p10eyqM zYaO+&^)Cb8GLU)D%!k&#WDdi6{kC39u}JTrh3jH{`uN^Gko60UF)|98-^Tx8Ac{?wx_!<^kTmd2K%T z);!MM`8oCXFCO2osy~%Jhx1sB%cpYf=Q7e`25S4fP#@f3p5=Z_1~Rvs`S_BTTru+x|4x<8PGQ< z`BReDMt}eT0t5&UAaIU=zQMd2L2boN2KvcBsn?asTt%*#FN<|&vDEt>q$lMwLfzRi zHduc$uzm*g4bG`VJ(mCh0t5&UAV8pGfy{@gDb}wk7&DfE`ZJL6GIObWF2nU=^b6|G z(~o09j_EJqzR5Yq#%F`}^7Uih{C_*^IOhaqyY=5b277-l*z+@qzfFJu0RjXF5FpUD zKtAiL9owlLtxM}t2G-9&=2SCJv*q&P-SwC2%lFbR$Q*i(b-(>zPwzd)uWmgTl}tOo zVEysK7|nR;3^?9>%e}F`MSuVS0t5&UAV8om0rLcE*rjV&zfY!m$Ux??GhegTJrL3> zc=h6Gv)7i7U+&Ei!{K<@v3uW`=h*x_x8;3U?y-^gxmFJ$?&*W|(v#_7T=E|K+yAD| zY9T;?009C72oNAZfWR68nKw{p_E%?G=hnFlESrJMUk`nRsq?(K{!r$^(7f82-z=@9^*8Ypd^lcwfVL8EfPAV7cs z0RjXF5I8N6`IJ&>AFo3lE(07s}^>Zurv-cs(GO%6-a!rivGiqOQe6F0VmlL*ZWnkM3 zv@kB$zF+bB2@oJafB*pk1PHV$;Mxt_)_3Y@-ZCHqwPhgti4X6to7oL!o-KWl+SY^D zYwVB#8Tc^+Id|AXfB*pk1PBlyK!5;&H3S@csRsSA2KE0kAp4WGU z$oY_SUuB`@4AgYLnKfz|zmos~0t5&UAV7csfpr8jA7t)Cy?6u3U46KuZR(l4+ z-c;{ko!rEe(J-v}N`L?X0t5&UAV7csfkg#UZ>aIMt?_vCmH`)SW4?F0YWey)vO)^W=j$aP081PBlyK!5-N0t5&USVJK9RcpDP z^4`oB$bbyUz_<+b)H4|Nr&oVm+6U?#$bbw~n}OWF-a>!?0RjXF5FkK+0D(0GuCHEJ zTPv9h95)xpiIWV-z^M%U@4x?AV;57!@4UQtU_WTySO#RE+6?5gyoCS(0t5&UAV7cs z0Rn3X1PBly zK!5-N0&5H8+NeF$Rod8RvS!N+XfB*pk z1PBlyK!Cu~0y)+)b4TjKdg?>#%DR#P8CW6%?y320kD6;WEn(_+*fwQA2DZ+?^JkB~ z4QBpDfB*pk1PBlyK!5;&1qD)XY+d8{d+KT#kO3LkF9R2k?=QF)ssC5$q3@SP;{baZ zI4=Xv9jkvI=sstJ(PRlK~m1DFbicye@T@wA<@)-9FoJ zP1~^7DzCRQ;QX=Ob{+mO0RjXF5FkK+0DM5etdlQQvw7C5FkK+009C7<_rAqzyEU1fpZRIKn7%>;SA(>ddqPE z`@-_(L(NfJCo(WD1G#6Jg#ZBp1PBlyK!5-N0!s_zI7@ZexVnrJCmE1|+A{F_Z@(_R z+i3FlW!|l}b?fySk7Pgwl7Z*X9yK{Gds74m5FkK+009C72<$JAdPD80_LKn`kb(9x zU_NR5KKA1Aefz?O&$}_cwKu+5r(f4;`olR6$U=Ys0RjXF5FkK+0D+|iKD@g&S7NS2 z24p}68q9!uW{%%uzIyT0HqhWUU~XBzpoa`({IU=rK!5-N0t5&UAV6RZf!tTEhq}sp zG=CriGSE{7*4SUP`a4~#$k@=+*xtP@3lHKdtU?y5FkK+009C72<$A7`(~;k z+o~b0PwP_#`p!W5!Io(Px!#BErtj^>`nUdT&A{6?ucr-o9!-D%0RjXF5FkK+0D+ML zIaaCGTFL8H=gWW$$iVj*_}_p3HFEU#utDJR;(`63YZb|W3{;tcj9(T41PBlyK!5-N z0t5)GA&@yH^KReIyYb*512Rx+1{`}jeNX8g2e$WGw|B2wc9VfzZ_Pr0009C72oNAZ zfB=D|1^)i)PkjT&*vWtl$Uq|*$e3=KCXil>`B(F=G9UxBX25x7)5d&{CP07y0RjXF z5FkKcyukBkkIb1gGG|z8&FgiWuaSXeGBAGR_sBs&|9qM3*@ijFz&;tsJzDgPhY{UZ z0t5&UAV7cs0RjY;6!`G&THiq5Kn7$$1}e{hdm_)@Gn((*XTFm=cNthF16O}s&L03h zlK=q%1PBlyK!5-N0!IWg-=ub1rgr1VQ3hmSj|{j!=8>J;i67Yq+Gh9IX8ldMTPg!j z9^N}K%K0$?0t5&UAV7cs0RjZ(3FJ7XrD_+yM;$B!G9UxnXTUYC=k5RUEPaFR^J zE(6PEAopsq5FkK+009C72oNAZU=4vBtE7fowua-$QwC&Us|@HH%-=KW8`w^_+D`o) zSz9Xu&N-XE5AaL^1PBlyK!5-N0tAi=WX@@=TE_2HC(D2g$iTW8&^I`~k305LeS>wg zY1`*31KVWa)5rJ6hB`kZK!5-N0t5&UAV7e?EP=oO`cvPa^82Ey%eSe^Idd*C1NsKD zc70!-q;FuGE^(Xon&fh72J8ps?*}}S009C72oNAZfB=Ex0?(g4(mT*QkO3KxfeJHV zzUcV=?buKC4b0C@ouBo1nJF;?zyJ2@O^ySy5FkK+009C72oNAZU}=F5@2<@kRybc+ zV(scRnNyL0Z8M;6Fn^DzZ?J7vjSakIV7&}n{c$;eF!W3U1PBlyK!5-N0t5&g5iozU zUM;6iW-9|SuvP~24UX*CPW;Gz&vv=icIo%Z*47z#^6=h?5zmha5FkK+009C72oNAJ zPawxBZC&H|d+KT#kO3JuF9VtTw#*aAwLRqTy!`RwCj;eV;O(2&^9DoDB0zuu0RjXF z5FkK+zzKmItEB!br~b1ptP2^Cfm0c9jOmFT+mRoivh{P@pA5)AO&PencyMII^CJQT z2oNAZfB*pk1PII*$n`F2s*k)@^}GzofDHUHf6(vvnK>?3c8nh~AOo#s;M2$V9gpnZ zIspO%2oNAZfB*pkYYF`Q*PrH5+#gv6WIzTg%RugJZJa`_ova2{^ybob4KOwtlBv z?VW+#wxEJR5eVlt6 z*jD%4R{d@H+ad!wf7n8R009C72oNAZfB=Ct1af`UE$SM7L;WlRG9UxzXW-R~r)%sA zn*5#a<8yxAY_kkypzI7>UOZ@W{Pv~@5FkK+009C72oTs`z&#_&t_RiY){hLx!0imU zF2%V0rfU@0ZfHQ?18Zd<*Wt0;EnuH!yIX6!^Lu5h)C}YtD+>Vv1PBlyK!5-N0tD6&xOjYD@4&H{ zG9UvoP)P<{m*ViAF!u>9b#Bk=GVd${GO$Jl@;Tl@fB*pk1PBlyK!5;&H3VD>S{=Ja z9qV_>l?<$tfn1luIR`)Q4Rh}YnOi4wT)E0X*%`>WRu%#T2oNAZfB*pk1PH7lka;LI zVc9jI^<(|WfD8;7aL&QcdqVd$uzd{Mhp)2GTL#?w=I4Ea?+_qBfB*pk1PBlyu#7RcHE(lXyM0E<8Tj<^y>kK##%^r~e0RjXF z5FkK+0D-jxa*nfkx$o!Yc<_(`87MgeuUX@3rr;jh4KPdTsK#RR(g+Q40YA z1PBlyK!5;&|Ht0l>qv2ReH%X?9|32;5%>r!Ifj&k$P$TV(h-qGHj)z(kenIG(S##s zmJkvN0U|)8^WJ~_)JxO!%yjoub?vqHuOz#N`RJ~?)>V7gTB|+~Kwt?1d8XQ_avPr` zkLCaeI53X`Bj2y+;;(b_mUGNGzyS{Yp92{aEeIfh00IagfB*srAb`LU1kQf>QSK%G z<^TscP{@JYqv!Ldk4sDmO}}UEub1;{^`GYHKjDZY2kJYJ<9G`K2q1s}0tg_000Iag zuml0;Zr0Bk<^5vFf$F=_xT@X$upr_<8OV82^`=+3l5Al7mPCJ*-iuyKmY** z5I_I{1X>~R=O4eS#}v9pNeelubxaBpkRL2kn}`@A@Cpj`)YA2$mE2q1s} z0tg_000IagFgJnZ8gjvQb3w5Y8xC;bCI@=%9q1Ww|0ehSj2|5M&w-o+WI+G{1Q0*~ z0R#|0009J+AfPrQ_m+EefCF_M=w0Z19Ll&-a+c^)fdk|wr|Y6Z+JgWjo?7f*Pd0M*Q`GR z2q1s}0tg_000Pwsoc;2n-ht;FaDW3GDC|JS@po@7%T7n>VduQr5o&z4Q7xMT9GKIA zjQ_PifBl#e2zzIluu9oZf-d-Cw_aw!}2C=zHe=+HuQNfw3L<{kNa4^4$B@m(;mj z?>4rp#wzjWz&sB0_N<%M=P&Dr00IagfB*srAb`L~1f1_VPu?c~#gPLXXvKky>n%R_ zAnTMGS1X*Xhkcd<9O%J;3jN}dax2@400IagfB*srAb>zG1d?m?kTcl^xjzRuz=5w0 zJbiTk%e{+Q=pSUBNiW0oZC~TVRV+Bbfw>&W_-a7_0R#|0009ILKmY**mLPC`_E7Jj z@N-e;$~(P}8U_dYb0D?))RfjZA2pxp`hxzPGERso2ikHV@Ke+hoe9?OtrB{$=G<$vlf9Lks`*Df`*E^80-GTrD z2q1s}0tg_000IasK_K_EkdIxTk70ui2RP7^1L+~;I)uxMS9j_o zeZ;o#xsn4Je=P_gfB*srAbOjV83jzorfB*srAbz^+5I_I{1Q0*~0R-kIkX+-g`HH`n^KyU#9N+*4IKTl8aNzU~ zT*cqXb2-2P4sd`29N+*4IIx}r@7`QWXb?aE0R#|0009ILKmdW+34H$a z@#@KgdwK_+bHD)(aDW3G-~b0Wz=2{8ymz^+5I_I{1Q0*~0R)yHaDMhso>(wXZ0TG#>*Cx?4sd`2 z9N+*4IBrIKTl8aDW3G-~b2Kap2vXO9>4E2q1s} z0tg_000IagFgt{7B;0+N# z009ILKmY**5I_Kd5cu~Rb5sT%B z&g)z6ygux)=Ku#dzyS_$pd|+`{yLY@AbNsIKTl8aDW3G-~a~-I`HQozs{W=cs~RX zKmY**5I_I{1Q0;r3xPaWt=Ds~Y?t#3Iluu9aDW3G-~b1Db>Qiv`!X5?5I_I{1Q0*~ z0R#|0U=aeB7q9dU3Z7rqYyNAy)OI+)0S<70103MM{tjIHb-qZB;7t)g009ILKmY** z5I_KdoU0^1+dn_U3M&q9fCC)h00%h0ffgLdwZj$!5I_I{1Q0*~0R#|0UdAw9^2CC9 zVhiWFSqJA^a)1LI-~b0Wz=8D~c=7!4;Mt7LM*sl?5I_I{1Q0*~fu#r}-;nREm+xVZ zJqI|z0S<70103MMF%IPXpPUC|K>z^+5I_I{1Q0*~0R-kI@ZtSieFN7UaDW3G-~b0W zzyS_$pr8Y}@2CX<1Q0*~0R#|0009ILSb{*Ft9HyePX5NZaU9?P2ROh14sd`299YkR zr;qM0ksf$Y1Q0*~0R#|0009ILK;Y`~;+4KZ!Sl=H=c{m3+zyS_$fCC)h00;JSAlD9C5I_I{1Q0*~ z0R#|00D&b41Q0*~0R#|0009IL_(CB0 zMoZ^SSr_N=aexCH-~b0WzyS_$;C2Uc{!h*WvLJu}0tg_000IagfB*t>6Ug<=^0M3W zGK?_d00%h00S<700~~10f!uf0f&c;tAblP0S<70 z103MMC=TRU-hu!E2q1s}0tg_000IasK_KTUjgnj0PIUwhaDW3G-~b0Wz=2*I$hE^3 z1Q0*~0R#|0009ILKwt?1xi@Mrxs&Zs2jBn)IKTl8aDW3G7|DS=v&@130tg_000Iag zfB*sr%uV3)r;k@p9^BJAaK9oBaDW3G-~b0WzyS^vbKu4E$8)C#-VXr;5I_I{1Q0*~ z0R#~ELLm9ZNatnQR_7sdfCC)h00%h00S@%wK+gZkc|aBf5I_I{1Q0*~0R#|0U~U4r zzPX3I$u`LUIluu9aDW3G-~b0kb0GH}wIF~10tg_000IagfB*tZ5Xf`YM$55muet&U zIKTl8aDW3G;6QZ;o<6$2M0(&o5kLR|1Q0*~0R#|00D-HEzs~gyirvq)dX8)T)oD1u z0S<70103MMJPur5ypqu%fB*srAbBF{SZI^ z0R#|0009ILKmdU+1d?ya2iwgD#YSv6zyS_$fCC)h00(wC@Z`b0tDFa9K>z^+5I_I{ z1Q0*~0R-kI@b1l}zJY5FIKTl8aDW3G-~b0WP|$(bFQ3hw9(X?l5I_I{1Q0*~0R#|0 z;0u8~S8dlhOupybHV$xr103K02ROh14z%mQ(?|DZGzcJo00IagfB*srAb`Lk1TOwM z*EcA5ewiG;-5g$Q#D)VL-~b0Wz=2aaaCz}+ksQIBB7gt_2q1s}0tg_000KEz=~TIm z&yh!SfCC)h00%h00S<6r9tY0P9!h2qKmY**5I_I{1Q0*~fkg=X`NyyN2Cg~a00%h0 z0S<70103K$K?gp(f4fMI;7t)g009ILKmY**5I|sT0-ry9lp_|*5uK+u&v|+{;>ZCG zaDW3G;6P;uat@FM0R#|0009ILKmY**5Lkl1i|3E^4)hKUB##aVafb#j0M9N+*4IKYAV9Ju)F zeD3tX`yqe;0tg_000IagfB*tt2>kxrPkIM>2OQu42ROh14sd`294O|%(?|DZGzcJo z00IagfB*srAb`Lk1ah8|^SZ z6c=&f00%h00S<70101-?fm}aqK>z^+5I_I{1Q0*~0R)yHko%+Fl#lqCe3b(n-~b0W zzyS_$fCKG0aDMi1iS)pGB7gt_2q1s}0tg_000LKk{_(56fqNBkfCC)h00%h00S<7W zpaUP?zm?G-fB*srAbS_P z6<-c;fCC)h00%hGp96VbnFRp^5I_I{1Q0*~0R#|Og211D{HkwI?7DP$e}8$un2r_G z^RtInuU|g9%6X#O=ZfYzm(%z2JHP+-lQG&D%>fQ@fCJY$@ZtU2B~k?Mi2wo!Abi%H8xyyn1 z9mqXLEeIfh00IagfB*srAb`LU1X72R$IYL|VTvgS#&saI{daFJml(3Yyl2MkvtNER zcDr{82ROh14t%*LZLtZ*8zX=K0tg_000IagfWXiM)St$!Z>nvJJqPA>;FR}_aJ|^P zv9mAW$pH>jbRfs^76cGL009ILKmY**5I|rF0=ai~#XQM6%kMeBfmR&Iy&`fQz^Rvv z-}&rTI9Lz+BnLRafzvy%#{6^Y`OfDdfB*srAbl;9Ns1Q0*~0R#|0009ILn2kVk4LO&* zn*$u+z+n#LK0&Q4`oD+8j_Y`FfCD`_aLVWFv@-8m4+IcE009ILKmY**5I7A1&)Aed z_mn@|7CtZSK#Mh+-+%i_5BO*9FYUhf=NJdtb0F8hSr9+~0R#|0009ILKmdUy2)NHh zdpV%ITueC7rUOqO-CtrDzVmzL+A+^MXww<{vbb=714}#5(lu~*p3uAj0tg_000Iag zfB*srG)3U-mmi(0>ReS0aDW51IM8x!#{G)aCYP>FdVh{_piKvI9B)AY0R#|0009IL zKmY**mLQN?m|U>UTu@xZg#&Fk(CW1Y)12`6(?{1F+|u{4FSW5RiGw(BV15U3{jdcA z1Q0*~0R#|0009ILn45rl)BO3HoETFM^yNUy*BVT7f@==?a?Thco*dx7z7FJhWflYw zKmY**5I_I{1Q0-A2?FX(a?^ctQ@_WR(HyAq+{yHSM~jW^H6C$*1049zfjsNRf&c;t zAb3KEZ$B)yPaDW3NIFNIIEC?Wg00Iag zfB*srAb`LU1kTSM>K(X0HU~Jsfh`9vFJ3J%f#*WK$UCb z7Mc9KDFO%}fB*srAb*A#bg=fL+4RO}mMoyGF|SYm<+ z2RJZ-1G#?Kf&c;tAb)X8c zbB=JJUk6@1e>``3;QbIl009ILKmY**5I_KdF9cF2lTY@WPl~PBa$tW4D)tRL<9`46 z+840m00(+>;KTd3vKj;sKmY**5I_I{1Q0-Ab^^&Ydd#70liC0WIB=)~75fHyzdS$G zbHC3S4)o}aj~|K z;Kl(C^y0wNNB0-W54M4J+E$RUr;J~2{RIIxk zx{ZF{e!&3_aG*~I-o3dTHI1>|2q1s}0tg_000IagupoiYpFUnad2mnfzh*AqP0nlLP4|TM$410R#|0009ILKmdUy2%Mii9J*%MQ|@S6 z)IvCLxC6OgTZJX(l#1`+@x=@?4sf7n2cABVI(BID~EZ|YhD`%mlpkM-jy2iA4q-J8ofrW@~p00IagfB*sr zAbtM~^fu4by;I5jW@9}~I9JtMamadtT zS|NY{0tg_000IagfIwdaa<0;CIfdfAime{$6a5 z&U>Hz@}qHAzm5YO-~b2i>_B?Z76cGL009ILKmY**5I|rF0?v6=OWmiI>UVfDngi({ ze0cwMwqZB@g1qNw9Br?$g99Akz_JeHJRl1K2q1s}0tg_000IagFgJnJ#pH9#=5zSs z%YnKMq_>)99~@Jc$=~D|2Kohc<8R*;I}UK5bq7v$kG;8b5buWo0tg_000IagfB*vR z6HwEV6StldTR)Dr;6TR8JQpU{4&=Yh>k;zx^!?Ou+>3~_EjVi(c*ueN9eDcae*4LW zHADab1Q0*~0R#|00D+POaxSgBvj4nNjK!D(9N+*4IKTl8^yfgXZL=VN00IagfB*sr zAb`|LWAhO}009ILKmY**5I|rl0?9Yzll|tCVk@?5 zI*@yz<$1yRFV9q&o*_KF=9;Jc?ELJZacIH*W%|Qm9M+r9c%I^rdt~QlnGf`mjTvi> z89tkT95~E@jM+I4$btX@2q1s}0tg_000IciP2j`(x9W8Z*6ZZIhvmO`)#r8g%a31b z(-~V1UD8`f{aWq%?jB4%n=n0e$$c*p2hYH{dmM~u7;=CEOFNK$umu4G5I_I{1Q0*~ z0R#|OfK`2EOL{JI){t^7g8spAu`uT0#sLnT z!hwv@76cGL009ILKmY**5I|rF0y!V`6nTx$kUw*vKL=8;n5$o~J-+E5?C(qZ1?jac zb-xPzgZ*P+oWqI(95}TD$6Q;sJ?Hv20tg_000IagfB*srAh3&o^GW5nr_OPGUVZ0I z^*qMhHz)T$;>>Pm!seL!LhYIhZu(w&5}rk|I|kgug##Q|(}7&`XF&h~1Q0*~0R#|0 z009J+Advf7teNllZ22|^>Nt>d+jIW;DHk>C-F~EZv(z;P*Tpy2HP~Nw#{(B!IKY9m z95_Y)__}$)_Ypt<0R#|0009ILK%id&sXKWVsa$uhT-WE;aWCUU&i`L)Nu8W8|M_z2 z-pyvr^V#B1XTKFIvEo4g4xH*5z|Fb9*9ahh00IagfB*srATS02^|=0XN_7Jg&%Aoj3eG0tg_000IagfB*sr z)FhDWo8@rx=5To8*^>inp5MMbs*e1;gFIuPmBl@ajB!1Uakhoe)g8!vM=b~-fB*sr zAb_EzhuT?R--6#{B!6F$_}~PpG4$W* zJv|?%-+kY>F18$~=Rl6-EeIfh00IagfB*srAb`LU1X3fEC)Udo#Z1hWa-gOA6K#)q zBi*m4rTY_Y$HD!Ij6X|_Ki-#H95~E@Tmxo7009ILKmY**5I_I{1ePF>d!x$H4$IN- z!mC#YTEFK(&cE-K{q}a{-b}47t~oFUZu{!roZ!HE4zxU9v^tMjF9Z-k009ILKmdUn z1ajVD&Xw`sIt)fSe^*YsUQUaB4fZWxYcS1dBivuVwdYGr>^EOsb5LWSHuj1W2l{s) z19^5pm8Iu; zb*1`G?rU|MSAK5n;Q$9Xu&e`>=8Y=H8P*d41Q0*~0R#|0pbY~02FvCWay5K8z=7Kw zc=F(08xz8wYteJPI`?6!vgCeBx4Gr##vcxFfCI}qpl?uR*0i1oAb&w~R9H=;l zPjiS@1Q0*~0R#|0009JUBB16Vf9oxOvpt-y=0L@oLp8?xtZ%jJJ~Fqzc>dV+2Cg^Y z00%g*q5~D@@SAc&KSKZk1Q0*~0R#|0U?QMzpvJmSJjPb9nOLUU&D$$$VIC(Kj$ojxtW#POi7=K*c${+FWOS z5kLR|1Q0*~0R+||pl=`#Yc&tEUfdkTfr>SU9%ECnZ(t5LPWHGj+9qCCa-iZIzE0NZ zQxQM_0R#|0009K55m0lGn^nrqtSeu8bl}DF$JM5e?yfbMys8!mbGR{aa@Q*tIIxZb z`UcfzPV0*R0tg_000IagurC2MhjnrcxgG8t;K1a-(?|FB9kQGFy*!U$TKC>wKfHh2 z1mk^IqR0D1_2!iAG1haSZwK@Z_MNf(9s&p;fB*srAb>zM0{RAh=Q46t@#R1b2P)O8 zbA3dwUcG+#tjsjtbN{ASzid~H@mQS1=@19>4a!WLRulmQ5I_I{1Q0;rKmz&(a-&0X zBfrZR4lLt9rFwO)kLcBd^RtHs&Z6u6dTLd@#=&+iGtS_OD+lIxK;K}!dC(^#fB*sr zAbEp5!?@aIBbN}Yh{m1Vcb2-3) zwH(klm}ye;HV7bq00IagfB*u`5zse~XRVcI`CRUC;9CbO+^cAWbKEcfI&W^S-d>@9 z|M|Q7{*CeB+wlPpJUGy%1NsKrGn9WLfB*srAbOjla9Q6D=K&!1%ZAtABHX1M4}^@_ccZ9MRt*fB*srAb5YRVRKi`n^i2(;Vu<5|%#j81n_gUX##JQry`$oNf`D}=LMQ!d6zBWE`fCDW% zpl>ki;f6Ou009ILKmY**5NL?Nh_#@Wa}Vn#H|78ba<2QkHSasMNOpfB zW5cNXq3z~=%MR!pG;_FOMG!y$0R#|0009JMCZKO17i&2evu^wx)q%5Ler)1CIl1;> z)EH#DTj}BCo=I7kkz--o%?BLdK#LCO8_b-XdOHLVKmY**5I_KdW(epT$iZ67!K@Q6 zIq<{P+P#BZ^PpC}e(xYXgScRyFfMX{0~~120eypJ(poEm00IagfB*srATToleS`M$ z4>_TjaDW4o18emTa-D&m!B-F0y6zy)qt-L{8i#4W@QMisIM9j%`UW#6r``?$1Q0*~ z0R#|0pcw-C2J)^}@-FMaMGowC;M6ti)Ih}I`&c9nr+nUl-p%(h;3Xy;;6Q5*=o>VX z)>;t+5I_I{1Q0*~ftd;D8_2&}%fGA-Cpoa&fz&2)&zhx{oZD_r*v)>^$@8;^i^d@_ z*1Neq4x2IYHC`Oxz*-LI8!UBH;(ZZ7009ILKmY**)+V4vB+pta&+@t4TFC-pjE|{x2 zJkNG>f75}Ch50Y}fBwsJhNrL3&mMmHS?*Q8`8U3v-=XQb40(3*#D99t_Yxz|37DVz zw#(CV8RGuLfAckQ@J#3Bi|qIJ|pv`6Ui&*Ym|8 zEK~mq+kKbBEc;OU3&yPZVu2-=9H`@fzQMjThTlT~0R#|0009ILs765FpiW*QhZHLg z+}(lH`%~wfWBK==zrURSE-uH#C4HMidN$X^EVb*rmpB|3ha3mQ<-0D4W$I_H%Q-F% z{Kbs}H6745xa;iU?-4)%0R#|0009JAC7^FmGe?nIiW>(Gb0D?M0&`;hX2Ng>W{O7=$J%edpNPX5b4}SRa zyyA(;TCaKsKYY1g&{{o%Deig)<|uO%2YPfs-(VW$y&`}B0tg_000Ic~MnK;{4%cH2 zXPd_M`V{wW-)An!^$sK7XFvBQYGFyeeB}G=$HBFG29pQwSv2<8Y;G3+9v#p(m_~T7 z2q1s}0tg_000O-c&^M67^_auirg6Pq^PV`n=7&+wvCg#?t<u zpl=|j8zHB&jbnJ;%KeG9$KjmoK9Ym2UGuRSkCEoQmhMlq8H0SCI@?J5gl!#T4ie`v ze15nuC_FHJIX%*Yr1f8jEa;xs?OsIiPQ_ zIjZ{_0R#|0009ILK%gfA`Ud0WH*#7r@6Canw_IV#vx0kLzqvhC`ZsZ<_j+}%Z>g~4 z`jkzFd~JT>00*i$pl?uNwy}-~Ab=>0H;|uI%g?MYXSW?lJ-*74XVq@6 ze=q;8(!Yr-J=d!%^=~dOUUj=a^s-;u4s$LCZgxQ5pvn=L^+W&x1Q0*~0R#{@lz_g$ z&AEVF2_Fu$?LdY5xNnZjJ=d!%^>2EvS6Axafu2SYHGXKmY**5I_KdeF^khH<905pWk4E zO-~L~>ls9ySYJ6Z2ab23+P&N}9zS_-ue&;F#kxt>xwm~Ub9%L<*L!#M=9cX_ zexAavZ*~>uziRV@^+f;y1Q0*~0R#|Ohk(98-?@xjReXDQpkm*k$Cy;Co9G*?bKJ50 zsrm-ysUF8f+cb{X73aV0aoE2RKmY**5I_I{1Q6IoK;K}TyhaWy);&7lK1FSKU9oP` z(**1J&~NUhm10xun|C<8vl5CRJPX4b~q=&1GW1f#V&h zb{tY|j7H^Me?O!6_WjH>foGTUP`SKmY**5I_KdLkQ>_$f-_| zQ~3;@E#*M9`?;T=J?yqdTB%;$YyT$uS*03Q&;6UF#ysz9KIK4L4pbU*56KbzE&>Q3 zfB*srAbr{H(*K+=Tm8IwYO|O30uEXaj z%=%_lY3{2sH&{;u5I_I{1Q0*~fi?)}8}yya$W_I61P7|*?7cqQU~;Tdy}IZAP2%z4 z{o6JUGHzLmp8Ge`e&N+T%z?fgsB#Q)%W&++2q1s}0tg_000IbnM?l}8?_5T%D!wB) zP~~1lBh*MM)T=KqUUhq|!L+|usAJ_GMZNcLrg(TY7jvLr2P%xW-z9*)fdB#sAb=(^xkI5I_I{1Q0*~fx8eGp?)G?>pNc)U-69ttzUC6!nFp|zL@(lwYrRSKmC?# zS*|rOZcpQdS1wd{pyjc))w$SuA%Fk^2q1s}0tnO~pl={wtDdh}e-2L$w0h0KDAyWH zamc*V>XQD_w4S}cwtUTjYYmLsz3+dvZyd+<4O*S?trr3aAbM zVa2+42U@L@jyTTd83L^>uU|eJre`qi>n+|l>fM{m;o>mG!>f6i1N}LmZ_wIoZ+#Fz z009ILKmY**<{~g+Jxk8kU(P0`Vmdj{+PUtdo>!e7PD{^79=TTC(sekwKhfDQKk6I& z@7`+DnB^4<4%Bu)-(aq+)cYWS00IagfB*srv_?SRpmwez*AzbveD6RjIehN(KXQCh zA8uhut$L*U6Ky+}9!~luEi5BF=VLn#{{8(}jR___IMB*5M+;-TbwB_C1Q0*~0R#}} zi-5j?oUMnP%{K6O+kv&uu}+;f_1f+A8tva}-zUfU?f;DpE%Xlb42;vGjeoXx4CnO? z`pS)M4FU)tfB*srAb`ME0{RAHoo{1HC$+Z*c0_-RB{I00IagfB*sroQiSttJAwVKYKX*bvM_=!}rZo9H{SrzQL)oR-c0a0tg_0 z00IagaB2ej2KDn5Ij0zM;ARI>vtIfcz3QVk#~_ZLs%EWsb9*dO=UVz2kUZZ*@8>KmY**5I_KdQxVWNsGY0GHN}qu`#X?p59aP4r1qMcL2BY+ zv0p4Q@1&+Y_mcDI)v5Q3L0pZqOWeyV`Hc0k`??%CY?A%Fk^2q1s} z0tmE1K;NKtt|HeIKMox3Kyv)M_7Bo8$a(Gh1&8yWoSmzGkRF7&?D#mO*D^={Ap4Z( z7#tsmX}|D_AqQ$Xpl{GhR%<;FKmY**5I_I{1m-57)>11skxPmf2j+7i_b*Dmzwdqb zQx{FIz`4}(#UMVXZk;&nd*A)^Z1TIQQR~~x8;iJ|9>TtTwe)NfJJ)&48wcY&o*by- zfWE=p*{SzK009ILKmY**5NL&fzCoS5L=Gud99Y(YjEDIzwbRrAQkR~-r#`(b`%Smg z&)JMaVll-*efo59i1QQo#327=-^lk8qs^H4`gC!@8fy-;H5p8}yhV+a?4MKmY**5I_Kd5YRVhKOd1Z ziV+7mzyS_$fCC)h00-K0K;J-4jQ|1&Ab;u&%Sa;^2Ptr;o|z za~h`S0+@48ABWAcDfPW+|M1L`)5oI1TxK14ai|0O2J&eH5I_I{1Q0*~fnEuyVaSgT z&5!&(XW~F=qRExty}8_Lc>MRDzrXyx=i1KePimj((_CJ>y88U-zB{m zqwaqW&pnohA$`UZEM0sTD!2q1s}0tg_0 zK&u4QDdcz~=6JUGaPLz~yKBE-dmPk1I6Mw>^b5A*kp4mH9{LCS$78O3!FDXtgK*ya z{&C1y9(VV-)@GaaP5;i=lxsoE6^F$FFT6%{K;Ph|(bUfnKmY**5I_I{1S%8IH<06v znB&>z{k_jUKj!WiY>$~a$2XrNF4xB;HT|5EzT}eHt8=ZdkHO^UDQZNy4rKZ~ue)QC z1zZKKmY**5I_Kd zo(Sk0$oEFi_iX?7`@mW~gK2ElJNPdqYxNAKIOrYx7l+jC=e~db#CV@qy@UVake)%D zTyrrteLr8ALopfM0eyotM`fRl00IagfB*srAh09>eFOR4==q-Q|8^h9Ikhb;$-_sE zO~&RHmegrSjzenIYxWE#2lNhpSp4}O6YJYvM<1I;9uI8$x5o>3{KrGz;I>iI&k;ZX z0R#|0009K563{o01Ihb1FgcKWc(=5SdQb7()4qjT_2k0H_h&if^9&~5*S(H>{YPtc zIrok-Z%*^3S1z^T>!joEB?ABi<(`HTu?;5AWZ4{>T634aW}{aNvil7UzoA#!Tyj00IagfB*sr zAW)NlzJZ)i&c}f`aCz~n$w6$iI_cWaG`LQJ(dwk>SGBr~HgC6d&Bvw_sTG<(&7U0L zfWE=z$mnYX5I_I{1Q0*~fu0DAR)3J+$?rJuy#pBsT3$xF<{~`xQygL ztLO7hW9=w)($;Gasgve8!M*PrEnjOe?GvNS-7WXg5{uEUhw6Rbw|$QNj&Y#%xqF&7 zydr=A0tg_000Iaga43OMY7fWc0scn*ID!MMUu!UpX}ynCsmHhUEaZv%_Uj1O9JG9` z!4!`?1EcrxsnvUSO);2WN4g%Wcjs)MVUq%_NeIJ2q1s}0tg_000O-b&^M4T-8EnG_q83WaE^P%+urNdRr)uHMbGC~ zSLxp*4!zf_vyBy&5w86(&)1&k#c$d8=^IoSsjVXd2q1s}0tg_0z%c~$4VKL%_E;}uC(-AuWtQ5?wg+UT(7Rwzv=n8AC>wy**8Xb?nldWnRRolwWb652Af9+zD586 z1Q0*~0R#}}iGaR=Tx-o-%V)3aU+(==Y3a3IU9Er9bG^D!|0erxul4Fm{hP!g&tfp2 ztvjFLkN;W@=o@4dv><>00tg_000Ic~NI>5}p0!q<<#X3}FZJI_OYV2xqc>Ha|GMc) zk8#L0Ra<(U!>f%=z4kPF-Op{;`o|wV#X#Sn+RSNv5kLR|1Q0*~0R;9X&}+@1r+mb= z$ZwZ&pjvIN*Yn&f)*O~{a?SV6d{}Mibq=p~-AAsESkqmf?Kpu0>p4(y4zD)XSziPY zKmY**5I_KdbqMrYbC5r+mp@@&hkcc2B2S~EzQIId{ns9!HBqr|(Cc~b>l=R98uU6g zna}IY=VG;9tn>{I8!!Df0tg_000IagfIuAr`UdOe8FD=A*LI-JNcDA1*JifF=U2P# zBhlz_?x=XbD1C#kW5b$X=^L2OmzW#8XAQ6P4b~ieeKrCJAbyZTb6q79=*{wSbtxLR~2hNJ@yUqtmkUWyEm8aQ}n-msK@cyHi_$P4pf{w zs*SPM7XbtiKmY**5I|rZ0{RB>q1*ByKkwI(bw-`zKegA|-0PRmj)$_+U!MK)V|O*e z%!k#M9{UEFbE++_H|TesbPOc6w>hA1P;JDvz6czBZ$6w!|jrdorIrKPoRJ=bCf0rUM5I7pwki?svW>r=>r;QoXw8d81Ojy665) z>hM*Tp8GdTk9*$V@lOv9=o?fSt*s{l2q1s}0tg_0z@Y?suK)Cqi`WJ^_0kShsaNNo zN=vg?^Zl#Tt9$R?RH|3^K7Z$aSQVC@`!{Qji9Xx$PZbCB4JwS())4^&5I_I{1Q0;r z7y`Z59jfFU)>D4GmIGDl)mhKA7@gvCtJJG|@82ZW6`sl4d;cbJs8WkN`{l>3$EH(^ ze?G%8P-O?Ij8(_viT(xw1Q0*~0R#|0U~B^V26DB^xtevK$KTv%qvd6k`+!tkE!=K*uHt@SseT2sBh5nC~e&kKmY**5I_I{1nxpW z-=N<-Mvf}B`#RA2H3xZq^S;|s`S((jZ@Er7!nFpIKP_K#FiIaSv1q*>=NQU7JB=A$ z$919gaq2F4qQ6A|0R#|0009IL7@L5ccicQjjw|+?4zzmB!D#36wp@Q0;aY=fpJ@4- zgWm7iHN_yWx#pnNu_@1n*<4p&n^QS3rUNa{+hb?BwjKcl5I_I{1Q0;rt_1WA#>{c# zw&K3+K+frJaT#rlYW3bMBR>1J#b<`+S;5=;ZtwqYacs(UK_m6idUwqBIVQS`1NsIn z9z9wo1Q0*~0R#|00D(COj8>DoOJ3k_<(0iVklJudOLFnvneW=Z*3NYwu~wb9w05rh z$hGR$p8YY>H6OdiDBp8D#DT94=o_?jE_knv1c?F~aEMq3!>EAJ8{gb5!=(2q1s}0tg_000K)A&^P%0_+L(h z$>>cPCe2qo=SZnnTj7?wTF!~&6`?;{`fWE@oVN)Y_ox%4Gp1LNH`l#y+zK=uZ%Tv~@ z^I3WZ-^U?iMDAmD%Ei5jzK_E+S9rx{bO%m7N1t-+^H~TWfB*srAb2JkNF? z?|#nlO`d#c4RsmR>??`WwEuWrJ|-Ar+@k{-Pb>%^fB*srAbsipHN z@6Ry~aDW3G-~b0Wz=0ke&^M4%BY*$`2q1s}0toa`Ubsb$+imt1Q0*~0R#|0K;J+IuoeDw|F(+D7d00IagfB*u$5|Dq^%TMH#eZ(x! zM$NNQ^PJA>mJjdW{{QEAs@W|Q_i#yZtLzo(y=I46Mn31t*dd__2f?Z3-tc+dxJ;!rx#O;_^`K^PI{6*~2q1s}0tg_0Ko11unYD8jxn`~Sr5=}>=dnv_N9i><|N3rm55I7{sZVfQ zj1ouV=iTCv8g=R|H}wf_i$nf4?_n&wTP)Itx$D@J+O@IiZgEJj;4WiR#Y65 z4DajyCUG^69u|wKUvQVc&2~J}Q%mjJ7rHggMDWQzlQ(<2q1s}0tg^bjevTW9O;-G$=~efO=>oC_Y1bi80T8w9*fj4 zb1w9fOXBQY>)T_IYi{Q37i`BgHE{Qmx;+j{k4=fOvFY|WWNbu7wWpKHx1NsJ=qm8c-KmY**5I_I{1bQMMkCZRnHDB`g-#fJQx$V2gAT{gn<8aFQ z=w?jRtiO%LDSJ1YaZt1VHVzr9mp-?BlmGN}#-=!6uJ{^{wZ{g2J)u4@+R-|pG#}hK5xLBR;b_V^eF(i1+YHAGfu!Dc1sx+Fu@VZnVwI9iO&1-?cXGTOR}vKmY**5I_Kd zngsL>mdhjLXydR=jlRWYw0lvsn!D$``;o?j^hsJ=M!XKEwOaM$#c0PKt&UA2_HafT zi*4&^jze4Bzo+rmD*^~0fB*srAb+;2v9^VK>2LF&{+;^HDEmZ9_a~b8Pp|1`j1q^|$EFdlhZ@B_+j*K}UVVdUWcP{y z0tg_000Iag&>Mk~YFYB9)8tR1ac=cmgK4Z9rB2#<-ag8GqguVzVA>x>y5^vTXHrkx zcfF2sJyh#slWPtf8;y3{Huv_nzK`!3e|--D1Q0*~0R#|00D*l7=o`ov=gyx-dcT&Q zg?!!qK0=+e_49eB*o-jVw%$ieEJk=na?95mO#8zK*F&}5N1I~dnghpzBOSZVy{-BN z(@5eK0R#|0009ILK%h4QBh<0x&JVnw{CaXA_pPt6^nPA%g&Ju*>HWOkDmBvd(x!d5 z*Vk5`Yp^|D_I?afrJguKFRj;o-FB^iOj>0>-yVPc8vz6mKmY**5I_Kd+X(cYv&*N} z&!>8h!EO6jD?jhOUR|Ys(|f(TO8+MNWzTb8mHtiQ(0jeQO8;hrYd?BE7O`#X9_#8G zWHh%RfB*srAb*P

tCsV6K7IiziwM9e?L9KN=vU}Q)P#&dyV7IYR4g!#wF{D00IagfB*srAh0e0eS;b~ ziQKWhIIX+q9{%aQ_6@3C_tEQl?iKq6z0QA!j}GhpMXzI1wcfJ6L3_s;j{R!HMBiZD zImD+UfB*srAbIZn&%mCL+yuQj1+wYfyMviH}md&Rm*kMm!( zd+KL@&^HK$^_O0In%9k4zVA43E(a>kf9sDp)&Kzn5I_I{1Q0-=0)ZauCUfNh-bYS( zoddZaceO>|pq+?a=T+tJSFD@#cz>MC>D88A$EMWbt1XqsAnWd!Z7l~X&VSX$ChLm; z0tg_000Iagunqxz19{e3c~<4mRhw&Pzj^oOa-G5R4xgHH!Yli~Yuzi>O|s6rwx#m- zj7{x~O?Ma~R{PV+?pybE+#f9#73aUz1{oiW00IagfB*srAW({czJWYzwES+4?X7q( zkY3N}b)TX(#;}TYlk>BO`UVXgvE&&gbBxQay+_5KW@`+sPaEUAINThEit}FsL%&r( z009ILKmY**5GX)E-{9t4K(2((bq*BRy}wz-UTbsu25s!4x#v%{rPukdS{!=3?j!R? zwXrFEj_c+--*01_6^AzCpl?uZ*0H__Ab7)By)`Y6nt9$L=Wc;dBukQJtEV(CHrKQK`glB(9Kcv!Q7_p29&O7-g0b9;8I(mkVk?cc<&O7-fV`!|*9)jjub zvQJc6dhOrzJonkQHjW+j4JwUv))fH+5I_I{1Q0;r5CZxJZR8wsznXEVQm@Xn5j{Ir zrC!~8|E5yCy4PnLO#AsE#{p~pZqNOj#3J|psj&3kznS(6uQiXgT92E)L4{GyIwF7o z0tg_000IacLqOj^4%K=-*7EvQsaN-U|M1C;D)s8#`!_jXxx&(W|0erIm3npW{hQqT zr^3>E|7MyiygKGx!U27Q3ZtBLL;wK<5I_I{1Q0lefWCnoYKa_bl=sX%XIfu+eGc!o zV-@bL-}60+rg&7SpNvo=&D`GlGQ!xDURvv8Q?8qHk0Qr{(-`jca2G1fd#%q0)(-&$ z5I_I{1Q0-=O#=D`hvx@!D$F(=X#JXlQTFlH&+CmRo8I*Lb?T2T-@j{wYYis1QkQRe zF*dakoe}zI(>~$V@!Jv(=o_>#mRSn~5I_I{1Q0*~f%yrHlD99BCwNc!XB=qxnu8I} z=iS^7bC2#8m(kAWZMlY${?ev1J%63~z13xuKHB70tJfTiHa1Q1@ap(&P6t|^^IDw) ztQP_ZAbqwUt7H%YJ_`s zZO(_SK7(bnv1v0VzIOaOmjn6+Egk_|Cj<~c009ILKmdU`2#i)&oGVB0KJv{?2T~_( z?OgX!>p`vca7L_EXWwY8hm-TFdw&M+=9n~now2F4bKOTBn_BDPjCMWRv`=_-{5H1( z`UZ1kq}~Go1Q0*~0R#|0phW`u2J)x5^QTeYFEzt(m;T-GH?xnvg4#uW7 z=BpO^%NZL-y!K|-e&Ks<>=)JJpl{GePHZa>KmY**5I_I{1imMrZy;x?p0BmG{%h9f z^bEf4&uiD`Mn1oNd!Jmhcc5qRb)PUcePyxj=qdlb=GdfX(1ru2^v!22A9Hz^+5I_I{1Q6(hfWE=<`GlNpJjU6tPJO<6?oBlMbq3q}TCO`d^*xIccjM~!`}`Wu zok%Tb+lkTto!-Hz$EMty$k_DlK9SzRsmG=~=fZUc-^O9|@xb=aT_us#)vZd}Y4NzudR!6g6wTo15cs z%9^#_&GoTJFXdF%;^^I6ABSCImG9wohy(ft-{+UUg8%{uAbit{5APR~aLW0%ODu`6`x5OJhxF5ytd-`yo!fqUEW*gxbUUY0%z5dj zEj2dfeT_|R#Q791`HXd9t#7czk(KvE009ILKmY**5NLsbn&Ub-hTLu}?sM1ZQVa6T z-Qyfe4Q;L(UHS#77pPMo9*dmwp1#*yOZo-rNf=uXi^beylV0Xw{5O4FVr<%ce_tOT z8|<*V$pL+Xxn^?jg8%{uAbrQP{zhIu2 zq(&E>$6O1N_~q~Q3+^6^^bc|$sblW-p1(=GM!(?haR|e(Jf@eEu_-ZhJaP9pWX?Ty zY|2<>Y?>zqj(u=E!~uPSLyu_vJ^~0JfB*srAb>y>0`h#h(IL5!-<{8w)SS};NWPi+ zb?SBbFEzO7>(t`%bI-DzkNYl%!#AA6dt#p&-1K$glb`F^EE|WZb{*Ce`}79#U;dq% zXnvNus$RizvDjy9N*wZ^v1z$DWUR_DS~zcxO}pj@W7D#6*fnqX9_Et+*+(r1AbkeIkW(Xo00IagfB*sr^h!YAK;ARuJzlxM0S<70 z103K02ROii{Tq4snWh)ZCx|=XAYSB&m5h_*tDGg)f}n5zWN4Vc4_}Z009ILKmY** z5a^JAnyS35O1_pFTWUa4UHXvSG4+Ym>QYClvc0XYXXOX35{>@AVMwKK8`t zcspo4ELuOd>uLS;4d$8!y$=EiAb_1OHuniV=Kc1CmV6j(UGxn$_gG&efB*srAb8D*EhxBfay&uu$xU0cY0u8&3fji*==Z)4N-acJR~Vbpav)%<*l{mf?|fB*sr zAb@voM*fxBxVq4{`$Fn7r>c)mW0IQnw=u}RdCK0+6o+j=m%HGYiPpDacABR)+Zl*Y7PBJ!qAB)zG<3?MbHRgtCpYVzR0tg_000Iag zfWVRj^bO=+E#-Gh-k0dM{U~)Iy@UVuhc$WzQ+!eva!%=gafpwnI;VQ#zxk>@_jP_S zHhpEW=A02H*IJBCUmaL;j!68COO4A$%!OfgZP`(e@NtW2D5dyNaNjMZzc zN3Jtid&$_8^X%7R(8A}YpSkuN5f6+_j$d2YAF>Ym25XPZJ|6)D5I_I{1Q0-ADFXTi z@~#&0t`_P+(>|8_?6<&Vjdf_H9+X(*o<$>H{}B&cSw@~G);zy`JO1gdt}!mO@R_OA zw@|B2Oh)f7x6sd=_6^sjIfiXve9||V_DHV?AbSXysNOS`UcqxEeIfh00IagfB*tL z5*VotR3Y~mZ5>-YpLcUVAEg%2dd@paowVgXTK0>Pt~qFVY|1qUql``I-?qGray?Y* zW7DYjv>WZ1vDNKu{TQm{G1R&tfB*srAbRZio)?Tlc&*z=?+7arc zEnjOe#bK0tv$T9Z?-Yv>=8Ot8pb@TzYWZ4&X@3~ynuC`6Xj3dk7@O8UZu9xObA}3Y z#6wAFrGXEw%0y&Lf@n)ZXjWRr)u**Mut79eS=;SLxqmeCoYkU8R51`?Vic z`Zu{QZK-1x@9S9kCI>3b-_xA7BAsejXZy}C;OruVUF?Xl74U!N~jnl~yP z^I2B}5I_I{1Q0*~f%OUK8(g0U$dQ&~lUjVGrPq3OrT$HP>9t;6t$)*Vy}DBWrssNf zwK`n-96cU0RO;Vkj>xqUJ;uQ{Eq6RR2it1nRrVDN0tg_000IagfB*vf5YRVhI~SQF zpK0+u_HnbN-|2Nfsn);gwI)>M{^8R;*>k;m+Qurcz1FKMJR^M5k<<-aoNMp8PSxt6 z@#e1E?C%{HAL>B0{bY0O@-+eoAb>gT=T>K|>~#*WHa7Lz(=;}n_SmS$adm`ks<@xDc&u%m5I_I{1Q0*~ z0R-A3&}+?Mggm9{Hdd_7_1ZV6bWfIyp85vu%pq0BpPSY@^I^56*RiQu9C{s_a!;~q z%T4>ZpXq}v;eftDwNb+QB7gt_2q1s}0toC&K;J+PwL}he(|fATwd35r<3uaJ*X#Xp zs@*RtKJ++u7@Leu$BuQq_6@3C_mTKLeRSWwihgcqU%u&hu$Q0d8yq`w`a1*=KmY** z5I_Kd)(P}j`{^YQ>2*6Q*5-Px3AH{x?OMMc_lt_PxgO_^inX5}uQ#anobZfOAKt(1 zvQ}+uQh)Dp&YgRkD$X6dj<0+V0R#|0009ILKmdVl0zKA#=FSJa-@JKq#lAtW_paU^ z4_g1X>kZn8P>*v*wdYgEh93I{nRBZxz4n$X_6=NbFz+#&W8m9)>KjxWv8^ux2q1s} z0tg_0z`g`}to_JwZqISnWK^+l&}(hZ^9|bAS1Q&Va!&Y~g1wK3NEB7gt_2q1s}0tl=_K;K}VoI&n&7{_&NKmJpD?Hg1( zm$cW~T*aD0k7HiNXSVk`hga)qroP_eK4EM+?AWG92W->r4piJvjz1#ymk1z$00Iag zfB*s`5$Lt%AkVoy&smdEwSBDD{iNFSczdnQy?Fk()GpEM{h~4^R$F=IbpS$P>=Uy ziObb;-PG56+y`?%?n+DgO+CgT+f?a1_n!MV)yAeg8>7d4qS`e$z4n%SbgQRrsy1F# zI^M9Z2q1s}0tg_000Qe0&^MT;4&GD#bBb-Llw~ods)xGv_5{Enk zy~@&a|E5yCy665)_K6Sg-&WZru6pg?oZ`65XB?joR63?Dvol*!1Q0*~0R#|00D&e5 z=o=iLC&;mmXIH6SoqVw8SXZf6_uRjUUzO_BJ@;=a)vJ5%-(;UiJ-))ybN?oBsPt^) zp66=YcKk8t((LpNDvZ?D5dj1cKmY**5J2D<0{RB+<{?YxIcvXvm2((VM_ilLlAm9t zUfp~Drb@lK_x?@xwJP=Mp8Ge6LzQ}U@BN#c!(UKBlOV{qntzD@-oWUlzw5$ z%LqOGo5n0ZTlZLA-=O8u!MY)U00IagfB*sr+=aj>bx}Fhx;fQ#{9CR?j8un64YJib z=_u#(PWwo_YjqjvnWHV&fkwF2VA>yAz8-3nKH3zEmizpUW!pT)o80OBwdMJu)#C~4 zg#ZEwAbwqo@4*c-P`GQXehxZuy#nQLZ(Z z_PbWEhZ^CYT~iG5+Uj!-MjM+pW8&*Jj^SH9eroYp!a5;<00IagfB*sr)FLoiZB&lc zMvk=^hZdg)n`Z@YuHy(_r%u}1{q;w92JbGnT6ehNV8JkAyqwN~2d~e-jTYZCCxv#ZF009ILKmY**5ZIr9zQMY=hFtGI{w?KP zqt=7sVrs*!+^=Z#dQeL}oYBs!-W-=(8kF}3PZ>p_zTtvqvL z^mp55dc1ZRzVuZ`y!K|=7xLOle>t_<&GquNSU)}@_D0QXRy6*#>+MCSv2A^KDOC!{yV*c%Zpd1+#SAr zmY%`4<5FU>M(;q+;QKgaY+B>F6RtD(J{Fr}m9J0FN8jM&Q=<<=009ILKmY**5I7M5 zeS_2I7jnAWnV+&|oj%3r_y4%g$y3&>Qy=ww%ia6NDQnhxH@jmn?IWkG59!@pAB$7g zto3fLk3+_$Q{BfaW0Pan>tivEQC?5a=#=xqiPA$Kg8%{uAbkN0(f8eS@X;eD8|@0tg_000Iagur>jG1G(6e zxzrr*9Y^QvAE*T#=X|aOn7c-o_j8TGVev?lFy|^ zm-~w!dro8SHIdv-E$D8}r$(2W+OhT0{9XQDzu<0hm}+$K`j{pAQT8Wuz}@1I@iVpl zyFB~A*mU<8Bo1*oy~tzd2xHScahPLlN*s+%^TeadF<#~M)HgWhD8%0&fB*srAbOUV}Iu2ea0sDNn1W9Q*6A>$6DWD*U_KvA%Fk^2q1s}0toa#K>cXGyg?3zB?mab z0S<70103K02gY$g-=K$#*ft=500IagfB*sr>>{9VFiu`0hZSoMaDW3G-~b0WzyS`- z=YYP!t_;)n5I_I{1Q0*~0R(y=pl>i=-XI6Vk^>yz00%h00S<701LHWLZ_qih!-_QrIKTl8aDW3G-~b2ab3orMVa1vQ z9N+*4IKTl8aDW5zIiPQ_E5r0X1Q0*~0R#|00D&F|=o`$JH^{-TKmY**5I_KdT?F(E#>s2suwu;t4sd`29N+*4IKYAV9MCt|m0|iG z0tg_000IagfItrf^bO|A8{}YEa)1LI-~b0WzyS_$U>pbZ4SL9kZ36-bAbv9gB~(s z+kgN92q1s}0tg_mi-5ktIC+g6R;)R|0S<70103K02RJaF1NsKLGECn?009ILKmY** z5a@w`zQKHXgB%P?4sd`29N+*4IKTl8jN^d5K@SAP~fCC)h00%h00S?UPfWE=5 z4Ab`zKmY**5I_I{1bQH#Z!llpAP2*e103K02ROh14sd`2<2ayi&_hOS8xTMM0R#|0 z009Jc5zsdnC$EviiZur~zyS_$fCC)h00-uCK;K|jhUt3y*1oRE&$r_jmb~ zZyv9gB~(s+kgN9 z2q1s}0tg_mi$IQbEwHexN-9N+*4IKTl8aDW3Gn8yKqgIyV>?;(Hy0tg_000Ic~ zK;Ye*OML^^8*qRF9N+*4IKTl8aDW3GIM9I}@ZCGaDW3G-~b0Wz=1Iwc=F)hb?KGwBY*$`2q1s}0tg_`1%aoJ?mM^Wz;lb_gJa}_ z;w;V_-~b0WzyS_$fCIZ6`2Dw^y2u4>1Of;kfB*srAb6wdF2A%=K`2u!T}C&fCC)h z00%h00S@ePK;NK?RMz01m3;5biIM=4LHC74sd`29N+*4IKTl8+~UCJPape80&N8X2q1s}0tg_000Mgu zc=F($9O0H6!FgH7pO=Llb{yaU2ROh14sd`2)g3td<;OiTE5CvO0tg_000IagfIvS4 zJmWwfS3Qrj{v7522ROh14sd`299YtUKmYi(p9IjBAbu#4i&i1}?I1X@t103K02ROh14sf8V z1Fv5`o6;t)2q1s}0tg_000IaMMIhJRRy~fl-f~b5aDW3G-~b0WzyS_$U?~Saynj1X zZfIi>KmY**5I_I{1Q6IH@buAry@Q|STTA6z-j`b(-~b0WzyS_$fCJSWI6r&1nKJnr z0R#|0009ILKmdW^2;}ESuR&8m$R;X~0R#|0009ILKmdVn3ApA!K3655vz}b#00%h00S<70 z0~}bw0oNLQn{9eK(D4sd`29N+*4I54*Zt~J=5Bl<1^2q1s}0tg_0 z00JS9^K9lm2KRpQIu3Av103K02ROh14sf7V2Y&zUC&?fJ2q1s}0tg_000IcyK;Ykh z{?<46Sx(n#PG`Ni$pH> z@8{e%4sd`29N+*4IKTl8)Nvp=&*x7cZ%>f?1_B5mfB*srAb7)DIo?j%d+CQ(t3M&q9fCC)h z00%h0fnFRq`{hSD9|8y%103K02ROh14vgV|p24ztp?60B0R#|0009ILKwz6ddIwJ*-Pb$NJKz8ZIKTl8 zaDW3G-~b1jb3o5vJA?CY1Q0*~0R#|0009I}L%=7$f-wd zAb~j-J=re)|9iIKTl8aDW3Gn8$(S79ZZf-A+6F8vz6mKmY**5I_I{1PT(k z`0Lzr4_u?Z*fr|%@OkoZ9C73T2ROh14sf7V2l5<)%Zpb9Ga;*t00IagfB*srAbqkQ_k& zppBeNZY&NQ-~b0WzyS_$fCH^KpkHuER^@jQKmY**5I_I{1Q6(xfc}9TwUr#zdT@~g z9N+*4IKTl8tmD9w2luYrkEqXl&sHIT00IagfB*srAaEA~xo2sfpZxo8Kka?5Q+eGw zc^&S!bASUJ-~b0WzyS`-;lTOX!!J3&UGfNjivR)$Ab`D%`W)L^5I_I{1Q0*~0R#|0 z0D-v)ynA!`<@}`d5!}=0G`XIf8fOl0fCC)h00%h0fqfmwF}gm%+*y$KLjVB;5I_I{ z1Q0*~fkO%8yrqBt`TI*hA!~-8 zWAEHwG{@rZJ9KCkYwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@ z0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VK zfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5 zV8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM z7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b* z1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd z0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwA zz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEj zFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r z3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@ z0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VK zfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5 zV8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM z7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b* z1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd z0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwA zz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEj zFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r z3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@ z0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VK zfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5 zV8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM z7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b* z1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd z0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwA yz<>b*1`HT5V8DO@0|pEjFkrxd0RsjM7%*VKfB^#r3>YwAz<>b*1`HT5@E!=`B85Q! literal 0 HcmV?d00001 diff --git a/contrib/windows/Inno.Setup/openzfs.ico b/contrib/windows/Inno.Setup/openzfs.ico new file mode 100644 index 0000000000000000000000000000000000000000..2b5058e8e67d01c8d4e4d583cccb96476d90531e GIT binary patch literal 318431 zcmb??by!tj(C(q7L*ghcAt)_<=AYKKK8- zpGR=B)|$O?=AD^$fj}@I6wvbn10n-CNP|EH!2dft|Gv{G!GZ{6Kp=nrfA23~K>V4o zAbtJ6??nk9kXZp7NI>9Uxey4134sSOG5ss20)aSu;6XGrf8WhtgFvgf@E|g>f8|&p zka!zB2n_!F9)=789go6;uzk88v4Ti}04nSoDWn!_1@`Jt1qCb-vEk3f*5DM`m*GO z*$Fm^V($O{@uQ0M^8u)Si+F3hu(MC}%I0xN;O37d!EegmmgB%3M8&u_(+K0!kg>69 zCzgJ24PzCoZF69FI99iCSUQKxdz$c_B`&B{_io7V-te#&HS^YTG6@+|AFE~H6mXVv z$iy)khR0;55e^sgFia<}G9Zz*tb};`GkLb4uggou+s0AMTXQ1tx56H`3GD+s`s10~ zPi~vTX2dUlh!C>h2W%=}5xj+%iJ2@Qc0!^0OyI8>M&gs`m}mYl;F5A|;ZSs8+t-0h zq7j2*17IV%NLqx?wA5lEMqnnF5-k$=Ri+>5$r`J)gU;ywg=Q7H>-{&kd$zlmyS6?L zY)hwnOKoi?+lEohNu#wmFYO=BEq)hbDvCDPDyrfmA~M_@a?DrhCh5BRCvUpcUXh}B zp8pixH{J7{uY@@*3QjsYs&)caVWePef}vHn2x~W(qsixXmoQ6hOz{c2548&=j$Hb* zK43B+9VzGPU+dn$3WhEBJw=c$Xelu{H5AD!VQOw39_s@Z5(YH;`^8tsjn3Dvi#3xN zE{10A6g&IKg^dTKB7555vfI`QKW)$#jAlei1WmdRJCkZK0bi!Jr3k#l$-!?6W|vLF z^*o4d*7du0xKkAAUAcHGHyCsECKQrg;N5>*7WotDXmz~e`jt_XiBu-1P{FnPg_a6l z?#D87j+%LqftaC~Zw@JLpxXTli@HU{Kli_p9;33D`_&6vkwg{8hr$he;Y*8jTHLi0 zLrqHPbctTR#Ky+XDXCuP`pE^lF$x&x->$iRmQK4mrH)Ilb*Z7HSULF9_rZ8zLc$JI zOL(2HJunW8)I<$#2>L-l2RB5UtzMI^p&6XolGh5&yWmZ+H&4G;`PJK;|8Q z;stsylL38Tw;u=|lHE>W{)zRPd<{Cx=VuQ7QdwuKGmqK@2%Q%NjTw49+%7mMd_=QA z_h-+D^=8JDD%Y-d1SA_><`Z8{EZ!EDnW@s?Z=(9IOyAwG=Z4z$nD=hJln9Rx^8LU(|BKC&jla@BB5gl8cgFR)-U ze&tM&LPA2;0*f5CqApFU;ygPS7t`yF>eRRP|666pM&=2PHj)W6(g`zc>}feVQDbA{ zVW?lhr1SIul9j!1ol5zSi@8+N7@?vP^H>8`zBjPH=g>vOzGUsZE5R~p_Ehrn_APkp zobV#FO40f1w2SIGk}B~Xmp3n+a3joMZZ9qT`gFO@9L?)+A~+;57j2U85vLW+QT1vzjDomXB7P`iaX#|+S7plY;k4V8^*Vn4+E#$OcL)~sEdr; z^^!GUvN8v#&>E7V;QRP0qbMW>5hLmkDsKfD3l8t;N{3Giw0T8!VDscu5ON!?IhrcG z$%MA~n)S1eSOm3s;7k4ok!X67y_o{s)XcQ$k+!Cg@-O9z%mbFkG7=Va zBKHNTx2~Y`c?Tcg2+Vkf4>!%oNYV_i3L%Wsc6V4-P~7oc+RTYKNFy3KeW=z{)}T$C zWZ|R>Ns$P<#Ed!N@_(BZ1@`o|axI0o1aS zI^)0ZR4E|7Conh`TnTvZjgu){Vdt70O>%lj{IJ+;Uhfht2zRaOk{1en*YeV?I~*(* z*HSz5$Sb>(hQ{AFt`XGs%YRoJVG#FP5x?aHd-Ydn)J(OTp7CScTg;q^PBd8PtoIj* zM^<_rYR+%ATJlcF;AgW6XaE3-XrcIla!5i3u&H-p;1jMKoVUFpWV;A<2~ zl|tKUHm+7Ce#d14S>T>;%iP!&?FJ>wUe#Y66qlWSpFJ~IDu<0NxY!|YBRTZQn(%aN zBz9;0=H*t~=eu;bj@bg8dhKGrztltrbPO5jSOkLtliTgN)d$nruNoiN+4Is{n3(cC zB+jng&-=$aYb{dSGWjcsP}bIdQ8-4(;p|DS*hyhn&D5leg~?M+$KSU~cw`zY46lE^ zZ(M94;wHB&;zvZ%YRi;bBS5M03xI4KtS_Zb#H>yuengb=?=QSAk0Bj>C1cbnxF4WVZp<`!yaA- zZA0wEez;$D)>qEe#|x0%_59i?_Q)C%wQj8+?d1+8YWButsO{?8P{FZUSfpXdt&z-AC2Chk`e83B z;o*%NNuXdma%vd?hXc{m2)skxarP+D-PdfTsuWvt+ zR@tpS++kziT^0J?Xf!$c+{F;3GU^5BBr>wLLcG{7#9?hAo$p5$!?N2h_SzhZhi5#Y zsCdI|3k}lID|c5N&b)cvjI0YBW*9m+qyB=QjFa@W%+}AHg+Lx_*+q`)Z z#?{J~P6Q=hxWNB(`XvC2h^qlx(Cg}`E&9blS0?Y}N?WZL55M1LHqA+hAo#;0O5g*e z3JiyR_Y1}+7;&z~1eHFo+Hi4qidOGQ=4-*0$w-8SePIv1rhJ+Z!!67|gAH%fO{1;CH49Bg}_Yk)d8Bi(-A1)h=qSO9ITme6# zYb{xW=F*|#U%OBg#e5_v zwnx1F>JCqPy{SH5=^6oA1~sTn?(OXmRDorbm;tz z0d@QkF7nygdQJvl-%IyV_SS|^*X;|(E6;ztV#V@M)Oqn0@2&md0FvJqY^iTj)kXy^ znGE2-4}NTN>H(=Iv|8*_ZB&{PDcSdzgu@C5Do50uo^P2t=b+HA zpf|qa6^@(cZ(tQmy*=g~^cEt=8#X+#D3WqQW917xyyr&ZmOv*=WJ%1_tZ$}P+LBnV zhm=e^{cAQybzsLcxr5I@NidwT{P7)A#PIG%2Q|uq^VJE`I!HfK*Ug3XMdX=b%`OkH zH_tn!nqlB-y>cnepW5qCG2k+5zU*_EcxXAlKz67lXIj9bI#q$ffH|3(hlJevoNrk# zjVC=HO$KMKdq0>xDq_Oy1aiP+AN1;);3Y6ttjzGfBo-E`gp2-l@pxCgFHyYxM4I*F z*UsygI*R}rE@cPzC(^3P`Qb;On+XfKOJI(?;6wi0mbUSmY@zZq(EF{onz&aXo!$%nd_ zBh!sA`nzvDBaDP((oGM^dDqm^1kR|?URC}1Nj@)Za}x9=tdzr9!jtzSNp6DDxZ)q| zkXIjAym+y(O=|+p#(Uquqmt*4Boza@XtM94uqRzC9mN+Zb-$MCy{Z}4DZ}qB397MW zAvDFuC5tYb2YRvg%I`H+ma2O8F5q{PwOr>qNRpbK8&ipEby~>EMAbFyJKy|PFC2kZ zm?e5Uae=-hRDWtm8gp~1m9W--@^s{JVy4ut z_vNh)!DwpK%EUFdeekqqvwlb4?BtP)g9ykYne5R;c$Q?%<{h%9A220#r13aZ1_6kdu!5_9GW|~ zXDxec?-7E9uubi6@mG{SVPDv%`!z2pG2`RdL?pd*uXoTw{I9EwK4{0XeKIIUTh$0W8(sqTSuqs;q>>;eb+zkWQIn-9*`|50%I|O!{+7^PBn8 zQ%lf>m2dYqXLldX)!POiA9l+^xg$fsWPXEqUZOl`VP-JK2(-dtIDOiRn|H?umB}%g z+@i}n5uU0qCQyF1X_ONo?!?-*^aMMtaJ0HJC(06uS}EcR-tHm4_hyCt(hbIMbZ0`g!cm%OriZf{=jjo=p;TC z=5j)``b^;R=t_o)Kl4kqp02&Hibk`(^dN^p)`#kvr^3sj|51E7~ui+Ph=L>j439^06A)5`b2-n1NIG+)EA$Qs79VjHw|x0!^o%$6hsr6B=^ku{YLcXD%dcfw-ua&b>7FvHegR5 z4G-U4j=FS;P20LXFt{$vvh0aE(3St=ON!K%bV5V3ezEZb8T2P)AsH+@Lpz}pZ+rGhpi9K8=kuudO zYOqXJIXYx$XY1KWF;RFW8y%8M(e8g~mAfoQYOx5q?)AL*N_06|X0giSgMu2ZC_hA) zYkc8pIYsXe7=(ddbdZz6o|^wfXaQ%Uy%bsXqwD4&lzl$|e?)wbU{8E@wZa`!c;@i3 z0!cW&o=SJAeR&~$rb4hXY9O>KxeI75!S!}O4`v@T%ftJfYXE)yd#jFm&BEFuB&Wwm z`+$q9{jcj9>5a;l6$6J!?H7kbv#WMXzR-FW64HKMz*G4_4ro9?^=%F=RFuuq($~M9 zq!@0F@i-*BgT`OSm45>K8Q~mPp8YuYxC*FQOgJHE>$5eROlW=hso0@uK8WZX~-m^%*8{H~p=WxAiAa zN}#n}F?uN^fBVLD=YyxtRq8`12JTN@uyJ$-7{^ zswmd~0oD)>H7D`yZ>h8NXgK8M{q=h=?$vVUCFjU%?)>G(0ZTw*5(i?78Ju6#x$#D| zp|#^NjS!Wc#vja$7ptO3MNOX`R9^LiuUA&IG zk2WseaAFVrDPM;96e5B7XC{0aGKg!53|OXMu>r>6Yw0Y z8ccUDWoa24F-FFDt`FLnm7>>o*9sv2H<_8$PNq;#b@rYff3OqLniX@J%-vO_d@uY+ zqu{4pX?liNVPRp@?WLHjAl+@P7Gh zCqZN#5B)+J;SpS;-)2OuXPOet;m!F})GQol8B(*$Iv@f-Le-G>n8cIy!l;$1&dD6k zgPIrlXmzyWm?FnN^yG)`%vYGSVK`uv}JzGK+Vk zgGOd`(@&f=TN*FB;O5h+>UGw;#K1?|hWVY3)9i40u%ROKc1N-&WS_ zM1v{k@vrrta)kFBbbq%zp-M?GQD+kan*c! zNeX*5p2I#XB!X_AEGONV==jz_!MXa98z<-c3js9%e^qk;!>un^2KNv2bqIE5{OEGm!w)WyIhA7~V%o}J=>tOg3ct3JH1-XCV4u)|`X7VmLSGvKMgrpih6*S;I%p370~+C>z03YDfaz{3*dBqArS26y*jl0bkb~E z^r2k)VviN|r8A+~=$Fe>Z%dgu(Sj3u0X0N059g$v0S7`TW}!DPrhmrJQ~S%4gN;?k zrIj=`m&obhWp^0 z?O4tFI9(Qr2)eXeplr)bFc7?Y4|B06dP~1_8w=<6E_XY`7P*v=BopiF`bOK>aR*~y zRkX~iA7cwifAvdS5V$6E!+^VXiZ;O2b=T2AU*hcZ*6Hc)vdpkTdb3>gPGPyuH}qXe z64!imRL>c5sRiz${djKv_fy{@Q-U?!?GMwI_YLR>pJx|>B@X(L$;WE~mPNE!#C{MF zoEU!uu(W|VZBHP}&cWn2<-$y#d+>uox~Y5?aG<~=F@Ul8fkCshCRuBfe$h~BRc2e? z=Q=fK#r%MqQGp(U!G@Oza|i8FcLhwgk?RS~{Y4sPPZ__r*zD)bxfQvxBiousUmdC5 zCf;4ZWPxh{&jDQXsTG0bYOLw@0$SF#T*{vRwqCsaT{+Z40@LUc(Dz2J`ZeUdqZcg! z+%5+?vFgH+8QP^CdWHRarangheT=dU#+{W_?YifLR{7ksjgODd@A1d2yejaNDIu(! z5Wolx`U<3eN~2u>!69k-v`}H2#vc|4vgVFC&oA6jNG7qNm8GActXGAVOt|Ih1^iA92bw^|fJj1r7n_%a;S< zPyDd%qO?7#Uk6`iUG9)WFh__61x8gyri*nUXoyBf+1ZA|p+T6)qjAwUq{W(z8x!A``8$M`a zS;zH2B@uQI_a#M%Q6d&0JDK`9=naL7=j%5y*x}z*qPac1dZgQeW=Va+mRRANB5~u0 z)0tbC!aa;5qGfO6ZsGbOvp%Gcq8u&7;e`_Vs|jr^|2!#E{K_2+zK87_5O;mkuN#u` zrr_6DUnHjh*Xf;}aA*`x3*QgD?Xc-2$dRj$7}f5ALfRytXXfyGasd!nZBfRhJ} zCZiR4$!&#uu`k@^%y;rZ{O!j)zSJ0^iH4UQaG#K}Z0l&U0^yL9?tG7#J_$0<6jP8G zH0-cDQUCKK54gc2IB8*h&TlB;BaM+fV#Z-4D)gMECfqwg^UUBo#s&L~Ufyz#BU`WI zwy6%r*LQkIzOwmQ^QHQ$?Q@yP+q?T$D=%%BkV5ON<8&~MQZ@g&7!(iq!L*#d+O9@8>X1ix$OKTRmVVZaM&x$_n(u_fSi3M z>1P3)mSPxk@@k{jFhc+2d{kwf0N)4wUPXiHeP9b7tg3tMU;&u%V8$ZaDMtdc#`eBH zy87(7GOKy=djs%Rq`@Eg1^t7LIm^El@s_Z3dh>%(P4teEU)zEvvD{=Si}_u5VL6Bb zwI1)Q&zLUtGH*0OU}ycl^Qm$ASVtbRBvH8ts8aUzQSnKk$ax#HkHo}3HznKp5+5Uf zYn0l%-^Jw$J4?!aey&FL+cA@;df$ld1D2>m5OukRlfkd*3BjWVyOFWyR!WNaz$EDM zf?*8k)Z|=A`gO$&E5F4qbhz38L4P#_d(Cb@4|u&YEGW_J=US2UV_ZWd!%Gq@2{rGm zxYwZm0Lwo~Pl$#z97(q9{^$dYEbP%d%n1oYM4VBV2n@z1hsbL-!ng;xrF3OyyE~j? zK^0nDtboau#{8FKg0g+;qG4HialNtkK=W(uGr7CJVz>E&Lw}SB)f(|{$?rr@i5($? z{sdtPv)Z54Z5ykq!D zWKbyWg z8|N)@q-iE&1S78K{yX8XYm4>x=eeGThx1NVTwh&1fAu~F#daluctEpS7VhMpk4QY{%A>jBcrPFO9XZZ1As{JY1Z^V^i2%e4}%{Q4j=&ykpJp!g+ImqiYpB_yWK*G!u~h3VB(}_W9+7O*u>%QS>R{CF4oHl9ak)vvzCtB< z{KvzHd9Kn~Y$4?SXqv&TNKAy=-yIC7jb{=^ucW_Bc(~-n5JhqOJRKm-WFFCfEr??R z@T|sw6g^$MI+Z7Ae>Q`$(tf{J2U;-_V`QaK4s&w9)Unyv#%47sK1Uy=`hECc_d}gu z(s|XaZ2A@tIiiD9)du$VF~DRHB%`P*ncUAzf;Wc^J54!Su=^pQfFwqZEk01@I9kX= z^73*qj&@(}@VbBZ z_GtW#we)8@8L(zrSl@eviXht*Owj_z|A(rGgZD*$1Lpc_lPspf)f`hvZVL(4ueUDIEGg@Kz`T5&c-m}ZgkI&ax z##PvUS~2Pbx%cSRIq&#Cb4PN|k&J#g{2Z7Lh2^m2!D|{U{k6b0?~z~j=>SwQ3@MdI zMVKk+D#FL>FXhM&sB)$a@iFH1(_IeY7V-v?n9tsv(LiaPN0wq6QgpgnUCBb0q07U^ zvRrCVLc?m34!co>kVp|}$>VVcX`++Elt8SWUt)%M(ho0;PWe4-#c0MWqxrAM(Sx;a z6?|fP9Ja<6d$W8eR%7t=X~c6+gV@aTO=?1Udoytag2uz`ew-TAE(k6To7blYzQ3Tx z&4}6F_E{MyD0tmAxZiqweSsyU4weK0a0eO$9YHYyi}i(a^y*9=e;)4YS8UFgJZ1uY z+r8AD)cM(Mjnml|Aehz3pQ@6TNMuF7b2%9g*yE;_!$`cdXab$@ell0V6H`+IbW+Xx zmnENv3>D9IDEu*qV~n_PdEE7n!L;b&cE91A4^kYDV2omDZYbx(*&H*x4%$8I4me}p zOtFI6+Yvu?{W?JV3G4qn&}q2@EWGyHcg;iH)F^UUNyKj_?G#f2*9RP4LMx3nrT#P8 zUA359k@!04%r#p-(yFSe*Eu6(ra(OCI1ju(4i|gE(z?}ai+~L{l>NTgSMV*-ag52H zZe87NSE)v|6Vs*J!lv)_$7yoSMg>%bkVsu+MnNH@IQ?TR;O8G^jO>juA(K~DxaCgnv-#0pnx)O=6pZs6GRCgF+w|Y&% z^XJLrQ2f5~c&od{uqvUzN7x1AiJQDNKm+x<5-Bx|XS|1m<~d#?3b^_6&DdR1ynJ9h!l5-o{0T$od&;O@!^m*TC?IzYzQc5udkg7USGIMlBr1s@-Qt+Vw4 zKBw-aLO~E1lhdq-9{F%`a>geU`C7P%&b7XO37UTmd0wh3Y?NX0=XY`XxAsKLYb9fJT+X%O!P1aA-;acxJA=w;tj zo2(f4`l4uSTeoSI{kBJSqJ^c?i)jHR?9G@kB8?Q#<>p*dQEuG$ooxE8b@wl*`26@=V*zIM zyWKG!3bnkPr?8u56rWep)^D>`t?x;rdo=W>@w-RM2`*E!idt$m85;FsqKZ0U-=yU+ z?zU!Y?K1fiGv3y`(STgiuHNZ)R$d;8;?Pqyxa=D+;Xs@voJ^~`qHC#|ONT>@?WxV$nsGF3(kx(EKnM}AcHp21^Mph|uTJ9MIUL9G z_xdoW33*<{XjT?Q24?vV@0ixb?=CHH%d3%Mym})87MkjhEsQP3be5B=9;wH7z#&sM z%pHqnD01tM_DaL*kXEz5Jow84VrljnUw>4CU9nhF=aA~%A7j)E8hL@=k3(e)9i8)w z2uhWqR;K(h9vkXP{XhijC;I0ry^NrAl?3aQrPW~oaY+!k$zpSu%pAU35d6;eSV`!`vj@0DN(Ca|u0`>jQituu2ZgeA z>Da6IFWhyOE~mNkJSM|Ep_eh(&sut}K|Uz5jMd-=cz)!IZ6+8IjnuXR?kRGNL5JIA zZ+;-Q&#E~x$&LRWi&z0qa@K1gpsExIDfd5?qZ4C$2n&t1TCmHjpWPi-um2p!lmj_! zh0z+K92p13XUG)E7Dy-4>^z$5KYGtQ*jABy1Q{HDYJ2+JD28bOtvK>Ukv!N@o;idHGIm}8hf{lg0VI7;?*50 zFfUl1uL~oCeli+2Sk(AMNz^_sK2bJNO@Hhc`L()$K z^W3x(UAzA(#%uQa%n7z!Te8V_r7KM11=Vt2=dZ6{n7K_#Rcls&#RSq;_as!;Mu*{p=>sqLn0lN0}PdYBfoRh zkidfiTm{~{4Lq=uLRf*U$@UGuVt!t}?XvNJ#LHa<9G7DTNPlE&&!5W+WUC2SLeR!L zbpTZSo&EK#V1HH`iO+lJcB^JPnN_&7eUEXvJdGn3 z^CA-E+Cz;ZWrUy`#faRJa`qD;%ZiXiBr+b)@DxxpdDfN^^@U&{lj3B4C_H-Mx?jSm z6TzecGh`<*Y5ZS6!h(T)AE*7M|2axbc1(f#FQtA?zK3EIj8dHpgxc4x{S?6Q{v{Fa z*!nCUs_g2pf!r?DP~W=v9o^MaXN}A>Js;62MN&J};x}HrlZWywLd7byu9WO>(#f-* zU=ifxE*ILi$5!6rdY@yL;YW&2HaY##^omPYMn89w%Fj;D1!aa~$;Wc~h&20w!U_uJb$M9f<_%GJ&F_=tJ;8ici z;(SdsPM?LhYRBR*1fgiO;Q)h-V&J>&>~=|)KCWz)jxw zhj;DhRZj}K#C}wpfj_Kh00&yPIqR_Qi!JqDa8yrS{9Bi-i9*BS;lz}fh1|0k;@d{E zt+{RjhX2j20xanNE)vjrJwSUSkB-g-{KWiP1>vfQJ0`p9Z7dM`^TbAy*db&IolGn` z?NLl6^ip=q5txed)=@EHcC0grk>S`kqT;~Cfv;V7Z2ij_V+QyxWvbHfdHlZCODTUz z-5>Qj{vx~LgctDb4@of-_~iB06BV;AWZb^@8t==%-xa&K zh|=V$zGoHT6m#AcYGXp;I@_wdy`}oagpGY zn+V?^)N^>5xJ^`7hl`OB4ZgP%pZ+tZK>J!>S>_0)IJt@{nkWdvc3ue5hqBP*kf&bo znC9+qWqW%t;WhTn(6F{Rbj9Fd_MQ77Tb<~nyin_1d3g;D1&PlI%y9O5pNXaq|2PXX zPSro?aZFiKW#Rz8lVkxfGC#<}G>U?epydXX%@J?(d|$czC?(~$JY3kQh)`QDcT>0H zN5FfJf;o)zBSo+c#-LPB&$aDiGSAk>jef;A2N}=ZM#}vDeg1PO+ElIV_P(>jbo7j|{Wd`Fs_6aR7mH!sc@r&JFYLf;@z|oEYgqF|_LAaxYfm0a6*0{FG>Hx> zVZ5alf?cy1H70M);1ZqbVS3H5K5*Dvl-;U<3PPpz*jJi|2i>?^YFQS*Cj_q0l*AE_ zn!YU4E{^Gn>Lph#$>yT|)!p@}dc<@dpfJhAyf2D;KM)Ifz`nbdg!V^2u~#gMbpsDw zQV~LAGYyyvCvn&8M>@OgB};2HMo~n5BWIOV(BuXtAE&a-gb{*P{hVimnXl$otNSWa zU##~Q`3t!oPVQ?jQrFf^7zf_pWn2m(qeR>fTeIuDVwLd9oU?SBur)o)$6;oUo!0K) zzXc9KivAmAk)Q(;{rQ)`;v$?yeGSWX&D#sd`9e zc1*r>=>_OXzq%I8cB-Lo>DN#X`oHCg=ataCE7qT9nfG{eMdwJW2elR>HwgT*Xjgrf zPkZAqdst0W{ezfW#VbU@!&g6olh=C^TbK|wfKyzv=t*_vP05+Ti2BxkIPE{GdA z6Gxkutn;?hSv`v!kWo?QI9Q>Fh*^Hgjj!(?xWBJCAS%W#w8tz2cvJsz)}00*yK+_s zMWn8VhVG3AJjD*S*jWO#R>?rcoN-a73-JLTlIl_*qisr%8#tG9r?6%A_;lFaX_EJ8 z$!^tk4$?xOFJ9Z{0F|fsIBBEe+WGZ3_jM`>fBo!cT>YD^J=eY0kYBe|bO7DMl%Tnl z(>nF)BnN;NuOr}@r%xTfC_3Y=O;F5pP5#hOd9mc!3Rk*=^p<(>lmz-T!Zpl7gt3%e z`SqW7lv<^GFo<`I_TVz&;{mbQqf8M=e&=xqM2znsRciASpHK)2$fAUUJui&6 z4d&s1Bg;@k2r}c~*p84pZdM`UQgS7UR&#nvLs{|58a4nAvze1Y{Vur|Z_D2z;ZK<>@%$_0zUqZmvEvpLS~293vEuo#iV5Vi+cv z&l8=i!5U|KaePcXTNn)MVTWgRwn(dcPGAV&@%=xO>^bazlIf7+j-|1_AJaMnGeH~T7 zff#_|eWeLxKDgb)%o-zFdP0VDW6pEKUx-xw*?ifuJ(sk<_h)lKmF#k%aP_qA4=xpDH{Cwn?6yJJ zKEV^GeA0dvaWB4|icl7DR62f(jr9BE51|kPbn=Qp2p9qJ6AcCan_u56YKf|o*HsmD zP(+t8h+7Tqa0%!eZ0jy0lXS)Jq6nciN>tnO(;8zfm&Z_FOydf;kxXFcI9m9DjygEzVb-+{L}_()qT;ap8-3lfG<21 zH4_Rx55>2=Cu9fr)!XoqKd$$)6)TYErC$NN9xS>!tjkZBNi$akCk75?W}Tjpzm#VD zBdUDOZj*+!^#{-XVD8M5|AsQ}7|lKO;%g3>pYJhz4~)?Yt+IKIrncHfCycgOB~ua& zd5djQo8gS%}2y<@XF~X(r;VG(y_@ zPP~(RxCk?hq}`Z(g!kQ_|7m(=rUP?h^!=???)0N*_8-ln!T z&%UOE!y0O900q#C6};K{Mp}m`Xdt4#aQ!pl0n_U(s+1Ss8GA%`9LVu%ypI9|?W6aD84Uv=07b=rI z!g}b-*t$*C5;)(A{YH|mu^tUg4j0(HzEwnHuLvKBq;pKEf`d5i$#(g&J^Oq0lYE;} z5*_q4^sMnxrx*AEkWNw1xZcZvRHi1cvD4W|y@G(@}e2x?H_9wjM4b z{Qjk;vUcCtJ{g{N4`u7Z+_`p-nj;oo;SEgwFth2DodvcB_jedeO-9IAYW{%;L$dcY zB2|W#$iESfG5NCNb|e(bG=IM#i;D5`3!;1f|6;=VVB}9j&Ufr&C~)foaG19sQ>VWK z$)IyvImjJEjbU4Sudm%lRG4#LD*DwYR8|Qv@l%Y;Zj!eEp2zN+Xuay*p(z#;F#n@U z4S*-KTP{n|FZLF_f&;~(i(uG5)bhTMD2q@v*QWY6?VYyT+@1-yaW!+yj>g8`<{Q`i zdOp|D|5Bl7J)Q!J3J4HY>8qpEhrDo=MEsf|=d1Ff2mv;3CMIV2815@Z2JPz-;asWl zSCuxM>}qP;p$>O{68EM;*LE@JNg()uhv|3^d28yq1hR+ISN^;BGLfg;=B|_+n(8%1m_m@qt{t-aufmQ(yvBaIOvr6ay66aIYR>uV-xHR^T z%`;cGDEi&-3({V}6PH=Li;?P)b?R2^Ej|%3%*Q^D9wI^W*6eSoHNE(sb4LF7^qM+}@~5Gcfj1m$U7z8%&LMT5{=>Q4qCE1HaA$|ycQ z)^nd&Q#u=YtCgj+gkKE!O)bmUBr8 zMDA8A$VyV?iROO)BzI70Wt1XBt4^k9>*bp!n<~?4M!y}BoLQ)sy=kkCkMC)3VmnMP zl+WOqhBan&T5-|sGs1TK7Skpxbo8})zHX?89N1mZP)Vdo#x@wA(S5F3lJspAh! zJT%_GLa?fg#GT_s@o-CZ=^4Xe5lTuhB)J73cbCC`V`c{zlFE*94_13T`oB82pOjx{ zIONvTzx(i5W7pngHHW-8GtO4Yarz%C;rLA5gmd7vOtRhuCKYB=vSjsWx)D44X&>iSVujvgy3~*i-h18|qLk=t+EjOI) zU`&Dhvc;a{LvRW|f8JYh&Ge)49nlTWSgVr1aWzM|kkb$Y08cxxa*@WLZbf{PuJZ=s z`TM}tyzk~=Zk5)Dnq>do(2OG%k9SP$B3cl0Vz*5I!0N)Lu2~@dD8_-`M%V!lqtaDw zuE}v48xT!Oof1}^@mKwByzM>#fcj`v`(L01L;j=rqYuvSnSh+@{H=EQxzx}ZaWnw} z6o9*-8c5HVBNw!TimqIb5;7=MAEs|Cmq~p;6H~CM=&2#{AIy5?vYG`t~D2VH}(m=#Z_;76cY>~#2z2!hnMvd1$T-BUf-zg%#{p{%4 z+&X?gCFC{ZuqQn5!0WKEU5jbIw4GzidMLZM(YQY_2{3)=x_4vejx(LxywqG{ra!0K zW|=HxK!B;(Q+Ck1Dp)Vqi_o?2^Fh)caAOjxb-uq)R-4rDTz06C^0=Kg3@85GxDW1_ zs4JU#J)py%#J2=8#;)RZRCQ>xZ6T+AB^n&B@_=-6HZm6XnmK~HD@0how1C}msvzoz zL?!y`(7eI*>wR(bUh6jiyl?i@WQW{9j<~teWvnqck8Z2Ze*@Pn?MpJx1J24hR)N`N zYhE`jw$-jO1UhJ0%t?*{Il8qF5w=DmvM2tYM_y)sf04I(nkQVY$C43RlYW#6w~eZd z9@FYe=kvS|`<3)AEP6t!jq=4fjgWzRiC=&P3pPhvc3ir64OFRJ-g-#mB{t-tbRwSAIe$WD`yH;GDTF>$a%@h&&c^u zAq3TmJ{>uShTQXUGLrfY zR^M>)6$%>|x7LCJa0I_A94o;*EdC$%&I3G->T1Iy%eKHan2s?R8CUJD-00o3KE8^(YQVOyL?7UA8 z ztQ{{i9=~wY?H;bWcln#Q4}N#Sv+s_2@RA>Q-SoDbrX6+6lD}N_-B%;3j(+9&qDy*y zaOLjf-}$Ye+w?EqT=wn*;dkpd95Lg~zg@8Lgqw~&_rLd+ZFK4Fy_c-0S$^N#>cW># zc)oA_i!Z)-@U8dETw}mH8`NBX*_ZoYICXx>h@N-e^4zyQA9-cysUNL5<~q`2*R9-j%pH3%@cG6%vxoE>d+$ruIOp_-cKV`ZU*qn&U8-(~ zE!7?I*gpnL*m}v#W!Jqs?c4u+JN|#~%ncN-`Q|ATf;KozN|L@1D&3|0F-q#0x@Z{@nJ^bUQ4{cZQ>anjMaL6kMKgwc( zJ})2D=gt@Q-Q!PxeRzW>w_EY*;7|YWvvYp<{F}AEo%#2PpS*O=Cu?`x;e?V||9fkv zUUU@(^f>a?lh}yRYprXeUDsPiKeqD6QQz0S@XWsFZFFs*&NN8jDQylUaX5lipt_1ewrykDQEMTB3hdg09Hi;VL& z8g|0lPmS2&%fjs&PJC_R$kT=#c+=fKpZCSE1NK|z{f6Cp)NeFt)`REnaPjPYcY5~K zO}iHzvcbH^`k%3L*(+a0=amk)`pS_vk0}`OYtg@c`|jLtue~>Pd*N^2oV;!C2dj@; z{^*1`&z~RqebHsdp0N4wBlZhlFyoUKx1Djuo|FIn$@?Fld~Bc7U)g5r)5D*7_wnLa zA6+zU`3;-D`@~V9qfYzhcinb*HX@Be|UH6gI}Hg+G%~S9?^HsC&n%t?e^9C10ERu!0qPb)ggc1vCCtR?loiG z(2($|hkn26_p1iKSeJjx{9yxU7A~EC_@;XtZVq-Y=`nbNe~!KF@dfigJ>ivgYF^lA znDJoGSzq2X?xdmzimTS2aQe$7Z!Ik>JaoWqGx`lV{e_J_E)37=dG&t3Jih;h0|q=- z*6qbBe*UFzuO4r|eZ`sUEj{|O6$^SET5-VjeYe?Tt3I=?sI9f8zgIpp$F}$O_sf4M z*ztjW7u~dE`HMTw>$laD>+kdQ28Dl#t>!rXjh=(HGk#sy_}|=L3$~jw@3)&4Egv=I zr}`Ivr&G^DuRm|A(f;^jIyfQZl}E08a!m0Z8%H?DF2$WH$K3@Gzue0WEcHZm84S6@- z*!{9yLeo}}&_xHFH(}`uuN=-nBgf=l*|YkmLkj18cHSwwg^PYY6b5deJ|OSpv%miE z)JvBxSb4$k-&Gzx>Y8ngvuh_!>apH>pHw}U|HJ2JpT7ux?i@O5)44?zKhF5#$mMsO zu>I4ekP^-Z8{c^zNL6%^gF{*6+zn<2A;w#rgr(sK; zI{RIt;_Yo;zWmayK6-JV(5SrS|G9hv9lmn(xo<=sD%tV)tIwM7a+mr?moTiss)Lsf zI_b7qSKa>Et(Wig*r1Qj4s3fQixT@x`TYD@=GxEqPQB%C*M407*Dvb2blZ2|JEm0r zaZcpbpB`kr^G&~ecle?Ge*fg&>6>Vx);#LBk))Z@J|84@Up~S?Ku9Ze95uotxUj4()ACG>a=>FNuYC@+?nO;$3tspsFmT=*(XW#ww!QVt4Y50T%xz|-1 zTfaN$q2D(6(@kp}vTMW2TPJm0eEZlP-n@@omyUSrnVMg|zj3RVZvOYDqfS3KwEg_U z=G=1O`JeVR4&0^akcJOut$d4N^WT1Xxl6An&RTciZR>7z)>Dt|IcHZgbMzT&zE^U8 z?d83G8h_>*Jr;cM@IL!qKC|M6zg_`cS~ld1-?zDAO4*+-s5B1yVCA%teR{p{!olHgZ}0i+`OA*Fa?8m_UcBXv>)bqh z&mJ4U9y`tEvk{+tHul8-oOAhw80)d0?ooMb-&NoJ{g@t)&AR0;h1Z|)pPTpmcJcOe zZ#m`ClP~_{!)KoRxX1bs6;_ z!Z}+yh|9co>tc)_YM6*R_`kjcd{(kD+?@YC z!5B~xE)C4^;maUp8<6dk1+byO9^vxP=Hc>S&zkb0F2FLoHvf;51&zAWKpwnpfvryf zH-itrH{g5lBlyXOFM~8}K(-?rk}b)mZlZ3TP+J!2Ut3hN$yHy~1T7 z)$@;lpTR00z6@H>22_{UgJU8k#XUi*mw`xGkx^S7G{WV9jp5^T@Fno=-j_jZ+Q1js z+-bCpHjY$;jM|bwtCa)n-rzmdhxD%iex2{zeQVl%d|I--%T)%j$5yfb8tQ(~Gg|h7 z|KmTNAOGH$M{C=_FS5m&ieN9$qGe!CX|Zuu`4H9pYVYg))7p9U)AMESWCOCrW5Qz# zjmY#-Emj6dL$gjB(*p+dCj!6C@7Kdl^>ACd zVUvLg$CMe=(b=CL&^UjjOkap(@)!%zncv(_Y{zgX8jEi(RgP`pb*+0`u6Xa z{~Wb}4{DevR$ZO#OJxdZO@rp*+{&Nt?#revZ2;S3twdS&=Y?6jA^RT-G#}lTdGyov zB0zhC}y)COdm*l70Wg3a{~=6XDHJ)2wk^WFWrxh-u# z>*XqBtJ%JM`}gbo9If-?%bRcie*GJ751sA4v)TVg)JeC4GeH=f0_3}ZIzany)&ZmNlb*7z8?EiJn!vyi+^2zM;CGO!{@lWg)-u+fRFbzL)fK@W#;TP@6=~O$7InvOb8r|CZ{pF7 zPCD+A;S$EwN0PsF=1dCYA#X!6m+bM|4M-O)S^m88-5KovbJkza4p$Vdeb3(e8&%VW zH0kP8l?08&(^#j3ZDK2jpqJ&qtNap$Q{8(JI^TTBjFEX-b8f$wA2%W;p*+HE3GM;e zC_yrPf4vW~^$vOyT8CGa4edU!tiMrLHnK_Ddam!b z%7~PQ)*{^TU?oT-!?V(I{{w(Eqq?dF_OZnXZT$hD(KPC@un9=4keDy}V=Yvm0G<3F$fPcQgE^ILprJKTNz>N)tMaibdx1>n z3%|qm$6Wgf-*Ko&KBgV17^XEUq|3V^+Fnn$>`7MatqQ7bl{CK!P)mIT`V#(bkZ!t} z#_P!TulD7n2=mZ2<-0QBj0qUn|tRfxReo4HeKq|>n4;LFJ4LVL`U`d+kiq}OorRzh( zTY0w_jIPyXKS){d>on88g|ihlOm!Y9mhK4Bd_B!{)6q|SoQ`b&kJpq9T?bfd_Vi!P zT;TkQpz1@7RmRK8>-{^_X*;SdmFh)$vj0k;GYh_S(6u^AAIr{|#=^jQ#D5~qbgjIn``?l6f9ZnB#cKe&*b_c4m|A3P{r7IN!!y#< z$s6>ww@TLEc4C6`lZP&@)L(fEq+?5`UN&(j##d9TYif}-=s}wG zX{MVFukmp@vi(2J*yF|^C3C337OKllom`rxPTr$Gnev+2@G0~GI3I~Lj|0D^k+#P1 zkF9eUXH6ym;wjxf0G$~XZy#RoFJ?@04ej|z<%+g7&Hl>2(oEMYe{SJAvi-k{e0T9W z_G9S{J;)aNgGbWTi78XnsS8SmTV#{|*K$T*O~p`+J)NIsy3+Bv3nmTeUQ^ZaT7Qo2iN%NMFWpUOo%$$B@?0NiY}CEYu+{r`&XSC>!flUK!Dua~Qq@tQi` z$>`!>@J$9fxxThEw64bZyj*o-268qSb75t{V$ysigLGeFPAepPj!g1)t^wh7Cg1v` zd1D6Y(r*qOxVW^4cc-`ZW)t-gnt37aiIDC7+jRhi@6KWSnhs@-l!9 zv3+yw=dKKFLFs;h4IX*w`1Q@cWwJ7$zJfVN7ajZ2+L?59OWe?sn$l2rPxB764QEZ> zLgoEvpmD2o(^b4XYAZq;d3gtfc`t_PD-dUEuq?xLGvT?T+rQ>zZpJ1yO*Rgy^$DOC z;ZFjb4Umy7NGC6n?$&kXMR_{^(d~+jy>unK=Dt47Lb{*w-WdxthUJzl>wtzBRi%US zct>taxRR46OlR!HO$X0I?_lcUHN#T}8Fgb>91y!?_jSi?uC*{55@ssM zWWLRw{(rFR<7&!+8!nik`M=fn7@>V)mbtQ^#$65q*;sR7rT-oD&Z4dvR6Qk-XCHge zGv{u0*IXIlWczK^?y>mo4EC?Qs*ZULETpZWagHs*6~i{5ozM?E*$KDfaeEy60xa3s z|0OHgv)0O}k9rxn7qIRnVIB4#{4WHDQO1g}y^UCl*N67eDBLE2TY<(d?de(1vJtH- zlf8(}MX`I0RVHdHiKgOAqWq1*<~CtmOY?dG>fgip7aj3#rDOk>%svA50F8wy&&vA? z;3r@YZ#_$fk0DpdbSK`a{b+w~K>ua{&wGPKK=WN~S?^oCN~fJ^|Cj9LLm(YK37!S- zfJE&bjVnr@3e(pAoz4DJmA7$+z-wM5JWc8a0QpPk}kw_)FiAV0{JOk_Z+e7rhe_)g7s%+ZJJc`**L4WKUPt zR1~dG8$I3xn%!%?Z?El{zl!c3P+23 z?YlBfJJeXuiQ%b3jPQh}dj}JxJFjw>vAD!6f3t|6?wH~gtbOgtm}#P)L z^X=bRXCI@>13*glA7=d{&>YdVY1)GNkaL$zDK_R*y000sGd1tnLB`40Epg|Abo-%K zhs%oB*0^RW7wIY8gMjADTFY*;m5y)!?m9bpLFK@%H6HggRWVLl#~3I!unl-816#O@ zHgzB3dp1MaEMt+I5a;d;()~Aax7K`D)nsq`hpVU1UzjkgEAdXtAl+=)ayoQ;`**kh zHyDH5ZsC+7qiSmMHSMy&`6WY`2M+!vLtFSS_CId^F@y8Ge51LB_V)?nnVT*RP9WWH zGfex;+VY_VUi$r2oawf((mcatd(wO}!*tWJ-E741?cd%0CEwdvBeyAI`U&e|b3J8g zK=t56pf-G_x_N~%ux(`Gh`h*T_dR3_Cecosz*>apc`ey!e}6_^$5HN?151);$~$x8 zeI=tTrn`UENq>x| zg!ad_3!LU-HTMb%2%|Bde`V1IKg9N@RhI{PFP^ZOQN=o=_)F zl;_rV^X)&e{YxKO%kTpFp2OO&ZK_I(d!KN`$h_)t0o9TH@jDz`1HJ-o`ct1&ed>B3 zo6(wQt=IVmxW$os12}{6#XZC2#T!Q|M|6oy9nqz>w0L9O_X1~v_dp^VN<(QWO{Fa! zJ^_jP$;ylJbXd3~*k5_At0?BI;J`Y1FX28Hya+N`=aGo#*7Ec1Ke6q^>)|K#tu^;r z_j?a`2s{ry18%k?_Z2|np*v81v}TgE@a8$lgLt3(0rx9FqA^R^*!w^>d>^_9PfqDfsykM%Zt?QZcnbsb9C)=Kyz15WFcKQesbB}xBtxT%}KwX(#AQU zX5yCSy3i&@=kTEZE$1|L#RlhrO!|}Bk2SZZgs~_OYnRxEX)GGIl|k7%ka+EJ{>n-A zskr&}-!k@pMWmu=Ey{p{wKsPMnkVGo zqH+S(BF!r^NH?B`_H^&tfBLrOrswalqscnUFmkqo!MQkTKgs+ubN>wcZqv5^|6}g9 zYbxvZ_Bq%v7U??xWx*2geFkZ}$s?EDefv+}*4*^kfDMfXE|%`%{9*K59OBIYYR9J@NM{{`&UqLw z8mV4d?*O(YZhKwFkW43T{=WTZ-~M&(?#>z5|5WTSo$+|74jM=E0-QFe*)bY4|SQ32n7^dPBd5M-pbPk-=e2J)&WZXxiVE^GUkhhBp? z7i5rbB3b0JpI`sCOxavPox00H?68{e%O$(?J>I&B!7hwLmScmN)EA0d7AYCbnY+o$ z9&6l;g{<{#TuMH)rYeOE&+zU~15qbFBFd?CpuOJPS6soV5=- zhG*?;M5b?E?1h@5Wn8*Z-BsHIw?xjqmp0u6f+Qb3SNjVJT-A zI<$q%ahH-H=OLTt!akEl8+?W`uovs?y6L<^`^CC_qk7uVE?S=<9ziB^&h0@LZam61 zr>d^b_61$FXO97z4{}Stqq}Pj9CNb4X}kx9My7WUdct^9gs*C z8V`C4Xzi(>`($H8Zxd!AHokRr<X&cll<=oN>gcz2l3LL>!q_f_mRCn*{`=#wR5-dYWITf@OTnv zO;!bKQ%AAO*2qT#^4dRqP%tligu9N9<=m6<;|6HXK{Bj4jdvOYmakdKvzC)k_08L*2a zXP)OzM>kqu8{OaZrC$a-)d3G7r+&b*9?iC9sX6bpv=hl#{hmpSDu#3JjJLJko&}o4 zfco|t*ce_rlWwd!DxLrE+2e-s9o^*fSC*R8H<+`_hN@@kI+6WLwl^|oxSsYfF3DY@)No(YXspgOt^&IAAPDcBVu}lI2R;io5Ikvf5Mv z>BcEiwYC4ymf1I*aSg_rHQu~GGS)dOsixbW!=Fq2x`y@}*EQk2=1)j9zvD?eRMysoHV6;We%|UHgXX zQaFcv{QjF6OQtfXTx#we6ODA!Z;$ab<~8?}-`GEw;H=2DemCelZJ;;2`eUJq?enT| zDd&)<@IKdf)j9J)vc0C0*}hd?pI{82Z~Po#Tid%m{mErp8;cpMroVCwI#NH+;@$sm zDvOUeBW^E!M=4T~d<;n6h}1j_^4uGI(n;m|4Kf?Qc=8D2xDz(Dn_kX5N0<)=8Vh}- zlj_N9S-&?@TD&8#S-)ds4`;W=S=)HiYm@vpUy7VrLl zb6KbiJW6|UBJ+ov9C(2G@9xwJq7*>v{yj55@=t6-;Zi)L+>(fFj#GjT)k??WB(Oq8PHg0XYH@0Q$~MB zURuvD=sq199X*cL4$cjiH*YWIQhCsKROqYl&6Kpili1Pq=;tr}^3A#1!M>Vv&DCoz z-wD;(2-v^YtLVHOU*EnAvSkCZC-x16j4Jkg=IW(0zriN(N*~X+e_sY!wSnt6dt+VM zST0}Db)kXT=w5T!*((1&9bcBL0St^@$ain5Du(225zv|~`U3RVlb(Oz+q7@fsoKC9 z*j9o1nz?#K$3pcDPxCv5spiiQ@0W>0Ht-Jb0M)IzYW_xe@=(S?&Alx8+2( zoQ~gDthE`VGa@20a(91BY>&Lwv*>KUbn@xP@#W@b1J_Un)&V*D4lvt6%vWIh^MF4V z>f5rLEvMuDG2<*dM_6-HJ27|P0sdI1|K4lCcmL`1F*%DfckQ3NQ~#T1nx55ya!9BC z@Z+@q_t9I-wenr$+-(Czn5Qu31K7iW{cA6SZ+E^7vSkCDI~6)!HkQkm=IUxIiZm8_ zgKz)M%D>LF(w-sVzrK8$$tPJ^d9?M&yGCDNVN``-a3 z@DAUO{T-(*i&sVjay|9R}_9>jjkgJz@UT$e{Q}p_TuUm7U%T8@_u!=w1{(JaO1__0s_*2QXFPS*@ot@KbtPwIY<$XLV~mB9 zw&mP_VaCa&qmWHeAN;NY{@GWl*N1&If4Ss`-txvfz{On{HWe@9};i z?;7hrj6T~)bfz&Mef#%rAiwsWzqaCEb%(yq*xFTlO6k4&kG=H{s0JNX*PoKNhkd({g+g{Y=I} zbN3yfxgVfC^5=j|^P5ZY|4zSlGxPlN*jTjYWk6T*c?3vh+)L{PkDx!?)jIP=eYV?W{qVaHFcqCJG%4gzj^YvPXmA6Um&*}Q4Z%hQ5M zK?7(meW_P^ohY2_zrweFZ+4bNyLgHEeX!=W?3dc_Ri#Dw*z_##>4i?HtsLHkHB)wS zYX1vEp!KqMdaW*xCgr&m`Sk7IV;!4ioqP|y8R6rDdHNPY{Dr%|kE1^8Qz@kRWKBs( zapFlC)g=Mr*fE3huz!syDb2RBvs_KrxBs@YiMM(09#LJ+`cC?&tmQQ3mk07#>)9VW zy$qzXw?$zuilo<({~cijzRZwv(wVEXo!|*incwBowsWU(*T@!v&Zdd&%XUPw~tJ9ta|u5 z+A+g4CSO|-?9Tm~dX_s#wQgTQ8)h=?`}KHF6fiba!1q1ZqwlQxeFaE`U!{F5^904v zS`%6pZ^LoF0i?=XDrx6pSl|9rvmY-yyq~>Uo7>0sW8*NJV`KLLFLV?}-}0GUJ#|=@ z__ezFzU_jEfi9G(68uv+C*WSjm-<>j>d|`_Hbv)#e!MG!7|S z!%r9s6wZtN+)X{biQ?HW^>0}()EB>dJknB{W1PlZ<9M4u7_Yo};Xl{I`1bG34!!E} z^BTUnfX&*?)qKXbtZ#pR?$KsX`)9=_#sbzT+OIJ4#EXj7Qh8GU!V7=!78>Zro7_2v zcqp{I^5!MYTo2>hzc<_Qs>jdStKxdj9=4@%$$=@@*r`{Q?Pkn(Sf5*?xvo{KQnCM) zl+6Kd_8;Xfq?X|ux}FY7~UPd^j?XSvtOyacrDz<0Qq_4OW~rtl?96y zmk(xLM98R{x)uLH@peVbKNWiIJ*;p4so9UEZ_P1joJ8_-nq$&_EA2zP6fBAT>sjs= z4eMX=MMv~R*K&{l*IpRCH%Y4bq(Amf_H&a zJU2ImpJ>LXJwAsseRR?) z08a7T+{D|2NTv1-uzvd3LB{FNGk;MYD8gOk)vXM=(aTkL-~PSZ#S671tl7t|;;$Ok z7|)+HG#_`ZUHbue(Y3TR{7z37v0s!N;Nm*|SV(ouRIX(_$A>={-ZJ8K$q64bw4=8@jZ*!d9Iy8Y5w1e~4O z6*|=(X?c-FTZZxN-x|J@3E?q;M$;>{fplk&drC;SdX;4*kD_R@$KKM{p*}%^NbM(yTHa+SFk%a@t#M! zcu(P-)*=ue+t*npDV^(l5oLfl&Fg>O4O(-wC-mO;$Xf&9b_31rANtHgh_}l<(rQcF z_L8=5|K9EYQ*?gFQDxibm6mN~+J^oYalQ!a#x}z5niOr~n(DH^MhnY^8u9dwD%(Nx zQ3nI1=S7DK^9lWigN`iUp($^)S6GWJ-ZXCIm6pau|Hin~hGFXZNZsycUL${^j2{?+ zx6q;Zf0hD0uj#f$IN$!g+lBP=7AU3O?!!77=3_%^vo3!WZnAx^V?UOj75Yq@;_-j4mdg;9Lb5k1j$vsLk9@g_d) z>FHT`Y@YoQ-PEcdAY7Ro1fcaqP`1;NNUJ>p%89j{8)~fYQ}Dx#FX- zKWg$8-AC|u+*9U_vnC33&ZsV}hpsIz7O&$$DrG?F-T<2LCK=M-MwaH8yQ%QzRleB_ z=i5JRZRT~hURV(_W=~vw4*aa~LyU#2Bf!?whF%W5$X)gE>2t;vbp?&~svAAaUH-sp zz5NEZzttkn^NFW7d%{pdZsHqSZvd~noi=NHQCH@6oA5Sw{7_@g_@UxW?**^CEoUxx z%SDn~9KG79gE)6u-|Y!(1Fc-|EBN-$SYB;OsHeWEYrohpr8A^7cXTQ6Vq;&?Pu$6A zTvN2=E`MlgUB4H4m(`XQ^|0?(kJ4h!p|rRsbT0Qu>nrMIwZ-h&G@s>;|1SJ{-A`~S z>p^s`MKgNL5wo8@Al|M3Uh>&iVSM}7_e0AIY3qz?rpKxYMaIJEtaFR5VfLby0(VI7QsBy|9&WZN^{nO8CDgv8{2d7JAv!>Mi zro30)7B4(DU@REdd~Y&)K#e6O<~;E!9`or%7A+CRxBs%>MSNez@f*|FDmF1xZQ~0) z>hVk3OM34+nOMyJP5&=F(z~#xY)FCluwUu}RKquPF7il=Gy_{D;tjv;fY*1^1yz#+ z-L)6Ze$m%4YAS|mp6n9P5_X#{n!f$(T;%dV@j~jW<4@SEQI(Miyc1~KVlyFh_>xCG z{zCY=nJ4z`5>HD+^(=S!LyK^p?BXTnTZ`344NvIb$eU=@O)1v9q!>Ccd!(f_!;$fu zcTwKruPASu!dq}5_@765$OPG#ysf4eW==3dd1cOEICQ+OxyzP4r%K1S|7d^j(r{(* z2J!7|bZbN^LK~n%)vsRrgp&U|VE@|Vw(JtF9A=uC{v$I->3s72WBy+Cq;xOg8$BD) z4{kJBbNokc(fT!2-uA1Vy-kyj@Hk0dK39-W< zc}+K);e7j#>icK(bTc>rJ5ZgfI$ySW6Odo3GM63A0cr~cV*h$h#lEaGmA?4cpE4l6 zsB=x;#BVyhy+iq#4K0lwS-j2VK7HPz{D?2f?f{T3Z&sdL>%X(uzuMbRf(O8x!0jFX zEq<@ZpnE4hD6VKc2t?DZEw8%o1EBVa{@v#A4g~1$DXkVU{|8#h}6FKce zn~nJ`=#|4;S#SYJWsc0sGrXBHxAXSTGLi}z?ftJ#|m3DLg2c42rN%_AGc+IE6-9kN4sJvO{?&>?tpd002JpNy%K<{&CjgibHdtH<#VD2N> z0y?*+Kr59zcnRB)?f)Zeb-ys*M2wtaioNj%c^kDQ<~K&KPC@q9L@LZVoklv=OY1oc4LDZ(C8+6aVWx z(vl5cMVZzdViVrNr^6e%VeC!)g^yF9=f$qQhUv)m|5OcY*g+Gy>p}CAi`n0Y%vH~N zmDPk(#|>dzyE#3xo-7Tl4=uHoz0iA-x_|xnaTAM4%|kDo&KiQ~dK@pb6lUy_vJvL~ zM2n8~UkkrrL&B&(;Dz23e6wIZV3BD2S3XahzLj{J=#dtC(}H6bPh%d%Vc%R#ZX2b* zn-~653Dc47f9c#Q!`A?f`m=6oCl{IPzL2}>9#8T@j|VTRC^DSr%_$#j%q<_>9a;~2 zr1x;7WKaRHSZe&&yzWBnfn-06M_RJ60~XSzm_?hyeuYaZ=cOYwPqx$}y@w-{LfwI# z6n0Nr&uqJR-}h5r*$;B6*r7nGl7^wj3&Ju?Wu z7d_H@fj;>bz=0NhijrVHbY^*^C3~GwQ(2_>1!sEE^HjXfFZ%ZHvA#y`-%-Cz3*KiK zReT#g{xa9Pln46~=Ft@N_;^iu(WchAVT)cwd(NZhkXJ&>ZOqiJH-)SGKZt~FXF%YN zy%ufC|Kq?O&+Yk9eG9|tms`9om^#Gh^@{qNfoae}9-iolw~AHU9BeG8SUs;LntWf9 z_Mgd{*LI%xeU$Y;j=Zg}GqudK?>zJ7mj86!J97OmnY_XH-65QR&|TjIT`+N&QNH`$ zYX5JB-}&Gd;8k`SL%fK3tY7(`1{e!C8!F5^|AI*&jn}BHvOEP^N>kwvuB#~St~CUr zH}$YBG~S|d&ulZEC*Xgq!ME|rK>rbP+9WhXA z!Nr^A!&4bg6>sN5Z_|sWm27z%f8}D&Fn~nei6gKd*6zzkqakOGnmDaXN$jE8m*m{2HtT>FCccJJAp=(G+dF ze=5(Nk+*NMAh%Rx>=w4O*uPu;efP}DZhM!XZ~vWJ|Lxs=Gv&X<>_0lTYo1x~Lx$~x zO!Mf+_sh7O4ahdjWvkh~psTU1ao~I4mcH-q%cd=DK(^`nP2^0<1az_g&|dtjuhW)! z^waj`?`8wC%{^tS*}gR2$9zd4v_JIi-!K0;Y6BlIKVIlGH=SvD&^iIoi+tajqj`6$ z6MXm1^S-zl+wav9_Rm@wg>;>X1vh{EE4b4bpaUMJkh>eYEen4|% znv2tXwAN(!@MVy$4akOMOCNw+z(m^mTSO{~@^pqvu3S1d7-&CjFWyOo=30qz&SNMK zP4VH&AblIqnI56N*heRu;yZw$F4mbyxnlc1AHD>93HTE5CE!cImw+z;Ujn`adfzl)x(gib09CC->&gIyimR?@sRFRjYn=av!~F)lW|Dh3@WOJB9DPYE^@iI~nNE?)}Mh z#4z&s>k5ome)AA^K7#69Zy2NF-1AXRZ~R8%7sk&DUw|?T<&H8;fyL|x}r!ilvoy@y9cS~P(_ug<* z7?oz6d~k!K(NU>2btea52Y1;s_7=V3(jAQkR%OH8g?^z6KW_ZH<<~0jcKI0f*m^8m ztL?{fF#oA-SFLJjbjq)f-J0%}XruZyeQfTs2lJv9j<2^zTT-*R%bv{(DaXGTlDg-+ zgg4zSqEUM`>D$~J`7vK#s5N%Exi{uZueG@MU_pZwv#EPt#BvWe32(W=ncVAFKU;Th z?xPjH!3wqd-_5;H;a4lHF@l@BsThawHib3PadWpRvN61yyG=2T;oaO#g{@X_V|be) zle*g!*%-d5d!zl^6jspC7$Gb{*T)_U8e%t8IPvbP4`Y|<-Vl2zux?`Et;aa`Lea6J zueK|riMz4-a#M_(yQ!F1vF&XbR!U9khr);|@90JN2J61CK0h^g7dpKod5-CJH;jo^ zTJh;xZu-x_eOCkhR>z3N(Fq#`4Tfc7P278@=bMm zTZ7Td9{wl8lyjlo{a2bS=y$ZuotC4i-_a4%qcL4Kcf*B_G0!mVYI7U?=wst=BmG7f zZ{hPAjLoFS?TvznVpw!|3GQkXG@xHKeVe$)^xM>(#$j0Pmfy`LV)F2A^xgOo5Am19 zXS|#4lem+~MtPD{V|X&tD1W{GEjqm$R_m+4i@OUQ+Q+{}?Ps*vUbg7do){^6n<#s$ zU!5*@O|K;RY8sfQb`-v`?uyR@^=)I_81GKK?dEPZlH!BwJz`ZwP29t)T>IU02%25y z`>~;!v&8IQQ3(_worokvA3D$IuRE?hV?3XM6a34UfG+`G0=@)%3HTE5CE!cImw+z; zUjn`ad3-Ue)D848HyD zroOb@>n0T&Va{XXOr&*4Ydp9U=-e88S5@C@*I70`T3-f|y>uX5+yTaO9?Lr6$|55& zW}CG29Un1*J=K&A<6Gdt5cF>X`W9b%I8UrI@~E?@{sDq@mBS2u+aZ%{PI-UM!wl&> zOzHk{&>3ag9`h_+1tWahX5O*s+knnPttkzv&R4l_k8o4Icm>&M|bI+A20oL=HpQQ)wXQU zGU&{_f51Cj{q(fTKk@bi>R)tb8Mf!VOLrO{NLT*{68BqQ|LxZPr90^`UHuZ}6t?9jJ}*G03!^@jEnhrnb&${|IEG?h*~rvd(^28kyLX#%PpR zl=J~f%AdVa^fM8oAPuII2zEd4sFi=PGB*REE2^{*DqJ~ z|0eyCy=y9mcBze>bzjX`)uKxJ9Oc0ck;S@da_n~zQ6XMKO^L67qIm2g#*C&{rdQP&C zQ07b;Vl1pM#~mjV#>;ny{!KX5^^G#M-^dT_2K2s7CO3uI71+r%KFbdb2Je7m^nN1D zggWvxZ-VnUT+Bw-C)}eR=p{=p-ai-hfA*Y-g?Y85?(@{hz@R-aivQ0%$m7mrV_c#K z^i8+dJkVQ4UE)|)Ed$^?E!5$GwV-vk2YR29ml42<-t1rBds)bLU!bM;UcCM?z0XDc zpQ87CHQ$wSyCf^x1!@nfot#VtHxgz|;6_fm7u2qJ0VJdM9AP#93Frl&XTDRs%!_?Y zeugBXr#8^S$*i#`3!Vm&@t2B!F6#dV&3j_$KK36jN zy-$CH_IV=xH`>#`JkV2rbZ-yzWW%!MWc;P#pNsl`58bP7cPwAjz38gAEB@zrl!a{M zK)w}9+r0TZg|%r$Waiegr{jQ^K83=}tS;Y-I==Zgr}f^7t%*?I(u;jcUuW^1Bjw2@ z&!#|v1;2C0ABNKZG^HG*VuLkje) zd?)+QRsE}s{T*yg-h0%RjmWDmrOcNG)$XbU4Zxl^ z#cRBu?%xBI(AqdsQQWm|`Y^Tk$a`=b{v``9b(5WDvgcgYzxbB?J_NUdv%#g{Szy=2 zSGZjU4h18@cAyMM-*!8YKJNkNfeV4^g>ON;oWAGzW^fGH68Wp%9TRhtUwl~A@1h}E zqIoZnj^o3K2iZd@*cOOK@p?HB@9|-Ow#qS)+fwU9_VRp)qsTb;5}B~T<3va zGXB%?Z_oO_i0@A4tAFHnN%o*Gc-Mn`?<2NBE7ES%b=k|pA!~?x#>80b}p7pQ%-iT~Bkgnq|jqBA^nB#go z5$4}1$o6sG$AiP{X^c!6)=08?iu!+y6-MWDpGbk;TZG*)7Ppa}ZlZm_zy|o=m;$|Y zbet$od)L4E_Sdp*W_WdZsQZ%1gN$nSA>*bx^35+_dib8(?3bp>YwAOQb*T3W>eY^{<26f;Rp!exlAYP6dTe};SF*dLV|9gYu!Q;TL1HD^r0*hnN zy}gc>-WB&a@P`=9a_*Iey}v15#FKavkADLzfZdkj`E8B+j-Y=pa?-e7HDd+q*6N&s zk|J~brD{q@^>APO&I3+m*NwKqp3gX8-^u%IYrfyLW^Y01K+V;xgWr6hG~MFoihF0! zzslHIywe$PZ9Z1Sy8|02QlI{|6!djlxHP0VP2TBxP#R`h(K(B@F5k)QtTX8U19URn zNmtU>oU#GxgQ?AM0QeRplbOo(e$4e5Cr)m@p1`yL{0O7>`v)HAB}=a*{X2vHH7A#J zT!(sKQ^LLFL0`*h&*_~ZRXbkt!={wx{T}GGq<)jp?hN{8Jy~c?dmk}gaZ!Dlb4Aa4 z(3j?*H&7h6OJkBieTKH|+a=Sz{5ymGH9xO5nVY&)R|^`eUjdTI?Op1MEfmM?ioQ=m z>k{Ts5A>3y*OLC7LI0nlgI!I19eq+Wl|?3_d+jSK3r++-fn@SlKXO8tal*)y=IzNb z3v?!uNpCdazVJXVS$Zw$-x>5T`Q1X9-H`t#`qx;I&OpL_2zVNJmG#r~_lKBkN2iWx zLR0S*+BYfA&tXn{9Wxd zdtXh@8pjcw{O-ej``WUi?n@?`{RfqM=JL#S4cXqv|h{$J|f(|fyTj_ z*szxe#o;{{;=ci;!)t5Dkq+Mk7l0GMY;Yw|A1z+L8rRbrJjRR6wV_AwUjx+clb^-6 z;wY|Y%mSh*+Dapyo)x||{_m#l39`0qfCmg@>}w0o1MOMkFfNkJ22_SXH}ZN3XpOvO zGYvQW{7hcXrwsR7IGy*q){rrmw1_>2xc39n?=QfK7dM(tVOz@WLwMT?BvW5j{FM&) z1Na)WRGwOs?v0e6^^`CBB{%jC$WLjT&nMSP(~olc)nS?gu{{`TMzg!J-AMF{{Wv=2f^I9J4 z&HsZrp4SIh8@W!RwUMrwOwofixW>_=H&nT(1-Z}$O4q)gXPt-2wyOm2*SPLRgn1F9 zn@(%Q`!D?Mr@YjatX{|Dnz0N$ln&5**ZZI~^4Ah+s13U}^SValgaOSY5@8J({=y%D z*88*s580&oSNbmn;<=?S?RSop7Hcf!vUVVsL^8gEbzhY4=Ib>jN3b!tvz_GeL;Cky zitpCAln&SjwCBE{#_X;HgQ_csarPMFder$fQ-WRS7Y@e#DiA*gl*pEH!Ea&JgaBvh zw7d-<7wKPZBh9zxLf+HmL2a02;50Bf2Hn33+FSePW!`6-!*`2U0kxCRe{>#g3hA}? zzMy40>Pxwe_gP_b`Cob{(YYD5ta)tdOLj&*sd3U=Zy)Ef&a%zhedt5=a!acp-3N@h z<%M~q6#?1Khfgm(nX0eyN6pvl5pHo4P#ZvNU`O$j*85x9IEzNi`zg<|7vrn?a^7} zo9p>}ku_`mK#S{t0pEsOsP_PU571Hig1N5WWc*1FleMO$mVIw6eUTRW*ZDlbmq9Y# zeE-&w(LK?9DXpP@?SmVlzThHX|2bp-8k^S`eM=XAlfKfS4$vHg)}^(#GOjhWcf_Fk z_Qt;(zn9gN6?fNp$Sr+oZ9UK&&#GiQtRpg-G_`(1$62f4|#J-KU+egtS~ z^&63?oONN=0m1$G*P4X(AgAx)^9c2mG-jr=d(1uE(Q^dl*BJ z2HL}1n$`jD94{N)$K2%CUUZy@PxS>ShIJ0gwANZ9YOYCsrJcx6?UnUwD>Pmv+YPMG zv(|njl4*PMyPNzZT<@B#sz7@Y7K{t3FSs}dWvDtyAyOS+7C#0ivc!z3lQZDrcUxc5jVDH$!p5yhXwO;o1UJ5e}s2{C=dz|?A zxqg31Y!6>T-8p$`&!F}WY7Zgn;gYbQ^| z0SAHaK&t8HLip1bmk&2;(4npx^0SzG@^X5D{G7IU(ooJAjehs5rX*l2nLa#OoLtDO z8xPW*bm*kwgl>uxi`!oHud{hJvGN|vkDlf3Ca*_WOWntsL#JPW3<8StSb{iiv~t0{ zGwWY#{VxGUj8U(vf1tWw?ryqKd;cP!u0qND5odvtNsLeBYGFeQ4HOLRUWcogpPbCSOq$NV-Z`z97lVl7jp zdA`%1lf*evw6(@IF3!=!`N1U}r?gw!t)uJzBl>8ADEEzRS#+UxYofUv;sgge#2HMy z_;*LVt{me)V+e6_`909l^{?|6cLH|(%T0dqI<)R_5A5g=XJ_2p=6~GcXx)HY94o!H z{O{=cmu&yaJAA!`e8W@FJ#NYNCR{<;rG2k`bByst_Ia6~al0K<#-Mw-S@KmJjf3gmaZS~)g z^#3yQp8>S*ME`n@m#g}A@%=mbWg2HDG+s$D9j7?)e75F&N74Vg)X_Ud7>`w*9Vs{G zW%Lez*QT3z*>a9|kT9C#CyqJKujhnuG&i(Eb&0}j{)@IG=_ZX+8r-4AJU@mx2I3N4 zaaMqMe%j)`qv-!)&T{T*&E=pIt!>j>j>hWTd&>PPc_f7J9r)>l9A#NPT?El$SrxCax*+>ndx|o6P+_soIOCYIO09Zo0%swPHV0w zY)|^vJjyW(H7CQm^#y!;PxCU=^?%0QZ7xS~j;&I&x%`;WUcq7g+$EPf zZumKtXU)qv#0eys%L$04J)OjH%FQ_?dmP^R33w}Y+-IaXtu<%Vp7j3(a?!X+6B+Zo zBaoky{kz2xokVd&)1FSUI6FAR*%`Mlfg^9&W~y zm|$Hwb-%YbR=%?FpUe7J|48#P`nU4>6Zf}*vKVwPH_KmPm8Z6fa|>yUw{mbBu=AFU z{8)L-#(%Ere;Kkn9vlFU1NJ!<={H`-l0|$R#Z4v7frM8a(TJC;Q#zt6p2XV;Ak{cZ z=T(quI`%m6@zcG}Mg6ZpHaqifH22@vPzQUMW3k0C=ihgNPNMlAGft#}u*?_Ksm_k> z`<44D7l__YHPL-PMqRXDpm%)IIN>R@@AYoreZs%ig*3RNp|s6(s`Z|e9Lmk_k0p!a z#&aU~T-5(1yyLq{R=V_!P0lXXoXf=t%Q>-|b0yo`UR8zvFzuoHpPWl$Ttg)$7 zjG&hBOx88)9WW8Z&vW?ofBUHae?)H+SX05cXL{d;)3`ZU7dkfsU+?Wp1~)UltTRkf zDfUgm&Yw7lxe0S^R5AL0!nc8TQwAPKS0U|ts$*^`lS}qfr!noaplm?%->O&qx*#`Y zAl<004r$&~bCKD&=1v&E{OK^Q#nih%?E$q3KLqMu{MSdD%0N0$J6}4PARX25{hv8A z(tGD8vX>gp(5zVEV{;59yuK3~A^z8K4k=Gil{dE_; zV*IXc7{{3ov1e0$a)>>f`Oo6Boql)2m?-jAdggunb6!MGm}kuX3L5l`dB&c}f9yG; z8_9!9{t?~8o-yBM&-HQsm~+hj3mQyko})L>^o_9fZ2H3Ps#Rt}vBzf5qiy~?|73;L ze_`|IU*r8LpZ4b_1q%Nr&j`r=91~EK>S}?levarC`_$Daptxt#$Nn6X$7+F@(N!Nb ztx#h2V>A>*1GqfreXYX8<>GJp#rsuKfhdO z8OC_F%?SSGOTd?aF9BZyz65*;`E3B_j+tw}wf@^jQyEA$lF=rd z_sKURqi1)fqg}1DsVNzy^-*^z0UURydOFU+?Bl3mgh(7V=tK2S%NPfO=*8TE-|noRtXy`Xpa=b)we>MZmn z%dLS|`R_*9Z+!V@&Hf}y$+Rux{~PJ*9E0CMs`j2qT&1UUMI+U8Gl}bzR$Itl=M>)v z#)JLAY@oJ)7d_~Vk$-{`upgKX9s^#^(~!IsR&f-!6x;(6om*nbOuUK5SwK9C_kV!z zffXj5|JIhj_BB@X{f3_W^WB!9v50e_x!3tJ4}e5@ko?X8>+&rcV}8jX_S%^HCH4Ng zE=gR`y&m+{8FTX{>TEi#-8T1EoQ-?3vLN0D*Ya(nNST?Ycx+VY>L{e80E_)GmjurB#{Ize8}TRgtdly`jG=r?yl-HCU5 zg1EOL=Weq3_>1tyNmB-irp`}tlhu>(rf-16M>Y9Se6S!vTsNAQduz*o24&u?cj7Zs z!#D_IQmT_4NsyP*^^G$8BL_(#08Y8;8nm)zRBEMTmGNH>;6loGKLzHg~p%Q!?Y&uXD7*r z&bFCE*hV|%nYiY+*XDEozXb8+H=jDYn{@`K&iv%u$2{C8gYOc=Romw*`0Qbgl~prd za7x8++5Q1wWrDc5CV%m>0!#ypJ%wbq+kuOL+NJVWe*XiW0MCN2fRzt@cSo{TSy9}P zKxcAlPofn@I@Wtc(7olZFsFf$lp)0p0oC22X@&Wcd!?zg#e;Z}pZ2+m$L-jA*q48e zn<)?P0xNDh|E+ER7LUKcx9XbZ;B#Q*^Jnfa0mFgX7VCpOzyrXFtG2#mqqvft)ppeX zO<+6Dz0~`19rHfg*N6|JIEpJ8R=+{%D-ET!9#GmNfOrv4N?T#X>l@&Gkf^*_ye9i^ zEBU+ebv5s`wXAYpQ!>Ol4?+Ed=Yd;1x!(&m2Am;WY{gR@A-DTLqPWkYFV4E;eaifX zF1=?=Udn=3CyASgZZf}IlE3Pi(IEacXU-9sa*3YJ?v@|f=#logA|y9(6i}ISi+`N< z(bm|`O^LcJUxr% z-oT!$rMp7>-Ud!_Gj+=)`RjazU;;VFhOPSAo^R!2*937#pA_q2F( zikqohF3DeVICc@=2diO>#=2Aw(_dHvzhw3B$u&~~U99${;_JQ|H{za>M1E?+D-A1I zGp@c_hTBZwruU8{f8}Kb*pK$Lc_zPkPI_se2W}?-wg2PutM*9zv+AJnj7M~r&eXSx zlm;~}F%HNtKCC^iXrDm59_klx56tgE@5jA(ijSMk`&_brD_@_I-%CLSI1bzntp1VO zkR6p8RNohKf!cvyE9cYAXk;joA-(XeN!DjCQ*I z?MeQxGjA{mG}h7cj@^Q@am@45wnK&=C6J@~TH6BZq*0CW^$gUvSPs&aVJ7j~ll(6Z zPaT{W=KJjNR}C`6j@Lkb|4bmmyEq%Gz2X$se~h!R^2;ZcDniC;4k^@{q+9BaQ0VH->oD7>()=wP$_=ZnBbJd1Ts% zJjO3g9aQr@pv9Hu81fRkpUT|`;3V=UZVCf7 z0{;Tpc@s0ejmP&u?q72gU>c17o2&9>eg__n8&3(rr&*8q)Ohjnfv?Ns`|vYCNE z^?z&gC|!>iKlXQ4GFiL%Lr?ipe`f_~?ffK4?|<+!LGqq`-0Jpy-PFP2S5Q6kD@a76 zCH(Ft&0fGwq$#;emyrsy9d~{kF_tI#%T!Yy&iRm54@5_9dxDhYoG7qp zfzhC?+6+!|Qk!5$@n*l|2GnklES%!DwA)3sCG7dKt$DKZ>Ge65pC0_TRh!vOUJK?= zq zJlysL@3%3Z3(W7>V-MN5$cyB!_VcrCoG-hMHO_WAI4p)U_)j!0XOG`f&(E9dzp_*Y zY9~|9qjN*^$X`qGoC#f(%Umj(;$G~04as6CaR3+O}TXG;e$`BSEXlr2v-(4J(~61=Le zR-K)dGN5()TF(zXy<2kWeb*AQ%N9-cpip<0tX^Z2i87>1-YIH}CR?AGE&kg(9op=n zjU~5iTza3!=D@3|SoI%}28IZnxHYWl{cmuV2*0d3~AmHwAhp37dgeKr(r>w!g+MM}usL-=ra0 zKx3_KHO?p**3-Z28C$O|e$+Or2dyn%FX^4fdiO3`@0g8?JSxAMYtWp7m-JgAjMjPX zyZ`ZjF-|$GzxdJEo7`F=txRdY3tyVc%7*x9Pa%KJ{YCfY|I$i%R^6_AEC<$HZUgt{ z#h`o1Nq!2m0_4*BfVo`l!%toYG)Jtt;%erM9p{pnW7b@==A6l+>fuc7BUya4Z7xRV zgMsS&6M)t++vEK=>K7nPXfXGBKj%ujh_wtMt!YSF1{4Npt%T-X9t01#Kz>?#VV6U; zpI4BE)^3_(ie|m`-K=Uv5bhLM9@4E7k+8PDivj!`pGfV9BGkI=L^8X)lTLvBhD}nMY z{x$Y@1F-gJeb4>V;2EHMg_EDc+yyNA5e?B2P0`Loj_Jm2Px8METi*<%rPGbIrcfypPtotz(7b ze_f!mV3+6lRTZH;bZOPATN1`9Q}(be`P^RQuRhdBV2)YMFCU)Az1sM8*>F}*ptt&z zk}Z%foYo$mH-GW~_h3tpinDN#%x!(sg0L$O{{|>k5gi*b+ zHaHf@&GNVYSKzicKB3f7oP&jNByn) zz%AfoV7Wccy&!+N$xmSv)^ZaK(GpFozpk{D<};u*-pRQt|EtLVM%ZUp{zJg24*v=F zS)eEOuC~A4$(9b{ZCC4m_GevOw^$!-HqTCTbxI?|J6UN`&XVr;j!z@q`&^X2+ADhi zvkuokkj(6Pl&x+mf3vQae52>j*u(vUv(4~ptn1a@c@x;fNtdM6xV}$gxlU~-d%R4Z zb5Z^pE0Zl7HI?&8d|HLZ6ox>6j)wbKjN zkycf7Kb~}c5D+i+aM^s$Mfod#s+UJ$i|g~hANV)0=UZcF=Y#%0^@ZNG%8xyq-0(B! zToqPvoW|dj25FgTl6LSRV5gPsXYsZhXv^^jz5A5cGh_dHR$D}433@kv36yW~dLuX+ zybR=~cckRD4EzA(_5rvw2Hnd|ehQ~fBFYU0(U7;ha*5dZ-3_5&YPWfD$i{# zfAKoAjHCd%81Z`?&LtH^Uz15XK%SmFHZLzuJO_f-PeIYF9etL4Btkf!Z&UgOj{ehv@yP zK7*ipr+P(kMMJbi^M2rzrg(|x&8gk$6wj+$Ys>!^@^dFpTl+HbH4v}gf`-_?p5-q8 z*+AhHPwroU_rTAwXT>=cRD)+@ZmJ*D=Mt>C;$PfX0fki@(Gm^O5=}kF^L9DUUU*Y` z*WyhyQblXa|8{uf9ZOj*HT$-8Jgcn}&$pfz(hpUCN@ZW~WGqlPg}3Txdsw}b_W>#| z7XiDx6kcu@0)_b_xF6W*=visZ0ZLP8JJn?hC!WQ-!p#G2yg8*|cWZ6=EB#-A`k6lg zJ3sNy@>e*8k9W7D!;%LjHChI`ea?@>iL2 z>d$v(8D^4q$x<@)I{vl^VV8kS()8o5mY-xPnI@IL##Cx1o9hnF@a3O1`#YUIbmlsM zqzWK_^aO^27kwRMtOLn#2=F3$GmOp?sfmt9jV9hpz7EpXfn>jHwBM52K1hA{y?TcE z{c-6|b=&cCiE^i4UNK}N4K?+Y3fq{+Rh z7t_6WR7bIUQ+Fn7z2o#)uxeEV66O_upC9MWeSIv5fepmpH^jL&-W#h{eQminxeuG} zCJ#nGx#en$3^M^q;9_tCgfcbEI_d69`$-1z%w@TPLMr`s^H z7iJg@6Myd#ca1KFvD-S)@S7FnMH;q`-7DSLeJrXy-8^Hrk-fSY5xM6_lvZB7?(-t` zBYWi|u7r7X;m0wp_%a(`mgCF#YTH2X8C*J%dp8gPc}9PJxpp&*@qGS6@GoBiz66>} zpt?fy*XB5BPx{+MU~8bhcYB0*R~~vs=iC#IRhsKyD$FtZy@>Nqpm~1HXSQc356a7) ztUEOKEH>A=!CDnl=NcbZ-ah~xZJtMYQr^sO>)K@?cOaX;8+5e1TlrDmlt+8^t+^4E z71@gwzN7tXUB2?@mVd47$z(l9uFFe#QyvrLUv-_Ad}vL`^WZda0nl1Zw|E*iKNFk{ z-bxVe-?1|cSH<3A)g|T{Qpt1*@L~hm z%D>iB{TZ0$h%@^Q?E&Cf_HCE@G=;0tn!xBfTZOB22v?(Ud~3p5Kd5li9m1WZaDuMb z{n7keYo+b!$3JH)|Dt&VSUZ-!_|SZ!)O`fXa7xpnr)Mu{iTuZ> zdn|mAcj_hTgp+5R-%dH#fwzcR7npUT>U-0c>m0(JW1Pr$z^ON_^S=l)!y#Ne4~gzu zGXK)YL=df)%)J6;op_Q1zb7c1_M*k@Av@6_+>-b;4vGVmhGlPw^6KQ*lKEG9{{U+P z@hS8J&j33wTEndR%?hXc2OPpZjhkqg;ds}I_Dv4q?DXB9TRQ)$hpq>Q1J#Slr*yRf zTnVlL9|D!z+rd#W=w5E}`x{VL#Zg?zMYR3|MB8g!=Olxc&cBuBukb18`Bkuu>M+*( zcj10FpuEX1KAha-*Ckw9th#d>pt$yMdQKOumH&AD=9}jW@ZDwAV~eAEXU=rU!x<5s z9iVkFtM^haa0nMqJJtKP$bSWSFs2@9o+oSQ9#lKzt4ehj6LpIX--{ z{OdieF!6CRy}uQDy@8eYjktfgGF&z%)QD}Ul5=5==(aF z{{sBPPdaHPi=%qm%FjF8@6Nd{`TPsJ1G$NY!emm%i^k_*N6;*vZVu|hDLgm~{0Nfe zClmiAT8F7KMgPCOGl7<)Did&r1Ri?=aXbMTk|-!^h~ONVku!n;IU~x^86?0cK}ZOq zgAfT5LY};YAft?s;IPS}A~S#zk$pU%ghgNwOju+mJT}=^*$K>iU-i8`_f@*OtNZm! z9y954zTCh5`~Uy_@2#p^x2pPeT>E(sO?&m$N&RVK{?U`UjC#^N85n;x%Oz*k9O-X;dffnIZ^r24!39i(|ni1 zc~*7^V*fva?}HuACmk!#@t?wHVOKZ@;@pth^;f=$xF3M>{b#(o56GTnorqt5+i&LY zJTs7U&6~vE!~XMkv*pL2dLB4`I*&H%!(gxHerfa7y^k#Wal8PZe%)&gmT_tMTFBfK z=K+r$kD|g}65DWR53ye{Ivs%&FPCqmyZ)lda5eIll@L(`L4iwyk)wt4mO0DFc*$V@Ltk;$~0Cx^0$xKf1K@%hC7mY%A;qqMxi`B z(YMn^BOmYWy~i)=|0k&H^z)CHk8{vu+Xm0WTOii?j5s?tDYY4h`PR0YJ^$D)*u=a# zQNBdX#qcuR56<}>=bn#Vm%PtVR%e|y+UlckCvu1}?7w|1+W(#IzvuO2?(aufg4;2d z-ynNU-UNP|=rbR-^_c>xPxNiKT;Bcnx}KeHp-ivyC-^;w@a`+U*V1Nl@I28s^=Y@? z(QkS6-{)7az&mg%>=;V7qu%3uuYl*l+ezIi$a$asJ`-~cmPh|>bRf zz4hAJXNf*LAZwHQ7{gcxgLZLWwo~dq(C7bkw6QjL{7!_WV6K*KtG8ae*oSM8*8=N2 zCQ_e0gFf#QeOoD)NBTXJgR6MdVN1J(Zv z`M$@xm_1*eT_Wy`EyJg?M8$;YXA-1nP z_Ow~syZ@#qdh4gpSe*HcMb7(-seEsyK1bwzPWb9a)a!FsQm6lzi3jgg&nM=RFFVzL z?@tZ@+g>033u0T!Bf+};;a#xK((NK2<5*|hc>k0781oj0?cB_|x}Hz0uUB@e|Hi%- zX2MS)U7IPN14qI=kmjL}dh7E(#xb5YsgE&Q^-29xxlI4PHl;rQmt_049TeUFZAM)u zo^uS=G4uI{eXQSq|L^Jg{|D({?N6(DT~7b(keIbsO9jX3tzI#ekp=7`=sTJjVl$mW9hF^xb3H_ zmw))B1Ap#f=t6!gC6@fIgui(4-{JQqj*eyQUo6A#F^rC7_)Uh9XzJC+qbjNdA&^ScdMIbz42 zuHH)aDJCy(SG(@G2>xBsQpgFCM$m@g5emnAFgMv)m8)ViO0bSOq_NcLt4*qpvN5fX&vl|cUz(%2=JWR)X_UY1 z{wsVECWG$=(%5&iJrR5-bQUaySg(9GSf`D)vCh(1#x(ZvP}E0l&Cy(2&Hp-l{7wt+ zHnZ>1%|(tmIJO5_7x+ucdmz>;ZxrgZ`8uTg=$H`GI@`D&qHn%zH~*`MxhBLMXC`g9 z&-YF0K8SorQhz)0UnKRDkkjMD=U6eWZP->m@3`G={;v@85ZD+#3myZ0kHE2g6>fyT z0q21^@H+Gcl+{_MjkfyeYYbx<)7akv+lu+-%XafOj>o0*u<;)N+x#F*gs0#IcoaMr z&jj-oWp&nRqwSvHJ~4)|jA`sf$G$oAC;xOTj_-;82VM1VJ$#lQ>%H!63-Opx_E|%$ z{|~;eH`M16;d=tdCw0-LsO(h!XVCu8kfX<}&yZr9j^$3F&OUqw)Wv#jvvVoLaR0=* zW@V@H_c)jd=IQwPe#vL_zGt_M-@%sx%Id7sMq7RKJqnD~EbmxvoN>_V{2Bez(mCXf z(Du2MBY6iljPlFidFe4P%6ouyBSBl^=$p<%Y3w4olljLsmXe1#$K!oHUgjDj~;@Q>NEvWp(~-w>ER2m7G)CqWs4ab4fqqI={?^>EL)j z4)X#}1f6x-q;pZ4TRnLJ_LJd2m^)Qi!o&N+Nv|BuhGF0jpJ^LkMm zm*cw@)bh;>$!~-?&%F#qvC8V7A;wm=8Q7XqU0MIs&Utr&aeDY$zM_xyU_Y;b)W041 z9by~?!-bEF^tFfN_;9PnJ zwAw~$?|b+Q0?KJ!yRvgj!?`s4#-``N&XBGdTW#|`{5K5TPuZ$}J$vVN=Xm}v*R2YV zqpZH#Pp=*I;xy8^Uh~*{8fFBX`_)<3$hSSsF;|5ReE4c$pBWp&oIlXL20j-6Wn$GFZ7 zKY`=maaat0f*%Hy)mf*Fw(VXc2O@uu^(|p7xB={!=kEpqWp&nRvlZwQ``=30$^73U zhdbd(I2-1IV|)jwn-@@4XPq|Ms?*mPD=PnsY1{ij&-ZvgbsF-za5Ticl+~RcblOe< zeT@>*y%Id6Br>#Ev8be)-8GVY%{^sxf_L(pl zye>JfKL^LdAz}MbN{{cO0?O*N)ka(I!91RfVJu^oxtH&6{;$!_1b&NX2xYuJczyDm zTnl_w;@I5>8wQlsS*J~UKeM2Fa`xSt3FwVoH11K~ZvJl(`#QK6u7Q)`ksvRjycEs? z$M`qkc%K(gR%e|y+UjE*>x`vr?CZg{V!rvZ-Tbd5rsv>J@Jfh(0p)7255(jDgr4en z9v*CL|8Ty$0#S;mDO42wzm4{YYcV9G`4NnR?IhF7Ul0TXOyM{~`uf5xKgg=jZeOLH{WzWGnd-btSJKFHrS z;Hqi<-IKGwNvNRv5ZH%Dz&UX+VNT}moXq$5mc5V9erNc9zdztHHkk03alWb@H!Fn| zKLXY75Aj8+>{F>^-!B?osq~s!SEaJhlK-8luv~l8sPWdvGH*X*_H6%mr-hZ-v+Kth zHL9xTYgc{R;p<(WHjh{mcy!1wDYNf=k#C=pytD5|#j;P_CjH6h-_Pw{{mCPaYaHiT z`aLcABGkU8wZY2oX)zof|FB-lej|f9XbU(BPFg|O_7>sYZ>^u`yat)u?XaS+|K?yW zS$>v*nUpI!x7fb9#JplTzd?Sk&mMw5=WJZp#cO_y9ZS~$)}?Dd>x^NnbUhh;Vrec} zUfJ&OVlq8I_tEFdVP&yEMpesIN#rgG`AA@ zJwP1SNA_nuxD9OYZTJFY{iK`+F^>}y{qD%xT6ZfLLmy)rJNl$$EBSfyMg!I z>RhLrlkIH^-ow5TP*!K1Hrnc=uQ7~eOm%5qQEnwaeV&J#!T3LcPk_&Ed=9V(9)w$9 zmw>Xzn04A{tB=0MFqSc6KKZiM{Gz}2I@^K!$2FVx%dzgzpttU@M1Ki#w!Wh@M`J{v zqO#rmyjPs`t?J)4eHeK$#CxtEA-mt?46tqMwE6I?>fbC+LN-QGUQyp}e){_zr3K)= zb3b_;O@m&zDxj>+I&HMoN8gx3qq0Bw#klX#)}Fv`G-uC*y#I-Hr-k3ia$ipbZDU=T zvX&oyeCIYhp60h0!)FEG2*2I82J+7n{|k`U2;ULFruGb?C^wG-8dKG=S3&5{c0VHo@vTFEK3{Z8^88oA{-$Mj_XCr!f~cO~wtZNxTD=i#FQ z%4zOpSwuhTZJb0!0MEc=Wh`~Adfj@sncM4is1D4t{E7}drt z6{@*sW9(QL#b~5=y;QVjYUCH|TpN0frD@gV>hYbOe?9|^K4nYuniA+wjz1=kVZ}Mt z=9)6|R?OXN>Wi>%K-o1_tZ$|4GuJp~)wwYHUafPZbEI>nb7q;iuVLr<$@PnC6H!)Y zU6~kp8?P1dJG}mlvFGl*5G!71-YXH?b+P&E0gj!=k0`6NPMduD&9>*8Q_1yxQ7rQ~ z2xh|b@GQ9Q6J>SQX`^jXoO=4&`k=)9M?HV%AK$%w4km%;!zvhr_Yp7`pbE1>HebXWewrRv&$hVXS;k>2|C6 zc`n=D_OL&^1D>0Xqw8bmKhJ^B2b9%W=eTLBkG_8m#%eS_w3455t@GKb;N1BL{4<;w zwjZNB8jb_UNxlo}tkXwZee`wPSjIGVI&M*JB|qC34^zNCd%QaadxLYE`^que9Xw~{ z3orq!(?(l;^fiXDjA>tDPH9=ok9y~3nO|UQyk0PNjPWS4=V{!w9gn}bt(>;4tv<2d zSjH^c$L139%GLwQ&I!?fMVHR&S$-~Fg1iOg9fKf8b1;`=UB$Jo`b=O;^p4}a6@+bX z5q|f#mfuRD{SgSSr>gI%2-6F^R>|H=@p`AY9CQ7!y7j|Gjj5}zWUr}wU*+qlBicIo&nIKwxa!;BQ?vkMpagIsFlq*Zgl=wi$D9 zTzG*QT2P&9>!?#!XI<3kqpm2fZ8mEEE$a8>er8ol*C@tyEp;FagF_(R zb1AE{F0S|W(bxTIteBUjIW=qFb==>BIerst%Q*iC>L!Ev$N`|vI&HMoN8LYwv080k z`-flwECSbMN5W$GWhh-IORCdGTYdC3M%s4S{--_{5ySCa6BdSeS5kHbl(oMGw9!_d zG)5|y8UNIN39-HeUjdKZTj5Ta7*JMco%=vrebN}I+-m#!z5%bnH((8L+~c}cS)Kb_ z8*S6}@^U-v8^blRefGH84?Nao5~#CIn|yn2zZdE}UR|4i4D^u=;4Y}=m)AXr-MR1w z=ng2Wvo3ERx1XnOD*PSn3~NKyepthKpAv1$l>do+JpPoM`FKv+e%59-`Gp{W~=B&)2<*z2x;p+a9lwyb>`?>PM8ez){o z@SO16FrMd1YM+;HqkbAV?~Ag!yj{_@=lUp!IeZM+Il;NX`NO%Qp09f6uB+f{aCbmi zonu?%o5pK*{&QR$r?bI$&c)vh@>!I-K(n#aubuX9!O!dNufV>Z4_5}1)mf*_?;-7P zUT&p*$MYRH7WRTS!FG3mhXcy$tkdSZpij}hXr+Dku`%tF&pM{S&*3?k8&Fnfoi^I) zqi??NZkK7_+#iD9!4Ys1c%AQoi{TzPHK44{I&HMoM_*&)$DvI7Z{YVuFs3iVuf|@=cBjrQW7$TG|Givn&$FMwi$U+&>LPHS66Ltw(nj0-JYdYC z_=i!~1y_UdCcwt%9i;F5r{I_tEF_GwurzV@%ft}qrJ4JfO# zE{*B+z-!W5;59t%--D3z*NW8dZt54pkpX3Osg1HRT!ZWY<`Cm6$Lm?%CvLxh&8Gs& zQJ*g_pl&r_<_Pa9CTH(de$c=t#&As_al2mm9QB`sEPkzxs?NH4KCSA0&^>jVY;2PF z_Q}3Aisf2M8rfLqSm(#zed4~Ux1T+}7bLirt#{1Fc8%dN6ZfyjlE;+CRzAM^8{l+^ z$G@^V>tf7&S?2h64dgw|k>GiBRzO*ub=u@(yWK3l{kRRB>z;sFa8*EAopsve`&=ge z<@g;B?yKo=2b>vDR%e|yS3tfGjq>+6a(?_ZoCuD!^UcKpWp&nRqiz1UY83ys*gHQy z1s*q!@nzt-CCcio(?(l;(mtf|r&6~xSD)IpDjUm~@%S7bwr%rY`iU`&ZOqynF;D_a zrAiMgq~;98W+?BihGaCiAL6^C>~E5IJ(RsOs;$F@ExQi0uP_oeYY~0)T_1h{o;Qz& z^08#Qo=#hR^tD~DZypOb!^!X}%mYX#5guNin>w5=XoKHN5?t+iP2f(&sT+4UiUa)OZ zRu^q9MIHyMgZtpy5bM(NI`oq2lplgkKwnW_0(m>H*G1QC+tk+_%*C8ifBk+Hs29UN z+jqynv2aYH|I5g~ggAZ|A*+k^mi6|(dhJ|)>3=~$SzQ{V-u{0P8;=w1e->o7mDMS0 zV>_vT-2Z<-zX+}d`*cGP EKdjmDR{#J2 literal 0 HcmV?d00001 diff --git a/contrib/windows/Inno.Setup/readme.txt b/contrib/windows/Inno.Setup/readme.txt new file mode 100644 index 000000000000..71a5eb395c1a --- /dev/null +++ b/contrib/windows/Inno.Setup/readme.txt @@ -0,0 +1,14 @@ + +** Open ZFS On Windows ** + +This is a beta release of Open ZFS for Windows. + +It is recommended that this software is only used on test VMs, with test POOLS +and test DATA. The Developer(s) can not be held responsible if you lose your +Windows, and/or data using this software. This is the wild west people, +the edge of the frontier. + +Having said that, if you find yourself in a boot loop - when Windows 10 fails to +boot two times in a row, you can enter the Advanced Console, and delete the +OpenZFS.SYS file from C:\Windows\Drivers to boot normally. + diff --git a/contrib/windows/OpenZFS/OpenZFS.sln b/contrib/windows/OpenZFS/OpenZFS.sln new file mode 100644 index 000000000000..4b5ec560e858 --- /dev/null +++ b/contrib/windows/OpenZFS/OpenZFS.sln @@ -0,0 +1,51 @@ + +Microsoft Visual Studio Solution File, Format Version 12.00 +# Visual Studio Version 16 +VisualStudioVersion = 16.0.30621.155 +MinimumVisualStudioVersion = 10.0.40219.1 +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "OpenZFS", "OpenZFS\OpenZFS.vcxproj", "{CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|ARM = Debug|ARM + Debug|ARM64 = Debug|ARM64 + Debug|x64 = Debug|x64 + Debug|x86 = Debug|x86 + Release|ARM = Release|ARM + Release|ARM64 = Release|ARM64 + Release|x64 = Release|x64 + Release|x86 = Release|x86 + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|ARM.ActiveCfg = Debug|ARM + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|ARM.Build.0 = Debug|ARM + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|ARM.Deploy.0 = Debug|ARM + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|ARM64.ActiveCfg = Debug|ARM64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|ARM64.Build.0 = Debug|ARM64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|ARM64.Deploy.0 = Debug|ARM64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|x64.ActiveCfg = Debug|x64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|x64.Build.0 = Debug|x64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|x64.Deploy.0 = Debug|x64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|x86.ActiveCfg = Debug|Win32 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|x86.Build.0 = Debug|Win32 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Debug|x86.Deploy.0 = Debug|Win32 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|ARM.ActiveCfg = Release|ARM + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|ARM.Build.0 = Release|ARM + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|ARM.Deploy.0 = Release|ARM + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|ARM64.ActiveCfg = Release|ARM64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|ARM64.Build.0 = Release|ARM64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|ARM64.Deploy.0 = Release|ARM64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|x64.ActiveCfg = Release|x64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|x64.Build.0 = Release|x64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|x64.Deploy.0 = Release|x64 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|x86.ActiveCfg = Release|Win32 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|x86.Build.0 = Release|Win32 + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE}.Release|x86.Deploy.0 = Release|Win32 + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection + GlobalSection(ExtensibilityGlobals) = postSolution + SolutionGuid = {1DB41F4F-FE9B-4EEB-A025-596E53980599} + EndGlobalSection +EndGlobal diff --git a/contrib/windows/OpenZFS/OpenZFS/OpenZFS.vcxproj b/contrib/windows/OpenZFS/OpenZFS/OpenZFS.vcxproj new file mode 100644 index 000000000000..5761b2a5b09d --- /dev/null +++ b/contrib/windows/OpenZFS/OpenZFS/OpenZFS.vcxproj @@ -0,0 +1,233 @@ + + + + + Debug + Win32 + + + Release + Win32 + + + Debug + x64 + + + Release + x64 + + + Debug + ARM + + + Release + ARM + + + Debug + ARM64 + + + Release + ARM64 + + + + {CE4D8C0F-7C0C-48EA-A286-A3FD540BA4DE} + {1bc93793-694f-48fe-9372-81e2b05556fd} + v4.5 + 12.0 + Debug + Win32 + OpenZFS + + + + Windows7 + true + WindowsKernelModeDriver10.0 + Driver + KMDF + Desktop + + + Windows7 + false + WindowsKernelModeDriver10.0 + Driver + KMDF + Desktop + + + Windows7 + true + WindowsKernelModeDriver10.0 + Driver + KMDF + Desktop + + + Windows7 + false + WindowsKernelModeDriver10.0 + Driver + KMDF + Desktop + + + Windows10 + true + WindowsKernelModeDriver10.0 + Driver + KMDF + Universal + + + Windows10 + false + WindowsKernelModeDriver10.0 + Driver + KMDF + Universal + + + Windows10 + true + WindowsKernelModeDriver10.0 + Driver + KMDF + Universal + + + Windows10 + false + WindowsKernelModeDriver10.0 + Driver + KMDF + Universal + + + + + + + + + + + DbgengKernelDebugger + true + true + + + DbgengKernelDebugger + true + true + + + DbgengKernelDebugger + true + false + + + $(IncludePath);$(KMDF_INC_PATH)$(KMDF_VER_PATH);../../../../include + + + DbgengKernelDebugger + true + false + + + $(IncludePath);$(KMDF_INC_PATH)$(KMDF_VER_PATH);../../../../include + + + DbgengKernelDebugger + + + DbgengKernelDebugger + + + DbgengKernelDebugger + + + DbgengKernelDebugger + + + + $(SolutionDir)..\..\..\include\os\windows\spl;$(SolutionDir)..\..\..\include\os\windows\zfs;$(SolutionDir)..\..\..\include\os\windows;%(AdditionalIncludeDirectories) + _KERNEL;_KERNEL_;_X86_=1;i386=1;STD_CALL;%(PreprocessorDefinitions) + Level3 + false + + + wdmsec.lib;Storport.lib;scsiwmi.lib;splkern.lib;icpkern.lib;luakern.lib;nvpairkern.lib;unicodekern.lib;zcommonkern.lib;zlibkern.lib;zstdkern.lib;zfskern.lib;zfskern_os.lib;%(AdditionalDependencies);$(KernelBufferOverflowLib);$(DDK_LIB_PATH)ntoskrnl.lib;$(DDK_LIB_PATH)hal.lib;$(DDK_LIB_PATH)wmilib.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfLdr.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfDriverEntry.lib + $(SolutionDir)..\..\..\out\build\x64-Debug\module\os\windows\spl;$(SolutionDir)..\..\..\out\build\x64-Debug\module\os\windows\zfs;$(SolutionDir)..\..\..\out\build\x64-Debug\module\icp;$(SolutionDir)..\..\..\out\build\x64-Debug\module\lua;$(SolutionDir)..\..\..\out\build\x64-Debug\module\nvpair;$(SolutionDir)..\..\..\out\build\x64-Debug\module\unicode;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zcommon;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zfs;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zstd;$(SolutionDir)..\..\..\out\build\x64-Debug\lib\os\windows\zlib-1.2.3;%(AdditionalLibraryDirectories) + + + /tr http://timestamp.digicert.com /td sha256 /fd sha256 %(AdditionalOptions) + + + + + $(SolutionDir)..\..\..\include\os\windows\spl;$(SolutionDir)..\..\..\include\os\windows\zfs;$(SolutionDir)..\..\..\include\os\windows;%(AdditionalIncludeDirectories) + _KERNEL;_KERNEL_;_X86_=1;i386=1;STD_CALL;%(PreprocessorDefinitions) + Level3 + false + + + wdmsec.lib;Storport.lib;scsiwmi.lib;splkern.lib;icpkern.lib;luakern.lib;nvpairkern.lib;unicodekern.lib;zcommonkern.lib;zlibkern.lib;zstdkern.lib;zfskern.lib;zfskern_os.lib;%(AdditionalDependencies);$(KernelBufferOverflowLib);$(DDK_LIB_PATH)ntoskrnl.lib;$(DDK_LIB_PATH)hal.lib;$(DDK_LIB_PATH)wmilib.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfLdr.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfDriverEntry.lib + $(SolutionDir)..\..\..\out\build\x64-Debug\module\os\windows\spl;$(SolutionDir)..\..\..\out\build\x64-Debug\module\os\windows\zfs;$(SolutionDir)..\..\..\out\build\x64-Debug\module\icp;$(SolutionDir)..\..\..\out\build\x64-Debug\module\lua;$(SolutionDir)..\..\..\out\build\x64-Debug\module\nvpair;$(SolutionDir)..\..\..\out\build\x64-Debug\module\unicode;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zcommon;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zfs;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zstd;$(SolutionDir)..\..\..\out\build\x64-Debug\lib\os\windows\zlib-1.2.3;%(AdditionalLibraryDirectories) + + + /tr http://timestamp.digicert.com /td sha256 /fd sha256 %(AdditionalOptions) + + + + + $(SolutionDir)..\..\..\include\os\windows\spl;$(SolutionDir)..\..\..\include\os\windows\zfs;$(SolutionDir)..\..\..\include\os\windows;%(AdditionalIncludeDirectories) + _KERNEL;_KERNEL_;_WIN64;_AMD64_;AMD64;%(PreprocessorDefinitions) + Level3 + false + + + $(SolutionDir)..\..\..\out\build\x64-Debug\module\os\windows\spl;$(SolutionDir)..\..\..\out\build\x64-Debug\module\os\windows\zfs;$(SolutionDir)..\..\..\out\build\x64-Debug\module\icp;$(SolutionDir)..\..\..\out\build\x64-Debug\module\lua;$(SolutionDir)..\..\..\out\build\x64-Debug\module\nvpair;$(SolutionDir)..\..\..\out\build\x64-Debug\module\unicode;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zcommon;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zfs;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zstd;$(SolutionDir)..\..\..\out\build\x64-Debug\lib\os\windows\zlib-1.2.3;%(AdditionalLibraryDirectories) + wdmsec.lib;Storport.lib;scsiwmi.lib;splkern.lib;icpkern.lib;luakern.lib;nvpairkern.lib;unicodekern.lib;zcommonkern.lib;zlibkern.lib;zstdkern.lib;zfskern.lib;zfskern_os.lib;%(AdditionalDependencies);$(KernelBufferOverflowLib);$(DDK_LIB_PATH)ntoskrnl.lib;$(DDK_LIB_PATH)hal.lib;$(DDK_LIB_PATH)wmilib.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfLdr.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfDriverEntry.lib + false + + + /tr http://timestamp.digicert.com /td sha256 /fd sha256 %(AdditionalOptions) + + + + + $(SolutionDir)..\..\..\include\os\windows\spl;$(SolutionDir)..\..\..\include\os\windows\zfs;$(SolutionDir)..\..\..\include\os\windows;%(AdditionalIncludeDirectories) + _KERNEL;_KERNEL_;_WIN64;_AMD64_;AMD64;%(PreprocessorDefinitions) + Level3 + false + + + $(SolutionDir)..\..\..\out\build\x64-Debug\module\os\windows\spl;$(SolutionDir)..\..\..\out\build\x64-Debug\module\os\windows\zfs;$(SolutionDir)..\..\..\out\build\x64-Debug\module\icp;$(SolutionDir)..\..\..\out\build\x64-Debug\module\lua;$(SolutionDir)..\..\..\out\build\x64-Debug\module\nvpair;$(SolutionDir)..\..\..\out\build\x64-Debug\module\unicode;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zcommon;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zfs;$(SolutionDir)..\..\..\out\build\x64-Debug\module\zstd;$(SolutionDir)..\..\..\out\build\x64-Debug\lib\os\windows\zlib-1.2.3;%(AdditionalLibraryDirectories) + wdmsec.lib;Storport.lib;scsiwmi.lib;splkern.lib;icpkern.lib;luakern.lib;nvpairkern.lib;unicodekern.lib;zcommonkern.lib;zlibkern.lib;zstdkern.lib;zfskern.lib;zfskern_os.lib;%(AdditionalDependencies);$(KernelBufferOverflowLib);$(DDK_LIB_PATH)ntoskrnl.lib;$(DDK_LIB_PATH)hal.lib;$(DDK_LIB_PATH)wmilib.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfLdr.lib;$(KMDF_LIB_PATH)$(KMDF_VER_PATH)\WdfDriverEntry.lib + false + + + /tr http://timestamp.digicert.com /td sha256 /fd sha256 %(AdditionalOptions) + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/contrib/windows/TestCert/README.md b/contrib/windows/TestCert/README.md new file mode 100644 index 000000000000..64fbf0e7dd14 --- /dev/null +++ b/contrib/windows/TestCert/README.md @@ -0,0 +1,15 @@ +### Development +- Install OpenZFS test certificate + - Install `test_sign_cert_nopass.pfx` (password: ) + - Certificate should be installed into + 1. "Personal" in "Current User" + +### Target +- Install OpenZFS test certificate + - Install `test_sign_cert_nopass.pfx` (password: ) + - Certificate should be installed into + 1. "Trusted Root Certification Authority" in "Local Computer" (not current user) *and* + 2. "Trusted Publishers" in "Local Computer" (not current user) +- Enable test signing + - `> bcdedit.exe /set TESTSIGNING ON` + - reboot the system to apply diff --git a/contrib/windows/TestCert/create_test_sign_cert.ps1 b/contrib/windows/TestCert/create_test_sign_cert.ps1 new file mode 100644 index 000000000000..87ddb232f40a --- /dev/null +++ b/contrib/windows/TestCert/create_test_sign_cert.ps1 @@ -0,0 +1,28 @@ +#cert stores +#cert:\localmachine\my +#Cert:\CurrentUser\My + +#config +$plaintextpwd = 'password1234' +$subject = "OpenZFS Test Signing Certificate" +$filename = 'test_sign_cert' +#$dirname = 'c:\' +$dirname = '' +$yearsvalid = 3 + +#generate +$date_now = Get-Date +$extended_date = $date_now.AddYears($yearsvalid) +$cert = New-SelfSignedCertificate -CertStoreLocation Cert:\CurrentUser\My -Type CodeSigningCert -Subject $subject -notafter $extended_date + +#export with password +$filepathpass = $dirname + $filename + '_pass.pfx' +$pwd = ConvertTo-SecureString -String $plaintextpwd -Force -AsPlainText +$path = 'cert:\CurrentUser\My\' + $cert.thumbprint +Export-PfxCertificate -cert $path -FilePath $filepathpass -Password $pwd + +#export "without" password +$filepathnopass = $dirname + $filename + '_nopass.pfx' +$passin = 'pass:' + $plaintextpwd +&"C:\Program Files\OpenSSL-Win64\bin\openssl.exe" pkcs12 -in $filepathpass -nodes -noenc -passin $passin | &"C:\Program Files\OpenSSL-Win64\bin\openssl.exe" pkcs12 -export -keypbe NONE -certpbe NONE -noenc -nomaciter -noiter -nomac -passout pass: -out $filepathnopass + diff --git a/contrib/windows/TestCert/test_sign_cert_nopass.pfx b/contrib/windows/TestCert/test_sign_cert_nopass.pfx new file mode 100644 index 0000000000000000000000000000000000000000..9f2bd19af368da1d87b8766645ed33452c9170b2 GIT binary patch literal 2259 zcmeHHX*8T^7Ot-ribA2ZeJN|j{vl!sC1UMF&?pHZNLivpwVk1(`8pBA8oNkh-_6*A zSd$P-Y#kAM5F(l&Vi_uq{+#K_$^4u7Kj%I7$Mf9x-uIs8-ur?Ob{GkuKnUB*#3C1E z8nw?3Fax;|wiyj!8(_KygivY!;X*~BAru7D;jo1=;r@zZV+ByT5b8Wkl9^clB3XFh zTqq?tKn~_*KnTf^i$VfOB&Ud)tEA{@w|Ya9pRCfTPOTvOw?_aR1_8>e$buv$6a|f9 zBU<>l(d~_`i8gNj0Yqz054tDagJ|I97vSmcNp%Tu14a4RRaHR^_^5&^YMS;iRR0%L zIf4IMG6<0Pz8C<5KtTW=0YU&P5&{6kNcZZCQB6_37P&2>?xe#Z{9ape`lqar#u(Sx z;de8lEv&?bm3D$d&!}=&-z`^MtcGC;{)E6<AoQUi_W~0E}Xe|kq z87qgpuH#x|Y>v^rhxIb0EM{DU$+eZF<44zOSLl7QRwRG<_}1^m;z$&L01AvjLy!aRj|d(Bj?riok~sn-oM34*8_3Fp zm1hDl7-kd-6geR|(7b3Kfd$u`39FBZQ{CaOd%qObwA|sJ{$6Rg?Xc1S0{MD^XfJrW zeUi@+zZc>uxXsn;y}c@7fyOELuQ(?$oF(<{F3Ql76NCRACn|h>a^vV!H6nE{Zfvf( zMmG&0m5$#O=Fqh~=!r~LFnU84Z^9MQ7I?)V>q+^Pa&EbYomBiR9~o@ciJpGj(+ZuP zR@Bjutkjds$lAy`vw%Ns_o;23y84^``WV|rp|;fFdkGd|d1{8(z#!$BVN|hQn$Kr& z+n-F}`0y&mrE~6A_79CDw_QoEv9M<(R!VmEgZ`gq1AM9_`x&p7qqA)cjxORTLuBjB z>h~3Omr9IF;;51qHvN@XHH!RAh4dehOEcbP6HUBg7}?puVw>ThiZm$sZ5dd8P=Tde z^=e`r!cp#P^=6;Hn%fXhNjjyAYbQI4!Jr`w07B>^*bcs5OxTI;g2^@{fcRc< zc(&0Hx&-zVz@Gn4_rIqbRs!K6!M+|A>-sZxw7}ifXfWwCPwDQh=Q)~Etu{R^MLb=O z`~r>nD+LTDgnhdfM8aS%3sHpm;w+fa$&3*v+P#EvlPEiu#G4UAJS>9~{^)oMi8D#9 z2Ii;_noFvTj?@l;*n>imEWg~$Y)+nl{sE!*_JNne>SiEfTxq!|$?C%6ynEgV3fYv? zQk@u5)806U*%6;e^tc;M$_EbPc}NP@gQ~^bTZhh74HC?zH$Jz{#OJea)p)m;={yk{ z);zD1K4^bS3vzTYe3fC|bP%3Wk=RnEW@ffpHjVDJvut=C+%=fQ+E2RMc=q zGv_tjRfj#Vo1=%f9g&fdhzaW{Tf*eH(r+_cw9?{fJ=>se?oXZsDQ@EEL6sF6-;*vGq$KIxpe zmb*YW$WD9rFj+0>e!y%79#LdNmFErwZ=6jqTFnq*ZF;kEJhGbNHKoZnFOw-Xy||bv z=(IwgG`PJP!qs=>8pqC?r2aCVSdX5|&ZquJoqy$}kzUlohLrk|Um=2`|hN&HFv(pMqX z>*8UXM)L~#Z=tA>-B({zHJTR`SAUT|E-o%LAWTg?wtxpzF|L5Dr9SwPE z$1pqFbkUu?7%u6B(LMuRTZC3^ComD-;FdL?+#WjoRMEoc4SUNu+xFrF zdu1Jp(?*D0psJ?9dW@kU5owV1W9A?0^$jevyFHb^#Nf>6E;W8-#c3K5FaOJb@lWi* B(B}XE literal 0 HcmV?d00001 diff --git a/contrib/windows/TestCert/test_sign_cert_pass.pfx b/contrib/windows/TestCert/test_sign_cert_pass.pfx new file mode 100644 index 0000000000000000000000000000000000000000..e789e3b376a6e3dc221612dfed8d2a517a91703b GIT binary patch literal 2646 zcmZXUXIN9&8ikV(5_*v$mjFU&QiMZG0FfrWBTYKer4x-25R?dV1rZdKjx?!40OT0HYRtrt|E89t;O( z;i)g6cL4c<3>}1*7>dCWkXX zsTkFDYeS3r_#hDY02B`$XS5boUQwzG=FMV12zr>YCbCCzRDUT{IjaqyqU-d*FlEYj5_!~| z^*3zPrmkQJft{Rv-t;*3k1=DPQujy^TqiU{Y{#_Ydbq_C8WCn_)Z$grY@K>mzFlvS zm`Z8{?D7)j*3h@39q+1(pieY!5Q5a!nNB~lukmA$5-jLa89ruXrjRnGNUsH+4-C!} z>K-w3p*IfC`8m4!h?u2|i@CRW3)S-4nXrfv{icT6z9;BBE5FH^MGE>9xfi zD9BcF7k&MSIUTDST;APF_R+EWoi>Wj#)T!d2LyzvDd z{pNzq;7h5i_s0d$+2m@6 z>_6huD=I!J&xh$E4p6P}!UPl3?hEW*IMqb2mTJ&uthQRhbxlU!fZ?;-kNv;(gaoKI zsnrgb!r8TQg;uKSu|{YQPdc64Hw+z0AmJMJzCoziV($2naM-9(Wwqy46Xjho=$`4e zvi3#DBuT8H#I($*JkhHyG6toN>&|iniUO+qAJ^oyI7yvTlfVwq?clIRhSoEBYn-%? zk}vnt!#wQ7$TPRc*p1FGj(SIotCKj7WKIQ|wpB=D;?V6ZzVy6(ZOlZrJ^ial&n2us zDL71lJ7Gp7zzZSMbqG4}Akbv`Ydt@A$6cF?DJXCk-#FE;bnv#?m@@DhCZ^m}ca5@t zFs6#OZIPnesnUMth=~e$E&}*z*fnPD@qdoyvyH6IJpbd^{0FjaJ3eaF|3W!CYQm*K zG)`E7S=rVir6*V>0-rON4DoI`%IEcI8QQfeO}P0ch9s#0v!u?(H_QV^>`Eu%9(Utj=c%xsoJr9tUa#|i%%H4Y@=|u zMcEIPCB3Ms94%;}Szso~?%fL{J{Of*qc2;p;H#0zu$M!4UTtWZy#nnruybxXM31T9 zg>LWiBONr8aDz#u8cCeJWBRkC?XtI5B2o!A?vpfxd2G{~1`Aa7Gbv&dkN$ij1(rJ$og9S;nIIaGDF^_&-<-(G8 zab8_jU00riZU9@o;6N_jz!R7DY7c&N|sRY&E3K=z< z+jcZs)K!gl^UxX%2d~Yglxi5dfc-y^Rmz0N2y`Fh2D%0UKxmLVS<)a`kPFC}OweQw zkn3x|2!_@Ce@3bA|Tu~^?1b9G-ExW?WskL{Z!L|OO6 zCwlF}+bC>`-i;bR6Ta%~;bgF@)ax$^FzHOLa~jkm!=iEW<>WQy8{#4O#8#`~~g>;pP)t5y@_ ztR?=K^G)2G z5$=sfii*ghz#7+suBUtRem}ya_dg#JnaFP z7Mr|ZIcV91R(5Kg4C}Ki_YW-l+={Q9g5)6X&cQRH-{u$3vsN&JBpjb@Vf7|Asr5_FSY+QI`n)g<>Y83+)M8G4g{R4#yB3~I)FK2UAOcj)f@<}MPf{k_JBo8SXiQI$h ztkE5g2jA!9(x`s7h(eX+^32Pm5g4z2k7{DJOkU>yJcG3hFOUdkzQ3?GVI_L%Wi@2Y zCvW$|Vg=+8?GzZC?4pf*(3Q;)r(Kbj zxdxX_Lef2_6u6(N@r@L$EZ;Z1_9EQ`i81OCZmkUZDQ|?p)y}%9Xx<;48;K;yqkrc~ z1ve&HTCxY8A%{fwnii3D_uMLTl39+=l@E{i0Npfu(r6GDN;bGNaBYv z)*4l@ylQLk`XaZf+OVoM=$RI5!cjK)g#g}$sY~Vp*MTbl97ZX|Kmp-|vi5eoJ)Sj0 reBek7a6Jh2G%Vu8tUy`s$uZ>b>#h>~7a)@(Yb-B7n!WHpfAs$V%8IZR literal 0 HcmV?d00001 diff --git a/contrib/windows/cmake/FindOpenSSL.cmake b/contrib/windows/cmake/FindOpenSSL.cmake new file mode 100644 index 000000000000..b1afa5f40122 --- /dev/null +++ b/contrib/windows/cmake/FindOpenSSL.cmake @@ -0,0 +1,675 @@ +# Distributed under the OSI-approved BSD 3-Clause License. See accompanying +# file Copyright.txt or https://cmake.org/licensing for details. + +#[=======================================================================[.rst: +FindOpenSSL +----------- + +Find the OpenSSL encryption library. + +This module finds an installed OpenSSL library and determines its version. + +.. versionadded:: 3.19 + When a version is requested, it can be specified as a simple value or as a + range. For a detailed description of version range usage and capabilities, + refer to the :command:`find_package` command. + +.. versionadded:: 3.18 + Support for OpenSSL 3.0. + +Optional COMPONENTS +^^^^^^^^^^^^^^^^^^^ + +.. versionadded:: 3.12 + +This module supports two optional COMPONENTS: ``Crypto`` and ``SSL``. Both +components have associated imported targets, as described below. + +Imported Targets +^^^^^^^^^^^^^^^^ + +.. versionadded:: 3.4 + +This module defines the following :prop_tgt:`IMPORTED` targets: + +``OpenSSL::SSL`` + The OpenSSL ``ssl`` library, if found. +``OpenSSL::Crypto`` + The OpenSSL ``crypto`` library, if found. +``OpenSSL::applink`` + .. versionadded:: 3.18 + + The OpenSSL ``applink`` components that might be need to be compiled into + projects under MSVC. This target is available only if found OpenSSL version + is not less than 0.9.8. By linking this target the above OpenSSL targets can + be linked even if the project has different MSVC runtime configurations with + the above OpenSSL targets. This target has no effect on platforms other than + MSVC. + +NOTE: Due to how ``INTERFACE_SOURCES`` are consumed by the consuming target, +unless you certainly know what you are doing, it is always preferred to link +``OpenSSL::applink`` target as ``PRIVATE`` and to make sure that this target is +linked at most once for the whole dependency graph of any library or +executable: + +.. code-block:: cmake + + target_link_libraries(myTarget PRIVATE OpenSSL::applink) + +Otherwise you would probably encounter unexpected random problems when building +and linking, as both the ISO C and the ISO C++ standard claims almost nothing +about what a link process should be. + +Result Variables +^^^^^^^^^^^^^^^^ + +This module will set the following variables in your project: + +``OPENSSL_FOUND`` + System has the OpenSSL library. If no components are requested it only + requires the crypto library. +``OPENSSL_INCLUDE_DIR`` + The OpenSSL include directory. +``OPENSSL_CRYPTO_LIBRARY`` + The OpenSSL crypto library. +``OPENSSL_CRYPTO_LIBRARIES`` + The OpenSSL crypto library and its dependencies. +``OPENSSL_SSL_LIBRARY`` + The OpenSSL SSL library. +``OPENSSL_SSL_LIBRARIES`` + The OpenSSL SSL library and its dependencies. +``OPENSSL_LIBRARIES`` + All OpenSSL libraries and their dependencies. +``OPENSSL_VERSION`` + This is set to ``$major.$minor.$revision$patch`` (e.g. ``0.9.8s``). +``OPENSSL_APPLINK_SOURCE`` + The sources in the target ``OpenSSL::applink`` that is mentioned above. This + variable shall always be undefined if found openssl version is less than + 0.9.8 or if platform is not MSVC. + +Hints +^^^^^ + +Set ``OPENSSL_ROOT_DIR`` to the root directory of an OpenSSL installation. + +.. versionadded:: 3.4 + Set ``OPENSSL_USE_STATIC_LIBS`` to ``TRUE`` to look for static libraries. + +.. versionadded:: 3.5 + Set ``OPENSSL_MSVC_STATIC_RT`` set ``TRUE`` to choose the MT version of the lib. +#]=======================================================================] + +macro(_OpenSSL_test_and_find_dependencies ssl_library crypto_library) + if((CMAKE_SYSTEM_NAME STREQUAL "Linux") AND + (("${ssl_library}" MATCHES "\\${CMAKE_STATIC_LIBRARY_SUFFIX}$") OR + ("${crypto_library}" MATCHES "\\${CMAKE_STATIC_LIBRARY_SUFFIX}$"))) + set(_OpenSSL_has_dependencies TRUE) + find_package(Threads) + else() + set(_OpenSSL_has_dependencies FALSE) + endif() +endmacro() + +function(_OpenSSL_add_dependencies libraries_var) + if(CMAKE_THREAD_LIBS_INIT) + list(APPEND ${libraries_var} ${CMAKE_THREAD_LIBS_INIT}) + endif() + list(APPEND ${libraries_var} ${CMAKE_DL_LIBS}) + set(${libraries_var} ${${libraries_var}} PARENT_SCOPE) +endfunction() + +function(_OpenSSL_target_add_dependencies target) + if(_OpenSSL_has_dependencies) + set_property( TARGET ${target} APPEND PROPERTY INTERFACE_LINK_LIBRARIES Threads::Threads ) + set_property( TARGET ${target} APPEND PROPERTY INTERFACE_LINK_LIBRARIES ${CMAKE_DL_LIBS} ) + endif() +endfunction() + +if (UNIX) + find_package(PkgConfig QUIET) + pkg_check_modules(_OPENSSL QUIET openssl) +endif () + +# Support preference of static libs by adjusting CMAKE_FIND_LIBRARY_SUFFIXES +if(OPENSSL_USE_STATIC_LIBS) + set(_openssl_ORIG_CMAKE_FIND_LIBRARY_SUFFIXES ${CMAKE_FIND_LIBRARY_SUFFIXES}) + if(WIN32) + set(CMAKE_FIND_LIBRARY_SUFFIXES .lib .a ${CMAKE_FIND_LIBRARY_SUFFIXES}) + else() + set(CMAKE_FIND_LIBRARY_SUFFIXES .a ) + endif() +endif() + +if (WIN32) + # http://www.slproweb.com/products/Win32OpenSSL.html + set(_OPENSSL_ROOT_HINTS + ${OPENSSL_ROOT_DIR} + "[HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\OpenSSL (32-bit)_is1;Inno Setup: App Path]" + "[HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Uninstall\\OpenSSL (64-bit)_is1;Inno Setup: App Path]" + ENV OPENSSL_ROOT_DIR + ) + + if("${CMAKE_SIZEOF_VOID_P}" STREQUAL "8") + set(_arch "Win64") + file(TO_CMAKE_PATH "$ENV{PROGRAMFILES}" _programfiles) + else() + set(_arch "Win32") + set(_progfiles_x86 "ProgramFiles(x86)") + if(NOT "$ENV{${_progfiles_x86}}" STREQUAL "") + # under windows 64 bit machine + file(TO_CMAKE_PATH "$ENV{${_progfiles_x86}}" _programfiles) + else() + # under windows 32 bit machine + file(TO_CMAKE_PATH "$ENV{ProgramFiles}" _programfiles) + endif() + endif() + + set(_OPENSSL_ROOT_PATHS + "${_programfiles}/OpenSSL" + "${_programfiles}/OpenSSL-${_arch}" + "C:/OpenSSL/" + "C:/OpenSSL-${_arch}/" + ) + unset(_programfiles) + unset(_arch) +else () + set(_OPENSSL_ROOT_HINTS + ${OPENSSL_ROOT_DIR} + ENV OPENSSL_ROOT_DIR + ) +endif () + +set(_OPENSSL_ROOT_HINTS_AND_PATHS + HINTS ${_OPENSSL_ROOT_HINTS} + PATHS ${_OPENSSL_ROOT_PATHS} + ) + +find_path(OPENSSL_INCLUDE_DIR + NAMES + openssl/ssl.h + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + HINTS + ${_OPENSSL_INCLUDEDIR} + ${_OPENSSL_INCLUDE_DIRS} + PATH_SUFFIXES + include +) + +if(WIN32 AND NOT CYGWIN) + if(MSVC) + # /MD and /MDd are the standard values - if someone wants to use + # others, the libnames have to change here too + # use also ssl and ssleay32 in debug as fallback for openssl < 0.9.8b + # enable OPENSSL_MSVC_STATIC_RT to get the libs build /MT (Multithreaded no-DLL) + # In Visual C++ naming convention each of these four kinds of Windows libraries has it's standard suffix: + # * MD for dynamic-release + # * MDd for dynamic-debug + # * MT for static-release + # * MTd for static-debug + + # Implementation details: + # We are using the libraries located in the VC subdir instead of the parent directory even though : + # libeay32MD.lib is identical to ../libeay32.lib, and + # ssleay32MD.lib is identical to ../ssleay32.lib + # enable OPENSSL_USE_STATIC_LIBS to use the static libs located in lib/VC/static + + if (OPENSSL_MSVC_STATIC_RT) + set(_OPENSSL_MSVC_RT_MODE "MT") + else () + set(_OPENSSL_MSVC_RT_MODE "MD") + endif () + + # Since OpenSSL 1.1, lib names are like libcrypto32MTd.lib and libssl32MTd.lib + if( "${CMAKE_SIZEOF_VOID_P}" STREQUAL "8" ) + set(_OPENSSL_MSVC_ARCH_SUFFIX "64") + else() + set(_OPENSSL_MSVC_ARCH_SUFFIX "32") + endif() + + if(OPENSSL_USE_STATIC_LIBS) + set(_OPENSSL_STATIC_SUFFIX + "_static" + ) + set(_OPENSSL_PATH_SUFFIXES + "lib/VC/static" + "VC/static" + "lib" + ) + else() + set(_OPENSSL_STATIC_SUFFIX + "" + ) + set(_OPENSSL_PATH_SUFFIXES + "lib/VC" + "VC" + "lib" + ) + endif () + + find_library(LIB_EAY_DEBUG + NAMES + # When OpenSSL is built with default options, the static library name is suffixed with "_static". + # Looking the "libcrypto_static.lib" with a higher priority than "libcrypto.lib" which is the + # import library of "libcrypto.dll". + libcrypto${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libcrypto${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libcrypto${_OPENSSL_STATIC_SUFFIX}d + libeay32${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libeay32${_OPENSSL_STATIC_SUFFIX}d + crypto${_OPENSSL_STATIC_SUFFIX}d + # When OpenSSL is built with the "-static" option, only the static build is produced, + # and it is not suffixed with "_static". + libcrypto${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libcrypto${_OPENSSL_MSVC_RT_MODE}d + libcryptod + libeay32${_OPENSSL_MSVC_RT_MODE}d + libeay32d + cryptod + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + PATH_SUFFIXES + ${_OPENSSL_PATH_SUFFIXES} + ) + + find_library(LIB_EAY_RELEASE + NAMES + # When OpenSSL is built with default options, the static library name is suffixed with "_static". + # Looking the "libcrypto_static.lib" with a higher priority than "libcrypto.lib" which is the + # import library of "libcrypto.dll". + libcrypto${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libcrypto${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libcrypto${_OPENSSL_STATIC_SUFFIX} + libeay32${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libeay32${_OPENSSL_STATIC_SUFFIX} + crypto${_OPENSSL_STATIC_SUFFIX} + # When OpenSSL is built with the "-static" option, only the static build is produced, + # and it is not suffixed with "_static". + libcrypto${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libcrypto${_OPENSSL_MSVC_RT_MODE} + libcrypto + libeay32${_OPENSSL_MSVC_RT_MODE} + libeay32 + crypto + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + PATH_SUFFIXES + ${_OPENSSL_PATH_SUFFIXES} + ) + + find_library(SSL_EAY_DEBUG + NAMES + # When OpenSSL is built with default options, the static library name is suffixed with "_static". + # Looking the "libssl_static.lib" with a higher priority than "libssl.lib" which is the + # import library of "libssl.dll". + libssl${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libssl${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libssl${_OPENSSL_STATIC_SUFFIX}d + ssleay32${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + ssleay32${_OPENSSL_STATIC_SUFFIX}d + ssl${_OPENSSL_STATIC_SUFFIX}d + # When OpenSSL is built with the "-static" option, only the static build is produced, + # and it is not suffixed with "_static". + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE}d + libssl${_OPENSSL_MSVC_RT_MODE}d + libssld + ssleay32${_OPENSSL_MSVC_RT_MODE}d + ssleay32d + ssld + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + PATH_SUFFIXES + ${_OPENSSL_PATH_SUFFIXES} + ) + + find_library(SSL_EAY_RELEASE + NAMES + # When OpenSSL is built with default options, the static library name is suffixed with "_static". + # Looking the "libssl_static.lib" with a higher priority than "libssl.lib" which is the + # import library of "libssl.dll". + libssl${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libssl${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libssl${_OPENSSL_STATIC_SUFFIX} + ssleay32${_OPENSSL_STATIC_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + ssleay32${_OPENSSL_STATIC_SUFFIX} + ssl${_OPENSSL_STATIC_SUFFIX} + # When OpenSSL is built with the "-static" option, only the static build is produced, + # and it is not suffixed with "_static". + libssl${_OPENSSL_MSVC_ARCH_SUFFIX}${_OPENSSL_MSVC_RT_MODE} + libssl${_OPENSSL_MSVC_RT_MODE} + libssl + ssleay32${_OPENSSL_MSVC_RT_MODE} + ssleay32 + ssl + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + PATH_SUFFIXES + ${_OPENSSL_PATH_SUFFIXES} + ) + + set(LIB_EAY_LIBRARY_DEBUG "${LIB_EAY_DEBUG}") + set(LIB_EAY_LIBRARY_RELEASE "${LIB_EAY_RELEASE}") + set(SSL_EAY_LIBRARY_DEBUG "${SSL_EAY_DEBUG}") + set(SSL_EAY_LIBRARY_RELEASE "${SSL_EAY_RELEASE}") + + include(${CMAKE_CURRENT_LIST_DIR}/SelectLibraryConfigurations.cmake) + select_library_configurations(LIB_EAY) + select_library_configurations(SSL_EAY) + + mark_as_advanced(LIB_EAY_LIBRARY_DEBUG LIB_EAY_LIBRARY_RELEASE + SSL_EAY_LIBRARY_DEBUG SSL_EAY_LIBRARY_RELEASE) + set(OPENSSL_SSL_LIBRARY ${SSL_EAY_LIBRARY} ) + set(OPENSSL_CRYPTO_LIBRARY ${LIB_EAY_LIBRARY} ) + elseif(MINGW) + # same player, for MinGW + set(LIB_EAY_NAMES crypto libeay32) + set(SSL_EAY_NAMES ssl ssleay32) + find_library(LIB_EAY + NAMES + ${LIB_EAY_NAMES} + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + PATH_SUFFIXES + "lib/MinGW" + "lib" + ) + + find_library(SSL_EAY + NAMES + ${SSL_EAY_NAMES} + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + PATH_SUFFIXES + "lib/MinGW" + "lib" + ) + + mark_as_advanced(SSL_EAY LIB_EAY) + set(OPENSSL_SSL_LIBRARY ${SSL_EAY} ) + set(OPENSSL_CRYPTO_LIBRARY ${LIB_EAY} ) + unset(LIB_EAY_NAMES) + unset(SSL_EAY_NAMES) + else() + # Not sure what to pick for -say- intel, let's use the toplevel ones and hope someone report issues: + find_library(LIB_EAY + NAMES + libcrypto + libeay32 + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + HINTS + ${_OPENSSL_LIBDIR} + PATH_SUFFIXES + lib + ) + + find_library(SSL_EAY + NAMES + libssl + ssleay32 + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + HINTS + ${_OPENSSL_LIBDIR} + PATH_SUFFIXES + lib + ) + + mark_as_advanced(SSL_EAY LIB_EAY) + set(OPENSSL_SSL_LIBRARY ${SSL_EAY} ) + set(OPENSSL_CRYPTO_LIBRARY ${LIB_EAY} ) + endif() +else() + + find_library(OPENSSL_SSL_LIBRARY + NAMES + ssl + ssleay32 + ssleay32MD + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + HINTS + ${_OPENSSL_LIBDIR} + ${_OPENSSL_LIBRARY_DIRS} + PATH_SUFFIXES + lib + ) + + find_library(OPENSSL_CRYPTO_LIBRARY + NAMES + crypto + NAMES_PER_DIR + ${_OPENSSL_ROOT_HINTS_AND_PATHS} + HINTS + ${_OPENSSL_LIBDIR} + ${_OPENSSL_LIBRARY_DIRS} + PATH_SUFFIXES + lib + ) + + mark_as_advanced(OPENSSL_CRYPTO_LIBRARY OPENSSL_SSL_LIBRARY) + +endif() + +set(OPENSSL_SSL_LIBRARIES ${OPENSSL_SSL_LIBRARY}) +set(OPENSSL_CRYPTO_LIBRARIES ${OPENSSL_CRYPTO_LIBRARY}) +set(OPENSSL_LIBRARIES ${OPENSSL_SSL_LIBRARIES} ${OPENSSL_CRYPTO_LIBRARIES} ) +_OpenSSL_test_and_find_dependencies("${OPENSSL_SSL_LIBRARY}" "${OPENSSL_CRYPTO_LIBRARY}") +if(_OpenSSL_has_dependencies) + _OpenSSL_add_dependencies( OPENSSL_SSL_LIBRARIES ) + _OpenSSL_add_dependencies( OPENSSL_CRYPTO_LIBRARIES ) + _OpenSSL_add_dependencies( OPENSSL_LIBRARIES ) +endif() + +function(from_hex HEX DEC) + string(TOUPPER "${HEX}" HEX) + set(_res 0) + string(LENGTH "${HEX}" _strlen) + + while (_strlen GREATER 0) + math(EXPR _res "${_res} * 16") + string(SUBSTRING "${HEX}" 0 1 NIBBLE) + string(SUBSTRING "${HEX}" 1 -1 HEX) + if (NIBBLE STREQUAL "A") + math(EXPR _res "${_res} + 10") + elseif (NIBBLE STREQUAL "B") + math(EXPR _res "${_res} + 11") + elseif (NIBBLE STREQUAL "C") + math(EXPR _res "${_res} + 12") + elseif (NIBBLE STREQUAL "D") + math(EXPR _res "${_res} + 13") + elseif (NIBBLE STREQUAL "E") + math(EXPR _res "${_res} + 14") + elseif (NIBBLE STREQUAL "F") + math(EXPR _res "${_res} + 15") + else() + math(EXPR _res "${_res} + ${NIBBLE}") + endif() + + string(LENGTH "${HEX}" _strlen) + endwhile() + + set(${DEC} ${_res} PARENT_SCOPE) +endfunction() + +if(OPENSSL_INCLUDE_DIR AND EXISTS "${OPENSSL_INCLUDE_DIR}/openssl/opensslv.h") + file(STRINGS "${OPENSSL_INCLUDE_DIR}/openssl/opensslv.h" openssl_version_str + REGEX "^#[\t ]*define[\t ]+OPENSSL_VERSION_NUMBER[\t ]+0x([0-9a-fA-F])+.*") + + if(openssl_version_str) + # The version number is encoded as 0xMNNFFPPS: major minor fix patch status + # The status gives if this is a developer or prerelease and is ignored here. + # Major, minor, and fix directly translate into the version numbers shown in + # the string. The patch field translates to the single character suffix that + # indicates the bug fix state, which 00 -> nothing, 01 -> a, 02 -> b and so + # on. + + string(REGEX REPLACE "^.*OPENSSL_VERSION_NUMBER[\t ]+0x([0-9a-fA-F])([0-9a-fA-F][0-9a-fA-F])([0-9a-fA-F][0-9a-fA-F])([0-9a-fA-F][0-9a-fA-F])([0-9a-fA-F]).*$" + "\\1;\\2;\\3;\\4;\\5" OPENSSL_VERSION_LIST "${openssl_version_str}") + list(GET OPENSSL_VERSION_LIST 0 OPENSSL_VERSION_MAJOR) + list(GET OPENSSL_VERSION_LIST 1 OPENSSL_VERSION_MINOR) + from_hex("${OPENSSL_VERSION_MINOR}" OPENSSL_VERSION_MINOR) + list(GET OPENSSL_VERSION_LIST 2 OPENSSL_VERSION_FIX) + from_hex("${OPENSSL_VERSION_FIX}" OPENSSL_VERSION_FIX) + list(GET OPENSSL_VERSION_LIST 3 OPENSSL_VERSION_PATCH) + + if (NOT OPENSSL_VERSION_PATCH STREQUAL "00") + from_hex("${OPENSSL_VERSION_PATCH}" _tmp) + # 96 is the ASCII code of 'a' minus 1 + math(EXPR OPENSSL_VERSION_PATCH_ASCII "${_tmp} + 96") + unset(_tmp) + # Once anyone knows how OpenSSL would call the patch versions beyond 'z' + # this should be updated to handle that, too. This has not happened yet + # so it is simply ignored here for now. + string(ASCII "${OPENSSL_VERSION_PATCH_ASCII}" OPENSSL_VERSION_PATCH_STRING) + endif () + + set(OPENSSL_VERSION "${OPENSSL_VERSION_MAJOR}.${OPENSSL_VERSION_MINOR}.${OPENSSL_VERSION_FIX}${OPENSSL_VERSION_PATCH_STRING}") + else () + # Since OpenSSL 3.0.0, the new version format is MAJOR.MINOR.PATCH and + # a new OPENSSL_VERSION_STR macro contains exactly that + file(STRINGS "${OPENSSL_INCLUDE_DIR}/openssl/opensslv.h" OPENSSL_VERSION_STR + REGEX "^#[\t ]*define[\t ]+OPENSSL_VERSION_STR[\t ]+\"([0-9])+\\.([0-9])+\\.([0-9])+\".*") + string(REGEX REPLACE "^.*OPENSSL_VERSION_STR[\t ]+\"([0-9]+\\.[0-9]+\\.[0-9]+)\".*$" + "\\1" OPENSSL_VERSION_STR "${OPENSSL_VERSION_STR}") + + set(OPENSSL_VERSION "${OPENSSL_VERSION_STR}") + + unset(OPENSSL_VERSION_STR) + endif () +endif () + +foreach(_comp IN LISTS OpenSSL_FIND_COMPONENTS) + if(_comp STREQUAL "Crypto") + if(EXISTS "${OPENSSL_INCLUDE_DIR}" AND + (EXISTS "${OPENSSL_CRYPTO_LIBRARY}" OR + EXISTS "${LIB_EAY_LIBRARY_DEBUG}" OR + EXISTS "${LIB_EAY_LIBRARY_RELEASE}") + ) + set(OpenSSL_${_comp}_FOUND TRUE) + else() + set(OpenSSL_${_comp}_FOUND FALSE) + endif() + elseif(_comp STREQUAL "SSL") + if(EXISTS "${OPENSSL_INCLUDE_DIR}" AND + (EXISTS "${OPENSSL_SSL_LIBRARY}" OR + EXISTS "${SSL_EAY_LIBRARY_DEBUG}" OR + EXISTS "${SSL_EAY_LIBRARY_RELEASE}") + ) + set(OpenSSL_${_comp}_FOUND TRUE) + else() + set(OpenSSL_${_comp}_FOUND FALSE) + endif() + else() + message(WARNING "${_comp} is not a valid OpenSSL component") + set(OpenSSL_${_comp}_FOUND FALSE) + endif() +endforeach() +unset(_comp) + +include(${CMAKE_CURRENT_LIST_DIR}/FindPackageHandleStandardArgs.cmake) +find_package_handle_standard_args(OpenSSL + REQUIRED_VARS + OPENSSL_CRYPTO_LIBRARY + OPENSSL_INCLUDE_DIR + VERSION_VAR + OPENSSL_VERSION + HANDLE_VERSION_RANGE + HANDLE_COMPONENTS + FAIL_MESSAGE + "Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR" +) + +mark_as_advanced(OPENSSL_INCLUDE_DIR) + +if(OPENSSL_FOUND) + if(NOT TARGET OpenSSL::Crypto AND + (EXISTS "${OPENSSL_CRYPTO_LIBRARY}" OR + EXISTS "${LIB_EAY_LIBRARY_DEBUG}" OR + EXISTS "${LIB_EAY_LIBRARY_RELEASE}") + ) + add_library(OpenSSL::Crypto UNKNOWN IMPORTED) + set_target_properties(OpenSSL::Crypto PROPERTIES + INTERFACE_INCLUDE_DIRECTORIES "${OPENSSL_INCLUDE_DIR}") + if(EXISTS "${OPENSSL_CRYPTO_LIBRARY}") + set_target_properties(OpenSSL::Crypto PROPERTIES + IMPORTED_LINK_INTERFACE_LANGUAGES "C" + IMPORTED_LOCATION "${OPENSSL_CRYPTO_LIBRARY}") + endif() + if(EXISTS "${LIB_EAY_LIBRARY_RELEASE}") + set_property(TARGET OpenSSL::Crypto APPEND PROPERTY + IMPORTED_CONFIGURATIONS RELEASE) + set_target_properties(OpenSSL::Crypto PROPERTIES + IMPORTED_LINK_INTERFACE_LANGUAGES_RELEASE "C" + IMPORTED_LOCATION_RELEASE "${LIB_EAY_LIBRARY_RELEASE}") + endif() + if(EXISTS "${LIB_EAY_LIBRARY_DEBUG}") + set_property(TARGET OpenSSL::Crypto APPEND PROPERTY + IMPORTED_CONFIGURATIONS DEBUG) + set_target_properties(OpenSSL::Crypto PROPERTIES + IMPORTED_LINK_INTERFACE_LANGUAGES_DEBUG "C" + IMPORTED_LOCATION_DEBUG "${LIB_EAY_LIBRARY_DEBUG}") + endif() + _OpenSSL_target_add_dependencies(OpenSSL::Crypto) + endif() + + if(NOT TARGET OpenSSL::SSL AND + (EXISTS "${OPENSSL_SSL_LIBRARY}" OR + EXISTS "${SSL_EAY_LIBRARY_DEBUG}" OR + EXISTS "${SSL_EAY_LIBRARY_RELEASE}") + ) + add_library(OpenSSL::SSL UNKNOWN IMPORTED) + set_target_properties(OpenSSL::SSL PROPERTIES + INTERFACE_INCLUDE_DIRECTORIES "${OPENSSL_INCLUDE_DIR}") + if(EXISTS "${OPENSSL_SSL_LIBRARY}") + set_target_properties(OpenSSL::SSL PROPERTIES + IMPORTED_LINK_INTERFACE_LANGUAGES "C" + IMPORTED_LOCATION "${OPENSSL_SSL_LIBRARY}") + endif() + if(EXISTS "${SSL_EAY_LIBRARY_RELEASE}") + set_property(TARGET OpenSSL::SSL APPEND PROPERTY + IMPORTED_CONFIGURATIONS RELEASE) + set_target_properties(OpenSSL::SSL PROPERTIES + IMPORTED_LINK_INTERFACE_LANGUAGES_RELEASE "C" + IMPORTED_LOCATION_RELEASE "${SSL_EAY_LIBRARY_RELEASE}") + endif() + if(EXISTS "${SSL_EAY_LIBRARY_DEBUG}") + set_property(TARGET OpenSSL::SSL APPEND PROPERTY + IMPORTED_CONFIGURATIONS DEBUG) + set_target_properties(OpenSSL::SSL PROPERTIES + IMPORTED_LINK_INTERFACE_LANGUAGES_DEBUG "C" + IMPORTED_LOCATION_DEBUG "${SSL_EAY_LIBRARY_DEBUG}") + endif() + if(TARGET OpenSSL::Crypto) + set_target_properties(OpenSSL::SSL PROPERTIES + INTERFACE_LINK_LIBRARIES OpenSSL::Crypto) + endif() + _OpenSSL_target_add_dependencies(OpenSSL::SSL) + endif() + + if("${OPENSSL_VERSION_MAJOR}.${OPENSSL_VERSION_MAJOR}.${OPENSSL_VERSION_FIX}" VERSION_GREATER_EQUAL "0.9.8") + if(MSVC) + if(EXISTS "${OPENSSL_INCLUDE_DIR}") + set(_OPENSSL_applink_paths PATHS ${OPENSSL_INCLUDE_DIR}) + endif() + find_file(OPENSSL_APPLINK_SOURCE + NAMES + openssl/applink.c + ${_OPENSSL_applink_paths} + NO_DEFAULT_PATH) + if(OPENSSL_APPLINK_SOURCE) + set(_OPENSSL_applink_interface_srcs ${OPENSSL_APPLINK_SOURCE}) + endif() + endif() + if(NOT TARGET OpenSSL::applink) + add_library(OpenSSL::applink INTERFACE IMPORTED) + set_property(TARGET OpenSSL::applink APPEND + PROPERTY INTERFACE_SOURCES + ${_OPENSSL_applink_interface_srcs}) + endif() + endif() +endif() + +# Restore the original find library ordering +if(OPENSSL_USE_STATIC_LIBS) + set(CMAKE_FIND_LIBRARY_SUFFIXES ${_openssl_ORIG_CMAKE_FIND_LIBRARY_SUFFIXES}) +endif() diff --git a/contrib/windows/cmake/FindPackageHandleStandardArgs.cmake b/contrib/windows/cmake/FindPackageHandleStandardArgs.cmake new file mode 100644 index 000000000000..fbcf7cd88bc9 --- /dev/null +++ b/contrib/windows/cmake/FindPackageHandleStandardArgs.cmake @@ -0,0 +1,605 @@ +# Distributed under the OSI-approved BSD 3-Clause License. See accompanying +# file Copyright.txt or https://cmake.org/licensing for details. + +#[=======================================================================[.rst: +FindPackageHandleStandardArgs +----------------------------- + +This module provides functions intended to be used in :ref:`Find Modules` +implementing :command:`find_package()` calls. + +.. command:: find_package_handle_standard_args + + This command handles the ``REQUIRED``, ``QUIET`` and version-related + arguments of :command:`find_package`. It also sets the + ``_FOUND`` variable. The package is considered found if all + variables listed contain valid results, e.g. valid filepaths. + + There are two signatures: + + .. code-block:: cmake + + find_package_handle_standard_args( + (DEFAULT_MSG|) + ... + ) + + find_package_handle_standard_args( + [FOUND_VAR ] + [REQUIRED_VARS ...] + [VERSION_VAR ] + [HANDLE_VERSION_RANGE] + [HANDLE_COMPONENTS] + [CONFIG_MODE] + [NAME_MISMATCHED] + [REASON_FAILURE_MESSAGE ] + [FAIL_MESSAGE ] + ) + + The ``_FOUND`` variable will be set to ``TRUE`` if all + the variables ``...`` are valid and any optional + constraints are satisfied, and ``FALSE`` otherwise. A success or + failure message may be displayed based on the results and on + whether the ``REQUIRED`` and/or ``QUIET`` option was given to + the :command:`find_package` call. + + The options are: + + ``(DEFAULT_MSG|)`` + In the simple signature this specifies the failure message. + Use ``DEFAULT_MSG`` to ask for a default message to be computed + (recommended). Not valid in the full signature. + + ``FOUND_VAR `` + .. deprecated:: 3.3 + + Specifies either ``_FOUND`` or + ``_FOUND`` as the result variable. This exists only + for compatibility with older versions of CMake and is now ignored. + Result variables of both names are always set for compatibility. + + ``REQUIRED_VARS ...`` + Specify the variables which are required for this package. + These may be named in the generated failure message asking the + user to set the missing variable values. Therefore these should + typically be cache entries such as ``FOO_LIBRARY`` and not output + variables like ``FOO_LIBRARIES``. + + .. versionchanged:: 3.18 + If ``HANDLE_COMPONENTS`` is specified, this option can be omitted. + + ``VERSION_VAR `` + Specify the name of a variable that holds the version of the package + that has been found. This version will be checked against the + (potentially) specified required version given to the + :command:`find_package` call, including its ``EXACT`` option. + The default messages include information about the required + version and the version which has been actually found, both + if the version is ok or not. + + ``HANDLE_VERSION_RANGE`` + .. versionadded:: 3.19 + + Enable handling of a version range, if one is specified. Without this + option, a developer warning will be displayed if a version range is + specified. + + ``HANDLE_COMPONENTS`` + Enable handling of package components. In this case, the command + will report which components have been found and which are missing, + and the ``_FOUND`` variable will be set to ``FALSE`` + if any of the required components (i.e. not the ones listed after + the ``OPTIONAL_COMPONENTS`` option of :command:`find_package`) are + missing. + + ``CONFIG_MODE`` + Specify that the calling find module is a wrapper around a + call to ``find_package( NO_MODULE)``. This implies + a ``VERSION_VAR`` value of ``_VERSION``. The command + will automatically check whether the package configuration file + was found. + + ``REASON_FAILURE_MESSAGE `` + .. versionadded:: 3.16 + + Specify a custom message of the reason for the failure which will be + appended to the default generated message. + + ``FAIL_MESSAGE `` + Specify a custom failure message instead of using the default + generated message. Not recommended. + + ``NAME_MISMATCHED`` + .. versionadded:: 3.17 + + Indicate that the ```` does not match + ``${CMAKE_FIND_PACKAGE_NAME}``. This is usually a mistake and raises a + warning, but it may be intentional for usage of the command for components + of a larger package. + +Example for the simple signature: + +.. code-block:: cmake + + find_package_handle_standard_args(LibXml2 DEFAULT_MSG + LIBXML2_LIBRARY LIBXML2_INCLUDE_DIR) + +The ``LibXml2`` package is considered to be found if both +``LIBXML2_LIBRARY`` and ``LIBXML2_INCLUDE_DIR`` are valid. +Then also ``LibXml2_FOUND`` is set to ``TRUE``. If it is not found +and ``REQUIRED`` was used, it fails with a +:command:`message(FATAL_ERROR)`, independent whether ``QUIET`` was +used or not. If it is found, success will be reported, including +the content of the first ````. On repeated CMake runs, +the same message will not be printed again. + +.. note:: + + If ```` does not match ``CMAKE_FIND_PACKAGE_NAME`` for the + calling module, a warning that there is a mismatch is given. The + ``FPHSA_NAME_MISMATCHED`` variable may be set to bypass the warning if using + the old signature and the ``NAME_MISMATCHED`` argument using the new + signature. To avoid forcing the caller to require newer versions of CMake for + usage, the variable's value will be used if defined when the + ``NAME_MISMATCHED`` argument is not passed for the new signature (but using + both is an error).. + +Example for the full signature: + +.. code-block:: cmake + + find_package_handle_standard_args(LibArchive + REQUIRED_VARS LibArchive_LIBRARY LibArchive_INCLUDE_DIR + VERSION_VAR LibArchive_VERSION) + +In this case, the ``LibArchive`` package is considered to be found if +both ``LibArchive_LIBRARY`` and ``LibArchive_INCLUDE_DIR`` are valid. +Also the version of ``LibArchive`` will be checked by using the version +contained in ``LibArchive_VERSION``. Since no ``FAIL_MESSAGE`` is given, +the default messages will be printed. + +Another example for the full signature: + +.. code-block:: cmake + + find_package(Automoc4 QUIET NO_MODULE HINTS /opt/automoc4) + find_package_handle_standard_args(Automoc4 CONFIG_MODE) + +In this case, a ``FindAutmoc4.cmake`` module wraps a call to +``find_package(Automoc4 NO_MODULE)`` and adds an additional search +directory for ``automoc4``. Then the call to +``find_package_handle_standard_args`` produces a proper success/failure +message. + +.. command:: find_package_check_version + + .. versionadded:: 3.19 + + Helper function which can be used to check if a ```` is valid + against version-related arguments of :command:`find_package`. + + .. code-block:: cmake + + find_package_check_version( + [HANDLE_VERSION_RANGE] + [RESULT_MESSAGE_VARIABLE ] + ) + + The ```` will hold a boolean value giving the result of the check. + + The options are: + + ``HANDLE_VERSION_RANGE`` + Enable handling of a version range, if one is specified. Without this + option, a developer warning will be displayed if a version range is + specified. + + ``RESULT_MESSAGE_VARIABLE `` + Specify a variable to get back a message describing the result of the check. + +Example for the usage: + +.. code-block:: cmake + + find_package_check_version(1.2.3 result HANDLE_VERSION_RANGE + RESULT_MESSAGE_VARIABLE reason) + if (result) + message (STATUS "${reason}") + else() + message (FATAL_ERROR "${reason}") + endif() +#]=======================================================================] + +include(${CMAKE_CURRENT_LIST_DIR}/FindPackageMessage.cmake) + + +cmake_policy(PUSH) +# numbers and boolean constants +cmake_policy (SET CMP0012 NEW) +# IN_LIST operator +cmake_policy (SET CMP0057 NEW) + + +# internal helper macro +macro(_FPHSA_FAILURE_MESSAGE _msg) + set (__msg "${_msg}") + if (FPHSA_REASON_FAILURE_MESSAGE) + string(APPEND __msg "\n Reason given by package: ${FPHSA_REASON_FAILURE_MESSAGE}\n") + endif() + if (${_NAME}_FIND_REQUIRED) + message(FATAL_ERROR "${__msg}") + else () + if (NOT ${_NAME}_FIND_QUIETLY) + message(STATUS "${__msg}") + endif () + endif () +endmacro() + + +# internal helper macro to generate the failure message when used in CONFIG_MODE: +macro(_FPHSA_HANDLE_FAILURE_CONFIG_MODE) + # _CONFIG is set, but FOUND is false, this means that some other of the REQUIRED_VARS was not found: + if(${_NAME}_CONFIG) + _FPHSA_FAILURE_MESSAGE("${FPHSA_FAIL_MESSAGE}: missing:${MISSING_VARS} (found ${${_NAME}_CONFIG} ${VERSION_MSG})") + else() + # If _CONSIDERED_CONFIGS is set, the config-file has been found, but no suitable version. + # List them all in the error message: + if(${_NAME}_CONSIDERED_CONFIGS) + set(configsText "") + list(LENGTH ${_NAME}_CONSIDERED_CONFIGS configsCount) + math(EXPR configsCount "${configsCount} - 1") + foreach(currentConfigIndex RANGE ${configsCount}) + list(GET ${_NAME}_CONSIDERED_CONFIGS ${currentConfigIndex} filename) + list(GET ${_NAME}_CONSIDERED_VERSIONS ${currentConfigIndex} version) + string(APPEND configsText "\n ${filename} (version ${version})") + endforeach() + if (${_NAME}_NOT_FOUND_MESSAGE) + if (FPHSA_REASON_FAILURE_MESSAGE) + string(PREPEND FPHSA_REASON_FAILURE_MESSAGE "${${_NAME}_NOT_FOUND_MESSAGE}\n ") + else() + set(FPHSA_REASON_FAILURE_MESSAGE "${${_NAME}_NOT_FOUND_MESSAGE}") + endif() + else() + string(APPEND configsText "\n") + endif() + _FPHSA_FAILURE_MESSAGE("${FPHSA_FAIL_MESSAGE} ${VERSION_MSG}, checked the following files:${configsText}") + + else() + # Simple case: No Config-file was found at all: + _FPHSA_FAILURE_MESSAGE("${FPHSA_FAIL_MESSAGE}: found neither ${_NAME}Config.cmake nor ${_NAME_LOWER}-config.cmake ${VERSION_MSG}") + endif() + endif() +endmacro() + + +function(FIND_PACKAGE_CHECK_VERSION version result) + cmake_parse_arguments (PARSE_ARGV 2 FPCV "HANDLE_VERSION_RANGE;NO_AUTHOR_WARNING_VERSION_RANGE" "RESULT_MESSAGE_VARIABLE" "") + + if (FPCV_UNPARSED_ARGUMENTS) + message (FATAL_ERROR "find_package_check_version(): ${FPCV_UNPARSED_ARGUMENTS}: unexpected arguments") + endif() + if ("RESULT_MESSAGE_VARIABLE" IN_LIST FPCV_KEYWORDS_MISSING_VALUES) + message (FATAL_ERROR "find_package_check_version(): RESULT_MESSAGE_VARIABLE expects an argument") + endif() + + set (${result} FALSE PARENT_SCOPE) + if (FPCV_RESULT_MESSAGE_VARIABLE) + unset (${FPCV_RESULT_MESSAGE_VARIABLE} PARENT_SCOPE) + endif() + + if (_CMAKE_FPHSA_PACKAGE_NAME) + set (package "${_CMAKE_FPHSA_PACKAGE_NAME}") + elseif (CMAKE_FIND_PACKAGE_NAME) + set (package "${CMAKE_FIND_PACKAGE_NAME}") + else() + message (FATAL_ERROR "find_package_check_version(): Cannot be used outside a 'Find Module'") + endif() + + if (NOT FPCV_NO_AUTHOR_WARNING_VERSION_RANGE + AND ${package}_FIND_VERSION_RANGE AND NOT FPCV_HANDLE_VERSION_RANGE) + message(AUTHOR_WARNING + "`find_package()` specify a version range but the option " + "HANDLE_VERSION_RANGE` is not passed to `find_package_check_version()`. " + "Only the lower endpoint of the range will be used.") + endif() + + + set (version_ok FALSE) + unset (version_msg) + + if (FPCV_HANDLE_VERSION_RANGE AND ${package}_FIND_VERSION_RANGE) + if ((${package}_FIND_VERSION_RANGE_MIN STREQUAL "INCLUDE" + AND version VERSION_GREATER_EQUAL ${package}_FIND_VERSION_MIN) + AND ((${package}_FIND_VERSION_RANGE_MAX STREQUAL "INCLUDE" + AND version VERSION_LESS_EQUAL ${package}_FIND_VERSION_MAX) + OR (${package}_FIND_VERSION_RANGE_MAX STREQUAL "EXCLUDE" + AND version VERSION_LESS ${package}_FIND_VERSION_MAX))) + set (version_ok TRUE) + set(version_msg "(found suitable version \"${version}\", required range is \"${${package}_FIND_VERSION_RANGE}\")") + else() + set(version_msg "Found unsuitable version \"${version}\", required range is \"${${package}_FIND_VERSION_RANGE}\"") + endif() + elseif (DEFINED ${package}_FIND_VERSION) + if(${package}_FIND_VERSION_EXACT) # exact version required + # count the dots in the version string + string(REGEX REPLACE "[^.]" "" version_dots "${version}") + # add one dot because there is one dot more than there are components + string(LENGTH "${version_dots}." version_dots) + if (version_dots GREATER ${package}_FIND_VERSION_COUNT) + # Because of the C++ implementation of find_package() ${package}_FIND_VERSION_COUNT + # is at most 4 here. Therefore a simple lookup table is used. + if (${package}_FIND_VERSION_COUNT EQUAL 1) + set(version_regex "[^.]*") + elseif (${package}_FIND_VERSION_COUNT EQUAL 2) + set(version_regex "[^.]*\\.[^.]*") + elseif (${package}_FIND_VERSION_COUNT EQUAL 3) + set(version_regex "[^.]*\\.[^.]*\\.[^.]*") + else() + set(version_regex "[^.]*\\.[^.]*\\.[^.]*\\.[^.]*") + endif() + string(REGEX REPLACE "^(${version_regex})\\..*" "\\1" version_head "${version}") + if (NOT ${package}_FIND_VERSION VERSION_EQUAL version_head) + set(version_msg "Found unsuitable version \"${version}\", but required is exact version \"${${package}_FIND_VERSION}\"") + else () + set(version_ok TRUE) + set(version_msg "(found suitable exact version \"${_FOUND_VERSION}\")") + endif () + else () + if (NOT ${package}_FIND_VERSION VERSION_EQUAL version) + set(version_msg "Found unsuitable version \"${version}\", but required is exact version \"${${package}_FIND_VERSION}\"") + else () + set(version_ok TRUE) + set(version_msg "(found suitable exact version \"${version}\")") + endif () + endif () + else() # minimum version + if (${package}_FIND_VERSION VERSION_GREATER version) + set(version_msg "Found unsuitable version \"${version}\", but required is at least \"${${package}_FIND_VERSION}\"") + else() + set(version_ok TRUE) + set(version_msg "(found suitable version \"${version}\", minimum required is \"${${package}_FIND_VERSION}\")") + endif() + endif() + else () + set(version_ok TRUE) + set(version_msg "(found version \"${version}\")") + endif() + + set (${result} ${version_ok} PARENT_SCOPE) + if (FPCV_RESULT_MESSAGE_VARIABLE) + set (${FPCV_RESULT_MESSAGE_VARIABLE} "${version_msg}" PARENT_SCOPE) + endif() +endfunction() + + +function(FIND_PACKAGE_HANDLE_STANDARD_ARGS _NAME _FIRST_ARG) + + # Set up the arguments for `cmake_parse_arguments`. + set(options CONFIG_MODE HANDLE_COMPONENTS NAME_MISMATCHED HANDLE_VERSION_RANGE) + set(oneValueArgs FAIL_MESSAGE REASON_FAILURE_MESSAGE VERSION_VAR FOUND_VAR) + set(multiValueArgs REQUIRED_VARS) + + # Check whether we are in 'simple' or 'extended' mode: + set(_KEYWORDS_FOR_EXTENDED_MODE ${options} ${oneValueArgs} ${multiValueArgs} ) + list(FIND _KEYWORDS_FOR_EXTENDED_MODE "${_FIRST_ARG}" INDEX) + + unset(FPHSA_NAME_MISMATCHED_override) + if (DEFINED FPHSA_NAME_MISMATCHED) + # If the variable NAME_MISMATCHED variable is set, error if it is passed as + # an argument. The former is for old signatures, the latter is for new + # signatures. + list(FIND ARGN "NAME_MISMATCHED" name_mismatched_idx) + if (NOT name_mismatched_idx EQUAL "-1") + message(FATAL_ERROR + "The `NAME_MISMATCHED` argument may only be specified by the argument or " + "the variable, not both.") + endif () + + # But use the variable if it is not an argument to avoid forcing minimum + # CMake version bumps for calling modules. + set(FPHSA_NAME_MISMATCHED_override "${FPHSA_NAME_MISMATCHED}") + endif () + + if(${INDEX} EQUAL -1) + set(FPHSA_FAIL_MESSAGE ${_FIRST_ARG}) + set(FPHSA_REQUIRED_VARS ${ARGN}) + set(FPHSA_VERSION_VAR) + else() + cmake_parse_arguments(FPHSA "${options}" "${oneValueArgs}" "${multiValueArgs}" ${_FIRST_ARG} ${ARGN}) + + if(FPHSA_UNPARSED_ARGUMENTS) + message(FATAL_ERROR "Unknown keywords given to FIND_PACKAGE_HANDLE_STANDARD_ARGS(): \"${FPHSA_UNPARSED_ARGUMENTS}\"") + endif() + + if(NOT FPHSA_FAIL_MESSAGE) + set(FPHSA_FAIL_MESSAGE "DEFAULT_MSG") + endif() + + # In config-mode, we rely on the variable _CONFIG, which is set by find_package() + # when it successfully found the config-file, including version checking: + if(FPHSA_CONFIG_MODE) + list(INSERT FPHSA_REQUIRED_VARS 0 ${_NAME}_CONFIG) + list(REMOVE_DUPLICATES FPHSA_REQUIRED_VARS) + set(FPHSA_VERSION_VAR ${_NAME}_VERSION) + endif() + + if(NOT FPHSA_REQUIRED_VARS AND NOT FPHSA_HANDLE_COMPONENTS) + message(FATAL_ERROR "No REQUIRED_VARS specified for FIND_PACKAGE_HANDLE_STANDARD_ARGS()") + endif() + endif() + + if (DEFINED FPHSA_NAME_MISMATCHED_override) + set(FPHSA_NAME_MISMATCHED "${FPHSA_NAME_MISMATCHED_override}") + endif () + + if (DEFINED CMAKE_FIND_PACKAGE_NAME + AND NOT FPHSA_NAME_MISMATCHED + AND NOT _NAME STREQUAL CMAKE_FIND_PACKAGE_NAME) + message(AUTHOR_WARNING + "The package name passed to `find_package_handle_standard_args` " + "(${_NAME}) does not match the name of the calling package " + "(${CMAKE_FIND_PACKAGE_NAME}). This can lead to problems in calling " + "code that expects `find_package` result variables (e.g., `_FOUND`) " + "to follow a certain pattern.") + endif () + + if (${_NAME}_FIND_VERSION_RANGE AND NOT FPHSA_HANDLE_VERSION_RANGE) + message(AUTHOR_WARNING + "`find_package()` specify a version range but the module ${_NAME} does " + "not support this capability. Only the lower endpoint of the range " + "will be used.") + endif() + + # to propagate package name to FIND_PACKAGE_CHECK_VERSION + set(_CMAKE_FPHSA_PACKAGE_NAME "${_NAME}") + + # now that we collected all arguments, process them + + if("x${FPHSA_FAIL_MESSAGE}" STREQUAL "xDEFAULT_MSG") + set(FPHSA_FAIL_MESSAGE "Could NOT find ${_NAME}") + endif() + + if (FPHSA_REQUIRED_VARS) + list(GET FPHSA_REQUIRED_VARS 0 _FIRST_REQUIRED_VAR) + endif() + + string(TOUPPER ${_NAME} _NAME_UPPER) + string(TOLOWER ${_NAME} _NAME_LOWER) + + if(FPHSA_FOUND_VAR) + set(_FOUND_VAR_UPPER ${_NAME_UPPER}_FOUND) + set(_FOUND_VAR_MIXED ${_NAME}_FOUND) + if(FPHSA_FOUND_VAR STREQUAL _FOUND_VAR_MIXED OR FPHSA_FOUND_VAR STREQUAL _FOUND_VAR_UPPER) + set(_FOUND_VAR ${FPHSA_FOUND_VAR}) + else() + message(FATAL_ERROR "The argument for FOUND_VAR is \"${FPHSA_FOUND_VAR}\", but only \"${_FOUND_VAR_MIXED}\" and \"${_FOUND_VAR_UPPER}\" are valid names.") + endif() + else() + set(_FOUND_VAR ${_NAME_UPPER}_FOUND) + endif() + + # collect all variables which were not found, so they can be printed, so the + # user knows better what went wrong (#6375) + set(MISSING_VARS "") + set(DETAILS "") + # check if all passed variables are valid + set(FPHSA_FOUND_${_NAME} TRUE) + foreach(_CURRENT_VAR ${FPHSA_REQUIRED_VARS}) + if(NOT ${_CURRENT_VAR}) + set(FPHSA_FOUND_${_NAME} FALSE) + string(APPEND MISSING_VARS " ${_CURRENT_VAR}") + else() + string(APPEND DETAILS "[${${_CURRENT_VAR}}]") + endif() + endforeach() + if(FPHSA_FOUND_${_NAME}) + set(${_NAME}_FOUND TRUE) + set(${_NAME_UPPER}_FOUND TRUE) + else() + set(${_NAME}_FOUND FALSE) + set(${_NAME_UPPER}_FOUND FALSE) + endif() + + # component handling + unset(FOUND_COMPONENTS_MSG) + unset(MISSING_COMPONENTS_MSG) + + if(FPHSA_HANDLE_COMPONENTS) + foreach(comp ${${_NAME}_FIND_COMPONENTS}) + if(${_NAME}_${comp}_FOUND) + + if(NOT DEFINED FOUND_COMPONENTS_MSG) + set(FOUND_COMPONENTS_MSG "found components:") + endif() + string(APPEND FOUND_COMPONENTS_MSG " ${comp}") + + else() + + if(NOT DEFINED MISSING_COMPONENTS_MSG) + set(MISSING_COMPONENTS_MSG "missing components:") + endif() + string(APPEND MISSING_COMPONENTS_MSG " ${comp}") + + if(${_NAME}_FIND_REQUIRED_${comp}) + set(${_NAME}_FOUND FALSE) + string(APPEND MISSING_VARS " ${comp}") + endif() + + endif() + endforeach() + set(COMPONENT_MSG "${FOUND_COMPONENTS_MSG} ${MISSING_COMPONENTS_MSG}") + string(APPEND DETAILS "[c${COMPONENT_MSG}]") + endif() + + # version handling: + set(VERSION_MSG "") + set(VERSION_OK TRUE) + + # check with DEFINED here as the requested or found version may be "0" + if (DEFINED ${_NAME}_FIND_VERSION) + if(DEFINED ${FPHSA_VERSION_VAR}) + set(_FOUND_VERSION ${${FPHSA_VERSION_VAR}}) + if (FPHSA_HANDLE_VERSION_RANGE) + set (FPCV_HANDLE_VERSION_RANGE HANDLE_VERSION_RANGE) + else() + set(FPCV_HANDLE_VERSION_RANGE NO_AUTHOR_WARNING_VERSION_RANGE) + endif() + find_package_check_version ("${_FOUND_VERSION}" VERSION_OK RESULT_MESSAGE_VARIABLE VERSION_MSG + ${FPCV_HANDLE_VERSION_RANGE}) + else() + # if the package was not found, but a version was given, add that to the output: + if(${_NAME}_FIND_VERSION_EXACT) + set(VERSION_MSG "(Required is exact version \"${${_NAME}_FIND_VERSION}\")") + elseif (FPHSA_HANDLE_VERSION_RANGE AND ${_NAME}_FIND_VERSION_RANGE) + set(VERSION_MSG "(Required is version range \"${${_NAME}_FIND_VERSION_RANGE}\")") + else() + set(VERSION_MSG "(Required is at least version \"${${_NAME}_FIND_VERSION}\")") + endif() + endif() + else () + # Check with DEFINED as the found version may be 0. + if(DEFINED ${FPHSA_VERSION_VAR}) + set(VERSION_MSG "(found version \"${${FPHSA_VERSION_VAR}}\")") + endif() + endif () + + if(VERSION_OK) + string(APPEND DETAILS "[v${${FPHSA_VERSION_VAR}}(${${_NAME}_FIND_VERSION})]") + else() + set(${_NAME}_FOUND FALSE) + endif() + + + # print the result: + if (${_NAME}_FOUND) + FIND_PACKAGE_MESSAGE(${_NAME} "Found ${_NAME}: ${${_FIRST_REQUIRED_VAR}} ${VERSION_MSG} ${COMPONENT_MSG}" "${DETAILS}") + else () + + if(FPHSA_CONFIG_MODE) + _FPHSA_HANDLE_FAILURE_CONFIG_MODE() + else() + if(NOT VERSION_OK) + set(RESULT_MSG) + if (_FIRST_REQUIRED_VAR) + string (APPEND RESULT_MSG "found ${${_FIRST_REQUIRED_VAR}}") + endif() + if (COMPONENT_MSG) + if (RESULT_MSG) + string (APPEND RESULT_MSG ", ") + endif() + string (APPEND RESULT_MSG "${FOUND_COMPONENTS_MSG}") + endif() + _FPHSA_FAILURE_MESSAGE("${FPHSA_FAIL_MESSAGE}: ${VERSION_MSG} (${RESULT_MSG})") + else() + _FPHSA_FAILURE_MESSAGE("${FPHSA_FAIL_MESSAGE} (missing:${MISSING_VARS}) ${VERSION_MSG}") + endif() + endif() + + endif () + + set(${_NAME}_FOUND ${${_NAME}_FOUND} PARENT_SCOPE) + set(${_NAME_UPPER}_FOUND ${${_NAME}_FOUND} PARENT_SCOPE) +endfunction() + + +cmake_policy(POP) diff --git a/contrib/windows/cmake/FindPackageMessage.cmake b/contrib/windows/cmake/FindPackageMessage.cmake new file mode 100644 index 000000000000..0628b9816911 --- /dev/null +++ b/contrib/windows/cmake/FindPackageMessage.cmake @@ -0,0 +1,48 @@ +# Distributed under the OSI-approved BSD 3-Clause License. See accompanying +# file Copyright.txt or https://cmake.org/licensing for details. + +#[=======================================================================[.rst: +FindPackageMessage +------------------ + +.. code-block:: cmake + + find_package_message( "message for user" "find result details") + +This function is intended to be used in FindXXX.cmake modules files. +It will print a message once for each unique find result. This is +useful for telling the user where a package was found. The first +argument specifies the name (XXX) of the package. The second argument +specifies the message to display. The third argument lists details +about the find result so that if they change the message will be +displayed again. The macro also obeys the QUIET argument to the +find_package command. + +Example: + +.. code-block:: cmake + + if(X11_FOUND) + find_package_message(X11 "Found X11: ${X11_X11_LIB}" + "[${X11_X11_LIB}][${X11_INCLUDE_DIR}]") + else() + ... + endif() +#]=======================================================================] + +function(find_package_message pkg msg details) + # Avoid printing a message repeatedly for the same find result. + if(NOT ${pkg}_FIND_QUIETLY) + string(REPLACE "\n" "" details "${details}") + set(DETAILS_VAR FIND_PACKAGE_MESSAGE_DETAILS_${pkg}) + if(NOT "${details}" STREQUAL "${${DETAILS_VAR}}") + # The message has not yet been printed. + message(STATUS "${msg}") + + # Save the find details in the cache to avoid printing the same + # message again. + set("${DETAILS_VAR}" "${details}" + CACHE INTERNAL "Details about finding ${pkg}") + endif() + endif() +endfunction() diff --git a/contrib/windows/cmake/FindWdk.cmake b/contrib/windows/cmake/FindWdk.cmake new file mode 100644 index 000000000000..827b49d8c766 --- /dev/null +++ b/contrib/windows/cmake/FindWdk.cmake @@ -0,0 +1,218 @@ +# Redistribution and use is allowed under the OSI-approved 3-clause BSD license. +# Copyright (c) 2018 Sergey Podobry (sergey.podobry at gmail.com). All rights reserved. + +#.rst: +# FindWDK +# ---------- +# +# This module searches for the installed Windows Development Kit (WDK) and +# exposes commands for creating kernel drivers and kernel libraries. +# +# Output variables: +# - `WDK_FOUND` -- if false, do not try to use WDK +# - `WDK_ROOT` -- where WDK is installed +# - `WDK_VERSION` -- the version of the selected WDK +# - `WDK_WINVER` -- the WINVER used for kernel drivers and libraries +# (default value is `0x0601` and can be changed per target or globally) +# +# Example usage: +# +# find_package(WDK REQUIRED) +# +# wdk_add_library(KmdfCppLib STATIC KMDF 1.15 +# KmdfCppLib.h +# KmdfCppLib.cpp +# ) +# target_include_directories(KmdfCppLib INTERFACE .) +# +# wdk_add_driver(KmdfCppDriver KMDF 1.15 +# Main.cpp +# ) +# target_link_libraries(KmdfCppDriver KmdfCppLib) +# + +if(DEFINED ENV{WDKContentRoot}) + file(GLOB WDK_NTDDK_FILES + "$ENV{WDKContentRoot}/Include/*/km/ntddk.h" + ) +else() + file(GLOB WDK_NTDDK_FILES + "C:/Program Files*/Windows Kits/10/Include/*/km/ntddk.h" + ) +endif() + +if(WDK_NTDDK_FILES) + list(GET WDK_NTDDK_FILES -1 WDK_LATEST_NTDDK_FILE) +endif() + +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(WDK REQUIRED_VARS WDK_LATEST_NTDDK_FILE) + +if (NOT WDK_LATEST_NTDDK_FILE) + return() +endif() + +get_filename_component(WDK_ROOT "${WDK_LATEST_NTDDK_FILE}" DIRECTORY) +get_filename_component(WDK_ROOT "${WDK_ROOT}" DIRECTORY) +get_filename_component(WDK_VERSION "${WDK_ROOT}" NAME) +get_filename_component(WDK_ROOT "${WDK_ROOT}" DIRECTORY) +get_filename_component(WDK_ROOT "${WDK_ROOT}" DIRECTORY) + +message(STATUS "WDK_ROOT: " "${WDK_ROOT}") +message(STATUS "WDK_VERSION: " "${WDK_VERSION}") + +set(WDK_WINVER "0x0601" CACHE STRING "Default WINVER for WDK targets") + +set(WDK_ADDITIONAL_FLAGS_FILE "${CMAKE_CURRENT_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/wdkflags.h") +file(WRITE "${WDK_ADDITIONAL_FLAGS_FILE}" "#pragma runtime_checks(\"suc\", off)") + +define_property(TARGET + PROPERTY WDK_TARGET + BRIEF_DOCS "Defines the target to be a WDK driver/library" + FULL_DOCS "Defines the target to be a WDK driver/library" +) + +list(APPEND WDK_COMPILE_FLAGS + "/X" + "/Zp8" # set struct alignment + "/GF" # enable string pooling + "/GR-" # disable RTTI + "/Gz" # __stdcall by default + "/kernel" # create kernel mode binary + "/FIwarning.h" # disable warnings in WDK headers + "/FI${WDK_ADDITIONAL_FLAGS_FILE}" # include file to disable RTC +) + +list(APPEND WDK_COMPILE_DEFINITIONS + "WINNT=1" +) +list(APPEND WDK_COMPILE_DEFINITIONS_DEBUG + "MSC_NOOPT" + "DEPRECATE_DDK_FUNCTIONS=1" + "DBG=1" +) + +if(CMAKE_SIZEOF_VOID_P EQUAL 4) + list(APPEND WDK_COMPILE_DEFINITIONS + "_X86_=1" + "i386=1" + "STD_CALL" + ) + set(WDK_PLATFORM "x86") +elseif(CMAKE_SIZEOF_VOID_P EQUAL 8) + list(APPEND WDK_COMPILE_DEFINITIONS + "_WIN64" + "_AMD64_" + "AMD64" + ) + set(WDK_PLATFORM "x64") +else() + message(FATAL_ERROR "Unsupported architecture") +endif() + +list(APPEND WDK_INCLUDE_DIRECTORIES + "${WDK_ROOT}/Include/${WDK_VERSION}/shared" + "${WDK_ROOT}/Include/${WDK_VERSION}/km" + "${WDK_ROOT}/Include/${WDK_VERSION}/km/crt" +) + +# Generate imported targets for WDK lib files +file(GLOB WDK_LIBRARIES "${WDK_ROOT}/Lib/${WDK_VERSION}/km/${WDK_PLATFORM}/*.lib") +foreach(LIBRARY IN LISTS WDK_LIBRARIES) + get_filename_component(LIBRARY_NAME "${LIBRARY}" NAME_WE) + string(TOUPPER ${LIBRARY_NAME} LIBRARY_NAME) + add_library(WDK::${LIBRARY_NAME} INTERFACE IMPORTED) + set_property(TARGET WDK::${LIBRARY_NAME} PROPERTY INTERFACE_LINK_LIBRARIES "${LIBRARY}") +endforeach(LIBRARY) +unset(WDK_LIBRARIES) + +function(wdk_add_driver _target) + cmake_parse_arguments(WDK "" "KMDF;WINVER" "" ${ARGN}) + + add_executable(${_target} ${WDK_UNPARSED_ARGUMENTS}) + + set_target_properties( + ${_target} PROPERTIES + SUFFIX ".sys" + WDK_TARGET 1 + COMPILE_PDB_NAME ${_target} + COMPILE_PDB_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} + ) + target_compile_options(${_target} PRIVATE ${WDK_COMPILE_FLAGS}) + target_compile_definitions(${_target} PRIVATE + ${WDK_COMPILE_DEFINITIONS} + $<$:${WDK_COMPILE_DEFINITIONS_DEBUG}> + _WIN32_WINNT=${WDK_WINVER} + ) + target_link_options(${_target} PRIVATE + "/MANIFEST:NO" + "/DRIVER" + "/OPT:REF" + "/INCREMENTAL:NO" + "/OPT:ICF" + "/SUBSYSTEM:NATIVE" + "/MERGE:_TEXT=.text" + "/MERGE:_PAGE=PAGE" + "/NODEFAULTLIB" # do not link default CRT + "/SECTION:INIT,d" + "/VERSION:10.0" + ) + + target_include_directories(${_target} SYSTEM PRIVATE ${WDK_INCLUDE_DIRECTORIES}) + target_link_libraries(${_target} PRIVATE WDK::NTOSKRNL WDK::HAL WDK::WMILIB) + + if (WDK_WINVER LESS 0x0602) + target_link_libraries(${_target} PRIVATE WDK::BUFFEROVERFLOWK) + else() + target_link_libraries(${_target} PRIVATE WDK::BUFFEROVERFLOWFASTFAILK) + endif() + + if(CMAKE_SIZEOF_VOID_P EQUAL 4) + target_link_libraries(${_target} PRIVATE WDK::MEMCMP) + endif() + + if(DEFINED WDK_KMDF) + target_include_directories(${_target} SYSTEM PRIVATE "${WDK_ROOT}/Include/wdf/kmdf/${WDK_KMDF}") + target_link_libraries(${_target} PRIVATE + "${WDK_ROOT}/Lib/wdf/kmdf/${WDK_PLATFORM}/${WDK_KMDF}/WdfDriverEntry.lib" + "${WDK_ROOT}/Lib/wdf/kmdf/${WDK_PLATFORM}/${WDK_KMDF}/WdfLdr.lib" + ) + + if(CMAKE_SIZEOF_VOID_P EQUAL 4) + target_link_options(${_target} PRIVATE "/ENTRY:FxDriverEntry@8") + elseif(CMAKE_SIZEOF_VOID_P EQUAL 8) + target_link_options(${_target} PRIVATE "/ENTRY:FxDriverEntry") + endif() + else() + if(CMAKE_SIZEOF_VOID_P EQUAL 4) + target_link_options(${_target} PRIVATE "/ENTRY:GsDriverEntry@8") + elseif(CMAKE_SIZEOF_VOID_P EQUAL 8) + target_link_options(${_target} PRIVATE "/ENTRY:GsDriverEntry") + endif() + endif() +endfunction() + +function(wdk_add_library _target) + cmake_parse_arguments(WDK "" "KMDF;WINVER" "" ${ARGN}) + + add_library(${_target} STATIC ${WDK_UNPARSED_ARGUMENTS}) + + set_target_properties( + ${_target} PROPERTIES + WDK_TARGET 1 + COMPILE_PDB_NAME ${_target} + COMPILE_PDB_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} + ) + target_compile_options(${_target} PRIVATE ${WDK_COMPILE_FLAGS}) + target_compile_definitions(${_target} PRIVATE + ${WDK_COMPILE_DEFINITIONS} + $<$:${WDK_COMPILE_DEFINITIONS_DEBUG}> + _WIN32_WINNT=${WDK_WINVER} + ) + + target_include_directories(${_target} SYSTEM PRIVATE ${WDK_INCLUDE_DIRECTORIES}) + + if(DEFINED WDK_KMDF) + target_include_directories(${_target} SYSTEM PRIVATE "${WDK_ROOT}/Include/wdf/kmdf/${WDK_KMDF}") + endif() +endfunction() diff --git a/contrib/windows/cmake/GetGitRevisionDescription.cmake b/contrib/windows/cmake/GetGitRevisionDescription.cmake new file mode 100644 index 000000000000..87f691ad8d2b --- /dev/null +++ b/contrib/windows/cmake/GetGitRevisionDescription.cmake @@ -0,0 +1,284 @@ +# - Returns a version string from Git +# +# These functions force a re-configure on each git commit so that you can +# trust the values of the variables in your build system. +# +# get_git_head_revision( [ALLOW_LOOKING_ABOVE_CMAKE_SOURCE_DIR]) +# +# Returns the refspec and sha hash of the current head revision +# +# git_describe( [ ...]) +# +# Returns the results of git describe on the source tree, and adjusting +# the output so that it tests false if an error occurs. +# +# git_describe_working_tree( [ ...]) +# +# Returns the results of git describe on the working tree (--dirty option), +# and adjusting the output so that it tests false if an error occurs. +# +# git_get_exact_tag( [ ...]) +# +# Returns the results of git describe --exact-match on the source tree, +# and adjusting the output so that it tests false if there was no exact +# matching tag. +# +# git_local_changes() +# +# Returns either "CLEAN" or "DIRTY" with respect to uncommitted changes. +# Uses the return code of "git diff-index --quiet HEAD --". +# Does not regard untracked files. +# +# Requires CMake 2.6 or newer (uses the 'function' command) +# +# Original Author: +# 2009-2020 Ryan Pavlik +# http://academic.cleardefinition.com +# +# Copyright 2009-2013, Iowa State University. +# Copyright 2013-2020, Ryan Pavlik +# Copyright 2013-2020, Contributors +# SPDX-License-Identifier: BSL-1.0 +# Distributed under the Boost Software License, Version 1.0. +# (See accompanying file LICENSE_1_0.txt or copy at +# http://www.boost.org/LICENSE_1_0.txt) + +if(__get_git_revision_description) + return() +endif() +set(__get_git_revision_description YES) + +# We must run the following at "include" time, not at function call time, +# to find the path to this module rather than the path to a calling list file +get_filename_component(_gitdescmoddir ${CMAKE_CURRENT_LIST_FILE} PATH) + +# Function _git_find_closest_git_dir finds the next closest .git directory +# that is part of any directory in the path defined by _start_dir. +# The result is returned in the parent scope variable whose name is passed +# as variable _git_dir_var. If no .git directory can be found, the +# function returns an empty string via _git_dir_var. +# +# Example: Given a path C:/bla/foo/bar and assuming C:/bla/.git exists and +# neither foo nor bar contain a file/directory .git. This wil return +# C:/bla/.git +# +function(_git_find_closest_git_dir _start_dir _git_dir_var) + set(cur_dir "${_start_dir}") + set(git_dir "${_start_dir}/.git") + while(NOT EXISTS "${git_dir}") + # .git dir not found, search parent directories + set(git_previous_parent "${cur_dir}") + get_filename_component(cur_dir ${cur_dir} DIRECTORY) + if(cur_dir STREQUAL git_previous_parent) + # We have reached the root directory, we are not in git + set(${_git_dir_var} + "" + PARENT_SCOPE) + return() + endif() + set(git_dir "${cur_dir}/.git") + endwhile() + set(${_git_dir_var} + "${git_dir}" + PARENT_SCOPE) +endfunction() + +function(get_git_head_revision _refspecvar _hashvar) + _git_find_closest_git_dir("${CMAKE_CURRENT_SOURCE_DIR}" GIT_DIR) + + if("${ARGN}" STREQUAL "ALLOW_LOOKING_ABOVE_CMAKE_SOURCE_DIR") + set(ALLOW_LOOKING_ABOVE_CMAKE_SOURCE_DIR TRUE) + else() + set(ALLOW_LOOKING_ABOVE_CMAKE_SOURCE_DIR FALSE) + endif() + if(NOT "${GIT_DIR}" STREQUAL "") + file(RELATIVE_PATH _relative_to_source_dir "${CMAKE_SOURCE_DIR}" + "${GIT_DIR}") + if("${_relative_to_source_dir}" MATCHES "[.][.]" AND NOT ALLOW_LOOKING_ABOVE_CMAKE_SOURCE_DIR) + # We've gone above the CMake root dir. + set(GIT_DIR "") + endif() + endif() + if("${GIT_DIR}" STREQUAL "") + set(${_refspecvar} + "GITDIR-NOTFOUND" + PARENT_SCOPE) + set(${_hashvar} + "GITDIR-NOTFOUND" + PARENT_SCOPE) + return() + endif() + + # Check if the current source dir is a git submodule or a worktree. + # In both cases .git is a file instead of a directory. + # + if(NOT IS_DIRECTORY ${GIT_DIR}) + # The following git command will return a non empty string that + # points to the super project working tree if the current + # source dir is inside a git submodule. + # Otherwise the command will return an empty string. + # + execute_process( + COMMAND "${GIT_EXECUTABLE}" rev-parse + --show-superproject-working-tree + WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}" + OUTPUT_VARIABLE out + ERROR_QUIET OUTPUT_STRIP_TRAILING_WHITESPACE) + if(NOT "${out}" STREQUAL "") + # If out is empty, GIT_DIR/CMAKE_CURRENT_SOURCE_DIR is in a submodule + file(READ ${GIT_DIR} submodule) + string(REGEX REPLACE "gitdir: (.*)$" "\\1" GIT_DIR_RELATIVE + ${submodule}) + string(STRIP ${GIT_DIR_RELATIVE} GIT_DIR_RELATIVE) + get_filename_component(SUBMODULE_DIR ${GIT_DIR} PATH) + get_filename_component(GIT_DIR ${SUBMODULE_DIR}/${GIT_DIR_RELATIVE} + ABSOLUTE) + set(HEAD_SOURCE_FILE "${GIT_DIR}/HEAD") + else() + # GIT_DIR/CMAKE_CURRENT_SOURCE_DIR is in a worktree + file(READ ${GIT_DIR} worktree_ref) + # The .git directory contains a path to the worktree information directory + # inside the parent git repo of the worktree. + # + string(REGEX REPLACE "gitdir: (.*)$" "\\1" git_worktree_dir + ${worktree_ref}) + string(STRIP ${git_worktree_dir} git_worktree_dir) + _git_find_closest_git_dir("${git_worktree_dir}" GIT_DIR) + set(HEAD_SOURCE_FILE "${git_worktree_dir}/HEAD") + endif() + else() + set(HEAD_SOURCE_FILE "${GIT_DIR}/HEAD") + endif() + set(GIT_DATA "${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/git-data") + if(NOT EXISTS "${GIT_DATA}") + file(MAKE_DIRECTORY "${GIT_DATA}") + endif() + + if(NOT EXISTS "${HEAD_SOURCE_FILE}") + return() + endif() + set(HEAD_FILE "${GIT_DATA}/HEAD") + configure_file("${HEAD_SOURCE_FILE}" "${HEAD_FILE}" COPYONLY) + + configure_file("${_gitdescmoddir}/GetGitRevisionDescription.cmake.in" + "${GIT_DATA}/grabRef.cmake" @ONLY) + include("${GIT_DATA}/grabRef.cmake") + + set(${_refspecvar} + "${HEAD_REF}" + PARENT_SCOPE) + set(${_hashvar} + "${HEAD_HASH}" + PARENT_SCOPE) +endfunction() + +function(git_describe _var) + if(NOT GIT_FOUND) + find_package(Git QUIET) + endif() + get_git_head_revision(refspec hash) + if(NOT GIT_FOUND) + set(${_var} + "GIT-NOTFOUND" + PARENT_SCOPE) + return() + endif() + if(NOT hash) + set(${_var} + "HEAD-HASH-NOTFOUND" + PARENT_SCOPE) + return() + endif() + + # TODO sanitize + #if((${ARGN}" MATCHES "&&") OR + # (ARGN MATCHES "||") OR + # (ARGN MATCHES "\\;")) + # message("Please report the following error to the project!") + # message(FATAL_ERROR "Looks like someone's doing something nefarious with git_describe! Passed arguments ${ARGN}") + #endif() + + #message(STATUS "Arguments to execute_process: ${ARGN}") + + execute_process( + COMMAND "${GIT_EXECUTABLE}" describe --tags --always ${hash} ${ARGN} + WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}" + RESULT_VARIABLE res + OUTPUT_VARIABLE out + ERROR_QUIET OUTPUT_STRIP_TRAILING_WHITESPACE) + if(NOT res EQUAL 0) + set(out "${out}-${res}-NOTFOUND") + endif() + + set(${_var} + "${out}" + PARENT_SCOPE) +endfunction() + +function(git_describe_working_tree _var) + if(NOT GIT_FOUND) + find_package(Git QUIET) + endif() + if(NOT GIT_FOUND) + set(${_var} + "GIT-NOTFOUND" + PARENT_SCOPE) + return() + endif() + + execute_process( + COMMAND "${GIT_EXECUTABLE}" describe --dirty ${ARGN} + WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}" + RESULT_VARIABLE res + OUTPUT_VARIABLE out + ERROR_QUIET OUTPUT_STRIP_TRAILING_WHITESPACE) + if(NOT res EQUAL 0) + set(out "${out}-${res}-NOTFOUND") + endif() + + set(${_var} + "${out}" + PARENT_SCOPE) +endfunction() + +function(git_get_exact_tag _var) + git_describe(out --exact-match ${ARGN}) + set(${_var} + "${out}" + PARENT_SCOPE) +endfunction() + +function(git_local_changes _var) + if(NOT GIT_FOUND) + find_package(Git QUIET) + endif() + get_git_head_revision(refspec hash) + if(NOT GIT_FOUND) + set(${_var} + "GIT-NOTFOUND" + PARENT_SCOPE) + return() + endif() + if(NOT hash) + set(${_var} + "HEAD-HASH-NOTFOUND" + PARENT_SCOPE) + return() + endif() + + execute_process( + COMMAND "${GIT_EXECUTABLE}" diff-index --quiet HEAD -- + WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}" + RESULT_VARIABLE res + OUTPUT_VARIABLE out + ERROR_QUIET OUTPUT_STRIP_TRAILING_WHITESPACE) + if(res EQUAL 0) + set(${_var} + "CLEAN" + PARENT_SCOPE) + else() + set(${_var} + "DIRTY" + PARENT_SCOPE) + endif() +endfunction() diff --git a/contrib/windows/cmake/GetGitRevisionDescription.cmake.in b/contrib/windows/cmake/GetGitRevisionDescription.cmake.in new file mode 100644 index 000000000000..116efc4e4eec --- /dev/null +++ b/contrib/windows/cmake/GetGitRevisionDescription.cmake.in @@ -0,0 +1,43 @@ +# +# Internal file for GetGitRevisionDescription.cmake +# +# Requires CMake 2.6 or newer (uses the 'function' command) +# +# Original Author: +# 2009-2010 Ryan Pavlik +# http://academic.cleardefinition.com +# Iowa State University HCI Graduate Program/VRAC +# +# Copyright 2009-2012, Iowa State University +# Copyright 2011-2015, Contributors +# Distributed under the Boost Software License, Version 1.0. +# (See accompanying file LICENSE_1_0.txt or copy at +# http://www.boost.org/LICENSE_1_0.txt) +# SPDX-License-Identifier: BSL-1.0 + +set(HEAD_HASH) + +file(READ "@HEAD_FILE@" HEAD_CONTENTS LIMIT 1024) + +string(STRIP "${HEAD_CONTENTS}" HEAD_CONTENTS) +if(HEAD_CONTENTS MATCHES "ref") + # named branch + string(REPLACE "ref: " "" HEAD_REF "${HEAD_CONTENTS}") + if(EXISTS "@GIT_DIR@/${HEAD_REF}") + configure_file("@GIT_DIR@/${HEAD_REF}" "@GIT_DATA@/head-ref" COPYONLY) + else() + configure_file("@GIT_DIR@/packed-refs" "@GIT_DATA@/packed-refs" COPYONLY) + file(READ "@GIT_DATA@/packed-refs" PACKED_REFS) + if(${PACKED_REFS} MATCHES "([0-9a-z]*) ${HEAD_REF}") + set(HEAD_HASH "${CMAKE_MATCH_1}") + endif() + endif() +else() + # detached HEAD + configure_file("@GIT_DIR@/HEAD" "@GIT_DATA@/head-ref" COPYONLY) +endif() + +if(NOT HEAD_HASH) + file(READ "@GIT_DATA@/head-ref" HEAD_HASH LIMIT 1024) + string(STRIP "${HEAD_HASH}" HEAD_HASH) +endif() diff --git a/contrib/windows/cmake/SelectLibraryConfigurations.cmake b/contrib/windows/cmake/SelectLibraryConfigurations.cmake new file mode 100644 index 000000000000..4c0e9a8c0a2f --- /dev/null +++ b/contrib/windows/cmake/SelectLibraryConfigurations.cmake @@ -0,0 +1,80 @@ +# Distributed under the OSI-approved BSD 3-Clause License. See accompanying +# file Copyright.txt or https://cmake.org/licensing for details. + +#[=======================================================================[.rst: +SelectLibraryConfigurations +--------------------------- + +.. code-block:: cmake + + select_library_configurations(basename) + +This macro takes a library base name as an argument, and will choose +good values for the variables + +:: + + basename_LIBRARY + basename_LIBRARIES + basename_LIBRARY_DEBUG + basename_LIBRARY_RELEASE + +depending on what has been found and set. + +If only ``basename_LIBRARY_RELEASE`` is defined, ``basename_LIBRARY`` will +be set to the release value, and ``basename_LIBRARY_DEBUG`` will be set +to ``basename_LIBRARY_DEBUG-NOTFOUND``. If only ``basename_LIBRARY_DEBUG`` +is defined, then ``basename_LIBRARY`` will take the debug value, and +``basename_LIBRARY_RELEASE`` will be set to ``basename_LIBRARY_RELEASE-NOTFOUND``. + +If the generator supports configuration types, then ``basename_LIBRARY`` +and ``basename_LIBRARIES`` will be set with debug and optimized flags +specifying the library to be used for the given configuration. If no +build type has been set or the generator in use does not support +configuration types, then ``basename_LIBRARY`` and ``basename_LIBRARIES`` +will take only the release value, or the debug value if the release one +is not set. +#]=======================================================================] + +# This macro was adapted from the FindQt4 CMake module and is maintained by Will +# Dicharry . + +macro(select_library_configurations basename) + if(NOT ${basename}_LIBRARY_RELEASE) + set(${basename}_LIBRARY_RELEASE "${basename}_LIBRARY_RELEASE-NOTFOUND" CACHE FILEPATH "Path to a library.") + endif() + if(NOT ${basename}_LIBRARY_DEBUG) + set(${basename}_LIBRARY_DEBUG "${basename}_LIBRARY_DEBUG-NOTFOUND" CACHE FILEPATH "Path to a library.") + endif() + + get_property(_isMultiConfig GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG) + if( ${basename}_LIBRARY_DEBUG AND ${basename}_LIBRARY_RELEASE AND + NOT ${basename}_LIBRARY_DEBUG STREQUAL ${basename}_LIBRARY_RELEASE AND + ( _isMultiConfig OR CMAKE_BUILD_TYPE ) ) + # if the generator is multi-config or if CMAKE_BUILD_TYPE is set for + # single-config generators, set optimized and debug libraries + set( ${basename}_LIBRARY "" ) + foreach( _libname IN LISTS ${basename}_LIBRARY_RELEASE ) + list( APPEND ${basename}_LIBRARY optimized "${_libname}" ) + endforeach() + foreach( _libname IN LISTS ${basename}_LIBRARY_DEBUG ) + list( APPEND ${basename}_LIBRARY debug "${_libname}" ) + endforeach() + elseif( ${basename}_LIBRARY_RELEASE ) + set( ${basename}_LIBRARY ${${basename}_LIBRARY_RELEASE} ) + elseif( ${basename}_LIBRARY_DEBUG ) + set( ${basename}_LIBRARY ${${basename}_LIBRARY_DEBUG} ) + else() + set( ${basename}_LIBRARY "${basename}_LIBRARY-NOTFOUND") + endif() + + set( ${basename}_LIBRARIES "${${basename}_LIBRARY}" ) + + if( ${basename}_LIBRARY ) + set( ${basename}_FOUND TRUE ) + endif() + + mark_as_advanced( ${basename}_LIBRARY_RELEASE + ${basename}_LIBRARY_DEBUG + ) +endmacro() diff --git a/contrib/windows/parsedump/README.md b/contrib/windows/parsedump/README.md new file mode 100644 index 000000000000..cf01578b5c3b --- /dev/null +++ b/contrib/windows/parsedump/README.md @@ -0,0 +1,16 @@ +# To extract crash dump for OpenZFS on Windows. + +Prepare the required software + +* Download and install cdb from Microsoft https://download.microsoft.com/download/7/9/6/7962e9ce-cd69-4574-978c-1202654bd729/windowssdk/Installers/X64%20Debuggers%20And%20Tools-x64_en-us.msi +* Install Python 3 from the Microsoft store https://apps.microsoft.com/store/detail/python-39/9P7QFQMJRFP7 + +Run the script + +* Browse to C:\Program Files\OpenZFS On Windows +* Run parsedump.bat + +Upload the results + +* Drag and drop C:\stack.txt and C:\cbuf.txt into the editor textbox when creating an issue on GitHub + diff --git a/contrib/windows/parsedump/parsedump.bat b/contrib/windows/parsedump/parsedump.bat new file mode 100644 index 000000000000..8e62c5513a3f --- /dev/null +++ b/contrib/windows/parsedump/parsedump.bat @@ -0,0 +1,34 @@ +@echo off + +REM https://superuser.com/a/852877 + +:: BatchGotAdmin +:------------------------------------- +REM --> Check for permissions +>nul 2>&1 "%SYSTEMROOT%\system32\cacls.exe" "%SYSTEMROOT%\system32\config\system" + +REM --> If error flag set, we do not have admin. +if '%errorlevel%' NEQ '0' ( + echo Requesting administrative privileges... + goto UACPrompt +) else ( goto gotAdmin ) + +:UACPrompt + echo Set UAC = CreateObject^("Shell.Application"^) > "%temp%\getadmin.vbs" + set params = %*:"="" + echo UAC.ShellExecute "cmd.exe", "/c %~s0 %params%", "", "runas", 1 >> "%temp%\getadmin.vbs" + + "%temp%\getadmin.vbs" + del "%temp%\getadmin.vbs" + exit /B + +:gotAdmin + pushd "%CD%" + CD /D "%~dp0" +:-------------------------------------- + +@echo on + + +python3 parsedump.py +pause diff --git a/contrib/windows/parsedump/parsedump.py b/contrib/windows/parsedump/parsedump.py new file mode 100644 index 000000000000..1478777bfe9a --- /dev/null +++ b/contrib/windows/parsedump/parsedump.py @@ -0,0 +1,47 @@ +# https://apps.microsoft.com/store/detail/python-39/9P7QFQMJRFP7 +# https://download.microsoft.com/download/7/9/6/ +# 7962e9ce-cd69-4574-978c-1202654bd729/windowssdk/ +# Installers/X64 Debuggers And Tools-x64_en-us.msi + +import re +import subprocess + +cdbstr = "C:\\Program Files\\Windows Kits\\10\\Debuggers\\x64\\cdb.exe" +dumpfilestr = "C:\\Windows\\MEMORY.DMP" +symbolstr = "srv*;C:\\Program Files\\OpenZFS On Windows\\symbols\\;" + + +def run(arg): + result = subprocess.run( + [cdbstr, "-z", dumpfilestr, "-c", arg, "-y", symbolstr], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + return re.search( + r"Reading initial command[\s\S]+quit:", result.stdout.decode() + ).group() + + +analyze = run("!analyze -v ; q") + +stack = run("k ; q") + +cbuf = run("dt OpenZFS!cbuf ; q") +# print(cbuf) +b = re.search(r"'dt OpenZFS!cbuf ; q'" + "[\\s\\S]+?(0x[0-9A-Za-z]{8}`[0-9A-Za-z]{8})", + cbuf) +cbufaddr = b.group()[-19:] + +cbuf2 = run( + ".writemem C:\\cbuf.txt " + cbufaddr + " L100000 ; q", +) + +with open("C:\\stack.txt", "w") as file: + file.write(analyze) + file.write("\n") + file.write(stack) + + +print("Please upload C:\\stack.txt and C:\\cbuf.txt " + "when creating an issue on GitHub") diff --git a/include/libzfs.h b/include/libzfs.h index a7037e3e6266..d75ad795d16d 100644 --- a/include/libzfs.h +++ b/include/libzfs.h @@ -712,6 +712,7 @@ _LIBZFS_H int zfs_snapshot(libzfs_handle_t *, const char *, boolean_t, _LIBZFS_H int zfs_snapshot_nvl(libzfs_handle_t *hdl, nvlist_t *snaps, nvlist_t *props); _LIBZFS_H int zfs_rollback(zfs_handle_t *, zfs_handle_t *, boolean_t); +_LIBZFS_H void zfs_rollback_os(struct zfs_handle *zhp); typedef struct renameflags { /* recursive rename */ @@ -901,6 +902,8 @@ _LIBZFS_H int zfs_mount_at(zfs_handle_t *, const char *, int, const char *); _LIBZFS_H int zfs_unmount(zfs_handle_t *, const char *, int); _LIBZFS_H int zfs_unmountall(zfs_handle_t *, int); _LIBZFS_H int zfs_mount_delegation_check(void); +_LIBZFS_H int zfs_snapshot_mount(zfs_handle_t *, const char *options, int); +_LIBZFS_H int zfs_snapshot_unmount(zfs_handle_t *, int); #if defined(__linux__) || defined(__APPLE__) _LIBZFS_H int zfs_parse_mount_options(const char *mntopts, diff --git a/include/libzutil.h b/include/libzutil.h index 237ff976ba62..95338fde2b99 100644 --- a/include/libzutil.h +++ b/include/libzutil.h @@ -108,7 +108,12 @@ _LIBZUTIL_H void update_vdev_config_dev_strs(nvlist_t *); /* * Default device paths */ +#ifdef _WIN32 +#define DISK_ROOT "\\\\?\\" +#else #define DISK_ROOT "/dev" +#endif + #define UDISK_ROOT "/dev/disk" #define ZVOL_ROOT "/dev/zvol" @@ -193,6 +198,9 @@ _LIBZUTIL_H ssize_t zfs_dirnamelen(const char *path); extern char **environ; _LIBZUTIL_H void zfs_setproctitle_init(int argc, char *argv[], char *envp[]); _LIBZUTIL_H void zfs_setproctitle(const char *fmt, ...); +#elif _WIN32 +#define zfs_setproctitle(fmt, ...) +#define zfs_setproctitle_init(x, y, z) ((void)0) #else #define zfs_setproctitle(fmt, ...) setproctitle(fmt, ##__VA_ARGS__) #define zfs_setproctitle_init(x, y, z) ((void)0) diff --git a/include/os/windows/Trace.h b/include/os/windows/Trace.h new file mode 100644 index 000000000000..f1403cf316aa --- /dev/null +++ b/include/os/windows/Trace.h @@ -0,0 +1,89 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2020, DataCore Software Corp. + */ + +#pragma once + +static const int TRACE_FATAL = 1; +static const int TRACE_ERROR = 2; +static const int TRACE_WARNING = 3; +static const int TRACE_INFO = 4; +static const int TRACE_VERBOSE = 5; +static const int TRACE_NOISY = 8; + +#ifdef WPPFILE +#define WPPNAME OpenZFSTraceGuid +#define WPPGUID c20c603c, afd4, 467d, bf76, c0a4c10553df + +#define WPP_DEFINE_DEFAULT_BITS \ + WPP_DEFINE_BIT(MYDRIVER_ALL_INFO) \ + WPP_DEFINE_BIT(TRACE_KDPRINT) \ + WPP_DEFINE_BIT(DEFAULT_TRACE_LEVEL) + +#undef WPP_DEFINE_CONTROL_GUID +#define WPP_CONTROL_GUIDS \ + WPP_DEFINE_CONTROL_GUID(WPPNAME, (WPPGUID), \ + WPP_DEFINE_DEFAULT_BITS) + +#define WPP_FLAGS_LEVEL_LOGGER(Flags, level) \ + WPP_LEVEL_LOGGER(Flags) + +#define WPP_FLAGS_LEVEL_ENABLED(Flags, level) \ + (WPP_LEVEL_ENABLED(Flags) && \ + WPP_CONTROL(WPP_BIT_ ## Flags).Level >= level) + +#define WPP_LEVEL_FLAGS_LOGGER(lvl, flags) \ + WPP_LEVEL_LOGGER(flags) + +#define WPP_LEVEL_FLAGS_ENABLED(lvl, flags) \ + (WPP_LEVEL_ENABLED(flags) && \ + WPP_CONTROL(WPP_BIT_ ## flags).Level >= lvl) + + +// begin_wpp config +// FUNC dprintf{FLAGS=MYDRIVER_ALL_INFO, LEVEL=TRACE_INFO}(MSG, ...); +// FUNC TraceEvent{FLAGS=MYDRIVER_ALL_INFO}(LEVEL, MSG, ...); +// end_wpp + +#define STRINGIZE_DETAIL(x) #x +#define STRINGIZE(x) STRINGIZE_DETAIL(x) + +#include STRINGIZE(WPPFILE) + +#else + +#undef WPP_INIT_TRACING +#define WPP_INIT_TRACING(...) ((void)(0, __VA_ARGS__)) + +#undef WPP_CLEANUP +#define WPP_CLEANUP(...) ((void)(0, __VA_ARGS__)) +#endif + +#ifndef WPP_CHECK_INIT +#define WPP_CHECK_INIT +#endif + + +void ZFSWppInit(PDRIVER_OBJECT pDriverObject, PUNICODE_STRING pRegistryPath); + +void ZFSWppCleanup(PDRIVER_OBJECT pDriverObject); diff --git a/include/os/windows/spl/rpc/types.h b/include/os/windows/spl/rpc/types.h new file mode 100644 index 000000000000..50b0459b81f0 --- /dev/null +++ b/include/os/windows/spl/rpc/types.h @@ -0,0 +1,29 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + * + * Copyright (c) 1989, 2011, Oracle and/or its affiliates. All rights reserved. + */ + +#ifndef _SPL_RPC_TYPES_H +#define _SPL_RPC_TYPES_H + +typedef int bool_t; + +#endif /* SPL_RPC_TYPES_H */ diff --git a/include/os/windows/spl/rpc/xdr.h b/include/os/windows/spl/rpc/xdr.h new file mode 100644 index 000000000000..64da4b8d2ab5 --- /dev/null +++ b/include/os/windows/spl/rpc/xdr.h @@ -0,0 +1,171 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + * + * Copyright (c) 1989, 2011, Oracle and/or its affiliates. All rights reserved. + */ +/* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ +/* + * Portions of this source code were derived from Berkeley + * 4.3 BSD under license from the Regents of the University of + * California. + */ + +/* + * xdr.h, External Data Representation Serialization Routines. + * + */ + +#ifndef _SPL_RPC_XDR_H +#define _SPL_RPC_XDR_H + + +#include +#include + +/* + * XDR enums and types. + */ +enum xdr_op { + XDR_ENCODE, + XDR_DECODE +}; + +struct xdr_ops; + +typedef struct { + struct xdr_ops *x_ops; + caddr_t x_addr; + caddr_t x_addr_end; + enum xdr_op x_op; +} XDR; + +typedef bool_t (*xdrproc_t)(XDR *xdrs, void *ptr); + +struct xdr_ops { + bool_t (*xdr_control)(XDR *, int, void *); + + bool_t (*xdr_char)(XDR *, char *); + bool_t (*xdr_u_short)(XDR *, unsigned short *); + bool_t (*xdr_u_int)(XDR *, unsigned *); + bool_t (*xdr_u_longlong_t)(XDR *, u_longlong_t *); + + bool_t (*xdr_opaque)(XDR *, caddr_t, const uint_t); + bool_t (*xdr_string)(XDR *, char **, const uint_t); + bool_t (*xdr_array)(XDR *, caddr_t *, uint_t *, const uint_t, + const uint_t, const xdrproc_t); +}; + +/* + * XDR control operator. + */ +#define XDR_GET_BYTES_AVAIL 1 + +struct xdr_bytesrec { + bool_t xc_is_last_record; + size_t xc_num_avail; +}; + +typedef struct xdr_bytesrec xdr_bytesrec; + +/* + * XDR functions. + */ +void xdrmem_create(XDR *xdrs, const caddr_t addr, const uint_t size, + const enum xdr_op op); + +#define xdr_destroy(xdrs) ((void) 0) + +#define xdr_control(xdrs, req, info) (xdrs)->x_ops->xdr_control((xdrs),\ + (req), (info)) + +/* + * For precaution, the following are defined as static inlines instead of macros + * to get some amount of type safety. + * + * Also, macros wouldn't work in the case where typecasting is done, because it + * must be possible to reference the functions' addresses by these names. + */ +static inline bool_t xdr_char(XDR *xdrs, char *cp) +{ + return (xdrs->x_ops->xdr_char(xdrs, cp)); +} + +static inline bool_t xdr_u_short(XDR *xdrs, unsigned short *usp) +{ + return (xdrs->x_ops->xdr_u_short(xdrs, usp)); +} + +static inline bool_t xdr_short(XDR *xdrs, short *sp) +{ + // BUILD_BUG_ON(sizeof(short) != 2); + return (xdrs->x_ops->xdr_u_short(xdrs, (unsigned short *) sp)); +} + +static inline bool_t xdr_u_int(XDR *xdrs, unsigned *up) +{ + return (xdrs->x_ops->xdr_u_int(xdrs, up)); +} + +static inline bool_t xdr_int(XDR *xdrs, int *ip) +{ + // BUILD_BUG_ON(sizeof(int) != 4); + return (xdrs->x_ops->xdr_u_int(xdrs, (unsigned *)ip)); +} + +static inline bool_t xdr_u_longlong_t(XDR *xdrs, u_longlong_t *ullp) +{ + return (xdrs->x_ops->xdr_u_longlong_t(xdrs, ullp)); +} + +static inline bool_t xdr_longlong_t(XDR *xdrs, longlong_t *llp) +{ + // BUILD_BUG_ON(sizeof(longlong_t) != 8); + return (xdrs->x_ops->xdr_u_longlong_t(xdrs, (u_longlong_t *)llp)); +} + +/* + * Fixed-length opaque data. + */ +static inline bool_t xdr_opaque(XDR *xdrs, caddr_t cp, const uint_t cnt) +{ + return (xdrs->x_ops->xdr_opaque(xdrs, cp, cnt)); +} + +/* + * Variable-length string. + * The *sp buffer must have (maxsize + 1) bytes. + */ +static inline bool_t xdr_string(XDR *xdrs, char **sp, const uint_t maxsize) +{ + return (xdrs->x_ops->xdr_string(xdrs, sp, maxsize)); +} + +/* + * Variable-length arrays. + */ +static inline bool_t xdr_array(XDR *xdrs, caddr_t *arrp, uint_t *sizep, + const uint_t maxsize, const uint_t elsize, const xdrproc_t elproc) +{ + return (xdrs->x_ops->xdr_array(xdrs, arrp, sizep, maxsize, elsize, + elproc)); +} + +#endif /* SPL_RPC_XDR_H */ diff --git a/include/os/windows/spl/spl-debug.h b/include/os/windows/spl/spl-debug.h new file mode 100644 index 000000000000..9dac67b30b3e --- /dev/null +++ b/include/os/windows/spl/spl-debug.h @@ -0,0 +1,61 @@ +/* + * Copyright (C) 2007-2010 Lawrence Livermore National Security, LLC. + * Copyright (C) 2007 The Regents of the University of California. + * Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). + * Written by Brian Behlendorf . + * UCRL-CODE-235197 + * + * This file is part of the SPL, Solaris Porting Layer. + * For details, see . + * + * The SPL is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; either version 2 of the License, or (at your + * option) any later version. + * + * The SPL is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + * + * You should have received a copy of the GNU General Public License along + * with the SPL. If not, see . + */ + +/* + * Available debug functions. These function should be used by any + * package which needs to integrate with the SPL log infrastructure. + * + * SDEBUG() - Log debug message with specified mask. + * SDEBUG_LIMIT() - Log just 1 debug message with specified mask. + * SWARN() - Log a warning message. + * SERROR() - Log an error message. + * SEMERG() - Log an emergency error message. + * SCONSOLE() - Log a generic message to the console. + * + * SENTRY - Log entry point to a function. + * SEXIT - Log exit point from a function. + * SRETURN(x) - Log return from a function. + * SGOTO(x, y) - Log goto within a function. + */ + +#ifndef _SPL_DEBUG_INTERNAL_H +#define _SPL_DEBUG_INTERNAL_H + +#include +#ifdef __cplusplus +// To make C++ happier about strnlen in kcdata.h +extern "C" { +#endif +#include +#ifdef __cplusplus +} +#endif + + + +void spl_backtrace(char *thesignal); +int getpcstack(uintptr_t *pcstack, int pcstack_limit); +void print_symbol(uintptr_t symbol); + +#endif /* SPL_DEBUG_INTERNAL_H */ diff --git a/include/os/windows/spl/sys/acl.h b/include/os/windows/spl/sys/acl.h new file mode 100644 index 000000000000..e7dcdfe697f6 --- /dev/null +++ b/include/os/windows/spl/sys/acl.h @@ -0,0 +1,128 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SPL_ACL_H +#define _SPL_ACL_H + +#include + +typedef struct ace { + uid_t a_who; + uint32_t a_access_mask; + uint16_t a_flags; + uint16_t a_type; +} ace_t; + +typedef struct ace_object { + uid_t a_who; /* uid or gid */ + uint32_t a_access_mask; /* read,write,... */ + uint16_t a_flags; /* see below */ + uint16_t a_type; /* allow or deny */ + uint8_t a_obj_type[16]; /* obj type */ + uint8_t a_inherit_obj_type[16]; /* inherit obj */ +} ace_object_t; + +#define MAX_ACL_ENTRIES 1024 + +#define ACE_READ_DATA 0x00000001 +#define ACE_LIST_DIRECTORY 0x00000001 +#define ACE_WRITE_DATA 0x00000002 +#define ACE_ADD_FILE 0x00000002 +#define ACE_APPEND_DATA 0x00000004 +#define ACE_ADD_SUBDIRECTORY 0x00000004 +#define ACE_READ_NAMED_ATTRS 0x00000008 +#define ACE_WRITE_NAMED_ATTRS 0x00000010 +#define ACE_EXECUTE 0x00000020 +#define ACE_DELETE_CHILD 0x00000040 +#define ACE_READ_ATTRIBUTES 0x00000080 +#define ACE_WRITE_ATTRIBUTES 0x00000100 +#define ACE_DELETE 0x00010000 +#define ACE_READ_ACL 0x00020000 +#define ACE_WRITE_ACL 0x00040000 +#define ACE_WRITE_OWNER 0x00080000 +#define ACE_SYNCHRONIZE 0x00100000 + +#define ACE_FILE_INHERIT_ACE 0x0001 +#define ACE_DIRECTORY_INHERIT_ACE 0x0002 +#define ACE_NO_PROPAGATE_INHERIT_ACE 0x0004 +#define ACE_INHERIT_ONLY_ACE 0x0008 +#define ACE_SUCCESSFUL_ACCESS_ACE_FLAG 0x0010 +#define ACE_FAILED_ACCESS_ACE_FLAG 0x0020 +#define ACE_IDENTIFIER_GROUP 0x0040 +#define ACE_INHERITED_ACE 0x0080 +#define ACE_OWNER 0x1000 +#define ACE_GROUP 0x2000 +#define ACE_EVERYONE 0x4000 + +#define ACE_ACCESS_ALLOWED_ACE_TYPE 0x0000 +#define ACE_ACCESS_DENIED_ACE_TYPE 0x0001 +#define ACE_SYSTEM_AUDIT_ACE_TYPE 0x0002 +#define ACE_SYSTEM_ALARM_ACE_TYPE 0x0003 + +#define ACL_AUTO_INHERIT 0x0001 +#define ACL_PROTECTED 0x0002 +#define ACL_DEFAULTED 0x0004 +#define ACL_FLAGS_ALL (ACL_AUTO_INHERIT|ACL_PROTECTED|ACL_DEFAULTED) + +#define ACE_ACCESS_ALLOWED_COMPOUND_ACE_TYPE 0x04 +#define ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE 0x05 +#define ACE_ACCESS_DENIED_OBJECT_ACE_TYPE 0x06 +#define ACE_SYSTEM_AUDIT_OBJECT_ACE_TYPE 0x07 +#define ACE_SYSTEM_ALARM_OBJECT_ACE_TYPE 0x08 +#define ACE_ACCESS_ALLOWED_CALLBACK_ACE_TYPE 0x09 +#define ACE_ACCESS_DENIED_CALLBACK_ACE_TYPE 0x0A +#define ACE_ACCESS_ALLOWED_CALLBACK_OBJECT_ACE_TYPE 0x0B +#define ACE_ACCESS_DENIED_CALLBACK_OBJECT_ACE_TYPE 0x0C +#define ACE_SYSTEM_AUDIT_CALLBACK_ACE_TYPE 0x0D +#define ACE_SYSTEM_ALARM_CALLBACK_ACE_TYPE 0x0E +#define ACE_SYSTEM_AUDIT_CALLBACK_OBJECT_ACE_TYPE 0x0F +#define ACE_SYSTEM_ALARM_CALLBACK_OBJECT_ACE_TYPE 0x10 + +#define ACE_ALL_TYPES 0x001F + +#define ACE_TYPE_FLAGS (ACE_OWNER|ACE_GROUP|ACE_EVERYONE|ACE_IDENTIFIER_GROUP) + +#define ACE_ALL_PERMS (ACE_READ_DATA|ACE_LIST_DIRECTORY|ACE_WRITE_DATA| \ + ACE_ADD_FILE|ACE_APPEND_DATA|ACE_ADD_SUBDIRECTORY| \ + ACE_READ_NAMED_ATTRS|ACE_WRITE_NAMED_ATTRS|ACE_EXECUTE| \ + ACE_DELETE_CHILD|ACE_READ_ATTRIBUTES|ACE_WRITE_ATTRIBUTES| \ + ACE_DELETE|ACE_READ_ACL|ACE_WRITE_ACL|ACE_WRITE_OWNER| \ + ACE_SYNCHRONIZE) + +#define VSA_ACE 0x0010 +#define VSA_ACECNT 0x0020 +#define VSA_ACE_ALLTYPES 0x0040 +#define VSA_ACE_ACLFLAGS 0x0080 + +typedef struct trivial_acl { + uint32_t allow0; /* allow mask for bits only in owner */ + uint32_t deny1; /* deny mask for bits not in owner */ + uint32_t deny2; /* deny mask for bits not in group */ + uint32_t owner; /* allow mask matching mode */ + uint32_t group; /* allow mask matching mode */ + uint32_t everyone; /* allow mask matching mode */ +} trivial_acl_t; + +#endif /* _SPL_ACL_H */ diff --git a/include/os/windows/spl/sys/acl_impl.h b/include/os/windows/spl/sys/acl_impl.h new file mode 100644 index 000000000000..96a4949c3e91 --- /dev/null +++ b/include/os/windows/spl/sys/acl_impl.h @@ -0,0 +1,5 @@ + +#ifndef _SPL_ACL_IMPL_H +#define _SPL_ACL_IMPL_H + +#endif /* _SPL_ACL_IMPL_H */ diff --git a/include/os/windows/spl/sys/atomic.h b/include/os/windows/spl/sys/atomic.h new file mode 100644 index 000000000000..1fd1d8478da2 --- /dev/null +++ b/include/os/windows/spl/sys/atomic.h @@ -0,0 +1,212 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Windows Atomic functions using GCC builtins. + * + * Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + * + */ + +#ifndef _SPL_ATOMIC_H +#define _SPL_ATOMIC_H + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/* The _nv variants return the NewValue */ + +/* + * Increment target + */ +static inline void atomic_inc_32(volatile uint32_t *target) +{ + InterlockedIncrement((volatile LONG *)target); +} +static inline void atomic_inc_64(volatile uint64_t *target) +{ + InterlockedIncrement64((volatile LONG64 *)target); +} +static inline int32_t atomic_inc_32_nv(volatile uint32_t *target) +{ + return (InterlockedIncrement((volatile LONG *)target)); +} +static inline int64_t atomic_inc_64_nv(volatile uint64_t *target) +{ + return (InterlockedIncrement64((volatile LONG64 *)target)); +} + +/* + * Decrement target + */ +static inline void atomic_dec_32(volatile uint32_t *target) +{ + InterlockedDecrement((volatile LONG *)target); +} +static inline void atomic_dec_64(volatile uint64_t *target) +{ + InterlockedDecrement64((volatile LONG64 *)target); +} +static inline int32_t atomic_dec_32_nv(volatile uint32_t *target) +{ + return (InterlockedDecrement((volatile LONG *)target)); +} +static inline int64_t atomic_dec_64_nv(volatile uint64_t *target) +{ + return (InterlockedDecrement64((volatile LONG64 *)target)); +} + + + + +/* + * Add delta to target + */ +static inline void +atomic_add_32(volatile uint32_t *target, int32_t delta) +{ + InterlockedExchangeAdd((volatile LONG *)target, delta); +} + +static inline uint32_t +atomic_add_32_nv(volatile uint32_t *target, int32_t delta) +{ + return (InterlockedExchangeAdd((volatile LONG *)target, delta) + delta); +} + +static inline void +atomic_add_64(volatile uint64_t *target, int64_t delta) +{ + InterlockedExchangeAdd64((volatile LONG64 *)target, delta); +} + +static inline uint64_t +atomic_add_64_nv(volatile uint64_t *target, int64_t delta) +{ + return (InterlockedExchangeAdd64((volatile LONG64 *)target, + delta) + delta); +} + +/* + * Subtract delta to target + */ +static inline void +atomic_sub_32(volatile uint32_t *target, int32_t delta) +{ + InterlockedExchangeAdd((volatile LONG *)target, -delta); +} + +static inline void +atomic_sub_64(volatile uint64_t *target, int64_t delta) +{ + InterlockedExchangeAdd64((volatile LONG64 *)target, -delta); +} + +static inline uint64_t +atomic_sub_64_nv(volatile uint64_t *target, int64_t delta) +{ + return (InterlockedExchangeAdd64((volatile LONG64 *)target, + -delta) - delta); +} + +/* + * logical OR bits with target + */ + +/* + * logical AND bits with target + */ + +/* + * Compare And Set + * if *arg1 == arg2, then set *arg1 = arg3; return old value. + */ + +static inline uint32_t +atomic_cas_32(volatile uint32_t *_target, uint32_t _cmp, uint32_t _new) +{ + return (InterlockedCompareExchange((volatile LONG *)_target, _new, + _cmp)); +} + +static inline uint64_t +atomic_cas_64(volatile uint64_t *_target, uint64_t _cmp, uint64_t _new) +{ + return (InterlockedCompareExchange64((volatile LONG64 *)_target, + _new, _cmp)); +} + +static inline uint32_t +atomic_swap_32(volatile uint32_t *_target, uint32_t _new) +{ + return (InterlockedExchange((volatile LONG *)_target, _new)); +} + +static inline uint64_t +atomic_swap_64(volatile uint64_t *_target, uint64_t _new) +{ + return (InterlockedExchange64((volatile LONG64 *)_target, _new)); +} + +extern void *atomic_cas_ptr(volatile void *_target, void *_cmp, void *_new); + +#if defined(__clang__) +static inline uint64_t +atomic_load_64(volatile uint64_t *target) +{ + return (__atomic_load_n(target, __ATOMIC_RELAXED)); +} + +static inline void +atomic_store_64(volatile uint64_t *target, uint64_t bits) +{ + return (__atomic_store_n(target, bits, __ATOMIC_RELAXED)); +} +#endif + +static inline void +membar_producer(void) +{ + _mm_sfence(); +} + +static inline void +membar_consumer(void) +{ + _mm_lfence(); +} + +static inline void +membar_sync(void) +{ + _mm_mfence(); +} + +#ifdef __cplusplus +} +#endif + +#endif /* _SPL_ATOMIC_H */ diff --git a/include/os/windows/spl/sys/attr.h b/include/os/windows/spl/sys/attr.h new file mode 100644 index 000000000000..ec9486197e3f --- /dev/null +++ b/include/os/windows/spl/sys/attr.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_ATTR_H +#define _SPL_ATTR_H + +#endif /* SPL_ATTR_H */ diff --git a/include/os/windows/spl/sys/avl_impl.h b/include/os/windows/spl/sys/avl_impl.h new file mode 100644 index 000000000000..8c2aa34df054 --- /dev/null +++ b/include/os/windows/spl/sys/avl_impl.h @@ -0,0 +1,144 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2004 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _AVL_IMPL_H +#define _AVL_IMPL_H + + + +/* + * This is a private header file. Applications should not directly include + * this file. + */ + +#include + +#ifdef __cplusplus +extern "C" { +#endif + + +/* + * generic AVL tree implementation for kernel use + * + * There are 5 pieces of information stored for each node in an AVL tree + * + * pointer to less than child + * pointer to greater than child + * a pointer to the parent of this node + * an indication [0/1] of which child I am of my parent + * a "balance" (-1, 0, +1) indicating which child tree is taller + * + * Since they only need 3 bits, the last two fields are packed into the + * bottom bits of the parent pointer on 64 bit machines to save on space. + */ + + +/* + * for 64 bit machines, avl_pcb contains parent pointer, balance and child_index + * values packed in the following manner: + * + * |63 3| 2 |1 0 | + * |-------------------------------------|-----------------|-------------| + * | avl_parent hi order bits | avl_child_index | avl_balance | + * | | | + 1 | + * |-------------------------------------|-----------------|-------------| + * + */ +struct avl_node { + struct avl_node *avl_child[2]; /* left/right children nodes */ + uintptr_t avl_pcb; /* parent, child_index, balance */ +}; + +/* + * macros to extract/set fields in avl_pcb + * + * pointer to the parent of the current node is the high order bits + */ +#define AVL_XPARENT(n) ((struct avl_node *)((n)->avl_pcb & ~7)) +#define AVL_SETPARENT(n, p) \ + ((n)->avl_pcb = (((n)->avl_pcb & 7) | (uintptr_t)(p))) + +/* + * index of this node in its parent's avl_child[]: bit #2 + */ +#define AVL_XCHILD(n) (((n)->avl_pcb >> 2) & 1) +#define AVL_SETCHILD(n, c) \ + ((n)->avl_pcb = (uintptr_t)(((n)->avl_pcb & ~4) | ((c) << 2))) + +/* + * balance indication for a node, lowest 2 bits. A valid balance is + * -1, 0, or +1, and is encoded by adding 1 to the value to get the + * unsigned values of 0, 1, 2. + */ +#define AVL_XBALANCE(n) ((int)(((n)->avl_pcb & 3) - 1)) +#define AVL_SETBALANCE(n, b) \ + ((n)->avl_pcb = (uintptr_t)((((n)->avl_pcb & ~3) | ((b) + 1)))) + + + + +/* + * switch between a node and data pointer for a given tree + * the value of "o" is tree->avl_offset + */ +#define AVL_NODE2DATA(n, o) ((void *)((uintptr_t)(n) - (o))) +#define AVL_DATA2NODE(d, o) ((struct avl_node *)((uintptr_t)(d) + (o))) + + + +/* + * macros used to create/access an avl_index_t + */ +#define AVL_INDEX2NODE(x) ((avl_node_t *)((x) & ~1)) +#define AVL_INDEX2CHILD(x) ((x) & 1) +#define AVL_MKINDEX(n, c) ((avl_index_t)(n) | (c)) + + +/* + * The tree structure. The fields avl_root, avl_compar, and avl_offset come + * first since they are needed for avl_find(). We want them to fit into + * a single 64 byte cache line to make avl_find() as fast as possible. + */ +struct avl_tree { + struct avl_node *avl_root; /* root node in tree */ + int (*avl_compar)(const void *, const void *); + uint32_t avl_offset; /* offsetof(type, avl_link_t field) */ + ulong_t avl_numnodes; /* number of nodes in the tree */ + uint32_t avl_size; /* sizeof user type struct */ +}; + + +/* + * This will only by used via AVL_NEXT() or AVL_PREV() + */ +extern void *avl_walk(struct avl_tree *, void *, int); + +#ifdef __cplusplus +} +#endif + +#endif /* _AVL_IMPL_H */ diff --git a/include/os/windows/spl/sys/bootconf.h b/include/os/windows/spl/sys/bootconf.h new file mode 100644 index 000000000000..81619bed776a --- /dev/null +++ b/include/os/windows/spl/sys/bootconf.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_BOOTCONF_H +#define _SPL_BOOTCONF_H + +#endif /* SPL_BOOTCONF_H */ diff --git a/include/os/windows/spl/sys/bootprops.h b/include/os/windows/spl/sys/bootprops.h new file mode 100644 index 000000000000..940d0d5fdc99 --- /dev/null +++ b/include/os/windows/spl/sys/bootprops.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_BOOTPROPS_H +#define _SPL_BOOTPROPS_H + +#endif /* SPL_BOOTPROPS_H */ diff --git a/include/os/windows/spl/sys/buf.h b/include/os/windows/spl/sys/buf.h new file mode 100644 index 000000000000..e66e088efa1a --- /dev/null +++ b/include/os/windows/spl/sys/buf.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_BUF_H +#define _SPL_BUF_H + +#endif /* SPL_BUF_H */ diff --git a/include/os/windows/spl/sys/byteorder.h b/include/os/windows/spl/sys/byteorder.h new file mode 100644 index 000000000000..638d0c1cad44 --- /dev/null +++ b/include/os/windows/spl/sys/byteorder.h @@ -0,0 +1,71 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#ifndef _SPL_BYTEORDER_H +#define _SPL_BYTEORDER_H + +#include + +#define LE_16(x) (x) +#define LE_32(x) (x) +#define LE_64(x) (x) + +#if defined(_MSC_VER) && !defined(__clang__) +#include +#define BE_16(x) _byteswap_ushort(x) +#define BE_32(x) _byteswap_ulong(x) fff= +#define BE_64(x) _byteswap_uint64(x) +#else +#define BE_16(x) __builtin_bswap16(x) +#define BE_32(x) __builtin_bswap32(x) +#define BE_64(x) __builtin_bswap64(x) +#endif + +#define BE_IN8(xa) \ + *((uint8_t *)(xa)) + +#define BE_IN16(xa) \ + (((uint16_t)BE_IN8(xa) << 8) | BE_IN8((uint8_t *)(xa)+1)) + +#define BE_IN32(xa) \ + (((uint32_t)BE_IN16(xa) << 16) | BE_IN16((uint8_t *)(xa)+2)) + +#if !defined(htonll) +#define htonll(x) BE_64(x) +#endif +#if !defined(ntohll) +#define ntohll(x) BE_64(x) +#endif +#if !defined(htonl) +#define htonl(x) BE_32(x) +#endif + +// I'm going to assume windows in LE for now +#define _ZFS_LITTLE_ENDIAN + + +#endif /* SPL_BYTEORDER_H */ diff --git a/include/os/windows/spl/sys/callb.h b/include/os/windows/spl/sys/callb.h new file mode 100644 index 000000000000..5b1074dd67bc --- /dev/null +++ b/include/os/windows/spl/sys/callb.h @@ -0,0 +1,67 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SPL_CALLB_H +#define _SPL_CALLB_H + +#include + +#define CALLB_CPR_ASSERT(cp) ASSERT(MUTEX_HELD((cp)->cc_lockp)); + +typedef struct callb_cpr { + kmutex_t *cc_lockp; +} callb_cpr_t; + +#define CALLB_CPR_INIT(cp, lockp, func, name) { \ + (cp)->cc_lockp = lockp; \ +} + +#define CALLB_CPR_SAFE_BEGIN(cp) { \ + CALLB_CPR_ASSERT(cp); \ +} + +#define CALLB_CPR_SAFE_END(cp, lockp) { \ + CALLB_CPR_ASSERT(cp); \ +} + +#define CALLB_CPR_EXIT(cp) { \ + ASSERT(MUTEX_HELD((cp)->cc_lockp)); \ + mutex_exit((cp)->cc_lockp); \ +} + + +#define CALLOUT_FLAG_ROUNDUP 0x1 +#define CALLOUT_FLAG_ABSOLUTE 0x2 +#define CALLOUT_FLAG_HRESTIME 0x4 +#define CALLOUT_FLAG_32BIT 0x8 + +/* Move me to more correct "sys/callo.h" file when convenient. */ +#define CALLOUT_NORMAL 1 +typedef uint64_t callout_id_t; +callout_id_t timeout_generic(int, void (*)(void *), void *, hrtime_t, hrtime_t, + int); + + +#endif /* _SPL_CALLB_H */ diff --git a/include/os/windows/spl/sys/cmn_err.h b/include/os/windows/spl/sys/cmn_err.h new file mode 100644 index 000000000000..abf8c7ab4214 --- /dev/null +++ b/include/os/windows/spl/sys/cmn_err.h @@ -0,0 +1,60 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_CMN_ERR_H +#define _SPL_CMN_ERR_H + +#include +#include + +#define CE_CONT 0 /* continuation */ +#define CE_NOTE 1 /* notice */ +#define CE_WARN 2 /* warning */ +#define CE_PANIC 3 /* panic */ +#define CE_IGNORE 4 /* print nothing */ + +extern void cmn_err(int, const char *, ...) + __attribute__((format(printf, 2, 3))); +extern void vcmn_err(int, const char *, va_list) + __attribute__((format(printf, 2, 0))); +extern void vpanic(const char *, va_list) + __attribute__((format(printf, 1, 0), __noreturn__)); + +#define fm_panic panic + +#define cmn_err_once(ce, ...) \ +{ \ + static volatile uint32_t printed = 0; \ + if (atomic_cas_32(&printed, 0, 1) == 0) { \ + cmn_err(ce, __VA_ARGS__); \ + } \ +} + +#define vcmn_err_once(ce, fmt, ap) \ +{ \ + static volatile uint32_t printed = 0; \ + if (atomic_cas_32(&printed, 0, 1) == 0) { \ + vcmn_err(ce, fmt, ap); \ + } \ +} + +#endif /* SPL_CMN_ERR_H */ diff --git a/include/os/windows/spl/sys/compress.h b/include/os/windows/spl/sys/compress.h new file mode 100644 index 000000000000..9811518b9287 --- /dev/null +++ b/include/os/windows/spl/sys/compress.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_COMPRESS_H +#define _SPL_COMPRESS_H + +#endif /* SPL_COMPRESS_H */ diff --git a/include/os/windows/spl/sys/condvar.h b/include/os/windows/spl/sys/condvar.h new file mode 100644 index 000000000000..6c21b6f3aae3 --- /dev/null +++ b/include/os/windows/spl/sys/condvar.h @@ -0,0 +1,125 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (C) 2017 Jorgen Lundman + */ + +#ifndef SPL_CONDVAR_H +#define SPL_CONDVAR_H + +#include + +struct kmutex; + +#define hz 119 /* frequency when using gethrtime() >> 23 for lbolt */ + +typedef enum { + CV_DEFAULT, + CV_DRIVER +} kcv_type_t; + +enum { + CV_SIGNAL = 0, + CV_BROADCAST = 1, + CV_MAX_EVENTS = 2 +}; + +struct cv { + KEVENT cv_kevent[CV_MAX_EVENTS]; // signal event, broadcast event + KSPIN_LOCK cv_waiters_count_lock; + uint32_t cv_waiters_count; + uint32_t cv_initialised; // Just used as sanity +}; + +typedef struct cv kcondvar_t; + +#define PRIBIO 1 +#define PCATCH 2 + +void spl_cv_init(kcondvar_t *cvp, char *name, kcv_type_t type, void *arg); +void spl_cv_destroy(kcondvar_t *cvp); +void spl_cv_signal(kcondvar_t *cvp); +void spl_cv_broadcast(kcondvar_t *cvp); +int spl_cv_wait(kcondvar_t *cvp, struct kmutex *mp, int flags, const char *msg); +int spl_cv_timedwait(kcondvar_t *cvp, struct kmutex *mp, clock_t tim, int flags, + const char *msg); +int cv_timedwait_hires(kcondvar_t *cvp, struct kmutex *mp, + hrtime_t tim, hrtime_t res, int flag); + +/* + * Use these wrapper macros to obtain the CV variable + * name to make ZFS more gdb debugging friendly! + * This name shows up as a thread's wait_event string. + */ +#define cv_wait(cvp, mp) \ + (void) spl_cv_wait((cvp), (mp), PRIBIO, #cvp) + +/* + * Linux provides a cv_wait_io so the schedular will know why we block. + * find Windows equivalent? + */ +#define cv_wait_io(cvp, mp) \ + (void) spl_cv_wait((cvp), (mp), PRIBIO, #cvp) + +#define cv_wait_idle(cvp, mp) \ + (void) spl_cv_wait((cvp), (mp), PRIBIO, #cvp) + +#define cv_timedwait(cvp, mp, tim) \ + spl_cv_timedwait((cvp), (mp), (tim), PRIBIO, #cvp) + +#define cv_timedwait_io(cvp, mp, tim) \ + spl_cv_timedwait((cvp), (mp), (tim), PRIBIO, #cvp) + +#define cv_timedwait_idle(cvp, mp, tim) \ + spl_cv_timedwait((cvp), (mp), (tim), PRIBIO, #cvp) + +#define cv_wait_interruptible(cvp, mp) \ + (void) spl_cv_wait((cvp), (mp), PRIBIO|PCATCH, #cvp) + +#define cv_timedwait_interruptible(cvp, mp, tim) \ + spl_cv_timedwait((cvp), (mp), (tim), PRIBIO|PCATCH, #cvp) + +/* cv_wait_sig is the correct name for cv_wait_interruptible */ +#define cv_wait_sig(cvp, mp) \ + spl_cv_wait((cvp), (mp), PRIBIO|PCATCH, #cvp) + +#define cv_wait_io_sig(cvp, mp) \ + spl_cv_wait((cvp), (mp), PRIBIO|PCATCH, #cvp) + +#define cv_timedwait_sig(cvp, mp, tim) \ + spl_cv_timedwait((cvp), (mp), (tim), PRIBIO|PCATCH, #cvp) + + +#define TICK_TO_NSEC(tick) ((hrtime_t)(tick) * 1000000000 / hz) +#define cv_reltimedwait(cvp, mp, tim, type) \ + cv_timedwait_hires((cvp), (mp), TICK_TO_NSEC((tim)), 0, 0) + +#define cv_timedwait_idle_hires(cvp, mp, tim, res, flag) \ + cv_timedwait_hires(cvp, mp, tim, res, flag) + +#define cv_init spl_cv_init +#define cv_destroy spl_cv_destroy +#define cv_broadcast spl_cv_broadcast +#define cv_signal spl_cv_signal + + +#endif diff --git a/include/os/windows/spl/sys/conf.h b/include/os/windows/spl/sys/conf.h new file mode 100644 index 000000000000..6b928f14b684 --- /dev/null +++ b/include/os/windows/spl/sys/conf.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_CONF_H +#define _SPL_CONF_H + +#endif /* SPL_CONF_H */ diff --git a/include/os/windows/spl/sys/console.h b/include/os/windows/spl/sys/console.h new file mode 100644 index 000000000000..968b2a8d6656 --- /dev/null +++ b/include/os/windows/spl/sys/console.h @@ -0,0 +1,30 @@ +/* + * Copyright (C) 2007-2010 Lawrence Livermore National Security, LLC. + * Copyright (C) 2007 The Regents of the University of California. + * Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). + * Written by Brian Behlendorf . + * UCRL-CODE-235197 + * + * This file is part of the SPL, Solaris Porting Layer. + * + * The SPL is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; either version 2 of the License, or (at your + * option) any later version. + * + * The SPL is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + * + * You should have received a copy of the GNU General Public License along + * with the SPL. If not, see . + */ + +#ifndef _SPL_CONSOLE_H +#define _SPL_CONSOLE_H + +#define console_vprintf(...) +#define console_printf(...) + +#endif /* _SPL_CONSOLE_H */ diff --git a/include/os/windows/spl/sys/cpupart.h b/include/os/windows/spl/sys/cpupart.h new file mode 100644 index 000000000000..201b6e106a14 --- /dev/null +++ b/include/os/windows/spl/sys/cpupart.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_CPUPART_H +#define _SPL_CPUPART_H + +#endif /* SPL_CPUPART_H */ diff --git a/include/os/windows/spl/sys/cpuvar.h b/include/os/windows/spl/sys/cpuvar.h new file mode 100644 index 000000000000..4f6d25073c51 --- /dev/null +++ b/include/os/windows/spl/sys/cpuvar.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_CPUVAR_H +#define _SPL_CPUVAR_H + +#endif /* SPL_CPUVAR_H */ diff --git a/include/os/windows/spl/sys/crc32.h b/include/os/windows/spl/sys/crc32.h new file mode 100644 index 000000000000..32169ed9e339 --- /dev/null +++ b/include/os/windows/spl/sys/crc32.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_CRC32_H +#define _SPL_CRC32_H + +#endif /* SPL_CRC32_H */ diff --git a/include/os/windows/spl/sys/cred.h b/include/os/windows/spl/sys/cred.h new file mode 100644 index 000000000000..e7750fe8dfc1 --- /dev/null +++ b/include/os/windows/spl/sys/cred.h @@ -0,0 +1,59 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#ifndef _SPL_CRED_H +#define _SPL_CRED_H + +#include +#include + +struct ucred; // fixme +typedef struct ucred cred_t; + +#define kcred (cred_t *)NULL +#define CRED() (cred_t *)NULL +#define KUID_TO_SUID(x) (x) +#define KGID_TO_SGID(x) (x) + +extern void crhold(cred_t *cr); +extern void crfree(cred_t *cr); +extern uid_t crgetuid(const cred_t *cr); +extern uid_t crgetruid(const cred_t *cr); +extern uid_t crgetsuid(const cred_t *cr); +extern uid_t crgetfsuid(const cred_t *cr); +extern gid_t crgetgid(const cred_t *cr); +extern gid_t crgetrgid(const cred_t *cr); +extern gid_t crgetsgid(const cred_t *cr); +extern gid_t crgetfsgid(const cred_t *cr); +extern int crgetngroups(const cred_t *cr); +extern gid_t *crgetgroups(const cred_t *cr); +extern void crgetgroupsfree(gid_t *gids); +extern int spl_cred_ismember_gid(cred_t *cr, gid_t gid); + +#define crgetsid(cred, i) (NULL) + +#endif /* _SPL_CRED_H */ diff --git a/include/os/windows/spl/sys/ctype.h b/include/os/windows/spl/sys/ctype.h new file mode 100644 index 000000000000..d71cddea228e --- /dev/null +++ b/include/os/windows/spl/sys/ctype.h @@ -0,0 +1,27 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#ifndef _SPL_CTYPE_H +#define _SPL_CTYPE_H + +#define iscntrl(C) (uchar(C) <= 0x1f || uchar(C) == 0x7f) + +#endif /* SPL_CTYPE_H */ diff --git a/include/os/windows/spl/sys/ddi.h b/include/os/windows/spl/sys/ddi.h new file mode 100644 index 000000000000..0ecb2e63aa28 --- /dev/null +++ b/include/os/windows/spl/sys/ddi.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_DDI_H +#define _SPL_DDI_H + +#endif /* SPL_DDI_H */ diff --git a/include/os/windows/spl/sys/debug.h b/include/os/windows/spl/sys/debug.h new file mode 100644 index 000000000000..b10238886272 --- /dev/null +++ b/include/os/windows/spl/sys/debug.h @@ -0,0 +1,251 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Copyright (c) 2012 by Delphix. All rights reserved. + */ + +/* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ + + +/* + * Available Solaris debug functions. All of the ASSERT() macros will be + * compiled out when NDEBUG is defined, this is the default behavior for + * the SPL. To enable assertions use the --enable-debug with configure. + * The VERIFY() functions are never compiled out and cannot be disabled. + * + * PANIC() - Panic the node and print message. + * ASSERT() - Assert X is true, if not panic. + * ASSERTF() - Assert X is true, if not panic and print message. + * ASSERTV() - Wraps a variable declaration which is only used by ASSERT(). + * ASSERT3S() - Assert signed X OP Y is true, if not panic. + * ASSERT3U() - Assert unsigned X OP Y is true, if not panic. + * ASSERT3P() - Assert pointer X OP Y is true, if not panic. + * ASSERT0() - Assert value is zero, if not panic. + * VERIFY() - Verify X is true, if not panic. + * VERIFY3S() - Verify signed X OP Y is true, if not panic. + * VERIFY3U() - Verify unsigned X OP Y is true, if not panic. + * VERIFY3P() - Verify pointer X OP Y is true, if not panic. + * VERIFY0() - Verify value is zero, if not panic. + */ + +#ifndef _SPL_DEBUG_H +#define _SPL_DEBUG_H + +#include +#include + +#if defined(_MSC_VER) && !defined(__clang__) + +#define unlikely(X) X +#define likely(X) X +#define __maybe_unused [[maybe_unused]] +#define __printflike(X, Y) +#define __unused [[maybe_unused]] +#define always_inline __forceinline +#define _Noreturn __declspec(noreturn) + +#else + +#define try __try +#define except __except + +#ifndef expect +#define expect(expr, value) (__builtin_expect((expr), (value))) +#endif +#define likely(x) __builtin_expect(!!(x), 1) +#define unlikely(x) __builtin_expect(!!(x), 0) + +#ifndef __maybe_unused +#define __maybe_unused __attribute__((unused)) +#endif +#define __printflike(a, b) __attribute__((__format__(__printf__, a, b))) + +#define __unused __attribute__((unused)) +#define _Noreturn __attribute__((__noreturn__)) + +#endif + +extern void _Noreturn panic(const char *fmt, ...); + + + +extern void printBuffer(const char *fmt, ...); + +#define LUDICROUS_SPEED // use circular buffer +// xprintf is always printed +// dprintf is printed in DEBUG builds +// IOLog is printed in DEBUG builds (legacy from osx) +// +#ifdef DBG /* Debugging Disabled */ +#ifdef LUDICROUS_SPEED +#define dprintf(...) printBuffer(__VA_ARGS__) +#define IOLog(...) printBuffer(__VA_ARGS__) +#define xprintf(...) KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, \ + __VA_ARGS__)) +#define TraceEvent(x, ...) + +#else // LUDICROUS_SPEED + +#undef KdPrintEx +#define KdPrintEx(_x_) DbgPrintEx _x_ +#define dprintf(...) KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, \ + __VA_ARGS__)) +#define IOLog(...) KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, \ + __VA_ARGS__)) +#define xprintf(...) KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, \ + __VA_ARGS__)) +#define TraceEvent(level, ...) KdPrintEx((DPFLTR_IHVDRIVER_ID, level, \ + __VA_ARGS__)) + +#endif // LUDICROUS_SPEED + +#define PANIC(fmt, ...) \ + do { \ + xprintf(fmt, __VA_ARGS__); \ + DbgBreakPoint(); \ + } while (0) + +#else // DBG + +#define TraceEvent(x, ...) +#define xprintf(...) DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, \ + __VA_ARGS__) +#define dprintf(...) +#define IOLog(...) +#define PANIC(fmt, ...) \ + do { \ + xprintf(fmt, __VA_ARGS__); \ + } while (0) +#endif + +#ifdef DBG /* Debugging Disabled */ + +/* Define SPL_DEBUG_STR to make clear which ASSERT definitions are used */ +#define SPL_DEBUG_STR " (DEBUG mode)" + +/* ASSERTION that is safe to use within the debug system */ +#define __ASSERT(cond) \ + do { \ + if (unlikely(!(cond))) { \ + printk(KERN_EMERG "ASSERTION(" #cond ") failed\n"); \ + BUG(); \ + } \ + } while (0) + +#define ASSERTF(cond, fmt, ...) \ + do { \ + if (unlikely(!(cond))) \ + PANIC("ASSERTION(" #cond ") failed: " fmt, \ + __VA_ARGS__); \ + } while (0) + +#define ASSERT3B(x, y, z) VERIFY3B(x, y, z) +#define ASSERT3S(x, y, z) VERIFY3S(x, y, z) +#define ASSERT3U(x, y, z) VERIFY3U(x, y, z) +#define ASSERT3P(x, y, z) VERIFY3P(x, y, z) +#define ASSERT0(x) VERIFY0(x) + +#define ASSERTV(x) x + +#ifndef ZFS_DEBUG +#define ZFS_DEBUG 1 +#endif + +#else /* Debugging Enabled */ + +/* Define SPL_DEBUG_STR to make clear which ASSERT definitions are used */ +#define SPL_DEBUG_STR "" + +#define __ASSERT(x) ((void)0) +#define ASSERTF(x, y, z, ...) ((void)0) +#define ASSERTV(x) + +#define ASSERT3B(x, y, z) ((void)0) +#define ASSERT3S(x, y, z) ((void)0) +#define ASSERT3U(x, y, z) ((void)0) +#define ASSERT3P(x, y, z) ((void)0) +#define ASSERT0(x) ((void)0) + +#endif /* DBG */ + +#define VERIFY3_IMPL(LEFT, OP, RIGHT, TYPE, FMT, CAST) \ + do { \ + TYPE _verify3_left = (TYPE)(LEFT); \ + TYPE _verify3_right = (TYPE)(RIGHT); \ + if (!(_verify3_left OP _verify3_right)) \ + PANIC("VERIFY3( %s " #OP " %s ) " \ + "failed (" FMT " " #OP " " FMT \ + ")\n", #LEFT, #RIGHT, \ + CAST(_verify3_left), CAST(_verify3_right)); \ + } while (0) + +#define VERIFY3B(x, y, z) VERIFY3_IMPL(x, y, z, int64_t, "%lld", (boolean_t)) +#define VERIFY3S(x, y, z) VERIFY3_IMPL(x, y, z, int64_t, "%lld", (long long)) +#define VERIFY3U(x, y, z) VERIFY3_IMPL(x, y, z, uint64_t, "%llu", \ + (unsigned long long)) +#define VERIFY3P(x, y, z) VERIFY3_IMPL(x, y, z, uintptr_t, "%p", (void *)) +#define VERIFY0(x) VERIFY3_IMPL(0, ==, x, int64_t, "%lld", (long long)) + +#define VERIFY(EX) do { \ + if (!(EX)) panic("PANIC: %s %s:%d\n", #EX, __FILE__, \ + __LINE__); \ + } while (0) + +/* + * IMPLY and EQUIV are assertions of the form: + * + * if (a) then (b) + * and + * if (a) then (b) *AND* if (b) then (a) + */ +#if DEBAG +#define IMPLY(A, B) \ + ((void)(((!(A)) || (B)) || \ + panic("(" #A ") implies (" #B ")", __FILE__, __LINE__))) +#define EQUIV(A, B) \ + ((void)((!!(A) == !!(B)) || \ + panic("(" #A ") is equivalent to (" #B ")", __FILE__, \ + __LINE__))) +#else +#define IMPLY(A, B) ((void)0) +#define EQUIV(A, B) ((void)0) +#endif + + +/* + * Compile-time assertion. The condition 'x' must be constant. + */ +#define CTASSERT_GLOBAL(x) _CTASSERT(x, __LINE__) +#define CTASSERT(x) { _CTASSERT(x, __LINE__); } +#define _CTASSERT(x, y) __CTASSERT(x, y) +#define __CTASSERT(x, y) \ + typedef char __attribute__((unused)) \ + __compile_time_assertion__ ## y[(x) ? 1 : -1] + +#define zfs_fallthrough __attribute__((__fallthrough__)) + +#endif /* SPL_DEBUG_H */ diff --git a/include/os/windows/spl/sys/dirent.h b/include/os/windows/spl/sys/dirent.h new file mode 100644 index 000000000000..4a8a08392443 --- /dev/null +++ b/include/os/windows/spl/sys/dirent.h @@ -0,0 +1,57 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_DIRENT_H +#define _SPL_DIRENT_H + +#include + +#define MAXNAMLEN 255 + +/* + * File types + */ +#define DT_UNKNOWN 0 +#define DT_FIFO 1 +#define DT_CHR 2 +#define DT_DIR 4 +#define DT_BLK 6 +#define DT_REG 8 +#define DT_LNK 10 +#define DT_SOCK 12 +#define DT_WHT 14 + +struct dirent { + uint64_t d_ino; /* file number of entry */ + uint64_t d_seekoff; /* seek offset (optional, used by servers) */ + uint16_t d_reclen; /* length of this record */ + uint16_t d_namlen; /* length of string in d_name */ + uint8_t d_type; /* file type, see below */ + char d_name[MAXPATHLEN]; /* entry name (up to MAXPATHLEN bytes) */ +}; + +#ifndef IFTODT +#define IFTODT(mode) (((mode) & 0170000) >> 12) +#endif +#define DTTOIF(dirtype) ((dirtype) << 12) + + +#endif /* SPL_DIRENT_H */ diff --git a/include/os/windows/spl/sys/disp.h b/include/os/windows/spl/sys/disp.h new file mode 100644 index 000000000000..4011902d2df9 --- /dev/null +++ b/include/os/windows/spl/sys/disp.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_DISP_H +#define _SPL_DISP_H + +#endif /* SPL_DISP_H */ diff --git a/include/os/windows/spl/sys/dkio.h b/include/os/windows/spl/sys/dkio.h new file mode 100644 index 000000000000..731c1f753895 --- /dev/null +++ b/include/os/windows/spl/sys/dkio.h @@ -0,0 +1,15 @@ + +#ifndef _SPL_DKIO_H +#define _SPL_DKIO_H + +struct dk_callback { + void (*dkc_callback)(void *dkc_cookie, int error); + void *dkc_cookie; + int dkc_flag; +}; + +#define DKIOC (0x04 << 8) +#define DKIOCFLUSHWRITECACHE (DKIOC | 34) +#define DKIOCTRIM (DKIOC | 35) + +#endif /* _SPL_DKIO_H */ diff --git a/include/os/windows/spl/sys/dklabel.h b/include/os/windows/spl/sys/dklabel.h new file mode 100644 index 000000000000..e70e88c1fb82 --- /dev/null +++ b/include/os/windows/spl/sys/dklabel.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_DKLABEL_H +#define _SPL_DKLABEL_H + +#endif /* _SPL_DKLABEL_H */ diff --git a/include/os/windows/spl/sys/dnlc.h b/include/os/windows/spl/sys/dnlc.h new file mode 100644 index 000000000000..8c542d6bc7ee --- /dev/null +++ b/include/os/windows/spl/sys/dnlc.h @@ -0,0 +1,38 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_DNLC_H +#define _SPL_DNLC_H + +/* + * Reduce the dcache and icache then reap the free'd slabs. Note the + * interface takes a reclaim percentage but we don't have easy access to + * the total number of entries to calculate the reclaim count. However, + * in practice this doesn't need to be even close to correct. We simply + * need to reclaim some useful fraction of the cache. The caller can + * determine if more needs to be done. + */ +static inline void +dnlc_reduce_cache(void *reduce_percent) +{ +} + +#endif /* SPL_DNLC_H */ diff --git a/include/os/windows/spl/sys/dumphdr.h b/include/os/windows/spl/sys/dumphdr.h new file mode 100644 index 000000000000..76cb792f9db5 --- /dev/null +++ b/include/os/windows/spl/sys/dumphdr.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_DUMPHDR_H +#define _SPL_DUMPHDR_H + +#endif /* SPL_DUMPHDR_H */ diff --git a/include/os/windows/spl/sys/errno.h b/include/os/windows/spl/sys/errno.h new file mode 100644 index 000000000000..a28a19ff7166 --- /dev/null +++ b/include/os/windows/spl/sys/errno.h @@ -0,0 +1,160 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2000 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ + +/* + * University Copyright- Copyright (c) 1982, 1986, 1988 + * The Regents of the University of California + * All Rights Reserved + * + * University Acknowledgment- Portions of this document are derived from + * software developed by the University of California, Berkeley, and its + * contributors. + */ + +#ifndef _SYS_ERRNO_H +#define _SYS_ERRNO_H + + +#ifdef __cplusplus +extern "C" { +#endif + +#ifdef _KERNEL +#define ERESTART (-1) /* restart syscall */ +#define EJUSTRETURN (-2) /* don't modify regs, just return */ +#endif + + +/* + * Error codes + */ + +#define EPERM 1 /* Not super-user */ +#define ENOENT 2 /* No such file or directory */ +#define ESRCH 3 /* No such process */ +#define EINTR 4 /* interrupted system call */ +#define EIO 5 /* I/O error */ +#define ENXIO 6 /* No such device or address */ +#define E2BIG 7 /* Arg list too long */ +#define ENOEXEC 8 /* Exec format error */ +#define EBADF 9 /* Bad file number */ +#define ECHILD 10 /* No children */ +#define EAGAIN 11 /* Resource temporarily unavailable */ +#define ENOMEM 12 /* Not enough core */ +#define EACCES 13 /* Permission denied */ +#define EFAULT 14 /* Bad address */ +#define ENOTBLK 15 /* Block device required */ +#define EBUSY 16 /* Mount device busy */ +#define EEXIST 17 /* File exists */ +#define EXDEV 18 /* Cross-device link */ +#define ENODEV 19 /* No such device */ +#define ENOTDIR 20 /* Not a directory */ +#define EISDIR 21 /* Is a directory */ +#define EINVAL 22 /* Invalid argument */ +#define ENFILE 23 /* File table overflow */ +#define EMFILE 24 /* Too many open files */ +#define ENOTTY 25 /* Inappropriate ioctl for device */ +#define EFBIG 27 /* File too large */ +#define ENOSPC 28 /* No space left on device */ +#define ESPIPE 29 /* Illegal seek */ +#define EROFS 30 /* Read only file system */ +#define EMLINK 31 /* Too many links */ +#define EPIPE 32 /* Broken pipe */ +#define EDOM 33 /* Math arg out of domain of func */ +#define ERANGE 34 /* Math result not representable */ + +#define EDEADLK 36 +#define ENAMETOOLONG 38 +#define ENOLCK 39 +#define ENOSYS 40 +#define ENOTEMPTY 41 +#define EILSEQ 42 /* Illegal byte sequence */ + +#define EDQUOT 49 /* Disc quota exceeded */ +#define EBADE 50 /* invalid exchange */ +#define ECKSUM EBADE + +#define ESHUTDOWN 58 /* Can't send after socket shutdown */ +#define ESTALE 70 /* Stale NFS file handle */ + +#ifndef _KERNEL +#define ERESTART 85 /* Interrupted system call should be restarted */ +#endif + +#define EADDRINUSE 100 +#define EADDRNOTAVAIL 101 +#define EAFNOSUPPORT 102 +#define EALREADY 103 +#define EBADMSG 104 +#define ECANCELED 105 +#define ECONNABORTED 106 +#define ECONNREFUSED 107 +#define ECONNRESET 108 +#define EDESTADDRREQ 109 +#define EHOSTUNREACH 110 +#define EIDRM 111 +#define EINPROGRESS 112 +#define EISCONN 113 +#define ELOOP 114 +#define EMSGSIZE 115 +#define ENETDOWN 116 +#define ENETRESET 117 +#define ENETUNREACH 118 +#define ENOBUFS 119 +#define ENODATA 120 +#define ENOLINK 121 +#define ENOMSG 122 +#define ENOPROTOOPT 123 +#define ENOSR 124 +#define ENOSTR 125 +#define ENOTCONN 126 +#define ENOTRECOVERABLE 127 +#define ENOTSOCK 128 +#define ENOTSUP 129 +#define EOPNOTSUPP 130 +#define EOTHER 131 +#define EOVERFLOW 132 +#define EOWNERDEAD 133 +#define EPROTO 134 +#define EPROTONOSUPPORT 135 +#define EPROTOTYPE 136 +#define ETIME 137 +#define ETIMEDOUT 138 +#define ETXTBSY 139 +#define EWOULDBLOCK 140 + +#define ENOTACTIVE 142 +#define ECHRNG 143 +#define EREMOTEIO 144 + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_ERRNO_H */ diff --git a/include/os/windows/spl/sys/extdirent.h b/include/os/windows/spl/sys/extdirent.h new file mode 100644 index 000000000000..aa2278f52fa3 --- /dev/null +++ b/include/os/windows/spl/sys/extdirent.h @@ -0,0 +1,7 @@ + +#ifndef _SPL_EXTDIRENT_H +#define _SPL_EXTDIRENT_H + +#define ED_CASE_CONFLICT 0x10 + +#endif /* _SPL_EXTDIRENT_H */ diff --git a/include/os/windows/spl/sys/fcntl.h b/include/os/windows/spl/sys/fcntl.h new file mode 100644 index 000000000000..d57c873b59cc --- /dev/null +++ b/include/os/windows/spl/sys/fcntl.h @@ -0,0 +1,61 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_FCNTL_H +#define _SPL_FCNTL_H + +#include + +#define _CRT_DECLARE_NONSTDC_NAMES 1 +#include + +#define F_FREESP 11 + +#define O_LARGEFILE 0 +#define O_RSYNC 0 +#define O_DIRECT 0 +#define O_SYNC 0 +#define O_DSYNC 0 +#define O_CLOEXEC 0 +#define O_NDELAY 0 + +#define F_RDLCK 1 /* shared or read lock */ +#define F_UNLCK 2 /* unlock */ +#define F_WRLCK 3 /* exclusive or write lock */ +#ifdef KERNEL +#define F_WAIT 0x010 /* Wait until lock is granted */ +#define F_FLOCK 0x020 /* Use flock(2) semantics for lock */ +#define F_POSIX 0x040 /* Use POSIX semantics for lock */ +#define F_PROV 0x080 /* Non-coalesced provisional lock */ +#define F_WAKE1_SAFE 0x100 /* its safe to only wake one waiter */ +#define F_ABORT 0x200 /* lock attempt aborted (force umount) */ +#define F_OFD_LOCK 0x400 /* Use "OFD" semantics for lock */ +#endif + +struct flock { + off_t l_start; /* starting offset */ + off_t l_len; /* len = 0 means until end of file */ + pid_t l_pid; /* lock owner */ + short l_type; /* lock type: read/write, etc. */ + short l_whence; /* type of l_start */ +}; + +#endif /* _SPL_FCNTL_H */ diff --git a/include/os/windows/spl/sys/file.h b/include/os/windows/spl/sys/file.h new file mode 100644 index 000000000000..38a05a311692 --- /dev/null +++ b/include/os/windows/spl/sys/file.h @@ -0,0 +1,54 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_FILE_H +#define _SPL_FILE_H + +#define FIGNORECASE 0x00080000 +#define FKIOCTL 0x80000000 +#define FCOPYSTR 0x40000000 + +#include + +struct spl_fileproc { + void *f_vnode; + list_node_t f_next; + uint64_t f_fd; + uint64_t f_offset; + void *f_proc; + void *f_fp; + int f_writes; + uint64_t f_file; + HANDLE f_handle; + void *f_fileobject; + void *f_deviceobject; +}; + +#define file_t struct spl_fileproc + +void *getf(uint64_t fd); +void releasef(uint64_t fd); +void releasefp(struct spl_fileproc *fp); + +/* O3X extended - get vnode from previos getf() */ +struct vnode *getf_vnode(void *fp); + +#endif /* SPL_FILE_H */ diff --git a/include/os/windows/spl/sys/ia32/asm_linkage.h b/include/os/windows/spl/sys/ia32/asm_linkage.h new file mode 100644 index 000000000000..081157dd0a0b --- /dev/null +++ b/include/os/windows/spl/sys/ia32/asm_linkage.h @@ -0,0 +1,181 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2008 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _IA32_SYS_ASM_LINKAGE_H +#define _IA32_SYS_ASM_LINKAGE_H + +#define RET ret + +/* Tell compiler to call assembler like Unix */ +#undef ASMABI +#define ASMABI __attribute__((sysv_abi)) + +#define ENDBR + +#define SECTION_TEXT .text +#define SECTION_STATIC .data + +#ifdef __cplusplus +extern "C" { +#endif + +#ifdef _ASM /* The remainder of this file is only for assembly files */ + + +/* + * make annoying differences in assembler syntax go away + */ + +/* + * D16 and A16 are used to insert instructions prefixes; the + * macros help the assembler code be slightly more portable. + */ +#if !defined(__GNUC_AS__) +/* + * /usr/ccs/bin/as prefixes are parsed as separate instructions + */ +#define D16 data16; +#define A16 addr16; + +/* + * (There are some weird constructs in constant expressions) + */ +#define _CONST(const) [const] +#define _BITNOT(const) -1!_CONST(const) +#define _MUL(a, b) _CONST(a \* b) + +#else +/* + * Why not use the 'data16' and 'addr16' prefixes .. well, the + * assembler doesn't quite believe in real mode, and thus argues with + * us about what we're trying to do. + */ +#define D16 .byte 0x66; +#define A16 .byte 0x67; + +#define _CONST(const) (const) +#define _BITNOT(const) ~_CONST(const) +#define _MUL(a, b) _CONST(a * b) + +#endif + +/* + * C pointers are different sizes between i386 and amd64. + * These constants can be used to compute offsets into pointer arrays. + */ +#if defined(__amd64) +#define CLONGSHIFT 3 +#define CLONGSIZE 8 +#define CLONGMASK 7 +#elif defined(__i386) +#define CLONGSHIFT 2 +#define CLONGSIZE 4 +#define CLONGMASK 3 +#endif + +/* + * Since we know we're either ILP32 or LP64 .. + */ +#define CPTRSHIFT CLONGSHIFT +#define CPTRSIZE CLONGSIZE +#define CPTRMASK CLONGMASK + +#if CPTRSIZE != (1 << CPTRSHIFT) || CLONGSIZE != (1 << CLONGSHIFT) +#error "inconsistent shift constants" +#endif + +#if CPTRMASK != (CPTRSIZE - 1) || CLONGMASK != (CLONGSIZE - 1) +#error "inconsistent mask constants" +#endif + +#define ASM_ENTRY_ALIGN 16 + +/* + * SSE register alignment and save areas + */ + +#define XMM_SIZE 16 +#define XMM_ALIGN 16 + +/* + * ENTRY provides the standard procedure entry code and an easy way to + * insert the calls to mcount for profiling. ENTRY_NP is identical, but + * never calls mcount. + */ +#define ENTRY(x) \ + .text; \ + .balign ASM_ENTRY_ALIGN; \ + .globl x; \ +x: MCOUNT(x) + +#define ENTRY_NP(x) \ + .text; \ + .balign ASM_ENTRY_ALIGN; \ + .globl x; \ +x: + +#define ENTRY_ALIGN(x, a) \ + .text; \ + .balign a; \ + .globl x; \ +x: + +#define FUNCTION(x) \ +x: + +/* + * ENTRY2 is identical to ENTRY but provides two labels for the entry point. + */ +#define ENTRY2(x, y) \ + .text; \ + .align ASM_ENTRY_ALIGN; \ + .globl x, y; \ +x:; \ +y: MCOUNT(x) + +#define ENTRY_NP2(x, y) \ + .text; \ + .align ASM_ENTRY_ALIGN; \ + .globl x, y; \ +x:; \ +y: + + +/* + * SET_SIZE trails a function and set the size for the ELF symbol table. + */ +#define SET_SIZE(x) + +#define SET_OBJ(x) + + +#endif /* _ASM */ + +#ifdef __cplusplus +} +#endif + +#endif /* _IA32_SYS_ASM_LINKAGE_H */ diff --git a/include/os/windows/spl/sys/idmap.h b/include/os/windows/spl/sys/idmap.h new file mode 100644 index 000000000000..80678fa8fb47 --- /dev/null +++ b/include/os/windows/spl/sys/idmap.h @@ -0,0 +1,27 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_IDMAP_H +#define _SPL_IDMAP_H + +#define IDMAP_WK_CREATOR_OWNER_UID 2147483648U + +#endif /* SPL_IDMAP_H */ diff --git a/include/os/windows/spl/sys/int_limits.h b/include/os/windows/spl/sys/int_limits.h new file mode 100644 index 000000000000..81f5caf9eaf6 --- /dev/null +++ b/include/os/windows/spl/sys/int_limits.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_INT_LIMITS_H +#define _SPL_INT_LIMITS_H + +#endif /* SPL_INT_LIMITS_H */ diff --git a/include/os/windows/spl/sys/int_types.h b/include/os/windows/spl/sys/int_types.h new file mode 100644 index 000000000000..1a04d92dfdf3 --- /dev/null +++ b/include/os/windows/spl/sys/int_types.h @@ -0,0 +1,27 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_INT_TYPES_H +#define _SPL_INT_TYPES_H + +#include + +#endif /* SPL_INT_TYPES_H */ diff --git a/include/os/windows/spl/sys/inttypes.h b/include/os/windows/spl/sys/inttypes.h new file mode 100644 index 000000000000..59d8efdb1edf --- /dev/null +++ b/include/os/windows/spl/sys/inttypes.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_INTTYPES_H +#define _SPL_INTTYPES_H + +#endif /* SPL_INTTYPES_H */ diff --git a/include/os/windows/spl/sys/ioctl.h b/include/os/windows/spl/sys/ioctl.h new file mode 100644 index 000000000000..61b0d9db35dc --- /dev/null +++ b/include/os/windows/spl/sys/ioctl.h @@ -0,0 +1,30 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. + */ + +#ifndef _SYS_IOCTL_H +#define _SYS_IOCTL_H + + +#endif /* _LIBSPL_SYS_STAT_H */ diff --git a/include/os/windows/spl/sys/isa_defs.h b/include/os/windows/spl/sys/isa_defs.h new file mode 100644 index 000000000000..ef50b166f2e8 --- /dev/null +++ b/include/os/windows/spl/sys/isa_defs.h @@ -0,0 +1,689 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2008 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SYS_ISA_DEFS_H +#define _SYS_ISA_DEFS_H + +/* + * This header file serves to group a set of well known defines and to + * set these for each instruction set architecture. These defines may + * be divided into two groups; characteristics of the processor and + * implementation choices for Solaris on a processor. + * + * Processor Characteristics: + * + * _LITTLE_ENDIAN / _BIG_ENDIAN: + * The natural byte order of the processor. A pointer to an int points + * to the least/most significant byte of that int. + * + * _STACK_GROWS_UPWARD / _STACK_GROWS_DOWNWARD: + * The processor specific direction of stack growth. A push onto the + * stack increases/decreases the stack pointer, so it stores data at + * successively higher/lower addresses. (Stackless machines ignored + * without regrets). + * + * _LONG_LONG_HTOL / _LONG_LONG_LTOH: + * A pointer to a long long points to the most/least significant long + * within that long long. + * + * _BIT_FIELDS_HTOL / _BIT_FIELDS_LTOH: + * The C compiler assigns bit fields from the high/low to the low/high end + * of an int (most to least significant vs. least to most significant). + * + * _IEEE_754: + * The processor (or supported implementations of the processor) + * supports the ieee-754 floating point standard. No other floating + * point standards are supported (or significant). Any other supported + * floating point formats are expected to be cased on the ISA processor + * symbol. + * + * _CHAR_IS_UNSIGNED / _CHAR_IS_SIGNED: + * The C Compiler implements objects of type `char' as `unsigned' or + * `signed' respectively. This is really an implementation choice of + * the compiler writer, but it is specified in the ABI and tends to + * be uniform across compilers for an instruction set architecture. + * Hence, it has the properties of a processor characteristic. + * + * _CHAR_ALIGNMENT / _SHORT_ALIGNMENT / _INT_ALIGNMENT / _LONG_ALIGNMENT / + * _LONG_LONG_ALIGNMENT / _DOUBLE_ALIGNMENT / _LONG_DOUBLE_ALIGNMENT / + * _POINTER_ALIGNMENT / _FLOAT_ALIGNMENT: + * The ABI defines alignment requirements of each of the primitive + * object types. Some, if not all, may be hardware requirements as + * well. The values are expressed in "byte-alignment" units. + * + * _MAX_ALIGNMENT: + * The most stringent alignment requirement as specified by the ABI. + * Equal to the maximum of all the above _XXX_ALIGNMENT values. + * + * _ALIGNMENT_REQUIRED: + * True or false (1 or 0) whether or not the hardware requires the ABI + * alignment. + * + * _LONG_LONG_ALIGNMENT_32 + * The 32-bit ABI supported by a 64-bit kernel may have different + * alignment requirements for primitive object types. The value of this + * identifier is expressed in "byte-alignment" units. + * + * _HAVE_CPUID_INSN + * This indicates that the architecture supports the 'cpuid' + * instruction as defined by Intel. (Intel allows other vendors + * to extend the instruction for their own purposes.) + * + * + * Implementation Choices: + * + * _ILP32 / _LP64: + * This specifies the compiler data type implementation as specified in + * the relevant ABI. The choice between these is strongly influenced + * by the underlying hardware, but is not absolutely tied to it. + * Currently only two data type models are supported: + * + * _ILP32: + * Int/Long/Pointer are 32 bits. This is the historical UNIX + * and Solaris implementation. Due to its historical standing, + * this is the default case. + * + * _LP64: + * Long/Pointer are 64 bits, Int is 32 bits. This is the chosen + * implementation for 64-bit ABIs such as SPARC V9. + * + * _I32LPx: + * A compilation environment where 'int' is 32-bit, and + * longs and pointers are simply the same size. + * + * In all cases, Char is 8 bits and Short is 16 bits. + * + * _SUNOS_VTOC_8 / _SUNOS_VTOC_16 / _SVR4_VTOC_16: + * This specifies the form of the disk VTOC (or label): + * + * _SUNOS_VTOC_8: + * This is a VTOC form which is upwardly compatible with the + * SunOS 4.x disk label and allows 8 partitions per disk. + * + * _SUNOS_VTOC_16: + * In this format the incore vtoc image matches the ondisk + * version. It allows 16 slices per disk, and is not + * compatible with the SunOS 4.x disk label. + * + * Note that these are not the only two VTOC forms possible and + * additional forms may be added. One possible form would be the + * SVr4 VTOC form. The symbol for that is reserved now, although + * it is not implemented. + * + * _SVR4_VTOC_16: + * This VTOC form is compatible with the System V Release 4 + * VTOC (as implemented on the SVr4 Intel and 3b ports) with + * 16 partitions per disk. + * + * + * _DMA_USES_PHYSADDR / _DMA_USES_VIRTADDR + * This describes the type of addresses used by system DMA: + * + * _DMA_USES_PHYSADDR: + * This type of DMA, used in the x86 implementation, + * requires physical addresses for DMA buffers. The 24-bit + * addresses used by some legacy boards is the source of the + * "low-memory" (<16MB) requirement for some devices using DMA. + * + * _DMA_USES_VIRTADDR: + * This method of DMA allows the use of virtual addresses for + * DMA transfers. + * + * _FIRMWARE_NEEDS_FDISK / _NO_FDISK_PRESENT + * This indicates the presence/absence of an fdisk table. + * + * _FIRMWARE_NEEDS_FDISK + * The fdisk table is required by system firmware. If present, + * it allows a disk to be subdivided into multiple fdisk + * partitions, each of which is equivalent to a separate, + * virtual disk. This enables the co-existence of multiple + * operating systems on a shared hard disk. + * + * _NO_FDISK_PRESENT + * If the fdisk table is absent, it is assumed that the entire + * media is allocated for a single operating system. + * + * _HAVE_TEM_FIRMWARE + * Defined if this architecture has the (fallback) option of + * using prom_* calls for doing I/O if a suitable kernel driver + * is not available to do it. + * + * _DONT_USE_1275_GENERIC_NAMES + * Controls whether or not device tree node names should + * comply with the IEEE 1275 "Generic Names" Recommended + * Practice. With _DONT_USE_GENERIC_NAMES, device-specific + * names identifying the particular device will be used. + * + * __i386_COMPAT + * This indicates whether the i386 ABI is supported as a *non-native* + * mode for the platform. When this symbol is defined: + * - 32-bit xstat-style system calls are enabled + * - 32-bit xmknod-style system calls are enabled + * - 32-bit system calls use i386 sizes -and- alignments + * + * Note that this is NOT defined for the i386 native environment! + * + * __x86 + * This is ONLY a synonym for defined(__i386) || defined(__amd64) + * which is useful only insofar as these two architectures share + * common attributes. Analogous to __sparc. + * + * _PSM_MODULES + * This indicates whether or not the implementation uses PSM + * modules for processor support, reading /etc/mach from inside + * the kernel to extract a list. + * + * _RTC_CONFIG + * This indicates whether or not the implementation uses /etc/rtc_config + * to configure the real-time clock in the kernel. + * + * _UNIX_KRTLD + * This indicates that the implementation uses a dynamically + * linked unix + krtld to form the core kernel image at boot + * time, or (in the absence of this symbol) a prelinked kernel image. + * + * _OBP + * This indicates the firmware interface is OBP. + * + * _SOFT_HOSTID + * This indicates that the implementation obtains the hostid + * from the file /etc/hostid, rather than from hardware. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +/* + * The following set of definitions characterize Solaris on AMD's + * 64-bit systems. + */ +#if defined(__x86_64) || defined(__amd64) + +#if !defined(__amd64) +#define __amd64 /* preferred guard */ +#endif + +#if !defined(__x86) +#define __x86 +#endif + +/* + * Define the appropriate "processor characteristics" + */ +#define _STACK_GROWS_DOWNWARD +#define _LONG_LONG_LTOH +#define _BIT_FIELDS_LTOH +#define _IEEE_754 +#define _CHAR_IS_SIGNED +#define _BOOL_ALIGNMENT 1 +#define _CHAR_ALIGNMENT 1 +#define _SHORT_ALIGNMENT 2 +#define _INT_ALIGNMENT 4 +#define _FLOAT_ALIGNMENT 4 +#define _FLOAT_COMPLEX_ALIGNMENT 4 +#define _LONG_ALIGNMENT 8 +#define _LONG_LONG_ALIGNMENT 8 +#define _DOUBLE_ALIGNMENT 8 +#define _DOUBLE_COMPLEX_ALIGNMENT 8 +#define _LONG_DOUBLE_ALIGNMENT 16 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 16 +#define _POINTER_ALIGNMENT 8 +#define _MAX_ALIGNMENT 16 +#define _ALIGNMENT_REQUIRED 1 + +/* + * Different alignment constraints for the i386 ABI in compatibility mode + */ +#define _LONG_LONG_ALIGNMENT_32 4 + +/* + * Define the appropriate "implementation choices". + */ +#if !defined(_LP64) +#error "_LP64 not defined" +#endif +#if !defined(_I32LPx) +#define _I32LPx +#endif +#define _SUNOS_VTOC_16 +#define _DMA_USES_PHYSADDR +#define _FIRMWARE_NEEDS_FDISK +#define __i386_COMPAT +#define _PSM_MODULES +#define _RTC_CONFIG +#define _SOFT_HOSTID +#define _DONT_USE_1275_GENERIC_NAMES +#define _HAVE_CPUID_INSN + +/* + * The feature test macro __i386 is generic for all processors implementing + * the Intel 386 instruction set or a superset of it. Specifically, this + * includes all members of the 386, 486, and Pentium family of processors. + */ +#elif defined(__i386) || defined(__i386__) + +#if !defined(__i386) +#define __i386 +#endif + +#if !defined(__x86) +#define __x86 +#endif + +/* + * Define the appropriate "processor characteristics" + */ +#define _STACK_GROWS_DOWNWARD +#define _LONG_LONG_LTOH +#define _BIT_FIELDS_LTOH +#define _IEEE_754 +#define _CHAR_IS_SIGNED +#define _BOOL_ALIGNMENT 1 +#define _CHAR_ALIGNMENT 1 +#define _SHORT_ALIGNMENT 2 +#define _INT_ALIGNMENT 4 +#define _FLOAT_ALIGNMENT 4 +#define _FLOAT_COMPLEX_ALIGNMENT 4 +#define _LONG_ALIGNMENT 4 +#define _LONG_LONG_ALIGNMENT 4 +#define _DOUBLE_ALIGNMENT 4 +#define _DOUBLE_COMPLEX_ALIGNMENT 4 +#define _LONG_DOUBLE_ALIGNMENT 4 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 4 +#define _POINTER_ALIGNMENT 4 +#define _MAX_ALIGNMENT 4 +#define _ALIGNMENT_REQUIRED 0 + +#define _LONG_LONG_ALIGNMENT_32 _LONG_LONG_ALIGNMENT + +/* + * Define the appropriate "implementation choices". + */ +#if !defined(_ILP32) +#define _ILP32 +#endif +#if !defined(_I32LPx) +#define _I32LPx +#endif +#define _SUNOS_VTOC_16 +#define _DMA_USES_PHYSADDR +#define _FIRMWARE_NEEDS_FDISK +#define _PSM_MODULES +#define _RTC_CONFIG +#define _SOFT_HOSTID +#define _DONT_USE_1275_GENERIC_NAMES +#define _HAVE_CPUID_INSN + +#elif defined(__aarch64__) + +/* + * Define the appropriate "processor characteristics" + */ +#define _STACK_GROWS_DOWNWARD +#define _LONG_LONG_LTOH +#define _BIT_FIELDS_LTOH +#define _IEEE_754 +#define _CHAR_IS_UNSIGNED +#define _BOOL_ALIGNMENT 1 +#define _CHAR_ALIGNMENT 1 +#define _SHORT_ALIGNMENT 2 +#define _INT_ALIGNMENT 4 +#define _FLOAT_ALIGNMENT 4 +#define _FLOAT_COMPLEX_ALIGNMENT 4 +#define _LONG_ALIGNMENT 8 +#define _LONG_LONG_ALIGNMENT 8 +#define _DOUBLE_ALIGNMENT 8 +#define _DOUBLE_COMPLEX_ALIGNMENT 8 +#define _LONG_DOUBLE_ALIGNMENT 16 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 16 +#define _POINTER_ALIGNMENT 8 +#define _MAX_ALIGNMENT 16 +#define _ALIGNMENT_REQUIRED 1 + +#define _LONG_LONG_ALIGNMENT_32 _LONG_LONG_ALIGNMENT + +/* + * Define the appropriate "implementation choices" + */ +#if !defined(_LP64) +#error "_LP64 not defined" +#endif +#define _SUNOS_VTOC_16 +#define _DMA_USES_PHYSADDR +#define _FIRMWARE_NEEDS_FDISK +#define _PSM_MODULES +#define _RTC_CONFIG +#define _DONT_USE_1275_GENERIC_NAMES +#define _HAVE_CPUID_INSN + +#elif defined(__riscv) + +/* + * Define the appropriate "processor characteristics" + */ +#define _STACK_GROWS_DOWNWARD +#define _LONG_LONG_LTOH +#define _BIT_FIELDS_LTOH +#define _IEEE_754 +#define _CHAR_IS_UNSIGNED +#define _BOOL_ALIGNMENT 1 +#define _CHAR_ALIGNMENT 1 +#define _SHORT_ALIGNMENT 2 +#define _INT_ALIGNMENT 4 +#define _FLOAT_ALIGNMENT 4 +#define _FLOAT_COMPLEX_ALIGNMENT 4 +#define _LONG_ALIGNMENT 8 +#define _LONG_LONG_ALIGNMENT 8 +#define _DOUBLE_ALIGNMENT 8 +#define _DOUBLE_COMPLEX_ALIGNMENT 8 +#define _LONG_DOUBLE_ALIGNMENT 16 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 16 +#define _POINTER_ALIGNMENT 8 +#define _MAX_ALIGNMENT 16 +#define _ALIGNMENT_REQUIRED 1 + +#define _LONG_LONG_ALIGNMENT_32 _LONG_LONG_ALIGNMENT + +/* + * Define the appropriate "implementation choices" + */ +#if !defined(_LP64) +#define _LP64 +#endif +#define _SUNOS_VTOC_16 +#define _DMA_USES_PHYSADDR +#define _FIRMWARE_NEEDS_FDISK +#define _PSM_MODULES +#define _RTC_CONFIG +#define _DONT_USE_1275_GENERIC_NAMES +#define _HAVE_CPUID_INSN + +#elif defined(__arm__) + +/* + * Define the appropriate "processor characteristics" + */ +#define _STACK_GROWS_DOWNWARD +#define _LONG_LONG_LTOH +#define _BIT_FIELDS_LTOH +#define _IEEE_754 +#define _CHAR_IS_SIGNED +#define _BOOL_ALIGNMENT 1 +#define _CHAR_ALIGNMENT 1 +#define _SHORT_ALIGNMENT 2 +#define _INT_ALIGNMENT 4 +#define _FLOAT_ALIGNMENT 4 +#define _FLOAT_COMPLEX_ALIGNMENT 4 +#define _LONG_ALIGNMENT 4 +#define _LONG_LONG_ALIGNMENT 4 +#define _DOUBLE_ALIGNMENT 4 +#define _DOUBLE_COMPLEX_ALIGNMENT 4 +#define _LONG_DOUBLE_ALIGNMENT 4 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 4 +#define _POINTER_ALIGNMENT 4 +#define _MAX_ALIGNMENT 4 +#define _ALIGNMENT_REQUIRED 0 + +#define _LONG_LONG_ALIGNMENT_32 _LONG_LONG_ALIGNMENT + +/* + * Define the appropriate "implementation choices". + */ +#if !defined(_ILP32) +#define _ILP32 +#endif +#if !defined(_I32LPx) +#define _I32LPx +#endif +#define _SUNOS_VTOC_16 +#define _DMA_USES_PHYSADDR +#define _FIRMWARE_NEEDS_FDISK +#define _PSM_MODULES +#define _RTC_CONFIG +#define _DONT_USE_1275_GENERIC_NAMES +#define _HAVE_CPUID_INSN + +#elif defined(__mips__) + +/* + * Define the appropriate "processor characteristics" + */ +#define _STACK_GROWS_DOWNWARD +#define _LONG_LONG_LTOH +#define _BIT_FIELDS_LTOH +#define _IEEE_754 +#define _CHAR_IS_SIGNED +#define _BOOL_ALIGNMENT 1 +#define _CHAR_ALIGNMENT 1 +#define _SHORT_ALIGNMENT 2 +#define _INT_ALIGNMENT 4 +#define _FLOAT_ALIGNMENT 4 +#define _FLOAT_COMPLEX_ALIGNMENT 4 +#if defined(__mips_n64) +#define _LONG_ALIGNMENT 8 +#define _LONG_LONG_ALIGNMENT 8 +#define _DOUBLE_ALIGNMENT 8 +#define _DOUBLE_COMPLEX_ALIGNMENT 8 +#define _LONG_DOUBLE_ALIGNMENT 8 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 8 +#define _POINTER_ALIGNMENT 8 +#define _MAX_ALIGNMENT 8 +#define _ALIGNMENT_REQUIRED 0 + +#define _LONG_LONG_ALIGNMENT_32 _INT_ALIGNMENT +/* + * Define the appropriate "implementation choices". + */ +#if !defined(_LP64) +#error "_LP64 not defined" +#endif +#else +#define _LONG_ALIGNMENT 4 +#define _LONG_LONG_ALIGNMENT 4 +#define _DOUBLE_ALIGNMENT 4 +#define _DOUBLE_COMPLEX_ALIGNMENT 4 +#define _LONG_DOUBLE_ALIGNMENT 4 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 4 +#define _POINTER_ALIGNMENT 4 +#define _MAX_ALIGNMENT 4 +#define _ALIGNMENT_REQUIRED 0 + +#define _LONG_LONG_ALIGNMENT_32 _LONG_LONG_ALIGNMENT + +/* + * Define the appropriate "implementation choices". + */ +#if !defined(_ILP32) +#define _ILP32 +#endif +#if !defined(_I32LPx) +#define _I32LPx +#endif +#endif +#define _SUNOS_VTOC_16 +#define _DMA_USES_PHYSADDR +#define _FIRMWARE_NEEDS_FDISK +#define _PSM_MODULES +#define _RTC_CONFIG +#define _DONT_USE_1275_GENERIC_NAMES +#define _HAVE_CPUID_INSN + +#elif defined(__powerpc__) + +#if defined(__BIG_ENDIAN__) +#define _BIT_FIELDS_HTOL +#else +#define _BIT_FIELDS_LTOH +#endif + +/* + * The following set of definitions characterize the Solaris on SPARC systems. + * + * The symbol __sparc indicates any of the SPARC family of processor + * architectures. This includes SPARC V7, SPARC V8 and SPARC V9. + * + * The symbol __sparcv8 indicates the 32-bit SPARC V8 architecture as defined + * by Version 8 of the SPARC Architecture Manual. (SPARC V7 is close enough + * to SPARC V8 for the former to be subsumed into the latter definition.) + * + * The symbol __sparcv9 indicates the 64-bit SPARC V9 architecture as defined + * by Version 9 of the SPARC Architecture Manual. + * + * The symbols __sparcv8 and __sparcv9 are mutually exclusive, and are only + * relevant when the symbol __sparc is defined. + */ +/* + * XXX Due to the existence of 5110166, "defined(__sparcv9)" needs to be added + * to support backwards builds. This workaround should be removed in s10_71. + */ +#elif defined(__sparc) || defined(__sparcv9) || defined(__sparc__) +#if !defined(__sparc) +#define __sparc +#endif + +/* + * You can be 32-bit or 64-bit, but not both at the same time. + */ +#if defined(__sparcv8) && defined(__sparcv9) +#error "SPARC Versions 8 and 9 are mutually exclusive choices" +#endif + +/* + * Existing compilers do not set __sparcv8. Years will transpire before + * the compilers can be depended on to set the feature test macro. In + * the interim, we'll set it here on the basis of historical behaviour; + * if you haven't asked for SPARC V9, then you must've meant SPARC V8. + */ +#if !defined(__sparcv9) && !defined(__sparcv8) +#define __sparcv8 +#endif + +/* + * Define the appropriate "processor characteristics" shared between + * all Solaris on SPARC systems. + */ +#define _STACK_GROWS_DOWNWARD +#define _LONG_LONG_HTOL +#define _BIT_FIELDS_HTOL +#define _IEEE_754 +#define _CHAR_IS_SIGNED +#define _BOOL_ALIGNMENT 1 +#define _CHAR_ALIGNMENT 1 +#define _SHORT_ALIGNMENT 2 +#define _INT_ALIGNMENT 4 +#define _FLOAT_ALIGNMENT 4 +#define _FLOAT_COMPLEX_ALIGNMENT 4 +#define _LONG_LONG_ALIGNMENT 8 +#define _DOUBLE_ALIGNMENT 8 +#define _DOUBLE_COMPLEX_ALIGNMENT 8 +#define _ALIGNMENT_REQUIRED 1 + +/* + * Define the appropriate "implementation choices" shared between versions. + */ +#define _SUNOS_VTOC_8 +#define _DMA_USES_VIRTADDR +#define _NO_FDISK_PRESENT +#define _HAVE_TEM_FIRMWARE +#define _OBP + +/* + * The following set of definitions characterize the implementation of + * 32-bit Solaris on SPARC V8 systems. + */ +#if defined(__sparcv8) + +/* + * Define the appropriate "processor characteristics" + */ +#define _LONG_ALIGNMENT 4 +#define _LONG_DOUBLE_ALIGNMENT 8 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 8 +#define _POINTER_ALIGNMENT 4 +#define _MAX_ALIGNMENT 8 + +#define _LONG_LONG_ALIGNMENT_32 _LONG_LONG_ALIGNMENT + +/* + * Define the appropriate "implementation choices" + */ +#define _ILP32 +#if !defined(_I32LPx) +#define _I32LPx +#endif + +/* + * The following set of definitions characterize the implementation of + * 64-bit Solaris on SPARC V9 systems. + */ +#elif defined(__sparcv9) + +/* + * Define the appropriate "processor characteristics" + */ +#define _LONG_ALIGNMENT 8 +#define _LONG_DOUBLE_ALIGNMENT 16 +#define _LONG_DOUBLE_COMPLEX_ALIGNMENT 16 +#define _POINTER_ALIGNMENT 8 +#define _MAX_ALIGNMENT 16 + +#define _LONG_LONG_ALIGNMENT_32 _LONG_LONG_ALIGNMENT + +/* + * Define the appropriate "implementation choices" + */ +#if !defined(_LP64) +#error "_LP64 not defined" +#endif +#if !defined(_I32LPx) +#define _I32LPx +#endif +#define _MULTI_DATAMODEL + +#else +#error "unknown SPARC version" +#endif + +/* + * #error is strictly ansi-C, but works as well as anything for K&R systems. + */ +#else +#error "ISA not supported" +#endif + +#if defined(_ILP32) && defined(_LP64) +#error "Both _ILP32 and _LP64 are defined" +#endif + +#define ____cacheline_aligned __attribute__((aligned(64))) + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_ISA_DEFS_H */ diff --git a/include/os/windows/spl/sys/kidmap.h b/include/os/windows/spl/sys/kidmap.h new file mode 100644 index 000000000000..2acae935b42f --- /dev/null +++ b/include/os/windows/spl/sys/kidmap.h @@ -0,0 +1,27 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_KIDMAP_H +#define _SPL_KIDMAP_H + +#include + +#endif /* SPL_KIDMAP_H */ diff --git a/include/os/windows/spl/sys/kmem.h b/include/os/windows/spl/sys/kmem.h new file mode 100644 index 000000000000..412db7600ead --- /dev/null +++ b/include/os/windows/spl/sys/kmem.h @@ -0,0 +1,160 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2008 MacZFS Project + * Copyright (C) 2013 Jorgen Lundman + * Copyright (C) 2017 Sean Doran + * + */ + +#ifndef _SPL_KMEM_H +#define _SPL_KMEM_H + +#include +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +// XNU total amount of memory +extern uint64_t physmem; + +#define KM_SLEEP 0x0000 /* can block for memory; success guaranteed */ +#define KM_NOSLEEP 0x0001 /* cannot block for memory; may fail */ +#define KM_PANIC 0x0002 /* if memory cannot be allocated, panic */ +#define KM_PUSHPAGE 0x0004 /* can block for memory; may use reserve */ +#define KM_NORMALPRI 0x0008 /* with KM_NOSLEEP, lower priority allocation */ +#define KM_NODEBUG 0x0010 /* NOT IMPLEMENTED ON OSX */ +#define KM_NO_VBA 0x0020 /* OSX: don't descend to the bucket layer */ +#define KM_VMFLAGS 0x00ff /* flags that must match VM_* flags */ + +#define KM_FLAGS 0xffff /* all settable kmem flags */ + + /* + * Kernel memory allocator: DDI interfaces. + * See kmem_alloc(9F) for details. + */ + +#define MALLOC(A, C, S, T, F) \ + (A) = (C)ExAllocatePoolWithTag(NonPagedPoolNx, (S), '!SFZ') +#define FREE(A, T) \ + ExFreePoolWithTag((A), '!SFZ') + +// Work around symbol collisions in XNU +#define kmem_alloc(size, kmflags) zfs_kmem_alloc((size), (kmflags)) +#define kmem_zalloc(size, kmflags) zfs_kmem_zalloc((size), (kmflags)) +#define kmem_free(buf, size) zfs_kmem_free((buf), (size)) + +void *zfs_kmem_alloc(size_t size, int kmflags); +void *zfs_kmem_zalloc(size_t size, int kmflags); +void zfs_kmem_free(void *buf, size_t size); + +void spl_kmem_init(uint64_t); +void spl_kmem_thread_init(); +void spl_kmem_mp_init(); +void spl_kmem_thread_fini(); +void spl_kmem_timer_fini(); +void spl_kmem_fini(); + +uint64_t kmem_size(void); +uint64_t kmem_used(void); +int64_t kmem_avail(void); +uint64_t kmem_num_pages_wanted(); +int spl_vm_pool_low(void); +int64_t spl_minimal_physmem_p(void); +int64_t spl_adjust_pressure(int64_t); +int64_t spl_free_wrapper(void); +int64_t spl_free_manual_pressure_wrapper(void); +boolean_t spl_free_fast_pressure_wrapper(void); +void spl_free_set_pressure(int64_t); +void spl_free_set_fast_pressure(boolean_t); +uint64_t spl_free_last_pressure_wrapper(void); + +#define KMC_NOTOUCH 0x00010000 +#define KMC_NODEBUG 0x00020000 +#define KMC_NOMAGAZINE 0x00040000 +#define KMC_NOHASH 0x00080000 +#define KMC_QCACHE 0x00100000 +#define KMC_KMEM_ALLOC 0x00200000 /* internal use only */ +#define KMC_IDENTIFIER 0x00400000 /* internal use only */ +#define KMC_PREFILL 0x00800000 +#define KMC_ARENA_SLAB 0x01000000 /* use a bigger kmem cache */ + +struct kmem_cache; + +typedef struct kmem_cache kmem_cache_t; + +/* Client response to kmem move callback */ +typedef enum kmem_cbrc { + KMEM_CBRC_YES, + KMEM_CBRC_NO, + KMEM_CBRC_LATER, + KMEM_CBRC_DONT_NEED, + KMEM_CBRC_DONT_KNOW +} kmem_cbrc_t; + +#define POINTER_IS_VALID(p) (!((uintptr_t)(p) & 0x3)) +#define POINTER_INVALIDATE(pp) (*(pp) = (void *)((uintptr_t)(*(pp)) | 0x1)) + +kmem_cache_t *kmem_cache_create(char *name, size_t bufsize, size_t align, + int (*constructor)(void *, void *, int), + void (*destructor)(void *, void *), + void (*reclaim)(void *), + void *_private, struct vmem *vmp, int cflags); +void kmem_cache_destroy(kmem_cache_t *cache); +void *kmem_cache_alloc(kmem_cache_t *cache, int flags); +void kmem_cache_free(kmem_cache_t *cache, void *buf); +void kmem_cache_free_to_slab(kmem_cache_t *cache, void *buf); +void kmem_cache_reap_now(kmem_cache_t *cache); +void kmem_depot_ws_zero(kmem_cache_t *cache); +void kmem_reap(void); +void kmem_reap_idspace(void); +kmem_cache_t *kmem_cache_buf_in_cache(kmem_cache_t *, void *); + +int kmem_debugging(void); +void kmem_cache_set_move(kmem_cache_t *, + kmem_cbrc_t (*)(void *, void *, size_t, void *)); + +extern char *kmem_asprintf(const char *fmt, ...); +extern char *kmem_strdup(const char *str); +extern void kmem_strfree(char *str); +extern char *kmem_vasprintf(const char *fmt, va_list ap); +extern int kmem_scnprintf(char *restrict str, size_t size, + const char *restrict fmt, ...); +extern char *kmem_strstr(const char *in, const char *str); +extern void strident_canon(char *s, size_t n); +extern uint64_t spl_kmem_cache_inuse(kmem_cache_t *cache); +extern uint64_t spl_kmem_cache_entry_size(kmem_cache_t *cache); +extern boolean_t spl_arc_no_grow(size_t, boolean_t, kmem_cache_t **); +extern boolean_t kmem_cache_reap_active(void); + +#ifdef __cplusplus +} +#endif + +#endif /* _SPL_KMEM_H */ diff --git a/include/os/windows/spl/sys/kmem_cache.h b/include/os/windows/spl/sys/kmem_cache.h new file mode 100644 index 000000000000..2dc08b171266 --- /dev/null +++ b/include/os/windows/spl/sys/kmem_cache.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#ifndef _SPL_KMEM_CACHE_H +#define _SPL_KMEM_CACHE_H + +#endif diff --git a/include/os/windows/spl/sys/kmem_impl.h b/include/os/windows/spl/sys/kmem_impl.h new file mode 100644 index 000000000000..10403215883b --- /dev/null +++ b/include/os/windows/spl/sys/kmem_impl.h @@ -0,0 +1,501 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 1994, 2010, Oracle and/or its affiliates. All rights reserved. + */ + +#ifndef _SYS_KMEM_IMPL_H +#define _SYS_KMEM_IMPL_H + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#pragma pack(2) + +/* + * kernel memory allocator: implementation-private data structures + * + * Lock order: + * 1. cache_lock + * 2. cc_lock in order by CPU ID + * 3. cache_depot_lock + * + * Do not call kmem_cache_alloc() or taskq_dispatch() while holding any of the + * above locks. + */ + +#define KMF_AUDIT 0x00000001 /* transaction auditing */ +#define KMF_DEADBEEF 0x00000002 /* deadbeef checking */ +#define KMF_REDZONE 0x00000004 /* redzone checking */ +#define KMF_CONTENTS 0x00000008 /* freed-buffer content logging */ +#define KMF_STICKY 0x00000010 /* if set, override /etc/system */ +#define KMF_NOMAGAZINE 0x00000020 /* disable per-cpu magazines */ +#define KMF_FIREWALL 0x00000040 /* put all bufs before unmapped pages */ +#define KMF_LITE 0x00000100 /* lightweight debugging */ + +#define KMF_HASH 0x00000200 /* cache has hash table */ +#define KMF_RANDOMIZE 0x00000400 /* randomize other kmem_flags */ + +#define KMF_DUMPDIVERT 0x00001000 /* use alternate memory at dump time */ +#define KMF_DUMPUNSAFE 0x00002000 /* flag caches used at dump time */ +#define KMF_PREFILL 0x00004000 /* Prefill the slab when created. */ + +#define KMF_BUFTAG (KMF_DEADBEEF | KMF_REDZONE) +#define KMF_TOUCH (KMF_BUFTAG | KMF_LITE | KMF_CONTENTS) +#define KMF_RANDOM (KMF_TOUCH | KMF_AUDIT | KMF_NOMAGAZINE) +#define KMF_DEBUG (KMF_RANDOM | KMF_FIREWALL) + +#define KMEM_STACK_DEPTH 15 + +#define KMEM_FREE_PATTERN 0xdeadbeefdeadbeefULL +#define KMEM_UNINITIALIZED_PATTERN 0xbaddcafebaddcafeULL +#define KMEM_REDZONE_PATTERN 0xfeedfacefeedfaceULL +#define KMEM_REDZONE_BYTE 0xbb + +/* + * Upstream platforms handle size == 0 as valid alloc, we + * can not return NULL, as that invalidates KM_SLEEP. So + * we return a valid hardcoded address, instead of actually taking up + * memory by fudging size to 1 byte. If read/writes are + * attempted, we will get page fault (which is correct, they + * asked for zero bytes after all) + */ +#define KMEM_ZERO_SIZE_PTR ((void *)16) + +/* + * Redzone size encodings for kmem_alloc() / kmem_free(). We encode the + * allocation size, rather than storing it directly, so that kmem_free() + * can distinguish frees of the wrong size from redzone violations. + * + * A size of zero is never valid. + */ +#define KMEM_SIZE_ENCODE(x) (251 * (x) + 1) +#define KMEM_SIZE_DECODE(x) ((x) / 251) +#define KMEM_SIZE_VALID(x) ((x) % 251 == 1 && (x) != 1) + +#define KMEM_ALIGN 8 /* min guaranteed alignment */ +#define KMEM_ALIGN_SHIFT 3 /* log2(KMEM_ALIGN) */ +#define KMEM_VOID_FRACTION 8 /* never waste more than 1/8 of slab */ + +#define KMEM_SLAB_IS_PARTIAL(sp) \ + ((sp)->slab_refcnt > 0 && (sp)->slab_refcnt < (sp)->slab_chunks) +#define KMEM_SLAB_IS_ALL_USED(sp) \ + ((sp)->slab_refcnt == (sp)->slab_chunks) + +/* + * The bufctl (buffer control) structure keeps some minimal information + * about each buffer: its address, its slab, and its current linkage, + * which is either on the slab's freelist (if the buffer is free), or + * on the cache's buf-to-bufctl hash table (if the buffer is allocated). + * In the case of non-hashed, or "raw", caches (the common case), only + * the freelist linkage is necessary: the buffer address is at a fixed + * offset from the bufctl address, and the slab is at the end of the page. + * + * NOTE: bc_next must be the first field; raw buffers have linkage only. + */ +typedef struct kmem_bufctl { + struct kmem_bufctl *bc_next; /* next bufctl struct */ + void *bc_addr; /* address of buffer */ + struct kmem_slab *bc_slab; /* controlling slab */ +} kmem_bufctl_t; + +/* + * The KMF_AUDIT version of the bufctl structure. The beginning of this + * structure must be identical to the normal bufctl structure so that + * pointers are interchangeable. + */ +typedef struct kmem_bufctl_audit { + struct kmem_bufctl *bc_next; /* next bufctl struct */ + void *bc_addr; /* address of buffer */ + struct kmem_slab *bc_slab; /* controlling slab */ + kmem_cache_t *bc_cache; /* controlling cache */ + hrtime_t bc_timestamp; /* transaction time */ + kthread_t *bc_thread; /* thread doing transaction */ + struct kmem_bufctl *bc_lastlog; /* last log entry */ + void *bc_contents; /* contents at last free */ + int bc_depth; /* stack depth */ + pc_t bc_stack[KMEM_STACK_DEPTH]; /* pc stack */ +} kmem_bufctl_audit_t; + +/* + * A kmem_buftag structure is appended to each buffer whenever any of the + * KMF_BUFTAG flags (KMF_DEADBEEF, KMF_REDZONE, KMF_VERIFY) are set. + */ +typedef struct kmem_buftag { + uint64_t bt_redzone; /* 64-bit redzone pattern */ + kmem_bufctl_t *bt_bufctl; /* bufctl */ + intptr_t bt_bxstat; /* bufctl ^ (alloc/free) */ +} kmem_buftag_t; + +/* + * A variant of the kmem_buftag structure used for KMF_LITE caches. + * Previous callers are stored in reverse chronological order. (i.e. most + * recent first) + */ +typedef struct kmem_buftag_lite { + kmem_buftag_t bt_buftag; /* a normal buftag */ + pc_t bt_history[1]; /* zero or more callers */ +} kmem_buftag_lite_t; + +#define KMEM_BUFTAG_LITE_SIZE(f) \ + (offsetof(kmem_buftag_lite_t, bt_history[f])) + +#define KMEM_BUFTAG(cp, buf) \ + ((kmem_buftag_t *)((char *)(buf) + (cp)->cache_buftag)) + +#define KMEM_BUFCTL(cp, buf) \ + ((kmem_bufctl_t *)((char *)(buf) + (cp)->cache_bufctl)) + +#define KMEM_BUF(cp, bcp) \ + ((void *)((char *)(bcp) - (cp)->cache_bufctl)) + +#define KMEM_SLAB(cp, buf) \ + ((kmem_slab_t *)P2END((uintptr_t)(buf), (cp)->cache_slabsize) - 1) + +/* + * Test for using alternate memory at dump time. + */ +#define KMEM_DUMP(cp) ((cp)->cache_flags & KMF_DUMPDIVERT) +#define KMEM_DUMPCC(ccp) ((ccp)->cc_flags & KMF_DUMPDIVERT) + +/* + * The "CPU" macro loads a cpu_t that refers to the cpu that the current + * thread is running on at the time the macro is executed. A context switch + * may occur immediately after loading this data structure, leaving this + * thread pointing at the cpu_t for the previous cpu. This is not a problem; + * we'd just end up checking the previous cpu's per-cpu cache, and then check + * the other layers of the kmem cache if need be. + * + * It's not even a problem if the old cpu gets DR'ed out during the context + * switch. The cpu-remove DR operation memset(0)s the cpu_t, but doesn't free + * it. So the cpu_t's cpu_cache_offset would read as 0, causing us to use + * cpu 0's per-cpu cache. + * + * So, there is no need to disable kernel preemption while using the CPU macro + * below since if we have been context switched, there will not be any + * correctness problem, just a momentary use of a different per-cpu cache. + */ + +#define KMEM_CPU_CACHE(cp) \ + (&cp->cache_cpu[cpu_number()]) + +#define KMOM_MAGAZINE_VALID(cp, mp) \ + (((kmem_slab_t *)P2END((uintptr_t)(mp), PAGESIZE) - 1)->slab_cache == \ + (cp)->cache_magtype->mt_cache) + +#define KMEM_MAGAZINE_VALID(cp, mp) \ + (((kmem_slab_t *)P2END((uintptr_t)(mp), PAGESIZE) - 1)->slab_cache == \ + (cp)->cache_magtype->mt_cache) + +#define KMEM_SLAB_OFFSET(sp, buf) \ + ((uint32_t)((uintptr_t)(buf) - (uintptr_t)((sp)->slab_base))) + +#define KMEM_SLAB_MEMBER(sp, buf) \ + (KMEM_SLAB_OFFSET(sp, buf) < (sp)->slab_cache->cache_slabsize) + +#define KMEM_BUFTAG_ALLOC 0xa110c8edUL +#define KMEM_BUFTAG_FREE 0xf4eef4eeUL + +/* slab_later_count thresholds */ +#define KMEM_DISBELIEF 3 + +/* slab_flags */ +#define KMEM_SLAB_NOMOVE 0x1 +#define KMEM_SLAB_MOVE_PENDING 0x2 + +typedef struct kmem_slab { + struct kmem_cache *slab_cache; /* controlling cache */ + void *slab_base; /* base of allocated memory */ + avl_node_t slab_link; /* slab linkage */ + struct kmem_bufctl *slab_head; /* first free buffer */ + long slab_refcnt; /* outstanding allocations */ + long slab_chunks; /* chunks (bufs) in this slab */ + uint32_t slab_stuck_offset; /* unmoved buffer offset */ + uint16_t slab_later_count; /* cf KMEM_CBRC_LATER */ + uint16_t slab_flags; /* bits to mark the slab */ + hrtime_t slab_create_time; /* when was slab created? */ +} kmem_slab_t; + +#define KMEM_HASH_INITIAL 64 + +#define KMEM_HASH(cp, buf) \ + ((cp)->cache_hash_table + \ + (((uintptr_t)(buf) >> (cp)->cache_hash_shift) & (cp)->cache_hash_mask)) + +#define KMEM_CACHE_NAMELEN 31 + +typedef struct kmem_magazine { + void *mag_next; + void *mag_round[1]; /* one or more rounds */ +} kmem_magazine_t; + +/* + * The magazine types for fast per-cpu allocation + */ +typedef struct kmem_magtype { + short mt_magsize; /* magazine size (number of rounds) */ + int mt_align; /* magazine alignment */ + uint32_t mt_minbuf; /* all smaller buffers qualify */ + uint32_t mt_maxbuf; /* no larger buffers qualify */ + kmem_cache_t *mt_cache; /* magazine cache */ +} kmem_magtype_t; + +#define KMEM_CPU_CACHE_SIZE 128 /* must be power of 2 */ +#define KMEM_CPU_PAD (KMEM_CPU_CACHE_SIZE - sizeof (kmutex_t) - \ + 2 * sizeof (uint64_t) - 2 * sizeof (void *) - sizeof (int) - \ + 5 * sizeof (short)) +#define KMEM_CACHE_SIZE(ncpus) \ + offsetof(kmem_cache_t, cache_cpu[ncpus]) + + /* Offset from kmem_cache->cache_cpu for per cpu caches */ +#define KMEM_CPU_CACHE_OFFSET(cpuid) \ + offsetof(kmem_cache_t, cache_cpu[cpuid]) - \ + offsetof(kmem_cache_t, cache_cpu) + +/* + * Per CPU cache data + */ +typedef struct kmem_cpu_cache { + kmutex_t cc_lock; /* protects this cpu's local cache */ + uint64_t cc_alloc; /* allocations from this cpu */ + uint64_t cc_free; /* frees to this cpu */ + kmem_magazine_t *cc_loaded; /* the currently loaded magazine */ + kmem_magazine_t *cc_ploaded; /* the previously loaded magazine */ + int cc_flags; /* CPU-local copy of cache_flags */ + short cc_rounds; /* number of objects in loaded mag */ + short cc_prounds; /* number of objects in previous mag */ + short cc_magsize; /* number of rounds in a full mag */ + short cc_dump_rounds; /* dump time copy of cc_rounds */ + short cc_dump_prounds; /* dump time copy of cc_prounds */ + char cc_pad[KMEM_CPU_PAD]; /* for nice alignment */ +} kmem_cpu_cache_t; + +/* + * The magazine lists used in the depot. + */ +typedef struct kmem_maglist { + kmem_magazine_t *ml_list; /* magazine list */ + uint64_t ml_total; /* number of magazines */ + uint64_t ml_min; /* min since last update */ + uint64_t ml_reaplimit; /* max reapable magazines */ + uint64_t ml_alloc; /* allocations from this list */ +} kmem_maglist_t; + +typedef struct kmem_defrag { + /* + * Statistics + */ + uint64_t kmd_callbacks; /* move callbacks */ + uint64_t kmd_yes; /* KMEM_CBRC_YES responses */ + uint64_t kmd_no; /* NO responses */ + uint64_t kmd_later; /* LATER responses */ + uint64_t kmd_dont_need; /* DONT_NEED responses */ + uint64_t kmd_dont_know; /* DONT_KNOW responses */ + uint64_t kmd_hunt_found; /* DONT_KNOW: # found in mag */ + uint64_t kmd_slabs_freed; /* slabs freed by moves */ + uint64_t kmd_defrags; /* kmem_cache_defrag() */ + uint64_t kmd_scans; /* kmem_cache_scan() */ + + /* + * Consolidator fields + */ + avl_tree_t kmd_moves_pending; /* buffer moves pending */ + list_t kmd_deadlist; /* deferred slab frees */ + uint32_t kmd_deadcount; /* # of slabs in kmd_deadlist */ + uint8_t kmd_reclaim_numer; /* slab usage threshold */ + uint8_t kmd_pad1; /* compiler padding */ + uint16_t kmd_consolidate; /* triggers consolidator */ + uint32_t kmd_pad2; /* compiler padding */ + uint32_t kmd_slabs_sought; /* reclaimable slabs sought */ + uint32_t kmd_slabs_found; /* reclaimable slabs found */ + uint32_t kmd_tries; /* nth scan interval counter */ + /* + * Fields used to ASSERT that the client does not kmem_cache_free() + * objects passed to the move callback. + */ + void *kmd_from_buf; /* object to move */ + void *kmd_to_buf; /* move destination */ + kthread_t *kmd_thread; /* thread calling move */ +} kmem_defrag_t; + +/* + * Cache callback function types + */ +typedef int (*constructor_fn_t)(void*, void*, int); +typedef void (*destructor_fn_t)(void*, void*); +typedef void (*reclaim_fn_t)(void*); + +/* + * Cache + */ +struct kmem_cache { + + /* + * Statistics + */ + uint64_t cache_slab_create; /* slab creates */ + uint64_t cache_slab_destroy; /* slab destroys */ + uint64_t cache_slab_alloc; /* slab layer allocations */ + uint64_t cache_slab_free; /* slab layer frees */ + uint64_t cache_alloc_fail; /* total failed allocations */ + uint64_t cache_buftotal; /* total buffers */ + uint64_t cache_bufmax; /* max buffers ever */ + uint64_t cache_bufslab; /* buffers free in slab layer */ + uint64_t cache_reap; /* cache reaps */ + uint64_t cache_rescale; /* hash table rescales */ + uint64_t cache_lookup_depth; /* hash lookup depth */ + uint64_t cache_depot_contention; /* mutex contention count */ + uint64_t cache_depot_contention_prev; /* previous snapshot */ + uint64_t cache_alloc_count; /* Number of allocations in cache */ + uint64_t no_vba_success; /* successful calls with KM_NO_VBA set */ + uint64_t no_vba_fail; + /* number of times we set arc growth suppression time */ + uint64_t arc_no_grow_set; + /* number of times spl_zio_is_suppressed returned true for this cache */ + uint64_t arc_no_grow; + + /* + * Cache properties + */ + char cache_name[KMEM_CACHE_NAMELEN + 1]; + uint32_t cache_bufsize; /* object size */ + uint32_t cache_align; /* object alignment */ + int (*cache_constructor)(void *, void *, int); + void (*cache_destructor)(void *, void *); + void (*cache_reclaim)(void *); + kmem_cbrc_t (*cache_move)(void *, void *, uint32_t, void *); + void *cache_private; /* opaque arg to callbacks */ + vmem_t *cache_arena; /* vmem source for slabs */ + int cache_cflags; /* cache creation flags */ + int cache_flags; /* various cache state info */ + uint32_t cache_mtbf; /* induced alloc failure rate */ + uint32_t cache_pad1; /* compiler padding */ + kstat_t *cache_kstat; /* exported statistics */ + list_node_t cache_link; /* cache linkage */ + + /* + * Slab layer + */ + kmutex_t cache_lock; /* protects slab layer */ + + uint32_t cache_chunksize; /* buf + alignment [+ debug] */ + uint32_t cache_slabsize; /* size of a slab */ + uint32_t cache_maxchunks; /* max buffers per slab */ + uint32_t cache_bufctl; /* buf-to-bufctl distance */ + uint32_t cache_buftag; /* buf-to-buftag distance */ + uint32_t cache_verify; /* bytes to verify */ + uint32_t cache_contents; /* bytes of saved content */ + uint32_t cache_color; /* next slab color */ + uint32_t cache_mincolor; /* maximum slab color */ + uint32_t cache_maxcolor; /* maximum slab color */ + uint32_t cache_hash_shift; /* get to interesting bits */ + uint32_t cache_hash_mask; /* hash table mask */ + list_t cache_complete_slabs; /* completely allocated slabs */ + uint32_t cache_complete_slab_count; + avl_tree_t cache_partial_slabs; /* partial slab freelist */ + uint32_t cache_partial_binshift; /* for AVL sort bins */ + kmem_cache_t *cache_bufctl_cache; /* source of bufctls */ + kmem_bufctl_t **cache_hash_table; /* hash table base */ + kmem_defrag_t *cache_defrag; /* slab consolidator fields */ + + /* + * Depot layer + */ + kmutex_t cache_depot_lock; /* protects depot */ + kmem_magtype_t *cache_magtype; /* magazine type */ + kmem_maglist_t cache_full; /* full magazines */ + kmem_maglist_t cache_empty; /* empty magazines */ + void *cache_dumpfreelist; /* heap during crash dump */ + void *cache_dumplog; /* log entry during dump */ + + /* + * Per CPU structures + */ + // XNU adjust to suit offsetof + kmem_cpu_cache_t cache_cpu[1]; /* per-cpu data */ +}; + +typedef struct kmem_cpu_log_header { + kmutex_t clh_lock; + char *clh_current; + uint32_t clh_avail; + int clh_chunk; + int clh_hits; +#if defined(SPL_DEBUG_MUTEX) + char clh_pad[128 - sizeof (kmutex_t) - sizeof (char *) - + sizeof (uint32_t) - 2 * sizeof (int)]; +#else + char clh_pad[128 - sizeof (kmutex_t) - sizeof (char *) - + sizeof (uint32_t) - 2 * sizeof (int)]; +#endif +} kmem_cpu_log_header_t; + +typedef struct kmem_log_header { + kmutex_t lh_lock; + char *lh_base; + uint32_t *lh_free; + uint32_t lh_chunksize; + uint32_t lh_nchunks; + uint32_t lh_head; + uint32_t lh_tail; + uint32_t lh_hits; + kmem_cpu_log_header_t lh_cpu[1]; /* ncpus actually allocated */ +} kmem_log_header_t; + + /* kmem_move kmm_flags */ +#define KMM_DESPERATE 0x1 +#define KMM_NOTIFY 0x2 +#define KMM_DEBUG 0x4 + +typedef struct kmem_move { + kmem_slab_t *kmm_from_slab; + void *kmm_from_buf; + void *kmm_to_buf; + avl_node_t kmm_entry; + int kmm_flags; +} kmem_move_t; + +/* + * In order to consolidate partial slabs, it must be possible for the cache to + * have partial slabs. + */ +#define KMEM_IS_MOVABLE(cp) \ + (((cp)->cache_chunksize * 2) <= (cp)->cache_slabsize) + +#pragma pack() + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/include/os/windows/spl/sys/kstat.h b/include/os/windows/spl/sys/kstat.h new file mode 100644 index 000000000000..43b47322c298 --- /dev/null +++ b/include/os/windows/spl/sys/kstat.h @@ -0,0 +1,370 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + * Copyright (c) 2013, 2020 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ + +#ifndef _SPL_KSTAT_H +#define _SPL_KSTAT_H + +#include +#include +#include +#include + +/* + * Kernel statistics driver (/dev/zfs) ioctls + * Defined outside the ZFS ioctls, and handled separately in + * zfs_vnops_windows.c + */ +#define SPLIOCTL_TYPE 40000 +#define KSTAT_IOC_CHAIN_ID CTL_CODE(SPLIOCTL_TYPE, 0x7FD, \ + METHOD_NEITHER, FILE_ANY_ACCESS) +#define KSTAT_IOC_READ CTL_CODE(SPLIOCTL_TYPE, 0x7FE, \ + METHOD_NEITHER, FILE_ANY_ACCESS) +#define KSTAT_IOC_WRITE CTL_CODE(SPLIOCTL_TYPE, 0x7FF, \ + METHOD_NEITHER, FILE_ANY_ACCESS) + +#define KSTAT_STRLEN 255 +#define KSTAT_RAW_MAX (128*1024) + +#if defined(_KERNEL) + +#define KSTAT_ENTER(k) \ + { kmutex_t *lp = (k)->ks_lock; if (lp) mutex_enter(lp); } + +#define KSTAT_EXIT(k) \ + { kmutex_t *lp = (k)->ks_lock; if (lp) mutex_exit(lp); } + +#define KSTAT_UPDATE(k, rw) (*(k)->ks_update)((k), (rw)) + +#define KSTAT_SNAPSHOT(k, buf, rw) (*(k)->ks_snapshot)((k), (buf), (rw)) + +#endif /* defined(_KERNEL) */ + +/* + * For reference valid classes are: + * disk, tape, net, controller, vm, kvm, hat, streams, kstat, misc + */ + +#define KSTAT_TYPE_RAW 0 /* can be anything; ks_ndata >= 1 */ +#define KSTAT_TYPE_NAMED 1 /* name/value pair; ks_ndata >= 1 */ +#define KSTAT_TYPE_INTR 2 /* interrupt stats; ks_ndata == 1 */ +#define KSTAT_TYPE_IO 3 /* I/O stats; ks_ndata == 1 */ +#define KSTAT_TYPE_TIMER 4 /* event timer; ks_ndata >= 1 */ +#define KSTAT_TYPE_TXG 5 /* txg sync; ks_ndata >= 1 */ +#define KSTAT_NUM_TYPES 6 + +#define KSTAT_DATA_CHAR 0 +#define KSTAT_DATA_INT32 1 +#define KSTAT_DATA_UINT32 2 +#define KSTAT_DATA_INT64 3 +#define KSTAT_DATA_UINT64 4 +#define KSTAT_DATA_LONG 5 +#define KSTAT_DATA_ULONG 6 +#define KSTAT_DATA_STRING 7 +#define KSTAT_NUM_DATAS 8 + +#define KSTAT_INTR_HARD 0 +#define KSTAT_INTR_SOFT 1 +#define KSTAT_INTR_WATCHDOG 2 +#define KSTAT_INTR_SPURIOUS 3 +#define KSTAT_INTR_MULTSVC 4 +#define KSTAT_NUM_INTRS 5 + +#define KSTAT_FLAG_VIRTUAL 0x01 +#define KSTAT_FLAG_VAR_SIZE 0x02 +#define KSTAT_FLAG_WRITABLE 0x04 +#define KSTAT_FLAG_PERSISTENT 0x08 +#define KSTAT_FLAG_DORMANT 0x10 +#define KSTAT_FLAG_UNSUPPORTED (KSTAT_FLAG_VAR_SIZE | KSTAT_FLAG_WRITABLE | \ + KSTAT_FLAG_PERSISTENT | KSTAT_FLAG_DORMANT) +#define KSTAT_FLAG_INVALID 0x20 +#define KSTAT_FLAG_LONGSTRINGS 0x40 +#define KSTAT_FLAG_NO_HEADERS 0x80 + +#define KS_MAGIC 0x9d9d9d9d + +#define KSTAT_NAMED_PTR(kptr) ((kstat_named_t *)(kptr)->ks_data) + + +/* Dynamic updates */ +#define KSTAT_READ 0 +#define KSTAT_WRITE 1 + +struct kstat; + +typedef int kid_t; /* unique kstat id */ +typedef int kstat_update_t(struct kstat *, int); /* dynamic update cb */ + +struct seq_file { + char *sf_buf; + size_t sf_size; +}; + +void seq_printf(struct seq_file *m, const char *fmt, ...); + +typedef struct kstat_raw_ops { + int (*headers)(char *buf, size_t size); + int (*seq_headers)(struct seq_file *); + int (*data)(char *buf, size_t size, void *data); + void *(*addr)(struct kstat_s *ksp, loff_t index); +} kstat_raw_ops_t; + +#pragma pack(4) +typedef struct kstat { + /* + * Fields relevant to both kernel and user + */ + hrtime_t ks_crtime; /* creation time (from gethrtime()) */ + struct kstat *ks_next; /* kstat chain linkage */ + kid_t ks_kid; /* unique kstat ID */ + char ks_module[KSTAT_STRLEN]; /* provider module name */ + uchar_t ks_resv; /* reserved, currently just padding */ + int ks_instance; /* provider module's instance */ + char ks_name[KSTAT_STRLEN]; /* kstat name */ + uchar_t ks_type; /* kstat data type */ + char ks_class[KSTAT_STRLEN]; /* kstat class */ + uchar_t ks_flags; /* kstat flags */ + void *ks_data; /* kstat type-specific data */ + uint_t ks_ndata; /* # of type-specific data records */ + size_t ks_data_size; /* total size of kstat data section */ + hrtime_t ks_snaptime; /* time of last data shapshot */ + int ks_returnvalue; + int ks_errnovalue; + + /* + * Fields relevant to kernel only + */ + kmutex_t ks_private_lock; /* kstat private data lock */ + kmutex_t *ks_lock; /* kstat data lock */ + int(*ks_update)(struct kstat *, int); /* dynamic update */ + int(*ks_snapshot)(struct kstat *, void *, int); + void *ks_private; /* arbitrary provider-private data */ + void *ks_private1; /* private data */ + kstat_raw_ops_t ks_raw_ops; /* ops table for raw type */ + char *ks_raw_buf; /* buf used for raw ops */ + size_t ks_raw_bufsize; /* size of raw ops buffer */ +} kstat_t; +#pragma pack() + +#pragma pack(4) +typedef struct kstat_named { + char name[KSTAT_STRLEN]; /* name of counter */ + uchar_t data_type; /* data type */ + union { + char c[16]; /* enough for 128-bit ints */ + int32_t i32; + uint32_t ui32; + struct { + union { + char *ptr; /* NULL-term string */ +#if defined(_KERNEL) && defined(_MULTI_DATAMODEL) + caddr32_t ptr32; +#endif + char __pad[8]; /* 64-bit padding */ + } addr; + uint32_t len; /* # bytes for strlen + '\0' */ + } str; +/* + * The int64_t and uint64_t types are not valid for a maximally conformant + * 32-bit compilation environment (cc -Xc) using compilers prior to the + * introduction of C99 conforming compiler (reference ISO/IEC 9899:1990). + * In these cases, the visibility of i64 and ui64 is only permitted for + * 64-bit compilation environments or 32-bit non-maximally conformant + * C89 or C90 ANSI C compilation environments (cc -Xt and cc -Xa). In the + * C99 ANSI C compilation environment, the long long type is supported. + * The _INT64_TYPE is defined by the implementation (see sys/int_types.h). + */ + int64_t i64; + uint64_t ui64; + + long l; + ulong_t ul; + + /* These structure members are obsolete */ + + longlong_t ll; + u_longlong_t ull; + float f; + double d; + } value; /* value of counter */ +} kstat_named_t; +#pragma pack() + + +#define KSTAT_NAMED_PTR(kptr) ((kstat_named_t *)(kptr)->ks_data) + +/* + * Retrieve the pointer of the string contained in the given named kstat. + */ +#define KSTAT_NAMED_STR_PTR(knptr) ((knptr)->value.str.addr.ptr) + +/* + * Retrieve the length of the buffer required to store the string in the given + * named kstat. + */ +#define KSTAT_NAMED_STR_BUFLEN(knptr) ((knptr)->value.str.len) + +typedef struct kstat_intr { + uint_t intrs[KSTAT_NUM_INTRS]; +} kstat_intr_t; + +typedef struct kstat_io { + u_longlong_t nread; /* number of bytes read */ + u_longlong_t nwritten; /* number of bytes written */ + uint_t reads; /* number of read operations */ + uint_t writes; /* number of write operations */ + hrtime_t wtime; /* cumulative wait (pre-service) time */ + hrtime_t wlentime; /* cumulative wait len*time product */ + hrtime_t wlastupdate; /* last time wait queue changed */ + hrtime_t rtime; /* cumulative run (service) time */ + hrtime_t rlentime; /* cumulative run length*time product */ + hrtime_t rlastupdate; /* last time run queue changed */ + uint_t wcnt; /* count of elements in wait state */ + uint_t rcnt; /* count of elements in run state */ +} kstat_io_t; + +typedef struct kstat_timer { + char name[KSTAT_STRLEN]; /* event name */ + u_longlong_t num_events; /* number of events */ + hrtime_t elapsed_time; /* cumulative elapsed time */ + hrtime_t min_time; /* shortest event duration */ + hrtime_t max_time; /* longest event duration */ + hrtime_t start_time; /* previous event start time */ + hrtime_t stop_time; /* previous event stop time */ +} kstat_timer_t; + +typedef struct +{ + uint64_t total; + uint64_t count; +} stat_pair; + +/* + * Somewhat awkward, as the functions are in spl-kstat, but struct + * refers to ZFS name, crossing the ZFS/SPL boundary separation. + */ +typedef struct { + uint64_t arcstat_hits; + uint64_t arcstat_misses; + uint64_t arcstat_total_demand_hits; + uint64_t arcstat_total_demand_miss; + uint64_t arcstat_perfetch_hits; + uint64_t arcstat_perfetch_miss; + uint64_t arcstat_size; + uint64_t arcstat_c; + uint64_t arcstat_mfu_hits; + uint64_t arcstat_mru_hits; + uint64_t arcstat_mru_ghost_hits; + uint64_t arcstat_mfu_ghost_hits; + uint64_t arcstat_evict_skip; + uint64_t arcstat_mutex_miss; + uint64_t arcstat_compressed_size; + uint64_t arcstat_uncompressed_size; + uint64_t arcstat_overhead_size; + uint64_t arcstat_read_ps; + uint64_t arcstat_metadata_accesses_ps; + uint64_t arcstat_metadata_hit_ps; + uint64_t arcstat_metadata_miss_ps; + uint64_t arcstat_perfetch_ps; + uint64_t arcstat_demand_ps; + uint64_t arcstat_l2_hits; + uint64_t arcstat_l2_misses; + uint64_t arcstat_l2_read_bytes; + uint64_t arcstat_l2_write_bytes; + uint64_t arcstat_l2_access_ps; + /* + * ZIL and SLOG counters + */ + uint64_t zil_commit_count; + uint64_t zil_commit_writer_count; + uint64_t zil_itx_count; + uint64_t zil_itx_indirect_count; + uint64_t zil_itx_indirect_bytes; + uint64_t zil_itx_copied_count; + uint64_t zil_itx_copied_bytes; + uint64_t zil_itx_needcopy_count; + uint64_t zil_itx_needcopy_bytes; + uint64_t zil_itx_metaslab_normal_count; + uint64_t zil_itx_metaslab_normal_bytes; + uint64_t zil_itx_metaslab_slog_count; + uint64_t zil_itx_metaslab_slog_bytes; +} cache_counters; + +void spl_kstat_init(void); +void spl_kstat_fini(void); + +typedef uint64_t zoneid_t; +#define ALL_ZONES 0 + +extern kstat_t *kstat_create(const char *, int, const char *, const char *, + uchar_t, uint_t, uchar_t); +extern kstat_t *kstat_create_zone(const char *, int, const char *, + const char *, uchar_t, uint_t, uchar_t, zoneid_t); +extern void kstat_install(kstat_t *); +extern void kstat_delete(kstat_t *); +extern void kstat_named_setstr(kstat_named_t *knp, const char *src); +extern void kstat_set_string(char *, const char *); +extern void kstat_delete_byname(const char *, int, const char *); +extern void kstat_delete_byname_zone(const char *, int, const char *, zoneid_t); +extern void kstat_named_init(kstat_named_t *, const char *, uchar_t); +extern void kstat_timer_init(kstat_timer_t *, const char *); +extern void kstat_waitq_enter(kstat_io_t *); +extern void kstat_waitq_exit(kstat_io_t *); +extern void kstat_runq_enter(kstat_io_t *); +extern void kstat_runq_exit(kstat_io_t *); +extern void kstat_waitq_to_runq(kstat_io_t *); +extern void kstat_runq_back_to_waitq(kstat_io_t *); +extern void kstat_timer_start(kstat_timer_t *); +extern void kstat_timer_stop(kstat_timer_t *); + +extern void kstat_zone_add(kstat_t *, zoneid_t); +extern void kstat_zone_remove(kstat_t *, zoneid_t); +extern int kstat_zone_find(kstat_t *, zoneid_t); +extern void __kstat_set_raw_ops(kstat_t *ksp, + int (*headers)(char *buf, size_t size), + int (*data)(char *buf, size_t size, void *data), + void* (*addr)(kstat_t *ksp, loff_t index)); + +extern void __kstat_set_seq_raw_ops(kstat_t *ksp, + int (*headers)(struct seq_file *), + int (*data)(char *buf, size_t size, void *data), + void* (*addr)(kstat_t *ksp, loff_t index)); + +extern kstat_t *kstat_hold_bykid(kid_t kid, zoneid_t); +extern kstat_t *kstat_hold_byname(const char *, int, const char *, zoneid_t); +extern void kstat_rele(kstat_t *); + +#define kstat_set_seq_raw_ops(k, h, d, a) __kstat_set_seq_raw_ops(k, h, d, a) +#define kstat_set_raw_ops(k, h, d, a) __kstat_set_raw_ops(k, h, d, a) + +int spl_kstat_chain_id(PDEVICE_OBJECT DiskDevice, PIRP Irp, + PIO_STACK_LOCATION IrpSp); +int spl_kstat_read(PDEVICE_OBJECT DiskDevice, PIRP Irp, + PIO_STACK_LOCATION IrpSp); +int spl_kstat_write(PDEVICE_OBJECT DiskDevice, PIRP Irp, + PIO_STACK_LOCATION IrpSp); + +#endif /* _SPL_KSTAT_H */ diff --git a/include/os/windows/spl/sys/linker_set.h b/include/os/windows/spl/sys/linker_set.h new file mode 100644 index 000000000000..fb02a90f5a11 --- /dev/null +++ b/include/os/windows/spl/sys/linker_set.h @@ -0,0 +1,163 @@ +/* + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD + * + * Copyright (c) 1999 John D. Polstra + * Copyright (c) 1999,2001 Peter Wemm + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * Copyright (c) 2022 Jorgen Lundman + * - Windows version (x64 clang) + */ + +#ifndef _SYS_LINKER_SET_H_ +#define _SYS_LINKER_SET_H_ + +/* From cdefs.h */ +#ifndef __has_attribute +#define __has_attribute(x) 0 +#endif + +#define __CONCAT1(x, y) x ## y +#define __CONCAT(x, y) __CONCAT1(x, y) +#define __STRING(x) #x /* stringify without expanding x */ +#define __XSTRING(x) __STRING(x) /* expand x, then stringify */ + +#define __used __attribute__((__used__)) +#define __GLOBL(sym) __asm__(".globl " __XSTRING(sym)) +#define __WEAK(sym) __asm__(".weak " __XSTRING(sym)) + +#define __weak_symbol /* __attribute__((__weak__)) */ + +#if __has_attribute(no_sanitize) && defined(__clang__) +#ifdef _KERNEL +#define __nosanitizeaddress __attribute__((no_sanitize("kernel-address"))) +#else +#define __nosanitizeaddress __attribute__((no_sanitize("address"))) +#endif +#endif + +/* + * The following macros are used to declare global sets of objects, which + * are collected by the linker into a `linker_set' as defined below. + * For ELF, this is done by constructing a separate segment for each set. + */ + +#if defined(__powerpc64__) && (!defined(_CALL_ELF) || _CALL_ELF == 1) +/* + * ELFv1 pointers to functions are actaully pointers to function + * descriptors. + * + * Move the symbol pointer from ".text" to ".data" segment, to make + * the GCC compiler happy: + */ +#define __MAKE_SET_CONST +#else +#define __MAKE_SET_CONST const +#endif + +/* + * The userspace address sanitizer inserts redzones around global variables, + * violating the assumption that linker set elements are packed. + */ +#ifdef _KERNEL +#define __NOASAN +#else +#define __NOASAN __nosanitizeaddress +#endif + +/* + * set = "mine". + * ".mine$a" sets the start + * ".mine$m" all entries + * ".mine$z" sets the end + */ + +#define __MAKE_SET_QV(set, sym, qv) \ + __declspec(selectany) __declspec(allocate("." #set "$a")) \ + __weak_symbol const void * qv __CONCAT(__start_set_, set); \ + __declspec(selectany) __declspec(allocate("." #set "$z")) \ + __weak_symbol const void * qv __CONCAT(__stop_set_, set); \ + __declspec(allocate("." #set "$m")) const void * qv \ + __NOASAN \ + __set_##set##_sym_##sym \ + __used = (void *)&(sym) + +#define __MAKE_SET(set, sym) __MAKE_SET_QV(set, sym, __MAKE_SET_CONST) + +/* + * Public macros. + */ +#define TEXT_SET(set, sym) __MAKE_SET(set, sym) +#define DATA_SET(set, sym) __MAKE_SET(set, sym) +#define DATA_WSET(set, sym) __MAKE_SET_QV(set, sym, /* */) +#define BSS_SET(set, sym) __MAKE_SET(set, sym) +#define ABS_SET(set, sym) __MAKE_SET(set, sym) +#define SET_ENTRY(set, sym) __MAKE_SET(set, sym) + +/* + * Initialize before referring to a given linker set. + * We can not do the direct "extern ptype" here, due to: + * "redeclaration of '__start_set_mine' with a different type: 'int *' vs + * 'void *'" + * So we declare a local variable '__xstart_set_mine' and point + * it at '__start_set_mine' with a cast to (ptype *). + * We also need to skip first object as it is the start value. + */ +#define SET_DECLARE(set, ptype) \ + extern const void * __weak_symbol __CONCAT(__start_set_, set); \ + extern const void * __weak_symbol __CONCAT(__stop_set_, set); \ + static void * __CONCAT(__xstart_set_, set); \ + static void * __CONCAT(__xstop_set_, set); \ + __CONCAT(__xstart_set_, set) = \ + (ptype * const) __weak_symbol &__CONCAT(__start_set_, set); \ + __CONCAT(__xstop_set_, set) = \ + (ptype * const) __weak_symbol &__CONCAT(__stop_set_, set); \ + __CONCAT(__xstart_set_, set) += sizeof (void *) + +#define SET_BEGIN(set) \ + (__CONCAT(__xstart_set_, set)) +#define SET_LIMIT(set) \ + (__CONCAT(__xstop_set_, set)) + +/* + * Iterate over all the elements of a set. + * + * Sets always contain addresses of things, and "pvar" points to words + * containing those addresses. Thus is must be declared as "type **pvar", + * and the address of each set item is obtained inside the loop by "*pvar". + */ +#define SET_FOREACH(pvar, set) \ + for (pvar = (__typeof__((pvar))) \ + SET_BEGIN(set); pvar < (__typeof__((pvar))) SET_LIMIT(set); pvar++) + +#define SET_ITEM(set, i) \ + ((SET_BEGIN(set))[i]) + +/* + * Provide a count of the items in a set. + */ +#define SET_COUNT(set) \ + (SET_LIMIT(set) - SET_BEGIN(set)) + +#endif /* _SYS_LINKER_SET_H_ */ diff --git a/include/os/windows/spl/sys/list.h b/include/os/windows/spl/sys/list.h new file mode 100644 index 000000000000..5dd7276d9d50 --- /dev/null +++ b/include/os/windows/spl/sys/list.h @@ -0,0 +1,143 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2008 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SPL_LIST_H +#define _SPL_LIST_H + +#include + +/* + * NOTE: I have implemented the Solaris list API in terms of the native + * linux API. This has certain advantages in terms of leveraging the linux + * list debugging infrastructure, but it also means that the internals of a + * list differ slightly than on Solaris. This is not a problem as long as + * all callers stick to the published API. The two major differences are: + * + * 1) A list_node_t is mapped to a linux list_head struct which changes + * the name of the list_next/list_prev pointers to next/prev respectively. + * + * 2) A list_node_t which is not attached to a list on Solaris is denoted + * by having its list_next/list_prev pointers set to NULL. Under linux + * the next/prev pointers are set to LIST_POISON1 and LIST_POISON2 + * respectively. At this moment this only impacts the implementation + * of the list_link_init() and list_link_active() functions. + */ + +typedef struct list_node { + struct list_node *list_next; + struct list_node *list_prev; +} list_node_t; + +typedef struct list { + size_t list_size; + size_t list_offset; + list_node_t list_head; +} list_t; + +void list_create(list_t *, size_t, size_t); +void list_destroy(list_t *); + +void list_insert_after(list_t *, void *, void *); +void list_insert_before(list_t *, void *, void *); +void list_insert_head(list_t *, void *); +void list_insert_tail(list_t *, void *); +void list_remove(list_t *, void *); +void list_move_tail(list_t *, list_t *); + +void *list_head(list_t *); +void *list_tail(list_t *); +void *list_next(list_t *, void *); +void *list_prev(list_t *, void *); + +int list_link_active(list_node_t *); +int list_is_empty(list_t *); + +#define LIST_POISON1 NULL +#define LIST_POISON2 NULL + +#define list_d2l(a, obj) \ + ((list_node_t *)(((uint64_t)obj) + (uint64_t)(a)->list_offset)) +#define list_object(a, node) \ + ((void *)(((uint64_t)node) - (uint64_t)(a)->list_offset)) +#define list_empty(a) ((a)->list_head.list_next == &(a)->list_head) + + +static inline void +list_link_init(list_node_t *node) +{ + node->list_next = LIST_POISON1; + node->list_prev = LIST_POISON2; +} + +static inline void +__list_del(list_node_t *prev, list_node_t *next) +{ + next->list_prev = prev; + prev->list_next = next; +} + +static inline void list_del(list_node_t *entry) +{ + __list_del(entry->list_prev, entry->list_next); + entry->list_next = LIST_POISON1; + entry->list_prev = LIST_POISON2; +} + +static inline void * +list_remove_head(list_t *list) +{ + list_node_t *head = list->list_head.list_next; + if (head == &list->list_head) + return (NULL); + + list_del(head); + return (list_object(list, head)); +} + +static inline void * +list_remove_tail(list_t *list) +{ + list_node_t *tail = list->list_head.list_prev; + if (tail == &list->list_head) + return (NULL); + + list_del(tail); + return (list_object(list, tail)); +} + +static inline void +list_link_replace(list_node_t *old_node, list_node_t *new_node) +{ + ASSERT(list_link_active(old_node)); + ASSERT(!list_link_active(new_node)); + + new_node->list_next = old_node->list_next; + new_node->list_prev = old_node->list_prev; + old_node->list_prev->list_next = new_node; + old_node->list_next->list_prev = new_node; + list_link_init(old_node); +} + +#endif /* SPL_LIST_H */ diff --git a/include/os/windows/spl/sys/lookasidelist.h b/include/os/windows/spl/sys/lookasidelist.h new file mode 100644 index 000000000000..434da1d789d3 --- /dev/null +++ b/include/os/windows/spl/sys/lookasidelist.h @@ -0,0 +1,51 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2020, DataCore Software Corp. + */ + +#ifndef _SPL_LOOKASIDELIST_H +#define _SPL_LOOKASIDELIST_H + +extern void *osif_malloc(uint64_t); +extern void osif_free(void *, uint64_t); + +#define ZFS_LookAsideList_DRV_TAG '!SFZ' + +#define LOOKASIDELIST_CACHE_NAMELEN 31 + +typedef struct lookasidelist_cache { + uint64_t cache_active_allocations; + uint64_t total_alloc; + uint64_t total_free; + size_t cache_chunksize; // cache object size + kstat_t *cache_kstat; + char cache_name[LOOKASIDELIST_CACHE_NAMELEN + 1]; + LOOKASIDE_LIST_EX lookasideField; +} lookasidelist_cache_t; + +lookasidelist_cache_t *lookasidelist_cache_create(char *name, size_t size); +void lookasidelist_cache_destroy(lookasidelist_cache_t *pLookasidelist_cache); +void* lookasidelist_cache_alloc(lookasidelist_cache_t *pLookasidelist_cache); +void lookasidelist_cache_free(lookasidelist_cache_t *pLookasidelist_cache, + void *buf); + +#endif diff --git a/include/os/windows/spl/sys/md5.h b/include/os/windows/spl/sys/md5.h new file mode 100644 index 000000000000..1781654cc96b --- /dev/null +++ b/include/os/windows/spl/sys/md5.h @@ -0,0 +1,70 @@ +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Cleaned up version of the md5.h header file from RFC 1321. + */ + +/* + * MD5.H - header file for MD5C.C + */ + +/* + * Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All + * rights reserved. + * + * License to copy and use this software is granted provided that it + * is identified as the "RSA Data Security, Inc. MD5 Message-Digest + * Algorithm" in all material mentioning or referencing this software + * or this function. + * + * License is also granted to make and use derivative works provided + * that such works are identified as "derived from the RSA Data + * Security, Inc. MD5 Message-Digest Algorithm" in all material + * mentioning or referencing the derived work. + * + * RSA Data Security, Inc. makes no representations concerning either + * the merchantability of this software or the suitability of this + * software for any particular purpose. It is provided "as is" + * without express or implied warranty of any kind. + * + * These notices must be retained in any copies of any part of this + * documentation and/or software. + */ + +#ifndef _SYS_MD5_H +#define _SYS_MD5_H + +#include /* for uint_* */ + +/* + * Definitions for MD5 hashing functions, conformant to RFC 1321 + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#define MD5_DIGEST_LENGTH 16 + +/* MD5 context. */ +typedef struct { + uint32_t state[4]; /* state (ABCD) */ + uint32_t count[2]; /* number of bits, modulo 2^64 (lsb first) */ + union { + uint8_t buf8[64]; /* undigested input */ + uint32_t buf32[16]; /* realigned input */ + } buf_un; +} MD5_CTX; + +void MD5Init(MD5_CTX *); +void MD5Update(MD5_CTX *, const void *, unsigned int); +void MD5Final(void *, MD5_CTX *); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_MD5_H */ diff --git a/include/os/windows/spl/sys/md5_consts.h b/include/os/windows/spl/sys/md5_consts.h new file mode 100644 index 000000000000..e767cc3bd909 --- /dev/null +++ b/include/os/windows/spl/sys/md5_consts.h @@ -0,0 +1,133 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) by 1998 Sun Microsystems, Inc. + * All rights reserved. + */ + +#ifndef _SYS_MD5_CONSTS_H +#define _SYS_MD5_CONSTS_H + +#pragma ident "%Z%%M% %I% %E% SMI" + +#ifdef __cplusplus +extern "C" { +#endif + +/* constants, as provided in RFC 1321 */ + +#define MD5_CONST_0 (uint32_t)0xd76aa478 +#define MD5_CONST_1 (uint32_t)0xe8c7b756 +#define MD5_CONST_2 (uint32_t)0x242070db +#define MD5_CONST_3 (uint32_t)0xc1bdceee +#define MD5_CONST_4 (uint32_t)0xf57c0faf +#define MD5_CONST_5 (uint32_t)0x4787c62a +#define MD5_CONST_6 (uint32_t)0xa8304613 +#define MD5_CONST_7 (uint32_t)0xfd469501 +#define MD5_CONST_8 (uint32_t)0x698098d8 +#define MD5_CONST_9 (uint32_t)0x8b44f7af +#define MD5_CONST_10 (uint32_t)0xffff5bb1 +#define MD5_CONST_11 (uint32_t)0x895cd7be +#define MD5_CONST_12 (uint32_t)0x6b901122 +#define MD5_CONST_13 (uint32_t)0xfd987193 +#define MD5_CONST_14 (uint32_t)0xa679438e +#define MD5_CONST_15 (uint32_t)0x49b40821 +#define MD5_CONST_16 (uint32_t)0xf61e2562 +#define MD5_CONST_17 (uint32_t)0xc040b340 +#define MD5_CONST_18 (uint32_t)0x265e5a51 +#define MD5_CONST_19 (uint32_t)0xe9b6c7aa +#define MD5_CONST_20 (uint32_t)0xd62f105d +#define MD5_CONST_21 (uint32_t)0x2441453 +#define MD5_CONST_22 (uint32_t)0xd8a1e681 +#define MD5_CONST_23 (uint32_t)0xe7d3fbc8 +#define MD5_CONST_24 (uint32_t)0x21e1cde6 +#define MD5_CONST_25 (uint32_t)0xc33707d6 +#define MD5_CONST_26 (uint32_t)0xf4d50d87 +#define MD5_CONST_27 (uint32_t)0x455a14ed +#define MD5_CONST_28 (uint32_t)0xa9e3e905 +#define MD5_CONST_29 (uint32_t)0xfcefa3f8 +#define MD5_CONST_30 (uint32_t)0x676f02d9 +#define MD5_CONST_31 (uint32_t)0x8d2a4c8a +#define MD5_CONST_32 (uint32_t)0xfffa3942 +#define MD5_CONST_33 (uint32_t)0x8771f681 +#define MD5_CONST_34 (uint32_t)0x6d9d6122 +#define MD5_CONST_35 (uint32_t)0xfde5380c +#define MD5_CONST_36 (uint32_t)0xa4beea44 +#define MD5_CONST_37 (uint32_t)0x4bdecfa9 +#define MD5_CONST_38 (uint32_t)0xf6bb4b60 +#define MD5_CONST_39 (uint32_t)0xbebfbc70 +#define MD5_CONST_40 (uint32_t)0x289b7ec6 +#define MD5_CONST_41 (uint32_t)0xeaa127fa +#define MD5_CONST_42 (uint32_t)0xd4ef3085 +#define MD5_CONST_43 (uint32_t)0x4881d05 +#define MD5_CONST_44 (uint32_t)0xd9d4d039 +#define MD5_CONST_45 (uint32_t)0xe6db99e5 +#define MD5_CONST_46 (uint32_t)0x1fa27cf8 +#define MD5_CONST_47 (uint32_t)0xc4ac5665 +#define MD5_CONST_48 (uint32_t)0xf4292244 +#define MD5_CONST_49 (uint32_t)0x432aff97 +#define MD5_CONST_50 (uint32_t)0xab9423a7 +#define MD5_CONST_51 (uint32_t)0xfc93a039 +#define MD5_CONST_52 (uint32_t)0x655b59c3 +#define MD5_CONST_53 (uint32_t)0x8f0ccc92 +#define MD5_CONST_54 (uint32_t)0xffeff47d +#define MD5_CONST_55 (uint32_t)0x85845dd1 +#define MD5_CONST_56 (uint32_t)0x6fa87e4f +#define MD5_CONST_57 (uint32_t)0xfe2ce6e0 +#define MD5_CONST_58 (uint32_t)0xa3014314 +#define MD5_CONST_59 (uint32_t)0x4e0811a1 +#define MD5_CONST_60 (uint32_t)0xf7537e82 +#define MD5_CONST_61 (uint32_t)0xbd3af235 +#define MD5_CONST_62 (uint32_t)0x2ad7d2bb +#define MD5_CONST_63 (uint32_t)0xeb86d391 + +/* initialization constants, as given in RFC 1321. used in MD5Init */ + +#define MD5_INIT_CONST_1 (uint32_t)0x67452301 +#define MD5_INIT_CONST_2 (uint32_t)0xefcdab89 +#define MD5_INIT_CONST_3 (uint32_t)0x98badcfe +#define MD5_INIT_CONST_4 (uint32_t)0x10325476 + +/* shift constants, as given in RFC 1321. used in MD5Transform */ + +#define MD5_SHIFT_11 7 +#define MD5_SHIFT_12 12 +#define MD5_SHIFT_13 17 +#define MD5_SHIFT_14 22 +#define MD5_SHIFT_21 5 +#define MD5_SHIFT_22 9 +#define MD5_SHIFT_23 14 +#define MD5_SHIFT_24 20 +#define MD5_SHIFT_31 4 +#define MD5_SHIFT_32 11 +#define MD5_SHIFT_33 16 +#define MD5_SHIFT_34 23 +#define MD5_SHIFT_41 6 +#define MD5_SHIFT_42 10 +#define MD5_SHIFT_43 15 +#define MD5_SHIFT_44 21 + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_MD5_CONSTS_H */ diff --git a/include/os/windows/spl/sys/misc.h b/include/os/windows/spl/sys/misc.h new file mode 100644 index 000000000000..23416362196b --- /dev/null +++ b/include/os/windows/spl/sys/misc.h @@ -0,0 +1,22 @@ +/* + * The SPL is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; either version 2 of the License, or (at your + * option) any later version. + * + * The SPL is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + * + * You should have received a copy of the GNU General Public License along + * with the SPL. If not, see . + * + * Copyright 2022 Andrew Innes + * + */ + +#ifndef _OS_WINDOWS_SPL_MISC_H +#define _OS_WINDOWS_SPL_MISC_H + +#endif diff --git a/include/os/windows/spl/sys/mntent.h b/include/os/windows/spl/sys/mntent.h new file mode 100644 index 000000000000..73308121f99d --- /dev/null +++ b/include/os/windows/spl/sys/mntent.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_MNTENT_H +#define _SPL_MNTENT_H + +#endif /* SPL_MNTENT_H */ diff --git a/include/os/windows/spl/sys/mod_os.h b/include/os/windows/spl/sys/mod_os.h new file mode 100644 index 000000000000..f7d850d0c1e0 --- /dev/null +++ b/include/os/windows/spl/sys/mod_os.h @@ -0,0 +1,344 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright(c) 2022 Jorgen Lundman + */ + +/* + * OpenZFS litter the source files with ZFS_MODULE_PARAMS + * which are tunables for the kernel. They are generally "static". + * + * For Windows, we collect all these into a "linker set", which + * we can iterate at start up, and add the tunables to the Registry. + * + * Having just a pointer to the variable isn't going to be enough + * so the macro will define a struct with: + * - ptr to tunable + * - name of tunable + * - submodule name + * - type of tunable (int, long, string) + * and this struct is put into the linker set. + * For _CALL style, we also define + * - func to call + * which allows a function to be called to sanitise the input. + */ + + +#ifndef _SPL_MOD_H +#define _SPL_MOD_H + +#ifdef __cplusplus +extern "C" { +#endif /* __cplusplus */ + +#include +#include + +#define MODULE_INIT(s) +#define MODULE_AUTHOR(s) +#define MODULE_LICENSE(s) +#define MODULE_VERSION(s) +#define ZFS_MODULE_DESCRIPTION(s) +#define ZFS_MODULE_AUTHOR(s) +#define ZFS_MODULE_LICENSE(s) +#define ZFS_MODULE_VERSION(s) + +#ifdef _MSC_VER +#define __init +#define __exit +#else +#define __init __attribute__((unused)) +#define __exit __attribute__((unused)) +#endif + +// Glancing at Linux kernel, module parameters limit: +#define LINUX_MAX_MODULE_PARAM_LEN 1024 + +/* + * The init/fini functions need to be called, but they are all static + */ +#define module_init(fn) \ + int wrap_ ## fn(void) \ + { \ + return (fn()); \ + } + +#define module_exit(fn) \ + void wrap_ ## fn(void) \ + { \ + fn(); \ + } + +#define module_param_named(a, b, c, d) + +#define module_init_early(fn) \ + void \ + wrap_ ## fn(void *dummy __unused) \ + { \ + fn(); \ + } + +typedef enum { + ZT_ZMOD_RD, + ZT_ZMOD_RW +} ztunable_perm; + +/* + * STRING is a bit awkward, Linux kernel use it as + * "char *s = NULL", so it is allocated elsewhere. + * But we also like to be able to use it with static + * areas, like *version = "openzfs-2.1.8", so we + * internally add a flag member, so we can know what to + * free. + */ +typedef enum { + ZT_FLAG_ALLOCATED = 0, + ZT_FLAG_STATIC = (1<<0), + ZT_FLAG_WRITEONLY = (1<<1), +} ztunable_flag; + +/* + * ZFS_MODULE_CALL() and VIRTUAL do not define a type (like ULONG) in the + * MACRO so, they are set to ZT_TYPE_NOTSET. The call ZT_GET_VALUE( ..., &type) + * is used to fetch the real type from each handler function. + * The handler functions are given and expected: + * function(struct ztunable_s *zt, void **ptr, ULONG *len, ULONG *type, \ + * boolean_t set) + * * GET: point 'ptr' to variable, set 'len' size of variable, set 'type' to + * real type. + * * SET: 'ptr' points to input, 'len' has size (for ASSERT), set 'type' to + * real type. + */ +typedef enum { + ZT_TYPE_NOTSET, // _CALL sets no type. + ZT_TYPE_INT, // Linux INT + ZT_TYPE_UINT, // Linux UINT + ZT_TYPE_LONG, // Linux LONG + ZT_TYPE_ULONG, // Linux ULONG + ZT_TYPE_STRING, // Linux STRING + ZT_TYPE_U64, // Future expansion + ZT_TYPE_S64, // Future expansion +} ztunable_type; + +// Enhance this to dynamic one day? +#define ZFS_MODULE_STRMAX 64 + +static inline uint64_t ZT_TYPE_REGISTRY(ztunable_type t) +{ + switch (t) { + case ZT_TYPE_INT: + case ZT_TYPE_UINT: + return (REG_DWORD); + // "long" on linux is 8 bytes (x64), and windows 4. we + // has special type for it, so for "ZT_" it is 8 bytes. + case ZT_TYPE_LONG: + case ZT_TYPE_ULONG: + return (REG_QWORD); + case ZT_TYPE_STRING: + return (REG_SZ); + case ZT_TYPE_U64: + case ZT_TYPE_S64: + return (REG_QWORD); + case ZT_TYPE_NOTSET: + /* not reached */ + ASSERT3U(t, !=, ZT_TYPE_NOTSET); + return (REG_NONE); + } + return (REG_NONE); +} + +static inline uint64_t ZT_TYPE_SIZE(ztunable_type t) +{ + switch (t) { + case ZT_TYPE_INT: + case ZT_TYPE_UINT: + return (sizeof (int)); + // For now, "long" on linux is 8 bytes, and windows 4. + case ZT_TYPE_LONG: + case ZT_TYPE_ULONG: + return (sizeof (uint64_t)); + case ZT_TYPE_STRING: + return (sizeof (uintptr_t)); + case ZT_TYPE_U64: + case ZT_TYPE_S64: + return (sizeof (uint64_t)); + case ZT_TYPE_NOTSET: + /* not reached */ + ASSERT3U(t, !=, ZT_TYPE_NOTSET); + return (0); + } + return (0); +} + +#define ZFS_MODULE_PARAM_ARGS \ + struct ztunable_s *zt, void **ptr, ULONG *len, ULONG *type, boolean_t set + +typedef struct ztunable_s { + void *zt_ptr; + int (*zt_func)(ZFS_MODULE_PARAM_ARGS); /* If SET this is a callout */ + const char *zt_name; + const char *zt_prefix; + const char *zt_desc; + ztunable_perm zt_perm; + ztunable_type zt_type; + ztunable_flag zt_flag; +} ztunable_t; + +static inline void +ZT_SET_VALUE(ztunable_t *zt, void **ptr, ULONG *len, ULONG *type) +{ + + if (zt->zt_func != NULL) { + zt->zt_func(zt, ptr, len, type, B_TRUE); + return; + } + + switch (zt->zt_type) { + case ZT_TYPE_INT: + case ZT_TYPE_UINT: + ASSERT3U(*len, >=, sizeof (int)); + *(int *)zt->zt_ptr = *(int *)*ptr; + return; + case ZT_TYPE_LONG: + case ZT_TYPE_ULONG: + ASSERT3U(*len, >=, sizeof (uint64_t)); + *(uint64_t *)zt->zt_ptr = *(uint64_t *)*ptr; + return; + case ZT_TYPE_STRING: + if (zt->zt_flag & ZT_FLAG_STATIC) { + strlcpy(zt->zt_ptr, *ptr, ZFS_MODULE_STRMAX); + return; + } + zt->zt_ptr = (void *)*ptr; + return; + case ZT_TYPE_U64: + case ZT_TYPE_S64: + ASSERT3U(*len, >=, sizeof (uint64_t)); + *(uint64_t *)zt->zt_ptr = *(uint64_t *)*ptr; + return; + case ZT_TYPE_NOTSET: + /* not reached */ + ASSERT3U(zt->zt_type, !=, ZT_TYPE_NOTSET); + return; + } +} + +// This SETs ptr to point to the value location. +static inline void +ZT_GET_VALUE(ztunable_t *zt, void **ptr, ULONG *len, ULONG *type) +{ + + if (zt->zt_func != NULL) { + zt->zt_func(zt, ptr, len, type, B_FALSE); + return; + } + + *len = ZT_TYPE_SIZE(zt->zt_type); + *type = zt->zt_type; + + switch (zt->zt_type) { + case ZT_TYPE_INT: + case ZT_TYPE_UINT: + case ZT_TYPE_LONG: + case ZT_TYPE_ULONG: + case ZT_TYPE_U64: + case ZT_TYPE_S64: + *ptr = zt->zt_ptr; + return; + case ZT_TYPE_STRING: + *ptr = zt->zt_ptr; + if (zt->zt_ptr != NULL) + *len = strlen(zt->zt_ptr); + return; + case ZT_TYPE_NOTSET: + /* not reached */ + ASSERT3U(zt->zt_type, !=, ZT_TYPE_NOTSET); + return; + } +} + +#define ZFS_MODULE_PARAM(scope_prefix, name_prefix, name, type, perm, desc) \ + static ztunable_t zt_ ## name_prefix ## name = { \ + .zt_ptr = &name_prefix ## name, \ + .zt_func = NULL, \ + .zt_name = #name_prefix #name, \ + .zt_prefix = #scope_prefix, \ + .zt_desc = #desc, \ + .zt_perm = __CONCAT(ZT_, perm), \ + .zt_type = ZT_TYPE_ ## type, \ + .zt_flag = ZT_FLAG_STATIC \ + }; \ + SET_ENTRY(zt, zt_ ## name_prefix ## name) + +/* Used only internally in Windows port */ +#define ZFS_MODULE_RAW(scope_prefix, name, variable, type, perm, flag, desc) \ + static ztunable_t zt_ ## variable = { \ + .zt_ptr = &variable, \ + .zt_func = NULL, \ + .zt_name = #name, \ + .zt_prefix = #scope_prefix, \ + .zt_desc = #desc, \ + .zt_perm = __CONCAT(ZT_, perm), \ + .zt_type = ZT_TYPE_ ## type, \ + .zt_flag = flag \ + }; \ + SET_ENTRY(zt, zt_ ## variable) + + +#define ZFS_MODULE_PARAM_CALL_IMPL( \ + scope_prefix, name_prefix, name, perm, func, args, desc) \ + static ztunable_t zt_ ## name_prefix ## name = { \ + .zt_ptr = args, \ + .zt_func = func, \ + .zt_name = #name_prefix #name, \ + .zt_prefix = #scope_prefix, \ + .zt_desc = #desc, \ + .zt_perm = __CONCAT(ZT_, perm), \ + .zt_type = ZT_TYPE_NOTSET, \ + .zt_flag = ZT_FLAG_STATIC \ + }; \ + SET_ENTRY(zt, zt_ ## name_prefix ## name) + +#define ZFS_MODULE_PARAM_CALL( \ + scope_prefix, name_prefix, name, func, _, perm, desc) \ + ZFS_MODULE_PARAM_CALL_IMPL(scope_prefix, name_prefix, name, perm, \ + func, &name_prefix ## name, desc) + +#define ZFS_MODULE_VIRTUAL_PARAM_CALL( \ + scope_prefix, name_prefix, name, func, _, perm, desc) \ + ZFS_MODULE_PARAM_CALL_IMPL(scope_prefix, name_prefix, name, perm, \ + win32_ ## func, NULL, desc) + +#define module_param_call(name, _set, _get, var, mode) \ + extern int win32_ ## _set(ZFS_MODULE_PARAM_ARGS); \ + ZFS_MODULE_PARAM_CALL_IMPL(zfs, /* */, name, ZMOD_RW, \ + win32_ ## _set, var, "xxx") + +struct zfs_kernel_param_s; +typedef struct zfs_kernel_param_s zfs_kernel_param_t; + +extern int param_set_uint(char *v, zfs_kernel_param_t *kp); +#ifdef __cplusplus +} +#endif /* __cplusplus */ + +#endif /* SPL_MOD_H */ diff --git a/include/os/windows/spl/sys/mount.h b/include/os/windows/spl/sys/mount.h new file mode 100644 index 000000000000..65727c273a66 --- /dev/null +++ b/include/os/windows/spl/sys/mount.h @@ -0,0 +1,141 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_MOUNT_H +#define _SPL_MOUNT_H + +#define MNT_WAIT 1 /* synchronized I/O file integrity completion */ +#define MNT_NOWAIT 2 /* start all I/O, but do not wait for it */ + +#define MNT_RDONLY 0x00000001 /* read only filesystem */ +#define MNT_SYNCHRONOUS 0x00000002 /* file system written synchronously */ +#define MNT_NOEXEC 0x00000004 /* can't exec from filesystem */ +#define MNT_NOSUID 0x00000008 /* don't honor setuid bits on fs */ +#define MNT_NODEV 0x00000010 /* don't interpret special files */ +#define MNT_UNION 0x00000020 /* union with underlying filesystem */ +#define MNT_ASYNC 0x00000040 /* file system written asynchronously */ +#define MNT_CPROTECT 0x00000080 /* file system supports content protection */ + +#define MNT_LOCAL 0x00001000 /* filesystem is stored locally */ +#define MNT_QUOTA 0x00002000 /* quotas are enabled on filesystem */ +#define MNT_ROOTFS 0x00004000 /* identifies the root filesystem */ +#define MNT_DOVOLFS 0x00008000 /* FS supports volfs (deprecated 10.5) */ + +#define MNT_DONTBROWSE 0x00100000 /* fs is not appropriate path to user data */ +#define MNT_IGNORE_OWNERSHIP 0x00200000 /* VFS will ignore ownership */ +#define MNT_AUTOMOUNTED 0x00400000 /* filesystem was mounted by automounter */ +#define MNT_JOURNALED 0x00800000 /* filesystem is journaled */ +#define MNT_NOUSERXATTR 0x01000000 /* Don't allow user extended attributes */ +#define MNT_DEFWRITE 0x02000000 /* filesystem should defer writes */ +#define MNT_MULTILABEL 0x04000000 /* MAC support for individual labels */ +#define MNT_NOATIME 0x10000000 /* disable update of file access time */ + +#define MNT_UPDATE 0x00010000 /* not a real mount, just an update */ +#define MNT_NOBLOCK 0x00020000 /* don't block unmount if not responding */ +#define MNT_RELOAD 0x00040000 /* reload filesystem data */ +#define MNT_FORCE 0x00080000 /* force unmount or readonly change */ +#define MNT_CMDFLAGS (MNT_UPDATE|MNT_NOBLOCK|MNT_RELOAD|MNT_FORCE) + +#define MNT_UNKNOWNPERMISSIONS MNT_IGNORE_OWNERSHIP + +#define MFSTYPENAMELEN 16 + +// Undo this OSX legacy +typedef struct fsid { int32_t val[2]; } fsid_t; + +struct vfsstatfs { + uint32_t f_bsize; /* fundamental file system block size */ + size_t f_iosize; /* optimal transfer block size */ + uint64_t f_blocks; /* total data blocks in file system */ + uint64_t f_bfree; /* free blocks in fs */ + uint64_t f_bavail; /* free blocks avail to non-superuser */ + uint64_t f_bused; /* free blocks avail to non-superuser */ + uint64_t f_files; /* total file nodes in file system */ + uint64_t f_ffree; /* free file nodes in fs */ + fsid_t f_fsid; /* file system id */ + uid_t f_owner; /* user that mounted the filesystem */ + uint64_t f_flags; /* copy of mount exported flags */ + char f_fstypename[MFSTYPENAMELEN]; /* fs type name inclus */ + char f_mntonname[MAXPATHLEN]; /* dir on which mounted */ + char f_mntfromname[MAXPATHLEN]; /* mounted filesystem */ + uint32_t f_fssubtype; /* fs sub-type (flavor) */ + void *f_reserved[2]; /* For future use == 0 */ +}; + +typedef enum _FSD_IDENTIFIER_TYPE { + MOUNT_TYPE_DGL = ':DGL', // Dokan Global + MOUNT_TYPE_DCB = ':DCB', // Disk Control Block + MOUNT_TYPE_VCB = ':VCB', // Volume Control Block + MOUNT_TYPE_FCB = ':FCB', // File Control Block + MOUNT_TYPE_CCB = ':CCB', // Context Control Block +} FSD_IDENTIFIER_TYPE; + +typedef enum mount_type mount_type_t; + +struct mount +{ + FSD_IDENTIFIER_TYPE type; + ULONG size; + void *fsprivate; + void *parent_device; // Only set so vcd can find dcb + PDEVICE_OBJECT deviceObject; + PDEVICE_OBJECT diskDeviceObject; + UNICODE_STRING bus_name; + UNICODE_STRING device_name; + UNICODE_STRING symlink_name; + UNICODE_STRING fs_name; + UNICODE_STRING name; + UNICODE_STRING uuid; + UNICODE_STRING mountpoint; + boolean_t justDriveLetter; + uint64_t volume_opens; + PVPB vpb; + + uint64_t mountflags; + + // NotifySync is used by notify directory change + PNOTIFY_SYNC NotifySync; + LIST_ENTRY DirNotifyList; +}; +typedef struct mount mount_t; +typedef struct mount vfsp_t; +#define LK_NOWAIT 1 + +int vfs_busy(mount_t *mp, int flags); +void vfs_unbusy(mount_t *mp); +int vfs_isrdonly(mount_t *mp); +void vfs_setrdonly(mount_t *mp); +void vfs_clearrdonly(mount_t *mp); + +void *vfs_fsprivate(mount_t *mp); +void vfs_setfsprivate(mount_t *mp, void *mntdata); +void vfs_clearflags(mount_t *mp, uint64_t flags); +void vfs_setflags(mount_t *mp, uint64_t flags); +struct vfsstatfs *vfs_statfs(mount_t *mp); +uint64_t vfs_flags(mount_t *mp); +void vfs_setlocklocal(mount_t *mp); +int vfs_typenum(mount_t *mp); +void vfs_getnewfsid(struct mount *mp); +int vfs_isunmount(mount_t *mp); +int vfs_iswriteupgrade(mount_t *mp); +void vfs_setextendedsecurity(mount_t *mp); + +#endif /* SPL_MOUNT_H */ diff --git a/include/os/windows/spl/sys/mutex.h b/include/os/windows/spl/sys/mutex.h new file mode 100644 index 000000000000..52d4f23ba529 --- /dev/null +++ b/include/os/windows/spl/sys/mutex.h @@ -0,0 +1,108 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#ifndef WINDOWS_MUTEX_H +#define WINDOWS_MUTEX_H + +#include // For SPL_DEBUG_MUTEX + +#ifdef _KERNEL + +#if __clang__ +// We set this in top CMakelists.txt but these are emitted by the +// pre-processor, so we need to push them here as well. +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wunknown-pragmas" +#pragma GCC diagnostic ignored "-Wignored-attributes" +#pragma GCC diagnostic ignored "-Wignored-pragma-intrinsic" +#endif + +#include +#include + +#if __clang__ +#pragma GCC diagnostic pop +#endif + +#include + + +typedef enum { + MUTEX_ADAPTIVE = 0, /* spin if owner is running, otherwise block */ + MUTEX_SPIN = 1, /* block interrupts and spin */ + MUTEX_DRIVER = 4, /* driver (DDI) mutex */ + MUTEX_DEFAULT = 6 /* kernel default mutex */ +} kmutex_type_t; + +typedef struct { + KEVENT opaque; +} mutex_t; + +/* + * Solaris kmutex defined. + * + * and is embedded into ZFS structures (see dbuf) so we need to match the + * size carefully. It appears to be 32 bytes. Or rather, it needs to be + * aligned. + */ + +typedef struct kmutex { + mutex_t m_lock; + void *m_owner; + KSPIN_LOCK m_destroy_lock; + unsigned int m_initialised; +} kmutex_t; + +#define MUTEX_HELD(x) (mutex_owned(x)) +#define MUTEX_NOT_HELD(x) (!mutex_owned(x)) + +#define mutex_init spl_mutex_init +void spl_mutex_init(kmutex_t *mp, char *name, kmutex_type_t type, void *ibc); + +#define mutex_enter spl_mutex_enter +void spl_mutex_enter(kmutex_t *mp); + +#define mutex_enter_nested(A, B) mutex_enter(A) +#define MUTEX_NOLOCKDEP 0 + +#define mutex_destroy spl_mutex_destroy +#define mutex_exit spl_mutex_exit +#define mutex_tryenter spl_mutex_tryenter +#define mutex_owned spl_mutex_owned +#define mutex_owner spl_mutex_owner + +void spl_mutex_destroy(kmutex_t *mp); +void spl_mutex_exit(kmutex_t *mp); +int spl_mutex_tryenter(kmutex_t *mp); +int spl_mutex_owned(kmutex_t *mp); +struct kthread *spl_mutex_owner(kmutex_t *mp); + +int spl_mutex_subsystem_init(void); +void spl_mutex_subsystem_fini(void); + +#endif // KERNEL +#endif diff --git a/include/os/windows/spl/sys/note.h b/include/os/windows/spl/sys/note.h new file mode 100644 index 000000000000..b460b2f71ac7 --- /dev/null +++ b/include/os/windows/spl/sys/note.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_NOTE_H +#define _SPL_NOTE_H + +#endif /* SPL_NOTE_H */ diff --git a/include/os/windows/spl/sys/param.h b/include/os/windows/spl/sys/param.h new file mode 100644 index 000000000000..bbe6b63c113f --- /dev/null +++ b/include/os/windows/spl/sys/param.h @@ -0,0 +1,34 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_PARAM_H +#define _SPL_PARAM_H + +/* Pages to bytes and back */ +#define ptob(pages) (pages << PAGE_SHIFT) +#define btop(bytes) (bytes >> PAGE_SHIFT) +#ifndef howmany +#define howmany(x, y) ((((x) % (y)) == 0) ? ((x) / (y)) : (((x) / (y)) + 1)) +#endif + +#define MAXUID UINT32_MAX + +#endif /* SPL_PARAM_H */ diff --git a/include/os/windows/spl/sys/policy.h b/include/os/windows/spl/sys/policy.h new file mode 100644 index 000000000000..3df37fc573dc --- /dev/null +++ b/include/os/windows/spl/sys/policy.h @@ -0,0 +1,88 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2008 MacZFS + * Copyright (C) 2013 Jorgen Lundman + * + */ + +#ifndef _SPL_POLICY_H +#define _SPL_POLICY_H + +#ifdef _KERNEL + +#include +#include + +struct vattr; + +int secpolicy_fs_unmount(cred_t *, struct mount *); +int secpolicy_nfs(const cred_t *); +int secpolicy_sys_config(const cred_t *, boolean_t); +int secpolicy_zfs(const cred_t *); +int secpolicy_zfs_proc(const cred_t *, proc_t *); +int secpolicy_zinject(const cred_t *); + +/* + * This function to be called from xxfs_setattr(). + * Must be called with the node's attributes read-write locked. + * + * cred_t * - acting credentials + * struct vnode * - vnode we're operating on + * struct vattr *va - new attributes, va_mask may be + * changed on return from a call + * struct vattr *oldva - old attributes, need include owner + * and mode only + * int flags - setattr flags + * int iaccess(void *node, int mode, cred_t *cr) + * - non-locking internal access function + * mode be checked + * w/ VREAD|VWRITE|VEXEC, not fs + * internal mode encoding. + * + * void *node - internal node (inode, tmpnode) to + * pass as arg to iaccess + */ +int secpolicy_vnode_setattr(cred_t *, struct vnode *, vattr_t *, + const vattr_t *, int, int (void *, int, cred_t *), void *); + +int secpolicy_vnode_stky_modify(const cred_t *); +int secpolicy_setid_setsticky_clear(struct vnode *vp, vattr_t *vap, + const vattr_t *ovap, cred_t *cr); + +int secpolicy_vnode_remove(struct vnode *, const cred_t *); +int secpolicy_vnode_create_gid(const cred_t *); +int secpolicy_vnode_setids_setgids(struct vnode *, const cred_t *, gid_t); +int secpolicy_vnode_setdac(struct vnode *, const cred_t *, uid_t); +int secpolicy_vnode_chown(struct vnode *, const cred_t *, uid_t); +struct znode; +int secpolicy_vnode_setid_retain(struct znode *, const cred_t *, boolean_t); +int secpolicy_xvattr(vattr_t *, uid_t, const cred_t *, mode_t); +int secpolicy_setid_clear(vattr_t *, const cred_t *); +int secpolicy_basic_link(const cred_t *); +int secpolicy_fs_mount_clearopts(const cred_t *, struct mount *); +int secpolicy_fs_mount(const cred_t *, struct vnode *, struct mount *); + +#endif /* _KERNEL */ + +#endif /* SPL_POLICY_H */ diff --git a/include/os/windows/spl/sys/priv.h b/include/os/windows/spl/sys/priv.h new file mode 100644 index 000000000000..a5239bb3ed8c --- /dev/null +++ b/include/os/windows/spl/sys/priv.h @@ -0,0 +1,525 @@ +/* + * Copyright (c) 2006 nCircle Network Security, Inc. + * All rights reserved. + * + * This software was developed by Robert N. M. Watson for the TrustedBSD + * Project under contract to nCircle Network Security, Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR, NCIRCLE NETWORK SECURITY, + * INC., OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED + * TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR + * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +/* + * Privilege checking interface for BSD kernel. + */ +#ifndef _SPL_PRIV_H +#define _SPL_PRIV_H + +/* + * Privilege list, sorted loosely by kernel subsystem. + * + * Think carefully before adding or reusing one of these privileges -- are + * there existing instances referring to the same privilege? Third party + * vendors may request the assignment of privileges to be used in loadable + * modules. Particular numeric privilege assignments are part of the + * loadable kernel module ABI, and should not be changed across minor + * releases. + * + * When adding a new privilege, remember to determine if it's appropriate for + * use in jail, and update the privilege switch in kern_jail.c as necessary. + */ + +/* + * Track beginning of privilege list. + */ +#define _PRIV_LOWEST 1 + +/* + * The remaining privileges typically correspond to one or a small + * number of specific privilege checks, and have (relatively) precise + * meanings. They are loosely sorted into a set of base system + * privileges, such as the ability to reboot, and then loosely by + * subsystem, indicated by a subsystem name. + */ +#define _PRIV_ROOT 1 /* Removed. */ +#define PRIV_ACCT 2 /* Manage process accounting. */ +#define PRIV_MAXFILES 3 /* Exceed system open files limit. */ +#define PRIV_MAXPROC 4 /* Exceed system processes limit. */ +#define PRIV_KTRACE 5 /* Set/clear KTRFAC_ROOT on ktrace. */ +#define PRIV_SETDUMPER 6 /* Configure dump device. */ +#define PRIV_REBOOT 8 /* Can reboot system. */ +#define PRIV_SWAPON 9 /* Can swapon(). */ +#define PRIV_SWAPOFF 10 /* Can swapoff(). */ +#define PRIV_MSGBUF 11 /* Can read kernel message buffer. */ +#define PRIV_IO 12 /* Can perform low-level I/O. */ +#define PRIV_KEYBOARD 13 /* Reprogram keyboard. */ +#define PRIV_DRIVER 14 /* Low-level driver privilege. */ +#define PRIV_ADJTIME 15 /* Set time adjustment. */ +#define PRIV_NTP_ADJTIME 16 /* Set NTP time adjustment. */ +#define PRIV_CLOCK_SETTIME 17 /* Can call clock_settime. */ +#define PRIV_SETTIMEOFDAY 18 /* Can call settimeofday. */ +#define _PRIV_SETHOSTID 19 /* Removed. */ +#define _PRIV_SETDOMAINNAME 20 /* Removed. */ + +/* + * Audit subsystem privileges. + */ +#define PRIV_AUDIT_CONTROL 40 /* Can configure audit. */ +#define PRIV_AUDIT_FAILSTOP 41 /* Can run during audit fail stop. */ +#define PRIV_AUDIT_GETAUDIT 42 /* Can get proc audit properties. */ +#define PRIV_AUDIT_SETAUDIT 43 /* Can set proc audit properties. */ +#define PRIV_AUDIT_SUBMIT 44 /* Can submit an audit record. */ + +/* + * Credential management privileges. + */ +#define PRIV_CRED_SETUID 50 /* setuid. */ +#define PRIV_CRED_SETEUID 51 /* seteuid to !ruid and !svuid. */ +#define PRIV_CRED_SETGID 52 /* setgid. */ +#define PRIV_CRED_SETEGID 53 /* setgid to !rgid and !svgid. */ +#define PRIV_CRED_SETGROUPS 54 /* Set process additional groups. */ +#define PRIV_CRED_SETREUID 55 /* setreuid. */ +#define PRIV_CRED_SETREGID 56 /* setregid. */ +#define PRIV_CRED_SETRESUID 57 /* setresuid. */ +#define PRIV_CRED_SETRESGID 58 /* setresgid. */ +#define PRIV_SEEOTHERGIDS 59 /* Exempt bsd.seeothergids. */ +#define PRIV_SEEOTHERUIDS 60 /* Exempt bsd.seeotheruids. */ + +/* + * Debugging privileges. + */ +#define PRIV_DEBUG_DIFFCRED 80 /* Exempt debugging other users. */ +#define PRIV_DEBUG_SUGID 81 /* Exempt debugging setuid proc. */ +#define PRIV_DEBUG_UNPRIV 82 /* Exempt unprivileged debug limit. */ +#define PRIV_DEBUG_DENIED 83 /* Exempt P2_NOTRACE. */ + +/* + * Dtrace privileges. + */ +#define PRIV_DTRACE_KERNEL 90 /* Allow use of DTrace on the kernel. */ +#define PRIV_DTRACE_PROC 91 /* Allow attaching DTrace to process. */ +#define PRIV_DTRACE_USER 92 /* Process may submit DTrace events. */ + +/* + * Firmware privilegs. + */ +#define PRIV_FIRMWARE_LOAD 100 /* Can load firmware. */ + +/* + * Jail privileges. + */ +#define PRIV_JAIL_ATTACH 110 /* Attach to a jail. */ +#define PRIV_JAIL_SET 111 /* Set jail parameters. */ +#define PRIV_JAIL_REMOVE 112 /* Remove a jail. */ + +/* + * Kernel environment priveleges. + */ +#define PRIV_KENV_SET 120 /* Set kernel env. variables. */ +#define PRIV_KENV_UNSET 121 /* Unset kernel env. variables. */ + +/* + * Loadable kernel module privileges. + */ +#define PRIV_KLD_LOAD 130 /* Load a kernel module. */ +#define PRIV_KLD_UNLOAD 131 /* Unload a kernel module. */ + +/* + * Privileges associated with the MAC Framework and specific MAC policy + * modules. + */ +#define PRIV_MAC_PARTITION 140 /* Privilege in mac_partition policy. */ +#define PRIV_MAC_PRIVS 141 /* Privilege in the mac_privs policy. */ + +/* + * Process-related privileges. + */ +#define PRIV_PROC_LIMIT 160 /* Exceed user process limit. */ +#define PRIV_PROC_SETLOGIN 161 /* Can call setlogin. */ +#define PRIV_PROC_SETRLIMIT 162 /* Can raise resources limits. */ +#define PRIV_PROC_SETLOGINCLASS 163 /* Can call setloginclass(2). */ + +/* + * System V IPC privileges. + */ +#define PRIV_IPC_READ 170 /* Can override IPC read perm. */ +#define PRIV_IPC_WRITE 171 /* Can override IPC write perm. */ +#define PRIV_IPC_ADMIN 172 /* Can override IPC owner-only perm. */ +#define PRIV_IPC_MSGSIZE 173 /* Exempt IPC message queue limit. */ + +/* + * POSIX message queue privileges. + */ +#define PRIV_MQ_ADMIN 180 /* Can override msgq owner-only perm. */ + +/* + * Performance monitoring counter privileges. + */ +#define PRIV_PMC_MANAGE 190 /* Can administer PMC. */ +#define PRIV_PMC_SYSTEM 191 /* Can allocate a system-wide PMC. */ + +/* + * Scheduling privileges. + */ +#define PRIV_SCHED_DIFFCRED 200 /* Exempt scheduling other users. */ +#define PRIV_SCHED_SETPRIORITY 201 /* Can set lower nice value for proc. */ +#define PRIV_SCHED_RTPRIO 202 /* Can set real time scheduling. */ +#define PRIV_SCHED_SETPOLICY 203 /* Can set scheduler policy. */ +#define PRIV_SCHED_SET 204 /* Can set thread scheduler. */ +#define PRIV_SCHED_SETPARAM 205 /* Can set thread scheduler params. */ +#define PRIV_SCHED_CPUSET 206 /* Can manipulate cpusets. */ +#define PRIV_SCHED_CPUSET_INTR 207 /* Can adjust IRQ to CPU binding. */ + +/* + * POSIX semaphore privileges. + */ +#define PRIV_SEM_WRITE 220 /* Can override sem write perm. */ + +/* + * Signal privileges. + */ +#define PRIV_SIGNAL_DIFFCRED 230 /* Exempt signalling other users. */ +#define PRIV_SIGNAL_SUGID 231 /* Non-conserv signal setuid proc. */ + +/* + * Sysctl privileges. + */ +#define PRIV_SYSCTL_DEBUG 240 /* Can invoke sysctl.debug. */ +#define PRIV_SYSCTL_WRITE 241 /* Can write sysctls. */ +#define PRIV_SYSCTL_WRITEJAIL 242 /* Can write sysctls, jail permitted. */ + +/* + * TTY privileges. + */ +#define PRIV_TTY_CONSOLE 250 /* Set console to tty. */ +#define PRIV_TTY_DRAINWAIT 251 /* Set tty drain wait time. */ +#define PRIV_TTY_DTRWAIT 252 /* Set DTR wait on tty. */ +#define PRIV_TTY_EXCLUSIVE 253 /* Override tty exclusive flag. */ +#define _PRIV_TTY_PRISON 254 /* Removed. */ +#define PRIV_TTY_STI 255 /* Simulate input on another tty. */ +#define PRIV_TTY_SETA 256 /* Set tty termios structure. */ + +/* + * UFS-specific privileges. + */ +#define PRIV_UFS_EXTATTRCTL 270 /* Can configure EAs on UFS1. */ +#define PRIV_UFS_QUOTAOFF 271 /* quotaoff(). */ +#define PRIV_UFS_QUOTAON 272 /* quotaon(). */ +#define PRIV_UFS_SETUSE 273 /* setuse(). */ + +/* + * ZFS-specific privileges. + */ +#define PRIV_ZFS_POOL_CONFIG 280 /* Can configure ZFS pools. */ +#define PRIV_ZFS_INJECT 281 /* Can inject faults to framework. */ +#define PRIV_ZFS_JAIL 282 /* Can attach/detach to/from jails. */ + +/* + * NFS-specific privileges. + */ +#define PRIV_NFS_DAEMON 290 /* Can become the NFS daemon. */ +#define PRIV_NFS_LOCKD 291 /* Can become NFS lock daemon. */ + +/* + * VFS privileges. + */ +#define PRIV_VFS_READ 310 /* Override vnode DAC read perm. */ +#define PRIV_VFS_WRITE 311 /* Override vnode DAC write perm. */ +#define PRIV_VFS_ADMIN 312 /* Override vnode DAC admin perm. */ +#define PRIV_VFS_EXEC 313 /* Override vnode DAC exec perm. */ +#define PRIV_VFS_LOOKUP 314 /* Override vnode DAC lookup perm. */ +#define PRIV_VFS_BLOCKRESERVE 315 /* Can use free block reserve. */ +#define PRIV_VFS_CHFLAGS_DEV 316 /* Can chflags() a device node. */ +#define PRIV_VFS_CHOWN 317 /* Can set user; group to non-member. */ +#define PRIV_VFS_CHROOT 318 /* chroot(). */ +#define PRIV_VFS_RETAINSUGID 319 /* Can retain sugid bits on change. */ +#define PRIV_VFS_EXCEEDQUOTA 320 /* Exempt from quota restrictions. */ +#define PRIV_VFS_EXTATTR_SYSTEM 321 /* Operate on system EA namespace. */ +#define PRIV_VFS_FCHROOT 322 /* fchroot(). */ +#define PRIV_VFS_FHOPEN 323 /* Can fhopen(). */ +#define PRIV_VFS_FHSTAT 324 /* Can fhstat(). */ +#define PRIV_VFS_FHSTATFS 325 /* Can fhstatfs(). */ +#define PRIV_VFS_GENERATION 326 /* stat() returns generation number. */ +#define PRIV_VFS_GETFH 327 /* Can retrieve file handles. */ +#define PRIV_VFS_GETQUOTA 328 /* getquota(). */ +#define PRIV_VFS_LINK 329 /* bsd.hardlink_check_uid */ +#define PRIV_VFS_MKNOD_BAD 330 /* Can mknod() to mark bad inodes. */ +#define PRIV_VFS_MKNOD_DEV 331 /* Can mknod() to create dev nodes. */ +#define PRIV_VFS_MKNOD_WHT 332 /* Can mknod() to create whiteout. */ +#define PRIV_VFS_MOUNT 333 /* Can mount(). */ +#define PRIV_VFS_MOUNT_OWNER 334 /* Can manage other users' fs. */ +#define PRIV_VFS_MOUNT_EXPORTED 335 /* Can set MNT_EXPORTED on mount. */ +#define PRIV_VFS_MOUNT_PERM 336 /* Override dev node perms at mount. */ +#define PRIV_VFS_MOUNT_SUIDDIR 337 /* Can set MNT_SUIDDIR on mount. */ +#define PRIV_VFS_MOUNT_NONUSER 338 /* Can perform a non-user mount. */ +#define PRIV_VFS_SETGID 339 /* Can setgid if not in group. */ +#define PRIV_VFS_SETQUOTA 340 /* setquota(). */ +#define PRIV_VFS_STICKYFILE 341 /* Can set sticky bit on file. */ +#define PRIV_VFS_SYSFLAGS 342 /* Can modify system flags. */ +#define PRIV_VFS_UNMOUNT 343 /* Can unmount(). */ +#define PRIV_VFS_STAT 344 /* Override vnode MAC stat perm. */ + +/* + * Virtual memory privileges. + */ +#define PRIV_VM_MADV_PROTECT 360 /* Can set MADV_PROTECT. */ +#define PRIV_VM_MLOCK 361 /* Can mlock(), mlockall(). */ +#define PRIV_VM_MUNLOCK 362 /* Can munlock(), munlockall(). */ +#define PRIV_VM_SWAP_NOQUOTA 363 +#define PRIV_VM_SWAP_NORLIMIT 364 + +/* + * Device file system privileges. + */ +#define PRIV_DEVFS_RULE 370 /* Can manage devfs rules. */ +#define PRIV_DEVFS_SYMLINK 371 /* Can create symlinks in devfs. */ + +/* + * Random number generator privileges. + */ +#define PRIV_RANDOM_RESEED 380 /* Closing /dev/random reseeds. */ + +/* + * Network stack privileges. + */ +#define PRIV_NET_BRIDGE 390 /* Administer bridge. */ +#define PRIV_NET_GRE 391 /* Administer GRE. */ +#define _PRIV_NET_PPP 392 /* Removed. */ +#define _PRIV_NET_SLIP 393 /* Removed. */ +#define PRIV_NET_BPF 394 /* Monitor BPF. */ +#define PRIV_NET_RAW 395 /* Open raw socket. */ +#define PRIV_NET_ROUTE 396 /* Administer routing. */ +#define PRIV_NET_TAP 397 /* Can open tap device. */ +#define PRIV_NET_SETIFMTU 398 /* Set interface MTU. */ +#define PRIV_NET_SETIFFLAGS 399 /* Set interface flags. */ +#define PRIV_NET_SETIFCAP 400 /* Set interface capabilities. */ +#define PRIV_NET_SETIFNAME 401 /* Set interface name. */ +#define PRIV_NET_SETIFMETRIC 402 /* Set interface metrics. */ +#define PRIV_NET_SETIFPHYS 403 /* Set interface physical layer prop. */ +#define PRIV_NET_SETIFMAC 404 /* Set interface MAC label. */ +#define PRIV_NET_ADDMULTI 405 /* Add multicast addr. to ifnet. */ +#define PRIV_NET_DELMULTI 406 /* Delete multicast addr. from ifnet. */ +#define PRIV_NET_HWIOCTL 407 /* Issue hardware ioctl on ifnet. */ +#define PRIV_NET_SETLLADDR 408 /* Set interface link-level address. */ +#define PRIV_NET_ADDIFGROUP 409 /* Add new interface group. */ +#define PRIV_NET_DELIFGROUP 410 /* Delete interface group. */ +#define PRIV_NET_IFCREATE 411 /* Create cloned interface. */ +#define PRIV_NET_IFDESTROY 412 /* Destroy cloned interface. */ +#define PRIV_NET_ADDIFADDR 413 /* Add protocol addr to interface. */ +#define PRIV_NET_DELIFADDR 414 /* Delete protocol addr on interface. */ +#define PRIV_NET_LAGG 415 /* Administer lagg interface. */ +#define PRIV_NET_GIF 416 /* Administer gif interface. */ +#define PRIV_NET_SETIFVNET 417 /* Move interface to vnet. */ +#define PRIV_NET_SETIFDESCR 418 /* Set interface description. */ +#define PRIV_NET_SETIFFIB 419 /* Set interface fib. */ +#define PRIV_NET_VXLAN 420 /* Administer vxlan. */ + +/* + * 802.11-related privileges. + */ +#define PRIV_NET80211_GETKEY 440 /* Query 802.11 keys. */ +#define PRIV_NET80211_MANAGE 441 /* Administer 802.11. */ + +/* + * Placeholder for AppleTalk privileges, not supported anymore. + */ +#define _PRIV_NETATALK_RESERVEDPORT 450 /* Bind low port number. */ + +/* + * ATM privileges. + */ +#define PRIV_NETATM_CFG 460 +#define PRIV_NETATM_ADD 461 +#define PRIV_NETATM_DEL 462 +#define PRIV_NETATM_SET 463 + +/* + * Bluetooth privileges. + */ +#define PRIV_NETBLUETOOTH_RAW 470 /* Open raw bluetooth socket. */ + +/* + * Netgraph and netgraph module privileges. + */ +#define PRIV_NETGRAPH_CONTROL 480 /* Open netgraph control socket. */ +#define PRIV_NETGRAPH_TTY 481 /* Configure tty for netgraph. */ + +/* + * IPv4 and IPv6 privileges. + */ +#define PRIV_NETINET_RESERVEDPORT 490 /* Bind low port number. */ +#define PRIV_NETINET_IPFW 491 /* Administer IPFW firewall. */ +#define PRIV_NETINET_DIVERT 492 /* Open IP divert socket. */ +#define PRIV_NETINET_PF 493 /* Administer pf firewall. */ +#define PRIV_NETINET_DUMMYNET 494 /* Administer DUMMYNET. */ +#define PRIV_NETINET_CARP 495 /* Administer CARP. */ +#define PRIV_NETINET_MROUTE 496 /* Administer multicast routing. */ +#define PRIV_NETINET_RAW 497 /* Open netinet raw socket. */ +#define PRIV_NETINET_GETCRED 498 /* Query netinet pcb credentials. */ +#define PRIV_NETINET_ADDRCTRL6 499 /* Administer IPv6 address scopes. */ +#define PRIV_NETINET_ND6 500 /* Administer IPv6 neighbor disc. */ +#define PRIV_NETINET_SCOPE6 501 /* Administer IPv6 address scopes. */ +#define PRIV_NETINET_ALIFETIME6 502 /* Administer IPv6 address lifetimes. */ +#define PRIV_NETINET_IPSEC 503 /* Administer IPSEC. */ +#define PRIV_NETINET_REUSEPORT 504 /* Allow [rapid] port/address reuse. */ +#define PRIV_NETINET_SETHDROPTS 505 /* Set certain IPv4/6 header options. */ +#define PRIV_NETINET_BINDANY 506 /* Allow bind to any address. */ +#define PRIV_NETINET_HASHKEY 507 /* Get and set hash keys for IPv4/6. */ + +/* + * Placeholders for IPX/SPX privileges, not supported any more. + */ +#define _PRIV_NETIPX_RESERVEDPORT 520 /* Bind low port number. */ +#define _PRIV_NETIPX_RAW 521 /* Open netipx raw socket. */ + +/* + * NCP privileges. + */ +#define PRIV_NETNCP 530 /* Use another user's connection. */ + +/* + * SMB privileges. + */ +#define PRIV_NETSMB 540 /* Use another user's connection. */ + +/* + * VM86 privileges. + */ +#define PRIV_VM86_INTCALL 550 /* Allow invoking vm86 int handlers. */ + +/* + * Set of reserved privilege values, which will be allocated to code as + * needed, in order to avoid renumbering later privileges due to insertion. + */ +#define _PRIV_RESERVED0 560 +#define _PRIV_RESERVED1 561 +#define _PRIV_RESERVED2 562 +#define _PRIV_RESERVED3 563 +#define _PRIV_RESERVED4 564 +#define _PRIV_RESERVED5 565 +#define _PRIV_RESERVED6 566 +#define _PRIV_RESERVED7 567 +#define _PRIV_RESERVED8 568 +#define _PRIV_RESERVED9 569 +#define _PRIV_RESERVED10 570 +#define _PRIV_RESERVED11 571 +#define _PRIV_RESERVED12 572 +#define _PRIV_RESERVED13 573 +#define _PRIV_RESERVED14 574 +#define _PRIV_RESERVED15 575 + +/* + * Define a set of valid privilege numbers that can be used by loadable + * modules that don't yet have privilege reservations. Ideally, these should + * not be used, since their meaning is opaque to any policies that are aware + * of specific privileges, such as jail, and as such may be arbitrarily + * denied. + */ +#define PRIV_MODULE0 600 +#define PRIV_MODULE1 601 +#define PRIV_MODULE2 602 +#define PRIV_MODULE3 603 +#define PRIV_MODULE4 604 +#define PRIV_MODULE5 605 +#define PRIV_MODULE6 606 +#define PRIV_MODULE7 607 +#define PRIV_MODULE8 608 +#define PRIV_MODULE9 609 +#define PRIV_MODULE10 610 +#define PRIV_MODULE11 611 +#define PRIV_MODULE12 612 +#define PRIV_MODULE13 613 +#define PRIV_MODULE14 614 +#define PRIV_MODULE15 615 + +/* + * DDB(4) privileges. + */ +#define PRIV_DDB_CAPTURE 620 /* Allow reading of DDB capture log. */ + +/* + * Arla/nnpfs privileges. + */ +#define PRIV_NNPFS_DEBUG 630 /* Perforn ARLA_VIOC_NNPFSDEBUG. */ + +/* + * cpuctl(4) privileges. + */ +#define PRIV_CPUCTL_WRMSR 640 /* Write model-specific register. */ +#define PRIV_CPUCTL_UPDATE 641 /* Update cpu microcode. */ + +/* + * Capi4BSD privileges. + */ +#define PRIV_C4B_RESET_CTLR 650 /* Load firmware, reset controller. */ +#define PRIV_C4B_TRACE 651 /* Unrestricted CAPI message tracing. */ + +/* + * OpenAFS privileges. + */ +#define PRIV_AFS_ADMIN 660 /* Can change AFS client settings. */ +#define PRIV_AFS_DAEMON 661 /* Can become the AFS daemon. */ + +/* + * Resource Limits privileges. + */ +#define PRIV_RCTL_GET_RACCT 670 +#define PRIV_RCTL_GET_RULES 671 +#define PRIV_RCTL_GET_LIMITS 672 +#define PRIV_RCTL_ADD_RULE 673 +#define PRIV_RCTL_REMOVE_RULE 674 + +/* + * mem(4) privileges. + */ +#define PRIV_KMEM_READ 680 /* Open mem/kmem for reading. */ +#define PRIV_KMEM_WRITE 681 /* Open mem/kmem for writing. */ + +/* + * Track end of privilege list. + */ +#define _PRIV_HIGHEST 682 + +/* + * Validate that a named privilege is known by the privilege system. Invalid + * privileges presented to the privilege system by a priv_check interface + * will result in a panic. This is only approximate due to sparse allocation + * of the privilege space. + */ +#define PRIV_VALID(x) ((x) > _PRIV_LOWEST && (x) < _PRIV_HIGHEST) + +#ifdef _KERNEL +/* + * Privilege check interfaces, modeled after historic suser() interfaces, but + * with the addition of a specific privilege name. No flags are currently + * defined for the API. Historically, flags specified using the real uid + * instead of the effective uid, and whether or not the check should be + * allowed in jail. + */ +struct thread; +struct ucred; +int priv_check(struct thread *td, int priv); +int priv_check_cred(struct ucred *cred, int priv, int flags); +#endif + +#endif /* _SPL_PRIV_H */ diff --git a/include/os/windows/spl/sys/proc.h b/include/os/windows/spl/sys/proc.h new file mode 100644 index 000000000000..40078d9b7e50 --- /dev/null +++ b/include/os/windows/spl/sys/proc.h @@ -0,0 +1,46 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_PROC_H +#define _SPL_PROC_H + +#include + +typedef struct _KPROCESS proc_t; + +extern proc_t p0; + +#define current_proc PsGetCurrentProcess +#define getpid() (pid_t)(uintptr_t)PsGetProcessId(PsGetCurrentProcess()) + +static inline boolean_t +zfs_proc_is_caller(proc_t *p) +{ + return (p == PsGetCurrentProcess()); +} + +static inline char * +getcomm(void) +{ + return ("procname"); // WIN32 me +} + +#endif /* SPL_PROC_H */ diff --git a/include/os/windows/spl/sys/processor.h b/include/os/windows/spl/sys/processor.h new file mode 100644 index 000000000000..c1710374bfed --- /dev/null +++ b/include/os/windows/spl/sys/processor.h @@ -0,0 +1,29 @@ + +#ifndef _SPL_PROCESSOR_H +#define _SPL_PROCESSOR_H + +#include + +extern uint32_t getcpuid(); + +typedef int processorid_t; + +#define CPUID_FEATURE_PCLMULQDQ (1<<1) +#define CPUID_FEATURE_MOVBE (1<<22) +#define CPUID_FEATURE_AES (1<<25) +#define CPUID_FEATURE_XSAVE (1<<26) +#define CPUID_FEATURE_OSXSAVE (1<<27) +#define CPUID_FEATURE_AVX1_0 (1<<28) + +#define CPUID_FEATURE_SSE (1<<25) +#define CPUID_FEATURE_SSE2 (1<<26) +#define CPUID_FEATURE_SSE3 (1<<0) +#define CPUID_FEATURE_SSSE3 (1<<9) +#define CPUID_FEATURE_SSE4_2 (1<<20) +#define CPUID_FEATURE_SSE4_1 (1<<19) + +#define CPUID_LEAF7_FEATURE_AVX2 (1<<5) +#define CPUID_LEAF7_FEATURE_AVX512F (1<<16) +#define CPUID_LEAF7_FEATURE_SHA_NI (1<<29) + +#endif /* _SPL_PROCESSOR_H */ diff --git a/include/os/windows/spl/sys/procfs_list.h b/include/os/windows/spl/sys/procfs_list.h new file mode 100644 index 000000000000..e024aeb1c595 --- /dev/null +++ b/include/os/windows/spl/sys/procfs_list.h @@ -0,0 +1,66 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#ifndef _SPL_PROCFS_LIST_H +#define _SPL_PROCFS_LIST_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +typedef struct procfs_list procfs_list_t; +struct procfs_list { + void *pl_private; + void *pl_next_data; + kmutex_t pl_lock; + list_t pl_list; + uint64_t pl_next_id; + int (*pl_show)(struct seq_file *f, void *p); + int (*pl_show_header)(struct seq_file *f); + int (*pl_clear)(procfs_list_t *procfs_list); + size_t pl_node_offset; +}; + +typedef struct procfs_list_node { + list_node_t pln_link; + uint64_t pln_id; +} procfs_list_node_t; + +void procfs_list_install(const char *module, + const char *submodule, + const char *name, + mode_t mode, + procfs_list_t *procfs_list, + int (*show)(struct seq_file *f, void *p), + int (*show_header)(struct seq_file *f), + int (*clear)(procfs_list_t *procfs_list), + size_t procfs_list_node_off); +void procfs_list_uninstall(procfs_list_t *procfs_list); +void procfs_list_destroy(procfs_list_t *procfs_list); +void procfs_list_add(procfs_list_t *procfs_list, void *p); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/include/os/windows/spl/sys/random.h b/include/os/windows/spl/sys/random.h new file mode 100644 index 000000000000..902ca6a63b0f --- /dev/null +++ b/include/os/windows/spl/sys/random.h @@ -0,0 +1,49 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#ifndef _SPL_RANDOM_H +#define _SPL_RANDOM_H + +extern int random_get_bytes(uint8_t *ptr, uint32_t len); +#define random_get_pseudo_bytes random_get_bytes + +static inline uint32_t +random_in_range(uint32_t range) +{ + uint32_t r; + + ASSERT(range != 0); + + if (range == 1) + return (0); + + random_get_bytes((void *)&r, sizeof (r)); + + return (r % range); +} + +#endif /* _SPL_RANDOM_H */ diff --git a/include/os/windows/spl/sys/resource.h b/include/os/windows/spl/sys/resource.h new file mode 100644 index 000000000000..5e1bf347e615 --- /dev/null +++ b/include/os/windows/spl/sys/resource.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_RESOURCE_H +#define _SPL_RESOURCE_H + +#endif /* SPL_RESOURCE_H */ diff --git a/include/os/windows/spl/sys/rwlock.h b/include/os/windows/spl/sys/rwlock.h new file mode 100644 index 000000000000..222ee3848d68 --- /dev/null +++ b/include/os/windows/spl/sys/rwlock.h @@ -0,0 +1,71 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (C) 2017 Jorgen Lundman + */ + +#ifndef _SPL_RWLOCK_H +#define _SPL_RWLOCK_H + +#include + +typedef enum { + RW_DRIVER = 2, + RW_DEFAULT = 4 +} krw_type_t; + +typedef enum { + RW_NONE = 0, + RW_WRITER = 1, + RW_READER = 2 +} krw_t; + +struct krwlock { + ERESOURCE rw_lock; /* opaque data */ + void *rw_owner; /* writer (exclusive) lock only */ + int rw_readers; /* reader lock only */ + int rw_pad; +}; +typedef struct krwlock krwlock_t; + +#define RW_NOLOCKDEP 0 + +#define RW_READ_HELD(x) (rw_read_held((x))) +#define RW_WRITE_HELD(x) (rw_write_held((x))) +#define RW_LOCK_HELD(x) (rw_lock_held((x))) +#define RW_ISWRITER(x) (rw_iswriter(x)) + +extern void rw_init(krwlock_t *, char *, krw_type_t, void *); +extern void rw_destroy(krwlock_t *); +extern void rw_enter(krwlock_t *, krw_t); +extern int rw_tryenter(krwlock_t *, krw_t); +extern void rw_exit(krwlock_t *); +extern void rw_downgrade(krwlock_t *); +extern int rw_tryupgrade(krwlock_t *); +extern int rw_write_held(krwlock_t *); +extern int rw_lock_held(krwlock_t *); +extern int rw_isinit(krwlock_t *); +extern int rw_read_held(krwlock_t *); + +int spl_rwlock_init(void); +void spl_rwlock_fini(void); + +#endif /* _SPL_RWLOCK_H */ diff --git a/include/os/windows/spl/sys/sched.h b/include/os/windows/spl/sys/sched.h new file mode 100644 index 000000000000..c4955c20e998 --- /dev/null +++ b/include/os/windows/spl/sys/sched.h @@ -0,0 +1,27 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef WINDOWS_SCHED_H +#define WINDOWS_SCHED_H + +#include + +#endif diff --git a/include/os/windows/spl/sys/seg_kmem.h b/include/os/windows/spl/sys/seg_kmem.h new file mode 100644 index 000000000000..eda9ecb8a1c9 --- /dev/null +++ b/include/os/windows/spl/sys/seg_kmem.h @@ -0,0 +1,92 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _VM_SEG_KMEM_H +#define _VM_SEG_KMEM_H + + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +/* + * VM - Kernel Segment Driver + */ + +#if defined(_KERNEL) + +extern uint64_t segkmem_total_allocated; + +extern vmem_t *abd_arena; + +/* + * segkmem page vnodes + */ +#define kvp (kvps[KV_KVP]) +#define zvp (kvps[KV_ZVP]) +#if defined(__sparc) +#define mpvp (kvps[KV_MPVP]) +#define promvp (kvps[KV_PROMVP]) +#endif /* __sparc */ + +void *segkmem_alloc(vmem_t *, uint32_t, int); +extern void segkmem_free(vmem_t *, void *, uint32_t); +extern void kernelheap_init(void); +extern void kernelheap_fini(void); +extern void *segkmem_zio_alloc(vmem_t *, uint32_t, int); +extern void segkmem_zio_free(vmem_t *, void *, uint32_t); +extern void segkmem_zio_init(void); +extern void segkmem_zio_fini(void); +extern void segkmem_abd_init(void); +extern void segkmem_abd_fini(void); + +/* + * Flags for segkmem_xalloc(). + * + * SEGKMEM_SHARELOCKED requests pages which are locked SE_SHARED to be + * returned rather than unlocked which is now the default. Note that + * memory returned by SEGKMEM_SHARELOCKED cannot be freed by segkmem_free(). + * This is a hack for seg_dev that should be cleaned up in the future. + */ +#define SEGKMEM_SHARELOCKED 0x20000 + +/* + * Large page for kmem caches support + */ + +#define SEGKMEM_USE_LARGEPAGES (segkmem_lpsize > PAGESIZE) + +#define IS_KMEM_VA_LARGEPAGE(vaddr) \ + (((vaddr) >= heap_lp_base) && ((vaddr) < heap_lp_end)) + +#endif /* _KERNEL */ + +#ifdef __cplusplus +} +#endif + +#endif /* _VM_SEG_KMEM_H */ diff --git a/include/os/windows/spl/sys/sid.h b/include/os/windows/spl/sys/sid.h new file mode 100644 index 000000000000..fab1cdf22d16 --- /dev/null +++ b/include/os/windows/spl/sys/sid.h @@ -0,0 +1,103 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SPL_SID_H +#define _SPL_SID_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +typedef struct ksiddomain { + char *kd_name; +} ksiddomain_t; + +typedef enum ksid_index { + KSID_USER, + KSID_GROUP, + KSID_OWNER, + KSID_COUNT +} ksid_index_t; + +typedef int ksid_t; + +/* Should be in kidmap.h */ +typedef int32_t idmap_stat; + +static inline ksiddomain_t * +ksid_lookupdomain(const char *dom) +{ + ksiddomain_t *kd; + int len = strlen(dom); + + kd = (ksiddomain_t *)kmem_zalloc(sizeof (ksiddomain_t), KM_SLEEP); + kd->kd_name = (char *)kmem_zalloc(len + 1, KM_SLEEP); + memcpy(kd->kd_name, dom, len); + + return (kd); +} + +static inline void +ksiddomain_rele(ksiddomain_t *ksid) +{ + kmem_free(ksid->kd_name, strlen(ksid->kd_name) + 1); + kmem_free(ksid, sizeof (ksiddomain_t)); +} + +#define UID_NOBODY 65534 +#define GID_NOBODY 65534 + +static __inline uint_t +ksid_getid(ksid_t *ks) +{ + panic("%s has been unexpectedly called", __func__); + return (0); +} + +static __inline const char * +ksid_getdomain(ksid_t *ks) +{ + panic("%s has been unexpectedly called", __func__); + return (0); +} + +static __inline uint_t +ksid_getrid(ksid_t *ks) +{ + panic("%s has been unexpectedly called", __func__); + return (0); +} + +#define kidmap_getsidbyuid(zone, uid, sid_prefix, rid) (1) +#define kidmap_getsidbygid(zone, gid, sid_prefix, rid) (1) + +#ifdef __cplusplus +} +#endif + +#endif /* _SPL_SID_H */ diff --git a/include/os/windows/spl/sys/signal.h b/include/os/windows/spl/sys/signal.h new file mode 100644 index 000000000000..30ff1c4533d9 --- /dev/null +++ b/include/os/windows/spl/sys/signal.h @@ -0,0 +1,83 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ + +/* + * University Copyright- Copyright (c) 1982, 1986, 1988 + * The Regents of the University of California + * All Rights Reserved + * + * University Acknowledgment- Portions of this document are derived from + * software developed by the University of California, Berkeley, and its + * contributors. + */ + + +#ifndef _SPL_SIGNAL_H +#define _SPL_SIGNAL_H + +#include +#include +#include + +#define FORREAL 0 /* Usual side-effects */ +#define JUSTLOOKING 1 /* Don't stop the process */ + +struct proc; + +typedef struct __siginfo { + /* Windows version goes here */ + void *si_addr; +} siginfo_t; + +/* + * The "why" argument indicates the allowable side-effects of the call: + * + * FORREAL: Extract the next pending signal from p_sig into p_cursig; + * stop the process if a stop has been requested or if a traced signal + * is pending. + * + * JUSTLOOKING: Don't stop the process, just indicate whether or not + * a signal might be pending (FORREAL is needed to tell for sure). + */ +#define threadmask (sigmask(SIGILL)|sigmask(SIGTRAP)|\ + sigmask(SIGIOT)|sigmask(SIGEMT)| \ + sigmask(SIGFPE)|sigmask(SIGBUS)| \ + sigmask(SIGSEGV)|sigmask(SIGSYS)| \ + sigmask(SIGPIPE)|sigmask(SIGKILL)| \ + sigmask(SIGTERM)|sigmask(SIGINT)) + +static inline int +issig(int why) +{ + return (0); +} + +#define signal_pending(p) issig(0) + +#endif /* SPL_SIGNAL_H */ diff --git a/include/os/windows/spl/sys/simd.h b/include/os/windows/spl/sys/simd.h new file mode 100644 index 000000000000..cf13da1ceab0 --- /dev/null +++ b/include/os/windows/spl/sys/simd.h @@ -0,0 +1,757 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (C) 2016 Gvozden Neskovic . + */ + +/* + * USER API: + * + * Kernel fpu methods: + * kfpu_begin() + * kfpu_end() + * + * SIMD support: + * + * Following functions should be called to determine whether CPU feature + * is supported. All functions are usable in kernel and user space. + * If a SIMD algorithm is using more than one instruction set + * all relevant feature test functions should be called. + * + * Supported features: + * zfs_sse_available() + * zfs_sse2_available() + * zfs_sse3_available() + * zfs_ssse3_available() + * zfs_sse4_1_available() + * zfs_sse4_2_available() + * + * zfs_avx_available() + * zfs_avx2_available() + * + * zfs_bmi1_available() + * zfs_bmi2_available() + * + * zfs_avx512f_available() + * zfs_avx512cd_available() + * zfs_avx512er_available() + * zfs_avx512pf_available() + * zfs_avx512bw_available() + * zfs_avx512dq_available() + * zfs_avx512vl_available() + * zfs_avx512ifma_available() + * zfs_avx512vbmi_available() + * + * NOTE(AVX-512VL): If using AVX-512 instructions with 128Bit registers + * also add zfs_avx512vl_available() to feature check. + */ + +#ifndef _SIMD_X86_H +#define _SIMD_X86_H + +#include +#include + +/* only for __x86 */ +#if defined(__x86) + +#include + +#if defined(_KERNEL) + +#ifndef __clang__ +#include +#endif + +#ifdef _WIN32 +// XNU fpu.h +static inline uint64_t +xgetbv(uint32_t c) +{ + uint32_t mask_hi, mask_lo; + __asm__ __volatile__("xgetbv" : "=a"(mask_lo), "=d"(mask_hi) : "c" (c)); + return (((uint64_t)mask_hi<<32) + (uint64_t)mask_lo); +} + +#endif + +extern uint64_t spl_cpuid_features(void); +extern uint64_t spl_cpuid_leaf7_features(void); + +#define ZFS_ASM_BUG() { ASSERT(0); } break + +#define kfpu_allowed() 1 + +#endif + +#define kfpu_init() (0) +#define kfpu_fini() do {} while (0) + +extern uint32_t kfpu_state; + +#define kfpu_begin() \ + NTSTATUS saveStatus = STATUS_INVALID_PARAMETER; \ + XSTATE_SAVE SaveState; \ + saveStatus = KeSaveExtendedProcessorState(kfpu_state, &SaveState); + +#define kfpu_end() \ + if (NT_SUCCESS(saveStatus)) \ + KeRestoreExtendedProcessorState(&SaveState); + +#define kfpu_begin_ctx(O) \ + (O)->saveStatus = \ + KeSaveExtendedProcessorState(kfpu_state, &(O)->SaveState); + +#define kfpu_end_ctx(O) \ + if (NT_SUCCESS(((O)->saveStatus))) \ + KeRestoreExtendedProcessorState(&(O)->SaveState); + +/* + * CPUID feature tests for user-space. Linux kernel provides an interface for + * CPU feature testing. + */ +#if !defined(_KERNEL) + +#include + +#define ZFS_ASM_BUG() { assert(0); } break + +/* + * x86 registers used implicitly by CPUID + */ +typedef enum cpuid_regs { + EAX = 0, + EBX, + ECX, + EDX, + CPUID_REG_CNT = 4 +} cpuid_regs_t; + +/* + * List of instruction sets identified by CPUID + */ +typedef enum cpuid_inst_sets { + SSE = 0, + SSE2, + SSE3, + SSSE3, + SSE4_1, + SSE4_2, + OSXSAVE, + AVX, + AVX2, + BMI1, + BMI2, + AVX512F, + AVX512CD, + AVX512DQ, + AVX512BW, + AVX512IFMA, + AVX512VBMI, + AVX512PF, + AVX512ER, + AVX512VL, + AES, + PCLMULQDQ +} cpuid_inst_sets_t; + +/* + * Instruction set descriptor. + */ +typedef struct cpuid_feature_desc { + uint32_t leaf; /* CPUID leaf */ + uint32_t subleaf; /* CPUID sub-leaf */ + uint32_t flag; /* bit mask of the feature */ + cpuid_regs_t reg; /* which CPUID return register to test */ +} cpuid_feature_desc_t; + +#define _AVX512F_BIT (1U << 16) +#define _AVX512CD_BIT (_AVX512F_BIT | (1U << 28)) +#define _AVX512DQ_BIT (_AVX512F_BIT | (1U << 17)) +#define _AVX512BW_BIT (_AVX512F_BIT | (1U << 30)) +#define _AVX512IFMA_BIT (_AVX512F_BIT | (1U << 21)) +#define _AVX512VBMI_BIT (1U << 1) /* AVX512F_BIT is on another leaf */ +#define _AVX512PF_BIT (_AVX512F_BIT | (1U << 26)) +#define _AVX512ER_BIT (_AVX512F_BIT | (1U << 27)) +#define _AVX512VL_BIT (1U << 31) /* if used also check other levels */ +#define _AES_BIT (1U << 25) +#define _PCLMULQDQ_BIT (1U << 1) + +/* + * Descriptions of supported instruction sets + */ +static const cpuid_feature_desc_t spl_cpuid_features[] = { + [SSE] = {1U, 0U, 1U << 25, EDX }, + [SSE2] = {1U, 0U, 1U << 26, EDX }, + [SSE3] = {1U, 0U, 1U << 0, ECX }, + [SSSE3] = {1U, 0U, 1U << 9, ECX }, + [SSE4_1] = {1U, 0U, 1U << 19, ECX }, + [SSE4_2] = {1U, 0U, 1U << 20, ECX }, + [OSXSAVE] = {1U, 0U, 1U << 27, ECX }, + [AVX] = {1U, 0U, 1U << 28, ECX }, + [AVX2] = {7U, 0U, 1U << 5, EBX }, + [BMI1] = {7U, 0U, 1U << 3, EBX }, + [BMI2] = {7U, 0U, 1U << 8, EBX }, + [AVX512F] = {7U, 0U, _AVX512F_BIT, EBX }, + [AVX512CD] = {7U, 0U, _AVX512CD_BIT, EBX }, + [AVX512DQ] = {7U, 0U, _AVX512DQ_BIT, EBX }, + [AVX512BW] = {7U, 0U, _AVX512BW_BIT, EBX }, + [AVX512IFMA] = {7U, 0U, _AVX512IFMA_BIT, EBX }, + [AVX512VBMI] = {7U, 0U, _AVX512VBMI_BIT, ECX }, + [AVX512PF] = {7U, 0U, _AVX512PF_BIT, EBX }, + [AVX512ER] = {7U, 0U, _AVX512ER_BIT, EBX }, + [AVX512VL] = {7U, 0U, _AVX512ER_BIT, EBX }, + [AES] = {1U, 0U, _AES_BIT, ECX }, + [PCLMULQDQ] = {1U, 0U, _PCLMULQDQ_BIT, ECX }, +}; + +/* + * Check if OS supports AVX and AVX2 by checking XCR0 + * Only call this function if CPUID indicates that AVX feature is + * supported by the CPU, otherwise it might be an illegal instruction. + */ +static inline uint64_t +xgetbv(uint32_t index) +{ + uint32_t eax, edx; + /* xgetbv - instruction byte code */ + __asm__ __volatile__(".byte 0x0f; .byte 0x01; .byte 0xd0" + : "=a" (eax), "=d" (edx) + : "c" (index)); + + return ((((uint64_t)edx)<<32) | (uint64_t)eax); +} + +/* + * Check if CPU supports a feature + */ +static inline boolean_t +__cpuid_check_feature(const cpuid_feature_desc_t *desc) +{ + uint32_t r[CPUID_REG_CNT]; + + if (__get_cpuid_max(0, NULL) >= desc->leaf) { + /* + * __cpuid_count is needed to properly check + * for AVX2. It is a macro, so return parameters + * are passed by value. + */ + __cpuid_count(desc->leaf, desc->subleaf, + r[EAX], r[EBX], r[ECX], r[EDX]); + return ((r[desc->reg] & desc->flag) == desc->flag); + } + return (B_FALSE); +} + +#define CPUID_FEATURE_CHECK(name, id) \ +static inline boolean_t \ +__cpuid_has_ ## name(void) \ +{ \ + return (__cpuid_check_feature(&spl_cpuid_features[id])); \ +} + +/* + * Define functions for user-space CPUID features testing + */ +CPUID_FEATURE_CHECK(sse, SSE); +CPUID_FEATURE_CHECK(sse2, SSE2); +CPUID_FEATURE_CHECK(sse3, SSE3); +CPUID_FEATURE_CHECK(ssse3, SSSE3); +CPUID_FEATURE_CHECK(sse4_1, SSE4_1); +CPUID_FEATURE_CHECK(sse4_2, SSE4_2); +CPUID_FEATURE_CHECK(avx, AVX); +CPUID_FEATURE_CHECK(avx2, AVX2); +CPUID_FEATURE_CHECK(osxsave, OSXSAVE); +CPUID_FEATURE_CHECK(bmi1, BMI1); +CPUID_FEATURE_CHECK(bmi2, BMI2); +CPUID_FEATURE_CHECK(avx512f, AVX512F); +CPUID_FEATURE_CHECK(avx512cd, AVX512CD); +CPUID_FEATURE_CHECK(avx512dq, AVX512DQ); +CPUID_FEATURE_CHECK(avx512bw, AVX512BW); +CPUID_FEATURE_CHECK(avx512ifma, AVX512IFMA); +CPUID_FEATURE_CHECK(avx512vbmi, AVX512VBMI); +CPUID_FEATURE_CHECK(avx512pf, AVX512PF); +CPUID_FEATURE_CHECK(avx512er, AVX512ER); +CPUID_FEATURE_CHECK(avx512vl, AVX512VL); +CPUID_FEATURE_CHECK(aes, AES); +CPUID_FEATURE_CHECK(pclmulqdq, PCLMULQDQ); + +#endif /* !defined(_KERNEL) */ + + +/* + * Detect register set support + */ +static inline boolean_t +__simd_state_enabled(const uint64_t state) +{ + boolean_t has_osxsave; + uint64_t xcr0; + +#if defined(_KERNEL) + has_osxsave = !!(spl_cpuid_features() & CPUID_FEATURE_OSXSAVE); +#elif !defined(_KERNEL) + has_osxsave = __cpuid_has_osxsave(); +#endif + if (!has_osxsave) + return (B_FALSE); + + xcr0 = xgetbv(0); + return ((xcr0 & state) == state); +} + +#define _XSTATE_SSE_AVX (0x2 | 0x4) +#define _XSTATE_AVX512 (0xE0 | _XSTATE_SSE_AVX) + +#define __ymm_enabled() __simd_state_enabled(_XSTATE_SSE_AVX) +#define __zmm_enabled() __simd_state_enabled(_XSTATE_AVX512) + + +/* + * Check if SSE instruction set is available + */ +static inline boolean_t +zfs_sse_available(void) +{ +#if defined(_KERNEL) + return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE)); +#elif !defined(_KERNEL) + return (__cpuid_has_sse()); +#endif +} + +/* + * Check if SSE2 instruction set is available + */ +static inline boolean_t +zfs_sse2_available(void) +{ +#if defined(_KERNEL) + return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE2)); +#elif !defined(_KERNEL) + return (__cpuid_has_sse2()); +#endif +} + +/* + * Check if SSE3 instruction set is available + */ +static inline boolean_t +zfs_sse3_available(void) +{ +#if defined(_KERNEL) + return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE3)); +#elif !defined(_KERNEL) + return (__cpuid_has_sse3()); +#endif +} + +/* + * Check if SSSE3 instruction set is available + */ +static inline boolean_t +zfs_ssse3_available(void) +{ +#if defined(_KERNEL) + return (!!(spl_cpuid_features() & CPUID_FEATURE_SSSE3)); +#elif !defined(_KERNEL) + return (__cpuid_has_ssse3()); +#endif +} + +/* + * Check if SSE4.1 instruction set is available + */ +static inline boolean_t +zfs_sse4_1_available(void) +{ +#if defined(_KERNEL) + return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE4_1)); +#elif !defined(_KERNEL) + return (__cpuid_has_sse4_1()); +#endif +} + +/* + * Check if SSE4.2 instruction set is available + */ +static inline boolean_t +zfs_sse4_2_available(void) +{ +#if defined(_KERNEL) + return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE4_2)); +#elif !defined(_KERNEL) + return (__cpuid_has_sse4_2()); +#endif +} + +/* + * Check if AVX instruction set is available + */ +static inline boolean_t +zfs_avx_available(void) +{ + boolean_t has_avx; +#if defined(_KERNEL) + return (!!(spl_cpuid_features() & CPUID_FEATURE_AVX1_0)); +#elif !defined(_KERNEL) + has_avx = __cpuid_has_avx(); +#endif + + return (has_avx && __ymm_enabled()); +} + +/* + * Check if AVX2 instruction set is available + */ +static inline boolean_t +zfs_avx2_available(void) +{ + boolean_t has_avx2; +#if defined(_KERNEL) +#if defined(HAVE_AVX2) + has_avx2 = (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_AVX2)); +#else + has_avx2 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx2 = __cpuid_has_avx2(); +#endif + + return (has_avx2 && __ymm_enabled()); +} + +/* + * Check if BMI1 instruction set is available + */ +static inline boolean_t +zfs_bmi1_available(void) +{ +#if defined(_KERNEL) +#if defined(CPUID_LEAF7_FEATURE_BMI1) + return (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_BMI1)); +#else + return (B_FALSE); +#endif +#elif !defined(_KERNEL) + return (__cpuid_has_bmi1()); +#endif +} + +/* + * Check if BMI2 instruction set is available + */ +static inline boolean_t +zfs_bmi2_available(void) +{ +#if defined(_KERNEL) +#if defined(CPUID_LEAF7_FEATURE_BMI2) + return (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_BMI2)); +#else + return (B_FALSE); +#endif +#elif !defined(_KERNEL) + return (__cpuid_has_bmi2()); +#endif +} + +/* + * Check if AES instruction set is available + */ +static inline boolean_t +zfs_aes_available(void) +{ +#if defined(_KERNEL) +#if defined(HAVE_AES) + return (!!(spl_cpuid_features() & CPUID_FEATURE_AES)); +#else + return (B_FALSE); +#endif +#elif !defined(_KERNEL) + return (__cpuid_has_aes()); +#endif +} + +/* + * Check if PCLMULQDQ instruction set is available + */ +static inline boolean_t +zfs_pclmulqdq_available(void) +{ +#if defined(_KERNEL) +#if defined(HAVE_PCLMULQDQ) + return (!!(spl_cpuid_features() & CPUID_FEATURE_PCLMULQDQ)); +#else + return (B_FALSE); +#endif +#elif !defined(_KERNEL) + return (__cpuid_has_pclmulqdq()); +#endif +} + +/* + * AVX-512 family of instruction sets: + * + * AVX512F Foundation + * AVX512CD Conflict Detection Instructions + * AVX512ER Exponential and Reciprocal Instructions + * AVX512PF Prefetch Instructions + * + * AVX512BW Byte and Word Instructions + * AVX512DQ Double-word and Quadword Instructions + * AVX512VL Vector Length Extensions + * + * AVX512IFMA Integer Fused Multiply Add (Not supported by kernel 4.4) + * AVX512VBMI Vector Byte Manipulation Instructions + */ + + +/* Check if AVX512F instruction set is available */ +static inline boolean_t +zfs_avx512f_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512F) && defined(CPUID_LEAF7_FEATURE_AVX512F) + return (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_AVX512F)); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512f(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +/* Check if AVX512CD instruction set is available */ +static inline boolean_t +zfs_avx512cd_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512F) && defined(HAVE_AVX512CD) && \ + defined(CPUID_LEAF7_FEATURE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512CD) + has_avx512 = (spl_cpuid_leaf7_features() & + (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512CD)) == + (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512CD); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512cd(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +/* Check if AVX512ER instruction set is available */ +static inline boolean_t +zfs_avx512er_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512F) && defined(HAVE_AVX512ER) && \ + defined(CPUID_LEAF7_FEATURE_AVX512ER) + has_avx512 = (spl_cpuid_leaf7_features() & + (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512ER)) == + (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512ER); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512er(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +/* Check if AVX512PF instruction set is available */ +static inline boolean_t +zfs_avx512pf_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512PF) && defined(HAVE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512PF) + has_avx512 = (spl_cpuid_leaf7_features() & + (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512PF)) == + (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512PF); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512pf(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +/* Check if AVX512BW instruction set is available */ +static inline boolean_t +zfs_avx512bw_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512BW) && defined(HAVE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512BW) + has_avx512 = (spl_cpuid_leaf7_features() & + (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512BW)) == + (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512BW); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512bw(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +/* Check if AVX512DQ instruction set is available */ +static inline boolean_t +zfs_avx512dq_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512DQ) && defined(HAVE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512DQ) + has_avx512 = (spl_cpuid_leaf7_features() & + (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512DQ)) == + (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512DQ); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512dq(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +/* Check if AVX512VL instruction set is available */ +static inline boolean_t +zfs_avx512vl_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512VL) && defined(HAVE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512VL) + has_avx512 = (spl_cpuid_leaf7_features() & + (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512VL)) == + (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512VL); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512vl(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +/* Check if AVX512IFMA instruction set is available */ +static inline boolean_t +zfs_avx512ifma_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512IFMA) && defined(HAVE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512IFMA) + has_avx512 = (spl_cpuid_leaf7_features() & + (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512IFMA)) == + (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512IFMA); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512ifma(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +/* Check if AVX512VBMI instruction set is available */ +static inline boolean_t +zfs_avx512vbmi_available(void) +{ + boolean_t has_avx512 = B_FALSE; + +#if defined(_KERNEL) +#if defined(HAVE_AVX512VBMI) && defined(HAVE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512F) && \ + defined(CPUID_LEAF7_FEATURE_AVX512VBMI) + has_avx512 = (spl_cpuid_leaf7_features() & + (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512VBMI)) == + (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512VBMI); +#else + has_avx512 = B_FALSE; +#endif +#elif !defined(_KERNEL) + has_avx512 = __cpuid_has_avx512f() && + __cpuid_has_avx512vbmi(); +#endif + + return (has_avx512 && __zmm_enabled()); +} + +static inline boolean_t +zfs_movbe_available(void) +{ +#if defined(_KERNEL) +#if defined(HAVE_MOVBE) + return (!!(spl_cpuid_features() & CPUID_FEATURE_MOVBE)); +#else + return (B_FALSE); +#endif + return (B_FALSE); +#endif +} + +/* + * Check if SHA_NI instruction set is available + */ +static inline boolean_t +zfs_shani_available(void) +{ +#if defined(HAVE_SHA_NI) + return (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_SHA_NI)); +#else + return (B_FALSE); +#endif +} + +#endif /* defined(__x86) */ + +#endif /* _SIMD_X86_H */ diff --git a/include/os/windows/spl/sys/stat.h b/include/os/windows/spl/sys/stat.h new file mode 100644 index 000000000000..2eff738cd18f --- /dev/null +++ b/include/os/windows/spl/sys/stat.h @@ -0,0 +1,84 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_STAT_H +#define _SPL_STAT_H + +#ifndef S_IFMT +/* File type */ +#define S_IFMT 0170000 /* [XSI] type of file mask */ +#define S_IFIFO 0010000 /* [XSI] named pipe (fifo) */ +#define S_IFCHR 0020000 /* [XSI] character special */ +#define S_IFDIR 0040000 /* [XSI] directory */ +#define S_IFBLK 0060000 /* [XSI] block special */ +#define S_IFREG 0100000 /* [XSI] regular */ +#define S_IFLNK 0120000 /* [XSI] symbolic link */ +#define S_IFSOCK 0140000 /* [XSI] socket */ +#if !defined(_POSIX_C_SOURCE) +#define S_IFWHT 0160000 /* OBSOLETE: whiteout */ +#endif +/* File mode */ +/* Read, write, execute/search by owner */ +#define S_IRWXU 0000700 /* [XSI] RWX mask for owner */ +#define S_IRUSR 0000400 /* [XSI] R for owner */ +#define S_IWUSR 0000200 /* [XSI] W for owner */ +#define S_IXUSR 0000100 /* [XSI] X for owner */ +/* Read, write, execute/search by group */ +#define S_IRWXG 0000070 /* [XSI] RWX mask for group */ +#define S_IRGRP 0000040 /* [XSI] R for group */ +#define S_IWGRP 0000020 /* [XSI] W for group */ +#define S_IXGRP 0000010 /* [XSI] X for group */ +/* Read, write, execute/search by others */ +#define S_IRWXO 0000007 /* [XSI] RWX mask for other */ +#define S_IROTH 0000004 /* [XSI] R for other */ +#define S_IWOTH 0000002 /* [XSI] W for other */ +#define S_IXOTH 0000001 /* [XSI] X for other */ + +#define S_ISUID 0004000 /* [XSI] set user id on execution */ +#define S_ISGID 0002000 /* [XSI] set group id on execution */ +#define S_ISVTX 0001000 /* [XSI] directory restrcted delete */ + +#if !defined(_POSIX_C_SOURCE) +#define S_ISTXT S_ISVTX /* sticky bit: not supported */ +#define S_IREAD S_IRUSR /* backward compatability */ +#define S_IWRITE S_IWUSR /* backward compatability */ +#define S_IEXEC S_IXUSR /* backward compatability */ +#endif +#endif /* !S_IFMT */ + +/* + * [XSI] The following macros shall be provided to test whether a file is + * of the specified type. The value m supplied to the macros is the value + * of st_mode from a stat structure. The macro shall evaluate to a non-zero + * value if the test is true; 0 if the test is false. + */ +#define S_ISBLK(m) (((m)& S_IFMT) == S_IFBLK) /* block special */ +#define S_ISCHR(m) (((m)& S_IFMT) == S_IFCHR) /* char special */ +#define S_ISDIR(m) (((m)& S_IFMT) == S_IFDIR) /* directory */ +#define S_ISFIFO(m) (((m)& S_IFMT) == S_IFIFO) /* fifo or socket */ +#define S_ISREG(m) (((m)& S_IFMT) == S_IFREG) /* regular file */ +#define S_ISLNK(m) (((m)& S_IFMT) == S_IFLNK) /* symbolic link */ +#define S_ISSOCK(m) (((m)& S_IFMT) == S_IFSOCK) /* socket */ +#if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) +#define S_ISWHT(m) (((m)& S_IFMT) == S_IFWHT) /* OBSOLETE: whiteout */ +#endif + +#endif /* SPL_STAT_H */ diff --git a/include/os/windows/spl/sys/string.h b/include/os/windows/spl/sys/string.h new file mode 100644 index 000000000000..8d5b87aabdeb --- /dev/null +++ b/include/os/windows/spl/sys/string.h @@ -0,0 +1,29 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_STRING_H +#define _SPL_STRING_H + +size_t strlcpy(char *dst, const char *src, size_t dstsize); + +#include + +#endif /* SPL_DISP_H */ diff --git a/include/os/windows/spl/sys/stropts.h b/include/os/windows/spl/sys/stropts.h new file mode 100644 index 000000000000..41e77010ec1d --- /dev/null +++ b/include/os/windows/spl/sys/stropts.h @@ -0,0 +1,168 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2013 Jorgen Lundman + * + */ + +#ifndef _SPL_STROPTS_H +#define _SPL_STROPTS_H + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define isprint(c) ((c) >= ' ' && (c) <= '~') + +/* + * Find highest one bit set. + * Returns bit number + 1 of highest bit that is set, otherwise returns 0. + * High order bit is 31 (or 63 in _LP64 kernel). + */ +static inline int +highbit64(unsigned long long i) +{ + register int h = 1; + + if (i == 0) + return (0); + if (i & 0xffffffff00000000ull) { + h += 32; i >>= 32; + } + if (i & 0xffff0000) { + h += 16; i >>= 16; + } + if (i & 0xff00) { + h += 8; i >>= 8; + } + if (i & 0xf0) { + h += 4; i >>= 4; + } + if (i & 0xc) { + h += 2; i >>= 2; + } + if (i & 0x2) { + h += 1; + } + return (h); +} + +static inline int +highbit(unsigned long long i) +{ + register int h = 1; + + if (i == 0) + return (0); + if (i & 0xffffffff00000000ull) { + h += 32; i >>= 32; + } + if (i & 0xffff0000) { + h += 16; i >>= 16; + } + if (i & 0xff00) { + h += 8; i >>= 8; + } + if (i & 0xf0) { + h += 4; i >>= 4; + } + if (i & 0xc) { + h += 2; i >>= 2; + } + if (i & 0x2) { + h += 1; + } + return (h); +} + +/* + * Find lowest one bit set. + * Returns bit number + 1 of lowest bit that is set, otherwise returns 0. + * Low order bit is 0. + */ +static inline int +lowbit(unsigned long long i) +{ + register int h = 1; + + if (i == 0) + return (0); + + if (!(i & 0xffffffff)) { + h += 32; i >>= 32; + } + if (!(i & 0xffff)) { + h += 16; i >>= 16; + } + if (!(i & 0xff)) { + h += 8; i >>= 8; + } + if (!(i & 0xf)) { + h += 4; i >>= 4; + } + if (!(i & 0x3)) { + h += 2; i >>= 2; + } + if (!(i & 0x1)) { + h += 1; + } + return (h); +} + +static inline int +is_ascii_str(const char *str) +{ + unsigned char ch; + + while ((ch = (unsigned char)*str++) != '\0') { + if (ch >= 0x80) + return (0); + } + return (1); +} + +static inline void * +kmemchr(const void *s, int c, size_t n) +{ + if (n != 0) { + const unsigned char *p = (const unsigned char *)s; + do { + if (*p++ == (unsigned char)c) + return ((void *)(uintptr_t)(p - 1)); + } while (--n != 0); + } + return (NULL); +} + +#define LONG_BIT 64 +#define IDX(c) ((unsigned char)(c) / LONG_BIT) +#define BIT(c) ((unsigned long)1 << ((unsigned char)(c) % LONG_BIT)) + +#ifdef __cplusplus +} +#endif + +#endif /* SPL_STROPTS_H */ diff --git a/include/os/windows/spl/sys/sunddi.h b/include/os/windows/spl/sys/sunddi.h new file mode 100644 index 000000000000..1646a07a85f8 --- /dev/null +++ b/include/os/windows/spl/sys/sunddi.h @@ -0,0 +1,194 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 1990, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2012 Garrett D'Amore . All rights reserved. + * Copyright (c) 2012 by Delphix. All rights reserved. + */ + +#ifndef _SPL_SUNDDI_H +#define _SPL_SUNDDI_H + +#include +#include +#include +#include +#include +#include +#include + +typedef int ddi_devid_t; + +#define DDI_DEV_T_NONE ((dev_t)-1) +#define DDI_DEV_T_ANY ((dev_t)-2) +#define DI_MAJOR_T_UNKNOWN ((major_t)0) + +#define DDI_PROP_DONTPASS 0x0001 +#define DDI_PROP_CANSLEEP 0x0002 + +#define DDI_SUCCESS 0 +#define DDI_FAILURE -1 + +#define ddi_prop_lookup_string(x1, x2, x3, x4, x5) (*x5 = NULL) +#define ddi_prop_free(x) (void)0 +#define ddi_root_node() (void)0 + +#define isalnum(ch) (isalpha(ch) || isdigit(ch)) +#define isalpha(ch) (isupper(ch) || islower(ch)) +#define isdigit(ch) ((ch) >= '0' && (ch) <= '9') +#define islower(ch) ((ch) >= 'a' && (ch) <= 'z') +#define isspace(ch) (((ch) == ' ') || ((ch) == '\r') || ((ch) == '\n') || \ + ((ch) == '\t') || ((ch) == '\f')) +#define isupper(ch) ((ch) >= 'A' && (ch) <= 'Z') +#define isxdigit(ch) (isdigit(ch) || ((ch) >= 'a' && (ch) <= 'f') || \ + ((ch) >= 'A' && (ch) <= 'F')) +#define isgraph(C) ((C) >= 0x21 && (C) <= 0x7E) +#define ispunct(C) (((C) >= 0x21 && (C) <= 0x2F) || \ + ((C) >= 0x3A && (C) <= 0x40) || \ + ((C) >= 0x5B && (C) <= 0x60) || \ + ((C) >= 0x7B && (C) <= 0x7E)) + +#define DIGIT(x) \ + (isdigit(x) ? (x) - '0' : islower(x) ? (x) + 10 - 'a' : (x) + 10 - 'A') + +#define MBASE ('z' - 'a' + 1 + 10) +/* + * The following macro is a version of isalnum() that limits alphabetic + * characters to the ranges a-z and A-Z; locale dependent characters will not + * return 1. The members of a-z and A-Z are assumed to be in ascending order + * and contiguous. + */ +#define lisalnum(x) \ + (isdigit(x) || ((x) >= 'a' && (x) <= 'z') || ((x) >= 'A' && (x) <= 'Z')) + +// Define proper Solaris API calls, and clean ZFS up to use +#define copyin(from, to, len) ddi_copyin((from), (to), (len), 0) +int ddi_copyin(const void *from, void *to, size_t len, int flags); +#define copyout(from, to, len) ddi_copyout((from), (to), (len), 0) +int ddi_copyout(const void *from, void *to, size_t len, int flags); +#define copyinstr(from, to, len, done) \ + ddi_copyinstr((from), (to), (len), (done)) +int ddi_copyinstr(const void *uaddr, void *kaddr, size_t len, size_t *done); + +int ddi_copysetup(void *to, size_t len, void **out_buffer, PMDL *out_mdl); + +extern int ddi_strtol(const char *str, char **nptr, int base, long *result); + +#ifndef OTYPCNT +#define OTYPCNT 5 +#define OTYP_BLK 0 +#define OTYP_MNT 1 +#define OTYP_CHR 2 +#define OTYP_SWP 3 +#define OTYP_LYR 4 +#endif + +#define P2END(x, align) (-(~((uint64_t)x) & -((uint64_t)align))) + +#define ddi_name_to_major(name) devsw_name2blk(name, NULL, 0) + +struct dev_info { + dev_t dev; // Major / Minor + void *devc; + void *devb; +}; +typedef struct dev_info dev_info_t; + +int ddi_strtoull(const char *, char **, int, unsigned long long *); +int ddi_strtoll(const char *, char **, int, long long *); +int ddi_strtoul(const char *, char **, int, unsigned long *); +int ddi_strtol(const char *, char **, int, long *); +int ddi_soft_state_init(void **, uint32_t, uint32_t); +int ddi_soft_state_zalloc(void *, int); +void *ddi_get_soft_state(void *, int); +void ddi_soft_state_free(void *, int); +void ddi_soft_state_fini(void **); +int ddi_create_minor_node(dev_info_t *, char *, int, + minor_t, char *, int); +void ddi_remove_minor_node(dev_info_t *, char *); + +int ddi_driver_major(dev_info_t *); + +typedef void *ldi_ident_t; + +#define DDI_SUCCESS 0 +#define DDI_FAILURE -1 + +#define DDI_PSEUDO "" + +#define ddi_prop_update_int64(a, b, c, d) DDI_SUCCESS +#define ddi_prop_update_string(a, b, c, d) DDI_SUCCESS + +#define bioerror(bp, er) (buf_seterror((bp), (er))) +#define biodone(bp) buf_biodone(bp) + +static inline long ddi_fls(long mask) { \ + /* Algorithm courtesy of Steve Chessin. */ \ + while (mask) { \ + long nx; \ + if ((nx = (mask & (mask - 1))) == 0) \ + break; \ + mask = nx; \ + } \ + return (__lzcnt(mask)); \ +} + +// find_first_bits_de_bruijn(unsigned nums[ARRAY_SIZE]) +static inline long +ddi_ffs(long mask) +{ + static const int MultiplyDeBruijnBitPosition[32] = { + 1, 2, 29, 3, 30, 15, 25, 4, 31, 23, 21, 16, 26, 18, 5, 9, + 32, 28, 14, 24, 22, 20, 17, 8, 27, 13, 19, 7, 12, 6, 11, 10 + }; + return MultiplyDeBruijnBitPosition[ + ((unsigned)((mask & -mask) * 0x077CB531U)) >> 27]; +} + + + + +#define getminor(X) minor((X)) + + + +/* + * This data structure is entirely private to the soft state allocator. + */ +struct i_ddi_soft_state { + void **array; /* the array of pointers */ + kmutex_t lock; /* serialize access to this struct */ + uint32_t size; /* how many bytes per state struct */ + uint32_t n_items; /* how many structs herein */ + struct i_ddi_soft_state *next; /* 'dirty' elements */ +}; + +#define MIN_N_ITEMS 8 /* 8 void *'s == 32 bytes */ + +extern unsigned long long strlcpy(char *, const char *, unsigned long long); +extern size_t strlcat(char *dst, const char *s2, size_t n); + +uint32_t +ddi_strcspn(const char *__restrict s, const char *__restrict charset); +#define strcspn ddi_strcspn + +#endif /* SPL_SUNDDI_H */ diff --git a/include/os/windows/spl/sys/sunldi.h b/include/os/windows/spl/sys/sunldi.h new file mode 100644 index 000000000000..4b493cc5587f --- /dev/null +++ b/include/os/windows/spl/sys/sunldi.h @@ -0,0 +1,32 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 1994, 2010, Oracle and/or its affiliates. All rights reserved. + */ + +#ifndef _SPL_SUNLDI_H +#define _SPL_SUNLDI_H + +#include + +#define SECTOR_SIZE 512 + +#endif /* SPL_SUNLDI_H */ diff --git a/include/os/windows/spl/sys/sysevent.h b/include/os/windows/spl/sys/sysevent.h new file mode 100644 index 000000000000..6510297d601f --- /dev/null +++ b/include/os/windows/spl/sys/sysevent.h @@ -0,0 +1,36 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SYS_SYSEVENT_H +#define _SYS_SYSEVENT_H + +#include + +typedef struct sysevent { + nvlist_t *resource; +} sysevent_t; + +#endif diff --git a/include/os/windows/spl/sys/sysevent/eventdefs.h b/include/os/windows/spl/sys/sysevent/eventdefs.h new file mode 100644 index 000000000000..b16885e6ddfc --- /dev/null +++ b/include/os/windows/spl/sys/sysevent/eventdefs.h @@ -0,0 +1,140 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + * Copyright 2017 Joyent, Inc. + */ + +#ifndef _SYS_SYSEVENT_EVENTDEFS_H +#define _SYS_SYSEVENT_EVENTDEFS_H + +#ifdef __cplusplus +extern "C" { +#endif + +/* + * eventdefs.h contains public definitions for sysevent types (classes + * and subclasses). All additions/removal/changes are subject + * to PSARC approval. + */ + +/* Sysevent Class definitions */ +#define EC_NONE "EC_none" +#define EC_PRIV "EC_priv" +#define EC_PLATFORM "EC_platform" /* events private to platform */ +#define EC_DR "EC_dr" /* Dynamic reconfiguration event class */ +#define EC_ENV "EC_env" /* Environmental monitor event class */ +#define EC_DOMAIN "EC_domain" /* Domain event class */ +#define EC_AP_DRIVER "EC_ap_driver" /* Alternate Pathing event class */ +#define EC_IPMP "EC_ipmp" /* IP Multipathing event class */ +#define EC_DEV_ADD "EC_dev_add" /* device add event class */ +#define EC_DEV_REMOVE "EC_dev_remove" /* device remove event class */ +#define EC_DEV_BRANCH "EC_dev_branch" /* device tree branch event class */ +#define EC_DEV_STATUS "EC_dev_status" /* device status event class */ +#define EC_FM "EC_fm" /* FMA error report event */ +#define EC_ZFS "EC_zfs" /* ZFS event */ +#define EC_DATALINK "EC_datalink" /* datalink event */ +#define EC_VRRP "EC_vrrp" /* VRRP event */ + +/* + * EC_DEV_ADD and EC_DEV_REMOVE subclass definitions - supporting attributes + * (name/value pairs) are found in sys/sysevent/dev.h + */ +#define ESC_DISK "disk" /* disk device */ +#define ESC_NETWORK "network" /* network interface */ +#define ESC_PRINTER "printer" /* printer device */ +#define ESC_LOFI "lofi" /* lofi device */ + +/* + * EC_DEV_BRANCH subclass definitions:supporting attributes (name/value pairs) + * are found in sys/sysevent/dev.h + */ + +/* device tree branch added */ +#define ESC_DEV_BRANCH_ADD "dev_branch_add" + +/* device tree branch removed */ +#define ESC_DEV_BRANCH_REMOVE "dev_branch_remove" + +/* + * EC_DEV_STATUS subclass definitions + * + * device capacity dynamically changed + */ +#define ESC_DEV_DLE "dev_dle" + +/* LUN has received an eject request from the user */ +#define ESC_DEV_EJECT_REQUEST "dev_eject_request" + +/* FMA Fault and Error event protocol subclass */ +#define ESC_FM_ERROR "error" +#define ESC_FM_ERROR_REPLAY "error_replay" + +/* + * ZFS subclass definitions: supporting attributes (name/value paris) are found + * in sys/fs/zfs.h + */ +#define ESC_ZFS_RESILVER_START "resilver_start" +#define ESC_ZFS_RESILVER_FINISH "resilver_finish" +#define ESC_ZFS_VDEV_REMOVE "vdev_remove" +#define ESC_ZFS_VDEV_REMOVE_AUX "vdev_remove_aux" +#define ESC_ZFS_VDEV_REMOVE_DEV "vdev_remove_dev" +#define ESC_ZFS_POOL_CREATE "pool_create" +#define ESC_ZFS_POOL_DESTROY "pool_destroy" +#define ESC_ZFS_POOL_IMPORT "pool_import" +#define ESC_ZFS_POOL_EXPORT "pool_export" +#define ESC_ZFS_VDEV_ADD "vdev_add" +#define ESC_ZFS_VDEV_ATTACH "vdev_attach" +#define ESC_ZFS_VDEV_CLEAR "vdev_clear" +#define ESC_ZFS_VDEV_CHECK "vdev_check" +#define ESC_ZFS_VDEV_ONLINE "vdev_online" +#define ESC_ZFS_CONFIG_SYNC "config_sync" +#define ESC_ZFS_SCRUB_START "scrub_start" +#define ESC_ZFS_SCRUB_FINISH "scrub_finish" +#define ESC_ZFS_SCRUB_ABORT "scrub_abort" +#define ESC_ZFS_SCRUB_RESUME "scrub_resume" +#define ESC_ZFS_SCRUB_PAUSED "scrub_paused" +#define ESC_ZFS_VDEV_SPARE "vdev_spare" +#define ESC_ZFS_VDEV_AUTOEXPAND "vdev_autoexpand" +#define ESC_ZFS_BOOTFS_VDEV_ATTACH "bootfs_vdev_attach" +#define ESC_ZFS_POOL_REGUID "pool_reguid" +#define ESC_ZFS_HISTORY_EVENT "history_event" +#define ESC_ZFS_TRIM_START "trim_start" +#define ESC_ZFS_TRIM_FINISH "trim_finish" +#define ESC_ZFS_TRIM_CANCEL "trim_cancel" +#define ESC_ZFS_TRIM_RESUME "trim_resume" +#define ESC_ZFS_TRIM_SUSPEND "trim_suspend" +#define ESC_ZFS_ERRORSCRUB_START "errorscrub_start" +#define ESC_ZFS_ERRORSCRUB_FINISH "errorscrub_finish" +#define ESC_ZFS_ERRORSCRUB_ABORT "errorscrub_abort" +#define ESC_ZFS_ERRORSCRUB_RESUME "errorscrub_resume" +#define ESC_ZFS_ERRORSCRUB_PAUSED "errorscrub_paused" +/* + * datalink subclass definitions. + */ +#define ESC_DATALINK_PHYS_ADD "datalink_phys_add" /* new physical link */ + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_SYSEVENT_EVENTDEFS_H */ diff --git a/include/os/windows/spl/sys/sysmacros.h b/include/os/windows/spl/sys/sysmacros.h new file mode 100644 index 000000000000..1bedb83b59f5 --- /dev/null +++ b/include/os/windows/spl/sys/sysmacros.h @@ -0,0 +1,253 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#ifndef _SPL_SYSMACROS_H +#define _SPL_SYSMACROS_H + +#include +#include +#include +#include +#include + +#ifndef _KERNEL +#define _KERNEL __KERNEL__ +#endif + +#define FALSE 0 +#define TRUE 1 + +#define NBBY 8 + +#define MAXMSGLEN 256 +#define MAXNAMELEN 256 +#define MAXPATHLEN 1024 +#define MAXOFFSET_T LLONG_MAX +#define DEV_BSIZE 512 +#define DEV_BSHIFT 9 /* log2(DEV_BSIZE) */ + +#define proc_pageout NULL +// #define curproc (struct proc *)PsGetCurrentProcess() +#define curproc (proc_t *)PsGetCurrentProcess() + +extern uint32_t cpu_number(void); +#define CPU_SEQID (cpu_number()) +#define CPU_SEQID_UNSTABLE (cpu_number()) +#define _NOTE(x) +#define is_system_labeled() 0 + +extern unsigned int max_ncpus; + +#ifndef RLIM64_INFINITY +#define RLIM64_INFINITY (~0ULL) +#endif + +/* + * 0..MAX_PRIO-1: Process priority + * 0..MAX_RT_PRIO-1: RT priority tasks + * MAX_RT_PRIO..MAX_PRIO-1: SCHED_NORMAL tasks + * + * Treat shim tasks as SCHED_NORMAL tasks + */ + +/* + * In OSX, the kernel thread priorities start at 81 and goes to + * 95 MAXPRI_KERNEL. BASEPRI_REALTIME starts from 96. Since + * swap priority is at 92. Most ZFS priorities should probably + * stay below this, but kmem_reap needs to be higher. + */ +#define minclsyspri 81 /* BASEPRI_KERNEL */ +#define defclsyspri 81 /* BASEPRI_KERNEL */ +#define maxclsyspri 89 + +#define NICE_TO_PRIO(nice) (MAX_RT_PRIO + (nice) + 20) +#define PRIO_TO_NICE(prio) ((prio) - MAX_RT_PRIO - 20) + +/* + * Missing macros + */ +#define PAGESIZE PAGE_SIZE + +/* from Solaris sys/byteorder.h */ +#define BSWAP_8(x) ((x) & 0xff) +#define BSWAP_16(x) ((BSWAP_8(x) << 8) | BSWAP_8((x) >> 8)) +#define BSWAP_32(x) ((BSWAP_16(x) << 16) | BSWAP_16((x) >> 16)) +#define BSWAP_64(x) ((BSWAP_32(x) << 32) | BSWAP_32((x) >> 32)) + +/* Dtrace probes do not exist in the linux kernel */ +#ifdef DTRACE_PROBE +#undef DTRACE_PROBE +#endif /* DTRACE_PROBE */ +#define DTRACE_PROBE(a) ((void)0) + +#ifdef DTRACE_PROBE1 +#undef DTRACE_PROBE1 +#endif /* DTRACE_PROBE1 */ +#define DTRACE_PROBE1(a, b, c) ((void)0) + +#ifdef DTRACE_PROBE2 +#undef DTRACE_PROBE2 +#endif /* DTRACE_PROBE2 */ +#define DTRACE_PROBE2(a, b, c, d, e) ((void)0) + +#ifdef DTRACE_PROBE3 +#undef DTRACE_PROBE3 +#endif /* DTRACE_PROBE3 */ +#define DTRACE_PROBE3(a, b, c, d, e, f, g) ((void)0) + +#ifdef DTRACE_PROBE4 +#undef DTRACE_PROBE4 +#endif /* DTRACE_PROBE4 */ +#define DTRACE_PROBE4(a, b, c, d, e, f, g, h, i) ((void)0) + +/* Missing globals */ +extern char spl_version[32]; +extern char hw_serial[11]; + +/* Missing misc functions */ +extern uint32_t zone_get_hostid(void *zone); +extern void spl_setup(void); +extern void spl_cleanup(void); + +#define boot_ncpus max_ncpus +#define SET_ERROR(err) \ + (__set_error(__FILE__, __func__, __LINE__, err), err) + +#define NBITSMINOR 20 +#define MINORMASK ((1UL<> NBITSMINOR) +#define minor(x) ((x) & MINORMASK) + +#define makedev(x, y) (((x) << NBITSMINOR) | ((y) & MINORMASK)) +#define makedevice(maj, min) makedev(maj, min) + +/* common macros */ +#ifndef MIN +#define MIN(a, b) ((a) < (b) ? (a) : (b)) +#endif +#ifndef MAX +#define MAX(a, b) ((a) < (b) ? (b) : (a)) +#endif +#ifndef ABS +#define ABS(a) ((a) < 0 ? -(a) : (a)) +#endif +#ifndef DIV_ROUND_UP +#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d)) +#endif + +#ifndef roundup +#define roundup(x, y) ((((x) + ((y) - 1)) / (y)) * (y)) +#endif + +/* + * Compatibility macros/typedefs needed for Solaris -> Linux port + * For some reason Windows makes some of these signed, and everything + * goes to hell. + * But may have put in too many (uint64_t), check this + */ +#define P2ALIGN(x, align) (((uint64_t)x) & -((uint64_t)align)) +#define P2CROSS(x, y, align) \ + ((((uint64_t)x) ^ ((uint64_t)y)) > ((uint64_t)align) - 1) +#define P2ROUNDUP(x, align) (-(-((uint64_t)x) & -((uint64_t)align))) +#define P2PHASE(x, align) (((uint64_t)x) & (((uint64_t)align) - 1)) +#define P2NPHASE(x, align) (-((uint64_t)x) & (((uint64_t)align) - 1)) +#define ISP2(x) ((((uint64_t)x) & (((uint64_t)x) - 1)) == 0) +#define IS_P2ALIGNED(v, a) ((((uintptr_t)(v)) & ((uintptr_t)(a) - 1)) == 0) +#define P2BOUNDARY(off, len, align) \ + ((((uint64_t)off) ^ (((uint64_t)off) + \ + ((uint64_t)len) - 1)) > ((uint64_t)align) - 1) + +/* + * Typed version of the P2* macros. These macros should be used to ensure + * that the result is correctly calculated based on the data type of (x), + * which is passed in as the last argument, regardless of the data + * type of the alignment. For example, if (x) is of type uint64_t, + * and we want to round it up to a page boundary using "PAGESIZE" as + * the alignment, we can do either + * + * P2ROUNDUP(x, (uint64_t)PAGESIZE) + * or + * P2ROUNDUP_TYPED(x, PAGESIZE, uint64_t) + */ +#define P2ALIGN_TYPED(x, align, type) \ + ((type)((uint64_t)x) & -(type)((uint64_t)align)) +#define P2PHASE_TYPED(x, align, type) \ + ((type)((uint64_t)x) & ((type)((uint64_t)align) - 1)) +#define P2NPHASE_TYPED(x, align, type) \ + (-(type)((uint64_t)x) & ((type)((uint64_t)align) - 1)) +#define P2ROUNDUP_TYPED(x, align, type) \ + (-(-(type)((uint64_t)x) & -(type)((uint64_t)align))) +#define P2END_TYPED(x, align, type) \ + (-(~(type)((uint64_t)x) & -(type)((uint64_t)align))) +#define P2PHASEUP_TYPED(x, align, phase, type) \ + ((type)((uint64_t)phase) - (((type)((uint64_t)phase) - \ + (type)((uint64_t)x)) & -(type)((uint64_t)align))) +#define P2CROSS_TYPED(x, y, align, type) \ + (((type)((uint64_t)x) ^ (type)((uint64_t)y)) > \ + (type)((uint64_t)align) - 1) +#define P2SAMEHIGHBIT_TYPED(x, y, type) \ + (((type)((uint64_t)x) ^ (type)((uint64_t)y)) < \ + ((type)((uint64_t)x) & (type)((uint64_t)y))) + +/* + * P2* Macros from Illumos + */ + +/* + * return x rounded up to the next phase (offset) within align. + * phase should be < align. + * eg, P2PHASEUP(0x1234, 0x100, 0x10) == 0x1310 (0x13*align + phase) + * eg, P2PHASEUP(0x5600, 0x100, 0x10) == 0x5610 (0x56*align + phase) + */ +#define P2PHASEUP(x, align, phase) \ + (((uint64_t)phase) - ((((uint64_t)phase) - \ + ((uint64_t)x)) & -((uint64_t)align))) + +/* + * Return TRUE if they have the same highest bit set. + * eg, P2SAMEHIGHBIT(0x1234, 0x1001) == TRUE (the high bit is 0x1000) + * eg, P2SAMEHIGHBIT(0x1234, 0x3010) == FALSE (high bit of 0x3010 is 0x2000) + */ +#define P2SAMEHIGHBIT(x, y) \ + ((((uint64_t)x) ^ ((uint64_t)y)) < (((uint64_t)x) & ((uint64_t)y))) + +/* + * End Illumos copy-fest + */ + +/* avoid any possibility of clashing with version */ +#if defined(_KERNEL) && !defined(_KMEMUSER) && !defined(offsetof) +#include +#endif + +#define IS_INDEXABLE(arg) (sizeof (arg[0])) +#define IS_ARRAY(arg) \ + (IS_INDEXABLE(arg) && (((void *) &arg) == ((void *) arg))) + +#define ARRAY_SIZE(a) (sizeof (a) / sizeof (a[0])) + +#endif /* _SPL_SYSMACROS_H */ diff --git a/include/os/windows/spl/sys/systeminfo.h b/include/os/windows/spl/sys/systeminfo.h new file mode 100644 index 000000000000..95a7511a6897 --- /dev/null +++ b/include/os/windows/spl/sys/systeminfo.h @@ -0,0 +1,33 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_SYSTEMINFO_H +#define _SPL_SYSTEMINFO_H + +#define HW_INVALID_HOSTID 0xFFFFFFFF /* an invalid hostid */ + +/* minimum buffer size needed to hold a decimal or hex hostid string */ +#define HW_HOSTID_LEN 11 + +const char *spl_panicstr(void); +int spl_system_inshutdown(void); + +#endif /* SPL_SYSTEMINFO_H */ diff --git a/include/os/windows/spl/sys/systm.h b/include/os/windows/spl/sys/systm.h new file mode 100644 index 000000000000..fe6cabe80fd7 --- /dev/null +++ b/include/os/windows/spl/sys/systm.h @@ -0,0 +1,125 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright(c) 2019 Jorgen Lundman + */ + +#ifndef _SPL_SYSTM_H +#define _SPL_SYSTM_H + +#include + +typedef uintptr_t pc_t; + +// Find a header to place this? +struct bsd_timeout_wrapper { + uint32_t flag; // Must be first + uint32_t init; + void(*func)(void *); + void *arg; + KTIMER timer; +}; + +/* + * bsd_timeout will create a new thread, and the new thread will + * first sleep the desired duration, then call the wanted function + */ +#define BSD_TIMEOUT_MAGIC 0x42994299 +static inline void bsd_timeout_handler(void *arg) +{ + struct bsd_timeout_wrapper *btw = arg; + KeWaitForSingleObject(&btw->timer, Executive, KernelMode, TRUE, NULL); + if (btw->init == BSD_TIMEOUT_MAGIC) + btw->func(btw->arg); + thread_exit(); +} + +static inline void bsd_untimeout(void(*func)(void *), void *ID) +{ +/* + * Unfortunately, calling KeCancelTimer() does not Signal (or abort) any thread + * sitting in KeWaitForSingleObject() so they would wait forever. Instead we + * change the timeout to be now, so that the threads can exit. + */ + struct bsd_timeout_wrapper *btw = (struct bsd_timeout_wrapper *)ID; + LARGE_INTEGER p = { .QuadPart = -1 }; + VERIFY3P(btw, !=, NULL); + // If timer was armed, release it. + if (btw->init == BSD_TIMEOUT_MAGIC) { + btw->init = 0; // stop it from running func() + KeSetTimer(&btw->timer, p, NULL); + } +} + +static inline void bsd_timeout(void *FUNC, void *ID, struct timespec *TIM) +{ + LARGE_INTEGER duetime; + struct bsd_timeout_wrapper *btw = (struct bsd_timeout_wrapper *)ID; + void(*func)(void *) = FUNC; + if (btw == NULL) { + dprintf("%s NULL ID is not implemented\n", __func__); + return; + } + duetime.QuadPart = -((int64_t)(SEC2NSEC100(TIM->tv_sec) + + NSEC2NSEC100(TIM->tv_nsec))); + btw->func = func; + btw->arg = ID; + /* Global vars are guaranteed set to 0, still is this secure enough? */ + if (btw->init != BSD_TIMEOUT_MAGIC) { + btw->init = BSD_TIMEOUT_MAGIC; + KeInitializeTimer(&btw->timer); + } + if (!KeSetTimer(&btw->timer, duetime, NULL)) { + func((ID)); + } else { + /* Another option would have been to use taskq, it can cancel */ + thread_create(NULL, 0, bsd_timeout_handler, ID, 0, NULL, + TS_RUN, minclsyspri); + } +} + +/* + * Unfortunately, calling KeCancelTimer() does not Signal (or abort) any thread + * sitting in KeWaitForSingleObject() so they would wait forever. Call this + * function only when there are no threads waiting in bsd_timeout_handler(). + * Unloading the driver with loaded timer object can cause bugcheck when the + * timer fires. + */ +static inline void bsd_timeout_cancel(void *ID) +{ + struct bsd_timeout_wrapper *btw = (struct bsd_timeout_wrapper *)ID; + + if (btw == NULL) { + dprintf("%s NULL ID is not implemented\n", __func__); + return; + } + + if (btw->func != NULL) { + if (KeCancelTimer(&btw->timer)) { + dprintf("timer object was loaded.Cancelled it.\n"); + } else { + dprintf("timer object is not loaded.\n"); + } + } +} + +#endif /* SPL_SYSTM_H */ diff --git a/include/os/windows/spl/sys/taskq.h b/include/os/windows/spl/sys/taskq.h new file mode 100644 index 000000000000..6a700d877379 --- /dev/null +++ b/include/os/windows/spl/sys/taskq.h @@ -0,0 +1,121 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ +/* + * Copyright (C) 2015 Jorgen Lundman + */ + +#ifndef _SYS_TASKQ_H +#define _SYS_TASKQ_H + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define TASKQ_NAMELEN 31 + +typedef struct taskq taskq_t; +typedef uintptr_t taskqid_t; +typedef void (task_func_t)(void *); +struct taskq_ent; +struct proc; + +/* New ZFS expects to find taskq_ent_t as well */ +#include + +/* + * Public flags for taskq_create(): bit range 0-15 + */ +#define TASKQ_PREPOPULATE 0x0001 /* Prepopulate with threads and data */ +#define TASKQ_CPR_SAFE 0x0002 /* Use CPR safe protocol */ +#define TASKQ_DYNAMIC 0x0004 /* Use dynamic thread scheduling */ +#define TASKQ_THREADS_CPU_PCT 0x0008 /* number of threads as % of ncpu */ +#define TASKQ_DC_BATCH 0x0010 /* Taskq uses SDC in batch mode */ + +#ifdef _WIN32 +#define TASKQ_TIMESHARE 0x0020 /* macOS dynamic thread priority */ +#define TASKQ_REALLY_DYNAMIC 0x0040 /* don't filter out TASKQ_DYNAMIC */ +#endif +/* + * Flags for taskq_dispatch. TQ_SLEEP/TQ_NOSLEEP should be same as + * KM_SLEEP/KM_NOSLEEP. + */ +#define TQ_SLEEP 0x00 /* Can block for memory */ +#define TQ_NOSLEEP 0x01 /* cannot block for memory; may fail */ +#define TQ_NOQUEUE 0x02 /* Do not enqueue if can't dispatch */ +#define TQ_NOALLOC 0x04 /* cannot allocate memory; may fail */ +#define TQ_FRONT 0x08 /* Put task at the front of the queue */ +#define TQ_DELAYED 0x10 /* dispatch_delay, clean up after */ + +#define TASKQID_INVALID ((taskqid_t)0) + +#ifdef _KERNEL + +extern taskq_t *system_taskq; +/* Global dynamic task queue for long delay */ +extern taskq_t *system_delay_taskq; + +extern int spl_taskq_init(void); +extern void spl_taskq_fini(void); +extern void taskq_mp_init(void); + +extern taskq_t *taskq_create(const char *, int, pri_t, int, int, uint_t); +extern taskq_t *taskq_create_instance(const char *, int, int, pri_t, int, + int, uint_t); +extern taskq_t *taskq_create_proc(const char *, int, pri_t, int, int, + proc_t *, uint_t); +extern taskq_t *taskq_create_sysdc(const char *, int, int, int, + proc_t *, uint_t, uint_t); +extern taskqid_t taskq_dispatch(taskq_t *, task_func_t, void *, uint_t); +extern void nulltask(void *); +extern void taskq_destroy(taskq_t *); +extern void taskq_wait(taskq_t *); +#define HAVE_TASKQ_WAIT_ID +extern void taskq_wait_id(taskq_t *, taskqid_t); +extern void taskq_suspend(taskq_t *); +extern int taskq_suspended(taskq_t *); +extern void taskq_resume(taskq_t *); +extern int taskq_member(taskq_t *, struct kthread *); +extern int taskq_cancel_id(taskq_t *, taskqid_t); +extern taskq_t *taskq_of_curthread(void); +extern int taskq_empty_ent(struct taskq_ent *); + +extern void taskq_wait_outstanding(taskq_t *, taskqid_t); + +extern void system_taskq_init(void); +extern void system_taskq_fini(void); + +#endif /* _KERNEL */ + +extern int EMPTY_TASKQ(taskq_t *tq); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_TASKQ_H */ diff --git a/include/os/windows/spl/sys/taskq_impl.h b/include/os/windows/spl/sys/taskq_impl.h new file mode 100644 index 000000000000..ca8a49332b91 --- /dev/null +++ b/include/os/windows/spl/sys/taskq_impl.h @@ -0,0 +1,181 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ +/* + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. + */ +/* + * Copyright (C) 2015 Jorgen Lundman + */ + + +#ifndef _SYS_TASKQ_IMPL_H +#define _SYS_TASKQ_IMPL_H + +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct taskq_bucket taskq_bucket_t; + +typedef struct taskq_ent { + struct taskq_ent *tqent_next; + struct taskq_ent *tqent_prev; + task_func_t *tqent_func; + void *tqent_arg; + union { + taskq_bucket_t *tqent_bucket; + uintptr_t tqent_flags; + } tqent_un; + kthread_t *tqent_thread; + kcondvar_t tqent_cv; + kmutex_t tqent_delay_lock; + kcondvar_t tqent_delay_cv; + clock_t tqent_delay_time; + /* Used to simulate TS_STOPPED */ + kmutex_t tqent_thread_lock; + kcondvar_t tqent_thread_cv; +} taskq_ent_t; + +#define TQENT_FLAG_PREALLOC 0x1 +#define TQENT_FLAG_DELAYED 0x2 + +/* + * Taskq Statistics fields are not protected by any locks. + */ +typedef struct tqstat { + uint_t tqs_hits; + uint_t tqs_misses; + uint_t tqs_overflow; /* no threads to allocate */ + uint_t tqs_tcreates; /* threads created */ + uint_t tqs_tdeaths; /* threads died */ + uint_t tqs_maxthreads; /* max # of alive threads */ + uint_t tqs_nomem; /* # of times there were no memory */ + uint_t tqs_disptcreates; +} tqstat_t; + +/* + * Per-CPU hash bucket manages taskq_bent_t structures using freelist. + */ +struct taskq_bucket { + kmutex_t tqbucket_lock; + taskq_t *tqbucket_taskq; /* Enclosing taskq */ + taskq_ent_t tqbucket_freelist; + uint_t tqbucket_nalloc; /* # of allocated entries */ + uint_t tqbucket_nfree; /* # of free entries */ + kcondvar_t tqbucket_cv; + ushort_t tqbucket_flags; + hrtime_t tqbucket_totaltime; + tqstat_t tqbucket_stat; +}; + +/* + * Bucket flags. + */ +#define TQBUCKET_CLOSE 0x01 +#define TQBUCKET_SUSPEND 0x02 + +#define TASKQ_INTERFACE_FLAGS 0x0000ffff /* defined in */ + +/* + * taskq implementation flags: bit range 16-31 + */ +#define TASKQ_CHANGING 0x00010000 /* nthreads != target */ +#define TASKQ_SUSPENDED 0x00020000 /* taskq is suspended */ +#define TASKQ_NOINSTANCE 0x00040000 /* no instance number */ +#define TASKQ_THREAD_CREATED 0x00080000 /* a thread has been created */ +#define TASKQ_DUTY_CYCLE 0x00100000 /* using the SDC class */ + +struct taskq { + char tq_name[TASKQ_NAMELEN + 1]; + kmutex_t tq_lock; + krwlock_t tq_threadlock; + kcondvar_t tq_dispatch_cv; + kcondvar_t tq_wait_cv; + kcondvar_t tq_exit_cv; + pri_t tq_pri; /* Scheduling priority */ + uint_t tq_flags; + int tq_active; + int tq_nthreads; + int tq_nthreads_target; + int tq_nthreads_max; + int tq_threads_ncpus_pct; + int tq_nalloc; + int tq_minalloc; + int tq_maxalloc; + kcondvar_t tq_maxalloc_cv; + int tq_maxalloc_wait; + taskq_ent_t *tq_freelist; + taskq_ent_t tq_task; + int tq_maxsize; + taskq_bucket_t *tq_buckets; /* Per-cpu array of buckets */ + int tq_instance; + uint_t tq_nbuckets; /* # of buckets (2^n) */ + union { + kthread_t *_tq_thread; + kthread_t **_tq_threadlist; + } tq_thr; + + list_node_t tq_cpupct_link; /* linkage for taskq_cpupct_list */ + proc_t *tq_proc; /* process for taskq threads */ + int tq_cpupart; /* cpupart id bound to */ + uint_t tq_DC; /* duty cycle for SDC */ + + /* + * Statistics. + */ + kstat_t *tq_kstat; /* Exported statistics */ + hrtime_t tq_totaltime; /* Time spent processing tasks */ + uint64_t tq_tasks; /* Total # of tasks posted */ + uint64_t tq_executed; /* Total # of tasks executed */ + int tq_maxtasks; /* Max number of tasks in the queue */ + int tq_tcreates; + int tq_tdeaths; +}; + +/* Special form of taskq dispatch that uses preallocated entries. */ +void taskq_dispatch_ent(taskq_t *, task_func_t, void *, uint_t, taskq_ent_t *); + +#define tq_thread tq_thr._tq_thread +#define tq_threadlist tq_thr._tq_threadlist + +/* The MAX guarantees we have at least one thread */ +#define TASKQ_THREADS_PCT(ncpus, pct) MAX(((ncpus) * (pct)) / 100, 1) + +/* Extra ZOL / Apple */ +extern void taskq_init_ent(taskq_ent_t *t); +extern taskqid_t taskq_dispatch_delay(taskq_t *tq, task_func_t func, void *arg, + uint_t flags, clock_t expire_time); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_TASKQ_IMPL_H */ diff --git a/include/os/windows/spl/sys/thread.h b/include/os/windows/spl/sys/thread.h new file mode 100644 index 000000000000..d66082ec07f7 --- /dev/null +++ b/include/os/windows/spl/sys/thread.h @@ -0,0 +1,103 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2013 Jorgen Lundman + * + */ + +#ifndef _SPL_THREAD_H +#define _SPL_THREAD_H + +#include +#include +#include +#include + +typedef struct _KTHREAD kthread_t; +typedef struct _KTHREAD thread_t; + +/* + * Thread interfaces + */ +#define TP_MAGIC 0x53535353 + +#define TS_FREE 0x00 /* Thread at loose ends */ +#define TS_SLEEP 0x01 /* Awaiting an event */ +#define TS_RUN 0x02 /* Runnable, but not yet on a processor */ +#define TS_ONPROC 0x04 /* Thread is being run on a processor */ +#define TS_ZOMB 0x08 /* Thread has died but hasn't been reaped */ +#define TS_STOPPED 0x10 /* Stopped, initial state */ +#define TS_WAIT 0x20 /* Waiting to become runnable */ + + +typedef void (*thread_func_t)(void *); + +// This should be ThreadId, but that dies in taskq_member, +// for now, dsl_pool_sync_context calls it instead. +#define current_thread PsGetCurrentThread +#define curthread ((void *)current_thread()) /* current thread pointer */ +#define curproj (ttoproj(curthread)) /* current project pointer */ + +#define thread_join(t) VERIFY(0) + +// Drop the p0 argument, not used. + +#ifdef SPL_DEBUG_THREAD + +#define thread_create(A, B, C, D, E, F, G, H) \ + spl_thread_create(A, B, C, D, E, G, __FILE__, __LINE__, H) +extern kthread_t *spl_thread_create(caddr_t stk, size_t stksize, + void (*proc)(void *), void *arg, size_t len, /* proc_t *pp, */ + int state, char *, int, pri_t pri); +#define thread_create_named(name, A, B, C, D, E, F, G, H) \ + spl_thread_create(A, B, C, D, E, G, __FILE__, __LINE__, H) +#else + +#define thread_create(A, B, C, D, E, F, G, H) \ + spl_thread_create(A, B, C, D, E, G, H) +extern kthread_t *spl_thread_create(caddr_t stk, size_t stksize, + void (*proc)(void *), void *arg, size_t len, /* proc_t *pp, */ + int state, pri_t pri); +#define thread_create_named(name, A, B, C, D, E, F, G, H) \ + spl_thread_create(A, B, C, D, E, G, H) +#endif + +#define thread_exit spl_thread_exit +extern void __declspec(noreturn) spl_thread_exit(void); + +extern kthread_t *spl_current_thread(void); + +#define delay windows_delay +#define IOSleep windows_delay +extern void windows_delay(int); + +#define KPREEMPT_SYNC 0 +static inline void kpreempt(int flags) +{ + (void) flags; + LARGE_INTEGER interval; + interval.QuadPart = 0; + KeDelayExecutionThread(KernelMode, FALSE, &interval); +} + +#endif /* _SPL_THREAD_H */ diff --git a/include/os/windows/spl/sys/time.h b/include/os/windows/spl/sys/time.h new file mode 100644 index 000000000000..336f16e845c1 --- /dev/null +++ b/include/os/windows/spl/sys/time.h @@ -0,0 +1,116 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2008 MacZFS + * Copyright (C) 2013 Jorgen Lundman + * + */ + +#ifndef _SPL_TIME_H +#define _SPL_TIME_H + +typedef long long hrtime_t; + +#include +#include +#include +struct timespec; + +#if defined(CONFIG_64BIT) +#define TIME_MAX INT64_MAX +#define TIME_MIN INT64_MIN +#else +#define TIME_MAX INT32_MAX +#define TIME_MIN INT32_MIN +#endif + +#define SEC 1 +#define MILLISEC 1000 +#define MICROSEC 1000000 +#define NANOSEC 1000000000 + +#define NSEC2SEC(n) ((n) / (NANOSEC / SEC)) +#define SEC2NSEC(m) ((hrtime_t)(m) * (NANOSEC / SEC)) + +/* Already defined in include/linux/time.h */ +#undef CLOCK_THREAD_CPUTIME_ID +#undef CLOCK_REALTIME +#undef CLOCK_MONOTONIC +#undef CLOCK_PROCESS_CPUTIME_ID + +typedef enum clock_type { + __CLOCK_REALTIME0 = 0, /* obsolete; same as CLOCK_REALTIME */ + CLOCK_VIRTUAL = 1, /* thread's user-level CPU clock */ + CLOCK_THREAD_CPUTIME_ID = 2, /* thread's user+system CPU clock */ + CLOCK_REALTIME = 3, /* wall clock */ + CLOCK_MONOTONIC = 4, /* high resolution monotonic clock */ + CLOCK_PROCESS_CPUTIME_ID = 5, /* process's user+system CPU clock */ + CLOCK_HIGHRES = CLOCK_MONOTONIC, /* alternate name */ + CLOCK_PROF = CLOCK_THREAD_CPUTIME_ID, /* alternate name */ +} clock_type_t; + +#define TIMESPEC_OVERFLOW(ts) \ + ((ts)->tv_sec < TIME_MIN || (ts)->tv_sec > TIME_MAX) + +extern hrtime_t gethrtime(void); +extern void gethrestime(struct timespec *); +extern time_t gethrestime_sec(void); +extern void hrt2ts(hrtime_t hrt, struct timespec *tsp); + +#define MSEC2NSEC(m) ((hrtime_t)(m) * (NANOSEC / MILLISEC)) +#define USEC2NSEC(u) ((hrtime_t)(u) * (NANOSEC / MICROSEC)) +#define NSEC2MSEC(n) ((n) / (NANOSEC / MILLISEC)) +#define NSEC2USEC(n) ((n) / (NANOSEC / MICROSEC)) + +// Windows 100NS +#define SEC2NSEC100(n) ((n) * 10000000ULL) +#define NSEC2NSEC100(n) ((n) / 100ULL) + +#define SEC_TO_TICK(sec) ((sec) * hz) +#define NSEC_TO_TICK(nsec) ((nsec) / (NANOSEC / hz)) + +#define NSEC2USEC(n) ((n) / (NANOSEC / MICROSEC)) + +/* + * ZFS time is 2* 64bit values, which are seconds, and nanoseconds since 1970 + * Windows time is 1 64bit value; representing the number of + * 100-nanosecond intervals since January 1, 1601 (UTC). + * There's 116444736000000000 100ns between 1601 and 1970 + */ + +// I think these functions handle sec correctly, but nsec should be */100 +#define TIME_WINDOWS_TO_UNIX(WT, UT) do { \ + uint64_t unixepoch = (WT) - 116444736000000000ULL; \ + (UT)[0] = /* seconds */ unixepoch / 10000000ULL; \ + (UT)[1] = /* remainding nsec */ unixepoch - ((UT)[0] * 10000000ULL); \ + } while (0) + +#define TIME_UNIX_TO_WINDOWS(UT, WT) do { \ + (WT) = ((UT)[1]) + ((UT)[0] * 10000000ULL) + 116444736000000000ULL; \ + } while (0) + +#define TIME_UNIX_TO_WINDOWS_EX(SEC, USEC, WT) do { \ + (WT) = (USEC) + ((SEC) * 10000000ULL) + 116444736000000000ULL; \ + } while (0) + +#endif /* _SPL_TIME_H */ diff --git a/include/os/windows/spl/sys/timer.h b/include/os/windows/spl/sys/timer.h new file mode 100644 index 000000000000..27ea8e1a2bfa --- /dev/null +++ b/include/os/windows/spl/sys/timer.h @@ -0,0 +1,77 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2008 MacZFS + * Copyright (C) 2013 Jorgen Lundman + * + */ + +#ifndef _SPL_TIMER_H +#define _SPL_TIMER_H + +#include + +// Typical timespec is smaller, but we need to retain the precision +// to copy time between Unix and Windows. +struct timespec { + uint64_t tv_sec; + uint64_t tv_nsec; +}; + +#define lbolt zfs_lbolt() +#define lbolt64 zfs_lbolt() + +#define ddi_get_lbolt() (zfs_lbolt()) +#define ddi_get_lbolt64() (zfs_lbolt()) + +#define typecheck(type, x) \ + ( \ + { type __dummy; \ + typeof(x) __dummy2; \ + (void) (&__dummy == &__dummy2); \ + 1; \ + }) + +#pragma error(disable: 4296) +#define ddi_time_before(a, b) ((a) - (b) < 0) +#define ddi_time_after(a, b) ddi_time_before(b, a) + +#define ddi_time_before64(a, b) ((a) - (b) < 0) +#define ddi_time_after64(a, b) ddi_time_before64(b, a) +#pragma error(default: 4296) + +extern uint64_t zfs_lbolt(void); + +#define usleep_range(wakeup, whocares) \ + do { \ + hrtime_t delta = wakeup - gethrtime(); \ + if (delta > 0) { \ + LARGE_INTEGER ft; \ + ft.QuadPart = -((__int64)NSEC2NSEC100(delta)); \ + (void) KeWaitForMultipleObjects(0, NULL, WaitAny, \ + Executive, KernelMode, FALSE, &ft, NULL); \ + } \ + } while (0) + + +#endif /* _SPL_TIMER_H */ diff --git a/include/os/windows/spl/sys/trace.h b/include/os/windows/spl/sys/trace.h new file mode 100644 index 000000000000..5e33920c8195 --- /dev/null +++ b/include/os/windows/spl/sys/trace.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#ifndef _SPL_TRACE_H +#define _SPL_TRACE_H + +#endif diff --git a/include/os/windows/spl/sys/trace_zfs.h b/include/os/windows/spl/sys/trace_zfs.h new file mode 100644 index 000000000000..d9639d27b60e --- /dev/null +++ b/include/os/windows/spl/sys/trace_zfs.h @@ -0,0 +1 @@ +/* keep me */ diff --git a/include/os/windows/spl/sys/tsd.h b/include/os/windows/spl/sys/tsd.h new file mode 100644 index 000000000000..d54c2a30e244 --- /dev/null +++ b/include/os/windows/spl/sys/tsd.h @@ -0,0 +1,57 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2008 MacZFS + * Copyright (C) 2013 Jorgen Lundman + * + */ + + +#ifndef _SPL_TSD_H +#define _SPL_TSD_H + +#include +#include + +#define TSD_HASH_TABLE_BITS_DEFAULT 9 +#define TSD_KEYS_MAX 32768 +#define DTOR_PID (PID_MAX_LIMIT+1) +#define PID_KEY (TSD_KEYS_MAX+1) + +typedef void (*dtor_func_t)(void *); + +extern int tsd_set(uint_t, void *); +extern void *tsd_get(uint_t); +typedef struct _KTHREAD thread_t; + +extern void *tsd_get_by_thread(uint_t, thread_t *); +extern void tsd_create(uint_t *, dtor_func_t); +extern void tsd_destroy(uint_t *); +extern void tsd_exit(void); + +uint64_t spl_tsd_size(void); +void tsd_thread_exit(void); +int spl_tsd_init(void); +void spl_tsd_fini(void); + +#endif /* _SPL_TSD_H */ diff --git a/include/os/windows/spl/sys/types.h b/include/os/windows/spl/sys/types.h new file mode 100644 index 000000000000..253a54297dab --- /dev/null +++ b/include/os/windows/spl/sys/types.h @@ -0,0 +1,188 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ +#ifndef _SPL_TYPES_H +#define _SPL_TYPES_H + +// use ntintsafe.h ? +typedef enum { B_FALSE = 0, B_TRUE = 1 } boolean_t; +typedef short pri_t; +typedef int int32_t; +typedef unsigned long ulong_t; +typedef unsigned long long uint64_t; +typedef unsigned long long u_longlong_t; +typedef unsigned long long rlim64_t; +typedef unsigned long long loff_t; +#define _CLOCK_T_DEFINED +typedef unsigned long long clock_t; +typedef long long int64_t; +typedef long long longlong_t; +typedef unsigned char uchar_t; +typedef unsigned int uint_t; +typedef unsigned int uint32_t; +typedef unsigned short ushort_t; +typedef void *spinlock_t; +typedef long long offset_t; +typedef long long off_t; +typedef struct timespec timestruc_t; /* definition per SVr4 */ +typedef struct timespec timespec_t; +typedef ulong_t pgcnt_t; +typedef unsigned int mode_t; +#define NODEV32 (dev32_t)(-1) +typedef uint_t minor_t; +typedef char *caddr_t; +typedef unsigned char uint8_t; +typedef char int8_t; +typedef short int int16_t; +typedef unsigned short int uint16_t; +typedef unsigned long long uid_t; +typedef unsigned long long gid_t; +typedef unsigned int pid_t; + +typedef int64_t ssize_t; +typedef uint64_t vm_offset_t; +typedef uint64_t dev_t; +#define NGROUPS 16 + +typedef unsigned short umode_t; +typedef uint64_t user_addr_t; +typedef uint64_t user_size_t; +typedef uint64_t ino64_t; + +typedef unsigned char uuid_t[16]; + +typedef void zidmap_t; + +// clang spits out "atomics are disabled" - change code to use atomic() calls. +// #define _Atomic + +#define PATH_MAX 1024 +#define Z_OK 0 + +struct buf; +typedef struct buf buf_t; +typedef unsigned int uInt; +#include +#include +typedef uintptr_t pc_t; + +#include +#include +#include +#include +#include +#include + + +#define snprintf _snprintf +#define vprintf(...) vKdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_INFO_LEVEL, \ + __VA_ARGS__)) +#define vsnprintf _vsnprintf + +#ifndef ULLONG_MAX +#define ULLONG_MAX (~0ULL) +#endif + +#ifndef LLONG_MAX +#define LLONG_MAX ((long long)(~0ULL>>1)) +#endif + +#include +#define FREAD 0x0001 +#define FWRITE 0x0002 + +#define FCREAT O_CREAT +#define FTRUNC O_TRUNC +#define FEXCL O_EXCL +#define FNOCTTY O_NOCTTY +#define FNOFOLLOW O_NOFOLLOW +#define FAPPEND O_APPEND + + +#define FSYNC 0x10 /* file (data+inode) integrity while writing */ +#define FDSYNC 0x40 /* file data only integrity while writing */ +#define FRSYNC 0x8000 /* sync read operations at same level of */ +/* integrity as specified for writes by */ +/* FSYNC and FDSYNC flags */ +#define FOFFMAX 0x2000 /* large file */ + +#define EXPORT_SYMBOL(X) +#define module_param(X, Y, Z) +#define MODULE_PARM_DESC(X, Y) + +#define B_WRITE 0x00000000 /* Write buffer (pseudo flag). */ +#define B_READ 0x00000001 /* Read buffer. */ +#define B_ASYNC 0x00000002 /* Start I/O, do not wait. */ +#define B_NOCACHE 0x00000004 /* Do not cache block after use. */ +#define B_PHYS 0x00000020 /* I/O to user memory. */ +#define B_PASSIVE 0x00000800 /* PASSIVE I/Os are ignored by THROTTLE */ +#define B_BUSY B_PHYS + +#ifdef __GNUC__ +#define member_type(type, member) __typeof__(((type *)0)->member) +#else +#define member_type(type, member) void +#endif + +#define container_of(ptr, type, member) ((type *)\ + ((char *)(member_type(type, member)*)\ + { ptr } - offsetof(type, member))) + +#define strtok_r strtok_s +#define strcasecmp _stricmp + +struct mount; +typedef struct mount mount_t; + +struct kauth_cred; +typedef struct kauth_cred kauth_cred_t; +struct kauth_acl; +typedef struct kauth_acl kauth_acl_t; +#define KAUTH_FILESEC_NONE ((kauth_filesec_t)0) + +struct kauth_ace_rights; +typedef struct kauth_ace_rights kauth_ace_rights_t; + +extern int groupmember(gid_t gid, kauth_cred_t *cred); + +typedef struct { +#define KAUTH_GUID_SIZE 16 /* 128-bit identifier */ + unsigned char g_guid[KAUTH_GUID_SIZE]; +} guid_t; + +#pragma warning(disable: 4296) // expression is always true +#pragma error(disable: 4296) // expression is always true +#pragma warning(disable: 4703) // potentially uninit local pointer variable + +#define LINK_MAX 32767 /* max file link count */ + +#define FNV1_32A_INIT ((uint32_t)0x811c9dc5) +uint32_t fnv_32a_str(const char *str, uint32_t hval); +uint32_t fnv_32a_buf(void *buf, size_t len, uint32_t hval); + +typedef struct timespec inode_timespec_t; + +#endif /* _SPL_TYPES_H */ diff --git a/include/os/windows/spl/sys/types32.h b/include/os/windows/spl/sys/types32.h new file mode 100644 index 000000000000..44634fad098f --- /dev/null +++ b/include/os/windows/spl/sys/types32.h @@ -0,0 +1,13 @@ + +#ifndef _SPL_TYPES32_H +#define _SPL_TYPES32_H + +#include +#include + +typedef uint32_t caddr32_t; +typedef int32_t daddr32_t; +typedef int32_t time32_t; +typedef uint32_t size32_t; + +#endif /* _SPL_TYPES32_H */ diff --git a/include/os/windows/spl/sys/uio.h b/include/os/windows/spl/sys/uio.h new file mode 100644 index 000000000000..26fad35f19df --- /dev/null +++ b/include/os/windows/spl/sys/uio.h @@ -0,0 +1,194 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2014 Garrett D'Amore + * + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + * + * Copyright 2013 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2015, Joyent, Inc. All rights reserved. + */ + +/* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ + +/* + * University Copyright- Copyright (c) 1982, 1986, 1988 + * The Regents of the University of California + * All Rights Reserved + * + * University Acknowledgment- Portions of this document are derived from + * software developed by the University of California, Berkeley, and its + * contributors. + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#ifndef _SYS_UIO_H +#define _SYS_UIO_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +/* + * I/O parameter information. A uio structure describes the I/O which + * is to be performed by an operation. Typically the data movement will + * be performed by a routine such as uiomove(), which updates the uio + * structure to reflect what was done. + */ + +typedef struct iovec { + void *iov_base; + size_t iov_len; +} iovec_t; + + +/* + * I/O direction. + */ +typedef enum zfs_uio_rw { UIO_READ, UIO_WRITE } zfs_uio_rw_t; + +/* + * Segment flag values. + */ +typedef enum zfs_uio_seg { UIO_USERSPACE, UIO_SYSSPACE, UIO_USERISPACE } + zfs_uio_seg_t; + + +typedef struct zfs_uio { + const struct iovec *uio_iov; + int uio_iovcnt; + int uio_index; + off_t uio_loffset; + zfs_uio_seg_t uio_segflg; + boolean_t uio_fault_disable; + uint16_t uio_fmode; + uint16_t uio_extflg; + ssize_t uio_resid; + size_t uio_skip; +} zfs_uio_t; + +static inline zfs_uio_seg_t +zfs_uio_segflg(zfs_uio_t *uio) +{ + return (uio->uio_segflg); +} + +static inline int +zfs_uio_iovcnt(zfs_uio_t *uio) +{ + return (uio->uio_iovcnt); +} + +static inline off_t +zfs_uio_offset(zfs_uio_t *uio) +{ + return (uio->uio_loffset); +} + +static inline size_t +zfs_uio_resid(zfs_uio_t *uio) +{ + return (uio->uio_resid); +} + +static inline int +zfs_uio_skip(zfs_uio_t *uio) +{ + return (uio->uio_skip); +} + +static inline int /* lundman extension */ +zfs_uio_index(zfs_uio_t *uio) +{ + return (uio->uio_index); +} + +static inline void +zfs_uio_setoffset(zfs_uio_t *uio, off_t off) +{ + uio->uio_loffset = off; +} + +static inline void +zfs_uio_setskip(zfs_uio_t *uio, int skip) +{ + uio->uio_skip = skip; +} + +static inline void /* lundman extension */ +zfs_uio_setindex(zfs_uio_t *uio, int index) +{ + uio->uio_index = index; +} + +static inline void +zfs_uio_advance(zfs_uio_t *uio, size_t size) +{ + uio->uio_resid -= size; + uio->uio_loffset += size; +} + +/* zfs_uio_iovlen(uio, 0) = uio_curriovlen() */ +static inline uint64_t +zfs_uio_iovlen(zfs_uio_t *uio, unsigned int idx) +{ + return (uio->uio_iov[idx].iov_len); +} + +static inline void * +zfs_uio_iovbase(zfs_uio_t *uio, unsigned int idx) +{ + return (uio->uio_iov[(idx)].iov_base); +} + +static inline void +zfs_uio_iovec_init(zfs_uio_t *uio, const struct iovec *iov, + unsigned long nr_segs, off_t offset, zfs_uio_seg_t seg, ssize_t resid, + size_t skip) +{ + uio->uio_iov = iov; + uio->uio_iovcnt = nr_segs; + uio->uio_loffset = offset; + uio->uio_segflg = seg; + uio->uio_fmode = 0; + uio->uio_extflg = 0; + uio->uio_resid = resid; + uio->uio_skip = skip; +} + +extern int zfs_uio_prefaultpages(ssize_t, zfs_uio_t *); +#define zfs_uio_fault_disable(uio, set) +#define zfs_uio_fault_move(p, n, rw, u) zfs_uiomove((p), (n), (rw), (u)) + + + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_UIO_H */ diff --git a/include/os/windows/spl/sys/unistd.h b/include/os/windows/spl/sys/unistd.h new file mode 100644 index 000000000000..7f53079756e9 --- /dev/null +++ b/include/os/windows/spl/sys/unistd.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_UNISTD_H +#define _SPL_UNISTD_H + +#endif /* SPL_UNISTD_H */ diff --git a/include/os/windows/spl/sys/utfconv.h b/include/os/windows/spl/sys/utfconv.h new file mode 100644 index 000000000000..33d23f1f19df --- /dev/null +++ b/include/os/windows/spl/sys/utfconv.h @@ -0,0 +1,26 @@ + +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#ifndef _SPL_SYS_UTFCONV_H +#define _SPL_SYS_UTFCONV_H + +#endif diff --git a/include/os/windows/spl/sys/utsname.h b/include/os/windows/spl/sys/utsname.h new file mode 100644 index 000000000000..a500251b79f7 --- /dev/null +++ b/include/os/windows/spl/sys/utsname.h @@ -0,0 +1,47 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ + + +/* + * Copyright 2004 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + + +#ifndef _SPL_UTSNAME_H +#define _SPL_UTSNAME_H + +#define _SYS_NMLN 257 +struct opensolaris_utsname { + char sysname[_SYS_NMLN]; + char nodename[_SYS_NMLN]; + char release[_SYS_NMLN]; + char version[_SYS_NMLN]; + char machine[_SYS_NMLN]; +}; + +typedef struct opensolaris_utsname utsname_t; +extern utsname_t *utsname(void); + +#endif /* SPL_UTSNAME_H */ diff --git a/include/os/windows/spl/sys/varargs.h b/include/os/windows/spl/sys/varargs.h new file mode 100644 index 000000000000..1b8b1fc16d55 --- /dev/null +++ b/include/os/windows/spl/sys/varargs.h @@ -0,0 +1,27 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_VARARGS_H +#define _SPL_VARARGS_H + +#define __va_list va_list + +#endif /* SPL_VARARGS_H */ diff --git a/include/os/windows/spl/sys/vfs.h b/include/os/windows/spl/sys/vfs.h new file mode 100644 index 000000000000..4df14d04f055 --- /dev/null +++ b/include/os/windows/spl/sys/vfs.h @@ -0,0 +1,82 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 1988, 2010, Oracle and/or its affiliates. All rights reserved. + */ + +/* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ + +/* + * Portions of this source code were derived from Berkeley 4.3 BSD + * under license from the Regents of the University of California. + */ + +#ifndef _SPL_ZFS_H +#define _SPL_ZFS_H + +#include +#include + +#define MAXFIDSZ 64 + +typedef struct mount vfs_t; + +#define vn_vfswlock(vp) (0) +#define vn_vfsunlock(vp) +#define VFS_HOLD(vfsp) +#define VFS_RELE(vfsp) + +/* + * File identifier. Should be unique per filesystem on a single + * machine. This is typically called by a stateless file server + * in order to generate "file handles". + * + * Do not change the definition of struct fid ... fid_t without + * letting the CacheFS group know about it! They will have to do at + * least two things, in the same change that changes this structure: + * 1. change CFSVERSION in usr/src/uts/common/sys/fs/cachefs_fs.h + * 2. put the old version # in the canupgrade array + * in cachfs_upgrade() in usr/src/cmd/fs.d/cachefs/fsck/fsck.c + * This is necessary because CacheFS stores FIDs on disk. + * + * Many underlying file systems cast a struct fid into other + * file system dependent structures which may require 4 byte alignment. + * Because a fid starts with a short it may not be 4 byte aligned, the + * fid_pad will force the alignment. + */ + +#define MAXFIDSZ 64 +#define OLD_MAXFIDSZ 16 + +typedef struct fid { + union { + long fid_pad; + struct { + ushort_t len; /* length of data in bytes */ + char data[MAXFIDSZ]; /* data (variable len) */ + } _fid; + } un; +} fid_t; + +extern void (*mountroot_post_hook)(void); + +#endif /* SPL_ZFS_H */ diff --git a/include/os/windows/spl/sys/vfs_opreg.h b/include/os/windows/spl/sys/vfs_opreg.h new file mode 100644 index 000000000000..ee3489048681 --- /dev/null +++ b/include/os/windows/spl/sys/vfs_opreg.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_OPREG_H +#define _SPL_OPREG_H + +#endif /* SPL_OPREG_H */ diff --git a/include/os/windows/spl/sys/vmem.h b/include/os/windows/spl/sys/vmem.h new file mode 100644 index 000000000000..d342dc8346ec --- /dev/null +++ b/include/os/windows/spl/sys/vmem.h @@ -0,0 +1,178 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012 by Delphix. All rights reserved. + */ + +#ifndef _SYS_VMEM_H +#define _SYS_VMEM_H + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define KMEM_QUANTUM (PAGESIZE) + +/* + * Per-allocation flags + */ +#define VM_SLEEP 0x00000000 /* same as KM_SLEEP */ +#define VM_NOSLEEP 0x00000001 /* same as KM_NOSLEEP */ +#define VM_PANIC 0x00000002 /* same as KM_PANIC */ +#define VM_PUSHPAGE 0x00000004 /* same as KM_PUSHPAGE */ +#define VM_NORMALPRI 0x00000008 /* same as KM_NORMALPRI */ +#define VM_NODEBUG 0x00000010 /* matches KM_NODEBUG */ +#define VM_NO_VBA 0x00000020 /* do not descend to bucket */ +#define VM_KMFLAGS 0x000000ff /* flags to match KM_* flags */ +#define VM_BESTFIT 0x00000100 +#define VM_FIRSTFIT 0x00000200 +#define VM_NEXTFIT 0x00000400 + +/* + * The following flags are restricted for use only within the kernel. + * VM_MEMLOAD is for use by the HAT to avoid infinite recursion. + * VM_NORELOC is used by the kernel when static VA->PA mappings are required. + */ +#define VM_MEMLOAD 0x00000800 +#define VM_NORELOC 0x00001000 + +/* + * VM_ABORT requests that vmem_alloc() *ignore* the VM_SLEEP/VM_NOSLEEP flags + * and forgo reaping if the allocation or attempted import, fails. This + * flag is a segkmem-specific flag, and should not be used by anyone else. + */ +#define VM_ABORT 0x00002000 + +/* + * VM_ENDALLOC requests that large addresses be preferred in allocations. + * Has no effect if VM_NEXTFIT is active. + */ +#define VM_ENDALLOC 0x00004000 + +#define VM_FLAGS 0x0000FFFF + +/* + * Arena creation flags + */ +#define VMC_POPULATOR 0x00010000 +#define VMC_NO_QCACHE 0x00020000 /* cannot use quantum caches */ +#define VMC_IDENTIFIER 0x00040000 /* not backed by memory */ + +#define VMC_DUMPSAFE 0x00200000 /* can use alternate dump memory */ + +/* keep span creation time, newest spans to front */ +#define VMC_TIMEFREE 0x01000000 + +/* must accompany VMC_TIMEFREE, oldest spans to front */ +#define VMC_OLDFIRST 0x02000000 +/* + * internal use only; the import function uses the vmem_ximport_t interface + * and may increase the request size if it so desires. + * VMC_XALIGN, for use with vmem_xcreate, specifies that + * the address returned by the import function will be + * aligned according to the alignment argument. + */ +#define VMC_XALLOC 0x00080000 +#define VMC_XALIGN 0x00100000 +#define VMC_FLAGS 0xFFFF0000 + +/* + * Public segment types + */ +#define VMEM_ALLOC 0x01 +#define VMEM_FREE 0x02 + +/* + * Implementation-private segment types + */ +#define VMEM_SPAN 0x10 +#define VMEM_ROTOR 0x20 +#define VMEM_WALKER 0x40 + +/* + * VMEM_REENTRANT indicates to vmem_walk() that the callback routine may + * call back into the arena being walked, so vmem_walk() must drop the + * arena lock before each callback. The caveat is that since the arena + * isn't locked, its state can change. Therefore it is up to the callback + * routine to handle cases where the segment isn't of the expected type. + * For example, we use this to walk heap_arena when generating a crash dump; + * see segkmem_dump() for sample usage. + */ +#define VMEM_REENTRANT 0x80000000 + +struct vmem; + +typedef struct vmem vmem_t; +typedef void *(vmem_alloc_t)(vmem_t *, size_t, int); +typedef void (vmem_free_t)(vmem_t *, void *, size_t); + +/* + * Alternate import style; the requested size is passed in a pointer, + * which can be increased by the import function if desired. + */ +typedef void *(vmem_ximport_t)(vmem_t *, size_t *, size_t, int); + +#ifdef _KERNEL +extern vmem_t *vmem_init(const char *, void *, size_t, size_t, + vmem_alloc_t *, vmem_free_t *); +extern void vmem_fini(vmem_t *); +extern void vmem_timer_fini(); +extern void vmem_update(void *); +extern int vmem_is_populator(); +extern size_t vmem_seg_size; +#endif + +extern vmem_t *vmem_create(const char *, void *, size_t, size_t, + vmem_alloc_t *, vmem_free_t *, vmem_t *, size_t, int); +extern vmem_t *vmem_xcreate(const char *, void *, size_t, size_t, + vmem_ximport_t *, vmem_free_t *, vmem_t *, size_t, int); +extern void vmem_destroy(vmem_t *); +extern void *vmem_alloc_impl(vmem_t *, size_t, int); +extern void *vmem_xalloc(vmem_t *, size_t, size_t, size_t, size_t, + void *, void *, int); +extern void vmem_free_impl(vmem_t *, void *, size_t); +extern void vmem_xfree(vmem_t *, void *, size_t); +extern void *vmem_add(vmem_t *, void *, size_t, int); +extern int vmem_contains(vmem_t *, void *, size_t); +extern void vmem_walk(vmem_t *, int, void (*)(void *, void *, size_t), + void *); +extern size_t vmem_size(vmem_t *, int); +extern size_t vmem_size_locked(vmem_t *, int); +extern size_t vmem_size_semi_atomic(vmem_t *, int); +extern void vmem_qcache_reap(vmem_t *vmp); +extern int64_t vmem_buckets_size(int); + +/* + * Since Linux code uses vmem_alloc()/vmem_free and it is not + * the illumos one, wrap kmem_alloc()/kmem_free. + */ +#define vmem_free(A, B) zfs_kmem_free((A), (B)) +#define vmem_alloc(A, B) zfs_kmem_alloc((A), (B)) +#define vmem_zalloc(A, B) zfs_kmem_zalloc((A), (B)) + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_VMEM_H */ diff --git a/include/os/windows/spl/sys/vmem_impl.h b/include/os/windows/spl/sys/vmem_impl.h new file mode 100644 index 000000000000..83be8db402a1 --- /dev/null +++ b/include/os/windows/spl/sys/vmem_impl.h @@ -0,0 +1,155 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 1999-2001, 2003 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SYS_VMEM_IMPL_H +#define _SYS_VMEM_IMPL_H + +#include +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + + typedef struct vmem_seg vmem_seg_t; + +#define VMEM_STACK_DEPTH 20 + +struct vmem_seg { + /* + * The first four fields must match vmem_freelist_t exactly. + */ + uintptr_t vs_start; /* start of segment (inclusive) */ + uintptr_t vs_end; /* end of segment (exclusive) */ + vmem_seg_t *vs_knext; /* next of kin (alloc, free, span) */ + vmem_seg_t *vs_kprev; /* prev of kin */ + + vmem_seg_t *vs_anext; /* next in arena */ + vmem_seg_t *vs_aprev; /* prev in arena */ + uint8_t vs_type; /* alloc, free, span */ + uint8_t vs_import; /* non-zero if segment was imported */ + uint8_t vs_depth; /* stack depth if KMF_AUDIT active */ + /* + * if VM_FREESORT is set on the arena, then + * this field is set at span creation time. + */ + hrtime_t vs_span_createtime; + /* + * The following fields are present only when KMF_AUDIT is set. + */ + kthread_t *vs_thread; + hrtime_t vs_timestamp; + pc_t vs_stack[VMEM_STACK_DEPTH]; +}; + +typedef struct vmem_freelist { + uintptr_t vs_start; /* always zero */ + uintptr_t vs_end; /* segment size */ + vmem_seg_t *vs_knext; /* next of kin */ + vmem_seg_t *vs_kprev; /* prev of kin */ +} vmem_freelist_t; + +#define VS_SIZE(vsp) ((vsp)->vs_end - (vsp)->vs_start) + + /* + * Segment hashing + */ +#define VMEM_HASH_INDEX(a, s, q, m) \ +((((a) + ((a) >> (s)) + ((a) >> ((s) << 1))) >> (q)) & (m)) + +#define VMEM_HASH(vmp, addr) \ +(&(vmp)->vm_hash_table[VMEM_HASH_INDEX(addr, \ +(vmp)->vm_hash_shift, (vmp)->vm_qshift, (vmp)->vm_hash_mask)]) + +#define VMEM_QCACHE_SLABSIZE(max) \ + MAX(1 << highbit(3 * (max)), 64) + +#define VMEM_NAMELEN 30 +#define VMEM_HASH_INITIAL 16 +#define VMEM_NQCACHE_MAX 16 +#define VMEM_FREELISTS (sizeof (void *) * 8) + +typedef struct vmem_kstat { + kstat_named_t vk_mem_inuse; /* memory in use */ + kstat_named_t vk_mem_import; /* memory imported */ + kstat_named_t vk_mem_total; /* total memory in arena */ + kstat_named_t vk_source_id; /* vmem id of vmem source */ + kstat_named_t vk_alloc; /* number of allocations */ + kstat_named_t vk_free; /* number of frees */ + kstat_named_t vk_wait; /* number of allocations that waited */ + kstat_named_t vk_fail; /* number of allocations that failed */ + kstat_named_t vk_lookup; /* hash lookup count */ + kstat_named_t vk_search; /* freelist search count */ + kstat_named_t vk_populate_fail; /* populates that failed */ + kstat_named_t vk_contains; /* vmem_contains() calls */ + kstat_named_t vk_contains_search; /* vmem_contains() search cnt */ + kstat_named_t vk_parent_alloc; /* called the source allocator */ + kstat_named_t vk_parent_free; /* called the source free function */ + kstat_named_t vk_threads_waiting; /* threads in cv_wait in vmem */ + kstat_named_t vk_excess; /* count of retained excess imports */ +} vmem_kstat_t; + +struct vmem { + char vm_name[VMEM_NAMELEN]; /* arena name */ + kcondvar_t vm_cv; /* cv for blocking allocations */ + kmutex_t vm_lock; /* arena lock */ + uint32_t vm_id; /* vmem id */ + hrtime_t vm_createtime; + uint32_t vm_mtbf; /* induced alloc failure rate */ + int vm_cflags; /* arena creation flags */ + int vm_qshift; /* log2(vm_quantum) */ + uint32_t vm_quantum; /* vmem quantum */ + uint32_t vm_qcache_max; /* maximum size to front by kmem */ + uint32_t vm_min_import; /* smallest amount to import */ + void *(*vm_source_alloc)(vmem_t *, uint32_t, int); + void (*vm_source_free)(vmem_t *, void *, uint32_t); + vmem_t *vm_source; /* vmem source for imported memory */ + vmem_t *vm_next; /* next in vmem_list */ + kstat_t *vm_ksp; /* kstat */ + ssize_t vm_nsegfree; /* number of free vmem_seg_t's */ + vmem_seg_t *vm_segfree; /* free vmem_seg_t list */ + vmem_seg_t **vm_hash_table; /* allocated-segment hash table */ + uint32_t vm_hash_mask; /* hash_size - 1 */ + uint32_t vm_hash_shift; /* log2(vm_hash_mask + 1) */ + ulong_t vm_freemap; /* bitmap of non-empty freelists */ + vmem_seg_t vm_seg0; /* anchor segment */ + vmem_seg_t vm_rotor; /* rotor for VM_NEXTFIT allocations */ + vmem_seg_t *vm_hash0[VMEM_HASH_INITIAL]; /* initial hash table */ + void *vm_qcache[VMEM_NQCACHE_MAX]; /* quantum caches */ + vmem_freelist_t vm_freelist[VMEM_FREELISTS + 1]; /* power-of-2 flists */ + vmem_kstat_t vm_kstat; /* kstat data */ +}; +typedef struct vmem vmem_t; + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_VMEM_IMPL_H */ diff --git a/include/os/windows/spl/sys/vmsystm.h b/include/os/windows/spl/sys/vmsystm.h new file mode 100644 index 000000000000..3a605a2b9b53 --- /dev/null +++ b/include/os/windows/spl/sys/vmsystm.h @@ -0,0 +1,29 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_VMSYSTM_H +#define _SPL_VMSYSTM_H + +#include + +#define xcopyout copyout + +#endif /* SPL_VMSYSTM_H */ diff --git a/include/os/windows/spl/sys/vnode.h b/include/os/windows/spl/sys/vnode.h new file mode 100644 index 000000000000..701729f49ed4 --- /dev/null +++ b/include/os/windows/spl/sys/vnode.h @@ -0,0 +1,552 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#ifndef _SPL_VNODE_H +#define _SPL_VNODE_H + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +/* Enable to track all IOCOUNT */ +// #define DEBUG_IOCOUNT + + +/* + * Lets define a vnode struct that will hold everything needed for Windows + * request to be handled. + */ +#define VNODE_DEAD 1 +#define VNODE_MARKTERM 2 +#define VNODE_NEEDINACTIVE 4 +#define VNODE_MARKROOT 8 +#define VNODE_SIZECHANGE 16 +#define VNODE_EASIZE 32 +#define VNODE_FLUSHING 64 +#define VNODE_VALIDBITS 127 + +/* v_unlink flags */ +#define UNLINK_DELETE_ON_CLOSE (1 << 0) // 1 +#define UNLINK_DELETED (1 << 1) // 2 + +#include +typedef struct vnode_fileobjects { + avl_node_t avlnode; + void *fileobject; + int remove; +} vnode_fileobjects_t; + + +#pragma pack(8) +struct vnode { + // Windows specific header, has to be first. + FSRTL_ADVANCED_FCB_HEADER FileHeader; + // Mutex for locking access to FileHeader. + FAST_MUTEX AdvancedFcbHeaderMutex; + // mmap file access struct + SECTION_OBJECT_POINTERS SectionObjectPointers; + + OPLOCK oplock; + + // Our implementation data fields + // KSPIN_LOCK v_spinlock; + kmutex_t v_mutex; + + mount_t *v_mount; + uint32_t v_flags; + uint32_t v_iocount; // Short term holds + uint32_t v_usecount; // Long term holds + uint32_t v_type; + uint32_t v_unlink; + REPARSE_DATA_BUFFER *v_reparse; + size_t v_reparse_size; + uint32_t v_unused; + void *v_data; + uint64_t v_id; + uint64_t v_easize; + hrtime_t v_age; // How long since entered DEAD + + // Other Windows entries + // Must be 8byte aligned + ERESOURCE resource; // Holder for FileHeader.Resource + ERESOURCE pageio_resource; // Holder for FileHeader.PageIoResource + FILE_LOCK lock; + SECURITY_DESCRIPTOR *security_descriptor; + SHARE_ACCESS share_access; + + list_node_t v_list; // vnode_all_list member node. + + avl_tree_t v_fileobjects; // All seen FOs that point to this +}; +typedef struct vnode vnode_t; +#pragma pack() + +struct vfs_context; +typedef struct vfs_context *vfs_context_t; + +struct caller_context; +typedef struct caller_context caller_context_t; +typedef int vcexcl_t; + +enum vcexcl { NONEXCL, EXCL }; + +#define VSUID 0x800 /* 04000 */ /* set user id on execution */ +#define VSGID 0x400 /* 02000 */ /* set group id on execution */ +#define VSVTX 0x200 /* 01000 */ /* save swapped text even after use */ +#define VREAD 0x100 /* 00400 */ /* read, write, execute permissions */ +#define VWRITE 0x080 /* 00200 */ +#define VEXEC 0x040 /* 00100 */ + +/* + * Vnode types. VNON means no type. + */ +enum vtype { + /* 0 */ + VNON, + /* 1 - 5 */ + VREG, VDIR, VBLK, VCHR, VLNK, + /* 6 - 10 */ + VSOCK, VFIFO, VBAD, VSTR, VCPLX +}; + +extern enum vtype iftovt_tab[]; +extern int vttoif_tab[]; + +#define IFTOVT(mode) (iftovt_tab[((mode)& S_IFMT) >> 12]) +#define VTTOIF(indx) (vttoif_tab[(int)(indx)]) +#define MAKEIMODE(indx, mode) (int)(VTTOIF(indx) | (mode)) + +#define ATTR_NOACLCHECK 0x20 + +#define F_SEEK_HOLE 0 + +/* + * Windows uses separate vnop getfileinformation to deal with XATTRs, so + * we never get vop&XVATTR set from VFS. All internal checks for it in + * ZFS is not required. + */ +#define ATTR_XVATTR 0 +#define AT_XVATTR ATTR_XVATTR + +#define B_INVAL 0x01 +#define B_TRUNC 0x02 + +#define CREATE_XATTR_DIR 0x04 /* Create extended attr dir */ + + +#define DNLC_NO_VNODE (struct vnode *)(-1) + + +#define IS_DEVVP(vp) \ + (vnode_ischr(vp) || vnode_isblk(vp) || vnode_isfifo(vp)) + +#define VNODE_ATTR_va_rdev (1LL << 0) /* 00000001 */ +#define VNODE_ATTR_va_nlink (1LL << 1) /* 00000002 */ +#define VNODE_ATTR_va_total_size (1LL << 2) /* 00000004 */ +#define VNODE_ATTR_va_total_alloc (1LL << 3) /* 00000008 */ +#define VNODE_ATTR_va_data_size (1LL << 4) /* 00000010 */ +#define VNODE_ATTR_va_data_alloc (1LL << 5) /* 00000020 */ +#define VNODE_ATTR_va_iosize (1LL << 6) /* 00000040 */ +#define VNODE_ATTR_va_uid (1LL << 7) /* 00000080 */ +#define VNODE_ATTR_va_gid (1LL << 8) /* 00000100 */ +#define VNODE_ATTR_va_mode (1LL << 9) /* 00000200 */ +#define VNODE_ATTR_va_flags (1LL << 10) /* 00000400 */ +#define VNODE_ATTR_va_acl (1LL << 11) /* 00000800 */ +#define VNODE_ATTR_va_create_time (1LL << 12) /* 00001000 */ +#define VNODE_ATTR_va_access_time (1LL << 13) /* 00002000 */ +#define VNODE_ATTR_va_modify_time (1LL << 14) /* 00004000 */ +#define VNODE_ATTR_va_change_time (1LL << 15) /* 00008000 */ +#define VNODE_ATTR_va_backup_time (1LL << 16) /* 00010000 */ +#define VNODE_ATTR_va_fileid (1LL << 17) /* 00020000 */ +#define VNODE_ATTR_va_linkid (1LL << 18) /* 00040000 */ +#define VNODE_ATTR_va_parentid (1LL << 19) /* 00080000 */ +#define VNODE_ATTR_va_fsid (1LL << 20) /* 00100000 */ +#define VNODE_ATTR_va_filerev (1LL << 21) /* 00200000 */ +#define VNODE_ATTR_va_gen (1LL << 22) /* 00400000 */ +#define VNODE_ATTR_va_encoding (1LL << 23) /* 00800000 */ +#define VNODE_ATTR_va_type (1LL << 24) /* 01000000 */ +#define VNODE_ATTR_va_name (1LL << 25) /* 02000000 */ +#define VNODE_ATTR_va_uuuid (1LL << 26) /* 04000000 */ +#define VNODE_ATTR_va_guuid (1LL << 27) /* 08000000 */ +#define VNODE_ATTR_va_nchildren (1LL << 28) /* 10000000 */ +#define VNODE_ATTR_va_dirlinkcount (1LL << 29) /* 20000000 */ +#define VNODE_ATTR_va_addedtime (1LL << 30) /* 40000000 */ + +enum rm { RMFILE, RMDIRECTORY }; /* rm or rmdir (remove) */ +enum create { CRCREAT, CRMKNOD, CRMKDIR }; /* reason for create */ + +#define va_mask va_active +#define va_nodeid va_fileid +#define va_nblocks va_filerev + + +/* + * vnode attr translations + */ +#define ATTR_TYPE VNODE_ATTR_va_type +#define ATTR_MODE VNODE_ATTR_va_mode +#define ATTR_ACL VNODE_ATTR_va_acl +#define ATTR_UID VNODE_ATTR_va_uid +#define ATTR_GID VNODE_ATTR_va_gid +#define ATTR_ATIME VNODE_ATTR_va_access_time +#define ATTR_MTIME VNODE_ATTR_va_modify_time +#define ATTR_CTIME VNODE_ATTR_va_change_time +#define ATTR_CRTIME VNODE_ATTR_va_create_time +#define ATTR_SIZE VNODE_ATTR_va_data_size +#define ATTR_NOSET 0 + +#define va_size va_data_size +#define va_atime va_access_time +#define va_mtime va_modify_time +#define va_ctime va_change_time +#define va_crtime va_create_time +#define va_bytes va_data_size + + +// TBD - this comes from XNU, to assist with compiling right now, but +// this struct should be replaced with whatever we cook up for Windows +struct vnode_attr { + uint64_t va_supported; + uint64_t va_active; + int va_vaflags; + dev_t va_rdev; /* device id (device nodes only) */ + uint64_t va_nlink; /* number of references to this file */ + uint64_t va_total_size; /* size in bytes of all forks */ + uint64_t va_total_alloc; /* disk space used by all forks */ + uint64_t va_data_size; /* size in bytes of the fork */ + uint64_t va_data_alloc; /* disk space used by the fork */ + uint32_t va_iosize; /* optimal I/O blocksize */ + + /* file security information */ + uid_t va_uid; /* owner UID */ + gid_t va_gid; /* owner GID */ + mode_t va_mode; /* posix permissions */ + uint32_t va_flags; /* file flags */ + struct kauth_acl *va_acl; /* access control list */ + + struct timespec va_create_time; /* time of creation */ + struct timespec va_access_time; /* time of last access */ + struct timespec va_modify_time; /* time of last data modification */ + struct timespec va_change_time; /* time of last metadata change */ + struct timespec va_backup_time; /* time of last backup */ + + uint64_t va_fileid; /* file unique ID in filesystem */ + uint64_t va_linkid; /* file link unique ID */ + uint64_t va_parentid; /* parent ID */ + uint32_t va_fsid; /* filesystem ID */ + uint64_t va_filerev; /* file revision counter */ + + enum vtype va_type; /* file type (create only) */ + char *va_name; /* Name for ATTR_CMN_NAME; */ + /* MAXPATHLEN bytes */ + +}; +typedef struct vnode_attr vattr; +typedef struct vnode_attr vattr_t; + +/* vsa_mask values */ +#define VSA_ACL 0x0001 +#define VSA_ACLCNT 0x0002 +#define VSA_DFACL 0x0004 +#define VSA_DFACLCNT 0x0008 +#define VSA_ACE 0x0010 +#define VSA_ACECNT 0x0020 +#define VSA_ACE_ALLTYPES 0x0040 +#define VSA_ACE_ACLFLAGS 0x0080 /* get/set ACE ACL flags */ + +/* + * component name operations (for VNOP_LOOKUP) + */ +// Unfortunately 'DELETE' is a Win32 define as well. +// We should consider moving all these to VN_* +#define LOOKUP 0 /* perform name lookup only */ +#define CREATE 1 /* setup for file creation */ +#define VN_DELETE 2 /* setup for file deletion */ +#define RENAME 3 /* setup for file renaming */ +#define OPMASK 3 /* mask for operation */ + +/* + * component name operational modifier flags + */ +#define FOLLOW 0x00000040 /* follow symbolic links */ +#define NOTRIGGER 0x10000000 /* don't trigger automounts */ + +/* + * component name parameter descriptors. + */ +#define ISDOTDOT 0x00002000 /* current component name is .. */ +#define MAKEENTRY 0x00004000 /* entry is to be added to name cache */ +#define ISLASTCN 0x00008000 /* this is last component of pathname */ +#define ISWHITEOUT 0x00020000 /* OBSOLETE: found whiteout */ +#define DOWHITEOUT 0x00040000 /* OBSOLETE: do whiteouts */ + + +struct componentname { + uint32_t cn_nameiop; + uint32_t cn_flags; + char *cn_pnbuf; + int cn_pnlen; + char *cn_nameptr; + int cn_namelen; +}; + +extern struct vnode *vn_alloc(int flag); + +extern int vn_open(char *pnamep, enum zfs_uio_seg seg, int filemode, + int createmode, + struct vnode **vpp, enum create crwhy, mode_t umask); + +extern int vn_openat(char *pnamep, enum zfs_uio_seg seg, int filemode, + int createmode, struct vnode **vpp, enum create crwhy, + mode_t umask, struct vnode *startvp); + +#define vn_renamepath(tdvp, svp, tnm, lentnm) do { } while (0) +#define vn_free(vp) do { } while (0) +#define vn_pages_remove(vp, fl, op) do { } while (0) + +// OSX kernel has a vn_rdwr, let's work around it. +extern int zfs_vn_rdwr(enum zfs_uio_rw rw, struct vnode *vp, caddr_t base, + ssize_t len, offset_t offset, enum zfs_uio_seg seg, + int ioflag, rlim64_t ulimit, cred_t *cr, + ssize_t *residp); + +#define vn_rdwr(rw, vp, base, len, off, seg, flg, limit, cr, resid) \ + zfs_vn_rdwr((rw), (vp), (base), (len), (off), (seg), (flg),\ + (limit), (cr), (resid)) + +extern int vn_remove(char *fnamep, enum zfs_uio_seg seg, enum rm dirflag); +extern int vn_rename(char *from, char *to, enum zfs_uio_seg seg); + +#define LK_RETRY 0 +#define LK_SHARED 0 +#define VN_UNLOCK(vp) +static inline int vn_lock(struct vnode *vp, int fl) { return 0; } + +// KERNEL + +#ifdef DEBUG_IOCOUNT +#define VN_HOLD(vp) vnode_getwithref(vp, __FILE__, __LINE__) +#define VN_RELE(vp) \ + do { \ + if ((vp) && (vp) != DNLC_NO_VNODE) \ + vnode_put(vp, __FILE__, __LINE__); \ + } while (0) +#define vnode_getwithvid(V, ID) \ + vnode_debug_getwithvid((V), (ID), __FILE__, __LINE__) + +#else + +#define VN_HOLD(vp) vnode_getwithref(vp) +#define VN_RELE(vp) vnode_put(vp); + +#endif + +void spl_rele_async(void *arg); +void vn_rele_async(struct vnode *vp, void *taskq); + +#define VN_RELE_ASYNC(vp, tq) vn_rele_async((vp), (tq)) + +#define vn_exists(vp) +#define vn_is_readonly(vp) vnode_vfsisrdonly(vp) + +#define VATTR_NULL(v) do { } while (0) + +extern int +VOP_CLOSE(struct vnode *vp, int flag, int count, offset_t off, + void *cr, void *); +extern int +VOP_FSYNC(struct vnode *vp, int flags, void* unused, void *); +extern int +VOP_SPACE(HANDLE h, int cmd, struct flock *fl, int flags, offset_t off, + cred_t *cr, void *ctx); + +extern int VOP_GETATTR(struct vnode *vp, vattr_t *vap, int flags, + void *x3, void *x4); + +#define VOP_UNLOCK(vp, fl) do { } while (0) + +void vfs_mountedfrom(struct mount *vfsp, char *osname); + +extern struct vnode *dnlc_lookup(struct vnode *dvp, char *name); +extern int dnlc_purge_vfsp(struct mount *mp, int flags); +extern void dnlc_remove(struct vnode *vp, char *name); +extern void dnlc_update(struct vnode *vp, char *name, + struct vnode *tp); + +extern struct vnode *rootdir; + +static inline int +chklock(struct vnode *vp, int iomode, unsigned long long offset, + ssize_t len, int fmode, void *ct) +{ + return (0); +} + +#define vn_has_cached_data(VP) 0 +/* (VTOZ(VP)->z_is_mapped || vnode_isswap(VP) || win_has_cached_data(VP)) */ + +static inline int +win_has_cached_data(struct vnode *vp) +{ + int ret = 0; + PFILE_OBJECT pfo = + CcGetFileObjectFromSectionPtrsRef(&vp->SectionObjectPointers); + if (pfo) { + ret = CcIsFileCached(pfo); + ObDereferenceObject(pfo); + } + return (ret); +} + + +#define vn_ismntpt(vp) (vnode_mountedhere(vp) != NULL) + +void spl_vnode_fini(void); +int spl_vnode_init(void); + +extern int spl_vfs_root(mount_t *mount, struct vnode **vp); +#define VFS_ROOT(V, L, VP) spl_vfs_root((V), (VP)) + +extern void cache_purgevfs(mount_t *mp); + +int spl_vn_rdwr(enum zfs_uio_rw rw, struct vnode *vp, caddr_t base, ssize_t len, + offset_t offset, enum zfs_uio_seg seg, int ioflag, rlim64_t ulimit, + cred_t *cr, ssize_t *residp); + +extern int spl_vnode_notify(struct vnode *vp, uint32_t type, + struct vnode_attr *vap); +extern int spl_vfs_get_notify_attributes(struct vnode_attr *vap); +extern void spl_hijack_mountroot(void *func); +extern void spl_setrootvnode(struct vnode *vp); + +struct vnode *getrootdir(void); +void spl_vfs_start(void); + +int vnode_vfsisrdonly(vnode_t *vp); +uint64_t vnode_vid(vnode_t *vp); +int vnode_isreg(vnode_t *vp); +int vnode_isdir(vnode_t *vp); +#ifdef DEBUG_IOCOUNT +int vnode_debug_getwithvid(vnode_t *, uint64_t, char *, int); +int vnode_getwithref(vnode_t *vp, char *, int); +int vnode_put(vnode_t *vp, char *, int); +void vnode_check_iocount(void); +#else +int vnode_getwithvid(vnode_t *, uint64_t); +int vnode_getwithref(vnode_t *vp); +int vnode_put(vnode_t *vp); +#endif + +void *vnode_fsnode(struct vnode *dvp); +enum vtype vnode_vtype(vnode_t *vp); +int vnode_isblk(vnode_t *vp); +int vnode_ischr(vnode_t *vp); +int vnode_isswap(vnode_t *vp); +int vnode_isfifo(vnode_t *vp); +int vnode_islnk(vnode_t *vp); +mount_t *vnode_mountedhere(vnode_t *vp); +void ubc_setsize(struct vnode *, uint64_t); +int vnode_isinuse(vnode_t *vp, uint64_t refcnt); +int vnode_isidle(vnode_t *vp); +int vnode_recycle(vnode_t *vp); +int vnode_isvroot(vnode_t *vp); +mount_t *vnode_mount(vnode_t *vp); +void vnode_clearfsnode(vnode_t *vp); +void vnode_create(mount_t *, void *v_data, int type, int flags, + struct vnode **vpp); +int vnode_ref(vnode_t *vp); +void vnode_rele(vnode_t *vp); +void *vnode_sectionpointer(vnode_t *vp); +void *vnode_security(vnode_t *vp); +void vnode_setsecurity(vnode_t *vp, void *sd); +void vnode_couplefileobject(vnode_t *vp, FILE_OBJECT *fileobject, + uint64_t size); +void vnode_decouplefileobject(vnode_t *vp, FILE_OBJECT *fileobject); +void vnode_setsizechange(vnode_t *vp, int set); +int vnode_sizechange(vnode_t *vp); +int vnode_isrecycled(vnode_t *vp); +dev_t vnode_specrdev(vnode_t *vp); +void cache_purge(vnode_t *vp); +void cache_purge_negatives(vnode_t *vp); +int vnode_removefsref(vnode_t *vp); +int vnode_iocount(vnode_t *vp); +void vnode_pager_setsize(void *fo, vnode_t *vp, uint64_t size, boolean_t delay); +void vnode_set_reparse(struct vnode *vp, REPARSE_DATA_BUFFER *rpp, size_t size); +ULONG vnode_get_reparse_tag(struct vnode *vp); +int vnode_get_reparse_point(struct vnode *vp, REPARSE_DATA_BUFFER **rpp, + size_t *size); + + +#define VNODE_READDIR_EXTENDED 1 + +#define SKIPSYSTEM 0x0001 /* vflush: skip vnodes marked VSYSTEM */ +#define FORCECLOSE 0x0002 /* vflush: force file closeure */ +#define WRITECLOSE 0x0004 /* vflush: only close writeable files */ +#define SKIPSWAP 0x0008 /* vflush: skip vnodes marked VSWAP */ +#define SKIPROOT 0x0010 /* vflush: skip root vnodes marked VROOT */ +#define VNODELOCKED 0x0100 /* vflush: vnode already called to recycle */ +#define NULLVP NULL + +#define LOOKUP_XATTR 0x2 + +int vflush(struct mount *mp, struct vnode *skipvp, int flags); +int vnode_fileobject_add(vnode_t *vp, void *fo); +int vnode_fileobject_remove(vnode_t *vp, void *fo); +int vnode_fileobject_empty(vnode_t *vp, int locked); +int vnode_fileobject_member(vnode_t *vp, void *fo); + +void vnode_lock(vnode_t *vp); +void vnode_unlock(vnode_t *vp); +int vnode_drain_delayclose(int); +int vnode_easize(struct vnode *vp, uint64_t *size); +void vnode_set_easize(struct vnode *vp, uint64_t size); +void vnode_clear_easize(struct vnode *vp); +int vnode_flushcache(vnode_t *vp, FILE_OBJECT *fileobject, boolean_t); + +int kernel_ioctl(PDEVICE_OBJECT DeviceObject, PFILE_OBJECT, long cmd, + void *inbuf, uint32_t inlen, void *outbuf, uint32_t outlen); + +/* Linux TRIM API */ +int blk_queue_discard(PDEVICE_OBJECT dev); +int blk_queue_discard_secure(PDEVICE_OBJECT dev); +int blk_queue_nonrot(PDEVICE_OBJECT dev); +int blkdev_issue_discard_bytes(PDEVICE_OBJECT dev, uint64_t offset, + uint64_t size, uint32_t flags); + +POPLOCK vp_oplock(struct vnode *vp); + +#endif /* SPL_VNODE_H */ diff --git a/include/os/windows/spl/sys/wmsum.h b/include/os/windows/spl/sys/wmsum.h new file mode 100644 index 000000000000..a024cdc27a69 --- /dev/null +++ b/include/os/windows/spl/sys/wmsum.h @@ -0,0 +1,72 @@ +/* + * CDDL HEADER START + * + * This file and its contents are supplied under the terms of the + * Common Development and Distribution License ("CDDL"), version 1.0. + * You may only use this file in accordance with the terms of version + * 1.0 of the CDDL. + * + * A full copy of the text of the CDDL should have accompanied this + * source. A copy of the CDDL is also available via the Internet at + * http://www.illumos.org/license/CDDL. + * + * CDDL HEADER END + */ + +/* + * wmsum counters are a reduced version of aggsum counters, optimized for + * write-mostly scenarios. They do not provide optimized read functions, + * but instead allow much cheaper add function. The primary usage is + * infrequently read statistic counters, not requiring exact precision. + * + * Find a Windows variant of whatever this is + * + * Copyright (c) 2017 Jorgen Lundman + */ + +#ifndef _SYS_WMSUM_H +#define _SYS_WMSUM_H + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +struct wmsum +{ + uint64_t wm_value; +}; + +typedef struct wmsum wmsum_t; + +static inline void +wmsum_init(wmsum_t *ws, uint64_t value) +{ + ws->wm_value = value; +} + +static inline void +wmsum_fini(wmsum_t *ws) +{ +} + +static inline uint64_t +wmsum_value(wmsum_t *ws) +{ + return (atomic_add_64_nv(&ws->wm_value, 0)); +} + +static inline void +wmsum_add(wmsum_t *ws, int64_t delta) +{ + atomic_add_64(&ws->wm_value, delta); +} + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_WMSUM_H */ diff --git a/include/os/windows/spl/sys/zmod.h b/include/os/windows/spl/sys/zmod.h new file mode 100644 index 000000000000..1218bcc283a1 --- /dev/null +++ b/include/os/windows/spl/sys/zmod.h @@ -0,0 +1,121 @@ +/* + * zlib.h -- interface of the 'zlib' general purpose compression library + * version 1.2.5, April 19th, 2010 + * + * Copyright (C) 1995-2010 Jean-loup Gailly and Mark Adler + * + * This software is provided 'as-is', without any express or implied + * warranty. In no event will the authors be held liable for any damages + * arising from the use of this software. + * + * Permission is granted to anyone to use this software for any purpose, + * including commercial applications, and to alter it and redistribute it + * freely, subject to the following restrictions: + * + * 1. The origin of this software must not be misrepresented; you must not + * claim that you wrote the original software. If you use this software + * in a product, an acknowledgment in the product documentation would be + * appreciated but is not required. + * 2. Altered source versions must be plainly marked as such, and must not be + * misrepresented as being the original software. + * 3. This notice may not be removed or altered from any source distribution. + * + * Jean-loup Gailly + * Mark Adler + */ + +#ifndef _SPL_ZMOD_H +#define _SPL_ZMOD_H + +#include +#include +#include + +struct _zmemheader { + uint64_t length; + char data[0]; +}; + +static inline void * +zfs_zalloc(void* opaque, uInt items, uInt size) +{ + struct _zmemheader *hdr; + uint32_t alloc_size = (items * size) + sizeof (uint64_t); + hdr = kmem_zalloc(alloc_size, KM_SLEEP); + hdr->length = alloc_size; + return (&hdr->data); +} + +static inline void +zfs_zfree(void *opaque, void *addr) +{ + struct _zmemheader *hdr; + hdr = addr; + hdr--; + kmem_free(hdr, (uint32_t)hdr->length); +} + +/* + * Uncompress the buffer 'src' into the buffer 'dst'. The caller must store + * the expected decompressed data size externally so it can be passed in. + * The resulting decompressed size is then returned through dstlen. This + * function return Z_OK on success, or another error code on failure. + */ +static inline int + z_uncompress(void *dst, uint32_t *dstlen, const void *src, uint32_t srclen) +{ + z_stream zs; + int err; + + memset(&zs, 0, sizeof (zs)); + zs.next_in = (uchar_t *)src; + zs.avail_in = srclen; + zs.next_out = dst; + zs.avail_out = *dstlen; + zs.zalloc = zfs_zalloc; + zs.zfree = zfs_zfree; + if ((err = inflateInit(&zs)) != Z_OK) + return (err); + if ((err = inflate(&zs, Z_FINISH)) != Z_STREAM_END) { + (void) inflateEnd(&zs); + return (err == Z_OK ? Z_BUF_ERROR : err); + } + *dstlen = zs.total_out; + return (inflateEnd(&zs)); +} + +static inline int +z_compress_level(void *dst, uint32_t *dstlen, const void *src, uint32_t srclen, + int level) +{ + z_stream zs; + int err; + memset(&zs, 0, sizeof (zs)); + zs.next_in = (uchar_t *)src; + zs.avail_in = srclen; + zs.next_out = dst; + zs.avail_out = *dstlen; + zs.zalloc = zfs_zalloc; + zs.zfree = zfs_zfree; + if ((err = deflateInit(&zs, level)) != Z_OK) + return (err); + if ((err = deflate(&zs, Z_FINISH)) != Z_STREAM_END) { + (void) deflateEnd(&zs); + return (err == Z_OK ? Z_BUF_ERROR : err); + } + *dstlen = zs.total_out; + return (deflateEnd(&zs)); +} + +static inline int +z_compress(void *dst, uint32_t *dstlen, const void *src, uint32_t srclen) +{ + return (z_compress_level(dst, dstlen, src, srclen, + Z_DEFAULT_COMPRESSION)); +} + + +int spl_zlib_init(void); +void spl_zlib_fini(void); + +#endif /* SPL_ZMOD_H */ diff --git a/include/os/windows/spl/sys/zone.h b/include/os/windows/spl/sys/zone.h new file mode 100644 index 000000000000..6408d4939551 --- /dev/null +++ b/include/os/windows/spl/sys/zone.h @@ -0,0 +1,34 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_ZONE_H +#define _SPL_ZONE_H + +#include + +#define GLOBAL_ZONEID 0 + +#define zone_dataset_visible(x, y) (1) +#define INGLOBALZONE(z) (1) + +#define crgetzoneid(x) (GLOBAL_ZONEID) + +#endif /* SPL_ZONE_H */ diff --git a/include/os/windows/spl/unistd.h b/include/os/windows/spl/unistd.h new file mode 100644 index 000000000000..7f53079756e9 --- /dev/null +++ b/include/os/windows/spl/unistd.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_UNISTD_H +#define _SPL_UNISTD_H + +#endif /* SPL_UNISTD_H */ diff --git a/include/os/windows/spl_config.h b/include/os/windows/spl_config.h new file mode 100644 index 000000000000..24fa133b359e --- /dev/null +++ b/include/os/windows/spl_config.h @@ -0,0 +1,102 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* spl_config.h. This file is not autogenerated on Windows. */ + +/* Copyright(c) 2015 Jorgen Lundman */ + +/* Define to 1 to enable basic kmem accounting */ +#define DEBUG_KMEM 1 + +/* Define to 1 to enable detailed kmem tracking */ +/* #undef DEBUG_KMEM_TRACKING */ + +/* Define to 1 to enable basic debug logging */ +/* #undef DEBUG_LOG */ + +/* Define to 1 if you have the header file. */ +#define HAVE_DLFCN_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_INTTYPES_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_MEMORY_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STDINT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STDLIB_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRINGS_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRING_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_UNISTD_H 1 + +/* Define to the sub-directory where libtool stores uninstalled libraries. */ +#define LT_OBJDIR ".libs/" + +/* Define to 1 to enable mutex leak accounting */ +/* #undef SPL_DEBUG_MUTEX */ + +/* Define the kernel dependencies version. */ +#define SPL_KERNEL_DEPENDENCIES "28" + +/* Define the project alias string. */ +#define SPL_META_ALIAS "spl-1.6.0-7-gc1b4a00" + +/* Define the project author. */ +/* #undef SPL_META_AUTHOR */ + +/* Define the project release date. */ +/* #undef SPL_META_DATA */ + +/* Define the libtool library 'age' version information. */ +/* #undef SPL_META_LT_AGE */ + +/* Define the libtool library 'current' version information. */ +/* #undef SPL_META_LT_CURRENT */ + +/* Define the libtool library 'revision' version information. */ +/* #undef SPL_META_LT_REVISION */ + +/* Define the project name. */ +#define SPL_META_NAME "spl" + +/* Define the project release. */ +#define SPL_META_RELEASE "7-gc1b4a00" + +/* Define the project version. */ +#define SPL_META_VERSION "0.2.3" + +/* Define ZFS_BOOT to enable kext load at boot */ +#define ZFS_BOOT 1 diff --git a/include/os/windows/zfs/sys/fs/zfsdi.h b/include/os/windows/zfs/sys/fs/zfsdi.h new file mode 100644 index 000000000000..609b5ebbd1b0 --- /dev/null +++ b/include/os/windows/zfs/sys/fs/zfsdi.h @@ -0,0 +1,146 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2020, DataCore Software Corp. + */ + +#ifndef _SYS_ZFSZVOLDI_H +#define _SYS_ZFSZVOLDI_H + +#include +#ifdef __cplusplus +extern "C" { +#endif + +/* + * The structures in this file are passed between kernel drivers who need to use + * the zvol Direct Interface (DI). + * + * To resolve the interface use an IRP_MJ_PNP irp, initialize the stack the + * following way and send it to the ZFS_DEV_KERNEL device object (use + * IoGetDeviceObjectPointer() to resolve it) + * + * pStack->MinorFunction = IRP_MN_QUERY_INTERFACE; + * pStack->Parameters.QueryInterface.InterfaceType = (LPGUID)&ZFSZVOLDI_GUID; + * pStack->Parameters.QueryInterface.Size = sizeof(zfsdizvol_t); + * pStack->Parameters.QueryInterface.Version = ZFSZVOLDI_VERSION; + * pStack->Parameters.QueryInterface.Interface = (PINTERFACE); + * pStack->Parameters.QueryInterface.InterfaceSpecificData = ; + * + * The full T10 value string is of the form <8-bytes-vendorid + + * vendorspecific>: "OpenZFS poolname/zvolname" + * + * Upon a STATUS_SUCCESS return value the Context field in the INTERFACE + * header portion of the zfsdizvol_t structure will be initialized. That Context + * value is opaque, identifies the ZVOL being accessed, and must be set in all + * interface calls. + */ + +#define ZFS_DEV_KERNEL L"\\Device\\ZFSCTL" +DEFINE_GUID(ZFSZVOLDI_GUID, 0x904ca0cdl, 0x6ae1, 0x4acb, 0xb8, 0xb9, 0x2a, 0x00, + 0x2e, 0xd1, 0x10, 0xd4); + +#define ZFSZVOLDI_VERSION 1 // Interface version + +/* + * I/O Flags definition + * + * ZFSZVOLFG_AlwaysPend: + * When set, forces ZFS to always complete the I/O asynchronously through its + * own threading model. The caller will always get STATUS_PENDING back from + * the interface request. + * When not set, ZFS will use the caller's thread context to perform the direct + * interface I/O (if at all possible). + */ +#define ZFSZVOLFG_AlwaysPend 0x1 + +/* + * I/O descriptor + * + * Each I/O call through the interface requires a zfsiodesc_t control block. + * The first 4 fields must be initialized, others are for use by the caller if + * needed and serve as optional callback routine and context values to that + * callback. + * + * The ZFS driver will make its own copy of this control block if it can't + * handle the request synchronously so it can be allocated on the caller's + * stack (no need to perform a dynamic allocation for each I/O). + * + * Input values: + * + * Buffer: must be in system address space. The caller is responsible for + * getting that address mapped properly. + * ByteOffset: I/O offset, in bytes. + * Length: I/O length, in bytes + * Flags: 0 by default, see Flags definition above. + * + * Optional Caller input values: + * + * Cb: Caller callback routine. NULL if none needed. + * Synchronous I/O return: the callback is always called prior to + * returning synchronously to the caller. + * Asynchronous I/O return: the callback is always called and the caller + * would have gotten STATUS_PENDING back from the request. + * + * - pIo: could be different than the one passed in the request. However + * all fields will be copied from the request intact. + * - status: ultimate outcome for the I/O. + * - bPendingReturned: if TRUE, the caller has gotten STATUS_PENDING on + * its request. This can be used to set the upstream IRP pending. + * If FALSE, the request has been processed synchronously and the + * caller will get control just after the callback has run. + * ex: if the caller has logic to wait on an event in case of + * STATUS_PENDING, this flag can be used to skip setting the event + * since the request was not pended. + * + * CbParm[]: 4 context values that the caller can use in its callback routine. + * + */ +typedef struct zfsiodesc { + PVOID Buffer; + LONGLONG ByteOffset; + ULONG Length; + ULONG Flags; // Optional flags (see ZFSZVOLFG_xxxxxxx) + // Optional I/O initiator's callback routine and parameter (must set + // to NULL if not used) + // Always called at final I/O completion by ZFS (whether the I/O is + // synchronous or not). + void(*Cb)(struct zfsiodesc *pIo, NTSTATUS status, BOOLEAN bPendingReturned); + PVOID CbParm[4]; +} zfsiodesc_t; + +/* + * Query interface descriptor. + */ +typedef struct { + INTERFACE header; + NTSTATUS(*Read)(PVOID Context, zfsiodesc_t *pIo); + NTSTATUS(*Write)(PVOID Context, zfsiodesc_t *pIo); + // Add new interface routines here. do not modify the existing order. +} zfsdizvol_t; + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_ZFSZVOLDI_H */ diff --git a/include/os/windows/zfs/sys/policy.h b/include/os/windows/zfs/sys/policy.h new file mode 100644 index 000000000000..18a6f40d05ad --- /dev/null +++ b/include/os/windows/zfs/sys/policy.h @@ -0,0 +1,25 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef POLICY_H_INCLUDED +#define POLICY_H_INCLUDED + +#endif diff --git a/include/os/windows/zfs/sys/vdev_disk_os.h b/include/os/windows/zfs/sys/vdev_disk_os.h new file mode 100644 index 000000000000..f1f08f395a91 --- /dev/null +++ b/include/os/windows/zfs/sys/vdev_disk_os.h @@ -0,0 +1,38 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#ifndef _ZFS_VDEV_DISK_OS_H +#define _ZFS_VDEV_DISK_OS_H + +typedef struct vdev_disk { + ddi_devid_t vd_devid; + char *vd_minor; + list_t vd_ldi_cbs; + boolean_t vd_ldi_offline; + HANDLE vd_lh; + PFILE_OBJECT vd_FileObject; + PDEVICE_OBJECT vd_DeviceObject; + PDEVICE_OBJECT vd_ExclusiveObject; + uint64_t vdev_win_offset; /* soft partition start */ + uint64_t vdev_win_length; /* soft partition length */ +} vdev_disk_t; + +#endif diff --git a/include/os/windows/zfs/sys/wzvol.h b/include/os/windows/zfs/sys/wzvol.h new file mode 100644 index 000000000000..3f4d2e8d011e --- /dev/null +++ b/include/os/windows/zfs/sys/wzvol.h @@ -0,0 +1,493 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2018 by Jorgen Lundman . + */ + +#ifndef _SPL_WZVOL_H +#define _SPL_WZVOL_H + +#include +#include +#include +#include +#include +#include + +#define VENDOR_ID L"OpenZFS " +#define VENDOR_ID_ascii "OpenZFS " +#define PRODUCT_ID L"WinZVOL " +#define PRODUCT_ID_ascii "WinZVOL " +#define PRODUCT_REV L"1.00" +#define PRODUCT_REV_ascii "1.00" +#define MP_TAG_GENERAL 'LOVZ' + +#define WZOL_MAX_TARGETS 16 // 8! A bit low + +#define MAX_TARGETS WZOL_MAX_TARGETS +#define MAX_LUNS 24 +#define MP_MAX_TRANSFER_SIZE (32 * 1024) +#define TIME_INTERVAL (1 * 1000 * 1000) // 1 second. +#define DEVLIST_BUFFER_SIZE 1024 +#define DEVICE_NOT_FOUND 0xFF +#define SECTOR_NOT_FOUND 0xFFFF + +// Minimum size required for Disk Manager +#define MINIMUM_DISK_SIZE (1540 * 1024) + +#define MAXIMUM_MAP_DISK_SIZE (256 * 1024) + +#define MP_BLOCK_SIZE (512) +#define BUF_SIZE (1540 * 1024) +#define MAX_BLOCKS (BUF_SIZE / MP_BLOCK_SIZE) + +#define DEFAULT_BREAK_ON_ENTRY 0 +#define DEFAULT_DEBUG_LEVEL 2 +#define DEFAULT_INITIATOR_ID 7 +#define DEFAULT_VIRTUAL_DISK_SIZE (8 * 1024 * 1024) +#define DEFAULT_PHYSICAL_DISK_SIZE DEFAULT_VIRTUAL_DISK_SIZE +#define DEFAULT_USE_LBA_LIST 0 +#define DEFAULT_NUMBER_OF_BUSES 1 +#define DEFAULT_NbrVirtDisks 1 +#define DEFAULT_NbrLUNsperHBA 400 +#define DEFAULT_NbrLUNsperTarget 32 +#define DEFAULT_bCombineVirtDisks FALSE + +#define GET_FLAG(Flags, Bit) ((Flags) & (Bit)) +#define SET_FLAG(Flags, Bit) ((Flags) |= (Bit)) +#define CLEAR_FLAG(Flags, Bit) ((Flags) &= ~(Bit)) + +typedef struct _DEVICE_LIST DEVICE_LIST, *pDEVICE_LIST; +typedef struct _wzvolDriverInfo wzvolDriverInfo, *pwzvolDriverInfo; +typedef struct _MP_REG_INFO MP_REG_INFO, *pMP_REG_INFO; +typedef struct _HW_LU_EXTENSION HW_LU_EXTENSION, *pHW_LU_EXTENSION; +typedef struct _HW_LU_EXTENSION_MPIO + HW_LU_EXTENSION_MPIO, *pHW_LU_EXTENSION_MPIO; +typedef struct _LBA_LIST LBA_LIST, *PLBA_LIST; + +extern wzvolDriverInfo STOR_wzvolDriverInfo; + +typedef struct _MP_REG_INFO { + UNICODE_STRING VendorId; + UNICODE_STRING ProductId; + UNICODE_STRING ProductRevision; + ULONG BreakOnEntry; // Break into debugger + ULONG DebugLevel; // Debug log level + ULONG InitiatorID; // Adapter's target ID + ULONG VirtualDiskSize; // Disk size to be reported + ULONG PhysicalDiskSize; // Disk size to be allocated + ULONG NbrVirtDisks; // Number of virtual disks. + ULONG NbrLUNsperHBA; // Number of LUNs per HBA + // really is the amount of zvols we can present through StorPort + + ULONG NbrLUNsperTarget; // Number of LUNs per Target. + ULONG bCombineVirtDisks; // 0 => do not combine virtual/MPIO +} WZVOL_REG_INFO, *pWZVOL_REG_INFO; + +typedef struct _wzvolContext { + PVOID zv; + PIO_REMOVE_LOCK pIoRemLock; + volatile ULONGLONG refCnt; +} wzvolContext, *pwzvolContext; + + +// The master miniport object. In effect, an extension of the driver object +// for the miniport. +typedef struct _wzvolDriverInfo { + WZVOL_REG_INFO wzvolRegInfo; + KSPIN_LOCK DrvInfoLock; + // Lock for ListMPIOExt, header of list of HW_LU_EXTENSION_MPIO objects + KSPIN_LOCK MPIOExtLock; + KSPIN_LOCK SrbExtLock; // Lock for ListSrbExt + LIST_ENTRY ListMPHBAObj; // Header of list of HW_HBA_EXT + LIST_ENTRY ListMPIOExt; // Header HW_LU_EXTENSION_MPIO + LIST_ENTRY ListSrbExt; // Header HW_SRB_EXTENSION + PDRIVER_OBJECT pDriverObj; + wzvolContext *zvContextArray; + ULONG DrvInfoNbrMPHBAObj; // Count of items in ListMPHBAObj. + ULONG DrvInfoNbrMPIOExtObj; // Count of items in ListMPIOExt. + UCHAR MaximumNumberOfLogicalUnits; + UCHAR MaximumNumberOfTargets; + UCHAR NumberOfBuses; +} wzvolDriverInfo, *pwzvolDriverInfo; + +typedef struct _LUNInfo { + UCHAR bReportLUNsDontUse; + UCHAR bIODontUse; +} LUNInfo, *pLUNInfo; + +#define DISK_DEVICE 0x00 + +typedef struct _MP_DEVICE_INFO { + UCHAR DeviceType; + UCHAR TargetID; + UCHAR LunID; +} MP_DEVICE_INFO, *pMP_DEVICE_INFO; + +typedef struct _MP_DEVICE_LIST { + ULONG DeviceCount; + MP_DEVICE_INFO DeviceInfo[1]; +} MP_DEVICE_LIST, *pMP_DEVICE_LIST; + +#define LUNInfoMax 8 + +// Adapter device-object extension allocated by StorPort. +typedef struct _HW_HBA_EXT { + LIST_ENTRY List; // Pointers to next&prev HW_HBA_EXT. + LIST_ENTRY LUList; // Pointers to HW_LU_EXTENSION. + LIST_ENTRY MPIOLunList; + pwzvolDriverInfo pwzvolDrvObj; + PDRIVER_OBJECT pDrvObj; + SCSI_WMILIB_CONTEXT WmiLibContext; + PIRP pReverseCallIrp; + KSPIN_LOCK WkItemsLock; + KSPIN_LOCK WkRoutinesLock; + KSPIN_LOCK MPHBAObjLock; + KSPIN_LOCK LUListLock; + ULONG SRBsSeen; + ULONG WMISRBsSeen; + ULONG NbrMPIOLuns; + ULONG NbrLUNsperHBA; + ULONG Test; + UCHAR HostTargetId; + UCHAR AdapterState; + UCHAR VendorId[9]; + UCHAR ProductId[17]; + UCHAR ProductRevision[5]; + + BOOLEAN bDontReport; // TRUE => no Report LUNs. + BOOLEAN bReportAdapterDone; + // To be set only by a kernel debugger. + LUNInfo LUNInfoArray[LUNInfoMax]; +} HW_HBA_EXT, *pHW_HBA_EXT; + +// Collector for LUNs that are represented by MPIO as 1 pseudo-LUN. +typedef struct _HW_LU_EXTENSION_MPIO { + LIST_ENTRY List; // Ptrs next&prev HW_LU_EXTENSION_MPIO + LIST_ENTRY LUExtList; // Header of list of HW_LU_EXTENSION. + KSPIN_LOCK LUExtMPIOLock; + ULONG NbrRealLUNs; + SCSI_ADDRESS ScsiAddr; + PUCHAR pDiskBuf; + USHORT MaxBlocks; + // At present, this is set only by a kernel debugger, for testing. + BOOLEAN bIsMissingOnAnyPath; +} HW_LU_EXTENSION_MPIO, *pHW_LU_EXTENSION_MPIO; + +// Flag definitions for LUFlags. + +#define LU_DEVICE_INITIALIZED 0x0001 +#define LU_MPIO_MAPPED 0x0004 + +// LUN extension allocated by StorPort. +typedef struct _HW_LU_EXTENSION { + LIST_ENTRY List; + LIST_ENTRY MPIOList; + pHW_LU_EXTENSION_MPIO pLUMPIOExt; + PUCHAR pDiskBuf; + ULONG LUFlags; + USHORT MaxBlocks; + USHORT BlocksUsed; + BOOLEAN bIsMissing; + UCHAR DeviceType; + UCHAR TargetId; + UCHAR Lun; +} HW_LU_EXTENSION, *pHW_LU_EXTENSION; + + +typedef enum { + ActionRead, + ActionWrite, + ActionUnmap +} MpWkRtnAction; + + +typedef struct _MP_WorkRtnParms { + pHW_HBA_EXT pHBAExt; + PSCSI_REQUEST_BLOCK pSrb; + PEPROCESS pReqProcess; + MpWkRtnAction Action; + ULONG SecondsToDelay; + /* ZFS ZVOLDI */ + void* zv; + zfsiodesc_t ioDesc; + taskq_ent_t ent; + CHAR pQueueWorkItem[1]; + // IO_WORKITEM struct: keep at the end (dynamically allocated). +} MP_WorkRtnParms, *pMP_WorkRtnParms; + +typedef struct _HW_SRB_EXTENSION { + SCSIWMI_REQUEST_CONTEXT WmiRequestContext; + LIST_ENTRY QueuedForProcessing; + volatile ULONG Cancelled; + PSCSI_REQUEST_BLOCK pSrbBackPtr; + MP_WorkRtnParms WkRtnParms; + // keep at the end of this block (pQueueWorkItem dynamically allocated) +} HW_SRB_EXTENSION, *PHW_SRB_EXTENSION; + +enum ResultType { + ResultDone, + ResultQueued +}; + +#define RegWkBfrSz 0x1000 + +typedef struct _RegWorkBuffer { + pHW_HBA_EXT pAdapterExt; + UCHAR Work[256]; +} RegWorkBuffer, *pRegWorkBuffer; + + +ULONG +wzvol_HwFindAdapter( + __in pHW_HBA_EXT DevExt, + __in PVOID HwContext, + __in PVOID BusInfo, + __in PVOID LowerDevice, + __in PCHAR ArgumentString, + __in __out PPORT_CONFIGURATION_INFORMATION ConfigInfo, + __out PBOOLEAN Again +); + +void +wzvol_HwTimer( + __in pHW_HBA_EXT DevExt +); + +BOOLEAN +wzvol_HwInitialize( + __in pHW_HBA_EXT +); + +void +wzvol_HwReportAdapter( + __in pHW_HBA_EXT +); + +void +wzvol_HwReportLink( + __in pHW_HBA_EXT +); + +void +wzvol_HwReportLog(__in pHW_HBA_EXT); + +void +wzvol_HwFreeAdapterResources( + __in pHW_HBA_EXT +); + +BOOLEAN +wzvol_HwStartIo( + __in pHW_HBA_EXT, + __in PSCSI_REQUEST_BLOCK +); + +BOOLEAN +wzvol_HwResetBus( + __in pHW_HBA_EXT, + __in ULONG +); + +SCSI_ADAPTER_CONTROL_STATUS +wzvol_HwAdapterControl( + __in pHW_HBA_EXT DevExt, + __in SCSI_ADAPTER_CONTROL_TYPE ControlType, + __in PVOID Parameters +); + +UCHAR +ScsiExecuteMain( + __in pHW_HBA_EXT DevExt, + __in PSCSI_REQUEST_BLOCK, + __in PUCHAR +); + +UCHAR +ScsiExecute( + __in pHW_HBA_EXT DevExt, + __in PSCSI_REQUEST_BLOCK Srb +); + +UCHAR +ScsiOpInquiry( + __in pHW_HBA_EXT DevExt, + __in PSCSI_REQUEST_BLOCK Srb +); + +UCHAR +ScsiOpReadCapacity( + IN pHW_HBA_EXT DevExt, + IN PSCSI_REQUEST_BLOCK Srb +); + +UCHAR +ScsiOpReadCapacity16( + IN pHW_HBA_EXT DevExt, + IN PSCSI_REQUEST_BLOCK Srb +); + +UCHAR +ScsiOpRead( + IN pHW_HBA_EXT DevExt, + IN PSCSI_REQUEST_BLOCK Srb, + IN PUCHAR Action +); + +UCHAR +ScsiOpWrite( + IN pHW_HBA_EXT DevExt, + IN PSCSI_REQUEST_BLOCK Srb, + IN PUCHAR Action +); + +UCHAR +ScsiOpUnmap( + IN pHW_HBA_EXT DevExt, + IN PSCSI_REQUEST_BLOCK Srb, + IN PUCHAR Action +); + +UCHAR +ScsiOpModeSense( + IN pHW_HBA_EXT DevExt, + IN PSCSI_REQUEST_BLOCK pSrb +); + +UCHAR +ScsiOpReportLuns( + IN pHW_HBA_EXT DevExt, + IN PSCSI_REQUEST_BLOCK Srb +); + +void +wzvol_QueryRegParameters( + IN PUNICODE_STRING, + IN pMP_REG_INFO +); + +NTSTATUS +wzvol_CreateDeviceList( + __in pHW_HBA_EXT, + __in ULONG +); + +UCHAR +wzvol_GetDeviceType( + __in pHW_HBA_EXT DevExt, + __in UCHAR PathId, + __in UCHAR TargetId, + __in UCHAR Lun +); + +UCHAR wzvol_FindRemovedDevice( + __in pHW_HBA_EXT, + __in PSCSI_REQUEST_BLOCK +); + +void wzvol_StopAdapter( + __in pHW_HBA_EXT DevExt +); + +void +wzvol_TracingInit( + __in PVOID, + __in PVOID +); + +void +wzvol_TracingCleanup(__in PVOID); + +void +wzvol_ProcServReq( + __in pHW_HBA_EXT, + __in PIRP +); + +void +wzvol_CompServReq( + __in pHW_HBA_EXT +); + +UCHAR +ScsiOpVPD( + __in pHW_HBA_EXT, + __in PSCSI_REQUEST_BLOCK, + __in PVOID +); + +void +InitializeWmiContext(__in pHW_HBA_EXT); + +BOOLEAN +HandleWmiSrb( + __in pHW_HBA_EXT, + __in __out PSCSI_WMI_REQUEST_BLOCK +); + +UCHAR +ScsiReadWriteSetup( + __in pHW_HBA_EXT pDevExt, + __in PSCSI_REQUEST_BLOCK pSrb, + __in MpWkRtnAction WkRtnAction, + __in PUCHAR pResult +); + +void +wzvol_GeneralWkRtn( + __in PVOID, + __in PVOID +); + +ULONG +wzvol_ThreadWkRtn(__in PVOID); + +void +wzvol_WkRtn(IN PVOID); + +void +wzvol_CompleteIrp( + __in pHW_HBA_EXT, + __in PIRP +); + +void +wzvol_QueueServiceIrp( + __in pHW_HBA_EXT pDevExt, + __in PIRP pIrp +); + +void +wzvol_ProcServReq( + __in pHW_HBA_EXT pDevExt, + __in PIRP pIrp +); + +void +wzvol_CompServReq( + __in pHW_HBA_EXT pDevExt +); + +extern int zvol_start(PDRIVER_OBJECT DriverObject, + PUNICODE_STRING pRegistryPath); + +#endif diff --git a/include/os/windows/zfs/sys/wzvolwmi.h b/include/os/windows/zfs/sys/wzvolwmi.h new file mode 100644 index 000000000000..0ee22d856fdb --- /dev/null +++ b/include/os/windows/zfs/sys/wzvolwmi.h @@ -0,0 +1,4884 @@ +#ifndef _mpwmi_h_ +#define _mpwmi_h_ + +// ************************************************************************** +// +// hbapiwmi.h +// +// Module: WDM classes to expose HBA api data from drivers +// +// Purpose: Contains WDM classes that specify the HBA data to be exposed +// via the HBA api set. +// +// NOTE: This file contains information that is based upon: +// SM-HBA Version 1.0 and FC-HBA 2.18 specification. +// +// Please specify which WMI interfaces the provider will implement by +// defining MS_SM_HBA_API or MSFC_HBA_API before including this file. +// That is: +// +// #define MS_SM_HBA_API +// #include +// +// - or - +// +// #define MSFC_HBA_API +// #include +// +// +// Copyright (c) 2001 Microsoft Corporation +// +// ************************************************************************** + + +#define MSFC_HBAPortStatisticsGuid \ + { 0x3ce7904f, 0x459f, 0x480d, \ + { 0x9a, 0x3c, 0x01, 0x3e, 0xde, 0x3b, 0xdd, 0xe8 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_HBAPortStatistics_GUID, \ + 0x3ce7904f, 0x459f, 0x480d, 0x9a, 0x3c, 0x01, 0x3e, 0xde, 0x3b, \ + 0xdd, 0xe8); +#endif + +typedef struct _MSFC_HBAPortStatistics +{ + // + LONGLONG SecondsSinceLastReset; +#define MSFC_HBAPortStatistics_SecondsSinceLastReset_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_SecondsSinceLastReset_ID 1 + + // + LONGLONG TxFrames; +#define MSFC_HBAPortStatistics_TxFrames_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_TxFrames_ID 2 + + // + LONGLONG TxWords; +#define MSFC_HBAPortStatistics_TxWords_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_TxWords_ID 3 + + // + LONGLONG RxFrames; +#define MSFC_HBAPortStatistics_RxFrames_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_RxFrames_ID 4 + + // + LONGLONG RxWords; +#define MSFC_HBAPortStatistics_RxWords_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_RxWords_ID 5 + + // + LONGLONG LIPCount; +#define MSFC_HBAPortStatistics_LIPCount_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_LIPCount_ID 6 + + // + LONGLONG NOSCount; +#define MSFC_HBAPortStatistics_NOSCount_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_NOSCount_ID 7 + + // + LONGLONG ErrorFrames; +#define MSFC_HBAPortStatistics_ErrorFrames_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_ErrorFrames_ID 8 + + // + LONGLONG DumpedFrames; +#define MSFC_HBAPortStatistics_DumpedFrames_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_DumpedFrames_ID 9 + + // + LONGLONG LinkFailureCount; +#define MSFC_HBAPortStatistics_LinkFailureCount_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_LinkFailureCount_ID 10 + + // + LONGLONG LossOfSyncCount; +#define MSFC_HBAPortStatistics_LossOfSyncCount_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_LossOfSyncCount_ID 11 + + // + LONGLONG LossOfSignalCount; +#define MSFC_HBAPortStatistics_LossOfSignalCount_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_LossOfSignalCount_ID 12 + + // + LONGLONG PrimitiveSeqProtocolErrCount; +#define MSFC_HBAPortStatistics_PrimitiveSeqProtocolErrCount_SIZE \ + sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_PrimitiveSeqProtocolErrCount_ID 13 + + // + LONGLONG InvalidTxWordCount; +#define MSFC_HBAPortStatistics_InvalidTxWordCount_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_InvalidTxWordCount_ID 14 + + // + LONGLONG InvalidCRCCount; +#define MSFC_HBAPortStatistics_InvalidCRCCount_SIZE sizeof (LONGLONG) +#define MSFC_HBAPortStatistics_InvalidCRCCount_ID 15 + +} MSFC_HBAPortStatistics, *PMSFC_HBAPortStatistics; + +#define MSFC_HBAPortStatistics_SIZE (FIELD_OFFSET(MSFC_HBAPortStatistics, \ + InvalidCRCCount) + MSFC_HBAPortStatistics_InvalidCRCCount_SIZE) + +// HBAFC3MgmtInfo - HBAFC3MgmtInfo +#define HBAFC3MgmtInfoGuid \ + { 0x5966a24f, 0x6aa5, 0x418e, \ + { 0xb7, 0x5c, 0x2f, 0x21, 0x4d, 0xfb, 0x4b, 0x18 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(HBAFC3MgmtInfo_GUID, \ + 0x5966a24f, 0x6aa5, 0x418e, 0xb7, 0x5c, 0x2f, 0x21, + 0x4d, 0xfb, 0x4b, 0x18); +#endif + + +typedef struct _HBAFC3MgmtInfo +{ + // + ULONGLONG UniqueAdapterId; +#define HBAFC3MgmtInfo_UniqueAdapterId_SIZE sizeof (ULONGLONG) +#define HBAFC3MgmtInfo_UniqueAdapterId_ID 1 + + // + UCHAR wwn[8]; +#define HBAFC3MgmtInfo_wwn_SIZE sizeof (UCHAR[8]) +#define HBAFC3MgmtInfo_wwn_ID 2 + + // + ULONG unittype; +#define HBAFC3MgmtInfo_unittype_SIZE sizeof (ULONG) +#define HBAFC3MgmtInfo_unittype_ID 3 + + // + ULONG PortId; +#define HBAFC3MgmtInfo_PortId_SIZE sizeof (ULONG) +#define HBAFC3MgmtInfo_PortId_ID 4 + + // + ULONG NumberOfAttachedNodes; +#define HBAFC3MgmtInfo_NumberOfAttachedNodes_SIZE sizeof (ULONG) +#define HBAFC3MgmtInfo_NumberOfAttachedNodes_ID 5 + + // + USHORT IPVersion; +#define HBAFC3MgmtInfo_IPVersion_SIZE sizeof (USHORT) +#define HBAFC3MgmtInfo_IPVersion_ID 6 + + // + USHORT UDPPort; +#define HBAFC3MgmtInfo_UDPPort_SIZE sizeof (USHORT) +#define HBAFC3MgmtInfo_UDPPort_ID 7 + + // + UCHAR IPAddress[16]; +#define HBAFC3MgmtInfo_IPAddress_SIZE sizeof (UCHAR[16]) +#define HBAFC3MgmtInfo_IPAddress_ID 8 + + // + USHORT reserved; +#define HBAFC3MgmtInfo_reserved_SIZE sizeof (USHORT) +#define HBAFC3MgmtInfo_reserved_ID 9 + + // + USHORT TopologyDiscoveryFlags; +#define HBAFC3MgmtInfo_TopologyDiscoveryFlags_SIZE sizeof (USHORT) +#define HBAFC3MgmtInfo_TopologyDiscoveryFlags_ID 10 + + // + ULONG reserved1; +#define HBAFC3MgmtInfo_reserved1_SIZE sizeof (ULONG) +#define HBAFC3MgmtInfo_reserved1_ID 11 + +} HBAFC3MgmtInfo, *PHBAFC3MgmtInfo; + +#define HBAFC3MgmtInfo_SIZE (FIELD_OFFSET(HBAFC3MgmtInfo, reserved1) + \ + HBAFC3MgmtInfo_reserved1_SIZE) + +// HBAScsiID - HBAScsiID +#define HBAScsiIDGuid \ + { 0xa76f5058, 0xb1f0, 0x4622, \ + { 0x9e, 0x88, 0x5c, 0xc4, 0x1e, 0x34, 0x45, 0x4a } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(HBAScsiID_GUID, \ + 0xa76f5058, 0xb1f0, 0x4622, 0x9e, 0x88, 0x5c, 0xc4, \ + 0x1e, 0x34, 0x45, 0x4a); +#endif + + +typedef struct _HBAScsiID +{ + // + ULONG ScsiBusNumber; +#define HBAScsiID_ScsiBusNumber_SIZE sizeof (ULONG) +#define HBAScsiID_ScsiBusNumber_ID 1 + + // + ULONG ScsiTargetNumber; +#define HBAScsiID_ScsiTargetNumber_SIZE sizeof (ULONG) +#define HBAScsiID_ScsiTargetNumber_ID 2 + + // + ULONG ScsiOSLun; +#define HBAScsiID_ScsiOSLun_SIZE sizeof (ULONG) +#define HBAScsiID_ScsiOSLun_ID 3 + + // ****************************************************************** + // + // This used to be a string type, but we made this a fixed length + // array so the WmiSizeIs() will work correctly for structs that + // contain this type. + // Please note that this should still be treated as a string. + // The first uint16 must hold the length of string (in bytes). + // + // ****************************************************************** + + // + USHORT OSDeviceName[257]; +#define HBAScsiID_OSDeviceName_SIZE sizeof (USHORT[257]) +#define HBAScsiID_OSDeviceName_ID 4 + +} HBAScsiID, *PHBAScsiID; + +#define HBAScsiID_SIZE \ + (FIELD_OFFSET(HBAScsiID, OSDeviceName) + HBAScsiID_OSDeviceName_SIZE) + +// MSFC_LinkEvent - MSFC_LinkEvent + + + + +// +// Event types. +// +// These match the definitions in hbaapi.h and must be kept in sync. +// +/* Adapter Level Events */ +#define HBA_EVENT_ADAPTER_UNKNOWN 0x100 +#define HBA_EVENT_ADAPTER_ADD 0x101 +#define HBA_EVENT_ADAPTER_REMOVE 0x102 +#define HBA_EVENT_ADAPTER_CHANGE 0x103 + +/* Port Level Events */ +#define HBA_EVENT_PORT_UNKNOWN 0x200 +#define HBA_EVENT_PORT_OFFLINE 0x201 +#define HBA_EVENT_PORT_ONLINE 0x202 +#define HBA_EVENT_PORT_NEW_TARGETS 0x203 +#define HBA_EVENT_PORT_FABRIC 0x204 +#define HBA_EVENT_PORT_BROADCAST_CHANGE 0x205 +#define HBA_EVENT_PORT_BROADCAST_D24_0 0x206 +#define HBA_EVENT_PORT_BROADCAST_D27_4 0x207 +#define HBA_EVENT_PORT_BROADCAST_SES 0x208 +#define HBA_EVENT_PORT_BROADCAST_D01_4 0x209 +#define HBA_EVENT_PORT_BROADCAST_D04_7 0x20a +#define HBA_EVENT_PORT_BROADCAST_D16_7 0x20b +#define HBA_EVENT_PORT_BROADCAST_D29_7 0x20c +#define HBA_EVENT_PORT_ALL 0x2ff + +/* Port Statistics Events */ +#define HBA_EVENT_PORT_STAT_THRESHOLD 0x301 +#define HBA_EVENT_PORT_STAT_GROWTH 0x302 + +/* Phy Statistics Events */ +#define HBA_EVENT_PHY_STAT_THRESHOLD 0x351 +#define HBA_EVENT_PHY_STAT_GROWTH 0x352 + +/* Target Level Events */ +#define HBA_EVENT_TARGET_UNKNOWN 0x400 +#define HBA_EVENT_TARGET_OFFLINE 0x401 +#define HBA_EVENT_TARGET_ONLINE 0x402 +#define HBA_EVENT_TARGET_REMOVED 0x403 + +/* Fabric Link Events */ +#define HBA_EVENT_LINK_UNKNOWN 0x500 +#define HBA_EVENT_LINK_INCIDENT 0x501 + +#define MSFC_LinkEventGuid \ + { 0xc66015ee, 0x014b, 0x498a, \ + { 0x94, 0x51, 0x99, 0xfe, 0xad, 0x0a, 0xb4, 0x51 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_LinkEvent_GUID, \ + 0xc66015ee, 0x014b, 0x498a, 0x94, 0x51, 0x99, 0xfe, \ + 0xad, 0x0a, 0xb4, 0x51); +#endif + +typedef struct _MSFC_LinkEvent +{ + // + ULONG EventType; +#define MSFC_LinkEvent_EventType_SIZE sizeof (ULONG) +#define MSFC_LinkEvent_EventType_ID 1 + + // + UCHAR AdapterWWN[8]; +#define MSFC_LinkEvent_AdapterWWN_SIZE sizeof (UCHAR[8]) +#define MSFC_LinkEvent_AdapterWWN_ID 2 + + // + ULONG RLIRBufferSize; +#define MSFC_LinkEvent_RLIRBufferSize_SIZE sizeof (ULONG) +#define MSFC_LinkEvent_RLIRBufferSize_ID 3 + + // + UCHAR RLIRBuffer[1]; +#define MSFC_LinkEvent_RLIRBuffer_ID 4 + +} MSFC_LinkEvent, *PMSFC_LinkEvent; + +// MSFC_FCAdapterHBAAttributes - MSFC_FCAdapterHBAAttributes + +#ifndef MS_SM_HBA_API +#ifndef MSFC_HBA_API +// +// if neither defined then default to MSFC +// +#define MSFC_HBA_API +#endif +#endif + +#ifdef MSFC_HBA_API + +#define MSFC_FCAdapterHBAAttributesGuid \ + { 0xf8f3ea26, 0xab2c, 0x4593, \ + { 0x8b, 0x84, 0xc5, 0x64, 0x28, 0xe6, 0xbe, 0xdb } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_FCAdapterHBAAttributes_GUID, \ + 0xf8f3ea26, 0xab2c, 0x4593, 0x8b, 0x84, 0xc5, 0x64,\ + 0x28, 0xe6, 0xbe, 0xdb); +#endif + + +typedef struct _MSFC_FCAdapterHBAAttributes +{ + // + ULONGLONG UniqueAdapterId; +#define MSFC_FCAdapterHBAAttributes_UniqueAdapterId_SIZE sizeof (ULONGLONG) +#define MSFC_FCAdapterHBAAttributes_UniqueAdapterId_ID 1 + + // + ULONG HBAStatus; +#define MSFC_FCAdapterHBAAttributes_HBAStatus_SIZE sizeof (ULONG) +#define MSFC_FCAdapterHBAAttributes_HBAStatus_ID 2 + + // + UCHAR NodeWWN[8]; +#define MSFC_FCAdapterHBAAttributes_NodeWWN_SIZE sizeof (UCHAR[8]) +#define MSFC_FCAdapterHBAAttributes_NodeWWN_ID 3 + + // + ULONG VendorSpecificID; +#define MSFC_FCAdapterHBAAttributes_VendorSpecificID_SIZE sizeof (ULONG) +#define MSFC_FCAdapterHBAAttributes_VendorSpecificID_ID 4 + + // + ULONG NumberOfPorts; +#define MSFC_FCAdapterHBAAttributes_NumberOfPorts_SIZE sizeof (ULONG) +#define MSFC_FCAdapterHBAAttributes_NumberOfPorts_ID 5 + + // ****************************************************************** + // + // The string type is variable length (up to MaxLen). + // Each string starts with a ushort that holds the strings length + // (in bytes) followed by the WCHARs that make up the string. + // + // ****************************************************************** + + // + WCHAR Manufacturer[64 + 1]; +#define MSFC_FCAdapterHBAAttributes_Manufacturer_ID 6 + + // + WCHAR SerialNumber[64 + 1]; +#define MSFC_FCAdapterHBAAttributes_SerialNumber_ID 7 + + // + WCHAR Model[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_Model_ID 8 + + // + WCHAR ModelDescription[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_ModelDescription_ID 9 + + // + WCHAR NodeSymbolicName[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_NodeSymbolicName_ID 10 + + // + WCHAR HardwareVersion[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_HardwareVersion_ID 11 + + // + WCHAR DriverVersion[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_DriverVersion_ID 12 + + // + WCHAR OptionROMVersion[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_OptionROMVersion_ID 13 + + // + WCHAR FirmwareVersion[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_FirmwareVersion_ID 14 + + // + WCHAR DriverName[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_DriverName_ID 15 + + // + WCHAR MfgDomain[256 + 1]; +#define MSFC_FCAdapterHBAAttributes_MfgDomain_ID 16 + +} MSFC_FCAdapterHBAAttributes, *PMSFC_FCAdapterHBAAttributes; + +// MSFC_HBAPortAttributesResults - MSFC_HBAPortAttributesResults +#define MSFC_HBAPortAttributesResultsGuid \ + { 0xa76bd4e3, 0x9961, 0x4d9b, \ + { 0xb6, 0xbe, 0x86, 0xe6, 0x98, 0x26, 0x0f, 0x68 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_HBAPortAttributesResults_GUID, \ + 0xa76bd4e3, 0x9961, 0x4d9b, 0xb6, 0xbe, 0x86, 0xe6,\ + 0x98, 0x26, 0x0f, 0x68); +#endif + +typedef struct _MSFC_HBAPortAttributesResults +{ + // + UCHAR NodeWWN[8]; +#define MSFC_HBAPortAttributesResults_NodeWWN_SIZE sizeof (UCHAR[8]) +#define MSFC_HBAPortAttributesResults_NodeWWN_ID 1 + + // + UCHAR PortWWN[8]; +#define MSFC_HBAPortAttributesResults_PortWWN_SIZE sizeof (UCHAR[8]) +#define MSFC_HBAPortAttributesResults_PortWWN_ID 2 + + // + ULONG PortFcId; +#define MSFC_HBAPortAttributesResults_PortFcId_SIZE sizeof (ULONG) +#define MSFC_HBAPortAttributesResults_PortFcId_ID 3 + + // + ULONG PortType; +#define MSFC_HBAPortAttributesResults_PortType_SIZE sizeof (ULONG) +#define MSFC_HBAPortAttributesResults_PortType_ID 4 + + // + ULONG PortState; +#define MSFC_HBAPortAttributesResults_PortState_SIZE sizeof (ULONG) +#define MSFC_HBAPortAttributesResults_PortState_ID 5 + + // + ULONG PortSupportedClassofService; +#define MSFC_HBAPortAttributesResults_PortSupportedClassofService_SIZE \ + sizeof (ULONG) +#define MSFC_HBAPortAttributesResults_PortSupportedClassofService_ID 6 + + // + UCHAR PortSupportedFc4Types[32]; +#define MSFC_HBAPortAttributesResults_PortSupportedFc4Types_SIZE \ + sizeof (UCHAR[32]) +#define MSFC_HBAPortAttributesResults_PortSupportedFc4Types_ID 7 + + // + UCHAR PortActiveFc4Types[32]; +#define MSFC_HBAPortAttributesResults_PortActiveFc4Types_SIZE \ + sizeof (UCHAR[32]) +#define MSFC_HBAPortAttributesResults_PortActiveFc4Types_ID 8 + + // + ULONG PortSupportedSpeed; +#define MSFC_HBAPortAttributesResults_PortSupportedSpeed_SIZE sizeof (ULONG) +#define MSFC_HBAPortAttributesResults_PortSupportedSpeed_ID 9 + + // + ULONG PortSpeed; +#define MSFC_HBAPortAttributesResults_PortSpeed_SIZE sizeof (ULONG) +#define MSFC_HBAPortAttributesResults_PortSpeed_ID 10 + + // + ULONG PortMaxFrameSize; +#define MSFC_HBAPortAttributesResults_PortMaxFrameSize_SIZE sizeof (ULONG) +#define MSFC_HBAPortAttributesResults_PortMaxFrameSize_ID 11 + + // + UCHAR FabricName[8]; +#define MSFC_HBAPortAttributesResults_FabricName_SIZE sizeof (UCHAR[8]) +#define MSFC_HBAPortAttributesResults_FabricName_ID 12 + + // + ULONG NumberofDiscoveredPorts; +#define MSFC_HBAPortAttributesResults_NumberofDiscoveredPorts_SIZE \ + sizeof (ULONG) +#define MSFC_HBAPortAttributesResults_NumberofDiscoveredPorts_ID 13 + +} MSFC_HBAPortAttributesResults, *PMSFC_HBAPortAttributesResults; + +#define MSFC_HBAPortAttributesResults_SIZE \ + (FIELD_OFFSET(MSFC_HBAPortAttributesResults, NumberofDiscoveredPorts) +\ + MSFC_HBAPortAttributesResults_NumberofDiscoveredPorts_SIZE) + +// MSFC_FibrePortHBAAttributes - MSFC_FibrePortHBAAttributes +#define MSFC_FibrePortHBAAttributesGuid \ + { 0x61b397fd, 0xf5ae, 0x4950, { 0x97, 0x58, 0x0e, 0xe5, 0x98,\ + 0xe3, 0xc6, 0xe6 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_FibrePortHBAAttributes_GUID, \ + 0x61b397fd, 0xf5ae, 0x4950, 0x97, 0x58, 0x0e, 0xe5,\ + 0x98, 0xe3, 0xc6, 0xe6); +#endif + + +typedef struct _MSFC_FibrePortHBAAttributes +{ + // + ULONGLONG UniquePortId; +#define MSFC_FibrePortHBAAttributes_UniquePortId_SIZE sizeof (ULONGLONG) +#define MSFC_FibrePortHBAAttributes_UniquePortId_ID 1 + + // + ULONG HBAStatus; +#define MSFC_FibrePortHBAAttributes_HBAStatus_SIZE sizeof (ULONG) +#define MSFC_FibrePortHBAAttributes_HBAStatus_ID 2 + + // + MSFC_HBAPortAttributesResults Attributes; +#define MSFC_FibrePortHBAAttributes_Attributes_SIZE \ + sizeof (MSFC_HBAPortAttributesResults) +#define MSFC_FibrePortHBAAttributes_Attributes_ID 3 + +} MSFC_FibrePortHBAAttributes, *PMSFC_FibrePortHBAAttributes; + +#define MSFC_FibrePortHBAAttributes_SIZE \ + (FIELD_OFFSET(MSFC_FibrePortHBAAttributes, Attributes) + \ + MSFC_FibrePortHBAAttributes_Attributes_SIZE) + +// MSFC_FibrePortHBAStatistics - MSFC_FibrePortHBAStatistics +#define MSFC_FibrePortHBAStatisticsGuid \ + { 0x27efaba4, 0x362a, 0x4f20, \ + { 0x92, 0x0b, 0xed, 0x66, 0xe2, 0x80, 0xfc, 0xf5 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_FibrePortHBAStatistics_GUID, \ + 0x27efaba4, 0x362a, 0x4f20, 0x92, 0x0b, 0xed, 0x66,\ + 0xe2, 0x80, 0xfc, 0xf5); +#endif + + +typedef struct _MSFC_FibrePortHBAStatistics +{ + // + ULONGLONG UniquePortId; +#define MSFC_FibrePortHBAStatistics_UniquePortId_SIZE sizeof (ULONGLONG) +#define MSFC_FibrePortHBAStatistics_UniquePortId_ID 1 + + // + ULONG HBAStatus; +#define MSFC_FibrePortHBAStatistics_HBAStatus_SIZE sizeof (ULONG) +#define MSFC_FibrePortHBAStatistics_HBAStatus_ID 2 + + // + MSFC_HBAPortStatistics Statistics; +#define MSFC_FibrePortHBAStatistics_Statistics_SIZE \ + sizeof (MSFC_HBAPortStatistics) +#define MSFC_FibrePortHBAStatistics_Statistics_ID 3 + +} MSFC_FibrePortHBAStatistics, *PMSFC_FibrePortHBAStatistics; + +#define MSFC_FibrePortHBAStatistics_SIZE \ + (FIELD_OFFSET(MSFC_FibrePortHBAStatistics, Statistics) + \ + MSFC_FibrePortHBAStatistics_Statistics_SIZE) + +// MSFC_FibrePortHBAMethods - MSFC_FibrePortHBAMethods +#define MSFC_FibrePortHBAMethodsGuid \ + { 0xe693553e, 0xedf6, 0x4d57, \ + { 0xbf, 0x08, 0xef, 0xca, 0xae, 0x1a, 0x2e, 0x1c } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_FibrePortHBAMethods_GUID, \ + 0xe693553e, 0xedf6, 0x4d57, 0xbf, 0x08, 0xef, 0xca, \ + 0xae, 0x1a, 0x2e, 0x1c); +#endif + +// +// Method id definitions for MSFC_FibrePortHBAMethods +#define ResetStatistics 1 + +// MSFC_FC4STATISTICS - MSFC_FC4STATISTICS +#define MSFC_FC4STATISTICSGuid \ + { 0xca8e7fe6, 0xb85e, 0x497f, \ + { 0x88, 0x58, 0x9b, 0x5d, 0x93, 0xa6, 0x6f, 0xe1 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_FC4STATISTICS_GUID, \ + 0xca8e7fe6, 0xb85e, 0x497f, 0x88, 0x58, 0x9b, 0x5d, \ + 0x93, 0xa6, 0x6f, 0xe1); +#endif + + +typedef struct _MSFC_FC4STATISTICS +{ + // + ULONGLONG InputRequests; +#define MSFC_FC4STATISTICS_InputRequests_SIZE sizeof (ULONGLONG) +#define MSFC_FC4STATISTICS_InputRequests_ID 1 + + // + ULONGLONG OutputRequests; +#define MSFC_FC4STATISTICS_OutputRequests_SIZE sizeof (ULONGLONG) +#define MSFC_FC4STATISTICS_OutputRequests_ID 2 + + // + ULONGLONG ControlRequests; +#define MSFC_FC4STATISTICS_ControlRequests_SIZE sizeof (ULONGLONG) +#define MSFC_FC4STATISTICS_ControlRequests_ID 3 + + // + ULONGLONG InputMegabytes; +#define MSFC_FC4STATISTICS_InputMegabytes_SIZE sizeof (ULONGLONG) +#define MSFC_FC4STATISTICS_InputMegabytes_ID 4 + + // + ULONGLONG OutputMegabytes; +#define MSFC_FC4STATISTICS_OutputMegabytes_SIZE sizeof (ULONGLONG) +#define MSFC_FC4STATISTICS_OutputMegabytes_ID 5 + +} MSFC_FC4STATISTICS, *PMSFC_FC4STATISTICS; + +#define MSFC_FC4STATISTICS_SIZE \ + (FIELD_OFFSET(MSFC_FC4STATISTICS, OutputMegabytes) +\ + MSFC_FC4STATISTICS_OutputMegabytes_SIZE) + +// MSFC_EventBuffer - MSFC_EventBuffer +#define MSFC_EventBufferGuid \ + { 0x623f4588, 0xcf01, 0x4f0e, \ + { 0xb1, 0x97, 0xab, 0xbe, 0xe5, 0xe0, 0xcf, 0xf3 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_EventBuffer_GUID, \ + 0x623f4588, 0xcf01, 0x4f0e, 0xb1, 0x97, 0xab, 0xbe, \ + 0xe5, 0xe0, 0xcf, 0xf3); +#endif + + +typedef struct _MSFC_EventBuffer +{ + // + ULONG EventType; +#define MSFC_EventBuffer_EventType_SIZE sizeof (ULONG) +#define MSFC_EventBuffer_EventType_ID 1 + + // + ULONG EventInfo[4]; +#define MSFC_EventBuffer_EventInfo_SIZE sizeof (ULONG[4]) +#define MSFC_EventBuffer_EventInfo_ID 2 + +} MSFC_EventBuffer, *PMSFC_EventBuffer; + +#define MSFC_EventBuffer_SIZE \ + (FIELD_OFFSET(MSFC_EventBuffer, EventInfo) + \ + MSFC_EventBuffer_EventInfo_SIZE) + +// MSFC_HBAAdapterMethods - MSFC_HBAAdapterMethods +#define MSFC_HBAAdapterMethodsGuid \ + { 0xdf87d4ed, 0x4612, 0x4d12, \ + { 0x85, 0xfb, 0x83, 0x57, 0x4e, 0xc3, 0x4b, 0x7c } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_HBAAdapterMethods_GUID, \ + 0xdf87d4ed, 0x4612, 0x4d12, 0x85, 0xfb, 0x83, 0x57, \ + 0x4e, 0xc3, 0x4b, 0x7c); +#endif + +// +// Method id definitions for MSFC_HBAAdapterMethods +#define GetDiscoveredPortAttributes 1 +typedef struct _GetDiscoveredPortAttributes_IN +{ + // + ULONG PortIndex; +#define GetDiscoveredPortAttributes_IN_PortIndex_SIZE sizeof (ULONG) +#define GetDiscoveredPortAttributes_IN_PortIndex_ID 1 + + // + ULONG DiscoveredPortIndex; +#define GetDiscoveredPortAttributes_IN_DiscoveredPortIndex_SIZE sizeof (ULONG) +#define GetDiscoveredPortAttributes_IN_DiscoveredPortIndex_ID 2 + +} GetDiscoveredPortAttributes_IN, *PGetDiscoveredPortAttributes_IN; + +#define GetDiscoveredPortAttributes_IN_SIZE \ + (FIELD_OFFSET(GetDiscoveredPortAttributes_IN, DiscoveredPortIndex) +\ + GetDiscoveredPortAttributes_IN_DiscoveredPortIndex_SIZE) + +typedef struct _GetDiscoveredPortAttributes_OUT +{ + // + ULONG HBAStatus; +#define GetDiscoveredPortAttributes_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetDiscoveredPortAttributes_OUT_HBAStatus_ID 3 + + // + MSFC_HBAPortAttributesResults PortAttributes; +#define GetDiscoveredPortAttributes_OUT_PortAttributes_SIZE \ + sizeof (MSFC_HBAPortAttributesResults) +#define GetDiscoveredPortAttributes_OUT_PortAttributes_ID 4 + +} GetDiscoveredPortAttributes_OUT, *PGetDiscoveredPortAttributes_OUT; + +#define GetDiscoveredPortAttributes_OUT_SIZE \ + (FIELD_OFFSET(GetDiscoveredPortAttributes_OUT, PortAttributes) + \ + GetDiscoveredPortAttributes_OUT_PortAttributes_SIZE) + +#define GetPortAttributesByWWN 2 +typedef struct _GetPortAttributesByWWN_IN +{ + // + UCHAR wwn[8]; +#define GetPortAttributesByWWN_IN_wwn_SIZE sizeof (UCHAR[8]) +#define GetPortAttributesByWWN_IN_wwn_ID 1 + +} GetPortAttributesByWWN_IN, *PGetPortAttributesByWWN_IN; + +#define GetPortAttributesByWWN_IN_SIZE \ + (FIELD_OFFSET(GetPortAttributesByWWN_IN, wwn) + \ + GetPortAttributesByWWN_IN_wwn_SIZE) + +typedef struct _GetPortAttributesByWWN_OUT +{ + // + ULONG HBAStatus; +#define GetPortAttributesByWWN_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetPortAttributesByWWN_OUT_HBAStatus_ID 2 + + // + MSFC_HBAPortAttributesResults PortAttributes; +#define GetPortAttributesByWWN_OUT_PortAttributes_SIZE \ + sizeof (MSFC_HBAPortAttributesResults) +#define GetPortAttributesByWWN_OUT_PortAttributes_ID 3 + +} GetPortAttributesByWWN_OUT, *PGetPortAttributesByWWN_OUT; + +#define GetPortAttributesByWWN_OUT_SIZE \ + (FIELD_OFFSET(GetPortAttributesByWWN_OUT, PortAttributes) +\ + GetPortAttributesByWWN_OUT_PortAttributes_SIZE) + +#define RefreshInformation 3 +#define SendCTPassThru 4 +typedef struct _SendCTPassThru_IN +{ + // + UCHAR PortWWN[8]; +#define SendCTPassThru_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SendCTPassThru_IN_PortWWN_ID 1 + + // + ULONG RequestBufferCount; +#define SendCTPassThru_IN_RequestBufferCount_SIZE sizeof (ULONG) +#define SendCTPassThru_IN_RequestBufferCount_ID 2 + + // + UCHAR RequestBuffer[1]; +#define SendCTPassThru_IN_RequestBuffer_ID 3 + +} SendCTPassThru_IN, *PSendCTPassThru_IN; + +typedef struct _SendCTPassThru_OUT +{ + // + ULONG HBAStatus; +#define SendCTPassThru_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SendCTPassThru_OUT_HBAStatus_ID 4 + + // + ULONG TotalResponseBufferCount; +#define SendCTPassThru_OUT_TotalResponseBufferCount_SIZE sizeof (ULONG) +#define SendCTPassThru_OUT_TotalResponseBufferCount_ID 5 + + // + ULONG ActualResponseBufferCount; +#define SendCTPassThru_OUT_ActualResponseBufferCount_SIZE sizeof (ULONG) +#define SendCTPassThru_OUT_ActualResponseBufferCount_ID 6 + + +#define SendCTPassThru_OUT_ResponseBuffer_SIZE_HINT 768 + + // + UCHAR ResponseBuffer[1]; +#define SendCTPassThru_OUT_ResponseBuffer_ID 7 + +} SendCTPassThru_OUT, *PSendCTPassThru_OUT; + +#define SendRNID 5 +typedef struct _SendRNID_IN +{ + // + UCHAR wwn[8]; +#define SendRNID_IN_wwn_SIZE sizeof (UCHAR[8]) +#define SendRNID_IN_wwn_ID 1 + + // + ULONG wwntype; +#define SendRNID_IN_wwntype_SIZE sizeof (ULONG) +#define SendRNID_IN_wwntype_ID 2 + +} SendRNID_IN, *PSendRNID_IN; + +#define SendRNID_IN_SIZE \ + (FIELD_OFFSET(SendRNID_IN, wwntype) + SendRNID_IN_wwntype_SIZE) + +typedef struct _SendRNID_OUT +{ + // + ULONG HBAStatus; +#define SendRNID_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SendRNID_OUT_HBAStatus_ID 3 + + // + ULONG ResponseBufferCount; +#define SendRNID_OUT_ResponseBufferCount_SIZE sizeof (ULONG) +#define SendRNID_OUT_ResponseBufferCount_ID 4 + + +#define SendRNID_OUT_ResponseBuffer_SIZE_HINT 76 + + // + UCHAR ResponseBuffer[1]; +#define SendRNID_OUT_ResponseBuffer_ID 5 + +} SendRNID_OUT, *PSendRNID_OUT; + +#define SendRNIDV2 6 +typedef struct _SendRNIDV2_IN +{ + // + UCHAR PortWWN[8]; +#define SendRNIDV2_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SendRNIDV2_IN_PortWWN_ID 1 + + // + UCHAR DestWWN[8]; +#define SendRNIDV2_IN_DestWWN_SIZE sizeof (UCHAR[8]) +#define SendRNIDV2_IN_DestWWN_ID 2 + + // + ULONG DestFCID; +#define SendRNIDV2_IN_DestFCID_SIZE sizeof (ULONG) +#define SendRNIDV2_IN_DestFCID_ID 3 + + // + ULONG NodeIdDataFormat; +#define SendRNIDV2_IN_NodeIdDataFormat_SIZE sizeof (ULONG) +#define SendRNIDV2_IN_NodeIdDataFormat_ID 4 + +} SendRNIDV2_IN, *PSendRNIDV2_IN; + +#define SendRNIDV2_IN_SIZE \ + (FIELD_OFFSET(SendRNIDV2_IN, NodeIdDataFormat) + \ + SendRNIDV2_IN_NodeIdDataFormat_SIZE) + +typedef struct _SendRNIDV2_OUT +{ + // + ULONG HBAStatus; +#define SendRNIDV2_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SendRNIDV2_OUT_HBAStatus_ID 5 + + // + ULONG TotalRspBufferSize; +#define SendRNIDV2_OUT_TotalRspBufferSize_SIZE sizeof (ULONG) +#define SendRNIDV2_OUT_TotalRspBufferSize_ID 6 + + // + ULONG ActualRspBufferSize; +#define SendRNIDV2_OUT_ActualRspBufferSize_SIZE sizeof (ULONG) +#define SendRNIDV2_OUT_ActualRspBufferSize_ID 7 + + +#define SendRNIDV2_OUT_RspBuffer_SIZE_HINT 76 + + // + UCHAR RspBuffer[1]; +#define SendRNIDV2_OUT_RspBuffer_ID 8 + +} SendRNIDV2_OUT, *PSendRNIDV2_OUT; + +#define GetFC3MgmtInfo 7 +typedef struct _GetFC3MgmtInfo_OUT +{ + // + ULONG HBAStatus; +#define GetFC3MgmtInfo_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetFC3MgmtInfo_OUT_HBAStatus_ID 1 + + // + HBAFC3MgmtInfo MgmtInfo; +#define GetFC3MgmtInfo_OUT_MgmtInfo_SIZE sizeof (HBAFC3MgmtInfo) +#define GetFC3MgmtInfo_OUT_MgmtInfo_ID 2 + +} GetFC3MgmtInfo_OUT, *PGetFC3MgmtInfo_OUT; + +#define GetFC3MgmtInfo_OUT_SIZE \ + (FIELD_OFFSET(GetFC3MgmtInfo_OUT, MgmtInfo) + \ + GetFC3MgmtInfo_OUT_MgmtInfo_SIZE) + +#define SetFC3MgmtInfo 8 +typedef struct _SetFC3MgmtInfo_IN +{ + // + HBAFC3MgmtInfo MgmtInfo; +#define SetFC3MgmtInfo_IN_MgmtInfo_SIZE sizeof (HBAFC3MgmtInfo) +#define SetFC3MgmtInfo_IN_MgmtInfo_ID 1 + +} SetFC3MgmtInfo_IN, *PSetFC3MgmtInfo_IN; + +#define SetFC3MgmtInfo_IN_SIZE \ + (FIELD_OFFSET(SetFC3MgmtInfo_IN, MgmtInfo) + \ + SetFC3MgmtInfo_IN_MgmtInfo_SIZE) + +typedef struct _SetFC3MgmtInfo_OUT +{ + // + ULONG HBAStatus; +#define SetFC3MgmtInfo_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SetFC3MgmtInfo_OUT_HBAStatus_ID 2 + +} SetFC3MgmtInfo_OUT, *PSetFC3MgmtInfo_OUT; + +#define SetFC3MgmtInfo_OUT_SIZE \ + (FIELD_OFFSET(SetFC3MgmtInfo_OUT, HBAStatus) + \ + SetFC3MgmtInfo_OUT_HBAStatus_SIZE) + +#define SendRPL 9 +typedef struct _SendRPL_IN +{ + // + UCHAR PortWWN[8]; +#define SendRPL_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SendRPL_IN_PortWWN_ID 1 + + // + UCHAR AgentWWN[8]; +#define SendRPL_IN_AgentWWN_SIZE sizeof (UCHAR[8]) +#define SendRPL_IN_AgentWWN_ID 2 + + // + ULONG agent_domain; +#define SendRPL_IN_agent_domain_SIZE sizeof (ULONG) +#define SendRPL_IN_agent_domain_ID 3 + + // + ULONG portIndex; +#define SendRPL_IN_portIndex_SIZE sizeof (ULONG) +#define SendRPL_IN_portIndex_ID 4 + +} SendRPL_IN, *PSendRPL_IN; + +#define SendRPL_IN_SIZE \ + (FIELD_OFFSET(SendRPL_IN, portIndex) + SendRPL_IN_portIndex_SIZE) + +typedef struct _SendRPL_OUT +{ + // + ULONG HBAStatus; +#define SendRPL_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SendRPL_OUT_HBAStatus_ID 5 + + // + ULONG TotalRspBufferSize; +#define SendRPL_OUT_TotalRspBufferSize_SIZE sizeof (ULONG) +#define SendRPL_OUT_TotalRspBufferSize_ID 6 + + // + ULONG ActualRspBufferSize; +#define SendRPL_OUT_ActualRspBufferSize_SIZE sizeof (ULONG) +#define SendRPL_OUT_ActualRspBufferSize_ID 7 + + +#define SendRPL_OUT_RspBuffer_SIZE_HINT 28 // 12+16*n + + // + UCHAR RspBuffer[1]; +#define SendRPL_OUT_RspBuffer_ID 8 + +} SendRPL_OUT, *PSendRPL_OUT; + +#define SendRPS 10 +typedef struct _SendRPS_IN +{ + // + UCHAR PortWWN[8]; +#define SendRPS_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SendRPS_IN_PortWWN_ID 1 + + // + UCHAR AgentWWN[8]; +#define SendRPS_IN_AgentWWN_SIZE sizeof (UCHAR[8]) +#define SendRPS_IN_AgentWWN_ID 2 + + // + UCHAR ObjectWWN[8]; +#define SendRPS_IN_ObjectWWN_SIZE sizeof (UCHAR[8]) +#define SendRPS_IN_ObjectWWN_ID 3 + + // + ULONG AgentDomain; +#define SendRPS_IN_AgentDomain_SIZE sizeof (ULONG) +#define SendRPS_IN_AgentDomain_ID 4 + + // + ULONG ObjectPortNumber; +#define SendRPS_IN_ObjectPortNumber_SIZE sizeof (ULONG) +#define SendRPS_IN_ObjectPortNumber_ID 5 + +} SendRPS_IN, *PSendRPS_IN; + +#define SendRPS_IN_SIZE \ + (FIELD_OFFSET(SendRPS_IN, ObjectPortNumber) + \ + SendRPS_IN_ObjectPortNumber_SIZE) + +typedef struct _SendRPS_OUT +{ + // + ULONG HBAStatus; +#define SendRPS_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SendRPS_OUT_HBAStatus_ID 6 + + // + ULONG TotalRspBufferSize; +#define SendRPS_OUT_TotalRspBufferSize_SIZE sizeof (ULONG) +#define SendRPS_OUT_TotalRspBufferSize_ID 7 + + // + ULONG ActualRspBufferSize; +#define SendRPS_OUT_ActualRspBufferSize_SIZE sizeof (ULONG) +#define SendRPS_OUT_ActualRspBufferSize_ID 8 + +#define SendRPS_OUT_RspBuffer_SIZE_HINT 64 + + // + UCHAR RspBuffer[1]; +#define SendRPS_OUT_RspBuffer_ID 9 + +} SendRPS_OUT, *PSendRPS_OUT; + +#define SendSRL 11 +typedef struct _SendSRL_IN +{ + // + UCHAR PortWWN[8]; +#define SendSRL_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SendSRL_IN_PortWWN_ID 1 + + // + UCHAR WWN[8]; +#define SendSRL_IN_WWN_SIZE sizeof (UCHAR[8]) +#define SendSRL_IN_WWN_ID 2 + + // + ULONG Domain; +#define SendSRL_IN_Domain_SIZE sizeof (ULONG) +#define SendSRL_IN_Domain_ID 3 + +} SendSRL_IN, *PSendSRL_IN; + +#define SendSRL_IN_SIZE \ + (FIELD_OFFSET(SendSRL_IN, Domain) + SendSRL_IN_Domain_SIZE) + +typedef struct _SendSRL_OUT +{ + // + ULONG HBAStatus; +#define SendSRL_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SendSRL_OUT_HBAStatus_ID 4 + + // + ULONG TotalRspBufferSize; +#define SendSRL_OUT_TotalRspBufferSize_SIZE sizeof (ULONG) +#define SendSRL_OUT_TotalRspBufferSize_ID 5 + + // + ULONG ActualRspBufferSize; +#define SendSRL_OUT_ActualRspBufferSize_SIZE sizeof (ULONG) +#define SendSRL_OUT_ActualRspBufferSize_ID 6 + +#define SendSRL_OUT_RspBuffer_SIZE_HINT 8 + + // + UCHAR RspBuffer[1]; +#define SendSRL_OUT_RspBuffer_ID 7 + +} SendSRL_OUT, *PSendSRL_OUT; + +#define SendLIRR 12 +typedef struct _SendLIRR_IN +{ + // + UCHAR SourceWWN[8]; +#define SendLIRR_IN_SourceWWN_SIZE sizeof (UCHAR[8]) +#define SendLIRR_IN_SourceWWN_ID 1 + + // + UCHAR DestWWN[8]; +#define SendLIRR_IN_DestWWN_SIZE sizeof (UCHAR[8]) +#define SendLIRR_IN_DestWWN_ID 2 + + // + UCHAR Function; +#define SendLIRR_IN_Function_SIZE sizeof (UCHAR) +#define SendLIRR_IN_Function_ID 3 + + // + UCHAR Type; +#define SendLIRR_IN_Type_SIZE sizeof (UCHAR) +#define SendLIRR_IN_Type_ID 4 + +} SendLIRR_IN, *PSendLIRR_IN; + +#define SendLIRR_IN_SIZE \ + (FIELD_OFFSET(SendLIRR_IN, Type) + SendLIRR_IN_Type_SIZE) + +typedef struct _SendLIRR_OUT +{ + // + ULONG HBAStatus; +#define SendLIRR_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SendLIRR_OUT_HBAStatus_ID 5 + + // + ULONG TotalRspBufferSize; +#define SendLIRR_OUT_TotalRspBufferSize_SIZE sizeof (ULONG) +#define SendLIRR_OUT_TotalRspBufferSize_ID 6 + + // + ULONG ActualRspBufferSize; +#define SendLIRR_OUT_ActualRspBufferSize_SIZE sizeof (ULONG) +#define SendLIRR_OUT_ActualRspBufferSize_ID 7 + + +#define SendLIRR_OUT_RspBuffer_SIZE_HINT 8 + + // + UCHAR RspBuffer[1]; +#define SendLIRR_OUT_RspBuffer_ID 8 + +} SendLIRR_OUT, *PSendLIRR_OUT; + +#define GetFC4Statistics 13 +typedef struct _GetFC4Statistics_IN +{ + // + UCHAR PortWWN[8]; +#define GetFC4Statistics_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define GetFC4Statistics_IN_PortWWN_ID 1 + + // + UCHAR FC4Type; +#define GetFC4Statistics_IN_FC4Type_SIZE sizeof (UCHAR) +#define GetFC4Statistics_IN_FC4Type_ID 2 + +} GetFC4Statistics_IN, *PGetFC4Statistics_IN; + +#define GetFC4Statistics_IN_SIZE \ + (FIELD_OFFSET(GetFC4Statistics_IN, FC4Type) + \ + GetFC4Statistics_IN_FC4Type_SIZE) + +typedef struct _GetFC4Statistics_OUT +{ + // + ULONG HBAStatus; +#define GetFC4Statistics_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetFC4Statistics_OUT_HBAStatus_ID 3 + + // + MSFC_FC4STATISTICS FC4Statistics; +#define GetFC4Statistics_OUT_FC4Statistics_SIZE sizeof (MSFC_FC4STATISTICS) +#define GetFC4Statistics_OUT_FC4Statistics_ID 4 + +} GetFC4Statistics_OUT, *PGetFC4Statistics_OUT; + +#define GetFC4Statistics_OUT_SIZE \ + (FIELD_OFFSET(GetFC4Statistics_OUT, FC4Statistics) + \ + GetFC4Statistics_OUT_FC4Statistics_SIZE) + +#define GetFCPStatistics 14 +typedef struct _GetFCPStatistics_IN +{ + // + HBAScsiID ScsiId; +#define GetFCPStatistics_IN_ScsiId_SIZE sizeof (HBAScsiID) +#define GetFCPStatistics_IN_ScsiId_ID 1 + +} GetFCPStatistics_IN, *PGetFCPStatistics_IN; + +#define GetFCPStatistics_IN_SIZE \ + (FIELD_OFFSET(GetFCPStatistics_IN, ScsiId) + \ + GetFCPStatistics_IN_ScsiId_SIZE) + +typedef struct _GetFCPStatistics_OUT +{ + // + ULONG HBAStatus; +#define GetFCPStatistics_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetFCPStatistics_OUT_HBAStatus_ID 2 + + // + MSFC_FC4STATISTICS FC4Statistics; +#define GetFCPStatistics_OUT_FC4Statistics_SIZE sizeof (MSFC_FC4STATISTICS) +#define GetFCPStatistics_OUT_FC4Statistics_ID 3 + +} GetFCPStatistics_OUT, *PGetFCPStatistics_OUT; + +#define GetFCPStatistics_OUT_SIZE \ + (FIELD_OFFSET(GetFCPStatistics_OUT, FC4Statistics) + \ + GetFCPStatistics_OUT_FC4Statistics_SIZE) + +#define ScsiInquiry 15 +typedef struct _ScsiInquiry_IN +{ + // + UCHAR Cdb[6]; +#define ScsiInquiry_IN_Cdb_SIZE sizeof (UCHAR[6]) +#define ScsiInquiry_IN_Cdb_ID 1 + + // + UCHAR HbaPortWWN[8]; +#define ScsiInquiry_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define ScsiInquiry_IN_HbaPortWWN_ID 2 + + // + UCHAR DiscoveredPortWWN[8]; +#define ScsiInquiry_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define ScsiInquiry_IN_DiscoveredPortWWN_ID 3 + + // + ULONGLONG FcLun; +#define ScsiInquiry_IN_FcLun_SIZE sizeof (ULONGLONG) +#define ScsiInquiry_IN_FcLun_ID 4 + +} ScsiInquiry_IN, *PScsiInquiry_IN; + +#define ScsiInquiry_IN_SIZE \ + (FIELD_OFFSET(ScsiInquiry_IN, FcLun) + ScsiInquiry_IN_FcLun_SIZE) + +typedef struct _ScsiInquiry_OUT +{ + // + ULONG HBAStatus; +#define ScsiInquiry_OUT_HBAStatus_SIZE sizeof (ULONG) +#define ScsiInquiry_OUT_HBAStatus_ID 5 + + // + ULONG ResponseBufferSize; +#define ScsiInquiry_OUT_ResponseBufferSize_SIZE sizeof (ULONG) +#define ScsiInquiry_OUT_ResponseBufferSize_ID 6 + + // + ULONG SenseBufferSize; +#define ScsiInquiry_OUT_SenseBufferSize_SIZE sizeof (ULONG) +#define ScsiInquiry_OUT_SenseBufferSize_ID 7 + + // + UCHAR ScsiStatus; +#define ScsiInquiry_OUT_ScsiStatus_SIZE sizeof (UCHAR) +#define ScsiInquiry_OUT_ScsiStatus_ID 8 + + +#define ScsiInquiry_OUT_ResponseBuffer_SIZE_HINT 96 + + // + UCHAR ResponseBuffer[1]; +#define ScsiInquiry_OUT_ResponseBuffer_ID 9 + + // +// UCHAR SenseBuffer[1]; +#define ScsiInquiry_OUT_SenseBuffer_ID 10 + +} ScsiInquiry_OUT, *PScsiInquiry_OUT; + +#define ScsiReadCapacity 16 +typedef struct _ScsiReadCapacity_IN +{ + // + UCHAR Cdb[10]; +#define ScsiReadCapacity_IN_Cdb_SIZE sizeof (UCHAR[10]) +#define ScsiReadCapacity_IN_Cdb_ID 1 + + // + UCHAR HbaPortWWN[8]; +#define ScsiReadCapacity_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define ScsiReadCapacity_IN_HbaPortWWN_ID 2 + + // + UCHAR DiscoveredPortWWN[8]; +#define ScsiReadCapacity_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define ScsiReadCapacity_IN_DiscoveredPortWWN_ID 3 + + // + ULONGLONG FcLun; +#define ScsiReadCapacity_IN_FcLun_SIZE sizeof (ULONGLONG) +#define ScsiReadCapacity_IN_FcLun_ID 4 + +} ScsiReadCapacity_IN, *PScsiReadCapacity_IN; + +#define ScsiReadCapacity_IN_SIZE \ + (FIELD_OFFSET(ScsiReadCapacity_IN, FcLun) + \ + ScsiReadCapacity_IN_FcLun_SIZE) + +typedef struct _ScsiReadCapacity_OUT +{ + // + ULONG HBAStatus; +#define ScsiReadCapacity_OUT_HBAStatus_SIZE sizeof (ULONG) +#define ScsiReadCapacity_OUT_HBAStatus_ID 5 + + // + ULONG ResponseBufferSize; +#define ScsiReadCapacity_OUT_ResponseBufferSize_SIZE sizeof (ULONG) +#define ScsiReadCapacity_OUT_ResponseBufferSize_ID 6 + + // + ULONG SenseBufferSize; +#define ScsiReadCapacity_OUT_SenseBufferSize_SIZE sizeof (ULONG) +#define ScsiReadCapacity_OUT_SenseBufferSize_ID 7 + + // + UCHAR ScsiStatus; +#define ScsiReadCapacity_OUT_ScsiStatus_SIZE sizeof (UCHAR) +#define ScsiReadCapacity_OUT_ScsiStatus_ID 8 + + +#define ScsiReadCapacity_OUT_ResponseBuffer_SIZE_HINT 16 + + // + UCHAR ResponseBuffer[1]; +#define ScsiReadCapacity_OUT_ResponseBuffer_ID 9 + + // +// UCHAR SenseBuffer[1]; +#define ScsiReadCapacity_OUT_SenseBuffer_ID 10 + +} ScsiReadCapacity_OUT, *PScsiReadCapacity_OUT; + +#define ScsiReportLuns 17 +typedef struct _ScsiReportLuns_IN +{ + // + UCHAR Cdb[12]; +#define ScsiReportLuns_IN_Cdb_SIZE sizeof (UCHAR[12]) +#define ScsiReportLuns_IN_Cdb_ID 1 + + // + UCHAR HbaPortWWN[8]; +#define ScsiReportLuns_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define ScsiReportLuns_IN_HbaPortWWN_ID 2 + + // + UCHAR DiscoveredPortWWN[8]; +#define ScsiReportLuns_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define ScsiReportLuns_IN_DiscoveredPortWWN_ID 3 + +} ScsiReportLuns_IN, *PScsiReportLuns_IN; + +#define ScsiReportLuns_IN_SIZE \ + (FIELD_OFFSET(ScsiReportLuns_IN, DiscoveredPortWWN) + \ + ScsiReportLuns_IN_DiscoveredPortWWN_SIZE) + +typedef struct _ScsiReportLuns_OUT +{ + // + ULONG HBAStatus; +#define ScsiReportLuns_OUT_HBAStatus_SIZE sizeof (ULONG) +#define ScsiReportLuns_OUT_HBAStatus_ID 4 + + // + ULONG ResponseBufferSize; +#define ScsiReportLuns_OUT_ResponseBufferSize_SIZE sizeof (ULONG) +#define ScsiReportLuns_OUT_ResponseBufferSize_ID 5 + + // + ULONG SenseBufferSize; +#define ScsiReportLuns_OUT_SenseBufferSize_SIZE sizeof (ULONG) +#define ScsiReportLuns_OUT_SenseBufferSize_ID 6 + + // + UCHAR ScsiStatus; +#define ScsiReportLuns_OUT_ScsiStatus_SIZE sizeof (UCHAR) +#define ScsiReportLuns_OUT_ScsiStatus_ID 7 + + +#define ScsiReportLuns_OUT_ResponseBuffer_SIZE_HINT 16 // 8+8*number_of_luns + + // + UCHAR ResponseBuffer[1]; +#define ScsiReportLuns_OUT_ResponseBuffer_ID 8 + + // +// UCHAR SenseBuffer[1]; +#define ScsiReportLuns_OUT_SenseBuffer_ID 9 + +} ScsiReportLuns_OUT, *PScsiReportLuns_OUT; + +#define GetEventBuffer 18 +typedef struct _GetEventBuffer_OUT +{ + // + ULONG HBAStatus; +#define GetEventBuffer_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetEventBuffer_OUT_HBAStatus_ID 1 + + // + ULONG EventCount; +#define GetEventBuffer_OUT_EventCount_SIZE sizeof (ULONG) +#define GetEventBuffer_OUT_EventCount_ID 2 + + // + MSFC_EventBuffer Events[1]; +#define GetEventBuffer_OUT_Events_ID 3 + +} GetEventBuffer_OUT, *PGetEventBuffer_OUT; + +#define SendRLS 19 +typedef struct _SendRLS_IN +{ + // + UCHAR PortWWN[8]; +#define SendRLS_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SendRLS_IN_PortWWN_ID 1 + + // + UCHAR DestWWN[8]; +#define SendRLS_IN_DestWWN_SIZE sizeof (UCHAR[8]) +#define SendRLS_IN_DestWWN_ID 2 + +} SendRLS_IN, *PSendRLS_IN; + +#define SendRLS_IN_SIZE \ + (FIELD_OFFSET(SendRLS_IN, DestWWN) + SendRLS_IN_DestWWN_SIZE) + +typedef struct _SendRLS_OUT +{ + // + ULONG HBAStatus; +#define SendRLS_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SendRLS_OUT_HBAStatus_ID 3 + + // + ULONG TotalRspBufferSize; +#define SendRLS_OUT_TotalRspBufferSize_SIZE sizeof (ULONG) +#define SendRLS_OUT_TotalRspBufferSize_ID 4 + + // + ULONG ActualRspBufferSize; +#define SendRLS_OUT_ActualRspBufferSize_SIZE sizeof (ULONG) +#define SendRLS_OUT_ActualRspBufferSize_ID 5 + + +#define SendRLS_OUT_RspBuffer_SIZE_HINT 28 + + // + UCHAR RspBuffer[1]; +#define SendRLS_OUT_RspBuffer_ID 6 + +} SendRLS_OUT, *PSendRLS_OUT; + + +// HBAFCPID - HBAFCPID +#define HBAFCPIDGuid \ + { 0xff02bc96, 0x7fb0, 0x4bac, \ + { 0x8f, 0x97, 0xc7, 0x1e, 0x49, 0x5f, 0xa6, 0x98 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(HBAFCPID_GUID, \ + 0xff02bc96, 0x7fb0, 0x4bac, 0x8f, 0x97, 0xc7, 0x1e, \ + 0x49, 0x5f, 0xa6, 0x98); +#endif + + +typedef struct _HBAFCPID +{ + // + ULONG Fcid; +#define HBAFCPID_Fcid_SIZE sizeof (ULONG) +#define HBAFCPID_Fcid_ID 1 + + // + UCHAR NodeWWN[8]; +#define HBAFCPID_NodeWWN_SIZE sizeof (UCHAR[8]) +#define HBAFCPID_NodeWWN_ID 2 + + // + UCHAR PortWWN[8]; +#define HBAFCPID_PortWWN_SIZE sizeof (UCHAR[8]) +#define HBAFCPID_PortWWN_ID 3 + + // + ULONGLONG FcpLun; +#define HBAFCPID_FcpLun_SIZE sizeof (ULONGLONG) +#define HBAFCPID_FcpLun_ID 4 + +} HBAFCPID, *PHBAFCPID; + +#define HBAFCPID_SIZE (FIELD_OFFSET(HBAFCPID, FcpLun) + HBAFCPID_FcpLun_SIZE) + +// HBAFCPScsiEntry - HBAFCPScsiEntry +#define HBAFCPScsiEntryGuid \ + { 0x77ca1248, 0x1505, 0x4221, \ + { 0x8e, 0xb6, 0xbb, 0xb6, 0xec, 0x77, 0x1a, 0x87 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(HBAFCPScsiEntry_GUID, \ + 0x77ca1248, 0x1505, 0x4221, 0x8e, 0xb6, 0xbb, 0xb6, \ + 0xec, 0x77, 0x1a, 0x87); +#endif + + +typedef struct _HBAFCPScsiEntry +{ + // + HBAFCPID FCPId; +#define HBAFCPScsiEntry_FCPId_SIZE sizeof (HBAFCPID) +#define HBAFCPScsiEntry_FCPId_ID 1 + + // + UCHAR Luid[256]; +#define HBAFCPScsiEntry_Luid_SIZE sizeof (UCHAR[256]) +#define HBAFCPScsiEntry_Luid_ID 2 + + // + HBAScsiID ScsiId; +#define HBAFCPScsiEntry_ScsiId_SIZE sizeof (HBAScsiID) +#define HBAFCPScsiEntry_ScsiId_ID 3 + +} HBAFCPScsiEntry, *PHBAFCPScsiEntry; + +#define HBAFCPScsiEntry_SIZE \ + (FIELD_OFFSET(HBAFCPScsiEntry, ScsiId) + HBAFCPScsiEntry_ScsiId_SIZE) + +// HBAFCPBindingEntry - HBAFCPBindingEntry +#define HBAFCPBindingEntryGuid \ + { 0xfceff8b7, 0x9d6b, 0x4115, \ + { 0x84, 0x22, 0x05, 0x99, 0x24, 0x51, 0xa6, 0x29 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(HBAFCPBindingEntry_GUID, \ + 0xfceff8b7, 0x9d6b, 0x4115, 0x84, 0x22, 0x05, 0x99, \ + 0x24, 0x51, 0xa6, 0x29); +#endif + + +typedef struct _HBAFCPBindingEntry +{ + // + ULONG Type; +#define HBAFCPBindingEntry_Type_SIZE sizeof (ULONG) +#define HBAFCPBindingEntry_Type_ID 1 + + // + HBAFCPID FCPId; +#define HBAFCPBindingEntry_FCPId_SIZE sizeof (HBAFCPID) +#define HBAFCPBindingEntry_FCPId_ID 2 + + // + HBAScsiID ScsiId; +#define HBAFCPBindingEntry_ScsiId_SIZE sizeof (HBAScsiID) +#define HBAFCPBindingEntry_ScsiId_ID 3 + +} HBAFCPBindingEntry, *PHBAFCPBindingEntry; + +#define HBAFCPBindingEntry_SIZE \ + (FIELD_OFFSET(HBAFCPBindingEntry, ScsiId) + \ + HBAFCPBindingEntry_ScsiId_SIZE) + +// HBAFCPBindingEntry2 - HBAFCPBindingEntry2 +#define HBAFCPBindingEntry2Guid \ + { 0x3a1e7679, 0x4b1f, 0x4f31, \ + { 0xa8, 0xae, 0xfe, 0x92, 0x78, 0x73, 0x09, 0x24 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(HBAFCPBindingEntry2_GUID, \ + 0x3a1e7679, 0x4b1f, 0x4f31, 0xa8, 0xae, 0xfe, 0x92, \ + 0x78, 0x73, 0x09, 0x24); +#endif + + +typedef struct _HBAFCPBindingEntry2 +{ + // + ULONG Type; +#define HBAFCPBindingEntry2_Type_SIZE sizeof (ULONG) +#define HBAFCPBindingEntry2_Type_ID 1 + + // + HBAFCPID FCPId; +#define HBAFCPBindingEntry2_FCPId_SIZE sizeof (HBAFCPID) +#define HBAFCPBindingEntry2_FCPId_ID 2 + + // + UCHAR Luid[256]; +#define HBAFCPBindingEntry2_Luid_SIZE sizeof (UCHAR[256]) +#define HBAFCPBindingEntry2_Luid_ID 3 + + // + HBAScsiID ScsiId; +#define HBAFCPBindingEntry2_ScsiId_SIZE sizeof (HBAScsiID) +#define HBAFCPBindingEntry2_ScsiId_ID 4 + +} HBAFCPBindingEntry2, *PHBAFCPBindingEntry2; + +#define HBAFCPBindingEntry2_SIZE \ + (FIELD_OFFSET(HBAFCPBindingEntry2, ScsiId) + \ + HBAFCPBindingEntry2_ScsiId_SIZE) + +// MSFC_HBAFCPInfo - MSFC_HBAFCPInfo +#define MSFC_HBAFCPInfoGuid \ + { 0x7a1fc391, 0x5b23, 0x4c19, \ + { 0xb0, 0xeb, 0xb1, 0xae, 0xf5, 0x90, 0x50, 0xc3 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_HBAFCPInfo_GUID, \ + 0x7a1fc391, 0x5b23, 0x4c19, 0xb0, 0xeb, 0xb1, 0xae, \ + 0xf5, 0x90, 0x50, 0xc3); +#endif + +// +// Method id definitions for MSFC_HBAFCPInfo +#define GetFcpTargetMapping 1 +typedef struct _GetFcpTargetMapping_IN +{ + // + UCHAR HbaPortWWN[8]; +#define GetFcpTargetMapping_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define GetFcpTargetMapping_IN_HbaPortWWN_ID 1 + + // + ULONG InEntryCount; +#define GetFcpTargetMapping_IN_InEntryCount_SIZE sizeof (ULONG) +#define GetFcpTargetMapping_IN_InEntryCount_ID 2 + +} GetFcpTargetMapping_IN, *PGetFcpTargetMapping_IN; + +#define GetFcpTargetMapping_IN_SIZE \ + (FIELD_OFFSET(GetFcpTargetMapping_IN, InEntryCount) + \ + GetFcpTargetMapping_IN_InEntryCount_SIZE) + +typedef struct _GetFcpTargetMapping_OUT +{ + // + ULONG HBAStatus; +#define GetFcpTargetMapping_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetFcpTargetMapping_OUT_HBAStatus_ID 3 + + // + ULONG TotalEntryCount; +#define GetFcpTargetMapping_OUT_TotalEntryCount_SIZE sizeof (ULONG) +#define GetFcpTargetMapping_OUT_TotalEntryCount_ID 4 + + // + ULONG OutEntryCount; +#define GetFcpTargetMapping_OUT_OutEntryCount_SIZE sizeof (ULONG) +#define GetFcpTargetMapping_OUT_OutEntryCount_ID 5 + + // + HBAFCPScsiEntry Entry[1]; +#define GetFcpTargetMapping_OUT_Entry_ID 6 + +} GetFcpTargetMapping_OUT, *PGetFcpTargetMapping_OUT; + +#define GetFcpPersistentBinding 2 +typedef struct _GetFcpPersistentBinding_IN +{ + // + ULONG InEntryCount; +#define GetFcpPersistentBinding_IN_InEntryCount_SIZE sizeof (ULONG) +#define GetFcpPersistentBinding_IN_InEntryCount_ID 1 + +} GetFcpPersistentBinding_IN, *PGetFcpPersistentBinding_IN; + +#define GetFcpPersistentBinding_IN_SIZE \ + (FIELD_OFFSET(GetFcpPersistentBinding_IN, InEntryCount) + \ + GetFcpPersistentBinding_IN_InEntryCount_SIZE) + +typedef struct _GetFcpPersistentBinding_OUT +{ + // + ULONG HBAStatus; +#define GetFcpPersistentBinding_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetFcpPersistentBinding_OUT_HBAStatus_ID 2 + + // + ULONG TotalEntryCount; +#define GetFcpPersistentBinding_OUT_TotalEntryCount_SIZE sizeof (ULONG) +#define GetFcpPersistentBinding_OUT_TotalEntryCount_ID 3 + + // + ULONG OutEntryCount; +#define GetFcpPersistentBinding_OUT_OutEntryCount_SIZE sizeof (ULONG) +#define GetFcpPersistentBinding_OUT_OutEntryCount_ID 4 + + // + HBAFCPBindingEntry Entry[1]; +#define GetFcpPersistentBinding_OUT_Entry_ID 5 + +} GetFcpPersistentBinding_OUT, *PGetFcpPersistentBinding_OUT; + +#define GetBindingCapability 3 +typedef struct _GetBindingCapability_IN +{ + // + UCHAR PortWWN[8]; +#define GetBindingCapability_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define GetBindingCapability_IN_PortWWN_ID 1 + +} GetBindingCapability_IN, *PGetBindingCapability_IN; + +#define GetBindingCapability_IN_SIZE \ + (FIELD_OFFSET(GetBindingCapability_IN, PortWWN) + \ + GetBindingCapability_IN_PortWWN_SIZE) + +typedef struct _GetBindingCapability_OUT +{ + // + ULONG HBAStatus; +#define GetBindingCapability_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetBindingCapability_OUT_HBAStatus_ID 2 + + // + ULONG BindType; +#define GetBindingCapability_OUT_BindType_SIZE sizeof (ULONG) +#define GetBindingCapability_OUT_BindType_ID 3 + +} GetBindingCapability_OUT, *PGetBindingCapability_OUT; + +#define GetBindingCapability_OUT_SIZE \ + (FIELD_OFFSET(GetBindingCapability_OUT, BindType) + \ + GetBindingCapability_OUT_BindType_SIZE) + +#define GetBindingSupport 4 +typedef struct _GetBindingSupport_IN +{ + // + UCHAR PortWWN[8]; +#define GetBindingSupport_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define GetBindingSupport_IN_PortWWN_ID 1 + +} GetBindingSupport_IN, *PGetBindingSupport_IN; + +#define GetBindingSupport_IN_SIZE \ + (FIELD_OFFSET(GetBindingSupport_IN, PortWWN) + \ + GetBindingSupport_IN_PortWWN_SIZE) + +typedef struct _GetBindingSupport_OUT +{ + // + ULONG HBAStatus; +#define GetBindingSupport_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetBindingSupport_OUT_HBAStatus_ID 2 + + // + ULONG BindType; +#define GetBindingSupport_OUT_BindType_SIZE sizeof (ULONG) +#define GetBindingSupport_OUT_BindType_ID 3 + +} GetBindingSupport_OUT, *PGetBindingSupport_OUT; + +#define GetBindingSupport_OUT_SIZE \ + (FIELD_OFFSET(GetBindingSupport_OUT, BindType) + \ + GetBindingSupport_OUT_BindType_SIZE) + +#define SetBindingSupport 5 +typedef struct _SetBindingSupport_IN +{ + // + UCHAR PortWWN[8]; +#define SetBindingSupport_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SetBindingSupport_IN_PortWWN_ID 1 + + // + ULONG BindType; +#define SetBindingSupport_IN_BindType_SIZE sizeof (ULONG) +#define SetBindingSupport_IN_BindType_ID 2 + +} SetBindingSupport_IN, *PSetBindingSupport_IN; + +#define SetBindingSupport_IN_SIZE \ + (FIELD_OFFSET(SetBindingSupport_IN, BindType) + \ + SetBindingSupport_IN_BindType_SIZE) + +typedef struct _SetBindingSupport_OUT +{ + // + ULONG HBAStatus; +#define SetBindingSupport_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SetBindingSupport_OUT_HBAStatus_ID 3 + +} SetBindingSupport_OUT, *PSetBindingSupport_OUT; + +#define SetBindingSupport_OUT_SIZE \ + (FIELD_OFFSET(SetBindingSupport_OUT, HBAStatus) + \ + SetBindingSupport_OUT_HBAStatus_SIZE) + +#define GetPersistentBinding2 6 +typedef struct _GetPersistentBinding2_IN +{ + // + UCHAR PortWWN[8]; +#define GetPersistentBinding2_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define GetPersistentBinding2_IN_PortWWN_ID 1 + + // + ULONG InEntryCount; +#define GetPersistentBinding2_IN_InEntryCount_SIZE sizeof (ULONG) +#define GetPersistentBinding2_IN_InEntryCount_ID 2 + +} GetPersistentBinding2_IN, *PGetPersistentBinding2_IN; + +#define GetPersistentBinding2_IN_SIZE \ + (FIELD_OFFSET(GetPersistentBinding2_IN, InEntryCount) + \ + GetPersistentBinding2_IN_InEntryCount_SIZE) + +typedef struct _GetPersistentBinding2_OUT +{ + // + ULONG HBAStatus; +#define GetPersistentBinding2_OUT_HBAStatus_SIZE sizeof (ULONG) +#define GetPersistentBinding2_OUT_HBAStatus_ID 3 + + // + ULONG TotalEntryCount; +#define GetPersistentBinding2_OUT_TotalEntryCount_SIZE sizeof (ULONG) +#define GetPersistentBinding2_OUT_TotalEntryCount_ID 4 + + // + ULONG OutEntryCount; +#define GetPersistentBinding2_OUT_OutEntryCount_SIZE sizeof (ULONG) +#define GetPersistentBinding2_OUT_OutEntryCount_ID 5 + + // + HBAFCPBindingEntry2 Bindings[1]; +#define GetPersistentBinding2_OUT_Bindings_ID 6 + +} GetPersistentBinding2_OUT, *PGetPersistentBinding2_OUT; + +// ********************************************************************* +// +// A call to HBA_SetPersistentBindingV2 will call SetPersistentEntry +// once for each binding entry. +// Each binding entry that SetPersistentEntry accepts will be stored +// in the registry. +// +// Persistent bindings are stored in the registry under: +// +// System\CurrentControlSet\Control\Storage\FC\ +// +// under the REG_BINARY key Bindings is the struct: +// +// typedef struct { +// ULONG Version; +// HBA_FCPBINDING2 Bindings; +// } HBAP_PERSISTENT_BINDINGS, *PHBAP_PERSISTENT_BINDINGS; +// +// This is done so that storport capable drivers may have access to +// this information during boot +// +// ******************************************************************** + +#define HBA_REGISTRY_BINDING_VERSION (1) +#define HBA_REGISTRY_BINDING_RELATIVE_PATH \ + L"System\\CurrentControlSet\\Control\\Storage\\FC" +#define HBA_REGISTRY_BINDING_KEY L"Bindings" + + +#define SetPersistentEntry 7 +typedef struct _SetPersistentEntry_IN +{ + // + UCHAR PortWWN[8]; +#define SetPersistentEntry_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SetPersistentEntry_IN_PortWWN_ID 1 + + // + HBAFCPBindingEntry2 Binding; +#define SetPersistentEntry_IN_Binding_SIZE sizeof (HBAFCPBindingEntry2) +#define SetPersistentEntry_IN_Binding_ID 2 + +} SetPersistentEntry_IN, *PSetPersistentEntry_IN; + +#define SetPersistentEntry_IN_SIZE \ + (FIELD_OFFSET(SetPersistentEntry_IN, Binding) + \ + SetPersistentEntry_IN_Binding_SIZE) + +typedef struct _SetPersistentEntry_OUT +{ + // + ULONG HBAStatus; +#define SetPersistentEntry_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SetPersistentEntry_OUT_HBAStatus_ID 3 + +} SetPersistentEntry_OUT, *PSetPersistentEntry_OUT; + +#define SetPersistentEntry_OUT_SIZE \ + (FIELD_OFFSET(SetPersistentEntry_OUT, HBAStatus) + \ + SetPersistentEntry_OUT_HBAStatus_SIZE) + +#define RemovePersistentEntry 8 +typedef struct _RemovePersistentEntry_IN +{ + // + UCHAR PortWWN[8]; +#define RemovePersistentEntry_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define RemovePersistentEntry_IN_PortWWN_ID 1 + + // + HBAFCPBindingEntry2 Binding; +#define RemovePersistentEntry_IN_Binding_SIZE sizeof (HBAFCPBindingEntry2) +#define RemovePersistentEntry_IN_Binding_ID 2 + +} RemovePersistentEntry_IN, *PRemovePersistentEntry_IN; + +#define RemovePersistentEntry_IN_SIZE \ + (FIELD_OFFSET(RemovePersistentEntry_IN, Binding) + \ + RemovePersistentEntry_IN_Binding_SIZE) + +typedef struct _RemovePersistentEntry_OUT +{ + // + ULONG HBAStatus; +#define RemovePersistentEntry_OUT_HBAStatus_SIZE sizeof (ULONG) +#define RemovePersistentEntry_OUT_HBAStatus_ID 3 + +} RemovePersistentEntry_OUT, *PRemovePersistentEntry_OUT; + +#define RemovePersistentEntry_OUT_SIZE \ + (FIELD_OFFSET(RemovePersistentEntry_OUT, HBAStatus) + \ + RemovePersistentEntry_OUT_HBAStatus_SIZE) + + +// MSFC_AdapterEvent - MSFC_AdapterEvent +#define MSFC_AdapterEventGuid \ + { 0xe9e47403, 0xd1d7, 0x43f8, \ + { 0x8e, 0xe3, 0x53, 0xcd, 0xbf, 0xff, 0x56, 0x46 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_AdapterEvent_GUID, \ + 0xe9e47403, 0xd1d7, 0x43f8, 0x8e, 0xe3, 0x53, 0xcd, \ + 0xbf, 0xff, 0x56, 0x46); +#endif + +typedef struct _MSFC_AdapterEvent +{ + // + ULONG EventType; +#define MSFC_AdapterEvent_EventType_SIZE sizeof (ULONG) +#define MSFC_AdapterEvent_EventType_ID 1 + + // + UCHAR PortWWN[8]; +#define MSFC_AdapterEvent_PortWWN_SIZE sizeof (UCHAR[8]) +#define MSFC_AdapterEvent_PortWWN_ID 2 + +} MSFC_AdapterEvent, *PMSFC_AdapterEvent; + +#define MSFC_AdapterEvent_SIZE \ + (FIELD_OFFSET(MSFC_AdapterEvent, PortWWN) + \ + MSFC_AdapterEvent_PortWWN_SIZE) + +// MSFC_PortEvent - MSFC_PortEvent +#define MSFC_PortEventGuid \ + { 0x095fbe97, 0x3876, 0x48ef, \ + { 0x8a, 0x04, 0x1c, 0x55, 0x93, 0x5d, 0x0d, 0xf5 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_PortEvent_GUID, \ + 0x095fbe97, 0x3876, 0x48ef, 0x8a, 0x04, 0x1c, 0x55, \ + 0x93, 0x5d, 0x0d, 0xf5); +#endif + +typedef struct _MSFC_PortEvent +{ + // + ULONG EventType; +#define MSFC_PortEvent_EventType_SIZE sizeof (ULONG) +#define MSFC_PortEvent_EventType_ID 1 + + // + ULONG FabricPortId; +#define MSFC_PortEvent_FabricPortId_SIZE sizeof (ULONG) +#define MSFC_PortEvent_FabricPortId_ID 2 + + // + UCHAR PortWWN[8]; +#define MSFC_PortEvent_PortWWN_SIZE sizeof (UCHAR[8]) +#define MSFC_PortEvent_PortWWN_ID 3 + +} MSFC_PortEvent, *PMSFC_PortEvent; + +#define MSFC_PortEvent_SIZE \ + (FIELD_OFFSET(MSFC_PortEvent, PortWWN) + MSFC_PortEvent_PortWWN_SIZE) + +// MSFC_TargetEvent - MSFC_TargetEvent +#define MSFC_TargetEventGuid \ + { 0xcfa6ef26, 0x8675, 0x4e27, \ + { 0x9a, 0x0b, 0xb4, 0xa8, 0x60, 0xdd, 0xd0, 0xf3 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_TargetEvent_GUID, \ + 0xcfa6ef26, 0x8675, 0x4e27, 0x9a, 0x0b, 0xb4, 0xa8, \ + 0x60, 0xdd, 0xd0, 0xf3); +#endif + +typedef struct _MSFC_TargetEvent +{ + // + ULONG EventType; +#define MSFC_TargetEvent_EventType_SIZE sizeof (ULONG) +#define MSFC_TargetEvent_EventType_ID 1 + + // + UCHAR PortWWN[8]; +#define MSFC_TargetEvent_PortWWN_SIZE sizeof (UCHAR[8]) +#define MSFC_TargetEvent_PortWWN_ID 2 + + // + UCHAR DiscoveredPortWWN[8]; +#define MSFC_TargetEvent_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define MSFC_TargetEvent_DiscoveredPortWWN_ID 3 + +} MSFC_TargetEvent, *PMSFC_TargetEvent; + +#define MSFC_TargetEvent_SIZE \ + (FIELD_OFFSET(MSFC_TargetEvent, DiscoveredPortWWN) + \ + MSFC_TargetEvent_DiscoveredPortWWN_SIZE) + +// MSFC_EventControl - MSFC_EventControl +#define MSFC_EventControlGuid \ + { 0xa251ccb3, 0x5ab0, 0x411b, \ + { 0x87, 0x71, 0x54, 0x30, 0xef, 0x53, 0xa2, 0x6c } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_EventControl_GUID, \ + 0xa251ccb3, 0x5ab0, 0x411b, 0x87, 0x71, 0x54, 0x30, \ + 0xef, 0x53, 0xa2, 0x6c); +#endif + +// +// Method id definitions for MSFC_EventControl +#define AddTarget 10 +typedef struct _AddTarget_IN +{ + // + UCHAR HbaPortWWN[8]; +#define AddTarget_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define AddTarget_IN_HbaPortWWN_ID 1 + + // + UCHAR DiscoveredPortWWN[8]; +#define AddTarget_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define AddTarget_IN_DiscoveredPortWWN_ID 2 + + // + ULONG AllTargets; +#define AddTarget_IN_AllTargets_SIZE sizeof (ULONG) +#define AddTarget_IN_AllTargets_ID 3 + +} AddTarget_IN, *PAddTarget_IN; + +#define AddTarget_IN_SIZE \ + (FIELD_OFFSET(AddTarget_IN, AllTargets) + AddTarget_IN_AllTargets_SIZE) + +typedef struct _AddTarget_OUT +{ + // + ULONG HBAStatus; +#define AddTarget_OUT_HBAStatus_SIZE sizeof (ULONG) +#define AddTarget_OUT_HBAStatus_ID 4 + +} AddTarget_OUT, *PAddTarget_OUT; + +#define AddTarget_OUT_SIZE \ + (FIELD_OFFSET(AddTarget_OUT, HBAStatus) + AddTarget_OUT_HBAStatus_SIZE) + +#define RemoveTarget 11 +typedef struct _RemoveTarget_IN +{ + // + UCHAR HbaPortWWN[8]; +#define RemoveTarget_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define RemoveTarget_IN_HbaPortWWN_ID 1 + + // + UCHAR DiscoveredPortWWN[8]; +#define RemoveTarget_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define RemoveTarget_IN_DiscoveredPortWWN_ID 2 + + // + ULONG AllTargets; +#define RemoveTarget_IN_AllTargets_SIZE sizeof (ULONG) +#define RemoveTarget_IN_AllTargets_ID 3 + +} RemoveTarget_IN, *PRemoveTarget_IN; + +#define RemoveTarget_IN_SIZE \ + (FIELD_OFFSET(RemoveTarget_IN, AllTargets) + \ + RemoveTarget_IN_AllTargets_SIZE) + +typedef struct _RemoveTarget_OUT +{ + // + ULONG HBAStatus; +#define RemoveTarget_OUT_HBAStatus_SIZE sizeof (ULONG) +#define RemoveTarget_OUT_HBAStatus_ID 4 + +} RemoveTarget_OUT, *PRemoveTarget_OUT; + +#define RemoveTarget_OUT_SIZE \ + (FIELD_OFFSET(RemoveTarget_OUT, HBAStatus) + \ + RemoveTarget_OUT_HBAStatus_SIZE) + +#define AddPort 20 +typedef struct _AddPort_IN +{ + // + UCHAR PortWWN[8]; +#define AddPort_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define AddPort_IN_PortWWN_ID 1 + +} AddPort_IN, *PAddPort_IN; + +#define AddPort_IN_SIZE \ + (FIELD_OFFSET(AddPort_IN, PortWWN) + AddPort_IN_PortWWN_SIZE) + +typedef struct _AddPort_OUT +{ + // + ULONG HBAStatus; +#define AddPort_OUT_HBAStatus_SIZE sizeof (ULONG) +#define AddPort_OUT_HBAStatus_ID 2 + +} AddPort_OUT, *PAddPort_OUT; + +#define AddPort_OUT_SIZE \ + (FIELD_OFFSET(AddPort_OUT, HBAStatus) + AddPort_OUT_HBAStatus_SIZE) + +#define RemovePort 21 +typedef struct _RemovePort_IN +{ + // + UCHAR PortWWN[8]; +#define RemovePort_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define RemovePort_IN_PortWWN_ID 1 + +} RemovePort_IN, *PRemovePort_IN; + +#define RemovePort_IN_SIZE \ + (FIELD_OFFSET(RemovePort_IN, PortWWN) + RemovePort_IN_PortWWN_SIZE) + +typedef struct _RemovePort_OUT +{ + // + ULONG HBAStatus; +#define RemovePort_OUT_HBAStatus_SIZE sizeof (ULONG) +#define RemovePort_OUT_HBAStatus_ID 2 + +} RemovePort_OUT, *PRemovePort_OUT; + +#define RemovePort_OUT_SIZE \ + (FIELD_OFFSET(RemovePort_OUT, HBAStatus) + \ + RemovePort_OUT_HBAStatus_SIZE) + +#define AddLink 30 +typedef struct _AddLink_OUT +{ + // + ULONG HBAStatus; +#define AddLink_OUT_HBAStatus_SIZE sizeof (ULONG) +#define AddLink_OUT_HBAStatus_ID 1 + +} AddLink_OUT, *PAddLink_OUT; + +#define AddLink_OUT_SIZE \ + (FIELD_OFFSET(AddLink_OUT, HBAStatus) + AddLink_OUT_HBAStatus_SIZE) + +#define RemoveLink 31 +typedef struct _RemoveLink_OUT +{ + // + ULONG HBAStatus; +#define RemoveLink_OUT_HBAStatus_SIZE sizeof (ULONG) +#define RemoveLink_OUT_HBAStatus_ID 1 + +} RemoveLink_OUT, *PRemoveLink_OUT; + +#define RemoveLink_OUT_SIZE \ + (FIELD_OFFSET(RemoveLink_OUT, HBAStatus) + \ + RemoveLink_OUT_HBAStatus_SIZE) + + +// MS_SM_AdapterInformationQuery - MS_SM_AdapterInformationQuery + + +#endif // MSFC_HBA_API + +#ifdef MS_SM_HBA_API + +#define MS_SM_AdapterInformationQueryGuid \ + { 0xbdc67efa, 0xe5e7, 0x4777, \ + { 0xb1, 0x3c, 0x62, 0x14, 0x59, 0x65, 0x70, 0x99 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_AdapterInformationQuery_GUID, \ + 0xbdc67efa, 0xe5e7, 0x4777, 0xb1, 0x3c, 0x62, 0x14, \ + 0x59, 0x65, 0x70, 0x99); +#endif + + +typedef struct _MS_SM_AdapterInformationQuery +{ + // + ULONGLONG UniqueAdapterId; +#define MS_SM_AdapterInformationQuery_UniqueAdapterId_SIZE sizeof (ULONGLONG) +#define MS_SM_AdapterInformationQuery_UniqueAdapterId_ID 1 + + // + ULONG HBAStatus; +#define MS_SM_AdapterInformationQuery_HBAStatus_SIZE sizeof (ULONG) +#define MS_SM_AdapterInformationQuery_HBAStatus_ID 2 + + // + ULONG NumberOfPorts; +#define MS_SM_AdapterInformationQuery_NumberOfPorts_SIZE sizeof (ULONG) +#define MS_SM_AdapterInformationQuery_NumberOfPorts_ID 3 + + // + ULONG VendorSpecificID; +#define MS_SM_AdapterInformationQuery_VendorSpecificID_SIZE sizeof (ULONG) +#define MS_SM_AdapterInformationQuery_VendorSpecificID_ID 4 + + + + // ****************************************************************** + // + // The string type is variable length (up to MaxLen). + // Each string starts with a ushort that holds the strings length + // (in bytes) followed by the WCHARs that make up the string. + // + // ****************************************************************** + + + // + WCHAR Manufacturer[64 + 1]; +#define MS_SM_AdapterInformationQuery_Manufacturer_ID 5 + + // + WCHAR SerialNumber[64 + 1]; +#define MS_SM_AdapterInformationQuery_SerialNumber_ID 6 + + // + WCHAR Model[256 + 1]; +#define MS_SM_AdapterInformationQuery_Model_ID 7 + + // + WCHAR ModelDescription[256 + 1]; +#define MS_SM_AdapterInformationQuery_ModelDescription_ID 8 + + // + WCHAR HardwareVersion[256 + 1]; +#define MS_SM_AdapterInformationQuery_HardwareVersion_ID 9 + + // + WCHAR DriverVersion[256 + 1]; +#define MS_SM_AdapterInformationQuery_DriverVersion_ID 10 + + // + WCHAR OptionROMVersion[256 + 1]; +#define MS_SM_AdapterInformationQuery_OptionROMVersion_ID 11 + + // + WCHAR FirmwareVersion[256 + 1]; +#define MS_SM_AdapterInformationQuery_FirmwareVersion_ID 12 + + // + WCHAR DriverName[256 + 1]; +#define MS_SM_AdapterInformationQuery_DriverName_ID 13 + + // + WCHAR HBASymbolicName[256 + 1]; +#define MS_SM_AdapterInformationQuery_HBASymbolicName_ID 14 + + // + WCHAR RedundantOptionROMVersion[256 + 1]; +#define MS_SM_AdapterInformationQuery_RedundantOptionROMVersion_ID 15 + + // + WCHAR RedundantFirmwareVersion[256 + 1]; +#define MS_SM_AdapterInformationQuery_RedundantFirmwareVersion_ID 16 + + // + WCHAR MfgDomain[256 + 1]; +#define MS_SM_AdapterInformationQuery_MfgDomain_ID 17 + +} MS_SM_AdapterInformationQuery, *PMS_SM_AdapterInformationQuery; + +// MS_SMHBA_FC_Port - MS_SMHBA_FC_Port +#define MS_SMHBA_FC_PortGuid \ + { 0x96b827a7, 0x2b4a, 0x49c8, \ + { 0x90, 0x97, 0x07, 0x82, 0x00, 0xc5, 0xa5, 0xcd } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_FC_Port_GUID, \ + 0x96b827a7, 0x2b4a, 0x49c8, 0x90, 0x97, 0x07, 0x82, \ + 0x00, 0xc5, 0xa5, 0xcd); +#endif + + +typedef struct _MS_SMHBA_FC_Port +{ + // + UCHAR NodeWWN[8]; +#define MS_SMHBA_FC_Port_NodeWWN_SIZE sizeof (UCHAR[8]) +#define MS_SMHBA_FC_Port_NodeWWN_ID 1 + + // + UCHAR PortWWN[8]; +#define MS_SMHBA_FC_Port_PortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SMHBA_FC_Port_PortWWN_ID 2 + + // + ULONG FcId; +#define MS_SMHBA_FC_Port_FcId_SIZE sizeof (ULONG) +#define MS_SMHBA_FC_Port_FcId_ID 3 + + // + ULONG PortSupportedClassofService; +#define MS_SMHBA_FC_Port_PortSupportedClassofService_SIZE sizeof (ULONG) +#define MS_SMHBA_FC_Port_PortSupportedClassofService_ID 4 + + // + UCHAR PortSupportedFc4Types[32]; +#define MS_SMHBA_FC_Port_PortSupportedFc4Types_SIZE sizeof (UCHAR[32]) +#define MS_SMHBA_FC_Port_PortSupportedFc4Types_ID 5 + + // + UCHAR PortActiveFc4Types[32]; +#define MS_SMHBA_FC_Port_PortActiveFc4Types_SIZE sizeof (UCHAR[32]) +#define MS_SMHBA_FC_Port_PortActiveFc4Types_ID 6 + + // + UCHAR FabricName[8]; +#define MS_SMHBA_FC_Port_FabricName_SIZE sizeof (UCHAR[8]) +#define MS_SMHBA_FC_Port_FabricName_ID 7 + + // + ULONG NumberofDiscoveredPorts; +#define MS_SMHBA_FC_Port_NumberofDiscoveredPorts_SIZE sizeof (ULONG) +#define MS_SMHBA_FC_Port_NumberofDiscoveredPorts_ID 8 + + // + UCHAR NumberofPhys; +#define MS_SMHBA_FC_Port_NumberofPhys_SIZE sizeof (UCHAR) +#define MS_SMHBA_FC_Port_NumberofPhys_ID 9 + + // + WCHAR PortSymbolicName[256 + 1]; +#define MS_SMHBA_FC_Port_PortSymbolicName_ID 10 + +} MS_SMHBA_FC_Port, *PMS_SMHBA_FC_Port; + +// MS_SMHBA_SAS_Port - MS_SMHBA_SAS_Port +#define MS_SMHBA_SAS_PortGuid \ + { 0xb914e34f, 0x7b80, 0x46b0, \ + { 0x80, 0x34, 0x6d, 0x9b, 0x68, 0x9e, 0x1d, 0xdd } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_SAS_Port_GUID, \ + 0xb914e34f, 0x7b80, 0x46b0, 0x80, 0x34, 0x6d, 0x9b, \ + 0x68, 0x9e, 0x1d, 0xdd); +#endif + +typedef struct _MS_SMHBA_SAS_Port +{ + // + ULONG PortProtocol; +#define MS_SMHBA_SAS_Port_PortProtocol_SIZE sizeof (ULONG) +#define MS_SMHBA_SAS_Port_PortProtocol_ID 1 + + // + UCHAR LocalSASAddress[8]; +#define MS_SMHBA_SAS_Port_LocalSASAddress_SIZE sizeof (UCHAR[8]) +#define MS_SMHBA_SAS_Port_LocalSASAddress_ID 2 + + // + UCHAR AttachedSASAddress[8]; +#define MS_SMHBA_SAS_Port_AttachedSASAddress_SIZE sizeof (UCHAR[8]) +#define MS_SMHBA_SAS_Port_AttachedSASAddress_ID 3 + + // + ULONG NumberofDiscoveredPorts; +#define MS_SMHBA_SAS_Port_NumberofDiscoveredPorts_SIZE sizeof (ULONG) +#define MS_SMHBA_SAS_Port_NumberofDiscoveredPorts_ID 4 + + // + ULONG NumberofPhys; +#define MS_SMHBA_SAS_Port_NumberofPhys_SIZE sizeof (ULONG) +#define MS_SMHBA_SAS_Port_NumberofPhys_ID 5 + +} MS_SMHBA_SAS_Port, *PMS_SMHBA_SAS_Port; + +#define MS_SMHBA_SAS_Port_SIZE \ + (FIELD_OFFSET(MS_SMHBA_SAS_Port, NumberofPhys) + \ + MS_SMHBA_SAS_Port_NumberofPhys_SIZE) + +// MS_SMHBA_PORTATTRIBUTES - MS_SMHBA_PORTATTRIBUTES +#define MS_SMHBA_PORTATTRIBUTESGuid \ + { 0x50a97b2d, 0x99ad, 0x4cf9, \ + { 0x84, 0x37, 0xb4, 0xea, 0x0c, 0x07, 0xbe, 0x4c } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_PORTATTRIBUTES_GUID, \ + 0x50a97b2d, 0x99ad, 0x4cf9, 0x84, 0x37, 0xb4, 0xea, \ + 0x0c, 0x07, 0xbe, 0x4c); +#endif + +typedef struct _MS_SMHBA_PORTATTRIBUTES +{ + // + ULONG PortType; +#define MS_SMHBA_PORTATTRIBUTES_PortType_SIZE sizeof (ULONG) +#define MS_SMHBA_PORTATTRIBUTES_PortType_ID 1 + + // + ULONG PortState; +#define MS_SMHBA_PORTATTRIBUTES_PortState_SIZE sizeof (ULONG) +#define MS_SMHBA_PORTATTRIBUTES_PortState_ID 2 + + // + ULONG PortSpecificAttributesSize; +#define MS_SMHBA_PORTATTRIBUTES_PortSpecificAttributesSize_SIZE sizeof (ULONG) +#define MS_SMHBA_PORTATTRIBUTES_PortSpecificAttributesSize_ID 3 + + // + WCHAR OSDeviceName[256 + 1]; +#define MS_SMHBA_PORTATTRIBUTES_OSDeviceName_ID 4 + + // + ULONGLONG Reserved; +#define MS_SMHBA_PORTATTRIBUTES_Reserved_SIZE sizeof (ULONGLONG) +#define MS_SMHBA_PORTATTRIBUTES_Reserved_ID 5 + + // + UCHAR PortSpecificAttributes[1]; +#define MS_SMHBA_PORTATTRIBUTES_PortSpecificAttributes_ID 6 + +} MS_SMHBA_PORTATTRIBUTES, *PMS_SMHBA_PORTATTRIBUTES; + +// MS_SMHBA_PROTOCOLSTATISTICS - MS_SMHBA_PROTOCOLSTATISTICS +#define MS_SMHBA_PROTOCOLSTATISTICSGuid \ + { 0xb557bd86, 0x4128, 0x4d5c, \ + { 0xb6, 0xe6, 0xb6, 0x5f, 0x9b, 0xd6, 0x87, 0x22 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_PROTOCOLSTATISTICS_GUID, \ + 0xb557bd86, 0x4128, 0x4d5c, 0xb6, 0xe6, 0xb6, 0x5f, \ + 0x9b, 0xd6, 0x87, 0x22); +#endif + + +typedef struct _MS_SMHBA_PROTOCOLSTATISTICS +{ + // + LONGLONG SecondsSinceLastReset; +#define MS_SMHBA_PROTOCOLSTATISTICS_SecondsSinceLastReset_SIZE \ + sizeof (LONGLONG) +#define MS_SMHBA_PROTOCOLSTATISTICS_SecondsSinceLastReset_ID 1 + + // + LONGLONG InputRequests; +#define MS_SMHBA_PROTOCOLSTATISTICS_InputRequests_SIZE sizeof (LONGLONG) +#define MS_SMHBA_PROTOCOLSTATISTICS_InputRequests_ID 2 + + // + LONGLONG OutputRequests; +#define MS_SMHBA_PROTOCOLSTATISTICS_OutputRequests_SIZE sizeof (LONGLONG) +#define MS_SMHBA_PROTOCOLSTATISTICS_OutputRequests_ID 3 + + // + LONGLONG ControlRequests; +#define MS_SMHBA_PROTOCOLSTATISTICS_ControlRequests_SIZE sizeof (LONGLONG) +#define MS_SMHBA_PROTOCOLSTATISTICS_ControlRequests_ID 4 + + // + LONGLONG InputMegabytes; +#define MS_SMHBA_PROTOCOLSTATISTICS_InputMegabytes_SIZE sizeof (LONGLONG) +#define MS_SMHBA_PROTOCOLSTATISTICS_InputMegabytes_ID 5 + + // + LONGLONG OutputMegabytes; +#define MS_SMHBA_PROTOCOLSTATISTICS_OutputMegabytes_SIZE sizeof (LONGLONG) +#define MS_SMHBA_PROTOCOLSTATISTICS_OutputMegabytes_ID 6 + +} MS_SMHBA_PROTOCOLSTATISTICS, *PMS_SMHBA_PROTOCOLSTATISTICS; + +#define MS_SMHBA_PROTOCOLSTATISTICS_SIZE \ + (FIELD_OFFSET(MS_SMHBA_PROTOCOLSTATISTICS, OutputMegabytes) + \ + MS_SMHBA_PROTOCOLSTATISTICS_OutputMegabytes_SIZE) + +// MS_SMHBA_SASPHYSTATISTICS - MS_SMHBA_SASPHYSTATISTICS +#define MS_SMHBA_SASPHYSTATISTICSGuid \ + { 0xbd458e7d, 0xc40a, 0x4401, \ + { 0xa1, 0x79, 0x11, 0x91, 0x9c, 0xbc, 0xc5, 0xc6 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_SASPHYSTATISTICS_GUID, \ + 0xbd458e7d, 0xc40a, 0x4401, 0xa1, 0x79, 0x11, 0x91, \ + 0x9c, 0xbc, 0xc5, 0xc6); +#endif + + +typedef struct _MS_SMHBA_SASPHYSTATISTICS +{ + // + LONGLONG SecondsSinceLastReset; +#define MS_SMHBA_SASPHYSTATISTICS_SecondsSinceLastReset_SIZE sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_SecondsSinceLastReset_ID 1 + + // + LONGLONG TxFrames; +#define MS_SMHBA_SASPHYSTATISTICS_TxFrames_SIZE sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_TxFrames_ID 2 + + // + LONGLONG TxWords; +#define MS_SMHBA_SASPHYSTATISTICS_TxWords_SIZE sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_TxWords_ID 3 + + // + LONGLONG RxFrames; +#define MS_SMHBA_SASPHYSTATISTICS_RxFrames_SIZE sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_RxFrames_ID 4 + + // + LONGLONG RxWords; +#define MS_SMHBA_SASPHYSTATISTICS_RxWords_SIZE sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_RxWords_ID 5 + + // + LONGLONG InvalidDwordCount; +#define MS_SMHBA_SASPHYSTATISTICS_InvalidDwordCount_SIZE sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_InvalidDwordCount_ID 6 + + // + LONGLONG RunningDisparityErrorCount; +#define MS_SMHBA_SASPHYSTATISTICS_RunningDisparityErrorCount_SIZE \ + sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_RunningDisparityErrorCount_ID 7 + + // + LONGLONG LossofDwordSyncCount; +#define MS_SMHBA_SASPHYSTATISTICS_LossofDwordSyncCount_SIZE sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_LossofDwordSyncCount_ID 8 + + // + LONGLONG PhyResetProblemCount; +#define MS_SMHBA_SASPHYSTATISTICS_PhyResetProblemCount_SIZE sizeof (LONGLONG) +#define MS_SMHBA_SASPHYSTATISTICS_PhyResetProblemCount_ID 9 + +} MS_SMHBA_SASPHYSTATISTICS, *PMS_SMHBA_SASPHYSTATISTICS; + +#define MS_SMHBA_SASPHYSTATISTICS_SIZE \ + (FIELD_OFFSET(MS_SMHBA_SASPHYSTATISTICS, PhyResetProblemCount) + \ + MS_SMHBA_SASPHYSTATISTICS_PhyResetProblemCount_SIZE) + +// MS_SMHBA_FC_PHY - MS_SMHBA_FC_PHY +#define MS_SMHBA_FC_PHYGuid \ + { 0xfb66c8fe, 0x1da0, 0x48a2, \ + { 0x92, 0xdb, 0x02, 0xc3, 0x41, 0x14, 0x3c, 0x46 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_FC_PHY_GUID, \ + 0xfb66c8fe, 0x1da0, 0x48a2, 0x92, 0xdb, 0x02, 0xc3, \ + 0x41, 0x14, 0x3c, 0x46); +#endif + + +typedef struct _MS_SMHBA_FC_PHY +{ + // + ULONG PhySupportSpeed; +#define MS_SMHBA_FC_PHY_PhySupportSpeed_SIZE sizeof (ULONG) +#define MS_SMHBA_FC_PHY_PhySupportSpeed_ID 1 + + // + ULONG PhySpeed; +#define MS_SMHBA_FC_PHY_PhySpeed_SIZE sizeof (ULONG) +#define MS_SMHBA_FC_PHY_PhySpeed_ID 2 + + // + UCHAR PhyType; +#define MS_SMHBA_FC_PHY_PhyType_SIZE sizeof (UCHAR) +#define MS_SMHBA_FC_PHY_PhyType_ID 3 + + // + ULONG MaxFrameSize; +#define MS_SMHBA_FC_PHY_MaxFrameSize_SIZE sizeof (ULONG) +#define MS_SMHBA_FC_PHY_MaxFrameSize_ID 4 + +} MS_SMHBA_FC_PHY, *PMS_SMHBA_FC_PHY; + +#define MS_SMHBA_FC_PHY_SIZE \ + (FIELD_OFFSET(MS_SMHBA_FC_PHY, MaxFrameSize) + \ + MS_SMHBA_FC_PHY_MaxFrameSize_SIZE) + +// MS_SMHBA_SAS_PHY - MS_SMHBA_SAS_PHY +#define MS_SMHBA_SAS_PHYGuid \ + { 0xdde0a090, 0x96bc, 0x452b, \ + { 0x9a, 0x64, 0x6f, 0xbb, 0x6a, 0x19, 0xc4, 0x7d } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_SAS_PHY_GUID, \ + 0xdde0a090, 0x96bc, 0x452b, 0x9a, 0x64, 0x6f, 0xbb, \ + 0x6a, 0x19, 0xc4, 0x7d); +#endif + + +typedef struct _MS_SMHBA_SAS_PHY +{ + // + UCHAR PhyIdentifier; +#define MS_SMHBA_SAS_PHY_PhyIdentifier_SIZE sizeof (UCHAR) +#define MS_SMHBA_SAS_PHY_PhyIdentifier_ID 1 + + // + ULONG NegotiatedLinkRate; +#define MS_SMHBA_SAS_PHY_NegotiatedLinkRate_SIZE sizeof (ULONG) +#define MS_SMHBA_SAS_PHY_NegotiatedLinkRate_ID 2 + + // + ULONG ProgrammedMinLinkRate; +#define MS_SMHBA_SAS_PHY_ProgrammedMinLinkRate_SIZE sizeof (ULONG) +#define MS_SMHBA_SAS_PHY_ProgrammedMinLinkRate_ID 3 + + // + ULONG HardwareMinLinkRate; +#define MS_SMHBA_SAS_PHY_HardwareMinLinkRate_SIZE sizeof (ULONG) +#define MS_SMHBA_SAS_PHY_HardwareMinLinkRate_ID 4 + + // + ULONG ProgrammedMaxLinkRate; +#define MS_SMHBA_SAS_PHY_ProgrammedMaxLinkRate_SIZE sizeof (ULONG) +#define MS_SMHBA_SAS_PHY_ProgrammedMaxLinkRate_ID 5 + + // + ULONG HardwareMaxLinkRate; +#define MS_SMHBA_SAS_PHY_HardwareMaxLinkRate_SIZE sizeof (ULONG) +#define MS_SMHBA_SAS_PHY_HardwareMaxLinkRate_ID 6 + + // + UCHAR domainPortWWN[8]; +#define MS_SMHBA_SAS_PHY_domainPortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SMHBA_SAS_PHY_domainPortWWN_ID 7 + +} MS_SMHBA_SAS_PHY, *PMS_SMHBA_SAS_PHY; + +#define MS_SMHBA_SAS_PHY_SIZE \ + (FIELD_OFFSET(MS_SMHBA_SAS_PHY, domainPortWWN) + \ + MS_SMHBA_SAS_PHY_domainPortWWN_SIZE) + +// MS_SM_PortInformationMethods - MS_SM_PortInformationMethods +#define MS_SM_PortInformationMethodsGuid \ + { 0x5b6a8b86, 0x708d, 0x4ec6, \ + { 0x82, 0xa6, 0x39, 0xad, 0xcf, 0x6f, 0x64, 0x33 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_PortInformationMethods_GUID, \ + 0x5b6a8b86, 0x708d, 0x4ec6, 0x82, 0xa6, 0x39, 0xad, \ + 0xcf, 0x6f, 0x64, 0x33); +#endif + +// +// Method id definitions for MS_SM_PortInformationMethods +#define SM_GetPortType 1 +typedef struct _SM_GetPortType_IN +{ + // + ULONG PortIndex; +#define SM_GetPortType_IN_PortIndex_SIZE sizeof (ULONG) +#define SM_GetPortType_IN_PortIndex_ID 1 + +} SM_GetPortType_IN, *PSM_GetPortType_IN; + +#define SM_GetPortType_IN_SIZE \ + (FIELD_OFFSET(SM_GetPortType_IN, PortIndex) + \ + SM_GetPortType_IN_PortIndex_SIZE) + +typedef struct _SM_GetPortType_OUT +{ + // + ULONG HBAStatus; +#define SM_GetPortType_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetPortType_OUT_HBAStatus_ID 2 + + // + ULONG PortType; +#define SM_GetPortType_OUT_PortType_SIZE sizeof (ULONG) +#define SM_GetPortType_OUT_PortType_ID 3 + +} SM_GetPortType_OUT, *PSM_GetPortType_OUT; + +#define SM_GetPortType_OUT_SIZE \ + (FIELD_OFFSET(SM_GetPortType_OUT, PortType) + \ + SM_GetPortType_OUT_PortType_SIZE) + +#define SM_GetAdapterPortAttributes 2 +typedef struct _SM_GetAdapterPortAttributes_IN +{ + // + ULONG PortIndex; +#define SM_GetAdapterPortAttributes_IN_PortIndex_SIZE sizeof (ULONG) +#define SM_GetAdapterPortAttributes_IN_PortIndex_ID 1 + + +#define SM_PORT_SPECIFIC_ATTRIBUTES_MAXSIZE \ + max(sizeof (MS_SMHBA_FC_Port), sizeof (MS_SMHBA_SAS_Port)) + // + ULONG PortSpecificAttributesMaxSize; +#define SM_GetAdapterPortAttributes_IN_PortSpecificAttributesMaxSize_SIZE \ + sizeof (ULONG) +#define SM_GetAdapterPortAttributes_IN_PortSpecificAttributesMaxSize_ID 2 + +} SM_GetAdapterPortAttributes_IN, *PSM_GetAdapterPortAttributes_IN; + +#define SM_GetAdapterPortAttributes_IN_SIZE \ + (FIELD_OFFSET(SM_GetAdapterPortAttributes_IN, \ + PortSpecificAttributesMaxSize) + \ + SM_GetAdapterPortAttributes_IN_PortSpecificAttributesMaxSize_SIZE) + +typedef struct _SM_GetAdapterPortAttributes_OUT +{ + // + ULONG HBAStatus; +#define SM_GetAdapterPortAttributes_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetAdapterPortAttributes_OUT_HBAStatus_ID 3 + + // + MS_SMHBA_PORTATTRIBUTES PortAttributes; +#define SM_GetAdapterPortAttributes_OUT_PortAttributes_SIZE \ + sizeof (MS_SMHBA_PORTATTRIBUTES) +#define SM_GetAdapterPortAttributes_OUT_PortAttributes_ID 4 + +} SM_GetAdapterPortAttributes_OUT, *PSM_GetAdapterPortAttributes_OUT; + +#define SM_GetAdapterPortAttributes_OUT_SIZE \ + (FIELD_OFFSET(SM_GetAdapterPortAttributes_OUT, PortAttributes) + \ + SM_GetAdapterPortAttributes_OUT_PortAttributes_SIZE) + +#define SM_GetDiscoveredPortAttributes 3 +typedef struct _SM_GetDiscoveredPortAttributes_IN +{ + // + ULONG PortIndex; +#define SM_GetDiscoveredPortAttributes_IN_PortIndex_SIZE sizeof (ULONG) +#define SM_GetDiscoveredPortAttributes_IN_PortIndex_ID 1 + + // + ULONG DiscoveredPortIndex; +#define SM_GetDiscoveredPortAttributes_IN_DiscoveredPortIndex_SIZE \ + sizeof (ULONG) +#define SM_GetDiscoveredPortAttributes_IN_DiscoveredPortIndex_ID 2 + + // + ULONG PortSpecificAttributesMaxSize; +#define SM_GetDiscoveredPortAttributes_IN_PortSpecificAttributesMaxSize_SIZE \ + sizeof (ULONG) +#define SM_GetDiscoveredPortAttributes_IN_PortSpecificAttributesMaxSize_ID 3 + +} SM_GetDiscoveredPortAttributes_IN, *PSM_GetDiscoveredPortAttributes_IN; + +#define SM_GetDiscoveredPortAttributes_IN_SIZE \ + (FIELD_OFFSET(SM_GetDiscoveredPortAttributes_IN, \ + PortSpecificAttributesMaxSize) + \ + SM_GetDiscoveredPortAttributes_IN_PortSpecificAttributesMaxSize_SIZE) + +typedef struct _SM_GetDiscoveredPortAttributes_OUT +{ + // + ULONG HBAStatus; +#define SM_GetDiscoveredPortAttributes_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetDiscoveredPortAttributes_OUT_HBAStatus_ID 4 + + // + MS_SMHBA_PORTATTRIBUTES PortAttributes; +#define SM_GetDiscoveredPortAttributes_OUT_PortAttributes_SIZE \ + sizeof (MS_SMHBA_PORTATTRIBUTES) +#define SM_GetDiscoveredPortAttributes_OUT_PortAttributes_ID 5 + +} SM_GetDiscoveredPortAttributes_OUT, *PSM_GetDiscoveredPortAttributes_OUT; + +#define SM_GetDiscoveredPortAttributes_OUT_SIZE \ + (FIELD_OFFSET(SM_GetDiscoveredPortAttributes_OUT, PortAttributes) + \ + SM_GetDiscoveredPortAttributes_OUT_PortAttributes_SIZE) + +#define SM_GetPortAttributesByWWN 4 +typedef struct _SM_GetPortAttributesByWWN_IN +{ + // + UCHAR PortWWN[8]; +#define SM_GetPortAttributesByWWN_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetPortAttributesByWWN_IN_PortWWN_ID 1 + + // + UCHAR DomainPortWWN[8]; +#define SM_GetPortAttributesByWWN_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetPortAttributesByWWN_IN_DomainPortWWN_ID 2 + + // + ULONG PortSpecificAttributesMaxSize; +#define SM_GetPortAttributesByWWN_IN_PortSpecificAttributesMaxSize_SIZE \ + sizeof (ULONG) +#define SM_GetPortAttributesByWWN_IN_PortSpecificAttributesMaxSize_ID 3 + +} SM_GetPortAttributesByWWN_IN, *PSM_GetPortAttributesByWWN_IN; + +#define SM_GetPortAttributesByWWN_IN_SIZE \ + (FIELD_OFFSET(SM_GetPortAttributesByWWN_IN, \ + PortSpecificAttributesMaxSize) + \ + SM_GetPortAttributesByWWN_IN_PortSpecificAttributesMaxSize_SIZE) + +typedef struct _SM_GetPortAttributesByWWN_OUT +{ + // + ULONG HBAStatus; +#define SM_GetPortAttributesByWWN_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetPortAttributesByWWN_OUT_HBAStatus_ID 4 + + // + MS_SMHBA_PORTATTRIBUTES PortAttributes; +#define SM_GetPortAttributesByWWN_OUT_PortAttributes_SIZE \ + sizeof (MS_SMHBA_PORTATTRIBUTES) +#define SM_GetPortAttributesByWWN_OUT_PortAttributes_ID 5 + +} SM_GetPortAttributesByWWN_OUT, *PSM_GetPortAttributesByWWN_OUT; + +#define SM_GetPortAttributesByWWN_OUT_SIZE \ + (FIELD_OFFSET(SM_GetPortAttributesByWWN_OUT, PortAttributes) + \ + SM_GetPortAttributesByWWN_OUT_PortAttributes_SIZE) + +#define SM_GetProtocolStatistics 5 +typedef struct _SM_GetProtocolStatistics_IN +{ + // + ULONG PortIndex; +#define SM_GetProtocolStatistics_IN_PortIndex_SIZE sizeof (ULONG) +#define SM_GetProtocolStatistics_IN_PortIndex_ID 1 + + // + ULONG ProtocolType; +#define SM_GetProtocolStatistics_IN_ProtocolType_SIZE sizeof (ULONG) +#define SM_GetProtocolStatistics_IN_ProtocolType_ID 2 + +} SM_GetProtocolStatistics_IN, *PSM_GetProtocolStatistics_IN; + +#define SM_GetProtocolStatistics_IN_SIZE \ + (FIELD_OFFSET(SM_GetProtocolStatistics_IN, ProtocolType) + \ + SM_GetProtocolStatistics_IN_ProtocolType_SIZE) + +typedef struct _SM_GetProtocolStatistics_OUT +{ + // + ULONG HBAStatus; +#define SM_GetProtocolStatistics_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetProtocolStatistics_OUT_HBAStatus_ID 3 + + // + MS_SMHBA_PROTOCOLSTATISTICS ProtocolStatistics; +#define SM_GetProtocolStatistics_OUT_ProtocolStatistics_SIZE \ + sizeof (MS_SMHBA_PROTOCOLSTATISTICS) +#define SM_GetProtocolStatistics_OUT_ProtocolStatistics_ID 4 + +} SM_GetProtocolStatistics_OUT, *PSM_GetProtocolStatistics_OUT; + +#define SM_GetProtocolStatistics_OUT_SIZE \ + (FIELD_OFFSET(SM_GetProtocolStatistics_OUT, ProtocolStatistics) + \ + SM_GetProtocolStatistics_OUT_ProtocolStatistics_SIZE) + +#define SM_GetPhyStatistics 6 +typedef struct _SM_GetPhyStatistics_IN +{ + // + ULONG PortIndex; +#define SM_GetPhyStatistics_IN_PortIndex_SIZE sizeof (ULONG) +#define SM_GetPhyStatistics_IN_PortIndex_ID 1 + + // + ULONG PhyIndex; +#define SM_GetPhyStatistics_IN_PhyIndex_SIZE sizeof (ULONG) +#define SM_GetPhyStatistics_IN_PhyIndex_ID 2 + + // + ULONG InNumOfPhyCounters; +#define SM_GetPhyStatistics_IN_InNumOfPhyCounters_SIZE sizeof (ULONG) +#define SM_GetPhyStatistics_IN_InNumOfPhyCounters_ID 3 + +} SM_GetPhyStatistics_IN, *PSM_GetPhyStatistics_IN; + +#define SM_GetPhyStatistics_IN_SIZE \ + (FIELD_OFFSET(SM_GetPhyStatistics_IN, InNumOfPhyCounters) + \ + SM_GetPhyStatistics_IN_InNumOfPhyCounters_SIZE) + +typedef struct _SM_GetPhyStatistics_OUT +{ + // + ULONG HBAStatus; +#define SM_GetPhyStatistics_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetPhyStatistics_OUT_HBAStatus_ID 4 + + // + ULONG TotalNumOfPhyCounters; +#define SM_GetPhyStatistics_OUT_TotalNumOfPhyCounters_SIZE sizeof (ULONG) +#define SM_GetPhyStatistics_OUT_TotalNumOfPhyCounters_ID 5 + + // + ULONG OutNumOfPhyCounters; +#define SM_GetPhyStatistics_OUT_OutNumOfPhyCounters_SIZE sizeof (ULONG) +#define SM_GetPhyStatistics_OUT_OutNumOfPhyCounters_ID 6 + + // + LONGLONG PhyCounter[1]; +#define SM_GetPhyStatistics_OUT_PhyCounter_ID 7 + +} SM_GetPhyStatistics_OUT, *PSM_GetPhyStatistics_OUT; + +#define SM_GetFCPhyAttributes 7 +typedef struct _SM_GetFCPhyAttributes_IN +{ + // + ULONG PortIndex; +#define SM_GetFCPhyAttributes_IN_PortIndex_SIZE sizeof (ULONG) +#define SM_GetFCPhyAttributes_IN_PortIndex_ID 1 + + // + ULONG PhyIndex; +#define SM_GetFCPhyAttributes_IN_PhyIndex_SIZE sizeof (ULONG) +#define SM_GetFCPhyAttributes_IN_PhyIndex_ID 2 + +} SM_GetFCPhyAttributes_IN, *PSM_GetFCPhyAttributes_IN; + +#define SM_GetFCPhyAttributes_IN_SIZE \ + (FIELD_OFFSET(SM_GetFCPhyAttributes_IN, PhyIndex) + \ + SM_GetFCPhyAttributes_IN_PhyIndex_SIZE) + +typedef struct _SM_GetFCPhyAttributes_OUT +{ + // + ULONG HBAStatus; +#define SM_GetFCPhyAttributes_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetFCPhyAttributes_OUT_HBAStatus_ID 3 + + // + MS_SMHBA_FC_PHY PhyType; +#define SM_GetFCPhyAttributes_OUT_PhyType_SIZE sizeof (MS_SMHBA_FC_PHY) +#define SM_GetFCPhyAttributes_OUT_PhyType_ID 4 + +} SM_GetFCPhyAttributes_OUT, *PSM_GetFCPhyAttributes_OUT; + +#define SM_GetFCPhyAttributes_OUT_SIZE \ + (FIELD_OFFSET(SM_GetFCPhyAttributes_OUT, PhyType) + \ + SM_GetFCPhyAttributes_OUT_PhyType_SIZE) + +#define SM_GetSASPhyAttributes 8 +typedef struct _SM_GetSASPhyAttributes_IN +{ + // + ULONG PortIndex; +#define SM_GetSASPhyAttributes_IN_PortIndex_SIZE sizeof (ULONG) +#define SM_GetSASPhyAttributes_IN_PortIndex_ID 1 + + // + ULONG PhyIndex; +#define SM_GetSASPhyAttributes_IN_PhyIndex_SIZE sizeof (ULONG) +#define SM_GetSASPhyAttributes_IN_PhyIndex_ID 2 + +} SM_GetSASPhyAttributes_IN, *PSM_GetSASPhyAttributes_IN; + +#define SM_GetSASPhyAttributes_IN_SIZE \ + (FIELD_OFFSET(SM_GetSASPhyAttributes_IN, PhyIndex) + \ + SM_GetSASPhyAttributes_IN_PhyIndex_SIZE) + +typedef struct _SM_GetSASPhyAttributes_OUT +{ + // + ULONG HBAStatus; +#define SM_GetSASPhyAttributes_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetSASPhyAttributes_OUT_HBAStatus_ID 3 + + // + MS_SMHBA_SAS_PHY PhyType; +#define SM_GetSASPhyAttributes_OUT_PhyType_SIZE sizeof (MS_SMHBA_SAS_PHY) +#define SM_GetSASPhyAttributes_OUT_PhyType_ID 4 + +} SM_GetSASPhyAttributes_OUT, *PSM_GetSASPhyAttributes_OUT; + +#define SM_GetSASPhyAttributes_OUT_SIZE \ + (FIELD_OFFSET(SM_GetSASPhyAttributes_OUT, PhyType) + \ + SM_GetSASPhyAttributes_OUT_PhyType_SIZE) + +#define SM_RefreshInformation 10 + +// MS_SMHBA_PORTLUN - MS_SMHBA_PORTLUN +#define MS_SMHBA_PORTLUNGuid \ + { 0x0669d100, 0x066e, 0x4e49, \ + { 0xa6, 0x8c, 0xe0, 0x51, 0x99, 0x59, 0x61, 0x32 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_PORTLUN_GUID, \ + 0x0669d100, 0x066e, 0x4e49, 0xa6, 0x8c, 0xe0, 0x51, \ + 0x99, 0x59, 0x61, 0x32); +#endif + + +typedef struct _MS_SMHBA_PORTLUN +{ + // + UCHAR PortWWN[8]; +#define MS_SMHBA_PORTLUN_PortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SMHBA_PORTLUN_PortWWN_ID 1 + + // + UCHAR domainPortWWN[8]; +#define MS_SMHBA_PORTLUN_domainPortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SMHBA_PORTLUN_domainPortWWN_ID 2 + + // + ULONGLONG TargetLun; +#define MS_SMHBA_PORTLUN_TargetLun_SIZE sizeof (ULONGLONG) +#define MS_SMHBA_PORTLUN_TargetLun_ID 3 + +} MS_SMHBA_PORTLUN, *PMS_SMHBA_PORTLUN; + +#define MS_SMHBA_PORTLUN_SIZE \ + (FIELD_OFFSET(MS_SMHBA_PORTLUN, TargetLun) + \ + MS_SMHBA_PORTLUN_TargetLun_SIZE) + +// MS_SMHBA_SCSIENTRY - MS_SMHBA_SCSIENTRY +#define MS_SMHBA_SCSIENTRYGuid \ + { 0x125d41bc, 0x7643, 0x4155, \ + { 0xb8, 0x1c, 0xe2, 0xf1, 0x28, 0xad, 0x1f, 0xb4 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_SCSIENTRY_GUID, \ + 0x125d41bc, 0x7643, 0x4155, 0xb8, 0x1c, 0xe2, 0xf1, \ + 0x28, 0xad, 0x1f, 0xb4); +#endif + + +typedef struct _MS_SMHBA_SCSIENTRY +{ + // + MS_SMHBA_PORTLUN PortLun; +#define MS_SMHBA_SCSIENTRY_PortLun_SIZE sizeof (MS_SMHBA_PORTLUN) +#define MS_SMHBA_SCSIENTRY_PortLun_ID 1 + + // + UCHAR LUID[256]; +#define MS_SMHBA_SCSIENTRY_LUID_SIZE sizeof (UCHAR[256]) +#define MS_SMHBA_SCSIENTRY_LUID_ID 2 + + // + HBAScsiID ScsiId; +#define MS_SMHBA_SCSIENTRY_ScsiId_SIZE sizeof (HBAScsiID) +#define MS_SMHBA_SCSIENTRY_ScsiId_ID 3 + +} MS_SMHBA_SCSIENTRY, *PMS_SMHBA_SCSIENTRY; + +#define MS_SMHBA_SCSIENTRY_SIZE \ + (FIELD_OFFSET(MS_SMHBA_SCSIENTRY, ScsiId) + \ + MS_SMHBA_SCSIENTRY_ScsiId_SIZE) + +// MS_SMHBA_BINDINGENTRY - MS_SMHBA_BINDINGENTRY +#define MS_SMHBA_BINDINGENTRYGuid \ + { 0x65bfb548, 0xd00a, 0x4d4c, \ + { 0xa3, 0x57, 0x7d, 0xaa, 0x23, 0xbc, 0x2e, 0x3d } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SMHBA_BINDINGENTRY_GUID, \ + 0x65bfb548, 0xd00a, 0x4d4c, 0xa3, 0x57, 0x7d, 0xaa, \ + 0x23, 0xbc, 0x2e, 0x3d); +#endif + + +typedef struct _MS_SMHBA_BINDINGENTRY +{ + // + ULONG type; +#define MS_SMHBA_BINDINGENTRY_type_SIZE sizeof (ULONG) +#define MS_SMHBA_BINDINGENTRY_type_ID 1 + + // + MS_SMHBA_PORTLUN PortLun; +#define MS_SMHBA_BINDINGENTRY_PortLun_SIZE sizeof (MS_SMHBA_PORTLUN) +#define MS_SMHBA_BINDINGENTRY_PortLun_ID 2 + + // + UCHAR LUID[256]; +#define MS_SMHBA_BINDINGENTRY_LUID_SIZE sizeof (UCHAR[256]) +#define MS_SMHBA_BINDINGENTRY_LUID_ID 3 + + // + ULONG Status; +#define MS_SMHBA_BINDINGENTRY_Status_SIZE sizeof (ULONG) +#define MS_SMHBA_BINDINGENTRY_Status_ID 4 + + // + HBAScsiID ScsiId; +#define MS_SMHBA_BINDINGENTRY_ScsiId_SIZE sizeof (HBAScsiID) +#define MS_SMHBA_BINDINGENTRY_ScsiId_ID 5 + +} MS_SMHBA_BINDINGENTRY, *PMS_SMHBA_BINDINGENTRY; + +#define MS_SMHBA_BINDINGENTRY_SIZE \ + (FIELD_OFFSET(MS_SMHBA_BINDINGENTRY, ScsiId) + \ + MS_SMHBA_BINDINGENTRY_ScsiId_SIZE) + +// MS_SM_TargetInformationMethods - MS_SM_TargetInformationMethods +#define MS_SM_TargetInformationMethodsGuid \ + { 0x93545055, 0xab4c, 0x4e80, \ + { 0x84, 0xae, 0x6a, 0x86, 0xa2, 0xdc, 0x4b, 0x84 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_TargetInformationMethods_GUID, \ + 0x93545055, 0xab4c, 0x4e80, 0x84, 0xae, 0x6a, 0x86, \ + 0xa2, 0xdc, 0x4b, 0x84); +#endif + +// +// Method id definitions for MS_SM_TargetInformationMethods +#define SM_GetTargetMapping 1 +typedef struct _SM_GetTargetMapping_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_GetTargetMapping_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetTargetMapping_IN_HbaPortWWN_ID 1 + + // + UCHAR DomainPortWWN[8]; +#define SM_GetTargetMapping_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetTargetMapping_IN_DomainPortWWN_ID 2 + + // + ULONG InEntryCount; +#define SM_GetTargetMapping_IN_InEntryCount_SIZE sizeof (ULONG) +#define SM_GetTargetMapping_IN_InEntryCount_ID 3 + +} SM_GetTargetMapping_IN, *PSM_GetTargetMapping_IN; + +#define SM_GetTargetMapping_IN_SIZE \ + (FIELD_OFFSET(SM_GetTargetMapping_IN, InEntryCount) + \ + SM_GetTargetMapping_IN_InEntryCount_SIZE) + +typedef struct _SM_GetTargetMapping_OUT +{ + // + ULONG HBAStatus; +#define SM_GetTargetMapping_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetTargetMapping_OUT_HBAStatus_ID 4 + + // + ULONG TotalEntryCount; +#define SM_GetTargetMapping_OUT_TotalEntryCount_SIZE sizeof (ULONG) +#define SM_GetTargetMapping_OUT_TotalEntryCount_ID 5 + + // + ULONG OutEntryCount; +#define SM_GetTargetMapping_OUT_OutEntryCount_SIZE sizeof (ULONG) +#define SM_GetTargetMapping_OUT_OutEntryCount_ID 6 + + // + MS_SMHBA_SCSIENTRY Entry[1]; +#define SM_GetTargetMapping_OUT_Entry_ID 7 + +} SM_GetTargetMapping_OUT, *PSM_GetTargetMapping_OUT; + +#define SM_GetBindingCapability 2 +typedef struct _SM_GetBindingCapability_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_GetBindingCapability_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetBindingCapability_IN_HbaPortWWN_ID 1 + + // + UCHAR DomainPortWWN[8]; +#define SM_GetBindingCapability_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetBindingCapability_IN_DomainPortWWN_ID 2 + +} SM_GetBindingCapability_IN, *PSM_GetBindingCapability_IN; + +#define SM_GetBindingCapability_IN_SIZE \ + (FIELD_OFFSET(SM_GetBindingCapability_IN, DomainPortWWN) + \ + SM_GetBindingCapability_IN_DomainPortWWN_SIZE) + +typedef struct _SM_GetBindingCapability_OUT +{ + // + ULONG HBAStatus; +#define SM_GetBindingCapability_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetBindingCapability_OUT_HBAStatus_ID 3 + + // + ULONG Flags; +#define SM_GetBindingCapability_OUT_Flags_SIZE sizeof (ULONG) +#define SM_GetBindingCapability_OUT_Flags_ID 4 + +} SM_GetBindingCapability_OUT, *PSM_GetBindingCapability_OUT; + +#define SM_GetBindingCapability_OUT_SIZE \ + (FIELD_OFFSET(SM_GetBindingCapability_OUT, Flags) + \ + SM_GetBindingCapability_OUT_Flags_SIZE) + +#define SM_GetBindingSupport 3 +typedef struct _SM_GetBindingSupport_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_GetBindingSupport_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetBindingSupport_IN_HbaPortWWN_ID 1 + + // + UCHAR DomainPortWWN[8]; +#define SM_GetBindingSupport_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetBindingSupport_IN_DomainPortWWN_ID 2 + +} SM_GetBindingSupport_IN, *PSM_GetBindingSupport_IN; + +#define SM_GetBindingSupport_IN_SIZE \ + (FIELD_OFFSET(SM_GetBindingSupport_IN, DomainPortWWN) + \ + SM_GetBindingSupport_IN_DomainPortWWN_SIZE) + +typedef struct _SM_GetBindingSupport_OUT +{ + // + ULONG HBAStatus; +#define SM_GetBindingSupport_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetBindingSupport_OUT_HBAStatus_ID 3 + + // + ULONG Flags; +#define SM_GetBindingSupport_OUT_Flags_SIZE sizeof (ULONG) +#define SM_GetBindingSupport_OUT_Flags_ID 4 + +} SM_GetBindingSupport_OUT, *PSM_GetBindingSupport_OUT; + +#define SM_GetBindingSupport_OUT_SIZE \ + (FIELD_OFFSET(SM_GetBindingSupport_OUT, Flags) + \ + SM_GetBindingSupport_OUT_Flags_SIZE) + +#define SM_SetBindingSupport 4 +typedef struct _SM_SetBindingSupport_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SetBindingSupport_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SetBindingSupport_IN_HbaPortWWN_ID 1 + + // + UCHAR DomainPortWWN[8]; +#define SM_SetBindingSupport_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SetBindingSupport_IN_DomainPortWWN_ID 2 + + // + ULONG Flags; +#define SM_SetBindingSupport_IN_Flags_SIZE sizeof (ULONG) +#define SM_SetBindingSupport_IN_Flags_ID 3 + +} SM_SetBindingSupport_IN, *PSM_SetBindingSupport_IN; + +#define SM_SetBindingSupport_IN_SIZE \ + (FIELD_OFFSET(SM_SetBindingSupport_IN, Flags) + \ + SM_SetBindingSupport_IN_Flags_SIZE) + +typedef struct _SM_SetBindingSupport_OUT +{ + // + ULONG HBAStatus; +#define SM_SetBindingSupport_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SetBindingSupport_OUT_HBAStatus_ID 4 + +} SM_SetBindingSupport_OUT, *PSM_SetBindingSupport_OUT; + +#define SM_SetBindingSupport_OUT_SIZE \ + (FIELD_OFFSET(SM_SetBindingSupport_OUT, HBAStatus) + \ + SM_SetBindingSupport_OUT_HBAStatus_SIZE) + +#define SM_GetPersistentBinding 5 +typedef struct _SM_GetPersistentBinding_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_GetPersistentBinding_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetPersistentBinding_IN_HbaPortWWN_ID 1 + + // + UCHAR DomainPortWWN[8]; +#define SM_GetPersistentBinding_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_GetPersistentBinding_IN_DomainPortWWN_ID 2 + + // + ULONG InEntryCount; +#define SM_GetPersistentBinding_IN_InEntryCount_SIZE sizeof (ULONG) +#define SM_GetPersistentBinding_IN_InEntryCount_ID 3 + +} SM_GetPersistentBinding_IN, *PSM_GetPersistentBinding_IN; + +#define SM_GetPersistentBinding_IN_SIZE \ + (FIELD_OFFSET(SM_GetPersistentBinding_IN, InEntryCount) + \ + SM_GetPersistentBinding_IN_InEntryCount_SIZE) + +typedef struct _SM_GetPersistentBinding_OUT +{ + // + ULONG HBAStatus; +#define SM_GetPersistentBinding_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetPersistentBinding_OUT_HBAStatus_ID 4 + + // + ULONG TotalEntryCount; +#define SM_GetPersistentBinding_OUT_TotalEntryCount_SIZE sizeof (ULONG) +#define SM_GetPersistentBinding_OUT_TotalEntryCount_ID 5 + + // + ULONG OutEntryCount; +#define SM_GetPersistentBinding_OUT_OutEntryCount_SIZE sizeof (ULONG) +#define SM_GetPersistentBinding_OUT_OutEntryCount_ID 6 + + // + MS_SMHBA_BINDINGENTRY Entry[1]; +#define SM_GetPersistentBinding_OUT_Entry_ID 7 + +} SM_GetPersistentBinding_OUT, *PSM_GetPersistentBinding_OUT; + +#define SM_SetPersistentBinding 6 +typedef struct _SM_SetPersistentBinding_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SetPersistentBinding_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SetPersistentBinding_IN_HbaPortWWN_ID 1 + + // + UCHAR DomainPortWWN[8]; +#define SM_SetPersistentBinding_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SetPersistentBinding_IN_DomainPortWWN_ID 2 + + // + ULONG InEntryCount; +#define SM_SetPersistentBinding_IN_InEntryCount_SIZE sizeof (ULONG) +#define SM_SetPersistentBinding_IN_InEntryCount_ID 3 + + // + MS_SMHBA_BINDINGENTRY Entry[1]; +#define SM_SetPersistentBinding_IN_Entry_ID 4 + +} SM_SetPersistentBinding_IN, *PSM_SetPersistentBinding_IN; + +typedef struct _SM_SetPersistentBinding_OUT +{ + // + ULONG HBAStatus; +#define SM_SetPersistentBinding_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SetPersistentBinding_OUT_HBAStatus_ID 5 + + // + ULONG OutStatusCount; +#define SM_SetPersistentBinding_OUT_OutStatusCount_SIZE sizeof (ULONG) +#define SM_SetPersistentBinding_OUT_OutStatusCount_ID 6 + + // + ULONG EntryStatus[1]; +#define SM_SetPersistentBinding_OUT_EntryStatus_ID 7 + +} SM_SetPersistentBinding_OUT, *PSM_SetPersistentBinding_OUT; + +#define SM_RemovePersistentBinding 7 +typedef struct _SM_RemovePersistentBinding_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_RemovePersistentBinding_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_RemovePersistentBinding_IN_HbaPortWWN_ID 1 + + // + UCHAR DomainPortWWN[8]; +#define SM_RemovePersistentBinding_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_RemovePersistentBinding_IN_DomainPortWWN_ID 2 + + // + ULONG EntryCount; +#define SM_RemovePersistentBinding_IN_EntryCount_SIZE sizeof (ULONG) +#define SM_RemovePersistentBinding_IN_EntryCount_ID 3 + + // + MS_SMHBA_BINDINGENTRY Entry[1]; +#define SM_RemovePersistentBinding_IN_Entry_ID 4 + +} SM_RemovePersistentBinding_IN, *PSM_RemovePersistentBinding_IN; + +typedef struct _SM_RemovePersistentBinding_OUT +{ + // + ULONG HBAStatus; +#define SM_RemovePersistentBinding_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_RemovePersistentBinding_OUT_HBAStatus_ID 5 + +} SM_RemovePersistentBinding_OUT, *PSM_RemovePersistentBinding_OUT; + +#define SM_RemovePersistentBinding_OUT_SIZE \ + (FIELD_OFFSET(SM_RemovePersistentBinding_OUT, HBAStatus) + \ + SM_RemovePersistentBinding_OUT_HBAStatus_SIZE) + +#define SM_GetLUNStatistics 8 +typedef struct _SM_GetLUNStatistics_IN +{ + // + HBAScsiID Lunit; +#define SM_GetLUNStatistics_IN_Lunit_SIZE sizeof (HBAScsiID) +#define SM_GetLUNStatistics_IN_Lunit_ID 1 + +} SM_GetLUNStatistics_IN, *PSM_GetLUNStatistics_IN; + +#define SM_GetLUNStatistics_IN_SIZE \ + (FIELD_OFFSET(SM_GetLUNStatistics_IN, Lunit) + \ + SM_GetLUNStatistics_IN_Lunit_SIZE) + +typedef struct _SM_GetLUNStatistics_OUT +{ + // + ULONG HBAStatus; +#define SM_GetLUNStatistics_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetLUNStatistics_OUT_HBAStatus_ID 2 + + // + MS_SMHBA_PROTOCOLSTATISTICS ProtocolStatistics; +#define SM_GetLUNStatistics_OUT_ProtocolStatistics_SIZE \ + sizeof (MS_SMHBA_PROTOCOLSTATISTICS) +#define SM_GetLUNStatistics_OUT_ProtocolStatistics_ID 3 + +} SM_GetLUNStatistics_OUT, *PSM_GetLUNStatistics_OUT; + +#define SM_GetLUNStatistics_OUT_SIZE \ + (FIELD_OFFSET(SM_GetLUNStatistics_OUT, ProtocolStatistics) + \ + SM_GetLUNStatistics_OUT_ProtocolStatistics_SIZE) + + +// MS_SM_ScsiInformationMethods - MS_SM_ScsiInformationMethods +#define MS_SM_ScsiInformationMethodsGuid \ + { 0xb6661e6f, 0x075e, 0x4209, \ + { 0xae, 0x20, 0xfe, 0x81, 0xdb, 0x03, 0xd9, 0x79 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_ScsiInformationMethods_GUID, \ + 0xb6661e6f, 0x075e, 0x4209, 0xae, 0x20, 0xfe, 0x81, \ + 0xdb, 0x03, 0xd9, 0x79); +#endif + +// +// Method id definitions for MS_SM_ScsiInformationMethods +#define SM_ScsiInquiry 1 +typedef struct _SM_ScsiInquiry_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_ScsiInquiry_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiInquiry_IN_HbaPortWWN_ID 1 + + // + UCHAR DiscoveredPortWWN[8]; +#define SM_ScsiInquiry_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiInquiry_IN_DiscoveredPortWWN_ID 2 + + // + UCHAR DomainPortWWN[8]; +#define SM_ScsiInquiry_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiInquiry_IN_DomainPortWWN_ID 3 + + // + ULONGLONG SmhbaLUN; +#define SM_ScsiInquiry_IN_SmhbaLUN_SIZE sizeof (ULONGLONG) +#define SM_ScsiInquiry_IN_SmhbaLUN_ID 4 + + // + UCHAR Cdb[6]; +#define SM_ScsiInquiry_IN_Cdb_SIZE sizeof (UCHAR[6]) +#define SM_ScsiInquiry_IN_Cdb_ID 5 + + // + ULONG InRespBufferMaxSize; +#define SM_ScsiInquiry_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_ScsiInquiry_IN_InRespBufferMaxSize_ID 6 + + // + ULONG InSenseBufferMaxSize; +#define SM_ScsiInquiry_IN_InSenseBufferMaxSize_SIZE sizeof (ULONG) +#define SM_ScsiInquiry_IN_InSenseBufferMaxSize_ID 7 + +} SM_ScsiInquiry_IN, *PSM_ScsiInquiry_IN; + +#define SM_ScsiInquiry_IN_SIZE \ + (FIELD_OFFSET(SM_ScsiInquiry_IN, InSenseBufferMaxSize) + \ + SM_ScsiInquiry_IN_InSenseBufferMaxSize_SIZE) + +typedef struct _SM_ScsiInquiry_OUT +{ + // + ULONG HBAStatus; +#define SM_ScsiInquiry_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_ScsiInquiry_OUT_HBAStatus_ID 8 + + // + UCHAR ScsiStatus; +#define SM_ScsiInquiry_OUT_ScsiStatus_SIZE sizeof (UCHAR) +#define SM_ScsiInquiry_OUT_ScsiStatus_ID 9 + + // + ULONG OutRespBufferSize; +#define SM_ScsiInquiry_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_ScsiInquiry_OUT_OutRespBufferSize_ID 10 + + // + ULONG OutSenseBufferSize; +#define SM_ScsiInquiry_OUT_OutSenseBufferSize_SIZE sizeof (ULONG) +#define SM_ScsiInquiry_OUT_OutSenseBufferSize_ID 11 + + // + UCHAR RespBuffer[1]; +#define SM_ScsiInquiry_OUT_RespBuffer_ID 12 + + // +// UCHAR SenseBuffer[1]; +#define SM_ScsiInquiry_OUT_SenseBuffer_ID 13 + +} SM_ScsiInquiry_OUT, *PSM_ScsiInquiry_OUT; + +#define SM_ScsiReportLuns 2 +typedef struct _SM_ScsiReportLuns_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_ScsiReportLuns_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiReportLuns_IN_HbaPortWWN_ID 1 + + // + UCHAR DiscoveredPortWWN[8]; +#define SM_ScsiReportLuns_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiReportLuns_IN_DiscoveredPortWWN_ID 2 + + // + UCHAR DomainPortWWN[8]; +#define SM_ScsiReportLuns_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiReportLuns_IN_DomainPortWWN_ID 3 + + // + UCHAR Cdb[12]; +#define SM_ScsiReportLuns_IN_Cdb_SIZE sizeof (UCHAR[12]) +#define SM_ScsiReportLuns_IN_Cdb_ID 4 + + // + ULONG InRespBufferMaxSize; +#define SM_ScsiReportLuns_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_ScsiReportLuns_IN_InRespBufferMaxSize_ID 5 + + // + ULONG InSenseBufferMaxSize; +#define SM_ScsiReportLuns_IN_InSenseBufferMaxSize_SIZE sizeof (ULONG) +#define SM_ScsiReportLuns_IN_InSenseBufferMaxSize_ID 6 + +} SM_ScsiReportLuns_IN, *PSM_ScsiReportLuns_IN; + +#define SM_ScsiReportLuns_IN_SIZE \ + (FIELD_OFFSET(SM_ScsiReportLuns_IN, InSenseBufferMaxSize) + \ + SM_ScsiReportLuns_IN_InSenseBufferMaxSize_SIZE) + +typedef struct _SM_ScsiReportLuns_OUT +{ + // + ULONG HBAStatus; +#define SM_ScsiReportLuns_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_ScsiReportLuns_OUT_HBAStatus_ID 7 + + // + UCHAR ScsiStatus; +#define SM_ScsiReportLuns_OUT_ScsiStatus_SIZE sizeof (UCHAR) +#define SM_ScsiReportLuns_OUT_ScsiStatus_ID 8 + + // + ULONG TotalRespBufferSize; +#define SM_ScsiReportLuns_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_ScsiReportLuns_OUT_TotalRespBufferSize_ID 9 + + // + ULONG OutRespBufferSize; +#define SM_ScsiReportLuns_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_ScsiReportLuns_OUT_OutRespBufferSize_ID 10 + + // + ULONG OutSenseBufferSize; +#define SM_ScsiReportLuns_OUT_OutSenseBufferSize_SIZE sizeof (ULONG) +#define SM_ScsiReportLuns_OUT_OutSenseBufferSize_ID 11 + + // + UCHAR RespBuffer[1]; +#define SM_ScsiReportLuns_OUT_RespBuffer_ID 12 + + // +// UCHAR SenseBuffer[1]; +#define SM_ScsiReportLuns_OUT_SenseBuffer_ID 13 + +} SM_ScsiReportLuns_OUT, *PSM_ScsiReportLuns_OUT; + +#define SM_ScsiReadCapacity 3 +typedef struct _SM_ScsiReadCapacity_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_ScsiReadCapacity_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiReadCapacity_IN_HbaPortWWN_ID 1 + + // + UCHAR DiscoveredPortWWN[8]; +#define SM_ScsiReadCapacity_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiReadCapacity_IN_DiscoveredPortWWN_ID 2 + + // + UCHAR DomainPortWWN[8]; +#define SM_ScsiReadCapacity_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_ScsiReadCapacity_IN_DomainPortWWN_ID 3 + + // + ULONGLONG SmhbaLUN; +#define SM_ScsiReadCapacity_IN_SmhbaLUN_SIZE sizeof (ULONGLONG) +#define SM_ScsiReadCapacity_IN_SmhbaLUN_ID 4 + + // + UCHAR Cdb[16]; +#define SM_ScsiReadCapacity_IN_Cdb_SIZE sizeof (UCHAR[16]) +#define SM_ScsiReadCapacity_IN_Cdb_ID 5 + + // + ULONG InRespBufferMaxSize; +#define SM_ScsiReadCapacity_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_ScsiReadCapacity_IN_InRespBufferMaxSize_ID 6 + + // + ULONG InSenseBufferMaxSize; +#define SM_ScsiReadCapacity_IN_InSenseBufferMaxSize_SIZE sizeof (ULONG) +#define SM_ScsiReadCapacity_IN_InSenseBufferMaxSize_ID 7 + +} SM_ScsiReadCapacity_IN, *PSM_ScsiReadCapacity_IN; + +#define SM_ScsiReadCapacity_IN_SIZE \ + (FIELD_OFFSET(SM_ScsiReadCapacity_IN, InSenseBufferMaxSize) + \ + SM_ScsiReadCapacity_IN_InSenseBufferMaxSize_SIZE) + +typedef struct _SM_ScsiReadCapacity_OUT +{ + // + ULONG HBAStatus; +#define SM_ScsiReadCapacity_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_ScsiReadCapacity_OUT_HBAStatus_ID 8 + + // + UCHAR ScsiStatus; +#define SM_ScsiReadCapacity_OUT_ScsiStatus_SIZE sizeof (UCHAR) +#define SM_ScsiReadCapacity_OUT_ScsiStatus_ID 9 + + // + ULONG OutRespBufferSize; +#define SM_ScsiReadCapacity_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_ScsiReadCapacity_OUT_OutRespBufferSize_ID 10 + + // + ULONG OutSenseBufferSize; +#define SM_ScsiReadCapacity_OUT_OutSenseBufferSize_SIZE sizeof (ULONG) +#define SM_ScsiReadCapacity_OUT_OutSenseBufferSize_ID 11 + + // + UCHAR RespBuffer[1]; +#define SM_ScsiReadCapacity_OUT_RespBuffer_ID 12 + + // +// UCHAR SenseBuffer[1]; +#define SM_ScsiReadCapacity_OUT_SenseBuffer_ID 13 + +} SM_ScsiReadCapacity_OUT, *PSM_ScsiReadCapacity_OUT; + + +// MS_SM_FabricAndDomainManagementMethods - +// MS_SM_FabricAndDomainManagementMethods +#define MS_SM_FabricAndDomainManagementMethodsGuid \ + { 0x467fea10, 0x701b, 0x4388, \ + { 0x91, 0x7f, 0x73, 0x06, 0x20, 0xce, 0xa3, 0x28 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_FabricAndDomainManagementMethods_GUID, \ + 0x467fea10, 0x701b, 0x4388, 0x91, 0x7f, 0x73, 0x06, \ + 0x20, 0xce, 0xa3, 0x28); +#endif + +// +// Method id definitions for MS_SM_FabricAndDomainManagementMethods +#define SM_SendTEST 1 +typedef struct _SM_SendTEST_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendTEST_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendTEST_IN_HbaPortWWN_ID 1 + + // + UCHAR DestWWN[8]; +#define SM_SendTEST_IN_DestWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendTEST_IN_DestWWN_ID 2 + + // + ULONG DestFCID; +#define SM_SendTEST_IN_DestFCID_SIZE sizeof (ULONG) +#define SM_SendTEST_IN_DestFCID_ID 3 + + // + ULONG ReqBufferSize; +#define SM_SendTEST_IN_ReqBufferSize_SIZE sizeof (ULONG) +#define SM_SendTEST_IN_ReqBufferSize_ID 4 + + // + UCHAR ReqBuffer[1]; +#define SM_SendTEST_IN_ReqBuffer_ID 5 + +} SM_SendTEST_IN, *PSM_SendTEST_IN; + +typedef struct _SM_SendTEST_OUT +{ + // + ULONG HBAStatus; +#define SM_SendTEST_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendTEST_OUT_HBAStatus_ID 6 + +} SM_SendTEST_OUT, *PSM_SendTEST_OUT; + +#define SM_SendTEST_OUT_SIZE \ + (FIELD_OFFSET(SM_SendTEST_OUT, HBAStatus) + \ + SM_SendTEST_OUT_HBAStatus_SIZE) + +#define SM_SendECHO 2 +typedef struct _SM_SendECHO_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendECHO_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendECHO_IN_HbaPortWWN_ID 1 + + // + UCHAR DestWWN[8]; +#define SM_SendECHO_IN_DestWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendECHO_IN_DestWWN_ID 2 + + // + ULONG DestFCID; +#define SM_SendECHO_IN_DestFCID_SIZE sizeof (ULONG) +#define SM_SendECHO_IN_DestFCID_ID 3 + + // + ULONG InRespBufferMaxSize; +#define SM_SendECHO_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendECHO_IN_InRespBufferMaxSize_ID 4 + + // + ULONG ReqBufferSize; +#define SM_SendECHO_IN_ReqBufferSize_SIZE sizeof (ULONG) +#define SM_SendECHO_IN_ReqBufferSize_ID 5 + + // + UCHAR ReqBuffer[1]; +#define SM_SendECHO_IN_ReqBuffer_ID 6 + +} SM_SendECHO_IN, *PSM_SendECHO_IN; + +typedef struct _SM_SendECHO_OUT +{ + // + ULONG HBAStatus; +#define SM_SendECHO_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendECHO_OUT_HBAStatus_ID 7 + + // + ULONG OutRespBufferSize; +#define SM_SendECHO_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendECHO_OUT_OutRespBufferSize_ID 8 + + // + UCHAR RespBuffer[1]; +#define SM_SendECHO_OUT_RespBuffer_ID 9 + +} SM_SendECHO_OUT, *PSM_SendECHO_OUT; + +#define SM_SendSMPPassThru 3 +typedef struct _SM_SendSMPPassThru_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendSMPPassThru_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendSMPPassThru_IN_HbaPortWWN_ID 1 + + // + UCHAR DestPortWWN[8]; +#define SM_SendSMPPassThru_IN_DestPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendSMPPassThru_IN_DestPortWWN_ID 2 + + // + UCHAR DomainPortWWN[8]; +#define SM_SendSMPPassThru_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendSMPPassThru_IN_DomainPortWWN_ID 3 + + // + ULONG InRespBufferMaxSize; +#define SM_SendSMPPassThru_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendSMPPassThru_IN_InRespBufferMaxSize_ID 4 + + // + ULONG ReqBufferSize; +#define SM_SendSMPPassThru_IN_ReqBufferSize_SIZE sizeof (ULONG) +#define SM_SendSMPPassThru_IN_ReqBufferSize_ID 5 + + // + UCHAR ReqBuffer[1]; +#define SM_SendSMPPassThru_IN_ReqBuffer_ID 6 + +} SM_SendSMPPassThru_IN, *PSM_SendSMPPassThru_IN; + +typedef struct _SM_SendSMPPassThru_OUT +{ + // + ULONG HBAStatus; +#define SM_SendSMPPassThru_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendSMPPassThru_OUT_HBAStatus_ID 7 + + // + ULONG TotalRespBufferSize; +#define SM_SendSMPPassThru_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendSMPPassThru_OUT_TotalRespBufferSize_ID 8 + + // + ULONG OutRespBufferSize; +#define SM_SendSMPPassThru_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendSMPPassThru_OUT_OutRespBufferSize_ID 9 + + // + UCHAR RespBuffer[1]; +#define SM_SendSMPPassThru_OUT_RespBuffer_ID 10 + +} SM_SendSMPPassThru_OUT, *PSM_SendSMPPassThru_OUT; + +#define SM_SendCTPassThru 10 +typedef struct _SM_SendCTPassThru_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendCTPassThru_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendCTPassThru_IN_HbaPortWWN_ID 1 + + // + ULONG InRespBufferMaxSize; +#define SM_SendCTPassThru_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendCTPassThru_IN_InRespBufferMaxSize_ID 2 + + // + ULONG ReqBufferSize; +#define SM_SendCTPassThru_IN_ReqBufferSize_SIZE sizeof (ULONG) +#define SM_SendCTPassThru_IN_ReqBufferSize_ID 3 + + // + UCHAR ReqBuffer[1]; +#define SM_SendCTPassThru_IN_ReqBuffer_ID 4 + +} SM_SendCTPassThru_IN, *PSM_SendCTPassThru_IN; + +typedef struct _SM_SendCTPassThru_OUT +{ + // + ULONG HBAStatus; +#define SM_SendCTPassThru_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendCTPassThru_OUT_HBAStatus_ID 5 + + // + ULONG TotalRespBufferSize; +#define SM_SendCTPassThru_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendCTPassThru_OUT_TotalRespBufferSize_ID 6 + + // + ULONG OutRespBufferSize; +#define SM_SendCTPassThru_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendCTPassThru_OUT_OutRespBufferSize_ID 7 + + // + UCHAR RespBuffer[1]; +#define SM_SendCTPassThru_OUT_RespBuffer_ID 8 + +} SM_SendCTPassThru_OUT, *PSM_SendCTPassThru_OUT; + +#define SM_GetRNIDMgmtInfo 11 +typedef struct _SM_GetRNIDMgmtInfo_OUT +{ + // + ULONG HBAStatus; +#define SM_GetRNIDMgmtInfo_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_GetRNIDMgmtInfo_OUT_HBAStatus_ID 1 + + // + HBAFC3MgmtInfo MgmtInfo; +#define SM_GetRNIDMgmtInfo_OUT_MgmtInfo_SIZE sizeof (HBAFC3MgmtInfo) +#define SM_GetRNIDMgmtInfo_OUT_MgmtInfo_ID 2 + +} SM_GetRNIDMgmtInfo_OUT, *PSM_GetRNIDMgmtInfo_OUT; + +#define SM_GetRNIDMgmtInfo_OUT_SIZE \ + (FIELD_OFFSET(SM_GetRNIDMgmtInfo_OUT, MgmtInfo) + \ + SM_GetRNIDMgmtInfo_OUT_MgmtInfo_SIZE) + +#define SM_SetRNIDMgmtInfo 12 +typedef struct _SM_SetRNIDMgmtInfo_IN +{ + // + HBAFC3MgmtInfo MgmtInfo; +#define SM_SetRNIDMgmtInfo_IN_MgmtInfo_SIZE sizeof (HBAFC3MgmtInfo) +#define SM_SetRNIDMgmtInfo_IN_MgmtInfo_ID 1 + +} SM_SetRNIDMgmtInfo_IN, *PSM_SetRNIDMgmtInfo_IN; + +#define SM_SetRNIDMgmtInfo_IN_SIZE \ + (FIELD_OFFSET(SM_SetRNIDMgmtInfo_IN, MgmtInfo) + \ + SM_SetRNIDMgmtInfo_IN_MgmtInfo_SIZE) + +typedef struct _SM_SetRNIDMgmtInfo_OUT +{ + // + ULONG HBAStatus; +#define SM_SetRNIDMgmtInfo_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SetRNIDMgmtInfo_OUT_HBAStatus_ID 2 + +} SM_SetRNIDMgmtInfo_OUT, *PSM_SetRNIDMgmtInfo_OUT; + +#define SM_SetRNIDMgmtInfo_OUT_SIZE \ + (FIELD_OFFSET(SM_SetRNIDMgmtInfo_OUT, HBAStatus) + \ + SM_SetRNIDMgmtInfo_OUT_HBAStatus_SIZE) + +#define SM_SendRNID 13 +typedef struct _SM_SendRNID_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendRNID_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRNID_IN_HbaPortWWN_ID 1 + + // + UCHAR DestWWN[8]; +#define SM_SendRNID_IN_DestWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRNID_IN_DestWWN_ID 2 + + // + ULONG DestFCID; +#define SM_SendRNID_IN_DestFCID_SIZE sizeof (ULONG) +#define SM_SendRNID_IN_DestFCID_ID 3 + + // + ULONG NodeIdDataFormat; +#define SM_SendRNID_IN_NodeIdDataFormat_SIZE sizeof (ULONG) +#define SM_SendRNID_IN_NodeIdDataFormat_ID 4 + + // + ULONG InRespBufferMaxSize; +#define SM_SendRNID_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendRNID_IN_InRespBufferMaxSize_ID 5 + +} SM_SendRNID_IN, *PSM_SendRNID_IN; + +#define SM_SendRNID_IN_SIZE \ + (FIELD_OFFSET(SM_SendRNID_IN, InRespBufferMaxSize) + \ + SM_SendRNID_IN_InRespBufferMaxSize_SIZE) + +typedef struct _SM_SendRNID_OUT +{ + // + ULONG HBAStatus; +#define SM_SendRNID_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendRNID_OUT_HBAStatus_ID 6 + + // + ULONG TotalRespBufferSize; +#define SM_SendRNID_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendRNID_OUT_TotalRespBufferSize_ID 7 + + // + ULONG OutRespBufferSize; +#define SM_SendRNID_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendRNID_OUT_OutRespBufferSize_ID 8 + + // + UCHAR RespBuffer[1]; +#define SM_SendRNID_OUT_RespBuffer_ID 9 + +} SM_SendRNID_OUT, *PSM_SendRNID_OUT; + +#define SM_SendRPL 14 +typedef struct _SM_SendRPL_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendRPL_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRPL_IN_HbaPortWWN_ID 1 + + // + UCHAR AgentWWN[8]; +#define SM_SendRPL_IN_AgentWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRPL_IN_AgentWWN_ID 2 + + // + ULONG AgentDomain; +#define SM_SendRPL_IN_AgentDomain_SIZE sizeof (ULONG) +#define SM_SendRPL_IN_AgentDomain_ID 3 + + // + ULONG PortIndex; +#define SM_SendRPL_IN_PortIndex_SIZE sizeof (ULONG) +#define SM_SendRPL_IN_PortIndex_ID 4 + + // + ULONG InRespBufferMaxSize; +#define SM_SendRPL_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendRPL_IN_InRespBufferMaxSize_ID 5 + +} SM_SendRPL_IN, *PSM_SendRPL_IN; + +#define SM_SendRPL_IN_SIZE \ + (FIELD_OFFSET(SM_SendRPL_IN, InRespBufferMaxSize) + \ + SM_SendRPL_IN_InRespBufferMaxSize_SIZE) + +typedef struct _SM_SendRPL_OUT +{ + // + ULONG HBAStatus; +#define SM_SendRPL_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendRPL_OUT_HBAStatus_ID 6 + + // + ULONG TotalRespBufferSize; +#define SM_SendRPL_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendRPL_OUT_TotalRespBufferSize_ID 7 + + // + ULONG OutRespBufferSize; +#define SM_SendRPL_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendRPL_OUT_OutRespBufferSize_ID 8 + + // + UCHAR RespBuffer[1]; +#define SM_SendRPL_OUT_RespBuffer_ID 9 + +} SM_SendRPL_OUT, *PSM_SendRPL_OUT; + +#define SM_SendRPS 15 +typedef struct _SM_SendRPS_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendRPS_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRPS_IN_HbaPortWWN_ID 1 + + // + UCHAR AgentWWN[8]; +#define SM_SendRPS_IN_AgentWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRPS_IN_AgentWWN_ID 2 + + // + UCHAR ObjectWWN[8]; +#define SM_SendRPS_IN_ObjectWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRPS_IN_ObjectWWN_ID 3 + + // + ULONG AgentDomain; +#define SM_SendRPS_IN_AgentDomain_SIZE sizeof (ULONG) +#define SM_SendRPS_IN_AgentDomain_ID 4 + + // + ULONG ObjectPortNumber; +#define SM_SendRPS_IN_ObjectPortNumber_SIZE sizeof (ULONG) +#define SM_SendRPS_IN_ObjectPortNumber_ID 5 + + // + ULONG InRespBufferMaxSize; +#define SM_SendRPS_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendRPS_IN_InRespBufferMaxSize_ID 6 + +} SM_SendRPS_IN, *PSM_SendRPS_IN; + +#define SM_SendRPS_IN_SIZE \ + (FIELD_OFFSET(SM_SendRPS_IN, InRespBufferMaxSize) + \ + SM_SendRPS_IN_InRespBufferMaxSize_SIZE) + +typedef struct _SM_SendRPS_OUT +{ + // + ULONG HBAStatus; +#define SM_SendRPS_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendRPS_OUT_HBAStatus_ID 7 + + // + ULONG TotalRespBufferSize; +#define SM_SendRPS_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendRPS_OUT_TotalRespBufferSize_ID 8 + + // + ULONG OutRespBufferSize; +#define SM_SendRPS_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendRPS_OUT_OutRespBufferSize_ID 9 + + // + UCHAR RespBuffer[1]; +#define SM_SendRPS_OUT_RespBuffer_ID 10 + +} SM_SendRPS_OUT, *PSM_SendRPS_OUT; + +#define SM_SendSRL 16 +typedef struct _SM_SendSRL_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendSRL_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendSRL_IN_HbaPortWWN_ID 1 + + // + UCHAR WWN[8]; +#define SM_SendSRL_IN_WWN_SIZE sizeof (UCHAR[8]) +#define SM_SendSRL_IN_WWN_ID 2 + + // + ULONG Domain; +#define SM_SendSRL_IN_Domain_SIZE sizeof (ULONG) +#define SM_SendSRL_IN_Domain_ID 3 + + // + ULONG InRespBufferMaxSize; +#define SM_SendSRL_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendSRL_IN_InRespBufferMaxSize_ID 4 + +} SM_SendSRL_IN, *PSM_SendSRL_IN; + +#define SM_SendSRL_IN_SIZE \ + (FIELD_OFFSET(SM_SendSRL_IN, InRespBufferMaxSize) + \ + SM_SendSRL_IN_InRespBufferMaxSize_SIZE) + +typedef struct _SM_SendSRL_OUT +{ + // + ULONG HBAStatus; +#define SM_SendSRL_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendSRL_OUT_HBAStatus_ID 5 + + // + ULONG TotalRespBufferSize; +#define SM_SendSRL_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendSRL_OUT_TotalRespBufferSize_ID 6 + + // + ULONG OutRespBufferSize; +#define SM_SendSRL_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendSRL_OUT_OutRespBufferSize_ID 7 + + // + UCHAR RespBuffer[1]; +#define SM_SendSRL_OUT_RespBuffer_ID 8 + +} SM_SendSRL_OUT, *PSM_SendSRL_OUT; + +#define SM_SendLIRR 17 +typedef struct _SM_SendLIRR_IN +{ + // + UCHAR SourceWWN[8]; +#define SM_SendLIRR_IN_SourceWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendLIRR_IN_SourceWWN_ID 1 + + // + UCHAR DestWWN[8]; +#define SM_SendLIRR_IN_DestWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendLIRR_IN_DestWWN_ID 2 + + // + UCHAR Function; +#define SM_SendLIRR_IN_Function_SIZE sizeof (UCHAR) +#define SM_SendLIRR_IN_Function_ID 3 + + // + UCHAR Type; +#define SM_SendLIRR_IN_Type_SIZE sizeof (UCHAR) +#define SM_SendLIRR_IN_Type_ID 4 + + // + ULONG InRespBufferMaxSize; +#define SM_SendLIRR_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendLIRR_IN_InRespBufferMaxSize_ID 5 + +} SM_SendLIRR_IN, *PSM_SendLIRR_IN; + +#define SM_SendLIRR_IN_SIZE \ + (FIELD_OFFSET(SM_SendLIRR_IN, InRespBufferMaxSize) + \ + SM_SendLIRR_IN_InRespBufferMaxSize_SIZE) + +typedef struct _SM_SendLIRR_OUT +{ + // + ULONG HBAStatus; +#define SM_SendLIRR_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendLIRR_OUT_HBAStatus_ID 6 + + // + ULONG TotalRespBufferSize; +#define SM_SendLIRR_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendLIRR_OUT_TotalRespBufferSize_ID 7 + + // + ULONG OutRespBufferSize; +#define SM_SendLIRR_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendLIRR_OUT_OutRespBufferSize_ID 8 + + // + UCHAR RespBuffer[1]; +#define SM_SendLIRR_OUT_RespBuffer_ID 9 + +} SM_SendLIRR_OUT, *PSM_SendLIRR_OUT; + +#define SM_SendRLS 18 +typedef struct _SM_SendRLS_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_SendRLS_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRLS_IN_HbaPortWWN_ID 1 + + // + UCHAR DestWWN[8]; +#define SM_SendRLS_IN_DestWWN_SIZE sizeof (UCHAR[8]) +#define SM_SendRLS_IN_DestWWN_ID 2 + + // + ULONG InRespBufferMaxSize; +#define SM_SendRLS_IN_InRespBufferMaxSize_SIZE sizeof (ULONG) +#define SM_SendRLS_IN_InRespBufferMaxSize_ID 3 + +} SM_SendRLS_IN, *PSM_SendRLS_IN; + +#define SM_SendRLS_IN_SIZE \ + (FIELD_OFFSET(SM_SendRLS_IN, InRespBufferMaxSize) + \ + SM_SendRLS_IN_InRespBufferMaxSize_SIZE) + +typedef struct _SM_SendRLS_OUT +{ + // + ULONG HBAStatus; +#define SM_SendRLS_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_SendRLS_OUT_HBAStatus_ID 4 + + // + ULONG TotalRespBufferSize; +#define SM_SendRLS_OUT_TotalRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendRLS_OUT_TotalRespBufferSize_ID 5 + + // + ULONG OutRespBufferSize; +#define SM_SendRLS_OUT_OutRespBufferSize_SIZE sizeof (ULONG) +#define SM_SendRLS_OUT_OutRespBufferSize_ID 6 + + // + UCHAR RespBuffer[1]; +#define SM_SendRLS_OUT_RespBuffer_ID 7 + +} SM_SendRLS_OUT, *PSM_SendRLS_OUT; + + +// MS_SM_AdapterEvent - MS_SM_AdapterEvent +#define MS_SM_AdapterEventGuid \ + { 0x7944cf67, 0x697b, 0x4432, \ + { 0x95, 0x3e, 0x1f, 0xda, 0xda, 0x88, 0x43, 0x61 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_AdapterEvent_GUID, \ + 0x7944cf67, 0x697b, 0x4432, 0x95, 0x3e, 0x1f, 0xda, \ + 0xda, 0x88, 0x43, 0x61); +#endif + + +typedef struct _MS_SM_AdapterEvent +{ + // + ULONG EventType; +#define MS_SM_AdapterEvent_EventType_SIZE sizeof (ULONG) +#define MS_SM_AdapterEvent_EventType_ID 1 + + // + UCHAR PortWWN[8]; +#define MS_SM_AdapterEvent_PortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SM_AdapterEvent_PortWWN_ID 2 + +} MS_SM_AdapterEvent, *PMS_SM_AdapterEvent; + +#define MS_SM_AdapterEvent_SIZE \ + (FIELD_OFFSET(MS_SM_AdapterEvent, PortWWN) + \ + MS_SM_AdapterEvent_PortWWN_SIZE) + +// MS_SM_PortEvent - MS_SM_PortEvent +#define MS_SM_PortEventGuid \ + { 0x0f760256, 0x8fc6, 0x47ad, \ + { 0x9d, 0x2e, 0xf0, 0xd6, 0x98, 0x01, 0xde, 0x7c } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_PortEvent_GUID, \ + 0x0f760256, 0x8fc6, 0x47ad, 0x9d, 0x2e, 0xf0, 0xd6, \ + 0x98, 0x01, 0xde, 0x7c); +#endif + + +typedef struct _MS_SM_PortEvent +{ + // + ULONG EventType; +#define MS_SM_PortEvent_EventType_SIZE sizeof (ULONG) +#define MS_SM_PortEvent_EventType_ID 1 + + // + ULONG FabricPortId; +#define MS_SM_PortEvent_FabricPortId_SIZE sizeof (ULONG) +#define MS_SM_PortEvent_FabricPortId_ID 2 + + // + UCHAR PortWWN[8]; +#define MS_SM_PortEvent_PortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SM_PortEvent_PortWWN_ID 3 + +} MS_SM_PortEvent, *PMS_SM_PortEvent; + +#define MS_SM_PortEvent_SIZE \ + (FIELD_OFFSET(MS_SM_PortEvent, PortWWN) + MS_SM_PortEvent_PortWWN_SIZE) + +// MS_SM_TargetEvent - MS_SM_TargetEvent +#define MS_SM_TargetEventGuid \ + { 0x6e2d8b73, 0xf928, 0x4da9, \ + { 0xbd, 0xa1, 0xae, 0x54, 0x18, 0x9a, 0x38, 0x25 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_TargetEvent_GUID, \ + 0x6e2d8b73, 0xf928, 0x4da9, 0xbd, 0xa1, 0xae, 0x54, \ + 0x18, 0x9a, 0x38, 0x25); +#endif + + +typedef struct _MS_SM_TargetEvent +{ + // + ULONG EventType; +#define MS_SM_TargetEvent_EventType_SIZE sizeof (ULONG) +#define MS_SM_TargetEvent_EventType_ID 1 + + // + UCHAR PortWWN[8]; +#define MS_SM_TargetEvent_PortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SM_TargetEvent_PortWWN_ID 2 + + // + UCHAR DiscoveredPortWWN[8]; +#define MS_SM_TargetEvent_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SM_TargetEvent_DiscoveredPortWWN_ID 3 + + // + UCHAR DomainPortWWN[8]; +#define MS_SM_TargetEvent_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define MS_SM_TargetEvent_DomainPortWWN_ID 4 + +} MS_SM_TargetEvent, *PMS_SM_TargetEvent; + +#define MS_SM_TargetEvent_SIZE \ + (FIELD_OFFSET(MS_SM_TargetEvent, DomainPortWWN) + \ + MS_SM_TargetEvent_DomainPortWWN_SIZE) + +// MS_SM_EventControl - MS_SM_EventControl +#define MS_SM_EventControlGuid \ + { 0xd6145693, 0x5988, 0x457f, \ + { 0x85, 0x81, 0x9a, 0x01, 0x57, 0xb5, 0x86, 0x90 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MS_SM_EventControl_GUID, \ + 0xd6145693, 0x5988, 0x457f, 0x85, 0x81, 0x9a, 0x01, \ + 0x57, 0xb5, 0x86, 0x90); +#endif + +// +// Method id definitions for MS_SM_EventControl +#define SM_AddTarget 1 +typedef struct _SM_AddTarget_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_AddTarget_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_AddTarget_IN_HbaPortWWN_ID 1 + + // + UCHAR DiscoveredPortWWN[8]; +#define SM_AddTarget_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_AddTarget_IN_DiscoveredPortWWN_ID 2 + + // + UCHAR DomainPortWWN[8]; +#define SM_AddTarget_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_AddTarget_IN_DomainPortWWN_ID 3 + + // + ULONG AllTargets; +#define SM_AddTarget_IN_AllTargets_SIZE sizeof (ULONG) +#define SM_AddTarget_IN_AllTargets_ID 4 + +} SM_AddTarget_IN, *PSM_AddTarget_IN; + +#define SM_AddTarget_IN_SIZE \ + (FIELD_OFFSET(SM_AddTarget_IN, AllTargets) + \ + SM_AddTarget_IN_AllTargets_SIZE) + +typedef struct _SM_AddTarget_OUT +{ + // + ULONG HBAStatus; +#define SM_AddTarget_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_AddTarget_OUT_HBAStatus_ID 5 + +} SM_AddTarget_OUT, *PSM_AddTarget_OUT; + +#define SM_AddTarget_OUT_SIZE \ + (FIELD_OFFSET(SM_AddTarget_OUT, HBAStatus) + \ + SM_AddTarget_OUT_HBAStatus_SIZE) + +#define SM_RemoveTarget 2 +typedef struct _SM_RemoveTarget_IN +{ + // + UCHAR HbaPortWWN[8]; +#define SM_RemoveTarget_IN_HbaPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_RemoveTarget_IN_HbaPortWWN_ID 1 + + // + UCHAR DiscoveredPortWWN[8]; +#define SM_RemoveTarget_IN_DiscoveredPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_RemoveTarget_IN_DiscoveredPortWWN_ID 2 + + // + UCHAR DomainPortWWN[8]; +#define SM_RemoveTarget_IN_DomainPortWWN_SIZE sizeof (UCHAR[8]) +#define SM_RemoveTarget_IN_DomainPortWWN_ID 3 + + // + ULONG AllTargets; +#define SM_RemoveTarget_IN_AllTargets_SIZE sizeof (ULONG) +#define SM_RemoveTarget_IN_AllTargets_ID 4 + +} SM_RemoveTarget_IN, *PSM_RemoveTarget_IN; + +#define SM_RemoveTarget_IN_SIZE \ + (FIELD_OFFSET(SM_RemoveTarget_IN, AllTargets) + \ + SM_RemoveTarget_IN_AllTargets_SIZE) + +typedef struct _SM_RemoveTarget_OUT +{ + // + ULONG HBAStatus; +#define SM_RemoveTarget_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_RemoveTarget_OUT_HBAStatus_ID 5 + +} SM_RemoveTarget_OUT, *PSM_RemoveTarget_OUT; + +#define SM_RemoveTarget_OUT_SIZE \ + (FIELD_OFFSET(SM_RemoveTarget_OUT, HBAStatus) + \ + SM_RemoveTarget_OUT_HBAStatus_SIZE) + +#define SM_AddPort 3 +typedef struct _SM_AddPort_IN +{ + // + UCHAR PortWWN[8]; +#define SM_AddPort_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SM_AddPort_IN_PortWWN_ID 1 + + // + ULONG EventType; +#define SM_AddPort_IN_EventType_SIZE sizeof (ULONG) +#define SM_AddPort_IN_EventType_ID 2 + +} SM_AddPort_IN, *PSM_AddPort_IN; + +#define SM_AddPort_IN_SIZE \ + (FIELD_OFFSET(SM_AddPort_IN, EventType) + SM_AddPort_IN_EventType_SIZE) + +typedef struct _SM_AddPort_OUT +{ + // + ULONG HBAStatus; +#define SM_AddPort_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_AddPort_OUT_HBAStatus_ID 3 + +} SM_AddPort_OUT, *PSM_AddPort_OUT; + +#define SM_AddPort_OUT_SIZE \ + (FIELD_OFFSET(SM_AddPort_OUT, HBAStatus) + \ + SM_AddPort_OUT_HBAStatus_SIZE) + +#define SM_RemovePort 4 +typedef struct _SM_RemovePort_IN +{ + // + UCHAR PortWWN[8]; +#define SM_RemovePort_IN_PortWWN_SIZE sizeof (UCHAR[8]) +#define SM_RemovePort_IN_PortWWN_ID 1 + + // + ULONG EventType; +#define SM_RemovePort_IN_EventType_SIZE sizeof (ULONG) +#define SM_RemovePort_IN_EventType_ID 2 + +} SM_RemovePort_IN, *PSM_RemovePort_IN; + +#define SM_RemovePort_IN_SIZE \ + (FIELD_OFFSET(SM_RemovePort_IN, EventType) + \ + SM_RemovePort_IN_EventType_SIZE) + +typedef struct _SM_RemovePort_OUT +{ + // + ULONG HBAStatus; +#define SM_RemovePort_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_RemovePort_OUT_HBAStatus_ID 3 + +} SM_RemovePort_OUT, *PSM_RemovePort_OUT; + +#define SM_RemovePort_OUT_SIZE \ + (FIELD_OFFSET(SM_RemovePort_OUT, HBAStatus) + \ + SM_RemovePort_OUT_HBAStatus_SIZE) + +#define SM_AddLink 10 +typedef struct _SM_AddLink_OUT +{ + // + ULONG HBAStatus; +#define SM_AddLink_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_AddLink_OUT_HBAStatus_ID 1 + +} SM_AddLink_OUT, *PSM_AddLink_OUT; + +#define SM_AddLink_OUT_SIZE \ + (FIELD_OFFSET(SM_AddLink_OUT, HBAStatus) + \ + SM_AddLink_OUT_HBAStatus_SIZE) + +#define SM_RemoveLink 11 +typedef struct _SM_RemoveLink_OUT +{ + // + ULONG HBAStatus; +#define SM_RemoveLink_OUT_HBAStatus_SIZE sizeof (ULONG) +#define SM_RemoveLink_OUT_HBAStatus_ID 1 + +} SM_RemoveLink_OUT, *PSM_RemoveLink_OUT; + +#define SM_RemoveLink_OUT_SIZE \ + (FIELD_OFFSET(SM_RemoveLink_OUT, HBAStatus) + \ + SM_RemoveLink_OUT_HBAStatus_SIZE) + + +// MSFC_TM - MSFC_TM + +#endif // MS_SM_HBA_API + +#define MSFC_TMGuid \ + { 0x8cf4c7eb, 0xa286, 0x409d, \ + { 0x9e, 0xb9, 0x29, 0xd7, 0xe0, 0xe9, 0xf4, 0xfa } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(MSFC_TM_GUID, \ + 0x8cf4c7eb, 0xa286, 0x409d, 0x9e, 0xb9, 0x29, 0xd7, \ + 0xe0, 0xe9, 0xf4, 0xfa); +#endif + + +typedef struct _MSFC_TM +{ + // + ULONG tm_sec; +#define MSFC_TM_tm_sec_SIZE sizeof (ULONG) +#define MSFC_TM_tm_sec_ID 1 + + // + ULONG tm_min; +#define MSFC_TM_tm_min_SIZE sizeof (ULONG) +#define MSFC_TM_tm_min_ID 2 + + // + ULONG tm_hour; +#define MSFC_TM_tm_hour_SIZE sizeof (ULONG) +#define MSFC_TM_tm_hour_ID 3 + + // + ULONG tm_mday; +#define MSFC_TM_tm_mday_SIZE sizeof (ULONG) +#define MSFC_TM_tm_mday_ID 4 + + // + ULONG tm_mon; +#define MSFC_TM_tm_mon_SIZE sizeof (ULONG) +#define MSFC_TM_tm_mon_ID 5 + + // + ULONG tm_year; +#define MSFC_TM_tm_year_SIZE sizeof (ULONG) +#define MSFC_TM_tm_year_ID 6 + + // + ULONG tm_wday; +#define MSFC_TM_tm_wday_SIZE sizeof (ULONG) +#define MSFC_TM_tm_wday_ID 7 + + // + ULONG tm_yday; +#define MSFC_TM_tm_yday_SIZE sizeof (ULONG) +#define MSFC_TM_tm_yday_ID 8 + + // + ULONG tm_isdst; +#define MSFC_TM_tm_isdst_SIZE sizeof (ULONG) +#define MSFC_TM_tm_isdst_ID 9 + +} MSFC_TM, *PMSFC_TM; + +#define MSFC_TM_SIZE (FIELD_OFFSET(MSFC_TM, tm_isdst) + MSFC_TM_tm_isdst_SIZE) + +// GmDemoDriver - GmDemoDriver +// GmDemoDriver Schema +#define GmDemoDriverGuid \ + { 0x33168f61, 0x67a8, 0x408e, \ + { 0xb2, 0x62, 0x12, 0x40, 0xaa, 0xc0, 0x34, 0x47 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(GmDemoDriver_GUID, \ + 0x33168f61, 0x67a8, 0x408e, 0xb2, 0x62, 0x12, 0x40,\ + 0xaa, 0xc0, 0x34, 0x47); +#endif + + +typedef struct _GmDemoDriver +{ + // The Answer + ULONG TheAnswer; +#define GmDemoDriver_TheAnswer_SIZE sizeof (ULONG) +#define GmDemoDriver_TheAnswer_ID 1 + + // The Next Answer + ULONG TheNextAnswer; +#define GmDemoDriver_TheNextAnswer_SIZE sizeof (ULONG) +#define GmDemoDriver_TheNextAnswer_ID 2 + + // SRBs seen + ULONG SRBsSeen; +#define GmDemoDriver_SRBsSeen_SIZE sizeof (ULONG) +#define GmDemoDriver_SRBsSeen_ID 3 + + // WMI SRBs seen + ULONG WMISRBsSeen; +#define GmDemoDriver_WMISRBsSeen_SIZE sizeof (ULONG) +#define GmDemoDriver_WMISRBsSeen_ID 4 + +} GmDemoDriver, *PGmDemoDriver; + +#define GmDemoDriver_SIZE \ + (FIELD_OFFSET(GmDemoDriver, WMISRBsSeen) + \ + GmDemoDriver_WMISRBsSeen_SIZE) + +// GmDemoDriver2 - GmDemoDriver2 +// GmDemoDriver Schema2 +#define GmDemoDriver2Guid \ + { 0x33168f62, 0x67a8, 0x408e, \ + { 0xb2, 0x62, 0x12, 0x40, 0xaa, 0xc0, 0x34, 0x47 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(GmDemoDriver2_GUID, \ + 0x33168f62, 0x67a8, 0x408e, 0xb2, 0x62, 0x12, 0x40, \ + 0xaa, 0xc0, 0x34, 0x47); +#endif + + +typedef struct _GmDemoDriver2 +{ + // Number of array elements + ULONG NumberElements; +#define GmDemoDriver2_NumberElements_SIZE sizeof (ULONG) +#define GmDemoDriver2_NumberElements_ID 1 + + // The array + ULONG UlongArray[1]; +#define GmDemoDriver2_UlongArray_ID 2 + +} GmDemoDriver2, *PGmDemoDriver2; + +// GmDemoDriverSrbActivity - GmDemoDriverSrbActivity +// Performance counter class that keeps counts of SRBs +#define GmDemoDriverSrbActivityGuid \ + { 0x33168f63, 0x67a8, 0x408e, \ + { 0xb2, 0x62, 0x12, 0x40, 0xaa, 0xc0, 0x34, 0x47 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(GmDemoDriverSrbActivity_GUID, \ + 0x33168f63, 0x67a8, 0x408e, 0xb2, 0x62, 0x12, 0x40, \ + 0xaa, 0xc0, 0x34, 0x47); +#endif + + +typedef struct _GmDemoDriverSrbActivity +{ + // Count of CREATE SRBs received + ULONG TotalCreateSRBs; +#define GmDemoDriverSrbActivity_TotalCreateSRBs_SIZE sizeof (ULONG) +#define GmDemoDriverSrbActivity_TotalCreateSRBs_ID 1 + + // Count of CLOSE SRBs received + ULONG TotalCloseSRBs; +#define GmDemoDriverSrbActivity_TotalCloseSRBs_SIZE sizeof (ULONG) +#define GmDemoDriverSrbActivity_TotalCloseSRBs_ID 2 + + // Count of IOCTL SRBs received + ULONG TotalIoCtlSrbs; +#define GmDemoDriverSrbActivity_TotalIoCtlSrbs_SIZE sizeof (ULONG) +#define GmDemoDriverSrbActivity_TotalIoCtlSrbs_ID 3 + +} GmDemoDriverSrbActivity, *PGmDemoDriverSrbActivity; + +#define GmDemoDriverSrbActivity_SIZE \ + (FIELD_OFFSET(GmDemoDriverSrbActivity, TotalIoCtlSrbs) + \ + GmDemoDriverSrbActivity_TotalIoCtlSrbs_SIZE) + +// GmDrvDrvMethod - GmDrvDrvMethod +// WMI method +#define GmDrvDrvMethodGuid \ + { 0x33168f64, 0x67a8, 0x408e, \ + { 0xb2, 0x62, 0x12, 0x40, 0xaa, 0xc0, 0x34, 0x47 } } + +#if ! (defined(MIDL_PASS)) +DEFINE_GUID(GmDrvDrvMethod_GUID, \ + 0x33168f64, 0x67a8, 0x408e, 0xb2, 0x62, 0x12, 0x40, \ + 0xaa, 0xc0, 0x34, 0x47); +#endif + +// +// Method id definitions for GmDrvDrvMethod +#define GmDrvDemoMethod1 1 +typedef struct _GmDrvDemoMethod1_IN +{ + // + ULONG inDatum; +#define GmDrvDemoMethod1_IN_inDatum_SIZE sizeof (ULONG) +#define GmDrvDemoMethod1_IN_inDatum_ID 1 + +} GmDrvDemoMethod1_IN, *PGmDrvDemoMethod1_IN; + +#define GmDrvDemoMethod1_IN_SIZE \ + (FIELD_OFFSET(GmDrvDemoMethod1_IN, inDatum) + \ + GmDrvDemoMethod1_IN_inDatum_SIZE) + +typedef struct _GmDrvDemoMethod1_OUT +{ + // + ULONG outDatum; +#define GmDrvDemoMethod1_OUT_outDatum_SIZE sizeof (ULONG) +#define GmDrvDemoMethod1_OUT_outDatum_ID 2 + +} GmDrvDemoMethod1_OUT, *PGmDrvDemoMethod1_OUT; + +#define GmDrvDemoMethod1_OUT_SIZE \ + (FIELD_OFFSET(GmDrvDemoMethod1_OUT, outDatum) + \ + GmDrvDemoMethod1_OUT_outDatum_SIZE) + +#define GmDrvDemoMethod2 2 +typedef struct _GmDrvDemoMethod2_IN +{ + // + ULONG inDatum1; +#define GmDrvDemoMethod2_IN_inDatum1_SIZE sizeof (ULONG) +#define GmDrvDemoMethod2_IN_inDatum1_ID 1 + + // + ULONG inDatum2; +#define GmDrvDemoMethod2_IN_inDatum2_SIZE sizeof (ULONG) +#define GmDrvDemoMethod2_IN_inDatum2_ID 2 + +} GmDrvDemoMethod2_IN, *PGmDrvDemoMethod2_IN; + +#define GmDrvDemoMethod2_IN_SIZE \ + (FIELD_OFFSET(GmDrvDemoMethod2_IN, inDatum2) + \ + GmDrvDemoMethod2_IN_inDatum2_SIZE) + +typedef struct _GmDrvDemoMethod2_OUT +{ + // + ULONG outDatum1; +#define GmDrvDemoMethod2_OUT_outDatum1_SIZE sizeof (ULONG) +#define GmDrvDemoMethod2_OUT_outDatum1_ID 3 + +} GmDrvDemoMethod2_OUT, *PGmDrvDemoMethod2_OUT; + +#define GmDrvDemoMethod2_OUT_SIZE \ + (FIELD_OFFSET(GmDrvDemoMethod2_OUT, outDatum1) + \ + GmDrvDemoMethod2_OUT_outDatum1_SIZE) + +#define GmDrvDemoMethod3 3 +typedef struct _GmDrvDemoMethod3_IN +{ + // + ULONG inDatum1; +#define GmDrvDemoMethod3_IN_inDatum1_SIZE sizeof (ULONG) +#define GmDrvDemoMethod3_IN_inDatum1_ID 1 + + // + ULONG inDatum2; +#define GmDrvDemoMethod3_IN_inDatum2_SIZE sizeof (ULONG) +#define GmDrvDemoMethod3_IN_inDatum2_ID 2 + +} GmDrvDemoMethod3_IN, *PGmDrvDemoMethod3_IN; + +#define GmDrvDemoMethod3_IN_SIZE \ + (FIELD_OFFSET(GmDrvDemoMethod3_IN, inDatum2) + \ + GmDrvDemoMethod3_IN_inDatum2_SIZE) + +typedef struct _GmDrvDemoMethod3_OUT +{ + // + ULONG outDatum1; +#define GmDrvDemoMethod3_OUT_outDatum1_SIZE sizeof (ULONG) +#define GmDrvDemoMethod3_OUT_outDatum1_ID 3 + + // + ULONG outDatum2; +#define GmDrvDemoMethod3_OUT_outDatum2_SIZE sizeof (ULONG) +#define GmDrvDemoMethod3_OUT_outDatum2_ID 4 + +} GmDrvDemoMethod3_OUT, *PGmDrvDemoMethod3_OUT; + +#define GmDrvDemoMethod3_OUT_SIZE \ + (FIELD_OFFSET(GmDrvDemoMethod3_OUT, outDatum2) + \ + GmDrvDemoMethod3_OUT_outDatum2_SIZE) + + +#endif diff --git a/include/os/windows/zfs/sys/zconf.h b/include/os/windows/zfs/sys/zconf.h new file mode 100644 index 000000000000..27bf81752514 --- /dev/null +++ b/include/os/windows/zfs/sys/zconf.h @@ -0,0 +1,119 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _ZCONF_H +#define _ZCONF_H + +#pragma ident "%Z%%M% %I% %E% SMI" + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/* + * We don't want to turn on zlib's debugging. + */ +#undef DEBUG + +/* + * We define our own memory allocation and deallocation routines that use kmem. + */ +#ifndef MY_ZCALLOC +#define MY_ZCALLOC +#endif + +/* + * We don't define HAVE_MEMCPY here, but do in zutil.c, and implement our + * our versions of zmemcpy(), zmemzero(), and zmemcmp(). + */ + +/* + * We have a sufficiently capable compiler as to not need zlib's compiler hack. + */ +#define NO_DUMMY_DECL + +#define compressBound(len) (len + (len >> 12) + (len >> 14) + 11) + +#define z_off_t off_t +#define OF(p) p +#define ZEXTERN extern +#define ZEXPORT +#define ZEXPORTVA +#define FAR + +#define deflateInit_ z_deflateInit_ +#define deflate z_deflate +#define deflateEnd z_deflateEnd +#define inflateInit_ z_inflateInit_ +#define inflate z_inflate +#define inflateEnd z_inflateEnd +#define deflateInit2_ z_deflateInit2_ +#define deflateSetDictionary z_deflateSetDictionary +#define deflateCopy z_deflateCopy +#define deflateReset z_deflateReset +#define deflateParams z_deflateParams +#define deflateBound z_deflateBound +#define deflatePrime z_deflatePrime +#define inflateInit2_ z_inflateInit2_ +#define inflateSetDictionary z_inflateSetDictionary +#define inflateSync z_inflateSync +#define inflateSyncPoint z_inflateSyncPoint +#define inflateCopy z_inflateCopy +#define inflateReset z_inflateReset +#define inflateBack z_inflateBack +#define inflateBackEnd z_inflateBackEnd +#define compress zz_compress +// #define compress2 zz_compress2 +// #define uncompress zz_uncompress +#define adler32 z_adler32 +#define crc32 z_crc32 +#define get_crc_table z_get_crc_table +#define zError z_zError + +#define MAX_MEM_LEVEL 9 +#define MAX_WBITS 15 + +typedef unsigned char Byte; +typedef unsigned int uInt; +typedef unsigned long uLong; +typedef Byte Bytef; +typedef char charf; +typedef int intf; +typedef uInt uIntf; +typedef uLong uLongf; +typedef void *voidpc; +typedef void *voidpf; +typedef void *voidp; + +#ifdef __cplusplus +} +#endif + +#endif /* _ZCONF_H */ diff --git a/include/os/windows/zfs/sys/zfs_bootenv_os.h b/include/os/windows/zfs/sys/zfs_bootenv_os.h new file mode 100644 index 000000000000..76907d921bce --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_bootenv_os.h @@ -0,0 +1,25 @@ +/* + * This file and its contents are supplied under the terms of the + * Common Development and Distribution License ("CDDL"), version 1.0. + * You may only use this file in accordance with the terms of version + * 1.0 of the CDDL. + * + * A full copy of the text of the CDDL should have accompanied this + * source. A copy of the CDDL is also available via the Internet at + * http://www.illumos.org/license/CDDL. + */ + +#ifndef _ZFS_BOOTENV_OS_H +#define _ZFS_BOOTENV_OS_H + +#ifdef __cplusplus +extern "C" { +#endif + +#define BOOTENV_OS BE_WINDOWS_VENDOR + +#ifdef __cplusplus +} +#endif + +#endif /* _ZFS_BOOTENV_OS_H */ diff --git a/include/os/windows/zfs/sys/zfs_context_os.h b/include/os/windows/zfs/sys/zfs_context_os.h new file mode 100644 index 000000000000..0a26bc0dda5c --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_context_os.h @@ -0,0 +1,219 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* Copyright(c) 2015 Jorgen Lundman */ + +#ifndef _SPL_ZFS_CONTEXT_OS_H +#define _SPL_ZFS_CONTEXT_OS_H + +#include +#include +#include + +#define ZIO_OS_FIELDS \ + struct { \ + IRP *irp; \ + void *b_addr; \ + IO_STATUS_BLOCK IoStatus; \ + PIO_WORKITEM work_item; \ + } windows; + + +#define MSEC_TO_TICK(msec) ((msec) / (MILLISEC / hz)) + +#define KMALLOC_MAX_SIZE (128 * 1024) // Win32 MAXPHYS ? + +#define MNTTYPE_ZFS_SUBTYPE ('Z'<<24|'F'<<16|'S'<<8) + +#ifndef MAX_UPL_TRANSFER +#define MAX_UPL_TRANSFER 256 +#endif + +#define flock64_t struct flock + +struct spa_iokit; +typedef struct spa_iokit spa_iokit_t; + +#define noinline __attribute__((noinline)) + +/* really? */ +#define kpreempt_disable() ((void)0) +#define kpreempt_enable() ((void)0) +#define cond_resched() (void)YieldProcessor() +#define schedule() (void)YieldProcessor() + +#define current curthread + +#define vmem_alloc(A, B) zfs_kmem_alloc((A), (B)) + +extern boolean_t ml_set_interrupts_enabled(boolean_t); +extern PDRIVER_OBJECT WIN_DriverObject; + +/* + * Ok this is pretty gross - until we can get rid of it from lua - + * it works as long as it doesn't parse strings + */ +#define sscanf sscanf_s + +#define DIRENT_RECLEN(namelen) (((namelen) + 7) & ~7) + +/* Make sure kmem and vmem are already included */ +#include +#include + +#include + +typedef int fstrans_cookie_t; +#define spl_fstrans_mark() (0) +#define spl_fstrans_unmark(x) (x = 0) + +// "zfs send" will try to use a new thread to send, which is +// not allowed (thread can't use HANDLE from userland unless +// it is exactly the same process). Set this here, to call +// "zfs send" directly. +#define HAVE_LARGE_STACKS 1 + + + +#ifdef _KERNEL + +struct hlist_node { + struct hlist_node *next, **pprev; +}; + +struct hlist_head { + struct hlist_node *first; +}; + +typedef struct { + volatile int counter; +} atomic_t; + +#define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x)) + +#define barrier() __asm__ __volatile__("": : :"memory") +#define smp_rmb() barrier() + +#define READ_ONCE(x) ( \ +{ \ + __typeof(x) __var = ( \ + { \ + barrier(); \ + ACCESS_ONCE(x); \ + }); \ + barrier(); \ + __var; \ + }) + +#define WRITE_ONCE(x, v) do { \ + barrier(); \ + ACCESS_ONCE(x) = (v); \ + barrier(); \ + } while (0) + +/* BEGIN CSTYLED */ +#define hlist_for_each(p, head) \ + for (p = (head)->first; p; p = (p)->next) + +#define hlist_entry(ptr, type, field) container_of(ptr, type, field) +/* END CSTYLED */ + +static inline void +hlist_add_head(struct hlist_node *n, struct hlist_head *h) +{ + n->next = h->first; + if (h->first != NULL) + h->first->pprev = &n->next; + WRITE_ONCE(h->first, n); + n->pprev = &h->first; +} + +static inline void +hlist_del(struct hlist_node *n) +{ + WRITE_ONCE(*(n->pprev), n->next); + if (n->next != NULL) + n->next->pprev = n->pprev; +} + + +#define HLIST_HEAD_INIT { } +#define HLIST_HEAD(name) struct hlist_head name = HLIST_HEAD_INIT +#define INIT_HLIST_HEAD(head) (head)->first = NULL + +/* BEGIN CSTYLED */ +#define INIT_HLIST_NODE(node) \ + do { \ + (node)->next = NULL; \ + (node)->pprev = NULL; \ + } while (0) + +/* END CSTYLED */ + +static inline int +atomic_read(const atomic_t *v) +{ + return (READ_ONCE(v->counter)); +} + +static inline int +atomic_inc(atomic_t *v) +{ + return (__sync_fetch_and_add(&v->counter, 1) + 1); +} + +static inline int +atomic_dec(atomic_t *v) +{ + return (__sync_fetch_and_add(&v->counter, -1) - 1); +} + +extern void kx_qsort(void *array, size_t nm, size_t member_size, + int (*cmpf)(const void *, const void *)); +// #define qsort kx_qsort + +#define strstr kmem_strstr + + +#define task_io_account_read(n) +#define task_io_account_write(n) + +#ifndef SEEK_HOLE +#define SEEK_HOLE 3 +#endif + +#ifndef SEEK_DATA +#define SEEK_DATA 4 +#endif + +#endif // _KERNEL + +#define FSCTL_ZFS_VOLUME_MOUNTPOINT CTL_CODE(FILE_DEVICE_UNKNOWN, \ + 0x8ff, METHOD_BUFFERED, FILE_ANY_ACCESS) +typedef struct { + int len; + WCHAR buffer[1]; // make this dynamic? +} fsctl_zfs_volume_mountpoint_t; + + + +#endif diff --git a/include/os/windows/zfs/sys/zfs_ctldir.h b/include/os/windows/zfs/sys/zfs_ctldir.h new file mode 100644 index 000000000000..7d077c5a424f --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_ctldir.h @@ -0,0 +1,195 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (C) 2011 Lawrence Livermore National Security, LLC. + * Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). + * LLNL-CODE-403049. + * Rewritten for Linux by: + * Rohan Puri + * Brian Behlendorf + * Copyright(c) 2022 Jorgen Lundman + */ + +#ifndef _ZFS_CTLDIR_H +#define _ZFS_CTLDIR_H + +#include +#include +#include +#include + +#define ZFS_CTLDIR_NAME ".zfs" +#define ZFS_SNAPDIR_NAME "snapshot" +#define ZFS_SHAREDIR_NAME "shares" + +#define zfs_has_ctldir(zdp) \ + ((zdp)->z_id == ZTOZSB(zdp)->z_root && \ + (ZTOZSB(zdp)->z_ctldir != NULL)) +#define zfs_show_ctldir(zdp) \ + (zfs_has_ctldir(zdp) && \ + (ZTOZSB(zdp)->z_show_ctldir)) + +struct path; + +extern int zfs_expire_snapshot; + +// Fix me Windows. - we don't want these, remove them +// once we Windowsify this file. +struct vnop_readdir_args { + struct vnode *a_vp; + struct uio *a_uio; + int a_flags; + int *a_eofflag; + int *a_numdirent; +}; + +struct vnop_getattr_args { + struct vnode *a_vp; + struct vnode_vattr *a_vap; +}; + +struct vnop_open_args { + struct vnode *a_vp; + int a_mode; +}; + +struct vnop_close_args { + struct vnode *a_vp; + int a_fflag; +}; + +struct vnop_access_args { + struct vnodeop_desc *a_desc; + struct vnode a_vp; + int a_action; +}; + +struct vnop_lookup_args { + struct vnode *a_dvp; + struct vnode **a_vpp; + struct componentname *a_cnp; +}; + +struct vnop_mkdir_args { + struct vnode *a_dvp; + struct vnode **a_vpp; + struct componentname *a_cnp; + struct vnode_vattr *a_vap; +}; + +struct vnop_rmdir_args { + struct vnode *a_dvp; + struct vnode *a_vp; + struct componentname *a_cnp; +}; + +struct vnop_reclaim_args { + struct vnode *a_vp; +}; + +struct vnop_inactive_args { + struct vnode *a_vp; +}; + + +/* zfsctl generic functions */ +extern int zfsctl_create(zfsvfs_t *); +extern void zfsctl_destroy(zfsvfs_t *); +extern struct vnode *zfsctl_root(znode_t *); +extern void zfsctl_init(void); +extern void zfsctl_fini(void); +extern boolean_t zfsctl_is_node(znode_t *zp); +extern boolean_t zfsctl_is_leafnode(znode_t *zp); + +extern boolean_t zfsctl_is_snapdir(struct vnode *ip); +extern int zfsctl_fid(struct vnode *ip, fid_t *fidp); + +/* zfsctl '.zfs' functions */ +extern int zfsctl_root_lookup(struct vnode *dip, char *name, + znode_t **zpp, int flags, cred_t *cr, int *direntflags, + struct componentname *realpnp); +extern struct vnode *zfs_root_dotdot(struct vnode *vp); + +/* zfsctl '.zfs/snapshot' functions */ +extern int zfsctl_snapdir_lookup(struct vnode *dip, char *name, + struct vnode **ipp, int flags, cred_t *cr, int *direntflags, + struct componentname *realpnp); +extern int zfsctl_snapdir_rename(struct vnode *sdip, char *sname, + struct vnode *tdip, char *tname, cred_t *cr, int flags); +extern int zfsctl_snapdir_remove(struct vnode *dip, char *name, cred_t *cr, + int flags); +extern int zfsctl_snapdir_mkdir(struct vnode *dip, char *dirname, vattr_t *vap, + struct vnode **ipp, cred_t *cr, int flags); +extern int zfsctl_snapshot_mount(struct vnode *, int flags); +extern int zfsctl_snapshot_unmount(const char *, int flags); +extern int zfsctl_snapshot_unmount_node(struct vnode *, const char *, + int flags); +extern int zfsctl_snapshot_unmount_delay(spa_t *spa, uint64_t objsetid, + int delay); +extern int zfsctl_snapdir_vget(struct mount *sb, uint64_t objsetid, + int gen, struct vnode **ipp); +extern int zfsctl_mkdir(znode_t *dzp, znode_t **zpp, char *dirname); +extern int zfsctl_set_reparse_point(znode_t *zp, REPARSE_DATA_BUFFER *rdb, + size_t size); +extern int zfsctl_delete_reparse_point(znode_t *zp); +extern ULONG zfsctl_get_reparse_tag(znode_t *zp); +extern int zfsctl_get_reparse_point(znode_t *zp, REPARSE_DATA_BUFFER **buffer, + size_t *size); + + +/* zfsctl '.zfs/shares' functions */ +extern int zfsctl_shares_lookup(struct vnode *dip, char *name, + struct vnode **ipp, int flags, cred_t *cr, int *direntflags, + struct componentname *realpnp); + +extern int zfsctl_vnop_lookup(struct vnop_lookup_args *); +extern int zfsctl_vnop_getattr(struct vnop_getattr_args *); +extern int zfsctl_vnop_readdir(struct vnop_readdir_args *); +extern int zfsctl_vnop_mkdir(struct vnop_mkdir_args *); +extern int zfsctl_vnop_rmdir(struct vnop_rmdir_args *); +extern int zfsctl_vnop_access(struct vnop_access_args *); +extern int zfsctl_vnop_open(struct vnop_open_args *); +extern int zfsctl_vnop_close(struct vnop_close_args *); +extern int zfsctl_vnop_inactive(struct vnop_inactive_args *); +extern int zfsctl_vnop_reclaim(struct vnop_reclaim_args *); + +extern void zfs_ereport_snapshot_post(const char *subclass, spa_t *spa, + const char *name); + +extern void zfsctl_mount_signal(zfsvfs_t *zfsvfs, char *, boolean_t); + + +/* + * These vnodes numbers are reserved for the .zfs control directory. + * It is important that they be no larger that 48-bits because only + * 6 bytes are reserved in the NFS file handle for the object number. + * However, they should be as large as possible to avoid conflicts + * with the objects which are assigned monotonically by the dmu. + */ +#define ZFSCTL_INO_ROOT 0x0000FFFFFFFFFFFFULL +#define ZFSCTL_INO_SHARES 0x0000FFFFFFFFFFFEULL +#define ZFSCTL_INO_SNAPDIR 0x0000FFFFFFFFFFFDULL +#define ZFSCTL_INO_SNAPDIRS 0x0000FFFFFFFFFFFCULL + +#define ZFSCTL_EXPIRE_SNAPSHOT 300 + +#endif /* _ZFS_CTLDIR_H */ diff --git a/include/os/windows/zfs/sys/zfs_dir.h b/include/os/windows/zfs/sys/zfs_dir.h new file mode 100644 index 000000000000..cfee82308a7d --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_dir.h @@ -0,0 +1,82 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SYS_FS_ZFS_DIR_H +#define _SYS_FS_ZFS_DIR_H + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/* zfs_dirent_lock() flags */ +#define ZNEW 0x0001 /* entry should not exist */ +#define ZEXISTS 0x0002 /* entry should exist */ +#define ZSHARED 0x0004 /* shared access (zfs_dirlook()) */ +#define ZXATTR 0x0008 /* we want the xattr dir */ +#define ZRENAMING 0x0010 /* znode is being renamed */ +#define ZCILOOK 0x0020 /* case-insensitive lookup requested */ +#define ZCIEXACT 0x0040 /* c-i requires c-s match (rename) */ +#define ZHAVELOCK 0x0080 /* z_name_lock is already held */ + +/* mknode flags */ +#define IS_ROOT_NODE 0x01 /* create a root node */ +#define IS_XATTR 0x02 /* create an extended attribute node */ +#define IS_REPLAY 0x04 /* we are replaying intent log */ + +extern int zfs_dirent_lock(zfs_dirlock_t **dlpp, znode_t *dzp, char *name, + znode_t **zpp, int flag, int *direntflags, + struct componentname *realpnp); + +extern void zfs_dirent_unlock(zfs_dirlock_t *); +extern int zfs_link_create(zfs_dirlock_t *, znode_t *, dmu_tx_t *, int); +extern int zfs_link_destroy(zfs_dirlock_t *, znode_t *, dmu_tx_t *, int, + boolean_t *); + +extern int zfs_dirlook(znode_t *, char *name, znode_t **, int, + int *deflg, struct componentname *rpnp); + +extern void zfs_mknode(znode_t *dzp, vattr_t *vap, dmu_tx_t *tx, cred_t *cr, + uint_t flag, znode_t **zpp, zfs_acl_ids_t *acl_ids); + +extern void zfs_rmnode(znode_t *); +extern void zfs_dl_name_switch(zfs_dirlock_t *dl, char *new, char **old); +extern boolean_t zfs_dirempty(znode_t *); +extern void zfs_unlinked_add(znode_t *, dmu_tx_t *); +extern void zfs_unlinked_drain(zfsvfs_t *zfsvfs); +extern void zfs_unlinked_drain_stop_wait(zfsvfs_t *zfsvfs); +extern int zfs_sticky_remove_access(znode_t *, znode_t *, cred_t *cr); + +extern int zfs_get_xattrdir(znode_t *, znode_t **, cred_t *, int); +extern int zfs_make_xattrdir(znode_t *, vattr_t *, znode_t **, cred_t *); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_FS_ZFS_DIR_H */ diff --git a/include/os/windows/zfs/sys/zfs_ioctl_compat.h b/include/os/windows/zfs/sys/zfs_ioctl_compat.h new file mode 100644 index 000000000000..b68e01455670 --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_ioctl_compat.h @@ -0,0 +1,217 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2013 Jorgen Lundan . + */ + +#ifndef _SYS_ZFS_IOCTL_COMPAT_H +#define _SYS_ZFS_IOCTL_COMPAT_H + +#include +#include +#include +#include +#include + +#ifdef _KERNEL +#include +#endif /* _KERNEL */ + +#ifdef __cplusplus +extern "C" { +#endif + +#define ZFSIOCTL_TYPE 40000 +#define ZFSIOCTL_BASE 0x800 + +#define DEVICE_FUNCTION_FROM_CTL_CODE(X) (((X)>>2) & 0x1fff) + +/* + * Backwards ioctl compatibility + */ + +/* ioctl versions for vfs.zfs.version.ioctl */ +#define ZFS_IOCVER_UNDEF -1 +#define ZFS_IOCVER_NONE 0 +#define ZFS_IOCVER_1_9_4 1 +#define ZFS_IOCVER_ZOF 15 + +/* compatibility conversion flag */ +#define ZFS_CMD_COMPAT_NONE 0 +#define ZFS_CMD_COMPAT_V15 1 +#define ZFS_CMD_COMPAT_V28 2 + +#define ZFS_IOC_COMPAT_PASS 254 +#define ZFS_IOC_COMPAT_FAIL 255 + +#define ZFS_IOCREQ(ioreq) ((ioreq) & 0xff) + +typedef struct zfs_iocparm { + uint32_t zfs_ioctl_version; + uint64_t zfs_cmd; + uint64_t zfs_cmd_size; + + /* + * ioctl() return codes can not be used to communicate - + * as Windows will skip copyout() if there is an error, so it + * is passed along in this wrapping structure. + */ + int zfs_ioc_error; /* ioctl error value */ +} zfs_iocparm_t; + +typedef struct zfs_cmd_1_9_4 +{ + char zc_name[MAXPATHLEN]; /* name of pool or dataset */ + uint64_t zc_nvlist_src; /* really (char *) */ + uint64_t zc_nvlist_src_size; + uint64_t zc_nvlist_dst; /* really (char *) */ + uint64_t zc_nvlist_dst_size; + boolean_t zc_nvlist_dst_filled; /* put an nvlist in dst? */ + int zc_pad2; + + /* + * The following members are for legacy ioctls which haven't been + * converted to the new method. + */ + uint64_t zc_history; /* really (char *) */ + char zc_value[MAXPATHLEN * 2]; + char zc_string[MAXNAMELEN]; + uint64_t zc_guid; + uint64_t zc_nvlist_conf; /* really (char *) */ + uint64_t zc_nvlist_conf_size; + uint64_t zc_cookie; + uint64_t zc_objset_type; + uint64_t zc_perm_action; + uint64_t zc_history_len; + uint64_t zc_history_offset; + uint64_t zc_obj; + uint64_t zc_iflags; /* internal to zfs(7fs) */ + zfs_share_t zc_share; + dmu_objset_stats_t zc_objset_stats; + struct drr_begin zc_begin_record; + zinject_record_t zc_inject_record; + uint32_t zc_defer_destroy; + uint32_t zc_flags; + uint64_t zc_action_handle; + int zc_cleanup_fd; + uint8_t zc_simple; + uint8_t zc_pad3[3]; + boolean_t zc_resumable; + uint32_t zc_pad4; + uint64_t zc_sendobj; + uint64_t zc_fromobj; + uint64_t zc_createtxg; + zfs_stat_t zc_stat; + int zc_ioc_error; /* ioctl error value */ + uint64_t zc_dev; /* OSX doesn't have ddi_driver_major */ +} zfs_cmd_1_9_4_t; + +// Figure this out +unsigned static long zfs_ioctl_1_9_4[] = +{ + // ZFS_IOC_POOL_CREATE = _IOWR('Z', 0, struct zfs_cmd), + + 0, /* 0 ZFS_IOC_POOL_CREATE */ + 1, /* 1 ZFS_IOC_POOL_DESTROY */ + 2, /* 2 ZFS_IOC_POOL_IMPORT */ + 3, /* 3 ZFS_IOC_POOL_EXPORT */ + 4, /* 4 ZFS_IOC_POOL_CONFIGS */ + 5, /* 5 ZFS_IOC_POOL_STATS */ + 6, /* 6 ZFS_IOC_POOL_TRYIMPORT */ + 7, /* 7 ZFS_IOC_POOL_SCRUB */ + 8, /* 8 ZFS_IOC_POOL_FREEZE */ + 9, /* 9 ZFS_IOC_POOL_UPGRADE */ + 10, /* 10 ZFS_IOC_POOL_GET_HISTORY */ + 11, /* 11 ZFS_IOC_VDEV_ADD */ + 12, /* 12 ZFS_IOC_VDEV_REMOVE */ + 13, /* 13 ZFS_IOC_VDEV_SET_STATE */ + 14, /* 14 ZFS_IOC_VDEV_ATTACH */ + 15, /* 15 ZFS_IOC_VDEV_DETACH */ + 16, /* 16 ZFS_IOC_VDEV_SETPATH */ + 18, /* 17 ZFS_IOC_OBJSET_STATS */ + 19, /* 18 ZFS_IOC_OBJSET_ZPLPROPS */ + 20, /* 19 ZFS_IOC_DATASET_LIST_NEXT */ + 21, /* 20 ZFS_IOC_SNAPSHOT_LIST_NEXT */ + 22, /* 21 ZFS_IOC_SET_PROP */ + ZFS_IOC_COMPAT_PASS, /* 22 ZFS_IOC_CREATE_MINOR */ + ZFS_IOC_COMPAT_PASS, /* 23 ZFS_IOC_REMOVE_MINOR */ + 23, /* 24 ZFS_IOC_CREATE */ + 24, /* 25 ZFS_IOC_DESTROY */ + 25, /* 26 ZFS_IOC_ROLLBACK */ + 26, /* 27 ZFS_IOC_RENAME */ + 27, /* 28 ZFS_IOC_RECV */ + 28, /* 29 ZFS_IOC_SEND */ + 29, /* 30 ZFS_IOC_INJECT_FAULT */ + 30, /* 31 ZFS_IOC_CLEAR_FAULT */ + 31, /* 32 ZFS_IOC_INJECT_LIST_NEXT */ + 32, /* 33 ZFS_IOC_ERROR_LOG */ + 33, /* 34 ZFS_IOC_CLEAR */ + 34, /* 35 ZFS_IOC_PROMOTE */ + 35, /* 36 ZFS_IOC_DESTROY_SNAPS */ + 36, /* 37 ZFS_IOC_SNAPSHOT */ + 37, /* 38 ZFS_IOC_DSOBJ_TO_DSNAME */ + 38, /* 39 ZFS_IOC_OBJ_TO_PATH */ + 39, /* 40 ZFS_IOC_POOL_SET_PROPS */ + 40, /* 41 ZFS_IOC_POOL_GET_PROPS */ + 41, /* 42 ZFS_IOC_SET_FSACL */ + 42, /* 43 ZFS_IOC_GET_FSACL */ + ZFS_IOC_COMPAT_PASS, /* 44 ZFS_IOC_ISCSI_PERM_CHECK */ + 43, /* 45 ZFS_IOC_SHARE */ + 44, /* 46 ZFS_IOC_IHNERIT_PROP */ + 58, /* 47 ZFS_IOC_JAIL */ + 59, /* 48 ZFS_IOC_UNJAIL */ + 45, /* 49 ZFS_IOC_SMB_ACL */ + 46, /* 50 ZFS_IOC_USERSPACE_ONE */ + 47, /* 51 ZFS_IOC_USERSPACE_MANY */ + 48, /* 52 ZFS_IOC_USERSPACE_UPGRADE */ + 17, /* 53 ZFS_IOC_SETFRU */ +}; + +#ifdef _KERNEL +int zfs_ioctl_compat_pre(zfs_cmd_t *, int *, const int); +void zfs_ioctl_compat_post(zfs_cmd_t *, const int, const int); +nvlist_t *zfs_ioctl_compat_innvl(zfs_cmd_t *, nvlist_t *, const int, + const int); +nvlist_t *zfs_ioctl_compat_outnvl(zfs_cmd_t *, nvlist_t *, const int, + const int); +#endif /* _KERNEL */ +void zfs_cmd_compat_get(zfs_cmd_t *, caddr_t, const int); +void zfs_cmd_compat_put(zfs_cmd_t *, caddr_t, const int, const int); +uint64_t zfs_ioc_unregister_fs(void); + +int wrap_avl_init(void); +int wrap_unicode_init(void); +int wrap_nvpair_init(void); +int wrap_zcommon_init(void); +int wrap_icp_init(void); +int wrap_lua_init(void); +void wrap_avl_fini(void); +void wrap_unicode_fini(void); +void wrap_nvpair_fini(void); +void wrap_zcommon_fini(void); +void wrap_icp_fini(void); +void wrap_lua_fini(void); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_ZFS_IOCTL_COMPAT_H */ diff --git a/include/os/windows/zfs/sys/zfs_mount.h b/include/os/windows/zfs/sys/zfs_mount.h new file mode 100644 index 000000000000..7c2d749c0861 --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_mount.h @@ -0,0 +1,73 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SYS_ZFS_MOUNT_H_ +#define _SYS_ZFS_MOUNT_H_ + +struct zfs_mount_args { + const char *fspec; /* block special device to mount */ + int mflag; + char *optptr; + int optlen; + int struct_size; +}; + +/* + * Flag bits passed to mount(2). + */ +#define MS_RDONLY 0x0001 /* Read-only */ +#define MS_FSS 0x0002 /* Old (4-argument) mount (compatibility) */ +#define MS_DATA 0x0004 /* 6-argument mount */ +#define MS_NOSUID 0x0010 /* Setuid programs disallowed */ +#define MS_REMOUNT 0x0020 /* Remount */ +#define MS_NOTRUNC 0x0040 /* Return ENAMETOOLONG for long filenames */ +#define MS_OVERLAY 0x0080 /* Allow overlay mounts */ +#define MS_OPTIONSTR 0x0100 /* Data is a an in/out option string */ +#define MS_GLOBAL 0x0200 /* Clustering: Mount into global name space */ +#define MS_FORCE 0x0400 /* Forced unmount */ +#define MS_NOMNTTAB 0x0800 /* Don't show mount in mnttab */ +/* + * Additional flag bits that domount() is prepared to interpret, but that + * can't be passed through mount(2). + */ +#define MS_SYSSPACE 0x0008 /* Mounta already in kernel space */ +#define MS_NOSPLICE 0x1000 /* Don't splice fs instance into name space */ +#define MS_NOCHECK 0x2000 /* Clustering: suppress mount busy checks */ +/* + * Mask to sift out flag bits allowable from mount(2). + */ +#define MS_MASK \ + (MS_RDONLY|MS_FSS|MS_DATA|MS_NOSUID|MS_REMOUNT|MS_NOTRUNC|MS_OVERLAY|\ + MS_OPTIONSTR|MS_GLOBAL|MS_NOMNTTAB) + +/* + * Mask to sift out flag bits allowable from umount2(2). + */ + +#define MS_UMOUNT_MASK (MS_FORCE) + +/* + * Maximum option string length accepted or returned by mount(2). + */ +#define MAX_MNTOPT_STR 1024 /* max length of mount options string */ + + +#endif /* _SYS_ZFS_IOCTL_H */ diff --git a/include/os/windows/zfs/sys/zfs_vfsops_os.h b/include/os/windows/zfs/sys/zfs_vfsops_os.h new file mode 100644 index 000000000000..c1e598be5a71 --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_vfsops_os.h @@ -0,0 +1,306 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright(c) 2015 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ + +#ifndef _SYS_FS_ZFS_VFSOPS_OS_H +#define _SYS_FS_ZFS_VFSOPS_OS_H + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +struct zfs_sb; +struct znode; + +/* + * Status of the zfs_unlinked_drain thread. + */ +typedef enum drain_state { + ZFS_DRAIN_SHUTDOWN = 0, + ZFS_DRAIN_RUNNING, + ZFS_DRAIN_SHUTDOWN_REQ +} drain_state_t; + + +typedef struct zfsvfs zfsvfs_t; + +struct zfsvfs { + vfs_t *z_vfs; /* generic fs struct */ + zfsvfs_t *z_parent; /* parent fs */ + objset_t *z_os; /* objset reference */ + uint64_t z_root; /* id of root znode */ + uint64_t z_unlinkedobj; /* id of unlinked zapobj */ + uint64_t z_max_blksz; /* maximum block size for files */ + uint64_t z_fuid_obj; /* fuid table object number */ + uint64_t z_fuid_size; /* fuid table size */ + avl_tree_t z_fuid_idx; /* fuid tree keyed by index */ + avl_tree_t z_fuid_domain; /* fuid tree keyed by domain */ + krwlock_t z_fuid_lock; /* fuid lock */ + boolean_t z_fuid_loaded; /* fuid tables are loaded */ + boolean_t z_fuid_dirty; /* need to sync fuid table ? */ + struct zfs_fuid_info *z_fuid_replay; /* fuid info for replay */ + uint64_t z_assign; /* TXG_NOWAIT or set by zil_replay() */ + zilog_t *z_log; /* intent log pointer */ + uint_t z_acl_mode; /* acl chmod/mode behavior */ + uint_t z_acl_inherit; /* acl inheritance behavior */ + zfs_case_t z_case; /* case-sense */ + boolean_t z_utf8; /* utf8-only */ + int z_norm; /* normalization flags */ + boolean_t z_atime; /* enable atimes mount option */ + boolean_t z_unmounted; /* unmounted */ + rrmlock_t z_teardown_lock; + krwlock_t z_teardown_inactive_lock; + list_t z_all_znodes; /* all vnodes in the fs */ + kmutex_t z_znodes_lock; /* lock for z_all_znodes */ + struct vnode *z_ctldir; /* .zfs directory pointer */ + uint64_t z_ctldir_startid; /* Start of snapdir range */ + boolean_t z_show_ctldir; /* expose .zfs in the root dir */ + boolean_t z_issnap; /* true if this is a snapshot */ + boolean_t z_vscan; /* virus scan on/off */ + boolean_t z_use_fuids; /* version allows fuids */ + boolean_t z_replay; /* set during ZIL replay */ + boolean_t z_use_sa; /* version allow system attributes */ + boolean_t z_xattr_sa; /* allow xattrs to be stores as SA */ + uint64_t z_version; + uint64_t z_shares_dir; /* hidden shares dir */ + kmutex_t z_lock; + dataset_kstats_t z_kstat; /* fs kstats */ + + /* for controlling async zfs_unlinked_drain */ + kmutex_t z_drain_lock; + kcondvar_t z_drain_cv; + drain_state_t z_drain_state; + + uint64_t z_userquota_obj; + uint64_t z_groupquota_obj; + uint64_t z_userobjquota_obj; + uint64_t z_groupobjquota_obj; + uint64_t z_projectquota_obj; + uint64_t z_projectobjquota_obj; + +#ifdef _WIN32 + dev_t z_rdev; /* proxy device for mount */ + boolean_t z_rdonly; /* is mount read-only? */ + time_t z_mount_time; /* mount timestamp (for Spotlight) */ + time_t z_last_unmount_time; /* unmount timestamp (for Spotlight) */ + boolean_t z_xattr; /* enable atimes mount option */ + + avl_tree_t z_hardlinks; /* linkid hash avl tree for vget */ + avl_tree_t z_hardlinks_linkid; /* sorted on linkid */ + krwlock_t z_hardlinks_lock; /* lock to access z_hardlinks */ + + uint64_t z_notification_conditions; /* HFSIOC_VOLUME_STATUS */ + uint64_t z_freespace_notify_warninglimit; + uint64_t z_freespace_notify_dangerlimit; + uint64_t z_freespace_notify_desiredlevel; + + void *z_devdisk; /* Hold fake disk if prop devdisk is on */ + + uint64_t z_findernotify_space; + +#endif + uint64_t z_replay_eof; /* New end of file - replay only */ + sa_attr_type_t *z_attr_table; /* SA attr mapping->id */ + + uint64_t z_hold_size; /* znode hold array size */ + avl_tree_t *z_hold_trees; /* znode hold trees */ + kmutex_t *z_hold_locks; /* znode hold locks */ + taskqid_t z_drain_task; /* task id for the unlink drain task */ +}; +#define ZFS_OBJ_MTX_SZ 64 + +struct hardlinks_struct { + avl_node_t hl_node; + avl_node_t hl_node_linkid; + uint64_t hl_parent; // parentid of entry + uint64_t hl_fileid; // the fileid (z_id) for vget + uint32_t hl_linkid; // the linkid, persistent over renames + char hl_name[PATH_MAX]; // cached name for vget +}; +typedef struct hardlinks_struct hardlinks_t; + +#define ZFS_SUPER_MAGIC 0x2fc12fc1 + +#define ZSB_XATTR 0x0001 /* Enable user xattrs */ + +#define ZFS_TEARDOWN_INIT(zfsvfs) \ + rrm_init(&(zfsvfs)->z_teardown_lock, B_FALSE) + +#define ZFS_TEARDOWN_DESTROY(zfsvfs) \ + rrm_destroy(&(zfsvfs)->z_teardown_lock) + +#define ZFS_TEARDOWN_ENTER_READ(zfsvfs, tag) \ + rrm_enter_read(&(zfsvfs)->z_teardown_lock, tag); + +#define ZFS_TEARDOWN_EXIT_READ(zfsvfs, tag) \ + rrm_exit(&(zfsvfs)->z_teardown_lock, tag) + +#define ZFS_TEARDOWN_ENTER_WRITE(zfsvfs, tag) \ + rrm_enter(&(zfsvfs)->z_teardown_lock, RW_WRITER, tag) + +#define ZFS_TEARDOWN_EXIT_WRITE(zfsvfs) \ + rrm_exit(&(zfsvfs)->z_teardown_lock, tag) + +#define ZFS_TEARDOWN_EXIT(zfsvfs, tag) \ + rrm_exit(&(zfsvfs)->z_teardown_lock, tag) + +#define ZFS_TEARDOWN_READ_HELD(zfsvfs) \ + RRM_READ_HELD(&(zfsvfs)->z_teardown_lock) + +#define ZFS_TEARDOWN_WRITE_HELD(zfsvfs) \ + RRM_WRITE_HELD(&(zfsvfs)->z_teardown_lock) + +#define ZFS_TEARDOWN_HELD(zfsvfs) \ + RRM_LOCK_HELD(&(zfsvfs)->z_teardown_lock) + +/* + * Normal filesystems (those not under .zfs/snapshot) have a total + * file ID size limited to 12 bytes (including the length field) due to + * NFSv2 protocol's limitation of 32 bytes for a filehandle. For historical + * reasons, this same limit is being imposed by the Solaris NFSv3 implementation + * (although the NFSv3 protocol actually permits a maximum of 64 bytes). It + * is not possible to expand beyond 12 bytes without abandoning support + * of NFSv2. + * + * For normal filesystems, we partition up the available space as follows: + * 2 bytes fid length (required) + * 6 bytes object number (48 bits) + * 4 bytes generation number (32 bits) + * + * We reserve only 48 bits for the object number, as this is the limit + * currently defined and imposed by the DMU. + */ +typedef struct zfid_short { + uint16_t zf_len; + uint8_t zf_object[6]; /* obj[i] = obj >> (8 * i) */ + uint8_t zf_gen[4]; /* gen[i] = gen >> (8 * i) */ +} zfid_short_t; + +/* + * Filesystems under .zfs/snapshot have a total file ID size of 22 bytes + * (including the length field). This makes files under .zfs/snapshot + * accessible by NFSv3 and NFSv4, but not NFSv2. + * + * For files under .zfs/snapshot, we partition up the available space + * as follows: + * 2 bytes fid length (required) + * 6 bytes object number (48 bits) + * 4 bytes generation number (32 bits) + * 6 bytes objset id (48 bits) + * 4 bytes currently just zero (32 bits) + * + * We reserve only 48 bits for the object number and objset id, as these are + * the limits currently defined and imposed by the DMU. + */ +typedef struct zfid_long { + zfid_short_t z_fid; + uint8_t zf_setid[6]; /* obj[i] = obj >> (8 * i) */ + uint8_t zf_setgen[4]; /* gen[i] = gen >> (8 * i) */ +} zfid_long_t; + +#define SHORT_FID_LEN (sizeof (zfid_short_t) - sizeof (uint16_t)) +#define LONG_FID_LEN (sizeof (zfid_long_t) - sizeof (uint16_t)) + +extern uint_t zfs_fsyncer_key; + +extern int zfs_suspend_fs(zfsvfs_t *zfsvfs); +extern int zfs_resume_fs(zfsvfs_t *zfsvfs, struct dsl_dataset *ds); +extern int zfs_userspace_one(zfsvfs_t *zfsvfs, zfs_userquota_prop_t type, + const char *domain, uint64_t rid, uint64_t *valuep); +extern int zfs_userspace_many(zfsvfs_t *zfsvfs, zfs_userquota_prop_t type, + uint64_t *cookiep, void *vbuf, uint64_t *bufsizep); +extern int zfs_set_userquota(zfsvfs_t *zfsvfs, zfs_userquota_prop_t type, + const char *domain, uint64_t rid, uint64_t quota); +extern boolean_t zfs_owner_overquota(zfsvfs_t *zfsvfs, struct znode *, + boolean_t isgroup); +extern boolean_t zfs_fuid_overquota(zfsvfs_t *zfsvfs, boolean_t isgroup, + uint64_t fuid); +extern int zfs_set_version(zfsvfs_t *zfsvfs, uint64_t newvers); +extern int zfsvfs_create_impl(zfsvfs_t **zfvp, zfsvfs_t *zfsvfs, objset_t *os); + +extern int zfs_get_zplprop(objset_t *os, zfs_prop_t prop, + uint64_t *value); + +extern int zfs_sb_create(const char *name, zfsvfs_t **zfsvfsp); +extern int zfs_sb_setup(zfsvfs_t *zfsvfs, boolean_t mounting); +extern void zfs_sb_free(zfsvfs_t *zfsvfs); +extern int zfs_check_global_label(const char *dsname, const char *hexsl); +extern boolean_t zfs_is_readonly(zfsvfs_t *zfsvfs); + + + + +extern int zfs_vfs_init(struct vfsconf *vfsp); +extern int zfs_vfs_start(struct mount *mp, int flags, vfs_context_t context); +extern int zfs_vfs_mount(struct mount *mp, vnode_t *devvp, + user_addr_t data, vfs_context_t context); +extern int zfs_vfs_unmount(struct mount *mp, int mntflags, + vfs_context_t context); +extern int zfs_vfs_root(struct mount *mp, vnode_t **vpp, + vfs_context_t context); +extern int zfs_vfs_vget(struct mount *mp, ino64_t ino, vnode_t **vpp, + vfs_context_t context); +extern int zfs_vfs_sync(struct mount *mp, int waitfor, vfs_context_t context); +extern int zfs_vfs_fhtovp(struct mount *mp, int fhlen, unsigned char *fhp, + vnode_t **vpp, vfs_context_t context); +extern int zfs_vfs_vptofh(vnode_t *vp, int *fhlenp, unsigned char *fhp, + vfs_context_t context); +extern int zfs_vfs_sysctl(int *name, uint_t namelen, user_addr_t oldp, + size_t *oldlenp, user_addr_t newp, size_t newlen, vfs_context_t context); +extern int zfs_vfs_quotactl(struct mount *mp, int cmds, uid_t uid, + caddr_t datap, vfs_context_t context); +extern int zfs_vfs_mountroot(struct mount *mp, struct vnode *vp, + vfs_context_t context); + +extern void zfs_init(void); +extern void zfs_fini(void); + +extern int zfs_vnode_lock(vnode_t *vp, int flags); +extern void zfs_freevfs(struct mount *vfsp); + +extern int zfsvfs_create(const char *name, boolean_t rd, zfsvfs_t **zfvp); +extern void zfsvfs_free(zfsvfs_t *zfsvfs); + +extern int zfs_get_temporary_prop(dsl_dataset_t *ds, zfs_prop_t zfs_prop, + uint64_t *val, char *setpoint); + +extern int zfs_end_fs(zfsvfs_t *zfsvfs, struct dsl_dataset *ds); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_FS_ZFS_VFSOPS_H */ diff --git a/include/os/windows/zfs/sys/zfs_vnops_os.h b/include/os/windows/zfs/sys/zfs_vnops_os.h new file mode 100644 index 000000000000..30a346e16c4c --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_vnops_os.h @@ -0,0 +1,168 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2015, Jorgen Lundman + */ + +#ifndef _SYS_FS_ZFS_VNOPS_OS_H +#define _SYS_FS_ZFS_VNOPS_OS_H + +#include +#include +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define KAUTH_WKG_NOT 0 /* not a well-known GUID */ +#define KAUTH_WKG_OWNER 1 +#define KAUTH_WKG_GROUP 2 +#define KAUTH_WKG_NOBODY 3 +#define KAUTH_WKG_EVERYBODY 4 + +struct emitdir_ptr { + char *alloc_buf; /* output buffer */ + char *bufptr; /* starts at alloc_buf, increments */ + int bufsize; /* total size of alloc_buf */ + int outcount; /* starts at 0, approaches bufsize */ + ULONG *next_offset; /* ptr to previous nextoffset */ + int last_alignment; /* How much was last alignment */ + uint64_t offset; /* dirindex, 0=".", 1="..", 2=".zfs" */ + int numdirent; + int dirlisttype; /* Win struct to use */ +}; + +typedef struct emitdir_ptr emitdir_ptr_t; + +extern int zfs_remove(znode_t *dzp, char *name, cred_t *cr, int flags); +extern int zfs_mkdir(znode_t *dzp, char *dirname, vattr_t *vap, + znode_t **zpp, cred_t *cr, int flags, vsecattr_t *vsecp, + zidmap_t *mnt_ns); +extern int zfs_rmdir(znode_t *dzp, char *name, znode_t *cwd, + cred_t *cr, int flags); +extern int zfs_setattr(znode_t *zp, vattr_t *vap, int flag, cred_t *cr, + zidmap_t *mnt_ns); +extern int zfs_rename(znode_t *sdzp, char *snm, znode_t *tdzp, + char *tnm, cred_t *cr, int flags, uint64_t rflags, vattr_t *wo_vap, + zidmap_t *mnt_ns); +extern int zfs_symlink(znode_t *dzp, char *name, vattr_t *vap, + char *link, znode_t **zpp, cred_t *cr, int flags, zidmap_t *mnt_ns); +extern int zfs_link(znode_t *tdzp, znode_t *sp, + char *name, cred_t *cr, int flags); +extern int zfs_space(znode_t *zp, int cmd, struct flock *bfp, int flag, + offset_t offset, cred_t *cr); +extern int zfs_create(znode_t *dzp, char *name, vattr_t *vap, int excl, + int mode, znode_t **zpp, cred_t *cr, int flag, vsecattr_t *vsecp, + zidmap_t *mnt_ns); +extern int zfs_setsecattr(znode_t *zp, vsecattr_t *vsecp, int flag, + cred_t *cr); +extern int zfs_write_simple(znode_t *zp, const void *data, size_t len, + loff_t pos, size_t *resid); + +extern int zfs_open(struct vnode *ip, int mode, int flag, cred_t *cr); +extern int zfs_close(struct vnode *ip, int flag, cred_t *cr); +extern int zfs_lookup(znode_t *dzp, char *nm, znode_t **zpp, + int flags, cred_t *cr, int *direntflags, struct componentname *realpnp); +extern int zfs_ioctl(vnode_t *vp, ulong_t com, intptr_t data, int flag, + cred_t *cred, int *rvalp, caller_context_t *ct); +extern int zfs_readdir(vnode_t *vp, emitdir_ptr_t *, cred_t *cr, + zfs_dirlist_t *zccb, int flags); +extern int zfs_readdir_emitdir(zfsvfs_t *zfsvfs, const char *name, + emitdir_ptr_t *ctx, zfs_dirlist_t *zccb, ino64_t objnum); +extern void zfs_readdir_complete(emitdir_ptr_t *ctx); + +extern int zfs_fsync(znode_t *zp, int syncflag, cred_t *cr); +extern int zfs_getattr(vnode_t *vp, vattr_t *vap, int flags, + cred_t *cr, caller_context_t *ct); +extern int zfs_readlink(vnode_t *vp, zfs_uio_t *uio, cred_t *cr); + +extern void zfs_inactive(vnode_t *vp); + +/* zfs_vops_windows.c calls */ +extern int zfs_znode_getvnode(znode_t *zp, znode_t *dzp, zfsvfs_t *zfsvfs); + +extern void getnewvnode_reserve(int num); +extern void getnewvnode_drop_reserve(void); +extern int zfs_vfsops_init(void); +extern int zfs_vfsops_fini(void); +extern int zfs_znode_asyncgetvnode(znode_t *zp, zfsvfs_t *zfsvfs); +extern void zfs_znode_asyncput(znode_t *zp); +extern int zfs_znode_asyncwait(znode_t *zp); + +/* zfs_vnops_windows_lib calls */ +extern int zfs_ioflags(int ap_ioflag); +extern int zfs_getattr_znode_unlocked(struct vnode *vp, vattr_t *vap); +extern int ace_trivial_common(void *acep, int aclcnt, + uintptr_t (*walk)(void *, uintptr_t, int aclcnt, + uint16_t *, uint16_t *, uint32_t *)); + +extern int zpl_obtain_xattr(struct znode *, const char *name, mode_t mode, + cred_t *cr, struct vnode **vpp, int flag); +extern int zpl_xattr_filldir(struct vnode *, zfs_uio_t *uio, const char *, + int name_len, FILE_FULL_EA_INFORMATION **previous_ea); +extern int zpl_xattr_list(struct vnode *, zfs_uio_t *, ssize_t *, cred_t *); +extern int zpl_xattr_get(struct vnode *ip, const char *name, zfs_uio_t *uio, + ssize_t *retsize, cred_t *cr); +extern int zpl_xattr_set(struct vnode *, const char *, zfs_uio_t *uio, + int flags, cred_t *cr); + +extern uint32_t getuseraccess(znode_t *zp, vfs_context_t ctx); +extern void zfs_zrele_async(znode_t *zp); + +extern int zfsctl_readdir(vnode_t *vp, emitdir_ptr_t *ctx, cred_t *cr, + zfs_dirlist_t *zccb, int flags); + +/* + * Windows ACL Helper funcions + */ +#define KAUTH_WKG_NOT 0 /* not a well-known GUID */ +#define KAUTH_WKG_OWNER 1 +#define KAUTH_WKG_GROUP 2 +#define KAUTH_WKG_NOBODY 3 +#define KAUTH_WKG_EVERYBODY 4 + +extern int kauth_wellknown_guid(guid_t *guid); +extern void aces_from_acl(ace_t *aces, int *nentries, struct kauth_acl *k_acl, + int *seen_type); +extern void nfsacl_set_wellknown(int wkg, guid_t *guid); +extern int zfs_addacl_trivial(znode_t *zp, ace_t *aces, int *nentries, + int seen_type); + +extern struct vnodeopv_desc zfs_dvnodeop_opv_desc; +extern struct vnodeopv_desc zfs_fvnodeop_opv_desc; +extern struct vnodeopv_desc zfs_symvnodeop_opv_desc; +extern struct vnodeopv_desc zfs_xdvnodeop_opv_desc; +extern struct vnodeopv_desc zfs_evnodeop_opv_desc; +extern struct vnodeopv_desc zfs_fifonodeop_opv_desc; +extern struct vnodeopv_desc zfs_ctldir_opv_desc; +extern int (**zfs_ctldirops)(void *); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_FS_ZFS_VNOPS_H */ diff --git a/include/os/windows/zfs/sys/zfs_windows.h b/include/os/windows/zfs/sys/zfs_windows.h new file mode 100644 index 000000000000..6fd602410ad6 --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_windows.h @@ -0,0 +1,221 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#ifndef SYS_WINDOWS_H_INCLUDED +#define SYS_WINDOWS_H_INCLUDED + + +#include + +extern PDEVICE_OBJECT ioctlDeviceObject; +extern PDEVICE_OBJECT fsDiskDeviceObject; + +#define ZFS_SERIAL (ULONG)'wZFS' +#define VOLUME_LABEL L"ZFS" + + +// We have to remember "query directory" related items, like index and +// search pattern. This is attached in IRP_MJ_CREATE to fscontext2 +#define ZFS_DIRLIST_MAGIC 0x6582feac +struct zfs_dirlist { + uint32_t magic; // Identifier + uint32_t dir_eof; // Directory listing completed? + uint64_t uio_offset; // Directory list offset + uint64_t ea_index; // EA list offset + uint32_t deleteonclose; // Marked for deletion + uint32_t ContainsWildCards; // searchname has wildcards + UNICODE_STRING searchname; // Search pattern + + uint64_t cacheinit; + uint64_t real_file_id; +}; + +typedef struct zfs_dirlist zfs_dirlist_t; + +extern uint64_t zfs_module_busy; + +extern CACHE_MANAGER_CALLBACKS CacheManagerCallbacks; + + +extern NTSTATUS dev_ioctl(PDEVICE_OBJECT DeviceObject, ULONG ControlCode, + PVOID InputBuffer, ULONG InputBufferSize, PVOID OutputBuffer, + ULONG OutputBufferSize, BOOLEAN Override, IO_STATUS_BLOCK* iosb); + +extern int zfs_windows_mount(zfs_cmd_t *zc); +extern int zfs_windows_unmount(zfs_cmd_t *zc); +extern NTSTATUS zfsdev_ioctl(PDEVICE_OBJECT DeviceObject, PIRP Irp, int flag); +extern void zfs_windows_vnops_callback(PDEVICE_OBJECT deviceObject); +extern void zfs_send_notify(zfsvfs_t *zfsvfs, char *name, int, + ULONG FilterMatch, ULONG Action); +extern void zfs_send_notify_stream(zfsvfs_t *, char *, int, ULONG, + ULONG, char *stream); +extern int zfs_set_security(struct vnode *vp, struct vnode *dvp); +extern uint64_t zfs_sid2uid(SID *sid); + +BOOLEAN vattr_apply_lx_ea(vattr_t *vap, PFILE_FULL_EA_INFORMATION ea); +NTSTATUS vnode_apply_eas(struct vnode *vp, PFILE_FULL_EA_INFORMATION eas, + ULONG eaLength, PULONG pEaErrorOffset); + +/* Main function to handle all VFS "vnops" */ +extern _Function_class_(DRIVER_DISPATCH) NTSTATUS + dispatcher(_In_ PDEVICE_OBJECT DeviceObject, _Inout_ PIRP Irp); + +extern NTSTATUS zfsdev_open(dev_t dev, PIRP Irp); +extern NTSTATUS zfsdev_release(dev_t dev, PIRP Irp); + +extern int zfs_vnop_recycle(znode_t *zp, int force); +extern uint64_t zfs_blksz(znode_t *zp); + +inline static uint64_t +allocationsize(struct znode *zp) +{ + if (zp->z_size == 0) + return (0ULL); + return (P2ROUNDUP(zp->z_size, zfs_blksz(zp))); +} + +extern int zfs_vnop_mount(PDEVICE_OBJECT DiskDevice, PIRP Irp, + PIO_STACK_LOCATION IrpSp); + +extern int zfs_build_path(znode_t *start_zp, znode_t *start_parent, + char **fullpath, uint32_t *returnsize, uint32_t *start_zp_offset); +extern int zfs_build_path_stream(znode_t *start_zp, znode_t *start_parent, + char **fullpath, uint32_t *returnsize, uint32_t *start_zp_offset, char *); + +extern int xattr_protected(char *name); +extern int xattr_stream(char *name); +extern uint64_t xattr_getsize(struct vnode *vp); +extern char *major2str(int major, int minor); +extern char *common_status_str(NTSTATUS Status); +extern char *create_options(ULONG options); +extern char *create_reply(NTSTATUS, ULONG reply); +extern void latency_stats(uint64_t *histo, unsigned int buckets, + stat_pair *lat); +extern size_t get_reparse_point_impl(znode_t *zp, char *buffer, size_t outlen); + +/* zfs_vnop_windows_lib.h */ +extern int AsciiStringToUnicodeString(char *in, PUNICODE_STRING out); +extern void FreeUnicodeString(PUNICODE_STRING s); +extern int zfs_vfs_uuid_gen(const char *osname, uuid_t uuid); +extern int zfs_vfs_uuid_unparse(uuid_t uuid, char *dst); +extern int zfs_vnop_ioctl_fullfsync(struct vnode *, vfs_context_t *, + zfsvfs_t *); +extern int zfs_setwinflags(znode_t *zp, uint32_t winflags); +extern uint32_t zfs_getwinflags(znode_t *zp); +extern NTSTATUS zfs_setunlink(FILE_OBJECT *fo, vnode_t *dvp); +extern int zfs_find_dvp_vp(zfsvfs_t *, char *, int finalpartmaynotexist, + int finalpartmustnotexist, char **lastname, struct vnode **dvpp, + struct vnode **vpp, int flags, ULONG options); +extern ULONG get_reparse_tag(znode_t *zp); + +/* IRP_MJ_SET_INFORMATION helpers */ +extern NTSTATUS set_file_basic_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS set_file_disposition_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, boolean_t); +extern NTSTATUS set_file_disposition_information_ex(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS set_file_endoffile_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS set_file_link_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS set_file_rename_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS set_file_valid_data_length_information( + PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp); +extern NTSTATUS set_file_position_information(PDEVICE_OBJECT DeviceObject, + PIRP Irp, PIO_STACK_LOCATION IrpSp); + +/* IRP_MJ_GET_INFORMATION helpers */ +extern NTSTATUS file_basic_information(PDEVICE_OBJECT, PIRP, PIO_STACK_LOCATION, + FILE_BASIC_INFORMATION *); +extern NTSTATUS file_compression_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_COMPRESSION_INFORMATION *); +extern NTSTATUS file_standard_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_STANDARD_INFORMATION *); +extern NTSTATUS file_position_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_POSITION_INFORMATION *); +extern NTSTATUS file_ea_information(PDEVICE_OBJECT, PIRP, PIO_STACK_LOCATION, + FILE_EA_INFORMATION *); +extern NTSTATUS file_alignment_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_ALIGNMENT_INFORMATION *); +extern NTSTATUS file_network_open_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_NETWORK_OPEN_INFORMATION *); +extern NTSTATUS file_standard_link_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_STANDARD_LINK_INFORMATION *); +extern NTSTATUS file_id_information(PDEVICE_OBJECT, PIRP, PIO_STACK_LOCATION, + FILE_ID_INFORMATION *); +extern NTSTATUS file_case_sensitive_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_CASE_SENSITIVE_INFORMATION *); +extern NTSTATUS file_stat_information(PDEVICE_OBJECT, PIRP, PIO_STACK_LOCATION, + FILE_STAT_INFORMATION *); +extern NTSTATUS file_stat_lx_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_STAT_LX_INFORMATION *); +extern NTSTATUS file_name_information(PDEVICE_OBJECT, PIRP, PIO_STACK_LOCATION, + FILE_NAME_INFORMATION *, PULONG usedspace, int normalize); +extern NTSTATUS file_remote_protocol_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_REMOTE_PROTOCOL_INFORMATION *); +extern NTSTATUS file_stream_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_STREAM_INFORMATION *); +extern NTSTATUS file_attribute_tag_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_ATTRIBUTE_TAG_INFORMATION *tag); +extern NTSTATUS file_internal_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_INTERNAL_INFORMATION *infernal); +extern NTSTATUS file_hard_link_information(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION, FILE_LINKS_INFORMATION *links); + +/* IRP_MJ_DEVICE_CONTROL helpers */ +extern NTSTATUS QueryCapabilities(PDEVICE_OBJECT, PIRP, PIO_STACK_LOCATION); +extern NTSTATUS ioctl_query_device_name(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_disk_get_drive_geometry(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_disk_get_drive_geometry_ex(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_disk_get_partition_info(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_disk_get_partition_info_ex(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_disk_get_length_info(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_volume_is_io_capable(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_storage_get_hotplug_info(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_storage_query_property(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_query_unique_id(PDEVICE_OBJECT, PIRP, PIO_STACK_LOCATION); +extern NTSTATUS ioctl_mountdev_query_suggested_link_name(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_mountdev_query_stable_guid(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern NTSTATUS ioctl_query_stable_guid(PDEVICE_OBJECT, PIRP, + PIO_STACK_LOCATION); +extern void strupper(char *s, size_t max); +extern NTSTATUS fsctl_zfs_volume_mountpoint(PDEVICE_OBJECT DeviceObject, + PIRP Irp, PIO_STACK_LOCATION IrpSp); +extern NTSTATUS fsctl_set_zero_data(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp); + +#endif diff --git a/include/os/windows/zfs/sys/zfs_znode_impl.h b/include/os/windows/zfs/sys/zfs_znode_impl.h new file mode 100644 index 000000000000..a3cbf3e87d07 --- /dev/null +++ b/include/os/windows/zfs/sys/zfs_znode_impl.h @@ -0,0 +1,217 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2015 by Delphix. All rights reserved. + * Copyright (c) 2014 Integros [integros.com] + * Copyright 2016 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2015 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ + +#ifndef _WIN_ZFS_SYS_ZNODE_IMPL_H +#define _WIN_ZFS_SYS_ZNODE_IMPL_H + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define ZFS_UIMMUTABLE 0x0000001000000000ull // OSX +#define ZFS_UAPPENDONLY 0x0000004000000000ull // OSX + +// #define ZFS_IMMUTABLE (ZFS_UIMMUTABLE | ZFS_SIMMUTABLE) +// #define ZFS_APPENDONLY (ZFS_UAPPENDONLY | ZFS_SAPPENDONLY) + +#define ZFS_TRACKED 0x0010000000000000ull +#define ZFS_COMPRESSED 0x0020000000000000ull + +#define ZFS_SIMMUTABLE 0x0040000000000000ull +#define ZFS_SAPPENDONLY 0x0080000000000000ull + +#define SA_ZPL_ADDTIME(z) z->z_attr_table[ZPL_ADDTIME] +#define SA_ZPL_DOCUMENTID(z) z->z_attr_table[ZPL_DOCUMENTID] + +#define ZGET_FLAG_UNLINKED (1<<0) /* Also lookup unlinked */ +#define ZGET_FLAG_ASYNC (1<<3) /* taskq the vnode_create call */ + +extern int zfs_zget_ext(zfsvfs_t *zfsvfs, uint64_t obj_num, + struct znode **zpp, int flags); + + +/* + * Directory entry locks control access to directory entries. + * They are used to protect creates, deletes, and renames. + * Each directory znode has a mutex and a list of locked names. + */ +#define ZNODE_OS_FIELDS \ + struct zfsvfs *z_zfsvfs; \ + struct vnode *z_vnode; \ + uint64_t z_uid; \ + uint64_t z_gid; \ + uint64_t z_gen; \ + uint64_t z_atime[2]; \ + uint64_t z_links; \ + uint32_t z_vid; \ + boolean_t z_is_mapped; \ + taskq_ent_t z_attach_taskq; \ + kcondvar_t z_attach_cv; \ + kmutex_t z_attach_lock; \ + hrtime_t z_snap_mount_time; \ + krwlock_t z_map_lock; \ + boolean_t z_fastpath; \ + uint32_t z_name_len; \ + uint32_t z_name_offset; \ + char *z_name_cache; + +#define ZFS_LINK_MAX UINT64_MAX + +/* + * ZFS minor numbers can refer to either a control device instance or + * a zvol. Depending on the value of zss_type, zss_data points to either + * a zvol_state_t or a zfs_onexit_t. + */ +enum zfs_soft_state_type { + ZSST_ZVOL, + ZSST_CTLDEV +}; + +typedef struct zfs_soft_state { + enum zfs_soft_state_type zss_type; + void *zss_data; +} zfs_soft_state_t; + +extern minor_t zfsdev_minor_alloc(void); + +/* + * Convert between znode pointers and vnode pointers + */ +#define ZTOV(ZP) ((ZP)->z_vnode) +#define ZTOI(ZP) ((ZP)->z_vnode) +#define VTOZ(VP) ((znode_t *)vnode_fsnode((VP))) +#define ITOZ(VP) ((znode_t *)vnode_fsnode((VP))) + +#define VTOM(VP) ((mount_t *)vnode_mount((VP))) + +/* These are not used so far, VN_HOLD returncode must be checked. */ +#define zhold(zp) VN_HOLD(ZTOV(zp)) +#define zrele(zp) VN_RELE(ZTOV(zp)) + +#define ZTOZSB(zp) ((zp)->z_zfsvfs) +#define ITOZSB(vp) ((zfsvfs_t *)vfs_fsprivate(vnode_mount(vp))) +#define ZTOTYPE(zp) (vnode_vtype(ZTOV(zp))) +#define ZTOGID(zp) ((zp)->z_gid) +#define ZTOUID(zp) ((zp)->z_uid) +#define ZTONLNK(zp) ((zp)->z_links) +#define Z_ISBLK(type) ((type) == VBLK) +#define Z_ISCHR(type) ((type) == VCHR) +#define Z_ISLNK(type) ((type) == VLNK) +#define Z_ISDIR(type) ((type) == VDIR) + +#define zn_has_cached_data(zp, start, end) (zp->z_is_mapped) +#define zn_flush_cached_data(zp, sync) /* find solution */ + +#define zn_rlimit_fsize(size) (0) +#define zn_rlimit_fsize_uio(zp, uio) (0) + +/* Called on entry to each ZFS inode and vfs operation. */ +static inline int +zfs_enter(zfsvfs_t *zfsvfs, const char *tag) +{ + ZFS_TEARDOWN_ENTER_READ(zfsvfs, tag); + if (unlikely(zfsvfs->z_unmounted)) { + ZFS_TEARDOWN_EXIT_READ(zfsvfs, tag); + return (SET_ERROR(EIO)); + } + return (0); +} + +/* Must be called before exiting the operation. */ +static inline void +zfs_exit(zfsvfs_t *zfsvfs, const char *tag) +{ + ZFS_TEARDOWN_EXIT_READ(zfsvfs, tag); +} + +/* + * Macros for dealing with dmu_buf_hold + */ +#define ZFS_OBJ_MTX_SZ 64 +#define ZFS_OBJ_MTX_MAX (1024 * 1024) +#define ZFS_OBJ_HASH(zfsvfs, obj) ((obj) & ((zfsvfs->z_hold_size) - 1)) + +extern unsigned int zfs_object_mutex_size; + +/* Encode ZFS stored time values from a struct timespec */ +#define ZFS_TIME_ENCODE(tp, stmp) \ + { \ + (stmp)[0] = (uint64_t)(tp)->tv_sec; \ + (stmp)[1] = (uint64_t)(tp)->tv_nsec; \ + } + +/* Decode ZFS stored time values to a struct timespec */ +#define ZFS_TIME_DECODE(tp, stmp) \ + { \ + (tp)->tv_sec = (time_t)(stmp)[0]; \ + (tp)->tv_nsec = (long)(stmp)[1]; \ +} + +#define ZFS_ACCESSTIME_STAMP(zfsvfs, zp) \ + if ((zfsvfs)->z_atime && !vfs_isrdonly(zfsvfs->z_vfs)) \ + zfs_tstamp_update_setup_ext(zp, ACCESSED, NULL, NULL, B_FALSE); + +extern void zfs_tstamp_update_setup_ext(struct znode *, + uint_t, uint64_t [2], uint64_t [2], boolean_t); +extern void zfs_tstamp_update_setup(struct znode *, + uint_t, uint64_t [2], uint64_t [2]); +extern void zfs_znode_free(struct znode *); + +extern zil_get_data_t zfs_get_data; +extern zil_replay_func_t *const zfs_replay_vector[TX_MAX_TYPE]; +extern int zfsfstype; + +extern int zfs_znode_parent_and_name(struct znode *zp, struct znode **dzpp, + char *buf); +extern uint32_t zfs_getbsdflags(struct znode *zp); +extern void zfs_setattr_generate_id(struct znode *, uint64_t, char *name); + +extern int zfs_setattr_set_documentid(struct znode *zp, + boolean_t update_flags); + +void zfs_setbsdflags(struct znode *, uint32_t bsdflags); +uint32_t zfs_getbsdflags(struct znode *zp); + +#ifdef __cplusplus +} +#endif + +#endif /* _MACOS_SYS_FS_ZFS_ZNODE_H */ diff --git a/include/os/windows/zfs/sys/zlib.h b/include/os/windows/zfs/sys/zlib.h new file mode 100644 index 000000000000..5c44a53ec85b --- /dev/null +++ b/include/os/windows/zfs/sys/zlib.h @@ -0,0 +1,1356 @@ +/* + * zlib.h -- interface of the 'zlib' general purpose compression library + * version 1.2.3, July 18th, 2005 + * + * Copyright (C) 1995-2005 Jean-loup Gailly and Mark Adler + * + * This software is provided 'as-is', without any express or implied + * warranty. In no event will the authors be held liable for any damages + * arising from the use of this software. + * + * Permission is granted to anyone to use this software for any purpose, + * including commercial applications, and to alter it and redistribute it + * freely, subject to the following restrictions: + * + * 1. The origin of this software must not be misrepresented; you must not + * claim that you wrote the original software. If you use this software + * in a product, an acknowledgment in the product documentation would be + * appreciated but is not required. + * 2. Altered source versions must be plainly marked as such, and must not be + * misrepresented as being the original software. + * 3. This notice may not be removed or altered from any source distribution. + * + * Jean-loup Gailly Mark Adler + * jloup@gzip.org madler@alumni.caltech.edu + * + * + * The data format used by the zlib library is described by RFCs (Request for + * Comments) 1950 to 1952 in the files http://www.ietf.org/rfc/rfc1950.txt + * (zlib format), rfc1951.txt (deflate format) and rfc1952.txt (gzip format). + */ + +#ifndef _ZLIB_H +#define _ZLIB_H + +#pragma ident "%Z%%M% %I% %E% SMI" + +#include "zconf.h" + +#ifdef __cplusplus +extern "C" { +#endif + +#define ZLIB_VERSION "1.2.3" +#define ZLIB_VERNUM 0x1230 + +/* + * The 'zlib' compression library provides in-memory compression and + * decompression functions, including integrity checks of the uncompressed + * data. This version of the library supports only one compression method + * (deflation) but other algorithms will be added later and will have the same + * stream interface. + * + * Compression can be done in a single step if the buffers are large + * enough (for example if an input file is mmap'ed), or can be done by + * repeated calls of the compression function. In the latter case, the + * application must provide more input and/or consume the output + * (providing more output space) before each call. + * + * The compressed data format used by default by the in-memory functions is + * the zlib format, which is a zlib wrapper documented in RFC 1950, wrapped + * around a deflate stream, which is itself documented in RFC 1951. + * + * The library also supports reading and writing files in gzip (.gz) format + * with an interface similar to that of stdio using the functions that start + * with "gz". The gzip format is different from the zlib format. gzip is a + * gzip wrapper, documented in RFC 1952, wrapped around a deflate stream. + * + * This library can optionally read and write gzip streams in memory as well. + * + * The zlib format was designed to be compact and fast for use in memory + * and on communications channels. The gzip format was designed for single- + * file compression on file systems, has a larger header than zlib to maintain + * directory information, and uses a different, slower check method than zlib. + * + * The library does not install any signal handler. The decoder checks + * the consistency of the compressed data, so the library should never + * crash even in case of corrupted input. + */ + +typedef voidpf (*alloc_func) OF((voidpf opaque, uInt items, uInt size)); +typedef void (*free_func) OF((voidpf opaque, voidpf address)); + +struct internal_state; + +typedef struct z_stream_s { + Bytef *next_in; /* next input byte */ + uInt avail_in; /* number of bytes available at next_in */ + uLong total_in; /* total nb of input bytes read so far */ + + Bytef *next_out; /* next output byte should be put there */ + uInt avail_out; /* remaining free space at next_out */ + uLong total_out; /* total nb of bytes output so far */ + + char *msg; /* last error message, NULL if no error */ + struct internal_state FAR *state; /* not visible by applications */ + + alloc_func zalloc; /* used to allocate the internal state */ + free_func zfree; /* used to free the internal state */ + voidpf opaque; /* private data object passed to zalloc and zfree */ + + int data_type; /* best guess about the data type: binary or text */ + uLong adler; /* adler32 value of the uncompressed data */ + uLong reserved; /* reserved for future use */ +} z_stream; + +typedef z_stream FAR *z_streamp; + +/* + * gzip header information passed to and from zlib routines. See RFC 1952 + * for more details on the meanings of these fields. + */ +typedef struct gz_header_s { + int text; /* true if compressed data believed to be text */ + uLong time; /* modification time */ + int xflags; /* extra flags (not used when writing a gzip file) */ + int os; /* operating system */ + Bytef *extra; /* pointer to extra field or Z_NULL if none */ + uInt extra_len; /* extra field length (valid if extra != Z_NULL) */ + uInt extra_max; /* space at extra (only when reading header) */ + Bytef *name; /* pointer to zero-terminated file name or Z_NULL */ + uInt name_max; /* space at name (only when reading header) */ + Bytef *comment; /* pointer to zero-terminated comment or Z_NULL */ + uInt comm_max; /* space at comment (only when reading header) */ + int hcrc; /* true if there was or will be a header crc */ + int done; /* true when done reading gzip header (not used */ + /* when writing a gzip file) */ +} gz_header; + +typedef gz_header FAR *gz_headerp; + +/* + * The application must update next_in and avail_in when avail_in has + * dropped to zero. It must update next_out and avail_out when avail_out + * has dropped to zero. The application must initialize zalloc, zfree and + * opaque before calling the init function. All other fields are set by the + * compression library and must not be updated by the application. + * + * The opaque value provided by the application will be passed as the first + * parameter for calls of zalloc and zfree. This can be useful for custom + * memory management. The compression library attaches no meaning to the + * opaque value. + * + * zalloc must return Z_NULL if there is not enough memory for the object. + * If zlib is used in a multi-threaded application, zalloc and zfree must be + * thread safe. + * + * On 16-bit systems, the functions zalloc and zfree must be able to allocate + * exactly 65536 bytes, but will not be required to allocate more than this + * if the symbol MAXSEG_64K is defined (see zconf.h). WARNING: On MSDOS, + * pointers returned by zalloc for objects of exactly 65536 bytes *must* + * have their offset normalized to zero. The default allocation function + * provided by this library ensures this (see zutil.c). To reduce memory + * requirements and avoid any allocation of 64K objects, at the expense of + * compression ratio, compile the library with -DMAX_WBITS=14 (see zconf.h). + * + * The fields total_in and total_out can be used for statistics or + * progress reports. After compression, total_in holds the total size of + * the uncompressed data and may be saved for use in the decompressor + * (particularly if the decompressor wants to decompress everything in + * a single step). + */ + + /* constants */ + +#define Z_NO_FLUSH 0 +#define Z_PARTIAL_FLUSH 1 /* will be removed, use Z_SYNC_FLUSH instead */ +#define Z_SYNC_FLUSH 2 +#define Z_FULL_FLUSH 3 +#define Z_FINISH 4 +#define Z_BLOCK 5 +/* Allowed flush values; see deflate() and inflate() below for details */ + +#define Z_OK 0 +#define Z_STREAM_END 1 +#define Z_NEED_DICT 2 +#define Z_ERRNO (-1) +#define Z_STREAM_ERROR (-2) +#define Z_DATA_ERROR (-3) +#define Z_MEM_ERROR (-4) +#define Z_BUF_ERROR (-5) +#define Z_VERSION_ERROR (-6) +/* + * Return codes for the compression/decompression functions. Negative + * values are errors, positive values are used for special but normal events. + */ + +#define Z_NO_COMPRESSION 0 +#define Z_BEST_SPEED 1 +#define Z_BEST_COMPRESSION 9 +#define Z_DEFAULT_COMPRESSION (-1) +/* compression levels */ + +#define Z_FILTERED 1 +#define Z_HUFFMAN_ONLY 2 +#define Z_RLE 3 +#define Z_FIXED 4 +#define Z_DEFAULT_STRATEGY 0 +/* compression strategy; see deflateInit2() below for details */ + +#define Z_BINARY 0 +#define Z_TEXT 1 +#define Z_ASCII Z_TEXT /* for compatibility with 1.2.2 and earlier */ +#define Z_UNKNOWN 2 +/* Possible values of the data_type field (though see inflate()) */ + +#define Z_DEFLATED 8 +/* The deflate compression method (the only one supported in this version) */ + +#define Z_NULL 0 /* for initializing zalloc, zfree, opaque */ + +#define zlib_version zlibVersion() +/* for compatibility with versions < 1.0.2 */ + +/* basic functions */ + +ZEXTERN const char *ZEXPORT zlibVersion OF((void)); +/* + * The application can compare zlibVersion and ZLIB_VERSION for consistency. + * If the first character differs, the library code actually used is + * not compatible with the zlib.h header file used by the application. + * This check is automatically made by deflateInit and inflateInit. + */ + +/* + * ZEXTERN int ZEXPORT deflateInit OF((z_streamp strm, int level)); + * + * Initializes the internal stream state for compression. The fields + * zalloc, zfree and opaque must be initialized before by the caller. + * If zalloc and zfree are set to Z_NULL, deflateInit updates them to + * use default allocation functions. + * + * The compression level must be Z_DEFAULT_COMPRESSION, or between 0 and 9: + * 1 gives best speed, 9 gives best compression, 0 gives no compression at + * all (the input data is simply copied a block at a time). + * Z_DEFAULT_COMPRESSION requests a default compromise between speed and + * compression (currently equivalent to level 6). + * + * deflateInit returns Z_OK if success, Z_MEM_ERROR if there was not + * enough memory, Z_STREAM_ERROR if level is not a valid compression level, + * Z_VERSION_ERROR if the zlib library version (zlib_version) is incompatible + * with the version assumed by the caller (ZLIB_VERSION). + * msg is set to null if there is no error message. deflateInit does not + * perform any compression: this will be done by deflate(). + */ + +ZEXTERN int ZEXPORT deflate OF((z_streamp strm, int flush)); +/* + * deflate compresses as much data as possible, and stops when the input + * buffer becomes empty or the output buffer becomes full. It may introduce some + * output latency (reading input without producing any output) except when + * forced to flush. + * + * The detailed semantics are as follows. deflate performs one or both of the + * following actions: + * + * - Compress more input starting at next_in and update next_in and avail_in + * accordingly. If not all input can be processed (because there is not + * enough room in the output buffer), next_in and avail_in are updated and + * processing will resume at this point for the next call of deflate(). + * + * - Provide more output starting at next_out and update next_out and avail_out + * accordingly. This action is forced if the parameter flush is non zero. + * Forcing flush frequently degrades the compression ratio, so this parameter + * should be set only when necessary (in interactive applications). + * Some output may be provided even if flush is not set. + * + * Before the call of deflate(), the application should ensure that at least + * one of the actions is possible, by providing more input and/or consuming + * more output, and updating avail_in or avail_out accordingly; avail_out + * should never be zero before the call. The application can consume the + * compressed output when it wants, for example when the output buffer is full + * (avail_out == 0), or after each call of deflate(). If deflate returns Z_OK + * and with zero avail_out, it must be called again after making room in the + * output buffer because there might be more output pending. + * + * Normally the parameter flush is set to Z_NO_FLUSH, which allows deflate to + * decide how much data to accumualte before producing output, in order to + * maximize compression. + * + * If the parameter flush is set to Z_SYNC_FLUSH, all pending output is + * flushed to the output buffer and the output is aligned on a byte boundary, so + * that the decompressor can get all input data available so far. (In particular + * avail_in is zero after the call if enough output space has been provided + * before the call.) Flushing may degrade compression for some compression + * algorithms and so it should be used only when necessary. + * + * If flush is set to Z_FULL_FLUSH, all output is flushed as with + * Z_SYNC_FLUSH, and the compression state is reset so that decompression can + * restart from this point if previous compressed data has been damaged or if + * random access is desired. Using Z_FULL_FLUSH too often can seriously degrade + * compression. + * + * If deflate returns with avail_out == 0, this function must be called again + * with the same value of the flush parameter and more output space (updated + * avail_out), until the flush is complete (deflate returns with non-zero + * avail_out). In the case of a Z_FULL_FLUSH or Z_SYNC_FLUSH, make sure that + * avail_out is greater than six to avoid repeated flush markers due to + * avail_out == 0 on return. + * + * If the parameter flush is set to Z_FINISH, pending input is processed, + * pending output is flushed and deflate returns with Z_STREAM_END if there + * was enough output space; if deflate returns with Z_OK, this function must be + * called again with Z_FINISH and more output space (updated avail_out) but no + * more input data, until it returns with Z_STREAM_END or an error. After + * deflate has returned Z_STREAM_END, the only possible operations on the + * stream are deflateReset or deflateEnd. + * + * Z_FINISH can be used immediately after deflateInit if all the compression + * is to be done in a single step. In this case, avail_out must be at least + * the value returned by deflateBound (see below). If deflate does not return + * Z_STREAM_END, then it must be called again as described above. + * + * deflate() sets strm->adler to the adler32 checksum of all input read + * so far (that is, total_in bytes). + * + * deflate() may update strm->data_type if it can make a good guess about + * the input data type (Z_BINARY or Z_TEXT). In doubt, the data is considered + * binary. This field is only for information purposes and does not affect + * the compression algorithm in any manner. + * + * deflate() returns Z_OK if some progress has been made (more input + * processed or more output produced), Z_STREAM_END if all input has been + * consumed and all output has been produced (only when flush is set to + * Z_FINISH), Z_STREAM_ERROR if the stream state was inconsistent (for example + * if next_in or next_out was NULL), Z_BUF_ERROR if no progress is possible + * (for example avail_in or avail_out was zero). Note that Z_BUF_ERROR is not + * fatal, and deflate() can be called again with more input and more output + * space to continue compressing. + */ + +ZEXTERN int ZEXPORT deflateEnd OF((z_streamp strm)); +/* + * All dynamically allocated data structures for this stream are freed. + * This function discards any unprocessed input and does not flush any + * pending output. + * + * deflateEnd returns Z_OK if success, Z_STREAM_ERROR if the + * stream state was inconsistent, Z_DATA_ERROR if the stream was freed + * prematurely (some input or output was discarded). In the error case, + * msg may be set but then points to a static string (which must not be + * deallocated). + */ + + +/* + * ZEXTERN int ZEXPORT inflateInit OF((z_streamp strm)); + * + * Initializes the internal stream state for decompression. The fields + * next_in, avail_in, zalloc, zfree and opaque must be initialized before by + * the caller. If next_in is not Z_NULL and avail_in is large enough (the exact + * value depends on the compression method), inflateInit determines the + * compression method from the zlib header and allocates all data structures + * accordingly; otherwise the allocation will be deferred to the first call of + * inflate. If zalloc and zfree are set to Z_NULL, inflateInit updates them to + * use default allocation functions. + * + * inflateInit returns Z_OK if success, Z_MEM_ERROR if there was not enough + * memory, Z_VERSION_ERROR if the zlib library version is incompatible with the + * version assumed by the caller. msg is set to null if there is no error + * message. inflateInit does not perform any decompression apart from reading + * the zlib header if present: this will be done by inflate(). (So next_in and + * avail_in may be modified, but next_out and avail_out are unchanged.) + */ + + +ZEXTERN int ZEXPORT inflate OF((z_streamp strm, int flush)); +/* + * inflate decompresses as much data as possible, and stops when the input + * buffer becomes empty or the output buffer becomes full. It may introduce + * some output latency (reading input without producing any output) except when + * forced to flush. + * + * The detailed semantics are as follows. inflate performs one or both of the + * following actions: + * + * - Decompress more input starting at next_in and update next_in and avail_in + * accordingly. If not all input can be processed (because there is not + * enough room in the output buffer), next_in is updated and processing + * will resume at this point for the next call of inflate(). + * + * - Provide more output starting at next_out and update next_out and avail_out + * accordingly. inflate() provides as much output as possible, until there + * is no more input data or no more space in the output buffer (see below + * about the flush parameter). + * + * Before the call of inflate(), the application should ensure that at least + * one of the actions is possible, by providing more input and/or consuming + * more output, and updating the next_* and avail_* values accordingly. + * The application can consume the uncompressed output when it wants, for + * example when the output buffer is full (avail_out == 0), or after each + * call of inflate(). If inflate returns Z_OK and with zero avail_out, it + * must be called again after making room in the output buffer because there + * might be more output pending. + * + * The flush parameter of inflate() can be Z_NO_FLUSH, Z_SYNC_FLUSH, + * Z_FINISH, or Z_BLOCK. Z_SYNC_FLUSH requests that inflate() flush as much + * output as possible to the output buffer. Z_BLOCK requests that inflate() stop + * if and when it gets to the next deflate block boundary. When decoding the + * zlib or gzip format, this will cause inflate() to return immediately after + * the header and before the first block. When doing a raw inflate, inflate() + * will go ahead and process the first block, and will return when it gets to + * the end of that block, or when it runs out of data. + * + * The Z_BLOCK option assists in appending to or combining deflate streams. + * Also to assist in this, on return inflate() will set strm->data_type to the + * number of unused bits in the last byte taken from strm->next_in, plus 64 + * if inflate() is currently decoding the last block in the deflate stream, + * plus 128 if inflate() returned immediately after decoding an end-of-block + * code or decoding the complete header up to just before the first byte of the + * deflate stream. The end-of-block will not be indicated until all of the + * uncompressed data from that block has been written to strm->next_out. The + * number of unused bits may in general be greater than seven, except when + * bit 7 of data_type is set, in which case the number of unused bits will be + * less than eight. + * + * inflate() should normally be called until it returns Z_STREAM_END or an + * error. However if all decompression is to be performed in a single step + * (a single call of inflate), the parameter flush should be set to + * Z_FINISH. In this case all pending input is processed and all pending + * output is flushed; avail_out must be large enough to hold all the + * uncompressed data. (The size of the uncompressed data may have been saved + * by the compressor for this purpose.) The next operation on this stream must + * be inflateEnd to deallocate the decompression state. The use of Z_FINISH + * is never required, but can be used to inform inflate that a faster approach + * may be used for the single inflate() call. + * + * In this implementation, inflate() always flushes as much output as + * possible to the output buffer, and always uses the faster approach on the + * first call. So the only effect of the flush parameter in this implementation + * is on the return value of inflate(), as noted below, or when it returns early + * because Z_BLOCK is used. + * + * If a preset dictionary is needed after this call (see inflateSetDictionary + * below), inflate sets strm->adler to the adler32 checksum of the dictionary + * chosen by the compressor and returns Z_NEED_DICT; otherwise it sets + * strm->adler to the adler32 checksum of all output produced so far (that is, + * total_out bytes) and returns Z_OK, Z_STREAM_END or an error code as described + * below. At the end of the stream, inflate() checks that its computed adler32 + * checksum is equal to that saved by the compressor and returns Z_STREAM_END + * only if the checksum is correct. + * + * inflate() will decompress and check either zlib-wrapped or gzip-wrapped + * deflate data. The header type is detected automatically. Any information + * contained in the gzip header is not retained, so applications that need that + * information should instead use raw inflate, see inflateInit2() below, or + * inflateBack() and perform their own processing of the gzip header and + * trailer. + * + * inflate() returns Z_OK if some progress has been made (more input processed + * or more output produced), Z_STREAM_END if the end of the compressed data has + * been reached and all uncompressed output has been produced, Z_NEED_DICT if a + * preset dictionary is needed at this point, Z_DATA_ERROR if the input data was + * corrupted (input stream not conforming to the zlib format or incorrect check + * value), Z_STREAM_ERROR if the stream structure was inconsistent (for example + * if next_in or next_out was NULL), Z_MEM_ERROR if there was not enough memory, + * Z_BUF_ERROR if no progress is possible or if there was not enough room in the + * output buffer when Z_FINISH is used. Note that Z_BUF_ERROR is not fatal, and + * inflate() can be called again with more input and more output space to + * continue decompressing. If Z_DATA_ERROR is returned, the application may then + * call inflateSync() to look for a good compression block if a partial recovery + * of the data is desired. + */ + + +ZEXTERN int ZEXPORT inflateEnd OF((z_streamp strm)); +/* + * All dynamically allocated data structures for this stream are freed. + * This function discards any unprocessed input and does not flush any + * pending output. + * + * inflateEnd returns Z_OK if success, Z_STREAM_ERROR if the stream state + * was inconsistent. In the error case, msg may be set but then points to a + * static string (which must not be deallocated). + */ + +/* Advanced functions */ + +/* + * The following functions are needed only in some special applications. + */ + +/* + * ZEXTERN int ZEXPORT deflateInit2 OF((z_streamp strm, + * int level, + * int method, + * int windowBits, + * int memLevel, + * int strategy)); + * + * This is another version of deflateInit with more compression options. The + * fields next_in, zalloc, zfree and opaque must be initialized before by + * the caller. + * + * The method parameter is the compression method. It must be Z_DEFLATED in + * this version of the library. + * + * The windowBits parameter is the base two logarithm of the window size + * (the size of the history buffer). It should be in the range 8..15 for this + * version of the library. Larger values of this parameter result in better + * compression at the expense of memory usage. The default value is 15 if + * deflateInit is used instead. + * + * windowBits can also be -8..-15 for raw deflate. In this case, -windowBits + * determines the window size. deflate() will then generate raw deflate data + * with no zlib header or trailer, and will not compute an adler32 check value. + * + * windowBits can also be greater than 15 for optional gzip encoding. Add + * 16 to windowBits to write a simple gzip header and trailer around the + * compressed data instead of a zlib wrapper. The gzip header will have no + * file name, no extra data, no comment, no modification time (set to zero), + * no header crc, and the operating system will be set to 255 (unknown). If a + * gzip stream is being written, strm->adler is a crc32 instead of an adler32. + * + * The memLevel parameter specifies how much memory should be allocated + * for the internal compression state. memLevel=1 uses minimum memory but + * is slow and reduces compression ratio; memLevel=9 uses maximum memory + * for optimal speed. The default value is 8. See zconf.h for total memory + * usage as a function of windowBits and memLevel. + * + * The strategy parameter is used to tune the compression algorithm. Use the + * value Z_DEFAULT_STRATEGY for normal data, Z_FILTERED for data produced by a + * filter (or predictor), Z_HUFFMAN_ONLY to force Huffman encoding only (no + * string match), or Z_RLE to limit match distances to one (run-length + * encoding). Filtered data consists mostly of small values with a somewhat + * random distribution. In this case, the compression algorithm is tuned to + * compress them better. The effect of Z_FILTERED is to force more Huffman + * coding and less string matching; it is somewhat intermediate between + * Z_DEFAULT and Z_HUFFMAN_ONLY. Z_RLE is designed to be almost as fast as + * Z_HUFFMAN_ONLY, but give better compression for PNG image data. The strategy + * parameter only affects the compression ratio but not the correctness of the + * compressed output even if it is not set appropriately. Z_FIXED prevents the + * use of dynamic Huffman codes, allowing for a simpler decoder for special + * applications. + * + * deflateInit2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + * memory, Z_STREAM_ERROR if a parameter is invalid (such as an invalid + * method). msg is set to null if there is no error message. deflateInit2 does + * not perform any compression: this will be done by deflate(). + */ + +ZEXTERN int ZEXPORT deflateSetDictionary OF((z_streamp strm, + const Bytef *dictionary, uInt dictLength)); +/* + * Initializes the compression dictionary from the given byte sequence + * without producing any compressed output. This function must be called + * immediately after deflateInit, deflateInit2 or deflateReset, before any + * call of deflate. The compressor and decompressor must use exactly the same + * dictionary (see inflateSetDictionary). + * + * The dictionary should consist of strings (byte sequences) that are likely + * to be encountered later in the data to be compressed, with the most commonly + * used strings preferably put towards the end of the dictionary. Using a + * dictionary is most useful when the data to be compressed is short and can be + * predicted with good accuracy; the data can then be compressed better than + * with the default empty dictionary. + * + * Depending on the size of the compression data structures selected by + * deflateInit or deflateInit2, a part of the dictionary may in effect be + * discarded, for example if the dictionary is larger than the window size in + * deflate or deflate2. Thus the strings most likely to be useful should be + * put at the end of the dictionary, not at the front. In addition, the + * current implementation of deflate will use at most the window size minus + * 262 bytes of the provided dictionary. + * + * Upon return of this function, strm->adler is set to the adler32 value + * of the dictionary; the decompressor may later use this value to determine + * which dictionary has been used by the compressor. (The adler32 value + * applies to the whole dictionary even if only a subset of the dictionary is + * actually used by the compressor.) If a raw deflate was requested, then the + * adler32 value is not computed and strm->adler is not set. + * + * deflateSetDictionary returns Z_OK if success, or Z_STREAM_ERROR if a + * parameter is invalid (such as NULL dictionary) or the stream state is + * inconsistent (for example if deflate has already been called for this stream + * or if the compression method is bsort). deflateSetDictionary does not + * perform any compression: this will be done by deflate(). + */ + +ZEXTERN int ZEXPORT deflateCopy OF((z_streamp dest, + z_streamp source)); +/* + * Sets the destination stream as a complete copy of the source stream. + * + * This function can be useful when several compression strategies will be + * tried, for example when there are several ways of pre-processing the input + * data with a filter. The streams that will be discarded should then be freed + * by calling deflateEnd. Note that deflateCopy duplicates the internal + * compression state which can be quite large, so this strategy is slow and + * can consume lots of memory. + * + * deflateCopy returns Z_OK if success, Z_MEM_ERROR if there was not + * enough memory, Z_STREAM_ERROR if the source stream state was inconsistent + * (such as zalloc being NULL). msg is left unchanged in both source and + * destination. + */ + +ZEXTERN int ZEXPORT deflateReset OF((z_streamp strm)); +/* + * This function is equivalent to deflateEnd followed by deflateInit, + * but does not free and reallocate all the internal compression state. + * The stream will keep the same compression level and any other attributes + * that may have been set by deflateInit2. + * + * deflateReset returns Z_OK if success, or Z_STREAM_ERROR if the source + * stream state was inconsistent (such as zalloc or state being NULL). + */ + +ZEXTERN int ZEXPORT deflateParams OF((z_streamp strm, + int level, int strategy)); +/* + * Dynamically update the compression level and compression strategy. The + * interpretation of level and strategy is as in deflateInit2. This can be + * used to switch between compression and straight copy of the input data, or + * to switch to a different kind of input data requiring a different + * strategy. If the compression level is changed, the input available so far + * is compressed with the old level (and may be flushed); the new level will + * take effect only at the next call of deflate(). + * + * Before the call of deflateParams, the stream state must be set as for + * a call of deflate(), since the currently available input may have to + * be compressed and flushed. In particular, strm->avail_out must be non-zero. + * + * deflateParams returns Z_OK if success, Z_STREAM_ERROR if the source + * stream state was inconsistent or if a parameter was invalid, Z_BUF_ERROR + * if strm->avail_out was zero. + */ + +ZEXTERN int ZEXPORT deflateTune OF((z_streamp strm, + int good_length, int max_lazy, int nice_length, int max_chain)); +/* + * Fine tune deflate's internal compression parameters. This should only be + * used by someone who understands the algorithm used by zlib's deflate for + * searching for the best matching string, and even then only by the most + * fanatic optimizer trying to squeeze out the last compressed bit for their + * specific input data. Read the deflate.c source code for the meaning of the + * max_lazy, good_length, nice_length, and max_chain parameters. + * + * deflateTune() can be called after deflateInit() or deflateInit2(), and + * returns Z_OK on success, or Z_STREAM_ERROR for an invalid deflate stream. + */ + +ZEXTERN uLong ZEXPORT deflateBound OF((z_streamp strm, + uLong sourceLen)); +/* + * deflateBound() returns an upper bound on the compressed size after + * deflation of sourceLen bytes. It must be called after deflateInit() + * or deflateInit2(). This would be used to allocate an output buffer + * for deflation in a single pass, and so would be called before deflate(). + */ + +ZEXTERN int ZEXPORT deflatePrime OF((z_streamp strm, + int bits, int value)); +/* + * deflatePrime() inserts bits in the deflate output stream. The intent + * is that this function is used to start off the deflate output with the + * bits leftover from a previous deflate stream when appending to it. As such, + * this function can only be used for raw deflate, and must be used before the + * first deflate() call after a deflateInit2() or deflateReset(). bits must be + * less than or equal to 16, and that many of the least significant bits of + * value will be inserted in the output. + * + * deflatePrime returns Z_OK if success, or Z_STREAM_ERROR if the source + * stream state was inconsistent. + */ + +ZEXTERN int ZEXPORT deflateSetHeader OF((z_streamp strm, + gz_headerp head)); +/* + * deflateSetHeader() provides gzip header information for when a gzip + * stream is requested by deflateInit2(). deflateSetHeader() may be called + * after deflateInit2() or deflateReset() and before the first call of + * deflate(). The text, time, os, extra field, name, and comment information + * in the provided gz_header structure are written to the gzip header (xflag is + * ignored -- the extra flags are set according to the compression level). The + * caller must assure that, if not Z_NULL, name and comment are terminated with + * a zero byte, and that if extra is not Z_NULL, that extra_len bytes are + * available there. If hcrc is true, a gzip header crc is included. Note that + * the current versions of the command-line version of gzip (up through version + * 1.3.x) do not support header crc's, and will report that it is a "multi-part + * gzip file" and give up. + * + * If deflateSetHeader is not used, the default gzip header has text false, + * the time set to zero, and os set to 255, with no extra, name, or comment + * fields. The gzip header is returned to the default state by deflateReset(). + * + * deflateSetHeader returns Z_OK if success, or Z_STREAM_ERROR if the source + * stream state was inconsistent. + */ + +/* + * ZEXTERN int ZEXPORT inflateInit2 OF((z_streamp strm, + * int windowBits)); + * + * This is another version of inflateInit with an extra parameter. The + * fields next_in, avail_in, zalloc, zfree and opaque must be initialized + * before by the caller. + * + * The windowBits parameter is the base two logarithm of the maximum window + * size (the size of the history buffer). It should be in the range 8..15 for + * this version of the library. The default value is 15 if inflateInit is used + * instead. windowBits must be greater than or equal to the windowBits value + * provided to deflateInit2() while compressing, or it must be equal to 15 if + * deflateInit2() was not used. If a compressed stream with a larger window + * size is given as input, inflate() will return with the error code + * Z_DATA_ERROR instead of trying to allocate a larger window. + * + * windowBits can also be -8..-15 for raw inflate. In this case, -windowBits + * determines the window size. inflate() will then process raw deflate data, + * not looking for a zlib or gzip header, not generating a check value, and not + * looking for any check values for comparison at the end of the stream. This + * is for use with other formats that use the deflate compressed data format + * such as zip. Those formats provide their own check values. If a custom + * format is developed using the raw deflate format for compressed data, it is + * recommended that a check value such as an adler32 or a crc32 be applied to + * the uncompressed data as is done in the zlib, gzip, and zip formats. For + * most applications, the zlib format should be used as is. Note that comments + * above on the use in deflateInit2() applies to the magnitude of windowBits. + * + * windowBits can also be greater than 15 for optional gzip decoding. Add + * 32 to windowBits to enable zlib and gzip decoding with automatic header + * detection, or add 16 to decode only the gzip format (the zlib format will + * return a Z_DATA_ERROR). If a gzip stream is being decoded, strm->adler is + * a crc32 instead of an adler32. + * + * inflateInit2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + * memory, Z_STREAM_ERROR if a parameter is invalid (such as a null strm). msg + * is set to null if there is no error message. inflateInit2 does not perform + * any decompression apart from reading the zlib header if present: this will + * be done by inflate(). (So next_in and avail_in may be modified, but next_out + * and avail_out are unchanged.) + */ + +ZEXTERN int ZEXPORT inflateSetDictionary OF((z_streamp strm, + const Bytef *dictionary, uInt dictLength)); +/* + * Initializes the decompression dictionary from the given uncompressed byte + * sequence. This function must be called immediately after a call of inflate, + * if that call returned Z_NEED_DICT. The dictionary chosen by the compressor + * can be determined from the adler32 value returned by that call of inflate. + * The compressor and decompressor must use exactly the same dictionary (see + * deflateSetDictionary). For raw inflate, this function can be called + * immediately after inflateInit2() or inflateReset() and before any call of + * inflate() to set the dictionary. The application must insure that the + * dictionary that was used for compression is provided. + * + * inflateSetDictionary returns Z_OK if success, Z_STREAM_ERROR if a + * parameter is invalid (such as NULL dictionary) or the stream state is + * inconsistent, Z_DATA_ERROR if the given dictionary doesn't match the + * expected one (incorrect adler32 value). inflateSetDictionary does not + * perform any decompression: this will be done by subsequent calls of + * inflate(). + */ + +ZEXTERN int ZEXPORT inflateSync OF((z_streamp strm)); +/* + * Skips invalid compressed data until a full flush point (see above the + * description of deflate with Z_FULL_FLUSH) can be found, or until all + * available input is skipped. No output is provided. + * + * inflateSync returns Z_OK if a full flush point has been found, Z_BUF_ERROR + * if no more input was provided, Z_DATA_ERROR if no flush point has been found, + * or Z_STREAM_ERROR if the stream structure was inconsistent. In the success + * case, the application may save the current current value of total_in which + * indicates where valid compressed data was found. In the error case, the + * application may repeatedly call inflateSync, providing more input each time, + * until success or end of the input data. + */ + +ZEXTERN int ZEXPORT inflateCopy OF((z_streamp dest, + z_streamp source)); +/* + * Sets the destination stream as a complete copy of the source stream. + * + * This function can be useful when randomly accessing a large stream. The + * first pass through the stream can periodically record the inflate state, + * allowing restarting inflate at those points when randomly accessing the + * stream. + * + * inflateCopy returns Z_OK if success, Z_MEM_ERROR if there was not + * enough memory, Z_STREAM_ERROR if the source stream state was inconsistent + * (such as zalloc being NULL). msg is left unchanged in both source and + * destination. + */ + +ZEXTERN int ZEXPORT inflateReset OF((z_streamp strm)); +/* + * This function is equivalent to inflateEnd followed by inflateInit, + * but does not free and reallocate all the internal decompression state. + * The stream will keep attributes that may have been set by inflateInit2. + * + * inflateReset returns Z_OK if success, or Z_STREAM_ERROR if the source + * stream state was inconsistent (such as zalloc or state being NULL). + */ + +ZEXTERN int ZEXPORT inflatePrime OF((z_streamp strm, + int bits, int value)); +/* + * This function inserts bits in the inflate input stream. The intent is + * that this function is used to start inflating at a bit position in the + * middle of a byte. The provided bits will be used before any bytes are used + * from next_in. This function should only be used with raw inflate, and + * should be used before the first inflate() call after inflateInit2() or + * inflateReset(). bits must be less than or equal to 16, and that many of the + * least significant bits of value will be inserted in the input. + * + * inflatePrime returns Z_OK if success, or Z_STREAM_ERROR if the source + * stream state was inconsistent. + */ + +ZEXTERN int ZEXPORT inflateGetHeader OF((z_streamp strm, + gz_headerp head)); +/* + * inflateGetHeader() requests that gzip header information be stored in the + * provided gz_header structure. inflateGetHeader() may be called after + * inflateInit2() or inflateReset(), and before the first call of inflate(). + * As inflate() processes the gzip stream, head->done is zero until the header + * is completed, at which time head->done is set to one. If a zlib stream is + * being decoded, then head->done is set to -1 to indicate that there will be + * no gzip header information forthcoming. Note that Z_BLOCK can be used to + * force inflate() to return immediately after header processing is complete + * and before any actual data is decompressed. + * + * The text, time, xflags, and os fields are filled in with the gzip header + * contents. hcrc is set to true if there is a header CRC. (The header CRC + * was valid if done is set to one.) If extra is not Z_NULL, then extra_max + * contains the maximum number of bytes to write to extra. Once done is true, + * extra_len contains the actual extra field length, and extra contains the + * extra field, or that field truncated if extra_max is less than extra_len. + * If name is not Z_NULL, then up to name_max characters are written there, + * terminated with a zero unless the length is greater than name_max. If + * comment is not Z_NULL, then up to comm_max characters are written there, + * terminated with a zero unless the length is greater than comm_max. When + * any of extra, name, or comment are not Z_NULL and the respective field is + * not present in the header, then that field is set to Z_NULL to signal its + * absence. This allows the use of deflateSetHeader() with the returned + * structure to duplicate the header. However if those fields are set to + * allocated memory, then the application will need to save those pointers + * elsewhere so that they can be eventually freed. + * + * If inflateGetHeader is not used, then the header information is simply + * discarded. The header is always checked for validity, including the header + * CRC if present. inflateReset() will reset the process to discard the header + * information. The application would need to call inflateGetHeader() again to + * retrieve the header from the next gzip stream. + * + * inflateGetHeader returns Z_OK if success, or Z_STREAM_ERROR if the source + * stream state was inconsistent. + */ + +/* + * ZEXTERN int ZEXPORT inflateBackInit OF((z_streamp strm, int windowBits, + * unsigned char FAR *window)); + * + * Initialize the internal stream state for decompression using inflateBack() + * calls. The fields zalloc, zfree and opaque in strm must be initialized + * before the call. If zalloc and zfree are Z_NULL, then the default library- + * derived memory allocation routines are used. windowBits is the base two + * logarithm of the window size, in the range 8..15. window is a caller + * supplied buffer of that size. Except for special applications where it is + * assured that deflate was used with small window sizes, windowBits must be 15 + * and a 32K byte window must be supplied to be able to decompress general + * deflate streams. + * + * See inflateBack() for the usage of these routines. + * + * inflateBackInit will return Z_OK on success, Z_STREAM_ERROR if any of + * the paramaters are invalid, Z_MEM_ERROR if the internal state could not + * be allocated, or Z_VERSION_ERROR if the version of the library does not + * match the version of the header file. + */ + +typedef unsigned (*in_func) OF((void FAR *, unsigned char FAR * FAR *)); +typedef int (*out_func) OF((void FAR *, unsigned char FAR *, unsigned)); + +ZEXTERN int ZEXPORT inflateBack OF((z_streamp strm, + in_func in, void FAR *in_desc, + out_func out, void FAR *out_desc)); +/* + * inflateBack() does a raw inflate with a single call using a call-back + * interface for input and output. This is more efficient than inflate() for + * file i/o applications in that it avoids copying between the output and the + * sliding window by simply making the window itself the output buffer. This + * function trusts the application to not change the output buffer passed by + * the output function, at least until inflateBack() returns. + * + * inflateBackInit() must be called first to allocate the internal state + * and to initialize the state with the user-provided window buffer. + * inflateBack() may then be used multiple times to inflate a complete, raw + * deflate stream with each call. inflateBackEnd() is then called to free + * the allocated state. + * + * A raw deflate stream is one with no zlib or gzip header or trailer. + * This routine would normally be used in a utility that reads zip or gzip + * files and writes out uncompressed files. The utility would decode the + * header and process the trailer on its own, hence this routine expects + * only the raw deflate stream to decompress. This is different from the + * normal behavior of inflate(), which expects either a zlib or gzip header and + * trailer around the deflate stream. + * + * inflateBack() uses two subroutines supplied by the caller that are then + * called by inflateBack() for input and output. inflateBack() calls those + * routines until it reads a complete deflate stream and writes out all of the + * uncompressed data, or until it encounters an error. The function's + * parameters and return types are defined above in the in_func and out_func + * typedefs. inflateBack() will call in(in_desc, &buf) which should return the + * number of bytes of provided input, and a pointer to that input in buf. If + * there is no input available, in() must return zero--buf is ignored in that + * case--and inflateBack() will return a buffer error. inflateBack() will call + * out(out_desc, buf, len) to write the uncompressed data buf[0..len-1]. out() + * should return zero on success, or non-zero on failure. If out() returns + * non-zero, inflateBack() will return with an error. Neither in() nor out() + * are permitted to change the contents of the window provided to + * inflateBackInit(), which is also the buffer that out() uses to write from. + * The length written by out() will be at most the window size. Any non-zero + * amount of input may be provided by in(). + * + * For convenience, inflateBack() can be provided input on the first call by + * setting strm->next_in and strm->avail_in. If that input is exhausted, then + * in() will be called. Therefore strm->next_in must be initialized before + * calling inflateBack(). If strm->next_in is Z_NULL, then in() will be called + * immediately for input. If strm->next_in is not Z_NULL, then strm->avail_in + * must also be initialized, and then if strm->avail_in is not zero, input will + * initially be taken from strm->next_in[0 .. strm->avail_in - 1]. + * + * The in_desc and out_desc parameters of inflateBack() is passed as the + * first parameter of in() and out() respectively when they are called. These + * descriptors can be optionally used to pass any information that the caller- + * supplied in() and out() functions need to do their job. + * + * On return, inflateBack() will set strm->next_in and strm->avail_in to + * pass back any unused input that was provided by the last in() call. The + * return values of inflateBack() can be Z_STREAM_END on success, Z_BUF_ERROR + * if in() or out() returned an error, Z_DATA_ERROR if there was a format + * error in the deflate stream (in which case strm->msg is set to indicate the + * nature of the error), or Z_STREAM_ERROR if the stream was not properly + * initialized. In the case of Z_BUF_ERROR, an input or output error can be + * distinguished using strm->next_in which will be Z_NULL only if in() returned + * an error. If strm->next is not Z_NULL, then the Z_BUF_ERROR was due to + * out() returning non-zero. (in() will always be called before out(), so + * strm->next_in is assured to be defined if out() returns non-zero.) Note + * that inflateBack() cannot return Z_OK. + */ + +ZEXTERN int ZEXPORT inflateBackEnd OF((z_streamp strm)); +/* + * All memory allocated by inflateBackInit() is freed. + * + * inflateBackEnd() returns Z_OK on success, or Z_STREAM_ERROR if the stream + * state was inconsistent. + */ + +ZEXTERN uLong ZEXPORT zlibCompileFlags OF((void)); +/* + * Return flags indicating compile-time options. + * + * Type sizes, two bits each, 00 = 16 bits, 01 = 32, 10 = 64, 11 = other: + * 1.0: size of uInt + * 3.2: size of uLong + * 5.4: size of voidpf (pointer) + * 7.6: size of z_off_t + * + * Compiler, assembler, and debug options: + * 8: DEBUG + * 9: ASMV or ASMINF -- use ASM code + * 10: ZLIB_WINAPI -- exported functions use the WINAPI calling convention + * 11: 0 (reserved) + * + * One-time table building (smaller code, but not thread-safe if true): + * 12: BUILDFIXED -- build static block decoding tables when needed + * 13: DYNAMIC_CRC_TABLE -- build CRC calculation tables when needed + * 14,15: 0 (reserved) + * + * Library content (indicates missing functionality): + * 16: NO_GZCOMPRESS -- gz* functions cannot compress (to avoid linking + * deflate code when not needed) + * 17: NO_GZIP -- deflate can't write gzip streams, and inflate can't detect + * and decode gzip streams (to avoid linking crc code) + * 18-19: 0 (reserved) + * + * Operation variations (changes in library functionality): + * 20: PKZIP_BUG_WORKAROUND -- slightly more permissive inflate + * 21: FASTEST -- deflate algorithm with only one, lowest compression level + * 22,23: 0 (reserved) + * + * The sprintf variant used by gzprintf (zero is best): + * 24: 0 = vs*, 1 = s* -- 1 means limited to 20 arguments after the format + * 25: 0 = *nprintf, 1 = *printf -- 1 means gzprintf() not secure! + * 26: 0 = returns value, 1 = void -- 1 means inferred string length returned + * + * Remainder: + * 27-31: 0 (reserved) + */ + + +/* utility functions */ + +/* + * The following utility functions are implemented on top of the + * basic stream-oriented functions. To simplify the interface, some + * default options are assumed (compression level and memory usage, + * standard memory allocation functions). The source code of these + * utility functions can easily be modified if you need special options. + */ + +ZEXTERN int ZEXPORT compress OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen)); +/* + * Compresses the source buffer into the destination buffer. sourceLen is + * the byte length of the source buffer. Upon entry, destLen is the total + * size of the destination buffer, which must be at least the value returned + * by compressBound(sourceLen). Upon exit, destLen is the actual size of the + * compressed buffer. + * This function can be used to compress a whole file at once if the + * input file is mmap'ed. + * compress returns Z_OK if success, Z_MEM_ERROR if there was not + * enough memory, Z_BUF_ERROR if there was not enough room in the output + * buffer. + */ + +ZEXTERN int ZEXPORT compress2 OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen, + int level)); +/* + * Compresses the source buffer into the destination buffer. The level + * parameter has the same meaning as in deflateInit. sourceLen is the byte + * length of the source buffer. Upon entry, destLen is the total size of the + * destination buffer, which must be at least the value returned by + * compressBound(sourceLen). Upon exit, destLen is the actual size of the + * compressed buffer. + * + * compress2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + * memory, Z_BUF_ERROR if there was not enough room in the output buffer, + * Z_STREAM_ERROR if the level parameter is invalid. + */ + +ZEXTERN uLong ZEXPORT compressBound OF((uLong sourceLen)); +/* + * compressBound() returns an upper bound on the compressed size after + * compress() or compress2() on sourceLen bytes. It would be used before + * a compress() or compress2() call to allocate the destination buffer. + */ + +ZEXTERN int ZEXPORT uncompress OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen)); +/* + * Decompresses the source buffer into the destination buffer. sourceLen is + * the byte length of the source buffer. Upon entry, destLen is the total + * size of the destination buffer, which must be large enough to hold the + * entire uncompressed data. (The size of the uncompressed data must have + * been saved previously by the compressor and transmitted to the decompressor + * by some mechanism outside the scope of this compression library.) + * Upon exit, destLen is the actual size of the compressed buffer. + * This function can be used to decompress a whole file at once if the + * input file is mmap'ed. + * + * uncompress returns Z_OK if success, Z_MEM_ERROR if there was not + * enough memory, Z_BUF_ERROR if there was not enough room in the output + * buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete. + */ + +typedef voidp gzFile; + +ZEXTERN gzFile ZEXPORT gzopen OF((const char *path, const char *mode)); +/* + * Opens a gzip (.gz) file for reading or writing. The mode parameter + * is as in fopen ("rb" or "wb") but can also include a compression level + * ("wb9") or a strategy: 'f' for filtered data as in "wb6f", 'h' for + * Huffman only compression as in "wb1h", or 'R' for run-length encoding + * as in "wb1R". (See the description of deflateInit2 for more information + * about the strategy parameter.) + * + * gzopen can be used to read a file which is not in gzip format; in this + * case gzread will directly read from the file without decompression. + * + * gzopen returns NULL if the file could not be opened or if there was + * insufficient memory to allocate the (de)compression state; errno + * can be checked to distinguish the two cases (if errno is zero, the + * zlib error is Z_MEM_ERROR). + */ + +ZEXTERN gzFile ZEXPORT gzdopen OF((int fd, const char *mode)); +/* + * gzdopen() associates a gzFile with the file descriptor fd. File + * descriptors are obtained from calls like open, dup, creat, pipe or + * fileno (in the file has been previously opened with fopen). + * The mode parameter is as in gzopen. + * The next call of gzclose on the returned gzFile will also close the + * file descriptor fd, just like fclose(fdopen(fd), mode) closes the file + * descriptor fd. If you want to keep fd open, use gzdopen(dup(fd), mode). + * gzdopen returns NULL if there was insufficient memory to allocate + * the (de)compression state. + * + */ + +ZEXTERN int ZEXPORT gzsetparams OF((gzFile file, int level, int strategy)); +/* + * Dynamically update the compression level or strategy. See the description + * of deflateInit2 for the meaning of these parameters. + * gzsetparams returns Z_OK if success, or Z_STREAM_ERROR if the file was not + * opened for writing. + */ + +ZEXTERN int ZEXPORT gzread OF((gzFile file, voidp buf, unsigned len)); +/* + * Reads the given number of uncompressed bytes from the compressed file. + * If the input file was not in gzip format, gzread copies the given number + * of bytes into the buffer. + * gzread returns the number of uncompressed bytes actually read (0 for + * end of file, -1 for error). + */ + +ZEXTERN int ZEXPORT gzwrite OF((gzFile file, + voidpc buf, unsigned len)); +/* + * Writes the given number of uncompressed bytes into the compressed file. + * gzwrite returns the number of uncompressed bytes actually written + * (0 in case of error). + */ + +ZEXTERN int ZEXPORTVA gzprintf OF((gzFile file, const char *format, ...)); +/* + * Converts, formats, and writes the args to the compressed file under + * control of the format string, as in fprintf. gzprintf returns the number of + * uncompressed bytes actually written (0 in case of error). The number of + * uncompressed bytes written is limited to 4095. The caller should assure that + * this limit is not exceeded. If it is exceeded, then gzprintf() will return + * return an error (0) with nothing written. In this case, there may also be a + * buffer overflow with unpredictable consequences, which is possible only if + * zlib was compiled with the insecure functions sprintf() or vsprintf() + * because the secure snprintf() or vsnprintf() functions were not available. + */ + +ZEXTERN int ZEXPORT gzputs OF((gzFile file, const char *s)); +/* + * Writes the given null-terminated string to the compressed file, excluding + * the terminating null character. + * gzputs returns the number of characters written, or -1 in case of error. + */ + +ZEXTERN char *ZEXPORT gzgets OF((gzFile file, char *buf, int len)); +/* + * Reads bytes from the compressed file until len-1 characters are read, or + * a newline character is read and transferred to buf, or an end-of-file + * condition is encountered. The string is then terminated with a null + * character. + * gzgets returns buf, or Z_NULL in case of error. + */ + +ZEXTERN int ZEXPORT gzputc OF((gzFile file, int c)); +/* + * Writes c, converted to an unsigned char, into the compressed file. + * gzputc returns the value that was written, or -1 in case of error. + */ + +ZEXTERN int ZEXPORT gzgetc OF((gzFile file)); +/* + * Reads one byte from the compressed file. gzgetc returns this byte + * or -1 in case of end of file or error. + */ + +ZEXTERN int ZEXPORT gzungetc OF((int c, gzFile file)); +/* + * Push one character back onto the stream to be read again later. + * Only one character of push-back is allowed. gzungetc() returns the + * character pushed, or -1 on failure. gzungetc() will fail if a + * character has been pushed but not read yet, or if c is -1. The pushed + * character will be discarded if the stream is repositioned with gzseek() + * or gzrewind(). + */ + +ZEXTERN int ZEXPORT gzflush OF((gzFile file, int flush)); +/* + * Flushes all pending output into the compressed file. The parameter + * flush is as in the deflate() function. The return value is the zlib + * error number (see function gzerror below). gzflush returns Z_OK if + * the flush parameter is Z_FINISH and all output could be flushed. + * gzflush should be called only when strictly necessary because it can + * degrade compression. + */ + +ZEXTERN z_off_t ZEXPORT gzseek OF((gzFile file, + z_off_t offset, int whence)); +/* + * Sets the starting position for the next gzread or gzwrite on the + * given compressed file. The offset represents a number of bytes in the + * uncompressed data stream. The whence parameter is defined as in lseek(2); + * the value SEEK_END is not supported. + * If the file is opened for reading, this function is emulated but can be + * extremely slow. If the file is opened for writing, only forward seeks are + * supported; gzseek then compresses a sequence of zeroes up to the new + * starting position. + * + * gzseek returns the resulting offset location as measured in bytes from + * the beginning of the uncompressed stream, or -1 in case of error, in + * particular if the file is opened for writing and the new starting position + * would be before the current position. + */ + +ZEXTERN int ZEXPORT gzrewind OF((gzFile file)); +/* + * Rewinds the given file. This function is supported only for reading. + * + * gzrewind(file) is equivalent to (int)gzseek(file, 0L, SEEK_SET) + */ + +ZEXTERN z_off_t ZEXPORT gztell OF((gzFile file)); +/* + * Returns the starting position for the next gzread or gzwrite on the + * given compressed file. This position represents a number of bytes in the + * uncompressed data stream. + * + * gztell(file) is equivalent to gzseek(file, 0L, SEEK_CUR) + */ + +ZEXTERN int ZEXPORT gzeof OF((gzFile file)); +/* + * Returns 1 when EOF has previously been detected reading the given + * input stream, otherwise zero. + */ + +ZEXTERN int ZEXPORT gzdirect OF((gzFile file)); +/* + * Returns 1 if file is being read directly without decompression, otherwise + * zero. + */ + +ZEXTERN int ZEXPORT gzclose OF((gzFile file)); +/* + * Flushes all pending output if necessary, closes the compressed file + * and deallocates all the (de)compression state. The return value is the zlib + * error number (see function gzerror below). + */ + +ZEXTERN const char *ZEXPORT gzerror OF((gzFile file, int *errnum)); +/* + * Returns the error message for the last error which occurred on the + * given compressed file. errnum is set to zlib error number. If an + * error occurred in the file system and not in the compression library, + * errnum is set to Z_ERRNO and the application may consult errno + * to get the exact error code. + */ + +ZEXTERN void ZEXPORT gzclearerr OF((gzFile file)); +/* + * Clears the error and end-of-file flags for file. This is analogous to the + * clearerr() function in stdio. This is useful for continuing to read a gzip + * file that is being written concurrently. + */ + +/* checksum functions */ + +/* + * These functions are not related to compression but are exported + * anyway because they might be useful in applications using the + * compression library. + */ + +ZEXTERN uLong ZEXPORT adler32 OF((uLong adler, const Bytef *buf, uInt len)); +/* + * Update a running Adler-32 checksum with the bytes buf[0..len-1] and + * return the updated checksum. If buf is NULL, this function returns + * the required initial value for the checksum. + * An Adler-32 checksum is almost as reliable as a CRC32 but can be computed + * much faster. Usage example: + * + * uLong adler = adler32(0L, Z_NULL, 0); + * + * while (read_buffer(buffer, length) != EOF) { + * adler = adler32(adler, buffer, length); + * } + * if (adler != original_adler) error(); + */ + +ZEXTERN uLong ZEXPORT adler32_combine OF((uLong adler1, uLong adler2, + z_off_t len2)); +/* + * Combine two Adler-32 checksums into one. For two sequences of bytes, seq1 + * and seq2 with lengths len1 and len2, Adler-32 checksums were calculated for + * each, adler1 and adler2. adler32_combine() returns the Adler-32 checksum of + * seq1 and seq2 concatenated, requiring only adler1, adler2, and len2. + */ + +ZEXTERN uLong ZEXPORT crc32 OF((uLong crc, const Bytef *buf, uInt len)); +/* + * Update a running CRC-32 with the bytes buf[0..len-1] and return the + * updated CRC-32. If buf is NULL, this function returns the required initial + * value for the for the crc. Pre- and post-conditioning (one's complement) is + * performed within this function so it shouldn't be done by the application. + * Usage example: + * + * uLong crc = crc32(0L, Z_NULL, 0); + * + * while (read_buffer(buffer, length) != EOF) { + * crc = crc32(crc, buffer, length); + * } + * if (crc != original_crc) error(); + */ + +ZEXTERN uLong ZEXPORT crc32_combine OF((uLong crc1, uLong crc2, z_off_t len2)); + +/* + * Combine two CRC-32 check values into one. For two sequences of bytes, + * seq1 and seq2 with lengths len1 and len2, CRC-32 check values were + * calculated for each, crc1 and crc2. crc32_combine() returns the CRC-32 + * check value of seq1 and seq2 concatenated, requiring only crc1, crc2, and + * len2. + */ + + +/* various hacks, don't look :) */ + +/* + * deflateInit and inflateInit are macros to allow checking the zlib version + * and the compiler's view of z_stream: + */ +ZEXTERN int ZEXPORT deflateInit_ OF((z_streamp strm, int level, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT inflateInit_ OF((z_streamp strm, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT deflateInit2_ OF((z_streamp strm, int level, int method, + int windowBits, int memLevel, + int strategy, const char *version, + int stream_size)); +ZEXTERN int ZEXPORT inflateInit2_ OF((z_streamp strm, int windowBits, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT inflateBackInit_ OF((z_streamp strm, int windowBits, + unsigned char FAR *window, + const char *version, + int stream_size)); +#define deflateInit(strm, level) \ + deflateInit_((strm), (level), ZLIB_VERSION, sizeof (z_stream)) +#define inflateInit(strm) \ + inflateInit_((strm), ZLIB_VERSION, sizeof (z_stream)) +#define deflateInit2(strm, level, method, windowBits, memLevel, strategy) \ + deflateInit2_((strm), (level), (method), (windowBits), (memLevel), \ + (strategy), ZLIB_VERSION, sizeof (z_stream)) +#define inflateInit2(strm, windowBits) \ + inflateInit2_((strm), (windowBits), ZLIB_VERSION, sizeof (z_stream)) +#define inflateBackInit(strm, windowBits, window) \ + inflateBackInit_((strm), (windowBits), (window), \ + ZLIB_VERSION, sizeof (z_stream)) + + +#if !defined(_ZUTIL_H) && !defined(NO_DUMMY_DECL) + struct internal_state {int dummy; }; /* hack for buggy compilers */ +#endif + +ZEXTERN const char *ZEXPORT zError OF((int)); +ZEXTERN int ZEXPORT inflateSyncPoint OF((z_streamp z)); +ZEXTERN const uLongf *ZEXPORT get_crc_table OF((void)); + +#ifdef __cplusplus +} +#endif + +#endif /* _ZLIB_H */ diff --git a/include/os/windows/zfs/sys/zpl.h b/include/os/windows/zfs/sys/zpl.h new file mode 100644 index 000000000000..5d391c6a960c --- /dev/null +++ b/include/os/windows/zfs/sys/zpl.h @@ -0,0 +1,27 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#ifndef _SYS_ZPL_H +#define _SYS_ZPL_H + + + + +#endif // _SYS_ZPL_H diff --git a/include/os/windows/zfs/sys/zvol_os.h b/include/os/windows/zfs/sys/zvol_os.h new file mode 100644 index 000000000000..283778d24cc6 --- /dev/null +++ b/include/os/windows/zfs/sys/zvol_os.h @@ -0,0 +1,83 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* Copyright (c) 2015 Jorgen Lundman */ + +#ifndef _SYS_ZVOL_OS_h +#define _SYS_ZVOL_OS_h + +#ifdef __cplusplus +extern "C" { +#endif + +/* struct wrapper for IOKit class */ +typedef struct zvol_iokit zvol_iokit_t; +typedef struct zvol_state zvol_state_t; +struct iomem; + +#define ZVOL_WRITE_SYNC 0x10 + +struct zvol_state_os { + dev_t zso_dev; /* device id */ + + uint32_t zso_open_count; /* open counts */ + uint64_t zso_openflags; /* Remember flags used at open */ + uint8_t zso_target_id; + uint8_t zso_lun_id; + // for I/O drainage (see assign_targetid, clear_targetid) + void *zso_target_context; +}; + +extern int zvol_os_ioctl(dev_t, unsigned long, caddr_t, + int isblk, cred_t *, int *rvalp); +extern int zvol_os_open_zv(zvol_state_t *, int, int, struct proc *p); +extern int zvol_os_open(dev_t dev, int flag, int otyp, struct proc *p); +extern int zvol_os_close_zv(zvol_state_t *, int, int, struct proc *p); +extern int zvol_os_close(dev_t dev, int flag, int otyp, struct proc *p); +extern int zvol_os_read(dev_t dev, zfs_uio_t *uio, int p); +extern int zvol_os_write(dev_t dev, zfs_uio_t *uio, int p); + +extern int zvol_os_read_zv(zvol_state_t *zv, zfs_uio_t *, int); +extern int zvol_os_write_zv(zvol_state_t *zv, zfs_uio_t *, int); +extern int zvol_os_unmap(zvol_state_t *zv, uint64_t off, uint64_t bytes); + +extern void zvol_os_strategy(struct buf *bp); +extern int zvol_os_get_volume_blocksize(dev_t dev); + +extern void zvol_os_lock_zv(zvol_state_t *zv); +extern void zvol_os_unlock_zv(zvol_state_t *zv); + +extern void *zvolRemoveDevice(zvol_iokit_t *iokitdev); +extern int zvolRemoveDeviceTerminate(void *iokitdev); +extern int zvolCreateNewDevice(zvol_state_t *zv); +extern int zvolRegisterDevice(zvol_state_t *zv); + +extern int zvolRenameDevice(zvol_state_t *zv); +extern int zvolSetVolsize(zvol_state_t *zv); +extern void zvol_os_attach(char *name); +extern void zvol_os_detach_zv(zvol_state_t *zv); +extern void zvol_os_detach(char *name); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/include/os/windows/zfs/zfs_config.h b/include/os/windows/zfs/zfs_config.h new file mode 100644 index 000000000000..d1b08be6a3c1 --- /dev/null +++ b/include/os/windows/zfs/zfs_config.h @@ -0,0 +1,152 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + + /* zfs_config.h. This file is not autogenerated on Windows. */ + + /* Copyright(c) 2015 Jorgen Lundman */ + +#include + +/* Define to 1 to enabled dmu tx validation */ +/* #undef DEBUG_DMU_TX */ + +#define SYSCONFDIR "\\SystemRoot\\System32\\drivers" // Windwosify me +#define PKGDATADIR "\\SystemRoot\\System32\\drivers" // Windwosify me + +#define TEXT_DOMAIN "zfs-windows-user" + +/* Path where the Filesystems bundle is installed. */ +#define FILESYSTEMS_PREFIX "\\SystemRoot\\System32\\drivers" + +/* Define to 1 if you have the header file. */ +#define HAVE_DLFCN_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_INTTYPES_H 1 + +/* Define if you have libblkid */ +/* #undef HAVE_LIBBLKID */ + +/* Define if you have libuuid */ +#define HAVE_LIBUUID 1 + +#define HAVE_XDR_BYTESREC 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_MEMORY_H 1 + +/* Define to 1 if you have the `mlockall' function. */ +#define HAVE_MLOCKALL 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STDINT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STDLIB_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRINGS_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRING_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_UNISTD_H 1 + +#define HAVE_SETMNTENT + +/* Define if you have zlib */ +#define HAVE_ZLIB 1 + +#define HAVE_LOFF_T 1 + +#define HAVE_USLEEP 1 + +/* These control which assembler files to use */ +#define HAVE_SSE2 1 +#define HAVE_SSSE3 1 +#define HAVE_SSE4_1 +#define HAVE_AVX 1 +#define HAVE_AVX2 1 +#define HAVE_PCLMULQDQ 1 +#define HAVE_MOVBE 1 +#define HAVE_SHA_NI 1 +#define HAVE_AES 1 +#define HAVE_AVX512F 1 +#define HAVE_AVX512CD 1 +#define HAVE_AVX512ER 1 +#define HAVE_AVX512BW 1 +#define HAVE_AVX512DQ 1 +#define HAVE_AVX512VL 1 +#define HAVE_AVX512IFMA 1 +#define HAVE_AVX512VBMI 1 +#define HAVE_AVX512PF 1 + + +/* Path where the kernel module is installed. */ +#define KERNEL_MODPREFIX "/Library/Extensions" + +/* Define to the sub-directory where libtool stores uninstalled libraries. */ +#define LT_OBJDIR ".libs/" + +/* Define to a directory where mount(2) will look for mount_zfs. */ +#define MOUNTEXECDIR "${exec_prefix}/sbin" + +/* Define ZFS_BOOT to enable kext load at boot */ +#define ZFS_BOOT 1 + +/* zfs debugging enabled */ + +/* Define the project author. */ +#define ZFS_META_AUTHOR "OpenZFS on Windows" + +/* Define the project release date. */ +/* #undef ZFS_META_DATA */ + +/* Define the project license. */ +#define ZFS_META_LICENSE "CDDL" + +/* Define the libtool library 'age' version information. */ +/* #undef ZFS_META_LT_AGE */ + +/* Define the libtool library 'current' version information. */ +/* #undef ZFS_META_LT_CURRENT */ + +/* Define the libtool library 'revision' version information. */ +/* #undef ZFS_META_LT_REVISION */ + +/* Define the project name. */ +#define ZFS_META_NAME "zfs" + +/* Define the project release. */ +#define ZFS_META_RELEASE "2" + +/* Define the project version. */ +#define ZFS_META_VERSION "2.1.99" + +/* Define the project alias string. */ +#define ZFS_META_ALIAS ZFS_META_GITREV diff --git a/include/sys/abd.h b/include/sys/abd.h index 750f9986c1da..62db375b5638 100644 --- a/include/sys/abd.h +++ b/include/sys/abd.h @@ -60,7 +60,9 @@ typedef struct abd { union { struct abd_scatter { uint_t abd_offset; -#if defined(__FreeBSD__) && defined(_KERNEL) +#if defined(_KERNEL) && \ + (defined(__FreeBSD__) || defined(__APPLE__) || defined(_WIN32)) + uint_t abd_chunk_size; void *abd_chunks[1]; /* actually variable-length */ #else uint_t abd_nents; diff --git a/include/sys/abd_impl.h b/include/sys/abd_impl.h index 40546d4af137..ebce2f014747 100644 --- a/include/sys/abd_impl.h +++ b/include/sys/abd_impl.h @@ -95,6 +95,9 @@ void abd_iter_unmap(struct abd_iter *); #if defined(__FreeBSD__) #define abd_enter_critical(flags) critical_enter() #define abd_exit_critical(flags) critical_exit() +#elif defined(_WIN32) +#define abd_enter_critical(flags) +#define abd_exit_critical(flags) #else #define abd_enter_critical(flags) local_irq_save(flags) #define abd_exit_critical(flags) local_irq_restore(flags) diff --git a/include/sys/fm/fs/zfs.h b/include/sys/fm/fs/zfs.h index b9bac7e252e5..ada2dce9e4f1 100644 --- a/include/sys/fm/fs/zfs.h +++ b/include/sys/fm/fs/zfs.h @@ -55,6 +55,8 @@ extern "C" { #define FM_EREPORT_ZFS_PROBE_FAILURE "probe_failure" #define FM_EREPORT_ZFS_LOG_REPLAY "log_replay" #define FM_EREPORT_ZFS_CONFIG_CACHE_WRITE "config_cache_write" +#define FM_EREPORT_ZFS_SNAPSHOT_MOUNT "snapshot_mount" +#define FM_EREPORT_ZFS_SNAPSHOT_UNMOUNT "snapshot_unmount" #define FM_EREPORT_PAYLOAD_ZFS_POOL "pool" #define FM_EREPORT_PAYLOAD_ZFS_POOL_FAILMODE "pool_failmode" diff --git a/include/sys/fs/zfs.h b/include/sys/fs/zfs.h index 93193fa142da..08664dae47be 100644 --- a/include/sys/fs/zfs.h +++ b/include/sys/fs/zfs.h @@ -191,6 +191,8 @@ typedef enum { ZFS_PROP_REDACTED, ZFS_PROP_REDACT_SNAPS, ZFS_PROP_SNAPSHOTS_CHANGED, + ZFS_PROP_MIMIC, /* Windows: mimic=ntfs */ + ZFS_PROP_DRIVELETTER, ZFS_NUM_PROPS } zfs_prop_t; @@ -543,6 +545,13 @@ typedef enum zfs_key_location { ZFS_KEYLOCATION_LOCATIONS } zfs_keylocation_t; +typedef enum zfs_mimic { + ZFS_MIMIC_OFF = 0, + ZFS_MIMIC_HFS, + ZFS_MIMIC_APFS, + ZFS_MIMIC_NTFS +} zfs_mimic_t; + #define DEFAULT_PBKDF2_ITERATIONS 350000 #define MIN_PBKDF2_ITERATIONS 100000 @@ -943,7 +952,11 @@ typedef struct zpool_load_policy { * userland. */ #define ZPOOL_CACHE_BOOT "/boot/zfs/zpool.cache" +#ifdef _WIN32 +#define ZPOOL_CACHE "\\SystemRoot\\System32\\drivers\\zpool.cache" +#else #define ZPOOL_CACHE "/etc/zfs/zpool.cache" +#endif /* * Settings for zpool compatibility features files */ @@ -1322,7 +1335,15 @@ typedef struct ddt_histogram { #define ZVOL_DRIVER "zvol" #define ZFS_DRIVER "zfs" + +#if defined(_WIN32) +#define ZFS_DEV "\\\\.\\ZFS" +#define ZFS_DEV_DOS L"\\DosDevices\\Global\\ZFS" +#define ZFS_DEV_KERNEL L"\\Device\\ZFSCTL" +#define ZFS_GLOBAL_FS_DISK_DEVICE_NAME L"\\ZFS" +#else #define ZFS_DEV "/dev/zfs" +#endif #define ZFS_DEVDIR "/dev" #define ZFS_SUPER_MAGIC 0x2fc12fc1 @@ -1378,7 +1399,7 @@ typedef enum zfs_ioc { /* * Core features - 88/128 numbers reserved. */ -#ifdef __FreeBSD__ +#if defined(__FreeBSD__) || defined(_WIN32) ZFS_IOC_FIRST = 0, #else ZFS_IOC_FIRST = ('Z' << 8), @@ -1487,6 +1508,9 @@ typedef enum zfs_ioc { ZFS_IOC_USERNS_DETACH = ZFS_IOC_UNJAIL, /* 0x86 (Linux) */ ZFS_IOC_SET_BOOTENV, /* 0x87 */ ZFS_IOC_GET_BOOTENV, /* 0x88 */ + ZFS_IOC_UNREGISTER_FS, /* 0x89 (Windows) */ + ZFS_IOC_MOUNT, /* 0x8a (Windows) */ + ZFS_IOC_UNMOUNT, /* 0x8b (Windows) */ ZFS_IOC_LAST } zfs_ioc_t; diff --git a/include/sys/mntent.h b/include/sys/mntent.h index 5bb7e080cda8..6685e6ccfae6 100644 --- a/include/sys/mntent.h +++ b/include/sys/mntent.h @@ -79,6 +79,9 @@ #elif defined(__FreeBSD__) #define MNTOPT_SETUID "setuid" /* Set uid allowed */ #define MNTOPT_NOSETUID "nosetuid" /* Set uid not allowed */ +#elif defined(_WIN32) +#define MNTOPT_SETUID "setuid" /* Set uid allowed */ +#define MNTOPT_NOSETUID "nosetuid" /* Set uid not allowed */ #else #error "unknown OS" #endif diff --git a/include/sys/vdev_file.h b/include/sys/vdev_file.h index fddecbfe1ab5..dfd3095ae941 100644 --- a/include/sys/vdev_file.h +++ b/include/sys/vdev_file.h @@ -35,6 +35,10 @@ extern "C" { typedef struct vdev_file { zfs_file_t *vf_file; +#ifdef _WIN32 + uint64_t vdev_win_offset; /* soft partition start */ + uint64_t vdev_win_length; /* soft partition length */ +#endif } vdev_file_t; extern void vdev_file_init(void); diff --git a/include/sys/zfs_bootenv.h b/include/sys/zfs_bootenv.h index 7af0a57dd008..7dbfe8f67e14 100644 --- a/include/sys/zfs_bootenv.h +++ b/include/sys/zfs_bootenv.h @@ -28,8 +28,10 @@ extern "C" { #define BE_ILLUMOS_VENDOR "illumos" #define BE_FREEBSD_VENDOR "freebsd" +#define BE_WINDOWS_VENDOR "windows" #define BE_GRUB_VENDOR "grub" #define BE_LINUX_VENDOR "linux" +#define BE_WINDOWS_VENDOR "windows" #include diff --git a/include/sys/zfs_debug.h b/include/sys/zfs_debug.h index a1dfef1d89ff..88061fa1af8b 100644 --- a/include/sys/zfs_debug.h +++ b/include/sys/zfs_debug.h @@ -84,6 +84,7 @@ extern void __dprintf(boolean_t dprint, const char *file, const char *func, __dprintf(B_FALSE, __FILE__, __func__, __LINE__, __VA_ARGS__) #ifdef ZFS_DEBUG +#undef dprintf /* * To enable this: * diff --git a/include/sys/zfs_file.h b/include/sys/zfs_file.h index e944165adc40..6927476ddb8b 100644 --- a/include/sys/zfs_file.h +++ b/include/sys/zfs_file.h @@ -31,6 +31,8 @@ typedef struct zfs_file { } zfs_file_t; #elif defined(__linux__) || defined(__FreeBSD__) typedef struct file zfs_file_t; +#elif defined(_WIN32) +typedef struct spl_fileproc zfs_file_t; #else #error "unknown OS" #endif diff --git a/include/sys/zfs_ioctl_impl.h b/include/sys/zfs_ioctl_impl.h index cb852c5577fd..810bfb94045a 100644 --- a/include/sys/zfs_ioctl_impl.h +++ b/include/sys/zfs_ioctl_impl.h @@ -80,6 +80,10 @@ void zfs_ioctl_register(const char *, zfs_ioc_t, zfs_ioc_func_t *, zfs_secpolicy_func_t *, zfs_ioc_namecheck_t, zfs_ioc_poolcheck_t, boolean_t, boolean_t, const zfs_ioc_key_t *, size_t); +void zfs_ioctl_register_legacy(zfs_ioc_t ioc, zfs_ioc_legacy_func_t *func, + zfs_secpolicy_func_t *secpolicy, zfs_ioc_namecheck_t namecheck, + boolean_t log_history, zfs_ioc_poolcheck_t pool_check); + uint64_t zfs_max_nvlist_src_size_os(void); void zfs_ioctl_update_mount_cache(const char *dsname); void zfs_ioctl_init_os(void); diff --git a/include/sys/zio.h b/include/sys/zio.h index 85217b873dc8..b6137d876c9a 100644 --- a/include/sys/zio.h +++ b/include/sys/zio.h @@ -521,6 +521,10 @@ struct zio { kcondvar_t io_cv; int io_allocator; +#ifdef ZIO_OS_FIELDS + ZIO_OS_FIELDS +#endif + /* FMA state */ zio_cksum_report_t *io_cksum_report; uint64_t io_ena; diff --git a/include/sys/zio_checksum.h b/include/sys/zio_checksum.h index 9fb79ab4a54b..b00deb290e46 100644 --- a/include/sys/zio_checksum.h +++ b/include/sys/zio_checksum.h @@ -69,6 +69,10 @@ typedef struct zio_abd_checksum_data { fletcher_4_ctx_t *acd_ctx; zio_cksum_t *acd_zcp; void *acd_private; +#if defined(_WIN32) && defined(_KERNEL) + NTSTATUS saveStatus; + XSTATE_SAVE SaveState; +#endif } zio_abd_checksum_data_t; typedef void zio_abd_checksum_init_t(zio_abd_checksum_data_t *); diff --git a/include/sys/zvol_impl.h b/include/sys/zvol_impl.h index 3243917bcd3f..6a7ed805d37b 100644 --- a/include/sys/zvol_impl.h +++ b/include/sys/zvol_impl.h @@ -17,6 +17,9 @@ * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END + * + * Portions Copyright 2022 Andrew Innes + * */ #ifndef _SYS_ZVOL_IMPL_H @@ -75,6 +78,7 @@ extern unsigned int zvol_inhibit_dev; */ zvol_state_t *zvol_find_by_name_hash(const char *name, uint64_t hash, int mode); +zvol_state_t *zvol_find_by_name(const char *name, int mode); int zvol_first_open(zvol_state_t *zv, boolean_t readonly); uint64_t zvol_name_hash(const char *name); void zvol_remove_minors_impl(const char *name); diff --git a/include/zfs_gitrev.h.win b/include/zfs_gitrev.h.win new file mode 100644 index 000000000000..a754208e0b71 --- /dev/null +++ b/include/zfs_gitrev.h.win @@ -0,0 +1,3 @@ +// This file is only used on Windows and CMake, other platforms +// see scripts/make-gitrev.sh +#define ZFS_META_GITREV "@GIT_GITREV@" diff --git a/lib/CMakeLists.txt b/lib/CMakeLists.txt new file mode 100644 index 000000000000..9d89f4ff083b --- /dev/null +++ b/lib/CMakeLists.txt @@ -0,0 +1,23 @@ + +include_directories(PRIVATE "${CMAKE_SOURCE_DIR}/lib/libspl/include/os/windows" "${CMAKE_SOURCE_DIR}/lib/libspl/include" "${CMAKE_SOURCE_DIR}/include" + "${CMAKE_SOURCE_DIR}/lib/os/windows/libpthread" "${OPENSSL_INCLUDE_DIR}") + +add_definitions(-D_CRT_SECURE_NO_WARNINGS) +add_definitions(-D_CRT_LOADCFG_DISABLE_CET) + +add_subdirectory(libavl) +add_subdirectory(libefi) +add_subdirectory(libicp) +add_subdirectory(libnvpair) +add_subdirectory(libshare) +add_subdirectory(libspl) +add_subdirectory(libtpool) +add_subdirectory(libunicode) +add_subdirectory(libuutil) +add_subdirectory(libzfs) +add_subdirectory(libzfs_core) +add_subdirectory(libzfsbootenv) +add_subdirectory(libzpool) +add_subdirectory(libzstd) +add_subdirectory(libzutil) +add_subdirectory(os/windows) diff --git a/lib/libavl/CMakeLists.txt b/lib/libavl/CMakeLists.txt new file mode 100644 index 000000000000..726560c555ad --- /dev/null +++ b/lib/libavl/CMakeLists.txt @@ -0,0 +1,8 @@ + +use_clang() + +set(AVL_MODULE_DIR "../../module/avl") +add_library(libavl + "${AVL_MODULE_DIR}/avl.c" +) +target_link_libraries(libavl PRIVATE libspl) diff --git a/lib/libefi/CMakeLists.txt b/lib/libefi/CMakeLists.txt new file mode 100644 index 000000000000..80c0645552d2 --- /dev/null +++ b/lib/libefi/CMakeLists.txt @@ -0,0 +1,7 @@ + +use_clang() + +add_library(libefi + rdwr_efi_windows.c +) +target_link_libraries(libefi PRIVATE libuuid zlib) diff --git a/lib/libefi/rdwr_efi_windows.c b/lib/libefi/rdwr_efi_windows.c new file mode 100644 index 000000000000..e5359744e3e5 --- /dev/null +++ b/lib/libefi/rdwr_efi_windows.c @@ -0,0 +1,1511 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2002, 2010, Oracle and/or its affiliates. All rights reserved. + */ + +/* Copyright(c) 2015 Jorgen Lundman */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#if defined(__linux__) +#include +#endif + +static struct uuid_to_ptag { + struct uuid uuid; +} conversion_array[] = { + { EFI_UNUSED }, + { EFI_BOOT }, + { EFI_ROOT }, + { EFI_SWAP }, + { EFI_USR }, + { EFI_BACKUP }, + { EFI_UNUSED }, /* STAND is never used */ + { EFI_VAR }, + { EFI_HOME }, + { EFI_ALTSCTR }, + { EFI_UNUSED }, /* CACHE (cachefs) is never used */ + { EFI_RESERVED }, + { EFI_SYSTEM }, + { EFI_LEGACY_MBR }, + { EFI_SYMC_PUB }, + { EFI_SYMC_CDS }, + { EFI_MSFT_RESV }, + { EFI_DELL_BASIC }, + { EFI_DELL_RAID }, + { EFI_DELL_SWAP }, + { EFI_DELL_LVM }, + { EFI_DELL_RESV }, + { EFI_AAPL_HFS }, + { EFI_AAPL_UFS }, + { EFI_FREEBSD_ZFS }, + { EFI_MIDNIGHTBSD_ZFS }, + { EFI_RHT_DATA } +}; + +#ifdef DEBUG +int efi_debug = 1; +#else +int efi_debug = 0; +#endif + +static int efi_read(int, struct dk_gpt *); + +/* + * Return a 32-bit CRC of the contents of the buffer. Pre-and-post + * one's conditioning will be handled by crc32() internally. + */ +static uint32_t +efi_crc32(const unsigned char *buf, unsigned int size) +{ + uint32_t crc = crc32(0, Z_NULL, 0); + + crc = crc32(crc, buf, size); + + return (crc); +} + +static int +read_disk_info(int fd, diskaddr_t *capacity, uint_t *lbsize) +{ + DISK_GEOMETRY_EX geometry_ex; + DWORD len; + + LARGE_INTEGER large; + if (DeviceIoControl(ITOH(fd), IOCTL_DISK_GET_DRIVE_GEOMETRY_EX, NULL, 0, + &geometry_ex, sizeof (geometry_ex), &len, NULL)) { + + *lbsize = (uint_t)geometry_ex.Geometry.BytesPerSector; + *capacity = (diskaddr_t)geometry_ex.DiskSize.QuadPart; + *capacity /= *lbsize; // Capacity is in # blocks + return (0); + } + + return (0); +} + +static int +efi_get_info(int fd, struct dk_cinfo *dki_info) +{ + int rval = 0; +#if defined(__linux__) + char *path; + char *dev_path; + + memset(dki_info, 0, sizeof (*dki_info)); + + path = calloc(PATH_MAX, 1); + if (path == NULL) + goto error; + + /* + * The simplest way to get the partition number under linux is + * to parse it out of the /dev/ block device name. + * The kernel creates this using the partition number when it + * populates /dev/ so it may be trusted. The tricky bit here is + * that the naming convention is based on the block device type. + * So we need to take this in to account when parsing out the + * partition information. Another issue is that the libefi API + * API only provides the open fd and not the file path. To handle + * this realpath(3) is used to resolve the block device name from + * /proc/self/fd/. Aside from the partition number we collect + * some additional device info. + */ + (void) sprintf(path, "/proc/self/fd/%d", fd); + dev_path = realpath(path, NULL); + free(path); + + if (dev_path == NULL) + goto error; + + if ((strncmp(dev_path, "/dev/sd", 7) == 0)) { + strcpy(dki_info->dki_cname, "sd"); + dki_info->dki_ctype = DKC_SCSI_CCS; + rval = sscanf(dev_path, "/dev/%[a-zA-Z]%hu", + dki_info->dki_dname, + &dki_info->dki_partition); + } else if ((strncmp(dev_path, "/dev/hd", 7) == 0)) { + strcpy(dki_info->dki_cname, "hd"); + dki_info->dki_ctype = DKC_DIRECT; + rval = sscanf(dev_path, "/dev/%[a-zA-Z]%hu", + dki_info->dki_dname, + &dki_info->dki_partition); + } else if ((strncmp(dev_path, "/dev/md", 7) == 0)) { + strcpy(dki_info->dki_cname, "pseudo"); + dki_info->dki_ctype = DKC_MD; + strcpy(dki_info->dki_dname, "md"); + rval = sscanf(dev_path, "/dev/md%[0-9]p%hu", + dki_info->dki_dname + 2, + &dki_info->dki_partition); + } else if ((strncmp(dev_path, "/dev/vd", 7) == 0)) { + strcpy(dki_info->dki_cname, "vd"); + dki_info->dki_ctype = DKC_MD; + rval = sscanf(dev_path, "/dev/%[a-zA-Z]%hu", + dki_info->dki_dname, + &dki_info->dki_partition); + } else if ((strncmp(dev_path, "/dev/xvd", 8) == 0)) { + strcpy(dki_info->dki_cname, "xvd"); + dki_info->dki_ctype = DKC_MD; + rval = sscanf(dev_path, "/dev/%[a-zA-Z]%hu", + dki_info->dki_dname, + &dki_info->dki_partition); + } else if ((strncmp(dev_path, "/dev/zd", 7) == 0)) { + strcpy(dki_info->dki_cname, "zd"); + dki_info->dki_ctype = DKC_MD; + rval = sscanf(dev_path, "/dev/%[a-zA-Z]%hu", + dki_info->dki_dname, + &dki_info->dki_partition); + } else if ((strncmp(dev_path, "/dev/dm-", 8) == 0)) { + strcpy(dki_info->dki_cname, "pseudo"); + dki_info->dki_ctype = DKC_VBD; + strcpy(dki_info->dki_dname, "dm-"); + rval = sscanf(dev_path, "/dev/dm-%[0-9]p%hu", + dki_info->dki_dname + 3, + &dki_info->dki_partition); + } else if ((strncmp(dev_path, "/dev/ram", 8) == 0)) { + strcpy(dki_info->dki_cname, "pseudo"); + dki_info->dki_ctype = DKC_PCMCIA_MEM; + strcpy(dki_info->dki_dname, "ram"); + rval = sscanf(dev_path, "/dev/ram%[0-9]p%hu", + dki_info->dki_dname + 3, + &dki_info->dki_partition); + } else if ((strncmp(dev_path, "/dev/loop", 9) == 0)) { + strcpy(dki_info->dki_cname, "pseudo"); + dki_info->dki_ctype = DKC_VBD; + strcpy(dki_info->dki_dname, "loop"); + rval = sscanf(dev_path, "/dev/loop%[0-9]p%hu", + dki_info->dki_dname + 4, + &dki_info->dki_partition); + } else { + strcpy(dki_info->dki_dname, "unknown"); + strcpy(dki_info->dki_cname, "unknown"); + dki_info->dki_ctype = DKC_UNKNOWN; + } + + switch (rval) { + case 0: + errno = EINVAL; + goto error; + case 1: + dki_info->dki_partition = 0; + } + + free(dev_path); +#elif __APPLE__ + // DKIOCISVIRTUAL 32bit + // DKIOCISSOLIDSTATE 32bit + char pathbuf[PATH_MAX]; + ushort_t poi; + if (fcntl(fd, F_GETPATH, pathbuf) >= 0) { + if ((strncmp(pathbuf, "/dev/disk", 9) == 0)) { + strcpy(dki_info->dki_cname, "disk"); + dki_info->dki_ctype = DKC_DIRECT; + rval = sscanf(pathbuf, "/dev/disk%hus%hu", + &poi, + &dki_info->dki_partition); + + switch (rval) { + case 0: + errno = EINVAL; + goto error; + case 1: + dki_info->dki_partition = 0; + } + strlcpy(dki_info->dki_dname, + &pathbuf[5], + sizeof (dki_info->dki_dname)); + } + + /* + * rdisk in OSX do not have partitions, also it will fail. Use + * disk instead. + */ + if ((strncmp(pathbuf, "/dev/rdisk", 10) == 0)) { + strcpy(dki_info->dki_cname, "disk"); + dki_info->dki_ctype = DKC_DIRECT; + dki_info->dki_partition = 0; + + rval = sscanf(pathbuf, "/dev/r%[a-zA-Z0-9]", + dki_info->dki_dname); + } + + if (efi_debug) + (void) fprintf(stderr, + "rval %d, name '%s' and part %d\n", + rval, + dki_info->dki_dname, + dki_info->dki_partition); + } + + if (osx_device_isvirtual(dki_info->dki_dname)) { + dki_info->dki_ctype = DKC_VBD; + if (efi_debug) + (void) fprintf(stderr, + "'%s' is virtual\n", + pathbuf); + } else { + if (efi_debug) + (void) fprintf(stderr, + "'%s' is not virtual\n", + pathbuf); + } +#elif _WIN32 + + // Ask it for partitions + PARTITION_INFORMATION partInfo; + DWORD retcount = 0; + int err; + err = DeviceIoControl(ITOH(fd), + IOCTL_DISK_GET_PARTITION_INFO, + (LPVOID)NULL, + (DWORD)0, + (LPVOID)&partInfo, + sizeof (partInfo), + &retcount, + (LPOVERLAPPED)NULL); + if (err) { + dki_info->dki_partition = 0; + dki_info->dki_ctype = DKC_DIRECT; + strlcpy(dki_info->dki_dname, + "getnamehere", + sizeof (dki_info->dki_dname)); + } else { + err = GetLastError(); + } + +#endif + return (0); +error: + if (efi_debug) + (void) fprintf(stderr, "DKIOCINFO errno 0x%x\n", errno); + + switch (errno) { + case EIO: + return (VT_EIO); + case EINVAL: + return (VT_EINVAL); + default: + return (VT_ERROR); + } +} + +/* + * the number of blocks the EFI label takes up (round up to nearest + * block) + */ +#define NBLOCKS(p, l) (1 + ((((p) * (int)sizeof (efi_gpe_t)) + \ + ((l) - 1)) / (l))) +/* number of partitions -- limited by what we can malloc */ +#define MAX_PARTS ((4294967295UL - sizeof (struct dk_gpt)) / \ + sizeof (struct dk_part)) + +int +efi_alloc_and_init(int fd, uint32_t nparts, struct dk_gpt **vtoc) +{ + diskaddr_t capacity = 0; + uint_t lbsize = 0; + uint_t nblocks; + size_t length; + struct dk_gpt *vptr; + struct uuid uuid; + struct dk_cinfo dki_info; + + if (read_disk_info(fd, &capacity, &lbsize) != 0) + return (-1); + + if (efi_get_info(fd, &dki_info) != 0) + return (-1); + + if (dki_info.dki_partition != 0) + return (-1); + + if ((dki_info.dki_ctype == DKC_PCMCIA_MEM) || + (dki_info.dki_ctype == DKC_VBD) || + (dki_info.dki_ctype == DKC_UNKNOWN)) + return (-1); + + if (lbsize == 0) + return (-1); + + nblocks = NBLOCKS(nparts, lbsize); + if ((nblocks * lbsize) < EFI_MIN_ARRAY_SIZE + lbsize) { + /* 16K plus one block for the GPT */ + nblocks = EFI_MIN_ARRAY_SIZE / lbsize + 1; + } + + if (nparts > MAX_PARTS) { + if (efi_debug) { + (void) fprintf(stderr, + "the maximum number of partitions supported is %llu\n", + MAX_PARTS); + } + return (-1); + } + + length = sizeof (struct dk_gpt) + + sizeof (struct dk_part) * (nparts - 1); + + if ((*vtoc = calloc(length, 1)) == NULL) + return (-1); + + vptr = *vtoc; + + vptr->efi_version = EFI_VERSION_CURRENT; + vptr->efi_lbasize = lbsize; + vptr->efi_nparts = nparts; + /* + * add one block here for the PMBR; on disks with a 512 byte + * block size and 128 or fewer partitions, efi_first_u_lba + * should work out to "34" + */ + vptr->efi_first_u_lba = nblocks + 1; + vptr->efi_last_lba = capacity - 1; + vptr->efi_altern_lba = capacity -1; + vptr->efi_last_u_lba = vptr->efi_last_lba - nblocks; + + (void) uuid_generate((uchar_t *)&uuid); + UUID_LE_CONVERT(vptr->efi_disk_uguid, uuid); + return (0); +} + +/* + * Read EFI - return partition number upon success. + */ +int +efi_alloc_and_read(int fd, struct dk_gpt **vtoc) +{ + int rval; + uint32_t nparts; + int length; + + /* figure out the number of entries that would fit into 16K */ + nparts = EFI_MIN_ARRAY_SIZE / sizeof (efi_gpe_t); + length = (int) sizeof (struct dk_gpt) + + (int) sizeof (struct dk_part) * (nparts - 1); + if ((*vtoc = calloc(length, 1)) == NULL) + return (VT_ERROR); + + (*vtoc)->efi_nparts = nparts; + rval = efi_read(fd, *vtoc); + + if ((rval == VT_EINVAL) && (*vtoc)->efi_nparts > nparts) { + void *tmp; + length = (int) sizeof (struct dk_gpt) + + (int) sizeof (struct dk_part) * + ((*vtoc)->efi_nparts - 1); + nparts = (*vtoc)->efi_nparts; + if ((tmp = realloc(*vtoc, length)) == NULL) { + free (*vtoc); + *vtoc = NULL; + return (VT_ERROR); + } else { + *vtoc = tmp; + rval = efi_read(fd, *vtoc); + } + } + + if (rval < 0) { + if (efi_debug) { + (void) fprintf(stderr, + "read of EFI table failed, rval=%d\n", rval); + } + free (*vtoc); + *vtoc = NULL; + } + + return (rval); +} + +static int +efi_ioctl(int fd, int cmd, dk_efi_t *dk_ioc) +{ + void *data = dk_ioc->dki_data; + int error; +#if defined(__linux__) || defined(__APPLE__) || defined(_WIN32) + diskaddr_t capacity; + uint_t lbsize; + + /* + * When the IO is not being performed in kernel as an ioctl we need + * to know the sector size so we can seek to the proper byte offset. + */ + if (read_disk_info(fd, &capacity, &lbsize) == -1) { + if (efi_debug) + (void) fprintf(stderr, "unable to read disk info: %d", + errno); + + errno = EIO; + return (-1); + } + + switch (cmd) { + case DKIOCGETEFI: + if (lbsize == 0) { + if (efi_debug) + (void) fprintf(stderr, "DKIOCGETEFI assuming " + "LBA %d bytes\n", DEV_BSIZE); + + lbsize = DEV_BSIZE; + } + + error = lseek(fd, dk_ioc->dki_lba * lbsize, SEEK_SET); + if (error == -1) { + if (efi_debug) + (void) fprintf(stderr, "DKIOCGETEFI lseek " + "error: %d\n", errno); + return (error); + } + + error = read(fd, data, dk_ioc->dki_length); + if (error == -1) { + if (efi_debug) + (void) fprintf(stderr, "DKIOCGETEFI read " + "error: %d\n", errno); + return (error); + } + + if (error != dk_ioc->dki_length) { + if (efi_debug) + (void) fprintf(stderr, "DKIOCGETEFI short " + "read of %d bytes\n", error); + errno = EIO; + return (-1); + } + error = 0; + break; + + case DKIOCSETEFI: + if (lbsize == 0) { + if (efi_debug) + (void) fprintf(stderr, "DKIOCSETEFI unknown " + "LBA size\n"); + errno = EIO; + return (-1); + } + + error = lseek(fd, dk_ioc->dki_lba * lbsize, SEEK_SET); + if (error == -1) { + if (efi_debug) + (void) fprintf(stderr, "DKIOCSETEFI lseek " + "error: %d\n", errno); + return (error); + } + + error = write(fd, data, dk_ioc->dki_length); + if (error == -1) { + if (efi_debug) + (void) fprintf(stderr, "DKIOCSETEFI write " + "error: %d\n", errno); + return (error); + } + + if (error != dk_ioc->dki_length) { + if (efi_debug) + (void) fprintf(stderr, "DKIOCSETEFI short " + "write of %d bytes\n", error); + errno = EIO; + return (-1); + } + + /* Sync the new EFI table to disk */ + error = fsync(fd); + + if (error == -1) + return (error); + + /* Ensure any local disk cache is also flushed */ +#if defined(__linux__) + if (ioctl(fd, BLKFLSBUF, 0) == -1) + return (error); +#endif + error = 0; + break; + + default: + if (efi_debug) + (void) fprintf(stderr, "unsupported ioctl()\n"); + + errno = EIO; + return (-1); + } +#else + dk_ioc->dki_data_64 = (uint64_t)(uintptr_t)data; + error = ioctl(fd, cmd, (void *)dk_ioc); + dk_ioc->dki_data = data; +#endif + return (error); +} + +int +efi_rescan(int fd) +{ +#if defined(__linux__) + int retry = 10; + int error; + + /* Notify the kernel a devices partition table has been updated */ + while ((error = ioctl(fd, BLKRRPART)) != 0) { + if ((--retry == 0) || (errno != EBUSY)) { + (void) fprintf(stderr, "the kernel failed to rescan " + "the partition table: %d\n", errno); + return (-1); + } + usleep(50000); + } +#endif + + return (0); +} + +static int +check_label(int fd, dk_efi_t *dk_ioc) +{ + efi_gpt_t *efi; + uint_t crc; + + if (efi_ioctl(fd, DKIOCGETEFI, dk_ioc) == -1) { + switch (errno) { + case EIO: + return (VT_EIO); + default: + return (VT_ERROR); + } + } + efi = dk_ioc->dki_data; + if (efi->efi_gpt_Signature != LE_64(EFI_SIGNATURE)) { + if (efi_debug) + (void) fprintf(stderr, + "Bad EFI signature: 0x%llx != 0x%llx\n", + (long long)efi->efi_gpt_Signature, + (long long)LE_64(EFI_SIGNATURE)); + return (VT_EINVAL); + } + + /* + * check CRC of the header; the size of the header should + * never be larger than one block + */ + crc = efi->efi_gpt_HeaderCRC32; + efi->efi_gpt_HeaderCRC32 = 0; + len_t headerSize = (len_t)LE_32(efi->efi_gpt_HeaderSize); + + if (headerSize < EFI_MIN_LABEL_SIZE || headerSize > EFI_LABEL_SIZE) { + if (efi_debug) + (void) fprintf(stderr, + "Invalid EFI HeaderSize %llu. Assuming %d.\n", + headerSize, EFI_MIN_LABEL_SIZE); + } + + if ((headerSize > dk_ioc->dki_length) || + crc != LE_32(efi_crc32((unsigned char *)efi, headerSize))) { + if (efi_debug) + (void) fprintf(stderr, + "Bad EFI CRC: 0x%x != 0x%x\n", + crc, LE_32(efi_crc32((unsigned char *)efi, + headerSize))); + return (VT_EINVAL); + } + + return (0); +} + +static int +efi_read(int fd, struct dk_gpt *vtoc) +{ + int i, j; + int label_len; + int rval = 0; + int md_flag = 0; + int vdc_flag = 0; + diskaddr_t capacity = 0; + uint_t lbsize = 0; + struct dk_minfo disk_info; + dk_efi_t dk_ioc; + efi_gpt_t *efi; + efi_gpe_t *efi_parts; + struct dk_cinfo dki_info; + uint32_t user_length; + boolean_t legacy_label = B_FALSE; + + /* + * get the partition number for this file descriptor. + */ + if ((rval = efi_get_info(fd, &dki_info)) != 0) + return (rval); + + if ((strncmp(dki_info.dki_cname, "pseudo", 7) == 0) && + (strncmp(dki_info.dki_dname, "md", 3) == 0)) { + md_flag++; + } else if ((strncmp(dki_info.dki_cname, "vdc", 4) == 0) && + (strncmp(dki_info.dki_dname, "vdc", 4) == 0)) { + /* + * The controller and drive name "vdc" (virtual disk client) + * indicates a LDoms virtual disk. + */ + vdc_flag++; + } + + /* get the LBA size */ + if (read_disk_info(fd, &capacity, &lbsize) == -1) { + if (efi_debug) { + (void) fprintf(stderr, + "unable to read disk info: %d", + errno); + } + return (VT_EINVAL); + } + + disk_info.dki_lbsize = lbsize; + disk_info.dki_capacity = capacity; + + if (disk_info.dki_lbsize == 0) { + if (efi_debug) { + (void) fprintf(stderr, + "efi_read: assuming LBA 512 bytes\n"); + } + disk_info.dki_lbsize = DEV_BSIZE; + } + /* + * Read the EFI GPT to figure out how many partitions we need + * to deal with. + */ + dk_ioc.dki_lba = 1; + if (NBLOCKS(vtoc->efi_nparts, disk_info.dki_lbsize) < 34) { + label_len = EFI_MIN_ARRAY_SIZE + disk_info.dki_lbsize; + } else { + label_len = vtoc->efi_nparts * (int) sizeof (efi_gpe_t) + + disk_info.dki_lbsize; + if (label_len % disk_info.dki_lbsize) { + /* pad to physical sector size */ + label_len += disk_info.dki_lbsize; + label_len &= ~(disk_info.dki_lbsize - 1); + } + } + + dk_ioc.dki_data = (void *)umem_alloc_aligned(label_len, + disk_info.dki_lbsize, UMEM_DEFAULT); + if (dk_ioc.dki_data == NULL) + return (VT_ERROR); + + memset(dk_ioc.dki_data, 0, label_len); + dk_ioc.dki_length = disk_info.dki_lbsize; + user_length = vtoc->efi_nparts; + efi = dk_ioc.dki_data; + if (md_flag) { + dk_ioc.dki_length = label_len; + if (efi_ioctl(fd, DKIOCGETEFI, &dk_ioc) == -1) { + switch (errno) { + case EIO: + return (VT_EIO); + default: + return (VT_ERROR); + } + } + } else if ((rval = check_label(fd, &dk_ioc)) == VT_EINVAL) { + /* + * No valid label here; try the alternate. Note that here + * we just read GPT header and save it into dk_ioc.data, + * Later, we will read GUID partition entry array if we + * can get valid GPT header. + */ + + /* + * This is a workaround for legacy systems. In the past, the + * last sector of SCSI disk was invisible on x86 platform. At + * that time, backup label was saved on the next to the last + * sector. It is possible for users to move a disk from previous + * solaris system to present system. Here, we attempt to search + * legacy backup EFI label first. + */ + dk_ioc.dki_lba = disk_info.dki_capacity - 2; + dk_ioc.dki_length = disk_info.dki_lbsize; + rval = check_label(fd, &dk_ioc); + if (rval == VT_EINVAL) { + /* + * we didn't find legacy backup EFI label, try to + * search backup EFI label in the last block. + */ + dk_ioc.dki_lba = disk_info.dki_capacity - 1; + dk_ioc.dki_length = disk_info.dki_lbsize; + rval = check_label(fd, &dk_ioc); + if (rval == 0) { + legacy_label = B_TRUE; + if (efi_debug) + (void) fprintf(stderr, + "efi_read: primary label corrupt; " + "using EFI backup label located on" + " the last block\n"); + } + } else { + if ((efi_debug) && (rval == 0)) + (void) fprintf(stderr, "efi_read: primary label" + " corrupt; using legacy EFI backup label " + " located on the next to last block\n"); + } + + if (rval == 0) { + dk_ioc.dki_lba = LE_64(efi->efi_gpt_PartitionEntryLBA); + vtoc->efi_flags |= EFI_GPT_PRIMARY_CORRUPT; + vtoc->efi_nparts = + LE_32(efi->efi_gpt_NumberOfPartitionEntries); + /* + * Partition tables are between backup GPT header + * table and ParitionEntryLBA (the starting LBA of + * the GUID partition entries array). Now that we + * already got valid GPT header and saved it in + * dk_ioc.dki_data, we try to get GUID partition + * entry array here. + */ + /* LINTED */ + dk_ioc.dki_data = (efi_gpt_t *)((char *)dk_ioc.dki_data + + disk_info.dki_lbsize); + if (legacy_label) + dk_ioc.dki_length = disk_info.dki_capacity - 1 - + dk_ioc.dki_lba; + else + dk_ioc.dki_length = disk_info.dki_capacity - 2 - + dk_ioc.dki_lba; + dk_ioc.dki_length *= disk_info.dki_lbsize; + if (dk_ioc.dki_length > + ((len_t)label_len - sizeof (*dk_ioc.dki_data))) { + rval = VT_EINVAL; + } else { + /* + * read GUID partition entry array + */ + rval = efi_ioctl(fd, DKIOCGETEFI, &dk_ioc); + } + } + + } else if (rval == 0) { + + dk_ioc.dki_lba = LE_64(efi->efi_gpt_PartitionEntryLBA); + /* LINTED */ + dk_ioc.dki_data = (efi_gpt_t *)((char *)dk_ioc.dki_data + + disk_info.dki_lbsize); + dk_ioc.dki_length = label_len - disk_info.dki_lbsize; + rval = efi_ioctl(fd, DKIOCGETEFI, &dk_ioc); + + } else if (vdc_flag && rval == VT_ERROR && errno == EINVAL) { + /* + * When the device is a LDoms virtual disk, the DKIOCGETEFI + * ioctl can fail with EINVAL if the virtual disk backend + * is a ZFS volume serviced by a domain running an old version + * of Solaris. This is because the DKIOCGETEFI ioctl was + * initially incorrectly implemented for a ZFS volume and it + * expected the GPT and GPE to be retrieved with a single ioctl. + * So we try to read the GPT and the GPE using that old style + * ioctl. + */ + dk_ioc.dki_lba = 1; + dk_ioc.dki_length = label_len; + rval = check_label(fd, &dk_ioc); + } + + if (rval < 0) { + umem_free_aligned(efi, label_len); + return (rval); + } + + /* LINTED -- always longlong aligned */ + efi_parts = (efi_gpe_t *)(((char *)efi) + disk_info.dki_lbsize); + + /* + * Assemble this into a "dk_gpt" struct for easier + * digestibility by applications. + */ + vtoc->efi_version = LE_32(efi->efi_gpt_Revision); + vtoc->efi_nparts = LE_32(efi->efi_gpt_NumberOfPartitionEntries); + vtoc->efi_part_size = LE_32(efi->efi_gpt_SizeOfPartitionEntry); + vtoc->efi_lbasize = disk_info.dki_lbsize; + vtoc->efi_last_lba = disk_info.dki_capacity - 1; + vtoc->efi_first_u_lba = LE_64(efi->efi_gpt_FirstUsableLBA); + vtoc->efi_last_u_lba = LE_64(efi->efi_gpt_LastUsableLBA); + vtoc->efi_altern_lba = LE_64(efi->efi_gpt_AlternateLBA); + UUID_LE_CONVERT(vtoc->efi_disk_uguid, efi->efi_gpt_DiskGUID); + + /* + * If the array the user passed in is too small, set the length + * to what it needs to be and return + */ + if (user_length < vtoc->efi_nparts) { + return (VT_EINVAL); + } + + for (i = 0; i < vtoc->efi_nparts; i++) { + + UUID_LE_CONVERT(vtoc->efi_parts[i].p_guid, + efi_parts[i].efi_gpe_PartitionTypeGUID); + + for (j = 0; + j < sizeof (conversion_array) + / sizeof (struct uuid_to_ptag); j++) { + + if (memcmp(&vtoc->efi_parts[i].p_guid, + &conversion_array[j].uuid, + sizeof (struct uuid)) == 0) { + vtoc->efi_parts[i].p_tag = j; + if (j == 0x18) { + j = 0x02; + } + break; + } + } + if (vtoc->efi_parts[i].p_tag == V_UNASSIGNED) + continue; + vtoc->efi_parts[i].p_flag = + LE_16(efi_parts[i].efi_gpe_Attributes.PartitionAttrs); + vtoc->efi_parts[i].p_start = + LE_64(efi_parts[i].efi_gpe_StartingLBA); + vtoc->efi_parts[i].p_size = + LE_64(efi_parts[i].efi_gpe_EndingLBA) - + vtoc->efi_parts[i].p_start + 1; + for (j = 0; j < EFI_PART_NAME_LEN; j++) { + vtoc->efi_parts[i].p_name[j] = + (uchar_t)LE_16( + efi_parts[i].efi_gpe_PartitionName[j]); + } + + UUID_LE_CONVERT(vtoc->efi_parts[i].p_uguid, + efi_parts[i].efi_gpe_UniquePartitionGUID); + } + umem_free_aligned(efi, label_len); + + return (dki_info.dki_partition); +} + +/* writes a "protective" MBR */ +static int +write_pmbr(int fd, struct dk_gpt *vtoc) +{ + dk_efi_t dk_ioc; + struct mboot mb; + uchar_t *cp; + diskaddr_t size_in_lba; + uchar_t *buf; + int len; + + len = (vtoc->efi_lbasize == 0) ? sizeof (mb) : vtoc->efi_lbasize; + buf = (void *)umem_alloc_aligned(len, len, UMEM_DEFAULT); + if (buf == NULL) + return (VT_ERROR); + + /* + * Preserve any boot code and disk signature if the first block is + * already an MBR. + */ + memset(buf, 0, len); + dk_ioc.dki_lba = 0; + dk_ioc.dki_length = len; + /* LINTED -- always longlong aligned */ + dk_ioc.dki_data = (efi_gpt_t *)buf; + if (efi_ioctl(fd, DKIOCGETEFI, &dk_ioc) == -1) { + (void) memcpy(&mb, buf, sizeof (mb)); + memset(&mb, 0, sizeof (mb)); + mb.signature = LE_16(MBB_MAGIC); + } else { + (void) memcpy(&mb, buf, sizeof (mb)); + if (mb.signature != LE_16(MBB_MAGIC)) { + memset(&mb, 0, sizeof (mb)); + mb.signature = LE_16(MBB_MAGIC); + } + } + + memset(&mb.parts, 0, sizeof (mb.parts)); + cp = (uchar_t *)&mb.parts[0]; + /* bootable or not */ + *cp++ = 0; + /* beginning CHS; 0xffffff if not representable */ + *cp++ = 0xff; + *cp++ = 0xff; + *cp++ = 0xff; + /* OS type */ + *cp++ = EFI_PMBR; + /* ending CHS; 0xffffff if not representable */ + *cp++ = 0xff; + *cp++ = 0xff; + *cp++ = 0xff; + /* starting LBA: 1 (little endian format) by EFI definition */ + *cp++ = 0x01; + *cp++ = 0x00; + *cp++ = 0x00; + *cp++ = 0x00; + /* ending LBA: last block on the disk (little endian format) */ + size_in_lba = vtoc->efi_last_lba; + if (size_in_lba < 0xffffffff) { + *cp++ = (size_in_lba & 0x000000ff); + *cp++ = (size_in_lba & 0x0000ff00) >> 8; + *cp++ = (size_in_lba & 0x00ff0000) >> 16; + *cp++ = (size_in_lba & 0xff000000) >> 24; + } else { + *cp++ = 0xff; + *cp++ = 0xff; + *cp++ = 0xff; + *cp++ = 0xff; + } + + (void) (void *) memcpy(buf, &mb, sizeof (mb)); + /* LINTED -- always longlong aligned */ + dk_ioc.dki_data = (efi_gpt_t *)buf; + dk_ioc.dki_lba = 0; + dk_ioc.dki_length = len; + if (efi_ioctl(fd, DKIOCSETEFI, &dk_ioc) == -1) { + umem_free_aligned(buf, len); + switch (errno) { + case EIO: + return (VT_EIO); + case EINVAL: + return (VT_EINVAL); + default: + return (VT_ERROR); + } + } + umem_free_aligned(buf, len); + return (0); +} + +/* make sure the user specified something reasonable */ +static int +check_input(struct dk_gpt *vtoc) +{ + int resv_part = -1; + int i, j; + diskaddr_t istart, jstart, isize, jsize, endsect; + + /* + * Sanity-check the input (make sure no partitions overlap) + */ + for (i = 0; i < vtoc->efi_nparts; i++) { + /* It can't be unassigned and have an actual size */ + if ((vtoc->efi_parts[i].p_tag == V_UNASSIGNED) && + (vtoc->efi_parts[i].p_size != 0)) { + if (efi_debug) { + (void) fprintf(stderr, "partition %d is " + "\"unassigned\" but has a size of %llu", + i, vtoc->efi_parts[i].p_size); + } + return (VT_EINVAL); + } + if (vtoc->efi_parts[i].p_tag == V_UNASSIGNED) { + if (uuid_is_null((uchar_t *)&vtoc->efi_parts[i].p_guid)) + continue; + /* we have encountered an unknown uuid */ + vtoc->efi_parts[i].p_tag = 0xff; + } + if (vtoc->efi_parts[i].p_tag == V_RESERVED) { + if (resv_part != -1) { + if (efi_debug) { + (void) fprintf(stderr, "found " + "duplicate reserved partition " + "at %d\n", i); + } + return (VT_EINVAL); + } + resv_part = i; + } + if ((vtoc->efi_parts[i].p_start < vtoc->efi_first_u_lba) || + (vtoc->efi_parts[i].p_start > vtoc->efi_last_u_lba)) { + if (efi_debug) { + (void) fprintf(stderr, + "Partition %d starts at %llu. ", + i, + vtoc->efi_parts[i].p_start); + (void) fprintf(stderr, + "It must be between %llu and %llu.\n", + vtoc->efi_first_u_lba, + vtoc->efi_last_u_lba); + } + return (VT_EINVAL); + } + if ((vtoc->efi_parts[i].p_start + + vtoc->efi_parts[i].p_size < + vtoc->efi_first_u_lba) || + (vtoc->efi_parts[i].p_start + + vtoc->efi_parts[i].p_size > + vtoc->efi_last_u_lba + 1)) { + if (efi_debug) { + (void) fprintf(stderr, + "Partition %d ends at %llu. ", + i, + vtoc->efi_parts[i].p_start + + vtoc->efi_parts[i].p_size); + (void) fprintf(stderr, + "It must be between %llu and %llu.\n", + vtoc->efi_first_u_lba, + vtoc->efi_last_u_lba); + } + return (VT_EINVAL); + } + + for (j = 0; j < vtoc->efi_nparts; j++) { + isize = vtoc->efi_parts[i].p_size; + jsize = vtoc->efi_parts[j].p_size; + istart = vtoc->efi_parts[i].p_start; + jstart = vtoc->efi_parts[j].p_start; + if ((i != j) && (isize != 0) && (jsize != 0)) { + endsect = jstart + jsize -1; + if ((jstart <= istart) && + (istart <= endsect)) { + if (efi_debug) { + (void) fprintf(stderr, + "Partition %d overlaps " + "partition %d.", i, j); + } + return (VT_EINVAL); + } + } + } + } + /* just a warning for now */ + if ((resv_part == -1) && efi_debug) { + (void) fprintf(stderr, + "no reserved partition found\n"); + } + return (0); +} + +/* + * add all the unallocated space to the current label + */ +int +efi_use_whole_disk(int fd) +{ + struct dk_gpt *efi_label; + int rval; + int i; + uint_t resv_index = 0, data_index = 0; + diskaddr_t resv_start = 0, data_start = 0; + diskaddr_t difference; + + rval = efi_alloc_and_read(fd, &efi_label); + if (rval < 0) { + return (rval); + } + + /* + * If alter_lba is 1, we are using the backup label. + * Since we can locate the backup label by disk capacity, + * there must be no unallocated space. + */ + if ((efi_label->efi_altern_lba == 1) || (efi_label->efi_altern_lba + >= efi_label->efi_last_lba)) { + if (efi_debug) { + (void) fprintf(stderr, + "efi_use_whole_disk: requested space not found\n"); + } + efi_free(efi_label); + return (VT_ENOSPC); + } + + difference = efi_label->efi_last_lba - efi_label->efi_altern_lba; + + /* + * Find the last physically non-zero partition. + * This is the reserved partition. + */ + for (i = 0; i < efi_label->efi_nparts; i ++) { + if (resv_start < efi_label->efi_parts[i].p_start) { + resv_start = efi_label->efi_parts[i].p_start; + resv_index = i; + } + } + + /* + * Find the last physically non-zero partition before that. + * This is the data partition. + */ + for (i = 0; i < resv_index; i ++) { + if (data_start < efi_label->efi_parts[i].p_start) { + data_start = efi_label->efi_parts[i].p_start; + data_index = i; + } + } + + /* + * Move the reserved partition. There is currently no data in + * here except fabricated devids (which get generated via + * efi_write()). So there is no need to copy data. + */ + efi_label->efi_parts[data_index].p_size += difference; + efi_label->efi_parts[resv_index].p_start += difference; + efi_label->efi_last_u_lba += difference; + + rval = efi_write(fd, efi_label); + if (rval < 0) { + if (efi_debug) { + (void) fprintf(stderr, + "efi_use_whole_disk:fail to write label, rval=%d\n", + rval); + } + efi_free(efi_label); + return (rval); + } + + efi_free(efi_label); + return (0); +} + + +/* + * write EFI label and backup label + */ +int +efi_write(int fd, struct dk_gpt *vtoc) +{ + dk_efi_t dk_ioc; + efi_gpt_t *efi; + efi_gpe_t *efi_parts; + int i, j; + struct dk_cinfo dki_info; + int rval; + int md_flag = 0; + int nblocks; + diskaddr_t lba_backup_gpt_hdr; + + if ((rval = efi_get_info(fd, &dki_info)) != 0) + return (rval); + + /* check if we are dealing wih a metadevice */ + if ((strncmp(dki_info.dki_cname, "pseudo", 7) == 0) && + (strncmp(dki_info.dki_dname, "md", 3) == 0)) { + md_flag = 1; + } + + if (check_input(vtoc)) { + /* + * not valid; if it's a metadevice just pass it down + * because SVM will do its own checking + */ + if (md_flag == 0) { + return (VT_EINVAL); + } + } + + dk_ioc.dki_lba = 1; + if (NBLOCKS(vtoc->efi_nparts, vtoc->efi_lbasize) < 34) { + dk_ioc.dki_length = EFI_MIN_ARRAY_SIZE + vtoc->efi_lbasize; + } else { + dk_ioc.dki_length = (len_t)NBLOCKS(vtoc->efi_nparts, + vtoc->efi_lbasize) * + vtoc->efi_lbasize; + } + + /* + * the number of blocks occupied by GUID partition entry array + */ + nblocks = dk_ioc.dki_length / vtoc->efi_lbasize - 1; + + /* + * Backup GPT header is located on the block after GUID + * partition entry array. Here, we calculate the address + * for backup GPT header. + */ + lba_backup_gpt_hdr = vtoc->efi_last_u_lba + 1 + nblocks; + dk_ioc.dki_data = (void *)umem_alloc_aligned(dk_ioc.dki_length, + vtoc->efi_lbasize, UMEM_DEFAULT); + if (dk_ioc.dki_data == NULL) + return (VT_ERROR); + + memset(dk_ioc.dki_data, 0, dk_ioc.dki_length); + efi = dk_ioc.dki_data; + + /* stuff user's input into EFI struct */ + efi->efi_gpt_Signature = LE_64(EFI_SIGNATURE); + efi->efi_gpt_Revision = LE_32(vtoc->efi_version); /* 0x02000100 */ + efi->efi_gpt_HeaderSize = LE_32(sizeof (struct efi_gpt) - LEN_EFI_PAD); + efi->efi_gpt_Reserved1 = 0; + efi->efi_gpt_MyLBA = LE_64(1ULL); + efi->efi_gpt_AlternateLBA = LE_64(lba_backup_gpt_hdr); + efi->efi_gpt_FirstUsableLBA = LE_64(vtoc->efi_first_u_lba); + efi->efi_gpt_LastUsableLBA = LE_64(vtoc->efi_last_u_lba); + efi->efi_gpt_PartitionEntryLBA = LE_64(2ULL); + efi->efi_gpt_NumberOfPartitionEntries = LE_32(vtoc->efi_nparts); + efi->efi_gpt_SizeOfPartitionEntry = LE_32(sizeof (struct efi_gpe)); + UUID_LE_CONVERT(efi->efi_gpt_DiskGUID, vtoc->efi_disk_uguid); + + /* LINTED -- always longlong aligned */ + efi_parts = (efi_gpe_t *)((char *)dk_ioc.dki_data + vtoc->efi_lbasize); + + for (i = 0; i < vtoc->efi_nparts; i++) { + for (j = 0; + j < sizeof (conversion_array) / + sizeof (struct uuid_to_ptag); j++) { + + if (vtoc->efi_parts[i].p_tag == j) { + UUID_LE_CONVERT( + efi_parts[i].efi_gpe_PartitionTypeGUID, + conversion_array[j].uuid); + break; + } + } + + if (j == sizeof (conversion_array) / + sizeof (struct uuid_to_ptag)) { + /* + * If we didn't have a matching uuid match, bail here. + * Don't write a label with unknown uuid. + */ + if (efi_debug) { + (void) fprintf(stderr, + "Unknown uuid for p_tag %d\n", + vtoc->efi_parts[i].p_tag); + } + return (VT_EINVAL); + } + + /* Zero's should be written for empty partitions */ + if (vtoc->efi_parts[i].p_tag == V_UNASSIGNED) + continue; + + efi_parts[i].efi_gpe_StartingLBA = + LE_64(vtoc->efi_parts[i].p_start); + efi_parts[i].efi_gpe_EndingLBA = + LE_64(vtoc->efi_parts[i].p_start + + vtoc->efi_parts[i].p_size - 1); + efi_parts[i].efi_gpe_Attributes.PartitionAttrs = + LE_16(vtoc->efi_parts[i].p_flag); + for (j = 0; j < EFI_PART_NAME_LEN; j++) { + efi_parts[i].efi_gpe_PartitionName[j] = + LE_16((ushort_t)vtoc->efi_parts[i].p_name[j]); + } + if ((vtoc->efi_parts[i].p_tag != V_UNASSIGNED) && + uuid_is_null((uchar_t *)&vtoc->efi_parts[i].p_uguid)) { + (void) uuid_generate((uchar_t *) + &vtoc->efi_parts[i].p_uguid); + } + memcpy(&efi_parts[i].efi_gpe_UniquePartitionGUID, + &vtoc->efi_parts[i].p_uguid, + sizeof (uuid_t)); + } + efi->efi_gpt_PartitionEntryArrayCRC32 = + LE_32(efi_crc32((unsigned char *)efi_parts, + vtoc->efi_nparts * (int)sizeof (struct efi_gpe))); + efi->efi_gpt_HeaderCRC32 = + LE_32(efi_crc32((unsigned char *)efi, + LE_32(efi->efi_gpt_HeaderSize))); + + if (efi_ioctl(fd, DKIOCSETEFI, &dk_ioc) == -1) { + umem_free_aligned(dk_ioc.dki_data, dk_ioc.dki_length); + switch (errno) { + case EIO: + return (VT_EIO); + case EINVAL: + return (VT_EINVAL); + default: + return (VT_ERROR); + } + } + /* if it's a metadevice we're done */ + if (md_flag) { + umem_free_aligned(dk_ioc.dki_data, dk_ioc.dki_length); + return (0); + } + + /* write backup partition array */ + dk_ioc.dki_lba = vtoc->efi_last_u_lba + 1; + dk_ioc.dki_length -= vtoc->efi_lbasize; + /* LINTED */ + dk_ioc.dki_data = (efi_gpt_t *)((char *)dk_ioc.dki_data + + vtoc->efi_lbasize); + + if (efi_ioctl(fd, DKIOCSETEFI, &dk_ioc) == -1) { + /* + * we wrote the primary label okay, so don't fail + */ + if (efi_debug) { + (void) fprintf(stderr, + "write of backup partitions to block %llu " + "failed, errno %d\n", + vtoc->efi_last_u_lba + 1, + errno); + } + } + /* + * now swap MyLBA and AlternateLBA fields and write backup + * partition table header + */ + dk_ioc.dki_lba = lba_backup_gpt_hdr; + dk_ioc.dki_length = vtoc->efi_lbasize; + /* LINTED */ + dk_ioc.dki_data = (efi_gpt_t *)((char *)dk_ioc.dki_data - + vtoc->efi_lbasize); + efi->efi_gpt_AlternateLBA = LE_64(1ULL); + efi->efi_gpt_MyLBA = LE_64(lba_backup_gpt_hdr); + efi->efi_gpt_PartitionEntryLBA = LE_64(vtoc->efi_last_u_lba + 1); + efi->efi_gpt_HeaderCRC32 = 0; + efi->efi_gpt_HeaderCRC32 = + LE_32(efi_crc32((unsigned char *)dk_ioc.dki_data, + LE_32(efi->efi_gpt_HeaderSize))); + + if (efi_ioctl(fd, DKIOCSETEFI, &dk_ioc) == -1) { + if (efi_debug) { + (void) fprintf(stderr, + "write of backup header to block %llu failed, " + "errno %d\n", + lba_backup_gpt_hdr, + errno); + } + } + /* write the PMBR */ + (void) write_pmbr(fd, vtoc); + umem_free_aligned(dk_ioc.dki_data, dk_ioc.dki_length); + + return (0); +} + +void +efi_free(struct dk_gpt *ptr) +{ + free(ptr); +} + +/* + * Input: File descriptor + * Output: 1 if disk has an EFI label, or > 2TB with no VTOC or legacy MBR. + * Otherwise 0. + */ +int +efi_type(int fd) +{ +#if 0 + struct vtoc vtoc; + struct extvtoc extvtoc; + + if (ioctl(fd, DKIOCGEXTVTOC, &extvtoc) == -1) { + if (errno == ENOTSUP) + return (1); + else if (errno == ENOTTY) { + if (ioctl(fd, DKIOCGVTOC, &vtoc) == -1) + if (errno == ENOTSUP) + return (1); + } + } + return (0); +#else + return (ENOSYS); +#endif +} + +void +efi_err_check(struct dk_gpt *vtoc) +{ + int resv_part = -1; + int i, j; + diskaddr_t istart, jstart, isize, jsize, endsect; + int overlap = 0; + + /* + * make sure no partitions overlap + */ + for (i = 0; i < vtoc->efi_nparts; i++) { + /* It can't be unassigned and have an actual size */ + if ((vtoc->efi_parts[i].p_tag == V_UNASSIGNED) && + (vtoc->efi_parts[i].p_size != 0)) { + (void) fprintf(stderr, + "partition %d is \"unassigned\" but has a size " + "of %llu\n", i, vtoc->efi_parts[i].p_size); + } + if (vtoc->efi_parts[i].p_tag == V_UNASSIGNED) { + continue; + } + if (vtoc->efi_parts[i].p_tag == V_RESERVED) { + if (resv_part != -1) { + (void) fprintf(stderr, + "found duplicate reserved partition at " + "%d\n", i); + } + resv_part = i; + if (vtoc->efi_parts[i].p_size != EFI_MIN_RESV_SIZE) + (void) fprintf(stderr, + "Warning: reserved partition size must " + "be %d sectors\n", EFI_MIN_RESV_SIZE); + } + if ((vtoc->efi_parts[i].p_start < vtoc->efi_first_u_lba) || + (vtoc->efi_parts[i].p_start > vtoc->efi_last_u_lba)) { + (void) fprintf(stderr, + "Partition %d starts at %llu\n", + i, + vtoc->efi_parts[i].p_start); + (void) fprintf(stderr, + "It must be between %llu and %llu.\n", + vtoc->efi_first_u_lba, + vtoc->efi_last_u_lba); + } + if ((vtoc->efi_parts[i].p_start + + vtoc->efi_parts[i].p_size < + vtoc->efi_first_u_lba) || + (vtoc->efi_parts[i].p_start + + vtoc->efi_parts[i].p_size > + vtoc->efi_last_u_lba + 1)) { + (void) fprintf(stderr, + "Partition %d ends at %llu\n", + i, + vtoc->efi_parts[i].p_start + + vtoc->efi_parts[i].p_size); + (void) fprintf(stderr, + "It must be between %llu and %llu.\n", + vtoc->efi_first_u_lba, + vtoc->efi_last_u_lba); + } + + for (j = 0; j < vtoc->efi_nparts; j++) { + isize = vtoc->efi_parts[i].p_size; + jsize = vtoc->efi_parts[j].p_size; + istart = vtoc->efi_parts[i].p_start; + jstart = vtoc->efi_parts[j].p_start; + if ((i != j) && (isize != 0) && (jsize != 0)) { + endsect = jstart + jsize -1; + if ((jstart <= istart) && + (istart <= endsect)) { + if (!overlap) { + (void) fprintf(stderr, + "label error: EFI Labels do not " + "support overlapping partitions\n"); + } + (void) fprintf(stderr, + "Partition %d overlaps partition " + "%d.\n", i, j); + overlap = 1; + } + } + } + } + /* make sure there is a reserved partition */ + if (resv_part == -1) { + (void) fprintf(stderr, + "no reserved partition found\n"); + } +} diff --git a/lib/libicp/CMakeLists.txt b/lib/libicp/CMakeLists.txt new file mode 100644 index 000000000000..7e14c6e3e603 --- /dev/null +++ b/lib/libicp/CMakeLists.txt @@ -0,0 +1,64 @@ + +use_clang() + +set(ICP_MODULE_DIR "../../module/icp") + +add_library(libicp + "${ICP_MODULE_DIR}/algs/aes/aes_impl_aesni.c" + "${ICP_MODULE_DIR}/algs/aes/aes_impl_generic.c" + "${ICP_MODULE_DIR}/algs/aes/aes_impl_x86-64.c" + "${ICP_MODULE_DIR}/algs/aes/aes_impl.c" + "${ICP_MODULE_DIR}/algs/aes/aes_modes.c" + "${ICP_MODULE_DIR}/algs/blake3/blake3.c" + "${ICP_MODULE_DIR}/algs/blake3/blake3_generic.c" + "${ICP_MODULE_DIR}/algs/blake3/blake3_impl.c" + "${ICP_MODULE_DIR}/algs/edonr/edonr.c" + "${ICP_MODULE_DIR}/algs/modes/cbc.c" + "${ICP_MODULE_DIR}/algs/modes/ccm.c" + "${ICP_MODULE_DIR}/algs/modes/ctr.c" + "${ICP_MODULE_DIR}/algs/modes/ecb.c" + "${ICP_MODULE_DIR}/algs/modes/gcm.c" + "${ICP_MODULE_DIR}/algs/modes/gcm_generic.c" + "${ICP_MODULE_DIR}/algs/modes/gcm_pclmulqdq.c" + "${ICP_MODULE_DIR}/algs/modes/modes.c" + "${ICP_MODULE_DIR}/algs/sha2/sha2_generic.c" + "${ICP_MODULE_DIR}/algs/sha2/sha256_impl.c" + "${ICP_MODULE_DIR}/algs/sha2/sha512_impl.c" + "${ICP_MODULE_DIR}/algs/skein/skein.c" + "${ICP_MODULE_DIR}/algs/skein/skein_block.c" + "${ICP_MODULE_DIR}/algs/skein/skein_iv.c" + "${ICP_MODULE_DIR}/api/kcf_cipher.c" + "${ICP_MODULE_DIR}/api/kcf_ctxops.c" + "${ICP_MODULE_DIR}/api/kcf_mac.c" + "${ICP_MODULE_DIR}/core/kcf_callprov.c" + "${ICP_MODULE_DIR}/core/kcf_mech_tabs.c" + "${ICP_MODULE_DIR}/core/kcf_prov_lib.c" + "${ICP_MODULE_DIR}/core/kcf_prov_tabs.c" + "${ICP_MODULE_DIR}/core/kcf_sched.c" + "${ICP_MODULE_DIR}/illumos-crypto.c" + "${ICP_MODULE_DIR}/io/aes.c" + "${ICP_MODULE_DIR}/io/sha2_mod.c" + "${ICP_MODULE_DIR}/io/skein_mod.c" + "${ICP_MODULE_DIR}/spi/kcf_spi.c" + + "${ICP_MODULE_DIR}/asm-x86_64/aes/aeskey.c" + "${ICP_MODULE_DIR}/asm-x86_64/aes/aes_aesni.S" + "${ICP_MODULE_DIR}/asm-x86_64/aes/aes_amd64.S" + "${ICP_MODULE_DIR}/asm-x86_64/modes/aesni-gcm-x86_64.S" + "${ICP_MODULE_DIR}/asm-x86_64/modes/gcm_pclmulqdq.S" + "${ICP_MODULE_DIR}/asm-x86_64/modes/ghash-x86_64.S" + + "${ICP_MODULE_DIR}/asm-x86_64/blake3/blake3_avx2.S" + "${ICP_MODULE_DIR}/asm-x86_64/blake3/blake3_avx512.S" + "${ICP_MODULE_DIR}/asm-x86_64/blake3/blake3_sse2.S" + "${ICP_MODULE_DIR}/asm-x86_64/blake3/blake3_sse41.S" + "${ICP_MODULE_DIR}/asm-x86_64/sha2/sha256-x86_64.S" + "${ICP_MODULE_DIR}/asm-x86_64/sha2/sha512-x86_64.S" + +) + +# Add windows/assembler sources here too. + + +target_include_directories(libicp PRIVATE "${ICP_MODULE_DIR}/include") +target_link_libraries(libicp PUBLIC libspl libpthread) diff --git a/lib/libnvpair/CMakeLists.txt b/lib/libnvpair/CMakeLists.txt new file mode 100644 index 000000000000..a95ddb4ce88b --- /dev/null +++ b/lib/libnvpair/CMakeLists.txt @@ -0,0 +1,14 @@ + +use_clang() + +set(NVPAIR_MODULE_DIR "../../module/nvpair") + +add_library(libnvpair + "${NVPAIR_MODULE_DIR}/fnvpair.c" + "${NVPAIR_MODULE_DIR}/nvpair.c" + "${NVPAIR_MODULE_DIR}/nvpair_alloc_fixed.c" + libnvpair.c + libnvpair_json.c + nvpair_alloc_system.c +) +target_link_libraries(libnvpair PUBLIC libspl) diff --git a/lib/libshare/CMakeLists.txt b/lib/libshare/CMakeLists.txt new file mode 100644 index 000000000000..3bde4e95edc1 --- /dev/null +++ b/lib/libshare/CMakeLists.txt @@ -0,0 +1,12 @@ + +use_clang() + +include_directories(PRIVATE .) + +add_library(libshare + libshare.c + os/windows/nfs.c + os/windows/smb.c +) +target_include_directories(libshare PRIVATE "${CMAKE_SOURCE_DIR}/lib/libzfs") +target_link_libraries(libshare PUBLIC libspl) diff --git a/lib/libshare/os/windows/nfs.c b/lib/libshare/os/windows/nfs.c new file mode 100644 index 000000000000..f1b3634edec1 --- /dev/null +++ b/lib/libshare/os/windows/nfs.c @@ -0,0 +1,414 @@ +/* + * Copyright (c) 2007 Pawel Jakub Dawidek + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * Copyright (c) 2020 by Delphix. All rights reserved. + */ + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "libzfs_impl.h" +#include "libshare_impl.h" +#include "nfs.h" + +#define _PATH_MOUNTDPID "/var/run/mountd.pid" +#define FILE_HEADER "# !!! DO NOT EDIT THIS FILE MANUALLY !!!\n\n" +#define OPTSSIZE 1024 +#define MAXLINESIZE (PATH_MAX + OPTSSIZE) +#define ZFS_EXPORTS_FILE "/etc/zfs/exports" +#define ZFS_EXPORTS_LOCK ZFS_EXPORTS_FILE".lock" + +static sa_fstype_t *nfs_fstype; + +static int nfs_lock_fd = -1; +#define LOCK_EX 1 +#define LOCK_SH 2 +#define LOCK_UN 3 +#define LOCK_NB 4 + +/* Fix me when we add shares to windows */ +static inline flock(int fd, int type) +{ + return (0); +} +/* + * The nfs_exports_[lock|unlock] is used to guard against conconcurrent + * updates to the exports file. Each protocol is responsible for + * providing the necessary locking to ensure consistency. + */ +static int +nfs_exports_lock(void) +{ + nfs_lock_fd = open(ZFS_EXPORTS_LOCK, + O_RDWR | O_CREAT, 0600); + if (nfs_lock_fd == -1) { + fprintf(stderr, "failed to lock %s: %s\n", + ZFS_EXPORTS_LOCK, strerror(errno)); + return (errno); + } + if (flock(nfs_lock_fd, LOCK_EX) != 0) { + fprintf(stderr, "failed to lock %s: %s\n", + ZFS_EXPORTS_LOCK, strerror(errno)); + return (errno); + } + return (0); +} + +static void +nfs_exports_unlock(void) +{ + verify(nfs_lock_fd > 0); + + if (flock(nfs_lock_fd, LOCK_UN) != 0) { + fprintf(stderr, "failed to unlock %s: %s\n", + ZFS_EXPORTS_LOCK, strerror(errno)); + } + close(nfs_lock_fd); + nfs_lock_fd = -1; +} + +/* + * Read one line from a file. Skip comments, empty lines and a line with a + * mountpoint specified in the 'skip' argument. + * + * NOTE: This function returns a static buffer and thus is not thread-safe. + */ +static char * +zgetline(FILE *fd, const char *skip) +{ + static char line[MAXLINESIZE]; + size_t len, skiplen = 0; + char *s, last; + + if (skip != NULL) + skiplen = strlen(skip); + for (;;) { + s = fgets(line, sizeof (line), fd); + if (s == NULL) + return (NULL); + /* Skip empty lines and comments. */ + if (line[0] == '\n' || line[0] == '#') + continue; + len = strlen(line); + if (line[len - 1] == '\n') + line[len - 1] = '\0'; + last = line[skiplen]; + /* Skip the given mountpoint. */ + if (skip != NULL && strncmp(skip, line, skiplen) == 0 && + (last == '\t' || last == ' ' || last == '\0')) { + continue; + } + break; + } + return (line); +} + +/* + * This function translate options to a format acceptable by exports(5), eg. + * + * -ro -network=192.168.0.0 -mask=255.255.255.0 -maproot=0 \ + * zfs.freebsd.org 69.147.83.54 + * + * Accepted input formats: + * + * ro,network=192.168.0.0,mask=255.255.255.0,maproot=0,zfs.freebsd.org + * ro network=192.168.0.0 mask=255.255.255.0 maproot=0 zfs.freebsd.org + * -ro,-network=192.168.0.0,-mask=255.255.255.0,-maproot=0,zfs.freebsd.org + * -ro -network=192.168.0.0 -mask=255.255.255.0 -maproot=0 \ + * zfs.freebsd.org + * + * Recognized keywords: + * + * ro, maproot, mapall, mask, network, sec, alldirs, public, webnfs, + * index, quiet + * + * NOTE: This function returns a static buffer and thus is not thread-safe. + */ +static char * +translate_opts(const char *shareopts) +{ + static const char *known_opts[] = { "ro", "maproot", "mapall", "mask", + "network", "sec", "alldirs", "public", "webnfs", "index", "quiet", + NULL }; + static char newopts[OPTSSIZE]; + char oldopts[OPTSSIZE]; + char *o, *s = NULL; + unsigned int i; + size_t len; + + strlcpy(oldopts, shareopts, sizeof (oldopts)); + newopts[0] = '\0'; + s = oldopts; + while ((o = strsep(&s, "-, ")) != NULL) { + if (o[0] == '\0') + continue; + for (i = 0; known_opts[i] != NULL; i++) { + len = strlen(known_opts[i]); + if (strncmp(known_opts[i], o, len) == 0 && + (o[len] == '\0' || o[len] == '=')) { + strlcat(newopts, "-", sizeof (newopts)); + break; + } + } + strlcat(newopts, o, sizeof (newopts)); + strlcat(newopts, " ", sizeof (newopts)); + } + return (newopts); +} + +static char * +nfs_init_tmpfile(void) +{ + char *tmpfile = NULL; + + if (asprintf(&tmpfile, "%s%s", ZFS_EXPORTS_FILE, ".XXXXXXXX") == -1) { + fprintf(stderr, "Unable to allocate buffer for temporary " + "file name\n"); + return (NULL); + } + + int fd = mkstemp(tmpfile); + if (fd == -1) { + fprintf(stderr, "Unable to create temporary file: %s", + strerror(errno)); + free(tmpfile); + return (NULL); + } + close(fd); + return (tmpfile); +} + +static int +nfs_fini_tmpfile(char *tmpfile) +{ + if (rename(tmpfile, ZFS_EXPORTS_FILE) == -1) { + fprintf(stderr, "Unable to rename %s: %s\n", tmpfile, + strerror(errno)); + unlink(tmpfile); + free(tmpfile); + return (SA_SYSTEM_ERR); + } + free(tmpfile); + return (SA_OK); +} + +/* + * This function copies all entries from the exports file to "filename", + * omitting any entries for the specified mountpoint. + */ +static int +nfs_copy_entries(char *filename, const char *mountpoint) +{ + int error = SA_OK; + char *line; + + /* + * If the file doesn't exist then there is nothing more + * we need to do. + */ + FILE *oldfp = fopen(ZFS_EXPORTS_FILE, "r"); + if (oldfp == NULL) + return (SA_OK); + + FILE *newfp = fopen(filename, "w+"); + fputs(FILE_HEADER, newfp); + while ((line = zgetline(oldfp, mountpoint)) != NULL) + fprintf(newfp, "%s\n", line); + if (ferror(oldfp) != 0) { + error = ferror(oldfp); + } + if (error == 0 && ferror(newfp) != 0) { + error = ferror(newfp); + } + + if (fclose(newfp) != 0) { + fprintf(stderr, "Unable to close file %s: %s\n", + filename, strerror(errno)); + error = error != 0 ? error : SA_SYSTEM_ERR; + } + fclose(oldfp); + + return (error); +} + +static int +nfs_enable_share(sa_share_impl_t impl_share) +{ + char *filename = NULL; + int error; + + if ((filename = nfs_init_tmpfile()) == NULL) + return (SA_SYSTEM_ERR); + + error = nfs_exports_lock(); + if (error != 0) { + unlink(filename); + free(filename); + return (error); + } + + error = nfs_copy_entries(filename, impl_share->sa_mountpoint); + if (error != SA_OK) { + unlink(filename); + free(filename); + nfs_exports_unlock(); + return (error); + } + + FILE *fp = fopen(filename, "a+"); + if (fp == NULL) { + fprintf(stderr, "failed to open %s file: %s", filename, + strerror(errno)); + unlink(filename); + free(filename); + nfs_exports_unlock(); + return (SA_SYSTEM_ERR); + } + char *shareopts = impl_share->sa_shareopts; + if (strcmp(shareopts, "on") == 0) + shareopts = ""; + + if (fprintf(fp, "%s\t%s\n", impl_share->sa_mountpoint, + translate_opts(shareopts)) < 0) { + fprintf(stderr, "failed to write to %s\n", filename); + fclose(fp); + unlink(filename); + free(filename); + nfs_exports_unlock(); + return (SA_SYSTEM_ERR); + } + + if (fclose(fp) != 0) { + fprintf(stderr, "Unable to close file %s: %s\n", + filename, strerror(errno)); + unlink(filename); + free(filename); + nfs_exports_unlock(); + return (SA_SYSTEM_ERR); + } + error = nfs_fini_tmpfile(filename); + nfs_exports_unlock(); + return (error); +} + +static int +nfs_disable_share(sa_share_impl_t impl_share) +{ + int error; + char *filename = NULL; + + if ((filename = nfs_init_tmpfile()) == NULL) + return (SA_SYSTEM_ERR); + + error = nfs_exports_lock(); + if (error != 0) { + unlink(filename); + free(filename); + return (error); + } + + error = nfs_copy_entries(filename, impl_share->sa_mountpoint); + if (error != SA_OK) { + unlink(filename); + free(filename); + nfs_exports_unlock(); + return (error); + } + + error = nfs_fini_tmpfile(filename); + nfs_exports_unlock(); + return (error); +} + +/* + * NOTE: This function returns a static buffer and thus is not thread-safe. + */ +static boolean_t +nfs_is_shared(sa_share_impl_t impl_share) +{ + static char line[MAXLINESIZE]; + char *s, last; + size_t len; + char *mntpoint = impl_share->sa_mountpoint; + size_t mntlen = strlen(mntpoint); + + FILE *fp = fopen(ZFS_EXPORTS_FILE, "r"); + if (fp == NULL) + return (B_FALSE); + + for (;;) { + s = fgets(line, sizeof (line), fp); + if (s == NULL) + return (B_FALSE); + /* Skip empty lines and comments. */ + if (line[0] == '\n' || line[0] == '#') + continue; + len = strlen(line); + if (line[len - 1] == '\n') + line[len - 1] = '\0'; + last = line[mntlen]; + /* Skip the given mountpoint. */ + if (strncmp(mntpoint, line, mntlen) == 0 && + (last == '\t' || last == ' ' || last == '\0')) { + fclose(fp); + return (B_TRUE); + } + } + fclose(fp); + return (B_FALSE); +} + +static int +nfs_validate_shareopts(const char *shareopts) +{ + return (SA_OK); +} + +/* + * Commit the shares by restarting mountd. + */ +static int +nfs_commit_shares(void) +{ + return (SA_OK); +} + +const sa_fstype_t libshare_nfs_type = { + .enable_share = nfs_enable_share, + .disable_share = nfs_disable_share, + .is_shared = nfs_is_shared, + + .validate_shareopts = nfs_validate_shareopts, + .commit_shares = nfs_commit_shares, +}; diff --git a/lib/libshare/os/windows/smb.c b/lib/libshare/os/windows/smb.c new file mode 100644 index 000000000000..27e107f4ca7d --- /dev/null +++ b/lib/libshare/os/windows/smb.c @@ -0,0 +1,94 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2020 by Delphix. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "libshare_impl.h" +#include "smb.h" + + +/* + * Enables SMB sharing for the specified share. + */ +static int +smb_enable_share(sa_share_impl_t impl_share) +{ + fprintf(stderr, "No SMB support in FreeBSD yet.\n"); + return (SA_NOT_SUPPORTED); +} +/* + * Disables SMB sharing for the specified share. + */ +static int +smb_disable_share(sa_share_impl_t impl_share) +{ + fprintf(stderr, "No SMB support in FreeBSD yet.\n"); + return (SA_NOT_SUPPORTED); +} + +/* + * Checks whether the specified SMB share options are syntactically correct. + */ +static int +smb_validate_shareopts(const char *shareopts) +{ + fprintf(stderr, "No SMB support in FreeBSD yet.\n"); + return (SA_NOT_SUPPORTED); +} + +/* + * Checks whether a share is currently active. + */ +static boolean_t +smb_is_share_active(sa_share_impl_t impl_share) +{ + return (B_FALSE); +} + +static int +smb_update_shares(void) +{ + /* Not implemented */ + return (0); +} + +const sa_fstype_t libshare_smb_type = { + .enable_share = smb_enable_share, + .disable_share = smb_disable_share, + .is_shared = smb_is_share_active, + + .validate_shareopts = smb_validate_shareopts, + .commit_shares = smb_update_shares, +}; diff --git a/lib/libspl/CMakeLists.txt b/lib/libspl/CMakeLists.txt new file mode 100644 index 000000000000..ad650783bd91 --- /dev/null +++ b/lib/libspl/CMakeLists.txt @@ -0,0 +1,35 @@ + +use_clang() + +add_library(libspl + atomic.c + list.c + mkdirp.c + strlcat.c + strlcpy.c + timestamp.c + assert.c + os/windows/posix.c + #fdatasync.c + #gethrestime.c + #gethrtime.c + # + os/windows/getmntany.c + os/windows/getopt.c + #os/windows/getoptl.c + os/windows/gethostid.c + os/windows/page.c + #os/windows/regex.c + os/windows/signal.c + #crc32.c + os/windows/weakpragma.c + os/windows/uio.c + os/windows/xdr.c + os/windows/xdr_array.c + os/windows/xdr_float.c + os/windows/xdr_mem.c + #os/windows/zmount.c + os/windows/zone.c +) + +target_link_libraries(libspl PUBLIC libpthread libregex PRIVATE advapi32 shell32) diff --git a/lib/libspl/atomic.c b/lib/libspl/atomic.c index 8cc350710ba0..33057353a7ce 100644 --- a/lib/libspl/atomic.c +++ b/lib/libspl/atomic.c @@ -339,7 +339,7 @@ atomic_swap_ptr(volatile void *target, void *bits) return (__atomic_exchange_n((void **)target, bits, __ATOMIC_SEQ_CST)); } -#ifndef _LP64 +#if !defined(_LP64) || defined(_WIN32) uint64_t atomic_load_64(volatile uint64_t *target) { diff --git a/lib/libspl/include/os/windows/aio.h b/lib/libspl/include/os/windows/aio.h new file mode 100644 index 000000000000..ec02dae5fc17 --- /dev/null +++ b/lib/libspl/include/os/windows/aio.h @@ -0,0 +1,71 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#ifndef _SPL_AIO_H +#define _SPL_AIO_H + +#include + +#define LIO_NOWAIT 0 +#define LIO_WAIT 1 + +#define LIO_NOP 0 +#define LIO_READ 0x01 /* Must match value of FREAD in sys/file.h */ +#define LIO_WRITE 0x02 /* Must match value of FWRITE in sys/file.h */ + +typedef struct aiocb { + int aio_fildes; + volatile void *aio_buf; /* buffer location */ + size_t aio_nbytes; /* length of transfer */ + off_t aio_offset; /* file offset */ + int aio_reqprio; /* request priority offset */ + // struct sigevent aio_sigevent; /* notification type */ + int aio_lio_opcode; /* listio operation */ + // aio_result_t aio_resultp; /* results */ + int aio_state; /* state flag for List I/O */ + int aio__pad[1]; /* extension padding */ +} aiocb_t; + + +static inline int lio_listio(int mode, struct aiocb *aiocb_list[], + int nitems, void * sevp) +{ + errno = EIO; + return (-1); +} + +static inline int +aio_error(const struct aiocb *aiocbp) +{ + return (EOPNOTSUPP); +} + +static inline ssize_t +aio_return(const struct aiocb *aiocbp) +{ + return (EOPNOTSUPP); +} + +#endif diff --git a/lib/libspl/include/os/windows/dirent.h b/lib/libspl/include/os/windows/dirent.h new file mode 100644 index 000000000000..801528631768 --- /dev/null +++ b/lib/libspl/include/os/windows/dirent.h @@ -0,0 +1,563 @@ +/* + * MIT License + * Copyright (c) 2019 win32ports + * Permission is hereby granted, free of charge, to any person obtaining + * a copy of this software and associated documentation files + * (the "Software"), to deal in the Software without restriction, + * including without limitation the rights to use, copy, modify, merge, + * publish, distribute, sublicense, and/or sell copies of the Software, + * and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * The above copyright notice and this permission notice shall be included + * in all copies or substantial portions of the Software. + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE + * OR OTHER DEALINGS IN THE SOFTWARE. + */ + +#pragma once + +#ifndef __DIRENT_H_9DE6B42C_8D0C_4D31_A8EF_8E4C30E6C46A__ +#define __DIRENT_H_9DE6B42C_8D0C_4D31_A8EF_8E4C30E6C46A__ + +#ifndef _WIN32 + +#pragma message("this dirent.h implementation is for Windows only!") + +#else /* _WIN32 */ + +#ifdef __cplusplus +extern "C" { +#endif /* __cplusplus */ + +#include +#include + +#if defined(_MSC_VER) +// These are now defined in stat.h +// #define S_IRUSR S_IREAD /* read, user */ +// #define S_IWUSR S_IWRITE /* write, user */ +// #define S_IXUSR 0 /* execute, user */ +// #define S_IRGRP 0 /* read, group */ +// #define S_IWGRP 0 /* write, group */ +// #define S_IXGRP 0 /* execute, group */ +// #define S_IROTH 0 /* read, others */ +// #define S_IWOTH 0 /* write, others */ +// #define S_IXOTH 0 /* execute, others */ +#endif + + +#ifndef NAME_MAX +#define NAME_MAX 260 +#endif /* NAME_MAX */ + +#ifndef DT_UNKNOWN +#define DT_UNKNOWN 0 +#endif /* DT_UNKNOWN */ + +#ifndef DT_FIFO +#define DT_FIFO 1 +#endif /* DT_FIFO */ + +#ifndef DT_CHR +#define DT_CHR 2 +#endif /* DT_CHR */ + +#ifndef DT_DIR +#define DT_DIR 4 +#endif /* DT_DIR */ + +#ifndef DT_BLK +#define DT_BLK 6 +#endif /* DT_BLK */ + +#ifndef DT_REG +#define DT_REG 8 +#endif /* DT_REF */ + +#ifndef DT_LNK +#define DT_LNK 10 +#endif /* DT_LNK */ + +#ifndef DT_SOCK +#define DT_SOCK 12 +#endif /* DT_SOCK */ + +#ifndef DT_WHT +#define DT_WHT 14 +#endif /* DT_WHT */ + +#ifndef _DIRENT_HAVE_D_NAMLEN +#define _DIRENT_HAVE_D_NAMLEN 1 +#endif /* _DIRENT_HAVE_D_NAMLEN */ + +#ifndef _DIRENT_HAVE_D_RECLEN +#define _DIRENT_HAVE_D_RECLEN 1 +#endif /* _DIRENT_HAVE_D_RECLEN */ + +#ifndef _DIRENT_HAVE_D_OFF +#define _DIRENT_HAVE_D_OFF 1 +#endif /* _DIRENT_HAVE_D_OFF */ + +#ifndef _DIRENT_HAVE_D_TYPE +#define _DIRENT_HAVE_D_TYPE 1 +#endif /* _DIRENT_HAVE_D_TYPE */ + +#ifndef NTFS_MAX_PATH +#define NTFS_MAX_PATH 32768 +#endif /* NTFS_MAX_PATH */ + +typedef void* DIR; + +typedef struct ino_t +{ + unsigned long long serial; + unsigned char fileid[16]; +} __ino_t; + +struct dirent +{ + __ino_t d_ino; + off_t d_off; + unsigned short d_reclen; + unsigned char d_namelen; + unsigned char d_type; + char d_name[NAME_MAX]; +}; + +struct __dir +{ + struct dirent *entries; + int fd; + long int count; + long int index; +}; + +#if __clang__ +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wunused-function" +#endif + +static int +closedir(DIR* dirp) +{ + struct __dir *data = NULL; + if (!dirp) { + errno = EBADF; + return (-1); + } + data = (struct __dir *)dirp; + CloseHandle((HANDLE)data->fd); + free(data->entries); + free(data); + return (0); +} + +static void +__seterrno(int value) +{ +#ifdef _MSC_VER + _set_errno(value); +#else /* _MSC_VER */ + errno = value; +#endif /* _MSC_VER */ +} + +static int +__islink(const wchar_t *name, char *buffer) +{ + DWORD io_result = 0; + DWORD bytes_returned = 0; + HANDLE hFile = CreateFileW(name, 0, 0, NULL, OPEN_EXISTING, + FILE_FLAG_OPEN_REPARSE_POINT | FILE_FLAG_BACKUP_SEMANTICS, 0); + if (hFile == INVALID_HANDLE_VALUE) + return (0); + + io_result = DeviceIoControl(hFile, FSCTL_GET_REPARSE_POINT, NULL, 0, + buffer, MAXIMUM_REPARSE_DATA_BUFFER_SIZE, &bytes_returned, NULL); + + CloseHandle(hFile); + + if (io_result == 0) + return (0); + + return (((REPARSE_GUID_DATA_BUFFER*)buffer)->ReparseTag == + IO_REPARSE_TAG_SYMLINK); +} + +static __ino_t +__inode(const wchar_t *name) +{ + __ino_t value = { 0 }; + BOOL result; + FILE_ID_INFO fileid; + BY_HANDLE_FILE_INFORMATION info; + HANDLE hFile = CreateFileW(name, GENERIC_READ, FILE_SHARE_READ, + NULL, OPEN_EXISTING, 0, 0); + if (hFile == INVALID_HANDLE_VALUE) + return (value); + + result = GetFileInformationByHandleEx(hFile, FileIdInfo, + &fileid, sizeof (fileid)); + if (result) { + value.serial = fileid.VolumeSerialNumber; + memcpy(value.fileid, fileid.FileId.Identifier, 16); + } else { + result = GetFileInformationByHandle(hFile, &info); + if (result) { + value.serial = info.dwVolumeSerialNumber; + memcpy(value.fileid + 8, &info.nFileIndexHigh, 4); + memcpy(value.fileid + 12, &info.nFileIndexLow, 4); + } + } + CloseHandle(hFile); + return (value); +} + +static DIR * +__internal_opendir(wchar_t *wname, int size) +{ + struct __dir *data = NULL; + struct dirent *tmp_entries = NULL; + static char default_char = '?'; + // static wchar_t* prefix = L"\\\\?\\"; + static wchar_t *suffix = L"\\*.*"; + int extra_prefix = 4; /* use prefix "\\?\" to handle long file names */ + static int extra_suffix = 4; /* use suffix "\*.*" to find everything */ + WIN32_FIND_DATAW w32fd = { 0 }; + HANDLE hFindFile = INVALID_HANDLE_VALUE; + static int grow_factor = 2; + char *buffer = NULL; + + memcpy(wname + extra_prefix + size - 1, suffix, + sizeof (wchar_t) * extra_prefix); + wname[size + extra_prefix + extra_suffix - 1] = 0; + + if (memcmp(wname, L"\\\\?\\", + // if (memcmp(wname + extra_prefix, L"\\\\?\\", + sizeof (wchar_t) * extra_prefix) == 0) { + wname += extra_prefix; + extra_prefix = 0; + } + + hFindFile = FindFirstFileW(wname, &w32fd); + if (INVALID_HANDLE_VALUE == hFindFile) { + __seterrno(ENOENT); + return (NULL); + } + + data = (struct __dir *)malloc(sizeof (struct __dir)); + if (!data) + goto out_of_memory; + wname[extra_prefix + size - 1] = 0; + data->fd = (int)(uintptr_t)CreateFileW(wname, 0, 0, NULL, OPEN_EXISTING, + FILE_FLAG_BACKUP_SEMANTICS, 0); + wname[extra_prefix + size - 1] = L'\\'; + data->count = 16; + data->index = 0; + data->entries = (struct dirent *)malloc( + sizeof (struct dirent) * data->count); + if (!data->entries) + goto out_of_memory; + buffer = (char *)malloc(MAXIMUM_REPARSE_DATA_BUFFER_SIZE); + if (!buffer) + goto out_of_memory; + do { + WideCharToMultiByte(CP_UTF8, 0, w32fd.cFileName, -1, + data->entries[data->index].d_name, NAME_MAX, + &default_char, NULL); + + memcpy(wname + extra_prefix + size, w32fd.cFileName, + sizeof (wchar_t) * NAME_MAX); + + if (((w32fd.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) == + FILE_ATTRIBUTE_REPARSE_POINT) && __islink(wname, buffer)) + data->entries[data->index].d_type = DT_LNK; + else if ((w32fd.dwFileAttributes & FILE_ATTRIBUTE_DEVICE) == + FILE_ATTRIBUTE_DEVICE) + data->entries[data->index].d_type = DT_CHR; + else if ((w32fd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == + FILE_ATTRIBUTE_DIRECTORY) + data->entries[data->index].d_type = DT_DIR; + else + data->entries[data->index].d_type = DT_REG; + + data->entries[data->index].d_ino = __inode(wname); + data->entries[data->index].d_reclen = sizeof (struct dirent); + data->entries[data->index].d_namelen = + (unsigned char)wcslen(w32fd.cFileName); + data->entries[data->index].d_off = 0; + + if (++data->index == data->count) { + tmp_entries = (struct dirent *)realloc(data->entries, + sizeof (struct dirent) * data->count * grow_factor); + if (!tmp_entries) + goto out_of_memory; + data->entries = tmp_entries; + data->count *= grow_factor; + } + } + while (FindNextFileW(hFindFile, &w32fd) != 0) + ; + + free(buffer); + FindClose(hFindFile); + + data->count = data->index; + data->index = 0; + return ((DIR*)data); + +out_of_memory: + if (data) { + if (INVALID_HANDLE_VALUE != (HANDLE)data->fd) + CloseHandle((HANDLE)data->fd); + free(data->entries); + } + free(buffer); + free(data); + if (INVALID_HANDLE_VALUE != hFindFile) + FindClose(hFindFile); + __seterrno(ENOMEM); + return (NULL); +} + +static wchar_t * +__get_buffer(void) +{ + wchar_t *name = (wchar_t *)malloc( + sizeof (wchar_t) * (NTFS_MAX_PATH + NAME_MAX + 8)); + if (name) + memcpy(name, L"\\\\?\\", sizeof (wchar_t) * 4); + return (name); +} + +static DIR * +opendir(const char *name) +{ + DIR *dirp = NULL; + wchar_t *wname = __get_buffer(); + int size = 0; + if (!wname) { + errno = ENOMEM; + return (NULL); + } + size = MultiByteToWideChar(CP_UTF8, 0, name, -1, wname + 4, + NTFS_MAX_PATH); + if (0 == size) { + free(wname); + return (NULL); + } + dirp = __internal_opendir(wname, size); + free(wname); + return (dirp); +} + +static DIR * +_wopendir(const wchar_t *name) +{ + DIR *dirp = NULL; + wchar_t *wname = __get_buffer(); + int size = 0; + if (!wname) { + errno = ENOMEM; + return (NULL); + } + size = (int)wcslen(name); + if (size > NTFS_MAX_PATH) { + free(wname); + return (NULL); + } + memcpy(wname + 4, name, sizeof (wchar_t) * (size + 1)); + dirp = __internal_opendir(wname, size + 1); + free(wname); + return (dirp); +} + +static DIR * +fdopendir(int fd) +{ + DIR *dirp = NULL; + wchar_t *wname = __get_buffer(); + int size = 0; + if (!wname) { + errno = ENOMEM; + return (NULL); + } + size = GetFinalPathNameByHandleW((HANDLE)fd, wname, + NTFS_MAX_PATH, FILE_NAME_NORMALIZED); + if (0 == size) { + free(wname); + errno = ENOTDIR; + return (NULL); + } + dirp = __internal_opendir(wname, size + 1); + free(wname); + return (dirp); +} + +static struct dirent * +readdir(DIR *dirp) +{ + struct __dir *data = (struct __dir *)dirp; + if (!data) { + errno = EBADF; + return (NULL); + } + if (data->index < data->count) { + return (&data->entries[data->index++]); + } + return (NULL); +} + +static int +readdir_r(DIR *dirp, struct dirent *entry, struct dirent **result) +{ + struct __dir *data = (struct __dir *)dirp; + if (!data) { + return (EBADF); + } + if (data->index < data->count) { + if (entry) + memcpy(entry, &data->entries[data->index++], + sizeof (struct dirent)); + if (result) + *result = entry; + } else if (result) + *result = NULL; + return (0); +} + +static void +seekdir(DIR *dirp, long int offset) +{ + if (dirp) { + struct __dir *data = (struct __dir *)dirp; + data->index = (offset < data->count) ? offset : data->index; + } +} + +static void +rewinddir(DIR *dirp) +{ + seekdir(dirp, 0); +} + +static long int +telldir(DIR *dirp) +{ + if (!dirp) { + errno = EBADF; + return (-1); + } + return (((struct __dir *)dirp)->count); +} + +static int +dirfd(DIR *dirp) +{ + if (!dirp) { + errno = EINVAL; + return (-1); + } + return (((struct __dir *)dirp)->fd); +} + +static int +scandir(const char *dirp, struct dirent ***namelist, + int (*filter)(const struct dirent *), + int (*compar)(const struct dirent **, const struct dirent **)) +{ + struct dirent **entries = NULL, **tmp_entries = NULL; + long int i = 0, index = 0, count = 16; + DIR *d = opendir(dirp); + struct __dir *data = (struct __dir *)d; + if (!data) { + closedir(d); + __seterrno(ENOENT); + return (-1); + } + entries = (struct dirent **) malloc(sizeof (struct dirent *) * count); + if (!entries) { + closedir(d); + __seterrno(ENOMEM); + return (-1); + } + for (i = 0; i < data->count; ++i) { + if (!filter || filter(&data->entries[i])) { + entries[index] = (struct dirent *) + malloc(sizeof (struct dirent)); + if (!entries[index]) { + closedir(d); + for (i = 0; i < index; ++i) + free(entries[index]); + free(entries); + __seterrno(ENOMEM); + return (-1); + } + memcpy(entries[index], &data->entries[i], + sizeof (struct dirent)); + if (++index == count) { + tmp_entries = (struct dirent **)realloc(entries, + sizeof (struct dirent *) * count * 2); + if (!tmp_entries) { + closedir(d); + for (i = 0; i < index; ++i) + free(entries[index - 1]); + free(entries); + __seterrno(ENOMEM); + return (-1); + } + entries = tmp_entries; + count *= 2; + } + } + } + // qsort(entries, index, sizeof (struct dirent*), compar); + entries[index] = NULL; + if (namelist) + *namelist = entries; + closedir(d); + return (0); +} + +static int +alphasort(const void *a, const void *b) +{ + struct dirent **dira = (struct dirent **)a, + **dirb = (struct dirent **)b; + if (!dira || !dirb) + return (0); + return (strcoll((*dira)->d_name, (*dirb)->d_name)); +} + +static int +__strverscmp(const char *s1, const char *s2) +{ + return (alphasort(s1, s2)); +} + +static int +versionsort(const void *a, const void *b) +{ + struct dirent **dira = (struct dirent **)a, + **dirb = (struct dirent **)b; + if (!dira || !dirb) + return (0); + return (__strverscmp((*dira)->d_name, (*dirb)->d_name)); +} + +#if __clang__ +#pragma GCC diagnostic pop +#endif + +#ifdef __cplusplus +} +#endif /* __cplusplus */ + +#endif /* _WIN32 */ + +#endif /* __DIRENT_H_9DE6B42C_8D0C_4D31_A8EF_8E4C30E6C46A__ */ diff --git a/lib/libspl/include/os/windows/dlfcn.h b/lib/libspl/include/os/windows/dlfcn.h new file mode 100644 index 000000000000..a01270be4701 --- /dev/null +++ b/lib/libspl/include/os/windows/dlfcn.h @@ -0,0 +1,26 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_WINDOWS_DLFCN_H +#define _LIBSPL_WINDOWS_DLFCN_H + +#endif /* _LIBSPL_WINDOWS_DLFCN_H */ diff --git a/lib/libspl/include/os/windows/err.h b/lib/libspl/include/os/windows/err.h new file mode 100644 index 000000000000..0baa7277891b --- /dev/null +++ b/lib/libspl/include/os/windows/err.h @@ -0,0 +1,48 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_ERR_H +#define _SPL_ERR_H + +#include + +void err(int, const char *, ...) _Noreturn __printf0like(2, 3); +void errx(int, const char *, ...) _Noreturn __printf0like(2, 3); +void warnx(const char *, ...) __printflike(1, 2); + +inline static void +err(int x, const char *f, ...) +{ +} + +inline static void +errx(int x, const char *f, ...) +{ + exit(x); +} + +inline static void +warnx(const char *, ...) +{ + exit(1); +} + +#endif diff --git a/lib/libspl/include/os/windows/fcntl.h b/lib/libspl/include/os/windows/fcntl.h new file mode 100644 index 000000000000..8c92570c4fcf --- /dev/null +++ b/lib/libspl/include/os/windows/fcntl.h @@ -0,0 +1,48 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_WINDOWS_SYS_FCNTL_H +#define _LIBSPL_WINDOWS_SYS_FCNTL_H + +#include_next + +#define O_LARGEFILE 0 +#define O_RSYNC 0 +#define O_DIRECT 0 +#define O_SYNC 0 +#define O_DSYNC 0 +#define O_CLOEXEC 0 +#define O_NDELAY 0 +#define O_NOCTTY 0 + +#define F_SETFD 2 +#define FD_CLOEXEC 1 + +#define O_DIRECTORY 0x1000000 + +/* + * Special value used to indicate openat should use the current + * working directory. + */ +#define AT_FDCWD -100 + +#endif /* _LIBSPL_SYS_FCNTL_H */ diff --git a/lib/libspl/include/os/windows/getopt.h b/lib/libspl/include/os/windows/getopt.h new file mode 100644 index 000000000000..0c2d8c9bcb79 --- /dev/null +++ b/lib/libspl/include/os/windows/getopt.h @@ -0,0 +1,45 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef LIBSPL_GETOPT_H_INCLUDED +#define LIBSPL_GETOPT_H_INCLUDED + +#define no_argument 0 +#define required_argument 1 +#define optional_argument 2 + +struct option +{ + const char *name; + int has_arg; + int *flag; + int val; +}; + +extern int getopt(int, char * const *, const char *); +extern int getopt_long(int, char * const *, const char *, + const struct option *, int *); +extern int getopt_long_only(int, char * const *, const char *, + const struct option *, int *); +extern int getsubopt(char **optionsp, char *tokens[], char **valuep); + +#endif // LIBSPL_GETOPT_H_INCLUDED diff --git a/lib/libspl/include/os/windows/grp.h b/lib/libspl/include/os/windows/grp.h new file mode 100644 index 000000000000..0a9ebe6cda90 --- /dev/null +++ b/lib/libspl/include/os/windows/grp.h @@ -0,0 +1,38 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#ifndef _SPL_GRP_H +#define _SPL_GRP_H + +struct group { /* see getgrent(3C) */ + char *gr_name; + char *gr_passwd; + gid_t gr_gid; + char **gr_mem; +}; + +extern struct group *getgrnam(const char *); /* MT-unsafe */ + +#endif diff --git a/lib/libspl/include/os/windows/langinfo.h b/lib/libspl/include/os/windows/langinfo.h new file mode 100644 index 000000000000..53e326346c40 --- /dev/null +++ b/lib/libspl/include/os/windows/langinfo.h @@ -0,0 +1,29 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#ifndef _SPL_LANGINFO_h +#define _SPL_LANGINFO_h + +#define _DATE_FMT 1 + +typedef int nl_item; + +char *nl_langinfo(nl_item item); +#endif diff --git a/lib/libspl/include/os/windows/libgen.h b/lib/libspl/include/os/windows/libgen.h new file mode 100644 index 000000000000..fa2c10584fa0 --- /dev/null +++ b/lib/libspl/include/os/windows/libgen.h @@ -0,0 +1,36 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2003 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _LIBSPL_LIBGEN_H +#define _LIBSPL_LIBGEN_H + +#include + +extern int mkdirp(const char *, mode_t); +extern char *basename(char *); + + +#endif /* _LIBSPL_LIBGEN_H */ diff --git a/lib/libspl/include/os/windows/libintl.h b/lib/libspl/include/os/windows/libintl.h new file mode 100644 index 000000000000..c9b6e15696c8 --- /dev/null +++ b/lib/libspl/include/os/windows/libintl.h @@ -0,0 +1,35 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef LIBSPL_LIBINTL_H +#define LIBSPL_LIBINTL_H + +#ifdef HAVE_GETTEXT + +#else +#define gettext(str) (str) +#define dgettext(domain, str) (str) +#define textdomain(domain) (domain) +#endif + + +#endif diff --git a/lib/libspl/include/os/windows/mntent.h b/lib/libspl/include/os/windows/mntent.h new file mode 100644 index 000000000000..69220538a986 --- /dev/null +++ b/lib/libspl/include/os/windows/mntent.h @@ -0,0 +1,159 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + * + * Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T + * All Rights Reserved + */ + +#ifndef _WINDOWS_MNTENT_H +#define _WINDOWS_MNTENT_H + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define MNTTAB "nul" +#define VFSTAB "/etc/vfstab" +#define MNTMAXSTR 128 + +#define MNTTYPE_ZFS "zfs" /* ZFS file system */ +#define MNTTYPE_UFS "ufs" /* Unix file system */ +#define MNTTYPE_NFS "nfs" /* NFS file system */ +#define MNTTYPE_NFS3 "nfs3" /* NFS Version 3 file system */ +#define MNTTYPE_NFS4 "nfs4" /* NFS Version 4 file system */ +#define MNTTYPE_CACHEFS "cachefs" /* Cache File System */ +#define MNTTYPE_PCFS "pcfs" /* PC (MSDOS) file system */ +#define MNTTYPE_PC MNTTYPE_PCFS /* Deprecated name; use MNTTYPE_PCFS */ +#define MNTTYPE_LOFS "lofs" /* Loop back file system */ +#define MNTTYPE_LO MNTTYPE_LOFS /* Deprecated name; use MNTTYPE_LOFS */ +#define MNTTYPE_HSFS "hsfs" /* High Sierra (9660) file system */ +#define MNTTYPE_SWAP "swap" /* Swap file system */ +#define MNTTYPE_TMPFS "tmpfs" /* Tmp volatile file system */ +#define MNTTYPE_AUTOFS "autofs" /* Automounter ``file'' system */ +#define MNTTYPE_MNTFS "mntfs" /* In-kernel mnttab */ +#define MNTTYPE_DEV "dev" /* /dev file system */ +#define MNTTYPE_CTFS "ctfs" /* Contract file system */ +#define MNTTYPE_OBJFS "objfs" /* Kernel object file system */ +#define MNTTYPE_SHAREFS "sharefs" /* Kernel sharetab file system */ + + +#define MNTOPT_RO "ro" /* Read only */ +#define MNTOPT_RW "rw" /* Read/write */ +#define MNTOPT_RQ "rq" /* Read/write with quotas */ +#define MNTOPT_QUOTA "quota" /* Check quotas */ +#define MNTOPT_NOQUOTA "noquota" /* Don't check quotas */ +#define MNTOPT_ONERROR "onerror" /* action to taken on error */ +#define MNTOPT_SOFT "soft" /* Soft mount */ +#define MNTOPT_SEMISOFT "semisoft" /* partial soft, uncommited interface */ +#define MNTOPT_HARD "hard" /* Hard mount */ +#define MNTOPT_SUID "suid" /* Both setuid and devices allowed */ +#define MNTOPT_NOSUID "nosuid" /* Neither setuid nor devices allowed */ +#define MNTOPT_DEVICES "devices" /* Device-special allowed */ +#define MNTOPT_NODEVICES "nodevices" /* Device-special disallowed */ +#define MNTOPT_SETUID "setuid" /* Set uid allowed */ +#define MNTOPT_NOSETUID "nosetuid" /* Set uid not allowed */ +#define MNTOPT_GRPID "grpid" /* SysV-compatible gid on create */ +#define MNTOPT_REMOUNT "remount" /* Change mount options */ +#define MNTOPT_NOSUB "nosub" /* Disallow mounts on subdirs */ +#define MNTOPT_MULTI "multi" /* Do multi-component lookup */ +#define MNTOPT_INTR "intr" /* Allow NFS ops to be interrupted */ +#define MNTOPT_NOINTR "nointr" /* Don't allow interrupted ops */ +#define MNTOPT_PORT "port" /* NFS server IP port number */ +#define MNTOPT_SECURE "secure" /* Secure (AUTH_DES) mounting */ +#define MNTOPT_RSIZE "rsize" /* Max NFS read size (bytes) */ +#define MNTOPT_WSIZE "wsize" /* Max NFS write size (bytes) */ +#define MNTOPT_TIMEO "timeo" /* NFS timeout (1/10 sec) */ +#define MNTOPT_RETRANS "retrans" /* Max retransmissions (soft mnts) */ +#define MNTOPT_ACTIMEO "actimeo" /* Attr cache timeout (sec) */ +#define MNTOPT_ACREGMIN "acregmin" /* Min attr cache timeout (files) */ +#define MNTOPT_ACREGMAX "acregmax" /* Max attr cache timeout (files) */ +#define MNTOPT_ACDIRMIN "acdirmin" /* Min attr cache timeout (dirs) */ +#define MNTOPT_ACDIRMAX "acdirmax" /* Max attr cache timeout (dirs) */ +#define MNTOPT_NOAC "noac" /* Don't cache attributes at all */ +#define MNTOPT_NOCTO "nocto" /* No close-to-open consistency */ +#define MNTOPT_BG "bg" /* Do mount retries in background */ +#define MNTOPT_FG "fg" /* Do mount retries in foreground */ +#define MNTOPT_RETRY "retry" /* Number of mount retries */ +#define MNTOPT_DEV "dev" /* Device id of mounted fs */ +#define MNTOPT_POSIX "posix" /* Get static pathconf for mount */ +#define MNTOPT_MAP "map" /* Automount map */ +#define MNTOPT_DIRECT "direct" /* Automount direct map mount */ +#define MNTOPT_INDIRECT "indirect" /* Automount indirect map mount */ +#define MNTOPT_LLOCK "llock" /* Local locking (no lock manager) */ +#define MNTOPT_IGNORE "ignore" /* Ignore this entry */ +#define MNTOPT_VERS "vers" /* protocol version number indicator */ +#define MNTOPT_PROTO "proto" /* protocol network_id indicator */ +#define MNTOPT_SEC "sec" /* Security flavor indicator */ +#define MNTOPT_SYNCDIR "syncdir" /* Synchronous local directory ops */ +#define MNTOPT_NOSETSEC "nosec" /* Do no allow setting sec attrs */ +#define MNTOPT_NOPRINT "noprint" /* Do not print messages */ +#define MNTOPT_LARGEFILES "largefiles" /* allow large files */ +#define MNTOPT_NOLARGEFILES "nolargefiles" /* don't allow large files */ +#define MNTOPT_FORCEDIRECTIO "forcedirectio" /* Force DirectIO on all files */ +#define MNTOPT_NOFORCEDIRECTIO "noforcedirectio" /* No Force DirectIO */ +#define MNTOPT_DISABLEDIRECTIO "disabledirectio" /* Disable DirectIO ioctls */ +#define MNTOPT_PUBLIC "public" /* Use NFS public file handlee */ +#define MNTOPT_LOGGING "logging" /* enable logging */ +#define MNTOPT_NOLOGGING "nologging" /* disable logging */ +#define MNTOPT_ATIME "atime" /* update atime for files */ +#define MNTOPT_NOATIME "noatime" /* do not update atime for files */ +#define MNTOPT_GLOBAL "global" /* Cluster-wide global mount */ +#define MNTOPT_NOGLOBAL "noglobal" /* Mount local to single node */ +#define MNTOPT_DFRATIME "dfratime" /* Deferred access time updates */ +#define MNTOPT_NODFRATIME "nodfratime" /* No Deferred access time updates */ +#define MNTOPT_NBMAND "nbmand" /* allow non-blocking mandatory locks */ +#define MNTOPT_NONBMAND "nonbmand" /* deny non-blocking mandatory locks */ +#define MNTOPT_XATTR "xattr" /* enable extended attributes */ +#define MNTOPT_NOXATTR "noxattr" /* disable extended attributes */ +#define MNTOPT_EXEC "exec" /* enable executables */ +#define MNTOPT_NOEXEC "noexec" /* disable executables */ +#define MNTOPT_RESTRICT "restrict" /* restricted autofs mount */ +#define MNTOPT_BROWSE "browse" /* browsable autofs mount */ +#define MNTOPT_NOBROWSE "nobrowse" /* non-browsable autofs mount */ +#define MNTOPT_OWNERS "owners" /* not ignore ownership */ +#define MNTOPT_NOOWNERS "noowners" /* ignore ownership */ + +#define MNT_RDONLY 0x00000001 /* read only filesystem */ +#define MNT_NOEXEC 0x00000004 /* can't exec from filesystem */ +#define MNT_NOSUID 0x00000008 /* don't honor setuid bits on fs */ +#define MNT_NODEV 0x00000010 /* don't interpret special files */ +#define MNT_UNION 0x00000020 /* union with underlying filesystem */ +#define MNT_ASYNC 0x00000040 /* file system written asynchronously */ + +#define MNT_NOATIME 0x10000000 /* disable update of file access time */ + +#define MNT_UPDATE 0x00010000 /* not a real mount, just an update */ +#define MNT_RELOAD 0x00040000 /* reload filesystem data */ +#define MNT_FORCE 0x00080000 /* force unmount or readonly change */ + +#define MNT_WAIT 1 /* synchronized I/O file integrity completion */ +#define MNT_NOWAIT 2 /* start all I/O, but do not wait for it */ + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_MNTENT_H */ diff --git a/lib/libspl/include/os/windows/poll.h b/lib/libspl/include/os/windows/poll.h new file mode 100644 index 000000000000..546c13113c6b --- /dev/null +++ b/lib/libspl/include/os/windows/poll.h @@ -0,0 +1,26 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_WINDOWS_POLL_H +#define _LIBSPL_WINDOWS_POLL_H + +#endif /* _LIBSPL_WINDOWS_POLL_H */ diff --git a/lib/libspl/include/os/windows/pwd.h b/lib/libspl/include/os/windows/pwd.h new file mode 100644 index 000000000000..93c2b3cf3cf4 --- /dev/null +++ b/lib/libspl/include/os/windows/pwd.h @@ -0,0 +1,45 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#ifndef _SPL_PWD_H +#define _SPL_PWD_H + +struct passwd { + char *pw_name; + char *pw_passwd; + uid_t pw_uid; + gid_t pw_gid; + char *pw_age; + char *pw_comment; + char *pw_gecos; + char *pw_dir; + char *pw_shell; +}; + + +extern struct passwd *getpwnam(const char *); /* MT-unsafe */ + + +#endif diff --git a/lib/libspl/include/os/windows/regex.h b/lib/libspl/include/os/windows/regex.h new file mode 100644 index 000000000000..1199a2d445c1 --- /dev/null +++ b/lib/libspl/include/os/windows/regex.h @@ -0,0 +1,145 @@ +/* $NetBSD: regex.h,v 1.16 2021/02/23 17:14:42 christos Exp $ */ + +/* + * Copyright (c) 1992, 1993 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer of the University of Toronto. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)regex.h 8.2 (Berkeley) 1/3/94 + */ + +/* + * Copyright (c) 1992 Henry Spencer. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer of the University of Toronto. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. All advertising materials mentioning features or use of this software + * must display the following acknowledgement: + * This product includes software developed by the University of + * California, Berkeley and its contributors. + * 4. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)regex.h 8.2 (Berkeley) 1/3/94 + */ + +#ifndef _REGEX_H_ +#define _REGEX_H_ + +// #include +#include +#include + +#define _DIAGASSERT ASSERT +#define __UNCONST(x) (x) +#define __arraycount(x) ARRAYSIZE(x) +/* types */ +typedef off_t regoff_t; + +typedef struct { + int re_magic; + size_t re_nsub; /* number of parenthesized subexpressions */ + const char *re_endp; /* end pointer for REG_PEND */ + struct re_guts *re_g; /* none of your business :-) */ +} regex_t; + +typedef struct { + regoff_t rm_so; /* start of match */ + regoff_t rm_eo; /* end of match */ +} regmatch_t; + +/* regcomp() flags */ +#define REG_BASIC 0000 +#define REG_EXTENDED 0001 +#define REG_ICASE 0002 +#define REG_NOSUB 0004 +#define REG_NEWLINE 0010 +#define REG_NOSPEC 0020 +#define REG_PEND 0040 +#define REG_DUMP 0200 +#define REG_GNU 0400 + +/* regerror() flags */ +#define REG_NOMATCH 1 +#define REG_BADPAT 2 +#define REG_ECOLLATE 3 +#define REG_ECTYPE 4 +#define REG_EESCAPE 5 +#define REG_ESUBREG 6 +#define REG_EBRACK 7 +#define REG_EPAREN 8 +#define REG_EBRACE 9 +#define REG_BADBR 10 +#define REG_ERANGE 11 +#define REG_ESPACE 12 +#define REG_BADRPT 13 +#define REG_EMPTY 14 +#define REG_ASSERT 15 +#define REG_INVARG 16 +#define REG_ILLSEQ 17 +#define REG_ATOI 255 /* convert name to number (!) */ +#define REG_ITOA 0400 /* convert number to name (!) */ + +/* regexec() flags */ +#define REG_NOTBOL 00001 +#define REG_NOTEOL 00002 +#define REG_STARTEND 00004 +#define REG_TRACE 00400 /* tracing of execution */ +#define REG_LARGE 01000 /* force large representation */ +#define REG_BACKR 02000 /* force use of backref code */ + +int regcomp(regex_t *__restrict, const char *__restrict, int); +size_t regerror(int, const regex_t *__restrict, char *__restrict, size_t); +int regexec(const regex_t *__restrict, + const char *__restrict, size_t, regmatch_t [], int); +void regfree(regex_t *); + +#endif /* !_REGEX_H_ */ diff --git a/lib/libspl/include/os/windows/rpc/types.h b/lib/libspl/include/os/windows/rpc/types.h new file mode 100644 index 000000000000..9a6fc83e0631 --- /dev/null +++ b/lib/libspl/include/os/windows/rpc/types.h @@ -0,0 +1,38 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef LIBSPL_RPC_TYPES_H +#define LIBSPL_RPC_TYPES_H + +#include + +#include + + +typedef int32_t rpc_inline_t; + + + +#endif /* LIBSPL_RPC_TYPES_H */ diff --git a/lib/libspl/include/os/windows/rpc/xdr.h b/lib/libspl/include/os/windows/rpc/xdr.h new file mode 100644 index 000000000000..9ad2770df953 --- /dev/null +++ b/lib/libspl/include/os/windows/rpc/xdr.h @@ -0,0 +1,300 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 1984 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _rpc_windows_xdr_h +#define _rpc_windows_xdr_h + +/* + * xdr.h, External Data Representation Serialization Routines. + */ + +#include +#include +#include +#include + + +/* + * XDR provides a conventional way for converting between C data + * types and an external bit-string representation. Library supplied + * routines provide for the conversion on built-in C data types. These + * routines and utility routines defined here are used to help implement + * a type encode/decode routine for each user-defined type. + * + * Each data type provides a single procedure which takes two arguments: + * + * bool_t + * xdrproc(xdrs, argresp) + * XDR *xdrs; + * *argresp; + * + * xdrs is an instance of a XDR handle, to which or from which the data + * type is to be converted. argresp is a pointer to the structure to be + * converted. The XDR handle contains an operation field which indicates + * which of the operations (ENCODE, DECODE * or FREE) is to be performed. + * + * XDR_DECODE may allocate space if the pointer argresp is null. This + * data can be freed with the XDR_FREE operation. + * + * We write only one procedure per data type to make it easy + * to keep the encode and decode procedures for a data type consistent. + * In many cases the same code performs all operations on a user defined type, + * because all the hard work is done in the component type routines. + * decode as a series of calls on the nested data types. + */ + +/* + * Xdr operations. XDR_ENCODE causes the type to be encoded into the + * stream. XDR_DECODE causes the type to be extracted from the stream. + * XDR_FREE can be used to release the space allocated by an XDR_DECODE + * request. + */ +enum xdr_op { + XDR_ENCODE = 0, + XDR_DECODE = 1, + XDR_FREE = 2 +}; + +#define XDR_GET_BYTES_AVAIL 1 + +/* + * This is the number of bytes per unit of external data. + */ +#define BYTES_PER_XDR_UNIT (4) +#define RNDUP(x) ((((x) + BYTES_PER_XDR_UNIT - 1) / BYTES_PER_XDR_UNIT) \ + * BYTES_PER_XDR_UNIT) + +/* + * A xdrproc_t exists for each data type which is to be encoded or decoded. + * + * The second argument to the xdrproc_t is a pointer to an opaque pointer. + * The opaque pointer generally points to a structure of the data type + * to be decoded. If this pointer is 0, then the type routines should + * allocate dynamic storage of the appropriate size and return it. + * bool_t (*xdrproc_t)(XDR *, caddr_t *); + */ +typedef bool_t (*xdrproc_t)(); + +typedef struct xdr_bytesrec { + bool_t xc_is_last_record; + uint32_t xc_num_avail; +} xdr_bytesrec_t; + +// typedef struct xdr_bytesrec xdr_bytesrec; + +/* + * The XDR handle. + * Contains operation which is being applied to the stream, + * an operations vector for the paticular implementation (e.g. see xdr_mem.c), + * and two private fields for the use of the particular impelementation. + */ +typedef struct { + enum xdr_op x_op; /* operation; fast additional param */ + struct xdr_ops { + bool_t (*x_getlong)(); /* get a long from underlying stream */ + bool_t (*x_putlong)(); /* put a long to " */ + bool_t (*x_getbytes)(); /* get some bytes from " */ + bool_t (*x_putbytes)(); /* put some bytes to " */ + uint_t (*x_getpostn)(); /* returns bytes off from beginning */ + bool_t (*x_setpostn)(); /* lets you reposition the stream */ + long *(*x_inline)(); /* buf quick ptr to buffered data */ + void (*x_destroy)(); /* free privates of this xdr_stream */ + bool_t(*x_control)(struct XDR *, int, void *); + bool_t(*x_getint32)(struct XDR *, int32_t *); + bool_t(*x_putint32)(struct XDR *, int32_t *); + } *x_ops; + caddr_t x_public; /* users' data */ + caddr_t x_private; /* pointer to private data */ + caddr_t x_base; /* private used for position info */ + int x_handy; /* extra private word */ +} XDR; + +#include_next + +/* + * Operations defined on a XDR handle + * + * XDR *xdrs; + * long *longp; + * caddr_t addr; + * u_int len; + * u_int pos; + */ +#define XDR_GETLONG(xdrs, longp) \ + (*(xdrs)->x_ops->x_getlong)(xdrs, longp) +#define xdr_getlong(xdrs, longp) \ + (*(xdrs)->x_ops->x_getlong)(xdrs, longp) + +#define XDR_PUTLONG(xdrs, longp) \ + (*(xdrs)->x_ops->x_putlong)(xdrs, longp) +#define xdr_putlong(xdrs, longp) \ + (*(xdrs)->x_ops->x_putlong)(xdrs, longp) + +#define XDR_GETBYTES(xdrs, addr, len) \ + (*(xdrs)->x_ops->x_getbytes)(xdrs, addr, len) +#define xdr_getbytes(xdrs, addr, len) \ + (*(xdrs)->x_ops->x_getbytes)(xdrs, addr, len) + +#define XDR_PUTBYTES(xdrs, addr, len) \ + (*(xdrs)->x_ops->x_putbytes)(xdrs, addr, len) +#define xdr_putbytes(xdrs, addr, len) \ + (*(xdrs)->x_ops->x_putbytes)(xdrs, addr, len) + +#define XDR_GETPOS(xdrs) \ + (*(xdrs)->x_ops->x_getpostn)(xdrs) +#define xdr_getpos(xdrs) \ + (*(xdrs)->x_ops->x_getpostn)(xdrs) + +#define XDR_SETPOS(xdrs, pos) \ + (*(xdrs)->x_ops->x_setpostn)(xdrs, pos) +#define xdr_setpos(xdrs, pos) \ + (*(xdrs)->x_ops->x_setpostn)(xdrs, pos) + +#define XDR_INLINE(xdrs, len) \ + (*(xdrs)->x_ops->x_inline)(xdrs, len) +#define xdr_inline(xdrs, len) \ + (*(xdrs)->x_ops->x_inline)(xdrs, len) + +#define XDR_DESTROY(xdrs) \ + (*(xdrs)->x_ops->x_destroy)(xdrs) +#define xdr_destroy(xdrs) XDR_DESTROY(xdrs) + +#define XDR_GETINT32(xdrs, int32p) \ + (*(xdrs)->x_ops->x_getint32)(xdrs, int32p) +#define xdr_getint32(xdrs, int32p) \ + (*(xdrs)->x_ops->x_getint32)(xdrs, int32p) + +#define XDR_PUTINT32(xdrs, int32p) \ + (*(xdrs)->x_ops->x_putint32)(xdrs, int32p) +#define xdr_putint32(xdrs, int32p) \ + (*(xdrs)->x_ops->x_putint32)(xdrs, int32p) + +/* + * Support struct for discriminated unions. + * You create an array of xdrdiscrim structures, terminated with + * a entry with a null procedure pointer. The xdr_union routine gets + * the discriminant value and then searches the array of structures + * for a matching value. If a match is found the associated xdr routine + * is called to handle that part of the union. If there is + * no match, then a default routine may be called. + * If there is no match and no default routine it is an error. + */ +#define NULL_xdrproc_t ((xdrproc_t)0) +struct xdr_discrim { + int value; + xdrproc_t proc; +}; + +/* + * In-line routines for fast encode/decode of primitve data types. + * Caveat emptor: these use single memory cycles to get the + * data from the underlying buffer, and will fail to operate + * properly if the data is not aligned. The standard way to use these + * is to say: + * if ((buf = XDR_INLINE(xdrs, count)) == NULL) + * return (FALSE); + * <<< macro calls >>> + * where ``count'' is the number of bytes of data occupied + * by the primitive data types. + * + * N.B. and frozen for all time: each data type here uses 4 bytes + * of external representation. + */ +#define IXDR_GET_LONG(buf) ((long)ntohl((ulong_t)*(buf)++)) +#define IXDR_PUT_LONG(buf, v) (*(buf)++ = (long)htonl((ulong_t)v)) + +#define IXDR_GET_BOOL(buf) ((bool_t)IXDR_GET_LONG(buf)) +#define IXDR_GET_ENUM(buf, t) ((t)IXDR_GET_LONG(buf)) +#define IXDR_GET_U_LONG(buf) ((ulong_t)IXDR_GET_LONG(buf)) +#define IXDR_GET_SHORT(buf) ((short)IXDR_GET_LONG(buf)) +#define IXDR_GET_U_SHORT(buf) ((ushort_t)IXDR_GET_LONG(buf)) + +#define IXDR_PUT_BOOL(buf, v) IXDR_PUT_LONG((buf), ((long)(v))) +#define IXDR_PUT_ENUM(buf, v) IXDR_PUT_LONG((buf), ((long)(v))) +#define IXDR_PUT_U_LONG(buf, v) IXDR_PUT_LONG((buf), ((long)(v))) +#define IXDR_PUT_SHORT(buf, v) IXDR_PUT_LONG((buf), ((long)(v))) +#define IXDR_PUT_U_SHORT(buf, v) IXDR_PUT_LONG((buf), ((long)(v))) + +/* + * These are the "generic" xdr routines. + */ +extern bool_t xdr_void(); +extern bool_t xdr_int(); +extern bool_t xdr_u_int(); +extern bool_t xdr_long(); +extern bool_t xdr_u_long(); +extern bool_t xdr_short(); +extern bool_t xdr_u_short(); +extern bool_t xdr_bool(); +extern bool_t xdr_enum(); +extern bool_t xdr_array(); +extern bool_t xdr_bytes(); +extern bool_t xdr_opaque(); +extern bool_t xdr_string(); +extern bool_t xdr_union(); +extern void xdr_free(); +extern bool_t xdr_char(); +extern bool_t xdr_u_char(); +extern bool_t xdr_vector(); +extern bool_t xdr_float(); +extern bool_t xdr_double(); +extern bool_t xdr_reference(); +extern bool_t xdr_pointer(); +extern bool_t xdr_wrapstring(); +extern bool_t xdr_longlong_t(XDR *xdrs, longlong_t *hp); +extern bool_t xdr_u_longlong_t(XDR *xdrs, u_longlong_t *hp); +extern bool_t xdr_control(XDR *xdrs, int request, void *info); + +/* + * Common opaque bytes objects used by many rpc protocols; + * declared here due to commonality. + */ +#define MAX_NETOBJ_SZ 1024 +struct netobj { + uint_t n_len; + char *n_bytes; +}; +typedef struct netobj netobj; +extern bool_t xdr_netobj(); + +/* + * These are the public routines for the various implementations of + * xdr streams. + */ +extern void xdrmem_create(); /* XDR using memory buffers */ +extern void xdrstdio_create(); /* XDR using stdio library */ +extern void xdrrec_create(); /* XDR pseudo records for tcp */ +extern bool_t xdrrec_endofrecord(); /* make end of xdr record */ +extern int xdrrec_readbytes(); /* like a read on a pipe */ +extern bool_t xdrrec_skiprecord(); /* move to beginning of next record */ +extern bool_t xdrrec_eof(); /* true if no more input */ + +#define XDR_PEEK 2 +#define XDR_SKIPBYTES 3 +#define XDR_RDMAGET 4 +#define XDR_RDMASET 5 + +#endif /* !_rpc_xdr_h */ diff --git a/lib/libspl/include/os/windows/sched.h b/lib/libspl/include/os/windows/sched.h new file mode 100644 index 000000000000..63d29b89b6f5 --- /dev/null +++ b/lib/libspl/include/os/windows/sched.h @@ -0,0 +1,40 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_SCHED_H +#define _SPL_SCHED_H + + +struct sched_param { + int32_t sched_priority; + int32_t sched_curpriority; + union { + int32_t reserved[8]; + struct { + int32_t __ss_low_priority; + int32_t __ss_max_repl; + struct timespec __ss_repl_period; + struct timespec __ss_init_budget; + } __ss; + } __ss_un; +}; + +#endif diff --git a/lib/libspl/include/os/windows/search.h b/lib/libspl/include/os/windows/search.h new file mode 100644 index 000000000000..1813c09dcd5b --- /dev/null +++ b/lib/libspl/include/os/windows/search.h @@ -0,0 +1,55 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_SEARCH_H +#define _SPL_SEARCH_H +#include + +typedef enum { + FIND, + ENTER +} ACTION; + +typedef struct entry { + char *key; + void *data; +} ENTRY; + + +/* Implement me for zstream decompress */ +inline static int +hcreate(size_t n) +{ + return (0); +} + +inline static void +hdestroy(void) +{ +} + +static inline ENTRY * +hsearch(ENTRY entry, ACTION action) +{ + return (NULL); +} + +#endif diff --git a/lib/libspl/include/os/windows/signal.h b/lib/libspl/include/os/windows/signal.h new file mode 100644 index 000000000000..652fe8d5185a --- /dev/null +++ b/lib/libspl/include/os/windows/signal.h @@ -0,0 +1,93 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SPL_SIGNAL_H +#define _SPL_SIGNAL_H + +// #include +// #include +// #include +// #include +#include_next + +#define FORREAL 0 /* Usual side-effects */ +#define JUSTLOOKING 1 /* Don't stop the process */ + + +#define SIGPIPE 0 + +struct proc; + +// extern int +// thread_issignal(struct proc *, thread_t, sigset_t); +#define SA_SIGINFO 0x00000008 + +typedef struct __siginfo { + /* Windows version goes here */ + void *si_addr; +} siginfo_t; + +typedef struct { + unsigned long sig[1]; +} sigset_t; + +typedef void (*sighandler_t) (int); + +struct sigaction { + union + { + /* Used if SA_SIGINFO is not set. */ + sighandler_t sa_handler; + /* Used if SA_SIGINFO is set. */ + void (*sa_sigaction) (int, siginfo_t *, void *); + }; + sigset_t sa_mask; + /* Special flags. */ + int sa_flags; +}; + +/* + * The "why" argument indicates the allowable side-effects of the call: + * + * FORREAL: Extract the next pending signal from p_sig into p_cursig; + * stop the process if a stop has been requested or if a traced signal + * is pending. + * + * JUSTLOOKING: Don't stop the process, just indicate whether or not + * a signal might be pending (FORREAL is needed to tell for sure). + */ +#define threadmask \ + (sigmask(SIGILL)|sigmask(SIGTRAP)| \ + sigmask(SIGIOT)|sigmask(SIGEMT)| \ + sigmask(SIGFPE)|sigmask(SIGBUS)| \ + sigmask(SIGSEGV)|sigmask(SIGSYS)| \ + sigmask(SIGPIPE)|sigmask(SIGKILL)| \ + sigmask(SIGTERM)|sigmask(SIGINT)) + +static inline int +issig(int why) +{ + return (0); +} + +#define signal_pending(p) issig(0) + +#endif /* SPL_SIGNAL_H */ diff --git a/lib/libspl/include/os/windows/stdio.h b/lib/libspl/include/os/windows/stdio.h new file mode 100644 index 000000000000..030d00be328c --- /dev/null +++ b/lib/libspl/include/os/windows/stdio.h @@ -0,0 +1,34 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_WINDOWS_STDIO_H +#define _LIBSPL_WINDOWS_STDIO_H + +#include_next + +/* + * This header is mostly here to include sysmacros, as so many places + * appear to get MAX() and ISP() from somewhere else on other platforms. + */ +#include + +#endif diff --git a/lib/libspl/include/os/windows/string.h b/lib/libspl/include/os/windows/string.h new file mode 100644 index 000000000000..6081cf15d61f --- /dev/null +++ b/lib/libspl/include/os/windows/string.h @@ -0,0 +1,76 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_WINDOWS_STRING_H +#define _LIBSPL_WINDOWS_STRING_H + +#include_next + +static inline char * +strtok_r(char *s, const char *delim, char **last) +{ + char *spanp, *tok; + int c, sc; + + if (s == NULL && (s = *last) == NULL) + return (NULL); + + /* + * Skip (span) leading delimiters (s += strspn(s, delim), sort of). + */ +cont: + c = *s++; + for (spanp = (char *)delim; (sc = *spanp++) != 0; ) { + if (c == sc) + goto cont; + } + + if (c == 0) { /* no non-delimiter characters */ + *last = NULL; + return (NULL); + } + tok = s - 1; + + /* + * Scan token (scan for delimiters: s += strcspn(s, delim), sort of). + * Note that delim must have one NUL; we stop if we see that, too. + */ + for (;;) { + c = *s++; + spanp = (char *)delim; + do { + if ((sc = *spanp++) == c) { + if (c == 0) + s = NULL; + else + s[-1] = '\0'; + *last = s; + return (tok); + } + } while (sc != 0); + } + /* NOTREACHED */ +} + +char *strsep(char **stringp, const char *delim); + +#endif diff --git a/lib/libspl/include/os/windows/sys/byteorder.h b/lib/libspl/include/os/windows/sys/byteorder.h new file mode 100644 index 000000000000..50b5af3b9c95 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/byteorder.h @@ -0,0 +1,218 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ + +/* + * University Copyright- Copyright (c) 1982, 1986, 1988 + * The Regents of the University of California + * All Rights Reserved + * + * University Acknowledgment- Portions of this document are derived from + * software developed by the University of California, Berkeley, and its + * contributors. + */ + +#ifndef _SYS_BYTEORDER_H +#define _SYS_BYTEORDER_H + +#include +#include + +#if defined(__GNUC__) && defined(_ASM_INLINES) && \ + (defined(__i386) || defined(__amd64)) +#include +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +/* + * macros for conversion between host and (internet) network byte order + */ + +#if defined(_BIG_ENDIAN) && !defined(ntohl) && !defined(__lint) +/* big-endian */ +#define ntohl(x) (x) +#define ntohs(x) (x) +#define htonl(x) (x) +#define htons(x) (x) + +#elif !defined(ntohl) /* little-endian */ + +#ifndef _IN_PORT_T +#define _IN_PORT_T +typedef uint16_t in_port_t; +#endif + +#ifndef _IN_ADDR_T +#define _IN_ADDR_T +typedef uint32_t in_addr_t; +#endif + +#endif + + + +#if !defined(_XPG4_2) || defined(__EXTENSIONS__) + +/* + * Macros to reverse byte order + */ +#define BSWAP_8(x) ((x) & 0xff) +#define BSWAP_16(x) ((BSWAP_8(x) << 8) | BSWAP_8((x) >> 8)) +#define BSWAP_32(x) ((BSWAP_16(x) << 16) | BSWAP_16((x) >> 16)) +#define BSWAP_64(x) ((BSWAP_32(x) << 32) | BSWAP_32((x) >> 32)) + +#define BMASK_8(x) ((x) & 0xff) +#define BMASK_16(x) ((x) & 0xffff) +#define BMASK_32(x) ((x) & 0xffffffff) +#define BMASK_64(x) (x) + +/* + * Macros to convert from a specific byte order to/from native byte order + */ +#ifdef _BIG_ENDIAN +#define BE_8(x) BMASK_8(x) +#define BE_16(x) BMASK_16(x) +#define BE_32(x) BMASK_32(x) +#define BE_64(x) BMASK_64(x) +#define LE_8(x) BSWAP_8(x) +#define LE_16(x) BSWAP_16(x) +#define LE_32(x) BSWAP_32(x) +#define LE_64(x) BSWAP_64(x) +#else +#define LE_8(x) BMASK_8(x) +#define LE_16(x) BMASK_16(x) +#define LE_32(x) BMASK_32(x) +#define LE_64(x) BMASK_64(x) +#define BE_8(x) BSWAP_8(x) +#define BE_16(x) BSWAP_16(x) +#define BE_32(x) BSWAP_32(x) +#define BE_64(x) BSWAP_64(x) +#endif + +#ifdef _BIG_ENDIAN +static __inline__ uint64_t +htonll(uint64_t n) +{ + return (n); +} + +static __inline__ uint64_t +ntohll(uint64_t n) +{ + return (n); +} + +#else +#if 0 +static __inline__ uint64_t +htonll(uint64_t n) +{ + return (_OSSwapInt64(n)); +} + +static __inline__ uint64_t +ntohll(uint64_t n) +{ + return (_OSSwapInt64(n)); +} +#endif + +#endif + +/* + * Macros to read unaligned values from a specific byte order to + * native byte order + */ + +#define BE_IN8(xa) \ + *((uint8_t *)(xa)) + +#define BE_IN16(xa) \ + (((uint16_t)BE_IN8(xa) << 8) | BE_IN8((uint8_t *)(xa)+1)) + +#define BE_IN32(xa) \ + (((uint32_t)BE_IN16(xa) << 16) | BE_IN16((uint8_t *)(xa)+2)) + +#define BE_IN64(xa) \ + (((uint64_t)BE_IN32(xa) << 32) | BE_IN32((uint8_t *)(xa)+4)) + +#define LE_IN8(xa) \ + *((uint8_t *)(xa)) + +#define LE_IN16(xa) \ + (((uint16_t)LE_IN8((uint8_t *)(xa) + 1) << 8) | LE_IN8(xa)) + +#define LE_IN32(xa) \ + (((uint32_t)LE_IN16((uint8_t *)(xa) + 2) << 16) | LE_IN16(xa)) + +#define LE_IN64(xa) \ + (((uint64_t)LE_IN32((uint8_t *)(xa) + 4) << 32) | LE_IN32(xa)) + +/* + * Macros to write unaligned values from native byte order to a specific byte + * order. + */ + +#define BE_OUT8(xa, yv) *((uint8_t *)(xa)) = (uint8_t)(yv); + +#define BE_OUT16(xa, yv) \ + BE_OUT8((uint8_t *)(xa) + 1, yv); \ + BE_OUT8((uint8_t *)(xa), (yv) >> 8); + +#define BE_OUT32(xa, yv) \ + BE_OUT16((uint8_t *)(xa) + 2, yv); \ + BE_OUT16((uint8_t *)(xa), (yv) >> 16); + +#define BE_OUT64(xa, yv) \ + BE_OUT32((uint8_t *)(xa) + 4, yv); \ + BE_OUT32((uint8_t *)(xa), (yv) >> 32); + +#define LE_OUT8(xa, yv) *((uint8_t *)(xa)) = (uint8_t)(yv); + +#define LE_OUT16(xa, yv) \ + LE_OUT8((uint8_t *)(xa), yv); \ + LE_OUT8((uint8_t *)(xa) + 1, (yv) >> 8); + +#define LE_OUT32(xa, yv) \ + LE_OUT16((uint8_t *)(xa), yv); \ + LE_OUT16((uint8_t *)(xa) + 2, (yv) >> 16); + +#define LE_OUT64(xa, yv) \ + LE_OUT32((uint8_t *)(xa), yv); \ + LE_OUT32((uint8_t *)(xa) + 4, (yv) >> 32); + +#endif /* !defined(_XPG4_2) || defined(__EXTENSIONS__) */ + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_BYTEORDER_H */ diff --git a/lib/libspl/include/os/windows/sys/errno.h b/lib/libspl/include/os/windows/sys/errno.h new file mode 100644 index 000000000000..2ca882dd4ec2 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/errno.h @@ -0,0 +1,30 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_SYS_ERRNO_H +#define _LIBSPL_SYS_ERRNO_H + +#include +#define ECKSUM EBADE +#define ERESTART 85 /* Interrupted system call should be restarted */ + +#endif diff --git a/lib/libspl/include/os/windows/sys/file.h b/lib/libspl/include/os/windows/sys/file.h new file mode 100644 index 000000000000..25bb4366035e --- /dev/null +++ b/lib/libspl/include/os/windows/sys/file.h @@ -0,0 +1,48 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _LIBSPL_SYS_FILE_H +#define _LIBSPL_SYS_FILE_H + +#define FREAD 1 +#define FWRITE 2 +#define FAPPEND 8 + +#define FCREAT O_CREAT +#define FTRUNC O_TRUNC +#define FOFFMAX 0 +#define FSYNC O_SYNC +#define FDSYNC O_DSYNC +#define FRSYNC O_RSYNC +#define FEXCL O_EXCL + +#define O_DIRECT 0 + +#define FNODSYNC 0x10000 /* fsync pseudo flag */ +#define FNOFOLLOW 0x20000 /* don't follow symlinks */ +#define FIGNORECASE 0x80000 /* request case-insensitive lookups */ + +#endif diff --git a/lib/libspl/include/os/windows/sys/ia32/asm_linkage.h b/lib/libspl/include/os/windows/sys/ia32/asm_linkage.h new file mode 100644 index 000000000000..ba9b781ebf5a --- /dev/null +++ b/lib/libspl/include/os/windows/sys/ia32/asm_linkage.h @@ -0,0 +1,183 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2008 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _IA32_SYS_ASM_LINKAGE_H +#define _IA32_SYS_ASM_LINKAGE_H + +#if defined(__linux__) && defined(CONFIG_SLS) +#define RET ret; int3 +#else +#define RET ret +#endif + +/* Tell compiler to call assembler like Unix */ +#undef ASMABI +#define ASMABI __attribute__((sysv_abi)) + +#define ENDBR + +#define SECTION_TEXT .text +#define SECTION_STATIC .data + +#ifdef __cplusplus +extern "C" { +#endif + +#ifdef _ASM /* The remainder of this file is only for assembly files */ + +/* + * make annoying differences in assembler syntax go away + */ + +/* + * D16 and A16 are used to insert instructions prefixes; the + * macros help the assembler code be slightly more portable. + */ +#if !defined(__GNUC_AS__) +/* + * /usr/ccs/bin/as prefixes are parsed as separate instructions + */ +#define D16 data16; +#define A16 addr16; + +/* + * (There are some weird constructs in constant expressions) + */ +#define _CONST(const) [const] +#define _BITNOT(const) -1!_CONST(const) +#define _MUL(a, b) _CONST(a \* b) + +#else +/* + * Why not use the 'data16' and 'addr16' prefixes .. well, the + * assembler doesn't quite believe in real mode, and thus argues with + * us about what we're trying to do. + */ +#define D16 .byte 0x66; +#define A16 .byte 0x67; + +#define _CONST(const) (const) +#define _BITNOT(const) ~_CONST(const) +#define _MUL(a, b) _CONST(a * b) + +#endif + +/* + * C pointers are different sizes between i386 and amd64. + * These constants can be used to compute offsets into pointer arrays. + */ +#if defined(__amd64) +#define CLONGSHIFT 3 +#define CLONGSIZE 8 +#define CLONGMASK 7 +#elif defined(__i386) +#define CLONGSHIFT 2 +#define CLONGSIZE 4 +#define CLONGMASK 3 +#endif + +/* + * Since we know we're either ILP32 or LP64 .. + */ +#define CPTRSHIFT CLONGSHIFT +#define CPTRSIZE CLONGSIZE +#define CPTRMASK CLONGMASK + +#if CPTRSIZE != (1 << CPTRSHIFT) || CLONGSIZE != (1 << CLONGSHIFT) +#error "inconsistent shift constants" +#endif + +#if CPTRMASK != (CPTRSIZE - 1) || CLONGMASK != (CLONGSIZE - 1) +#error "inconsistent mask constants" +#endif + +#define ASM_ENTRY_ALIGN 16 + +/* + * SSE register alignment and save areas + */ + +#define XMM_SIZE 16 +#define XMM_ALIGN 16 + +/* + * ENTRY provides the standard procedure entry code and an easy way to + * insert the calls to mcount for profiling. ENTRY_NP is identical, but + * never calls mcount. + */ +#define ENTRY(x) \ + .text; \ + .align ASM_ENTRY_ALIGN; \ + .globl x; \ +x: MCOUNT(x) + +#define ENTRY_NP(x) \ + .text; \ + .align ASM_ENTRY_ALIGN; \ + .globl x; \ +x: + +#define ENTRY_ALIGN(x, a) \ + .text; \ + .balign a; \ + .globl x; \ +x: + +#define FUNCTION(x) \ +x: + +/* + * ENTRY2 is identical to ENTRY but provides two labels for the entry point. + */ +#define ENTRY2(x, y) \ + .text; \ + .align ASM_ENTRY_ALIGN; \ + .globl x, y; \ +x:; \ +y: MCOUNT(x) + +#define ENTRY_NP2(x, y) \ + .text; \ + .align ASM_ENTRY_ALIGN; \ + .globl x, y; \ +x:; \ +y: + + +/* + * SET_SIZE trails a function and set the size for the ELF symbol table. + */ +#define SET_SIZE(x) + +#define SET_OBJ(x) + +#endif /* _ASM */ + +#ifdef __cplusplus +} +#endif + +#endif /* _IA32_SYS_ASM_LINKAGE_H */ diff --git a/lib/libspl/include/os/windows/sys/ioctl.h b/lib/libspl/include/os/windows/sys/ioctl.h new file mode 100644 index 000000000000..340895f7602c --- /dev/null +++ b/lib/libspl/include/os/windows/sys/ioctl.h @@ -0,0 +1,28 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_SYS_IOCTL_H +#define _LIBSPL_SYS_IOCTL_H + +#include + +#endif /* _LIBSPL_SYS_STAT_H */ diff --git a/lib/libspl/include/os/windows/sys/kstat.h b/lib/libspl/include/os/windows/sys/kstat.h new file mode 100644 index 000000000000..91f5a9f8383e --- /dev/null +++ b/lib/libspl/include/os/windows/sys/kstat.h @@ -0,0 +1,852 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SYS_KSTAT_H +#define _SYS_KSTAT_H + + + +/* + * Definition of general kernel statistics structures and /dev/kstat ioctls + */ + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +typedef int kid_t; /* unique kstat id */ + +/* + * Kernel statistics driver (/dev/kstat) ioctls + */ + +#define SPLIOCTL_TYPE 40000 +#define KSTAT_IOC_CHAIN_ID CTL_CODE(SPLIOCTL_TYPE, 0x7FD, \ + METHOD_NEITHER, FILE_ANY_ACCESS) +#define KSTAT_IOC_READ CTL_CODE(SPLIOCTL_TYPE, 0x7FE, \ + METHOD_NEITHER, FILE_ANY_ACCESS) +#define KSTAT_IOC_WRITE CTL_CODE(SPLIOCTL_TYPE, 0x7FF, \ + METHOD_NEITHER, FILE_ANY_ACCESS) + +/* + * /dev/kstat ioctl usage (kd denotes /dev/kstat descriptor): + * + * kcid = ioctl(kd, KSTAT_IOC_CHAIN_ID, NULL); + * kcid = ioctl(kd, KSTAT_IOC_READ, kstat_t *); + * kcid = ioctl(kd, KSTAT_IOC_WRITE, kstat_t *); + */ + +#define KSTAT_STRLEN 255 /* 254 chars + NULL; must be 16 * n - 1 */ + +/* + * The generic kstat header + */ + +typedef struct kstat_raw_ops { + int (*headers)(char *buf, size_t size); + int (*seq_headers)(struct seq_file *); + int (*data)(char *buf, size_t size, void *data); + void *(*addr)(struct kstat_s *ksp, loff_t index); +} kstat_raw_ops_t; + +/* + * This is a bit unfortunate, we store a mutex in kernel which userland needs + * to match in size + */ +struct kernel_mutex +{ + unsigned char opaque[0x28]; +}; + +#pragma pack(4) +typedef struct kstat { + /* + * Fields relevant to both kernel and user + */ + hrtime_t ks_crtime; /* creation time (from gethrtime()) */ + struct kstat *ks_next; /* kstat chain linkage */ + kid_t ks_kid; /* unique kstat ID */ + char ks_module[KSTAT_STRLEN]; /* provider module name */ + uchar_t ks_resv; /* reserved, currently just padding */ + int ks_instance; /* provider module's instance */ + char ks_name[KSTAT_STRLEN]; /* kstat name */ + uchar_t ks_type; /* kstat data type */ + char ks_class[KSTAT_STRLEN]; /* kstat class */ + uchar_t ks_flags; /* kstat flags */ + void *ks_data; /* kstat type-specific data */ + uint_t ks_ndata; /* # of type-specific data records */ + size_t ks_data_size; /* total size of kstat data section */ + hrtime_t ks_snaptime; /* time of last data snapshot */ + int ks_returnvalue; + int ks_errnovalue; + + /* + * Fields relevant to kernel only + */ + struct kernel_mutex ks_private_lock; /* kstat private data lock */ + struct kernel_mutex *ks_lock; /* kstat data lock */ + int(*ks_update)(struct kstat *, int); /* dynamic update */ + int(*ks_snapshot)(struct kstat *, void *, int); + void *ks_private; /* arbitrary provider-private data */ + void *ks_private1; /* private data */ + kstat_raw_ops_t ks_raw_ops; /* ops table for raw type */ + char *ks_raw_buf; /* buf used for raw ops */ + size_t ks_raw_bufsize; /* size of raw ops buffer */ +} kstat_t; +#pragma pack() + +#ifdef _SYSCALL32 + +typedef int32_t kid32_t; + +typedef struct kstat32 { + /* + * Fields relevant to both kernel and user + */ + hrtime_t ks_crtime; + caddr32_t ks_next; /* struct kstat pointer */ + kid32_t ks_kid; + char ks_module[KSTAT_STRLEN]; + uint8_t ks_resv; + int32_t ks_instance; + char ks_name[KSTAT_STRLEN]; + uint8_t ks_type; + char ks_class[KSTAT_STRLEN]; + uint8_t ks_flags; + caddr32_t ks_data; /* type-specific data */ + uint32_t ks_ndata; + size32_t ks_data_size; + hrtime_t ks_snaptime; + /* + * Fields relevant to kernel only (only needed here for padding) + */ + int32_t _ks_update; + caddr32_t _ks_private; + int32_t _ks_snapshot; + caddr32_t _ks_lock; +} kstat32_t; + +#endif /* _SYSCALL32 */ + +/* + * kstat structure and locking strategy + * + * Each kstat consists of a header section (a kstat_t) and a data section. + * The system maintains a set of kstats, protected by kstat_chain_lock. + * kstat_chain_lock protects all additions to/deletions from this set, + * as well as all changes to kstat headers. kstat data sections are + * *optionally* protected by the per-kstat ks_lock. If ks_lock is non-NULL, + * kstat clients (e.g. /dev/kstat) will acquire this lock for all of their + * operations on that kstat. It is up to the kstat provider to decide whether + * guaranteeing consistent data to kstat clients is sufficiently important + * to justify the locking cost. Note, however, that most statistic updates + * already occur under one of the provider's mutexes, so if the provider sets + * ks_lock to point to that mutex, then kstat data locking is free. + * + * NOTE: variable-size kstats MUST employ kstat data locking, to prevent + * data-size races with kstat clients. + * + * NOTE: ks_lock is really of type (kmutex_t *); it is declared as (void *) + * in the kstat header so that users don't have to be exposed to all of the + * kernel's lock-related data structures. + */ + +#if defined(_KERNEL) + +#define KSTAT_ENTER(k) \ + { kmutex_t *lp = (k)->ks_lock; if (lp) mutex_enter(lp); } + +#define KSTAT_EXIT(k) \ + { kmutex_t *lp = (k)->ks_lock; if (lp) mutex_exit(lp); } + +#define KSTAT_UPDATE(k, rw) (*(k)->ks_update)((k), (rw)) + +#define KSTAT_SNAPSHOT(k, buf, rw) (*(k)->ks_snapshot)((k), (buf), (rw)) + +#endif /* defined(_KERNEL) */ + +/* + * kstat time + * + * All times associated with kstats (e.g. creation time, snapshot time, + * kstat_timer_t and kstat_io_t timestamps, etc.) are 64-bit nanosecond values, + * as returned by gethrtime(). The accuracy of these timestamps is machine + * dependent, but the precision (units) is the same across all platforms. + */ + +/* + * kstat identity (KID) + * + * Each kstat is assigned a unique KID (kstat ID) when it is added to the + * global kstat chain. The KID is used as a cookie by /dev/kstat to + * request information about the corresponding kstat. There is also + * an identity associated with the entire kstat chain, kstat_chain_id, + * which is bumped each time a kstat is added or deleted. /dev/kstat uses + * the chain ID to detect changes in the kstat chain (e.g., a new disk + * coming online) between ioctl()s. + */ + +/* + * kstat module, kstat instance + * + * ks_module and ks_instance contain the name and instance of the module + * that created the kstat. In cases where there can only be one instance, + * ks_instance is 0. The kernel proper (/kernel/unix) uses "unix" as its + * module name. + */ + +/* + * kstat name + * + * ks_name gives a meaningful name to a kstat. The full kstat namespace + * is module.instance.name, so the name only need be unique within a + * module. kstat_create() will fail if you try to create a kstat with + * an already-used (ks_module, ks_instance, ks_name) triplet. Spaces are + * allowed in kstat names, but strongly discouraged, since they hinder + * awk-style processing at user level. + */ + +/* + * kstat type + * + * The kstat mechanism provides several flavors of kstat data, defined + * below. The "raw" kstat type is just treated as an array of bytes; you + * can use this to export any kind of data you want. + * + * Some kstat types allow multiple data structures per kstat, e.g. + * KSTAT_TYPE_NAMED; others do not. This is part of the spec for each + * kstat data type. + * + * User-level tools should *not* rely on the #define KSTAT_NUM_TYPES. To + * get this information, read out the standard system kstat "kstat_types". + */ + +#define KSTAT_TYPE_RAW 0 /* can be anything */ + /* ks_ndata >= 1 */ +#define KSTAT_TYPE_NAMED 1 /* name/value pair */ + /* ks_ndata >= 1 */ +#define KSTAT_TYPE_INTR 2 /* interrupt statistics */ + /* ks_ndata == 1 */ +#define KSTAT_TYPE_IO 3 /* I/O statistics */ + /* ks_ndata == 1 */ +#define KSTAT_TYPE_TIMER 4 /* event timer */ + /* ks_ndata >= 1 */ + +#define KSTAT_NUM_TYPES 5 + +/* + * kstat class + * + * Each kstat can be characterized as belonging to some broad class + * of statistics, e.g. disk, tape, net, vm, streams, etc. This field + * can be used as a filter to extract related kstats. The following + * values are currently in use: disk, tape, net, controller, vm, kvm, + * hat, streams, kstat, and misc. (The kstat class encompasses things + * like kstat_types.) + */ + +/* + * kstat flags + * + * Any of the following flags may be passed to kstat_create(). They are + * all zero by default. + * + * KSTAT_FLAG_VIRTUAL: + * + * Tells kstat_create() not to allocate memory for the + * kstat data section; instead, you will set the ks_data + * field to point to the data you wish to export. This + * provides a convenient way to export existing data + * structures. + * + * KSTAT_FLAG_VAR_SIZE: + * + * The size of the kstat you are creating will vary over time. + * For example, you may want to use the kstat mechanism to + * export a linked list. NOTE: The kstat framework does not + * manage the data section, so all variable-size kstats must be + * virtual kstats. Moreover, variable-size kstats MUST employ + * kstat data locking to prevent data-size races with kstat + * clients. See the section on "kstat snapshot" for details. + * + * KSTAT_FLAG_WRITABLE: + * + * Makes the kstat's data section writable by root. + * The ks_snapshot routine (see below) does not need to check for + * this; permission checking is handled in the kstat driver. + * + * KSTAT_FLAG_PERSISTENT: + * + * Indicates that this kstat is to be persistent over time. + * For persistent kstats, kstat_delete() simply marks the + * kstat as dormant; a subsequent kstat_create() reactivates + * the kstat. This feature is provided so that statistics + * are not lost across driver close/open (e.g., raw disk I/O + * on a disk with no mounted partitions.) + * NOTE: Persistent kstats cannot be virtual, since ks_data + * points to garbage as soon as the driver goes away. + * + * The following flags are maintained by the kstat framework: + * + * KSTAT_FLAG_DORMANT: + * + * For persistent kstats, indicates that the kstat is in the + * dormant state (e.g., the corresponding device is closed). + * + * KSTAT_FLAG_INVALID: + * + * This flag is set when a kstat is in a transitional state, + * e.g. between kstat_create() and kstat_install(). + * kstat clients must not attempt to access the kstat's data + * if this flag is set. + */ + +#define KSTAT_FLAG_VIRTUAL 0x01 +#define KSTAT_FLAG_VAR_SIZE 0x02 +#define KSTAT_FLAG_WRITABLE 0x04 +#define KSTAT_FLAG_PERSISTENT 0x08 +#define KSTAT_FLAG_DORMANT 0x10 +#define KSTAT_FLAG_INVALID 0x20 +#define KSTAT_FLAG_LONGSTRINGS 0x40 +#define KSTAT_FLAG_NO_HEADERS 0x80 + +/* + * Dynamic update support + * + * The kstat mechanism allows for an optional ks_update function to update + * kstat data. This is useful for drivers where the underlying device + * keeps cheap hardware stats, but extraction is expensive. Instead of + * constantly keeping the kstat data section up to date, you can supply a + * ks_update function which updates the kstat's data section on demand. + * To take advantage of this feature, simply set the ks_update field before + * calling kstat_install(). + * + * The ks_update function, if supplied, must have the following structure: + * + * int + * foo_kstat_update(kstat_t *ksp, int rw) + * { + * if (rw == KSTAT_WRITE) { + * ... update the native stats from ksp->ks_data; + * return EACCES if you don't support this + * } else { + * ... update ksp->ks_data from the native stats + * } + * } + * + * The ks_update return codes are: 0 for success, EACCES if you don't allow + * KSTAT_WRITE, and EIO for any other type of error. + * + * In general, the ks_update function may need to refer to provider-private + * data; for example, it may need a pointer to the provider's raw statistics. + * The ks_private field is available for this purpose. Its use is entirely + * at the provider's discretion. + * + * All variable-size kstats MUST supply a ks_update routine, which computes + * and sets ks_data_size (and ks_ndata if that is meaningful), since these + * are needed to perform kstat snapshots (see below). + * + * No kstat locking should be done inside the ks_update routine. The caller + * will already be holding the kstat's ks_lock (to ensure consistent data). + */ + +#define KSTAT_READ 0 +#define KSTAT_WRITE 1 + +/* + * Kstat snapshot + * + * In order to get a consistent view of a kstat's data, clients must obey + * the kstat's locking strategy. However, these clients may need to perform + * operations on the data which could cause a fault (e.g. copyout()), or + * operations which are simply expensive. Doing so could cause deadlock + * (e.g. if you're holding a disk's kstat lock which is ultimately required + * to resolve a copyout() fault), performance degradation (since the providers' + * activity is serialized at the kstat lock), device timing problems, etc. + * + * To avoid these problems, kstat data is provided via snapshots. Taking + * a snapshot is a simple process: allocate a wired-down kernel buffer, + * acquire the kstat's data lock, copy the data into the buffer ("take the + * snapshot"), and release the lock. This ensures that the kstat's data lock + * will be held as briefly as possible, and that no faults will occur while + * the lock is held. + * + * Normally, the snapshot is taken by default_kstat_snapshot(), which + * timestamps the data (sets ks_snaptime), copies it, and does a little + * massaging to deal with incomplete transactions on i/o kstats. However, + * this routine only works for kstats with contiguous data (the typical case). + * If you create a kstat whose data is, say, a linked list, you must provide + * your own ks_snapshot routine. The routine you supply must have the + * following prototype (replace "foo" with something appropriate): + * + * int foo_kstat_snapshot(kstat_t *ksp, void *buf, int rw); + * + * The minimal snapshot routine -- one which copies contiguous data that + * doesn't need any massaging -- would be this: + * + * ksp->ks_snaptime = gethrtime(); + * if (rw == KSTAT_WRITE) + * memcpy(ksp->ks_data, buf, ksp->ks_data_size); + * else + * memcpy(buf, ksp->ks_data, ksp->ks_data_size); + * return (0); + * + * A more illuminating example is taking a snapshot of a linked list: + * + * ksp->ks_snaptime = gethrtime(); + * if (rw == KSTAT_WRITE) + * return (EACCES); ... See below ... + * for (foo = first_foo; foo; foo = foo->next) { + * memcpy((char *) buf, (char *) foo, sizeof (struct foo)); + * buf = ((struct foo *) buf) + 1; + * } + * return (0); + * + * In the example above, we have decided that we don't want to allow + * KSTAT_WRITE access, so we return EACCES if this is attempted. + * + * The key points are: + * + * (1) ks_snaptime must be set (via gethrtime()) to timestamp the data. + * (2) Data gets copied from the kstat to the buffer on KSTAT_READ, + * and from the buffer to the kstat on KSTAT_WRITE. + * (3) ks_snapshot return values are: 0 for success, EACCES if you + * don't allow KSTAT_WRITE, and EIO for any other type of error. + * + * Named kstats (see section on "Named statistics" below) containing long + * strings (KSTAT_DATA_STRING) need special handling. The kstat driver + * assumes that all strings are copied into the buffer after the array of + * named kstats, and the pointers (KSTAT_NAMED_STR_PTR()) are updated to point + * into the copy within the buffer. The default snapshot routine does this, + * but overriding routines should contain at least the following: + * + * if (rw == KSTAT_READ) { + * kstat_named_t *knp = buf; + * char *end = knp + ksp->ks_ndata; + * uint_t i; + * + * ... Do the regular copy ... + * memcpy(buf, ksp->ks_data, sizeof (kstat_named_t) * ksp->ks_ndata); + * + * for (i = 0; i < ksp->ks_ndata; i++, knp++) { + * if (knp[i].data_type == KSTAT_DATA_STRING && + * KSTAT_NAMED_STR_PTR(knp) != NULL) { + * memcpy(end, KSTAT_NAMED_STR_PTR(knp), + * KSTAT_NAMED_STR_BUFLEN(knp)); + * KSTAT_NAMED_STR_PTR(knp) = end; + * end += KSTAT_NAMED_STR_BUFLEN(knp); + * } + * } + */ + +/* + * Named statistics. + * + * List of arbitrary name=value statistics. + */ + +#pragma pack(4) +typedef struct kstat_named { + char name[KSTAT_STRLEN]; /* name of counter */ + uchar_t data_type; /* data type */ + union { + char c[16]; /* enough for 128-bit ints */ + int32_t i32; + uint32_t ui32; + struct { + union { + char *ptr; /* NULL-term string */ +#if defined(_KERNEL) && defined(_MULTI_DATAMODEL) + caddr32_t ptr32; +#endif + char __pad[8]; /* 64-bit padding */ + } addr; + uint32_t len; /* # bytes for strlen + '\0' */ + } str; +/* + * The int64_t and uint64_t types are not valid for a maximally conformant + * 32-bit compilation environment (cc -Xc) using compilers prior to the + * introduction of C99 conforming compiler (reference ISO/IEC 9899:1990). + * In these cases, the visibility of i64 and ui64 is only permitted for + * 64-bit compilation environments or 32-bit non-maximally conformant + * C89 or C90 ANSI C compilation environments (cc -Xt and cc -Xa). In the + * C99 ANSI C compilation environment, the long long type is supported. + * The _INT64_TYPE is defined by the implementation (see sys/int_types.h). + */ +#if defined(_INT64_TYPE) + int64_t i64; + uint64_t ui64; +#endif + long l; + ulong_t ul; + + /* These structure members are obsolete */ + + longlong_t ll; + u_longlong_t ull; + float f; + double d; + } value; /* value of counter */ +} kstat_named_t; +#pragma pack() + +#define KSTAT_DATA_CHAR 0 +#define KSTAT_DATA_INT32 1 +#define KSTAT_DATA_UINT32 2 +#define KSTAT_DATA_INT64 3 +#define KSTAT_DATA_UINT64 4 + +#if !defined(_LP64) +#define KSTAT_DATA_LONG KSTAT_DATA_INT32 +#define KSTAT_DATA_ULONG KSTAT_DATA_UINT32 +#else +#if !defined(_KERNEL) +#define KSTAT_DATA_LONG KSTAT_DATA_INT64 +#define KSTAT_DATA_ULONG KSTAT_DATA_UINT64 +#else +#define KSTAT_DATA_LONG 7 /* only visible to the kernel */ +#define KSTAT_DATA_ULONG 8 /* only visible to the kernel */ +#endif /* !_KERNEL */ +#endif /* !_LP64 */ + +/* + * Statistics exporting named kstats with long strings (KSTAT_DATA_STRING) + * may not make the assumption that ks_data_size is equal to (ks_ndata * sizeof + * (kstat_named_t)). ks_data_size in these cases is equal to the sum of the + * amount of space required to store the strings (ie, the sum of + * KSTAT_NAMED_STR_BUFLEN() for all KSTAT_DATA_STRING statistics) plus the + * space required to store the kstat_named_t's. + * + * The default update routine will update ks_data_size automatically for + * variable-length kstats containing long strings (using the default update + * routine only makes sense if the string is the only thing that is changing + * in size, and ks_ndata is constant). Fixed-length kstats containing long + * strings must explicitly change ks_data_size (after creation but before + * initialization) to reflect the correct amount of space required for the + * long strings and the kstat_named_t's. + */ +#define KSTAT_DATA_STRING 9 + +/* These types are obsolete */ + +#define KSTAT_DATA_LONGLONG KSTAT_DATA_INT64 +#define KSTAT_DATA_ULONGLONG KSTAT_DATA_UINT64 +#define KSTAT_DATA_FLOAT 5 +#define KSTAT_DATA_DOUBLE 6 + +#define KSTAT_NAMED_PTR(kptr) ((kstat_named_t *)(kptr)->ks_data) + +/* + * Retrieve the pointer of the string contained in the given named kstat. + */ +#define KSTAT_NAMED_STR_PTR(knptr) ((knptr)->value.str.addr.ptr) + +/* + * Retrieve the length of the buffer required to store the string in the given + * named kstat. + */ +#define KSTAT_NAMED_STR_BUFLEN(knptr) ((knptr)->value.str.len) + +/* + * Interrupt statistics. + * + * An interrupt is a hard interrupt (sourced from the hardware device + * itself), a soft interrupt (induced by the system via the use of + * some system interrupt source), a watchdog interrupt (induced by + * a periodic timer call), spurious (an interrupt entry point was + * entered but there was no interrupt condition to service), + * or multiple service (an interrupt condition was detected and + * serviced just prior to returning from any of the other types). + * + * Measurement of the spurious class of interrupts is useful for + * autovectored devices in order to pinpoint any interrupt latency + * problems in a particular system configuration. + * + * Devices that have more than one interrupt of the same + * type should use multiple structures. + */ + +#define KSTAT_INTR_HARD 0 +#define KSTAT_INTR_SOFT 1 +#define KSTAT_INTR_WATCHDOG 2 +#define KSTAT_INTR_SPURIOUS 3 +#define KSTAT_INTR_MULTSVC 4 + +#define KSTAT_NUM_INTRS 5 + +typedef struct kstat_intr { + uint_t intrs[KSTAT_NUM_INTRS]; /* interrupt counters */ +} kstat_intr_t; + +#define KSTAT_INTR_PTR(kptr) ((kstat_intr_t *)(kptr)->ks_data) + +/* + * I/O statistics. + */ + +typedef struct kstat_io { + + /* + * Basic counters. + * + * The counters should be updated at the end of service + * (e.g., just prior to calling biodone()). + */ + + u_longlong_t nread; /* number of bytes read */ + u_longlong_t nwritten; /* number of bytes written */ + uint_t reads; /* number of read operations */ + uint_t writes; /* number of write operations */ + + /* + * Accumulated time and queue length statistics. + * + * Accumulated time statistics are kept as a running sum + * of "active" time. Queue length statistics are kept as a + * running sum of the product of queue length and elapsed time + * at that length -- i.e., a Riemann sum for queue length + * integrated against time. (You can also think of the active time + * as a Riemann sum, for the boolean function (queue_length > 0) + * integrated against time, or you can think of it as the + * Lebesgue measure of the set on which queue_length > 0.) + * + * ^ + * | _________ + * 8 | i4 | + * | | | + * Queue 6 | | + * Length | _________ | | + * 4 | i2 |_______| | + * | | i3 | + * 2_______| | + * | i1 | + * |_______________________________| + * Time-> t1 t2 t3 t4 + * + * At each change of state (entry or exit from the queue), + * we add the elapsed time (since the previous state change) + * to the active time if the queue length was non-zero during + * that interval; and we add the product of the elapsed time + * times the queue length to the running length*time sum. + * + * This method is generalizable to measuring residency + * in any defined system: instead of queue lengths, think + * of "outstanding RPC calls to server X". + * + * A large number of I/O subsystems have at least two basic + * "lists" of transactions they manage: one for transactions + * that have been accepted for processing but for which processing + * has yet to begin, and one for transactions which are actively + * being processed (but not done). For this reason, two cumulative + * time statistics are defined here: wait (pre-service) time, + * and run (service) time. + * + * All times are 64-bit nanoseconds (hrtime_t), as returned by + * gethrtime(). + * + * The units of cumulative busy time are accumulated nanoseconds. + * The units of cumulative length*time products are elapsed time + * times queue length. + * + * Updates to the fields below are performed implicitly by calls to + * these five functions: + * + * kstat_waitq_enter() + * kstat_waitq_exit() + * kstat_runq_enter() + * kstat_runq_exit() + * + * kstat_waitq_to_runq() (see below) + * kstat_runq_back_to_waitq() (see below) + * + * Since kstat_waitq_exit() is typically followed immediately + * by kstat_runq_enter(), there is a single kstat_waitq_to_runq() + * function which performs both operations. This is a performance + * win since only one timestamp is required. + * + * In some instances, it may be necessary to move a request from + * the run queue back to the wait queue, e.g. for write throttling. + * For these situations, call kstat_runq_back_to_waitq(). + * + * These fields should never be updated by any other means. + */ + + hrtime_t wtime; /* cumulative wait (pre-service) time */ + hrtime_t wlentime; /* cumulative wait length*time product */ + hrtime_t wlastupdate; /* last time wait queue changed */ + hrtime_t rtime; /* cumulative run (service) time */ + hrtime_t rlentime; /* cumulative run length*time product */ + hrtime_t rlastupdate; /* last time run queue changed */ + + uint_t wcnt; /* count of elements in wait state */ + uint_t rcnt; /* count of elements in run state */ + +} kstat_io_t; + +#define KSTAT_IO_PTR(kptr) ((kstat_io_t *)(kptr)->ks_data) + +/* + * Event timer statistics - cumulative elapsed time and number of events. + * + * Updates to these fields are performed implicitly by calls to + * kstat_timer_start() and kstat_timer_stop(). + */ + +typedef struct kstat_timer { + char name[KSTAT_STRLEN]; /* event name */ + uchar_t resv; /* reserved */ + u_longlong_t num_events; /* number of events */ + hrtime_t elapsed_time; /* cumulative elapsed time */ + hrtime_t min_time; /* shortest event duration */ + hrtime_t max_time; /* longest event duration */ + hrtime_t start_time; /* previous event start time */ + hrtime_t stop_time; /* previous event stop time */ +} kstat_timer_t; + +#define KSTAT_TIMER_PTR(kptr) ((kstat_timer_t *)(kptr)->ks_data) + +#if defined(_KERNEL) + +#include + +extern kid_t kstat_chain_id; /* bumped at each state change */ +extern void kstat_init(void); /* initialize kstat framework */ + +/* + * Adding and deleting kstats. + * + * The typical sequence to add a kstat is: + * + * ksp = kstat_create(module, instance, name, class, type, ndata, flags); + * if (ksp) { + * ... provider initialization, if necessary + * kstat_install(ksp); + * } + * + * There are three logically distinct steps here: + * + * Step 1: System Initialization (kstat_create) + * + * kstat_create() performs system initialization. kstat_create() + * allocates memory for the entire kstat (header plus data), initializes + * all header fields, initializes the data section to all zeroes, assigns + * a unique KID, and puts the kstat onto the system's kstat chain. + * The returned kstat is marked invalid (KSTAT_FLAG_INVALID is set), + * because the provider (caller) has not yet had a chance to initialize + * the data section. + * + * By default, kstats are exported to all zones on the system. A kstat may be + * created via kstat_create_zone() to specify a zone to which the statistics + * should be exported. kstat_zone_add() may be used to specify additional + * zones to which the statistics are to be exported. + * + * Step 2: Provider Initialization + * + * The provider performs any necessary initialization of the data section, + * e.g. setting the name fields in a KSTAT_TYPE_NAMED. Virtual kstats set + * the ks_data field at this time. The provider may also set the ks_update, + * ks_snapshot, ks_private, and ks_lock fields if necessary. + * + * Step 3: Installation (kstat_install) + * + * Once the kstat is completely initialized, kstat_install() clears the + * INVALID flag, thus making the kstat accessible to the outside world. + * kstat_install() also clears the DORMANT flag for persistent kstats. + * + * Removing a kstat from the system + * + * kstat_delete(ksp) removes ksp from the kstat chain and frees all + * associated system resources. NOTE: When you call kstat_delete(), + * you must NOT be holding that kstat's ks_lock. Otherwise, you may + * deadlock with a kstat reader. + * + * Persistent kstats + * + * From the provider's point of view, persistence is transparent. The only + * difference between ephemeral (normal) kstats and persistent kstats + * is that you pass KSTAT_FLAG_PERSISTENT to kstat_create(). Magically, + * this has the effect of making your data visible even when you're + * not home. Persistence is important to tools like iostat, which want + * to get a meaningful picture of disk activity. Without persistence, + * raw disk i/o statistics could never accumulate: they would come and + * go with each open/close of the raw device. + * + * The magic of persistence works by slightly altering the behavior of + * kstat_create() and kstat_delete(). The first call to kstat_create() + * creates a new kstat, as usual. However, kstat_delete() does not + * actually delete the kstat: it performs one final update of the data + * (i.e., calls the ks_update routine), marks the kstat as dormant, and + * sets the ks_lock, ks_update, ks_private, and ks_snapshot fields back + * to their default values (since they might otherwise point to garbage, + * e.g. if the provider is going away). kstat clients can still access + * the dormant kstat just like a live kstat; they just continue to see + * the final data values as long as the kstat remains dormant. + * All subsequent kstat_create() calls simply find the already-existing, + * dormant kstat and return a pointer to it, without altering any fields. + * The provider then performs its usual initialization sequence, and + * calls kstat_install(). kstat_install() uses the old data values to + * initialize the native data (i.e., ks_update is called with KSTAT_WRITE), + * thus making it seem like you were never gone. + */ + +extern kstat_t *kstat_create(const char *, int, const char *, const char *, + uchar_t, uint_t, uchar_t); +extern kstat_t *kstat_create_zone(const char *, int, const char *, + const char *, uchar_t, uint_t, uchar_t, zoneid_t); +extern void kstat_install(kstat_t *); +extern void kstat_delete(kstat_t *); +extern void kstat_named_setstr(kstat_named_t *knp, const char *src); +extern void kstat_set_string(char *, const char *); +extern void kstat_delete_byname(const char *, int, const char *); +extern void kstat_delete_byname_zone(const char *, int, const char *, zoneid_t); +extern void kstat_named_init(kstat_named_t *, const char *, uchar_t); +extern void kstat_timer_init(kstat_timer_t *, const char *); +extern void kstat_waitq_enter(kstat_io_t *); +extern void kstat_waitq_exit(kstat_io_t *); +extern void kstat_runq_enter(kstat_io_t *); +extern void kstat_runq_exit(kstat_io_t *); +extern void kstat_waitq_to_runq(kstat_io_t *); +extern void kstat_runq_back_to_waitq(kstat_io_t *); +extern void kstat_timer_start(kstat_timer_t *); +extern void kstat_timer_stop(kstat_timer_t *); + +extern void kstat_zone_add(kstat_t *, zoneid_t); +extern void kstat_zone_remove(kstat_t *, zoneid_t); +extern int kstat_zone_find(kstat_t *, zoneid_t); + +extern kstat_t *kstat_hold_bykid(kid_t kid, zoneid_t); +extern kstat_t *kstat_hold_byname(const char *, int, const char *, zoneid_t); +extern void kstat_rele(kstat_t *); + +#endif /* defined(_KERNEL) */ + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_KSTAT_H */ diff --git a/lib/libspl/include/os/windows/sys/mman.h b/lib/libspl/include/os/windows/sys/mman.h new file mode 100644 index 000000000000..9d9dc2ccff22 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/mman.h @@ -0,0 +1,38 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + + +#ifndef _LIBSPL_WINDOWS_SYS_MMAN_H +#define _LIBSPL_WINDOWS_SYS_MMAN_H + +#define PROT_READ 0x1 /* pages can be read */ +#define PROT_WRITE 0x2 /* pages can be written */ +#define PROT_EXEC 0x4 /* pages can be executed */ + +#define MAP_SHARED 1 /* share changes */ +#define MAP_PRIVATE 2 /* changes are private */ + +#define MAP_FAILED ((void *) -1) + + +int mprotect(void *addr, size_t len, int prot); + +#endif diff --git a/lib/libspl/include/os/windows/sys/mnttab.h b/lib/libspl/include/os/windows/sys/mnttab.h new file mode 100644 index 000000000000..8998c833d274 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/mnttab.h @@ -0,0 +1,87 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SYS_MNTTAB_H +#define _SYS_MNTTAB_H + +#include +#include +#include +#include +#include +#include +#include + +#ifdef MNTTAB +#undef MNTTAB +#endif /* MNTTAB */ + +/* + * mnttab file is updated by kernel to show current mounts on + * other platforms, there is no such file in Windows. We call + * xxx() instead, but build a "mnttab" list to be + * compatible. But since the existance of the MNTTAB is required + * (and fails silently) the "fd" work has been removed. + */ + +#ifdef MNTTAB +#undef MNTTAB +#endif /* MNTTAB */ +#define MNTTAB "nul" + +#define umount2(p, f) unmount(p, f) + +#define MNT_LINE_MAX 4096 + +#define MNT_TOOLONG 1 /* entry exceeds MNT_LINE_MAX */ +#define MNT_TOOMANY 2 /* too many fields in line */ +#define MNT_TOOFEW 3 /* too few fields in line */ + +struct mnttab { + char *mnt_special; + char *mnt_mountp; + char *mnt_fstype; + char *mnt_mntopts; + uint_t mnt_major; + uint_t mnt_minor; + uint32_t mnt_fssubtype; +}; +#define extmnttab mnttab + +extern DIR *fdopendir(int fd); +extern int openat64(int, const char *, int, ...); + +// From FreeBSD +extern int getmntany(FILE *fd, struct mnttab *mgetp, struct mnttab *mrefp); +extern int getmntent(FILE *fp, struct mnttab *mp); +extern char *hasmntopt(struct mnttab *mnt, char *opt); +FILE *setmntent(const char *filename, const char *type); + +extern void statfs2mnttab(struct statfs *sfs, struct mnttab *mp); + +#ifndef AT_SYMLINK_NOFOLLOW +#define AT_SYMLINK_NOFOLLOW 0x100 +#endif + +extern int fstatat64(int, const char *, struct _stat64 *, int); + +#endif diff --git a/lib/libspl/include/os/windows/sys/mount.h b/lib/libspl/include/os/windows/sys/mount.h new file mode 100644 index 000000000000..b6968856d843 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/mount.h @@ -0,0 +1,122 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_SYS_MOUNT_H +#define _LIBSPL_SYS_MOUNT_H + +#include +#include +#include +#include +#include +#include + +/* + * Some old glibc headers don't define BLKGETSIZE64 + * and we don't want to require the kernel headers + */ +#if !defined(BLKGETSIZE64) +#define BLKGETSIZE64 _IOR(0x12, 114, size_t) +#endif + +/* + * Some old glibc headers don't correctly define MS_DIRSYNC and + * instead use the enum name S_WRITE. When using these older + * headers define MS_DIRSYNC to be S_WRITE. + */ +#if !defined(MS_DIRSYNC) +#define MS_DIRSYNC S_WRITE +#endif + +/* + * Some old glibc headers don't correctly define MS_POSIXACL and + * instead leave it undefined. When using these older headers define + * MS_POSIXACL to the reserved value of (1<<16). + */ +#if !defined(MS_POSIXACL) +#define MS_POSIXACL (1<<16) +#endif + +#define MS_USERS (MS_NOEXEC|MS_NOSUID|MS_NODEV) +#define MS_OWNER (MS_NOSUID|MS_NODEV) +#define MS_GROUP (MS_NOSUID|MS_NODEV) +#define MS_COMMENT 0 + +#include + + +#ifdef __LINUX__ +/* + * Older glibc headers did not define all the available + * umount2(2) flags. Both MNT_FORCE and MNT_DETACH are supported in the + * kernel back to 2.4.11 so we define them correctly if they are missing. + */ +#ifdef MNT_FORCE +#define MS_FORCE MNT_FORCE +#else +#define MS_FORCE 0x00000001 +#endif /* MNT_FORCE */ + +#ifdef MNT_DETACH +#define MS_DETACH MNT_DETACH +#else +#define MS_DETACH 0x00000002 +#endif /* MNT_DETACH */ + +/* + * Overlay mount is default in Linux, but for solaris/zfs + * compatibility, MS_OVERLAY is defined to explicitly have the user + * provide a flag (-O) to mount over a non empty directory. + */ +#define MS_OVERLAY 0x00000004 + +#define MS_RDONLY 0x0001 /* Read-only */ +#define MS_OPTIONSTR 0x0100 /* Data is an in/out option string */ +#define MS_NOMNTTAB 0x0800 /* Don't show mount in mnttab */ +#endif /* __LINUX__ */ +/* + * MS_CRYPT indicates that encryption keys should be loaded if they are not + * already available. This is not defined in glibc, but it is never seen by + * the kernel so it will not cause any problems. + */ +#define MS_CRYPT 0x00000008 + +#define MFSTYPENAMELEN 16 + +struct statfs { + uint32_t f_bsize; /* fundamental file system block size */ + uint64_t f_blocks; /* total data blocks in file system */ + uint64_t f_bfree; /* free blocks in fs */ + uint64_t f_bavail; /* free blocks avail to non-superuser */ + uint64_t f_files; /* total file nodes in file system */ + uint64_t f_ffree; /* free file nodes in fs */ + uint32_t f_type; /* type of filesystem */ + uint32_t f_flags; /* copy of mount exported flags */ + uint32_t f_fssubtype; /* fs sub-type (flavor) */ + char f_fstypename[MFSTYPENAMELEN]; /* fs type name */ + char f_mntonname[MAXPATHLEN]; /* dir on which mounted */ + char f_mntfromname[MAXPATHLEN]; /* mounted filesystem */ +}; + +int statfs(const char *path, struct statfs *buf); + +#endif /* _LIBSPL_SYS_MOUNT_H */ diff --git a/lib/libspl/include/os/windows/sys/param.h b/lib/libspl/include/os/windows/sys/param.h new file mode 100644 index 000000000000..477cd61768b1 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/param.h @@ -0,0 +1,66 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _LIBSPL_WINDOWS_SYS_PARAM_H +#define _LIBSPL_WINDOWS_SYS_PARAM_H + +#include +#include + +/* + * File system parameters and macros. + * + * The file system is made out of blocks of at most MAXBSIZE units, + * with smaller units (fragments) only in the last direct block. + * MAXBSIZE primarily determines the size of buffers in the buffer + * pool. It may be made larger without any effect on existing + * file systems; however making it smaller make make some file + * systems unmountable. + * + * Note that the blocked devices are assumed to have DEV_BSIZE + * "sectors" and that fragments must be some multiple of this size. + */ +// #define MAXBSIZE 8192 +#define DEV_BSIZE 512 +#define DEV_BSHIFT 9 /* log2(DEV_BSIZE) */ + +#define MAXNAMELEN 256 +#define MAXOFFSET_T _I64_MAX + +#define UID_NOBODY 60001 /* user ID no body */ +#define GID_NOBODY UID_NOBODY +#define UID_NOACCESS 60002 /* user ID no access */ + +#define MAXUID UINT32_MAX /* max user id */ +#define MAXPROJID MAXUID /* max project id */ + +#ifndef PAGESIZE +#define PAGESIZE (8192) // SYSTEM_INFO.dwPageSize; +#endif /* PAGESIZE */ + +#define NBBY 8 + +#endif diff --git a/lib/libspl/include/os/windows/sys/resource.h b/lib/libspl/include/os/windows/sys/resource.h new file mode 100644 index 000000000000..6a15078ff23c --- /dev/null +++ b/lib/libspl/include/os/windows/sys/resource.h @@ -0,0 +1,26 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_RESOURCE_H +#define _LIBSPL_RESOURCE_H + +#endif diff --git a/lib/libspl/include/os/windows/sys/socket.h b/lib/libspl/include/os/windows/sys/socket.h new file mode 100644 index 000000000000..f22a6cee1392 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/socket.h @@ -0,0 +1,31 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (t he "License"). You may not use this file except in compliance + * wit h the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright(c) 2017 Jorgen Lundman + */ + +#ifndef _LIBSPL_SYS_SOCKET_H +#define _LIBSPL_SYS_SOCKET_H + +#include + +#endif diff --git a/lib/libspl/include/os/windows/sys/stat.h b/lib/libspl/include/os/windows/sys/stat.h new file mode 100644 index 000000000000..32972a2cc09d --- /dev/null +++ b/lib/libspl/include/os/windows/sys/stat.h @@ -0,0 +1,99 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_WINDOWS_SYS_STAT_H +#define _LIBSPL_WINDOWS_SYS_STAT_H + +#define NOWOSIXYET +#include_next +#undef NOWOSIXYET +#include + +/* File type and permission flags for stat() */ +/* We can not use Windows' stat.h versions, they are too small */ +#undef S_IFMT +#undef S_IFCHR +#undef S_IFDIR +#undef S_IFREG + +#define S_IFMT 0170000 /* [XSI] type of file mask */ +#define S_IFIFO 0010000 /* [XSI] named pipe (fifo) */ +#define S_IFCHR 0020000 /* [XSI] character special */ +#define S_IFDIR 0040000 /* [XSI] directory */ +#define S_IFBLK 0060000 /* [XSI] block special */ +#define S_IFREG 0100000 /* [XSI] regular */ +#define S_IFLNK 0120000 /* [XSI] symbolic link */ +#define S_IFSOCK 0140000 /* [XSI] socket */ +#if !defined(_POSIX_C_SOURCE) +#define S_IFWHT 0160000 /* OBSOLETE: whiteout */ +#endif +/* File mode */ +/* Read, write, execute/search by owner */ +#define S_IRWXU 0000700 /* [XSI] RWX mask for owner */ +#define S_IRUSR 0000400 /* [XSI] R for owner */ +#define S_IWUSR 0000200 /* [XSI] W for owner */ +#define S_IXUSR 0000100 /* [XSI] X for owner */ +/* Read, write, execute/search by group */ +#define S_IRWXG 0000070 /* [XSI] RWX mask for group */ +#define S_IRGRP 0000040 /* [XSI] R for group */ +#define S_IWGRP 0000020 /* [XSI] W for group */ +#define S_IXGRP 0000010 /* [XSI] X for group */ +/* Read, write, execute/search by others */ +#define S_IRWXO 0000007 /* [XSI] RWX mask for other */ +#define S_IROTH 0000004 /* [XSI] R for other */ +#define S_IWOTH 0000002 /* [XSI] W for other */ +#define S_IXOTH 0000001 /* [XSI] X for other */ + +#define S_ISUID 0004000 /* [XSI] set user id on execution */ +#define S_ISGID 0002000 /* [XSI] set group id on execution */ +#define S_ISVTX 0001000 /* [XSI] directory restrcted delete */ + +#if !defined(_POSIX_C_SOURCE) +#ifndef S_ISTXT +#define S_ISTXT S_ISVTX /* sticky bit: not supported */ +#endif +#ifndef S_IREAD +#define S_IREAD S_IRUSR /* backward compatability */ +#endif +#ifndef S_IWRITE +#define S_IWRITE S_IWUSR /* backward compatability */ +#endif +#ifndef S_IEXEC +#define S_IEXEC S_IXUSR /* backward compatability */ +#endif +#endif + +/* + * File type macros. Note that block devices, sockets and links cannot be + * distinguished on Windows and the macros S_ISBLK, S_ISSOCK and S_ISLNK are + * only defined for compatibility. These macros should always return false + * on Windows. + */ +#define S_ISFIFO(mode) (((mode) & S_IFMT) == S_IFIFO) +#define S_ISDIR(mode) (((mode) & S_IFMT) == S_IFDIR) +#define S_ISREG(mode) (((mode) & S_IFMT) == S_IFREG) +#define S_ISLNK(mode) (((mode) & S_IFMT) == S_IFLNK) +#define S_ISSOCK(mode) (((mode) & S_IFMT) == S_IFSOCK) +#define S_ISCHR(mode) (((mode) & S_IFMT) == S_IFCHR) +#define S_ISBLK(mode) (((mode) & S_IFMT) == S_IFBLK) + +#endif /* _LIBSPL_SYS_STAT_H */ diff --git a/lib/libspl/include/os/windows/sys/sysmacros.h b/lib/libspl/include/os/windows/sys/sysmacros.h new file mode 100644 index 000000000000..fa6c8f98eb33 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/sysmacros.h @@ -0,0 +1,119 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _LIBSPL_SYS_SYSMACROS_H +#define _LIBSPL_SYS_SYSMACROS_H + +// #include_next + +/* common macros */ +#ifndef MIN +#define MIN(a, b) ((a) < (b) ? (a) : (b)) +#endif +#ifndef MAX +#define MAX(a, b) ((a) < (b) ? (b) : (a)) +#endif +#ifndef ABS +#define ABS(a) ((a) < 0 ? -(a) : (a)) +#endif +#ifndef ARRAY_SIZE +#define ARRAY_SIZE(a) (sizeof (a) / sizeof (a[0])) +#endif +#ifndef DIV_ROUND_UP +#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d)) +#endif + +#define makedevice(maj, min) makedev(maj, min) +#define _sysconf(a) sysconf(a) + + +#ifdef _MSC_VER +#define _Noreturn +#else +#define _Noreturn __attribute__((__noreturn__)) +#endif + +/* + * Compatibility macros/typedefs needed for Solaris -> Windows port + */ +// unary minus operator applied to unsigned type +#pragma warning(disable: 4146) +#define P2ALIGN(x, align) ((x) & -(align)) +#define P2CROSS(x, y, align) (((x) ^ (y)) > (align) - 1) +#define P2ROUNDUP(x, align) (-(-(x) & -(align))) +#define P2ROUNDUP_TYPED(x, align, type) \ + (-(-(type)(x) & -(type)(align))) +#define P2BOUNDARY(off, len, align) \ + (((off) ^ ((off) + (len) - 1)) > (align) - 1) +#define P2PHASE(x, align) ((x) & ((align) - 1)) +#define P2NPHASE(x, align) (-(x) & ((align) - 1)) +#define P2NPHASE_TYPED(x, align, type) \ + (-(type)(x) & ((type)(align) - 1)) +#define ISP2(x) (((x) & ((x) - 1)) == 0) +#define IS_P2ALIGNED(v, a) ((((uintptr_t)(v)) & ((uintptr_t)(a) - 1)) == 0) +#pragma warning(default: 4146) + +/* + * Typed version of the P2* macros. These macros should be used to ensure + * that the result is correctly calculated based on the data type of (x), + * which is passed in as the last argument, regardless of the data + * type of the alignment. For example, if (x) is of type uint64_t, + * and we want to round it up to a page boundary using "PAGESIZE" as + * the alignment, we can do either + * P2ROUNDUP(x, (uint64_t)PAGESIZE) + * or + * P2ROUNDUP_TYPED(x, PAGESIZE, uint64_t) + */ +#define P2ALIGN_TYPED(x, align, type) \ + ((type)(x) & -(type)(align)) +#define P2PHASE_TYPED(x, align, type) \ + ((type)(x) & ((type)(align) - 1)) +#define P2NPHASE_TYPED(x, align, type) \ + (-(type)(x) & ((type)(align) - 1)) +#define P2ROUNDUP_TYPED(x, align, type) \ + (-(-(type)(x) & -(type)(align))) +#define P2END_TYPED(x, align, type) \ + (-(~(type)(x) & -(type)(align))) +#define P2PHASEUP_TYPED(x, align, phase, type) \ + ((type)(phase) - (((type)(phase) - (type)(x)) & -(type)(align))) +#define P2CROSS_TYPED(x, y, align, type) \ + (((type)(x) ^ (type)(y)) > (type)(align) - 1) +#define P2SAMEHIGHBIT_TYPED(x, y, type) \ + (((type)(x) ^ (type)(y)) < ((type)(x) & (type)(y))) + + +/* avoid any possibility of clashing with version */ +#if defined(_KERNEL) && !defined(_KMEMUSER) && !defined(offsetof) +#define offsetof(s, m) ((size_t)(&(((s *)0)->m))) +#endif + +#ifndef RLIM64_INFINITY +#define RLIM64_INFINITY (~0ULL) +#endif + +#define _LITTLE_ENDIAN + +#endif /* _LIBSPL_SYS_SYSMACROS_H */ diff --git a/lib/libspl/include/os/windows/sys/time.h b/lib/libspl/include/os/windows/sys/time.h new file mode 100644 index 000000000000..b212e32b5f70 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/time.h @@ -0,0 +1,104 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _LIBSPL_WINDOWS_SYS_TIME_H +#define _LIBSPL_WINDOWS_SYS_TIME_H + +// #include_next +#include +#if !defined(_WINSOCKAPI_) && !defined(_WINSOCK2API_) +#include +#pragma comment(lib, "Ws2_32.lib") +#endif + +typedef struct timespec inode_timespec_t; + +#ifndef SEC +#define SEC 1 +#endif + +#ifndef MILLISEC +#define MILLISEC 1000 +#endif + +#ifndef MICROSEC +#define MICROSEC 1000000 +#endif + +#ifndef NANOSEC +#define NANOSEC 1000000000 +#endif + +#ifndef NSEC_PER_USEC +#define NSEC_PER_USEC 1000L +#endif + +#ifndef MSEC2NSEC +#define MSEC2NSEC(m) ((hrtime_t)(m) * (NANOSEC / MILLISEC)) +#endif + +#ifndef USEC2NSEC +#define USEC2NSEC(u) ((hrtime_t)(u) * (NANOSEC / MICROSEC)) +#endif + + +#ifndef NSEC2MSEC +#define NSEC2MSEC(n) ((n) / (NANOSEC / MILLISEC)) +#endif + +#define NSEC2SEC(n) ((n) / (NANOSEC / SEC)) +#define SEC2NSEC(m) ((hrtime_t)(m) * (NANOSEC / SEC)) +#define NSEC2USEC(n) ((n) / (NANOSEC / MICROSEC)) + +/* Technically timeval lives in winsock2.h */ +#if 0 +#ifndef _WINSOCK2API_ +struct timeval { + long tv_sec; /* seconds */ + long tv_usec; /* and microseconds */ +}; +#endif +#endif + +typedef enum clock_type { + __CLOCK_REALTIME0 = 0, /* obsolete; same as CLOCK_REALTIME */ + CLOCK_VIRTUAL = 1, /* thread's user-level CPU clock */ + CLOCK_THREAD_CPUTIME_ID = 2, /* thread's user+system CPU clock */ + CLOCK_REALTIME = 3, /* wall clock */ + CLOCK_MONOTONIC = 4, /* high resolution monotonic clock */ + CLOCK_PROCESS_CPUTIME_ID = 5, /* process's user+system CPU clock */ + CLOCK_HIGHRES = CLOCK_MONOTONIC, /* alternate name */ + CLOCK_PROF = CLOCK_THREAD_CPUTIME_ID, /* alternate name */ +} clock_type_t; + +extern void gethrestime(timestruc_t *); + +const char *win_ctime_r(char *buffer, size_t bufsize, time_t cur_time); +#define ctime_r(CLOCK, STR) win_ctime_r((STR), sizeof ((STR)), (CLOCK)) + +struct tm *localtime_r(const time_t *clock, struct tm *result); + +#endif /* _LIBSPL_SYS_TIME_H */ diff --git a/lib/libspl/include/os/windows/sys/timer.h b/lib/libspl/include/os/windows/sys/timer.h new file mode 100644 index 000000000000..65bdd7b25695 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/timer.h @@ -0,0 +1,26 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_TIMER_H +#define _LIBSPL_TIMER_H + +#endif /* _LIBSPL_TIMER_H */ diff --git a/lib/libspl/include/os/windows/sys/types.h b/lib/libspl/include/os/windows/sys/types.h new file mode 100644 index 000000000000..09bcfde399de --- /dev/null +++ b/lib/libspl/include/os/windows/sys/types.h @@ -0,0 +1,225 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright(c) 2017 Jorgen Lundman + */ + +/* + * Unfortunately Windows does not have a #include_next so we + * renamed this wrapper, and it will have to be manually included + * after each sys/types.h include + */ + + +#ifndef _LIBSPL_SYS_W32_TYPES_H +#define _LIBSPL_SYS_W32_TYPES_H + +#include + +/* More #includes at end - after basic types */ + +typedef enum boolean bool_t; + +typedef unsigned char uchar_t; +typedef unsigned short ushort_t; +typedef unsigned int uint_t; +typedef unsigned long ulong_t; +typedef short int16_t; +typedef unsigned short uint16_t; +typedef long long longlong_t; +typedef unsigned long long u_longlong_t; +typedef signed char int8_t; +typedef unsigned char uint8_t; +typedef unsigned char uchar_t; + +typedef longlong_t offset_t; +typedef u_longlong_t u_offset_t; +typedef u_longlong_t len_t; +typedef longlong_t diskaddr_t; + +typedef ulong_t pfn_t; /* page frame number */ +typedef ulong_t pgcnt_t; /* number of pages */ +typedef long spgcnt_t; /* signed number of pages */ + +typedef longlong_t hrtime_t; +typedef struct timespec timestruc_t; +typedef struct timespec timespec_t; + +typedef short pri_t; + + + +typedef int major_t; +// typedef uint_t minor_t; +typedef int pid_t; + +typedef ushort_t mode_t; /* old file attribute type */ +typedef short index_t; + +typedef unsigned long long rlim64_t; + +typedef uint64_t uid_t; +typedef uint64_t gid_t; +typedef uint64_t user_addr_t; +typedef int64_t user_ssize_t; +typedef uint64_t user_size_t; +typedef long clock_t; + +typedef unsigned long ulong_t; +typedef unsigned char uchar_t; + +typedef char *caddr_t; + + +typedef uint32_t dev_t; + +typedef int64_t ssize_t; + +#define F_OK 0 +#define W_OK 2 +#define R_OK 4 + + +/* MAXPATHLEN need to match between kernel and userland. MAX_PATH is only 260 */ +// #define MAXPATHLEN MAX_PATH +#define MAXPATHLEN 1024 +#define PATH_MAX MAX_PATH + +typedef struct timespec timestruc_t; /* definition per SVr4 */ +typedef struct timespec timespec_t; + + +/* + * Definitions remaining from previous partial support for 64-bit file + * offsets. This partial support for devices greater than 2gb requires + * compiler support for long long. + */ +#ifdef _LONG_LONG_LTOH +typedef union { + offset_t _f; /* Full 64 bit offset value */ + struct { + int32_t _l; /* lower 32 bits of offset value */ + int32_t _u; /* upper 32 bits of offset value */ + } _p; +} lloff_t; +#endif + +#ifdef _LONG_LONG_HTOL +typedef union { + offset_t _f; /* Full 64 bit offset value */ + struct { + int32_t _u; /* upper 32 bits of offset value */ + int32_t _l; /* lower 32 bits of offset value */ + } _p; +} lloff_t; +#endif + +#define ENOTBLK 15 /* Block device required */ +#define EDQUOT 49 /* Disc quota exceeded */ +#define EBADE 50 /* invalid exchange */ +#define ESHUTDOWN 58 /* Can't send after socket shutdown */ +#define ESTALE 70 /* Stale NFS file handle */ + +#define ENOTACTIVE 142 +#define ECHRNG 143 +#define EREMOTEIO 144 + +#define O_SHLOCK 0 + + +// #ifdef _MSC_VER +#ifndef __clang__ + +#define INT_MAX 2147483647 + +#define DBL_DIG 15 +#define DBL_MAX 1.7976931348623157081452E+308 +#define DBL_MIN 2.2250738585072013830903E-308 + +#define FLT_DIG 6 +#define FLT_MAX 3.4028234663852885981170E+38F +#define FLT_MIN 1.1754943508222875079688E-38F +#endif + +#define STDIN_FILENO HTOI(GetStdHandle(STD_INPUT_HANDLE)) +#define STDOUT_FILENO HTOI(GetStdHandle(STD_OUTPUT_HANDLE)) +#define STDERR_FILENO HTOI(GetStdHandle(STD_ERROR_HANDLE)) +#define O_EXLOCK 0 + +#define alloca _alloca +int posix_memalign(void **memptr, size_t alignment, size_t size); + +#ifndef MIN +#define MIN(a, b) (((a) < (b)) ? (a) : (b)) +#endif + +#define sleep(x) Sleep(x * 1000) + +#define lstat _stat64 +#define unlink _unlink +#define strdup _strdup + +#define MFSTYPENAMELEN 16 +#define MNAMELEN MAXPATHLEN + +#define roundup(x, y) ((((x) + ((y) - 1)) / (y)) * (y)) +#define howmany(x, y) ((((x) % (y)) == 0) ? ((x) / (y)) : (((x) / (y)) + 1)) + +#define RLIMIT_NOFILE 8 /* number of open files */ +typedef uint64_t rlim_t; +struct rlimit { + rlim_t rlim_cur; /* current (soft) limit */ + rlim_t rlim_max; /* hard limit */ +}; + +#define _Atomic + +#ifndef SIGTSTP +#define SIGTSTP 0 +#endif + +#define strcasecmp _stricmp + +int nanosleep(const struct timespec *rqtp, struct timespec *rmtp); + +#include +#include +#include_next + +#include /* for NBBY */ +#include + +#include +#include + +/* Windows defines off_t as "long" and functions take it as parameter. */ +#include +typedef uint64_t zoff_t; +#define off_t zoff_t + + +/* Now replace POSIX calls with our versions. */ +#ifndef NOWOSIXYET +#include +#endif + +#endif diff --git a/lib/libspl/include/os/windows/sys/types32.h b/lib/libspl/include/os/windows/sys/types32.h new file mode 100644 index 000000000000..5a4da61636d0 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/types32.h @@ -0,0 +1,109 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _SYS_TYPES32_H +#define _SYS_TYPES32_H + + + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define WIN32_LEAN_AND_MEAN +#include +#include + +/* + * Interoperability types for programs. Used for: + * + * Crossing between 32-bit and 64-bit domains. + * + * On disk data formats such as filesystem meta data + * and disk label. + * + * Note: Applications should never include this + * header file. + */ +typedef int int32_t; +typedef unsigned int uint32_t; + +typedef uint32_t caddr32_t; +typedef int32_t daddr32_t; +typedef int32_t off32_t; +typedef uint32_t ino32_t; +typedef int32_t blkcnt32_t; +typedef uint32_t fsblkcnt32_t; +typedef uint32_t fsfilcnt32_t; +typedef int32_t id32_t; +typedef uint32_t major32_t; +typedef uint32_t minor32_t; +typedef int32_t key32_t; +typedef uint32_t mode32_t; +typedef uint32_t uid32_t; +typedef uint32_t gid32_t; +typedef uint32_t nlink32_t; +typedef uint32_t dev32_t; +typedef int32_t pid32_t; +typedef uint32_t size32_t; +typedef int32_t ssize32_t; +typedef int32_t time32_t; +typedef int32_t clock32_t; +typedef long long int64_t; +typedef unsigned long long uint64_t; +#ifndef loff_t +typedef unsigned long long loff_t; +#endif +typedef uint64_t a_offset; +typedef uint64_t user_addr_t; +typedef uint64_t user_size_t; +typedef char *caddr_t; +typedef unsigned short umode_t; +typedef unsigned short mode_t; + +struct timeval32 { + time32_t tv_sec; /* seconds */ + int32_t tv_usec; /* and microseconds */ +}; + +typedef struct timespec32 { + time32_t tv_sec; /* seconds */ + int32_t tv_nsec; /* and nanoseconds */ +} timespec32_t; + +typedef struct timespec32 timestruc32_t; + +typedef struct itimerspec32 { + struct timespec32 it_interval; + struct timespec32 it_value; +} itimerspec32_t; + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_TYPES32_H */ diff --git a/lib/libspl/include/os/windows/sys/uio.h b/lib/libspl/include/os/windows/sys/uio.h new file mode 100644 index 000000000000..1d23a05880c7 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/uio.h @@ -0,0 +1,52 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2005 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ + +/* + * University Copyright- Copyright (c) 1982, 1986, 1988 + * The Regents of the University of California + * All Rights Reserved + * + * University Acknowledgment- Portions of this document are derived from + * software developed by the University of California, Berkeley, and its + * contributors. + */ + +#ifndef _LIBSPL_WINDOWS_SYS_UIO_H +#define _LIBSPL_WINDOWS_SYS_UIO_H + +struct iovec { + void *iov_base; /* Base address. */ + uint32_t iov_len; /* Length. */ +}; + +typedef struct iovec iovec_t; + +#include_next + +#endif /* _WINDOWS_SYS_UIO_H */ diff --git a/lib/libspl/include/os/windows/sys/utsname.h b/lib/libspl/include/os/windows/sys/utsname.h new file mode 100644 index 000000000000..6075b7ef9377 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/utsname.h @@ -0,0 +1,41 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENLIBSPLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENLIBSPLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _LIBSPL_UTSNAME_H +#define _LIBSPL_UTSNAME_H + +#define _SYS_NMLN 257 +struct utsname { + char sysname[_SYS_NMLN]; + char nodename[_SYS_NMLN]; + char release[_SYS_NMLN]; + char version[_SYS_NMLN]; + char machine[_SYS_NMLN]; +}; + +typedef struct utsname utsname_t; + +#endif /* _LIBSPL_UTSNAME_H */ diff --git a/lib/libspl/include/os/windows/sys/vfs.h b/lib/libspl/include/os/windows/sys/vfs.h new file mode 100644 index 000000000000..a0447a5a1c09 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/vfs.h @@ -0,0 +1,26 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef LIBSPL_VFS_H_INCLUDED +#define LIBSPL_VFS_H_INCLUDED + +#endif diff --git a/lib/libspl/include/os/windows/sys/wait.h b/lib/libspl/include/os/windows/sys/wait.h new file mode 100644 index 000000000000..2ca58e62584b --- /dev/null +++ b/lib/libspl/include/os/windows/sys/wait.h @@ -0,0 +1,26 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _LIBSPL_WAIT_H +#define _LIBSPL_WAIT_H + +#endif diff --git a/lib/libspl/include/os/windows/sys/zfs_context_os.h b/lib/libspl/include/os/windows/sys/zfs_context_os.h new file mode 100644 index 000000000000..773d5f38f7cf --- /dev/null +++ b/lib/libspl/include/os/windows/sys/zfs_context_os.h @@ -0,0 +1,52 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef ZFS_CONTEXT_OS_H_ +#define ZFS_CONTEXT_OS_H_ + +#ifdef _WIN32 +#include +#endif + +#define ZFS_EXPORTS_PATH "/etc/exports" +#define MNTTYPE_ZFS_SUBTYPE ('Z'<<24|'F'<<16|'S'<<8) + +struct spa_iokit; +typedef struct spa_iokit spa_iokit_t; + +typedef off_t loff_t; + +struct zfs_handle; + +#define noinline __attribute__((noinline)) + +extern void libzfs_macos_wrapfd(int *srcfd, boolean_t send); + +#define FSCTL_ZFS_VOLUME_MOUNTPOINT CTL_CODE(FILE_DEVICE_UNKNOWN, \ + 0x8ff, METHOD_BUFFERED, FILE_ANY_ACCESS) +typedef struct +{ + int len; + WCHAR buffer[1]; // make this dynamic? +} fsctl_zfs_volume_mountpoint_t; + +#endif diff --git a/lib/libspl/include/os/windows/sys/zfs_mount.h b/lib/libspl/include/os/windows/sys/zfs_mount.h new file mode 100644 index 000000000000..fb0444048567 --- /dev/null +++ b/lib/libspl/include/os/windows/sys/zfs_mount.h @@ -0,0 +1,74 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#ifndef _SYS_ZFS_MOUNT_H_ +#define _SYS_ZFS_MOUNT_H_ + +struct zfs_mount_args { + const char *fspec; /* block special device to mount */ + int mflag; + char *optptr; + int optlen; + int struct_size; +}; + +/* + * Flag bits passed to mount(2). + */ +#define MS_RDONLY 0x0001 /* Read-only */ +#define MS_FSS 0x0002 /* Old (4-argument) mount (compatibility) */ +#define MS_DATA 0x0004 /* 6-argument mount */ +#define MS_NOSUID 0x0010 /* Setuid programs disallowed */ +#define MS_REMOUNT 0x0020 /* Remount */ +#define MS_NOTRUNC 0x0040 /* Return ENAMETOOLONG for long filenames */ +#define MS_OVERLAY 0x0080 /* Allow overlay mounts */ +#define MS_OPTIONSTR 0x0100 /* Data is a an in/out option string */ +#define MS_GLOBAL 0x0200 /* Clustering: Mount into global name space */ +#define MS_FORCE 0x0400 /* Forced unmount */ +#define MS_NOMNTTAB 0x0800 /* Don't show mount in mnttab */ +/* + * Additional flag bits that domount() is prepared to interpret, but that + * can't be passed through mount(2). + */ +#define MS_SYSSPACE 0x0008 /* Mounta already in kernel space */ +#define MS_NOSPLICE 0x1000 /* Don't splice fs instance into name space */ +#define MS_NOCHECK 0x2000 /* Clustering: suppress mount busy checks */ +/* + * Mask to sift out flag bits allowable from mount(2). + */ +#define MS_MASK \ + (MS_RDONLY|MS_FSS|MS_DATA|MS_NOSUID|MS_REMOUNT|MS_NOTRUNC|MS_OVERLAY|\ + MS_OPTIONSTR|MS_GLOBAL|MS_NOMNTTAB) + +/* + * Mask to sift out flag bits allowable from umount2(2). + */ + +#define MS_UMOUNT_MASK (MS_FORCE) + +/* + * Maximum option string length accepted or returned by mount(2). + */ +#define MAX_MNTOPT_STR 1024 /* max length of mount options string */ + + +#endif /* _SYS_ZFS_IOCTL_H */ diff --git a/lib/libspl/include/os/windows/termios.h b/lib/libspl/include/os/windows/termios.h new file mode 100644 index 000000000000..d75eb6cfa425 --- /dev/null +++ b/lib/libspl/include/os/windows/termios.h @@ -0,0 +1,50 @@ +/* + * CDDL HEADER START + * + * This file and its contents are supplied under the terms of the + * Common Development and Distribution License ("CDDL"), version 1.0. + * You may only use this file in accordance with the terms of version + * 1.0 of the CDDL. + * + * A full copy of the text of the CDDL should have accompanied this + * source. A copy of the CDDL is also available via the Internet at + * http://www.illumos.org/license/CDDL. + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017, Jorgen Lundman. All rights reserved. + */ + +#ifndef _TERMIOS_H +#define _TERMIOS_H + +#define ECHOE 0x00000002 /* visually erase chars */ +#define ECHOK 0x00000004 /* echo NL after line kill */ +#define ECHO 0x00000008 /* enable echoing */ +#define ECHONL 0x00000010 /* echo NL even if ECHO is off */ + +#define TCSAFLUSH 2 /* drain output, flush input */ + +#define TIOCGWINSZ (104) +#define TIOCSWINSZ (103) + +struct termios +{ + int c_lflag; +}; + +struct winsize { + unsigned short ws_row; /* rows, in characters */ + unsigned short ws_col; /* columns, in character */ + unsigned short ws_xpixel; /* horizontal size, pixels */ + unsigned short ws_ypixel; /* vertical size, pixels */ +}; + +int tcgetattr(int fildes, struct termios *termios_p); + +int tcsetattr(int fildes, int optional_actions, + const struct termios *termios_p); + +#endif diff --git a/lib/libspl/include/os/windows/unistd.h b/lib/libspl/include/os/windows/unistd.h new file mode 100644 index 000000000000..3e124b90322e --- /dev/null +++ b/lib/libspl/include/os/windows/unistd.h @@ -0,0 +1,92 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2008 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +#ifndef _LIBSPL_WINDOWS_UNISTD_H +#define _LIBSPL_WINDOWS_UNISTD_H + +#include +#include +#define issetugid() (geteuid() == 0 || getegid() == 0) + +#include + +#include +#include +#include + +extern int opterr; +extern int optind; +extern int optopt; +extern int optreset; +extern char *optarg; + +#include +#include +#include + +#define _SC_PAGESIZE 11 +#define _SC_PAGE_SIZE _SC_PAGESIZE +#define _SC_NPROCESSORS_ONLN 15 +#define _SC_PHYS_PAGES 500 + +#define X_OK 1 + +size_t strlcpy(register char *s, register const char *t, register size_t n); + +size_t strlcat(register char *s, register const char *t, register size_t n); + +ssize_t getline(char **linep, size_t *linecapp, FILE *stream); + +// int pread_win(HANDLE h, void *buf, size_t nbyte, off_t offset); +int pipe(int fildes[2]); +char *realpath(const char *file_name, char *resolved_name); +int usleep(__int64 usec); +int vasprintf(char **strp, const char *fmt, va_list ap); +int asprintf(char **strp, const char *fmt, ...); +int strncasecmp(const char *s1, const char *s2, size_t n); +int readlink(const char *path, char *buf, size_t bufsize); +const char *getexecname(void); +uint64_t geteuid(void); + +struct zfs_cmd; +int mkstemp(char *tmpl); +int64_t gethrtime(void); +struct timezone; +int gettimeofday(struct timeval *tp, struct timezone *tzp); +void flockfile(FILE *file); +void funlockfile(FILE *file); +unsigned long gethostid(void); +char *strndup(const char *src, size_t size); +int setrlimit(int resource, const struct rlimit *rlp); + +struct group *getgrgid(uint64_t gid); +struct passwd *getpwuid(uint64_t uid); +void syslog(int priority, const char *message, ...); +void closelog(void); + +int unmount(const char *dir, int flags); + +#endif /* _LIBSPL_WINDOWS_UNISTD_H */ diff --git a/lib/libspl/include/os/windows/uuid/uuid.h b/lib/libspl/include/os/windows/uuid/uuid.h new file mode 100644 index 000000000000..7d0b6add494c --- /dev/null +++ b/lib/libspl/include/os/windows/uuid/uuid.h @@ -0,0 +1,108 @@ +/* + * Public include file for the UUID library + * + * Copyright (C) 1996, 1997, 1998 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#ifndef _UUID_UUID_H +#define _UUID_UUID_H + +#include +#ifndef _WIN32 +#include +#endif +#include + +typedef unsigned char uuid_t[16]; + +/* UUID Variant definitions */ +#define UUID_VARIANT_NCS 0 +#define UUID_VARIANT_DCE 1 +#define UUID_VARIANT_MICROSOFT 2 +#define UUID_VARIANT_OTHER 3 + +/* UUID Type definitions */ +#define UUID_TYPE_DCE_TIME 1 +#define UUID_TYPE_DCE_RANDOM 4 + +/* Allow UUID constants to be defined */ +#ifdef __GNUC__ +#define UUID_DEFINE(name, u0, u1, u2, u3, u4, u5, u6, u7, u8, u9, u10, \ + u11, u12, u13, u14, u15) \ + static const uuid_t name __attribute__((unused)) = {u0, u1, u2, u3, \ + u4, u5, u6, u7, u8, u9, u10, u11, u12, u13, u14, u15} +#else +#define UUID_DEFINE(name, u0, u1, u2, u3, u4, u5, u6, u7, u8, u9, u10, \ + u11, u12, u13, u14, u15) \ + static const uuid_t name = {u0, u1, u2, u3, u4, u5, u6, u7, u8, u9, \ + u10, u11, u12, u13, u14, u15} +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +/* clear.c */ +void uuid_clear(uuid_t uu); + +/* compare.c */ +int uuid_compare(const uuid_t uu1, const uuid_t uu2); + +/* copy.c */ +void uuid_copy(uuid_t dst, const uuid_t src); + +/* gen_uuid.c */ +void uuid_generate(uuid_t out); +void uuid_generate_random(uuid_t out); +void uuid_generate_time(uuid_t out); +int uuid_generate_time_safe(uuid_t out); + +/* isnull.c */ +int uuid_is_null(const uuid_t uu); + +/* parse.c */ +int uuid_parse(const char *in, uuid_t uu); + +/* unparse.c */ +void uuid_unparse(const uuid_t uu, char *out); +void uuid_unparse_lower(const uuid_t uu, char *out); +void uuid_unparse_upper(const uuid_t uu, char *out); + +/* uuid_time.c */ +time_t uuid_time(const uuid_t uu, struct timeval *ret_tv); +int uuid_type(const uuid_t uu); +int uuid_variant(const uuid_t uu); + +#ifdef __cplusplus +} +#endif + +#endif /* _UUID_UUID_H */ diff --git a/lib/libspl/include/os/windows/wosix.h b/lib/libspl/include/os/windows/wosix.h new file mode 100644 index 000000000000..1c56458da299 --- /dev/null +++ b/lib/libspl/include/os/windows/wosix.h @@ -0,0 +1,168 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright(c) 2019 Jorgen Lundman + */ + +#ifndef WOSIX_HEADER +#define WOSIX_HEADER + +/* + * Replace all the normal POSIX calls; open, read, write, close, lseek, fstat + * As we have to use HANDLEs to open devices, we add a shim-layer to handle + * int fd and the change in underlying API calls. + * First, include the header that defines them in Windows. + */ +#include +#include +#include +#include + +#define HTOI(H) ((int)(unsigned __int64)(H)) +#define ITOH(I) ((HANDLE)(unsigned __int64)(I)) + +/* keep the struct type before we #define */ +/* We only use _stat64 in Windows, no reason to mix */ +struct wosix_stat { + union { + struct _stat64; + }; +}; + +struct stat64 { + union { + struct _stat64; + }; +}; + +#define readdir64 readdir +#define dirent64 dirent +#define statfs64 statfs + + +extern int wosix_fsync(int fd); +extern int wosix_open(const char *path, int oflag, ...); +extern int wosix_close(int fd); +struct zfs_iocparm; +extern int wosix_ioctl(int fd, unsigned long request, struct zfs_iocparm *zc); +extern int wosix_read(int fd, void *data, uint32_t len); +extern int wosix_write(int fd, const void *data, uint32_t len); +extern int wosix_isatty(int fd); +extern int wosix_mkdir(const char *path, mode_t mode); +extern int wosix_pwrite(int fd, const void *buf, size_t nbyte, off_t offset); +extern int wosix_pread(int fd, void *buf, size_t nbyte, off_t offset); +extern int wosix_stat(char *path, struct _stat64 *st); +extern int wosix_fstat(int fd, struct _stat64 *st); +extern int wosix_fstat_blk(int fd, struct _stat64 *st); +extern uint64_t wosix_lseek(int fd, uint64_t offset, int seek); +extern int wosix_fdatasync(int fd); +extern int wosix_ftruncate(int fd, off_t length); +extern int wosix_socketpair(int domain, int type, int protocol, + int socket_vector[2]); +extern int wosix_dup2(int fildes, int fildes2); +extern int wosix_pipe(int fildes[2]); +extern void *wosix_mmap(void *addr, size_t len, int prot, int flags, + int fildes, off_t off); +extern int wosix_munmap(void *addr, size_t len); + +#define wosix_fileno(X) (_get_osfhandle(_fileno((X)))) + +extern FILE *wosix_fdopen(int fildes, const char *mode); +extern FILE *wosix_freopen(const char *path, const char *mode, FILE *stream); + +/* Technically not needed, but handle mode */ +extern FILE *wosix_fopen(const char *name, const char *mode); + +/* + * Thin wrapper for the POSIX IO calls, to translate to HANDLEs + * + * Currently it "hopes" that HANDLE value will fit in type "int". + * This could be improved in future. + */ +#undef open +#define open wosix_open +#undef openat +#define openat wosix_openat +#undef open64 +#define open64 wosix_open +#undef close +#define close wosix_close +#undef ioctl +#define ioctl wosix_ioctl +#undef lseek +#define lseek wosix_lseek +#undef fsync +#define fsync wosix_fsync +#undef read +#define read wosix_read +#undef write +#define write wosix_write +#undef fileno +#define fileno wosix_fileno +#undef isatty +#define isatty wosix_isatty +#undef mkdir +#define mkdir wosix_mkdir +#undef pread +#define pread wosix_pread +#define pread64 wosix_pread +#undef pwrite +#define pwrite wosix_pwrite +#define pwrite64 wosix_pwrite +#undef fstat +#define fstat wosix_fstat +#undef lstat +#define lstat wosix_lstat +#undef lstat64 +#define lstat64 wosix_lstat +#undef _fstat64 +#define _fstat64 wosix_fstat +#undef fstat64 +#define fstat64 wosix_fstat +#undef fstat_blk +#define fstat_blk wosix_fstat_blk +#undef fstat64_blk +#define fstat64_blk wosix_fstat_blk +#undef stat +#define stat wosix_stat +#undef stat64 +#define stat64 wosix_stat +#undef fdatasync +#define fdatasync wosix_fdatasync +#undef ftruncate +#define ftruncate wosix_ftruncate +#undef socketpair +#define socketpair wosix_socketpair +#undef fdopen +#define fdopen wosix_fdopen +#undef freopen +#define freopen wosix_freopen +// Alas, edonr.c has a member named "pipe". +#ifndef _SYS_EDONR_H_ +#undef pipe +#define pipe wosix_pipe +#endif +#define pipe2(X, Y) wosix_pipe(X) +#define mmap wosix_mmap +#define munmap wosix_munmap +#define fopen wosix_fopen +#endif /* WOSIX_HEADER */ diff --git a/lib/libspl/include/rpc/xdr.h b/lib/libspl/include/rpc/xdr.h index 882b72456c21..9aeeaf0e5077 100644 --- a/lib/libspl/include/rpc/xdr.h +++ b/lib/libspl/include/rpc/xdr.h @@ -32,7 +32,9 @@ #ifndef LIBSPL_RPC_XDR_H #define LIBSPL_RPC_XDR_H +#ifndef _WIN32 #include_next +#endif #ifdef xdr_control /* if e.g. using tirpc */ #undef xdr_control diff --git a/lib/libspl/include/sys/debug.h b/lib/libspl/include/sys/debug.h index ef3f0afb68c3..6f8f564aa7b9 100644 --- a/lib/libspl/include/sys/debug.h +++ b/lib/libspl/include/sys/debug.h @@ -29,12 +29,33 @@ #include +#if defined(_MSC_VER) && !defined(__clang__) + +#ifndef __printflike +#define __printflike(x, y) +#endif + +#ifndef __printf0like +#define __printf0like(x, y) +#endif + +#ifndef __maybe_unused +#define __maybe_unused +#endif + +#else + #ifndef __printflike #define __printflike(x, y) __attribute__((__format__(__printf__, x, y))) #endif +#ifndef __printf0like +#define __printf0like(x, y) __attribute__((__format__(__printf0__, x, y))) +#endif + #ifndef __maybe_unused #define __maybe_unused __attribute__((unused)) #endif #endif +#endif diff --git a/lib/libspl/include/sys/uio.h b/lib/libspl/include/sys/uio.h index e9e21819d4f8..bcbf7a36b712 100644 --- a/lib/libspl/include/sys/uio.h +++ b/lib/libspl/include/sys/uio.h @@ -41,7 +41,10 @@ #define _LIBSPL_SYS_UIO_H #include + +#ifndef _WIN32 #include_next +#endif #ifdef __APPLE__ #include @@ -50,7 +53,7 @@ #include typedef struct iovec iovec_t; -#if defined(__linux__) || defined(__APPLE__) +#if defined(__linux__) || defined(__APPLE__) || defined(_WIN32) typedef enum zfs_uio_rw { UIO_READ = 0, UIO_WRITE = 1, diff --git a/lib/libspl/os/windows/gethostid.c b/lib/libspl/os/windows/gethostid.c new file mode 100644 index 000000000000..e62256c9cc00 --- /dev/null +++ b/lib/libspl/os/windows/gethostid.c @@ -0,0 +1,33 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2020, Jorgen Lundman + */ + +#include +#include +#include + +unsigned long +get_system_hostid(void) +{ + return (gethostid()); +} diff --git a/lib/libspl/os/windows/getmntany.c b/lib/libspl/os/windows/getmntany.c new file mode 100644 index 000000000000..c2a89ebaa548 --- /dev/null +++ b/lib/libspl/os/windows/getmntany.c @@ -0,0 +1,640 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2005 Sun Microsystems, Inc. All rights reserved. + * Copyright 2006 Ricardo Correia. All rights reserved. + * Use is subject to license terms. + * Copyright(c) 2017 Jorgen Lundman + */ + +/* Copyright (c) 1988 AT&T */ +/* All Rights Reserved */ +#include +#include +#include +#include /* for isspace() */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#ifdef _WIN32 +#include +#endif + +#include + +#include +#include +#include + +#define DIFF(xx) ((mrefp->xx != NULL) && \ + (mgetp->xx == NULL || strcmp(mrefp->xx, mgetp->xx) != 0)) + +static struct statfs *gsfs = NULL; +static int allfs = 0; +static dllinit = 1; + +NTSTATUS(__stdcall *NtFsControlFile)( + HANDLE fileHandle, + HANDLE event, + PIO_APC_ROUTINE apcRoutine, + PVOID apcContext, + PIO_STATUS_BLOCK ioStatusBlock, + ULONG fsControlCode, + PVOID inputBuffer, + ULONG inputBufferLength, + PVOID outputBuffer, + ULONG outputBufferLength +); + + +typedef struct _MOUNTDEV_NAME +{ + USHORT NameLength; + WCHAR Name[1]; +} MOUNTDEV_NAME, * PMOUNTDEV_NAME; +#define IOCTL_MOUNTDEV_QUERY_DEVICE_NAME 0x004d0008 +typedef struct _MOUNTDEV_UNIQUE_ID +{ + USHORT UniqueIdLength; + UCHAR UniqueId[1]; +} MOUNTDEV_UNIQUE_ID; +typedef MOUNTDEV_UNIQUE_ID* PMOUNTDEV_UNIQUE_ID; +#define IOCTL_MOUNTDEV_QUERY_UNIQUE_ID 0x4d0000 + + +/* + * We will also query the extended filesystem capabilities API, to lookup + * other mount options, for example, XATTR. We can not use the MNTNOUSERXATTR + * option due to VFS rejecting with EACCESS. + */ + +static int +chdir_block_begin(int newroot_fd) +{ + int cwdfd, error; + + cwdfd = open(".", O_RDONLY /* | O_DIRECTORY */); + if (cwdfd == -1) + return (-1); + +// if (fchdir(newroot_fd) == -1) { +// error = errno; +// (void) close(cwdfd); +// errno = error; +// return (-1); +// } + return (cwdfd); +} + +static void +chdir_block_end(int cwdfd) +{ + int error = errno; +// (void) fchdir(cwdfd); + (void) close(cwdfd); + errno = error; +} + +int +openat64(int dirfd, const char *path, int flags, ...) +{ + int cwdfd, filefd; + + if ((cwdfd = chdir_block_begin(dirfd)) == -1) + return (-1); + + if ((flags & O_CREAT) != 0) { + va_list ap; + int mode; + + va_start(ap, flags); + mode = va_arg(ap, int); + va_end(ap); + + filefd = open(path, flags, mode); + } else + filefd = open(path, flags); + + chdir_block_end(cwdfd); + return (filefd); +} + +int +fstatat64(int dirfd, const char *path, struct _stat64 *statbuf, int flag) +{ + int cwdfd, error; + + if ((cwdfd = chdir_block_begin(dirfd)) == -1) + return (-1); + + // if (flag == AT_SYMLINK_NOFOLLOW) + // error = lstat(path, statbuf); + // else + error = _stat64(path, statbuf); + + chdir_block_end(cwdfd); + return (error); +} + + +static char * +mntopt(char **p) +{ + char *cp = *p; + char *retstr; + + while (*cp && isspace(*cp)) + cp++; + + retstr = cp; + while (*cp && *cp != ',') + cp++; + + if (*cp) { + *cp = '\0'; + cp++; + } + + *p = cp; + return (retstr); +} + +char * +hasmntopt(struct mnttab *mnt, char *opt) +{ + char tmpopts[256]; + char *f, *opts = tmpopts; + + if (mnt->mnt_mntopts == NULL) + return (NULL); + (void) strlcpy(opts, mnt->mnt_mntopts, 256); + f = mntopt(&opts); + for (; *f; f = mntopt(&opts)) { + if (strncmp(opt, f, strlen(opt)) == 0) + return (f - tmpopts + mnt->mnt_mntopts); + } + return (NULL); +} + +static void +optadd(char *mntopts, size_t size, const char *opt) +{ + + if (mntopts[0] != '\0') + strlcat(mntopts, ",", size); + strlcat(mntopts, opt, size); +} + +void +statfs2mnttab(struct statfs *sfs, struct mnttab *mp) +{ + static char mntopts[MNTMAXSTR]; + long flags; + + mntopts[0] = '\0'; + + flags = sfs->f_flags; +#define OPTADD(opt) optadd(mntopts, sizeof (mntopts), (opt)) + if (flags & MNT_RDONLY) + OPTADD(MNTOPT_RO); + else + OPTADD(MNTOPT_RW); + + if (flags & MNT_UPDATE) + OPTADD(MNTOPT_REMOUNT); + if (flags & MNT_NOATIME) + OPTADD(MNTOPT_NOATIME); + else + OPTADD(MNTOPT_ATIME); +#if 0 + { + struct attrBufS attrBuf; + attrlist_t attrList; + + memset(&attrList, 0, sizeof (attrList)); + attrList.bitmapcount = ATTR_BIT_MAP_COUNT; + attrList.volattr = ATTR_VOL_INFO|ATTR_VOL_CAPABILITIES; + + if (getattrlist(sfs->f_mntonname, &attrList, &attrBuf, + sizeof (attrBuf), 0) == 0) { + + if (attrBuf.caps[VOL_CAPABILITIES_INTERFACES] & + VOL_CAP_INT_EXTENDED_ATTR) { + OPTADD(MNTOPT_XATTR); + } else { + OPTADD(MNTOPT_NOXATTR); + } // If EXTENDED + } // if getattrlist + } +#endif + if (flags & MNT_NOEXEC) + OPTADD(MNTOPT_NOEXEC); + else + OPTADD(MNTOPT_EXEC); + if (flags & MNT_NODEV) + OPTADD(MNTOPT_NODEVICES); + else + OPTADD(MNTOPT_DEVICES); +// if (flags & MNT_DONTBROWSE) +// OPTADD(MNTOPT_NOBROWSE); +// else +// OPTADD(MNTOPT_BROWSE); +// if (flags & MNT_IGNORE_OWNERSHIP) +// OPTADD(MNTOPT_NOOWNERS); +// else +// OPTADD(MNTOPT_OWNERS); + +#undef OPTADD + + mp->mnt_special = sfs->f_mntfromname; + mp->mnt_mountp = sfs->f_mntonname; + mp->mnt_fstype = sfs->f_fstypename; + mp->mnt_mntopts = mntopts; + mp->mnt_fssubtype = sfs->f_fssubtype; +} + +void +DisplayVolumePaths(char *VolumeName, char *out, int len) +{ + DWORD CharCount = MAX_PATH + 1; + char *Names = NULL; + char *NameIdx = NULL; + BOOL Success = FALSE; + + for (;;) { + // + // Allocate a buffer to hold the paths. + Names = (char *)malloc(CharCount); + + if (!Names) + return; + + // + // Obtain all of the paths + // for this volume. + Success = GetVolumePathNamesForVolumeName( + VolumeName, Names, CharCount, &CharCount); + + if (Success) break; + + if (GetLastError() != ERROR_MORE_DATA) break; + + // + // Try again with the + // new suggested size. + free(Names); + Names = NULL; + } + + if (Success) { + // + // Display the various paths. + for (NameIdx = Names; + NameIdx[0] != '\0'; + NameIdx += strlen(NameIdx) + 1) { + // printf(" %s", NameIdx); + snprintf(out, len, "%s%s", out, NameIdx); + } + // printf("\n"); + } + + if (Names != NULL) { + free(Names); + Names = NULL; + } +} + +static void +backslash2slash(char *s) +{ + while (*s != 0) { + if (*s == '\\') + *s = '/'; + s++; + } +} + +int +getfsstat(struct statfs *buf, int bufsize, int flags) +{ + char name[256]; + HANDLE vh; + int count = 0; + MOUNTDEV_UNIQUE_ID *UID = NULL; + + // If buf is NULL, return number of entries + vh = FindFirstVolume(name, sizeof (name)); + if (vh == INVALID_HANDLE_VALUE) + return (-1); + + do { + char *s = name; + fsctl_zfs_volume_mountpoint_t *fzvm = NULL; + + // Still room in out buffer? + if (buf && (bufsize < sizeof (struct statfs))) + break; + + + // We must skip the "\\?\" start + if (s[0] == '\\' && + s[1] == '\\' && + s[2] == '?' && + s[3] == '\\') + s = &s[4]; + // We must eat the final "\\" + int trail = strlen(name) - 1; + if (name[trail] == '\\') + name[trail] = 0; + + // Query DOS + char DeviceName[256], driveletter[256] = ""; + int CharCount; + CharCount = QueryDosDevice(s, DeviceName, sizeof (DeviceName)); + + // Restore trailing "\\" + if (name[trail] == 0) + name[trail] = '\\'; + + DisplayVolumePaths(name, driveletter, sizeof (driveletter)); + + // Open DeviceName, and query it for dataset name + HANDLE h; + + name[2] = '.'; // "\\?\" -> "\\.\" + + // We open the devices returned; like + // "'\\.\Volume{0b1bb601-af0b-32e8-a1d2-54c167af6277}'" and + // query its Unique ID, where we return the dataset name. "BOOM" + + h = CreateFile(name, 0, FILE_SHARE_READ | FILE_SHARE_WRITE, + NULL, OPEN_EXISTING, 0, NULL); + + if (h != INVALID_HANDLE_VALUE) { + char *dataset; + char uidbuffer[1024]; + UID = uidbuffer; + DWORD Size; + BOOL gotname = FALSE; + + gotname = DeviceIoControl(h, + IOCTL_MOUNTDEV_QUERY_UNIQUE_ID, + NULL, 0, UID, sizeof (uidbuffer) - 1, &Size, NULL); + // printf("deviocon %d: namelen %d\n", status, + // UID->UniqueIdLength); + if (gotname) { + // Kernel doesn't null terminate + UID->UniqueId[UID->UniqueIdLength] = 0; + } else + UID = NULL; + + if (NtFsControlFile != NULL) { + IO_STATUS_BLOCK iosb; + long Status; + + Status = NtFsControlFile(h, NULL, NULL, NULL, + &iosb, FSCTL_ZFS_VOLUME_MOUNTPOINT, NULL, + 0, fzvm, 0); + if (Status == STATUS_BUFFER_TOO_SMALL) { + // we are not given size back here, + // because of METHOD BUFFERED? + iosb.Information = 1024; + fzvm = malloc(iosb.Information); + if (fzvm) { + Status = NtFsControlFile(h, + NULL, NULL, NULL, &iosb, + FSCTL_ZFS_VOLUME_MOUNTPOINT, + NULL, 0, + fzvm, iosb.Information); + if (Status == 0) { + // Always "\??\E:\..." + fzvm->buffer[fzvm->len / + sizeof (WCHAR)] = 0; + } else { + free(fzvm); + fzvm = NULL; + } + } + } + } + CloseHandle(h); + } + + // We found a mount here, add it. + if (buf) { + memset(buf, 0, sizeof (*buf)); + if (UID) { + // Look up mountpoint + strlcpy(buf->f_mntfromname, UID->UniqueId, + sizeof (buf->f_mntfromname)); + strlcpy(buf->f_fstypename, MNTTYPE_ZFS, + sizeof (buf->f_fstypename)); + // If we have a driveletter, use it + // otherwise, lookup mountpoint, and if + // no mountpoint, the device exists, but + // is not mounted. + if (strlen(driveletter) > 2) { + strlcpy(buf->f_mntonname, driveletter, + sizeof (buf->f_mntonname)); + } else { + if (fzvm) { + snprintf(buf->f_mntonname, + sizeof (buf->f_mntonname), + "%ws", + &fzvm->buffer[4]); + } else { + strlcpy(buf->f_mntonname, + UID->UniqueId, + sizeof (buf->f_mntonname)); + } + } + } else { + // Not ZFS - do we care? + strlcpy(buf->f_mntfromname, DeviceName, + sizeof (buf->f_mntfromname)); + strlcpy(buf->f_fstypename, "UKN", + sizeof (buf->f_fstypename)); + strlcpy(buf->f_mntonname, name, + sizeof (buf->f_mntonname)); + } + + backslash2slash(buf->f_mntfromname); + backslash2slash(buf->f_mntonname); + + if (fzvm) + free(fzvm); + + UID = NULL; + // If it is mounted, add node. + if (buf->f_mntonname[0] != 0) { + buf++; // Go to next struct. + bufsize -= sizeof (*buf); + } + } + count++; + + } while (FindNextVolume(vh, name, sizeof (name)) != 0); + FindVolumeClose(vh); + return (count); +} + + + +static int +statfs_init(void) +{ + struct statfs *sfs; + int error; + + if (dllinit) { + dllinit = 0; + NtFsControlFile = (void *)GetProcAddress( + GetModuleHandle("ntdll.dll"), "NtFsControlFile"); + } + + if (gsfs != NULL) { + free(gsfs); + gsfs = NULL; + } + allfs = getfsstat(NULL, 0, MNT_NOWAIT); + if (allfs == -1) + goto fail; + gsfs = malloc(sizeof (gsfs[0]) * allfs * 2); + if (gsfs == NULL) + goto fail; + allfs = getfsstat(gsfs, (long)(sizeof (gsfs[0]) * allfs * 2), + MNT_NOWAIT); + if (allfs == -1) + goto fail; + sfs = realloc(gsfs, allfs * sizeof (gsfs[0])); + if (sfs != NULL) + gsfs = sfs; + return (0); +fail: + error = errno; + if (gsfs != NULL) + free(gsfs); + gsfs = NULL; + allfs = 0; + return (error); +} + +int +getmntany(FILE *fd, struct mnttab *mgetp, struct mnttab *mrefp) +{ + int i, error; + + error = statfs_init(); + if (error != 0) + return (error); + + for (i = 0; i < allfs; i++) { + statfs2mnttab(&gsfs[i], mgetp); + if (mrefp->mnt_special != NULL && mgetp->mnt_special != NULL && + strcmp(mrefp->mnt_special, mgetp->mnt_special) != 0) { + continue; + } + if (mrefp->mnt_mountp != NULL && mgetp->mnt_mountp != NULL && + strcmp(mrefp->mnt_mountp, mgetp->mnt_mountp) != 0) { + continue; + } + if (mrefp->mnt_fstype != NULL && mgetp->mnt_fstype != NULL && + strcmp(mrefp->mnt_fstype, mgetp->mnt_fstype) != 0) { + continue; + } + return (0); + } + return (-1); +} + +int +getmntent(FILE *fp, struct mnttab *mp) +{ + static int index = -1; + int error = 0; + + if (index < 0) { + error = statfs_init(); + } + + if (error != 0) + return (error); + + index++; + + // If we have finished "reading" the mnttab, reset it to + // start from the beginning, and return EOF. + if (index >= allfs) { + index = -1; + return (-1); + } + + statfs2mnttab(&gsfs[index], mp); + return (0); +} + +int +getextmntent(const char *path, struct extmnttab *entry, struct stat64 *statbuf) +{ + struct statfs sfs; + + if (strlen(path) >= MAXPATHLEN) { + (void) fprintf(stderr, "invalid object; pathname too long\n"); + return (-1); + } + + if (stat64(path, statbuf) != 0) { + (void) fprintf(stderr, "cannot open '%s': %s\n", + path, strerror(errno)); + return (-1); + } + + if (statfs(path, &sfs) != 0) { + (void) fprintf(stderr, "%s: %s\n", path, + strerror(errno)); + return (-1); + } + statfs2mnttab(&sfs, (struct mnttab *)entry); + return (0); +} + +FILE * +setmntent(const char *filename, const char *type) +{ + FILE *ret; + + if (tmpfile_s(&ret) == 0) + return (ret); + return (NULL); +} + +void +endmntent(FILE *fd) +{ + fclose(fd); +} diff --git a/lib/libspl/os/windows/getopt.c b/lib/libspl/os/windows/getopt.c new file mode 100644 index 000000000000..dc8996ecf760 --- /dev/null +++ b/lib/libspl/os/windows/getopt.c @@ -0,0 +1,314 @@ +/* $NetBSD: getopt.c,v 1.10 1997/07/21 14:08:51 jtc Exp $ */ + +/* + * Copyright (c) 1987, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. All advertising materials mentioning features or use of this software + * must display the following acknowledgement: + * This product includes software developed by the University of + * California, Berkeley and its contributors. + * 4. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ +#if HAVE_CONFIG_H +#include +#endif + +#if defined(LIBC_SCCS) && !defined(lint) +#if 0 +static char sccsid[] = "from: @(#)getopt.c 8.2 (Berkeley) 4/2/94"; +#else +__RCSID("$NetBSD: getopt.c,v 1.10 1997/07/21 14:08:51 jtc Exp $"); +#endif +#endif /* LIBC_SCCS and not lint */ + +#include +#include +#include +#include +#include +#include + + +#ifdef __weak_alias +__weak_alias(getopt, _getopt); +#endif + + +int opterr = 1, /* if error message should be printed */ + optind = 1, /* index into parent argv vector */ + optopt, /* character checked for validity */ + optreset; /* reset getopt */ +char *optarg; /* argument associated with option */ + +#define BADCH (int)'?' +#define BADARG (int)':' +#define EMSG "" + +/* + * getopt -- + * Parse argc/argv argument vector. + */ +int +getopt( + int nargc, + char *const *nargv, + const char *ostr) +{ + static char *place = EMSG; /* option letter processing */ + char *oli; /* option letter list index */ + + if (optreset || !*place) { /* update scanning pointer */ + optreset = 0; + if (optind >= nargc || *(place = nargv[optind]) != '-') { + place = EMSG; + return (-1); + } + if (place[1] && *++place == '-') { /* found "--" */ + ++optind; + place = EMSG; + return (-1); + } + } /* option letter okay? */ + if ((optopt = (int)*place++) == (int)':' || + !(oli = strchr(ostr, optopt))) { + /* + * if the user didn't specify '-' as an option, + * assume it means -1. + */ + if (optopt == (int)'-') + return (-1); + if (!*place) + ++optind; + if (opterr && *ostr != ':') + (void) fprintf(stderr, + ": illegal option -- %c\n", optopt); + return (BADCH); + } + if (*++oli != ':') { /* don't need argument */ + optarg = NULL; + if (!*place) + ++optind; + } else { /* need an argument */ + if (*place) /* no white space */ + optarg = place; + else if (nargc <= ++optind) { /* no arg */ + place = EMSG; + if (*ostr == ':') + return (BADARG); + if (opterr) + (void) fprintf(stderr, + ": option requires an argument -- %c\n", + optopt); + return (BADCH); + } else /* white space */ + optarg = nargv[optind]; + place = EMSG; + ++optind; + } + return (optopt); /* dump back option letter */ +} + +int +getsubopt( + char **optionsp, + char *tokens[], + char **valuep) +{ + register char *s = *optionsp, *p; + register int i, optlen; + + *valuep = NULL; + if (*s == '\0') + return (-1); + p = strchr(s, ','); /* find next option */ + if (p == NULL) { + p = s + strlen(s); + } else { + *p++ = '\0'; /* mark end and point to next */ + } + *optionsp = p; /* point to next option */ + p = strchr(s, '='); /* find value */ + if (p == NULL) { + optlen = strlen(s); + *valuep = NULL; + } else { + optlen = p - s; + *valuep = ++p; + } + for (i = 0; tokens[i] != NULL; i++) { + if ((optlen == strlen(tokens[i])) && + (strncmp(s, tokens[i], optlen) == 0)) + return (i); + } + /* no match, point value at option and return error */ + *valuep = s; + return (-1); +} + +static struct getopt_private_state { + const char *optptr; + const char *last_optstring; + char *const *last_argv; +} pvt; + +static inline const char *option_matches(const char *arg_str, + const char *opt_name) +{ + while (*arg_str != '\0' && *arg_str != '=') { + if (*arg_str++ != *opt_name++) + return (NULL); + } + + if (*opt_name) + return (NULL); + + return (arg_str); +} + +int +getopt_long(int argc, char *const *argv, const char *optstring, + const struct option *longopts, int *longindex) +{ + const char *carg; + const char *osptr; + int opt; + + /* + * getopt() relies on a number of different global state + * variables, which can make this really confusing if there is + * more than one use of getopt() in the same program. This + * attempts to detect that situation by detecting if the + * "optstring" or "argv" argument have changed since last time + * we were called; if so, reinitialize the query state. + */ + + if (optstring != pvt.last_optstring || argv != pvt.last_argv || + optind < 1 || optind > argc) { + /* optind doesn't match the current query */ + pvt.last_optstring = optstring; + pvt.last_argv = argv; + optind = 1; + pvt.optptr = NULL; + } + + carg = argv[optind]; + + /* First, eliminate all non-option cases */ + + if (!carg || carg[0] != '-' || !carg[1]) + return (-1); + + if (carg[1] == '-') { + const struct option *lo; + const char *opt_end = NULL; + + optind++; + + /* Either it's a long option, or it's -- */ + if (!carg[2]) { + /* It's -- */ + return (-1); + } + + for (lo = longopts; lo->name; lo++) { + if ((opt_end = option_matches(carg + 2, lo->name))) + break; + } + if (!opt_end) + return ('?'); + + if (longindex) + *longindex = lo - longopts; + + if (*opt_end == '=') { + if (lo->has_arg) + optarg = (char *)opt_end + 1; + else + return ('?'); + } else if (lo->has_arg == 1) { + if (!(optarg = argv[optind])) + return ('?'); + optind++; + } + + if (lo->flag) { + *lo->flag = lo->val; + return (0); + } else { + return (lo->val); + } + } + + if ((uintptr_t)(pvt.optptr - carg) > (uintptr_t)strlen(carg)) { + /* Someone frobbed optind, change to new opt. */ + pvt.optptr = carg + 1; + } + + opt = *pvt.optptr++; + + if (opt != ':' && (osptr = strchr(optstring, opt))) { + if (osptr[1] == ':') { + if (*pvt.optptr) { + /* + * Argument-taking option with attached + * argument + */ + optarg = (char *)pvt.optptr; + optind++; + } else { + /* + * Argument-taking option with non-attached + * argument + */ + if (argv[optind + 1]) { + optarg = (char *)argv[optind + 1]; + optind += 2; + } else { + /* Missing argument */ + optind++; + return (optstring[0] == ':') + ? ':' : '?'; + } + } + return (opt); + } else { + /* + * Non-argument-taking option + * pvt.optptr will remember the exact position to + * resume at + */ + if (!*pvt.optptr) + optind++; + return (opt); + } + } else { + /* Unknown option */ + optopt = opt; + if (!*pvt.optptr) + optind++; + return ('?'); + } +} diff --git a/lib/libspl/os/windows/getoptl.c b/lib/libspl/os/windows/getoptl.c new file mode 100644 index 000000000000..4966ec335f78 --- /dev/null +++ b/lib/libspl/os/windows/getoptl.c @@ -0,0 +1,152 @@ + +#include +#include +#include +#include + +char *optarg; +int optind, opterr, optopt; +static struct getopt_private_state { + const char *optptr; + const char *last_optstring; + char *const *last_argv; +} pvt; + +static inline const char *option_matches(const char *arg_str, + const char *opt_name) +{ + while (*arg_str != '\0' && *arg_str != '=') { + if (*arg_str++ != *opt_name++) + return (NULL); + } + + if (*opt_name) + return (NULL); + + return (arg_str); +} + +int +getopt_long(int argc, char *const *argv, const char *optstring, + const struct option *longopts, int *longindex) +{ + const char *carg; + const char *osptr; + int opt; + + /* + * getopt() relies on a number of different global state + * variables, which can make this really confusing if there is + * more than one use of getopt() in the same program. This + * attempts to detect that situation by detecting if the + * "optstring" or "argv" argument have changed since last time + * we were called; if so, reinitialize the query state. + */ + + if (optstring != pvt.last_optstring || argv != pvt.last_argv || + optind < 1 || optind > argc) { + /* optind doesn't match the current query */ + pvt.last_optstring = optstring; + pvt.last_argv = argv; + optind = 1; + pvt.optptr = NULL; + } + + carg = argv[optind]; + + /* First, eliminate all non-option cases */ + + if (!carg || carg[0] != '-' || !carg[1]) + return (-1); + + if (carg[1] == '-') { + const struct option *lo; + const char *opt_end = NULL; + + optind++; + + /* Either it's a long option, or it's -- */ + if (!carg[2]) { + /* It's -- */ + return (-1); + } + + for (lo = longopts; lo->name; lo++) { + if ((opt_end = option_matches(carg + 2, lo->name))) + break; + } + if (!opt_end) + return ('?'); + + if (longindex) + *longindex = lo - longopts; + + if (*opt_end == '=') { + if (lo->has_arg) + optarg = (char *)opt_end + 1; + else + return ('?'); + } else if (lo->has_arg == 1) { + if (!(optarg = argv[optind])) + return ('?'); + optind++; + } + + if (lo->flag) { + *lo->flag = lo->val; + return (0); + } else { + return (lo->val); + } + } + + if ((uintptr_t)(pvt.optptr - carg) > (uintptr_t)strlen(carg)) { + /* Someone frobbed optind, change to new opt. */ + pvt.optptr = carg + 1; + } + + opt = *pvt.optptr++; + + if (opt != ':' && (osptr = strchr(optstring, opt))) { + if (osptr[1] == ':') { + if (*pvt.optptr) { + /* + * Argument-taking option with attached + * argument + */ + optarg = (char *)pvt.optptr; + optind++; + } else { + /* + * Argument-taking option with non-attached + * argument + */ + if (argv[optind + 1]) { + optarg = (char *)argv[optind + 1]; + optind += 2; + } else { + /* Missing argument */ + optind++; + return (optstring[0] == ':') + ? ':' : '?'; + } + } + return (opt); + } else { + /* + * Non-argument-taking option + * pvt.optptr will remember the exact position to + * resume at + */ + if (!*pvt.optptr) + optind++; + return (opt); + } + } else { + /* Unknown option */ + optopt = opt; + if (!*pvt.optptr) + optind++; + return ('?'); + } +} diff --git a/lib/libspl/os/windows/page.c b/lib/libspl/os/windows/page.c new file mode 100644 index 000000000000..4ca46dad2f29 --- /dev/null +++ b/lib/libspl/os/windows/page.c @@ -0,0 +1,41 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#define _WINDOWS_MEAN_AND_LEAN +#include + +size_t pagesize = 0; + +size_t +spl_pagesize(void) +{ + if (pagesize == 0) { + SYSTEM_INFO sys; + GetSystemInfo(&sys); + pagesize = sys.dwPageSize; + } + return (pagesize); +} diff --git a/lib/libspl/os/windows/posix.c b/lib/libspl/os/windows/posix.c new file mode 100644 index 000000000000..467f727d24d2 --- /dev/null +++ b/lib/libspl/os/windows/posix.c @@ -0,0 +1,1825 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2017 Jorgen Lundman +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* Magic instruction to compiler to add library */ +#pragma comment(lib, "ws2_32.lib") + +void +clock_gettime(clock_type_t t, struct timespec *ts) +{ + LARGE_INTEGER time; + LARGE_INTEGER frequency; + FILETIME ft; + + switch (t) { + case CLOCK_MONOTONIC: + QueryPerformanceCounter(&time); + QueryPerformanceFrequency(&frequency); + ts->tv_sec = time.QuadPart / frequency.QuadPart; + ts->tv_nsec = 100*(long)(time.QuadPart % frequency.QuadPart); + break; + + case CLOCK_REALTIME: + GetSystemTimeAsFileTime(&ft); + time.LowPart = ft.dwLowDateTime; + time.HighPart = ft.dwHighDateTime; + time.QuadPart -= 116444736000000000; + ts->tv_sec = (long)(time.QuadPart / 10000000UL); + ts->tv_nsec = 100*(long)(time.QuadPart % 10000000UL); + break; + default: + ASSERT(0); + } +} + +void +gethrestime(inode_timespec_t *ts) +{ + struct timeval tv; + (void) gettimeofday(&tv, NULL); + ts->tv_sec = tv.tv_sec; + ts->tv_nsec = tv.tv_usec * NSEC_PER_USEC; +} + +uint64_t +gethrestime_sec(void) +{ + struct timeval tv; + (void) gettimeofday(&tv, NULL); + return (tv.tv_sec); +} + +hrtime_t +gethrtime(void) +{ + struct timespec ts; + (void) clock_gettime(CLOCK_MONOTONIC, &ts); + return ((((uint64_t)ts.tv_sec) * NANOSEC) + ts.tv_nsec); +} + +int +posix_memalign(void **memptr, size_t alignment, size_t size) +{ + void *ptr; + ptr = _aligned_malloc(size, alignment); + if (ptr == NULL) + return (ENOMEM); + *memptr = ptr; + return (0); +} + +const char * +getexecname(void) +{ + __declspec(thread) static char execname[32767 + 1]; + GetModuleFileNameA(NULL, execname, sizeof (execname)); + return (execname); +} + +struct passwd * +getpwnam(const char *login) +{ + return (NULL); +} + +struct passwd * +getgrnam(const char *group) +{ + return (NULL); +} + +struct tm * +localtime_r(const time_t *clock, struct tm *result) +{ + if (localtime_s(result, clock) == 0) + return (result); + // To avoid the ASSERT and abort(), make tm be something valid + memset(result, 0, sizeof (*result)); + result->tm_mday = 1; + return (NULL); +} + +char * +strsep(char **stringp, const char *delim) +{ + char *s; + const char *spanp; + int c, sc; + char *tok; + + if ((s = *stringp) == NULL) + return (NULL); + for (tok = s; /* empty */; ) { + c = *s++; + spanp = delim; + do { + if ((sc = *spanp++) == c) { + if (c == 0) + s = NULL; + else + s[-1] = 0; + *stringp = s; + return (tok); + } + } while (sc != 0); + } + /* NOTREACHED */ +} + +char * +realpath(const char *file_name, char *resolved_name) +{ + DWORD ret; + // If resolved_name is NULL, we allocate space. Otherwise we assume + // PATH_MAX - but pretty sure this style isn't used in ZFS + if (resolved_name == NULL) + resolved_name = malloc(PATH_MAX); + if (resolved_name == NULL) + return (NULL); + ret = GetFullPathName(file_name, PATH_MAX, resolved_name, NULL); + if (ret == 0) + return (NULL); + + return (resolved_name); +} + +int +statfs(const char *path, struct statfs *buf) +{ + ULARGE_INTEGER lpFreeBytesAvailable; + ULARGE_INTEGER lpTotalNumberOfBytes; + ULARGE_INTEGER lpTotalNumberOfFreeBytes; + uint64_t lbsize; + +#if 1 + if (GetDiskFreeSpaceEx(path, + &lpFreeBytesAvailable, + &lpTotalNumberOfBytes, + &lpTotalNumberOfFreeBytes)) { + return (-1); + } +#endif + + DISK_GEOMETRY_EX geometry_ex; + HANDLE handle; + DWORD len; + + handle = wosix_open(path, O_RDONLY | O_BINARY); + + if (!DeviceIoControl(handle, IOCTL_DISK_GET_DRIVE_GEOMETRY_EX, NULL, 0, + &geometry_ex, sizeof (geometry_ex), &len, NULL)) + return (-1); + wosix_close(handle); + lbsize = (uint_t)geometry_ex.Geometry.BytesPerSector; + + buf->f_bsize = lbsize; + buf->f_blocks = lpTotalNumberOfBytes.QuadPart / lbsize; + buf->f_bfree = lpTotalNumberOfFreeBytes.QuadPart / lbsize; + buf->f_bavail = lpTotalNumberOfFreeBytes.QuadPart / lbsize; + buf->f_type = 0; + strcpy(buf->f_fstypename, "fixme"); + strcpy(buf->f_mntonname, "fixme_to"); + strcpy(buf->f_mntfromname, "fixme_from"); + + return (0); +} + + +static const char letters[] = +"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"; +int +mkstemp(char *tmpl) +{ + int len; + char *XXXXXX; + static unsigned long long value; + unsigned long long random_time_bits; + unsigned int count; + int fd = -1; + int save_errno = errno; + +#define ATTEMPTS_MIN (62 * 62 * 62) + +#if ATTEMPTS_MIN < TMP_MAX + unsigned int attempts = TMP_MAX; +#else + unsigned int attempts = ATTEMPTS_MIN; +#endif + + len = strlen(tmpl); + if (len < 6 || strcmp(&tmpl[len - 6], "XXXXXX")) { + errno = EINVAL; + return (-1); + } + + XXXXXX = &tmpl[len - 6]; + + { + SYSTEMTIME stNow; + FILETIME ftNow; + + // get system time + GetSystemTime(&stNow); + stNow.wMilliseconds = 500; + if (!SystemTimeToFileTime(&stNow, &ftNow)) { + errno = -1; + return (-1); + } + + random_time_bits = + (((unsigned long long)ftNow.dwHighDateTime << 32) | + (unsigned long long)ftNow.dwLowDateTime); + } + value += random_time_bits ^ (unsigned long long)GetCurrentThreadId(); + + for (count = 0; count < attempts; value += 7777, ++count) { + unsigned long long v = value; + + /* Fill in the random bits. */ + XXXXXX[0] = letters[v % 62]; + v /= 62; + XXXXXX[1] = letters[v % 62]; + v /= 62; + XXXXXX[2] = letters[v % 62]; + v /= 62; + XXXXXX[3] = letters[v % 62]; + v /= 62; + XXXXXX[4] = letters[v % 62]; + v /= 62; + XXXXXX[5] = letters[v % 62]; + + fd = open(tmpl, + O_RDWR | O_CREAT | O_EXCL, _S_IREAD | _S_IWRITE); + if (fd >= 0) { + errno = save_errno; + return (fd); + } else if (errno != EEXIST) + return (-1); + } + + /* We got out of the loop because we ran out of combinations to try. */ + errno = EEXIST; + return (-1); +} + +int +readlink(const char *path, char *buf, size_t bufsize) +{ + return (-1); +} + +int +usleep(__int64 usec) +{ + HANDLE timer; + LARGE_INTEGER ft; + + ft.QuadPart = -(10 * usec); + + timer = CreateWaitableTimer(NULL, TRUE, NULL); + SetWaitableTimer(timer, &ft, 0, NULL, NULL, 0); + WaitForSingleObject(timer, INFINITE); + CloseHandle(timer); + return (0); +} + +int +nanosleep(const struct timespec *rqtp, struct timespec *rmtp) +{ + /* Declarations */ + HANDLE timer; /* Timer handle */ + LARGE_INTEGER li; /* Time defintion */ + /* Create timer */ + + // Negative means relative time, 100 nanosecs on Windows. + li.QuadPart = -((SEC2NSEC(rqtp->tv_sec) + rqtp->tv_nsec) / 100ULL); + + if (!(timer = CreateWaitableTimer(NULL, TRUE, NULL))) + return (FALSE); + + /* Set timer properties */ + if (!SetWaitableTimer(timer, &li, 0, NULL, NULL, FALSE)) { + CloseHandle(timer); + return (FALSE); + } + /* Start & wait for timer */ + WaitForSingleObject(timer, INFINITE); + /* Clean resources */ + CloseHandle(timer); + /* Slept without problems */ + return (0); +} + +int +strncasecmp(const char *s1, const char *s2, size_t n) +{ + if (n == 0) + return (0); + + while (n-- != 0 && tolower(*s1) == tolower(*s2)) { + if (n == 0 || *s1 == '\0' || *s2 == '\0') + break; + s1++; + s2++; + } + + return (tolower(*(unsigned char *)s1) - tolower(*(unsigned char *)s2)); +} + +#define DIRNAME 0 +#define BASENAME 1 + +#define M_FSDELIM(c) ((c) == '/' || (c) == '\\') +#define M_DRDELIM(c) (0) + +static char curdir[] = "."; +static char * +basedir(char *arg, int type) +{ + register char *cp, *path; + + if (arg == (char *)0 || *arg == '\0' || + (*arg == '.' && (arg[1] == '\0' || + (type == DIRNAME && arg[1] == '.' && arg[2] == '\0')))) + return (curdir); /* arg NULL, empty, ".", or ".." in DIRNAME */ + + if (M_DRDELIM(arg[1])) /* drive-specified pathnames */ + path = arg + 2; + else + path = arg; + + if (path[1] == '\0'&&M_FSDELIM(*path)) /* "/", or drive analog */ + return (arg); + + cp = strchr(path, '\0'); + cp--; + + while (cp != path && M_FSDELIM(*cp)) + *(cp--) = '\0'; + + for (; cp > path && !M_FSDELIM(*cp); cp--) + ; + + if (!M_FSDELIM(*cp)) + if (type == DIRNAME && path != arg) { + *path = '\0'; + return (arg); /* curdir on the specified drive */ + } else + return ((type == DIRNAME) ? curdir : path); + else if (cp == path && type == DIRNAME) { + cp[1] = '\0'; + return (arg); /* root directory involved */ + } else if (cp == path && cp[1] == '\0') + return (arg); + else if (type == BASENAME) + return (++cp); + *cp = '\0'; + return (arg); +} + +char * +dirname(char *arg) +{ + return (basedir(arg, DIRNAME)); +} + +char * +basename(char *arg) +{ + return (basedir(arg, BASENAME)); +} + +char * +getIoctlAsString(int cmdNo) +{ + switch (cmdNo) { + case 0x800: return "ZFS_IOC_FIRST"; + case 0x801: return "ZFS_IOC_POOL_DESTROY"; + case 0x802: return "ZFS_IOC_POOL_IMPORT"; + case 0x803: return "ZFS_IOC_POOL_EXPORT"; + case 0x804: return "ZFS_IOC_POOL_CONFIGS"; + case 0x805: return "ZFS_IOC_POOL_STATS"; + case 0x806: return "ZFS_IOC_POOL_TRYIMPORT"; + case 0x807: return "ZFS_IOC_POOL_SCAN"; + case 0x808: return "ZFS_IOC_POOL_FREEZE"; + case 0x809: return "ZFS_IOC_POOL_UPGRADE"; + case 0x80a: return "ZFS_IOC_POOL_GET_HISTORY"; + case 0x80b: return "ZFS_IOC_VDEV_ADD"; + case 0x80c: return "ZFS_IOC_VDEV_REMOVE"; + case 0x80d: return "ZFS_IOC_VDEV_SET_STATE"; + case 0x80e: return "ZFS_IOC_VDEV_ATTACH"; + case 0x80f: return "ZFS_IOC_VDEV_DETACH"; + case 0x810: return "ZFS_IOC_VDEV_SETPATH"; + case 0x811: return "ZFS_IOC_VDEV_SETFRU"; + case 0x812: return "ZFS_IOC_OBJSET_STATS"; + case 0x813: return "ZFS_IOC_OBJSET_ZPLPROPS"; + case 0x814: return "ZFS_IOC_DATASET_LIST_NEXT"; + case 0x815: return "ZFS_IOC_SNAPSHOT_LIST_NEXT"; + case 0x816: return "ZFS_IOC_SET_PROP"; + case 0x817: return "ZFS_IOC_CREATE"; + case 0x818: return "ZFS_IOC_DESTROY"; + case 0x819: return "ZFS_IOC_ROLLBACK"; + case 0x81a: return "ZFS_IOC_RENAME"; + case 0x81b: return "ZFS_IOC_RECV"; + case 0x81c: return "ZFS_IOC_SEND"; + case 0x81d: return "ZFS_IOC_INJECT_FAULT"; + case 0x81e: return "ZFS_IOC_CLEAR_FAULT"; + case 0x81f: return "ZFS_IOC_INJECT_LIST_NEXT"; + case 0x820: return "ZFS_IOC_ERROR_LOG"; + case 0x821: return "ZFS_IOC_CLEAR"; + case 0x822: return "ZFS_IOC_PROMOTE"; + case 0x823: return "ZFS_IOC_SNAPSHOT"; + case 0x824: return "ZFS_IOC_DSOBJ_TO_DSNAME"; + case 0x825: return "ZFS_IOC_OBJ_TO_PATH"; + case 0x826: return "ZFS_IOC_POOL_SET_PROPS"; + case 0x827: return "ZFS_IOC_POOL_GET_PROPS"; + case 0x828: return "ZFS_IOC_SET_FSACL"; + case 0x829: return "ZFS_IOC_GET_FSACL"; + case 0x82a: return "ZFS_IOC_SHARE"; + case 0x82b: return "ZFS_IOC_INHERIT_PROP"; + case 0x82c: return "ZFS_IOC_SMB_ACL"; + case 0x82d: return "ZFS_IOC_USERSPACE_ONE"; + case 0x82e: return "ZFS_IOC_USERSPACE_MANY"; + case 0x82f: return "ZFS_IOC_USERSPACE_UPGRADE"; + case 0x830: return "ZFS_IOC_HOLD"; + case 0x831: return "ZFS_IOC_RELEASE"; + case 0x832: return "ZFS_IOC_GET_HOLDS"; + case 0x833: return "ZFS_IOC_OBJSET_RECVD_PROPS"; + case 0x834: return "ZFS_IOC_VDEV_SPLIT"; + case 0x835: return "ZFS_IOC_NEXT_OBJ"; + case 0x836: return "ZFS_IOC_DIFF"; + case 0x837: return "ZFS_IOC_TMP_SNAPSHOT"; + case 0x838: return "ZFS_IOC_OBJ_TO_STATS"; + case 0x839: return "ZFS_IOC_SPACE_WRITTEN"; + case 0x83a: return "ZFS_IOC_SPACE_SNAPS"; + case 0x83b: return "ZFS_IOC_DESTROY_SNAPS"; + case 0x83c: return "ZFS_IOC_POOL_REGUID"; + case 0x83d: return "ZFS_IOC_POOL_REOPEN"; + case 0x83e: return "ZFS_IOC_SEND_PROGRESS"; + case 0x83f: return "ZFS_IOC_LOG_HISTORY"; + case 0x840: return "ZFS_IOC_SEND_NEW"; + case 0x841: return "ZFS_IOC_SEND_SPACE"; + case 0x842: return "ZFS_IOC_CLONE"; + case 0x843: return "ZFS_IOC_BOOKMARK"; + case 0x844: return "ZFS_IOC_GET_BOOKMARKS"; + case 0x845: return "ZFS_IOC_DESTROY_BOOKMARKS"; + case 0x846: return "ZFS_IOC_LOAD_KEY"; + case 0x847: return "ZFS_IOC_UNLOAD_KEY"; + case 0x848: return "ZFS_IOC_CHANGE_KEY"; + case 0x849: return "ZFS_IOC_REMAP"; + case 0x84a: return "ZFS_IOC_POOL_CHECKPOINT"; + case 0x84b: return "ZFS_IOC_POOL_DISCARD_CHECKPOINT"; + case 0x84c: return "ZFS_IOC_POOL_INITIALIZE"; + case 0x84d: return "ZFS_IOC_POOL_SYNC"; + case 0x84e: return "ZFS_IOC_CHANNEL_PROGRAM"; + case 0x84f: return "ZFS_IOC_TRIM"; + + case 0x880: return "ZFS_IOC_EVENTS_NEXT"; + case 0x881: return "ZFS_IOC_EVENTS_CLEAR"; + case 0x882: return "ZFS_IOC_EVENTS_SEEK"; + + case 0x8E0: return "ZFS_IOC_MOUNT"; + case 0x8E1: return "ZFS_IOC_UNMOUNT"; + case 0x8E2: return "ZFS_IOC_UNREGISTER_FS"; + + case 0x8E3: return "ZFS_IOC_LAST"; + default: return "unkown"; + } +} + + +int +vasprintf(char **strp, const char *fmt, va_list ap) +{ + int r = -1, size; + + size = _vscprintf(fmt, ap); + + if ((size >= 0) && (size < INT_MAX)) { + *strp = (char *)malloc(size + 1); + if (*strp) { + r = vsnprintf(*strp, size + 1, fmt, ap); + if ((r < 0) || (r > size)) { + r = -1; + free(*strp); + } + } + } else { + *strp = 0; + } + + return (r); +} + + +int +asprintf(char **strp, const char *fmt, ...) +{ + int r; + va_list ap; + va_start(ap, fmt); + r = vasprintf(strp, fmt, ap); + va_end(ap); + return (r); +} + + +int +gettimeofday(struct timeval *tp, struct timezone *tzp) +{ + // Note: some broken versions only have 8 trailing zero's, + // the correct epoch has 9 trailing zero's + static const uint64_t EPOCH = ((uint64_t)116444736000000000ULL); + + SYSTEMTIME system_time; + FILETIME file_time; + uint64_t time; + + GetSystemTime(&system_time); + SystemTimeToFileTime(&system_time, &file_time); + time = ((uint64_t)file_time.dwLowDateTime); + time += ((uint64_t)file_time.dwHighDateTime) << 32; + + tp->tv_sec = (long)((time - EPOCH) / 10000000L); + tp->tv_usec = (long)(system_time.wMilliseconds * 1000); + return (0); +} + + +void +flockfile(FILE *file) +{ +} + +void +funlockfile(FILE *file) +{ +} + +unsigned long +gethostid(void) +{ + LSTATUS Status; + unsigned long hostid = 0UL; + HKEY key; + DWORD type; + DWORD len; + + Status = RegOpenKeyEx(HKEY_LOCAL_MACHINE, + "SYSTEM\\ControlSet001\\Services\\OpenZFS", + 0, KEY_READ, &key); + if (Status != ERROR_SUCCESS) + return (0UL); + + len = sizeof (hostid); + Status = RegQueryValueEx(key, "hostid", NULL, &type, + (LPBYTE)&hostid, &len); + if (Status != ERROR_SUCCESS) + hostid = 0; + else + assert(type == REG_DWORD); + + RegCloseKey(key); + + return (hostid & 0xffffffff); +} + +uid_t +geteuid(void) +{ + return (0); // woah, root? +} + +struct passwd * +getpwuid(uid_t uid) +{ + return (NULL); +} + +const char * +win_ctime_r(char *buffer, size_t bufsize, time_t cur_time) +{ + ctime_s(buffer, bufsize, &cur_time); + return (buffer); +} + +uint64_t +GetFileDriveSize(HANDLE h) +{ + LARGE_INTEGER large; + + if (GetFileSizeEx(h, &large)) + return (large.QuadPart); + + PARTITION_INFORMATION_EX partInfo; + DWORD retcount = 0; + + if (DeviceIoControl(h, + IOCTL_DISK_GET_PARTITION_INFO_EX, + (LPVOID)NULL, + (DWORD)0, + (LPVOID)&partInfo, + sizeof (partInfo), + &retcount, + (LPOVERLAPPED)NULL)) { + return (partInfo.PartitionLength.QuadPart); + } + + + DISK_GEOMETRY_EX geometry_ex; + DWORD len; + if (DeviceIoControl(h, IOCTL_DISK_GET_DRIVE_GEOMETRY_EX, NULL, 0, + &geometry_ex, sizeof (geometry_ex), &len, NULL)) + return (geometry_ex.DiskSize.QuadPart); + + return (0); +} + + +void +openlog(const char *ident, int logopt, int facility) +{ + +} + +void +syslog(int priority, const char *message, ...) +{ + +} + +void +closelog(void) +{ + +} + +int +pipe(int fildes[2]) +{ + return (wosix_socketpair(AF_UNIX, SOCK_STREAM, 0, fildes)); +} + +struct group * +getgrgid(gid_t gid) +{ + return (NULL); +} + +int +unmount(const char *dir, int flags) +{ + return (-1); +} + +extern size_t +strlcpy(register char *s, register const char *t, register size_t n) +{ + const char *o = t; + + if (n) + do { + if (!--n) { + *s = 0; + break; + } + } while ((*s++ = *t++)); + if (!n) + while (*t++) + ; + return (t - o - 1); +} + +extern size_t +strlcat(register char *s, register const char *t, register size_t n) +{ + register size_t m; + const char *o = t; + + if ((m = n)) { + while (n && *s) { + n--; + s++; + } + m -= n; + if (n) + do { + if (!--n) { + *s = 0; + break; + } + } while ((*s++ = *t++)); + else + *s = 0; + } + if (!n) + while (*t++) + ; + return ((t - o) + m - 1); +} + +char * +strndup(const char *src, size_t size) +{ + char *r = _strdup(src); + if (r) { + r[size] = 0; + } + return (r); +} + +int +setrlimit(int resource, const struct rlimit *rlp) +{ + return (0); +} + +int +tcgetattr(int fildes, struct termios *termios_p) +{ + return (0); +} + +int +tcsetattr(int fildes, int optional_actions, + const struct termios *termios_p) +{ + return (0); +} + + +void +console_echo(boolean_t willecho) +{ + HANDLE hStdin = GetStdHandle(STD_INPUT_HANDLE); + int constype = isatty(HTOI(hStdin)); + switch (constype) { + case 0: + default: + return; + case 1: // dosbox + if (willecho) { + DWORD mode = 0; + GetConsoleMode(hStdin, &mode); + SetConsoleMode(hStdin, mode | (ENABLE_ECHO_INPUT)); + } else { + DWORD mode = 0; + GetConsoleMode(hStdin, &mode); + SetConsoleMode(hStdin, mode & (~ENABLE_ECHO_INPUT)); + } + return; + case 2: // mingw/cygwin + // Insert magic here + return; + } +} + +// Not really getline, just used for password input in libzfs_crypto.c +#define MAX_GETLINE 128 +ssize_t +getline(char **linep, size_t *linecapp, + FILE *stream) +{ + static char getpassbuf[MAX_GETLINE + 1]; + size_t i = 0; + + console_echo(FALSE); + + int c; + for (;;) { + c = getc(stream); + if ((c == '\r') || (c == '\n')) { + getpassbuf[i] = '\0'; + break; + } else if (i < MAX_GETLINE) { + getpassbuf[i++] = c; + } + if (i >= MAX_GETLINE) { + getpassbuf[i] = '\0'; + break; + } + } + + if (linep) *linep = strdup(getpassbuf); + if (linecapp) *linecapp = 1; + + console_echo(TRUE); + + return (i); +} + + +/* Windows POSIX wrappers */ + + +int +wosix_fsync(int fd) +{ + if (!FlushFileBuffers(ITOH(fd))) + return (EIO); + return (0); +} + +int +wosix_open(const char *inpath, int oflag, ...) +{ + HANDLE h; + DWORD mode = GENERIC_READ; // RDONLY=0, WRONLY=1, RDWR=2; + DWORD how = OPEN_EXISTING; + DWORD share = FILE_SHARE_READ | FILE_SHARE_WRITE; + char otherpath[MAXPATHLEN]; + char *path; + char *copy_path, *r; + + copy_path = strdup(inpath); + path = copy_path; + + /* Windows does not always handle mixed \\ and / in same path */ + r = copy_path; + while ((r = strchr(r, '/')) != NULL) + *r = '\\'; + + // This is wrong, not all bitfields + if (oflag&O_WRONLY) mode = GENERIC_WRITE; + if (oflag&O_RDWR) mode = GENERIC_READ | GENERIC_WRITE; + + switch (oflag&(O_CREAT | O_TRUNC | O_EXCL)) { + case O_CREAT: + how = OPEN_ALWAYS; + break; + case O_TRUNC: + how = TRUNCATE_EXISTING; + break; + case (O_CREAT | O_EXCL): + case (O_CREAT | O_EXCL | O_TRUNC): + // Only creating new implies starting from 0 + how = CREATE_NEW; + break; + case (O_CREAT | O_TRUNC): + how = CREATE_ALWAYS; + break; + default: + case O_EXCL: // Invalid, ignore bit - treat as normal open + how = OPEN_EXISTING; + break; + } + if (oflag&O_APPEND) mode |= FILE_APPEND_DATA; + +#ifdef O_EXLOCK + if (!(oflag&O_EXLOCK)) share |= FILE_SHARE_WRITE; +#endif + + // Win users might not supply \\?\ paths, make them so + if (path[1] == ':' && strncmp("\\\\.\\", path, 4) != 0) { + snprintf(otherpath, MAXPATHLEN, "\\\\.\\%s", &path[0]); + path = otherpath; + } + + // Support expansion of "SystemRoot" + if (strncmp(path, "\\SystemRoot\\", 12) == 0) { + snprintf(otherpath, MAXPATHLEN, "%s\\%s", + getenv("SystemRoot"), &path[12]); + path = otherpath; + } + + // Try to open verbatim, but if that fail, check if it is the + // "#offset#length#name" style, and try again. We let it fail first + // just in case someone names their file with a starting '#'. + + h = CreateFile(path, mode, share, NULL, how, + oflag & O_DIRECTORY ? FILE_FLAG_BACKUP_SEMANTICS : + FILE_ATTRIBUTE_NORMAL, + NULL); + + // Could be a directory (but we come from stat so no O_DIRECTORY) + if (h == INVALID_HANDLE_VALUE && GetLastError() == ERROR_ACCESS_DENIED) + h = CreateFile(path, mode, share, NULL, how, + FILE_FLAG_BACKUP_SEMANTICS, + NULL); + + if (h == INVALID_HANDLE_VALUE && path[0] == '#') { + char *end = NULL; + off_t offset; + size_t len; + offset = strtoull(&path[1], &end, 10); + while (end && *end == '#') end++; + len = strtoull(end, &end, 10); + while (end && *end == '#') end++; + + h = CreateFile(end, mode, share, NULL, how, + oflag & O_DIRECTORY ? FILE_FLAG_BACKUP_SEMANTICS : + FILE_ATTRIBUTE_NORMAL, + NULL); + if (h != INVALID_HANDLE_VALUE) { + // Upper layer probably handles this, but let's help + LARGE_INTEGER place; + place.QuadPart = offset; + SetFilePointerEx(h, place, NULL, FILE_BEGIN); + } + } + + // Also handle "/dev/" + if (strncmp("/dev/", path, 5) == 0) { + char tmp[PATH_MAX]; + snprintf(tmp, sizeof (tmp), "\\\\?\\%s", &path[5]); + h = CreateFile(tmp, mode, share, NULL, how, + FILE_ATTRIBUTE_NORMAL, NULL); + } + + // If we failed, translate error to posix + if (h == INVALID_HANDLE_VALUE) { + errno = EINVAL; + switch (GetLastError()) { + case ERROR_FILE_NOT_FOUND: + case ERROR_PATH_NOT_FOUND: + errno = ENOENT; + break; + case ERROR_ACCESS_DENIED: + errno = EACCES; + break; + case ERROR_FILE_EXISTS: + errno = EEXIST; + break; + case ERROR_SHARING_VIOLATION: + errno = EBUSY; // BSD: EWOULDBLOCK + // fall through + default: + fprintf(stderr, "wosix_open(%s): error %lu / 0x%lx\n", + path, GetLastError(), GetLastError()); + } + free(copy_path); + return (-1); + } + free(copy_path); + return (HTOI(h)); +} + +// Figure out when to call WSAStartup(); +static int posix_init_winsock = 0; + +int +wosix_close(int fd) +{ + HANDLE h = ITOH(fd); + + // Use CloseHandle() for everything except sockets. + if ((GetFileType(h) == FILE_TYPE_REMOTE) && + !GetNamedPipeInfo(h, NULL, NULL, NULL, NULL)) { + int err; + err = closesocket((SOCKET)h); + return (err); + } + + if (CloseHandle(h)) + return (0); + return (-1); +} + +int +wosix_ioctl(int fd, unsigned long request, zfs_iocparm_t *wrap) +{ + int error; + ULONG bytesReturned; + + error = DeviceIoControl(ITOH(fd), + (DWORD)request, + wrap, + (DWORD)sizeof (zfs_iocparm_t), + wrap, + (DWORD)sizeof (zfs_iocparm_t), + &bytesReturned, + NULL); + + if (error == 0) + error = GetLastError(); + else + error = 0; + +#ifdef DEBUG + fprintf(stderr, + " (ioctl 0x%x (%s) status %d bytes %ld)\n", + (request & 0x2ffc) >> 2, + getIoctlAsString((request & 0x2ffc) >> 2), error, + bytesReturned); + fflush(stderr); +#endif +#if 0 + for (int x = 0; x < 16; x++) + fprintf(stderr, "%02x ", ((unsigned char *)zc)[x]); + fprintf(stderr, "\n"); + fflush(stderr); + fprintf(stderr, + "returned ioctl on 0x%x (raw 0x%x) struct size %d in " + "%p:%d out %p:%d\n", + (request & 0x2ffc) >> 2, request, + sizeof (zfs_cmd_t), + zc->zc_nvlist_src, zc->zc_nvlist_src_size, + zc->zc_nvlist_dst, zc->zc_nvlist_dst_size); + fflush(stderr); +#endif + errno = error; + return (error); +} + +uint64_t +wosix_lseek(int fd, uint64_t offset, int seek) +{ + LARGE_INTEGER LOFF, LNEW; + int type = FILE_BEGIN; + + LOFF.QuadPart = offset; + switch (seek) { + case SEEK_SET: + type = FILE_BEGIN; + break; + case SEEK_CUR: + type = FILE_CURRENT; + break; + case SEEK_END: + type = FILE_END; + break; + } + if (!SetFilePointerEx(ITOH(fd), LOFF, &LNEW, type)) + return (-1); + return (LNEW.QuadPart); +} + +int +wosix_read(int fd, void *data, uint32_t len) +{ + DWORD red; + OVERLAPPED ow = {0}; + + if (GetFileType(ITOH(fd)) == FILE_TYPE_PIPE) { + if (!ReadFile(ITOH(fd), data, len, &red, &ow)) + return (-1); + } else { + if (!ReadFile(ITOH(fd), data, len, &red, NULL)) + return (-1); + } + + return (red); +} + +int +wosix_write(int fd, const void *data, uint32_t len) +{ + DWORD wrote; + OVERLAPPED ow = { 0 }; + + if (GetFileType(ITOH(fd)) == FILE_TYPE_PIPE) { + if (!WriteFile(ITOH(fd), data, len, &wrote, &ow)) + return (-1); + } else { + if (!WriteFile(ITOH(fd), data, len, &wrote, NULL)) + return (-1); + } + return (wrote); +} + +#define is_wprefix(s, prefix) \ + (wcsncmp((s), (prefix), sizeof (prefix) / sizeof (WCHAR) - 1) == 0) + +// Parts by: +// * Copyright(c) 2015 - 2017 K.Takata +// * You can redistribute it and /or modify it under the terms of either +// * the MIT license(as described below) or the Vim license. +// +// Extend isatty() slightly to return 1 for DOS Console, or +// 2 for cygwin/mingw - as we will have to do different things +// for NOECHO etc. +int +wosix_isatty(int fd) +{ + DWORD mode; + HANDLE h = ITOH(fd); + // int ret; + + // First, check if we are in a regular dos box, if yes, return. + // If not, check for cygwin ... + // check for mingw ... + // check for powershell ... + if (GetConsoleMode(h, &mode)) + return (1); + + // Not CMDbox, check mingw + if (GetFileType(h) == FILE_TYPE_PIPE) { + + int size = sizeof (FILE_NAME_INFO) + + sizeof (WCHAR) * (MAX_PATH - 1); + FILE_NAME_INFO* nameinfo; + WCHAR* p = NULL; + + nameinfo = malloc(size + sizeof (WCHAR)); + if (nameinfo != NULL) { + if (GetFileInformationByHandleEx(h, FileNameInfo, + nameinfo, size)) { + nameinfo->FileName[nameinfo->FileNameLength / + sizeof (WCHAR)] = L'\0'; + p = nameinfo->FileName; + if (is_wprefix(p, L"\\cygwin-")) { + p += 8; + } else if (is_wprefix(p, L"\\msys-")) { + p += 6; + } else { + p = NULL; + } + if (p != NULL) { + while (*p && isxdigit(*p)) + ++p; + if (is_wprefix(p, L"-pty")) { + p += 4; + } else { + p = NULL; + } + } + if (p != NULL) { + while (*p && isdigit(*p)) + ++p; + if (is_wprefix(p, L"-from-master")) { + // p += 12; + } else if (is_wprefix(p, + L"-to-master")) { + // p += 10; + } else { + p = NULL; + } + } + } + free(nameinfo); + if (p != NULL) + return (2); + } + } + + // Give up, it's not a TTY + return (0); +} + +// A bit different, just to wrap away the second argument +// Presumably _mkdir() sets errno, as EEXIST is tested. +int +wosix_mkdir(const char *path, mode_t mode) +{ + return (_mkdir(path)); +} + +int +wosix_stat(char *path, struct _stat64 *st) +{ + int fd; + int ret; + fd = wosix_open(path, O_RDONLY); + if (fd == -1) + return (-1); + ret = wosix_fstat(fd, st); + close(fd); + return (ret); +} + +int +wosix_lstat(char *path, struct _stat64 *st) +{ + int fd; + int ret; + + fd = wosix_open(path, O_RDONLY); + if (fd == -1) + return (-1); + ret = wosix_fstat(fd, st); // Fix me? Symlinks + close(fd); + return (ret); +} + +// Only fill in what we actually use in ZFS +// Mostly used to test for existance, st_mode, st_size +// also FIFO and BLK (fixme) +// Remember to convert between POSIX (S_IFDIR) and WINDOWS +// (_S_IFDIR) when required. +// Not that we call Windows _stat() in here. +int +wosix_fstat(int fd, struct _stat64 *st) +{ + HANDLE h = ITOH(fd); + BY_HANDLE_FILE_INFORMATION info; + + if (!GetFileInformationByHandle(h, &info)) + return (wosix_fstat_blk(fd, st)); + + st->st_dev = 0; + st->st_ino = 0; + st->st_mode = (info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) ? + S_IFDIR : S_IFREG; + st->st_nlink = + (info.nNumberOfLinks > SHRT_MAX ? SHRT_MAX : info.nNumberOfLinks); + st->st_uid = 0; + st->st_gid = 0; + st->st_rdev = 0; + st->st_size = + ((long long)info.nFileSizeHigh << 32ULL) | + (long long)info.nFileSizeLow; + st->st_atime = 0; + st->st_mtime = 0; + st->st_ctime = 0; + + return (0); +} + +int +wosix_fstat_blk(int fd, struct _stat64 *st) +{ + DISK_GEOMETRY_EX geometry_ex; + HANDLE handle = ITOH(fd); + DWORD len; + LARGE_INTEGER size; + + // Try device first + if (DeviceIoControl(handle, IOCTL_DISK_GET_DRIVE_GEOMETRY_EX, NULL, 0, + &geometry_ex, sizeof (geometry_ex), &len, NULL)) { + st->st_size = (diskaddr_t)geometry_ex.DiskSize.QuadPart; + st->st_mode = S_IFBLK; + return (0); + } + + // Try regular file + if (GetFileSizeEx(handle, &size)) { + st->st_size = (diskaddr_t)size.QuadPart; + st->st_mode = S_IFREG; + return (0); + } + + return (-1); // errno? +} + +// os specific files can call this directly. +int +pread_win(HANDLE h, void *buf, size_t nbyte, off_t offset) +{ + DWORD red; + LARGE_INTEGER large; + LARGE_INTEGER lnew; + // This code does all seeks based on "current" so we can + // pre-seek to offset start + + // Find current position + large.QuadPart = 0; + SetFilePointerEx(h, large, &lnew, FILE_CURRENT); + + // Seek to place to read + large.QuadPart = offset; + SetFilePointerEx(h, large, NULL, FILE_BEGIN); + + boolean_t ok; + + ok = ReadFile(h, buf, nbyte, &red, NULL); + + if (!ok) { + red = GetLastError(); + red = -red; + } + + // Restore position + SetFilePointerEx(h, lnew, NULL, FILE_BEGIN); + + return (red); +} + +int +wosix_pread(int fd, void *buf, size_t nbyte, off_t offset) +{ + return (pread_win(ITOH(fd), buf, nbyte, offset)); +} + +int +wosix_pwrite(int fd, const void *buf, size_t nbyte, off_t offset) +{ + HANDLE h = ITOH(fd); + DWORD wrote; + LARGE_INTEGER large; + LARGE_INTEGER lnew; + + // This code does all seeks based on "current" so we can + // pre-seek to offset start + + // Find current position + large.QuadPart = 0; + SetFilePointerEx(h, large, &lnew, FILE_CURRENT); + + // Seek to place to read + large.QuadPart = offset; + SetFilePointerEx(h, large, NULL, FILE_BEGIN); + + // Write + if (!WriteFile(h, buf, nbyte, &wrote, NULL)) + wrote = -GetLastError(); + + // Restore position + SetFilePointerEx(h, lnew, NULL, FILE_BEGIN); + + return (wrote); +} + +int +wosix_fdatasync(int fd) +{ + // if (fcntl(fd, F_FULLFSYNC) == -1) + // return -1; + return (0); +} + +int +wosix_ftruncate(int fd, off_t length) +{ + HANDLE h = ITOH(fd); + LARGE_INTEGER lnew; + + lnew.QuadPart = length; + if (SetFilePointerEx(h, lnew, NULL, FILE_BEGIN) && + SetEndOfFile(h)) + return (0); // Success + // errno? + return (-1); +} + +const char * +check_file_mode(const char *mode) +{ + /* Unknown mode causes abort() */ + if (strcmp(mode, "re") == 0) + return ("rb"); + if (strcmp(mode, "r") == 0) + return ("rb"); + return (mode); +} + +FILE * +wosix_fopen(const char *name, const char *mode) +{ + mode = check_file_mode(mode); +#undef fopen + return (fopen(name, mode)); +} + +FILE * +wosix_fdopen(int fd, const char *mode) +{ + // Convert HANDLE to int + int temp = _open_osfhandle((intptr_t)ITOH(fd), _O_APPEND | _O_RDONLY); + + if (temp == -1) { + return (NULL); + } + + mode = check_file_mode(mode); + // Convert int to FILE* + FILE *f = _fdopen(temp, mode); + + if (f == NULL) { + _close(temp); + return (NULL); + } + + // Why is this print required? + fprintf(stderr, "\r\n"); + + // fclose(f) will also call _close() on temp. + return (f); +} + +int +wosix_socketpair(int domain, int type, int protocol, int sv[2]) +{ + int temp, s1, s2, result; + struct sockaddr_in saddr; + int nameLen; + unsigned long option_arg = 1; + int err = 0; + WSADATA wsaData; + + // Do we need to init winsock? Is this the right way, should we + // add _init/_exit calls? If socketpair is the only winsock call + // we have, this might be ok. + if (posix_init_winsock == 0) { + posix_init_winsock = 1; + err = WSAStartup(MAKEWORD(2, 2), &wsaData); + if (err != 0) { + errno = err; + return (-1); + } + } + + nameLen = sizeof (saddr); + + /* ignore address family for now; just stay with AF_INET */ + temp = socket(AF_INET, SOCK_STREAM, 0); + if (temp == INVALID_SOCKET) { + int err = WSAGetLastError(); + errno = err; + return (-1); + } + + setsockopt(temp, SOL_SOCKET, SO_REUSEADDR, (void *)&option_arg, + sizeof (option_arg)); + + /* + * We *SHOULD* choose the correct sockaddr structure based + * on the address family requested... + */ + memset(&saddr, 0, sizeof (saddr)); + + saddr.sin_family = AF_INET; + saddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + saddr.sin_port = 0; // give me a port + + result = bind(temp, (struct sockaddr *)&saddr, nameLen); + if (result == SOCKET_ERROR) { + errno = WSAGetLastError(); + closesocket(temp); + return (-2); + } + + // Don't care about error here, the connect will fail instead + listen(temp, 1); + + // Fetch out the port that was given to us. + nameLen = sizeof (struct sockaddr_in); + + result = getsockname(temp, (struct sockaddr *)&saddr, &nameLen); + + if (result == INVALID_SOCKET) { + closesocket(temp); + return (-4); /* error case */ + } + + s1 = socket(AF_INET, SOCK_STREAM, 0); + if (s1 == INVALID_SOCKET) { + closesocket(temp); + return (-5); + } + + nameLen = sizeof (struct sockaddr_in); + + result = connect(s1, (struct sockaddr *)&saddr, nameLen); + + if (result == INVALID_SOCKET) { + closesocket(temp); + closesocket(s1); + return (-6); /* error case */ + } + + s2 = accept(temp, NULL, NULL); + + closesocket(temp); + + if (s2 == INVALID_SOCKET) { + closesocket(s1); + return (-7); + } + + sv[0] = s1; sv[1] = s2; + + if ((sv[0] < 0) || (sv[1] < 0)) + return (-8); + + return (0); /* normal case */ +} + +int +wosix_dup2(int fildes, int fildes2) +{ + return (-1); +} + +void * +wosix_mmap(void *addr, size_t len, int prot, int flags, + int fildes, off_t off) +{ + HANDLE h = ITOH(fildes); + HANDLE file_mapping; + int winprot = 0, winflags = 0; + void *mapaddr = NULL; + + /* Make a vague effort at matching flags */ + + if (prot & PROT_READ) { + winprot = PAGE_READONLY; + winflags = FILE_MAP_READ; + } + if (prot & PROT_WRITE) { + if (flags & MAP_PRIVATE) { + winprot = PAGE_WRITECOPY; + winflags = FILE_MAP_COPY; + } else if (flags & MAP_SHARED) { + winprot = PAGE_READWRITE; + winflags = FILE_MAP_WRITE; + } + } + + file_mapping = CreateFileMapping(h, NULL, winprot, + 0, 0, NULL); + if (file_mapping == NULL) + return (MAP_FAILED); + + mapaddr = (caddr_t)MapViewOfFileEx(file_mapping, winflags, + 0, off, len, (LPVOID) addr); + + CloseHandle(file_mapping); + if (mapaddr == NULL) + return (MAP_FAILED); + + return (mapaddr); +} + +int +wosix_munmap(void *addr, size_t len) +{ + return (int)(UnmapViewOfFile((LPVOID) addr)); +} + + + +static uint64_t GetLogicalProcessors(void); + +uint64_t +sysconf(int name) +{ + SYSTEM_INFO info; + MEMORYSTATUSEX status; + + switch (name) { + + case _SC_NPROCESSORS_ONLN: + return (GetLogicalProcessors()); + case _SC_PHYS_PAGES: + case _SC_PAGE_SIZE: + GetSystemInfo(&info); + if (name == _SC_PAGE_SIZE) + return (info.dwPageSize); + status.dwLength = sizeof (status); + GlobalMemoryStatusEx(&status); + return ((long)(status.ullTotalPhys / info.dwPageSize)); + default: + return (-1); + } +} + + +typedef BOOL(WINAPI *LPFN_GLPI)(PSYSTEM_LOGICAL_PROCESSOR_INFORMATION, + PDWORD); + +// Helper function to count set bits in the processor mask. +static DWORD +CountSetBits(ULONG_PTR bitMask) +{ + DWORD LSHIFT = sizeof (ULONG_PTR)*8 - 1; + DWORD bitSetCount = 0; + ULONG_PTR bitTest = (ULONG_PTR)1 << LSHIFT; + DWORD i; + + for (i = 0; i <= LSHIFT; ++i) { + bitSetCount += ((bitMask & bitTest)?1:0); + bitTest /= 2; + } + + return (bitSetCount); +} + +static uint64_t +GetLogicalProcessors(void) +{ + LPFN_GLPI glpi; + BOOL done = FALSE; + PSYSTEM_LOGICAL_PROCESSOR_INFORMATION buffer = NULL; + PSYSTEM_LOGICAL_PROCESSOR_INFORMATION ptr = NULL; + DWORD returnLength = 0; + uint64_t logicalProcessorCount = 0; + DWORD numaNodeCount = 0; + DWORD processorCoreCount = 0; + DWORD processorL1CacheCount = 0; + DWORD processorL2CacheCount = 0; + DWORD processorL3CacheCount = 0; + DWORD processorPackageCount = 0; + DWORD byteOffset = 0; + PCACHE_DESCRIPTOR Cache; + + glpi = (LPFN_GLPI) GetProcAddress( + GetModuleHandle(TEXT("kernel32")), + "GetLogicalProcessorInformation"); + if (NULL == glpi) + return (0); + + while (!done) { + DWORD rc = glpi(buffer, &returnLength); + + if (FALSE == rc) { + if (GetLastError() == ERROR_INSUFFICIENT_BUFFER) { + if (buffer) + free(buffer); + + buffer = (PSYSTEM_LOGICAL_PROCESSOR_INFORMATION) + malloc(returnLength); + + if (NULL == buffer) + return (0); + } else { + return (0); + } + } else { + done = TRUE; + } + } + + ptr = buffer; + + while (byteOffset + sizeof (SYSTEM_LOGICAL_PROCESSOR_INFORMATION) <= + returnLength) { + switch (ptr->Relationship) { + case RelationNumaNode: + // Non-NUMA systems report a single record of this type. + numaNodeCount++; + break; + + case RelationProcessorCore: + processorCoreCount++; + + // A hyperthreaded core supplies more than one + // logical processor. + logicalProcessorCount += + CountSetBits(ptr->ProcessorMask); + break; + + case RelationCache: + // Cache data is in ptr->Cache, one CACHE_DESCRIPTOR + // structure for each cache. + Cache = &ptr->Cache; + if (Cache->Level == 1) + processorL1CacheCount++; + else if (Cache->Level == 2) + processorL2CacheCount++; + else if (Cache->Level == 3) + processorL3CacheCount++; + break; + + case RelationProcessorPackage: + // Logical processors share a physical package. + processorPackageCount++; + break; + + default: + break; + } + byteOffset += sizeof (SYSTEM_LOGICAL_PROCESSOR_INFORMATION); + ptr++; + } + + free(buffer); + return (logicalProcessorCount); +} + +int +mprotect(void *addr, size_t len, int prot) +{ + // We can probably implement something using VirtualProtect() here. + return (0); +} + +int +getuid(void) +{ + return (1); +} + + +int +fcntl(int fildes, int cmd, /* arg */ ...) +{ + return (0); +} + +int +sched_yield(void) +{ + Sleep(0); + return (0); +} + +int +uname(struct utsname *buf) +{ + OSVERSIONINFOEX versionex; + SYSTEM_INFO info; + /* Fill in nodename. */ + if (gethostname(buf->nodename, sizeof (buf->nodename)) < 0) + strcpy(buf->nodename, "localhost"); + + versionex.dwOSVersionInfoSize = sizeof (OSVERSIONINFOEX); + // GetVersionEx(&versionex); + VerifyVersionInfo(&versionex, VER_MAJORVERSION | VER_MINORVERSION, 0); + snprintf(buf->sysname, sizeof (buf->sysname), "Windows_NT-%u.%u", + (unsigned int) versionex.dwMajorVersion, + (unsigned int) versionex.dwMinorVersion); + + GetSystemInfo(&info); + + switch (info.wProcessorArchitecture) { + case PROCESSOR_ARCHITECTURE_AMD64: + strcpy(buf->machine, "x86_64"); + break; + case PROCESSOR_ARCHITECTURE_IA64: + strcpy(buf->machine, "ia64"); + break; + case PROCESSOR_ARCHITECTURE_INTEL: + strcpy(buf->machine, "i386"); + break; + default: + strcpy(buf->machine, "unknown"); + break; + } + + return (0); +} + +char * +nl_langinfo(nl_item item) +{ + switch (item) { + /* nl_langinfo items of the LC_CTYPE category */ + case _DATE_FMT: + return ("%y/%m/%d"); + } + return (""); +} + +int +wosix_openat(int fd, const char *path, int oflag, ...) +{ + HANDLE h = ITOH(fd); + char fullpath[MAXPATHLEN]; + + if (fd == AT_FDCWD) + return (wosix_open(path, oflag)); + + /* + * Fetch the directory name, and stitch the name together. + * Another option is using NTCreateFile with RootDirectory=handle + */ + + if (GetFinalPathNameByHandleA(h, fullpath, + MAXPATHLEN, FILE_NAME_NORMALIZED) > 0) { + strlcat(fullpath, "/", MAXPATHLEN); + strlcat(fullpath, path, MAXPATHLEN); + return (wosix_open(fullpath, oflag)); + } + return (-1); +} + +/* + * This is a poor "port" of freopen() but, to date, it is only + * used to re-open the MNTTAB, of which we have none, and the return + * code is never used. + */ +FILE * +wosix_freopen(const char *path, const char *mode, FILE *stream) +{ + return ((FILE *)path); // Anything not NULL +} diff --git a/lib/libspl/os/windows/signal.c b/lib/libspl/os/windows/signal.c new file mode 100644 index 000000000000..56a7672c551e --- /dev/null +++ b/lib/libspl/os/windows/signal.c @@ -0,0 +1,92 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#include + +int +sigemptyset(sigset_t *set) +{ + // *set = 0; + return (0); +} + +int +sigfillset(sigset_t *set) +{ + // *set = ~(sigset_t)0; + return (0); +} + +int +sigaddset(sigset_t *set, int sig) +{ + // *set |= (1<<(sig-1)); + return (0); +} + +int +sigdelset(sigset_t *set, int sig) +{ + // *set &= ~(1<<(sig-1)); + return (0); +} + +int +sigismember(sigset_t *set, int sig) +{ + // return ((*set & (1<<(sig-1))) != 0); + return (0); +} + +int +sigaction(int sig, struct sigaction *sa, struct sigaction *osa) +{ + if (osa) + osa->sa_handler = signal(sig, + (void(__cdecl*)(int))sa->sa_handler); + else + signal(sig, (void(__cdecl*)(int))sa->sa_handler); + return (0); +} + +int +sigprocmask(int operation, sigset_t *set, sigset_t *oset) +{ + if (oset) + /* *oset = 0 */; + return (0); +} + +int +pause(void) +{ + +} + +int +kill(int pid, int sig) +{ + return (0); +} diff --git a/lib/libspl/os/windows/uio.c b/lib/libspl/os/windows/uio.c new file mode 100644 index 000000000000..04fe46f0a125 --- /dev/null +++ b/lib/libspl/os/windows/uio.c @@ -0,0 +1,113 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2017 Jorgen Lundman + +struct uio *uio_create( + int a_iovcount, /* max number of iovecs */ + off_t a_offset, /* current offset */ + int a_spacetype, /* type of address space */ + int a_iodirection) /* read or write flag */ +{ + return (NULL); +} + +user_addr_t +uio_curriovbase(struct uio *a_uio) +{ + return (0); +} + +int +uio_iovcnt(struct uio *a_uio) +{ + return (0); +} + +void +uio_free(struct uio *a_uio) +{ +} + +int +uio_addiov(struct uio *o, user_addr_t a_baseaddr, user_size_t a_length) +{ + return (-1); +} + +int +uio_getiov(struct uio *a_uio, + int a_index, + user_addr_t *a_baseaddr_p, + user_size_t *a_length_p) +{ + return (-1); +} + +user_size_t +uio_curriovlen(struct uio *a_uio) +{ + return (0); +} + +int +uio_isuserspace(struct uio *a_uio) +{ + return (0); +} + +user_size_t +uio_resid(struct uio *a_uio) +{ + return (0); +} + +void +uio_setrw(struct uio *a_uio, int a_value) +{ +} + +int +uiomove(const char *cp, int n, int r, struct uio *uio) +{ + return (0); +} + +void +uio_update(struct uio *a_uio, user_size_t a_count) +{ +} + +off_t +uio_offset(struct uio *a_uio) +{ + return (0); +} + +void +uio_reset(struct uio *a_uio, off_t a_offset, int a_spacetype, + int a_iodirection) +{ +} diff --git a/lib/libspl/os/windows/weakpragma.c b/lib/libspl/os/windows/weakpragma.c new file mode 100644 index 000000000000..43f631e0b98d --- /dev/null +++ b/lib/libspl/os/windows/weakpragma.c @@ -0,0 +1,43 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#include +#include +#include + + +/* No #pragma weaks here! */ +void +dmu_buf_add_ref(dmu_buf_t *db, const void *tag) +{ + dbuf_add_ref((dmu_buf_impl_t *)db, tag); +} + +boolean_t +dmu_buf_try_add_ref(dmu_buf_t *db, objset_t *os, uint64_t object, + uint64_t blkid, const void *tag) +{ + return (dbuf_try_add_ref(db, os, object, blkid, tag)); +} diff --git a/lib/libspl/os/windows/xdr.c b/lib/libspl/os/windows/xdr.c new file mode 100644 index 000000000000..a80d91cb9b8d --- /dev/null +++ b/lib/libspl/os/windows/xdr.c @@ -0,0 +1,729 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ +/* + * Portions of this source code were derived from Berkeley + * 4.3 BSD under license from the Regents of the University of + * California. + */ + +// #pragma ident "%Z%%M% %I% %E% SMI" + +/* + * Generic XDR routines implementation. + * + * These are the "generic" xdr routines used to serialize and de-serialize + * most common data items. See xdr.h for more info on the interface to + * xdr. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// warning C4133 : 'function' : incompatible types - from 'XDR *' to 'XDR *' +#pragma warning(disable: 4133) + +#ifndef _WIN32 +#pragma weak xdr_int64_t = xdr_hyper +#pragma weak xdr_uint64_t = xdr_u_hyper +#pragma weak xdr_int32_t = xdr_int +#pragma weak xdr_uint32_t = xdr_u_int +#pragma weak xdr_int16_t = xdr_short +#pragma weak xdr_uint16_t = xdr_u_short +#pragma weak xdr_int8_t = xdr_char +#pragma weak xdr_uint8_t = xdr_u_char + +/* + * The following routine was part of a workaround for an rpcgen + * that was fixed, this routine should be removed sometime. + */ +#pragma weak xdr_ulonglong_t = xdr_u_longlong_t +#endif + +/* + * constants specific to the xdr "protocol" + */ +#define XDR_FALSE ((uint_t)0) +#define XDR_TRUE ((uint_t)1) +#define LASTUNSIGNED ((uint_t)0-1) + +/* fragment size to use when doing an xdr_string() */ +#define FRAGMENT 65536 + +/* + * for unit alignment + */ +#define BYTES_PER_XDR_UNIT (4) + +static const char xdr_zero[BYTES_PER_XDR_UNIT] = { 0 }; + +/* + * Free a data structure using XDR + * Not a filter, but a convenient utility nonetheless + */ +void +xdr_free(xdrproc_t proc, char *objp) +{ + XDR x; + + x.x_op = XDR_FREE; + (*proc)(&x, objp); +} + +/* + * XDR nothing + */ +bool_t +xdr_void(void) +{ + return (B_TRUE); +} + +/* + * xdr_time_t sends time_t value over the wire. + * Due to RPC Protocol limitation, it can only send + * up to 32-bit integer quantity over the wire. + * + */ +bool_t +xdr_time_t(XDR *xdrs, time_t *tp) +{ + int32_t i; + + switch (xdrs->x_op) { + case XDR_ENCODE: + /* + * Check for the time overflow, when encoding it. + * Don't want to send OTW the time value too large to + * handle by the protocol. + */ +#if defined(_LP64) + if (*tp > INT32_MAX) + *tp = INT32_MAX; + else if (*tp < INT32_MIN) + *tp = INT32_MIN; +#endif + i = (int32_t)*tp; + return (XDR_PUTINT32(xdrs, &i)); + + case XDR_DECODE: + if (!XDR_GETINT32(xdrs, &i)) + return (FALSE); + *tp = (time_t)i; + return (TRUE); + + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR integers + */ +bool_t +xdr_int(XDR *xdrs, int *ip) +{ + switch (xdrs->x_op) { + case XDR_ENCODE: + return (XDR_PUTINT32(xdrs, ip)); + case XDR_DECODE: + return (XDR_GETINT32(xdrs, ip)); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR unsigned integers + */ +bool_t +xdr_u_int(XDR *xdrs, uint_t *up) +{ + switch (xdrs->x_op) { + case XDR_ENCODE: + return (XDR_PUTINT32(xdrs, (int *)up)); + case XDR_DECODE: + return (XDR_GETINT32(xdrs, (int *)up)); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * The definition of xdr_long()/xdr_u_long() is kept for backward + * compatibitlity. + * XDR long integers, same as xdr_u_long + */ +bool_t +xdr_long(XDR *xdrs, long *lp) +{ + int32_t i; + + switch (xdrs->x_op) { + case XDR_ENCODE: +#if defined(_LP64) + if ((*lp > INT32_MAX) || (*lp < INT32_MIN)) + return (FALSE); +#endif + i = (int32_t)*lp; + return (XDR_PUTINT32(xdrs, &i)); + case XDR_DECODE: + if (!XDR_GETINT32(xdrs, &i)) + return (FALSE); + *lp = (long)i; + return (TRUE); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR unsigned long integers + * same as xdr_long + */ +bool_t +xdr_u_long(XDR *xdrs, ulong_t *ulp) +{ + uint32_t ui; + + switch (xdrs->x_op) { + case XDR_ENCODE: +#if defined(_LP64) + if (*ulp > UINT32_MAX) + return (FALSE); +#endif + ui = (uint32_t)*ulp; + return (XDR_PUTINT32(xdrs, (int32_t *)&ui)); + case XDR_DECODE: + if (!XDR_GETINT32(xdrs, (int32_t *)&ui)) + return (FALSE); + *ulp = (ulong_t)ui; + return (TRUE); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR short integers + */ +bool_t +xdr_short(XDR *xdrs, short *sp) +{ + int32_t l; + + switch (xdrs->x_op) { + case XDR_ENCODE: + l = (int32_t)*sp; + return (XDR_PUTINT32(xdrs, &l)); + case XDR_DECODE: + if (!XDR_GETINT32(xdrs, &l)) + return (FALSE); + *sp = (short)l; + return (TRUE); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR unsigned short integers + */ +bool_t +xdr_u_short(XDR *xdrs, ushort_t *usp) +{ + uint_t i; + + switch (xdrs->x_op) { + case XDR_ENCODE: + i = (uint_t)*usp; + return (XDR_PUTINT32(xdrs, (int *)&i)); + case XDR_DECODE: + if (!XDR_GETINT32(xdrs, (int *)&i)) + return (FALSE); + *usp = (ushort_t)i; + return (TRUE); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + + +/* + * XDR a char + */ +bool_t +xdr_char(XDR *xdrs, char *cp) +{ + int i; + + switch (xdrs->x_op) { + case XDR_ENCODE: + i = (*cp); + return (XDR_PUTINT32(xdrs, &i)); + case XDR_DECODE: + if (!XDR_GETINT32(xdrs, &i)) + return (FALSE); + *cp = (char)i; + return (TRUE); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR an unsigned char + */ +bool_t +xdr_u_char(XDR *xdrs, uchar_t *cp) +{ + int i; + + switch (xdrs->x_op) { + case XDR_ENCODE: + i = (*cp); + return (XDR_PUTINT32(xdrs, &i)); + case XDR_DECODE: + if (!XDR_GETINT32(xdrs, &i)) + return (FALSE); + *cp = (uchar_t)i; + return (TRUE); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR booleans + */ +bool_t +xdr_bool(XDR *xdrs, bool_t *bp) +{ + int i; + + switch (xdrs->x_op) { + case XDR_ENCODE: + i = *bp ? XDR_TRUE : XDR_FALSE; + return (XDR_PUTINT32(xdrs, &i)); + case XDR_DECODE: + if (!XDR_GETINT32(xdrs, &i)) + return (FALSE); + *bp = (i == XDR_FALSE) ? FALSE : TRUE; + return (TRUE); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR enumerations + */ +bool_t +xdr_enum(XDR *xdrs, int *ep) +{ + enum sizecheck { SIZEVAL }; /* used to find the size of an enum */ + + /* + * enums are treated as ints + */ + /* CONSTCOND */ + assert(sizeof (enum sizecheck) == sizeof (int32_t)); + return (xdr_int(xdrs, (int *)ep)); +} + +/* + * XDR opaque data + * Allows the specification of a fixed size sequence of opaque bytes. + * cp points to the opaque object and cnt gives the byte length. + */ +bool_t +xdr_opaque(XDR *xdrs, caddr_t cp, const uint_t cnt) +{ + uint_t rndup; + char crud[BYTES_PER_XDR_UNIT]; + + /* + * if no data we are done + */ + if (cnt == 0) + return (TRUE); + + /* + * round byte count to full xdr units + */ + rndup = cnt % BYTES_PER_XDR_UNIT; + if ((int)rndup > 0) + rndup = BYTES_PER_XDR_UNIT - rndup; + + switch (xdrs->x_op) { + case XDR_DECODE: + if (!XDR_GETBYTES(xdrs, cp, cnt)) + return (FALSE); + if (rndup == 0) + return (TRUE); + return (XDR_GETBYTES(xdrs, crud, rndup)); + case XDR_ENCODE: + if (!XDR_PUTBYTES(xdrs, cp, cnt)) + return (FALSE); + if (rndup == 0) + return (TRUE); + return (XDR_PUTBYTES(xdrs, (caddr_t)&xdr_zero[0], rndup)); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +/* + * XDR counted bytes + * *cpp is a pointer to the bytes, *sizep is the count. + * If *cpp is NULL maxsize bytes are allocated + */ + +static const char xdr_err[] = "xdr_%s: out of memory"; + +bool_t +xdr_bytes(XDR *xdrs, char **cpp, uint_t *sizep, const uint_t maxsize) +{ + char *sp = *cpp; /* sp is the actual string pointer */ + uint_t nodesize; + + /* + * first deal with the length since xdr bytes are counted + * We decided not to use MACRO XDR_U_INT here, because the + * advantages here will be miniscule compared to xdr_bytes. + * This saved us 100 bytes in the library size. + */ + if (!xdr_u_int(xdrs, sizep)) + return (FALSE); + nodesize = *sizep; + if ((nodesize > maxsize) && (xdrs->x_op != XDR_FREE)) + return (FALSE); + + /* + * now deal with the actual bytes + */ + switch (xdrs->x_op) { + case XDR_DECODE: + if (nodesize == 0) + return (TRUE); + if (sp == NULL) + *cpp = sp = malloc(nodesize); + if (sp == NULL) { +// (void) syslog(LOG_ERR, xdr_err, (const char *)"bytes"); + return (FALSE); + } + /*FALLTHROUGH*/ + case XDR_ENCODE: + return (xdr_opaque(xdrs, sp, nodesize)); + case XDR_FREE: + if (sp != NULL) { + free(sp); + *cpp = NULL; + } + return (TRUE); + } + return (FALSE); +} + +/* + * Implemented here due to commonality of the object. + */ +bool_t +xdr_netobj(XDR *xdrs, struct netobj *np) +{ + return (xdr_bytes(xdrs, &np->n_bytes, &np->n_len, MAX_NETOBJ_SZ)); +} + +/* + * XDR a descriminated union + * Support routine for discriminated unions. + * You create an array of xdrdiscrim structures, terminated with + * an entry with a null procedure pointer. The routine gets + * the discriminant value and then searches the array of xdrdiscrims + * looking for that value. It calls the procedure given in the xdrdiscrim + * to handle the discriminant. If there is no specific routine a default + * routine may be called. + * If there is no specific or default routine an error is returned. + */ +bool_t +xdr_union(XDR *xdrs, int *dscmp, char *unp, + const struct xdr_discrim *choices, const xdrproc_t dfault) +{ + int dscm; + + /* + * we deal with the discriminator; it's an enum + */ + if (!xdr_enum(xdrs, dscmp)) + return (FALSE); + dscm = *dscmp; + + /* + * search choices for a value that matches the discriminator. + * if we find one, execute the xdr routine for that value. + */ + for (; choices->proc != NULL_xdrproc_t; choices++) { + if (choices->value == dscm) + return ((*(choices->proc))(xdrs, unp, LASTUNSIGNED)); + } + + /* + * no match - execute the default xdr routine if there is one + */ + return ((dfault == NULL_xdrproc_t) ? FALSE : + (*dfault)(xdrs, unp, LASTUNSIGNED)); +} + + +/* + * Non-portable xdr primitives. + * Care should be taken when moving these routines to new architectures. + */ + + +/* + * XDR null terminated ASCII strings + * xdr_string deals with "C strings" - arrays of bytes that are + * terminated by a NULL character. The parameter cpp references a + * pointer to storage; If the pointer is null, then the necessary + * storage is allocated. The last parameter is the max allowed length + * of the string as specified by a protocol. + */ +bool_t +xdr_string(XDR *xdrs, char **cpp, const uint_t maxsize) +{ + char *newsp, *sp = *cpp; /* sp is the actual string pointer */ + uint_t size, block; + uint64_t bytesread; + + /* + * first deal with the length since xdr strings are counted-strings + */ + switch (xdrs->x_op) { + case XDR_FREE: + if (sp == NULL) + return (TRUE); /* already free */ + /*FALLTHROUGH*/ + case XDR_ENCODE: + size = (sp != NULL) ? (uint_t)strlen(sp) : 0; + break; + } + /* + * We decided not to use MACRO XDR_U_INT here, because the + * advantages here will be miniscule compared to xdr_string. + * This saved us 100 bytes in the library size. + */ + if (!xdr_u_int(xdrs, &size)) + return (FALSE); + if (size > maxsize) + return (FALSE); + + /* + * now deal with the actual bytes + */ + switch (xdrs->x_op) { + case XDR_DECODE: + /* if buffer is already given, call xdr_opaque() directly */ + if (sp != NULL) { + if (!xdr_opaque(xdrs, sp, size)) + return (FALSE); + sp[size] = 0; + return (TRUE); + } + + /* + * We have to allocate a buffer of size 'size'. To avoid + * malloc()ing one huge chunk, we'll read the bytes in max + * FRAGMENT size blocks and keep realloc()ing. 'block' is + * the number of bytes to read in each xdr_opaque() and + * 'bytesread' is what we have already read. sp is NULL + * when we are in the loop for the first time. + */ + bytesread = 0; + do { + block = MIN(size - bytesread, FRAGMENT); + /* + * allocate enough for 'bytesread + block' bytes and + * one extra for the terminating NULL. + */ + newsp = realloc(sp, bytesread + block + 1); + if (newsp == NULL) { + if (sp != NULL) + free(sp); + return (FALSE); + } + sp = newsp; + if (!xdr_opaque(xdrs, &sp[bytesread], block)) { + free(sp); + return (FALSE); + } + bytesread += block; + } while (bytesread < size); + + sp[bytesread] = 0; /* terminate the string with a NULL */ + *cpp = sp; + return (TRUE); + case XDR_ENCODE: + return (xdr_opaque(xdrs, sp, size)); + case XDR_FREE: + free(sp); + *cpp = NULL; + return (TRUE); + } + return (FALSE); +} + +bool_t +xdr_hyper(XDR *xdrs, longlong_t *hp) +{ + if (xdrs->x_op == XDR_ENCODE) { +#if defined(_LONG_LONG_HTOL) + if (XDR_PUTINT32(xdrs, (int *)hp) == TRUE) + /* LINTED pointer cast */ + return (XDR_PUTINT32(xdrs, (int *)((char *)hp + + BYTES_PER_XDR_UNIT))); +#else + /* LINTED pointer cast */ + if (XDR_PUTINT32(xdrs, (int *)((char *)hp + + BYTES_PER_XDR_UNIT)) == TRUE) + return (XDR_PUTINT32(xdrs, (int32_t *)hp)); +#endif + return (FALSE); + } + + if (xdrs->x_op == XDR_DECODE) { +#if defined(_LONG_LONG_HTOL) + if (XDR_GETINT32(xdrs, (int *)hp) == FALSE || + /* LINTED pointer cast */ + (XDR_GETINT32(xdrs, (int *)((char *)hp + + BYTES_PER_XDR_UNIT)) == FALSE)) + return (FALSE); +#else + /* LINTED pointer cast */ + if ((XDR_GETINT32(xdrs, (int *)((char *)hp + + BYTES_PER_XDR_UNIT)) == FALSE) || + (XDR_GETINT32(xdrs, (int *)hp) == FALSE)) + return (FALSE); +#endif + return (TRUE); + } + return (TRUE); +} + +bool_t +xdr_u_hyper(XDR *xdrs, u_longlong_t *hp) +{ + return (xdr_hyper(xdrs, (longlong_t *)hp)); +} + +bool_t +xdr_longlong_t(XDR *xdrs, longlong_t *hp) +{ + return (xdr_hyper(xdrs, hp)); +} + +bool_t +xdr_u_longlong_t(XDR *xdrs, u_longlong_t *hp) +{ + return (xdr_hyper(xdrs, (longlong_t *)hp)); +} + +/* + * Wrapper for xdr_string that can be called directly from + * routines like clnt_call + */ +bool_t +xdr_wrapstring(XDR *xdrs, char **cpp) +{ + return (xdr_string(xdrs, cpp, LASTUNSIGNED)); +} + +#if 0 +bool_t +xdr_control(XDR *xdrs, int request, void *info) +{ + xdr_bytesrec_t *xptr; + int32_t *int32p; + int len; + + switch (request) { + case XDR_GET_BYTES_AVAIL: + xptr = (xdr_bytesrec_t *)info; + xptr->xc_is_last_record = TRUE; + xptr->xc_num_avail = xdrs->x_handy; + return (TRUE); + + case XDR_PEEK: + /* + * Return the next 4 byte unit in the XDR stream. + */ + if (xdrs->x_handy < sizeof (int32_t)) + return (FALSE); + int32p = (int32_t *)info; + *int32p = (int32_t)ntohl((uint32_t) + (*((int32_t *)(xdrs->x_private)))); + return (TRUE); + + case XDR_SKIPBYTES: + /* + * Skip the next N bytes in the XDR stream. + */ + int32p = (int32_t *)info; + len = RNDUP((int)(*int32p)); + if (xdrs->x_handy < len) + return (FALSE); + xdrs->x_handy -= len; + xdrs->x_private += len; + return (TRUE); + + } + return (FALSE); +} +#endif diff --git a/lib/libspl/os/windows/xdr_array.c b/lib/libspl/os/windows/xdr_array.c new file mode 100644 index 000000000000..46b6bff52b6f --- /dev/null +++ b/lib/libspl/os/windows/xdr_array.c @@ -0,0 +1,145 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ +/* + * Portions of this source code were derived from Berkeley + * 4.3 BSD under license from the Regents of the University of + * California. + */ + +/* + * Generic XDR routines impelmentation. + * + * These are the "non-trivial" xdr primitives used to serialize and + * de-serialize arrays. See xdr.h for more info on the interface to xdr. + */ + +#include +#include +#include + +#include +#include +#include +#include + +#define LASTUNSIGNED ((uint_t)0-1) + +char mem_err_msg_arr[] = "xdr_array: out of memory"; + +/* + * XDR an array of arbitrary elements + * *addrp is a pointer to the array, *sizep is the number of elements. + * If *addrp is NULL (*sizep * elsize) bytes are allocated. + * elsize is the size (in bytes) of each element, and elproc is the + * xdr procedure to call to handle each element of the array. + */ +bool_t +xdr_array(XDR *xdrs, caddr_t *addrp, uint_t *sizep, const uint_t maxsize, + const uint_t elsize, const xdrproc_t elproc) +{ + uint_t i; + caddr_t target = *addrp; + uint_t c; /* the actual element count */ + bool_t stat = TRUE; + uint_t nodesize; + + /* like strings, arrays are really counted arrays */ + if (!xdr_u_int(xdrs, sizep)) + return (FALSE); + c = *sizep; + if ((c > maxsize || LASTUNSIGNED / elsize < c) && + xdrs->x_op != XDR_FREE) + return (FALSE); + nodesize = c * elsize; + + /* + * if we are deserializing, we may need to allocate an array. + * We also save time by checking for a null array if we are freeing. + */ + if (target == NULL) + switch (xdrs->x_op) { + case XDR_DECODE: + if (c == 0) + return (TRUE); + *addrp = target = malloc(nodesize); + if (target == NULL) { + // (void) syslog(LOG_ERR, mem_err_msg_arr); + return (FALSE); + } + (void) memset(target, 0, nodesize); + break; + + case XDR_FREE: + return (TRUE); + } + + /* + * now we xdr each element of array + */ + for (i = 0; (i < c) && stat; i++) { + stat = (*elproc)(xdrs, target); + target += elsize; + } + + /* + * the array may need freeing + */ + if (xdrs->x_op == XDR_FREE) { + free(*addrp); + *addrp = NULL; + } + return (stat); +} + +/* + * xdr_vector(): + * + * XDR a fixed length array. Unlike variable-length arrays, + * the storage of fixed length arrays is static and unfreeable. + * > basep: base of the array + * > size: size of the array + * > elemsize: size of each element + * > xdr_elem: routine to XDR each element + */ +bool_t +xdr_vector(XDR *xdrs, char *basep, const uint_t nelem, + const uint_t elemsize, const xdrproc_t xdr_elem) +{ + uint_t i; + char *elptr; + + elptr = basep; + for (i = 0; i < nelem; i++) { + if (!(*xdr_elem)(xdrs, elptr, LASTUNSIGNED)) + return (FALSE); + elptr += elemsize; + } + return (TRUE); +} diff --git a/lib/libspl/os/windows/xdr_float.c b/lib/libspl/os/windows/xdr_float.c new file mode 100644 index 000000000000..bb007a6001cc --- /dev/null +++ b/lib/libspl/os/windows/xdr_float.c @@ -0,0 +1,319 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2006 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ +/* + * Portions of this source code were derived from Berkeley + * 4.3 BSD under license from the Regents of the University of + * California. + */ +/* + * Copyright 2011 Jason King. All rights reserved + */ + +/* + * Generic XDR routines impelmentation. + * + * These are the "floating point" xdr routines used to (de)serialize + * most common data items. See xdr.h for more info on the interface to + * xdr. + */ + +#include +#include +#include +#include +#include +#include + +// incompatible types - from 'XDR *' to 'XDR *' +#pragma warning(disable: 4133) + +#define _IEEE_754 +#ifdef _IEEE_754 + +/* + * The OTW format is IEEE 754 with big endian ordering. + */ +bool_t +xdr_float(XDR *xdrs, float *fp) +{ + switch (xdrs->x_op) { + + case XDR_ENCODE: + return (XDR_PUTINT32(xdrs, (int *)fp)); + + case XDR_DECODE: + return (XDR_GETINT32(xdrs, (int *)fp)); + + case XDR_FREE: + return (TRUE); + } + return (FALSE); +} + +bool_t +xdr_double(XDR *xdrs, double *dp) +{ + int64_t *i64p = (int64_t *)dp; + int64_t val; + bool_t ret; + + switch (xdrs->x_op) { + + case XDR_ENCODE: + val = BE_64(*i64p); + return (XDR_PUTBYTES(xdrs, (char *)&val, sizeof (val))); + + case XDR_DECODE: + ret = XDR_GETBYTES(xdrs, (char *)dp, sizeof (double)); + if (ret) + *i64p = BE_64(*i64p); + return (ret); + + case XDR_FREE: + return (TRUE); + } + + return (FALSE); +} + + +bool_t +xdr_quadruple(XDR *xdrs, long double *fp) +{ +/* + * The Sparc uses IEEE FP encoding, so just do a byte copy + */ + +#if !defined(sparc) + return (FALSE); +#else + switch (xdrs->x_op) { + case XDR_ENCODE: + return (XDR_PUTBYTES(xdrs, (char *)fp, sizeof (long double))); + case XDR_DECODE: + return (XDR_GETBYTES(xdrs, (char *)fp, sizeof (long double))); + case XDR_FREE: + return (TRUE); + } + return (FALSE); +#endif +} + +#else + +#warn No platform specific implementation defined for floats + +bool_t +xdr_float(XDR *xdrs, float *fp) +{ + /* + * Every machine can do this, its just not very efficient. + * In addtion, some rounding errors may occur do to the + * calculations involved. + */ + float f; + int neg = 0; + int exp = 0; + int32_t val; + + switch (xdrs->x_op) { + case XDR_ENCODE: + f = *fp; + if (f == 0) { + val = 0; + return (XDR_PUTINT32(xdrs, &val)); + } + if (f < 0) { + f = 0 - f; + neg = 1; + } + while (f < 1) { + f = f * 2; + --exp; + } + while (f >= 2) { + f = f/2; + ++exp; + } + if ((exp > 128) || (exp < -127)) { + /* over or under flowing ieee exponent */ + return (FALSE); + } + val = neg; + val = val << 8; /* for the exponent */ + val += 127 + exp; /* 127 is the bias */ + val = val << 23; /* for the mantissa */ + val += (int32_t)((f - 1) * 8388608); /* 2 ^ 23 */ + return (XDR_PUTINT32(xdrs, &val)); + + case XDR_DECODE: + /* + * It assumes that the decoding machine's float can represent + * any value in the range of + * ieee largest float = (2 ^ 128) * 0x1.fffff + * to + * ieee smallest float = (2 ^ -127) * 0x1.00000 + * In addtion, some rounding errors may occur do to the + * calculations involved. + */ + + if (!XDR_GETINT32(xdrs, (int32_t *)&val)) + return (FALSE); + neg = val & 0x80000000; + exp = (val & 0x7f800000) >> 23; + exp -= 127; /* subtract exponent base */ + f = (val & 0x007fffff) * 0.00000011920928955078125; + /* 2 ^ -23 */ + f++; + + while (exp != 0) { + if (exp < 0) { + f = f/2.0; + ++exp; + } else { + f = f * 2.0; + --exp; + } + } + + if (neg) + f = 0 - f; + + *fp = f; + return (TRUE); + + case XDR_FREE: + return (TRUE); + } + + return (FALSE); +} + +bool_t +xdr_double(XDR *xdrs, double *dp) +{ + /* + * Every machine can do this, its just not very efficient. + * In addtion, some rounding errors may occur do to the + * calculations involved. + */ + + int *lp; + double d; + int neg = 0; + int exp = 0; + int32_t val[2]; + + switch (xdrs->x_op) { + case XDR_ENCODE: + d = *dp; + if (d == 0) { + val[0] = 0; + val[1] = 0; + lp = val; + return (XDR_PUTINT32(xdrs, lp++) && + XDR_PUTINT32(xdrs, lp)); + } + if (d < 0) { + d = 0 - d; + neg = 1; + } + while (d < 1) { + d = d * 2; + --exp; + } + while (d >= 2) { + d = d/2; + ++exp; + } + if ((exp > 1024) || (exp < -1023)) { + /* over or under flowing ieee exponent */ + return (FALSE); + } + val[0] = (neg << 11); /* for the exponent */ + val[0] += 1023 + exp; /* 1023 is the bias */ + val[0] = val[0] << 20; /* for the mantissa */ + val[0] += (int32_t)((d - 1) * 1048576); /* 2 ^ 20 */ + val[1] += (uint32_t)((((d - 1) * 1048576) - val[0]) * + 4294967296); /* 2 ^ 32 */ + lp = val; + + return (XDR_PUTINT32(xdrs, lp++) && XDR_PUTINT32(xdrs, lp)); + + case XDR_DECODE: + /* + * It assumes that the decoding machine's + * double can represent any value in the range of + * ieee largest double = (2 ^ 1024) * 0x1.fffffffffffff + * to + * ieee smallest double = (2 ^ -1023) * 0x1.0000000000000 + * In addtion, some rounding errors may occur do to the + * calculations involved. + */ + + lp = val; + if (!XDR_GETINT32(xdrs, lp++) || !XDR_GETINT32(xdrs, lp)) + return (FALSE); + neg = val[0] & 0x80000000; + exp = (val[0] & 0x7ff00000) >> 20; + exp -= 1023; /* subtract exponent base */ + d = (val[0] & 0x000fffff) * 0.00000095367431640625; + /* 2 ^ -20 */ + d += (val[1] * 0.0000000000000002220446049250313); + /* 2 ^ -52 */ + d++; + while (exp != 0) { + if (exp < 0) { + d = d/2.0; + ++exp; + } else { + d = d * 2.0; + --exp; + } + } + if (neg) + d = 0 - d; + + *dp = d; + return (TRUE); + + case XDR_FREE: + return (TRUE); + } + + return (FALSE); +} + +bool_t +xdr_quadruple(XDR *xdrs, long double *fp) +{ + return (FALSE); +} + +#endif /* _IEEE_754 */ diff --git a/lib/libspl/os/windows/xdr_mem.c b/lib/libspl/os/windows/xdr_mem.c new file mode 100644 index 000000000000..6906e935c6f0 --- /dev/null +++ b/lib/libspl/os/windows/xdr_mem.c @@ -0,0 +1,267 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2005 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ +/* Copyright (c) 1983, 1984, 1985, 1986, 1987, 1988, 1989 AT&T */ +/* All Rights Reserved */ +/* + * Portions of this source code were derived from Berkeley + * 4.3 BSD under license from the Regents of the University of + * California. + */ + +/* + * xdr_mem.h, XDR implementation using memory buffers. + * + * If you have some data to be interpreted as external data representation + * or to be converted to external data representation in a memory buffer, + * then this is the package for you. + */ + +#include +#include +#include +#include +#include +#include + +static struct xdr_ops *xdrmem_ops(void); + +// formal parameter 1 different from declaration (XDR* != struct XDR *)? +#pragma warning(disable: 4028) + +/* + * Meaning of the private areas of the xdr struct for xdr_mem + * x_base : Base from where the xdr stream starts + * x_private : The current position of the stream. + * x_handy : The size of the stream buffer. + */ + +/* + * The procedure xdrmem_create initializes a stream descriptor for a + * memory buffer. + */ +void +xdrmem_create(XDR *xdrs, const caddr_t addr, const uint_t size, + const enum xdr_op op) +{ + caddr_t eaddr = addr; + + xdrs->x_op = op; + xdrs->x_ops = xdrmem_ops(); + xdrs->x_private = xdrs->x_base = 0; + /* + * We check here that the size is with in the range of the + * address space. If not we set x_handy to zero. This will cause + * all xdrmem entry points to fail. + */ + eaddr = addr + size; + + if (eaddr < addr) + xdrs->x_handy = 0; + else { + xdrs->x_handy = size; + xdrs->x_private = xdrs->x_base = addr; + } +} + + +static void +xdrmem_destroy(XDR *xdrs) +{ +} + +static bool_t +xdrmem_getlong(XDR *xdrs, long *lp) +{ + if (sizeof (int32_t) > (uint32_t)xdrs->x_handy) { + xdrs->x_private += (uint_t)xdrs->x_handy; + xdrs->x_handy = 0; + return (FALSE); + } + xdrs->x_handy -= sizeof (int32_t); + /* LINTED pointer cast */ + *lp = (int32_t)ntohl((uint32_t)(*((int32_t *)(xdrs->x_private)))); + xdrs->x_private += sizeof (int32_t); + return (TRUE); +} + +static bool_t +xdrmem_putlong(XDR *xdrs, long *lp) +{ +#if defined(_LP64) + if ((*lp > INT32_MAX) || (*lp < INT32_MIN)) + return (FALSE); +#endif + + if ((sizeof (int32_t) > (uint32_t)xdrs->x_handy)) { + xdrs->x_private += (uint_t)xdrs->x_handy; + xdrs->x_handy = 0; + return (FALSE); + } + xdrs->x_handy -= sizeof (int32_t); + /* LINTED pointer cast */ + *(int32_t *)xdrs->x_private = (int32_t)htonl((uint32_t)(*lp)); + xdrs->x_private += sizeof (int32_t); + return (TRUE); +} + +#if defined(_LP64) +static bool_t +xdrmem_getint32(XDR *xdrs, int32_t *ip) +{ + if (sizeof (int32_t) > (uint_t)xdrs->x_handy) { + xdrs->x_private += (uint_t)xdrs->x_handy; + xdrs->x_handy = 0; + return (FALSE); + } + xdrs->x_handy -= sizeof (int32_t); + /* LINTED pointer cast */ + *ip = (int32_t)ntohl((uint32_t)(*((int32_t *)(xdrs->x_private)))); + xdrs->x_private += sizeof (int32_t); + return (TRUE); +} + +static bool_t +xdrmem_putint32(XDR *xdrs, int32_t *ip) +{ + if (sizeof (int32_t) > (uint32_t)xdrs->x_handy) { + xdrs->x_private += (uint_t)xdrs->x_handy; + xdrs->x_handy = 0; + return (FALSE); + } + xdrs->x_handy -= sizeof (int32_t); + /* LINTED pointer cast */ + *(int32_t *)xdrs->x_private = (int32_t)htonl((uint32_t)(*ip)); + xdrs->x_private += sizeof (int32_t); + return (TRUE); +} +#endif /* _LP64 */ + +static bool_t +xdrmem_getbytes(XDR *xdrs, caddr_t addr, int len) +{ + if ((uint32_t)len > (uint32_t)xdrs->x_handy) { + xdrs->x_private += (uint_t)xdrs->x_handy; + xdrs->x_handy = 0; + return (FALSE); + } + xdrs->x_handy -= len; + (void) memcpy(addr, xdrs->x_private, (uint_t)len); + xdrs->x_private += (uint_t)len; + return (TRUE); +} + +static bool_t +xdrmem_putbytes(XDR *xdrs, caddr_t addr, int len) +{ + if ((uint32_t)len > (uint32_t)xdrs->x_handy) { + xdrs->x_private += (uint_t)xdrs->x_handy; + xdrs->x_handy = 0; + return (FALSE); + } + xdrs->x_handy -= len; + (void) memcpy(xdrs->x_private, addr, (uint_t)len); + xdrs->x_private += (uint_t)len; + return (TRUE); +} + +static uint_t +xdrmem_getpos(XDR *xdrs) +{ + return (uint_t)((uintptr_t)xdrs->x_private - (uintptr_t)xdrs->x_base); +} + +static bool_t +xdrmem_setpos(XDR *xdrs, uint_t pos) +{ + caddr_t newaddr = xdrs->x_base + pos; + caddr_t lastaddr = xdrs->x_private + (uint_t)xdrs->x_handy; + + if ((long)newaddr > (long)lastaddr) + return (FALSE); + xdrs->x_private = newaddr; + xdrs->x_handy = (int)((uintptr_t)lastaddr - (uintptr_t)newaddr); + return (TRUE); +} + +static rpc_inline_t * +xdrmem_inline(XDR *xdrs, int len) +{ + rpc_inline_t *buf = 0; + + if ((uint32_t)xdrs->x_handy >= (uint32_t)len) { + xdrs->x_handy -= len; + /* LINTED pointer cast */ + buf = (rpc_inline_t *)xdrs->x_private; + xdrs->x_private += (uint_t)len; + } + return (buf); +} + +static bool_t +xdrmem_control(XDR *xdrs, int request, void *info) +{ + struct xdr_bytesrec *xptr; + + switch (request) { + case XDR_GET_BYTES_AVAIL: + xptr = (struct xdr_bytesrec *)info; + xptr->xc_is_last_record = TRUE; + xptr->xc_num_avail = xdrs->x_handy; + return (TRUE); + default: + return (FALSE); + + } + +} + +static struct xdr_ops * +xdrmem_ops(void) +{ + static struct xdr_ops ops; +// extern mutex_t ops_lock; + +/* VARIABLES PROTECTED BY ops_lock: ops */ +// (void) mutex_lock(&ops_lock); + if (ops.x_getlong == NULL) { + ops.x_getlong = xdrmem_getlong; + ops.x_putlong = xdrmem_putlong; + ops.x_getbytes = xdrmem_getbytes; + ops.x_putbytes = xdrmem_putbytes; + ops.x_getpostn = xdrmem_getpos; + ops.x_setpostn = xdrmem_setpos; + ops.x_inline = xdrmem_inline; + ops.x_destroy = xdrmem_destroy; + ops.x_control = xdrmem_control; +#if defined(_LP64) + ops.x_getint32 = xdrmem_getint32; + ops.x_putint32 = xdrmem_putint32; +#endif + } +// (void) mutex_unlock(&ops_lock); + return (&ops); +} diff --git a/lib/libspl/os/windows/zone.c b/lib/libspl/os/windows/zone.c new file mode 100644 index 000000000000..a1e4f524094c --- /dev/null +++ b/lib/libspl/os/windows/zone.c @@ -0,0 +1,28 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include + +zoneid_t +getzoneid() +{ + return (GLOBAL_ZONEID); +} diff --git a/lib/libtpool/CMakeLists.txt b/lib/libtpool/CMakeLists.txt new file mode 100644 index 000000000000..2f8f78a360b6 --- /dev/null +++ b/lib/libtpool/CMakeLists.txt @@ -0,0 +1,8 @@ + +use_clang() + +add_library(libtpool + thread_pool.c +) +target_include_directories(libspl BEFORE PUBLIC include) +target_link_libraries(libspl PRIVATE libpthread advapi32 shell32) diff --git a/lib/libunicode/CMakeLists.txt b/lib/libunicode/CMakeLists.txt new file mode 100644 index 000000000000..470e9c1cb66b --- /dev/null +++ b/lib/libunicode/CMakeLists.txt @@ -0,0 +1,9 @@ + +use_clang() + +set(UNICODDE_MODULE_DIR "../../module/unicode") +add_library(libunicode + "${UNICODDE_MODULE_DIR}/u8_textprep.c" + "${UNICODDE_MODULE_DIR}/uconv.c" +) +target_link_libraries(libunicode PRIVATE libspl) diff --git a/lib/libuutil/CMakeLists.txt b/lib/libuutil/CMakeLists.txt new file mode 100644 index 000000000000..e289f343b026 --- /dev/null +++ b/lib/libuutil/CMakeLists.txt @@ -0,0 +1,13 @@ + +use_clang() + +add_library(libuutil + uu_alloc.c + uu_avl.c + uu_ident.c + uu_list.c + uu_misc.c +# uu_pname.c + uu_string.c +) +target_link_libraries(libuutil PUBLIC libspl libpthread) diff --git a/lib/libzfs/CMakeLists.txt b/lib/libzfs/CMakeLists.txt new file mode 100644 index 000000000000..2495c893fda4 --- /dev/null +++ b/lib/libzfs/CMakeLists.txt @@ -0,0 +1,38 @@ + +use_clang() + +add_library(libzfs + libzfs_changelist.c + libzfs_config.c + libzfs_crypto.c + libzfs_dataset.c + libzfs_diff.c + libzfs_import.c + libzfs_iter.c + libzfs_mount.c + libzfs_pool.c + libzfs_sendrecv.c + libzfs_status.c + libzfs_util.c + os/windows/libzfs_mount_os.c + os/windows/libzfs_pool_os.c + os/windows/libzfs_util_os.c +) + +#variable_watch(CRYPTO_STATIC) +# set(CRYPTO_STATIC "notset") +set(CMAKE_FIND_DEBUG_MODE TRUE) +find_library(CRYPTO_STATIC_TEST + NAMES + libcrypto64MTd + NAMES_PER_DIR + HINTS + "C:/Program Files/OpenSSL-Win64/lib/VC/static" + PATH_SUFFIXES + lib + REQUIRED + ) + +target_include_directories(libzfs PRIVATE "${CMAKE_SOURCE_DIR}/lib/libzfs") +target_link_libraries(libzfs PUBLIC libpthread zlib libzutil libshare libzfs_core libnvpair libuutil) +target_link_libraries(libzfs PRIVATE Crypt32.lib ${CRYPTO_STATIC_TEST}) diff --git a/lib/libzfs/libzfs.abi b/lib/libzfs/libzfs.abi index 6e53bcb41a87..e3cfbb08d88d 100644 --- a/lib/libzfs/libzfs.abi +++ b/lib/libzfs/libzfs.abi @@ -413,6 +413,7 @@ + @@ -430,7 +431,9 @@ + + @@ -1862,7 +1865,9 @@ - + + + @@ -7907,6 +7912,17 @@ + + + + + + + + + + + @@ -8134,6 +8150,10 @@ + + + + @@ -9033,7 +9053,12 @@ - + + + + + + @@ -9050,7 +9075,7 @@ - + diff --git a/lib/libzfs/libzfs_crypto.c b/lib/libzfs/libzfs_crypto.c index 8f2a50d55e87..f5231bdda968 100644 --- a/lib/libzfs/libzfs_crypto.c +++ b/lib/libzfs/libzfs_crypto.c @@ -79,6 +79,7 @@ static zfs_uri_handler_t uri_handlers[] = { { NULL, NULL } }; +#ifndef _WIN32 static int pkcs11_get_urandom(uint8_t *buf, size_t bytes) { @@ -101,6 +102,15 @@ pkcs11_get_urandom(uint8_t *buf, size_t bytes) return (bytes_read); } +#else +static int +pkcs11_get_urandom(uint8_t *buf, size_t bytes) +{ + // random_init()/random_fini() are empty + random_get_bytes((uint8_t *)buf, bytes); + return (bytes); +} +#endif static int zfs_prop_parse_keylocation(libzfs_handle_t *restrict hdl, const char *str, diff --git a/lib/libzfs/libzfs_dataset.c b/lib/libzfs/libzfs_dataset.c index fe9f3268d338..b87f32e09ba7 100644 --- a/lib/libzfs/libzfs_dataset.c +++ b/lib/libzfs/libzfs_dataset.c @@ -40,7 +40,6 @@ #include #include #include -#include #include #include #include @@ -904,8 +903,12 @@ libzfs_mnttab_find(libzfs_handle_t *hdl, const char *fsname, if (avl_numnodes(&hdl->libzfs_mnttab_cache)) libzfs_mnttab_fini(hdl); - +#ifdef WIN32 + // Unfortunately Windows abort() on unknown file mode + if ((mnttab = fopen(MNTTAB, "rb")) == NULL) +#else if ((mnttab = fopen(MNTTAB, "re")) == NULL) +#endif return (ENOENT); srch.mnt_special = (char *)fsname; @@ -1846,7 +1849,12 @@ zfs_prop_set_list(zfs_handle_t *zhp, nvlist_t *props) if (prop != ZFS_PROP_CANMOUNT || (fnvpair_value_uint64(elem) == ZFS_CANMOUNT_OFF && zfs_is_mounted(zhp, NULL))) { +#ifdef _WIN32 + cls[cl_idx] = changelist_gather(zhp, prop, + CL_GATHER_DONT_UNMOUNT, 0); +#else cls[cl_idx] = changelist_gather(zhp, prop, 0, 0); +#endif if (cls[cl_idx] == NULL) goto error; } @@ -2689,7 +2697,12 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char *propbuf, size_t proplen, if (literal || localtime_r(&time, &t) == NULL || - strftime(propbuf, proplen, "%a %b %e %k:%M %Y", + strftime(propbuf, proplen, +#ifdef _WIN32 + "%a %b %d %H:%M %Y", +#else + "%a %b %e %k:%M %Y", +#endif &t) == 0) (void) snprintf(propbuf, proplen, "%llu", (u_longlong_t)val); diff --git a/lib/libzfs/libzfs_iter.c b/lib/libzfs/libzfs_iter.c index 452d8fd6ab71..e1404475a3c1 100644 --- a/lib/libzfs/libzfs_iter.c +++ b/lib/libzfs/libzfs_iter.c @@ -634,17 +634,19 @@ zfs_iter_mounted(zfs_handle_t *zhp, zfs_iter_f func, void *data) continue; if ((mtab_zhp = zfs_open(zhp->zfs_hdl, entry.mnt_special, - ZFS_TYPE_FILESYSTEM)) == NULL) + ZFS_TYPE_FILESYSTEM|ZFS_TYPE_SNAPSHOT)) == NULL) continue; /* Ignore legacy mounts as they are user managed */ - verify(zfs_prop_get(mtab_zhp, ZFS_PROP_MOUNTPOINT, mnt_prop, - sizeof (mnt_prop), NULL, NULL, 0, B_FALSE) == 0); - if (strcmp(mnt_prop, "legacy") == 0) { - zfs_close(mtab_zhp); - continue; + if (mtab_zhp->zfs_type != ZFS_TYPE_SNAPSHOT) { + verify(zfs_prop_get(mtab_zhp, ZFS_PROP_MOUNTPOINT, + mnt_prop, sizeof (mnt_prop), NULL, NULL, 0, + B_FALSE) == 0); + if (strcmp(mnt_prop, "legacy") == 0) { + zfs_close(mtab_zhp); + continue; + } } - err = func(mtab_zhp, data); } diff --git a/lib/libzfs/libzfs_mount.c b/lib/libzfs/libzfs_mount.c index 5d1fe651c97e..3f20e4baf9d5 100644 --- a/lib/libzfs/libzfs_mount.c +++ b/lib/libzfs/libzfs_mount.c @@ -462,6 +462,7 @@ zfs_mount_at(zfs_handle_t *zhp, const char *options, int flags, strlcat(mntopts, "," MNTOPT_ZFSUTIL, sizeof (mntopts)); /* Create the directory if it doesn't already exist */ +#ifndef _WIN32 if (lstat(mountpoint, &buf) != 0) { if (mkdirp(mountpoint, 0755) != 0) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, @@ -472,7 +473,7 @@ zfs_mount_at(zfs_handle_t *zhp, const char *options, int flags, mountpoint)); } } - +#endif /* * Overlay mounts are enabled by default but may be disabled * via the 'overlay' property. The -O flag remains for compatibility. diff --git a/lib/libzfs/libzfs_pool.c b/lib/libzfs/libzfs_pool.c index d4af31c50cf8..f66c205fb13e 100644 --- a/lib/libzfs/libzfs_pool.c +++ b/lib/libzfs/libzfs_pool.c @@ -36,7 +36,6 @@ #include #include #include -#include #include #include #include @@ -4962,9 +4961,9 @@ zpool_load_compat(const char *compat, boolean_t *features, char *report, /* replace final newline with NULL to ensure string ends */ fc[fs.st_size - 1] = '\0'; - for (line = strtok_r(fc, "\n", &ls); + for (line = strtok_r(fc, "\r\n", &ls); line != NULL; - line = strtok_r(NULL, "\n", &ls)) { + line = strtok_r(NULL, "\r\n", &ls)) { /* discard comments */ char *r = strchr(line, '#'); if (r != NULL) diff --git a/lib/libzfs/libzfs_util.c b/lib/libzfs/libzfs_util.c index b94abea3d581..efcb0cd88135 100644 --- a/lib/libzfs/libzfs_util.c +++ b/lib/libzfs/libzfs_util.c @@ -41,7 +41,6 @@ #include #include #include -#include #include #include #if LIBFETCH_DYNAMIC @@ -902,6 +901,7 @@ libzfs_read_stdout_from_fd(int fd, char **lines[]) return (lines_cnt); } +#ifndef _WIN32 static int libzfs_run_process_impl(const char *path, char *argv[], char *env[], int flags, char **lines[], int *lines_cnt) @@ -967,6 +967,7 @@ libzfs_run_process_impl(const char *path, char *argv[], char *env[], int flags, return (-1); } +#endif int libzfs_run_process(const char *path, char *argv[], int flags) diff --git a/lib/libzfs/os/freebsd/libzfs_compat.c b/lib/libzfs/os/freebsd/libzfs_compat.c index d1c1fea7fb68..9740afd1f1f3 100644 --- a/lib/libzfs/os/freebsd/libzfs_compat.c +++ b/lib/libzfs/os/freebsd/libzfs_compat.c @@ -370,3 +370,10 @@ zfs_version_kernel(void) } return (version); } + +/* Called from the tail end of zfs_rollback() */ +void +zfs_rollback_os(zfs_handle_t *zhp) +{ + (void) zhp; +} diff --git a/lib/libzfs/os/freebsd/libzfs_zmount.c b/lib/libzfs/os/freebsd/libzfs_zmount.c index 34976f7bbf46..d3a4aca88198 100644 --- a/lib/libzfs/os/freebsd/libzfs_zmount.c +++ b/lib/libzfs/os/freebsd/libzfs_zmount.c @@ -137,3 +137,19 @@ zpool_disable_volume_os(const char *name) { (void) name; } + +/* Called for manual "zfs mount snapshot" */ +int +zfs_snapshot_mount(zfs_handle_t *zhp, const char *options, int flags) +{ + (void) zhp, (void) options, (void) flags; + return (0); +} + +/* Called for manual "zfs unmount snapshot" */ +int +zfs_snapshot_unmount(zfs_handle_t *zhp, int flags) +{ + (void) zhp, (void) flags; + return (0); +} diff --git a/lib/libzfs/os/linux/libzfs_mount_os.c b/lib/libzfs/os/linux/libzfs_mount_os.c index f0bf3dcc6c6b..ec34ba67d363 100644 --- a/lib/libzfs/os/linux/libzfs_mount_os.c +++ b/lib/libzfs/os/linux/libzfs_mount_os.c @@ -428,3 +428,19 @@ zpool_disable_volume_os(const char *name) { (void) name; } + +/* Called for manual "zfs mount snapshot" */ +int +zfs_snapshot_mount(zfs_handle_t *zhp, const char *options, int flags) +{ + (void) zhp, (void) options, (void) flags; + return (0); +} + +/* Called for manual "zfs unmount snapshot" */ +int +zfs_snapshot_unmount(zfs_handle_t *zhp, int flags) +{ + (void) zhp, (void) flags; + return (0); +} diff --git a/lib/libzfs/os/linux/libzfs_util_os.c b/lib/libzfs/os/linux/libzfs_util_os.c index 99faae66833e..fce22e73596c 100644 --- a/lib/libzfs/os/linux/libzfs_util_os.c +++ b/lib/libzfs/os/linux/libzfs_util_os.c @@ -278,3 +278,10 @@ zfs_userns(zfs_handle_t *zhp, const char *nspath, int attach) return (ret); } + +/* Called from the tail end of zfs_rollback() */ +void +zfs_rollback_os(zfs_handle_t *zhp) +{ + (void) zhp; +} diff --git a/lib/libzfs/os/windows/libzfs_mount_os.c b/lib/libzfs/os/windows/libzfs_mount_os.c new file mode 100644 index 000000000000..8f9356b8d7de --- /dev/null +++ b/lib/libzfs/os/windows/libzfs_mount_os.c @@ -0,0 +1,467 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2014, 2019 by Delphix. All rights reserved. + * Copyright 2016 Igor Kozhukhov + * Copyright 2017 RackTop Systems. + * Copyright (c) 2018 Datto Inc. + * Copyright 2018 OmniOS Community Edition (OmniOSce) Association. + */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "libzfs_impl.h" +#include + +#include + +/* + * if (zmount(zhp, zfs_get_name(zhp), mountpoint, MS_OPTIONSTR | flags, + * MNTTYPE_ZFS, NULL, 0, mntopts, sizeof (mntopts)) != 0) { + */ +int +do_mount(zfs_handle_t *zhp, const char *dir, const char *optptr, int mflag) +{ + int ret = 0; + int ispool = 0; + char driveletter[100] = "off"; + int hasprop = 0; + + /* mount 'spec' "tank/joe" on path 'dir' "/home/joe". */ +#ifdef DEBUG + fprintf(stderr, + "zmount running, emulating Unix mount: '%s'\r\n", + dir); + fflush(stderr); +#endif + zfs_cmd_t zc = { "\0" }; + + if (zhp) { + if (zhp->zpool_hdl && + strcmp(zpool_get_name(zhp->zpool_hdl), + zfs_get_name(zhp)) == 0) + ispool = 1; + + ret = zfs_prop_get(zhp, ZFS_PROP_DRIVELETTER, driveletter, + sizeof (driveletter), NULL, NULL, 0, B_FALSE); + + hasprop = ret ? 0 : 1; + if (!ret && + strncmp("-", driveletter, sizeof (driveletter)) == 0) + hasprop = 0; + } + /* + * if !hasprop and ispool -> hasprop=1 & driveletter=on + * if hasprop = on -> driveletter = ? + * if hasprop = off + */ + if (!hasprop && ispool) { + strcpy(driveletter, "on"); + hasprop = 1; + } + if (strcmp("off", driveletter) == 0) + hasprop = 0; + else if (strcmp("on", driveletter) == 0) + strcpy(driveletter, "?"); + + (void) strlcpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); + (void) strlcpy(zc.zc_value, dir, sizeof (zc.zc_value)); + + + if (zhp->zfs_type != ZFS_TYPE_SNAPSHOT) { + /* + * If hasprop is set, use 'driveletter' and ignore mountpoint + * path. if !hasprop && rootds same + */ + if (hasprop) { + /* We just pass "\\??\\X:" to kernel. */ + snprintf(zc.zc_value, sizeof (zc.zc_value), "\\??\\%c:", + tolower(driveletter[0])); + } else { + /* + * We are to mount with path. Attempt to find parent + * driveletter, if any. Otherwise assume c:/ + */ + driveletter[0] = 'c'; + + if (!ispool) { + char parent[ZFS_MAX_DATASET_NAME_LEN] = ""; + char *slashp; + struct mnttab entry = { 0 }; + + zfs_parent_name(zhp, parent, sizeof (parent)); + + while (strlen(parent) >= 1) { + if ((libzfs_mnttab_find(zhp->zfs_hdl, + parent, + &entry) == 0) && + (entry.mnt_mountp[1] == ':')) { + driveletter[0] = + entry.mnt_mountp[0]; +#ifdef DEBUG + fprintf(stderr, + "we think '%s' parent is '%s' and its mounts are: '%s'\r\n", + zfs_get_name(zhp), parent, entry.mnt_mountp); + fflush(stderr); +#endif + break; + } + if ((slashp = strrchr(parent, '/')) == + NULL) + break; + *slashp = '\0'; + } + + /* + * We need to skip the parent name part, in mountpoint "dir" here,ie + * if parent is "BOOM/lower" we need to skip to the 3nd slash + * in "/BOOM/lower/newfs" + * So, check if the mounted name is in the string + */ + // "BOOM" -> "/BOOM/" + snprintf(parent, sizeof (parent), "/%s/", + entry.mnt_special); + char *part = strstr(dir, parent); + if (part) dir = &part[strlen(parent) - 1]; + } + + snprintf(zc.zc_value, sizeof (zc.zc_value), + "\\??\\%c:%s", tolower(driveletter[0]), dir); + } + } else { + /* snapshot */ + snprintf(zc.zc_value, sizeof (zc.zc_value), "\\??\\%s", + dir); + zc.zc_cleanup_fd = MNT_RDONLY; + } + + /* Convert Unix slash to Win32 backslash */ + for (int i = 0; zc.zc_value[i]; i++) + if (zc.zc_value[i] == '/') + zc.zc_value[i] = '\\'; /* "\\??\\c:\\BOOM\\lower" */ +#ifdef DEBUG + fprintf(stderr, "zmount(%s,'%s') hasprop %d ispool %d\n", + zhp->zfs_name, zc.zc_value, hasprop, ispool); + fflush(stderr); +#endif + ret = zfs_ioctl(zhp->zfs_hdl, ZFS_IOC_MOUNT, &zc); + + + if (ret == 0) { + /* + * Tell Explorer we have a new drive + * Whats the deal here with this header file - + * did not like to be included. + * #include + */ + struct mnttab entry; + + /* Locate this mount */ + if (libzfs_mnttab_find(zhp->zfs_hdl, zhp->zfs_name, + &entry) == 0) { + /* + * If we get a driveletter, we tell Explorer. + * Otherwise not required. + */ + if (entry.mnt_mountp[1] == ':') { // "E:\ " -> "E:" + entry.mnt_mountp[2] = 0; + SHChangeNotify(SHCNE_DRIVEADD, SHCNF_PATH, + entry.mnt_mountp, NULL); + } + } + } + +#ifdef DEBUG + fprintf(stderr, "zmount(%s,%s) returns %d\n", + zhp->zfs_name, dir, ret); + + fprintf(stderr, "'%s' mounted on %s\r\n", zc.zc_name, zc.zc_value); +#endif + /* + * For BOOM, we get back + * "\\Device\\Volume{0b1bb601-af0b-32e8-a1d2-54c167af6277}\\" + * which is the volume name, and the FS device attached to it is: + * "\\\??\\\Volume{7cc383a0-beac-11e7-b56d-02150b22a130}" + * and if change that to + * "\\\\?\\Volume{7cc383a0-beac-11e7-b56d-02150b22a130}\\"; + * we can use GetVolumePathNamesForVolumeName() + * to get back "\\DosDevices\\E". + */ + + return (ret); +} + + +static int +do_unmount_impl(zfs_handle_t *zhp, const char *mntpt, int flags) +{ + int ret = 0; + + /* mount 'spec' "tank/joe" on path 'dir' "/home/joe". */ + fprintf(stderr, "zunmount(%s,%s) running\r\n", + zhp->zfs_name, mntpt); + fflush(stderr); + zfs_cmd_t zc = { "\0" }; + + (void) strlcpy(zc.zc_name, zhp->zfs_name, sizeof (zc.zc_name)); + (void) strlcpy(zc.zc_value, mntpt, sizeof (zc.zc_value)); + + ret = zfs_ioctl(zhp->zfs_hdl, ZFS_IOC_UNMOUNT, &zc); + + if (!ret) { + /* + * if mountpoint is a folder, we need to turn it back + * from JUNCTION to a real folder + */ + char mtpt_prop[ZFS_MAXPROPLEN]; + char driveletter[MAX_PATH]; + /* snapshots dont have mountpoint property */ + if (zfs_prop_get(zhp, ZFS_PROP_MOUNTPOINT, mtpt_prop, + sizeof (mtpt_prop), NULL, NULL, 0, B_FALSE) == 0) { + verify(zfs_prop_get(zhp, ZFS_PROP_DRIVELETTER, + driveletter, sizeof (driveletter), NULL, + NULL, 0, B_FALSE) == 0); + /* + * if mountpoint starts with '/' we assume that it is a + * path to a directory make sure we didn't mount as + * driveletter + */ + if (mtpt_prop && mtpt_prop[0] == '/' && + (strstr(driveletter, "-") != 0 || + strstr(driveletter, "off") != 0) && + (mntpt && strstr(mntpt, ":\\") == 0)) { + BOOL val = RemoveDirectoryA(mtpt_prop); + if (!val) { + } else { + val = CreateDirectoryA(mtpt_prop, NULL); + } + + } + } + } + + fprintf(stderr, "zunmount(%s,%s) returns %d\n", + zhp->zfs_name, mntpt, ret); + + return (ret); +} + + +void unmount_snapshots(zfs_handle_t *zhp, const char *mntpt, int flags); + +int +do_unmount(zfs_handle_t *zhp, const char *mntpt, int flags) +{ + + unmount_snapshots(zhp, mntpt, flags); + + return (do_unmount_impl(zhp, mntpt, flags)); +} + +/* + * Given "/Volumes/BOOM" look for any lower mounts with ".zfs/snapshot/" + * in them - issue unmount. + */ +void +unmount_snapshots(zfs_handle_t *zhp, const char *mntpt, int flags) +{ + struct mnttab entry; + int len = strlen(mntpt); + + while (getmntent(NULL, &entry) == 0) { + /* Starts with our mountpoint ? */ + if (strncmp(mntpt, entry.mnt_mountp, len) == 0) { + /* The next part is "/.zfs/snapshot/" ? */ + if (strncmp("/.zfs/snapshot/", &entry.mnt_mountp[len], + 15) == 0) { + /* Unmount it */ + do_unmount_impl(zhp, entry.mnt_mountp, + MS_FORCE); + } + } + } +} + +int +zfs_mount_delegation_check(void) +{ + return ((geteuid() != 0) ? EACCES : 0); +} + +static char * +zfs_snapshot_mountpoint(zfs_handle_t *zhp) +{ + char *dataset_name, *snapshot_mountpoint, *parent_mountpoint; + libzfs_handle_t *hdl = zhp->zfs_hdl; + zfs_handle_t *parent; + char *r; + + dataset_name = zfs_strdup(hdl, zhp->zfs_name); + if (dataset_name == NULL) { + (void) fprintf(stderr, gettext("not enough memory")); + return (NULL); + } + + r = strrchr(dataset_name, '@'); + + if (r == NULL) { + (void) fprintf(stderr, gettext("snapshot '%s' " + "has no '@'\n"), zhp->zfs_name); + free(dataset_name); + return (NULL); + } + + r[0] = 0; + + /* Open the dataset */ + if ((parent = zfs_open(hdl, dataset_name, + ZFS_TYPE_FILESYSTEM)) == NULL) { + (void) fprintf(stderr, + gettext("unable to open parent dataset '%s'\n"), + dataset_name); + free(dataset_name); + return (NULL); + } + + if (!zfs_is_mounted(parent, &parent_mountpoint)) { + (void) fprintf(stderr, + gettext("parent dataset '%s' must be mounted\n"), + dataset_name); + free(dataset_name); + zfs_close(parent); + return (NULL); + } + + zfs_close(parent); + + snapshot_mountpoint = + zfs_asprintf(hdl, "%s/.zfs/snapshot/%s/", + parent_mountpoint, &r[1]); + + free(dataset_name); + free(parent_mountpoint); + + return (snapshot_mountpoint); +} + +/* + * Mount a snapshot; called from "zfs mount dataset@snapshot". + * Given "dataset@snapshot" construct mountpoint path of the + * style "/mountpoint/dataset/.zfs/snapshot/$name/". Ensure + * parent "dataset" is mounted, then issue mount for snapshot. + */ +int +zfs_snapshot_mount(zfs_handle_t *zhp, const char *options, + int flags) +{ + int ret = 0; + char *mountpoint; + + /* + * The automounting will kick in, and zed mounts it - so + * we temporarily disable it + */ + uint64_t automount = 0; + uint64_t saved_automount = 0; + size_t len = sizeof (automount); + size_t slen = sizeof (saved_automount); + + if (zfs_is_mounted(zhp, NULL)) { + return (EBUSY); + } + + mountpoint = zfs_snapshot_mountpoint(zhp); + if (mountpoint == NULL) + return (EINVAL); + + ret = zfs_mount_at(zhp, options, MS_RDONLY | flags, + mountpoint); + + /* If zed is running, it can mount it before us */ + if (ret == -1 && errno == EINVAL) + ret = 0; + + if (ret == 0) { + (void) fprintf(stderr, + gettext("ZFS: snapshot mountpoint '%s'\n"), + mountpoint); + } + + free(mountpoint); + + return (ret); +} + +int +zfs_snapshot_unmount(zfs_handle_t *zhp, int flags) +{ + int ret = 0; + char *mountpoint; + + if (!zfs_is_mounted(zhp, NULL)) { + return (ENOENT); + } + + mountpoint = zfs_snapshot_mountpoint(zhp); + if (mountpoint == NULL) + return (EINVAL); + + ret = zfs_unmount(zhp, mountpoint, flags); + + free(mountpoint); + + return (ret); +} + +/* Called from the tail end of zpool_disable_datasets() */ +void +zpool_disable_datasets_os(zpool_handle_t *zhp, boolean_t force) +{ + (void) zhp, (void) force; +} + +/* Called from the tail end of zfs_unmount() */ +void +zpool_disable_volume_os(const char *name) +{ + (void) name; +} diff --git a/lib/libzfs/os/windows/libzfs_pool_os.c b/lib/libzfs/os/windows/libzfs_pool_os.c new file mode 100644 index 000000000000..9cb3dbdcc9be --- /dev/null +++ b/lib/libzfs/os/windows/libzfs_pool_os.c @@ -0,0 +1,343 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. + * Copyright 2016 Igor Kozhukhov + * Copyright (c) 2018 Datto Inc. + * Copyright (c) 2017 Open-E, Inc. All Rights Reserved. + * Copyright (c) 2017, Intel Corporation. + * Copyright (c) 2018, loli10K + * Copyright (c) 2017, Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "zfs_namecheck.h" +#include "zfs_prop.h" +#include "libzfs_impl.h" +#include "zfs_comutil.h" +#include "zfeature_common.h" + +/* + * If the device has being dynamically expanded then we need to relabel + * the disk to use the new unallocated space. + */ +int +zpool_relabel_disk(libzfs_handle_t *hdl, const char *path, const char *msg) +{ + int fd, error; + + if ((fd = open(path, O_RDWR|O_DIRECT)) < 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "cannot " + "relabel '%s': unable to open device: %d"), path, errno); + return (zfs_error(hdl, EZFS_OPENFAILED, msg)); + } + + /* + * It's possible that we might encounter an error if the device + * does not have any unallocated space left. If so, we simply + * ignore that error and continue on. + * + * Also, we don't call efi_rescan() - that would just return EBUSY. + * The module will do it for us in vdev_disk_open(). + */ + error = efi_use_whole_disk(fd); + + /* Flush the buffers to disk and invalidate the page cache. */ + (void) fsync(fd); +// (void) ioctl(fd, BLKFLSBUF); + + (void) close(fd); + if (error && error != VT_ENOSPC) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "cannot " + "relabel '%s': unable to read disk capacity"), path); + return (zfs_error(hdl, EZFS_NOCAP, msg)); + } + return (0); +} + +/* + * Read the EFI label from the config, if a label does not exist then + * pass back the error to the caller. If the caller has passed a non-NULL + * diskaddr argument then we set it to the starting address of the EFI + * partition. + */ +static int +read_efi_label(nvlist_t *config, diskaddr_t *sb) +{ + char *path; + int fd; + char diskname[MAXPATHLEN]; + int err = -1; + + if (nvlist_lookup_string(config, ZPOOL_CONFIG_PATH, &path) != 0) + return (err); + + (void) snprintf(diskname, sizeof (diskname), "%s%s", DISK_ROOT, + strrchr(path, '/')); + if ((fd = open(diskname, O_RDONLY|O_DIRECT)) >= 0) { + struct dk_gpt *vtoc; + + if ((err = efi_alloc_and_read(fd, &vtoc)) >= 0) { + if (sb != NULL) + *sb = vtoc->efi_parts[0].p_start; + efi_free(vtoc); + } + (void) close(fd); + } + return (err); +} + +/* + * determine where a partition starts on a disk in the current + * configuration + */ +static diskaddr_t +find_start_block(nvlist_t *config) +{ + nvlist_t **child; + uint_t c, children; + diskaddr_t sb = MAXOFFSET_T; + uint64_t wholedisk; + + if (nvlist_lookup_nvlist_array(config, + ZPOOL_CONFIG_CHILDREN, &child, &children) != 0) { + if (nvlist_lookup_uint64(config, + ZPOOL_CONFIG_WHOLE_DISK, + &wholedisk) != 0 || !wholedisk) { + return (MAXOFFSET_T); + } + if (read_efi_label(config, &sb) < 0) + sb = MAXOFFSET_T; + return (sb); + } + + for (c = 0; c < children; c++) { + sb = find_start_block(child[c]); + if (sb != MAXOFFSET_T) { + return (sb); + } + } + return (MAXOFFSET_T); +} + +static int +zpool_label_disk_check(char *path) +{ + struct dk_gpt *vtoc; + int fd, err; + + if ((fd = open(path, O_RDONLY|O_DIRECT)) < 0) + return (errno); + + if ((err = efi_alloc_and_read(fd, &vtoc)) != 0) { + (void) close(fd); + return (err); + } + + if (vtoc->efi_flags & EFI_GPT_PRIMARY_CORRUPT) { + efi_free(vtoc); + (void) close(fd); + return (EIDRM); + } + + efi_free(vtoc); + (void) close(fd); + return (0); +} + +/* + * Generate a unique partition name for the ZFS member. Partitions must + * have unique names to ensure udev will be able to create symlinks under + * /dev/disk/by-partlabel/ for all pool members. The partition names are + * of the form -. + */ +static void +zpool_label_name(char *label_name, int label_size) +{ + uint64_t id = 0; + int fd; + + fd = open("/dev/urandom", O_RDONLY); + if (fd >= 0) { + if (read(fd, &id, sizeof (id)) != sizeof (id)) + id = 0; + + close(fd); + } + + if (id == 0) + id = (((uint64_t)rand()) << 32) | (uint64_t)rand(); + + snprintf(label_name, label_size, "zfs-%016llx", (u_longlong_t)id); +} + +/* + * Label an individual disk. The name provided is the short name, + * stripped of any leading /dev path. + */ +int +zpool_label_disk(libzfs_handle_t *hdl, zpool_handle_t *zhp, const char *name) +{ + char path[MAXPATHLEN]; + struct dk_gpt *vtoc; + int rval, fd; + size_t resv = EFI_MIN_RESV_SIZE; + uint64_t slice_size; + diskaddr_t start_block; + char errbuf[1024]; + + /* prepare an error message just in case */ + (void) snprintf(errbuf, sizeof (errbuf), + dgettext(TEXT_DOMAIN, "cannot label '%s'"), name); + + if (zhp) { + nvlist_t *nvroot = fnvlist_lookup_nvlist(zhp->zpool_config, + ZPOOL_CONFIG_VDEV_TREE); + + if (zhp->zpool_start_block == 0) + start_block = find_start_block(nvroot); + else + start_block = zhp->zpool_start_block; + zhp->zpool_start_block = start_block; + } else { + /* new pool */ + start_block = NEW_START_BLOCK; + } + + (void) snprintf(path, sizeof (path), "%s%s", DISK_ROOT, name); + + if ((fd = open(path, O_RDWR|O_DIRECT|O_EXCL)) < 0) { + /* + * This shouldn't happen. We've long since verified that this + * is a valid device. + */ + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "cannot " + "label '%s': unable to open device: %d"), path, errno); + return (zfs_error(hdl, EZFS_OPENFAILED, errbuf)); + } + + if (efi_alloc_and_init(fd, EFI_NUMPAR, &vtoc) != 0) { + /* + * The only way this can fail is if we run out of memory, or we + * were unable to read the disk's capacity + */ + if (errno == ENOMEM) + (void) no_memory(hdl); + + (void) close(fd); + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "cannot " + "label '%s': unable to read disk capacity"), path); + + return (zfs_error(hdl, EZFS_NOCAP, errbuf)); + } + + slice_size = vtoc->efi_last_u_lba + 1; + slice_size -= EFI_MIN_RESV_SIZE; + if (start_block == MAXOFFSET_T) + start_block = NEW_START_BLOCK; + slice_size -= start_block; + slice_size = P2ALIGN(slice_size, PARTITION_END_ALIGNMENT); + + vtoc->efi_parts[0].p_start = start_block; + vtoc->efi_parts[0].p_size = slice_size; + + /* + * Why we use V_USR: V_BACKUP confuses users, and is considered + * disposable by some EFI utilities (since EFI doesn't have a backup + * slice). V_UNASSIGNED is supposed to be used only for zero size + * partitions, and efi_write() will fail if we use it. V_ROOT, V_BOOT, + * etc. were all pretty specific. V_USR is as close to reality as we + * can get, in the absence of V_OTHER. + */ + vtoc->efi_parts[0].p_tag = V_USR; + zpool_label_name(vtoc->efi_parts[0].p_name, EFI_PART_NAME_LEN); + + vtoc->efi_parts[8].p_start = slice_size + start_block; + vtoc->efi_parts[8].p_size = resv; + vtoc->efi_parts[8].p_tag = V_RESERVED; + + rval = efi_write(fd, vtoc); + + /* Flush the buffers to disk and invalidate the page cache. */ + (void) fsync(fd); +// (void) ioctl(fd, BLKFLSBUF); + + if (rval == 0) + rval = efi_rescan(fd); + + /* + * Some block drivers (like pcata) may not support EFI GPT labels. + * Print out a helpful error message directing the user to manually + * label the disk and give a specific slice. + */ + if (rval != 0) { + (void) close(fd); + efi_free(vtoc); + + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "try using " + "parted(8) and then provide a specific slice: %d"), rval); + return (zfs_error(hdl, EZFS_LABELFAILED, errbuf)); + } + + (void) close(fd); + efi_free(vtoc); + + (void) snprintf(path, sizeof (path), "%s%s", DISK_ROOT, name); + (void) zfs_append_partition(path, MAXPATHLEN); + + /* Wait to udev to signal use the device has settled. */ + rval = zpool_label_disk_wait(path, DISK_LABEL_WAIT); + if (rval) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "failed to " + "detect device partitions on '%s': %d"), path, rval); + return (zfs_error(hdl, EZFS_LABELFAILED, errbuf)); + } + + /* We can't be to paranoid. Read the label back and verify it. */ + (void) snprintf(path, sizeof (path), "%s%s", DISK_ROOT, name); + rval = zpool_label_disk_check(path); + if (rval) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "freshly written " + "EFI label on '%s' is damaged. Ensure\nthis device " + "is not in use, and is functioning properly: %d"), + path, rval); + return (zfs_error(hdl, EZFS_LABELFAILED, errbuf)); + } + return (0); +} diff --git a/lib/libzfs/os/windows/libzfs_util_os.c b/lib/libzfs/os/windows/libzfs_util_os.c new file mode 100644 index 000000000000..06aff439b007 --- /dev/null +++ b/lib/libzfs/os/windows/libzfs_util_os.c @@ -0,0 +1,297 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + * + * Copyright (c) 2017 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + * + */ + + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "libzfs_impl.h" +#include "zfs_prop.h" +#include +#include + +#define ZDIFF_SHARESDIR "/.zfs/shares/" + + +int +zfs_ioctl(libzfs_handle_t *hdl, int request, zfs_cmd_t *zc) +{ + return (lzc_ioctl_fd(hdl->libzfs_fd, request, zc)); +} + +const char * +libzfs_error_init(int error) +{ + switch (error) { + case ENXIO: + return (dgettext(TEXT_DOMAIN, "The ZFS modules are not " + "loaded.\nTry running '/sbin/kextload zfs.kext' as root " + "to load them.")); + case ENOENT: + return (dgettext(TEXT_DOMAIN, "/dev/zfs and /proc/self/mounts " + "are required.\nTry running 'udevadm trigger' and 'mount " + "-t proc proc /proc' as root.")); + case ENOEXEC: + return (dgettext(TEXT_DOMAIN, "The ZFS modules cannot be " + "auto-loaded.\nTry running '/sbin/kextload zfs.kext' as " + "root to manually load them.")); + case EACCES: + return (dgettext(TEXT_DOMAIN, "Permission denied the " + "ZFS utilities must be run as root.")); + default: + return (dgettext(TEXT_DOMAIN, "Failed to initialize the " + "libzfs library.")); + } +} + +static int +libzfs_module_loaded(const char *module) +{ + const char path_prefix[] = "/dev/"; + char path[256]; + + memcpy(path, path_prefix, sizeof (path_prefix) - 1); + strcpy(path + sizeof (path_prefix) - 1, module); + + return (access(path, F_OK) == 0); +} + +/* + * Verify the required ZFS_DEV device is available and optionally attempt + * to load the ZFS modules. Under normal circumstances the modules + * should already have been loaded by some external mechanism. + * + * Environment variables: + * - ZFS_MODULE_LOADING="YES|yes|ON|on" - Attempt to load modules. + * - ZFS_MODULE_TIMEOUT="" - Seconds to wait for ZFS_DEV + */ +static int +libzfs_load_module_impl(const char *module) +{ + return (0); +} + +int +libzfs_load_module(void) +{ + return (libzfs_load_module_impl(ZFS_DRIVER)); +} + +int +find_shares_object(differ_info_t *di) +{ + return (0); +} + +/* + * Fill given version buffer with zfs kernel version read from ZFS_SYSFS_DIR + * Returns 0 on success, and -1 on error (with errno set) + */ +char * +zfs_version_kernel(void) +{ + HKEY hKey; // SYSTEM\ControlSet001\Services\OpenZFS + LSTATUS status = RegOpenKeyExA(HKEY_LOCAL_MACHINE, + "SYSTEM\\ControlSet001\\Services\\OpenZFS", 0, KEY_READ, &hKey); + + if (status != ERROR_SUCCESS) + return (NULL); + + DWORD count = 0; + DWORD type; + + /* This must match module/os/windows/spl/spl-windows.c: zfs_version */ + status = RegQueryValueExA(hKey, "zfs_version", 0, &type, NULL, &count); + + char *version = malloc(count + 1); + if (version == NULL) + return (NULL); + + status = RegQueryValueExA(hKey, "zfs_version", 0, &type, version, + &count); + + RegCloseKey(hKey); + + if (status == ERROR_SUCCESS && + (type == REG_SZ)) { + version[count] = 0; + return (version); + } + return (NULL); +} + +static int +execvPe(const char *name, const char *path, char * const *argv, + char * const *envp) +{ + const char **memp; + size_t cnt, lp, ln; + int eacces, save_errno; + char *cur, buf[MAXPATHLEN]; + const char *p, *bp; + struct stat sb; + + eacces = 0; + + /* If it's an absolute or relative path name, it's easy. */ + if (strchr(name, '/')) { + bp = name; + cur = NULL; + goto retry; + } + bp = buf; + + /* If it's an empty path name, fail in the usual POSIX way. */ + if (*name == '\0') { + errno = ENOENT; + return (-1); + } + + cur = alloca(strlen(path) + 1); + if (cur == NULL) { + errno = ENOMEM; + return (-1); + } + strcpy(cur, path); + while ((p = strsep(&cur, ":")) != NULL) { + /* + * It's a SHELL path -- double, leading and trailing colons + * mean the current directory. + */ + if (*p == '\0') { + p = "."; + lp = 1; + } else + lp = strlen(p); + ln = strlen(name); + + /* + * If the path is too long complain. This is a possible + * security issue; given a way to make the path too long + * the user may execute the wrong program. + */ + if (lp + ln + 2 > sizeof (buf)) { + (void) write(STDERR_FILENO, "execvP: ", 8); + (void) write(STDERR_FILENO, p, lp); + (void) write(STDERR_FILENO, ": path too long\n", + 16); + continue; + } + memcpy(buf, p, lp); + buf[lp] = '/'; + memcpy(buf + lp + 1, name, ln); + buf[lp + ln + 1] = '\0'; + +retry: + (void) execve(bp, argv, envp); + switch (errno) { + case E2BIG: + goto done; + case ELOOP: + case ENAMETOOLONG: + case ENOENT: + break; + case ENOEXEC: + for (cnt = 0; argv[cnt]; ++cnt) + ; + memp = alloca((cnt + 2) * sizeof (char *)); + if (memp == NULL) { + goto done; + } + memp[0] = "sh"; + memp[1] = bp; + memcpy(memp + 2, argv + 1, cnt * sizeof (char *)); +// execve(_PATH_BSHELL, __DECONST(char **, memp), +// envp); + goto done; + case ENOMEM: + goto done; + case ENOTDIR: + break; + case ETXTBSY: + /* + * We used to retry here, but sh(1) doesn't. + */ + goto done; + default: + /* + * EACCES may be for an inaccessible directory or + * a non-executable file. Call stat() to decide + * which. This also handles ambiguities for EFAULT + * and EIO, and undocumented errors like ESTALE. + * We hope that the race for a stat() is unimportant. + */ + save_errno = errno; + if (stat(bp, &sb) != 0) + break; + if (save_errno == EACCES) { + eacces = 1; + continue; + } + errno = save_errno; + goto done; + } + } + if (eacces) + errno = EACCES; + else + errno = ENOENT; +done: + return (-1); +} + +void +zfs_rollback_os(zfs_handle_t *zhp) +{ + (void) zhp; +} + +int +libzfs_run_process_impl(const char *path, char *argv[], char *env[], + int flags, char **lines[], int *lines_cnt) +{ + return (0); +} + +int +zfs_destroy_snaps_nvl_os(libzfs_handle_t *hdl, nvlist_t *snaps) +{ + (void) hdl, (void) snaps; + return (0); +} diff --git a/lib/libzfs_core/CMakeLists.txt b/lib/libzfs_core/CMakeLists.txt new file mode 100644 index 000000000000..0445c96c6738 --- /dev/null +++ b/lib/libzfs_core/CMakeLists.txt @@ -0,0 +1,8 @@ + +use_clang() + +add_library(libzfs_core + libzfs_core.c + os/windows/libzfs_core_ioctl.c +) +target_link_libraries(libzfs_core PUBLIC libnvpair libpthread libzutil libnvpair) diff --git a/lib/libzfs_core/libzfs_core.c b/lib/libzfs_core/libzfs_core.c index c63a16de5ab6..f4cfed0f2387 100644 --- a/lib/libzfs_core/libzfs_core.c +++ b/lib/libzfs_core/libzfs_core.c @@ -26,6 +26,7 @@ * Copyright (c) 2017 Open-E, Inc. All Rights Reserved. * Copyright (c) 2019, 2020 by Christian Schwarz. All rights reserved. * Copyright (c) 2019 Datto Inc. + * Portions Copyright 2022 Andrew Innes */ /* @@ -91,10 +92,21 @@ #include #include #include + #if __FreeBSD__ #define BIG_PIPE_SIZE (64 * 1024) /* From sys/pipe.h */ #endif +#ifdef _WIN32 +/* + * 64k aswell + * https://stackoverflow.com/q/33553837 + * https://docs.microsoft.com/en-au/windows/win32/api/namedpipeapi/ + * nf-namedpipeapi-transactnamedpipe + */ +#define BIG_PIPE_SIZE (64 * 1024) +#endif + static int g_fd = -1; static pthread_mutex_t g_lock = PTHREAD_MUTEX_INITIALIZER; static int g_refcount; diff --git a/lib/libzfs_core/os/windows/libzfs_core_ioctl.c b/lib/libzfs_core/os/windows/libzfs_core_ioctl.c new file mode 100644 index 000000000000..941ab4cd4f6f --- /dev/null +++ b/lib/libzfs_core/os/windows/libzfs_core_ioctl.c @@ -0,0 +1,118 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#include +#include +#include +#include +#include + +static int +zcmd_ioctl_compat(int fd, int request, zfs_cmd_t *zc, const int cflag) +{ + int ret; + void *zc_c; + unsigned long ncmd; + zfs_iocparm_t *zip; + DWORD bytesReturned = 0; + + switch (cflag) { + case ZFS_CMD_COMPAT_NONE: + ncmd = CTL_CODE(ZFSIOCTL_TYPE, ZFSIOCTL_BASE + request, + METHOD_NEITHER, FILE_ANY_ACCESS); + + zip = malloc(sizeof (zfs_iocparm_t)); + zip->zfs_cmd = (uint64_t)zc; + zip->zfs_cmd_size = sizeof (zfs_cmd_t); + zip->zfs_ioctl_version = ZFS_IOCVER_ZOF; + zip->zfs_ioc_error = 0; + bytesReturned = sizeof (zfs_iocparm_t); + + // ret = ioctl(fd, ncmd, &zp); + ret = DeviceIoControl(ITOH(fd), + (DWORD)ncmd, + zip, + (DWORD)sizeof (zfs_iocparm_t), + zc, + (DWORD)sizeof (zfs_cmd_t), + &bytesReturned, + NULL); + + if (ret == 0) + ret = GetLastError(); + else + ret = 0; + + + /* + * If ioctl worked, get actual rc from kernel, which goes + * into errno, and return -1 if not-zero. + */ + if (ret == 0) { + errno = zip->zfs_ioc_error; + if (zip->zfs_ioc_error != 0) + ret = -1; + } + free(zip); + return (ret); + + default: + abort(); + return (EINVAL); + } + + /* Pass-through ioctl, rarely used if at all */ + + ret = ioctl(fd, ncmd, zc_c); + ASSERT0(ret); + + zfs_cmd_compat_get(zc, (caddr_t)zc_c, cflag); + free(zc_c); + + return (ret); +} + +/* + * This is the Windows version of ioctl(). Because the XNU kernel + * handles copyin() and copyout(), we must return success from the + * ioctl() handler (or it will not copyout() for userland), + * and instead embed the error return value in the zc structure. + */ +int +lzc_ioctl_fd(int fd, unsigned long request, zfs_cmd_t *zc) +{ + size_t oldsize; + int ret, cflag = ZFS_CMD_COMPAT_NONE; + + oldsize = zc->zc_nvlist_dst_size; + ret = zcmd_ioctl_compat(fd, request, zc, cflag); + + if (ret == 0 && oldsize < zc->zc_nvlist_dst_size) { + ret = -1; + errno = ENOMEM; + } + + return (ret); +} diff --git a/lib/libzfsbootenv/CMakeLists.txt b/lib/libzfsbootenv/CMakeLists.txt new file mode 100644 index 000000000000..7385cb804d09 --- /dev/null +++ b/lib/libzfsbootenv/CMakeLists.txt @@ -0,0 +1,12 @@ + +use_clang() + +# Unfortunately, libbootenv uses kernel headers as well. +include_directories(PRIVATE "${CMAKE_SOURCE_DIR}/include/os/windows/zfs") + +add_library(libzfsbootenv + lzbe_device.c + lzbe_pair.c + lzbe_util.c +) +target_link_libraries(libzfsbootenv PUBLIC libnvpair libpthread) diff --git a/lib/libzpool/CMakeLists.txt b/lib/libzpool/CMakeLists.txt new file mode 100644 index 000000000000..426d955e150e --- /dev/null +++ b/lib/libzpool/CMakeLists.txt @@ -0,0 +1,191 @@ + +use_clang() + +add_definitions(-DLIB_ZPOOL_BUILD -DDEBUG -UNDEBUG -DZFS_DEBUG) +# Unfortunately, libbootenv uses kernel headers as well. +include_directories(PRIVATE "${CMAKE_SOURCE_DIR}/include/os/windows/zfs") + + +set(MODULE_DIR "../../module") + +add_library(libzpool + "${MODULE_DIR}/lua/lapi.c" + "${MODULE_DIR}/lua/lauxlib.c" + "${MODULE_DIR}/lua/lbaselib.c" + "${MODULE_DIR}/lua/lcode.c" + "${MODULE_DIR}/lua/lcompat.c" + "${MODULE_DIR}/lua/lcorolib.c" + "${MODULE_DIR}/lua/lctype.c" + "${MODULE_DIR}/lua/ldebug.c" + "${MODULE_DIR}/lua/ldo.c" + "${MODULE_DIR}/lua/lfunc.c" + "${MODULE_DIR}/lua/lgc.c" + "${MODULE_DIR}/lua/llex.c" + "${MODULE_DIR}/lua/lmem.c" + "${MODULE_DIR}/lua/lobject.c" + "${MODULE_DIR}/lua/lopcodes.c" + "${MODULE_DIR}/lua/lparser.c" + "${MODULE_DIR}/lua/lstate.c" + "${MODULE_DIR}/lua/lstring.c" + "${MODULE_DIR}/lua/lstrlib.c" + "${MODULE_DIR}/lua/ltable.c" + "${MODULE_DIR}/lua/ltablib.c" + "${MODULE_DIR}/lua/ltm.c" + "${MODULE_DIR}/lua/lvm.c" + "${MODULE_DIR}/lua/lzio.c" + "${MODULE_DIR}/zcommon/cityhash.c" + "${MODULE_DIR}/zcommon/zfeature_common.c" + "${MODULE_DIR}/zcommon/zfs_comutil.c" + "${MODULE_DIR}/zcommon/zfs_deleg.c" + "${MODULE_DIR}/zcommon/zfs_fletcher.c" + "${MODULE_DIR}/zcommon/zfs_fletcher_avx512.c" + "${MODULE_DIR}/zcommon/zfs_fletcher_intel.c" + "${MODULE_DIR}/zcommon/zfs_fletcher_sse.c" + "${MODULE_DIR}/zcommon/zfs_fletcher_superscalar.c" + "${MODULE_DIR}/zcommon/zfs_fletcher_superscalar4.c" + "${MODULE_DIR}/zcommon/zfs_namecheck.c" + "${MODULE_DIR}/zcommon/zfs_prop.c" + "${MODULE_DIR}/zcommon/zpool_prop.c" + "${MODULE_DIR}/zcommon/zprop_common.c" + "${MODULE_DIR}/zfs/abd.c" + "${MODULE_DIR}/zfs/aggsum.c" + "${MODULE_DIR}/zfs/arc.c" + "${MODULE_DIR}/zfs/blake3_zfs.c" + "${MODULE_DIR}/zfs/blkptr.c" + "${MODULE_DIR}/zfs/bplist.c" + "${MODULE_DIR}/zfs/bpobj.c" + "${MODULE_DIR}/zfs/bptree.c" + "${MODULE_DIR}/zfs/btree.c" + "${MODULE_DIR}/zfs/bqueue.c" + "${MODULE_DIR}/zfs/brt.c" + "${MODULE_DIR}/zfs/dbuf.c" + "${MODULE_DIR}/zfs/dbuf_stats.c" + "${MODULE_DIR}/zfs/ddt.c" + "${MODULE_DIR}/zfs/ddt_zap.c" + "${MODULE_DIR}/zfs/dmu.c" + "${MODULE_DIR}/zfs/dmu_diff.c" + "${MODULE_DIR}/zfs/dmu_object.c" + "${MODULE_DIR}/zfs/dmu_objset.c" + "${MODULE_DIR}/zfs/dmu_recv.c" + "${MODULE_DIR}/zfs/dmu_redact.c" + "${MODULE_DIR}/zfs/dmu_send.c" + "${MODULE_DIR}/zfs/dmu_traverse.c" + "${MODULE_DIR}/zfs/dmu_tx.c" + "${MODULE_DIR}/zfs/dmu_zfetch.c" + "${MODULE_DIR}/zfs/dnode.c" + "${MODULE_DIR}/zfs/dnode_sync.c" + "${MODULE_DIR}/zfs/dsl_bookmark.c" + "${MODULE_DIR}/zfs/dsl_crypt.c" + "${MODULE_DIR}/zfs/dsl_dataset.c" + "${MODULE_DIR}/zfs/dsl_deadlist.c" + "${MODULE_DIR}/zfs/dsl_deleg.c" + "${MODULE_DIR}/zfs/dsl_destroy.c" + "${MODULE_DIR}/zfs/dsl_dir.c" + "${MODULE_DIR}/zfs/dsl_pool.c" + "${MODULE_DIR}/zfs/dsl_prop.c" + "${MODULE_DIR}/zfs/dsl_scan.c" + "${MODULE_DIR}/zfs/dsl_synctask.c" + "${MODULE_DIR}/zfs/dsl_userhold.c" + "${MODULE_DIR}/zfs/edonr_zfs.c" + "${MODULE_DIR}/zfs/fm.c" + "${MODULE_DIR}/zfs/gzip.c" + "${MODULE_DIR}/zfs/hkdf.c" + "${MODULE_DIR}/zfs/lz4.c" + "${MODULE_DIR}/zfs/lz4_zfs.c" + "${MODULE_DIR}/zfs/lzjb.c" + "${MODULE_DIR}/zfs/metaslab.c" + "${MODULE_DIR}/zfs/mmp.c" + "${MODULE_DIR}/zfs/multilist.c" + "${MODULE_DIR}/zfs/objlist.c" + "${MODULE_DIR}/zfs/pathname.c" + "${MODULE_DIR}/zfs/range_tree.c" + "${MODULE_DIR}/zfs/refcount.c" + "${MODULE_DIR}/zfs/rrwlock.c" + "${MODULE_DIR}/zfs/sa.c" + "${MODULE_DIR}/zfs/sha2_zfs.c" + "${MODULE_DIR}/zfs/skein_zfs.c" + "${MODULE_DIR}/zfs/spa.c" + "${MODULE_DIR}/zfs/space_map.c" + "${MODULE_DIR}/zfs/space_reftree.c" + "${MODULE_DIR}/zfs/spa_checkpoint.c" + "${MODULE_DIR}/zfs/spa_config.c" + "${MODULE_DIR}/zfs/spa_errlog.c" + "${MODULE_DIR}/zfs/spa_history.c" + "${MODULE_DIR}/zfs/spa_log_spacemap.c" + "${MODULE_DIR}/zfs/spa_misc.c" + "${MODULE_DIR}/zfs/spa_stats.c" + "${MODULE_DIR}/zfs/txg.c" + "${MODULE_DIR}/zfs/uberblock.c" + "${MODULE_DIR}/zfs/unique.c" + "${MODULE_DIR}/zfs/vdev.c" + "${MODULE_DIR}/zfs/vdev_draid.c" + "${MODULE_DIR}/zfs/vdev_draid_rand.c" + "${MODULE_DIR}/zfs/vdev_indirect.c" + "${MODULE_DIR}/zfs/vdev_indirect_births.c" + "${MODULE_DIR}/zfs/vdev_indirect_mapping.c" + "${MODULE_DIR}/zfs/vdev_initialize.c" + "${MODULE_DIR}/zfs/vdev_label.c" + "${MODULE_DIR}/zfs/vdev_mirror.c" + "${MODULE_DIR}/zfs/vdev_missing.c" + "${MODULE_DIR}/zfs/vdev_queue.c" + "${MODULE_DIR}/zfs/vdev_raidz.c" + "${MODULE_DIR}/zfs/vdev_raidz_math.c" + "${MODULE_DIR}/zfs/vdev_raidz_math_avx2.c" + "${MODULE_DIR}/zfs/vdev_raidz_math_avx512bw.c" + "${MODULE_DIR}/zfs/vdev_raidz_math_avx512f.c" + "${MODULE_DIR}/zfs/vdev_raidz_math_scalar.c" + "${MODULE_DIR}/zfs/vdev_raidz_math_sse2.c" + "${MODULE_DIR}/zfs/vdev_raidz_math_ssse3.c" + "${MODULE_DIR}/zfs/vdev_rebuild.c" + "${MODULE_DIR}/zfs/vdev_removal.c" + "${MODULE_DIR}/zfs/vdev_root.c" + "${MODULE_DIR}/zfs/vdev_trim.c" + "${MODULE_DIR}/zfs/zap.c" + "${MODULE_DIR}/zfs/zap_leaf.c" + "${MODULE_DIR}/zfs/zap_micro.c" + "${MODULE_DIR}/zfs/zcp.c" + "${MODULE_DIR}/zfs/zcp_get.c" + "${MODULE_DIR}/zfs/zcp_set.c" + "${MODULE_DIR}/zfs/zcp_global.c" + "${MODULE_DIR}/zfs/zcp_iter.c" + "${MODULE_DIR}/zfs/zcp_synctask.c" + "${MODULE_DIR}/zfs/zfeature.c" + "${MODULE_DIR}/zfs/zfs_byteswap.c" + "${MODULE_DIR}/zfs/zfs_chksum.c" + "${MODULE_DIR}/zfs/zfs_fm.c" + "${MODULE_DIR}/zfs/zfs_fuid.c" + "${MODULE_DIR}/zfs/zfs_ratelimit.c" + "${MODULE_DIR}/zfs/zfs_rlock.c" + "${MODULE_DIR}/zfs/zfs_sa.c" + "${MODULE_DIR}/zfs/zil.c" + "${MODULE_DIR}/zfs/zio.c" + "${MODULE_DIR}/zfs/zio_checksum.c" + "${MODULE_DIR}/zfs/zio_compress.c" + "${MODULE_DIR}/zfs/zio_inject.c" + "${MODULE_DIR}/zfs/zle.c" + "${MODULE_DIR}/zfs/zrlock.c" + "${MODULE_DIR}/zfs/zthr.c" + "${MODULE_DIR}/os/windows/zfs/vdev_file.c" + "${MODULE_DIR}/os/windows/zfs/zio_crypt.c" + "${MODULE_DIR}/os/windows/zfs/zfs_debug.c" + "${MODULE_DIR}/os/windows/zfs/zfs_racct.c" + "${MODULE_DIR}/os/windows/zfs/zfs_znode.c" + "${MODULE_DIR}/os/linux/zfs/abd_os.c" # Seems undesirable - one day fix. + "${MODULE_DIR}/os/linux/zfs/arc_os.c" + "kernel.c" + "taskq.c" + "util.c" +) + +target_link_libraries(libzpool PUBLIC + libicp + libnvpair + libunicode + libuutil + libpthread + libavl + libefi + libshare + zlib + libzstd +) diff --git a/lib/libzpool/kernel.c b/lib/libzpool/kernel.c index a9b9bf4c2ce5..468c90a2b86a 100644 --- a/lib/libzpool/kernel.c +++ b/lib/libzpool/kernel.c @@ -48,6 +48,9 @@ #include #include #include +#ifdef _WIN32 +#include +#endif /* * Emulation of kernel services in userland. @@ -131,8 +134,10 @@ zk_thread_create(void (*func)(void *), void *arg, size_t stksize, int state) * If this ever fails, it may be because the stack size is not a * multiple of system page size. */ +#ifndef _WIN32 VERIFY0(pthread_attr_setstacksize(&attr, stksize)); VERIFY0(pthread_attr_setguardsize(&attr, PAGESIZE)); +#endif VERIFY(ztw = malloc(sizeof (*ztw))); ztw->func = func; @@ -685,7 +690,11 @@ cmn_err(int ce, const char *fmt, ...) void delay(clock_t ticks) { +#if HAVE_USLEEP + usleep(ticks * 1000 / hz); +#else (void) poll(0, 0, ticks * (1000 / hz)); +#endif } /* @@ -720,6 +729,40 @@ const char *random_path = "/dev/random"; const char *urandom_path = "/dev/urandom"; static int random_fd = -1, urandom_fd = -1; + +#ifdef _WIN32 + +void +random_init(void) +{ +} + +void +random_fini(void) +{ +} + +static int +random_get_bytes_common(uint8_t *ptr, size_t len, int fd) +{ + size_t resid = len; + ssize_t bytes; + unsigned int number; + + while (resid != 0) { + rand_s(&number); + bytes = MIN(resid, sizeof (number)); + memcpy(ptr, &number, bytes); + ASSERT3S(bytes, >=, 0); + ptr += bytes; + resid -= bytes; + } + + return (0); +} + +#else /* Windows */ + void random_init(void) { @@ -754,6 +797,7 @@ random_get_bytes_common(uint8_t *ptr, size_t len, int fd) return (0); } +#endif int random_get_bytes(uint8_t *ptr, size_t len) diff --git a/lib/libzpool/util.c b/lib/libzpool/util.c index b9503d4e56d0..9d04f5542a7a 100644 --- a/lib/libzpool/util.c +++ b/lib/libzpool/util.c @@ -193,10 +193,11 @@ set_global_var_parse_kv(const char *arg, char **k_out, u_longlong_t *v_out) int set_global_var(char const *arg) { + int ret = 0; +#ifndef _WIN32 // Windowsify me void *zpoolhdl; char *varname; u_longlong_t val; - int ret; #ifndef _ZFS_LITTLE_ENDIAN /* @@ -240,6 +241,7 @@ set_global_var(char const *arg) out_free: free(varname); out_ret: +#endif return (ret); } diff --git a/lib/libzstd/CMakeLists.txt b/lib/libzstd/CMakeLists.txt new file mode 100644 index 000000000000..c2388c060804 --- /dev/null +++ b/lib/libzstd/CMakeLists.txt @@ -0,0 +1,31 @@ + +use_clang() + +set(ZSTD_MODULE_DIR "../../module/zstd") + +add_library(libzstd + "${ZSTD_MODULE_DIR}/lib/common/entropy_common.c" + "${ZSTD_MODULE_DIR}/lib/common/error_private.c" + "${ZSTD_MODULE_DIR}/lib/common/fse_decompress.c" + "${ZSTD_MODULE_DIR}/lib/common/pool.c" + "${ZSTD_MODULE_DIR}/lib/common/zstd_common.c" + "${ZSTD_MODULE_DIR}/lib/common/xxhash.c" + "${ZSTD_MODULE_DIR}/lib/compress/fse_compress.c" + "${ZSTD_MODULE_DIR}/lib/compress/hist.c" + "${ZSTD_MODULE_DIR}/lib/compress/huf_compress.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_compress_literals.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_compress_sequences.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_compress_superblock.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_compress.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_double_fast.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_fast.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_lazy.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_ldm.c" + "${ZSTD_MODULE_DIR}/lib/compress/zstd_opt.c" + "${ZSTD_MODULE_DIR}/lib/decompress/huf_decompress.c" + "${ZSTD_MODULE_DIR}/lib/decompress/zstd_ddict.c" + "${ZSTD_MODULE_DIR}/lib/decompress/zstd_decompress.c" + "${ZSTD_MODULE_DIR}/lib/decompress/zstd_decompress_block.c" + "${ZSTD_MODULE_DIR}/zfs_zstd.c" +) +target_link_libraries(libzstd PUBLIC libnvpair libpthread) diff --git a/lib/libzutil/CMakeLists.txt b/lib/libzutil/CMakeLists.txt new file mode 100644 index 000000000000..6aabab5cc1bc --- /dev/null +++ b/lib/libzutil/CMakeLists.txt @@ -0,0 +1,27 @@ + +use_clang() + +target_include_directories(libspl BEFORE PUBLIC "") + +add_library(libzutil + zutil_device_path.c + zutil_import.c + zutil_nicenum.c + zutil_pool.c + os/windows/zutil_compat.c + os/windows/zutil_device_path_os.c + os/windows/zutil_import_os.c +) + +target_link_libraries(libzutil PUBLIC + libicp + libnvpair + libunicode + libuutil + libpthread + libavl + libefi + libshare + libtpool + zlib +) diff --git a/lib/libzutil/os/windows/zutil_compat.c b/lib/libzutil/os/windows/zutil_compat.c new file mode 100644 index 000000000000..a93b7e9a0439 --- /dev/null +++ b/lib/libzutil/os/windows/zutil_compat.c @@ -0,0 +1,27 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include +#include +#include +#include +#include +#include diff --git a/lib/libzutil/os/windows/zutil_device_path_os.c b/lib/libzutil/os/windows/zutil_device_path_os.c new file mode 100644 index 000000000000..55656f1b7cbc --- /dev/null +++ b/lib/libzutil/os/windows/zutil_device_path_os.c @@ -0,0 +1,223 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + * + * Portions Copyright 2022 Andrew Innes + * + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include + +#include + +/* + * We don't strip/append partitions on FreeBSD. + */ + +/* + * Note: The caller must free the returned string. + */ +char * +zfs_strip_partition(const char *dev) +{ + unsigned int disk, slice; + char *partless; + + partless = strdup(dev); + + /* Ends with "diskNsP" - where 'N' and 'P' are integers - strip sP */ + if (sscanf(partless, "disk%us%u", &disk, &slice) == 2) { + char *r; + r = strrchr(partless, 's'); + if (r != NULL) + *r = 0; + } + + return (partless); +} + +/* + * When given PHYSICALDRIVE1 and we partition it for ZFS + * change the name to the partition, ie + * #offset#size#PHYSICALDRIVE1 + */ +int +zfs_append_partition(char *path, size_t max_len) +{ + int len = strlen(path); + int fd; + struct dk_gpt *vtoc; + + // Already expanded, nothing to do + if (path[0] == '#') + return (len); + + fd = open(path, O_RDONLY); + if (fd < 0) + return (len); + + if ((efi_alloc_and_read(fd, &vtoc)) == 0) { + for (int i = 0; i < vtoc->efi_nparts; i++) { + + if (vtoc->efi_parts[i].p_start == 0 && + vtoc->efi_parts[i].p_size == 0) + continue; + + if (tolower(vtoc->efi_parts[i].p_name[0]) == 'z' && + tolower(vtoc->efi_parts[i].p_name[1]) == 'f' && + tolower(vtoc->efi_parts[i].p_name[2]) == 's') { + size_t length = + vtoc->efi_parts[i].p_size * + vtoc->efi_lbasize; + off_t offset = + vtoc->efi_parts[i].p_start * + vtoc->efi_lbasize; + char *copypath = strdup(path); + snprintf(path, max_len, "#%llu#%llu#%s", + offset, length, copypath); + free(copypath); + len = strlen(path); + break; + } + + } + } else { + // If we can't read the partition, we are most likely + // creating a pool, and it will be slice 1, alas, we + // do not know the size/offset here, yet. + // which is why we call this function again after + // zpool_label_disk. + } + close(fd); + return (len); +} + +/* + * Strip the path from a device name. + * On FreeBSD we only want to remove "/dev/" from the beginning of + * paths if present. + */ +const char * +zfs_strip_path(const char *path) +{ + char *r; + r = strrchr(path, '/'); + if (r != NULL) + return (&r[1]); + r = strrchr(path, '\\'); + if (r != NULL) + return (&r[1]); + return (path); +} + +char * +zfs_get_underlying_path(const char *dev_name) +{ + + if (dev_name == NULL) + return (NULL); + + return (realpath(dev_name, NULL)); +} + +boolean_t +zfs_dev_is_whole_disk(const char *dev_name) +{ + struct dk_gpt *label = NULL; + int fd; + + if ((fd = open(dev_name, O_RDONLY | O_DIRECT)) < 0) + return (B_FALSE); + + if (efi_alloc_and_init(fd, EFI_NUMPAR, &label) != 0) { + (void) close(fd); + return (B_FALSE); + } + + efi_free(label); + (void) close(fd); + + return (B_TRUE); +} + +/* + * Wait up to timeout_ms for udev to set up the device node. The device is + * considered ready when libudev determines it has been initialized, all of + * the device links have been verified to exist, and it has been allowed to + * settle. At this point the device the device can be accessed reliably. + * Depending on the complexity of the udev rules this process could take + * several seconds. + */ +int +zpool_label_disk_wait(const char *path, int timeout_ms) +{ + int settle_ms = 50; + long sleep_ms = 10; + hrtime_t start, settle; + struct stat statbuf; + + return (0); + + start = gethrtime(); + settle = 0; + + do { + errno = 0; + if ((stat(path, &statbuf) == 0) && (errno == 0)) { + if (settle == 0) + settle = gethrtime(); + else if (NSEC2MSEC(gethrtime() - settle) >= settle_ms) + return (0); + } else if (errno != ENOENT) { + return (errno); + } + + usleep(sleep_ms * MILLISEC); + } while (NSEC2MSEC(gethrtime() - start) < timeout_ms); + + return (ENODEV); +} + + +boolean_t +is_mpath_whole_disk(const char *path) +{ + // Return TRUE here to have make_disks() call + // update_vdev_config_dev_strs() + return (B_TRUE); +} + +/* + * Return B_TRUE if device is a device mapper or multipath device. + * Return B_FALSE if not. + */ +boolean_t +zfs_dev_is_dm(const char *dev_name) +{ + return (B_FALSE); +} diff --git a/lib/libzutil/os/windows/zutil_import_os.c b/lib/libzutil/os/windows/zutil_import_os.c new file mode 100644 index 000000000000..679f509e8d53 --- /dev/null +++ b/lib/libzutil/os/windows/zutil_import_os.c @@ -0,0 +1,1170 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. + * Copyright 2015 RackTop Systems. + * Copyright (c) 2016, Intel Corporation. + * Portions Copyright 2022 Andrew Innes + */ + +/* + * Pool import support functions. + * + * Used by zpool, ztest, zdb, and zhack to locate importable configs. Since + * these commands are expected to run in the global zone, we can assume + * that the devices are all readable when called. + * + * To import a pool, we rely on reading the configuration information from the + * ZFS label of each device. If we successfully read the label, then we + * organize the configuration information in the following hierarchy: + * + * pool guid -> toplevel vdev guid -> label txg + * + * Duplicate entries matching this same tuple will be discarded. Once we have + * examined every device, we pick the best label txg config for each toplevel + * vdev. We then arrange these toplevel vdevs into a complete pool config, and + * update any paths that have changed. Finally, we attempt to import the pool + * using our derived config, and record the results. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include + +#include "zutil_import.h" + +#ifdef HAVE_LIBUDEV +#include +#include +#endif + +#define _WIN32_MEAN_AND_LEAN +#include +#include +#include +#pragma comment(lib, "setupapi.lib") + +/* + * We allow /dev/ to be search in DEBUG build + * DEFAULT_IMPORT_PATH_SIZE is decremented by one to remove /dev! + * See below in zpool_find_import_blkid() to skip. + */ +#define DEFAULT_IMPORT_PATH_SIZE 4 + +#define DEV_BYID_PATH "/private/var/run/disk/by-id" + +static char * +zpool_default_import_path[DEFAULT_IMPORT_PATH_SIZE] = { + "/private/var/run/disk/by-id", + "/private/var/run/disk/by-path", + "/private/var/run/disk/by-serial", + "/dev" /* Only with DEBUG build */ +}; + +extern uint64_t GetFileDriveSize(HANDLE); + +static boolean_t +is_watchdog_dev(char *dev) +{ + /* For 'watchdog' dev */ + if (strcmp(dev, "watchdog") == 0) + return (B_TRUE); + + /* For 'watchdog */ + if (strstr(dev, "watchdog") == dev && isdigit(dev[8])) + return (B_TRUE); + + return (B_FALSE); +} + +int +zfs_dev_flush(int fd) +{ +// return (ioctl(fd, BLKFLSBUF)); + return (0); +} + +/* + * Return the offset of the given label. + */ +static uint64_t +label_offset(uint64_t size, int l) +{ + ASSERT(P2PHASE_TYPED(size, sizeof (vdev_label_t), uint64_t) == 0); + return (l * sizeof (vdev_label_t) + (l < VDEV_LABELS / 2 ? + 0 : size - VDEV_LABELS * sizeof (vdev_label_t))); +} + +static int +zpool_read_label_win(HANDLE h, off_t offset, uint64_t len, + nvlist_t **config, int *num_labels) +{ + int l, count = 0; + vdev_label_t *label; + nvlist_t *expected_config = NULL; + uint64_t expected_guid = 0, size; + LARGE_INTEGER large; + uint64_t drivesize; + + *config = NULL; + + drivesize = len; + size = P2ALIGN_TYPED(drivesize, sizeof (vdev_label_t), uint64_t); + + if ((label = malloc(sizeof (vdev_label_t))) == NULL) + return (-1); + + for (l = 0; l < VDEV_LABELS; l++) { + uint64_t state, guid, txg; + + if (pread_win(h, label, sizeof (vdev_label_t), + label_offset(size, l) + offset) != sizeof (vdev_label_t)) + continue; + + if (nvlist_unpack(label->vl_vdev_phys.vp_nvlist, + sizeof (label->vl_vdev_phys.vp_nvlist), config, 0) != 0) + continue; + + if (nvlist_lookup_uint64(*config, ZPOOL_CONFIG_GUID, + &guid) != 0 || guid == 0) { + nvlist_free(*config); + continue; + } + + if (nvlist_lookup_uint64(*config, ZPOOL_CONFIG_POOL_STATE, + &state) != 0 || state > POOL_STATE_L2CACHE) { + nvlist_free(*config); + continue; + } + + if (state != POOL_STATE_SPARE && state != POOL_STATE_L2CACHE && + (nvlist_lookup_uint64(*config, ZPOOL_CONFIG_POOL_TXG, + &txg) != 0 || txg == 0)) { + nvlist_free(*config); + continue; + } + + if (expected_guid) { + if (expected_guid == guid) + count++; + + nvlist_free(*config); + } else { + expected_config = *config; + expected_guid = guid; + count++; + } + } + + if (num_labels != NULL) + *num_labels = count; + + free(label); + *config = expected_config; + + return (0); +} + +/* + * Somethings do not like mixing slashes with backslashes. So + * let's try using slashes with user facing output, zpool status etc. + * but internally we use backslashes in vdev_physpath + */ +static void +zfs_backslashes(char *s) +{ + char *r; + while ((r = strchr(s, '/')) != NULL) + *r = '\\'; +} + +static void +zfs_slashes(char *s) +{ + char *r; + while ((r = strchr(s, '\\')) != NULL) + *r = '/'; +} + +void +zpool_open_func(void *arg) +{ + rdsk_node_t *rn = arg; + libpc_handle_t *hdl = rn->rn_hdl; + struct stat64 statbuf; + nvlist_t *config; + char *bname, *dupname; + uint64_t vdev_guid = 0; + int error; + int num_labels = 0; + HANDLE h; + uint64_t offset = 0; + uint64_t len = 0; + uint64_t drive_len; + + // Check if this filename is encoded with "#start#len#name" + if (rn->rn_name[0] == '#') { + char *end = NULL; + + offset = strtoull(&rn->rn_name[1], &end, 10); + while (end && *end == '#') end++; + len = strtoull(end, &end, 10); + while (end && *end == '#') end++; + h = CreateFile(end, + GENERIC_READ, + FILE_SHARE_READ /* | FILE_SHARE_WRITE */, + NULL, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL /* | FILE_FLAG_OVERLAPPED */, + NULL); + if (h == INVALID_HANDLE_VALUE) { + int error = GetLastError(); + return; + } + LARGE_INTEGER place; + place.QuadPart = offset; + // If it fails, we cant read label + SetFilePointerEx(h, place, NULL, FILE_BEGIN); + drive_len = len; + + } else { + // We have no openat() - so stitch paths togther. + // char fullpath[MAX_PATH]; + // snprintf(fullpath, sizeof (fullpath), "%s%s", + // "", rn->rn_name); + zfs_backslashes(rn->rn_name); + h = CreateFile(rn->rn_name, + GENERIC_READ, + FILE_SHARE_READ /* | FILE_SHARE_WRITE */, + NULL, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL /* | FILE_FLAG_OVERLAPPED */, + NULL); + if (h == INVALID_HANDLE_VALUE) { + int error = GetLastError(); + return; + } + + drive_len = GetFileDriveSize(h); + } + + DWORD type = GetFileType(h); + + /* this file is too small to hold a zpool */ + if (type == FILE_TYPE_DISK && + drive_len < SPA_MINDEVSIZE) { + CloseHandle(h); + return; + } +// else if (type != FILE_TYPE_DISK) { + /* + * Try to read the disk label first so we don't have to + * open a bunch of minor nodes that can't have a zpool. + */ +// check_slices(rn->rn_avl, HTOI(h), rn->rn_name); +// } + + if ((zpool_read_label_win(h, offset, drive_len, &config, + &num_labels)) != 0) { + CloseHandle(h); + (void) no_memory(rn->rn_hdl); + return; + } + + if (num_labels == 0) { + CloseHandle(h); + nvlist_free(config); + return; + } + + + /* + * Check that the vdev is for the expected guid. Additional entries + * are speculatively added based on the paths stored in the labels. + * Entries with valid paths but incorrect guids must be removed. + */ + error = nvlist_lookup_uint64(config, ZPOOL_CONFIG_GUID, &vdev_guid); + if (error || (rn->rn_vdev_guid && rn->rn_vdev_guid != vdev_guid)) { + CloseHandle(h); + nvlist_free(config); + return; + } + + CloseHandle(h); + + rn->rn_config = config; + rn->rn_num_labels = num_labels; + + /* + * Add additional entries for paths described by this label. + */ + if (rn->rn_labelpaths) { + char *path = NULL; + char *devid = NULL; + char *env = NULL; + rdsk_node_t *slice; + avl_index_t where; + int timeout; + int error; + + if (label_paths(rn->rn_hdl, rn->rn_config, &path, &devid)) + return; + + env = getenv("ZPOOL_IMPORT_UDEV_TIMEOUT_MS"); + if ((env == NULL) || sscanf(env, "%d", &timeout) != 1 || + timeout < 0) { + timeout = DISK_LABEL_WAIT; + } + + /* + * Allow devlinks to stabilize so all paths are available. + */ + zpool_label_disk_wait(rn->rn_name, timeout); + + if (path != NULL) { + slice = zutil_alloc(hdl, sizeof (rdsk_node_t)); + // slice->rn_name = zutil_strdup(hdl, path); + slice->rn_name = zutil_strdup(hdl, rn->rn_name); + slice->rn_vdev_guid = vdev_guid; + slice->rn_avl = rn->rn_avl; + slice->rn_hdl = hdl; + slice->rn_order = IMPORT_ORDER_PREFERRED_1; + slice->rn_labelpaths = B_FALSE; + pthread_mutex_lock(rn->rn_lock); + if (avl_find(rn->rn_avl, slice, &where)) { + pthread_mutex_unlock(rn->rn_lock); + free(slice->rn_name); + free(slice); + } else { + avl_insert(rn->rn_avl, slice, where); + pthread_mutex_unlock(rn->rn_lock); + zpool_open_func(slice); + } + } + + if (devid != NULL) { + slice = zutil_alloc(hdl, sizeof (rdsk_node_t)); + error = asprintf(&slice->rn_name, "%s%s", + DEV_BYID_PATH, devid); + if (error == -1) { + free(slice); + return; + } + + slice->rn_vdev_guid = vdev_guid; + slice->rn_avl = rn->rn_avl; + slice->rn_hdl = hdl; + slice->rn_order = IMPORT_ORDER_PREFERRED_2; + slice->rn_labelpaths = B_FALSE; + pthread_mutex_lock(rn->rn_lock); + if (avl_find(rn->rn_avl, slice, &where)) { + pthread_mutex_unlock(rn->rn_lock); + free(slice->rn_name); + free(slice); + } else { + avl_insert(rn->rn_avl, slice, where); + pthread_mutex_unlock(rn->rn_lock); + zpool_open_func(slice); + } + } + } + +} + +const char * const * +zpool_default_search_paths(size_t *count) +{ + *count = DEFAULT_IMPORT_PATH_SIZE; + return ((const char * const *)zpool_default_import_path); +} + +/* + * Call Windows API to get list of physical disks, and iterate through them + * finding partitions. + */ +int +zpool_find_import_blkid(libpc_handle_t *hdl, pthread_mutex_t *lock, + avl_tree_t **slice_cache) +{ + int i, dirs; + struct dirent *dp; + char path[MAXPATHLEN]; + char *end, **dir; + size_t pathleft; + avl_index_t where; + rdsk_node_t *slice; + int error = 0; + void *cookie; + char rdsk[MAXPATHLEN]; + + HDEVINFO diskClassDevices; + GUID diskClassDeviceInterfaceGuid = GUID_DEVINTERFACE_DISK; + SP_DEVICE_INTERFACE_DATA deviceInterfaceData; + PSP_DEVICE_INTERFACE_DETAIL_DATA deviceInterfaceDetailData; + DWORD requiredSize; + DWORD deviceIndex; + + HANDLE disk = INVALID_HANDLE_VALUE; + STORAGE_DEVICE_NUMBER diskNumber; + DWORD bytesReturned; + + /* + * Go through and read the label configuration information from every + * possible device, organizing the information according to pool GUID + * and toplevel GUID. + */ + *slice_cache = zutil_alloc(hdl, sizeof (avl_tree_t)); + avl_create(*slice_cache, slice_cache_compare, + sizeof (rdsk_node_t), offsetof(rdsk_node_t, rn_node)); + + + + /* First, open all raw physical devices */ + + diskClassDevices = SetupDiGetClassDevs(&diskClassDeviceInterfaceGuid, + NULL, + NULL, + DIGCF_PRESENT | + DIGCF_DEVICEINTERFACE); + // CHK(INVALID_HANDLE_VALUE != diskClassDevices, + // "SetupDiGetClassDevs"); + + ZeroMemory(&deviceInterfaceData, sizeof (SP_DEVICE_INTERFACE_DATA)); + deviceInterfaceData.cbSize = sizeof (SP_DEVICE_INTERFACE_DATA); + deviceIndex = 0; + + while (SetupDiEnumDeviceInterfaces(diskClassDevices, + NULL, + &diskClassDeviceInterfaceGuid, + deviceIndex, + &deviceInterfaceData)) { + + ++deviceIndex; + + SetupDiGetDeviceInterfaceDetail(diskClassDevices, + &deviceInterfaceData, + NULL, + 0, + &requiredSize, + NULL); + // CHK(ERROR_INSUFFICIENT_BUFFER == GetLastError(), + // "SetupDiGetDeviceInterfaceDetail - 1"); + + deviceInterfaceDetailData = + (PSP_DEVICE_INTERFACE_DETAIL_DATA)malloc(requiredSize); + // CHK(NULL != deviceInterfaceDetailData, + // "malloc"); + + ZeroMemory(deviceInterfaceDetailData, requiredSize); + deviceInterfaceDetailData->cbSize = + sizeof (SP_DEVICE_INTERFACE_DETAIL_DATA); + + SetupDiGetDeviceInterfaceDetail(diskClassDevices, + &deviceInterfaceData, + deviceInterfaceDetailData, + requiredSize, + NULL, + NULL); + + // Here, the device path is something like + // " \\?\ide#diskvmware_virtual_ide_hard_drive___________" + // "00000001#5&1778b74b&0&0.0.0#{53f56307-b6bf-11d0-" + // "94f2-00a0c91efb8b}" + // and we create a path like + // "\\?\PhysicalDrive0" + // but perhaps it is better to use the full name of the device. + disk = CreateFile(deviceInterfaceDetailData->DevicePath, + 0 /* GENERIC_READ */, + FILE_SHARE_READ /* | FILE_SHARE_WRITE */, + NULL, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL /* | FILE_FLAG_OVERLAPPED */, + NULL); + if (disk == INVALID_HANDLE_VALUE) + continue; + + DeviceIoControl(disk, + IOCTL_STORAGE_GET_DEVICE_NUMBER, + NULL, + 0, + &diskNumber, + sizeof (STORAGE_DEVICE_NUMBER), + &bytesReturned, + NULL); + + fprintf(stderr, "path '%s'\n and '\\\\?\\PhysicalDrive%d'\n", + deviceInterfaceDetailData->DevicePath, + diskNumber.DeviceNumber); + fflush(stderr); + snprintf(rdsk, MAXPATHLEN, "\\\\.\\PHYSICALDRIVE%d", + diskNumber.DeviceNumber); + + // CloseHandle(disk); + +#if 0 + // This debug code was here to skip the boot disk, + // but it assumes the first disk is boot, which is wrong. + if (diskNumber.DeviceNumber == 0) { + CloseHandle(disk); + continue; + } +#endif + DWORD ior; + PDRIVE_LAYOUT_INFORMATION_EX partitions; + DWORD partitionsSize = sizeof (DRIVE_LAYOUT_INFORMATION_EX) + + 127 * sizeof (PARTITION_INFORMATION_EX); + partitions = + (PDRIVE_LAYOUT_INFORMATION_EX)malloc(partitionsSize); + if (DeviceIoControl(disk, IOCTL_DISK_GET_DRIVE_LAYOUT_EX, + NULL, 0, partitions, partitionsSize, &ior, NULL)) { + fprintf(stderr, "read partitions ok %d\n", + partitions->PartitionCount); + fflush(stderr); + + for (int i = 0; i < partitions->PartitionCount; i++) { + int add = 0; + switch (partitions->PartitionEntry[i].PartitionStyle) { + case PARTITION_STYLE_MBR: + fprintf(stderr, + " mbr %d: type %x off 0x%llx len 0x%llx\n", i, + partitions->PartitionEntry[i].Mbr.PartitionType, + partitions->PartitionEntry[i]. + StartingOffset.QuadPart, + partitions->PartitionEntry[i]. + PartitionLength.QuadPart); + fflush(stderr); + add = 1; + break; + case PARTITION_STYLE_GPT: + fprintf(stderr, + " gpt %d: type %llx off 0x%llx len 0x%llx\n", i, + partitions->PartitionEntry[i].Gpt.PartitionType, + partitions->PartitionEntry[i]. + StartingOffset.QuadPart, + partitions->PartitionEntry[i].PartitionLength. + QuadPart); + fflush(stderr); + add = 1; + break; + } + + if (add && + partitions->PartitionEntry[i].PartitionLength. + QuadPart > SPA_MINDEVSIZE) { + slice = zutil_alloc(hdl, sizeof (rdsk_node_t)); + + error = asprintf(&slice->rn_name, + "\\\\?\\Harddisk%uPartition%u", + diskNumber.DeviceNumber, i); + if (error == -1) { + free(slice); + continue; + } + + slice->rn_vdev_guid = 0; + slice->rn_lock = lock; + slice->rn_avl = *slice_cache; + slice->rn_hdl = hdl; + slice->rn_labelpaths = B_TRUE; + slice->rn_order = IMPORT_ORDER_PREFERRED_2; + + pthread_mutex_lock(lock); + if (avl_find(*slice_cache, slice, &where)) { + free(slice->rn_name); + free(slice); + } else { + avl_insert(*slice_cache, slice, where); + } + pthread_mutex_unlock(lock); + } + } + // in case we have a disk without partition, + // it would be possible that the + // disk itself contains a pool, so let's check that + if (partitions->PartitionCount == 0) { + + slice = zutil_alloc(hdl, sizeof (rdsk_node_t)); + + uint64_t size = GetFileDriveSize(disk); + + error = asprintf(&slice->rn_name, + "#%llu#%llu#%s", + 0ULL, size, + deviceInterfaceDetailData->DevicePath); + + if (error == -1) { + free(slice); + continue; + } + + slice->rn_vdev_guid = 0; + slice->rn_lock = lock; + slice->rn_avl = *slice_cache; + slice->rn_hdl = hdl; + slice->rn_labelpaths = B_TRUE; + slice->rn_order = + IMPORT_ORDER_SCAN_OFFSET + deviceIndex; + + pthread_mutex_lock(lock); + if (avl_find(*slice_cache, slice, &where)) { + free(slice->rn_name); + free(slice); + } else { + avl_insert(*slice_cache, slice, where); + } + pthread_mutex_unlock(lock); + } + + free(partitions); + } else { + fprintf(stderr, "read partitions ng\n"); + fflush(stderr); + } + + CloseHandle(disk); +#if 1 // efi + // Add the whole physical device, but lets also try + // to read EFI off it. + disk = CreateFile(deviceInterfaceDetailData->DevicePath, + GENERIC_READ, + FILE_SHARE_READ /* | FILE_SHARE_WRITE */, + NULL, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL /* | FILE_FLAG_OVERLAPPED */, + NULL); + + // On standard OsX created zpool, we expect: + // offset name + // 0x200 MBR partition protective GPT + // 0x400 EFI partition, s0 as ZFS + // 0x8410 "version" "name" "testpool" ZFS label + if (disk != INVALID_HANDLE_VALUE) { + fprintf(stderr, "asking libefi to read label\n"); + fflush(stderr); + int error; + struct dk_gpt *vtoc; + error = efi_alloc_and_read(HTOI(disk), &vtoc); + if (error >= 0) { + fprintf(stderr, + "EFI read OK, max partitions %d\n", + vtoc->efi_nparts); + fflush(stderr); + for (int i = 0; i < vtoc->efi_nparts; i++) { + + if (vtoc->efi_parts[i].p_start == 0 && + vtoc->efi_parts[i].p_size == 0) + continue; + + fprintf(stderr, + " part %d: offset %llx: len %llx: " + "tag: %x name: '%s'\n", + i, vtoc->efi_parts[i].p_start, + vtoc->efi_parts[i].p_size, + vtoc->efi_parts[i].p_tag, + vtoc->efi_parts[i].p_name); + fflush(stderr); + if (vtoc->efi_parts[i].p_start != 0 && + vtoc->efi_parts[i].p_size != 0) { + // Lets invent a naming scheme with start, + // and len in it. + + slice = zutil_alloc(hdl, + sizeof (rdsk_node_t)); + + error = asprintf(&slice->rn_name, "#%llu#%llu#%s", + vtoc->efi_parts[i].p_start * vtoc->efi_lbasize, + vtoc->efi_parts[i].p_size * vtoc->efi_lbasize, + deviceInterfaceDetailData->DevicePath); + if (error == -1) { + free(slice); + continue; + } + + slice->rn_vdev_guid = 0; + slice->rn_lock = lock; + slice->rn_avl = *slice_cache; + slice->rn_hdl = hdl; + slice->rn_labelpaths = B_TRUE; + slice->rn_order = IMPORT_ORDER_SCAN_OFFSET + i; + + pthread_mutex_lock(lock); + if (avl_find(*slice_cache, slice, &where)) { + free(slice->rn_name); + free(slice); + } else { + avl_insert(*slice_cache, slice, where); + } + pthread_mutex_unlock(lock); + } + } + } + efi_free(vtoc); + CloseHandle(disk); + } else { // Unable to open handle + fprintf(stderr, + "Unable to open disk, are we Administrator? " + "GetLastError() is 0x%x\n", + GetLastError()); + fflush(stderr); + } + +#endif + } // while SetupDiEnumDeviceInterfaces + + return (0); +} + + +/* + * Linux persistent device strings for vdev labels + * + * based on libudev for consistency with libudev disk add/remove events + */ + +typedef struct vdev_dev_strs { + char vds_devid[128]; + char vds_devphys[128]; +} vdev_dev_strs_t; + + +int +zfs_device_get_devid(struct udev_device *dev, char *bufptr, size_t buflen) +{ + return (ENODATA); +} + + +int +zfs_device_get_physical(struct udev_device *dev, char *bufptr, size_t buflen) +{ + return (ENODATA); +} + +/* + * Encode the persistent devices strings + * used for the vdev disk label + */ +static int +encode_device_strings(const char *path, vdev_dev_strs_t *ds, + boolean_t wholedisk) +{ + return (ENOENT); +} + +/* Given a "#1234#1234#/path/part" - find the path part only */ +static void +remove_partition_offset_hack(char *hacked_path, char **out_dev_path) +{ + uint64_t offset; + uint64_t len; + char *end = NULL; + + if (hacked_path[0] != '#') { + *out_dev_path = hacked_path; + return; + } + + end = hacked_path; + for (int i = 0; i < 3; i++) { + while (*end && *end != '#') { + end++; + } + if (*end == 0) + break; + end++; + } + *out_dev_path = end; +} + +static int +get_device_number(char *device_path, STORAGE_DEVICE_NUMBER *device_number) +{ + HANDLE hDevice = INVALID_HANDLE_VALUE; + DWORD returned = 0; + + hDevice = CreateFile(device_path, + GENERIC_READ, + FILE_SHARE_READ /* | FILE_SHARE_WRITE */, + NULL, + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL /* | FILE_FLAG_OVERLAPPED */, + NULL); + if (hDevice == INVALID_HANDLE_VALUE) { + // fprintf(stderr, "invalid handle value\n"); fflush(stderr); + return (GetLastError()); + } + + BOOL ret = DeviceIoControl(hDevice, + IOCTL_STORAGE_GET_DEVICE_NUMBER, NULL, 0, + (LPVOID)device_number, (DWORD)sizeof (*device_number), + (LPDWORD)&returned, (LPOVERLAPPED)NULL); + + CloseHandle(hDevice); + + if (!ret) { + // fprintf(stderr, "DeviceIoControl returned error\n"); + // fflush(stderr); + return (ERROR_INVALID_FUNCTION); + } + + return (ERROR_SUCCESS); +} + + +/* + * Update a leaf vdev's persistent device strings + * + * - only applies for a dedicated leaf vdev (aka whole disk) + * - updated during pool create|add|attach|import + * - used for matching device matching during auto-{online,expand,replace} + * - stored in a leaf disk config label (i.e. alongside 'path' NVP) + * - these strings are currently not used in kernel (i.e. for vdev_disk_open) + * + * single device node example: + * devid: 'scsi-MG03SCA300_350000494a8cb3d67-part1' + * phys_path: 'pci-0000:04:00.0-sas-0x50000394a8cb3d67-lun-0' + * + * multipath device node example: + * devid: 'dm-uuid-mpath-35000c5006304de3f' + * + * We also store the enclosure sysfs path for turning on enclosure LEDs + * (if applicable): + * vdev_enc_sysfs_path: '/sys/class/enclosure/11:0:1:0/SLOT 4' + */ +void +update_vdev_config_dev_strsXXXX(nvlist_t *nv) +{ + vdev_dev_strs_t vds; + char *env, *type, *path, *devid; + uint64_t wholedisk = 0; + int ret; + // Build a pretty vdev_path here + char *end = NULL; + STORAGE_DEVICE_NUMBER deviceNumber; + char udevpath[MAXPATHLEN]; + + if (nvlist_lookup_string(nv, ZPOOL_CONFIG_PATH, &path) != 0) + return; + nvlist_lookup_uint64(nv, ZPOOL_CONFIG_WHOLE_DISK, &wholedisk); + + fprintf(stderr, "working on dev '%s'\n", path); fflush(stderr); + + devid = strdup(path); + + HANDLE h; + h = CreateFile(path, GENERIC_READ, + FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL); + if (h != INVALID_HANDLE_VALUE) { + struct dk_gpt *vtoc; + if ((efi_alloc_and_read(HTOI(h), &vtoc)) == 0) { + // Slice 1 should be ZFS + fprintf(stderr, + "this code assumes ZFS is on partition 1\n"); + fflush(stderr); + snprintf(udevpath, MAXPATHLEN, "#%llu#%llu#%s", + vtoc->efi_parts[0].p_start * (uint64_t)vtoc->efi_lbasize, + vtoc->efi_parts[0].p_size * (uint64_t)vtoc->efi_lbasize, + path); + efi_free(vtoc); + path = udevpath; + } + CloseHandle(h); + } + + remove_partition_offset_hack(devid, &end); + + // If it is a device, clean that up - otherwise it is a filename pool + ret = get_device_number(end, &deviceNumber); + if (ret == 0) { + char *vdev_path; + + if (wholedisk) + asprintf(&vdev_path, "/dev/physicaldrive%lu", + deviceNumber.DeviceNumber); + else + asprintf(&vdev_path, "/dev/Harddisk%luPartition%lu", + deviceNumber.DeviceNumber, + deviceNumber.PartitionNumber); + + fprintf(stderr, "setting path here '%s'\r\n", vdev_path); + fflush(stderr); + fprintf(stderr, "setting physpath here '%s'\r\n", path); + fflush(stderr); + nvlist_remove_all(nv, ZPOOL_CONFIG_PHYS_PATH); + if (nvlist_add_string(nv, ZPOOL_CONFIG_PHYS_PATH, path) != 0) + return; + if (nvlist_add_string(nv, ZPOOL_CONFIG_VDEV_ENC_SYSFS_PATH, + strdup(path)) != 0) + return; + // This call frees the original "path", can't access after now + nvlist_remove_all(nv, ZPOOL_CONFIG_PATH); + if (nvlist_add_string(nv, ZPOOL_CONFIG_PATH, vdev_path) != 0) + return; + + } else { + // Not a disk, filepool. Fix path. + char *vdev_path; + + if (path[0] != '/') { + asprintf(&vdev_path, "\\??\\%s", path); + zfs_backslashes(vdev_path); + if (nvlist_add_string(nv, ZPOOL_CONFIG_PHYS_PATH, + vdev_path) != 0) + return; + + asprintf(&vdev_path, "//./%s", path); + zfs_slashes(vdev_path); + fprintf(stderr, "correcting path: '%s' \r\n", + vdev_path); + fflush(stderr); + if (nvlist_add_string(nv, ZPOOL_CONFIG_PATH, + vdev_path) != 0) + return; + + } + } + + free(devid); + +} + +/* + * Update a leaf vdev's persistent device strings + * + * - only applies for a dedicated leaf vdev (aka whole disk) + * - updated during pool create|add|attach|import + * - used for matching device matching during auto-{online,expand,replace} + * - stored in a leaf disk config label (i.e. alongside 'path' NVP) + * - these strings are currently not used in kernel (i.e. for vdev_disk_open) + * + * single device node example: + * devid: 'scsi-MG03SCA300_350000494a8cb3d67-part1' + * phys_path: 'pci-0000:04:00.0-sas-0x50000394a8cb3d67-lun-0' + * + * multipath device node example: + * devid: 'dm-uuid-mpath-35000c5006304de3f' + * + * We also store the enclosure sysfs path for turning on enclosure LEDs + * (if applicable): + * vdev_enc_sysfs_path: '/sys/class/enclosure/11:0:1:0/SLOT 4' + */ +void +update_vdev_config_dev_strs(nvlist_t *nv) +{ + /* + * First Windows work + */ + vdev_dev_strs_t vds; + char *env, *type, *path, *devid; + uint64_t wholedisk = 0; + int ret; + // Build a pretty vdev_path here + char *end = NULL; + STORAGE_DEVICE_NUMBER deviceNumber; + char udevpath[MAXPATHLEN]; + + if (nvlist_lookup_string(nv, ZPOOL_CONFIG_PATH, &path) != 0) + return; + nvlist_lookup_uint64(nv, ZPOOL_CONFIG_WHOLE_DISK, &wholedisk); + + fprintf(stderr, "working on dev '%s'\n", path); fflush(stderr); + + devid = strdup(path); + + HANDLE h; + h = CreateFile(path, GENERIC_READ, + FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL); + if (h != INVALID_HANDLE_VALUE) { + struct dk_gpt *vtoc; + if ((efi_alloc_and_read(HTOI(h), &vtoc)) == 0) { + // Slice 1 should be ZFS + fprintf(stderr, + "this code assumes ZFS is on partition 1\n"); + fflush(stderr); + snprintf(udevpath, MAXPATHLEN, "#%llu#%llu#%s", + vtoc->efi_parts[0].p_start * (uint64_t)vtoc-> + efi_lbasize, + vtoc->efi_parts[0].p_size * (uint64_t)vtoc-> + efi_lbasize, + path); + efi_free(vtoc); + path = udevpath; + } + CloseHandle(h); + } + + remove_partition_offset_hack(devid, &end); + + // If it is a device, clean that up - otherwise it is a filename pool + ret = get_device_number(end, &deviceNumber); + if (ret == 0) { + char *vdev_path; + + if (wholedisk) + asprintf(&vdev_path, "/dev/physicaldrive%lu", + deviceNumber.DeviceNumber); + else + asprintf(&vdev_path, "/dev/Harddisk%luPartition%lu", + deviceNumber.DeviceNumber, + deviceNumber.PartitionNumber); + + fprintf(stderr, "setting path here '%s'\r\n", vdev_path); + fflush(stderr); + fprintf(stderr, "setting physpath here '%s'\r\n", path); + fflush(stderr); + nvlist_remove_all(nv, ZPOOL_CONFIG_PHYS_PATH); + if (nvlist_add_string(nv, ZPOOL_CONFIG_PHYS_PATH, path) != 0) + return; + // This call frees the original "path", can't access after now + nvlist_remove_all(nv, ZPOOL_CONFIG_PATH); + if (nvlist_add_string(nv, ZPOOL_CONFIG_PATH, vdev_path) != 0) + return; + + } else { + // Not a disk, filepool. Fix path. + char *vdev_path; + + if (path[0] != '/') { + asprintf(&vdev_path, "\\??\\%s", path); + zfs_backslashes(vdev_path); + if (nvlist_add_string(nv, ZPOOL_CONFIG_PHYS_PATH, \ + vdev_path) != 0) + return; + + asprintf(&vdev_path, "//./%s", path); + zfs_slashes(vdev_path); + fprintf(stderr, "correcting path: '%s' \r\n", vdev_path); + fflush(stderr); + if (nvlist_add_string(nv, ZPOOL_CONFIG_PATH, vdev_path) != 0) + return; + + } + } + + free(devid); + + + + /* + * For the benefit of legacy ZFS implementations, allow + * for opting out of devid strings in the vdev label. + * + * example use: + * env ZFS_VDEV_DEVID_OPT_OUT=YES zpool import dozer + * + * explanation: + * Older ZFS on Linux implementations had issues when attempting to + * display pool config VDEV names if a "devid" NVP value is present + * in the pool's config. + * + * + * For example, a pool that originated on illumos platform would + * have a devid value in the config and "zpool status" would fail + * when listing the config. + * + * A pool can be stripped of any "devid" values on import or + * prevented from adding them on zpool create|add by setting + * ZFS_VDEV_DEVID_OPT_OUT. + */ + env = getenv("ZFS_VDEV_DEVID_OPT_OUT"); + if (env && (strtoul(env, NULL, 0) > 0 || + !strncasecmp(env, "YES", 3) || !strncasecmp(env, "ON", 2))) { + (void) nvlist_remove_all(nv, ZPOOL_CONFIG_DEVID); + (void) nvlist_remove_all(nv, ZPOOL_CONFIG_PHYS_PATH); + return; + } + + if (nvlist_lookup_string(nv, ZPOOL_CONFIG_TYPE, &type) != 0 || + strcmp(type, VDEV_TYPE_DISK) != 0) { + return; + } + if (nvlist_lookup_string(nv, ZPOOL_CONFIG_PATH, &path) != 0) + return; + (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_WHOLE_DISK, &wholedisk); + + /* + * Update device string values in the config nvlist. + */ + if (encode_device_strings(path, &vds, (boolean_t)wholedisk) == 0) { + (void) nvlist_add_string(nv, ZPOOL_CONFIG_DEVID, vds.vds_devid); + if (vds.vds_devphys[0] != '\0') { + (void) nvlist_add_string(nv, ZPOOL_CONFIG_PHYS_PATH, + vds.vds_devphys); + } + + } else { + /* Clear out any stale entries. */ + (void) nvlist_remove_all(nv, ZPOOL_CONFIG_DEVID); + // (void)nvlist_remove_all(nv, ZPOOL_CONFIG_PHYS_PATH); + (void) nvlist_remove_all(nv, ZPOOL_CONFIG_VDEV_ENC_SYSFS_PATH); + } +} + +/* + * The shared resolve shortname requires the shortname to exist in a directory + * which is not the case for us to handle "PHYSICALDRIVEx". The device object + * store is not opendir()able. + */ +int +zfs_resolve_shortname_os(const char *name, char *path, size_t len) +{ + /* Ok lets let them say just "PHYSICALDRIVEx" */ + if (!strncasecmp("PHYSICALDRIVE", name, 13)) { + // Convert to "\\?\PHYSICALDRIVEx" + snprintf(path, len, "\\\\?\\%s", name); + printf("Expanded path to '%s'\n", path); + return (0); + } + if (!strncasecmp("Harddisk", name, 8)) { + // Convert to "\\?\HarddiskXPartitionY" + snprintf(path, len, "\\\\?\\%s", name); + printf("Expanded path to '%s'\n", path); + return (0); + } + return (ENOENT); +} + +void +update_vdevs_config_dev_sysfs_path(nvlist_t *config) +{ + (void) config; +} diff --git a/lib/libzutil/zutil_device_path.c b/lib/libzutil/zutil_device_path.c index 0425018e1022..e89b6a789270 100644 --- a/lib/libzutil/zutil_device_path.c +++ b/lib/libzutil/zutil_device_path.c @@ -85,7 +85,12 @@ zfs_resolve_shortname(const char *name, char *path, size_t len) } } - return (errno = ENOENT); +#ifdef _WIN32 + /* Nothing found, attempt OS specific shortnames */ + if (zfs_resolve_shortname_os(name, path, len) == 0) + return (0); +#endif + return (ENOENT); } /* @@ -126,7 +131,12 @@ zfs_strcmp_shortname(const char *name, const char *cmp_name, int wholedisk) if (wholedisk) path_len = zfs_append_partition(path_name, MAXPATHLEN); +#ifdef _WIN32 + if ((path_len == cmp_len) && + strcasecmp(path_name, cmp_name) == 0) { +#else if ((path_len == cmp_len) && strcmp(path_name, cmp_name) == 0) { +#endif error = 0; break; } diff --git a/lib/libzutil/zutil_import.h b/lib/libzutil/zutil_import.h index da7941fa5307..3ca04b3121a5 100644 --- a/lib/libzutil/zutil_import.h +++ b/lib/libzutil/zutil_import.h @@ -57,5 +57,7 @@ typedef struct rdsk_node { int slice_cache_compare(const void *, const void *); void zpool_open_func(void *); +int zfs_resolve_shortname_os(const char *name, char *path, size_t len); + #endif /* _LIBZUTIL_ZUTIL_IMPORT_H_ */ diff --git a/lib/os/windows/CMakeLists.txt b/lib/os/windows/CMakeLists.txt new file mode 100644 index 000000000000..a1fede57bf98 --- /dev/null +++ b/lib/os/windows/CMakeLists.txt @@ -0,0 +1,5 @@ +add_subdirectory(zlib-1.2.13) +add_subdirectory(libpthread) +add_subdirectory(libkstat) +add_subdirectory(libuuid) +add_subdirectory(libregex) diff --git a/lib/os/windows/libkstat/CMakeLists.txt b/lib/os/windows/libkstat/CMakeLists.txt new file mode 100644 index 000000000000..64109da7b156 --- /dev/null +++ b/lib/os/windows/libkstat/CMakeLists.txt @@ -0,0 +1,8 @@ + +use_clang() + +add_library(libkstat + kstat.c + gmatch.c +) +target_link_libraries(libkstat PUBLIC libspl) diff --git a/lib/os/windows/libkstat/gmatch.c b/lib/os/windows/libkstat/gmatch.c new file mode 100644 index 000000000000..08a5a4fe9e19 --- /dev/null +++ b/lib/os/windows/libkstat/gmatch.c @@ -0,0 +1,167 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright 2008 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* Copyright (c) 1988 AT&T */ +/* All Rights Reserved */ + +#include +#include +#include +#include +// #include +// #include "_range.h" + +#define multibyte (0) + +#define Popwchar(p, c) \ + n = mbtowc(&cl, p, MB_LEN_MAX); \ + c = cl; \ + if (n <= 0) \ + return (0); \ + p += n + +#define WCHAR_CSMASK 0x30000000 +#define valid_range(c1, c2) \ + (((c1) & WCHAR_CSMASK) == ((c2) & WCHAR_CSMASK) && \ + ((c1) > 0xff || !iscntrl((int)c1)) && ((c2) > 0xff || \ + !iscntrl((int)c2))) + +int +gmatch(const char *s, const char *p) +{ + const char *olds; + wchar_t scc, c; + int n; + wchar_t cl; + + olds = s; + n = mbtowc(&cl, s, MB_LEN_MAX); + if (n <= 0) { + s++; + scc = n; + } else { + scc = cl; + s += n; + } + n = mbtowc(&cl, p, MB_LEN_MAX); + if (n < 0) + return (0); + if (n == 0) + return (scc == 0); + p += n; + c = cl; + + switch (c) { + case '[': + if (scc <= 0) + return (0); + { + int ok; + wchar_t lc = 0; + int notflag = 0; + + ok = 0; + if (*p == '!') { + notflag = 1; + p++; + } + Popwchar(p, c); + do + { + if (c == '-' && lc && *p != ']') { + Popwchar(p, c); + if (c == '\\') { + Popwchar(p, c); + } + if (notflag) { + if (!multibyte || + valid_range(lc, c)) { + if (scc < lc || scc > c) + ok++; + else + return (0); + } + } else { + if (!multibyte || + valid_range(lc, c)) + if (lc <= scc && + scc <= c) + ok++; + } + } else if (c == '\\') { + /* skip to quoted character */ + Popwchar(p, c); + } + lc = c; + if (notflag) { + if (scc != lc) + ok++; + else + return (0); + } + else + { + if (scc == lc) + ok++; + } + Popwchar(p, c); + } while (c != ']'); + return (ok ? gmatch(s, p) : 0); + } + + case '\\': + /* skip to quoted character and see if it matches */ + Popwchar(p, c); + /* FALLTHROUGH */ + + default: + if (c != scc) + return (0); + /* FALLTHROUGH */ + + case '?': + return (scc > 0 ? gmatch(s, p) : 0); + + case '*': + while (*p == '*') + p++; + + if (*p == 0) + return (1); + s = olds; + while (*s) { + if (gmatch(s, p)) + return (1); + n = mbtowc(&cl, s, MB_LEN_MAX); + if (n < 0) + /* skip past illegal byte sequence */ + s++; + else + s += n; + } + return (0); + } +} diff --git a/lib/os/windows/libkstat/kstat.c b/lib/os/windows/libkstat/kstat.c new file mode 100644 index 000000000000..636adfd85682 --- /dev/null +++ b/lib/os/windows/libkstat/kstat.c @@ -0,0 +1,386 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2004 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + * Copyright 2018 Jorgen Lundman . All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kstat.h" + +/*LINTLIBRARY*/ + +static void +kstat_zalloc(void **ptr, size_t size, int free_first) +{ + if (free_first) + free(*ptr); + *ptr = calloc(size, 1); +} + +static void +kstat_chain_free(kstat_ctl_t *kc) +{ + kstat_t *ksp, *nksp; + + ksp = kc->kc_chain; + while (ksp) { + nksp = ksp->ks_next; + free(ksp->ks_data); + free(ksp); + ksp = nksp; + } + kc->kc_chain = NULL; + kc->kc_chain_id = 0; +} + +kstat_ctl_t * +kstat_open(void) +{ + kstat_ctl_t *kc; + HANDLE h; + + // ZFSDEV - no includes + h = CreateFile("\\\\.\\ZFS", GENERIC_READ | GENERIC_WRITE, + 0, NULL, OPEN_EXISTING, 0, NULL); + if (h == INVALID_HANDLE_VALUE) + return (NULL); + + kstat_zalloc((void **)&kc, sizeof (kstat_ctl_t), 0); + if (kc == NULL) + return (NULL); + + kc->kc_kd = h; + kc->kc_chain = NULL; + kc->kc_chain_id = 0; + if (kstat_chain_update(kc) == -1) { + int saved_err = errno; + (void) kstat_close(kc); + errno = saved_err; + return (NULL); + } + return (kc); +} + +int +kstat_close(kstat_ctl_t *kc) +{ + int rc = 0; + + kstat_chain_free(kc); + CloseHandle(kc->kc_kd); + free(kc); + return (rc); +} + +int +kstat_ioctl(HANDLE hDevice, int request, kstat_t *ksp) +{ + int error; + ULONG bytesReturned; + + error = DeviceIoControl(hDevice, + (DWORD)request, + ksp, + (DWORD)sizeof (kstat_t), + ksp, + (DWORD)sizeof (kstat_t), + &bytesReturned, + NULL); + + // Windows: error from DeviceIoControl() is unlikely + if (error == 0) { + error = GetLastError(); + } else { + // More likely is we return error from kernel in errnovalue, + // or value in returnvalue + errno = ksp->ks_errnovalue; + error = ksp->ks_returnvalue; + } + + return (error); +} + +kid_t +kstat_read(kstat_ctl_t *kc, kstat_t *ksp, void *data) +{ + kid_t kcid; + + if (ksp->ks_data == NULL && ksp->ks_data_size > 0) { + kstat_zalloc(&ksp->ks_data, ksp->ks_data_size, 0); + if (ksp->ks_data == NULL) + return (-1); + } + while ((kcid = + (kid_t)kstat_ioctl(kc->kc_kd, KSTAT_IOC_READ, ksp)) == -1) { + if (errno == EAGAIN) { + (void) usleep(100); /* back off a moment */ + continue; /* and try again */ + } + /* + * Mating dance for variable-size kstats. + * You start with a buffer of a certain size, + * which you hope will hold all the data. + * If your buffer is too small, the kstat driver + * returns ENOMEM and sets ksp->ks_data_size to + * the current size of the kstat's data. You then + * resize your buffer and try again. In practice, + * this almost always converges in two passes. + */ + if (errno == ENOMEM && (ksp->ks_flags & + (KSTAT_FLAG_VAR_SIZE | KSTAT_FLAG_LONGSTRINGS))) { + kstat_zalloc(&ksp->ks_data, ksp->ks_data_size, 1); + if (ksp->ks_data != NULL) + continue; + } + return (-1); + } + if (data != NULL) { + (void) memcpy(data, ksp->ks_data, ksp->ks_data_size); + if (ksp->ks_type == KSTAT_TYPE_NAMED && + ksp->ks_data_size != + ksp->ks_ndata * sizeof (kstat_named_t)) { + /* + * Has KSTAT_DATA_STRING fields. Fix the pointers. + */ + uint_t i; + kstat_named_t *knp = data; + + for (i = 0; i < ksp->ks_ndata; i++, knp++) { + if (knp->data_type != KSTAT_DATA_STRING) + continue; + if (KSTAT_NAMED_STR_PTR(knp) == NULL) + continue; + /* + * The offsets of the strings within the + * buffers are the same, so add the offset of + * the string to the beginning of 'data' to fix + * the pointer so that strings in 'data' don't + * point at memory in 'ksp->ks_data'. + */ + KSTAT_NAMED_STR_PTR(knp) = (char *)data + + (KSTAT_NAMED_STR_PTR(knp) - + (char *)ksp->ks_data); + } + } + } + return (kcid); +} + +kid_t +kstat_write(kstat_ctl_t *kc, kstat_t *ksp, void *data) +{ + kid_t kcid; + + if (ksp->ks_data == NULL && ksp->ks_data_size > 0) { + kstat_zalloc(&ksp->ks_data, ksp->ks_data_size, 0); + if (ksp->ks_data == NULL) + return (-1); + } + if (data != NULL) { + (void) memcpy(ksp->ks_data, data, ksp->ks_data_size); + if (ksp->ks_type == KSTAT_TYPE_NAMED) { + kstat_named_t *oknp = data; + kstat_named_t *nknp = KSTAT_NAMED_PTR(ksp); + uint_t i; + + for (i = 0; i < ksp->ks_ndata; i++, oknp++, nknp++) { + if (nknp->data_type != KSTAT_DATA_STRING) + continue; + if (KSTAT_NAMED_STR_PTR(nknp) == NULL) + continue; + /* + * The buffer passed in as 'data' has string + * pointers that point within 'data'. Fix the + * pointers so they point into the same offset + * within the newly allocated buffer. + */ + KSTAT_NAMED_STR_PTR(nknp) = + (char *)ksp->ks_data + + (KSTAT_NAMED_STR_PTR(oknp) - (char *)data); + } + } + + } + while ((kcid = + (kid_t)kstat_ioctl(kc->kc_kd, KSTAT_IOC_WRITE, ksp)) == -1) { + if (errno == EAGAIN) { + (void) usleep(100); /* back off a moment */ + continue; /* and try again */ + } + break; + } + return (kcid); +} + +/* + * If the current KCID is the same as kc->kc_chain_id, return 0; + * if different, update the chain and return the new KCID. + * This operation is non-destructive for unchanged kstats. + */ +kid_t +kstat_chain_update(kstat_ctl_t *kc) +{ + kstat_t k0, *headers, *oksp, *nksp, **okspp, *next; + int i; + kid_t kcid; + kstat_t ksp = { 0 }; + + kcid = (kid_t)kstat_ioctl(kc->kc_kd, KSTAT_IOC_CHAIN_ID, &ksp); + if (kcid == -1) + return (-1); + if (kcid == kc->kc_chain_id) + return (0); + + /* + * kstat 0's data is the kstat chain, so we can get the chain + * by doing a kstat_read() of this kstat. The only fields the + * kstat driver needs are ks_kid (this identifies the kstat), + * ks_data (the pointer to our buffer), and ks_data_size (the + * size of our buffer). By zeroing the struct we set ks_data = NULL + * and ks_data_size = 0, so that kstat_read() will automatically + * determine the size and allocate space for us. We also fill in the + * name, so that truss can print something meaningful. + */ + memset(&k0, 0, sizeof (k0)); + (void) strlcpy(k0.ks_name, "kstat_headers", KSTAT_STRLEN); + + kcid = kstat_read(kc, &k0, NULL); + if (kcid == -1) { + free(k0.ks_data); + /* errno set by kstat_read() */ + return (-1); + } + headers = k0.ks_data; + + /* + * Chain the new headers together + */ + for (i = 1; i < k0.ks_ndata; i++) + headers[i - 1].ks_next = &headers[i]; + + headers[k0.ks_ndata - 1].ks_next = NULL; + + /* + * Remove all deleted kstats from the chain. + */ + nksp = headers; + okspp = &kc->kc_chain; + oksp = kc->kc_chain; + while (oksp != NULL) { + next = oksp->ks_next; + if (nksp != NULL && oksp->ks_kid == nksp->ks_kid) { + okspp = &oksp->ks_next; + nksp = nksp->ks_next; + } else { + *okspp = oksp->ks_next; + free(oksp->ks_data); + free(oksp); + } + oksp = next; + } + + /* + * Add all new kstats to the chain. + */ + while (nksp != NULL) { + kstat_zalloc((void **)okspp, sizeof (kstat_t), 0); + if ((oksp = *okspp) == NULL) { + free(headers); + return (-1); + } + *oksp = *nksp; + okspp = &oksp->ks_next; + oksp->ks_next = NULL; + oksp->ks_data = NULL; + nksp = nksp->ks_next; + } + + free(headers); + kc->kc_chain_id = kcid; + return (kcid); +} + +kstat_t * +kstat_lookup(kstat_ctl_t *kc, char *ks_module, int ks_instance, char *ks_name) +{ + kstat_t *ksp; + + for (ksp = kc->kc_chain; ksp; ksp = ksp->ks_next) { + if ((ks_module == NULL || + strcmp(ksp->ks_module, ks_module) == 0) && + (ks_instance == -1 || ksp->ks_instance == ks_instance) && + (ks_name == NULL || strcmp(ksp->ks_name, ks_name) == 0)) + return (ksp); + } + + errno = ENOENT; + return (NULL); +} + +void * +kstat_data_lookup(kstat_t *ksp, char *name) +{ + int i, size; + char *namep, *datap; + + switch (ksp->ks_type) { + + case KSTAT_TYPE_NAMED: + size = sizeof (kstat_named_t); + namep = KSTAT_NAMED_PTR(ksp)->name; + break; + + case KSTAT_TYPE_TIMER: + size = sizeof (kstat_timer_t); + namep = KSTAT_TIMER_PTR(ksp)->name; + break; + + default: + errno = EINVAL; + return (NULL); + } + + datap = ksp->ks_data; + for (i = 0; i < ksp->ks_ndata; i++) { + if (strcmp(name, namep) == 0) + return (datap); + namep += size; + datap += size; + } + errno = ENOENT; + return (NULL); +} diff --git a/lib/os/windows/libkstat/kstat.h b/lib/os/windows/libkstat/kstat.h new file mode 100644 index 000000000000..ccc65f90f1ef --- /dev/null +++ b/lib/os/windows/libkstat/kstat.h @@ -0,0 +1,60 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License, Version 1.0 only + * (the "License"). You may not use this file except in compliance + * with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 1992 by Sun Microsystems, Inc. + */ + +#ifndef _KSTAT_H +#define _KSTAT_H + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/* + * kstat_open() returns a pointer to a kstat_ctl_t. + * This is used for subsequent libkstat operations. + */ +typedef struct kstat_ctl { + kid_t kc_chain_id; /* current kstat chain ID */ + kstat_t *kc_chain; /* pointer to kstat chain */ + HANDLE kc_kd; /* libzfs_handle to /dev/zfs descriptor */ +} kstat_ctl_t; + +extern kstat_ctl_t *kstat_open(void); +extern int kstat_close(kstat_ctl_t *); +extern kid_t kstat_read(kstat_ctl_t *, kstat_t *, void *); +extern kid_t kstat_write(kstat_ctl_t *, kstat_t *, void *); +extern kid_t kstat_chain_update(kstat_ctl_t *); +extern kstat_t *kstat_lookup(kstat_ctl_t *, char *, int, char *); +extern void *kstat_data_lookup(kstat_t *, char *); + +extern int gmatch(const char *s, const char *p); + +#ifdef __cplusplus +} +#endif + +#endif /* _KSTAT_H */ diff --git a/lib/os/windows/libpthread/CMakeLists.txt b/lib/os/windows/libpthread/CMakeLists.txt new file mode 100644 index 000000000000..2c11f627f8b4 --- /dev/null +++ b/lib/os/windows/libpthread/CMakeLists.txt @@ -0,0 +1,7 @@ + +use_clang() + +add_library(libpthread + pthread.c +) +target_include_directories(libpthread INTERFACE "") diff --git a/lib/os/windows/libpthread/pthread.c b/lib/os/windows/libpthread/pthread.c new file mode 100644 index 000000000000..545a16fcb001 --- /dev/null +++ b/lib/os/windows/libpthread/pthread.c @@ -0,0 +1,51 @@ +/* + * Posix Threads library for Microsoft Windows + * + * Use at own risk, there is no implied warranty to this code. + * It uses undocumented features of Microsoft Windows that can change + * at any time in the future. + * + * (C) 2010 Lockless Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * + * + * * Redistributions of source code must retain the above copyright notice, + * this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright notice, + * this list of conditions and the following disclaimer in the documentation + * and/or other materials provided with the distribution. + * * Neither the name of Lockless Inc. nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AN ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR + * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include + +volatile long _pthread_cancelling; + +int _pthread_concur; + +/* Will default to zero as needed */ +pthread_once_t _pthread_tls_once; +DWORD _pthread_tls; + +/* Note initializer is zero, so this works */ +pthread_rwlock_t _pthread_key_lock; +long _pthread_key_max; +long _pthread_key_sch; +void (**_pthread_key_dest)(void *); diff --git a/lib/os/windows/libpthread/pthread.h b/lib/os/windows/libpthread/pthread.h new file mode 100644 index 000000000000..5a4ea6bcfe0b --- /dev/null +++ b/lib/os/windows/libpthread/pthread.h @@ -0,0 +1,1653 @@ +/* + * Posix Threads library for Microsoft Windows + * + * Use at own risk, there is no implied warranty to this code. + * It uses undocumented features of Microsoft Windows that can change + * at any time in the future. + * + * (C) 2010 Lockless Inc. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are met: + * + * + * * Redistributions of source code must retain the above copyright notice, + * this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright notice, + * this list of conditions and the following disclaimer in the documentation + * and/or other materials provided with the distribution. + * * Neither the name of Lockless Inc. nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AN ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR + * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING + * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS + * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/* + * You may want to use the MingW64 winpthreads library instead. + * It is based on this, but adds error checking. + */ + +/* + * Version 1.0.1 Released 2 Feb 2012 + * Fixes pthread_barrier_destroy() to wait for threads to exit the barrier. + */ + +#ifndef WIN_PTHREADS +#define WIN_PTHREADS + +/* Windows has own VT_ERROR to libefi one */ +#undef VT_ERROR +#define VT_ERROR WIN_VT_ERROR +#define _WIN32_MEAN_AND_LEAN +#include +#undef VT_ERROR + +#include +#include +#include +#include +#include +#include +#include + +// #define ETIMEDOUT 110 +// #define ENOTSUP 134 + +// warning C4018: '>': signed/unsigned mismatch +#pragma warning(disable: 4018) +#if __clang__ +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wunused-function" +#endif + +#define PTHREAD_CANCEL_DISABLE 0 +#define PTHREAD_CANCEL_ENABLE 0x01 + +#define PTHREAD_CANCEL_DEFERRED 0 +#define PTHREAD_CANCEL_ASYNCHRONOUS 0x02 + +#define PTHREAD_CREATE_JOINABLE 0 +#define PTHREAD_CREATE_DETACHED 0x04 + +#define PTHREAD_EXPLICT_SCHED 0 +#define PTHREAD_INHERIT_SCHED 0x08 + +#define PTHREAD_SCOPE_PROCESS 0 +#define PTHREAD_SCOPE_SYSTEM 0x10 + +#define PTHREAD_DEFAULT_ATTR (PTHREAD_CANCEL_ENABLE) + +#define PTHREAD_CANCELED ((void *) 0xDEADBEEFULL) + +#define PTHREAD_ONCE_INIT 0 +#define PTHREAD_MUTEX_INITIALIZER {(void*)-1, -1, 0, 0, 0, 0} +#define PTHREAD_RWLOCK_INITIALIZER {0} +#define PTHREAD_COND_INITIALIZER {0} +#define PTHREAD_BARRIER_INITIALIZER \ + {0, 0, PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER} +#define PTHREAD_SPINLOCK_INITIALIZER 0 + +#define PTHREAD_DESTRUCTOR_ITERATIONS 256 +#define PTHREAD_KEYS_MAX (1<<20) + +#define PTHREAD_MUTEX_NORMAL 0 +#define PTHREAD_MUTEX_ERRORCHECK 1 +#define PTHREAD_MUTEX_RECURSIVE 2 +#define PTHREAD_MUTEX_DEFAULT 3 +#define PTHREAD_MUTEX_SHARED 4 +#define PTHREAD_MUTEX_PRIVATE 0 +#define PTHREAD_PRIO_NONE 0 +#define PTHREAD_PRIO_INHERIT 8 +#define PTHREAD_PRIO_PROTECT 16 +#define PTHREAD_PRIO_MULT 32 +#define PTHREAD_PROCESS_SHARED 0 +#define PTHREAD_PROCESS_PRIVATE 1 + +#define PTHREAD_BARRIER_SERIAL_THREAD 1 + +#define PTHREAD_STACK_MIN 0 + +typedef struct _pthread_cleanup _pthread_cleanup; +struct _pthread_cleanup +{ + void (*func)(void *); + void *arg; + _pthread_cleanup *next; +}; + +struct _pthread_v +{ + void *ret_arg; + void *(* func)(void *); + _pthread_cleanup *clean; + HANDLE h; + int cancelled; + unsigned p_state; + int keymax; + void **keyval; + + jmp_buf jb; +}; + +typedef struct _pthread_v *pthread_t; + +typedef struct pthread_barrier_t pthread_barrier_t; +struct pthread_barrier_t +{ + int count; + int total; + CRITICAL_SECTION m; + CONDITION_VARIABLE cv; +}; + +typedef struct pthread_attr_t pthread_attr_t; +struct pthread_attr_t +{ + unsigned p_state; + void *stack; + size_t s_size; +}; + +typedef long pthread_once_t; +typedef unsigned pthread_mutexattr_t; +typedef SRWLOCK pthread_rwlock_t; +typedef CRITICAL_SECTION pthread_mutex_t; +typedef unsigned pthread_key_t; +typedef void *pthread_barrierattr_t; +typedef long pthread_spinlock_t; +typedef int pthread_condattr_t; +typedef CONDITION_VARIABLE pthread_cond_t; +typedef int pthread_rwlockattr_t; + +extern volatile long _pthread_cancelling; + +extern int _pthread_concur; + +/* Will default to zero as needed */ +extern pthread_once_t _pthread_tls_once; +extern DWORD _pthread_tls; + +/* Note initializer is zero, so this works */ +extern pthread_rwlock_t _pthread_key_lock; +extern long _pthread_key_max; +extern long _pthread_key_sch; +extern void (**_pthread_key_dest)(void *); + + +#define pthread_cleanup_push(F, A)\ +{\ + const _pthread_cleanup _pthread_cup = {(void (*)(void *))(F), (A), \ + pthread_self()->clean}; \ + _ReadWriteBarrier();\ + pthread_self()->clean = (_pthread_cleanup *) &_pthread_cup;\ + _ReadWriteBarrier() + +/* Note that if async cancelling is used, then there is a race here */ +#define pthread_cleanup_pop(E) \ + (pthread_self()->clean = _pthread_cup.next, \ + (E?_pthread_cup.func(_pthread_cup.arg):0)); \ +} + +static void _pthread_once_cleanup(pthread_once_t *o) +{ + *o = 0; +} + +static pthread_t pthread_self(void); +static int pthread_once(pthread_once_t *o, void (*func)(void)) +{ + long state = *o; + + _ReadWriteBarrier(); + + while (state != 1) { + if (!state) { + if (!_InterlockedCompareExchange(o, 2, 0)) { + /* Success */ + pthread_cleanup_push(_pthread_once_cleanup, o); + func(); + pthread_cleanup_pop(0); + + /* Mark as done */ + *o = 1; + + return (0); + } + } + + YieldProcessor(); + + _ReadWriteBarrier(); + + state = *o; + } + + /* Done */ + return (0); +} + +static int _pthread_once_raw(pthread_once_t *o, void (*func)(void)) +{ + long state = *o; + + _ReadWriteBarrier(); + + while (state != 1) { + if (!state) { + if (!_InterlockedCompareExchange(o, 2, 0)) { + /* Success */ + func(); + + /* Mark as done */ + *o = 1; + + return (0); + } + } + + YieldProcessor(); + + _ReadWriteBarrier(); + + state = *o; + } + + /* Done */ + return (0); +} + +static int pthread_mutex_lock(pthread_mutex_t *m) +{ + EnterCriticalSection(m); + return (0); +} + +static int pthread_mutex_unlock(pthread_mutex_t *m) +{ + LeaveCriticalSection(m); + return (0); +} + +static int pthread_mutex_trylock(pthread_mutex_t *m) +{ + return (TryEnterCriticalSection(m) ? 0 : EBUSY); +} + +static int pthread_mutex_init(pthread_mutex_t *m, pthread_mutexattr_t *a) +{ + (void) a; + InitializeCriticalSection(m); + + return (0); +} + +static int pthread_mutex_destroy(pthread_mutex_t *m) +{ + DeleteCriticalSection(m); + return (0); +} + +#define pthread_mutex_getprioceiling(M, P) ENOTSUP +#define pthread_mutex_setprioceiling(M, P) ENOTSUP + +static int pthread_equal(pthread_t t1, pthread_t t2) +{ + return (t1 == t2); +} + +static void pthread_testcancel(void); + +static int pthread_rwlock_init(pthread_rwlock_t *l, pthread_rwlockattr_t *a) +{ + (void) a; + InitializeSRWLock(l); + + return (0); +} + +static int pthread_rwlock_destroy(pthread_rwlock_t *l) +{ + (void) *l; + return (0); +} + +static int pthread_rwlock_rdlock(pthread_rwlock_t *l) +{ + pthread_testcancel(); + AcquireSRWLockShared(l); + + return (0); +} + +static int pthread_rwlock_wrlock(pthread_rwlock_t *l) +{ + pthread_testcancel(); + AcquireSRWLockExclusive(l); + + return (0); +} + +static void pthread_tls_init(void) +{ + _pthread_tls = TlsAlloc(); + + /* Cannot continue if out of indexes */ + if (_pthread_tls == TLS_OUT_OF_INDEXES) abort(); +} + +static int pthread_rwlock_unlock(pthread_rwlock_t *l); + +static void _pthread_cleanup_dest(pthread_t t) +{ + int i, j; + + for (j = 0; j < PTHREAD_DESTRUCTOR_ITERATIONS; j++) { + int flag = 0; + + for (i = 0; i < t->keymax; i++) { + void *val = t->keyval[i]; + + if (val) { + pthread_rwlock_rdlock(&_pthread_key_lock); + if ((uintptr_t)_pthread_key_dest[i] > 1) { + /* Call destructor */ + t->keyval[i] = NULL; + _pthread_key_dest[i](val); + flag = 1; + } + pthread_rwlock_unlock(&_pthread_key_lock); + } + } + + /* Nothing to do? */ + if (!flag) + return; + } +} + +static pthread_t pthread_self(void) +{ + pthread_t t; + + _pthread_once_raw(&_pthread_tls_once, pthread_tls_init); + + t = TlsGetValue(_pthread_tls); + + /* Main thread? */ + if (!t) { + t = malloc(sizeof (struct _pthread_v)); + + /* If cannot init main thread, then the only thing is abort */ + if (!t) abort(); + + t->ret_arg = NULL; + t->func = NULL; + t->clean = NULL; + t->cancelled = 0; + t->p_state = PTHREAD_DEFAULT_ATTR; + t->keymax = 0; + t->keyval = NULL; + t->h = GetCurrentThread(); + + /* Save for later */ + TlsSetValue(_pthread_tls, t); + + if (setjmp(t->jb)) { + /* Make sure we free ourselves if we are detached */ + if (!t->h) free(t); + + /* Time to die */ + _endthreadex(0); + } + } + + return (t); +} + +static int pthread_rwlock_unlock(pthread_rwlock_t *l) +{ + void *state = *(void **)l; + + if (state == (void *) 1) { + /* Known to be an exclusive lock */ + ReleaseSRWLockExclusive(l); + } else { + /* A shared unlock will work */ + ReleaseSRWLockShared(l); + } + + return (0); +} + + +static int pthread_rwlock_tryrdlock(pthread_rwlock_t *l) +{ + /* Get the current state of the lock */ + void *state = *(void **) l; + + if (!state) { + /* Unlocked to locked */ + if (!_InterlockedCompareExchangePointer((void *) l, + (void *)0x11, NULL)) + return (0); + return (EBUSY); + } + + /* A single writer exists */ + if (state == (void *) 1) + return (EBUSY); + + /* Multiple writers exist? */ + if ((uintptr_t)state & 14) + return (EBUSY); + + if (_InterlockedCompareExchangePointer((void *) l, + (void *)((uintptr_t)state + 16), state) == state) + return (0); + + return (EBUSY); +} + +static int pthread_rwlock_trywrlock(pthread_rwlock_t *l) +{ + /* Try to grab lock if it has no users */ + if (!_InterlockedCompareExchangePointer((void *) l, (void *)1, + NULL)) + return (0); + + return (EBUSY); +} + +static unsigned long long _pthread_time_in_ms(void) +{ + struct __timeb64 tb; + + _ftime64(&tb); + + return (tb.time * 1000 + tb.millitm); +} + +static unsigned long long +_pthread_time_in_ms_from_timespec(const struct timespec *ts) +{ + unsigned long long t = ts->tv_sec * 1000; + t += ts->tv_nsec / 1000000; + + return (t); +} + +static unsigned long long _pthread_rel_time_in_ms(const struct timespec *ts) +{ + unsigned long long t1 = _pthread_time_in_ms_from_timespec(ts); + unsigned long long t2 = _pthread_time_in_ms(); + + /* Prevent underflow */ + if (t1 < t2) + return (0); + return (t1 - t2); +} + +static int +pthread_rwlock_timedrdlock(pthread_rwlock_t *l, const struct timespec *ts) +{ + unsigned long long ct = _pthread_time_in_ms(); + unsigned long long t = _pthread_time_in_ms_from_timespec(ts); + + pthread_testcancel(); + + /* Use a busy-loop */ + while (1) { + /* Try to grab lock */ + if (!pthread_rwlock_tryrdlock(l)) + return (0); + + /* Get current time */ + ct = _pthread_time_in_ms(); + + /* Have we waited long enough? */ + if (ct > t) + return (ETIMEDOUT); + } +} + +static int +pthread_rwlock_timedwrlock(pthread_rwlock_t *l, const struct timespec *ts) +{ + unsigned long long ct = _pthread_time_in_ms(); + unsigned long long t = _pthread_time_in_ms_from_timespec(ts); + + pthread_testcancel(); + + /* Use a busy-loop */ + while (1) { + /* Try to grab lock */ + if (!pthread_rwlock_trywrlock(l)) + return (0); + + /* Get current time */ + ct = _pthread_time_in_ms(); + + /* Have we waited long enough? */ + if (ct > t) + return (ETIMEDOUT); + } +} + +static int pthread_get_concurrency(int *val) +{ + *val = _pthread_concur; + return (0); +} + +static int pthread_set_concurrency(int val) +{ + _pthread_concur = val; + return (0); +} + +#define pthread_getschedparam(T, P, S) ENOTSUP +#define pthread_setschedparam(T, P, S) ENOTSUP +#define pthread_getcpuclockid(T, C) ENOTSUP + +__declspec(noreturn) static int +pthread_exit(void *res) +{ + pthread_t t = pthread_self(); + + t->ret_arg = res; + + _pthread_cleanup_dest(t); + + longjmp(t->jb, 1); +} + + +static void _pthread_invoke_cancel(void) +{ + _pthread_cleanup *pcup; + + _InterlockedDecrement(&_pthread_cancelling); + + /* Call cancel queue */ + for (pcup = pthread_self()->clean; pcup; pcup = pcup->next) { + pcup->func(pcup->arg); + } + + pthread_exit(PTHREAD_CANCELED); +} + +static void pthread_testcancel(void) +{ + if (_pthread_cancelling) { + pthread_t t = pthread_self(); + + if (t->cancelled && (t->p_state & PTHREAD_CANCEL_ENABLE)) { + _pthread_invoke_cancel(); + } + } +} + + +static int pthread_cancel(pthread_t t) +{ + if (t->p_state & PTHREAD_CANCEL_ASYNCHRONOUS) { + /* Dangerous asynchronous cancelling */ + CONTEXT ctxt; + + /* Already done? */ + if (t->cancelled) + return (ESRCH); + + ctxt.ContextFlags = CONTEXT_CONTROL; + + SuspendThread(t->h); + GetThreadContext(t->h, &ctxt); +#ifdef _M_X64 + ctxt.Rip = (uintptr_t)_pthread_invoke_cancel; +#else + ctxt.Eip = (uintptr_t)_pthread_invoke_cancel; +#endif + SetThreadContext(t->h, &ctxt); + + /* Also try deferred Cancelling */ + t->cancelled = 1; + + /* Notify everyone to look */ + _InterlockedIncrement(&_pthread_cancelling); + + ResumeThread(t->h); + } else { + /* Safe deferred Cancelling */ + t->cancelled = 1; + + /* Notify everyone to look */ + _InterlockedIncrement(&_pthread_cancelling); + } + + return (0); +} + +static unsigned _pthread_get_state(pthread_attr_t *attr, unsigned flag) +{ + return (attr->p_state & flag); +} + +static int _pthread_set_state(pthread_attr_t *attr, unsigned flag, unsigned val) +{ + if (~flag & val) + return (EINVAL); + attr->p_state &= ~flag; + attr->p_state |= val; + + return (0); +} + +static int pthread_attr_init(pthread_attr_t *attr) +{ + attr->p_state = PTHREAD_DEFAULT_ATTR; + attr->stack = NULL; + attr->s_size = 0; + return (0); +} + +static int pthread_attr_destroy(pthread_attr_t *attr) +{ + /* No need to do anything */ + return (0); +} + + +static int pthread_attr_setdetachstate(pthread_attr_t *a, int flag) +{ + return (_pthread_set_state(a, PTHREAD_CREATE_DETACHED, flag)); +} + +static int pthread_attr_getdetachstate(pthread_attr_t *a, int *flag) +{ + *flag = _pthread_get_state(a, PTHREAD_CREATE_DETACHED); + return (0); +} + +static int pthread_attr_setinheritsched(pthread_attr_t *a, int flag) +{ + return (_pthread_set_state(a, PTHREAD_INHERIT_SCHED, flag)); +} + +static int pthread_attr_getinheritsched(pthread_attr_t *a, int *flag) +{ + *flag = _pthread_get_state(a, PTHREAD_INHERIT_SCHED); + return (0); +} + +static int pthread_attr_setscope(pthread_attr_t *a, int flag) +{ + return (_pthread_set_state(a, PTHREAD_SCOPE_SYSTEM, flag)); +} + +static int pthread_attr_getscope(pthread_attr_t *a, int *flag) +{ + *flag = _pthread_get_state(a, PTHREAD_SCOPE_SYSTEM); + return (0); +} + +static int pthread_attr_getstackaddr(pthread_attr_t *attr, void **stack) +{ + *stack = attr->stack; + return (0); +} + +static int pthread_attr_setstackaddr(pthread_attr_t *attr, void *stack) +{ + attr->stack = stack; + return (0); +} + +static int pthread_attr_getstacksize(pthread_attr_t *attr, size_t *size) +{ + *size = attr->s_size; + return (0); +} + +static int pthread_attr_setstacksize(pthread_attr_t *attr, size_t size) +{ + attr->s_size = size; + return (0); +} + +static int +pthread_attr_getstack(pthread_attr_t *attr, void **stack, size_t *size) +{ + *stack = attr->stack; + *size = attr->s_size; + return (0); +} + +static int pthread_attr_setstack(pthread_attr_t *attr, void *stack, size_t size) +{ + attr->stack = stack; + attr->s_size = size; + return (0); +} + +#define pthread_attr_getguardsize(A, S) ENOTSUP +#define pthread_attr_setguardsize(A, S) ENOTSUP +#define pthread_attr_getschedparam(A, S) ENOTSUP +#define pthread_attr_setschedparam(A, S) ENOTSUP +#define pthread_attr_getschedpolicy(A, S) ENOTSUP +#define pthread_attr_setschedpolicy(A, S) ENOTSUP + + +static int pthread_setcancelstate(int state, int *oldstate) +{ + pthread_t t = pthread_self(); + + if ((state & PTHREAD_CANCEL_ENABLE) != state) + return (EINVAL); + if (oldstate) *oldstate = t->p_state & PTHREAD_CANCEL_ENABLE; + t->p_state &= ~PTHREAD_CANCEL_ENABLE; + t->p_state |= state; + + return (0); +} + +static int pthread_setcanceltype(int type, int *oldtype) +{ + pthread_t t = pthread_self(); + + if ((type & PTHREAD_CANCEL_ASYNCHRONOUS) != type) + return (EINVAL); + if (oldtype) *oldtype = t->p_state & PTHREAD_CANCEL_ASYNCHRONOUS; + t->p_state &= ~PTHREAD_CANCEL_ASYNCHRONOUS; + t->p_state |= type; + + return (0); +} + +static int __stdcall pthread_create_wrapper(void *args) +{ + struct _pthread_v *tv = args; + + _pthread_once_raw(&_pthread_tls_once, pthread_tls_init); + + TlsSetValue(_pthread_tls, tv); + + if (!setjmp(tv->jb)) { + /* Call function and save return value */ + tv->ret_arg = tv->func(tv->ret_arg); + + /* Clean up destructors */ + _pthread_cleanup_dest(tv); + } + + /* If we exit too early, then we can race with create */ + while (tv->h == (HANDLE) -1) { + YieldProcessor(); + _ReadWriteBarrier(); + } + + /* Make sure we free ourselves if we are detached */ + if (!tv->h) free(tv); + + return (0); +} + +static int +pthread_create(pthread_t *th, pthread_attr_t *attr, + void *(*func)(void *), void *arg) +{ + struct _pthread_v *tv = malloc(sizeof (struct _pthread_v)); + unsigned ssize = 0; + + if (!tv) + return (1); + + *th = tv; + + /* Save data in pthread_t */ + tv->ret_arg = arg; + tv->func = func; + tv->clean = NULL; + tv->cancelled = 0; + tv->p_state = PTHREAD_DEFAULT_ATTR; + tv->keymax = 0; + tv->keyval = NULL; + tv->h = (HANDLE) -1; + + if (attr) { + tv->p_state = attr->p_state; + ssize = attr->s_size; + } + + /* Make sure tv->h has value of -1 */ + _ReadWriteBarrier(); + + tv->h = (HANDLE) _beginthreadex(NULL, ssize, + (_beginthreadex_proc_type) pthread_create_wrapper, tv, 0, NULL); + + /* Failed */ + if (!tv->h) + return (1); + + if (tv->p_state & PTHREAD_CREATE_DETACHED) { + CloseHandle(tv->h); + _ReadWriteBarrier(); + tv->h = 0; + } + + return (0); +} + +static int pthread_join(pthread_t t, void **res) +{ + struct _pthread_v *tv = t; + + pthread_testcancel(); + + WaitForSingleObject(tv->h, INFINITE); + CloseHandle(tv->h); + + /* Obtain return value */ + if (res) *res = tv->ret_arg; + + free(tv); + + return (0); +} + +static int pthread_detach(pthread_t t) +{ + struct _pthread_v *tv = t; + + /* + * This can't race with thread exit because + * our call would be undefined if called on a dead thread. + */ + + CloseHandle(tv->h); + _ReadWriteBarrier(); + tv->h = 0; + + return (0); +} + +static int pthread_mutexattr_init(pthread_mutexattr_t *a) +{ + *a = 0; + return (0); +} + +static int pthread_mutexattr_destroy(pthread_mutexattr_t *a) +{ + (void) a; + return (0); +} + +static int pthread_mutexattr_gettype(pthread_mutexattr_t *a, int *type) +{ + *type = *a & 3; + + return (0); +} + +static int pthread_mutexattr_settype(pthread_mutexattr_t *a, int type) +{ + if ((unsigned)type > 3) + return (EINVAL); + *a &= ~3; + *a |= type; + + return (0); +} + +static int pthread_mutexattr_getpshared(pthread_mutexattr_t *a, int *type) +{ + *type = *a & 4; + + return (0); +} + +static int pthread_mutexattr_setpshared(pthread_mutexattr_t *a, int type) +{ + if ((type & 4) != type) + return (EINVAL); + + *a &= ~4; + *a |= type; + + return (0); +} + +static int pthread_mutexattr_getprotocol(pthread_mutexattr_t *a, int *type) +{ + *type = *a & (8 + 16); + + return (0); +} + +static int pthread_mutexattr_setprotocol(pthread_mutexattr_t *a, int type) +{ + if ((type & (8 + 16)) != 8 + 16) + return (EINVAL); + + *a &= ~(8 + 16); + *a |= type; + + return (0); +} + +static int pthread_mutexattr_getprioceiling(pthread_mutexattr_t *a, int *prio) +{ + *prio = *a / PTHREAD_PRIO_MULT; + return (0); +} + +static int pthread_mutexattr_setprioceiling(pthread_mutexattr_t *a, int prio) +{ + *a &= (PTHREAD_PRIO_MULT - 1); + *a += prio * PTHREAD_PRIO_MULT; + + return (0); +} + +static int pthread_mutex_timedlock(pthread_mutex_t *m, struct timespec *ts) +{ + unsigned long long t, ct; + + struct _pthread_crit_t { + void *debug; + LONG count; + LONG r_count; + HANDLE owner; + HANDLE sem; + ULONG_PTR spin; + }; + + /* Try to lock it without waiting */ + if (!pthread_mutex_trylock(m)) + return (0); + + ct = _pthread_time_in_ms(); + t = _pthread_time_in_ms_from_timespec(ts); + + while (1) { + /* Have we waited long enough? */ + if (ct > t) + return (ETIMEDOUT); + + /* Wait on semaphore within critical section */ + WaitForSingleObject(((struct _pthread_crit_t *)m)->sem, t - ct); + + /* Try to grab lock */ + if (!pthread_mutex_trylock(m)) + return (0); + + /* Get current time */ + ct = _pthread_time_in_ms(); + } +} + +#define _PTHREAD_BARRIER_FLAG (1<<30) + +static int pthread_barrier_destroy(pthread_barrier_t *b) +{ + EnterCriticalSection(&b->m); + + while (b->total > _PTHREAD_BARRIER_FLAG) { + /* Wait until everyone exits the barrier */ + SleepConditionVariableCS(&b->cv, &b->m, INFINITE); + } + + LeaveCriticalSection(&b->m); + + DeleteCriticalSection(&b->m); + + return (0); +} + +static int pthread_barrier_init(pthread_barrier_t *b, void *attr, int count) +{ + /* Ignore attr */ + (void) attr; + + b->count = count; + b->total = 0; + + InitializeCriticalSection(&b->m); + InitializeConditionVariable(&b->cv); + + return (0); +} + +static int pthread_barrier_wait(pthread_barrier_t *b) +{ + EnterCriticalSection(&b->m); + + while (b->total > _PTHREAD_BARRIER_FLAG) { + /* Wait until everyone exits the barrier */ + SleepConditionVariableCS(&b->cv, &b->m, INFINITE); + } + + /* Are we the first to enter? */ + if (b->total == _PTHREAD_BARRIER_FLAG) b->total = 0; + + b->total++; + + if (b->total == b->count) { + b->total += _PTHREAD_BARRIER_FLAG - 1; + WakeAllConditionVariable(&b->cv); + + LeaveCriticalSection(&b->m); + + return (1); + } else { + while (b->total < _PTHREAD_BARRIER_FLAG) { + /* Wait until enough threads enter the barrier */ + SleepConditionVariableCS(&b->cv, &b->m, INFINITE); + } + + b->total--; + + /* Get entering threads to wake up */ + if (b->total == _PTHREAD_BARRIER_FLAG) + WakeAllConditionVariable(&b->cv); + + LeaveCriticalSection(&b->m); + + return (0); + } +} + +static int pthread_barrierattr_init(void **attr) +{ + *attr = NULL; + return (0); +} + +static int pthread_barrierattr_destroy(void **attr) +{ + /* Ignore attr */ + (void) attr; + + return (0); +} + +static int pthread_barrierattr_setpshared(void **attr, int s) +{ + *attr = (void *)(uintptr_t)s; + return (0); +} + +static int pthread_barrierattr_getpshared(void **attr, int *s) +{ + *s = (int)(size_t)*attr; + + return (0); +} + +static int pthread_key_create(pthread_key_t *key, void (* dest)(void *)) +{ + int i; + long nmax; + void (**d)(void *); + + if (!key) + return (EINVAL); + + pthread_rwlock_wrlock(&_pthread_key_lock); + + for (i = _pthread_key_sch; i < _pthread_key_max; i++) { + if (!_pthread_key_dest[i]) { + *key = i; + if (dest) { + _pthread_key_dest[i] = dest; + } else { + _pthread_key_dest[i] = (void(*)(void *))1; + } + pthread_rwlock_unlock(&_pthread_key_lock); + + return (0); + } + } + + for (i = 0; i < _pthread_key_sch; i++) { + if (!_pthread_key_dest[i]) { + *key = i; + if (dest) { + _pthread_key_dest[i] = dest; + } else { + _pthread_key_dest[i] = (void(*)(void *))1; + } + pthread_rwlock_unlock(&_pthread_key_lock); + + return (0); + } + } + + if (!_pthread_key_max) _pthread_key_max = 1; + if (_pthread_key_max == PTHREAD_KEYS_MAX) { + pthread_rwlock_unlock(&_pthread_key_lock); + + return (ENOMEM); + } + + nmax = _pthread_key_max * 2; + if (nmax > PTHREAD_KEYS_MAX) nmax = PTHREAD_KEYS_MAX; + + /* No spare room anywhere */ + d = realloc(_pthread_key_dest, nmax * sizeof (*d)); + if (!d) { + pthread_rwlock_unlock(&_pthread_key_lock); + + return (ENOMEM); + } + + /* Clear new region */ + memset((void *) &d[_pthread_key_max], 0, + (nmax-_pthread_key_max)*sizeof (void *)); + + /* Use new region */ + _pthread_key_dest = d; + _pthread_key_sch = _pthread_key_max + 1; + *key = _pthread_key_max; + _pthread_key_max = nmax; + + if (dest) { + _pthread_key_dest[*key] = dest; + } else { + _pthread_key_dest[*key] = (void(*)(void *))1; + } + + pthread_rwlock_unlock(&_pthread_key_lock); + + return (0); +} + +static int pthread_key_delete(pthread_key_t key) +{ + if (key > _pthread_key_max) + return (EINVAL); + if (!_pthread_key_dest) + return (EINVAL); + + pthread_rwlock_wrlock(&_pthread_key_lock); + _pthread_key_dest[key] = NULL; + + /* Start next search from our location */ + if (_pthread_key_sch > key) _pthread_key_sch = key; + + pthread_rwlock_unlock(&_pthread_key_lock); + + return (0); +} + +static void *pthread_getspecific(pthread_key_t key) +{ + pthread_t t = pthread_self(); + + if (key >= t->keymax) + return (NULL); + + return (t->keyval[key]); + +} + +static int pthread_setspecific(pthread_key_t key, const void *value) +{ + pthread_t t = pthread_self(); + + if (key >= t->keymax) { + int keymax = (key + 1) * 2; + void **kv = realloc(t->keyval, keymax * sizeof (void *)); + + if (!kv) + return (ENOMEM); + + /* Clear new region */ + memset(&kv[t->keymax], 0, (keymax - t->keymax)*sizeof (void*)); + + t->keyval = kv; + t->keymax = keymax; + } + + t->keyval[key] = (void *) value; + + return (0); +} + + +static int pthread_spin_init(pthread_spinlock_t *l, int pshared) +{ + (void) pshared; + + *l = 0; + return (0); +} + +static int pthread_spin_destroy(pthread_spinlock_t *l) +{ + (void) l; + return (0); +} + +/* No-fair spinlock due to lack of knowledge of thread number */ +static int pthread_spin_lock(pthread_spinlock_t *l) +{ + while (_InterlockedExchange(l, EBUSY)) { + /* Don't lock the bus whilst waiting */ + while (*l) { + YieldProcessor(); + + /* Compiler barrier. Prevent caching of *l */ + _ReadWriteBarrier(); + } + } + + return (0); +} + +static int pthread_spin_trylock(pthread_spinlock_t *l) +{ + return (_InterlockedExchange(l, EBUSY)); +} + +static int pthread_spin_unlock(pthread_spinlock_t *l) +{ + /* Compiler barrier. The store below acts with release symmantics */ + _ReadWriteBarrier(); + + *l = 0; + + return (0); +} + +static int pthread_cond_init(pthread_cond_t *c, pthread_condattr_t *a) +{ + (void) a; + + InitializeConditionVariable(c); + return (0); +} + +static int pthread_cond_signal(pthread_cond_t *c) +{ + WakeConditionVariable(c); + return (0); +} + +static int pthread_cond_broadcast(pthread_cond_t *c) +{ + WakeAllConditionVariable(c); + return (0); +} + +static int pthread_cond_wait(pthread_cond_t *c, pthread_mutex_t *m) +{ + pthread_testcancel(); + SleepConditionVariableCS(c, m, INFINITE); + return (0); +} + +static int pthread_cond_destroy(pthread_cond_t *c) +{ + (void) c; + return (0); +} + +static int +pthread_cond_timedwait(pthread_cond_t *c, pthread_mutex_t *m, + struct timespec *t) +{ + unsigned long long tm = _pthread_rel_time_in_ms(t); + + pthread_testcancel(); + + if (!SleepConditionVariableCS(c, m, tm)) + return (ETIMEDOUT); + + /* We can have a spurious wakeup after the timeout */ + if (!_pthread_rel_time_in_ms(t)) + return (ETIMEDOUT); + + return (0); +} + +static int pthread_condattr_destroy(pthread_condattr_t *a) +{ + (void) a; + return (0); +} + +#define pthread_condattr_getclock(A, C) ENOTSUP +#define pthread_condattr_setclock(A, C) ENOTSUP + +static int pthread_condattr_init(pthread_condattr_t *a) +{ + *a = 0; + return (0); +} + +static int pthread_condattr_getpshared(pthread_condattr_t *a, int *s) +{ + *s = *a; + return (0); +} + +static int pthread_condattr_setpshared(pthread_condattr_t *a, int s) +{ + *a = s; + return (0); +} + +static int pthread_rwlockattr_destroy(pthread_rwlockattr_t *a) +{ + (void) a; + return (0); +} + +static int pthread_rwlockattr_init(pthread_rwlockattr_t *a) +{ + *a = 0; + return (0); +} + +static int pthread_rwlockattr_getpshared(pthread_rwlockattr_t *a, int *s) +{ + *s = *a; + return (0); +} + +static int pthread_rwlockattr_setpshared(pthread_rwlockattr_t *a, int s) +{ + *a = s; + return (0); +} + + +/* No fork() in windows - so ignore this */ +#define pthread_atfork(F1, F2, F3) 0 + +/* Windows has rudimentary signals support */ +#define pthread_kill(T, S) 0 +#define pthread_sigmask(H, S1, S2) 0 + +#if 0 +/* Wrap cancellation points */ +#define accept(...) (pthread_testcancel(), accept(__VA_ARGS__)) +#define aio_suspend(...) (pthread_testcancel(), aio_suspend(__VA_ARGS__)) +#define clock_nanosleep(...) \ + (pthread_testcancel(), clock_nanosleep(__VA_ARGS__)) +#define close(...) (pthread_testcancel(), close(__VA_ARGS__)) +#define connect(...) (pthread_testcancel(), connect(__VA_ARGS__)) +#define creat(...) (pthread_testcancel(), creat(__VA_ARGS__)) +#define fcntl(...) (pthread_testcancel(), fcntl(__VA_ARGS__)) +#define fdatasync(...) (pthread_testcancel(), fdatasync(__VA_ARGS__)) +#define fsync(...) (pthread_testcancel(), fsync(__VA_ARGS__)) +#define getmsg(...) (pthread_testcancel(), getmsg(__VA_ARGS__)) +#define getpmsg(...) (pthread_testcancel(), getpmsg(__VA_ARGS__)) +#define lockf(...) (pthread_testcancel(), lockf(__VA_ARGS__)) +#define mg_receive(...) (pthread_testcancel(), mg_receive(__VA_ARGS__)) +#define mg_send(...) (pthread_testcancel(), mg_send(__VA_ARGS__)) +#define mg_timedreceive(...) \ + (pthread_testcancel(), mg_timedreceive(__VA_ARGS__)) +#define mg_timessend(...) (pthread_testcancel(), mg_timedsend(__VA_ARGS__)) +#define msgrcv(...) (pthread_testcancel(), msgrecv(__VA_ARGS__)) +#define msgsnd(...) (pthread_testcancel(), msgsnd(__VA_ARGS__)) +#define msync(...) (pthread_testcancel(), msync(__VA_ARGS__)) +#define nanosleep(...) (pthread_testcancel(), nanosleep(__VA_ARGS__)) +#define open(...) (pthread_testcancel(), open(__VA_ARGS__)) +#define pause(...) (pthread_testcancel(), pause(__VA_ARGS__)) +#define poll(...) (pthread_testcancel(), poll(__VA_ARGS__)) +#define pread(...) (pthread_testcancel(), pread(__VA_ARGS__)) +#define pselect(...) (pthread_testcancel(), pselect(__VA_ARGS__)) +#define putmsg(...) (pthread_testcancel(), putmsg(__VA_ARGS__)) +#define putpmsg(...) (pthread_testcancel(), putpmsg(__VA_ARGS__)) +#define pwrite(...) (pthread_testcancel(), pwrite(__VA_ARGS__)) +#define read(...) (pthread_testcancel(), read(__VA_ARGS__)) +#define readv(...) (pthread_testcancel(), readv(__VA_ARGS__)) +#define recv(...) (pthread_testcancel(), recv(__VA_ARGS__)) +#define recvfrom(...) (pthread_testcancel(), recvfrom(__VA_ARGS__)) +#define recvmsg(...) (pthread_testcancel(), recvmsg(__VA_ARGS__)) +#define select(...) (pthread_testcancel(), select(__VA_ARGS__)) +#define sem_timedwait(...) (pthread_testcancel(), sem_timedwait(__VA_ARGS__)) +#define sem_wait(...) (pthread_testcancel(), sem_wait(__VA_ARGS__)) +#define send(...) (pthread_testcancel(), send(__VA_ARGS__)) +#define sendmsg(...) (pthread_testcancel(), sendmsg(__VA_ARGS__)) +#define sendto(...) (pthread_testcancel(), sendto(__VA_ARGS__)) +#define sigpause(...) (pthread_testcancel(), sigpause(__VA_ARGS__)) +#define sigsuspend(...) (pthread_testcancel(), sigsuspend(__VA_ARGS__)) +#define sigwait(...) (pthread_testcancel(), sigwait(__VA_ARGS__)) +#define sigwaitinfo(...) (pthread_testcancel(), sigwaitinfo(__VA_ARGS__)) +#define sleep(...) (pthread_testcancel(), sleep(__VA_ARGS__)) +// #define Sleep(...) (pthread_testcancel(), Sleep(__VA_ARGS__)) +#define system(...) (pthread_testcancel(), system(__VA_ARGS__)) + + +#define access(...) (pthread_testcancel(), access(__VA_ARGS__)) +#define asctime(...) (pthread_testcancel(), asctime(__VA_ARGS__)) +#define asctime_r(...) (pthread_testcancel(), asctime_r(__VA_ARGS__)) +#define catclose(...) (pthread_testcancel(), catclose(__VA_ARGS__)) +#define catgets(...) (pthread_testcancel(), catgets(__VA_ARGS__)) +#define catopen(...) (pthread_testcancel(), catopen(__VA_ARGS__)) +#define closedir(...) (pthread_testcancel(), closedir(__VA_ARGS__)) +#define closelog(...) (pthread_testcancel(), closelog(__VA_ARGS__)) +#define ctermid(...) (pthread_testcancel(), ctermid(__VA_ARGS__)) +#define ctime(...) (pthread_testcancel(), ctime(__VA_ARGS__)) +#define ctime_r(...) (pthread_testcancel(), ctime_r(__VA_ARGS__)) +#define dbm_close(...) (pthread_testcancel(), dbm_close(__VA_ARGS__)) +#define dbm_delete(...) (pthread_testcancel(), dbm_delete(__VA_ARGS__)) +#define dbm_fetch(...) (pthread_testcancel(), dbm_fetch(__VA_ARGS__)) +#define dbm_nextkey(...) (pthread_testcancel(), dbm_nextkey(__VA_ARGS__)) +#define dbm_open(...) (pthread_testcancel(), dbm_open(__VA_ARGS__)) +#define dbm_store(...) (pthread_testcancel(), dbm_store(__VA_ARGS__)) +#define dlclose(...) (pthread_testcancel(), dlclose(__VA_ARGS__)) +#define dlopen(...) (pthread_testcancel(), dlopen(__VA_ARGS__)) +#define endgrent(...) (pthread_testcancel(), endgrent(__VA_ARGS__)) +#define endhostent(...) (pthread_testcancel(), endhostent(__VA_ARGS__)) +#define endnetent(...) (pthread_testcancel(), endnetent(__VA_ARGS__)) +#define endprotoent(...) (pthread_testcancel(), endprotoend(__VA_ARGS__)) +#define endpwent(...) (pthread_testcancel(), endpwent(__VA_ARGS__)) +#define endservent(...) (pthread_testcancel(), endservent(__VA_ARGS__)) +#define endutxent(...) (pthread_testcancel(), endutxent(__VA_ARGS__)) +#define fclose(...) (pthread_testcancel(), fclose(__VA_ARGS__)) +#define fflush(...) (pthread_testcancel(), fflush(__VA_ARGS__)) +#define fgetc(...) (pthread_testcancel(), fgetc(__VA_ARGS__)) +#define fgetpos(...) (pthread_testcancel(), fgetpos(__VA_ARGS__)) +#define fgets(...) (pthread_testcancel(), fgets(__VA_ARGS__)) +#define fgetwc(...) (pthread_testcancel(), fgetwc(__VA_ARGS__)) +#define fgetws(...) (pthread_testcancel(), fgetws(__VA_ARGS__)) +#define fmtmsg(...) (pthread_testcancel(), fmtmsg(__VA_ARGS__)) +#define fopen(...) (pthread_testcancel(), fopen(__VA_ARGS__)) +#define fpathconf(...) (pthread_testcancel(), fpathconf(__VA_ARGS__)) +#define fprintf(...) (pthread_testcancel(), fprintf(__VA_ARGS__)) +#define fputc(...) (pthread_testcancel(), fputc(__VA_ARGS__)) +#define fputs(...) (pthread_testcancel(), fputs(__VA_ARGS__)) +#define fputwc(...) (pthread_testcancel(), fputwc(__VA_ARGS__)) +#define fputws(...) (pthread_testcancel(), fputws(__VA_ARGS__)) +#define fread(...) (pthread_testcancel(), fread(__VA_ARGS__)) +#define freopen(...) (pthread_testcancel(), freopen(__VA_ARGS__)) +#define fscanf(...) (pthread_testcancel(), fscanf(__VA_ARGS__)) +#define fseek(...) (pthread_testcancel(), fseek(__VA_ARGS__)) +#define fseeko(...) (pthread_testcancel(), fseeko(__VA_ARGS__)) +#define fsetpos(...) (pthread_testcancel(), fsetpos(__VA_ARGS__)) +#define fstat(...) (pthread_testcancel(), fstat(__VA_ARGS__)) +#define ftell(...) (pthread_testcancel(), ftell(__VA_ARGS__)) +#define ftello(...) (pthread_testcancel(), ftello(__VA_ARGS__)) +#define ftw(...) (pthread_testcancel(), ftw(__VA_ARGS__)) +#define fwprintf(...) (pthread_testcancel(), fwprintf(__VA_ARGS__)) +#define fwrite(...) (pthread_testcancel(), fwrite(__VA_ARGS__)) +#define fwscanf(...) (pthread_testcancel(), fwscanf(__VA_ARGS__)) +#define getaddrinfo(...) (pthread_testcancel(), getaddrinfo(__VA_ARGS__)) +#define getc(...) (pthread_testcancel(), getc(__VA_ARGS__)) +#define getc_unlocked(...) (pthread_testcancel(), getc_unlocked(__VA_ARGS__)) +#define getchar(...) (pthread_testcancel(), getchar(__VA_ARGS__)) +#define getchar_unlocked(...) \ + (pthread_testcancel(), getchar_unlocked(__VA_ARGS__)) +#define getcwd(...) (pthread_testcancel(), getcwd(__VA_ARGS__)) +#define getdate(...) (pthread_testcancel(), getdate(__VA_ARGS__)) +#define getgrent(...) (pthread_testcancel(), getgrent(__VA_ARGS__)) +#define getgrgid(...) (pthread_testcancel(), getgrgid(__VA_ARGS__)) +#define getgrgid_r(...) (pthread_testcancel(), getgrgid_r(__VA_ARGS__)) +#define gergrnam(...) (pthread_testcancel(), getgrnam(__VA_ARGS__)) +#define getgrnam_r(...) (pthread_testcancel(), getgrnam_r(__VA_ARGS__)) +#define gethostbyaddr(...) (pthread_testcancel(), gethostbyaddr(__VA_ARGS__)) +#define gethostbyname(...) (pthread_testcancel(), gethostbyname(__VA_ARGS__)) +#define gethostent(...) (pthread_testcancel(), gethostent(__VA_ARGS__)) +#define gethostid(...) (pthread_testcancel(), gethostid(__VA_ARGS__)) +#define gethostname(...) (pthread_testcancel(), gethostname(__VA_ARGS__)) +#define getlogin(...) (pthread_testcancel(), getlogin(__VA_ARGS__)) +#define getlogin_r(...) (pthread_testcancel(), getlogin_r(__VA_ARGS__)) +#define getnameinfo(...) (pthread_testcancel(), getnameinfo(__VA_ARGS__)) +#define getnetbyaddr(...) (pthread_testcancel(), getnetbyaddr(__VA_ARGS__)) +#define getnetbyname(...) (pthread_testcancel(), getnetbyname(__VA_ARGS__)) +#define getnetent(...) (pthread_testcancel(), getnetent(__VA_ARGS__)) +#define getopt(...) (pthread_testcancel(), getopt(__VA_ARGS__)) +#define getprotobyname(...) (pthread_testcancel(), getprotobyname(__VA_ARGS__)) +#define getprotobynumber(...) \ + (pthread_testcancel(), getprotobynumber(__VA_ARGS__)) +#define getprotoent(...) (pthread_testcancel(), getprotoent(__VA_ARGS__)) +#define getpwent(...) (pthread_testcancel(), getpwent(__VA_ARGS__)) +#define getpwnam(...) (pthread_testcancel(), getpwnam(__VA_ARGS__)) +#define getpwnam_r(...) (pthread_testcancel(), getpwnam_r(__VA_ARGS__)) +#define getpwuid(...) (pthread_testcancel(), getpwuid(__VA_ARGS__)) +#define getpwuid_r(...) (pthread_testcancel(), getpwuid_r(__VA_ARGS__)) +#define gets(...) (pthread_testcancel(), gets(__VA_ARGS__)) +#define getservbyname(...) (pthread_testcancel(), getservbyname(__VA_ARGS__)) +#define getservbyport(...) (pthread_testcancel(), getservbyport(__VA_ARGS__)) +#define getservent(...) (pthread_testcancel(), getservent(__VA_ARGS__)) +#define getutxent(...) (pthread_testcancel(), getutxent(__VA_ARGS__)) +#define getutxid(...) (pthread_testcancel(), getutxid(__VA_ARGS__)) +#define getutxline(...) (pthread_testcancel(), getutxline(__VA_ARGS__)) +#undef getwc +#define getwc(...) (pthread_testcancel(), getwc(__VA_ARGS__)) +#undef getwchar +#define getwchar(...) (pthread_testcancel(), getwchar(__VA_ARGS__)) +#define getwd(...) (pthread_testcancel(), getwd(__VA_ARGS__)) +#define glob(...) (pthread_testcancel(), glob(__VA_ARGS__)) +#define iconv_close(...) (pthread_testcancel(), iconv_close(__VA_ARGS__)) +#define iconv_open(...) (pthread_testcancel(), iconv_open(__VA_ARGS__)) +#define ioctl(...) (pthread_testcancel(), ioctl(__VA_ARGS__)) +#define link(...) (pthread_testcancel(), link(__VA_ARGS__)) +#define localtime(...) (pthread_testcancel(), localtime(__VA_ARGS__)) +#define localtime_r(...) (pthread_testcancel(), localtime_r(__VA_ARGS__)) +#define lseek(...) (pthread_testcancel(), lseek(__VA_ARGS__)) +#define lstat(...) (pthread_testcancel(), lstat(__VA_ARGS__)) +#define mkstemp(...) (pthread_testcancel(), mkstemp(__VA_ARGS__)) +#define nftw(...) (pthread_testcancel(), nftw(__VA_ARGS__)) +#define opendir(...) (pthread_testcancel(), opendir(__VA_ARGS__)) +#define openlog(...) (pthread_testcancel(), openlog(__VA_ARGS__)) +#define pathconf(...) (pthread_testcancel(), pathconf(__VA_ARGS__)) +#define pclose(...) (pthread_testcancel(), pclose(__VA_ARGS__)) +#define perror(...) (pthread_testcancel(), perror(__VA_ARGS__)) +#define popen(...) (pthread_testcancel(), popen(__VA_ARGS__)) +#define posix_fadvise(...) (pthread_testcancel(), posix_fadvise(__VA_ARGS__)) +#define posix_fallocate(...) \ + (pthread_testcancel(), posix_fallocate(__VA_ARGS__)) +#define posix_madvise(...) (pthread_testcancel(), posix_madvise(__VA_ARGS__)) +#define posix_openpt(...) (pthread_testcancel(), posix_openpt(__VA_ARGS__)) +#define posix_spawn(...) (pthread_testcancel(), posix_spawn(__VA_ARGS__)) +#define posix_spawnp(...) (pthread_testcancel(), posix_spawnp(__VA_ARGS__)) +#define posix_trace_clear(...) \ + (pthread_testcancel(), posix_trace_clear(__VA_ARGS__)) +#define posix_trace_close(...) \ + (pthread_testcancel(), posix_trace_close(__VA_ARGS__)) +#define posix_trace_create(...) \ + (pthread_testcancel(), posix_trace_create(__VA_ARGS__)) +#define posix_trace_create_withlog(...) \ + (pthread_testcancel(), posix_trace_create_withlog(__VA_ARGS__)) +#define posix_trace_eventtypelist_getne(...) \ + (pthread_testcancel(), posix_trace_eventtypelist_getne(__VA_ARGS__)) +#define posix_trace_eventtypelist_rewin(...) \ + (pthread_testcancel(), posix_trace_eventtypelist_rewin(__VA_ARGS__)) +#define posix_trace_flush(...) \ + (pthread_testcancel(), posix_trace_flush(__VA_ARGS__)) +#define posix_trace_get_attr(...) \ + (pthread_testcancel(), posix_trace_get_attr(__VA_ARGS__)) +#define posix_trace_get_filter(...) \ + (pthread_testcancel(), posix_trace_get_filter(__VA_ARGS__)) +#define posix_trace_get_status(...) \ + (pthread_testcancel(), posix_trace_get_status(__VA_ARGS__)) +#define posix_trace_getnext_event(...) \ + (pthread_testcancel(), posix_trace_getnext_event(__VA_ARGS__)) +#define posix_trace_open(...) \ + (pthread_testcancel(), posix_trace_open(__VA_ARGS__)) +#define posix_trace_rewind(...) \ + (pthread_testcancel(), posix_trace_rewind(__VA_ARGS__)) +#define posix_trace_setfilter(...) \ + (pthread_testcancel(), posix_trace_setfilter(__VA_ARGS__)) +#define posix_trace_shutdown(...) \ + (pthread_testcancel(), posix_trace_shutdown(__VA_ARGS__)) +#define posix_trace_timedgetnext_event(...) \ + (pthread_testcancel(), posix_trace_timedgetnext_event(__VA_ARGS__)) +#define posix_typed_mem_open(...) \ + (pthread_testcancel(), posix_typed_mem_open(__VA_ARGS__)) +#define printf(...) (pthread_testcancel(), printf(__VA_ARGS__)) +#define putc(...) (pthread_testcancel(), putc(__VA_ARGS__)) +#define putc_unlocked(...) (pthread_testcancel(), putc_unlocked(__VA_ARGS__)) +#define putchar(...) (pthread_testcancel(), putchar(__VA_ARGS__)) +#define putchar_unlocked(...) \ + (pthread_testcancel(), putchar_unlocked(__VA_ARGS__)) +#define puts(...) (pthread_testcancel(), puts(__VA_ARGS__)) +#define pututxline(...) (pthread_testcancel(), pututxline(__VA_ARGS__)) +#undef putwc +#define putwc(...) (pthread_testcancel(), putwc(__VA_ARGS__)) +#undef putwchar +#define putwchar(...) (pthread_testcancel(), putwchar(__VA_ARGS__)) +#define readdir(...) (pthread_testcancel(), readdir(__VA_ARSG__)) +#define readdir_r(...) (pthread_testcancel(), readdir_r(__VA_ARGS__)) +#define remove(...) (pthread_testcancel(), remove(__VA_ARGS__)) +#define rename(...) (pthread_testcancel(), rename(__VA_ARGS__)) +#define rewind(...) (pthread_testcancel(), rewind(__VA_ARGS__)) +#define rewinddir(...) (pthread_testcancel(), rewinddir(__VA_ARGS__)) +#define scanf(...) (pthread_testcancel(), scanf(__VA_ARGS__)) +#define seekdir(...) (pthread_testcancel(), seekdir(__VA_ARGS__)) +#define semop(...) (pthread_testcancel(), semop(__VA_ARGS__)) +#define setgrent(...) (pthread_testcancel(), setgrent(__VA_ARGS__)) +#define sethostent(...) (pthread_testcancel(), sethostemt(__VA_ARGS__)) +#define setnetent(...) (pthread_testcancel(), setnetent(__VA_ARGS__)) +#define setprotoent(...) (pthread_testcancel(), setprotoent(__VA_ARGS__)) +#define setpwent(...) (pthread_testcancel(), setpwent(__VA_ARGS__)) +#define setservent(...) (pthread_testcancel(), setservent(__VA_ARGS__)) +#define setutxent(...) (pthread_testcancel(), setutxent(__VA_ARGS__)) +#define stat(...) (pthread_testcancel(), stat(__VA_ARGS__)) +#define strerror(...) (pthread_testcancel(), strerror(__VA_ARGS__)) +#define strerror_r(...) (pthread_testcancel(), strerror_r(__VA_ARGS__)) +#define strftime(...) (pthread_testcancel(), strftime(__VA_ARGS__)) +#define symlink(...) (pthread_testcancel(), symlink(__VA_ARGS__)) +#define sync(...) (pthread_testcancel(), sync(__VA_ARGS__)) +#define syslog(...) (pthread_testcancel(), syslog(__VA_ARGS__)) +#define tmpfile(...) (pthread_testcancel(), tmpfile(__VA_ARGS__)) +#define tmpnam(...) (pthread_testcancel(), tmpnam(__VA_ARGS__)) +#define ttyname(...) (pthread_testcancel(), ttyname(__VA_ARGS__)) +#define ttyname_r(...) (pthread_testcancel(), ttyname_r(__VA_ARGS__)) +#define tzset(...) (pthread_testcancel(), tzset(__VA_ARGS__)) +#define ungetc(...) (pthread_testcancel(), ungetc(__VA_ARGS__)) +#define ungetwc(...) (pthread_testcancel(), ungetwc(__VA_ARGS__)) +#define unlink(...) (pthread_testcancel(), unlink(__VA_ARGS__)) +#define vfprintf(...) (pthread_testcancel(), vfprintf(__VA_ARGS__)) +#define vfwprintf(...) (pthread_testcancel(), vfwprintf(__VA_ARGS__)) +#define vprintf(...) (pthread_testcancel(), vprintf(__VA_ARGS__)) +#define vwprintf(...) (pthread_testcancel(), vwprintf(__VA_ARGS__)) +#define wcsftime(...) (pthread_testcancel(), wcsftime(__VA_ARGS__)) +#define wordexp(...) (pthread_testcancel(), wordexp(__VA_ARGS__)) +#define wprintf(...) (pthread_testcancel(), wprintf(__VA_ARGS__)) +#define wscanf(...) (pthread_testcancel(), wscanf(__VA_ARGS__)) +#endif + +#if __clang__ +#pragma GCC diagnostic pop +#endif + + +#endif /* WIN_PTHREADS */ diff --git a/lib/os/windows/libregex/CMakeLists.txt b/lib/os/windows/libregex/CMakeLists.txt new file mode 100644 index 000000000000..4e63d291ee9d --- /dev/null +++ b/lib/os/windows/libregex/CMakeLists.txt @@ -0,0 +1,15 @@ + +use_clang() + +add_library(libregex + regcomp.c + regerror.c + regexec.c + regfree.c + regsub.c +) + +target_compile_definitions(libregex PUBLIC POSIX_MISTAKE) + +target_link_libraries(libregex PUBLIC libspl) + diff --git a/lib/os/windows/libregex/COPYRIGHT b/lib/os/windows/libregex/COPYRIGHT new file mode 100644 index 000000000000..f7a8f20c3b21 --- /dev/null +++ b/lib/os/windows/libregex/COPYRIGHT @@ -0,0 +1,54 @@ +$NetBSD: COPYRIGHT,v 1.5 2003/08/07 16:43:19 agc Exp $ + +Copyright 1992, 1993, 1994 Henry Spencer. All rights reserved. +This software is not subject to any license of the American Telephone +and Telegraph Company or of the Regents of the University of California. + +Permission is granted to anyone to use this software for any purpose on +any computer system, and to alter it and redistribute it, subject +to the following restrictions: + +1. The author is not responsible for the consequences of use of this + software, no matter how awful, even if they arise from flaws in it. + +2. The origin of this software must not be misrepresented, either by + explicit claim or by omission. Since few users ever read sources, + credits must appear in the documentation. + +3. Altered versions must be plainly marked as such, and must not be + misrepresented as being the original software. Since few users + ever read sources, credits must appear in the documentation. + +4. This notice may not be removed or altered. + +=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= +/*- + * Copyright (c) 1994 + * The Regents of the University of California. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)COPYRIGHT 8.1 (Berkeley) 3/16/94 + */ diff --git a/lib/os/windows/libregex/Makefile.inc b/lib/os/windows/libregex/Makefile.inc new file mode 100644 index 000000000000..03b85df3df29 --- /dev/null +++ b/lib/os/windows/libregex/Makefile.inc @@ -0,0 +1,14 @@ +# $NetBSD: Makefile.inc,v 1.9 2016/01/14 21:45:01 christos Exp $ +# @(#)Makefile.inc 8.1 (Berkeley) 6/4/93 + +# regex sources +.PATH: ${.CURDIR}/regex + +CPPFLAGS+=-DPOSIX_MISTAKE + +SRCS+= regcomp.c regerror.c regexec.c regfree.c regsub.c + +MAN+= regex.3 re_format.7 + +MLINKS+=regex.3 regcomp.3 regex.3 regexec.3 regex.3 regerror.3 \ + regex.3 regfree.3 regex.3 regnsub.3 regex.3 regasub.3 diff --git a/lib/os/windows/libregex/WHATSNEW b/lib/os/windows/libregex/WHATSNEW new file mode 100644 index 000000000000..93eb936060be --- /dev/null +++ b/lib/os/windows/libregex/WHATSNEW @@ -0,0 +1,95 @@ +# $NetBSD: WHATSNEW,v 1.6 1995/02/27 13:28:25 cgd Exp $ +# @(#)WHATSNEW 8.3 (Berkeley) 3/18/94 + +New in alpha3.4: The complex bug alluded to below has been fixed (in a +slightly kludgey temporary way that may hurt efficiency a bit; this is +another "get it out the door for 4.4" release). The tests at the end of +the tests file have accordingly been uncommented. The primary sign of +the bug was that something like a?b matching ab matched b rather than ab. +(The bug was essentially specific to this exact situation, else it would +have shown up earlier.) + +New in alpha3.3: The definition of word boundaries has been altered +slightly, to more closely match the usual programming notion that "_" +is an alphabetic. Stuff used for pre-ANSI systems is now in a subdir, +and the makefile no longer alludes to it in mysterious ways. The +makefile has generally been cleaned up some. Fixes have been made +(again!) so that the regression test will run without -DREDEBUG, at +the cost of weaker checking. A workaround for a bug in some folks' + has been added. And some more things have been added to +tests, including a couple right at the end which are commented out +because the code currently flunks them (complex bug; fix coming). +Plus the usual minor cleanup. + +New in alpha3.2: Assorted bits of cleanup and portability improvement +(the development base is now a BSDI system using GCC instead of an ancient +Sun system, and the newer compiler exposed some glitches). Fix for a +serious bug that affected REs using many [] (including REG_ICASE REs +because of the way they are implemented), *sometimes*, depending on +memory-allocation patterns. The header-file prototypes no longer name +the parameters, avoiding possible name conflicts. The possibility that +some clot has defined CHAR_MIN as (say) `-128' instead of `(-128)' is +now handled gracefully. "uchar" is no longer used as an internal type +name (too many people have the same idea). Still the same old lousy +performance, alas. + +New in alpha3.1: Basically nothing, this release is just a bookkeeping +convenience. Stay tuned. + +New in alpha3.0: Performance is no better, alas, but some fixes have been +made and some functionality has been added. (This is basically the "get +it out the door in time for 4.4" release.) One bug fix: regfree() didn't +free the main internal structure (how embarrassing). It is now possible +to put NULs in either the RE or the target string, using (resp.) a new +REG_PEND flag and the old REG_STARTEND flag. The REG_NOSPEC flag to +regcomp() makes all characters ordinary, so you can match a literal +string easily (this will become more useful when performance improves!). +There are now primitives to match beginnings and ends of words, although +the syntax is disgusting and so is the implementation. The REG_ATOI +debugging interface has changed a bit. And there has been considerable +internal cleanup of various kinds. + +New in alpha2.3: Split change list out of README, and moved flags notes +into Makefile. Macro-ized the name of regex(7) in regex(3), since it has +to change for 4.4BSD. Cleanup work in engine.c, and some new regression +tests to catch tricky cases thereof. + +New in alpha2.2: Out-of-date manpages updated. Regerror() acquires two +small extensions -- REG_ITOA and REG_ATOI -- which avoid debugging kludges +in my own test program and might be useful to others for similar purposes. +The regression test will now compile (and run) without REDEBUG. The +BRE \$ bug is fixed. Most uses of "uchar" are gone; it's all chars now. +Char/uchar parameters are now written int/unsigned, to avoid possible +portability problems with unpromoted parameters. Some unsigned casts have +been introduced to minimize portability problems with shifting into sign +bits. + +New in alpha2.1: Lots of little stuff, cleanup and fixes. The one big +thing is that regex.h is now generated, using mkh, rather than being +supplied in the distribution; due to circularities in dependencies, +you have to build regex.h explicitly by "make h". The two known bugs +have been fixed (and the regression test now checks for them), as has a +problem with assertions not being suppressed in the absence of REDEBUG. +No performance work yet. + +New in alpha2: Backslash-anything is an ordinary character, not an +error (except, of course, for the handful of backslashed metacharacters +in BREs), which should reduce script breakage. The regression test +checks *where* null strings are supposed to match, and has generally +been tightened up somewhat. Small bug fixes in parameter passing (not +harmful, but technically errors) and some other areas. Debugging +invoked by defining REDEBUG rather than not defining NDEBUG. + +New in alpha+3: full prototyping for internal routines, using a little +helper program, mkh, which extracts prototypes given in stylized comments. +More minor cleanup. Buglet fix: it's CHAR_BIT, not CHAR_BITS. Simple +pre-screening of input when a literal string is known to be part of the +RE; this does wonders for performance. + +New in alpha+2: minor bits of cleanup. Notably, the number "32" for the +word width isn't hardwired into regexec.c any more, the public header +file prototypes the functions if __STDC__ is defined, and some small typos +in the manpages have been fixed. + +New in alpha+1: improvements to the manual pages, and an important +extension, the REG_STARTEND option to regexec(). diff --git a/lib/os/windows/libregex/cname.h b/lib/os/windows/libregex/cname.h new file mode 100644 index 000000000000..e5447ded1b66 --- /dev/null +++ b/lib/os/windows/libregex/cname.h @@ -0,0 +1,142 @@ +/* $NetBSD: cname.h,v 1.8 2021/02/23 22:14:59 christos Exp $ */ + +/* + * SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 1992, 1993, 1994 Henry Spencer. + * Copyright (c) 1992, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)cname.h 8.3 (Berkeley) 3/20/94 + * $FreeBSD: head/lib/libc/regex/cname.h 326025 2017-11-20 19:49:47Z pfg $ + */ + +/* character-name table */ +static struct cname { + const char *name; + char code; +} cnames[] = { + {"NUL", '\0'}, + {"SOH", '\001'}, + {"STX", '\002'}, + {"ETX", '\003'}, + {"EOT", '\004'}, + {"ENQ", '\005'}, + {"ACK", '\006'}, + {"BEL", '\007'}, + {"alert", '\007'}, + {"BS", '\010'}, + {"backspace", '\b'}, + {"HT", '\011'}, + {"tab", '\t'}, + {"LF", '\012'}, + {"newline", '\n'}, + {"VT", '\013'}, + {"vertical-tab", '\v'}, + {"FF", '\014'}, + {"form-feed", '\f'}, + {"CR", '\015'}, + {"carriage-return", '\r'}, + {"SO", '\016'}, + {"SI", '\017'}, + {"DLE", '\020'}, + {"DC1", '\021'}, + {"DC2", '\022'}, + {"DC3", '\023'}, + {"DC4", '\024'}, + {"NAK", '\025'}, + {"SYN", '\026'}, + {"ETB", '\027'}, + {"CAN", '\030'}, + {"EM", '\031'}, + {"SUB", '\032'}, + {"ESC", '\033'}, + {"IS4", '\034'}, + {"FS", '\034'}, + {"IS3", '\035'}, + {"GS", '\035'}, + {"IS2", '\036'}, + {"RS", '\036'}, + {"IS1", '\037'}, + {"US", '\037'}, + {"space", ' '}, + {"exclamation-mark", '!'}, + {"quotation-mark", '"'}, + {"number-sign", '#'}, + {"dollar-sign", '$'}, + {"percent-sign", '%'}, + {"ampersand", '&'}, + {"apostrophe", '\''}, + {"left-parenthesis", '('}, + {"right-parenthesis", ')'}, + {"asterisk", '*'}, + {"plus-sign", '+'}, + {"comma", ','}, + {"hyphen", '-'}, + {"hyphen-minus", '-'}, + {"period", '.'}, + {"full-stop", '.'}, + {"slash", '/'}, + {"solidus", '/'}, + {"zero", '0'}, + {"one", '1'}, + {"two", '2'}, + {"three", '3'}, + {"four", '4'}, + {"five", '5'}, + {"six", '6'}, + {"seven", '7'}, + {"eight", '8'}, + {"nine", '9'}, + {"colon", ':'}, + {"semicolon", ';'}, + {"less-than-sign", '<'}, + {"equals-sign", '='}, + {"greater-than-sign", '>'}, + {"question-mark", '?'}, + {"commercial-at", '@'}, + {"left-square-bracket", '['}, + {"backslash", '\\'}, + {"reverse-solidus", '\\'}, + {"right-square-bracket", ']'}, + {"circumflex", '^'}, + {"circumflex-accent", '^'}, + {"underscore", '_'}, + {"low-line", '_'}, + {"grave-accent", '`'}, + {"left-brace", '{'}, + {"left-curly-bracket", '{'}, + {"vertical-line", '|'}, + {"right-brace", '}'}, + {"right-curly-bracket", '}'}, + {"tilde", '~'}, + {"DEL", '\177'}, + {NULL, 0} +}; diff --git a/lib/os/windows/libregex/engine.c b/lib/os/windows/libregex/engine.c new file mode 100644 index 000000000000..1126c7684093 --- /dev/null +++ b/lib/os/windows/libregex/engine.c @@ -0,0 +1,1278 @@ +/* $NetBSD: engine.c,v 1.29 2021/02/25 21:47:46 christos Exp $ */ + +/* + * SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 1992, 1993, 1994 Henry Spencer. + * Copyright (c) 1992, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)engine.c 8.5 (Berkeley) 3/20/94 + */ + +#include + +/* + * The matching engine and friends. This file is #included by regexec.c + * after suitable #defines of a variety of macros used herein, so that + * different state representations can be used without duplicating masses + * of code. + */ + +#ifdef SNAMES +#define stepback sstepback +#define matcher smatcher +#define walk swalk +#define dissect sdissect +#define backref sbackref +#define step sstep +#define print sprint +#define at sat +#define match smat +#endif +#ifdef LNAMES +#define stepback lstepback +#define matcher lmatcher +#define walk lwalk +#define dissect ldissect +#define backref lbackref +#define step lstep +#define print lprint +#define at lat +#define match lmat +#endif +#ifdef MNAMES +#define stepback mstepback +#define matcher mmatcher +#define walk mwalk +#define dissect mdissect +#define backref mbackref +#define step mstep +#define print mprint +#define at mat +#define match mmat +#endif + +/* another structure passed up and down to avoid zillions of parameters */ +struct match { + struct re_guts *g; + int eflags; + regmatch_t *pmatch; /* [nsub+1] (0 element unused) */ + const char *offp; /* offsets work from here */ + const char *beginp; /* start of string -- virtual NUL precedes */ + const char *endp; /* end of string -- virtual NUL here */ + const char *coldp; /* can be no match starting before here */ + const char **lastpos; /* [nplus+1] */ + STATEVARS; + states st; /* current states */ + states fresh; /* states for a fresh start */ + states tmp; /* temporary */ + states empty; /* empty set of states */ + mbstate_t mbs; /* multibyte conversion state */ +}; + +/* ========= begin header generated by ./mkh ========= */ +#ifdef __cplusplus +extern "C" { +#endif + +/* === engine.c === */ +static int matcher(struct re_guts *g, const char *string, size_t nmatch, + regmatch_t pmatch[], int eflags); +static const char *dissect(struct match *m, const char *start, const char *stop, + sopno startst, sopno stopst); +static const char *backref(struct match *m, const char *start, const char *stop, + sopno startst, sopno stopst, sopno lev, int); +static const char *walk(struct match *m, const char *start, const char *stop, + sopno startst, sopno stopst, bool fast); +static states step(struct re_guts *g, sopno start, sopno stop, states bef, + wint_t ch, states aft, int sflags); +#define MAX_RECURSION 100 +#define BOL (OUT-1) +#define EOL (BOL-1) +#define BOLEOL (BOL-2) +#define NOTHING (BOL-3) +#define BOW (BOL-4) +#define EOW (BOL-5) +#define BADCHAR (BOL-6) +#define NWBND (BOL-7) +#define NONCHAR(c) ((c) <= OUT) +/* sflags */ +#define SBOS 0x0001 +#define SEOS 0x0002 + +#ifdef REDEBUG +static void print(struct match *m, const char *caption, states st, + int ch, FILE *d); +#endif +#ifdef REDEBUG +static void at(struct match *m, const char *title, const char *start, + const char *stop, sopno startst, sopno stopst); +#endif +#ifdef REDEBUG +static const char *pchar(int ch); +#endif + +#ifdef __cplusplus +} +#endif +/* ========= end header generated by ./mkh ========= */ + +#ifdef REDEBUG +#define SP(t, s, c) print(m, t, s, c, stdout) +#define AT(t, p1, p2, s1, s2) at(m, t, p1, p2, s1, s2) +#define NOTE(str) { if (m->eflags®_TRACE) printf("=%s\n", (str)); } +#else +#define SP(t, s, c) /* nothing */ +#define AT(t, p1, p2, s1, s2) /* nothing */ +#define NOTE(s) /* nothing */ +#endif + +/* + * Given a multibyte string pointed to by start, step back nchar characters + * from current position pointed to by cur. + */ +static const char * +stepback(const char *start, const char *cur, int nchar) +{ +#ifdef NLS + const char *ret; + size_t wc, mbc; + mbstate_t mbs; + size_t clen; + + if (MB_CUR_MAX == 1) + goto out; + + ret = cur; + for (wc = nchar; wc > 0; wc--) { + for (mbc = 1; mbc <= MB_CUR_MAX; mbc++) { + if ((ret - mbc) < start) + return (NULL); + memset(&mbs, 0, sizeof (mbs)); + clen = mbrtowc(NULL, ret - mbc, mbc, &mbs); + if (clen != (size_t)-1 && clen != (size_t)-2) + break; + } + if (mbc > MB_CUR_MAX) + return (NULL); + ret -= mbc; + } + + return (ret); +out: +#endif + return ((cur - nchar) > start ? cur - nchar : NULL); +} + +/* + * matcher - the actual matching engine + * static int matcher(struct re_guts *g, const char *string, \ + * size_t nmatch, regmatch_t pmatch[], int eflags); + */ +static int /* 0 success, REG_NOMATCH failure */ +matcher(struct re_guts *g, + const char *string, + size_t nmatch, + regmatch_t pmatch[], + int eflags) +{ + const char *endp; + size_t i; + struct match mv; + struct match *m = &mv; + const char *dp = NULL; + const sopno gf = g->firststate+1; /* +1 for OEND */ + const sopno gl = g->laststate; + const char *start; + const char *stop; + /* Boyer-Moore algorithms variables */ + const char *pp; + size_t cj, mj; + const char *mustfirst; + const char *mustlast; + size_t *matchjump; + size_t *charjump; + int error = 0; + + _DIAGASSERT(g != NULL); + _DIAGASSERT(string != NULL); + /* pmatch checked below */ + + /* simplify the situation where possible */ + if (g->cflags®_NOSUB) + nmatch = 0; + if (eflags®_STARTEND) { + _DIAGASSERT(pmatch != NULL); + start = string + (size_t)pmatch[0].rm_so; + stop = string + (size_t)pmatch[0].rm_eo; + } else { + start = string; + stop = start + strlen(start); + } + if (stop < start) + return (REG_INVARG); + + /* prescreening; this does wonders for this rather slow code */ + if (g->must != NULL) { + if (g->charjump != NULL && g->matchjump != NULL) { + mustfirst = g->must; + mustlast = g->must + g->mlen - 1; + charjump = g->charjump; + matchjump = g->matchjump; + pp = mustlast; + for (dp = start+g->mlen-1; dp < stop; ) { + /* Fast skip non-matches */ + while (dp < stop && charjump[(int)*dp]) + dp += charjump[(int)*dp]; + + if (dp >= stop) + break; + + /* Greedy matcher */ + /* + * We depend on not being used for + * for strings of length 1 + */ + while (*--dp == *--pp && pp != mustfirst) + ; + + if (*dp == *pp) + break; + + /* Jump to next possible match */ + mj = matchjump[pp - mustfirst]; + cj = charjump[(int)*dp]; + dp += (cj < mj ? mj : cj); + pp = mustlast; + } + if (pp != mustfirst) + return (REG_NOMATCH); + } else { + for (dp = start; dp < stop; dp++) + if (*dp == g->must[0] && + (size_t)(stop - dp) >= g->mlen && + memcmp(dp, g->must, (size_t)g->mlen) == 0) + break; + if (dp == stop) /* we didn't find g->must */ + return (REG_NOMATCH); + } + } + + /* match struct setup */ + m->g = g; + m->eflags = eflags; + m->pmatch = NULL; + m->lastpos = NULL; + m->offp = string; + m->beginp = start; + m->endp = stop; + STATESETUP(m, 4); + SETUP(m->st); + SETUP(m->fresh); + SETUP(m->tmp); + SETUP(m->empty); + CLEAR(m->empty); + ZAPSTATE(&m->mbs); + + /* Adjust start according to moffset, to speed things up */ + if (dp != NULL && g->moffset > -1) { + const char *nstart; + + nstart = stepback(start, dp, g->moffset); + if (nstart != NULL) + start = nstart; + } + + SP("mloop", m->st, *start); + + /* this loop does only one repetition except for backrefs */ + for (;;) { + endp = walk(m, start, stop, gf, gl, true); + if (endp == NULL) { /* a miss */ + error = REG_NOMATCH; + goto done; + } + if (nmatch == 0 && !g->backrefs) + break; /* no further info needed */ + + /* where? */ + assert(m->coldp != NULL); + for (;;) { + NOTE("finding start"); + endp = walk(m, m->coldp, stop, gf, gl, false); + if (endp != NULL) + break; + assert(m->coldp < m->endp); + m->coldp += XMBRTOWC(NULL, m->coldp, + (size_t)(m->endp - m->coldp), &m->mbs, 0); + } + if (nmatch == 1 && !g->backrefs) + break; /* no further info needed */ + + /* oh my, he wants the subexpressions... */ + if (m->pmatch == NULL) + m->pmatch = (regmatch_t *)malloc((m->g->nsub + 1) * + sizeof (regmatch_t)); + if (m->pmatch == NULL) { + error = REG_ESPACE; + goto done; + } + for (i = 1; i <= m->g->nsub; i++) + m->pmatch[i].rm_so = m->pmatch[i].rm_eo = -1; + if (!g->backrefs && !(m->eflags®_BACKR)) { + NOTE("dissecting"); + dp = dissect(m, m->coldp, endp, gf, gl); + } else { + if (g->nplus > 0 && m->lastpos == NULL) + m->lastpos = malloc((g->nplus+1) * + sizeof (const char *)); + if (g->nplus > 0 && m->lastpos == NULL) { + error = REG_ESPACE; + goto done; + } + NOTE("backref dissect"); + dp = backref(m, m->coldp, endp, gf, gl, (sopno)0, 0); + } + if (dp != NULL) + break; + + /* uh-oh... we couldn't find a subexpression-level match */ + assert(g->backrefs); /* must be back references doing it */ + assert(g->nplus == 0 || m->lastpos != NULL); + for (;;) { + if (dp != NULL || endp <= m->coldp) + break; /* defeat */ + NOTE("backoff"); + endp = walk(m, m->coldp, endp-1, gf, gl, false); + if (endp == NULL) + break; /* defeat */ + /* try it on a shorter possibility */ +#ifndef NDEBUG + for (i = 1; i <= m->g->nsub; i++) { + assert(m->pmatch[i].rm_so == (regoff_t)-1); + assert(m->pmatch[i].rm_eo == (regoff_t)-1); + } +#endif + NOTE("backoff dissect"); + dp = backref(m, m->coldp, endp, gf, gl, (sopno)0, 0); + } + assert(dp == NULL || dp == endp); + if (dp != NULL) /* found a shorter one */ + break; + + /* despite initial appearances, there is no match here */ + NOTE("false alarm"); + /* recycle starting later */ + start = m->coldp + XMBRTOWC(NULL, m->coldp, + (size_t)(stop - m->coldp), &m->mbs, 0); + assert(start <= stop); + } + + /* fill in the details if requested */ + if (nmatch > 0) { + _DIAGASSERT(pmatch != NULL); + pmatch[0].rm_so = m->coldp - m->offp; + pmatch[0].rm_eo = endp - m->offp; + } + if (nmatch > 1) { + assert(m->pmatch != NULL); + for (i = 1; i < nmatch; i++) + if (i <= m->g->nsub) + pmatch[i] = m->pmatch[i]; + else { + pmatch[i].rm_so = (regoff_t)-1; + pmatch[i].rm_eo = (regoff_t)-1; + } + } + +done: + if (m->pmatch != NULL) { + free(m->pmatch); + m->pmatch = NULL; + } + if (m->lastpos != NULL) { + free(__UNCONST(m->lastpos)); + m->lastpos = NULL; + } + STATETEARDOWN(m); + return (error); +} + +/* + * dissect - figure out what matched what, no back references + * static const char *dissect(struct match *m, const char *start, \ + * const char *stop, sopno startst, sopno stopst); + */ +/* == stop (success) always */ +static const char * +dissect( + struct match *m, + const char *start, + const char *stop, + sopno startst, + sopno stopst) +{ + int i; + sopno ss; /* start sop of current subRE */ + sopno es; /* end sop of current subRE */ + const char *sp; /* start of string matched by it */ + const char *stp; /* string matched by it cannot pass here */ + const char *rest; /* start of rest of string */ + const char *tail; /* string unmatched by rest of RE */ + sopno ssub; /* start sop of subsubRE */ + sopno esub; /* end sop of subsubRE */ + const char *ssp; /* start of string matched by subsubRE */ + const char *sep; /* end of string matched by subsubRE */ + const char *oldssp; /* previous ssp */ + const char *dp __maybe_unused; + + _DIAGASSERT(m != NULL); + _DIAGASSERT(start != NULL); + _DIAGASSERT(stop != NULL); + + AT("diss", start, stop, startst, stopst); + sp = start; + for (ss = startst; ss < stopst; ss = es) { + /* identify end of subRE */ + es = ss; + switch (OP(m->g->strip[es])) { + case OPLUS_: + case OQUEST_: + es += OPND(m->g->strip[es]); + break; + case OCH_: + while (OP(m->g->strip[es]) != O_CH) + es += OPND(m->g->strip[es]); + break; + } + es++; + + /* figure out what it matched */ + switch (OP(m->g->strip[ss])) { + case OEND: + assert(nope); + break; + case OCHAR: + sp += XMBRTOWC(NULL, sp, (size_t)(stop - start), + &m->mbs, 0); + break; + case OBOL: + case OEOL: + case OBOW: + case OEOW: + case OBOS: + case OEOS: + case OWBND: + case ONWBND: + break; + case OANY: + case OANYOF: + sp += XMBRTOWC(NULL, sp, (size_t)(stop - start), + &m->mbs, 0); + break; + case OBACK_: + case O_BACK: + assert(nope); + break; + /* cases where length of match is hard to find */ + case OQUEST_: + stp = stop; + for (;;) { + /* how long could this one be? */ + rest = walk(m, sp, stp, ss, es, false); + assert(rest != NULL); /* it did match */ + /* could the rest match the rest? */ + tail = walk(m, rest, stop, es, stopst, false); + if (tail == stop) + break; /* yes! */ + /* no -- try a shorter match for this one */ + stp = rest - 1; + assert(stp >= sp); /* it did work */ + } + ssub = ss + 1; + esub = es - 1; + /* did innards match? */ + if (walk(m, sp, rest, ssub, esub, false) != NULL) { + dp = dissect(m, sp, rest, ssub, esub); + assert(dp == rest); + } else /* no */ + assert(sp == rest); + sp = rest; + break; + case OPLUS_: + stp = stop; + for (;;) { + /* how long could this one be? */ + rest = walk(m, sp, stp, ss, es, false); + assert(rest != NULL); /* it did match */ + /* could the rest match the rest? */ + tail = walk(m, rest, stop, es, stopst, false); + if (tail == stop) + break; /* yes! */ + /* no -- try a shorter match for this one */ + stp = rest - 1; + assert(stp >= sp); /* it did work */ + } + ssub = ss + 1; + esub = es - 1; + ssp = sp; + oldssp = ssp; + for (;;) { /* find last match of innards */ + sep = walk(m, ssp, rest, ssub, esub, false); + if (sep == NULL || sep == ssp) + break; /* failed or matched null */ + oldssp = ssp; /* on to next try */ + ssp = sep; + } + if (sep == NULL) { + /* last successful match */ + sep = ssp; + ssp = oldssp; + } + assert(sep == rest); /* must exhaust substring */ + assert(walk(m, ssp, sep, ssub, esub, false) == rest); + dp = dissect(m, ssp, sep, ssub, esub); + assert(dp == sep); + sp = rest; + break; + case OCH_: + stp = stop; + for (;;) { + /* how long could this one be? */ + rest = walk(m, sp, stp, ss, es, false); + assert(rest != NULL); /* it did match */ + /* could the rest match the rest? */ + tail = walk(m, rest, stop, es, stopst, false); + if (tail == stop) + break; /* yes! */ + /* no -- try a shorter match for this one */ + stp = rest - 1; + assert(stp >= sp); /* it did work */ + } + ssub = ss + 1; + esub = ss + OPND(m->g->strip[ss]) - 1; + assert(OP(m->g->strip[esub]) == OOR1); + for (;;) { /* find first matching branch */ + if (walk(m, sp, rest, ssub, esub, false) == + rest) + break; /* it matched all of it */ + /* that one missed, try next one */ + assert(OP(m->g->strip[esub]) == OOR1); + esub++; + assert(OP(m->g->strip[esub]) == OOR2); + ssub = esub + 1; + esub += OPND(m->g->strip[esub]); + if (OP(m->g->strip[esub]) == OOR2) + esub--; + else + assert(OP(m->g->strip[esub]) == O_CH); + } + dp = dissect(m, sp, rest, ssub, esub); + assert(dp == rest); + sp = rest; + break; + case O_PLUS: + case O_QUEST: + case OOR1: + case OOR2: + case O_CH: + assert(nope); + break; + case OLPAREN: + i = OPND(m->g->strip[ss]); + assert(0 < i && i <= m->g->nsub); + m->pmatch[i].rm_so = sp - m->offp; + break; + case ORPAREN: + i = OPND(m->g->strip[ss]); + assert(0 < i && i <= m->g->nsub); + m->pmatch[i].rm_eo = sp - m->offp; + break; + default: /* uh oh */ + assert(nope); + break; + } + } + + assert(sp == stop); + return (sp); +} + +#define ISBOW(m, sp) \ + (sp < m->endp && ISWORD(*sp) && \ + ((sp == m->beginp && !(m->eflags®_NOTBOL)) || \ + (sp > m->offp && !ISWORD(*(sp-1))))) +#define ISEOW(m, sp) \ + (((sp == m->endp && !(m->eflags®_NOTEOL)) || \ + (sp < m->endp && *sp == '\n' && \ + (m->g->cflags & REG_NEWLINE)) || \ + (sp < m->endp && !ISWORD(*sp))) && \ + (sp > m->beginp && ISWORD(*(sp-1)))) \ + +/* + * backref - figure out what matched what, figuring in back references + * static const char *backref(struct match *m, const char *start, \ + * const char *stop, sopno startst, sopno stopst, sopno lev); + */ +/* == stop (success) or NULL (failure) */ +static const char * +backref( + struct match *m, + const char *start, + const char *stop, + sopno startst, + sopno stopst, + sopno lev, /* PLUS nesting level */ + int rec) +{ + int i; + sopno ss; /* start sop of current subRE */ + const char *sp; /* start of string matched by it */ + sopno ssub; /* start sop of subsubRE */ + sopno esub; /* end sop of subsubRE */ + const char *ssp; /* start of string matched by subsubRE */ + const char *dp; + size_t len; + int hard; + sop s; + regoff_t offsave; + cset *cs; + wint_t wc; + + _DIAGASSERT(m != NULL); + _DIAGASSERT(start != NULL); + _DIAGASSERT(stop != NULL); + + AT("back", start, stop, startst, stopst); + sp = start; + + /* get as far as we can with easy stuff */ + hard = 0; + for (ss = startst; !hard && ss < stopst; ss++) + switch (OP(s = m->g->strip[ss])) { + case OCHAR: + if (sp == stop) + return (NULL); + sp += XMBRTOWC(&wc, sp, (size_t)(stop - sp), + &m->mbs, BADCHAR); + if (wc != (wint_t)OPND(s)) + return (NULL); + break; + case OANY: + if (sp == stop) + return (NULL); + sp += XMBRTOWC(&wc, sp, (size_t)(stop - sp), + &m->mbs, BADCHAR); + if (wc == BADCHAR) + return (NULL); + break; + case OANYOF: + if (sp == stop) + return (NULL); + cs = &m->g->sets[OPND(s)]; + sp += XMBRTOWC(&wc, sp, (size_t)(stop - sp), + &m->mbs, BADCHAR); + if (wc == BADCHAR || !CHIN(cs, wc)) + return (NULL); + break; + case OBOS: + if (sp == m->beginp && (m->eflags & REG_NOTBOL) == 0) { + /* yes */ + } else + return (NULL); + break; + case OEOS: + if (sp == m->endp && (m->eflags & REG_NOTEOL) == 0) { + /* yes */ + } else + return (NULL); + break; + case OBOL: + if ((sp == m->beginp && !(m->eflags®_NOTBOL)) || + (sp > m->offp && sp < m->endp && + *(sp-1) == '\n' && (m->g->cflags®_NEWLINE))) { + /* yes */ + } else + return (NULL); + break; + case OEOL: + if ((sp == m->endp && !(m->eflags®_NOTEOL)) || + (sp < m->endp && *sp == '\n' && + (m->g->cflags®_NEWLINE))) { + /* yes */ + } else + return (NULL); + break; + case OWBND: + if (ISBOW(m, sp) || ISEOW(m, sp)) { + /* yes */ + } else + return (NULL); + break; + case ONWBND: + if (((sp == m->beginp) && !ISWORD(*sp)) || + (sp == m->endp && !ISWORD(*(sp - 1)))) { + /* yes, beginning/end of subject */ + } else if (ISWORD(*(sp - 1)) == ISWORD(*sp)) { + /* yes, beginning/end of subject */ + } else + return (NULL); + break; + case OBOW: + if (ISBOW(m, sp)) { + /* yes */ + } else + return (NULL); + break; + case OEOW: + if (ISEOW(m, sp)) { + /* yes */ + } else + return (NULL); + break; + case O_QUEST: + break; + case OOR1: /* matches null but needs to skip */ + ss++; + s = m->g->strip[ss]; + do { + assert(OP(s) == OOR2); + ss += OPND(s); + } while (OP(s = m->g->strip[ss]) != O_CH); + /* note that the ss++ gets us past the O_CH */ + break; + default: /* have to make a choice */ + hard = 1; + break; + } + if (!hard) { /* that was it! */ + if (sp != stop) + return (NULL); + return (sp); + } + ss--; /* adjust for the for's final increment */ + + /* the hard stuff */ + AT("hard", sp, stop, ss, stopst); + s = m->g->strip[ss]; + switch (OP(s)) { + case OBACK_: /* the vilest depths */ + i = OPND(s); + assert(0 < i && i <= m->g->nsub); + if (m->pmatch[i].rm_eo == -1) + return (NULL); + assert(m->pmatch[i].rm_so != -1); + len = m->pmatch[i].rm_eo - m->pmatch[i].rm_so; + if (len == 0 && rec++ > MAX_RECURSION) + return (NULL); + assert(stop - m->beginp >= len); + if (sp > stop - len) + return (NULL); /* not enough left to match */ + ssp = m->offp + m->pmatch[i].rm_so; + if (memcmp(sp, ssp, len) != 0) + return (NULL); + while (m->g->strip[ss] != SOP(O_BACK, i)) + ss++; + return (backref(m, sp+len, stop, ss+1, stopst, lev, rec)); + case OQUEST_: /* to null or not */ + dp = backref(m, sp, stop, ss+1, stopst, lev, rec); + if (dp != NULL) + return (dp); /* not */ + return (backref(m, sp, stop, ss+OPND(s)+1, stopst, lev, rec)); + case OPLUS_: + assert(m->lastpos != NULL); + assert(lev+1 <= m->g->nplus); + m->lastpos[lev+1] = sp; + return (backref(m, sp, stop, ss+1, stopst, lev+1, rec)); + case O_PLUS: + if (sp == m->lastpos[lev]) /* last pass matched null */ + return (backref(m, sp, stop, ss+1, stopst, lev-1, rec)); + /* try another pass */ + m->lastpos[lev] = sp; + dp = backref(m, sp, stop, ss-OPND(s)+1, stopst, lev, rec); + if (dp == NULL) + return (backref(m, sp, stop, ss+1, stopst, lev-1, rec)); + else + return (dp); + case OCH_: /* find the right one, if any */ + ssub = ss + 1; + esub = ss + OPND(s) - 1; + assert(OP(m->g->strip[esub]) == OOR1); + for (;;) { /* find first matching branch */ + dp = backref(m, sp, stop, ssub, esub, lev, rec); + if (dp != NULL) + return (dp); + /* that one missed, try next one */ + if (OP(m->g->strip[esub]) == O_CH) + return (NULL); /* there is none */ + esub++; + assert(OP(m->g->strip[esub]) == OOR2); + ssub = esub + 1; + esub += OPND(m->g->strip[esub]); + if (OP(m->g->strip[esub]) == OOR2) + esub--; + else + assert(OP(m->g->strip[esub]) == O_CH); + } + /* NOTREACHED */ + break; + case OLPAREN: /* must undo assignment if rest fails */ + i = OPND(s); + assert(0 < i && i <= m->g->nsub); + offsave = m->pmatch[i].rm_so; + m->pmatch[i].rm_so = sp - m->offp; + dp = backref(m, sp, stop, ss+1, stopst, lev, rec); + if (dp != NULL) + return (dp); + m->pmatch[i].rm_so = offsave; + return (NULL); + case ORPAREN: /* must undo assignment if rest fails */ + i = OPND(s); + assert(0 < i && i <= m->g->nsub); + offsave = m->pmatch[i].rm_eo; + m->pmatch[i].rm_eo = sp - m->offp; + dp = backref(m, sp, stop, ss+1, stopst, lev, rec); + if (dp != NULL) + return (dp); + m->pmatch[i].rm_eo = offsave; + return (NULL); + default: /* uh oh */ + assert(nope); + break; + } + + /* "can't happen" */ + assert(nope); + /* NOTREACHED */ + return ("shut up gcc"); +} + +/* + * walk - step through the string either quickly or slowly + * static const char *walk(struct match *m, const char *start, \ + * const char *stop, sopno startst, sopno stopst, bool fast); + */ +/* where it ended, or NULL */ +static const char * +walk(struct match *m, const char *start, const char *stop, sopno startst, + sopno stopst, bool fast) +{ + states st = m->st; + states fresh = m->fresh; + states empty = m->empty; + states tmp = m->tmp; + const char *p = start; + wint_t c; + wint_t lastc; /* previous c */ + wint_t flagch; + int sflags; + const char *matchp; /* last p at which a match ended */ + size_t i, clen; + + _DIAGASSERT(m != NULL); + _DIAGASSERT(start != NULL); + _DIAGASSERT(stop != NULL); + + sflags = 0; + AT("walk", start, stop, startst, stopst); + CLEAR(st); + SET1(st, startst); + SP("sstart", st, *p); + st = step(m->g, startst, stopst, st, NOTHING, st, sflags); + if (fast) + ASSIGN(fresh, st); + matchp = NULL; + if (start == m->offp || (start == m->beginp && !(m->eflags®_NOTBOL))) + c = OUT; + else { + /* + * XXX Wrong if the previous character was multi-byte. + * Newline never is (in encodings supported by FreeBSD), + * so this only breaks the ISWORD tests below. + */ + c = (uch)*(start - 1); + } + for (;;) { + /* next character */ + lastc = c; + sflags = 0; + if (p == m->endp) { + c = OUT; + clen = 0; + } else + clen = XMBRTOWC(&c, p, (size_t)(m->endp - p), + &m->mbs, BADCHAR); + + if (fast && EQ(st, fresh)) + matchp = p; + + /* is there an EOL and/or BOL between lastc and c? */ + flagch = '\0'; + i = 0; + if ((lastc == '\n' && m->g->cflags®_NEWLINE) || + (lastc == OUT && !(m->eflags®_NOTBOL))) { + flagch = BOL; + i = m->g->nbol; + } + if ((c == '\n' && m->g->cflags®_NEWLINE) || + (c == OUT && !(m->eflags®_NOTEOL))) { + flagch = (flagch == BOL) ? BOLEOL : EOL; + i += m->g->neol; + } + if (lastc == OUT && (m->eflags & REG_NOTBOL) == 0) { + sflags |= SBOS; + /* Step one more for BOS. */ + i++; + } + if (c == OUT && (m->eflags & REG_NOTEOL) == 0) { + sflags |= SEOS; + /* Step one more for EOS. */ + i++; + } + if (i != 0) { + for (; i > 0; i--) + st = step(m->g, startst, stopst, st, flagch, st, + sflags); + SP("sboleol", st, c); + } + + /* how about a word boundary? */ + if ((flagch == BOL || (lastc != OUT && !ISWORD(lastc))) && + (c != OUT && ISWORD(c))) { + flagch = BOW; + } + if ((lastc != OUT && ISWORD(lastc)) && + (flagch == EOL || (c != OUT && !ISWORD(c)))) { + flagch = EOW; + } + if (flagch == BOW || flagch == EOW) { + st = step(m->g, startst, stopst, st, flagch, st, + sflags); + SP("sboweow", st, c); + } + if (lastc != OUT && c != OUT && + ISWORD(lastc) == ISWORD(c)) { + flagch = NWBND; + } else if ((lastc == OUT && !ISWORD(c)) || + (c == OUT && !ISWORD(lastc))) { + flagch = NWBND; + } + if (flagch == NWBND) { + st = step(m->g, startst, stopst, st, flagch, st, + sflags); + SP("snwbnd", st, c); + } + + /* are we done? */ + if (ISSET(st, stopst)) { + if (fast) + break; + else + matchp = p; + } + if (EQ(st, empty) || p == stop || clen > (size_t)(stop - p)) + break; /* NOTE BREAK OUT */ + + /* no, we must deal with this character */ + ASSIGN(tmp, st); + if (fast) + ASSIGN(st, fresh); + else + ASSIGN(st, empty); + assert(c != OUT); + st = step(m->g, startst, stopst, tmp, c, st, sflags); + SP("saft", st, c); + assert(EQ(step(m->g, startst, stopst, st, NOTHING, st, sflags), + st)); + p += clen; + } + + if (fast) { + assert(matchp != NULL); + m->coldp = matchp; + if (ISSET(st, stopst)) + return (p + XMBRTOWC(NULL, p, (size_t)(stop - p), + &m->mbs, 0)); + else + return (NULL); + } else + return (matchp); +} + +/* + * step - map set of states reachable before char to set reachable after + * static states step(struct re_guts *g, sopno start, sopno stop, \ + * states bef, int ch, states aft); + * #define BOL (OUT-1) + * #define EOL (BOL-1) + * #define BOLEOL (BOL-2) + * #define NOTHING (BOL-3) + * #define BOW (BOL-4) + * #define EOW (BOL-5) + * #define BADCHAR (BOL-6) + * #define NONCHAR(c) ((c) <= OUT) + */ +static states +step(struct re_guts *g, + sopno start, /* start state within strip */ + sopno stop, /* state after stop state within strip */ + states bef, /* states reachable before */ + wint_t ch, /* character or NONCHAR code */ + states aft, /* states already known reachable after */ + int sflags) /* state flags */ +{ + cset *cs; + sop s; + sopno pc; + onestate here; /* note, macros know this name */ + sopno look; + int i; + + _DIAGASSERT(g != NULL); + + for (pc = start, INIT(here, pc); pc != stop; pc++, INC(here)) { + s = g->strip[pc]; + switch (OP(s)) { + case OEND: + assert(pc == stop-1); + break; + case OCHAR: + /* only characters can match */ + assert(!NONCHAR(ch) || ch != OPND(s)); + if (ch == (wint_t)OPND(s)) + FWD(aft, bef, 1); + break; + case OBOS: + if ((ch == BOL || ch == BOLEOL) && (sflags & SBOS) != 0) + FWD(aft, bef, 1); + break; + case OEOS: + if ((ch == EOL || ch == BOLEOL) && (sflags & SEOS) != 0) + FWD(aft, bef, 1); + break; + case OBOL: + if (ch == BOL || ch == BOLEOL) + FWD(aft, bef, 1); + break; + case OEOL: + if (ch == EOL || ch == BOLEOL) + FWD(aft, bef, 1); + break; + case OBOW: + if (ch == BOW) + FWD(aft, bef, 1); + break; + case OEOW: + if (ch == EOW) + FWD(aft, bef, 1); + break; + case OWBND: + if (ch == BOW || ch == EOW) + FWD(aft, bef, 1); + break; + case ONWBND: + if (ch == NWBND) + FWD(aft, aft, 1); + break; + case OANY: + if (!NONCHAR(ch)) + FWD(aft, bef, 1); + break; + case OANYOF: + cs = &g->sets[OPND(s)]; + if (!NONCHAR(ch) && CHIN(cs, ch)) + FWD(aft, bef, 1); + break; + case OBACK_: /* ignored here */ + case O_BACK: + FWD(aft, aft, 1); + break; + case OPLUS_: /* forward, this is just an empty */ + FWD(aft, aft, 1); + break; + case O_PLUS: /* both forward and back */ + FWD(aft, aft, 1); + i = ISSETBACK(aft, OPND(s)); + BACK(aft, aft, OPND(s)); + if (!i && ISSETBACK(aft, OPND(s))) { + /* oho, must reconsider loop body */ + pc -= OPND(s) + 1; + INIT(here, pc); + } + break; + case OQUEST_: /* two branches, both forward */ + FWD(aft, aft, 1); + FWD(aft, aft, OPND(s)); + break; + case O_QUEST: /* just an empty */ + FWD(aft, aft, 1); + break; + case OLPAREN: /* not significant here */ + case ORPAREN: + FWD(aft, aft, 1); + break; + case OCH_: /* mark the first two branches */ + FWD(aft, aft, 1); + assert(OP(g->strip[pc+OPND(s)]) == OOR2); + FWD(aft, aft, OPND(s)); + break; + case OOR1: /* done a branch, find the O_CH */ + if (ISSTATEIN(aft, here)) { + for (look = 1; + OP(s = g->strip[pc+look]) != O_CH; + look += OPND(s)) + assert(OP(s) == OOR2); + FWD(aft, aft, look + 1); + } + break; + case OOR2: /* propagate OCH_'s marking */ + FWD(aft, aft, 1); + if (OP(g->strip[pc+OPND(s)]) != O_CH) { + assert(OP(g->strip[pc+OPND(s)]) == OOR2); + FWD(aft, aft, OPND(s)); + } + break; + case O_CH: /* just empty */ + FWD(aft, aft, 1); + break; + default: /* ooooops... */ + assert(nope); + break; + } + } + + return (aft); +} + +#ifdef REDEBUG +/* + * print - print a set of states + * #ifdef REDEBUG + * static void print(struct match *m, const char *caption, states st, \ + * int ch, FILE *d); + * #endif + */ +static void +print(struct match *m, + const char *caption, + states st, + int ch, + FILE *d) +{ + struct re_guts *g = m->g; + sopno i; + int first = 1; + + _DIAGASSERT(m != NULL); + _DIAGASSERT(caption != NULL); + + if (!(m->eflags®_TRACE)) + return; + + _DIAGASSERT(d != NULL); + + fprintf(d, "%s", caption); + if (ch != '\0') + fprintf(d, " %s", pchar(ch)); + for (i = 0; i < g->nstates; i++) + if (ISSET(st, i)) { + fprintf(d, "%s%lu", (first) ? "\t" : ", ", i); + first = 0; + } + fprintf(d, "\n"); +} + +/* + * at - print current situation + * #ifdef REDEBUG + * static void at(struct match *m, const char *title, const char *start, \ + * const char *stop, sopno startst, sopno stopst); + * #endif + */ +static void +at(struct match *m, + const char *title, + const char *start, + const char *stop, + sopno startst, + sopno stopst) +{ + + _DIAGASSERT(m != NULL); + _DIAGASSERT(title != NULL); + _DIAGASSERT(start != NULL); + _DIAGASSERT(stop != NULL); + + if (!(m->eflags®_TRACE)) + return; + + printf("%s %s-", title, pchar(*start)); + printf("%s ", pchar(*stop)); + printf("%ld-%ld\n", (long)startst, (long)stopst); +} + +#ifndef PCHARDONE +#define PCHARDONE /* never again */ +/* + * pchar - make a character printable + * #ifdef REDEBUG + * static const char *pchar(int ch); + * #endif + * + * Is this identical to regchar() over in debug.c? Well, yes. But a + * duplicate here avoids having a debugging-capable regexec.o tied to + * a matching debug.o, and this is convenient. It all disappears in + * the non-debug compilation anyway, so it doesn't matter much. + */ +/* -> representation */ +static const char * +pchar(int ch) +{ + static char pbuf[10]; + + if (isprint((uch)ch) || ch == ' ') + snprintf(pbuf, sizeof (pbuf), "%c", ch); + else + snprintf(pbuf, sizeof (pbuf), "\\%o", ch); + return (pbuf); +} +#endif +#endif + +#undef stepback +#undef matcher +#undef walk +#undef dissect +#undef backref +#undef step +#undef print +#undef at +#undef match diff --git a/lib/os/windows/libregex/re_format.7 b/lib/os/windows/libregex/re_format.7 new file mode 100644 index 000000000000..836e2529ee4a --- /dev/null +++ b/lib/os/windows/libregex/re_format.7 @@ -0,0 +1,494 @@ +.\" $NetBSD: re_format.7,v 1.14 2021/02/24 09:10:12 wiz Exp $ +.\" +.\" Copyright (c) 1992, 1993, 1994 Henry Spencer. +.\" Copyright (c) 1992, 1993, 1994 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" This code is derived from software contributed to Berkeley by +.\" Henry Spencer. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)re_format.7 8.3 (Berkeley) 3/20/94 +.\" $FreeBSD: head/lib/libc/regex/re_format.7 314373 2017-02-28 05:14:42Z glebius $ +.\" +.Dd February 22, 2021 +.Dt RE_FORMAT 7 +.Os +.Sh NAME +.Nm re_format +.Nd POSIX 1003.2 regular expressions +.Sh DESCRIPTION +Regular expressions +.Pq Dq RE Ns s , +as defined in +.St -p1003.2 , +come in two forms: +modern REs (roughly those of +.Xr egrep 1 ; +1003.2 calls these +.Dq extended +REs) +and obsolete REs (roughly those of +.Xr ed 1 ; +1003.2 +.Dq basic +REs). +Obsolete REs mostly exist for backward compatibility in some old programs; +they will be discussed at the end. +.St -p1003.2 +leaves some aspects of RE syntax and semantics open; +`\(dd' marks decisions on these aspects that +may not be fully portable to other +.St -p1003.2 +implementations. +.Pp +A (modern) RE is one\(dd or more non-empty\(dd +.Em branches , +separated by +.Ql \&| . +It matches anything that matches one of the branches. +.Pp +A branch is one\(dd or more +.Em pieces , +concatenated. +It matches a match for the first, followed by a match for the second, etc. +.Pp +A piece is an +.Em atom +possibly followed +by a single\(dd +.Ql \&* , +.Ql \&+ , +.Ql \&? , +or +.Em bound . +An atom followed by +.Ql \&* +matches a sequence of 0 or more matches of the atom. +An atom followed by +.Ql \&+ +matches a sequence of 1 or more matches of the atom. +An atom followed by +.Ql ?\& +matches a sequence of 0 or 1 matches of the atom. +.Pp +A +.Em bound +is +.Ql \&{ +followed by an unsigned decimal integer, +possibly followed by +.Ql \&, +possibly followed by another unsigned decimal integer, +always followed by +.Ql \&} . +The integers must lie between 0 and +.Dv RE_DUP_MAX +(255\(dd) inclusive, +and if there are two of them, the first may not exceed the second. +An atom followed by a bound containing one integer +.Em i +and no comma matches +a sequence of exactly +.Em i +matches of the atom. +An atom followed by a bound +containing one integer +.Em i +and a comma matches +a sequence of +.Em i +or more matches of the atom. +An atom followed by a bound +containing two integers +.Em i +and +.Em j +matches +a sequence of +.Em i +through +.Em j +(inclusive) matches of the atom. +.Pp +An atom is a regular expression enclosed in +.Ql () +(matching a match for the +regular expression), +an empty set of +.Ql () +(matching the null string)\(dd, +a +.Em bracket expression +(see below), +.Ql .\& +(matching any single character), +.Ql \&^ +(matching the null string at the beginning of a line), +.Ql \&$ +(matching the null string at the end of a line), a +.Ql \e +followed by one of the characters +.Ql ^.[$()|*+?{\e +(matching that character taken as an ordinary character), +a +.Ql \e +followed by any other character\(dd +(matching that character taken as an ordinary character, +as if the +.Ql \e +had not been present\(dd), +or a single character with no other significance (matching that character). +A +.Ql \&{ +followed by a character other than a digit is an ordinary +character, not the beginning of a bound\(dd. +It is illegal to end an RE with +.Ql \e . +.Pp +A +.Em bracket expression +is a list of characters enclosed in +.Ql [] . +It normally matches any single character from the list (but see below). +If the list begins with +.Ql \&^ , +it matches any single character +(but see below) +.Em not +from the rest of the list. +If two characters in the list are separated by +.Ql \&- , +this is shorthand +for the full +.Em range +of characters between those two (inclusive) in the +collating sequence, +.No e.g. Ql [0-9] +in ASCII matches any decimal digit. +It is illegal\(dd for two ranges to share an +endpoint, +.No e.g. Ql a-c-e . +Ranges are very collating-sequence-dependent, +and portable programs should avoid relying on them. +.Pp +To include a literal +.Ql \&] +in the list, make it the first character +(following a possible +.Ql \&^ ) . +To include a literal +.Ql \&- , +make it the first or last character, +or the second endpoint of a range. +To use a literal +.Ql \&- +as the first endpoint of a range, +enclose it in +.Ql [.\& +and +.Ql .]\& +to make it a collating element (see below). +With the exception of these and some combinations using +.Ql \&[ +(see next paragraphs), all other special characters, including +.Ql \e , +lose their special significance within a bracket expression. +.Pp +Within a bracket expression, a collating element (a character, +a multi-character sequence that collates as if it were a single character, +or a collating-sequence name for either) +enclosed in +.Ql [.\& +and +.Ql .]\& +stands for the +sequence of characters of that collating element. +The sequence is a single element of the bracket expression's list. +A bracket expression containing a multi-character collating element +can thus match more than one character, +e.g.\& if the collating sequence includes a +.Ql ch +collating element, +then the RE +.Ql [[.ch.]]*c +matches the first five characters +of +.Ql chchcc . +.Pp +Within a bracket expression, a collating element enclosed in +.Ql [= +and +.Ql =] +is an equivalence class, standing for the sequences of characters +of all collating elements equivalent to that one, including itself. +(If there are no other equivalent collating elements, +the treatment is as if the enclosing delimiters were +.Ql [.\& +and +.Ql .] . ) +For example, if +.Ql x +and +.Ql y +are the members of an equivalence class, +then +.Ql [[=x=]] , +.Ql [[=y=]] , +and +.Ql [xy] +are all synonymous. +An equivalence class may not\(dd be an endpoint +of a range. +.Pp +Within a bracket expression, the name of a +.Em character class +enclosed in +.Ql [: +and +.Ql :] +stands for the list of all characters belonging to that +class. +Standard character class names are: +.Bl -column "alnum" "digit" "xdigit" -offset indent +.It Em "alnum digit punct" +.It Em "alpha graph space" +.It Em "blank lower upper" +.It Em "cntrl print xdigit" +.El +.Pp +These stand for the character classes defined in +.Xr ctype 3 . +A locale may provide others. +A character class may not be used as an endpoint of a range. +.Pp +A bracketed expression like +.Ql [[:class:]] +can be used to match a single character that belongs to a character +class. +The reverse, matching any character that does not belong to a specific +class, the negation operator of bracket expressions may be used: +.Ql [^[:class:]] . +.Pp +There are two special cases\(dd of bracket expressions: +the bracket expressions +.Ql [[:<:]] +and +.Ql [[:>:]] +match the null string at the beginning and end of a word respectively. +A word is defined as a sequence of word characters +which is neither preceded nor followed by +word characters. +A word character is an +.Em alnum +character (as defined by +.Xr ctype 3 ) +or an underscore. +This is an extension, +compatible with but not specified by +.St -p1003.2 , +and should be used with +caution in software intended to be portable to other systems. +The additional word delimiters +.Ql \e< +and +.Ql \e> +are provided to ease compatibility with traditional +SVR4 +systems but are not portable and should be avoided. +.Pp +In the event that an RE could match more than one substring of a given +string, +the RE matches the one starting earliest in the string. +If the RE could match more than one substring starting at that point, +it matches the longest. +Subexpressions also match the longest possible substrings, subject to +the constraint that the whole match be as long as possible, +with subexpressions starting earlier in the RE taking priority over +ones starting later. +Note that higher-level subexpressions thus take priority over +their lower-level component subexpressions. +.Pp +Match lengths are measured in characters, not collating elements. +A null string is considered longer than no match at all. +For example, +.Ql bb* +matches the three middle characters of +.Ql abbbc , +.Ql (wee|week)(knights|nights) +matches all ten characters of +.Ql weeknights , +when +.Ql (.*).*\& +is matched against +.Ql abc +the parenthesized subexpression +matches all three characters, and +when +.Ql (a*)* +is matched against +.Ql bc +both the whole RE and the parenthesized +subexpression match the null string. +.Pp +If case-independent matching is specified, +the effect is much as if all case distinctions had vanished from the +alphabet. +When an alphabetic that exists in multiple cases appears as an +ordinary character outside a bracket expression, it is effectively +transformed into a bracket expression containing both cases, +.No e.g. Ql x +becomes +.Ql [xX] . +When it appears inside a bracket expression, all case counterparts +of it are added to the bracket expression, so that (e.g.) +.Ql [x] +becomes +.Ql [xX] +and +.Ql [^x] +becomes +.Ql [^xX] . +.Pp +No particular limit is imposed on the length of REs\(dd. +Programs intended to be portable should not employ REs longer +than 256 bytes, +as an implementation can refuse to accept such REs and remain +POSIX-compliant. +.Pp +Obsolete +.Pq Dq basic +regular expressions differ in several respects. +.Ql \&| +is an ordinary character and there is no equivalent +for its functionality. +.Ql \&+ +and +.Ql ?\& +are ordinary characters, and their functionality +can be expressed using bounds +.Po +.Ql {1,} +or +.Ql {0,1} +respectively +.Pc . +Also note that +.Ql x+ +in modern REs is equivalent to +.Ql xx* . +The delimiters for bounds are +.Ql \e{ +and +.Ql \e} , +with +.Ql \&{ +and +.Ql \&} +by themselves ordinary characters. +The parentheses for nested subexpressions are +.Ql \e( +and +.Ql \e) , +with +.Ql \&( +and +.Ql \&) +by themselves ordinary characters. +.Ql \&^ +is an ordinary character except at the beginning of the +RE or\(dd the beginning of a parenthesized subexpression, +.Ql \&$ +is an ordinary character except at the end of the +RE or\(dd the end of a parenthesized subexpression, +and +.Ql \&* +is an ordinary character if it appears at the beginning of the +RE or the beginning of a parenthesized subexpression +(after a possible leading +.Ql \&^ ) . +Finally, there is one new type of atom, a +.Em back reference : +.Ql \e +followed by a non-zero decimal digit +.Em d +matches the same sequence of characters +matched by the +.Em d Ns th +parenthesized subexpression +(numbering subexpressions by the positions of their opening parentheses, +left to right), +so that (e.g.) +.Ql \e([bc]\e)\e1 +matches +.Ql bb +or +.Ql cc +but not +.Ql bc . +.Sh SEE ALSO +.Xr regex 3 +.Rs +.%T Regular Expression Notation +.%R IEEE Std +.%N 1003.2 +.%P section 2.8 +.Re +.Sh BUGS +Having two kinds of REs is a botch. +.Pp +The current +.St -p1003.2 +spec says that +.Ql \&) +is an ordinary character in +the absence of an unmatched +.Ql \&( ; +this was an unintentional result of a wording error, +and change is likely. +Avoid relying on it. +.Pp +Back references are a dreadful botch, +posing major problems for efficient implementations. +They are also somewhat vaguely defined +(does +.Ql a\e(\e(b\e)*\e2\e)*d +match +.Ql abbbd ? ) . +Avoid using them. +.Pp +.St -p1003.2 +specification of case-independent matching is vague. +The +.Dq one case implies all cases +definition given above +is current consensus among implementors as to the right interpretation. +.Pp +The syntax for word boundaries is incredibly ugly. diff --git a/lib/os/windows/libregex/regcomp.c b/lib/os/windows/libregex/regcomp.c new file mode 100644 index 000000000000..ca90a3e03280 --- /dev/null +++ b/lib/os/windows/libregex/regcomp.c @@ -0,0 +1,2457 @@ +/* $NetBSD: regcomp.c,v 1.46 2021/03/11 15:00:29 christos Exp $ */ + +/* + * SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 1992, 1993, 1994 Henry Spencer. + * Copyright (c) 1992, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * Copyright (c) 2011 The FreeBSD Foundation + * All rights reserved. + * Portions of this software were developed by David Chisnall + * under sponsorship from the FreeBSD Foundation. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)regcomp.c 8.5 (Berkeley) 3/20/94 + */ + +#if HAVE_NBTOOL_CONFIG_H +#include "nbtool_config.h" +#endif + +// #include + +#define _OPENBSD_SOURCE + +#ifndef LIBHACK +#define REGEX_GNU_EXTENSIONS + +// #include "namespace.h" +#endif +#include +#include +#include +#include +#include +#include +#include +#include + +#if defined(__weak_alias) && !defined(LIBHACK) +__weak_alias(regcomp, _regcomp) +#endif + +#ifdef REGEX_LIBC_COLLATE +#include "collate.h" +#endif + +#include "utils.h" +#include "regex2.h" + +#include "cname.h" + +/* + * Branching context, used to keep track of branch state for all of the branch- + * aware functions. In addition to keeping track of branch positions for the + * p_branch_* functions, we use this to simplify some clumsiness in BREs for + * detection of whether ^ is acting as an anchor or being used erroneously and + * also for whether we're in a sub-expression or not. + */ +struct branchc { + sopno start; + sopno back; + sopno fwd; + + int nbranch; + int nchain; + bool outer; + bool terminate; +}; + +/* + * parse structure, passed up and down to avoid global variables and + * other clumsinesses + */ +struct parse { + const char *next; /* next character in RE */ + const char *end; /* end of string (-> NUL normally) */ + int error; /* has an error been seen? */ + int gnuext; + sop *strip; /* malloced strip */ + sopno ssize; /* malloced strip size (allocated) */ + sopno slen; /* malloced strip length (used) */ + size_t ncsalloc; /* number of csets allocated */ + struct re_guts *g; +#define NPAREN 10 /* we need to remember () 1-9 for back refs */ + sopno pbegin[NPAREN]; /* -> ( ([0] unused) */ + sopno pend[NPAREN]; /* -> ) ([0] unused) */ + bool allowbranch; /* can this expression branch? */ + bool bre; /* convenience; is this a BRE? */ + int pflags; /* other parsing flags -- legacy escapes? */ + bool (*parse_expr)(struct parse *, struct branchc *); + void (*pre_parse)(struct parse *, struct branchc *); + void (*post_parse)(struct parse *, struct branchc *); +}; + +#define PFLAG_LEGACY_ESC 0x00000001 + +/* ========= begin header generated by ./mkh ========= */ +#ifdef __cplusplus +extern "C" { +#endif + +/* === regcomp.c === */ +static bool p_ere_exp(struct parse *p, struct branchc *bc); +static void p_str(struct parse *p); +static int p_branch_eat_delim(struct parse *p, struct branchc *bc); +static void p_branch_ins_offset(struct parse *p, struct branchc *bc); +static void p_branch_fix_tail(struct parse *p, struct branchc *bc); +static bool p_branch_empty(struct parse *p, struct branchc *bc); +static bool p_branch_do(struct parse *p, struct branchc *bc); +static void p_bre_pre_parse(struct parse *p, struct branchc *bc); +static void p_bre_post_parse(struct parse *p, struct branchc *bc); +static void p_re(struct parse *p, int end1, int end2); +static bool p_simp_re(struct parse *p, struct branchc *bc); +static int p_count(struct parse *p); +static void p_bracket(struct parse *p); +static int p_range_cmp(wchar_t c1, wchar_t c2); +static void p_b_term(struct parse *p, cset *cs); +#ifdef REGEX_GNU_EXTENSIONS +static int p_b_pseudoclass(struct parse *p, char c); +#endif +static void p_b_cclass(struct parse *p, cset *cs); +static void p_b_cclass_named(struct parse *p, cset *cs, const char[]); +static void p_b_eclass(struct parse *p, cset *cs); +static wint_t p_b_symbol(struct parse *p); +static wint_t p_b_coll_elem(struct parse *p, wint_t endc); +static bool may_escape(struct parse *p, const wint_t ch); +static wint_t othercase(wint_t ch); +static void bothcases(struct parse *p, wint_t ch); +static void ordinary(struct parse *p, wint_t ch); +static void nonnewline(struct parse *p); +static void repeat(struct parse *p, sopno start, int from, int to); +static int seterr(struct parse *p, int e); +static cset *allocset(struct parse *p); +static void freeset(struct parse *p, cset *cs); +static void CHadd(struct parse *p, cset *cs, wint_t ch); +static void CHaddrange(struct parse *p, cset *cs, wint_t min, wint_t max); +static void CHaddtype(struct parse *p, cset *cs, wctype_t wct); +static wint_t singleton(cset *cs); +static sopno dupl(struct parse *p, sopno start, sopno finish); +static void doemit(struct parse *p, sop op, size_t opnd); +static void doinsert(struct parse *p, sop op, size_t opnd, sopno pos); +static void dofwd(struct parse *p, sopno pos, sop value); +static int enlarge(struct parse *p, sopno size); +static void stripsnug(struct parse *p, struct re_guts *g); +static void findmust(struct parse *p, struct re_guts *g); +static int altoffset(sop *scan, int offset); +static void computejumps(struct parse *p, struct re_guts *g); +static void computematchjumps(struct parse *p, struct re_guts *g); +static sopno pluscount(struct parse *p, struct re_guts *g); +static wint_t wgetnext(struct parse *p); + +#ifdef __cplusplus +} +#endif +/* ========= end header generated by ./mkh ========= */ + +static char nuls[10]; /* place to point scanner in event of error */ + +/* + * macros for use with parse structure + * BEWARE: these know that the parse structure is named `p' !!! + */ +#define PEEK() (*p->next) +#define PEEK2() (*(p->next+1)) +#define MORE() (p->next < p->end) +#define MORE2() (p->next+1 < p->end) +#define SEE(c) (MORE() && PEEK() == (c)) +#define SEETWO(a, b) (MORE() && MORE2() && PEEK() == (a) && PEEK2() == (b)) +#define SEESPEC(a) (p->bre ? SEETWO('\\', a) : SEE(a)) +#define EAT(c) ((SEE(c)) ? (NEXT(), 1) : 0) +#define EATTWO(a, b) ((SEETWO(a, b)) ? (NEXT2(), 1) : 0) +#define EATSPEC(a) (p->bre ? EATTWO('\\', a) : EAT(a)) +#define NEXT() (p->next++) +#define NEXT2() (p->next += 2) +#define NEXTn(n) (p->next += (n)) +#define GETNEXT() (*p->next++) +#define WGETNEXT() wgetnext(p) +#define SETERROR(e) seterr(p, (e)) +#define REQUIRE(co, e) ((co) || SETERROR(e)) +#define MUSTSEE(c, e) (REQUIRE(MORE() && PEEK() == (c), e)) +#define MUSTEAT(c, e) (REQUIRE(MORE() && GETNEXT() == (c), e)) +#define MUSTNOTSEE(c, e) (REQUIRE(!MORE() || PEEK() != (c), e)) +#define EMIT(op, sopnd) doemit(p, (op), (sopnd)) +#define INSERT(op, pos) doinsert(p, (op), HERE()-(pos)+1, pos) +#define AHEAD(pos) dofwd(p, pos, HERE()-(pos)) +#define ASTERN(sop, pos) EMIT(sop, HERE()-pos) +#define HERE() (p->slen) +#define THERE() (p->slen - 1) +#define THERETHERE() (p->slen - 2) +#define DROP(n) (p->slen -= (n)) + +/* Macro used by computejump()/computematchjump() */ +#ifndef MIN +#define MIN(a, b) ((a) < (b)?(a):(b)) +#endif + +#ifndef NLS +static const struct { + const char *name; + int (*func)(int); +} wctypes[] = { +#define ADD(x) { .name = # x, .func = is ## x } + ADD(alnum), + ADD(alpha), + ADD(blank), + ADD(cntrl), + ADD(digit), + ADD(graph), + ADD(lower), + ADD(print), + ADD(punct), + ADD(space), + ADD(upper), + ADD(xdigit), +#undef ADD +}; + +wctype_t +__regex_wctype(const char *str) +{ + for (size_t i = 0; i < __arraycount(wctypes); i++) { + if (strcmp(wctypes[i].name, str) == 0) + return ((wctype_t)(i + 1)); + } + return ((wctype_t)0); +} + +int +__regex_iswctype(wint_t c, wctype_t ct) +{ + if (ct == 0) + return (0); + return ((*wctypes[ct - 1].func)(c)); +} +#endif + +static int /* 0 success, otherwise REG_something */ +regcomp_internal(regex_t *__restrict preg, + const char *__restrict pattern, + int cflags, int pflags) +{ + struct parse pa; + struct re_guts *g; + struct parse *p = &pa; + int i; + size_t len; + size_t maxlen; +#ifdef REDEBUG +#define GOODFLAGS(f) (f) +#else +#define GOODFLAGS(f) ((f)&~REG_DUMP) +#endif + + _DIAGASSERT(preg != NULL); + _DIAGASSERT(pattern != NULL); + + cflags = GOODFLAGS(cflags); + if ((cflags®_EXTENDED) && (cflags®_NOSPEC)) + return (REG_INVARG); + + if (cflags®_PEND) { + if (preg->re_endp < pattern) + return (REG_INVARG); + len = preg->re_endp - pattern; + } else + len = strlen(pattern); + + /* do the mallocs early so failure handling is easy */ + g = malloc(sizeof (*g)); + if (g == NULL) + return (REG_ESPACE); + /* + * Limit the pattern space to avoid a 32-bit overflow on buffer + * extension. Also avoid any signed overflow in case of conversion + * so make the real limit based on a 31-bit overflow. + * + * Likely not applicable on 64-bit systems but handle the case + * generically (who are we to stop people from using ~715MB+ + * patterns?). + */ + maxlen = ((size_t)-1 >> 1) / sizeof (*p->strip) * 2 / 3; + if (len >= maxlen) { + free(g); + return (REG_ESPACE); + } + p->ssize = (sopno)(len / 2 * 3 + 1); /* ugh */ + assert(p->ssize >= len); + + p->strip = calloc(p->ssize, sizeof (*p->strip)); + p->slen = 0; + if (p->strip == NULL) { + free(g); + return (REG_ESPACE); + } + + /* set things up */ + p->g = g; + p->next = pattern; /* convenience; we do not modify it */ + p->end = p->next + len; + p->error = 0; + p->ncsalloc = 0; + p->pflags = pflags; + for (i = 0; i < NPAREN; i++) { + p->pbegin[i] = 0; + p->pend[i] = 0; + } +#ifdef REGEX_GNU_EXTENSIONS + if ((cflags & REG_GNU) == 0) { + p->gnuext = false; + p->allowbranch = (cflags & REG_EXTENDED) != 0; + } else + p->gnuext = p->allowbranch = true; +#else + p->gnuext = false; + p->allowbranch = (cflags & REG_EXTENDED) != 0; +#endif + if (cflags & REG_EXTENDED) { + p->bre = false; + p->parse_expr = p_ere_exp; + p->pre_parse = NULL; + p->post_parse = NULL; + } else { + p->bre = true; + p->parse_expr = p_simp_re; + p->pre_parse = p_bre_pre_parse; + p->post_parse = p_bre_post_parse; + } + g->sets = NULL; + g->ncsets = 0; + g->cflags = cflags; + g->iflags = 0; + g->nbol = 0; + g->neol = 0; + g->must = NULL; + g->moffset = -1; + g->charjump = NULL; + g->matchjump = NULL; + g->mlen = 0; + g->nsub = 0; + g->backrefs = 0; + + /* do it */ + EMIT(OEND, 0); + g->firststate = THERE(); + if (cflags & REG_NOSPEC) + p_str(p); + else + p_re(p, OUT, OUT); + EMIT(OEND, 0); + g->laststate = THERE(); + + /* tidy up loose ends and fill things in */ + stripsnug(p, g); + findmust(p, g); + /* + * only use Boyer-Moore algorithm if the pattern is bigger + * than three characters + */ + if (g->mlen > 3) { + computejumps(p, g); + computematchjumps(p, g); + if (g->matchjump == NULL && g->charjump != NULL) { + free(g->charjump); + g->charjump = NULL; + } + } + g->nplus = pluscount(p, g); + g->magic = MAGIC2; + preg->re_nsub = g->nsub; + preg->re_g = g; + preg->re_magic = MAGIC1; +#ifndef REDEBUG + /* not debugging, so can't rely on the assert() in regexec() */ + if (g->iflags&BAD) + SETERROR(REG_ASSERT); +#endif + + /* win or lose, we're done */ + if (p->error != 0) /* lose */ + regfree(preg); + return (p->error); +} + +/* + * regcomp - interface for parser and compilation + * extern int regcomp(regex_t *, const char *, int); + * #define REG_BASIC 0000 + * #define REG_EXTENDED 0001 + * #define REG_ICASE 0002 + * #define REG_NOSUB 0004 + * #define REG_NEWLINE 0010 + * #define REG_NOSPEC 0020 + * #define REG_PEND 0040 + * #define REG_DUMP 0200 + */ +int /* 0 success, otherwise REG_something */ +regcomp(regex_t *__restrict preg, + const char *__restrict pattern, + int cflags) +{ + + return (regcomp_internal(preg, pattern, cflags, 0)); +} + +/* + * p_ere_exp - parse one subERE, an atom possibly followed by a repetition op, + * return whether we should terminate or not + * static bool p_ere_exp(struct parse *p); + */ +static bool +p_ere_exp(struct parse *p, struct branchc *bc) +{ + char c; + wint_t wc; + sopno pos; + int count; + int count2; +#ifdef REGEX_GNU_EXTENSIONS + size_t i; + int handled; +#endif + sopno subno; + int wascaret = 0; + + _DIAGASSERT(p != NULL); + + (void) bc; + assert(MORE()); /* caller should have ensured this */ + c = GETNEXT(); + +#ifdef REGEX_GNU_EXTENSIONS + handled = 0; +#endif + pos = HERE(); + switch (c) { + case '(': + (void) REQUIRE(MORE(), REG_EPAREN); + p->g->nsub++; + subno = (sopno)p->g->nsub; + if (subno < NPAREN) + p->pbegin[subno] = HERE(); + EMIT(OLPAREN, subno); + if (!SEE(')')) + p_re(p, ')', IGN); + if (subno < NPAREN) { + p->pend[subno] = HERE(); + assert(p->pend[subno] != 0); + } + EMIT(ORPAREN, subno); + (void) MUSTEAT(')', REG_EPAREN); + break; +#ifndef POSIX_MISTAKE + case ')': /* happens only if no current unmatched ( */ + /* + * You may ask, why the ifndef? Because I didn't notice + * this until slightly too late for 1003.2, and none of the + * other 1003.2 regular-expression reviewers noticed it at + * all. So an unmatched ) is legal POSIX, at least until + * we can get it fixed. + */ + SETERROR(REG_EPAREN); + break; +#endif + case '^': + EMIT(OBOL, 0); + p->g->iflags |= USEBOL; + p->g->nbol++; + wascaret = 1; + break; + case '$': + EMIT(OEOL, 0); + p->g->iflags |= USEEOL; + p->g->neol++; + break; + case '|': + SETERROR(REG_EMPTY); + break; + case '*': + case '+': + case '?': + case '{': + SETERROR(REG_BADRPT); + break; + case '.': + if (p->g->cflags®_NEWLINE) + nonnewline(p); + else + EMIT(OANY, 0); + break; + case '[': + p_bracket(p); + break; + case '\\': + (void) REQUIRE(MORE(), REG_EESCAPE); + wc = WGETNEXT(); +#ifdef REGEX_GNU_EXTENSIONS + if (p->gnuext) { + handled = 1; + switch (wc) { + case '`': + EMIT(OBOS, 0); + break; + case '\'': + EMIT(OEOS, 0); + break; + case 'B': + EMIT(ONWBND, 0); + break; + case 'b': + EMIT(OWBND, 0); + break; + case 'W': + case 'w': + case 'S': + case 's': + p_b_pseudoclass(p, wc); + break; + case 'a': + ordinary(p, '\a'); + break; + case 'e': + ordinary(p, '\e'); + break; + case 'f': + ordinary(p, '\f'); + break; + case 'n': + ordinary(p, '\n'); + break; + case 'r': + ordinary(p, '\r'); + break; + case 't': + ordinary(p, '\t'); + break; + case 'v': + ordinary(p, '\v'); + break; + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + i = wc - '0'; + assert(i < NPAREN); + if (p->pend[i] != 0) { + assert(i <= p->g->nsub); + EMIT(OBACK_, i); + assert(p->pbegin[i] != 0); + assert(OP(p->strip[p->pbegin[i]]) == + OLPAREN); + assert(OP(p->strip[p->pend[i]]) == + ORPAREN); + (void) dupl(p, p->pbegin[i]+1, + p->pend[i]); + EMIT(O_BACK, i); + } else + SETERROR(REG_ESUBREG); + p->g->backrefs = 1; + break; + default: + handled = 0; + } +/* Don't proceed to the POSIX bits if we've already handled it */ + if (handled) + break; + } +#endif + switch (wc) { + case '<': + EMIT(OBOW, 0); + break; + case '>': + EMIT(OEOW, 0); + break; + default: + if (may_escape(p, wc)) + ordinary(p, wc); + else + SETERROR(REG_EESCAPE); + break; + } + break; + default: + if (p->error != 0) + return (false); + p->next--; + wc = WGETNEXT(); + ordinary(p, wc); + break; + } + + if (!MORE()) + return (false); + c = PEEK(); + /* we call { a repetition if followed by a digit */ + if (!(c == '*' || c == '+' || c == '?' || c == '{')) + return (false); /* no repetition, we're done */ + else if (c == '{') + (void) REQUIRE(MORE2() && \ + (isdigit((uch)PEEK2()) || PEEK2() == ','), REG_BADRPT); + NEXT(); + + (void) REQUIRE(!wascaret, REG_BADRPT); + switch (c) { + case '*': /* implemented as +? */ + /* this case does not require the (y|) trick, noKLUDGE */ + INSERT(OPLUS_, pos); + ASTERN(O_PLUS, pos); + INSERT(OQUEST_, pos); + ASTERN(O_QUEST, pos); + break; + case '+': + INSERT(OPLUS_, pos); + ASTERN(O_PLUS, pos); + break; + case '?': + /* KLUDGE: emit y? as (y|) until subtle bug gets fixed */ + INSERT(OCH_, pos); /* offset slightly wrong */ + ASTERN(OOR1, pos); /* this one's right */ + AHEAD(pos); /* fix the OCH_ */ + EMIT(OOR2, 0); /* offset very wrong... */ + AHEAD(THERE()); /* ...so fix it */ + ASTERN(O_CH, THERETHERE()); + break; + case '{': + count = p_count(p); + if (EAT(',')) { + if (isdigit((uch)PEEK())) { + count2 = p_count(p); + (void) REQUIRE(count <= count2, REG_BADBR); + } else /* single number with comma */ + count2 = REGINFINITY; + } else /* just a single number */ + count2 = count; + repeat(p, pos, count, count2); + if (!EAT('}')) { /* error heuristics */ + while (MORE() && PEEK() != '}') + NEXT(); + (void) REQUIRE(MORE(), REG_EBRACE); + SETERROR(REG_BADBR); + } + break; + } + + if (!MORE()) + return (false); + c = PEEK(); + if (!(c == '*' || c == '+' || c == '?' || + (c == '{' && MORE2() && isdigit((uch)PEEK2())))) + return (false); + SETERROR(REG_BADRPT); + return (false); +} + +/* + * p_str - string (no metacharacters) "parser" + * static void p_str(struct parse *p); + */ +static void +p_str(struct parse *p) +{ + (void) REQUIRE(MORE(), REG_EMPTY); + while (MORE()) + ordinary(p, WGETNEXT()); +} + +/* + * Eat consecutive branch delimiters for the kind of expression that we are + * parsing, return the number of delimiters that we ate. + */ +static int +p_branch_eat_delim(struct parse *p, struct branchc *bc) +{ + int nskip; + + (void) bc; + nskip = 0; + while (EATSPEC('|')) + ++nskip; + return (nskip); +} + +/* + * Insert necessary branch book-keeping operations. This emits a + * bogus 'next' offset, since we still have more to parse + */ +static void +p_branch_ins_offset(struct parse *p, struct branchc *bc) +{ + + if (bc->nbranch == 0) { + INSERT(OCH_, bc->start); /* offset is wrong */ + bc->fwd = bc->start; + bc->back = bc->start; + } + + ASTERN(OOR1, bc->back); + bc->back = THERE(); + AHEAD(bc->fwd); /* fix previous offset */ + bc->fwd = HERE(); + EMIT(OOR2, 0); /* offset is very wrong */ + ++bc->nbranch; +} + +/* + * Fix the offset of the tail branch, if we actually had any branches. + * This is to correct the bogus placeholder offset that we use. + */ +static void +p_branch_fix_tail(struct parse *p, struct branchc *bc) +{ + + /* Fix bogus offset at the tail if we actually have branches */ + if (bc->nbranch > 0) { + AHEAD(bc->fwd); + ASTERN(O_CH, bc->back); + } +} + +/* + * Signal to the parser that an empty branch has been encountered; this will, + * in the future, be used to allow for more permissive behavior with empty + * branches. The return value should indicate whether parsing may continue + * or not. + */ +static bool +p_branch_empty(struct parse *p, struct branchc *bc) +{ + + (void) bc; + SETERROR(REG_EMPTY); + return (false); +} + +/* + * Take care of any branching requirements. This includes inserting the + * appropriate branching instructions as well as eating all of the branch + * delimiters until we either run out of pattern or need to parse more pattern. + */ +static bool +p_branch_do(struct parse *p, struct branchc *bc) +{ + int ate = 0; + + ate = p_branch_eat_delim(p, bc); + if (ate == 0) + return (false); + else if ((ate > 1 || (bc->outer && !MORE())) && !p_branch_empty(p, bc)) +/* + * Halt parsing only if we have an empty branch and p_branch_empty + * indicates that we must not continue. In the future, this will not + * necessarily be an error. + */ + return (false); + p_branch_ins_offset(p, bc); + + return (true); +} + +static void +p_bre_pre_parse(struct parse *p, struct branchc *bc) +{ + + (void) bc; + /* + * Does not move cleanly into expression parser because of + * ordinary interpration of * at the beginning position of + * an expression. + */ + if (EAT('^')) { + EMIT(OBOL, 0); + p->g->iflags |= USEBOL; + p->g->nbol++; + } +} + +static void +p_bre_post_parse(struct parse *p, struct branchc *bc) +{ + + /* Expression is terminating due to EOL token */ + if (bc->terminate) { + DROP(1); + EMIT(OEOL, 0); + p->g->iflags |= USEEOL; + p->g->neol++; + } +} + +/* + * p_re - Top level parser, concatenation and BRE anchoring + * static void p_re(struct parse *p, int end1, int end2); + * Giving end1 as OUT essentially eliminates the end1/end2 check. + * + * This implementation is a bit of a kludge, in that a trailing $ is first + * taken as an ordinary character and then revised to be an anchor. + * The amount of lookahead needed to avoid this kludge is excessive. + */ +static void +p_re(struct parse *p, + int end1, /* first terminating character */ + int end2) /* second terminating character; ignored for EREs */ +{ + struct branchc bc; + + bc.nbranch = 0; + if (end1 == OUT && end2 == OUT) + bc.outer = true; + else + bc.outer = false; +#define SEEEND() (!p->bre ? SEE(end1) : SEETWO(end1, end2)) + for (;;) { + bc.start = HERE(); + bc.nchain = 0; + bc.terminate = false; + if (p->pre_parse != NULL) + p->pre_parse(p, &bc); + while (MORE() && + (!p->allowbranch || !SEESPEC('|')) && !SEEEND()) { + bc.terminate = p->parse_expr(p, &bc); + ++bc.nchain; + } + if (p->post_parse != NULL) + p->post_parse(p, &bc); + (void) REQUIRE(p->gnuext || HERE() != bc.start, REG_EMPTY); +#ifdef REGEX_GNU_EXTENSIONS + if (p->gnuext && HERE() == bc.start && !p_branch_empty(p, &bc)) + break; +#endif + if (!p->allowbranch) + break; + /* + * p_branch_do's return value indicates whether we should + * continue parsing or not. This is both for correctness and + * a slight optimization, because it will check if we've + * encountered an empty branch or the end of the string + * immediately following a branch delimiter. + */ + if (!p_branch_do(p, &bc)) + break; + } +#undef SEE_END + if (p->allowbranch) + p_branch_fix_tail(p, &bc); + assert(!MORE() || SEE(end1)); +} + +/* + * p_simp_re - parse a simple RE, an atom possibly followed by a repetition + * static bool p_simp_re(struct parse *p, struct branchc *bc); + */ +static bool /* was the simple RE an unbackslashed $? */ +p_simp_re(struct parse *p, struct branchc *bc) +{ + int c; + int cc; /* convenient/control character */ + int count; + int count2; + sopno pos; + bool handled; + size_t i; + wint_t wc; + sopno subno; +#define BACKSL (1<gnuext) { + handled = true; + switch (c) { + case BACKSL|'`': + EMIT(OBOS, 0); + break; + case BACKSL|'\'': + EMIT(OEOS, 0); + break; + case BACKSL|'B': + EMIT(ONWBND, 0); + break; + case BACKSL|'b': + EMIT(OWBND, 0); + break; + case BACKSL|'W': + case BACKSL|'w': + case BACKSL|'S': + case BACKSL|'s': + p_b_pseudoclass(p, cc); + break; + case BACKSL|'a': + ordinary(p, '\a'); + break; + case BACKSL|'e': + ordinary(p, '\e'); + break; + case BACKSL|'f': + ordinary(p, '\f'); + break; + case BACKSL|'n': + ordinary(p, '\n'); + break; + case BACKSL|'r': + ordinary(p, '\r'); + break; + case BACKSL|'t': + ordinary(p, '\t'); + break; + case BACKSL|'v': + ordinary(p, '\v'); + break; + default: + handled = false; + } + } +#endif + } + if (!handled) { + switch (c) { + case '.': + if (p->g->cflags®_NEWLINE) + nonnewline(p); + else + EMIT(OANY, 0); + break; + case '[': + p_bracket(p); + break; + case BACKSL|'<': + EMIT(OBOW, 0); + break; + case BACKSL|'>': + EMIT(OEOW, 0); + break; + case BACKSL|'{': + SETERROR(REG_BADRPT); + break; + case BACKSL|'(': + p->g->nsub++; + subno = (sopno)p->g->nsub; + if (subno < NPAREN) + p->pbegin[subno] = HERE(); + EMIT(OLPAREN, subno); + /* the MORE here is an error heuristic */ + if (MORE() && !SEETWO('\\', ')')) + p_re(p, '\\', ')'); + if (subno < NPAREN) { + p->pend[subno] = HERE(); + assert(p->pend[subno] != 0); + } + EMIT(ORPAREN, subno); + (void) REQUIRE(EATTWO('\\', ')'), REG_EPAREN); + break; + case BACKSL|')': /* should not get here -- must be user */ + SETERROR(REG_EPAREN); + break; + case BACKSL|'1': + case BACKSL|'2': + case BACKSL|'3': + case BACKSL|'4': + case BACKSL|'5': + case BACKSL|'6': + case BACKSL|'7': + case BACKSL|'8': + case BACKSL|'9': + i = (c&~BACKSL) - '0'; + assert(i < NPAREN); + if (p->pend[i] != 0) { + assert(i <= p->g->nsub); + EMIT(OBACK_, i); + assert(p->pbegin[i] != 0); + assert(OP(p->strip[p->pbegin[i]]) == OLPAREN); + assert(OP(p->strip[p->pend[i]]) == ORPAREN); + (void) dupl(p, p->pbegin[i]+1, p->pend[i]); + EMIT(O_BACK, i); + } else + SETERROR(REG_ESUBREG); + p->g->backrefs = 1; + break; + case '*': +/* + * Ordinary if used as the first character beyond BOL anchor of + * a (sub-)expression, counts as a bad repetition operator if it + * appears otherwise. + */ + (void) REQUIRE(bc->nchain == 0, REG_BADRPT); + /* FALLTHROUGH */ + default: + if (p->error != 0) + return (false); /* Definitely not $... */ + p->next--; + wc = WGETNEXT(); + if ((c & BACKSL) == 0 || may_escape(p, wc)) + ordinary(p, wc); + else + SETERROR(REG_EESCAPE); + break; + } + } + + if (EAT('*')) { /* implemented as +? */ + /* this case does not require the (y|) trick, noKLUDGE */ + INSERT(OPLUS_, pos); + ASTERN(O_PLUS, pos); + INSERT(OQUEST_, pos); + ASTERN(O_QUEST, pos); +#ifdef REGEX_GNU_EXTENSIONS + } else if (p->gnuext && EATTWO('\\', '?')) { + INSERT(OQUEST_, pos); + ASTERN(O_QUEST, pos); + } else if (p->gnuext && EATTWO('\\', '+')) { + INSERT(OPLUS_, pos); + ASTERN(O_PLUS, pos); +#endif + } else if (EATTWO('\\', '{')) { + count = p_count(p); + if (EAT(',')) { + if (MORE() && isdigit((uch)PEEK())) { + count2 = p_count(p); + (void) REQUIRE(count <= count2, REG_BADBR); + } else /* single number with comma */ + count2 = REGINFINITY; + } else /* just a single number */ + count2 = count; + repeat(p, pos, count, count2); + if (!EATTWO('\\', '}')) { /* error heuristics */ + while (MORE() && !SEETWO('\\', '}')) + NEXT(); + (void) REQUIRE(MORE(), REG_EBRACE); + SETERROR(REG_BADBR); + } + } else if (c == '$') /* $ (but not \$) ends it */ + return (true); + + return (false); +} + +/* + * p_count - parse a repetition count + * static int p_count(struct parse *p); + */ +static int /* the value */ +p_count(struct parse *p) +{ + int count = 0; + int ndigits = 0; + + while (MORE() && isdigit((uch)PEEK()) && count <= DUPMAX) { + count = count*10 + (GETNEXT() - '0'); + ndigits++; + } + + (void) REQUIRE(ndigits > 0 && count <= DUPMAX, REG_BADBR); + return (count); +} + +/* + * p_bracket - parse a bracketed character list + * static void p_bracket(struct parse *p); + */ +static void +p_bracket(struct parse *p) +{ + cset *cs; + wint_t ch; + + /* Dept of Truly Sickening Special-Case Kludges */ + if (p->next + 5 < p->end && strncmp(p->next, "[:<:]]", 6) == 0) { + EMIT(OBOW, 0); + NEXTn(6); + return; + } + if (p->next + 5 < p->end && strncmp(p->next, "[:>:]]", 6) == 0) { + EMIT(OEOW, 0); + NEXTn(6); + return; + } + + if ((cs = allocset(p)) == NULL) + return; + + if (p->g->cflags®_ICASE) + cs->icase = 1; + if (EAT('^')) + cs->invert = 1; + if (EAT(']')) + CHadd(p, cs, ']'); + else if (EAT('-')) + CHadd(p, cs, '-'); + while (MORE() && PEEK() != ']' && !SEETWO('-', ']')) + p_b_term(p, cs); + if (EAT('-')) + CHadd(p, cs, '-'); + (void) MUSTEAT(']', REG_EBRACK); + + if (p->error != 0) /* don't mess things up further */ + return; + + if (cs->invert && p->g->cflags®_NEWLINE) + cs->bmp['\n' >> 3] |= 1 << ('\n' & 7); + + if ((ch = singleton(cs)) != OUT) { /* optimize singleton sets */ + ordinary(p, ch); + freeset(p, cs); + } else + EMIT(OANYOF, (size_t)(cs - p->g->sets)); +} + +static int +p_range_cmp(wchar_t c1, wchar_t c2) +{ +#ifdef REGEX_LIBC_COLLATE + return (__wcollate_range_cmp(c1, c2)); +#elif defined(NLS) + /* Copied from libc/collate __wcollate_range_cmp */ + wchar_t s1[2], s2[2]; + + s1[0] = c1; + s1[1] = L'\0'; + s2[0] = c2; + s2[1] = L'\0'; + return (wcscoll(s1, s2)); +#else + char s1[2], s2[2]; + + s1[0] = (char)c1; + s1[1] = '\0'; + s2[0] = (char)c2; + s2[1] = '\0'; + return (strcoll(s1, s2)); +#endif +} + +/* + * p_b_term - parse one term of a bracketed character list + * static void p_b_term(struct parse *p, cset *cs); + */ +static void +p_b_term(struct parse *p, cset *cs) +{ + char c; + wint_t start, finish; + wint_t i; +#ifdef REGEX_LIBC_COLLATE + struct xlocale_collate *table = + (struct xlocale_collate *)__get_locale()->components[XLC_COLLATE]; +#endif + + _DIAGASSERT(p != NULL); + _DIAGASSERT(cs != NULL); + + /* classify what we've got */ + switch ((MORE()) ? PEEK() : '\0') { + case '[': + c = (MORE2()) ? PEEK2() : '\0'; + break; + case '-': + SETERROR(REG_ERANGE); + return; /* NOTE RETURN */ + default: + c = '\0'; + break; + } + + switch (c) { + case ':': /* character class */ + NEXT2(); + (void) REQUIRE(MORE(), REG_EBRACK); + c = PEEK(); + (void) REQUIRE(c != '-' && c != ']', REG_ECTYPE); + p_b_cclass(p, cs); + (void) REQUIRE(MORE(), REG_EBRACK); + (void) REQUIRE(EATTWO(':', ']'), REG_ECTYPE); + break; + case '=': /* equivalence class */ + NEXT2(); + (void) REQUIRE(MORE(), REG_EBRACK); + c = PEEK(); + (void) REQUIRE(c != '-' && c != ']', REG_ECOLLATE); + p_b_eclass(p, cs); + (void) REQUIRE(MORE(), REG_EBRACK); + (void) REQUIRE(EATTWO('=', ']'), REG_ECOLLATE); + break; + default: /* symbol, ordinary character, or range */ + start = p_b_symbol(p); + if (SEE('-') && MORE2() && PEEK2() != ']') { + /* range */ + NEXT(); + if (EAT('-')) + finish = '-'; + else + finish = p_b_symbol(p); + } else + finish = start; + if (start == finish) + CHadd(p, cs, start); + else { +#ifdef REGEX_LIBC_COLLATE + if (table->__collate_load_error || MB_CUR_MAX > 1) { +#else + if (MB_CUR_MAX > 1) { +#endif + (void) REQUIRE(start <= finish, REG_ERANGE); + CHaddrange(p, cs, start, finish); + } else { + (void) REQUIRE(p_range_cmp(start, finish) <= 0, + REG_ERANGE); + for (i = 0; i <= UCHAR_MAX; i++) { + if (p_range_cmp(start, i) <= 0 && + p_range_cmp(i, finish) <= 0) + CHadd(p, cs, i); + } + } + } + break; + } +} + +#ifdef REGEX_GNU_EXTENSIONS +/* + * p_b_pseudoclass - parse a pseudo-class (\w, \W, \s, \S) + * static int p_b_pseudoclass(struct parse *p, char c) + */ +static int +p_b_pseudoclass(struct parse *p, char c) +{ + cset *cs; + + if ((cs = allocset(p)) == NULL) + return (0); + + if (p->g->cflags®_ICASE) + cs->icase = 1; + + switch (c) { + case 'W': + cs->invert = 1; + /* FALLTHROUGH */ + case 'w': + p_b_cclass_named(p, cs, "alnum"); + break; + case 'S': + cs->invert = 1; + /* FALLTHROUGH */ + case 's': + p_b_cclass_named(p, cs, "space"); + break; + default: + return (0); + } + + EMIT(OANYOF, (size_t)(cs - p->g->sets)); + return (1); +} +#endif + +/* + * p_b_cclass - parse a character-class name and deal with it + * static void p_b_cclass(struct parse *p, cset *cs); + */ +static void +p_b_cclass(struct parse *p, cset *cs) +{ + const char *sp = p->next; + size_t len; + char clname[16]; + + while (MORE() && isalpha((uch)PEEK())) + NEXT(); + len = p->next - sp; + if (len >= sizeof (clname) - 1) { + SETERROR(REG_ECTYPE); + return; + } + memcpy(clname, sp, len); + clname[len] = '\0'; + + p_b_cclass_named(p, cs, clname); +} + +/* + * p_b_cclass_named - deal with a named character class + * static void p_b_cclass_named(struct parse *p, cset *cs, const char []); + */ +static void +p_b_cclass_named(struct parse *p, cset *cs, const char clname[]) +{ + wctype_t wct; + + if ((wct = wctype(clname)) == 0) { + SETERROR(REG_ECTYPE); + return; + } + CHaddtype(p, cs, wct); +} + +/* + * p_b_eclass - parse an equivalence-class name and deal with it + * static void p_b_eclass(struct parse *p, cset *cs); + * + * This implementation is incomplete. xxx + */ +static void +p_b_eclass(struct parse *p, cset *cs) +{ + wint_t c; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(cs != NULL); + + c = p_b_coll_elem(p, '='); + CHadd(p, cs, c); +} + +/* + * p_b_symbol - parse a character or [..]ed multicharacter collating symbol + * static wint_t p_b_symbol(struct parse *p); + */ +static wint_t /* value of symbol */ +p_b_symbol(struct parse *p) +{ + wint_t value; + + _DIAGASSERT(p != NULL); + + (void) REQUIRE(MORE(), REG_EBRACK); + if (!EATTWO('[', '.')) + return (WGETNEXT()); + + /* collating symbol */ + value = p_b_coll_elem(p, '.'); + (void) REQUIRE(EATTWO('.', ']'), REG_ECOLLATE); + return (value); +} + +/* + * p_b_coll_elem - parse a collating-element name and look it up + * static wint_t p_b_coll_elem(struct parse *p, wint_t endc); + */ +static wint_t /* value of collating element */ +p_b_coll_elem(struct parse *p, + wint_t endc) /* name ended by endc,']' */ +{ + const char *sp = p->next; + struct cname *cp; + size_t len; + + _DIAGASSERT(p != NULL); + + while (MORE() && !SEETWO(endc, ']')) + NEXT(); + if (!MORE()) { + SETERROR(REG_EBRACK); + return (0); + } + len = p->next - sp; + for (cp = cnames; cp->name != NULL; cp++) + if (strncmp(cp->name, sp, len) == 0 && strlen(cp->name) == len) + return (cp->code); /* known name */ +#ifdef NLS + mbstate_t mbs; + wchar_t wc; + size_t clen; + + memset(&mbs, 0, sizeof (mbs)); + if ((clen = mbrtowc(&wc, sp, len, &mbs)) == len) + return (wc); /* single character */ + else if (clen == (size_t)-1 || clen == (size_t)-2) + SETERROR(REG_ILLSEQ); + else + SETERROR(REG_ECOLLATE); /* neither */ + return (0); +#else + if (len == 1) + return (*sp); /* single character */ + SETERROR(REG_ECOLLATE); /* neither */ + return (0); +#endif +} + +/* + * may_escape - determine whether 'ch' is escape-able in the current context + * static int may_escape(struct parse *p, const wint_t ch) + */ +static bool +may_escape(struct parse *p, const wint_t ch) +{ + + if ((p->pflags & PFLAG_LEGACY_ESC) != 0) + return (true); + if (isalpha(ch) || ch == '\'' || ch == '`') + return (false); + return (true); +#ifdef NOTYET + /* + * Build a whitelist of characters that may be escaped to produce an + * ordinary in the current context. This assumes that these have not + * been otherwise interpreted as a special character. Escaping an + * ordinary character yields undefined results according to + * IEEE 1003.1-2008. Some extensions (notably, some GNU extensions) take + * advantage of this and use escaped ordinary characters to provide + * special meaning, e.g. \b, \B, \w, \W, \s, \S. + */ + switch (ch) { + case '|': + case '+': + case '?': + /* The above characters may not be escaped in BREs */ + if (!(p->g->cflags®_EXTENDED)) + return (false); + /* Fallthrough */ + case '(': + case ')': + case '{': + case '}': + case '.': + case '[': + case ']': + case '\\': + case '*': + case '^': + case '$': + return (true); + default: + return (false); + } +#endif +} + +/* + * othercase - return the case counterpart of an alphabetic + * static wint_t othercase(wint_t ch); + */ +static wint_t /* if no counterpart, return ch */ +othercase(wint_t ch) +{ + assert(iswalpha(ch)); + if (iswupper(ch)) + return (towlower(ch)); + else if (iswlower(ch)) + return (towupper(ch)); + else /* peculiar, but could happen */ + return (ch); +} + +/* + * bothcases - emit a dualcase version of a two-case character + * static void bothcases(struct parse *p, wint_t ch); + * + * Boy, is this implementation ever a kludge... + */ +static void +bothcases(struct parse *p, wint_t ch) +{ + const char *oldnext = p->next; + const char *oldend = p->end; + char bracket[3 + MB_LEN_MAX]; + size_t n; + + _DIAGASSERT(p != NULL); + + assert(othercase(ch) != ch); /* p_bracket() would recurse */ + p->next = bracket; +#ifdef NLS + mbstate_t mbs; + memset(&mbs, 0, sizeof (mbs)); + n = wcrtomb(bracket, ch, &mbs); + assert(n != (size_t)-1); +#else + n = 0; + bracket[n++] = ch; +#endif + bracket[n] = ']'; + bracket[n + 1] = '\0'; + p->end = bracket+n+1; + p_bracket(p); + assert(p->next == p->end); + p->next = oldnext; + p->end = oldend; +} + +/* + * ordinary - emit an ordinary character + * static void ordinary(struct parse *p, wint_t ch); + */ +static void +ordinary(struct parse *p, wint_t ch) +{ + cset *cs; + + _DIAGASSERT(p != NULL); + + if ((p->g->cflags®_ICASE) && iswalpha(ch) && othercase(ch) != ch) + bothcases(p, ch); + else if ((wint_t)(ch & OPDMASK) == ch) + EMIT(OCHAR, (size_t)ch); + else { + /* + * Kludge: character is too big to fit into an OCHAR operand. + * Emit a singleton set. + */ + if ((cs = allocset(p)) == NULL) + return; + CHadd(p, cs, ch); + EMIT(OANYOF, (size_t)(cs - p->g->sets)); + } +} + +/* + * nonnewline - emit REG_NEWLINE version of OANY + * static void nonnewline(struct parse *p); + * + * Boy, is this implementation ever a kludge... + */ +static void +nonnewline(struct parse *p) +{ + const char *oldnext = p->next; + const char *oldend = p->end; + char bracket[4]; + + _DIAGASSERT(p != NULL); + + p->next = bracket; + p->end = bracket+3; + bracket[0] = '^'; + bracket[1] = '\n'; + bracket[2] = ']'; + bracket[3] = '\0'; + p_bracket(p); + assert(p->next == bracket+3); + p->next = oldnext; + p->end = oldend; +} + +/* + * repeat - generate code for a bounded repetition, recursively if needed + * static void repeat(struct parse *p, sopno start, int from, int to); + */ +static void +repeat(struct parse *p, + sopno start, /* operand from here to end of strip */ + int from, /* repeated from this number */ + int to) /* to this number of times (maybe INFINITY) */ +{ + sopno finish = HERE(); +#define N 2 +#define INF 3 +#define REP(f, t) ((f)*8 + (t)) +#define MAP(n) (((n) <= 1) ? (n) : ((n) == REGINFINITY) ? INF : N) + sopno copy; + + _DIAGASSERT(p != NULL); + + if (p->error != 0) /* head off possible runaway recursion */ + return; + + assert(from <= to); + + switch (REP(MAP(from), MAP(to))) { + case REP(0, 0): /* must be user doing this */ + DROP(finish-start); /* drop the operand */ + break; + case REP(0, 1): /* as x{1,1}? */ + case REP(0, N): /* as x{1,n}? */ + case REP(0, INF): /* as x{1,}? */ + /* KLUDGE: emit y? as (y|) until subtle bug gets fixed */ + INSERT(OCH_, start); /* offset is wrong... */ + repeat(p, start+1, 1, to); + ASTERN(OOR1, start); + AHEAD(start); /* ... fix it */ + EMIT(OOR2, 0); + AHEAD(THERE()); + ASTERN(O_CH, THERETHERE()); + break; + case REP(1, 1): /* trivial case */ + /* done */ + break; + case REP(1, N): /* as x?x{1,n-1} */ + /* KLUDGE: emit y? as (y|) until subtle bug gets fixed */ + INSERT(OCH_, start); + ASTERN(OOR1, start); + AHEAD(start); + EMIT(OOR2, 0); /* offset very wrong... */ + AHEAD(THERE()); /* ...so fix it */ + ASTERN(O_CH, THERETHERE()); + copy = dupl(p, start+1, finish+1); + assert(copy == finish+4); + repeat(p, copy, 1, to-1); + break; + case REP(1, INF): /* as x+ */ + INSERT(OPLUS_, start); + ASTERN(O_PLUS, start); + break; + case REP(N, N): /* as xx{m-1,n-1} */ + copy = dupl(p, start, finish); + repeat(p, copy, from-1, to-1); + break; + case REP(N, INF): /* as xx{n-1,INF} */ + copy = dupl(p, start, finish); + repeat(p, copy, from-1, to); + break; + default: /* "can't happen" */ + SETERROR(REG_ASSERT); /* just in case */ + break; + } +} + +/* + * wgetnext - helper function for WGETNEXT() macro. Gets the next wide + * character from the parse struct, signals a REG_ILLSEQ error if the + * character can't be converted. Returns the number of bytes consumed. + */ +static wint_t +wgetnext(struct parse *p) +{ +#ifdef NLS + mbstate_t mbs; + wchar_t wc; + size_t n; + + memset(&mbs, 0, sizeof (mbs)); + n = mbrtowc(&wc, p->next, (size_t)(p->end - p->next), &mbs); + if (n == (size_t)-1 || n == (size_t)-2) { + SETERROR(REG_ILLSEQ); + return (0); + } + if (n == 0) + n = 1; + p->next += n; + return (wc); +#else + return (*p->next++); +#endif +} + +/* + * seterr - set an error condition + * static int seterr(struct parse *p, int e); + */ +static int /* useless but makes type checking happy */ +seterr(struct parse *p, int e) +{ + + _DIAGASSERT(p != NULL); + + if (p->error == 0) /* keep earliest error condition */ + p->error = e; + p->next = nuls; /* try to bring things to a halt */ + p->end = nuls; + return (0); /* make the return value well-defined */ +} + +#define MUL_NO_OVERFLOW ((size_t)1 << (sizeof (size_t) * 4)) + +static void * +reallocarray(void *optr, size_t nmemb, size_t size) +{ + if ((nmemb >= MUL_NO_OVERFLOW || size >= MUL_NO_OVERFLOW) && + nmemb > 0 && SIZE_MAX / nmemb < size) { + errno = ENOMEM; + return (NULL); + } + return (realloc(optr, size * nmemb)); +} + +/* + * allocset - allocate a set of characters for [] + * static cset *allocset(struct parse *p); + */ +static cset * +allocset(struct parse *p) +{ + cset *cs, *ncs; + + _DIAGASSERT(p != NULL); + + ncs = reallocarray(p->g->sets, p->g->ncsets + 1, sizeof (*ncs)); + if (ncs == NULL) { + SETERROR(REG_ESPACE); + return (NULL); + } + p->g->sets = ncs; + cs = &p->g->sets[p->g->ncsets++]; + memset(cs, 0, sizeof (*cs)); + + return (cs); +} + +/* + * freeset - free a now-unused set + * static void freeset(struct parse *p, cset *cs); + */ +static void +freeset(struct parse *p, cset *cs) +{ + cset *top; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(cs != NULL); + + top = &p->g->sets[p->g->ncsets]; + + free(cs->wides); + free(cs->ranges); + free(cs->types); + memset(cs, 0, sizeof (*cs)); + if (cs == top-1) /* recover only the easy case */ + p->g->ncsets--; +} + +/* + * singleton - Determine whether a set contains only one character, + * returning it if so, otherwise returning OUT. + */ +static wint_t +singleton(cset *cs) +{ + wint_t i, s, n; + + for (i = n = 0; i < NC; i++) + if (CHIN(cs, i)) { + n++; + s = i; + } + if (n == 1) + return (s); + if (cs->nwides == 1 && cs->nranges == 0 && cs->ntypes == 0 && + cs->icase == 0) + return (cs->wides[0]); + /* Don't bother handling the other cases. */ + return (OUT); +} + +/* + * CHadd - add character to character set. + */ +static void +CHadd(struct parse *p, cset *cs, wint_t ch) +{ + wint_t nch, *newwides; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(cs != NULL); + + assert(ch >= 0); + if (ch < NC) + cs->bmp[(unsigned)ch >> 3] |= 1 << (ch & 7); + else { + newwides = reallocarray(cs->wides, cs->nwides + 1, + sizeof (*cs->wides)); + if (newwides == NULL) { + SETERROR(REG_ESPACE); + return; + } + cs->wides = newwides; + cs->wides[cs->nwides++] = ch; + } + if (cs->icase) { + if ((nch = towlower(ch)) < NC) + cs->bmp[(unsigned)nch >> 3] |= 1 << (nch & 7); + if ((nch = towupper(ch)) < NC) + cs->bmp[(unsigned)nch >> 3] |= 1 << (nch & 7); + } +} + +/* + * CHaddrange - add all characters in the range [min,max] to a character set. + */ +static void +CHaddrange(struct parse *p, cset *cs, wint_t min, wint_t max) +{ + crange *newranges; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(cs != NULL); + + for (; min < NC && min <= max; min++) + CHadd(p, cs, min); + if (min >= max) + return; + newranges = reallocarray(cs->ranges, cs->nranges + 1, + sizeof (*cs->ranges)); + if (newranges == NULL) { + SETERROR(REG_ESPACE); + return; + } + cs->ranges = newranges; + cs->ranges[cs->nranges].min = min; + cs->ranges[cs->nranges].max = max; + cs->nranges++; +} + +/* + * CHaddtype - add all characters of a certain type to a character set. + */ +static void +CHaddtype(struct parse *p, cset *cs, wctype_t wct) +{ + wint_t i; + wctype_t *newtypes; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(cs != NULL); + + for (i = 0; i < NC; i++) + if (iswctype(i, wct)) + CHadd(p, cs, i); + newtypes = reallocarray(cs->types, cs->ntypes + 1, + sizeof (*cs->types)); + if (newtypes == NULL) { + SETERROR(REG_ESPACE); + return; + } + cs->types = newtypes; + cs->types[cs->ntypes++] = wct; +} + +/* + * dupl - emit a duplicate of a bunch of sops + * static sopno dupl(struct parse *p, sopno start, sopno finish); + */ +static sopno /* start of duplicate */ +dupl(struct parse *p, + sopno start, /* from here */ + sopno finish) /* to this less one */ +{ + sopno ret = HERE(); + sopno len = finish - start; + + _DIAGASSERT(p != NULL); + + assert(finish >= start); + if (len == 0) + return (ret); + if (!enlarge(p, p->ssize + len)) /* this many unexpected additions */ + return (ret); + (void) memcpy(p->strip + p->slen, + p->strip + start, len * sizeof (*p->strip)); + p->slen += len; + return (ret); +} + +/* + * doemit - emit a strip operator + * static void doemit(struct parse *p, sop op, size_t opnd); + * + * It might seem better to implement this as a macro with a function as + * hard-case backup, but it's just too big and messy unless there are + * some changes to the data structures. Maybe later. + */ +static void +doemit(struct parse *p, sop op, size_t opnd) +{ + /* avoid making error situations worse */ + if (p->error != 0) + return; + + _DIAGASSERT(p != NULL); + + /* deal with oversize operands ("can't happen", more or less) */ + assert(opnd < 1<slen >= p->ssize) + if (!enlarge(p, (p->ssize+1) / 2 * 3)) /* +50% */ + return; + + /* finally, it's all reduced to the easy case */ + p->strip[p->slen++] = (sopno)SOP(op, opnd); +} + +/* + * doinsert - insert a sop into the strip + * static void doinsert(struct parse *p, sop op, size_t opnd, sopno pos); + */ +static void +doinsert(struct parse *p, sop op, size_t opnd, sopno pos) +{ + sopno sn; + sop s; + int i; + + _DIAGASSERT(p != NULL); + + /* avoid making error situations worse */ + if (p->error != 0) + return; + + sn = HERE(); + EMIT(op, opnd); /* do checks, ensure space */ + assert(HERE() == sn+1); + s = p->strip[sn]; + + /* adjust paren pointers */ + assert(pos > 0); + for (i = 1; i < NPAREN; i++) { + if (p->pbegin[i] >= pos) { + p->pbegin[i]++; + } + if (p->pend[i] >= pos) { + p->pend[i]++; + } + } + + memmove(&p->strip[pos+1], &p->strip[pos], + (HERE()-pos-1)*sizeof (*p->strip)); + p->strip[pos] = s; +} + +/* + * dofwd - complete a forward reference + * static void dofwd(struct parse *p, sopno pos, sop value); + */ +static void +dofwd(struct parse *p, sopno pos, sop value) +{ + + _DIAGASSERT(p != NULL); + + /* avoid making error situations worse */ + if (p->error != 0) + return; + + assert(value < 1<strip[pos] = OP(p->strip[pos]) | value; +} + +/* + * enlarge - enlarge the strip + * static int enlarge(struct parse *p, sopno size); + */ +static int +enlarge(struct parse *p, sopno size) +{ + sop *sp; + + _DIAGASSERT(p != NULL); + + if (p->ssize >= size) + return (1); + + sp = reallocarray(p->strip, size, sizeof (*p->strip)); + if (sp == NULL) { + SETERROR(REG_ESPACE); + return (0); + } + p->strip = sp; + p->ssize = size; + return (1); +} + +/* + * stripsnug - compact the strip + * static void stripsnug(struct parse *p, struct re_guts *g); + */ +static void +stripsnug(struct parse *p, struct re_guts *g) +{ + + _DIAGASSERT(p != NULL); + _DIAGASSERT(g != NULL); + + g->nstates = p->slen; + g->strip = reallocarray(p->strip, p->slen, sizeof (*p->strip)); + if (g->strip == NULL) { + SETERROR(REG_ESPACE); + g->strip = p->strip; + } +} + +/* + * findmust - fill in must and mlen with longest mandatory literal string + * static void findmust(struct parse *p, struct re_guts *g); + * + * This algorithm could do fancy things like analyzing the operands of | + * for common subsequences. Someday. This code is simple and finds most + * of the interesting cases. + * + * Note that must and mlen got initialized during setup. + */ +static void +findmust(struct parse *p, struct re_guts *g) +{ + sop *scan; + sop *start = NULL; + sop *newstart = NULL; + sopno newlen; + sop s; + char *cp; + int offset; + mbstate_t mbs; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(g != NULL); + + /* avoid making error situations worse */ + if (p->error != 0) + return; + +#ifdef notyet + /* + * It's not generally safe to do a ``char'' substring search on + * multibyte character strings, but it's safe for at least + * UTF-8 (see RFC 3629). + */ + if (MB_CUR_MAX > 1 && + strcmp(_CurrentRuneLocale->__encoding, "UTF-8") != 0) + return; +#endif + + /* find the longest OCHAR sequence in strip */ + newlen = 0; + offset = 0; + g->moffset = 0; + scan = g->strip + 1; + do { + s = *scan++; + switch (OP(s)) { + case OCHAR: /* sequence member */ + if (newlen == 0) { /* new sequence */ + memset(&mbs, 0, sizeof (mbs)); + newstart = scan - 1; + } +#ifdef NLS + char buf[MB_LEN_MAX]; + size_t clen = wcrtomb(buf, (int)OPND(s), &mbs); + if (clen == (size_t)-1) + goto toohard; + newlen += (sopno)clen; +#else + newlen++; +#endif + break; + case OPLUS_: /* things that don't break one */ + case OLPAREN: + case ORPAREN: + break; + case OQUEST_: /* things that must be skipped */ + case OCH_: + offset = altoffset(scan, offset); + scan--; + do { + scan += OPND(s); + s = *scan; + /* assert() interferes w debug printouts */ + if (OP(s) != O_QUEST && + OP(s) != O_CH && OP(s) != OOR2) { + g->iflags |= BAD; + return; + } + } while (OP(s) != O_QUEST && OP(s) != O_CH); + /* FALLTHROUGH */ + case OBOW: /* things that break a sequence */ + case OEOW: + case OBOL: + case OEOL: + case OBOS: + case OEOS: + case OWBND: + case ONWBND: + case O_QUEST: + case O_CH: + case OEND: + if (newlen > (sopno)g->mlen) { /* ends one */ + start = newstart; + g->mlen = newlen; + if (offset > -1) { + g->moffset += offset; + offset = newlen; + } else + g->moffset = offset; + } else { + if (offset > -1) + offset += newlen; + } + newlen = 0; + break; + case OANY: + if (newlen > (sopno)g->mlen) { /* ends one */ + start = newstart; + g->mlen = newlen; + if (offset > -1) { + g->moffset += offset; + offset = newlen; + } else + g->moffset = offset; + } else { + if (offset > -1) + offset += newlen; + } + if (offset > -1) + offset++; + newlen = 0; + break; + case OANYOF: /* may or may not invalidate offset */ + /* First, everything as OANY */ + if (newlen > (sopno)g->mlen) { /* ends one */ + start = newstart; + g->mlen = newlen; + if (offset > -1) { + g->moffset += offset; + offset = newlen; + } else + g->moffset = offset; + } else { + if (offset > -1) + offset += newlen; + } + if (offset > -1) + offset++; + newlen = 0; + break; +#ifdef NLS +toohard: /* FALLTHROUGH */ +#endif + default: + /* + * Anything here makes it impossible or too hard + * to calculate the offset -- so we give up; + * save the last known good offset, in case the + * must sequence doesn't occur later. + */ + if (newlen > (sopno)g->mlen) { /* ends one */ + start = newstart; + g->mlen = newlen; + if (offset > -1) + g->moffset += offset; + else + g->moffset = offset; + } + offset = -1; + newlen = 0; + break; + } + } while (OP(s) != OEND); + + if (g->mlen == 0) { /* there isn't one */ + g->moffset = -1; + return; + } + + /* turn it into a character string */ + g->must = malloc((size_t)g->mlen + 1); + if (g->must == NULL) { /* argh; just forget it */ + g->mlen = 0; + g->moffset = -1; + return; + } + cp = g->must; + scan = start; + memset(&mbs, 0, sizeof (mbs)); + while (cp < g->must + g->mlen) { + while (OP(s = *scan++) != OCHAR) + continue; +#ifdef NLS + size_t clen = wcrtomb(cp, (int)OPND(s), &mbs); + assert(clen != (size_t)-1); + cp += clen; +#else + *cp++ = OPND(s); +#endif + } + assert(cp == g->must + g->mlen); + *cp++ = '\0'; /* just on general principles */ +} + +/* + * altoffset - choose biggest offset among multiple choices + * static int altoffset(sop *scan, int offset); + * + * Compute, recursively if necessary, the largest offset among multiple + * re paths. + */ +static int +altoffset(sop *scan, int offset) +{ + int largest; + int try; + sop s; + + _DIAGASSERT(scan != NULL); + + /* If we gave up already on offsets, return */ + if (offset == -1) + return (-1); + + largest = 0; + try = 0; + s = *scan++; + while (OP(s) != O_QUEST && OP(s) != O_CH) { + switch (OP(s)) { + case OOR1: + if (try > largest) + largest = try; + try = 0; + break; + case OQUEST_: + case OCH_: + try = altoffset(scan, try); + if (try == -1) + return (-1); + scan--; + do { + scan += OPND(s); + s = *scan; + if (OP(s) != O_QUEST && + OP(s) != O_CH && OP(s) != OOR2) + return (-1); + } while (OP(s) != O_QUEST && OP(s) != O_CH); + /* + * We must skip to the next position, or we'll + * leave altoffset() too early. + */ + scan++; + break; + case OANYOF: + case OCHAR: + case OANY: + try++; + /*FALLTHROUGH*/ + case OBOW: + case OEOW: + case OWBND: + case ONWBND: + case OLPAREN: + case ORPAREN: + case OOR2: + break; + default: + try = -1; + break; + } + if (try == -1) + return (-1); + s = *scan++; + } + + if (try > largest) + largest = try; + + return (largest + offset); +} + +/* + * computejumps - compute char jumps for BM scan + * static void computejumps(struct parse *p, struct re_guts *g); + * + * This algorithm assumes g->must exists and is has size greater than + * zero. It's based on the algorithm found on Computer Algorithms by + * Sara Baase. + * + * A char jump is the number of characters one needs to jump based on + * the value of the character from the text that was mismatched. + */ +static void +computejumps(struct parse *p, struct re_guts *g) +{ + int ch; + size_t mindex; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(g != NULL); + + /* Avoid making errors worse */ + if (p->error != 0) + return; + + g->charjump = calloc((NC_MAX + 1), sizeof (*g->charjump)); + if (g->charjump == NULL) /* Not a fatal error */ + return; + /* Adjust for signed chars, if necessary */ + g->charjump = &g->charjump[-(CHAR_MIN)]; + + /* + * If the character does not exist in the pattern, the jump + * is equal to the number of characters in the pattern. + */ + for (ch = CHAR_MIN; ch < (CHAR_MAX + 1); ch++) + g->charjump[ch] = g->mlen; + + /* + * If the character does exist, compute the jump that would + * take us to the last character in the pattern equal to it + * (notice that we match right to left, so that last character + * is the first one that would be matched). + */ + for (mindex = 0; mindex < g->mlen; mindex++) + g->charjump[(int)g->must[mindex]] = g->mlen - mindex - 1; +} + +/* + * computematchjumps - compute match jumps for BM scan + * static void computematchjumps(struct parse *p, struct re_guts *g); + * + * This algorithm assumes g->must exists and is has size greater than + * zero. It's based on the algorithm found on Computer Algorithms by + * Sara Baase. + * + * A match jump is the number of characters one needs to advance based + * on the already-matched suffix. + * Notice that all values here are minus (g->mlen-1), because of the way + * the search algorithm works. + */ +static void +computematchjumps(struct parse *p, struct re_guts *g) +{ + size_t mindex; /* General "must" iterator */ + size_t suffix; /* Keeps track of matching suffix */ + size_t ssuffix; /* Keeps track of suffixes' suffix */ + /* + * pmatches[k] points to the next i + * such that i+1...mlen is a substring + * of k+1...k+mlen-i-1 + */ + size_t *pmatches; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(g != NULL); + + /* Avoid making errors worse */ + if (p->error != 0) + return; + + pmatches = calloc(g->mlen, sizeof (*pmatches)); + if (pmatches == NULL) { + g->matchjump = NULL; + return; + } + + g->matchjump = calloc(g->mlen, sizeof (*g->matchjump)); + if (g->matchjump == NULL) { /* Not a fatal error */ + free(pmatches); + return; + } + + /* Set maximum possible jump for each character in the pattern */ + for (mindex = 0; mindex < g->mlen; mindex++) + g->matchjump[mindex] = 2 * g->mlen - mindex - 1; + + /* Compute pmatches[] */ + for (suffix = mindex = g->mlen; mindex-- > 0; suffix--) { + pmatches[mindex] = suffix; + + /* + * If a mismatch is found, interrupting the substring, + * compute the matchjump for that position. If no + * mismatch is found, then a text substring mismatched + * against the suffix will also mismatch against the + * substring. + */ + while (suffix < g->mlen && + g->must[mindex] != g->must[suffix]) { + g->matchjump[suffix] = MIN(g->matchjump[suffix], + g->mlen - mindex - 1); + suffix = pmatches[suffix]; + } + } + + /* + * Compute the matchjump up to the last substring found to jump + * to the beginning of the largest must pattern prefix matching + * it's own suffix. + */ + for (mindex = 0; mindex <= suffix; mindex++) + g->matchjump[mindex] = MIN(g->matchjump[mindex], + g->mlen + suffix - mindex); + + ssuffix = pmatches[suffix]; + while (suffix < g->mlen) { + while (suffix <= ssuffix && suffix < g->mlen) { + g->matchjump[suffix] = MIN(g->matchjump[suffix], + g->mlen + ssuffix - suffix); + suffix++; + } + if (suffix < g->mlen) + ssuffix = pmatches[ssuffix]; + } + + free(pmatches); +} + +/* + * pluscount - count + nesting + * static sopno pluscount(struct parse *p, struct re_guts *g); + */ +static sopno /* nesting depth */ +pluscount(struct parse *p, struct re_guts *g) +{ + sop *scan; + sop s; + sopno plusnest = 0; + sopno maxnest = 0; + + _DIAGASSERT(p != NULL); + _DIAGASSERT(g != NULL); + + if (p->error != 0) + return (0); /* there may not be an OEND */ + + scan = g->strip + 1; + do { + s = *scan++; + switch (OP(s)) { + case OPLUS_: + plusnest++; + break; + case O_PLUS: + if (plusnest > maxnest) + maxnest = plusnest; + plusnest--; + break; + } + } while (OP(s) != OEND); + if (plusnest != 0) + g->iflags |= BAD; + return (maxnest); +} diff --git a/lib/os/windows/libregex/regerror.c b/lib/os/windows/libregex/regerror.c new file mode 100644 index 000000000000..8bfc15ed38ee --- /dev/null +++ b/lib/os/windows/libregex/regerror.c @@ -0,0 +1,180 @@ +/* $NetBSD: regerror.c,v 1.25 2021/02/26 19:24:47 christos Exp $ */ + +/* + * SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 1992, 1993, 1994 Henry Spencer. + * Copyright (c) 1992, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)regerror.c 8.4 (Berkeley) 3/20/94 + */ + +#if HAVE_NBTOOL_CONFIG_H +#include "nbtool_config.h" +#endif + +// #include "namespace.h" +#include +#include +#include +#include +#include +#include + +#include "utils.h" + +#ifdef __weak_alias +__weak_alias(regerror, _regerror) +#endif + +/* ========= begin header generated by ./mkh ========= */ +#ifdef __cplusplus +extern "C" { +#endif + +/* === regerror.c === */ +static const char *regatoi(const regex_t *preg, char *localbufm, size_t buflen); + +#ifdef __cplusplus +} +#endif +/* ========= end header generated by ./mkh ========= */ +/* + * #define REG_NOMATCH 1 + * #define REG_BADPAT 2 + * #define REG_ECOLLATE 3 + * #define REG_ECTYPE 4 + * #define REG_EESCAPE 5 + * #define REG_ESUBREG 6 + * #define REG_EBRACK 7 + * #define REG_EPAREN 8 + * #define REG_EBRACE 9 + * #define REG_BADBR 10 + * #define REG_ERANGE 11 + * #define REG_ESPACE 12 + * #define REG_BADRPT 13 + * #define REG_EMPTY 14 + * #define REG_ASSERT 15 + * #define REG_INVARG 16 + * #define REG_ENOSYS 17 + * #define REG_ILLSEQ 18 + * #define REG_ATOI 255 // convert name to number (!) + * #define REG_ITOA 0400 // convert number to name (!) + */ + +static const struct rerr { + int code; + const char *name; + const char *explain; +} rerrs[] = { + {REG_NOMATCH, "REG_NOMATCH", "regexec() failed to match"}, + {REG_BADPAT, "REG_BADPAT", "invalid regular expression"}, + {REG_ECOLLATE, "REG_ECOLLATE", "invalid collating element"}, + {REG_ECTYPE, "REG_ECTYPE", "invalid character class"}, + {REG_EESCAPE, "REG_EESCAPE", "trailing backslash (\\)"}, + {REG_ESUBREG, "REG_ESUBREG", "invalid backreference number"}, + {REG_EBRACK, "REG_EBRACK", "brackets ([ ]) not balanced"}, + {REG_EPAREN, "REG_EPAREN", "parentheses not balanced"}, + {REG_EBRACE, "REG_EBRACE", "braces not balanced"}, + {REG_BADBR, "REG_BADBR", "invalid repetition count(s)"}, + {REG_ERANGE, "REG_ERANGE", "invalid character range"}, + {REG_ESPACE, "REG_ESPACE", "out of memory"}, + {REG_BADRPT, "REG_BADRPT", "repetition-operator operand invalid"}, + {REG_EMPTY, "REG_EMPTY", "empty (sub)expression"}, + {REG_ASSERT, "REG_ASSERT", "\"can't happen\" -- you found a bug"}, + {REG_INVARG, "REG_INVARG", "invalid argument to regex routine"}, + {REG_ILLSEQ, "REG_ILLSEQ", "illegal byte sequence"}, + {0, "", "*** unknown regexp error code ***"} +}; + +/* + * regerror - the interface to error numbers + * extern size_t regerror(int, const regex_t *, char *, size_t); + */ + +size_t +regerror(int errcode, + const regex_t *__restrict preg, + char *__restrict errbuf, + size_t errbuf_size) +{ + const struct rerr *r; + size_t len; + int target = errcode &~ REG_ITOA; + const char *s; + char convbuf[50]; + + _DIAGASSERT(errcode != REG_ATOI || preg != NULL); + _DIAGASSERT(errbuf != NULL); + + if (errcode == REG_ATOI) { + s = regatoi(preg, convbuf, sizeof (convbuf)); + } else { + for (r = rerrs; r->code != 0; r++) + if (r->code == target) + break; + + if (errcode®_ITOA) { + if (r->code != 0) + strlcpy(convbuf, r->name, sizeof (convbuf)); + else + snprintf(convbuf, sizeof (convbuf), "REG_0x%x", + target); + s = convbuf; + } else + s = r->explain; + } + + len = strlen(s) + 1; + if (errbuf_size > 0) + (void) strlcpy(errbuf, s, errbuf_size); + + return (len); +} + +/* + * regatoi - internal routine to implement REG_ATOI + * static char *regatoi(const regex_t *preg, char *localbuf); + */ +static const char * +regatoi(const regex_t *preg, char *localbuf, size_t buflen) +{ + const struct rerr *r; + + for (r = rerrs; r->code != 0; r++) + if (strcmp(r->name, preg->re_endp) == 0) + break; + if (r->code == 0) + return ("0"); + + snprintf(localbuf, buflen, "%d", r->code); + return (localbuf); +} diff --git a/lib/os/windows/libregex/regex.3 b/lib/os/windows/libregex/regex.3 new file mode 100644 index 000000000000..2f344496ddac --- /dev/null +++ b/lib/os/windows/libregex/regex.3 @@ -0,0 +1,861 @@ +.\" $NetBSD: regex.3,v 1.32 2021/03/11 18:49:18 christos Exp $ +.\" +.\" Copyright (c) 1992, 1993, 1994 Henry Spencer. +.\" Copyright (c) 1992, 1993, 1994 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" This code is derived from software contributed to Berkeley by +.\" Henry Spencer. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)regex.3 8.4 (Berkeley) 3/20/94 +.\" $FreeBSD: head/lib/libc/regex/regex.3 363817 2020-08-04 02:06:49Z kevans $ +.\" +.Dd March 11, 2021 +.Dt REGEX 3 +.Os +.Sh NAME +.Nm regcomp , +.Nm regexec , +.Nm regerror , +.Nm regfree , +.Nm regasub , +.Nm regnsub +.Nd regular-expression library +.Sh LIBRARY +.Lb libc +.Sh SYNOPSIS +.In regex.h +.Ft int +.Fo regcomp +.Fa "regex_t * restrict preg" "const char * restrict pattern" "int cflags" +.Fc +.Ft int +.Fo regexec +.Fa "const regex_t * restrict preg" "const char * restrict string" +.Fa "size_t nmatch" "regmatch_t pmatch[restrict]" "int eflags" +.Fc +.Ft size_t +.Fo regerror +.Fa "int errcode" "const regex_t * restrict preg" +.Fa "char * restrict errbuf" "size_t errbuf_size" +.Fc +.Ft void +.Fn regfree "regex_t *preg" +.Ft ssize_t +.Fn regnsub "char *buf" "size_t bufsiz" "const char *sub" "const regmatch_t *rm" "const char *str" +.Ft ssize_t +.Fn regasub "char **buf" "const char *sub" "const regmatch_t *rm" "const char *sstr" +.Sh DESCRIPTION +These routines implement +.St -p1003.2 +regular expressions +.Pq Do RE Dc Ns s ; +see +.Xr re_format 7 . +The +.Fn regcomp +function +compiles an RE written as a string into an internal form, +.Fn regexec +matches that internal form against a string and reports results, +.Fn regerror +transforms error codes from either into human-readable messages, +and +.Fn regfree +frees any dynamically-allocated storage used by the internal form +of an RE. +.Pp +The header +.In regex.h +declares two structure types, +.Ft regex_t +and +.Ft regmatch_t , +the former for compiled internal forms and the latter for match reporting. +It also declares the four functions, +a type +.Ft regoff_t , +and a number of constants with names starting with +.Dq Dv REG_ . +.Pp +The +.Fn regcomp +function +compiles the regular expression contained in the +.Fa pattern +string, +subject to the flags in +.Fa cflags , +and places the results in the +.Ft regex_t +structure pointed to by +.Fa preg . +The +.Fa cflags +argument +is the bitwise OR of zero or more of the following flags: +.Bl -tag -width REG_EXTENDED +.It Dv REG_EXTENDED +Compile modern +.Pq Dq extended +REs, +rather than the obsolete +.Pq Dq basic +REs that +are the default. +.It Dv REG_BASIC +This is a synonym for 0, +provided as a counterpart to +.Dv REG_EXTENDED +to improve readability. +.It Dv REG_NOSPEC +Compile with recognition of all special characters turned off. +All characters are thus considered ordinary, +so the +.Dq RE +is a literal string. +This is an extension, +compatible with but not specified by +.St -p1003.2 , +and should be used with +caution in software intended to be portable to other systems. +.Dv REG_EXTENDED +and +.Dv REG_NOSPEC +may not be used +in the same call to +.Fn regcomp . +.It Dv REG_ICASE +Compile for matching that ignores upper/lower case distinctions. +See +.Xr re_format 7 . +.It Dv REG_NOSUB +Compile for matching that need only report success or failure, +not what was matched. +.It Dv REG_NEWLINE +Compile for newline-sensitive matching. +By default, newline is a completely ordinary character with no special +meaning in either REs or strings. +With this flag, +.Ql [^ +bracket expressions and +.Ql .\& +never match newline, +a +.Ql ^\& +anchor matches the null string after any newline in the string +in addition to its normal function, +and the +.Ql $\& +anchor matches the null string before any newline in the +string in addition to its normal function. +.It Dv REG_PEND +The regular expression ends, +not at the first NUL, +but just before the character pointed to by the +.Va re_endp +member of the structure pointed to by +.Fa preg . +The +.Va re_endp +member is of type +.Ft "const char *" . +This flag permits inclusion of NULs in the RE; +they are considered ordinary characters. +This is an extension, +compatible with but not specified by +.St -p1003.2 , +and should be used with +caution in software intended to be portable to other systems. +.It Dv REG_GNU +Include GNU-inspired extensions: +.Pp +.Bl -tag -offset indent -width XX -compact +.It \eN +Use backreference +.Dv N +where +.Dv N +is a single digit number between +.Dv 1 +and +.Dv 9 . +.It \ea +Visual Bell +.It \eb +Match a position that is a word boundary. +.It \eB +Match a position that is a not word boundary. +.It \ef +Form Feed +.It \en +Line Feed +.It \er +Carriage return +.It \es +Alias for [[:space:]] +.It \eS +Alias for [^[:space:]] +.It \et +Horizontal Tab +.It \ev +Vertical Tab +.It \ew +Alias for [[:alnum:]_] +.It \eW +Alias for [^[:alnum:]_] +.It \e' +Matches the end of the subject string (the string to be matched). +.It \e` +Matches the beginning of the subject string. +.El +.Pp +This is an extension, +compatible with but not specified by +.St -p1003.2 , +and should be used with +caution in software intended to be portable to other systems. +.El +.Pp +When successful, +.Fn regcomp +returns 0 and fills in the structure pointed to by +.Fa preg . +One member of that structure +(other than +.Va re_endp ) +is publicized: +.Va re_nsub , +of type +.Ft size_t , +contains the number of parenthesized subexpressions within the RE +(except that the value of this member is undefined if the +.Dv REG_NOSUB +flag was used). +If +.Fn regcomp +fails, it returns a non-zero error code; +see +.Sx DIAGNOSTICS . +.Pp +The +.Fn regexec +function +matches the compiled RE pointed to by +.Fa preg +against the +.Fa string , +subject to the flags in +.Fa eflags , +and reports results using +.Fa nmatch , +.Fa pmatch , +and the returned value. +The RE must have been compiled by a previous invocation of +.Fn regcomp . +The compiled form is not altered during execution of +.Fn regexec , +so a single compiled RE can be used simultaneously by multiple threads. +.Pp +By default, +the NUL-terminated string pointed to by +.Fa string +is considered to be the text of an entire line, minus any terminating +newline. +The +.Fa eflags +argument is the bitwise OR of zero or more of the following flags: +.Bl -tag -width REG_STARTEND +.It Dv REG_NOTBOL +The first character of the string is treated as the continuation +of a line. +This means that the anchors +.Ql ^\& , +.Ql [[:<:]] , +and +.Ql \e< +do not match before it; but see +.Dv REG_STARTEND +below. +This does not affect the behavior of newlines under +.Dv REG_NEWLINE . +.It Dv REG_NOTEOL +The NUL terminating +the string +does not end a line, so the +.Ql $\& +anchor does not match before it. +This does not affect the behavior of newlines under +.Dv REG_NEWLINE . +.It Dv REG_STARTEND +The string is considered to start at +.Fa string No + +.Fa pmatch Ns [0]. Ns Fa rm_so +and to end before the byte located at +.Fa string No + +.Fa pmatch Ns [0]. Ns Fa rm_eo , +regardless of the value of +.Fa nmatch . +See below for the definition of +.Fa pmatch +and +.Fa nmatch . +This is an extension, +compatible with but not specified by +.St -p1003.2 , +and should be used with +caution in software intended to be portable to other systems. +.Pp +Without +.Dv REG_NOTBOL , +the position +.Fa rm_so +is considered the beginning of a line, such that +.Ql ^ +matches before it, and the beginning of a word if there is a word +character at this position, such that +.Ql [[:<:]] +and +.Ql \e< +match before it. +.Pp +With +.Dv REG_NOTBOL , +the character at position +.Fa rm_so +is treated as the continuation of a line, and if +.Fa rm_so +is greater than 0, the preceding character is taken into consideration. +If the preceding character is a newline and the regular expression was compiled +with +.Dv REG_NEWLINE , +.Ql ^ +matches before the string; if the preceding character is not a word character +but the string starts with a word character, +.Ql [[:<:]] +and +.Ql \e< +match before the string. +.El +.Pp +See +.Xr re_format 7 +for a discussion of what is matched in situations where an RE or a +portion thereof could match any of several substrings of +.Fa string . +.Pp +Normally, +.Fn regexec +returns 0 for success and the non-zero code +.Dv REG_NOMATCH +for failure. +Other non-zero error codes may be returned in exceptional situations; +see +.Sx DIAGNOSTICS . +.Pp +If +.Dv REG_NOSUB +was specified in the compilation of the RE, +or if +.Fa nmatch +is 0, +.Fn regexec +ignores the +.Fa pmatch +argument (but see below for the case where +.Dv REG_STARTEND +is specified). +Otherwise, +.Fa pmatch +points to an array of +.Fa nmatch +structures of type +.Ft regmatch_t . +Such a structure has at least the members +.Va rm_so +and +.Va rm_eo , +both of type +.Ft regoff_t +(a signed arithmetic type at least as large as an +.Ft off_t +and a +.Ft ssize_t ) , +containing respectively the offset of the first character of a substring +and the offset of the first character after the end of the substring. +Offsets are measured from the beginning of the +.Fa string +argument given to +.Fn regexec . +An empty substring is denoted by equal offsets, +both indicating the character following the empty substring. +.Pp +The 0th member of the +.Fa pmatch +array is filled in to indicate what substring of +.Fa string +was matched by the entire RE. +Remaining members report what substring was matched by parenthesized +subexpressions within the RE; +member +.Va i +reports subexpression +.Va i , +with subexpressions counted (starting at 1) by the order of their opening +parentheses in the RE, left to right. +Unused entries in the array (corresponding either to subexpressions that +did not participate in the match at all, or to subexpressions that do not +exist in the RE (that is, +.Va i +> +.Fa preg Ns -> Ns Va re_nsub ) ) +have both +.Va rm_so +and +.Va rm_eo +set to -1. +If a subexpression participated in the match several times, +the reported substring is the last one it matched. +(Note, as an example in particular, that when the RE +.Ql "(b*)+" +matches +.Ql bbb , +the parenthesized subexpression matches each of the three +.So Li b Sc Ns s +and then +an infinite number of empty strings following the last +.Ql b , +so the reported substring is one of the empties.) +.Pp +If +.Dv REG_STARTEND +is specified, +.Fa pmatch +must point to at least one +.Ft regmatch_t +(even if +.Fa nmatch +is 0 or +.Dv REG_NOSUB +was specified), +to hold the input offsets for +.Dv REG_STARTEND . +Use for output is still entirely controlled by +.Fa nmatch ; +if +.Fa nmatch +is 0 or +.Dv REG_NOSUB +was specified, +the value of +.Fa pmatch Ns [0] +will not be changed by a successful +.Fn regexec . +.Pp +The +.Fn regerror +function +maps a non-zero +.Fa errcode +from either +.Fn regcomp +or +.Fn regexec +to a human-readable, printable message. +If +.Fa preg +is +.No non\- Ns Dv NULL , +the error code should have arisen from use of +the +.Ft regex_t +pointed to by +.Fa preg , +and if the error code came from +.Fn regcomp , +it should have been the result from the most recent +.Fn regcomp +using that +.Ft regex_t . +The +.Po +.Fn regerror +may be able to supply a more detailed message using information +from the +.Ft regex_t . +.Pc +The +.Fn regerror +function +places the NUL-terminated message into the buffer pointed to by +.Fa errbuf , +limiting the length (including the NUL) to at most +.Fa errbuf_size +bytes. +If the whole message will not fit, +as much of it as will fit before the terminating NUL is supplied. +In any case, +the returned value is the size of buffer needed to hold the whole +message (including terminating NUL). +If +.Fa errbuf_size +is 0, +.Fa errbuf +is ignored but the return value is still correct. +.Pp +If the +.Fa errcode +given to +.Fn regerror +is first ORed with +.Dv REG_ITOA , +the +.Dq message +that results is the printable name of the error code, +e.g.\& +.Dq Dv REG_NOMATCH , +rather than an explanation thereof. +If +.Fa errcode +is +.Dv REG_ATOI , +then +.Fa preg +shall be +.No non\- Ns Dv NULL +and the +.Va re_endp +member of the structure it points to +must point to the printable name of an error code; +in this case, the result in +.Fa errbuf +is the decimal digits of +the numeric value of the error code +(0 if the name is not recognized). +.Dv REG_ITOA +and +.Dv REG_ATOI +are intended primarily as debugging facilities; +they are extensions, +compatible with but not specified by +.St -p1003.2 , +and should be used with +caution in software intended to be portable to other systems. +Be warned also that they are considered experimental and changes are possible. +.Pp +The +.Fn regfree +function +frees any dynamically-allocated storage associated with the compiled RE +pointed to by +.Fa preg . +The remaining +.Ft regex_t +is no longer a valid compiled RE +and the effect of supplying it to +.Fn regexec +or +.Fn regerror +is undefined. +.Pp +None of these functions references global variables except for tables +of constants; +all are safe for use from multiple threads if the arguments are safe. +.Pp +The +.Fn regnsub +and +.Fn regasub +functions perform substitutions using +.Xr sed 1 +like syntax. +They return the length of the string that would have been created +if there was enough space or +.Dv \-1 +on error, setting +.Dv errno . +The result +is being placed in +.Fa buf +which is user-supplied in +.Fn regnsub +and dynamically allocated in +.Fn regasub . +The +.Fa sub +argument contains a substitution string which might refer to the first +9 regular expression strings using +.Dq \e +to refer to the nth matched +item, or +.Dq & +(which is equivalent to +.Dq \e0 ) +to refer to the full match. +The +.Fa rm +array must be at least 10 elements long, and should contain the result +of the matches from a previous +.Fn regexec +call. +Only 10 elements of the +.Fa rm +array can be used. +The +.Fa str +argument contains the source string to apply the transformation to. +.Sh IMPLEMENTATION CHOICES +There are a number of decisions that +.St -p1003.2 +leaves up to the implementor, +either by explicitly saying +.Dq undefined +or by virtue of them being +forbidden by the RE grammar. +This implementation treats them as follows. +.Pp +See +.Xr re_format 7 +for a discussion of the definition of case-independent matching. +.Pp +There is no particular limit on the length of REs, +except insofar as memory is limited. +Memory usage is approximately linear in RE size, and largely insensitive +to RE complexity, except for bounded repetitions. +See +.Sx BUGS +for one short RE using them +that will run almost any system out of memory. +.Pp +A backslashed character other than one specifically given a magic meaning +by +.St -p1003.2 +(such magic meanings occur only in obsolete +.Bq Dq basic +REs) +is taken as an ordinary character. +.Pp +Any unmatched +.Ql [\& +is a +.Dv REG_EBRACK +error. +.Pp +Equivalence classes cannot begin or end bracket-expression ranges. +The endpoint of one range cannot begin another. +.Pp +.Dv RE_DUP_MAX , +the limit on repetition counts in bounded repetitions, is 255. +.Pp +A repetition operator +.Ql ( ?\& , +.Ql *\& , +.Ql +\& , +or bounds) +cannot follow another +repetition operator. +A repetition operator cannot begin an expression or subexpression +or follow +.Ql ^\& +or +.Ql |\& . +.Pp +.Ql |\& +cannot appear first or last in a (sub)expression or after another +.Ql |\& , +i.e., an operand of +.Ql |\& +cannot be an empty subexpression. +An empty parenthesized subexpression, +.Ql "()" , +is legal and matches an +empty (sub)string. +An empty string is not a legal RE. +.Pp +A +.Ql {\& +followed by a digit is considered the beginning of bounds for a +bounded repetition, which must then follow the syntax for bounds. +A +.Ql {\& +.Em not +followed by a digit is considered an ordinary character. +.Pp +.Ql ^\& +and +.Ql $\& +beginning and ending subexpressions in obsolete +.Pq Dq basic +REs are anchors, not ordinary characters. +.Sh DIAGNOSTICS +Non-zero error codes from +.Fn regcomp +and +.Fn regexec +include the following: +.Pp +.Bl -tag -width REG_ECOLLATE -compact +.It Dv REG_NOMATCH +The +.Fn regexec +function +failed to match +.It Dv REG_BADPAT +invalid regular expression +.It Dv REG_ECOLLATE +invalid collating element +.It Dv REG_ECTYPE +invalid character class +.It Dv REG_EESCAPE +.Ql \e +applied to unescapable character +.It Dv REG_ESUBREG +invalid backreference number +.It Dv REG_EBRACK +brackets +.Ql "[ ]" +not balanced +.It Dv REG_EPAREN +parentheses +.Ql "( )" +not balanced +.It Dv REG_EBRACE +braces +.Ql "{ }" +not balanced +.It Dv REG_BADBR +invalid repetition count(s) in +.Ql "{ }" +.It Dv REG_ERANGE +invalid character range in +.Ql "[ ]" +.It Dv REG_ESPACE +ran out of memory +.It Dv REG_BADRPT +.Ql ?\& , +.Ql *\& , +or +.Ql +\& +operand invalid +.It Dv REG_EMPTY +empty (sub)expression +.It Dv REG_ASSERT +cannot happen - you found a bug +.It Dv REG_INVARG +invalid argument, e.g.\& negative-length string +.It Dv REG_ILLSEQ +illegal byte sequence (bad multibyte character) +.El +.Sh SEE ALSO +.Xr grep 1 , +.Xr re_format 7 +.Pp +.St -p1003.2 , +sections 2.8 (Regular Expression Notation) +and +B.5 (C Binding for Regular Expression Matching). +.Sh HISTORY +Originally written by +.An Henry Spencer . +Altered for inclusion in the +.Bx 4.4 +distribution. +.Pp +The +.Fn regnsub +and +.Fn regasub +functions appeared in +.Nx 8 . +.Sh BUGS +This is an alpha release with known defects. +Please report problems. +.Pp +The back-reference code is subtle and doubts linger about its correctness +in complex cases. +.Pp +The +.Fn regexec +function +performance is poor. +This will improve with later releases. +The +.Fa nmatch +argument +exceeding 0 is expensive; +.Fa nmatch +exceeding 1 is worse. +The +.Fn regexec +function +is largely insensitive to RE complexity +.Em except +that back +references are massively expensive. +RE length does matter; in particular, there is a strong speed bonus +for keeping RE length under about 30 characters, +with most special characters counting roughly double. +.Pp +The +.Fn regcomp +function +implements bounded repetitions by macro expansion, +which is costly in time and space if counts are large +or bounded repetitions are nested. +An RE like, say, +.Ql "((((a{1,100}){1,100}){1,100}){1,100}){1,100}" +will (eventually) run almost any existing machine out of swap space. +.Pp +There are suspected problems with response to obscure error conditions. +Notably, +certain kinds of internal overflow, +produced only by truly enormous REs or by multiply nested bounded repetitions, +are probably not handled well. +.Pp +Due to a mistake in +.St -p1003.2 , +things like +.Ql "a)b" +are legal REs because +.Ql )\& +is +a special character only in the presence of a previous unmatched +.Ql (\& . +This cannot be fixed until the spec is fixed. +.Pp +The standard's definition of back references is vague. +For example, does +.Ql "a\e(\e(b\e)*\e2\e)*d" +match +.Ql "abbbd" ? +Until the standard is clarified, +behavior in such cases should not be relied on. +.Pp +The implementation of word-boundary matching is a bit of a kludge, +and bugs may lurk in combinations of word-boundary matching and anchoring. +.Pp +Word-boundary matching does not work properly in multibyte locales. diff --git a/lib/os/windows/libregex/regex2.h b/lib/os/windows/libregex/regex2.h new file mode 100644 index 000000000000..54f57437c84f --- /dev/null +++ b/lib/os/windows/libregex/regex2.h @@ -0,0 +1,207 @@ +/* $NetBSD: regex2.h,v 1.15 2021/02/24 18:13:21 christos Exp $ */ + +/* + * SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 1992, 1993, 1994 Henry Spencer. + * Copyright (c) 1992, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)regex2.h 8.4 (Berkeley) 3/20/94 + * $FreeBSD: head/lib/libc/regex/regex2.h 368359 2020-12-05 03:18:48Z kevans $ + */ + +/* + * First, the stuff that ends up in the outside-world include file + * typedef off_t regoff_t; + * typedef struct { + * int re_magic; + * size_t re_nsub; // number of parenthesized subexpressions + * const char *re_endp; // end pointer for REG_PEND + * struct re_guts *re_g; // none of your business :-) + * } regex_t; + * typedef struct { + * regoff_t rm_so; // start of match + * regoff_t rm_eo; // end of match + * } regmatch_t; + */ +/* + * internals of regex_t + */ +#define MAGIC1 ((('r'^0200)<<8) | 'e') + +/* + * The internal representation is a *strip*, a sequence of + * operators ending with an endmarker. (Some terminology etc. is a + * historical relic of earlier versions which used multiple strips.) + * Certain oddities in the representation are there to permit running + * the machinery backwards; in particular, any deviation from sequential + * flow must be marked at both its source and its destination. Some + * fine points: + * + * - OPLUS_ and O_PLUS are *inside* the loop they create. + * - OQUEST_ and O_QUEST are *outside* the bypass they create. + * - OCH_ and O_CH are *outside* the multi-way branch they create, while + * OOR1 and OOR2 are respectively the end and the beginning of one of + * the branches. Note that there is an implicit OOR2 following OCH_ + * and an implicit OOR1 preceding O_CH. + * + * In state representations, an operator's bit is on to signify a state + * immediately *preceding* "execution" of that operator. + */ +typedef uint32_t sop; /* strip operator */ +typedef uint32_t sopno; +#define OPRMASK 0xf8000000U +#define OPDMASK 0x07ffffffU +#define OPSHIFT (27U) +#define OP(n) ((n)&OPRMASK) +#define OPND(n) ((n)&OPDMASK) +#define SOP(op, opnd) ((op)|(opnd)) +/* operators meaning operand */ +/* (back, fwd are offsets) */ +#define OEND (1U<= 0); + if (ch < NC) + return (((cs->bmp[(unsigned)ch >> 3] & (1 << (ch & 7))) != 0) ^ + cs->invert); + for (i = 0; i < cs->nwides; i++) { + if (cs->icase) { + if (ch == towlower(cs->wides[i]) || + ch == towupper(cs->wides[i])) + return (!cs->invert); + } else if (ch == cs->wides[i]) + return (!cs->invert); + } + for (i = 0; i < cs->nranges; i++) + if (cs->ranges[i].min <= ch && ch <= cs->ranges[i].max) + return (!cs->invert); + for (i = 0; i < cs->ntypes; i++) + if (iswctype(ch, cs->types[i])) + return (!cs->invert); + return (cs->invert); +} + +static __inline int +CHIN(cset *cs, wint_t ch) +{ + + assert(ch >= 0); + if (ch < NC) + return (((cs->bmp[(unsigned)ch >> 3] & (1 << (ch & 7))) != 0) ^ + cs->invert); + else if (cs->icase) + return (CHIN1(cs, ch) || CHIN1(cs, towlower(ch)) || + CHIN1(cs, towupper(ch))); + else + return (CHIN1(cs, ch)); +} + +/* + * main compiled-expression structure + */ +struct re_guts { + int magic; +#define MAGIC2 ((('R'^0200)<<8)|'E') + sop *strip; /* malloced area for strip */ + size_t ncsets; /* number of csets in use */ + cset *sets; /* -> cset [ncsets] */ + int cflags; /* copy of regcomp() cflags argument */ + sopno nstates; /* = number of sops */ + sopno firststate; /* the initial OEND (normally 0) */ + sopno laststate; /* the final OEND */ + int iflags; /* internal flags */ +#define USEBOL 01 /* used ^ */ +#define USEEOL 02 /* used $ */ +#define BAD 04 /* something wrong */ + size_t nbol; /* number of ^ used */ + size_t neol; /* number of $ used */ + char *must; /* match must contain this string */ + int moffset; /* latest point at which must may be located */ + size_t *charjump; /* Boyer-Moore char jump table */ + size_t *matchjump; /* Boyer-Moore match jump table */ + size_t mlen; /* length of must */ + size_t nsub; /* copy of re_nsub */ + int backrefs; /* does it use back references? */ + sopno nplus; /* how deep does it nest +s? */ +}; + +/* misc utilities */ +#undef OUT +#define OUT (CHAR_MIN - 1) /* a non-character value */ +#define IGN (CHAR_MIN - 2) +#define ISWORD(c) (iswalnum((uch)(c)) || (c) == '_') diff --git a/lib/os/windows/libregex/regexec.c b/lib/os/windows/libregex/regexec.c new file mode 100644 index 000000000000..bb48316f1884 --- /dev/null +++ b/lib/os/windows/libregex/regexec.c @@ -0,0 +1,248 @@ +/* $NetBSD: regexec.c,v 1.26 2021/02/26 19:24:47 christos Exp $ */ + +/* + * SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 1992, 1993, 1994 Henry Spencer. + * Copyright (c) 1992, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)regexec.c 8.3 (Berkeley) 3/20/94 + */ + +#if HAVE_NBTOOL_CONFIG_H +#include "nbtool_config.h" +#endif + +/* + * the outer shell of regexec() + * + * This file includes engine.c three times, after muchos fiddling with the + * macros that code uses. This lets the same code operate on two different + * representations for state sets and characters. + */ + +#ifndef LIBHACK +// #include "namespace.h" +#endif +#include +#include +#include +#include +#include +#include +#include + +#if defined(__weak_alias) && !defined(LIBHACK) +__weak_alias(regexec, _regexec) +#endif + +#include "utils.h" +#include "regex2.h" + +static __inline size_t +xmbrtowc(wint_t *wi, const char *s, size_t n, mbstate_t *mbs, wint_t dummy) +{ +#ifdef NLS + size_t nr; + wchar_t wc; + + nr = mbrtowc(&wc, s, n, mbs); + if (wi != NULL) + *wi = wc; + if (nr == 0) + return (1); + else if (nr == (size_t)-1 || nr == (size_t)-2) { + memset(mbs, 0, sizeof (*mbs)); + if (wi != NULL) + *wi = dummy; + return (1); + } else + return (nr); +#else + if (wi) + *wi = *s; + return (1); +#endif +} + +static __inline size_t +xmbrtowc_dummy(wint_t *wi, + const char *s, + size_t n __maybe_unused, + mbstate_t *mbs __maybe_unused, + wint_t dummy __maybe_unused) +{ + + if (wi != NULL) + *wi = (unsigned char)*s; + return (1); +} + +/* macros for manipulating states, small version */ +#define states long +#define states1 states /* for later use in regexec() decision */ +#define CLEAR(v) ((v) = 0) +#define SET0(v, n) ((v) &= ~((unsigned long)1 << (n))) +#define SET1(v, n) ((v) |= (unsigned long)1 << (n)) +#define ISSET(v, n) (((v) & ((unsigned long)1 << (n))) != 0) +#define ASSIGN(d, s) ((d) = (s)) +#define EQ(a, b) ((a) == (b)) +#define STATEVARS long dummy /* dummy version */ +#define STATESETUP(m, n) /* nothing */ +#define STATETEARDOWN(m) /* nothing */ +#define SETUP(v) ((v) = 0) +#define onestate long +#define INIT(o, n) ((o) = (unsigned long)1 << (n)) +#define INC(o) ((o) <<= 1) +#define ISSTATEIN(v, o) (((v) & (o)) != 0) +/* some abbreviations; note that some of these know variable names! */ +/* do "if I'm here, I can also be there" etc without branches */ +#define FWD(dst, src, n) ((dst) |= ((unsigned long)(src)&(here)) << (n)) +#define BACK(dst, src, n) ((dst) |= ((unsigned long)(src)&(here)) >> (n)) +#define ISSETBACK(v, n) (((v) & ((unsigned long)here >> (n))) != 0) +/* no multibyte support */ +#define XMBRTOWC xmbrtowc_dummy +#define ZAPSTATE(mbs) ((void)(mbs)) +/* function names */ +#define SNAMES /* engine.c looks after details */ + +#include "engine.c" + +/* now undo things */ +#undef states +#undef CLEAR +#undef SET0 +#undef SET1 +#undef ISSET +#undef ASSIGN +#undef EQ +#undef STATEVARS +#undef STATESETUP +#undef STATETEARDOWN +#undef SETUP +#undef onestate +#undef INIT +#undef INC +#undef ISSTATEIN +#undef FWD +#undef BACK +#undef ISSETBACK +#undef SNAMES +#undef XMBRTOWC +#undef ZAPSTATE + +/* macros for manipulating states, large version */ +#define states char * +#define CLEAR(v) memset(v, 0, m->g->nstates) +#define SET0(v, n) ((v)[n] = 0) +#define SET1(v, n) ((v)[n] = 1) +#define ISSET(v, n) ((v)[n]) +#define ASSIGN(d, s) memcpy(d, s, m->g->nstates) +#define EQ(a, b) (memcmp(a, b, m->g->nstates) == 0) +#define STATEVARS long vn; char *space +#define STATESETUP(m, nv) { (m)->space = malloc((nv)*(m)->g->nstates); \ + if ((m)->space == NULL) \ + return (REG_ESPACE); \ + (m)->vn = 0; } +#define STATETEARDOWN(m) { free((m)->space); } +#define SETUP(v) ((v) = &m->space[m->vn++ * m->g->nstates]) +#define onestate long +#define INIT(o, n) ((o) = (n)) +#define INC(o) ((o)++) +#define ISSTATEIN(v, o) ((v)[o]) +/* some abbreviations; note that some of these know variable names! */ +/* do "if I'm here, I can also be there" etc without branches */ +#define FWD(dst, src, n) ((dst)[here+(n)] |= (src)[here]) +#define BACK(dst, src, n) ((dst)[here-(n)] |= (src)[here]) +#define ISSETBACK(v, n) ((v)[here - (n)]) +/* no multibyte support */ +#define XMBRTOWC xmbrtowc_dummy +#define ZAPSTATE(mbs) ((void)(mbs)) +/* function names */ +#define LNAMES /* flag */ + +#include "engine.c" + +/* multibyte character & large states version */ +#undef LNAMES +#undef XMBRTOWC +#undef ZAPSTATE +#define XMBRTOWC xmbrtowc +#define ZAPSTATE(mbs) memset((mbs), 0, sizeof (*(mbs))) +#define MNAMES + +#include "engine.c" + +/* + * regexec - interface for matching + * extern int regexec(const regex_t *, const char *, size_t, \ + * regmatch_t [], int); + * #define REG_NOTBOL 00001 + * #define REG_NOTEOL 00002 + * #define REG_STARTEND 00004 + * #define REG_TRACE 00400 // tracing of execution + * #define REG_LARGE 01000 // force large representation + * #define REG_BACKR 02000 // force use of backref code + * + * We put this here so we can exploit knowledge of the state representation + * when choosing which matcher to call. Also, by this point the matchers + * have been prototyped. + */ +int /* 0 success, REG_NOMATCH failure */ +regexec(const regex_t *__restrict preg, + const char *__restrict string, + size_t nmatch, + regmatch_t pmatch[__restrict], + int eflags) +{ + struct re_guts *g = preg->re_g; +#ifdef REDEBUG +#define GOODFLAGS(f) (f) +#else +#define GOODFLAGS(f) ((f)&(REG_NOTBOL|REG_NOTEOL|REG_STARTEND)) +#endif + _DIAGASSERT(preg != NULL); + _DIAGASSERT(string != NULL); + + if (preg->re_magic != MAGIC1 || g->magic != MAGIC2) + return (REG_BADPAT); + assert(!(g->iflags&BAD)); + if (g->iflags&BAD) /* backstop for no-debug case */ + return (REG_BADPAT); + eflags = GOODFLAGS(eflags); + + if (MB_CUR_MAX > 1) + return (mmatcher(g, string, nmatch, pmatch, eflags)); + else if (g->nstates <= CHAR_BIT*sizeof (states1) && !(eflags®_LARGE)) + return (smatcher(g, string, nmatch, pmatch, eflags)); + else + return (lmatcher(g, string, nmatch, pmatch, eflags)); +} diff --git a/lib/os/windows/libregex/regfree.c b/lib/os/windows/libregex/regfree.c new file mode 100644 index 000000000000..c82408a8810f --- /dev/null +++ b/lib/os/windows/libregex/regfree.c @@ -0,0 +1,97 @@ +/* $NetBSD: regfree.c,v 1.19 2021/02/26 19:24:47 christos Exp $ */ + +/* + * SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 1992, 1993, 1994 Henry Spencer. + * Copyright (c) 1992, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)regfree.c 8.3 (Berkeley) 3/20/94 + */ + +#if HAVE_NBTOOL_CONFIG_H +#include "nbtool_config.h" +#endif + +// #include "namespace.h" +#include +#include +#include +#include +#include + +#ifdef __weak_alias +__weak_alias(regfree, _regfree) +#endif + +#include "utils.h" +#include "regex2.h" + +/* + * regfree - free everything + * extern void regfree(regex_t *); + */ +void +regfree(regex_t *preg) +{ + struct re_guts *g; + unsigned int i; + + _DIAGASSERT(preg != NULL); + + _DIAGASSERT(preg->re_magic == MAGIC1); + if (preg->re_magic != MAGIC1) /* oops */ + return; /* nice to complain, but hard */ + + g = preg->re_g; + if (g == NULL || g->magic != MAGIC2) /* oops again */ + return; + preg->re_magic = 0; /* mark it invalid */ + g->magic = 0; /* mark it invalid */ + + if (g->strip != NULL) + free(g->strip); + if (g->sets != NULL) { + for (i = 0; i < g->ncsets; i++) { + free(g->sets[i].ranges); + free(g->sets[i].wides); + free(g->sets[i].types); + } + free(g->sets); + } + if (g->must != NULL) + free(g->must); + if (g->charjump != NULL) + free(&g->charjump[CHAR_MIN]); + if (g->matchjump != NULL) + free(g->matchjump); + free(g); +} diff --git a/lib/os/windows/libregex/regsub.c b/lib/os/windows/libregex/regsub.c new file mode 100644 index 000000000000..423377ed0deb --- /dev/null +++ b/lib/os/windows/libregex/regsub.c @@ -0,0 +1,160 @@ +/* $NetBSD: regsub.c,v 1.3 2016/02/29 22:10:13 aymeric Exp $ */ + +/* + * Copyright (c) 2015 The NetBSD Foundation, Inc. + * All rights reserved. + * + * This code is derived from software contributed to The NetBSD Foundation + * by Christos Zoulas. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS + * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include + +struct str { + char *s_ptr; + size_t s_max; + size_t s_len; + int s_fixed; +}; + +#define REINCR 64 + +static int +addspace(struct str *s, size_t len) +{ + void *v; + + if (s->s_max - s->s_len > len) + return (0); + + if (s->s_fixed) + return (-1); + + s->s_max += len + REINCR; + + v = realloc(s->s_ptr, s->s_max); + if (v == NULL) + return (-1); + s->s_ptr = v; + + return (0); +} + +static void +addchar(struct str *s, int c) +{ + if (addspace(s, 1) == -1) + s->s_len++; + else + s->s_ptr[s->s_len++] = c; + if (c == 0) { + --s->s_len; + s->s_ptr[s->s_max - 1] = c; + } +} + +static void +addnstr(struct str *s, const char *buf, size_t len) +{ + if (addspace(s, len) != -1) + memcpy(s->s_ptr + s->s_len, buf, len); + s->s_len += len; +} + +static int +initstr(struct str *s, char *buf, size_t len) +{ + s->s_max = len; + s->s_ptr = buf == NULL ? malloc(len) : buf; + s->s_fixed = buf != NULL; + s->s_len = 0; + return (s->s_ptr == NULL ? -1 : 0); +} + +static ssize_t +regsub1(char **buf, size_t len, const char *sub, + const regmatch_t *rm, const char *str) +{ + ssize_t i; + char c; + struct str s; + + if (initstr(&s, *buf, len) == -1) + return (-1); + + while ((c = *sub++) != '\0') { + + switch (c) { + case '&': + i = 0; + break; + case '\\': + if (isdigit((unsigned char)*sub)) + i = *sub++ - '0'; + else + i = -1; + break; + default: + i = -1; + break; + } + + if (i == -1) { + if (c == '\\' && (*sub == '\\' || *sub == '&')) + c = *sub++; + addchar(&s, c); + } else if (rm[i].rm_so != -1 && rm[i].rm_eo != -1) { + size_t l = (size_t)(rm[i].rm_eo - rm[i].rm_so); + addnstr(&s, str + rm[i].rm_so, l); + } + } + + addchar(&s, '\0'); + if (!s.s_fixed) { + if (s.s_len >= s.s_max) { + free(s.s_ptr); + return (-1); + } + *buf = s.s_ptr; + } + return (s.s_len); +} + +ssize_t +regnsub(char *buf, size_t len, const char *sub, const regmatch_t *rm, + const char *str) +{ + return (regsub1(&buf, len, sub, rm, str)); +} + +ssize_t +regasub(char **buf, const char *sub, const regmatch_t *rm, const char *str) +{ + *buf = NULL; + return (regsub1(buf, REINCR, sub, rm, str)); +} diff --git a/lib/os/windows/libregex/utils.h b/lib/os/windows/libregex/utils.h new file mode 100644 index 000000000000..618ff6d669e2 --- /dev/null +++ b/lib/os/windows/libregex/utils.h @@ -0,0 +1,84 @@ +/* $NetBSD: utils.h,v 1.9 2021/04/22 19:20:24 christos Exp $ */ + +/* + * SPDX-License-Identifier: BSD-3-Clause + * + * Copyright (c) 1992, 1993, 1994 Henry Spencer. + * Copyright (c) 1992, 1993, 1994 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Henry Spencer. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)utils.h 8.3 (Berkeley) 3/20/94 + * $FreeBSD: head/lib/libc/regex/utils.h 341838 2018-12-12 04:23:00Z yuripv $ + */ + +#ifdef NLS +#include +#include +#else +#include +#define wint_t regex_wint_t +#define mbstate_t regex_mbstate_t +#define wctype_t regex_wctype_t +typedef short wint_t; +typedef char mbstate_t; +typedef short wctype_t; +// #define iswupper(a) isupper(a) +// #define iswlower(a) islower(a) +// #define iswalpha(a) isalpha(a) +// #define iswalnum(a) isalnum(a) +#define towupper(a) toupper(a) +#define towlower(a) tolower(a) +extern wctype_t __regex_wctype(const char *); +extern int __regex_iswctype(wint_t, wctype_t); +// #define wctype(s) __regex_wctype(s) +// #define iswctype(c, t) __regex_iswctype((c), (t)) +#endif + +/* utility definitions */ +// Windows defines a float INFINITY +#define DUPMAX (~0ULL) /* xxx is this right? */ +#define REGINFINITY (DUPMAX + 1) + +#define NC_MAX (CHAR_MAX - CHAR_MIN + 1) +#define NC ((MB_CUR_MAX) == 1 ? (NC_MAX) : (128)) +typedef unsigned char uch; + +/* switch off assertions (if not already off) if no REDEBUG */ +#ifndef REDEBUG +#ifndef NDEBUG +#define NDEBUG /* no assertions please */ +#endif +#endif +#include + +/* for old systems with bcopy() but no memmove() */ +#ifdef USEBCOPY +#define memmove(d, s, c) bcopy(s, d, c) +#endif diff --git a/lib/os/windows/libuuid/CMakeLists.txt b/lib/os/windows/libuuid/CMakeLists.txt new file mode 100644 index 000000000000..ae683b74c871 --- /dev/null +++ b/lib/os/windows/libuuid/CMakeLists.txt @@ -0,0 +1,21 @@ + +use_clang() + +add_library(libuuid + clear.c + compare.c + copy.c + gen_uuid.c + isnull.c + pack.c + parse.c + randutils.c + unpack.c + unparse.c + uuid_time.c + randutils.h + uuid.h +) +target_compile_definitions(libuuid PUBLIC HAVE_STDINT_H) +target_link_libraries(libuuid PUBLIC libspl) +target_include_directories(libuuid PUBLIC "") diff --git a/lib/os/windows/libuuid/COPYING b/lib/os/windows/libuuid/COPYING new file mode 100644 index 000000000000..6935c2889323 --- /dev/null +++ b/lib/os/windows/libuuid/COPYING @@ -0,0 +1,5 @@ +This library is free software; you can redistribute it and/or +modify it under the terms of the Modified BSD License. + +The complete text of the license is available at the +Documentation/licenses/COPYING.BSD-3 file. diff --git a/lib/os/windows/libuuid/all-io.h b/lib/os/windows/libuuid/all-io.h new file mode 100644 index 000000000000..29959b568880 --- /dev/null +++ b/lib/os/windows/libuuid/all-io.h @@ -0,0 +1,83 @@ +/* + * No copyright is claimed. This code is in the public domain; do with + * it what you wish. + * + * Written by Karel Zak + * Petr Uzel + */ + +#ifndef UTIL_LINUX_ALL_IO_H +#define UTIL_LINUX_ALL_IO_H + +#include +#include +#include +#include + +#include "c.h" + +static inline int write_all(int fd, const void *buf, size_t count) +{ + while (count) { + ssize_t tmp; + + errno = 0; + tmp = write(fd, buf, count); + if (tmp > 0) { + count -= tmp; + if (count) + buf = (void *)((char *)buf + tmp); + } else if (errno != EINTR && errno != EAGAIN) + return (-1); + if (errno == EAGAIN) /* Try later, *sigh* */ + usleep(10000); + } + return (0); +} + +static inline int fwrite_all(const void *ptr, size_t size, + size_t nmemb, FILE *stream) +{ + while (nmemb) { + size_t tmp; + + errno = 0; + tmp = fwrite(ptr, size, nmemb, stream); + if (tmp > 0) { + nmemb -= tmp; + if (nmemb) + ptr = (void *)((char *)ptr + (tmp * size)); + } else if (errno != EINTR && errno != EAGAIN) + return (-1); + if (errno == EAGAIN) /* Try later, *sigh* */ + usleep(10000); + } + return (0); +} + +static inline ssize_t read_all(int fd, char *buf, size_t count) +{ + ssize_t ret; + ssize_t c = 0; + int tries = 0; + + memset(buf, 0, count); + while (count > 0) { + ret = read(fd, buf, count); + if (ret <= 0) { + if ((errno == EAGAIN || errno == EINTR || ret == 0) && + (tries++ < 5)) + continue; + return (c ? c : -1); + } + if (ret > 0) + tries = 0; + count -= ret; + buf += ret; + c += ret; + } + return (c); +} + + +#endif /* UTIL_LINUX_ALL_IO_H */ diff --git a/lib/os/windows/libuuid/c.h b/lib/os/windows/libuuid/c.h new file mode 100644 index 000000000000..4bd5798bf0b6 --- /dev/null +++ b/lib/os/windows/libuuid/c.h @@ -0,0 +1,328 @@ +/* + * Fundamental C definitions. + */ + +#ifndef UTIL_LINUX_C_H +#define UTIL_LINUX_C_H + +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + +#include +#include + +#ifdef HAVE_STDINT_H +#include +#else +#ifdef HAVE_INTTYPES_H +#include +#endif +#endif + +#include +#include +#include +#include +#include +#include + +#ifdef HAVE_ERR_H +#include +#endif + +#ifndef HAVE_USLEEP +#include +#endif + +/* + * Compiler specific stuff + */ +#ifndef __GNUC_PREREQ +#if defined __GNUC__ && defined __GNUC_MINOR__ +#define __GNUC_PREREQ(maj, min) \ + ((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min)) +#else +#define __GNUC_PREREQ(maj, min) 0 +#endif +#endif + +#ifdef __GNUC__ + +/* &a[0] degrades to a pointer: a different type from an array */ +#define __must_be_array(a) \ + UL_BUILD_BUG_ON_ZERO(__builtin_types_compatible_p(__typeof__(a), \ + __typeof__(&a[0]))) + +#define ignore_result(x) (/* */ {__typeof__(x) __dummy \ + __attribute__((__unused__)) = (x); (void) __dummy; \ +}) + +#else /* !__GNUC__ */ +#define __must_be_array(a) 0 +#define __attribute__(_arg_) +#define ignore_result(x) ((void) (x)) +#endif /* !__GNUC__ */ + +/* + * Function attributes + */ +#ifndef __ul_alloc_size +#if __GNUC_PREREQ(4, 3) +#define __ul_alloc_size(s) __attribute__((alloc_size(s))) +#else +#define __ul_alloc_size(s) +#endif +#endif + +#ifndef __ul_calloc_size +#if __GNUC_PREREQ(4, 3) +#define __ul_calloc_size(n, s) __attribute__((alloc_size(n, s))) +#else +#define __ul_calloc_size(n, s) +#endif +#endif + +/* + * Force a compilation error if condition is true, but also produce a + * result (of value 0 and type size_t), so the expression can be used + * e.g. in a structure initializer (or where-ever else comma expressions + * aren't permitted). + */ +#define UL_BUILD_BUG_ON_ZERO(e) (sizeof (struct { int:-!!(e); })) +#define BUILD_BUG_ON_NULL(e) ((void *)sizeof (struct { int:-!!(e); })) + +#ifndef ARRAY_SIZE +#define ARRAY_SIZE(arr) \ + (sizeof (arr) / sizeof ((arr)[0]) + __must_be_array(arr)) +#endif + +#ifndef PATH_MAX +#define PATH_MAX 4096 +#endif + +#ifndef TRUE +#define TRUE 1 +#endif + +#ifndef FALSE +#define FALSE 0 +#endif + +#ifndef min +#define min(x, y) (/* */ { \ + __typeof__(x) _min1 = (x); \ + __typeof__(y) _min2 = (y); \ + (void) (&_min1 == &_min2); \ + _min1 < _min2 ? _min1 : _min2; }) +#endif + +#ifndef max +#define max(x, y) (/* */ { \ + __typeof__(x) _max1 = (x); \ + __typeof__(y) _max2 = (y); \ + (void) (&_max1 == &_max2); \ + _max1 > _max2 ? _max1 : _max2; }) +#endif + +#ifndef offsetof +#define offsetof(TYPE, MEMBER) ((size_t)&((TYPE *)0)->MEMBER) +#endif + +#ifndef container_of +#define container_of(ptr, type, member) (/* */ { \ + const __typeof__(((type *)0)->member) *__mptr = (ptr); \ + (type *)((char *)__mptr - offsetof(type, member)); }) +#endif + +#ifndef HAVE_PROGRAM_INVOCATION_SHORT_NAME +#ifdef HAVE___PROGNAME +extern char *__progname; +#define program_invocation_short_name __progname +#else +#ifdef HAVE_GETEXECNAME +#define program_invocation_short_name \ + prog_inv_sh_nm_from_file(getexecname(), 0) +#else +#define program_invocation_short_name \ + prog_inv_sh_nm_from_file(__FILE__, 1) +#endif +static char prog_inv_sh_nm_buf[256]; +static inline char * +prog_inv_sh_nm_from_file(char *f, char stripext) +{ + char *t; + + if ((t = strrchr(f, '/')) != NULL) + t++; + else + t = f; + + strncpy(prog_inv_sh_nm_buf, t, sizeof (prog_inv_sh_nm_buf) - 1); + prog_inv_sh_nm_buf[sizeof (prog_inv_sh_nm_buf) - 1] = '\0'; + + if (stripext && (t = strrchr(prog_inv_sh_nm_buf, '.')) != NULL) + *t = '\0'; + + return (prog_inv_sh_nm_buf); +} +#endif +#endif + + +#ifndef HAVE_ERR_H +static inline void +errmsg(char doexit, int excode, char adderr, const char *fmt, ...) +{ + fprintf(stderr, "%s: ", program_invocation_short_name); + if (fmt != NULL) { + va_list argp; + va_start(argp, fmt); + vfprintf(stderr, fmt, argp); + va_end(argp); + if (adderr) + fprintf(stderr, ": "); + } + if (adderr) + fprintf(stderr, "%m"); + fprintf(stderr, "\n"); + if (doexit) + exit(excode); +} + +#ifndef HAVE_ERR +#define err(E, FMT, ...) errmsg(1, E, 1, FMT) +#endif + +#ifndef HAVE_ERRX +#define errx(E, FMT, ...) errmsg(1, E, 0, FMT) +#endif + +#ifndef HAVE_WARN +#define warn(FMT, ...) errmsg(0, 0, 1, FMT) +#endif + +#ifndef HAVE_WARNX +#define warnx(FMT, ...) errmsg(0, 0, 0, FMT) +#endif +#endif /* !HAVE_ERR_H */ + + +static inline __attribute__((const)) int is_power_of_2(unsigned long num) +{ + return (num != 0 && ((num & (num - 1)) == 0)); +} + +#ifndef HAVE_LOFF_T +typedef int64_t loff_t; +#endif + +#if !defined(HAVE_DIRFD) && \ + (!defined(HAVE_DECL_DIRFD) || HAVE_DECL_DIRFD == 0) && \ + defined(HAVE_DIR_DD_FD) +#include +#include +static inline int dirfd(DIR *d) +{ + return (d->dd_fd); +} +#endif + +/* + * Fallback defines for old versions of glibc + */ +#include + +#ifdef O_CLOEXEC +#define UL_CLOEXECSTR "e" +#else +#define UL_CLOEXECSTR "" +#endif + +#ifndef O_CLOEXEC +#define O_CLOEXEC 0 +#endif + + +#ifndef AI_ADDRCONFIG +#define AI_ADDRCONFIG 0x0020 +#endif + +#ifndef IUTF8 +#define IUTF8 0040000 +#endif + +/* + * MAXHOSTNAMELEN replacement + */ +static inline size_t get_hostname_max(void) +{ +#if HAVE_DECL__SC_HOST_NAME_MAX + long len = sysconf(_SC_HOST_NAME_MAX); + + if (0 < len) + return (len); +#endif + +#ifdef MAXHOSTNAMELEN + return (MAXHOSTNAMELEN); +#elif HOST_NAME_MAX + return (HOST_NAME_MAX); +#endif + return (64); +} + +#ifndef HAVE_USLEEP +/* + * This function is marked obsolete in POSIX.1-2001 and removed in + * POSIX.1-2008. It is replaced with nanosleep(). + */ +static inline int usleep(__int64 usec) +{ + HANDLE timer; + LARGE_INTEGER ft; + + ft.QuadPart = -(10 * usec); + + timer = CreateWaitableTimer(NULL, TRUE, NULL); + SetWaitableTimer(timer, &ft, 0, NULL, NULL, 0); + WaitForSingleObject(timer, INFINITE); + CloseHandle(timer); +} +#endif + +/* + * Constant strings for usage() functions. For more info see + * Documentation/howto-usage-function.txt and disk-utils/delpart.c + */ +#define USAGE_HEADER _("\nUsage:\n") +#define USAGE_OPTIONS _("\nOptions:\n") +#define USAGE_SEPARATOR _("\n") +#define USAGE_HELP _(" -h, --help display this help and exit\n") +#define USAGE_VERSION \ + _(" -V, --version output version information and exit\n") +#define USAGE_MAN_TAIL(_man) _("\nFor more details see %s.\n"), _man + +#define UTIL_LINUX_VERSION \ + _("%s from %s\n"), program_invocation_short_name, PACKAGE_STRING + +/* + * scanf modifiers for "strings allocation" + */ +#ifdef HAVE_SCANF_MS_MODIFIER +#define UL_SCNsA "%ms" +#elif defined(HAVE_SCANF_AS_MODIFIER) +#define UL_SCNsA "%as" +#endif + +/* + * seek stuff + */ +#ifndef SEEK_DATA +#define SEEK_DATA 3 +#endif +#ifndef SEEK_HOLE +#define SEEK_HOLE 4 +#endif + +#endif /* UTIL_LINUX_C_H */ diff --git a/lib/os/windows/libuuid/clear.c b/lib/os/windows/libuuid/clear.c new file mode 100644 index 000000000000..3d1ef1fe3788 --- /dev/null +++ b/lib/os/windows/libuuid/clear.c @@ -0,0 +1,43 @@ +/* + * clear.c -- Clear a UUID + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include "string.h" + +#include "uuidP.h" + +void +uuid_clear(uuid_t uu) +{ + memset(uu, 0, 16); +} diff --git a/lib/os/windows/libuuid/compare.c b/lib/os/windows/libuuid/compare.c new file mode 100644 index 000000000000..b9159acc2887 --- /dev/null +++ b/lib/os/windows/libuuid/compare.c @@ -0,0 +1,55 @@ +/* + * compare.c --- compare whether or not two UUIDs are the same + * + * Returns 0 if the two UUIDs are different, and 1 if they are the same. + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include "uuidP.h" +#include + +#define UUCMP(u1, u2) if (u1 != u2) return ((u1 < u2) ? -1 : 1); + +int +uuid_compare(const uuid_t uu1, const uuid_t uu2) +{ + struct uuid uuid1, uuid2; + + uuid_unpack(uu1, &uuid1); + uuid_unpack(uu2, &uuid2); + + UUCMP(uuid1.time_low, uuid2.time_low); + UUCMP(uuid1.time_mid, uuid2.time_mid); + UUCMP(uuid1.time_hi_and_version, uuid2.time_hi_and_version); + UUCMP(uuid1.clock_seq, uuid2.clock_seq); + return (memcmp(uuid1.node, uuid2.node, 6)); +} diff --git a/lib/os/windows/libuuid/copy.c b/lib/os/windows/libuuid/copy.c new file mode 100644 index 000000000000..6938aa17f7e0 --- /dev/null +++ b/lib/os/windows/libuuid/copy.c @@ -0,0 +1,46 @@ +/* + * copy.c --- copy UUIDs + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include "uuidP.h" + +void +uuid_copy(uuid_t dst, const uuid_t src) +{ + unsigned char *cp1; + const unsigned char *cp2; + int i; + + for (i = 0, cp1 = dst, cp2 = src; i < 16; i++) + *cp1++ = *cp2++; +} diff --git a/lib/os/windows/libuuid/gen_uuid.c b/lib/os/windows/libuuid/gen_uuid.c new file mode 100644 index 000000000000..f7488916e379 --- /dev/null +++ b/lib/os/windows/libuuid/gen_uuid.c @@ -0,0 +1,584 @@ +/* + * gen_uuid.c --- generate a DCE-compatible uuid + * + * Copyright (C) 1996, 1997, 1998, 1999 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +/* + * Force inclusion of SVID stuff since we need it if we're compiling in + * gcc-wall wall mode + */ +#define _SVID_SOURCE + +#ifdef _WIN32 +#define _WIN32_WINNT 0x0500 +#include +#define UUID MYUUID +#endif +#include +#ifdef HAVE_UNISTD_H +#include +#endif +#ifdef HAVE_STDLIB_H +#include +#endif +#include +#include +#include +#include +#include +#ifdef HAVE_SYS_TIME_H +#include +#endif +#include +#ifdef HAVE_SYS_FILE_H +#include +#endif +#ifdef HAVE_SYS_IOCTL_H +#include +#endif +#ifdef HAVE_SYS_SOCKET_H +#include +#endif +#ifdef HAVE_SYS_UN_H +#include +#endif +#ifdef HAVE_SYS_SOCKIO_H +#include +#endif +#ifdef HAVE_NET_IF_H +#include +#endif +#ifdef HAVE_NETINET_IN_H +#include +#endif +#ifdef HAVE_NET_IF_DL_H +#include +#endif +#if defined(__linux__) && defined(HAVE_SYS_SYSCALL_H) +#include +#endif + +#include + +#include "all-io.h" +#include "uuidP.h" +#include "uuidd.h" +#include "randutils.h" +#include "c.h" + +#ifdef HAVE_TLS +#define THREAD_LOCAL static __thread +#else +#define THREAD_LOCAL static +#endif + +#ifndef LOCK_EX +/* flock() replacement */ +#define LOCK_EX 1 +#define LOCK_SH 2 +#define LOCK_UN 3 +#define LOCK_NB 4 + +static int flock(int fd, int op) +{ + int rc = 0; + +#if defined(F_SETLK) && defined(F_SETLKW) + struct flock fl = {0}; + + switch (op & (LOCK_EX|LOCK_SH|LOCK_UN)) { + case LOCK_EX: + fl.l_type = F_WRLCK; + break; + + case LOCK_SH: + fl.l_type = F_RDLCK; + break; + + case LOCK_UN: + fl.l_type = F_UNLCK; + break; + + default: + errno = EINVAL; + return (-1); + } + + fl.l_whence = SEEK_SET; + rc = fcntl(fd, op & LOCK_NB ? F_SETLK : F_SETLKW, &fl); + + if (rc && (errno == EAGAIN)) + errno = EWOULDBLOCK; +#endif /* defined(F_SETLK) && defined(F_SETLKW) */ + + return (rc); +} + +#endif /* LOCK_EX */ + +/* + * Get the ethernet hardware address, if we can find it... + * + * XXX for a windows version, probably should use GetAdaptersInfo: + * http://www.codeguru.com/cpp/i-n/network/networkinformation/article.php/c5451 + * commenting out get_node_id just to get gen_uuid to compile under windows + * is not the right way to go! + */ +static int get_node_id(unsigned char *node_id) +{ +#ifdef HAVE_NET_IF_H + int sd; + struct ifreq ifr, *ifrp; + struct ifconf ifc; + char buf[1024]; + int n, i; + unsigned char *a; +#ifdef HAVE_NET_IF_DL_H + struct sockaddr_dl *sdlp; +#endif + +/* + * BSD 4.4 defines the size of an ifreq to be + * max(sizeof(ifreq), sizeof(ifreq.ifr_name)+ifreq.ifr_addr.sa_len + * However, under earlier systems, sa_len isn't present, so the size is + * just sizeof(struct ifreq) + */ +#ifdef HAVE_SA_LEN +#define ifreq_size(i) max(sizeof (struct ifreq),\ + sizeof ((i).ifr_name)+(i).ifr_addr.sa_len) +#else +#define ifreq_size(i) sizeof (struct ifreq) +#endif /* HAVE_SA_LEN */ + + sd = socket(AF_INET, SOCK_DGRAM, IPPROTO_IP); + if (sd < 0) { + return (-1); + } + memset(buf, 0, sizeof (buf)); + ifc.ifc_len = sizeof (buf); + ifc.ifc_buf = buf; + if (ioctl(sd, SIOCGIFCONF, (char *)&ifc) < 0) { + close(sd); + return (-1); + } + n = ifc.ifc_len; + for (i = 0; i < n; i += ifreq_size(*ifrp)) { + ifrp = (struct ifreq *)((char *)ifc.ifc_buf+i); + strncpy(ifr.ifr_name, ifrp->ifr_name, IFNAMSIZ); +#ifdef SIOCGIFHWADDR + if (ioctl(sd, SIOCGIFHWADDR, &ifr) < 0) + continue; + a = (unsigned char *) &ifr.ifr_hwaddr.sa_data; +#else +#ifdef SIOCGENADDR + if (ioctl(sd, SIOCGENADDR, &ifr) < 0) + continue; + a = (unsigned char *) ifr.ifr_enaddr; +#else +#ifdef HAVE_NET_IF_DL_H + sdlp = (struct sockaddr_dl *)&ifrp->ifr_addr; + if ((sdlp->sdl_family != AF_LINK) || (sdlp->sdl_alen != 6)) + continue; + a = (unsigned char *)&sdlp->sdl_data[sdlp->sdl_nlen]; +#else + /* + * XXX we don't have a way of getting the hardware + * address + */ + close(sd); + return (0); +#endif /* HAVE_NET_IF_DL_H */ +#endif /* SIOCGENADDR */ +#endif /* SIOCGIFHWADDR */ + if (!a[0] && !a[1] && !a[2] && !a[3] && !a[4] && !a[5]) + continue; + if (node_id) { + memcpy(node_id, a, 6); + close(sd); + return (1); + } + } + close(sd); +#endif + return (0); +} + +/* Assume that the gettimeofday() has microsecond granularity */ +#define MAX_ADJUSTMENT 10 + +/* + * Get clock from global sequence clock counter. + * + * Return -1 if the clock counter could not be opened/locked (in this case + * pseudorandom value is returned in @ret_clock_seq), otherwise return 0. + */ +static int +get_clock(uint32_t *clock_high, uint32_t *clock_low, + uint16_t *ret_clock_seq, int *num) +{ + THREAD_LOCAL int adjustment = 0; + THREAD_LOCAL struct timeval last = {0, 0}; + THREAD_LOCAL int state_fd = -2; + THREAD_LOCAL FILE *state_f; + THREAD_LOCAL uint16_t clock_seq; + struct timeval tv; + uint64_t clock_reg; + mode_t save_umask; + int len; + int ret = 0; + + if (state_fd == -2) { + save_umask = _umask(0); + state_fd = open(LIBUUID_CLOCK_FILE, O_RDWR|O_CREAT|O_CLOEXEC, + 0660); + (void) _umask(save_umask); + if (state_fd != -1) { + state_f = fdopen(state_fd, "r+" UL_CLOEXECSTR); + if (!state_f) { + close(state_fd); + state_fd = -1; + ret = -1; + } + } else + ret = -1; + } + if (state_fd >= 0) { + rewind(state_f); + while (flock(state_fd, LOCK_EX) < 0) { + if ((errno == EAGAIN) || (errno == EINTR)) + continue; + fclose(state_f); + close(state_fd); + state_fd = -1; + ret = -1; + break; + } + } + if (state_fd >= 0) { + unsigned int cl; + unsigned long tv1, tv2; + int a; + + if (fscanf(state_f, "clock: %04x tv: %lu %lu adj: %d\n", + &cl, &tv1, &tv2, &a) == 4) { + clock_seq = cl & 0x3FFF; + last.tv_sec = tv1; + last.tv_usec = tv2; + adjustment = a; + } + } + + if ((last.tv_sec == 0) && (last.tv_usec == 0)) { + random_get_bytes(&clock_seq, sizeof (clock_seq)); + clock_seq &= 0x3FFF; + gettimeofday(&last, 0); + last.tv_sec--; + } + +try_again: + gettimeofday(&tv, 0); + if ((tv.tv_sec < last.tv_sec) || + ((tv.tv_sec == last.tv_sec) && + (tv.tv_usec < last.tv_usec))) { + clock_seq = (clock_seq+1) & 0x3FFF; + adjustment = 0; + last = tv; + } else if ((tv.tv_sec == last.tv_sec) && + (tv.tv_usec == last.tv_usec)) { + if (adjustment >= MAX_ADJUSTMENT) + goto try_again; + adjustment++; + } else { + adjustment = 0; + last = tv; + } + + clock_reg = tv.tv_usec*10 + adjustment; + clock_reg += ((uint64_t)tv.tv_sec)*10000000; + clock_reg += (((uint64_t)0x01B21DD2) << 32) + 0x13814000; + + if (num && (*num > 1)) { + adjustment += *num - 1; + last.tv_usec += adjustment / 10; + adjustment = adjustment % 10; + last.tv_sec += last.tv_usec / 1000000; + last.tv_usec = last.tv_usec % 1000000; + } + + if (state_fd >= 0) { + rewind(state_f); + len = fprintf(state_f, + "clock: %04x tv: %016lu %08lu adj: %08d\n", + clock_seq, last.tv_sec, last.tv_usec, adjustment); + fflush(state_f); + if (ftruncate(state_fd, len) < 0) { + fprintf(state_f, " \n"); + fflush(state_f); + } + rewind(state_f); + flock(state_fd, LOCK_UN); + } + + *clock_high = clock_reg >> 32; + *clock_low = clock_reg; + *ret_clock_seq = clock_seq; + return (ret); +} + +#if defined(HAVE_UUIDD) && defined(HAVE_SYS_UN_H) +/* + * Try using the uuidd daemon to generate the UUID + * + * Returns 0 on success, non-zero on failure. + */ +static int +get_uuid_via_daemon(int op, uuid_t out, int *num) +{ + char op_buf[64]; + int op_len; + int s; + ssize_t ret; + int32_t reply_len = 0, expected = 16; + struct sockaddr_un srv_addr; + + if ((s = socket(AF_UNIX, SOCK_STREAM, 0)) < 0) + return (-1); + + srv_addr.sun_family = AF_UNIX; + strcpy(srv_addr.sun_path, UUIDD_SOCKET_PATH); + + if (connect(s, (const struct sockaddr *)&srv_addr, + sizeof (struct sockaddr_un)) < 0) + goto fail; + + op_buf[0] = op; + op_len = 1; + if (op == UUIDD_OP_BULK_TIME_UUID) { + memcpy(op_buf+1, num, sizeof (*num)); + op_len += sizeof (*num); + expected += sizeof (*num); + } + + ret = write(s, op_buf, op_len); + if (ret < 1) + goto fail; + + ret = read_all(s, (char *)&reply_len, sizeof (reply_len)); + if (ret < 0) + goto fail; + + if (reply_len != expected) + goto fail; + + ret = read_all(s, op_buf, reply_len); + + if (op == UUIDD_OP_BULK_TIME_UUID) + memcpy(op_buf+16, num, sizeof (int)); + + memcpy(out, op_buf, 16); + + close(s); + return ((ret == expected) ? 0 : -1); + +fail: + close(s); + return (-1); +} + +#else /* !defined(HAVE_UUIDD) && defined(HAVE_SYS_UN_H) */ +static int +get_uuid_via_daemon(int op, uuid_t out, int *num) +{ + return (-1); +} +#endif + +int +__uuid_generate_time(uuid_t out, int *num) +{ + static unsigned char node_id[6]; + static int has_init = 0; + struct uuid uu; + uint32_t clock_mid; + int ret; + + if (!has_init) { + if (get_node_id(node_id) <= 0) { + random_get_bytes(node_id, 6); + /* + * Set multicast bit, to prevent conflicts + * with IEEE 802 addresses obtained from + * network cards + */ + node_id[0] |= 0x01; + } + has_init = 1; + } + ret = get_clock(&clock_mid, &uu.time_low, &uu.clock_seq, num); + uu.clock_seq |= 0x8000; + uu.time_mid = (uint16_t)clock_mid; + uu.time_hi_and_version = ((clock_mid >> 16) & 0x0FFF) | 0x1000; + memcpy(uu.node, node_id, 6); + uuid_pack(&uu, out); + return (ret); +} + +/* + * Generate time-based UUID and store it to @out + * + * Tries to guarantee uniqueness of the generated UUIDs by obtaining + * them from the uuidd daemon, or, if uuidd is not usable, by using + * the global clock state counter (see get_clock()). If neither of + * these is possible (e.g. because of insufficient permissions), it generates + * the UUID anyway, but returns -1. Otherwise, returns 0. + */ +static int +uuid_generate_time_generic(uuid_t out) +{ +#ifdef HAVE_TLS + THREAD_LOCAL int num = 0; + THREAD_LOCAL struct uuid uu; + THREAD_LOCAL time_t last_time = 0; + time_t now; + + if (num > 0) { + now = time(0); + if (now > last_time+1) + num = 0; + } + if (num <= 0) { + num = 1000; + if (get_uuid_via_daemon(UUIDD_OP_BULK_TIME_UUID, + out, &num) == 0) { + last_time = time(0); + uuid_unpack(out, &uu); + num--; + return (0); + } + num = 0; + } + if (num > 0) { + uu.time_low++; + if (uu.time_low == 0) { + uu.time_mid++; + if (uu.time_mid == 0) + uu.time_hi_and_version++; + } + num--; + uuid_pack(&uu, out); + return (0); + } +#else + if (get_uuid_via_daemon(UUIDD_OP_TIME_UUID, out, 0) == 0) + return (0); +#endif + + return (__uuid_generate_time(out, 0)); +} + +/* + * Generate time-based UUID and store it to @out. + * + * Discards return value from uuid_generate_time_generic() + */ +void +uuid_generate_time(uuid_t out) +{ + (void) uuid_generate_time_generic(out); +} + +int +uuid_generate_time_safe(uuid_t out) +{ + return (uuid_generate_time_generic(out)); +} + +void +__uuid_generate_random(uuid_t out, int *num) +{ + uuid_t buf; + struct uuid uu; + int i, n; + + if (!num || !*num) + n = 1; + else + n = *num; + + for (i = 0; i < n; i++) { + random_get_bytes(buf, sizeof (buf)); + uuid_unpack(buf, &uu); + + uu.clock_seq = (uu.clock_seq & 0x3FFF) | 0x8000; + uu.time_hi_and_version = (uu.time_hi_and_version & 0x0FFF) + | 0x4000; + uuid_pack(&uu, out); + out += sizeof (uuid_t); + } +} + +void +uuid_generate_random(uuid_t out) +{ + int num = 1; + /* No real reason to use the daemon for random uuid's -- yet */ + + __uuid_generate_random(out, &num); +} + +/* + * Check whether good random source (/dev/random or /dev/urandom) + * is available. + */ +static int +have_random_source(void) +{ + struct stat s; + + return (!stat("/dev/random", &s) || !stat("/dev/urandom", &s)); +} + + +/* + * This is the generic front-end to uuid_generate_random and + * uuid_generate_time. It uses uuid_generate_random only if + * /dev/urandom is available, since otherwise we won't have + * high-quality randomness. + */ +void +uuid_generate(uuid_t out) +{ + if (have_random_source()) + uuid_generate_random(out); + else + uuid_generate_time(out); +} diff --git a/lib/os/windows/libuuid/isnull.c b/lib/os/windows/libuuid/isnull.c new file mode 100644 index 000000000000..866d8a06e7fa --- /dev/null +++ b/lib/os/windows/libuuid/isnull.c @@ -0,0 +1,48 @@ +/* + * isnull.c --- Check whether or not the UUID is null + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include "uuidP.h" + +/* Returns 1 if the uuid is the NULL uuid */ +int +uuid_is_null(const uuid_t uu) +{ + const unsigned char *cp; + int i; + + for (i = 0, cp = uu; i < 16; i++) + if (*cp++) + return (0); + return (1); +} diff --git a/lib/os/windows/libuuid/pack.c b/lib/os/windows/libuuid/pack.c new file mode 100644 index 000000000000..509867ecf814 --- /dev/null +++ b/lib/os/windows/libuuid/pack.c @@ -0,0 +1,69 @@ +/* + * Internal routine for packing UUIDs + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include +#include "uuidP.h" + +void +uuid_pack(const struct uuid *uu, uuid_t ptr) +{ + uint32_t tmp; + unsigned char *out = ptr; + + tmp = uu->time_low; + out[3] = (unsigned char) tmp; + tmp >>= 8; + out[2] = (unsigned char) tmp; + tmp >>= 8; + out[1] = (unsigned char) tmp; + tmp >>= 8; + out[0] = (unsigned char) tmp; + + tmp = uu->time_mid; + out[5] = (unsigned char) tmp; + tmp >>= 8; + out[4] = (unsigned char) tmp; + + tmp = uu->time_hi_and_version; + out[7] = (unsigned char) tmp; + tmp >>= 8; + out[6] = (unsigned char) tmp; + + tmp = uu->clock_seq; + out[9] = (unsigned char) tmp; + tmp >>= 8; + out[8] = (unsigned char) tmp; + + memcpy(out+10, uu->node, 6); +} diff --git a/lib/os/windows/libuuid/parse.c b/lib/os/windows/libuuid/parse.c new file mode 100644 index 000000000000..9948c4d3056d --- /dev/null +++ b/lib/os/windows/libuuid/parse.c @@ -0,0 +1,80 @@ +/* + * parse.c --- UUID parsing + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include +#include +#include +#include + +#include "uuidP.h" + +int +uuid_parse(const char *in, uuid_t uu) +{ + struct uuid uuid; + int i; + const char *cp; + char buf[3]; + + if (strlen(in) != 36) + return (-1); + for (i = 0, cp = in; i <= 36; i++, cp++) { + if ((i == 8) || (i == 13) || (i == 18) || + (i == 23)) { + if (*cp == '-') + continue; + else + return (-1); + } + if (i == 36) + if (*cp == 0) + continue; + if (!isxdigit(*cp)) + return (-1); + } + uuid.time_low = strtoul(in, NULL, 16); + uuid.time_mid = strtoul(in+9, NULL, 16); + uuid.time_hi_and_version = strtoul(in+14, NULL, 16); + uuid.clock_seq = strtoul(in+19, NULL, 16); + cp = in+24; + buf[2] = 0; + for (i = 0; i < 6; i++) { + buf[0] = *cp++; + buf[1] = *cp++; + uuid.node[i] = strtoul(buf, NULL, 16); + } + + uuid_pack(&uuid, uu); + return (0); +} diff --git a/lib/os/windows/libuuid/randutils.c b/lib/os/windows/libuuid/randutils.c new file mode 100644 index 000000000000..b3a618e3b9bb --- /dev/null +++ b/lib/os/windows/libuuid/randutils.c @@ -0,0 +1,126 @@ +/* + * General purpose random utilities + * + * Based on libuuid code. + * + * This file may be redistributed under the terms of the + * GNU Lesser General Public License. + */ +#include +#include +#include +#include +#include +#include + +#include "randutils.h" + +#ifdef HAVE_TLS +#define THREAD_LOCAL static __thread +#else +#define THREAD_LOCAL static +#endif + +#if defined(__linux__) && defined(__NR_gettid) && defined(HAVE_JRAND48) +#define DO_JRAND_MIX +THREAD_LOCAL unsigned short ul_jrand_seed[3]; +#endif + +#ifndef _WIN32 +int +random_get_fd(void) +{ + int i, fd; + struct timeval tv; + + gettimeofday(&tv, 0); + fd = open("/dev/urandom", O_RDONLY); + if (fd == -1) + fd = open("/dev/random", O_RDONLY | O_NONBLOCK); + if (fd >= 0) { + i = fcntl(fd, F_GETFD); + if (i >= 0) + fcntl(fd, F_SETFD, i | FD_CLOEXEC); + } + srand((getpid() << 16) ^ getuid() ^ tv.tv_sec ^ tv.tv_usec); + +#ifdef DO_JRAND_MIX + ul_jrand_seed[0] = getpid() ^ (tv.tv_sec & 0xFFFF); + ul_jrand_seed[1] = getppid() ^ (tv.tv_usec & 0xFFFF); + ul_jrand_seed[2] = (tv.tv_sec ^ tv.tv_usec) >> 16; +#endif + /* Crank the random number generator a few times */ + gettimeofday(&tv, 0); + for (i = (tv.tv_sec ^ tv.tv_usec) & 0x1F; i > 0; i--) + rand(); + return (fd); +} +#endif + +/* + * Generate a stream of random nbytes into buf. + * Use /dev/urandom if possible, and if not, + * use glibc pseudo-random functions. + */ +#ifndef _WIN32 +void +random_get_bytes(void *buf, size_t nbytes) +{ + size_t i, n = nbytes; + int fd = random_get_fd(); + int lose_counter = 0; + unsigned char *cp = (unsigned char *) buf; + + if (fd >= 0) { + while (n > 0) { + ssize_t x = read(fd, cp, n); + if (x <= 0) { + if (lose_counter++ > 16) + break; + continue; + } + n -= x; + cp += x; + lose_counter = 0; + } + + close(fd); + } + /* + * We do this all the time, but this is the only source of + * randomness if /dev/random/urandom is out to lunch. + */ + for (cp = buf, i = 0; i < nbytes; i++) + *cp++ ^= (rand() >> 7) & 0xFF; + +#ifdef DO_JRAND_MIX + { + unsigned short tmp_seed[3]; + + memcpy(tmp_seed, ul_jrand_seed, sizeof (tmp_seed)); + ul_jrand_seed[2] = ul_jrand_seed[2] ^ syscall(__NR_gettid); + for (cp = buf, i = 0; i < nbytes; i++) + *cp++ ^= (jrand48(tmp_seed) >> 7) & 0xFF; + memcpy(ul_jrand_seed, tmp_seed, + sizeof (ul_jrand_seed)-sizeof (unsigned short)); + } +#endif +} +#endif + +#ifdef TEST_PROGRAM +int +main(int argc __attribute__((__unused__)), + char *argv[] __attribute__((__unused__))) +{ + unsigned int v, i; + + /* generate and print 10 random numbers */ + for (i = 0; i < 10; i++) { + random_get_bytes(&v, sizeof (v)); + printf("%d\n", v); + } + + return (EXIT_SUCCESS); +} +#endif /* TEST_PROGRAM */ diff --git a/lib/os/windows/libuuid/randutils.h b/lib/os/windows/libuuid/randutils.h new file mode 100644 index 000000000000..e27456c61bc5 --- /dev/null +++ b/lib/os/windows/libuuid/randutils.h @@ -0,0 +1,13 @@ + +#ifndef UTIL_LINUX_RANDUTILS +#define UTIL_LINUX_RANDUTILS + +#ifdef HAVE_SRANDOM +#define srand(x) srandom(x) +#define rand() random() +#endif + +extern int random_get_fd(void); +extern void random_get_bytes(void *buf, size_t nbytes); + +#endif diff --git a/lib/os/windows/libuuid/test_uuid.c b/lib/os/windows/libuuid/test_uuid.c new file mode 100644 index 000000000000..a9d8b327b1b1 --- /dev/null +++ b/lib/os/windows/libuuid/test_uuid.c @@ -0,0 +1,181 @@ +/* + * tst_uuid.c --- test program from the UUID library + * + * Copyright (C) 1996, 1997, 1998 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#ifdef _WIN32 +#define _WIN32_WINNT 0x0500 +#include +#define UUID MYUUID +#endif + +#include +#include + +#include "uuid.h" + +static int +test_uuid(const char *uuid, int isValid) +{ + static const char *validStr[2] = {"invalid", "valid"}; + uuid_t uuidBits; + int parsedOk; + + parsedOk = uuid_parse(uuid, uuidBits) == 0; + + printf("%s is %s", uuid, validStr[isValid]); + if (parsedOk != isValid) { + printf(" but uuid_parse says %s\n", validStr[parsedOk]); + return (1); + } + printf(", OK\n"); + return (0); +} + +#ifdef __GNUC__ +#define ATTR(x) __attribute__(x) +#else +#define ATTR(x) +#endif + +int +main(int argc ATTR((unused)), char **argv ATTR((unused))) +{ + uuid_t buf, tst; + char str[100]; + struct timeval tv; + time_t time_reg; + unsigned char *cp; + int i; + int failed = 0; + int type, variant; + + uuid_generate(buf); + uuid_unparse(buf, str); + printf("UUID generate = %s\n", str); + printf("UUID: "); + for (i = 0, cp = (unsigned char *) &buf; i < 16; i++) { + printf("%02x", *cp++); + } + printf("\n"); + type = uuid_type(buf); variant = uuid_variant(buf); + printf("UUID type = %d, UUID variant = %d\n", type, variant); + if (variant != UUID_VARIANT_DCE) { + printf("Incorrect UUID Variant; was expecting DCE!\n"); + failed++; + } + printf("\n"); + + uuid_generate_random(buf); + uuid_unparse(buf, str); + printf("UUID random string = %s\n", str); + printf("UUID: "); + for (i = 0, cp = (unsigned char *) &buf; i < 16; i++) { + printf("%02x", *cp++); + } + printf("\n"); + type = uuid_type(buf); variant = uuid_variant(buf); + printf("UUID type = %d, UUID variant = %d\n", type, variant); + if (variant != UUID_VARIANT_DCE) { + printf("Incorrect UUID Variant; was expecting DCE!\n"); + failed++; + } + if (type != 4) { + printf("Incorrect UUID type; was expecting " + "4 (random type)!\n"); + failed++; + } + printf("\n"); + + uuid_generate_time(buf); + uuid_unparse(buf, str); + printf("UUID string = %s\n", str); + printf("UUID time: "); + for (i = 0, cp = (unsigned char *)&buf; i < 16; i++) { + printf("%02x", *cp++); + } + printf("\n"); + type = uuid_type(buf); variant = uuid_variant(buf); + printf("UUID type = %d, UUID variant = %d\n", type, variant); + if (variant != UUID_VARIANT_DCE) { + printf("Incorrect UUID Variant; was expecting DCE!\n"); + failed++; + } + if (type != 1) { + printf("Incorrect UUID type; was expecting " + "1 (time-based type)!\\n"); + failed++; + } + tv.tv_sec = 0; + tv.tv_usec = 0; + time_reg = uuid_time(buf, &tv); + printf("UUID time is: (%ld, %ld): %s\n", tv.tv_sec, tv.tv_usec, + ctime(&time_reg)); + uuid_parse(str, tst); + if (!uuid_compare(buf, tst)) + printf("UUID parse and compare succeeded.\n"); + else { + printf("UUID parse and compare failed!\n"); + failed++; + } + uuid_clear(tst); + if (uuid_is_null(tst)) + printf("UUID clear and is null succeeded.\n"); + else { + printf("UUID clear and is null failed!\n"); + failed++; + } + uuid_copy(buf, tst); + if (!uuid_compare(buf, tst)) + printf("UUID copy and compare succeeded.\n"); + else { + printf("UUID copy and compare failed!\n"); + failed++; + } + failed += test_uuid("84949cc5-4701-4a84-895b-354c584a981b", 1); + failed += test_uuid("84949CC5-4701-4A84-895B-354C584A981B", 1); + failed += test_uuid("84949cc5-4701-4a84-895b-354c584a981bc", 0); + failed += test_uuid("84949cc5-4701-4a84-895b-354c584a981", 0); + failed += test_uuid("84949cc5x4701-4a84-895b-354c584a981b", 0); + failed += test_uuid("84949cc504701-4a84-895b-354c584a981b", 0); + failed += test_uuid("84949cc5-470104a84-895b-354c584a981b", 0); + failed += test_uuid("84949cc5-4701-4a840895b-354c584a981b", 0); + failed += test_uuid("84949cc5-4701-4a84-895b0354c584a981b", 0); + failed += test_uuid("g4949cc5-4701-4a84-895b-354c584a981b", 0); + failed += test_uuid("84949cc5-4701-4a84-895b-354c584a981g", 0); + + if (failed) { + printf("%d failures.\n", failed); + exit(1); + } + return (0); +} diff --git a/lib/os/windows/libuuid/unpack.c b/lib/os/windows/libuuid/unpack.c new file mode 100644 index 000000000000..5b27f035df7f --- /dev/null +++ b/lib/os/windows/libuuid/unpack.c @@ -0,0 +1,63 @@ +/* + * Internal routine for unpacking UUID + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include +#include "uuidP.h" + +void +uuid_unpack(const uuid_t in, struct uuid *uu) +{ + const uint8_t *ptr = in; + uint32_t tmp; + + tmp = *ptr++; + tmp = (tmp << 8) | *ptr++; + tmp = (tmp << 8) | *ptr++; + tmp = (tmp << 8) | *ptr++; + uu->time_low = tmp; + + tmp = *ptr++; + tmp = (tmp << 8) | *ptr++; + uu->time_mid = tmp; + + tmp = *ptr++; + tmp = (tmp << 8) | *ptr++; + uu->time_hi_and_version = tmp; + + tmp = *ptr++; + tmp = (tmp << 8) | *ptr++; + uu->clock_seq = tmp; + + memcpy(uu->node, ptr, 6); +} diff --git a/lib/os/windows/libuuid/unparse.c b/lib/os/windows/libuuid/unparse.c new file mode 100644 index 000000000000..34fc18eb5e7e --- /dev/null +++ b/lib/os/windows/libuuid/unparse.c @@ -0,0 +1,80 @@ +/* + * unparse.c -- convert a UUID to string + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include + +#include "uuidP.h" + +static const char *fmt_lower = + "%08x-%04x-%04x-%02x%02x-%02x%02x%02x%02x%02x%02x"; + +static const char *fmt_upper = + "%08X-%04X-%04X-%02X%02X-%02X%02X%02X%02X%02X%02X"; + +#ifdef UUID_UNPARSE_DEFAULT_UPPER +#define FMT_DEFAULT fmt_upper +#else +#define FMT_DEFAULT fmt_lower +#endif + +static void +uuid_unparse_x(const uuid_t uu, char *out, const char *fmt) +{ + struct uuid uuid; + + uuid_unpack(uu, &uuid); + sprintf(out, fmt, + uuid.time_low, uuid.time_mid, uuid.time_hi_and_version, + uuid.clock_seq >> 8, uuid.clock_seq & 0xFF, + uuid.node[0], uuid.node[1], uuid.node[2], + uuid.node[3], uuid.node[4], uuid.node[5]); +} + +void +uuid_unparse_lower(const uuid_t uu, char *out) +{ + uuid_unparse_x(uu, out, fmt_lower); +} + +void +uuid_unparse_upper(const uuid_t uu, char *out) +{ + uuid_unparse_x(uu, out, fmt_upper); +} + +void +uuid_unparse(const uuid_t uu, char *out) +{ + uuid_unparse_x(uu, out, FMT_DEFAULT); +} diff --git a/lib/os/windows/libuuid/uuid.h b/lib/os/windows/libuuid/uuid.h new file mode 100644 index 000000000000..7d0b6add494c --- /dev/null +++ b/lib/os/windows/libuuid/uuid.h @@ -0,0 +1,108 @@ +/* + * Public include file for the UUID library + * + * Copyright (C) 1996, 1997, 1998 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#ifndef _UUID_UUID_H +#define _UUID_UUID_H + +#include +#ifndef _WIN32 +#include +#endif +#include + +typedef unsigned char uuid_t[16]; + +/* UUID Variant definitions */ +#define UUID_VARIANT_NCS 0 +#define UUID_VARIANT_DCE 1 +#define UUID_VARIANT_MICROSOFT 2 +#define UUID_VARIANT_OTHER 3 + +/* UUID Type definitions */ +#define UUID_TYPE_DCE_TIME 1 +#define UUID_TYPE_DCE_RANDOM 4 + +/* Allow UUID constants to be defined */ +#ifdef __GNUC__ +#define UUID_DEFINE(name, u0, u1, u2, u3, u4, u5, u6, u7, u8, u9, u10, \ + u11, u12, u13, u14, u15) \ + static const uuid_t name __attribute__((unused)) = {u0, u1, u2, u3, \ + u4, u5, u6, u7, u8, u9, u10, u11, u12, u13, u14, u15} +#else +#define UUID_DEFINE(name, u0, u1, u2, u3, u4, u5, u6, u7, u8, u9, u10, \ + u11, u12, u13, u14, u15) \ + static const uuid_t name = {u0, u1, u2, u3, u4, u5, u6, u7, u8, u9, \ + u10, u11, u12, u13, u14, u15} +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +/* clear.c */ +void uuid_clear(uuid_t uu); + +/* compare.c */ +int uuid_compare(const uuid_t uu1, const uuid_t uu2); + +/* copy.c */ +void uuid_copy(uuid_t dst, const uuid_t src); + +/* gen_uuid.c */ +void uuid_generate(uuid_t out); +void uuid_generate_random(uuid_t out); +void uuid_generate_time(uuid_t out); +int uuid_generate_time_safe(uuid_t out); + +/* isnull.c */ +int uuid_is_null(const uuid_t uu); + +/* parse.c */ +int uuid_parse(const char *in, uuid_t uu); + +/* unparse.c */ +void uuid_unparse(const uuid_t uu, char *out); +void uuid_unparse_lower(const uuid_t uu, char *out); +void uuid_unparse_upper(const uuid_t uu, char *out); + +/* uuid_time.c */ +time_t uuid_time(const uuid_t uu, struct timeval *ret_tv); +int uuid_type(const uuid_t uu); +int uuid_variant(const uuid_t uu); + +#ifdef __cplusplus +} +#endif + +#endif /* _UUID_UUID_H */ diff --git a/lib/os/windows/libuuid/uuidP.h b/lib/os/windows/libuuid/uuidP.h new file mode 100644 index 000000000000..9ab5b7e53d73 --- /dev/null +++ b/lib/os/windows/libuuid/uuidP.h @@ -0,0 +1,61 @@ +/* + * uuid.h -- private header file for uuids + * + * Copyright (C) 1996, 1997 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#include +#include + +#include "uuid.h" + +#define LIBUUID_CLOCK_FILE "/var/lib/libuuid/clock.txt" + +/* + * Offset between 15-Oct-1582 and 1-Jan-70 + */ +#define TIME_OFFSET_HIGH 0x01B21DD2 +#define TIME_OFFSET_LOW 0x13814000 + +struct uuid { + uint32_t time_low; + uint16_t time_mid; + uint16_t time_hi_and_version; + uint16_t clock_seq; + uint8_t node[6]; +}; + + +/* + * prototypes + */ +void uuid_pack(const struct uuid *uu, uuid_t ptr); +void uuid_unpack(const uuid_t in, struct uuid *uu); diff --git a/lib/os/windows/libuuid/uuid_time.c b/lib/os/windows/libuuid/uuid_time.c new file mode 100644 index 000000000000..3691edd4668a --- /dev/null +++ b/lib/os/windows/libuuid/uuid_time.c @@ -0,0 +1,175 @@ +/* + * uuid_time.c --- Interpret the time field from a uuid. This program + * violates the UUID abstraction barrier by reaching into the guts + * of a UUID and interpreting it. + * + * Copyright (C) 1998, 1999 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#ifdef _WIN32 +#define _WIN32_WINNT 0x0500 +#include +#define UUID MYUUID +#endif + +#include +#ifdef HAVE_UNISTD_H +#include +#endif +#include +#include +#ifdef HAVE_SYS_TIME_H +#include +#endif +#include + +#include "uuidP.h" + +time_t +uuid_time(const uuid_t uu, struct timeval *ret_tv) +{ + struct timeval tv; + struct uuid uuid; + uint32_t high; + uint64_t clock_reg; + + uuid_unpack(uu, &uuid); + + high = uuid.time_mid | ((uuid.time_hi_and_version & 0xFFF) << 16); + clock_reg = uuid.time_low | ((uint64_t)high << 32); + + clock_reg -= (((uint64_t)0x01B21DD2) << 32) + 0x13814000; + tv.tv_sec = clock_reg / 10000000; + tv.tv_usec = (clock_reg % 10000000) / 10; + + if (ret_tv) + *ret_tv = tv; + + return (tv.tv_sec); +} + +int +uuid_type(const uuid_t uu) +{ + struct uuid uuid; + + uuid_unpack(uu, &uuid); + return ((uuid.time_hi_and_version >> 12) & 0xF); +} + +int +uuid_variant(const uuid_t uu) +{ + struct uuid uuid; + int var; + + uuid_unpack(uu, &uuid); + var = uuid.clock_seq; + + if ((var & 0x8000) == 0) + return (UUID_VARIANT_NCS); + if ((var & 0x4000) == 0) + return (UUID_VARIANT_DCE); + if ((var & 0x2000) == 0) + return (UUID_VARIANT_MICROSOFT); + return (UUID_VARIANT_OTHER); +} + +#ifdef DEBUG +static const char * +variant_string(int variant) +{ + switch (variant) { + case UUID_VARIANT_NCS: + return ("NCS"); + case UUID_VARIANT_DCE: + return ("DCE"); + case UUID_VARIANT_MICROSOFT: + return ("Microsoft"); + default: + return ("Other"); + } +} + + +int +main(int argc, char **argv) +{ + uuid_t buf; + time_t time_reg; + struct timeval tv; + int type, variant; + + if (argc != 2) { + fprintf(stderr, "Usage: %s uuid\n", argv[0]); + exit(1); + } + if (uuid_parse(argv[1], buf)) { + fprintf(stderr, "Invalid UUID: %s\n", argv[1]); + exit(1); + } + variant = uuid_variant(buf); + type = uuid_type(buf); + time_reg = uuid_time(buf, &tv); + + printf("UUID variant is %d (%s)\n", variant, variant_string(variant)); + if (variant != UUID_VARIANT_DCE) { + printf("Warning: This program only knows how to interpret " + "DCE UUIDs.\n\tThe rest of the output is likely " + "to be incorrect!!\n"); + } + printf("UUID type is %d", type); + switch (type) { + case 1: + printf(" (time based)\n"); + break; + case 2: + printf(" (DCE)\n"); + break; + case 3: + printf(" (name-based)\n"); + break; + case 4: + printf(" (random)\n"); + break; + default: + printf("\n"); + } + if (type != 1) { + printf("Warning: not a time-based UUID, so UUID time " + "decoding will likely not work!\n"); + } + printf("UUID time is: (%ld, %ld): %s\n", tv.tv_sec, tv.tv_usec, + ctime(&time_reg)); + + return (0); +} +#endif diff --git a/lib/os/windows/libuuid/uuidd.h b/lib/os/windows/libuuid/uuidd.h new file mode 100644 index 000000000000..b71983768ea0 --- /dev/null +++ b/lib/os/windows/libuuid/uuidd.h @@ -0,0 +1,54 @@ +/* + * Definitions used by the uuidd daemon + * + * Copyright (C) 2007 Theodore Ts'o. + * + * %Begin-Header% + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, and the entire permission notice in its entirety, + * including the disclaimer of warranties. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED + * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, ALL OF + * WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT + * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE + * USE OF THIS SOFTWARE, EVEN IF NOT ADVISED OF THE POSSIBILITY OF SUCH + * DAMAGE. + * %End-Header% + */ + +#ifndef _UUID_UUIDD_H +#define _UUID_UUIDD_H + +#define UUIDD_DIR _PATH_LOCALSTATEDIR "/uuidd" +#define UUIDD_SOCKET_PATH UUIDD_DIR "/request" +#define UUIDD_PIDFILE_PATH UUIDD_DIR "/uuidd.pid" +#define UUIDD_PATH "/usr/sbin/uuidd" + +#define UUIDD_OP_GETPID 0 +#define UUIDD_OP_GET_MAXOP 1 +#define UUIDD_OP_TIME_UUID 2 +#define UUIDD_OP_RANDOM_UUID 3 +#define UUIDD_OP_BULK_TIME_UUID 4 +#define UUIDD_OP_BULK_RANDOM_UUID 5 +#define UUIDD_MAX_OP UUIDD_OP_BULK_RANDOM_UUID + +extern int __uuid_generate_time(uuid_t out, int *num); +extern void __uuid_generate_random(uuid_t out, int *num); + +#endif /* _UUID_UUID_H */ diff --git a/lib/os/windows/zlib-1.2.13/CMakeLists.txt b/lib/os/windows/zlib-1.2.13/CMakeLists.txt new file mode 100644 index 000000000000..841ba55231df --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/CMakeLists.txt @@ -0,0 +1,34 @@ + +use_clang() + +set(ZLIB_SOURCES + adler32.c + compress.c + crc32.c + deflate.c + #gzio.c + gzlib.c + gzread.c + gzwrite.c + gzclose.c + infback.c + inffast.c + inflate.c + inftrees.c + trees.c + uncompr.c + zutil.c + #contrib/masmx64/inffas8664.c + #contrib/masmx86/gvmat32c.c + contrib/minizip/ioapi.c + contrib/minizip/unzip.c + contrib/minizip/zip.c +) + +wdk_add_library(zlibkern ${ZLIB_SOURCES}) +target_include_directories(zlibkern PUBLIC "") +target_link_libraries(zlibkern PRIVATE splkern) +target_compile_definitions(zlibkern PUBLIC Z_PREFIX MY_ZCALLOC _KERNEL) + +add_library(zlib ${ZLIB_SOURCES}) +target_include_directories(zlib PUBLIC "") diff --git a/lib/os/windows/zlib-1.2.13/ChangeLog b/lib/os/windows/zlib-1.2.13/ChangeLog new file mode 100644 index 000000000000..457526bc6a51 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/ChangeLog @@ -0,0 +1,1590 @@ + + ChangeLog file for zlib + +Changes in 1.2.13 (13 Oct 2022) +- Fix configure issue that discarded provided CC definition +- Correct incorrect inputs provided to the CRC functions +- Repair prototypes and exporting of new CRC functions +- Fix inflateBack to detect invalid input with distances too far +- Have infback() deliver all of the available output up to any error +- Fix a bug when getting a gzip header extra field with inflate() +- Fix bug in block type selection when Z_FIXED used +- Tighten deflateBound bounds +- Remove deleted assembler code references +- Various portability and appearance improvements + +Changes in 1.2.12 (27 Mar 2022) +- Cygwin does not have _wopen(), so do not create gzopen_w() there +- Permit a deflateParams() parameter change as soon as possible +- Limit hash table inserts after switch from stored deflate +- Fix bug when window full in deflate_stored() +- Fix CLEAR_HASH macro to be usable as a single statement +- Avoid a conversion error in gzseek when off_t type too small +- Have Makefile return non-zero error code on test failure +- Avoid some conversion warnings in gzread.c and gzwrite.c +- Update use of errno for newer Windows CE versions +- Small speedup to inflate [psumbera] +- Return an error if the gzputs string length can't fit in an int +- Add address checking in clang to -w option of configure +- Don't compute check value for raw inflate if asked to validate +- Handle case where inflateSync used when header never processed +- Avoid the use of ptrdiff_t +- Avoid an undefined behavior of memcpy() in gzappend() +- Avoid undefined behaviors of memcpy() in gz*printf() +- Avoid an undefined behavior of memcpy() in _tr_stored_block() +- Make the names in functions declarations identical to definitions +- Remove old assembler code in which bugs have manifested +- Fix deflateEnd() to not report an error at start of raw deflate +- Add legal disclaimer to README +- Emphasize the need to continue decompressing gzip members +- Correct the initialization requirements for deflateInit2() +- Fix a bug that can crash deflate on some input when using Z_FIXED +- Assure that the number of bits for deflatePrime() is valid +- Use a structure to make globals in enough.c evident +- Use a macro for the printf format of big_t in enough.c +- Clean up code style in enough.c, update version +- Use inline function instead of macro for index in enough.c +- Clarify that prefix codes are counted in enough.c +- Show all the codes for the maximum tables size in enough.c +- Add gznorm.c example, which normalizes gzip files +- Fix the zran.c example to work on a multiple-member gzip file +- Add tables for crc32_combine(), to speed it up by a factor of 200 +- Add crc32_combine_gen() and crc32_combine_op() for fast combines +- Speed up software CRC-32 computation by a factor of 1.5 to 3 +- Use atomic test and set, if available, for dynamic CRC tables +- Don't bother computing check value after successful inflateSync() +- Correct comment in crc32.c +- Add use of the ARMv8 crc32 instructions when requested +- Use ARM crc32 instructions if the ARM architecture has them +- Explicitly note that the 32-bit check values are 32 bits +- Avoid adding empty gzip member after gzflush with Z_FINISH +- Fix memory leak on error in gzlog.c +- Fix error in comment on the polynomial representation of a byte +- Clarify gz* function interfaces, referring to parameter names +- Change macro name in inflate.c to avoid collision in VxWorks +- Correct typo in blast.c +- Improve portability of contrib/minizip +- Fix indentation in minizip's zip.c +- Replace black/white with allow/block. (theresa-m) +- minizip warning fix if MAXU32 already defined. (gvollant) +- Fix unztell64() in minizip to work past 4GB. (Daniël Hörchner) +- Clean up minizip to reduce warnings for testing +- Add fallthrough comments for gcc +- Eliminate use of ULL constants +- Separate out address sanitizing from warnings in configure +- Remove destructive aspects of make distclean +- Check for cc masquerading as gcc or clang in configure +- Fix crc32.c to compile local functions only if used + +Changes in 1.2.11 (15 Jan 2017) +- Fix deflate stored bug when pulling last block from window +- Permit immediate deflateParams changes before any deflate input + +Changes in 1.2.10 (2 Jan 2017) +- Avoid warnings on snprintf() return value +- Fix bug in deflate_stored() for zero-length input +- Fix bug in gzwrite.c that produced corrupt gzip files +- Remove files to be installed before copying them in Makefile.in +- Add warnings when compiling with assembler code + +Changes in 1.2.9 (31 Dec 2016) +- Fix contrib/minizip to permit unzipping with desktop API [Zouzou] +- Improve contrib/blast to return unused bytes +- Assure that gzoffset() is correct when appending +- Improve compress() and uncompress() to support large lengths +- Fix bug in test/example.c where error code not saved +- Remedy Coverity warning [Randers-Pehrson] +- Improve speed of gzprintf() in transparent mode +- Fix inflateInit2() bug when windowBits is 16 or 32 +- Change DEBUG macro to ZLIB_DEBUG +- Avoid uninitialized access by gzclose_w() +- Allow building zlib outside of the source directory +- Fix bug that accepted invalid zlib header when windowBits is zero +- Fix gzseek() problem on MinGW due to buggy _lseeki64 there +- Loop on write() calls in gzwrite.c in case of non-blocking I/O +- Add --warn (-w) option to ./configure for more compiler warnings +- Reject a window size of 256 bytes if not using the zlib wrapper +- Fix bug when level 0 used with Z_HUFFMAN or Z_RLE +- Add --debug (-d) option to ./configure to define ZLIB_DEBUG +- Fix bugs in creating a very large gzip header +- Add uncompress2() function, which returns the input size used +- Assure that deflateParams() will not switch functions mid-block +- Dramatically speed up deflation for level 0 (storing) +- Add gzfread(), duplicating the interface of fread() +- Add gzfwrite(), duplicating the interface of fwrite() +- Add deflateGetDictionary() function +- Use snprintf() for later versions of Microsoft C +- Fix *Init macros to use z_ prefix when requested +- Replace as400 with os400 for OS/400 support [Monnerat] +- Add crc32_z() and adler32_z() functions with size_t lengths +- Update Visual Studio project files [AraHaan] + +Changes in 1.2.8 (28 Apr 2013) +- Update contrib/minizip/iowin32.c for Windows RT [Vollant] +- Do not force Z_CONST for C++ +- Clean up contrib/vstudio [Roß] +- Correct spelling error in zlib.h +- Fix mixed line endings in contrib/vstudio + +Changes in 1.2.7.3 (13 Apr 2013) +- Fix version numbers and DLL names in contrib/vstudio/*/zlib.rc + +Changes in 1.2.7.2 (13 Apr 2013) +- Change check for a four-byte type back to hexadecimal +- Fix typo in win32/Makefile.msc +- Add casts in gzwrite.c for pointer differences + +Changes in 1.2.7.1 (24 Mar 2013) +- Replace use of unsafe string functions with snprintf if available +- Avoid including stddef.h on Windows for Z_SOLO compile [Niessink] +- Fix gzgetc undefine when Z_PREFIX set [Turk] +- Eliminate use of mktemp in Makefile (not always available) +- Fix bug in 'F' mode for gzopen() +- Add inflateGetDictionary() function +- Correct comment in deflate.h +- Use _snprintf for snprintf in Microsoft C +- On Darwin, only use /usr/bin/libtool if libtool is not Apple +- Delete "--version" file if created by "ar --version" [Richard G.] +- Fix configure check for veracity of compiler error return codes +- Fix CMake compilation of static lib for MSVC2010 x64 +- Remove unused variable in infback9.c +- Fix argument checks in gzlog_compress() and gzlog_write() +- Clean up the usage of z_const and respect const usage within zlib +- Clean up examples/gzlog.[ch] comparisons of different types +- Avoid shift equal to bits in type (caused endless loop) +- Fix uninitialized value bug in gzputc() introduced by const patches +- Fix memory allocation error in examples/zran.c [Nor] +- Fix bug where gzopen(), gzclose() would write an empty file +- Fix bug in gzclose() when gzwrite() runs out of memory +- Check for input buffer malloc failure in examples/gzappend.c +- Add note to contrib/blast to use binary mode in stdio +- Fix comparisons of differently signed integers in contrib/blast +- Check for invalid code length codes in contrib/puff +- Fix serious but very rare decompression bug in inftrees.c +- Update inflateBack() comments, since inflate() can be faster +- Use underscored I/O function names for WINAPI_FAMILY +- Add _tr_flush_bits to the external symbols prefixed by --zprefix +- Add contrib/vstudio/vc10 pre-build step for static only +- Quote --version-script argument in CMakeLists.txt +- Don't specify --version-script on Apple platforms in CMakeLists.txt +- Fix casting error in contrib/testzlib/testzlib.c +- Fix types in contrib/minizip to match result of get_crc_table() +- Simplify contrib/vstudio/vc10 with 'd' suffix +- Add TOP support to win32/Makefile.msc +- Support i686 and amd64 assembler builds in CMakeLists.txt +- Fix typos in the use of _LARGEFILE64_SOURCE in zconf.h +- Add vc11 and vc12 build files to contrib/vstudio +- Add gzvprintf() as an undocumented function in zlib +- Fix configure for Sun shell +- Remove runtime check in configure for four-byte integer type +- Add casts and consts to ease user conversion to C++ +- Add man pages for minizip and miniunzip +- In Makefile uninstall, don't rm if preceding cd fails +- Do not return Z_BUF_ERROR if deflateParam() has nothing to write + +Changes in 1.2.7 (2 May 2012) +- Replace use of memmove() with a simple copy for portability +- Test for existence of strerror +- Restore gzgetc_ for backward compatibility with 1.2.6 +- Fix build with non-GNU make on Solaris +- Require gcc 4.0 or later on Mac OS X to use the hidden attribute +- Include unistd.h for Watcom C +- Use __WATCOMC__ instead of __WATCOM__ +- Do not use the visibility attribute if NO_VIZ defined +- Improve the detection of no hidden visibility attribute +- Avoid using __int64 for gcc or solo compilation +- Cast to char * in gzprintf to avoid warnings [Zinser] +- Fix make_vms.com for VAX [Zinser] +- Don't use library or built-in byte swaps +- Simplify test and use of gcc hidden attribute +- Fix bug in gzclose_w() when gzwrite() fails to allocate memory +- Add "x" (O_EXCL) and "e" (O_CLOEXEC) modes support to gzopen() +- Fix bug in test/minigzip.c for configure --solo +- Fix contrib/vstudio project link errors [Mohanathas] +- Add ability to choose the builder in make_vms.com [Schweda] +- Add DESTDIR support to mingw32 win32/Makefile.gcc +- Fix comments in win32/Makefile.gcc for proper usage +- Allow overriding the default install locations for cmake +- Generate and install the pkg-config file with cmake +- Build both a static and a shared version of zlib with cmake +- Include version symbols for cmake builds +- If using cmake with MSVC, add the source directory to the includes +- Remove unneeded EXTRA_CFLAGS from win32/Makefile.gcc [Truta] +- Move obsolete emx makefile to old [Truta] +- Allow the use of -Wundef when compiling or using zlib +- Avoid the use of the -u option with mktemp +- Improve inflate() documentation on the use of Z_FINISH +- Recognize clang as gcc +- Add gzopen_w() in Windows for wide character path names +- Rename zconf.h in CMakeLists.txt to move it out of the way +- Add source directory in CMakeLists.txt for building examples +- Look in build directory for zlib.pc in CMakeLists.txt +- Remove gzflags from zlibvc.def in vc9 and vc10 +- Fix contrib/minizip compilation in the MinGW environment +- Update ./configure for Solaris, support --64 [Mooney] +- Remove -R. from Solaris shared build (possible security issue) +- Avoid race condition for parallel make (-j) running example +- Fix type mismatch between get_crc_table() and crc_table +- Fix parsing of version with "-" in CMakeLists.txt [Snider, Ziegler] +- Fix the path to zlib.map in CMakeLists.txt +- Force the native libtool in Mac OS X to avoid GNU libtool [Beebe] +- Add instructions to win32/Makefile.gcc for shared install [Torri] + +Changes in 1.2.6.1 (12 Feb 2012) +- Avoid the use of the Objective-C reserved name "id" +- Include io.h in gzguts.h for Microsoft compilers +- Fix problem with ./configure --prefix and gzgetc macro +- Include gz_header definition when compiling zlib solo +- Put gzflags() functionality back in zutil.c +- Avoid library header include in crc32.c for Z_SOLO +- Use name in GCC_CLASSIC as C compiler for coverage testing, if set +- Minor cleanup in contrib/minizip/zip.c [Vollant] +- Update make_vms.com [Zinser] +- Remove unnecessary gzgetc_ function +- Use optimized byte swap operations for Microsoft and GNU [Snyder] +- Fix minor typo in zlib.h comments [Rzesniowiecki] + +Changes in 1.2.6 (29 Jan 2012) +- Update the Pascal interface in contrib/pascal +- Fix function numbers for gzgetc_ in zlibvc.def files +- Fix configure.ac for contrib/minizip [Schiffer] +- Fix large-entry detection in minizip on 64-bit systems [Schiffer] +- Have ./configure use the compiler return code for error indication +- Fix CMakeLists.txt for cross compilation [McClure] +- Fix contrib/minizip/zip.c for 64-bit architectures [Dalsnes] +- Fix compilation of contrib/minizip on FreeBSD [Marquez] +- Correct suggested usages in win32/Makefile.msc [Shachar, Horvath] +- Include io.h for Turbo C / Borland C on all platforms [Truta] +- Make version explicit in contrib/minizip/configure.ac [Bosmans] +- Avoid warning for no encryption in contrib/minizip/zip.c [Vollant] +- Minor cleanup up contrib/minizip/unzip.c [Vollant] +- Fix bug when compiling minizip with C++ [Vollant] +- Protect for long name and extra fields in contrib/minizip [Vollant] +- Avoid some warnings in contrib/minizip [Vollant] +- Add -I../.. -L../.. to CFLAGS for minizip and miniunzip +- Add missing libs to minizip linker command +- Add support for VPATH builds in contrib/minizip +- Add an --enable-demos option to contrib/minizip/configure +- Add the generation of configure.log by ./configure +- Exit when required parameters not provided to win32/Makefile.gcc +- Have gzputc return the character written instead of the argument +- Use the -m option on ldconfig for BSD systems [Tobias] +- Correct in zlib.map when deflateResetKeep was added + +Changes in 1.2.5.3 (15 Jan 2012) +- Restore gzgetc function for binary compatibility +- Do not use _lseeki64 under Borland C++ [Truta] +- Update win32/Makefile.msc to build test/*.c [Truta] +- Remove old/visualc6 given CMakefile and other alternatives +- Update AS400 build files and documentation [Monnerat] +- Update win32/Makefile.gcc to build test/*.c [Truta] +- Permit stronger flushes after Z_BLOCK flushes +- Avoid extraneous empty blocks when doing empty flushes +- Permit Z_NULL arguments to deflatePending +- Allow deflatePrime() to insert bits in the middle of a stream +- Remove second empty static block for Z_PARTIAL_FLUSH +- Write out all of the available bits when using Z_BLOCK +- Insert the first two strings in the hash table after a flush + +Changes in 1.2.5.2 (17 Dec 2011) +- fix ld error: unable to find version dependency 'ZLIB_1.2.5' +- use relative symlinks for shared libs +- Avoid searching past window for Z_RLE strategy +- Assure that high-water mark initialization is always applied in deflate +- Add assertions to fill_window() in deflate.c to match comments +- Update python link in README +- Correct spelling error in gzread.c +- Fix bug in gzgets() for a concatenated empty gzip stream +- Correct error in comment for gz_make() +- Change gzread() and related to ignore junk after gzip streams +- Allow gzread() and related to continue after gzclearerr() +- Allow gzrewind() and gzseek() after a premature end-of-file +- Simplify gzseek() now that raw after gzip is ignored +- Change gzgetc() to a macro for speed (~40% speedup in testing) +- Fix gzclose() to return the actual error last encountered +- Always add large file support for windows +- Include zconf.h for windows large file support +- Include zconf.h.cmakein for windows large file support +- Update zconf.h.cmakein on make distclean +- Merge vestigial vsnprintf determination from zutil.h to gzguts.h +- Clarify how gzopen() appends in zlib.h comments +- Correct documentation of gzdirect() since junk at end now ignored +- Add a transparent write mode to gzopen() when 'T' is in the mode +- Update python link in zlib man page +- Get inffixed.h and MAKEFIXED result to match +- Add a ./config --solo option to make zlib subset with no library use +- Add undocumented inflateResetKeep() function for CAB file decoding +- Add --cover option to ./configure for gcc coverage testing +- Add #define ZLIB_CONST option to use const in the z_stream interface +- Add comment to gzdopen() in zlib.h to use dup() when using fileno() +- Note behavior of uncompress() to provide as much data as it can +- Add files in contrib/minizip to aid in building libminizip +- Split off AR options in Makefile.in and configure +- Change ON macro to Z_ARG to avoid application conflicts +- Facilitate compilation with Borland C++ for pragmas and vsnprintf +- Include io.h for Turbo C / Borland C++ +- Move example.c and minigzip.c to test/ +- Simplify incomplete code table filling in inflate_table() +- Remove code from inflate.c and infback.c that is impossible to execute +- Test the inflate code with full coverage +- Allow deflateSetDictionary, inflateSetDictionary at any time (in raw) +- Add deflateResetKeep and fix inflateResetKeep to retain dictionary +- Fix gzwrite.c to accommodate reduced memory zlib compilation +- Have inflate() with Z_FINISH avoid the allocation of a window +- Do not set strm->adler when doing raw inflate +- Fix gzeof() to behave just like feof() when read is not past end of file +- Fix bug in gzread.c when end-of-file is reached +- Avoid use of Z_BUF_ERROR in gz* functions except for premature EOF +- Document gzread() capability to read concurrently written files +- Remove hard-coding of resource compiler in CMakeLists.txt [Blammo] + +Changes in 1.2.5.1 (10 Sep 2011) +- Update FAQ entry on shared builds (#13) +- Avoid symbolic argument to chmod in Makefile.in +- Fix bug and add consts in contrib/puff [Oberhumer] +- Update contrib/puff/zeros.raw test file to have all block types +- Add full coverage test for puff in contrib/puff/Makefile +- Fix static-only-build install in Makefile.in +- Fix bug in unzGetCurrentFileInfo() in contrib/minizip [Kuno] +- Add libz.a dependency to shared in Makefile.in for parallel builds +- Spell out "number" (instead of "nb") in zlib.h for total_in, total_out +- Replace $(...) with `...` in configure for non-bash sh [Bowler] +- Add darwin* to Darwin* and solaris* to SunOS\ 5* in configure [Groffen] +- Add solaris* to Linux* in configure to allow gcc use [Groffen] +- Add *bsd* to Linux* case in configure [Bar-Lev] +- Add inffast.obj to dependencies in win32/Makefile.msc +- Correct spelling error in deflate.h [Kohler] +- Change libzdll.a again to libz.dll.a (!) in win32/Makefile.gcc +- Add test to configure for GNU C looking for gcc in output of $cc -v +- Add zlib.pc generation to win32/Makefile.gcc [Weigelt] +- Fix bug in zlib.h for _FILE_OFFSET_BITS set and _LARGEFILE64_SOURCE not +- Add comment in zlib.h that adler32_combine with len2 < 0 makes no sense +- Make NO_DIVIDE option in adler32.c much faster (thanks to John Reiser) +- Make stronger test in zconf.h to include unistd.h for LFS +- Apply Darwin patches for 64-bit file offsets to contrib/minizip [Slack] +- Fix zlib.h LFS support when Z_PREFIX used +- Add updated as400 support (removed from old) [Monnerat] +- Avoid deflate sensitivity to volatile input data +- Avoid division in adler32_combine for NO_DIVIDE +- Clarify the use of Z_FINISH with deflateBound() amount of space +- Set binary for output file in puff.c +- Use u4 type for crc_table to avoid conversion warnings +- Apply casts in zlib.h to avoid conversion warnings +- Add OF to prototypes for adler32_combine_ and crc32_combine_ [Miller] +- Improve inflateSync() documentation to note indeterminacy +- Add deflatePending() function to return the amount of pending output +- Correct the spelling of "specification" in FAQ [Randers-Pehrson] +- Add a check in configure for stdarg.h, use for gzprintf() +- Check that pointers fit in ints when gzprint() compiled old style +- Add dummy name before $(SHAREDLIBV) in Makefile [Bar-Lev, Bowler] +- Delete line in configure that adds -L. libz.a to LDFLAGS [Weigelt] +- Add debug records in assembler code [Londer] +- Update RFC references to use http://tools.ietf.org/html/... [Li] +- Add --archs option, use of libtool to configure for Mac OS X [Borstel] + +Changes in 1.2.5 (19 Apr 2010) +- Disable visibility attribute in win32/Makefile.gcc [Bar-Lev] +- Default to libdir as sharedlibdir in configure [Nieder] +- Update copyright dates on modified source files +- Update trees.c to be able to generate modified trees.h +- Exit configure for MinGW, suggesting win32/Makefile.gcc +- Check for NULL path in gz_open [Homurlu] + +Changes in 1.2.4.5 (18 Apr 2010) +- Set sharedlibdir in configure [Torok] +- Set LDFLAGS in Makefile.in [Bar-Lev] +- Avoid mkdir objs race condition in Makefile.in [Bowler] +- Add ZLIB_INTERNAL in front of internal inter-module functions and arrays +- Define ZLIB_INTERNAL to hide internal functions and arrays for GNU C +- Don't use hidden attribute when it is a warning generator (e.g. Solaris) + +Changes in 1.2.4.4 (18 Apr 2010) +- Fix CROSS_PREFIX executable testing, CHOST extract, mingw* [Torok] +- Undefine _LARGEFILE64_SOURCE in zconf.h if it is zero, but not if empty +- Try to use bash or ksh regardless of functionality of /bin/sh +- Fix configure incompatibility with NetBSD sh +- Remove attempt to run under bash or ksh since have better NetBSD fix +- Fix win32/Makefile.gcc for MinGW [Bar-Lev] +- Add diagnostic messages when using CROSS_PREFIX in configure +- Added --sharedlibdir option to configure [Weigelt] +- Use hidden visibility attribute when available [Frysinger] + +Changes in 1.2.4.3 (10 Apr 2010) +- Only use CROSS_PREFIX in configure for ar and ranlib if they exist +- Use CROSS_PREFIX for nm [Bar-Lev] +- Assume _LARGEFILE64_SOURCE defined is equivalent to true +- Avoid use of undefined symbols in #if with && and || +- Make *64 prototypes in gzguts.h consistent with functions +- Add -shared load option for MinGW in configure [Bowler] +- Move z_off64_t to public interface, use instead of off64_t +- Remove ! from shell test in configure (not portable to Solaris) +- Change +0 macro tests to -0 for possibly increased portability + +Changes in 1.2.4.2 (9 Apr 2010) +- Add consistent carriage returns to readme.txt's in masmx86 and masmx64 +- Really provide prototypes for *64 functions when building without LFS +- Only define unlink() in minigzip.c if unistd.h not included +- Update README to point to contrib/vstudio project files +- Move projects/vc6 to old/ and remove projects/ +- Include stdlib.h in minigzip.c for setmode() definition under WinCE +- Clean up assembler builds in win32/Makefile.msc [Rowe] +- Include sys/types.h for Microsoft for off_t definition +- Fix memory leak on error in gz_open() +- Symbolize nm as $NM in configure [Weigelt] +- Use TEST_LDSHARED instead of LDSHARED to link test programs [Weigelt] +- Add +0 to _FILE_OFFSET_BITS and _LFS64_LARGEFILE in case not defined +- Fix bug in gzeof() to take into account unused input data +- Avoid initialization of structures with variables in puff.c +- Updated win32/README-WIN32.txt [Rowe] + +Changes in 1.2.4.1 (28 Mar 2010) +- Remove the use of [a-z] constructs for sed in configure [gentoo 310225] +- Remove $(SHAREDLIB) from LIBS in Makefile.in [Creech] +- Restore "for debugging" comment on sprintf() in gzlib.c +- Remove fdopen for MVS from gzguts.h +- Put new README-WIN32.txt in win32 [Rowe] +- Add check for shell to configure and invoke another shell if needed +- Fix big fat stinking bug in gzseek() on uncompressed files +- Remove vestigial F_OPEN64 define in zutil.h +- Set and check the value of _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE +- Avoid errors on non-LFS systems when applications define LFS macros +- Set EXE to ".exe" in configure for MINGW [Kahle] +- Match crc32() in crc32.c exactly to the prototype in zlib.h [Sherrill] +- Add prefix for cross-compilation in win32/makefile.gcc [Bar-Lev] +- Add DLL install in win32/makefile.gcc [Bar-Lev] +- Allow Linux* or linux* from uname in configure [Bar-Lev] +- Allow ldconfig to be redefined in configure and Makefile.in [Bar-Lev] +- Add cross-compilation prefixes to configure [Bar-Lev] +- Match type exactly in gz_load() invocation in gzread.c +- Match type exactly of zcalloc() in zutil.c to zlib.h alloc_func +- Provide prototypes for *64 functions when building zlib without LFS +- Don't use -lc when linking shared library on MinGW +- Remove errno.h check in configure and vestigial errno code in zutil.h + +Changes in 1.2.4 (14 Mar 2010) +- Fix VER3 extraction in configure for no fourth subversion +- Update zlib.3, add docs to Makefile.in to make .pdf out of it +- Add zlib.3.pdf to distribution +- Don't set error code in gzerror() if passed pointer is NULL +- Apply destination directory fixes to CMakeLists.txt [Lowman] +- Move #cmakedefine's to a new zconf.in.cmakein +- Restore zconf.h for builds that don't use configure or cmake +- Add distclean to dummy Makefile for convenience +- Update and improve INDEX, README, and FAQ +- Update CMakeLists.txt for the return of zconf.h [Lowman] +- Update contrib/vstudio/vc9 and vc10 [Vollant] +- Change libz.dll.a back to libzdll.a in win32/Makefile.gcc +- Apply license and readme changes to contrib/asm686 [Raiter] +- Check file name lengths and add -c option in minigzip.c [Li] +- Update contrib/amd64 and contrib/masmx86/ [Vollant] +- Avoid use of "eof" parameter in trees.c to not shadow library variable +- Update make_vms.com for removal of zlibdefs.h [Zinser] +- Update assembler code and vstudio projects in contrib [Vollant] +- Remove outdated assembler code contrib/masm686 and contrib/asm586 +- Remove old vc7 and vc8 from contrib/vstudio +- Update win32/Makefile.msc, add ZLIB_VER_SUBREVISION [Rowe] +- Fix memory leaks in gzclose_r() and gzclose_w(), file leak in gz_open() +- Add contrib/gcc_gvmat64 for longest_match and inflate_fast [Vollant] +- Remove *64 functions from win32/zlib.def (they're not 64-bit yet) +- Fix bug in void-returning vsprintf() case in gzwrite.c +- Fix name change from inflate.h in contrib/inflate86/inffas86.c +- Check if temporary file exists before removing in make_vms.com [Zinser] +- Fix make install and uninstall for --static option +- Fix usage of _MSC_VER in gzguts.h and zutil.h [Truta] +- Update readme.txt in contrib/masmx64 and masmx86 to assemble + +Changes in 1.2.3.9 (21 Feb 2010) +- Expunge gzio.c +- Move as400 build information to old +- Fix updates in contrib/minizip and contrib/vstudio +- Add const to vsnprintf test in configure to avoid warnings [Weigelt] +- Delete zconf.h (made by configure) [Weigelt] +- Change zconf.in.h to zconf.h.in per convention [Weigelt] +- Check for NULL buf in gzgets() +- Return empty string for gzgets() with len == 1 (like fgets()) +- Fix description of gzgets() in zlib.h for end-of-file, NULL return +- Update minizip to 1.1 [Vollant] +- Avoid MSVC loss of data warnings in gzread.c, gzwrite.c +- Note in zlib.h that gzerror() should be used to distinguish from EOF +- Remove use of snprintf() from gzlib.c +- Fix bug in gzseek() +- Update contrib/vstudio, adding vc9 and vc10 [Kuno, Vollant] +- Fix zconf.h generation in CMakeLists.txt [Lowman] +- Improve comments in zconf.h where modified by configure + +Changes in 1.2.3.8 (13 Feb 2010) +- Clean up text files (tabs, trailing whitespace, etc.) [Oberhumer] +- Use z_off64_t in gz_zero() and gz_skip() to match state->skip +- Avoid comparison problem when sizeof(int) == sizeof(z_off64_t) +- Revert to Makefile.in from 1.2.3.6 (live with the clutter) +- Fix missing error return in gzflush(), add zlib.h note +- Add *64 functions to zlib.map [Levin] +- Fix signed/unsigned comparison in gz_comp() +- Use SFLAGS when testing shared linking in configure +- Add --64 option to ./configure to use -m64 with gcc +- Fix ./configure --help to correctly name options +- Have make fail if a test fails [Levin] +- Avoid buffer overrun in contrib/masmx64/gvmat64.asm [Simpson] +- Remove assembler object files from contrib + +Changes in 1.2.3.7 (24 Jan 2010) +- Always gzopen() with O_LARGEFILE if available +- Fix gzdirect() to work immediately after gzopen() or gzdopen() +- Make gzdirect() more precise when the state changes while reading +- Improve zlib.h documentation in many places +- Catch memory allocation failure in gz_open() +- Complete close operation if seek forward in gzclose_w() fails +- Return Z_ERRNO from gzclose_r() if close() fails +- Return Z_STREAM_ERROR instead of EOF for gzclose() being passed NULL +- Return zero for gzwrite() errors to match zlib.h description +- Return -1 on gzputs() error to match zlib.h description +- Add zconf.in.h to allow recovery from configure modification [Weigelt] +- Fix static library permissions in Makefile.in [Weigelt] +- Avoid warnings in configure tests that hide functionality [Weigelt] +- Add *BSD and DragonFly to Linux case in configure [gentoo 123571] +- Change libzdll.a to libz.dll.a in win32/Makefile.gcc [gentoo 288212] +- Avoid access of uninitialized data for first inflateReset2 call [Gomes] +- Keep object files in subdirectories to reduce the clutter somewhat +- Remove default Makefile and zlibdefs.h, add dummy Makefile +- Add new external functions to Z_PREFIX, remove duplicates, z_z_ -> z_ +- Remove zlibdefs.h completely -- modify zconf.h instead + +Changes in 1.2.3.6 (17 Jan 2010) +- Avoid void * arithmetic in gzread.c and gzwrite.c +- Make compilers happier with const char * for gz_error message +- Avoid unused parameter warning in inflate.c +- Avoid signed-unsigned comparison warning in inflate.c +- Indent #pragma's for traditional C +- Fix usage of strwinerror() in glib.c, change to gz_strwinerror() +- Correct email address in configure for system options +- Update make_vms.com and add make_vms.com to contrib/minizip [Zinser] +- Update zlib.map [Brown] +- Fix Makefile.in for Solaris 10 make of example64 and minizip64 [Torok] +- Apply various fixes to CMakeLists.txt [Lowman] +- Add checks on len in gzread() and gzwrite() +- Add error message for no more room for gzungetc() +- Remove zlib version check in gzwrite() +- Defer compression of gzprintf() result until need to +- Use snprintf() in gzdopen() if available +- Remove USE_MMAP configuration determination (only used by minigzip) +- Remove examples/pigz.c (available separately) +- Update examples/gun.c to 1.6 + +Changes in 1.2.3.5 (8 Jan 2010) +- Add space after #if in zutil.h for some compilers +- Fix relatively harmless bug in deflate_fast() [Exarevsky] +- Fix same problem in deflate_slow() +- Add $(SHAREDLIBV) to LIBS in Makefile.in [Brown] +- Add deflate_rle() for faster Z_RLE strategy run-length encoding +- Add deflate_huff() for faster Z_HUFFMAN_ONLY encoding +- Change name of "write" variable in inffast.c to avoid library collisions +- Fix premature EOF from gzread() in gzio.c [Brown] +- Use zlib header window size if windowBits is 0 in inflateInit2() +- Remove compressBound() call in deflate.c to avoid linking compress.o +- Replace use of errno in gz* with functions, support WinCE [Alves] +- Provide alternative to perror() in minigzip.c for WinCE [Alves] +- Don't use _vsnprintf on later versions of MSVC [Lowman] +- Add CMake build script and input file [Lowman] +- Update contrib/minizip to 1.1 [Svensson, Vollant] +- Moved nintendods directory from contrib to root +- Replace gzio.c with a new set of routines with the same functionality +- Add gzbuffer(), gzoffset(), gzclose_r(), gzclose_w() as part of above +- Update contrib/minizip to 1.1b +- Change gzeof() to return 0 on error instead of -1 to agree with zlib.h + +Changes in 1.2.3.4 (21 Dec 2009) +- Use old school .SUFFIXES in Makefile.in for FreeBSD compatibility +- Update comments in configure and Makefile.in for default --shared +- Fix test -z's in configure [Marquess] +- Build examplesh and minigzipsh when not testing +- Change NULL's to Z_NULL's in deflate.c and in comments in zlib.h +- Import LDFLAGS from the environment in configure +- Fix configure to populate SFLAGS with discovered CFLAGS options +- Adapt make_vms.com to the new Makefile.in [Zinser] +- Add zlib2ansi script for C++ compilation [Marquess] +- Add _FILE_OFFSET_BITS=64 test to make test (when applicable) +- Add AMD64 assembler code for longest match to contrib [Teterin] +- Include options from $SFLAGS when doing $LDSHARED +- Simplify 64-bit file support by introducing z_off64_t type +- Make shared object files in objs directory to work around old Sun cc +- Use only three-part version number for Darwin shared compiles +- Add rc option to ar in Makefile.in for when ./configure not run +- Add -WI,-rpath,. to LDFLAGS for OSF 1 V4* +- Set LD_LIBRARYN32_PATH for SGI IRIX shared compile +- Protect against _FILE_OFFSET_BITS being defined when compiling zlib +- Rename Makefile.in targets allstatic to static and allshared to shared +- Fix static and shared Makefile.in targets to be independent +- Correct error return bug in gz_open() by setting state [Brown] +- Put spaces before ;;'s in configure for better sh compatibility +- Add pigz.c (parallel implementation of gzip) to examples/ +- Correct constant in crc32.c to UL [Leventhal] +- Reject negative lengths in crc32_combine() +- Add inflateReset2() function to work like inflateEnd()/inflateInit2() +- Include sys/types.h for _LARGEFILE64_SOURCE [Brown] +- Correct typo in doc/algorithm.txt [Janik] +- Fix bug in adler32_combine() [Zhu] +- Catch missing-end-of-block-code error in all inflates and in puff + Assures that random input to inflate eventually results in an error +- Added enough.c (calculation of ENOUGH for inftrees.h) to examples/ +- Update ENOUGH and its usage to reflect discovered bounds +- Fix gzerror() error report on empty input file [Brown] +- Add ush casts in trees.c to avoid pedantic runtime errors +- Fix typo in zlib.h uncompress() description [Reiss] +- Correct inflate() comments with regard to automatic header detection +- Remove deprecation comment on Z_PARTIAL_FLUSH (it stays) +- Put new version of gzlog (2.0) in examples with interruption recovery +- Add puff compile option to permit invalid distance-too-far streams +- Add puff TEST command options, ability to read piped input +- Prototype the *64 functions in zlib.h when _FILE_OFFSET_BITS == 64, but + _LARGEFILE64_SOURCE not defined +- Fix Z_FULL_FLUSH to truly erase the past by resetting s->strstart +- Fix deflateSetDictionary() to use all 32K for output consistency +- Remove extraneous #define MIN_LOOKAHEAD in deflate.c (in deflate.h) +- Clear bytes after deflate lookahead to avoid use of uninitialized data +- Change a limit in inftrees.c to be more transparent to Coverity Prevent +- Update win32/zlib.def with exported symbols from zlib.h +- Correct spelling errors in zlib.h [Willem, Sobrado] +- Allow Z_BLOCK for deflate() to force a new block +- Allow negative bits in inflatePrime() to delete existing bit buffer +- Add Z_TREES flush option to inflate() to return at end of trees +- Add inflateMark() to return current state information for random access +- Add Makefile for NintendoDS to contrib [Costa] +- Add -w in configure compile tests to avoid spurious warnings [Beucler] +- Fix typos in zlib.h comments for deflateSetDictionary() +- Fix EOF detection in transparent gzread() [Maier] + +Changes in 1.2.3.3 (2 October 2006) +- Make --shared the default for configure, add a --static option +- Add compile option to permit invalid distance-too-far streams +- Add inflateUndermine() function which is required to enable above +- Remove use of "this" variable name for C++ compatibility [Marquess] +- Add testing of shared library in make test, if shared library built +- Use ftello() and fseeko() if available instead of ftell() and fseek() +- Provide two versions of all functions that use the z_off_t type for + binary compatibility -- a normal version and a 64-bit offset version, + per the Large File Support Extension when _LARGEFILE64_SOURCE is + defined; use the 64-bit versions by default when _FILE_OFFSET_BITS + is defined to be 64 +- Add a --uname= option to configure to perhaps help with cross-compiling + +Changes in 1.2.3.2 (3 September 2006) +- Turn off silly Borland warnings [Hay] +- Use off64_t and define _LARGEFILE64_SOURCE when present +- Fix missing dependency on inffixed.h in Makefile.in +- Rig configure --shared to build both shared and static [Teredesai, Truta] +- Remove zconf.in.h and instead create a new zlibdefs.h file +- Fix contrib/minizip/unzip.c non-encrypted after encrypted [Vollant] +- Add treebuild.xml (see http://treebuild.metux.de/) [Weigelt] + +Changes in 1.2.3.1 (16 August 2006) +- Add watcom directory with OpenWatcom make files [Daniel] +- Remove #undef of FAR in zconf.in.h for MVS [Fedtke] +- Update make_vms.com [Zinser] +- Use -fPIC for shared build in configure [Teredesai, Nicholson] +- Use only major version number for libz.so on IRIX and OSF1 [Reinholdtsen] +- Use fdopen() (not _fdopen()) for Interix in zutil.h [Bäck] +- Add some FAQ entries about the contrib directory +- Update the MVS question in the FAQ +- Avoid extraneous reads after EOF in gzio.c [Brown] +- Correct spelling of "successfully" in gzio.c [Randers-Pehrson] +- Add comments to zlib.h about gzerror() usage [Brown] +- Set extra flags in gzip header in gzopen() like deflate() does +- Make configure options more compatible with double-dash conventions + [Weigelt] +- Clean up compilation under Solaris SunStudio cc [Rowe, Reinholdtsen] +- Fix uninstall target in Makefile.in [Truta] +- Add pkgconfig support [Weigelt] +- Use $(DESTDIR) macro in Makefile.in [Reinholdtsen, Weigelt] +- Replace set_data_type() with a more accurate detect_data_type() in + trees.c, according to the txtvsbin.txt document [Truta] +- Swap the order of #include and #include "zlib.h" in + gzio.c, example.c and minigzip.c [Truta] +- Shut up annoying VS2005 warnings about standard C deprecation [Rowe, + Truta] (where?) +- Fix target "clean" from win32/Makefile.bor [Truta] +- Create .pdb and .manifest files in win32/makefile.msc [Ziegler, Rowe] +- Update zlib www home address in win32/DLL_FAQ.txt [Truta] +- Update contrib/masmx86/inffas32.asm for VS2005 [Vollant, Van Wassenhove] +- Enable browse info in the "Debug" and "ASM Debug" configurations in + the Visual C++ 6 project, and set (non-ASM) "Debug" as default [Truta] +- Add pkgconfig support [Weigelt] +- Add ZLIB_VER_MAJOR, ZLIB_VER_MINOR and ZLIB_VER_REVISION in zlib.h, + for use in win32/zlib1.rc [Polushin, Rowe, Truta] +- Add a document that explains the new text detection scheme to + doc/txtvsbin.txt [Truta] +- Add rfc1950.txt, rfc1951.txt and rfc1952.txt to doc/ [Truta] +- Move algorithm.txt into doc/ [Truta] +- Synchronize FAQ with website +- Fix compressBound(), was low for some pathological cases [Fearnley] +- Take into account wrapper variations in deflateBound() +- Set examples/zpipe.c input and output to binary mode for Windows +- Update examples/zlib_how.html with new zpipe.c (also web site) +- Fix some warnings in examples/gzlog.c and examples/zran.c (it seems + that gcc became pickier in 4.0) +- Add zlib.map for Linux: "All symbols from zlib-1.1.4 remain + un-versioned, the patch adds versioning only for symbols introduced in + zlib-1.2.0 or later. It also declares as local those symbols which are + not designed to be exported." [Levin] +- Update Z_PREFIX list in zconf.in.h, add --zprefix option to configure +- Do not initialize global static by default in trees.c, add a response + NO_INIT_GLOBAL_POINTERS to initialize them if needed [Marquess] +- Don't use strerror() in gzio.c under WinCE [Yakimov] +- Don't use errno.h in zutil.h under WinCE [Yakimov] +- Move arguments for AR to its usage to allow replacing ar [Marot] +- Add HAVE_VISIBILITY_PRAGMA in zconf.in.h for Mozilla [Randers-Pehrson] +- Improve inflateInit() and inflateInit2() documentation +- Fix structure size comment in inflate.h +- Change configure help option from --h* to --help [Santos] + +Changes in 1.2.3 (18 July 2005) +- Apply security vulnerability fixes to contrib/infback9 as well +- Clean up some text files (carriage returns, trailing space) +- Update testzlib, vstudio, masmx64, and masmx86 in contrib [Vollant] + +Changes in 1.2.2.4 (11 July 2005) +- Add inflatePrime() function for starting inflation at bit boundary +- Avoid some Visual C warnings in deflate.c +- Avoid more silly Visual C warnings in inflate.c and inftrees.c for 64-bit + compile +- Fix some spelling errors in comments [Betts] +- Correct inflateInit2() error return documentation in zlib.h +- Add zran.c example of compressed data random access to examples + directory, shows use of inflatePrime() +- Fix cast for assignments to strm->state in inflate.c and infback.c +- Fix zlibCompileFlags() in zutil.c to use 1L for long shifts [Oberhumer] +- Move declarations of gf2 functions to right place in crc32.c [Oberhumer] +- Add cast in trees.c t avoid a warning [Oberhumer] +- Avoid some warnings in fitblk.c, gun.c, gzjoin.c in examples [Oberhumer] +- Update make_vms.com [Zinser] +- Initialize state->write in inflateReset() since copied in inflate_fast() +- Be more strict on incomplete code sets in inflate_table() and increase + ENOUGH and MAXD -- this repairs a possible security vulnerability for + invalid inflate input. Thanks to Tavis Ormandy and Markus Oberhumer for + discovering the vulnerability and providing test cases +- Add ia64 support to configure for HP-UX [Smith] +- Add error return to gzread() for format or i/o error [Levin] +- Use malloc.h for OS/2 [Necasek] + +Changes in 1.2.2.3 (27 May 2005) +- Replace 1U constants in inflate.c and inftrees.c for 64-bit compile +- Typecast fread() return values in gzio.c [Vollant] +- Remove trailing space in minigzip.c outmode (VC++ can't deal with it) +- Fix crc check bug in gzread() after gzungetc() [Heiner] +- Add the deflateTune() function to adjust internal compression parameters +- Add a fast gzip decompressor, gun.c, to examples (use of inflateBack) +- Remove an incorrect assertion in examples/zpipe.c +- Add C++ wrapper in infback9.h [Donais] +- Fix bug in inflateCopy() when decoding fixed codes +- Note in zlib.h how much deflateSetDictionary() actually uses +- Remove USE_DICT_HEAD in deflate.c (would mess up inflate if used) +- Add _WIN32_WCE to define WIN32 in zconf.in.h [Spencer] +- Don't include stderr.h or errno.h for _WIN32_WCE in zutil.h [Spencer] +- Add gzdirect() function to indicate transparent reads +- Update contrib/minizip [Vollant] +- Fix compilation of deflate.c when both ASMV and FASTEST [Oberhumer] +- Add casts in crc32.c to avoid warnings [Oberhumer] +- Add contrib/masmx64 [Vollant] +- Update contrib/asm586, asm686, masmx86, testzlib, vstudio [Vollant] + +Changes in 1.2.2.2 (30 December 2004) +- Replace structure assignments in deflate.c and inflate.c with zmemcpy to + avoid implicit memcpy calls (portability for no-library compilation) +- Increase sprintf() buffer size in gzdopen() to allow for large numbers +- Add INFLATE_STRICT to check distances against zlib header +- Improve WinCE errno handling and comments [Chang] +- Remove comment about no gzip header processing in FAQ +- Add Z_FIXED strategy option to deflateInit2() to force fixed trees +- Add updated make_vms.com [Coghlan], update README +- Create a new "examples" directory, move gzappend.c there, add zpipe.c, + fitblk.c, gzlog.[ch], gzjoin.c, and zlib_how.html +- Add FAQ entry and comments in deflate.c on uninitialized memory access +- Add Solaris 9 make options in configure [Gilbert] +- Allow strerror() usage in gzio.c for STDC +- Fix DecompressBuf in contrib/delphi/ZLib.pas [ManChesTer] +- Update contrib/masmx86/inffas32.asm and gvmat32.asm [Vollant] +- Use z_off_t for adler32_combine() and crc32_combine() lengths +- Make adler32() much faster for small len +- Use OS_CODE in deflate() default gzip header + +Changes in 1.2.2.1 (31 October 2004) +- Allow inflateSetDictionary() call for raw inflate +- Fix inflate header crc check bug for file names and comments +- Add deflateSetHeader() and gz_header structure for custom gzip headers +- Add inflateGetheader() to retrieve gzip headers +- Add crc32_combine() and adler32_combine() functions +- Add alloc_func, free_func, in_func, out_func to Z_PREFIX list +- Use zstreamp consistently in zlib.h (inflate_back functions) +- Remove GUNZIP condition from definition of inflate_mode in inflate.h + and in contrib/inflate86/inffast.S [Truta, Anderson] +- Add support for AMD64 in contrib/inflate86/inffas86.c [Anderson] +- Update projects/README.projects and projects/visualc6 [Truta] +- Update win32/DLL_FAQ.txt [Truta] +- Avoid warning under NO_GZCOMPRESS in gzio.c; fix typo [Truta] +- Deprecate Z_ASCII; use Z_TEXT instead [Truta] +- Use a new algorithm for setting strm->data_type in trees.c [Truta] +- Do not define an exit() prototype in zutil.c unless DEBUG defined +- Remove prototype of exit() from zutil.c, example.c, minigzip.c [Truta] +- Add comment in zlib.h for Z_NO_FLUSH parameter to deflate() +- Fix Darwin build version identification [Peterson] + +Changes in 1.2.2 (3 October 2004) +- Update zlib.h comments on gzip in-memory processing +- Set adler to 1 in inflateReset() to support Java test suite [Walles] +- Add contrib/dotzlib [Ravn] +- Update win32/DLL_FAQ.txt [Truta] +- Update contrib/minizip [Vollant] +- Move contrib/visual-basic.txt to old/ [Truta] +- Fix assembler builds in projects/visualc6/ [Truta] + +Changes in 1.2.1.2 (9 September 2004) +- Update INDEX file +- Fix trees.c to update strm->data_type (no one ever noticed!) +- Fix bug in error case in inflate.c, infback.c, and infback9.c [Brown] +- Add "volatile" to crc table flag declaration (for DYNAMIC_CRC_TABLE) +- Add limited multitasking protection to DYNAMIC_CRC_TABLE +- Add NO_vsnprintf for VMS in zutil.h [Mozilla] +- Don't declare strerror() under VMS [Mozilla] +- Add comment to DYNAMIC_CRC_TABLE to use get_crc_table() to initialize +- Update contrib/ada [Anisimkov] +- Update contrib/minizip [Vollant] +- Fix configure to not hardcode directories for Darwin [Peterson] +- Fix gzio.c to not return error on empty files [Brown] +- Fix indentation; update version in contrib/delphi/ZLib.pas and + contrib/pascal/zlibpas.pas [Truta] +- Update mkasm.bat in contrib/masmx86 [Truta] +- Update contrib/untgz [Truta] +- Add projects/README.projects [Truta] +- Add project for MS Visual C++ 6.0 in projects/visualc6 [Cadieux, Truta] +- Update win32/DLL_FAQ.txt [Truta] +- Update list of Z_PREFIX symbols in zconf.h [Randers-Pehrson, Truta] +- Remove an unnecessary assignment to curr in inftrees.c [Truta] +- Add OS/2 to exe builds in configure [Poltorak] +- Remove err dummy parameter in zlib.h [Kientzle] + +Changes in 1.2.1.1 (9 January 2004) +- Update email address in README +- Several FAQ updates +- Fix a big fat bug in inftrees.c that prevented decoding valid + dynamic blocks with only literals and no distance codes -- + Thanks to "Hot Emu" for the bug report and sample file +- Add a note to puff.c on no distance codes case + +Changes in 1.2.1 (17 November 2003) +- Remove a tab in contrib/gzappend/gzappend.c +- Update some interfaces in contrib for new zlib functions +- Update zlib version number in some contrib entries +- Add Windows CE definition for ptrdiff_t in zutil.h [Mai, Truta] +- Support shared libraries on Hurd and KFreeBSD [Brown] +- Fix error in NO_DIVIDE option of adler32.c + +Changes in 1.2.0.8 (4 November 2003) +- Update version in contrib/delphi/ZLib.pas and contrib/pascal/zlibpas.pas +- Add experimental NO_DIVIDE #define in adler32.c + - Possibly faster on some processors (let me know if it is) +- Correct Z_BLOCK to not return on first inflate call if no wrap +- Fix strm->data_type on inflate() return to correctly indicate EOB +- Add deflatePrime() function for appending in the middle of a byte +- Add contrib/gzappend for an example of appending to a stream +- Update win32/DLL_FAQ.txt [Truta] +- Delete Turbo C comment in README [Truta] +- Improve some indentation in zconf.h [Truta] +- Fix infinite loop on bad input in configure script [Church] +- Fix gzeof() for concatenated gzip files [Johnson] +- Add example to contrib/visual-basic.txt [Michael B.] +- Add -p to mkdir's in Makefile.in [vda] +- Fix configure to properly detect presence or lack of printf functions +- Add AS400 support [Monnerat] +- Add a little Cygwin support [Wilson] + +Changes in 1.2.0.7 (21 September 2003) +- Correct some debug formats in contrib/infback9 +- Cast a type in a debug statement in trees.c +- Change search and replace delimiter in configure from % to # [Beebe] +- Update contrib/untgz to 0.2 with various fixes [Truta] +- Add build support for Amiga [Nikl] +- Remove some directories in old that have been updated to 1.2 +- Add dylib building for Mac OS X in configure and Makefile.in +- Remove old distribution stuff from Makefile +- Update README to point to DLL_FAQ.txt, and add comment on Mac OS X +- Update links in README + +Changes in 1.2.0.6 (13 September 2003) +- Minor FAQ updates +- Update contrib/minizip to 1.00 [Vollant] +- Remove test of gz functions in example.c when GZ_COMPRESS defined [Truta] +- Update POSTINC comment for 68060 [Nikl] +- Add contrib/infback9 with deflate64 decoding (unsupported) +- For MVS define NO_vsnprintf and undefine FAR [van Burik] +- Add pragma for fdopen on MVS [van Burik] + +Changes in 1.2.0.5 (8 September 2003) +- Add OF to inflateBackEnd() declaration in zlib.h +- Remember start when using gzdopen in the middle of a file +- Use internal off_t counters in gz* functions to properly handle seeks +- Perform more rigorous check for distance-too-far in inffast.c +- Add Z_BLOCK flush option to return from inflate at block boundary +- Set strm->data_type on return from inflate + - Indicate bits unused, if at block boundary, and if in last block +- Replace size_t with ptrdiff_t in crc32.c, and check for correct size +- Add condition so old NO_DEFLATE define still works for compatibility +- FAQ update regarding the Windows DLL [Truta] +- INDEX update: add qnx entry, remove aix entry [Truta] +- Install zlib.3 into mandir [Wilson] +- Move contrib/zlib_dll_FAQ.txt to win32/DLL_FAQ.txt; update [Truta] +- Adapt the zlib interface to the new DLL convention guidelines [Truta] +- Introduce ZLIB_WINAPI macro to allow the export of functions using + the WINAPI calling convention, for Visual Basic [Vollant, Truta] +- Update msdos and win32 scripts and makefiles [Truta] +- Export symbols by name, not by ordinal, in win32/zlib.def [Truta] +- Add contrib/ada [Anisimkov] +- Move asm files from contrib/vstudio/vc70_32 to contrib/asm386 [Truta] +- Rename contrib/asm386 to contrib/masmx86 [Truta, Vollant] +- Add contrib/masm686 [Truta] +- Fix offsets in contrib/inflate86 and contrib/masmx86/inffas32.asm + [Truta, Vollant] +- Update contrib/delphi; rename to contrib/pascal; add example [Truta] +- Remove contrib/delphi2; add a new contrib/delphi [Truta] +- Avoid inclusion of the nonstandard in contrib/iostream, + and fix some method prototypes [Truta] +- Fix the ZCR_SEED2 constant to avoid warnings in contrib/minizip + [Truta] +- Avoid the use of backslash (\) in contrib/minizip [Vollant] +- Fix file time handling in contrib/untgz; update makefiles [Truta] +- Update contrib/vstudio/vc70_32 to comply with the new DLL guidelines + [Vollant] +- Remove contrib/vstudio/vc15_16 [Vollant] +- Rename contrib/vstudio/vc70_32 to contrib/vstudio/vc7 [Truta] +- Update README.contrib [Truta] +- Invert the assignment order of match_head and s->prev[...] in + INSERT_STRING [Truta] +- Compare TOO_FAR with 32767 instead of 32768, to avoid 16-bit warnings + [Truta] +- Compare function pointers with 0, not with NULL or Z_NULL [Truta] +- Fix prototype of syncsearch in inflate.c [Truta] +- Introduce ASMINF macro to be enabled when using an ASM implementation + of inflate_fast [Truta] +- Change NO_DEFLATE to NO_GZCOMPRESS [Truta] +- Modify test_gzio in example.c to take a single file name as a + parameter [Truta] +- Exit the example.c program if gzopen fails [Truta] +- Add type casts around strlen in example.c [Truta] +- Remove casting to sizeof in minigzip.c; give a proper type + to the variable compared with SUFFIX_LEN [Truta] +- Update definitions of STDC and STDC99 in zconf.h [Truta] +- Synchronize zconf.h with the new Windows DLL interface [Truta] +- Use SYS16BIT instead of __32BIT__ to distinguish between + 16- and 32-bit platforms [Truta] +- Use far memory allocators in small 16-bit memory models for + Turbo C [Truta] +- Add info about the use of ASMV, ASMINF and ZLIB_WINAPI in + zlibCompileFlags [Truta] +- Cygwin has vsnprintf [Wilson] +- In Windows16, OS_CODE is 0, as in MSDOS [Truta] +- In Cygwin, OS_CODE is 3 (Unix), not 11 (Windows32) [Wilson] + +Changes in 1.2.0.4 (10 August 2003) +- Minor FAQ updates +- Be more strict when checking inflateInit2's windowBits parameter +- Change NO_GUNZIP compile option to NO_GZIP to cover deflate as well +- Add gzip wrapper option to deflateInit2 using windowBits +- Add updated QNX rule in configure and qnx directory [Bonnefoy] +- Make inflate distance-too-far checks more rigorous +- Clean up FAR usage in inflate +- Add casting to sizeof() in gzio.c and minigzip.c + +Changes in 1.2.0.3 (19 July 2003) +- Fix silly error in gzungetc() implementation [Vollant] +- Update contrib/minizip and contrib/vstudio [Vollant] +- Fix printf format in example.c +- Correct cdecl support in zconf.in.h [Anisimkov] +- Minor FAQ updates + +Changes in 1.2.0.2 (13 July 2003) +- Add ZLIB_VERNUM in zlib.h for numerical preprocessor comparisons +- Attempt to avoid warnings in crc32.c for pointer-int conversion +- Add AIX to configure, remove aix directory [Bakker] +- Add some casts to minigzip.c +- Improve checking after insecure sprintf() or vsprintf() calls +- Remove #elif's from crc32.c +- Change leave label to inf_leave in inflate.c and infback.c to avoid + library conflicts +- Remove inflate gzip decoding by default--only enable gzip decoding by + special request for stricter backward compatibility +- Add zlibCompileFlags() function to return compilation information +- More typecasting in deflate.c to avoid warnings +- Remove leading underscore from _Capital #defines [Truta] +- Fix configure to link shared library when testing +- Add some Windows CE target adjustments [Mai] +- Remove #define ZLIB_DLL in zconf.h [Vollant] +- Add zlib.3 [Rodgers] +- Update RFC URL in deflate.c and algorithm.txt [Mai] +- Add zlib_dll_FAQ.txt to contrib [Truta] +- Add UL to some constants [Truta] +- Update minizip and vstudio [Vollant] +- Remove vestigial NEED_DUMMY_RETURN from zconf.in.h +- Expand use of NO_DUMMY_DECL to avoid all dummy structures +- Added iostream3 to contrib [Schwardt] +- Replace rewind() with fseek() for WinCE [Truta] +- Improve setting of zlib format compression level flags + - Report 0 for huffman and rle strategies and for level == 0 or 1 + - Report 2 only for level == 6 +- Only deal with 64K limit when necessary at compile time [Truta] +- Allow TOO_FAR check to be turned off at compile time [Truta] +- Add gzclearerr() function [Souza] +- Add gzungetc() function + +Changes in 1.2.0.1 (17 March 2003) +- Add Z_RLE strategy for run-length encoding [Truta] + - When Z_RLE requested, restrict matches to distance one + - Update zlib.h, minigzip.c, gzopen(), gzdopen() for Z_RLE +- Correct FASTEST compilation to allow level == 0 +- Clean up what gets compiled for FASTEST +- Incorporate changes to zconf.in.h [Vollant] + - Refine detection of Turbo C need for dummy returns + - Refine ZLIB_DLL compilation + - Include additional header file on VMS for off_t typedef +- Try to use _vsnprintf where it supplants vsprintf [Vollant] +- Add some casts in inffast.c +- Enhance comments in zlib.h on what happens if gzprintf() tries to + write more than 4095 bytes before compression +- Remove unused state from inflateBackEnd() +- Remove exit(0) from minigzip.c, example.c +- Get rid of all those darn tabs +- Add "check" target to Makefile.in that does the same thing as "test" +- Add "mostlyclean" and "maintainer-clean" targets to Makefile.in +- Update contrib/inflate86 [Anderson] +- Update contrib/testzlib, contrib/vstudio, contrib/minizip [Vollant] +- Add msdos and win32 directories with makefiles [Truta] +- More additions and improvements to the FAQ + +Changes in 1.2.0 (9 March 2003) +- New and improved inflate code + - About 20% faster + - Does not allocate 32K window unless and until needed + - Automatically detects and decompresses gzip streams + - Raw inflate no longer needs an extra dummy byte at end + - Added inflateBack functions using a callback interface--even faster + than inflate, useful for file utilities (gzip, zip) + - Added inflateCopy() function to record state for random access on + externally generated deflate streams (e.g. in gzip files) + - More readable code (I hope) +- New and improved crc32() + - About 50% faster, thanks to suggestions from Rodney Brown +- Add deflateBound() and compressBound() functions +- Fix memory leak in deflateInit2() +- Permit setting dictionary for raw deflate (for parallel deflate) +- Fix const declaration for gzwrite() +- Check for some malloc() failures in gzio.c +- Fix bug in gzopen() on single-byte file 0x1f +- Fix bug in gzread() on concatenated file with 0x1f at end of buffer + and next buffer doesn't start with 0x8b +- Fix uncompress() to return Z_DATA_ERROR on truncated input +- Free memory at end of example.c +- Remove MAX #define in trees.c (conflicted with some libraries) +- Fix static const's in deflate.c, gzio.c, and zutil.[ch] +- Declare malloc() and free() in gzio.c if STDC not defined +- Use malloc() instead of calloc() in zutil.c if int big enough +- Define STDC for AIX +- Add aix/ with approach for compiling shared library on AIX +- Add HP-UX support for shared libraries in configure +- Add OpenUNIX support for shared libraries in configure +- Use $cc instead of gcc to build shared library +- Make prefix directory if needed when installing +- Correct Macintosh avoidance of typedef Byte in zconf.h +- Correct Turbo C memory allocation when under Linux +- Use libz.a instead of -lz in Makefile (assure use of compiled library) +- Update configure to check for snprintf or vsnprintf functions and their + return value, warn during make if using an insecure function +- Fix configure problem with compile-time knowledge of HAVE_UNISTD_H that + is lost when library is used--resolution is to build new zconf.h +- Documentation improvements (in zlib.h): + - Document raw deflate and inflate + - Update RFCs URL + - Point out that zlib and gzip formats are different + - Note that Z_BUF_ERROR is not fatal + - Document string limit for gzprintf() and possible buffer overflow + - Note requirement on avail_out when flushing + - Note permitted values of flush parameter of inflate() +- Add some FAQs (and even answers) to the FAQ +- Add contrib/inflate86/ for x86 faster inflate +- Add contrib/blast/ for PKWare Data Compression Library decompression +- Add contrib/puff/ simple inflate for deflate format description + +Changes in 1.1.4 (11 March 2002) +- ZFREE was repeated on same allocation on some error conditions + This creates a security problem described in + http://www.zlib.org/advisory-2002-03-11.txt +- Returned incorrect error (Z_MEM_ERROR) on some invalid data +- Avoid accesses before window for invalid distances with inflate window + less than 32K +- force windowBits > 8 to avoid a bug in the encoder for a window size + of 256 bytes. (A complete fix will be available in 1.1.5) + +Changes in 1.1.3 (9 July 1998) +- fix "an inflate input buffer bug that shows up on rare but persistent + occasions" (Mark) +- fix gzread and gztell for concatenated .gz files (Didier Le Botlan) +- fix gzseek(..., SEEK_SET) in write mode +- fix crc check after a gzeek (Frank Faubert) +- fix miniunzip when the last entry in a zip file is itself a zip file + (J Lillge) +- add contrib/asm586 and contrib/asm686 (Brian Raiter) + See http://www.muppetlabs.com/~breadbox/software/assembly.html +- add support for Delphi 3 in contrib/delphi (Bob Dellaca) +- add support for C++Builder 3 and Delphi 3 in contrib/delphi2 (Davide Moretti) +- do not exit prematurely in untgz if 0 at start of block (Magnus Holmgren) +- use macro EXTERN instead of extern to support DLL for BeOS (Sander Stoks) +- added a FAQ file + +- Support gzdopen on Mac with Metrowerks (Jason Linhart) +- Do not redefine Byte on Mac (Brad Pettit & Jason Linhart) +- define SEEK_END too if SEEK_SET is not defined (Albert Chin-A-Young) +- avoid some warnings with Borland C (Tom Tanner) +- fix a problem in contrib/minizip/zip.c for 16-bit MSDOS (Gilles Vollant) +- emulate utime() for WIN32 in contrib/untgz (Gilles Vollant) +- allow several arguments to configure (Tim Mooney, Frodo Looijaard) +- use libdir and includedir in Makefile.in (Tim Mooney) +- support shared libraries on OSF1 V4 (Tim Mooney) +- remove so_locations in "make clean" (Tim Mooney) +- fix maketree.c compilation error (Glenn, Mark) +- Python interface to zlib now in Python 1.5 (Jeremy Hylton) +- new Makefile.riscos (Rich Walker) +- initialize static descriptors in trees.c for embedded targets (Nick Smith) +- use "foo-gz" in example.c for RISCOS and VMS (Nick Smith) +- add the OS/2 files in Makefile.in too (Andrew Zabolotny) +- fix fdopen and halloc macros for Microsoft C 6.0 (Tom Lane) +- fix maketree.c to allow clean compilation of inffixed.h (Mark) +- fix parameter check in deflateCopy (Gunther Nikl) +- cleanup trees.c, use compressed_len only in debug mode (Christian Spieler) +- Many portability patches by Christian Spieler: + . zutil.c, zutil.h: added "const" for zmem* + . Make_vms.com: fixed some typos + . Make_vms.com: msdos/Makefile.*: removed zutil.h from some dependency lists + . msdos/Makefile.msc: remove "default rtl link library" info from obj files + . msdos/Makefile.*: use model-dependent name for the built zlib library + . msdos/Makefile.emx, nt/Makefile.emx, nt/Makefile.gcc: + new makefiles, for emx (DOS/OS2), emx&rsxnt and mingw32 (Windows 9x / NT) +- use define instead of typedef for Bytef also for MSC small/medium (Tom Lane) +- replace __far with _far for better portability (Christian Spieler, Tom Lane) +- fix test for errno.h in configure (Tim Newsham) + +Changes in 1.1.2 (19 March 98) +- added contrib/minzip, mini zip and unzip based on zlib (Gilles Vollant) + See http://www.winimage.com/zLibDll/unzip.html +- preinitialize the inflate tables for fixed codes, to make the code + completely thread safe (Mark) +- some simplifications and slight speed-up to the inflate code (Mark) +- fix gzeof on non-compressed files (Allan Schrum) +- add -std1 option in configure for OSF1 to fix gzprintf (Martin Mokrejs) +- use default value of 4K for Z_BUFSIZE for 16-bit MSDOS (Tim Wegner + Glenn) +- added os2/Makefile.def and os2/zlib.def (Andrew Zabolotny) +- add shared lib support for UNIX_SV4.2MP (MATSUURA Takanori) +- do not wrap extern "C" around system includes (Tom Lane) +- mention zlib binding for TCL in README (Andreas Kupries) +- added amiga/Makefile.pup for Amiga powerUP SAS/C PPC (Andreas Kleinert) +- allow "make install prefix=..." even after configure (Glenn Randers-Pehrson) +- allow "configure --prefix $HOME" (Tim Mooney) +- remove warnings in example.c and gzio.c (Glenn Randers-Pehrson) +- move Makefile.sas to amiga/Makefile.sas + +Changes in 1.1.1 (27 Feb 98) +- fix macros _tr_tally_* in deflate.h for debug mode (Glenn Randers-Pehrson) +- remove block truncation heuristic which had very marginal effect for zlib + (smaller lit_bufsize than in gzip 1.2.4) and degraded a little the + compression ratio on some files. This also allows inlining _tr_tally for + matches in deflate_slow +- added msdos/Makefile.w32 for WIN32 Microsoft Visual C++ (Bob Frazier) + +Changes in 1.1.0 (24 Feb 98) +- do not return STREAM_END prematurely in inflate (John Bowler) +- revert to the zlib 1.0.8 inflate to avoid the gcc 2.8.0 bug (Jeremy Buhler) +- compile with -DFASTEST to get compression code optimized for speed only +- in minigzip, try mmap'ing the input file first (Miguel Albrecht) +- increase size of I/O buffers in minigzip.c and gzio.c (not a big gain + on Sun but significant on HP) + +- add a pointer to experimental unzip library in README (Gilles Vollant) +- initialize variable gcc in configure (Chris Herborth) + +Changes in 1.0.9 (17 Feb 1998) +- added gzputs and gzgets functions +- do not clear eof flag in gzseek (Mark Diekhans) +- fix gzseek for files in transparent mode (Mark Diekhans) +- do not assume that vsprintf returns the number of bytes written (Jens Krinke) +- replace EXPORT with ZEXPORT to avoid conflict with other programs +- added compress2 in zconf.h, zlib.def, zlib.dnt +- new asm code from Gilles Vollant in contrib/asm386 +- simplify the inflate code (Mark): + . Replace ZALLOC's in huft_build() with single ZALLOC in inflate_blocks_new() + . ZALLOC the length list in inflate_trees_fixed() instead of using stack + . ZALLOC the value area for huft_build() instead of using stack + . Simplify Z_FINISH check in inflate() + +- Avoid gcc 2.8.0 comparison bug a little differently than zlib 1.0.8 +- in inftrees.c, avoid cc -O bug on HP (Farshid Elahi) +- in zconf.h move the ZLIB_DLL stuff earlier to avoid problems with + the declaration of FAR (Gilles Vollant) +- install libz.so* with mode 755 (executable) instead of 644 (Marc Lehmann) +- read_buf buf parameter of type Bytef* instead of charf* +- zmemcpy parameters are of type Bytef*, not charf* (Joseph Strout) +- do not redeclare unlink in minigzip.c for WIN32 (John Bowler) +- fix check for presence of directories in "make install" (Ian Willis) + +Changes in 1.0.8 (27 Jan 1998) +- fixed offsets in contrib/asm386/gvmat32.asm (Gilles Vollant) +- fix gzgetc and gzputc for big endian systems (Markus Oberhumer) +- added compress2() to allow setting the compression level +- include sys/types.h to get off_t on some systems (Marc Lehmann & QingLong) +- use constant arrays for the static trees in trees.c instead of computing + them at run time (thanks to Ken Raeburn for this suggestion). To create + trees.h, compile with GEN_TREES_H and run "make test" +- check return code of example in "make test" and display result +- pass minigzip command line options to file_compress +- simplifying code of inflateSync to avoid gcc 2.8 bug + +- support CC="gcc -Wall" in configure -s (QingLong) +- avoid a flush caused by ftell in gzopen for write mode (Ken Raeburn) +- fix test for shared library support to avoid compiler warnings +- zlib.lib -> zlib.dll in msdos/zlib.rc (Gilles Vollant) +- check for TARGET_OS_MAC in addition to MACOS (Brad Pettit) +- do not use fdopen for Metrowerks on Mac (Brad Pettit)) +- add checks for gzputc and gzputc in example.c +- avoid warnings in gzio.c and deflate.c (Andreas Kleinert) +- use const for the CRC table (Ken Raeburn) +- fixed "make uninstall" for shared libraries +- use Tracev instead of Trace in infblock.c +- in example.c use correct compressed length for test_sync +- suppress +vnocompatwarnings in configure for HPUX (not always supported) + +Changes in 1.0.7 (20 Jan 1998) +- fix gzseek which was broken in write mode +- return error for gzseek to negative absolute position +- fix configure for Linux (Chun-Chung Chen) +- increase stack space for MSC (Tim Wegner) +- get_crc_table and inflateSyncPoint are EXPORTed (Gilles Vollant) +- define EXPORTVA for gzprintf (Gilles Vollant) +- added man page zlib.3 (Rick Rodgers) +- for contrib/untgz, fix makedir() and improve Makefile + +- check gzseek in write mode in example.c +- allocate extra buffer for seeks only if gzseek is actually called +- avoid signed/unsigned comparisons (Tim Wegner, Gilles Vollant) +- add inflateSyncPoint in zconf.h +- fix list of exported functions in nt/zlib.dnt and mdsos/zlib.def + +Changes in 1.0.6 (19 Jan 1998) +- add functions gzprintf, gzputc, gzgetc, gztell, gzeof, gzseek, gzrewind and + gzsetparams (thanks to Roland Giersig and Kevin Ruland for some of this code) +- Fix a deflate bug occurring only with compression level 0 (thanks to + Andy Buckler for finding this one) +- In minigzip, pass transparently also the first byte for .Z files +- return Z_BUF_ERROR instead of Z_OK if output buffer full in uncompress() +- check Z_FINISH in inflate (thanks to Marc Schluper) +- Implement deflateCopy (thanks to Adam Costello) +- make static libraries by default in configure, add --shared option +- move MSDOS or Windows specific files to directory msdos +- suppress the notion of partial flush to simplify the interface + (but the symbol Z_PARTIAL_FLUSH is kept for compatibility with 1.0.4) +- suppress history buffer provided by application to simplify the interface + (this feature was not implemented anyway in 1.0.4) +- next_in and avail_in must be initialized before calling inflateInit or + inflateInit2 +- add EXPORT in all exported functions (for Windows DLL) +- added Makefile.nt (thanks to Stephen Williams) +- added the unsupported "contrib" directory: + contrib/asm386/ by Gilles Vollant + 386 asm code replacing longest_match() + contrib/iostream/ by Kevin Ruland + A C++ I/O streams interface to the zlib gz* functions + contrib/iostream2/ by Tyge Løvset + Another C++ I/O streams interface + contrib/untgz/ by "Pedro A. Aranda Guti\irrez" + A very simple tar.gz file extractor using zlib + contrib/visual-basic.txt by Carlos Rios + How to use compress(), uncompress() and the gz* functions from VB +- pass params -f (filtered data), -h (huffman only), -1 to -9 (compression + level) in minigzip (thanks to Tom Lane) + +- use const for rommable constants in deflate +- added test for gzseek and gztell in example.c +- add undocumented function inflateSyncPoint() (hack for Paul Mackerras) +- add undocumented function zError to convert error code to string + (for Tim Smithers) +- Allow compilation of gzio with -DNO_DEFLATE to avoid the compression code +- Use default memcpy for Symantec MSDOS compiler +- Add EXPORT keyword for check_func (needed for Windows DLL) +- add current directory to LD_LIBRARY_PATH for "make test" +- create also a link for libz.so.1 +- added support for FUJITSU UXP/DS (thanks to Toshiaki Nomura) +- use $(SHAREDLIB) instead of libz.so in Makefile.in (for HPUX) +- added -soname for Linux in configure (Chun-Chung Chen, +- assign numbers to the exported functions in zlib.def (for Windows DLL) +- add advice in zlib.h for best usage of deflateSetDictionary +- work around compiler bug on Atari (cast Z_NULL in call of s->checkfn) +- allow compilation with ANSI keywords only enabled for TurboC in large model +- avoid "versionString"[0] (Borland bug) +- add NEED_DUMMY_RETURN for Borland +- use variable z_verbose for tracing in debug mode (L. Peter Deutsch) +- allow compilation with CC +- defined STDC for OS/2 (David Charlap) +- limit external names to 8 chars for MVS (Thomas Lund) +- in minigzip.c, use static buffers only for 16-bit systems +- fix suffix check for "minigzip -d foo.gz" +- do not return an error for the 2nd of two consecutive gzflush() (Felix Lee) +- use _fdopen instead of fdopen for MSC >= 6.0 (Thomas Fanslau) +- added makelcc.bat for lcc-win32 (Tom St Denis) +- in Makefile.dj2, use copy and del instead of install and rm (Frank Donahoe) +- Avoid expanded $Id$. Use "rcs -kb" or "cvs admin -kb" to avoid Id expansion +- check for unistd.h in configure (for off_t) +- remove useless check parameter in inflate_blocks_free +- avoid useless assignment of s->check to itself in inflate_blocks_new +- do not flush twice in gzclose (thanks to Ken Raeburn) +- rename FOPEN as F_OPEN to avoid clash with /usr/include/sys/file.h +- use NO_ERRNO_H instead of enumeration of operating systems with errno.h +- work around buggy fclose on pipes for HP/UX +- support zlib DLL with BORLAND C++ 5.0 (thanks to Glenn Randers-Pehrson) +- fix configure if CC is already equal to gcc + +Changes in 1.0.5 (3 Jan 98) +- Fix inflate to terminate gracefully when fed corrupted or invalid data +- Use const for rommable constants in inflate +- Eliminate memory leaks on error conditions in inflate +- Removed some vestigial code in inflate +- Update web address in README + +Changes in 1.0.4 (24 Jul 96) +- In very rare conditions, deflate(s, Z_FINISH) could fail to produce an EOF + bit, so the decompressor could decompress all the correct data but went + on to attempt decompressing extra garbage data. This affected minigzip too +- zlibVersion and gzerror return const char* (needed for DLL) +- port to RISCOS (no fdopen, no multiple dots, no unlink, no fileno) +- use z_error only for DEBUG (avoid problem with DLLs) + +Changes in 1.0.3 (2 Jul 96) +- use z_streamp instead of z_stream *, which is now a far pointer in MSDOS + small and medium models; this makes the library incompatible with previous + versions for these models. (No effect in large model or on other systems.) +- return OK instead of BUF_ERROR if previous deflate call returned with + avail_out as zero but there is nothing to do +- added memcmp for non STDC compilers +- define NO_DUMMY_DECL for more Mac compilers (.h files merged incorrectly) +- define __32BIT__ if __386__ or i386 is defined (pb. with Watcom and SCO) +- better check for 16-bit mode MSC (avoids problem with Symantec) + +Changes in 1.0.2 (23 May 96) +- added Windows DLL support +- added a function zlibVersion (for the DLL support) +- fixed declarations using Bytef in infutil.c (pb with MSDOS medium model) +- Bytef is define's instead of typedef'd only for Borland C +- avoid reading uninitialized memory in example.c +- mention in README that the zlib format is now RFC1950 +- updated Makefile.dj2 +- added algorithm.doc + +Changes in 1.0.1 (20 May 96) [1.0 skipped to avoid confusion] +- fix array overlay in deflate.c which sometimes caused bad compressed data +- fix inflate bug with empty stored block +- fix MSDOS medium model which was broken in 0.99 +- fix deflateParams() which could generate bad compressed data +- Bytef is define'd instead of typedef'ed (work around Borland bug) +- added an INDEX file +- new makefiles for DJGPP (Makefile.dj2), 32-bit Borland (Makefile.b32), + Watcom (Makefile.wat), Amiga SAS/C (Makefile.sas) +- speed up adler32 for modern machines without auto-increment +- added -ansi for IRIX in configure +- static_init_done in trees.c is an int +- define unlink as delete for VMS +- fix configure for QNX +- add configure branch for SCO and HPUX +- avoid many warnings (unused variables, dead assignments, etc...) +- no fdopen for BeOS +- fix the Watcom fix for 32 bit mode (define FAR as empty) +- removed redefinition of Byte for MKWERKS +- work around an MWKERKS bug (incorrect merge of all .h files) + +Changes in 0.99 (27 Jan 96) +- allow preset dictionary shared between compressor and decompressor +- allow compression level 0 (no compression) +- add deflateParams in zlib.h: allow dynamic change of compression level + and compression strategy +- test large buffers and deflateParams in example.c +- add optional "configure" to build zlib as a shared library +- suppress Makefile.qnx, use configure instead +- fixed deflate for 64-bit systems (detected on Cray) +- fixed inflate_blocks for 64-bit systems (detected on Alpha) +- declare Z_DEFLATED in zlib.h (possible parameter for deflateInit2) +- always return Z_BUF_ERROR when deflate() has nothing to do +- deflateInit and inflateInit are now macros to allow version checking +- prefix all global functions and types with z_ with -DZ_PREFIX +- make falloc completely reentrant (inftrees.c) +- fixed very unlikely race condition in ct_static_init +- free in reverse order of allocation to help memory manager +- use zlib-1.0/* instead of zlib/* inside the tar.gz +- make zlib warning-free with "gcc -O3 -Wall -Wwrite-strings -Wpointer-arith + -Wconversion -Wstrict-prototypes -Wmissing-prototypes" +- allow gzread on concatenated .gz files +- deflateEnd now returns Z_DATA_ERROR if it was premature +- deflate is finally (?) fully deterministic (no matches beyond end of input) +- Document Z_SYNC_FLUSH +- add uninstall in Makefile +- Check for __cpluplus in zlib.h +- Better test in ct_align for partial flush +- avoid harmless warnings for Borland C++ +- initialize hash_head in deflate.c +- avoid warning on fdopen (gzio.c) for HP cc -Aa +- include stdlib.h for STDC compilers +- include errno.h for Cray +- ignore error if ranlib doesn't exist +- call ranlib twice for NeXTSTEP +- use exec_prefix instead of prefix for libz.a +- renamed ct_* as _tr_* to avoid conflict with applications +- clear z->msg in inflateInit2 before any error return +- initialize opaque in example.c, gzio.c, deflate.c and inflate.c +- fixed typo in zconf.h (_GNUC__ => __GNUC__) +- check for WIN32 in zconf.h and zutil.c (avoid farmalloc in 32-bit mode) +- fix typo in Make_vms.com (f$trnlnm -> f$getsyi) +- in fcalloc, normalize pointer if size > 65520 bytes +- don't use special fcalloc for 32 bit Borland C++ +- use STDC instead of __GO32__ to avoid redeclaring exit, calloc, etc... +- use Z_BINARY instead of BINARY +- document that gzclose after gzdopen will close the file +- allow "a" as mode in gzopen +- fix error checking in gzread +- allow skipping .gz extra-field on pipes +- added reference to Perl interface in README +- put the crc table in FAR data (I dislike more and more the medium model :) +- added get_crc_table +- added a dimension to all arrays (Borland C can't count) +- workaround Borland C bug in declaration of inflate_codes_new & inflate_fast +- guard against multiple inclusion of *.h (for precompiled header on Mac) +- Watcom C pretends to be Microsoft C small model even in 32 bit mode +- don't use unsized arrays to avoid silly warnings by Visual C++: + warning C4746: 'inflate_mask' : unsized array treated as '__far' + (what's wrong with far data in far model?) +- define enum out of inflate_blocks_state to allow compilation with C++ + +Changes in 0.95 (16 Aug 95) +- fix MSDOS small and medium model (now easier to adapt to any compiler) +- inlined send_bits +- fix the final (:-) bug for deflate with flush (output was correct but + not completely flushed in rare occasions) +- default window size is same for compression and decompression + (it's now sufficient to set MAX_WBITS in zconf.h) +- voidp -> voidpf and voidnp -> voidp (for consistency with other + typedefs and because voidnp was not near in large model) + +Changes in 0.94 (13 Aug 95) +- support MSDOS medium model +- fix deflate with flush (could sometimes generate bad output) +- fix deflateReset (zlib header was incorrectly suppressed) +- added support for VMS +- allow a compression level in gzopen() +- gzflush now calls fflush +- For deflate with flush, flush even if no more input is provided +- rename libgz.a as libz.a +- avoid complex expression in infcodes.c triggering Turbo C bug +- work around a problem with gcc on Alpha (in INSERT_STRING) +- don't use inline functions (problem with some gcc versions) +- allow renaming of Byte, uInt, etc... with #define +- avoid warning about (unused) pointer before start of array in deflate.c +- avoid various warnings in gzio.c, example.c, infblock.c, adler32.c, zutil.c +- avoid reserved word 'new' in trees.c + +Changes in 0.93 (25 June 95) +- temporarily disable inline functions +- make deflate deterministic +- give enough lookahead for PARTIAL_FLUSH +- Set binary mode for stdin/stdout in minigzip.c for OS/2 +- don't even use signed char in inflate (not portable enough) +- fix inflate memory leak for segmented architectures + +Changes in 0.92 (3 May 95) +- don't assume that char is signed (problem on SGI) +- Clear bit buffer when starting a stored block +- no memcpy on Pyramid +- suppressed inftest.c +- optimized fill_window, put longest_match inline for gcc +- optimized inflate on stored blocks +- untabify all sources to simplify patches + +Changes in 0.91 (2 May 95) +- Default MEM_LEVEL is 8 (not 9 for Unix) as documented in zlib.h +- Document the memory requirements in zconf.h +- added "make install" +- fix sync search logic in inflateSync +- deflate(Z_FULL_FLUSH) now works even if output buffer too short +- after inflateSync, don't scare people with just "lo world" +- added support for DJGPP + +Changes in 0.9 (1 May 95) +- don't assume that zalloc clears the allocated memory (the TurboC bug + was Mark's bug after all :) +- let again gzread copy uncompressed data unchanged (was working in 0.71) +- deflate(Z_FULL_FLUSH), inflateReset and inflateSync are now fully implemented +- added a test of inflateSync in example.c +- moved MAX_WBITS to zconf.h because users might want to change that +- document explicitly that zalloc(64K) on MSDOS must return a normalized + pointer (zero offset) +- added Makefiles for Microsoft C, Turbo C, Borland C++ +- faster crc32() + +Changes in 0.8 (29 April 95) +- added fast inflate (inffast.c) +- deflate(Z_FINISH) now returns Z_STREAM_END when done. Warning: this + is incompatible with previous versions of zlib which returned Z_OK +- work around a TurboC compiler bug (bad code for b << 0, see infutil.h) + (actually that was not a compiler bug, see 0.81 above) +- gzread no longer reads one extra byte in certain cases +- In gzio destroy(), don't reference a freed structure +- avoid many warnings for MSDOS +- avoid the ERROR symbol which is used by MS Windows + +Changes in 0.71 (14 April 95) +- Fixed more MSDOS compilation problems :( There is still a bug with + TurboC large model + +Changes in 0.7 (14 April 95) +- Added full inflate support +- Simplified the crc32() interface. The pre- and post-conditioning + (one's complement) is now done inside crc32(). WARNING: this is + incompatible with previous versions; see zlib.h for the new usage + +Changes in 0.61 (12 April 95) +- workaround for a bug in TurboC. example and minigzip now work on MSDOS + +Changes in 0.6 (11 April 95) +- added minigzip.c +- added gzdopen to reopen a file descriptor as gzFile +- added transparent reading of non-gziped files in gzread +- fixed bug in gzread (don't read crc as data) +- fixed bug in destroy (gzio.c) (don't return Z_STREAM_END for gzclose) +- don't allocate big arrays in the stack (for MSDOS) +- fix some MSDOS compilation problems + +Changes in 0.5: +- do real compression in deflate.c. Z_PARTIAL_FLUSH is supported but + not yet Z_FULL_FLUSH +- support decompression but only in a single step (forced Z_FINISH) +- added opaque object for zalloc and zfree +- added deflateReset and inflateReset +- added a variable zlib_version for consistency checking +- renamed the 'filter' parameter of deflateInit2 as 'strategy' + Added Z_FILTERED and Z_HUFFMAN_ONLY constants + +Changes in 0.4: +- avoid "zip" everywhere, use zlib instead of ziplib +- suppress Z_BLOCK_FLUSH, interpret Z_PARTIAL_FLUSH as block flush + if compression method == 8 +- added adler32 and crc32 +- renamed deflateOptions as deflateInit2, call one or the other but not both +- added the method parameter for deflateInit2 +- added inflateInit2 +- simplified considerably deflateInit and inflateInit by not supporting + user-provided history buffer. This is supported only in deflateInit2 + and inflateInit2 + +Changes in 0.3: +- prefix all macro names with Z_ +- use Z_FINISH instead of deflateEnd to finish compression +- added Z_HUFFMAN_ONLY +- added gzerror() diff --git a/lib/os/windows/zlib-1.2.13/FAQ b/lib/os/windows/zlib-1.2.13/FAQ new file mode 100644 index 000000000000..99b7cf92e454 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/FAQ @@ -0,0 +1,368 @@ + + Frequently Asked Questions about zlib + + +If your question is not there, please check the zlib home page +http://zlib.net/ which may have more recent information. +The lastest zlib FAQ is at http://zlib.net/zlib_faq.html + + + 1. Is zlib Y2K-compliant? + + Yes. zlib doesn't handle dates. + + 2. Where can I get a Windows DLL version? + + The zlib sources can be compiled without change to produce a DLL. See the + file win32/DLL_FAQ.txt in the zlib distribution. Pointers to the + precompiled DLL are found in the zlib web site at http://zlib.net/ . + + 3. Where can I get a Visual Basic interface to zlib? + + See + * http://marknelson.us/1997/01/01/zlib-engine/ + * win32/DLL_FAQ.txt in the zlib distribution + + 4. compress() returns Z_BUF_ERROR. + + Make sure that before the call of compress(), the length of the compressed + buffer is equal to the available size of the compressed buffer and not + zero. For Visual Basic, check that this parameter is passed by reference + ("as any"), not by value ("as long"). + + 5. deflate() or inflate() returns Z_BUF_ERROR. + + Before making the call, make sure that avail_in and avail_out are not zero. + When setting the parameter flush equal to Z_FINISH, also make sure that + avail_out is big enough to allow processing all pending input. Note that a + Z_BUF_ERROR is not fatal--another call to deflate() or inflate() can be + made with more input or output space. A Z_BUF_ERROR may in fact be + unavoidable depending on how the functions are used, since it is not + possible to tell whether or not there is more output pending when + strm.avail_out returns with zero. See http://zlib.net/zlib_how.html for a + heavily annotated example. + + 6. Where's the zlib documentation (man pages, etc.)? + + It's in zlib.h . Examples of zlib usage are in the files test/example.c + and test/minigzip.c, with more in examples/ . + + 7. Why don't you use GNU autoconf or libtool or ...? + + Because we would like to keep zlib as a very small and simple package. + zlib is rather portable and doesn't need much configuration. + + 8. I found a bug in zlib. + + Most of the time, such problems are due to an incorrect usage of zlib. + Please try to reproduce the problem with a small program and send the + corresponding source to us at zlib@gzip.org . Do not send multi-megabyte + data files without prior agreement. + + 9. Why do I get "undefined reference to gzputc"? + + If "make test" produces something like + + example.o(.text+0x154): undefined reference to `gzputc' + + check that you don't have old files libz.* in /usr/lib, /usr/local/lib or + /usr/X11R6/lib. Remove any old versions, then do "make install". + +10. I need a Delphi interface to zlib. + + See the contrib/delphi directory in the zlib distribution. + +11. Can zlib handle .zip archives? + + Not by itself, no. See the directory contrib/minizip in the zlib + distribution. + +12. Can zlib handle .Z files? + + No, sorry. You have to spawn an uncompress or gunzip subprocess, or adapt + the code of uncompress on your own. + +13. How can I make a Unix shared library? + + By default a shared (and a static) library is built for Unix. So: + + make distclean + ./configure + make + +14. How do I install a shared zlib library on Unix? + + After the above, then: + + make install + + However, many flavors of Unix come with a shared zlib already installed. + Before going to the trouble of compiling a shared version of zlib and + trying to install it, you may want to check if it's already there! If you + can #include , it's there. The -lz option will probably link to + it. You can check the version at the top of zlib.h or with the + ZLIB_VERSION symbol defined in zlib.h . + +15. I have a question about OttoPDF. + + We are not the authors of OttoPDF. The real author is on the OttoPDF web + site: Joel Hainley, jhainley@myndkryme.com. + +16. Can zlib decode Flate data in an Adobe PDF file? + + Yes. See http://www.pdflib.com/ . To modify PDF forms, see + http://sourceforge.net/projects/acroformtool/ . + +17. Why am I getting this "register_frame_info not found" error on Solaris? + + After installing zlib 1.1.4 on Solaris 2.6, running applications using zlib + generates an error such as: + + ld.so.1: rpm: fatal: relocation error: file /usr/local/lib/libz.so: + symbol __register_frame_info: referenced symbol not found + + The symbol __register_frame_info is not part of zlib, it is generated by + the C compiler (cc or gcc). You must recompile applications using zlib + which have this problem. This problem is specific to Solaris. See + http://www.sunfreeware.com for Solaris versions of zlib and applications + using zlib. + +18. Why does gzip give an error on a file I make with compress/deflate? + + The compress and deflate functions produce data in the zlib format, which + is different and incompatible with the gzip format. The gz* functions in + zlib on the other hand use the gzip format. Both the zlib and gzip formats + use the same compressed data format internally, but have different headers + and trailers around the compressed data. + +19. Ok, so why are there two different formats? + + The gzip format was designed to retain the directory information about a + single file, such as the name and last modification date. The zlib format + on the other hand was designed for in-memory and communication channel + applications, and has a much more compact header and trailer and uses a + faster integrity check than gzip. + +20. Well that's nice, but how do I make a gzip file in memory? + + You can request that deflate write the gzip format instead of the zlib + format using deflateInit2(). You can also request that inflate decode the + gzip format using inflateInit2(). Read zlib.h for more details. + +21. Is zlib thread-safe? + + Yes. However any library routines that zlib uses and any application- + provided memory allocation routines must also be thread-safe. zlib's gz* + functions use stdio library routines, and most of zlib's functions use the + library memory allocation routines by default. zlib's *Init* functions + allow for the application to provide custom memory allocation routines. + + Of course, you should only operate on any given zlib or gzip stream from a + single thread at a time. + +22. Can I use zlib in my commercial application? + + Yes. Please read the license in zlib.h. + +23. Is zlib under the GNU license? + + No. Please read the license in zlib.h. + +24. The license says that altered source versions must be "plainly marked". So + what exactly do I need to do to meet that requirement? + + You need to change the ZLIB_VERSION and ZLIB_VERNUM #defines in zlib.h. In + particular, the final version number needs to be changed to "f", and an + identification string should be appended to ZLIB_VERSION. Version numbers + x.x.x.f are reserved for modifications to zlib by others than the zlib + maintainers. For example, if the version of the base zlib you are altering + is "1.2.3.4", then in zlib.h you should change ZLIB_VERNUM to 0x123f, and + ZLIB_VERSION to something like "1.2.3.f-zachary-mods-v3". You can also + update the version strings in deflate.c and inftrees.c. + + For altered source distributions, you should also note the origin and + nature of the changes in zlib.h, as well as in ChangeLog and README, along + with the dates of the alterations. The origin should include at least your + name (or your company's name), and an email address to contact for help or + issues with the library. + + Note that distributing a compiled zlib library along with zlib.h and + zconf.h is also a source distribution, and so you should change + ZLIB_VERSION and ZLIB_VERNUM and note the origin and nature of the changes + in zlib.h as you would for a full source distribution. + +25. Will zlib work on a big-endian or little-endian architecture, and can I + exchange compressed data between them? + + Yes and yes. + +26. Will zlib work on a 64-bit machine? + + Yes. It has been tested on 64-bit machines, and has no dependence on any + data types being limited to 32-bits in length. If you have any + difficulties, please provide a complete problem report to zlib@gzip.org + +27. Will zlib decompress data from the PKWare Data Compression Library? + + No. The PKWare DCL uses a completely different compressed data format than + does PKZIP and zlib. However, you can look in zlib's contrib/blast + directory for a possible solution to your problem. + +28. Can I access data randomly in a compressed stream? + + No, not without some preparation. If when compressing you periodically use + Z_FULL_FLUSH, carefully write all the pending data at those points, and + keep an index of those locations, then you can start decompression at those + points. You have to be careful to not use Z_FULL_FLUSH too often, since it + can significantly degrade compression. Alternatively, you can scan a + deflate stream once to generate an index, and then use that index for + random access. See examples/zran.c . + +29. Does zlib work on MVS, OS/390, CICS, etc.? + + It has in the past, but we have not heard of any recent evidence. There + were working ports of zlib 1.1.4 to MVS, but those links no longer work. + If you know of recent, successful applications of zlib on these operating + systems, please let us know. Thanks. + +30. Is there some simpler, easier to read version of inflate I can look at to + understand the deflate format? + + First off, you should read RFC 1951. Second, yes. Look in zlib's + contrib/puff directory. + +31. Does zlib infringe on any patents? + + As far as we know, no. In fact, that was originally the whole point behind + zlib. Look here for some more information: + + http://www.gzip.org/#faq11 + +32. Can zlib work with greater than 4 GB of data? + + Yes. inflate() and deflate() will process any amount of data correctly. + Each call of inflate() or deflate() is limited to input and output chunks + of the maximum value that can be stored in the compiler's "unsigned int" + type, but there is no limit to the number of chunks. Note however that the + strm.total_in and strm_total_out counters may be limited to 4 GB. These + counters are provided as a convenience and are not used internally by + inflate() or deflate(). The application can easily set up its own counters + updated after each call of inflate() or deflate() to count beyond 4 GB. + compress() and uncompress() may be limited to 4 GB, since they operate in a + single call. gzseek() and gztell() may be limited to 4 GB depending on how + zlib is compiled. See the zlibCompileFlags() function in zlib.h. + + The word "may" appears several times above since there is a 4 GB limit only + if the compiler's "long" type is 32 bits. If the compiler's "long" type is + 64 bits, then the limit is 16 exabytes. + +33. Does zlib have any security vulnerabilities? + + The only one that we are aware of is potentially in gzprintf(). If zlib is + compiled to use sprintf() or vsprintf(), then there is no protection + against a buffer overflow of an 8K string space (or other value as set by + gzbuffer()), other than the caller of gzprintf() assuring that the output + will not exceed 8K. On the other hand, if zlib is compiled to use + snprintf() or vsnprintf(), which should normally be the case, then there is + no vulnerability. The ./configure script will display warnings if an + insecure variation of sprintf() will be used by gzprintf(). Also the + zlibCompileFlags() function will return information on what variant of + sprintf() is used by gzprintf(). + + If you don't have snprintf() or vsnprintf() and would like one, you can + find a portable implementation here: + + http://www.ijs.si/software/snprintf/ + + Note that you should be using the most recent version of zlib. Versions + 1.1.3 and before were subject to a double-free vulnerability, and versions + 1.2.1 and 1.2.2 were subject to an access exception when decompressing + invalid compressed data. + +34. Is there a Java version of zlib? + + Probably what you want is to use zlib in Java. zlib is already included + as part of the Java SDK in the java.util.zip package. If you really want + a version of zlib written in the Java language, look on the zlib home + page for links: http://zlib.net/ . + +35. I get this or that compiler or source-code scanner warning when I crank it + up to maximally-pedantic. Can't you guys write proper code? + + Many years ago, we gave up attempting to avoid warnings on every compiler + in the universe. It just got to be a waste of time, and some compilers + were downright silly as well as contradicted each other. So now, we simply + make sure that the code always works. + +36. Valgrind (or some similar memory access checker) says that deflate is + performing a conditional jump that depends on an uninitialized value. + Isn't that a bug? + + No. That is intentional for performance reasons, and the output of deflate + is not affected. This only started showing up recently since zlib 1.2.x + uses malloc() by default for allocations, whereas earlier versions used + calloc(), which zeros out the allocated memory. Even though the code was + correct, versions 1.2.4 and later was changed to not stimulate these + checkers. + +37. Will zlib read the (insert any ancient or arcane format here) compressed + data format? + + Probably not. Look in the comp.compression FAQ for pointers to various + formats and associated software. + +38. How can I encrypt/decrypt zip files with zlib? + + zlib doesn't support encryption. The original PKZIP encryption is very + weak and can be broken with freely available programs. To get strong + encryption, use GnuPG, http://www.gnupg.org/ , which already includes zlib + compression. For PKZIP compatible "encryption", look at + http://www.info-zip.org/ + +39. What's the difference between the "gzip" and "deflate" HTTP 1.1 encodings? + + "gzip" is the gzip format, and "deflate" is the zlib format. They should + probably have called the second one "zlib" instead to avoid confusion with + the raw deflate compressed data format. While the HTTP 1.1 RFC 2616 + correctly points to the zlib specification in RFC 1950 for the "deflate" + transfer encoding, there have been reports of servers and browsers that + incorrectly produce or expect raw deflate data per the deflate + specification in RFC 1951, most notably Microsoft. So even though the + "deflate" transfer encoding using the zlib format would be the more + efficient approach (and in fact exactly what the zlib format was designed + for), using the "gzip" transfer encoding is probably more reliable due to + an unfortunate choice of name on the part of the HTTP 1.1 authors. + + Bottom line: use the gzip format for HTTP 1.1 encoding. + +40. Does zlib support the new "Deflate64" format introduced by PKWare? + + No. PKWare has apparently decided to keep that format proprietary, since + they have not documented it as they have previous compression formats. In + any case, the compression improvements are so modest compared to other more + modern approaches, that it's not worth the effort to implement. + +41. I'm having a problem with the zip functions in zlib, can you help? + + There are no zip functions in zlib. You are probably using minizip by + Giles Vollant, which is found in the contrib directory of zlib. It is not + part of zlib. In fact none of the stuff in contrib is part of zlib. The + files in there are not supported by the zlib authors. You need to contact + the authors of the respective contribution for help. + +42. The match.asm code in contrib is under the GNU General Public License. + Since it's part of zlib, doesn't that mean that all of zlib falls under the + GNU GPL? + + No. The files in contrib are not part of zlib. They were contributed by + other authors and are provided as a convenience to the user within the zlib + distribution. Each item in contrib has its own license. + +43. Is zlib subject to export controls? What is its ECCN? + + zlib is not subject to export controls, and so is classified as EAR99. + +44. Can you please sign these lengthy legal documents and fax them back to us + so that we can use your software in our product? + + No. Go away. Shoo. diff --git a/lib/os/windows/zlib-1.2.13/INDEX b/lib/os/windows/zlib-1.2.13/INDEX new file mode 100644 index 000000000000..2ba064120486 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/INDEX @@ -0,0 +1,68 @@ +CMakeLists.txt cmake build file +ChangeLog history of changes +FAQ Frequently Asked Questions about zlib +INDEX this file +Makefile dummy Makefile that tells you to ./configure +Makefile.in template for Unix Makefile +README guess what +configure configure script for Unix +make_vms.com makefile for VMS +test/example.c zlib usages examples for build testing +test/minigzip.c minimal gzip-like functionality for build testing +test/infcover.c inf*.c code coverage for build coverage testing +treebuild.xml XML description of source file dependencies +zconf.h.cmakein zconf.h template for cmake +zconf.h.in zconf.h template for configure +zlib.3 Man page for zlib +zlib.3.pdf Man page in PDF format +zlib.map Linux symbol information +zlib.pc.in Template for pkg-config descriptor +zlib.pc.cmakein zlib.pc template for cmake +zlib2ansi perl script to convert source files for C++ compilation + +amiga/ makefiles for Amiga SAS C +as400/ makefiles for AS/400 +doc/ documentation for formats and algorithms +msdos/ makefiles for MSDOS +nintendods/ makefile for Nintendo DS +old/ makefiles for various architectures and zlib documentation + files that have not yet been updated for zlib 1.2.x +qnx/ makefiles for QNX +watcom/ makefiles for OpenWatcom +win32/ makefiles for Windows + + zlib public header files (required for library use): +zconf.h +zlib.h + + private source files used to build the zlib library: +adler32.c +compress.c +crc32.c +crc32.h +deflate.c +deflate.h +gzclose.c +gzguts.h +gzlib.c +gzread.c +gzwrite.c +infback.c +inffast.c +inffast.h +inffixed.h +inflate.c +inflate.h +inftrees.c +inftrees.h +trees.c +trees.h +uncompr.c +zutil.c +zutil.h + + source files for sample programs +See examples/README.examples + + unsupported contributions by third parties +See contrib/README.contrib diff --git a/lib/os/windows/zlib-1.2.13/README b/lib/os/windows/zlib-1.2.13/README new file mode 100644 index 000000000000..ba34d1894a9b --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/README @@ -0,0 +1,118 @@ +ZLIB DATA COMPRESSION LIBRARY + +zlib 1.2.13 is a general purpose data compression library. All the code is +thread safe. The data format used by the zlib library is described by RFCs +(Request for Comments) 1950 to 1952 in the files +http://tools.ietf.org/html/rfc1950 (zlib format), rfc1951 (deflate format) and +rfc1952 (gzip format). + +All functions of the compression library are documented in the file zlib.h +(volunteer to write man pages welcome, contact zlib@gzip.org). A usage example +of the library is given in the file test/example.c which also tests that +the library is working correctly. Another example is given in the file +test/minigzip.c. The compression library itself is composed of all source +files in the root directory. + +To compile all files and run the test program, follow the instructions given at +the top of Makefile.in. In short "./configure; make test", and if that goes +well, "make install" should work for most flavors of Unix. For Windows, use +one of the special makefiles in win32/ or contrib/vstudio/ . For VMS, use +make_vms.com. + +Questions about zlib should be sent to , or to Gilles Vollant + for the Windows DLL version. The zlib home page is +http://zlib.net/ . Before reporting a problem, please check this site to +verify that you have the latest version of zlib; otherwise get the latest +version and check whether the problem still exists or not. + +PLEASE read the zlib FAQ http://zlib.net/zlib_faq.html before asking for help. + +Mark Nelson wrote an article about zlib for the Jan. 1997 +issue of Dr. Dobb's Journal; a copy of the article is available at +http://marknelson.us/1997/01/01/zlib-engine/ . + +The changes made in version 1.2.13 are documented in the file ChangeLog. + +Unsupported third party contributions are provided in directory contrib/ . + +zlib is available in Java using the java.util.zip package, documented at +http://java.sun.com/developer/technicalArticles/Programming/compression/ . + +A Perl interface to zlib written by Paul Marquess is available +at CPAN (Comprehensive Perl Archive Network) sites, including +http://search.cpan.org/~pmqs/IO-Compress-Zlib/ . + +A Python interface to zlib written by A.M. Kuchling is +available in Python 1.5 and later versions, see +http://docs.python.org/library/zlib.html . + +zlib is built into tcl: http://wiki.tcl.tk/4610 . + +An experimental package to read and write files in .zip format, written on top +of zlib by Gilles Vollant , is available in the +contrib/minizip directory of zlib. + + +Notes for some targets: + +- For Windows DLL versions, please see win32/DLL_FAQ.txt + +- For 64-bit Irix, deflate.c must be compiled without any optimization. With + -O, one libpng test fails. The test works in 32 bit mode (with the -n32 + compiler flag). The compiler bug has been reported to SGI. + +- zlib doesn't work with gcc 2.6.3 on a DEC 3000/300LX under OSF/1 2.1 it works + when compiled with cc. + +- On Digital Unix 4.0D (formely OSF/1) on AlphaServer, the cc option -std1 is + necessary to get gzprintf working correctly. This is done by configure. + +- zlib doesn't work on HP-UX 9.05 with some versions of /bin/cc. It works with + other compilers. Use "make test" to check your compiler. + +- gzdopen is not supported on RISCOS or BEOS. + +- For PalmOs, see http://palmzlib.sourceforge.net/ + + +Acknowledgments: + + The deflate format used by zlib was defined by Phil Katz. The deflate and + zlib specifications were written by L. Peter Deutsch. Thanks to all the + people who reported problems and suggested various improvements in zlib; they + are too numerous to cite here. + +Copyright notice: + + (C) 1995-2022 Jean-loup Gailly and Mark Adler + + This software is provided 'as-is', without any express or implied + warranty. In no event will the authors be held liable for any damages + arising from the use of this software. + + Permission is granted to anyone to use this software for any purpose, + including commercial applications, and to alter it and redistribute it + freely, subject to the following restrictions: + + 1. The origin of this software must not be misrepresented; you must not + claim that you wrote the original software. If you use this software + in a product, an acknowledgment in the product documentation would be + appreciated but is not required. + 2. Altered source versions must be plainly marked as such, and must not be + misrepresented as being the original software. + 3. This notice may not be removed or altered from any source distribution. + + Jean-loup Gailly Mark Adler + jloup@gzip.org madler@alumni.caltech.edu + +If you use the zlib library in a product, we would appreciate *not* receiving +lengthy legal documents to sign. The sources are provided for free but without +warranty of any kind. The library has been entirely written by Jean-loup +Gailly and Mark Adler; it does not include third-party code. We make all +contributions to and distributions of this project solely in our personal +capacity, and are not conveying any rights to any intellectual property of +any third parties. + +If you redistribute modified sources, we would appreciate that you include in +the file ChangeLog history information documenting your changes. Please read +the FAQ for more information on the distribution of modified source versions. diff --git a/lib/os/windows/zlib-1.2.13/adler32.c b/lib/os/windows/zlib-1.2.13/adler32.c new file mode 100644 index 000000000000..d0be4380a39c --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/adler32.c @@ -0,0 +1,186 @@ +/* adler32.c -- compute the Adler-32 checksum of a data stream + * Copyright (C) 1995-2011, 2016 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* @(#) $Id$ */ + +#include "zutil.h" + +local uLong adler32_combine_ OF((uLong adler1, uLong adler2, z_off64_t len2)); + +#define BASE 65521U /* largest prime smaller than 65536 */ +#define NMAX 5552 +/* NMAX is the largest n such that 255n(n+1)/2 + (n+1)(BASE-1) <= 2^32-1 */ + +#define DO1(buf,i) {adler += (buf)[i]; sum2 += adler;} +#define DO2(buf,i) DO1(buf,i); DO1(buf,i+1); +#define DO4(buf,i) DO2(buf,i); DO2(buf,i+2); +#define DO8(buf,i) DO4(buf,i); DO4(buf,i+4); +#define DO16(buf) DO8(buf,0); DO8(buf,8); + +/* use NO_DIVIDE if your processor does not do division in hardware -- + try it both ways to see which is faster */ +#ifdef NO_DIVIDE +/* note that this assumes BASE is 65521, where 65536 % 65521 == 15 + (thank you to John Reiser for pointing this out) */ +# define CHOP(a) \ + do { \ + unsigned long tmp = a >> 16; \ + a &= 0xffffUL; \ + a += (tmp << 4) - tmp; \ + } while (0) +# define MOD28(a) \ + do { \ + CHOP(a); \ + if (a >= BASE) a -= BASE; \ + } while (0) +# define MOD(a) \ + do { \ + CHOP(a); \ + MOD28(a); \ + } while (0) +# define MOD63(a) \ + do { /* this assumes a is not negative */ \ + z_off64_t tmp = a >> 32; \ + a &= 0xffffffffL; \ + a += (tmp << 8) - (tmp << 5) + tmp; \ + tmp = a >> 16; \ + a &= 0xffffL; \ + a += (tmp << 4) - tmp; \ + tmp = a >> 16; \ + a &= 0xffffL; \ + a += (tmp << 4) - tmp; \ + if (a >= BASE) a -= BASE; \ + } while (0) +#else +# define MOD(a) a %= BASE +# define MOD28(a) a %= BASE +# define MOD63(a) a %= BASE +#endif + +/* ========================================================================= */ +uLong ZEXPORT adler32_z(adler, buf, len) + uLong adler; + const Bytef *buf; + z_size_t len; +{ + unsigned long sum2; + unsigned n; + + /* split Adler-32 into component sums */ + sum2 = (adler >> 16) & 0xffff; + adler &= 0xffff; + + /* in case user likes doing a byte at a time, keep it fast */ + if (len == 1) { + adler += buf[0]; + if (adler >= BASE) + adler -= BASE; + sum2 += adler; + if (sum2 >= BASE) + sum2 -= BASE; + return adler | (sum2 << 16); + } + + /* initial Adler-32 value (deferred check for len == 1 speed) */ + if (buf == Z_NULL) + return 1L; + + /* in case short lengths are provided, keep it somewhat fast */ + if (len < 16) { + while (len--) { + adler += *buf++; + sum2 += adler; + } + if (adler >= BASE) + adler -= BASE; + MOD28(sum2); /* only added so many BASE's */ + return adler | (sum2 << 16); + } + + /* do length NMAX blocks -- requires just one modulo operation */ + while (len >= NMAX) { + len -= NMAX; + n = NMAX / 16; /* NMAX is divisible by 16 */ + do { + DO16(buf); /* 16 sums unrolled */ + buf += 16; + } while (--n); + MOD(adler); + MOD(sum2); + } + + /* do remaining bytes (less than NMAX, still just one modulo) */ + if (len) { /* avoid modulos if none remaining */ + while (len >= 16) { + len -= 16; + DO16(buf); + buf += 16; + } + while (len--) { + adler += *buf++; + sum2 += adler; + } + MOD(adler); + MOD(sum2); + } + + /* return recombined sums */ + return adler | (sum2 << 16); +} + +/* ========================================================================= */ +uLong ZEXPORT adler32(adler, buf, len) + uLong adler; + const Bytef *buf; + uInt len; +{ + return adler32_z(adler, buf, len); +} + +/* ========================================================================= */ +local uLong adler32_combine_(adler1, adler2, len2) + uLong adler1; + uLong adler2; + z_off64_t len2; +{ + unsigned long sum1; + unsigned long sum2; + unsigned rem; + + /* for negative len, return invalid adler32 as a clue for debugging */ + if (len2 < 0) + return 0xffffffffUL; + + /* the derivation of this formula is left as an exercise for the reader */ + MOD63(len2); /* assumes len2 >= 0 */ + rem = (unsigned)len2; + sum1 = adler1 & 0xffff; + sum2 = rem * sum1; + MOD(sum2); + sum1 += (adler2 & 0xffff) + BASE - 1; + sum2 += ((adler1 >> 16) & 0xffff) + ((adler2 >> 16) & 0xffff) + BASE - rem; + if (sum1 >= BASE) sum1 -= BASE; + if (sum1 >= BASE) sum1 -= BASE; + if (sum2 >= ((unsigned long)BASE << 1)) sum2 -= ((unsigned long)BASE << 1); + if (sum2 >= BASE) sum2 -= BASE; + return sum1 | (sum2 << 16); +} + +/* ========================================================================= */ +uLong ZEXPORT adler32_combine(adler1, adler2, len2) + uLong adler1; + uLong adler2; + z_off_t len2; +{ + return adler32_combine_(adler1, adler2, len2); +} + +uLong ZEXPORT adler32_combine64(adler1, adler2, len2) + uLong adler1; + uLong adler2; + z_off64_t len2; +{ + return adler32_combine_(adler1, adler2, len2); +} diff --git a/lib/os/windows/zlib-1.2.13/compress.c b/lib/os/windows/zlib-1.2.13/compress.c new file mode 100644 index 000000000000..2ad5326c14ec --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/compress.c @@ -0,0 +1,86 @@ +/* compress.c -- compress a memory buffer + * Copyright (C) 1995-2005, 2014, 2016 Jean-loup Gailly, Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* @(#) $Id$ */ + +#define ZLIB_INTERNAL +#include "zlib.h" + +/* =========================================================================== + Compresses the source buffer into the destination buffer. The level + parameter has the same meaning as in deflateInit. sourceLen is the byte + length of the source buffer. Upon entry, destLen is the total size of the + destination buffer, which must be at least 0.1% larger than sourceLen plus + 12 bytes. Upon exit, destLen is the actual size of the compressed buffer. + + compress2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_BUF_ERROR if there was not enough room in the output buffer, + Z_STREAM_ERROR if the level parameter is invalid. +*/ +int ZEXPORT compress2(dest, destLen, source, sourceLen, level) + Bytef *dest; + uLongf *destLen; + const Bytef *source; + uLong sourceLen; + int level; +{ + z_stream stream; + int err; + const uInt max = (uInt)-1; + uLong left; + + left = *destLen; + *destLen = 0; + + stream.zalloc = (alloc_func)0; + stream.zfree = (free_func)0; + stream.opaque = (voidpf)0; + + err = deflateInit(&stream, level); + if (err != Z_OK) return err; + + stream.next_out = dest; + stream.avail_out = 0; + stream.next_in = (z_const Bytef *)source; + stream.avail_in = 0; + + do { + if (stream.avail_out == 0) { + stream.avail_out = left > (uLong)max ? max : (uInt)left; + left -= stream.avail_out; + } + if (stream.avail_in == 0) { + stream.avail_in = sourceLen > (uLong)max ? max : (uInt)sourceLen; + sourceLen -= stream.avail_in; + } + err = deflate(&stream, sourceLen ? Z_NO_FLUSH : Z_FINISH); + } while (err == Z_OK); + + *destLen = stream.total_out; + deflateEnd(&stream); + return err == Z_STREAM_END ? Z_OK : err; +} + +/* =========================================================================== + */ +int ZEXPORT compress(dest, destLen, source, sourceLen) + Bytef *dest; + uLongf *destLen; + const Bytef *source; + uLong sourceLen; +{ + return compress2(dest, destLen, source, sourceLen, Z_DEFAULT_COMPRESSION); +} + +/* =========================================================================== + If the default memLevel or windowBits for deflateInit() is changed, then + this function needs to be updated. + */ +uLong ZEXPORT compressBound(sourceLen) + uLong sourceLen; +{ + return sourceLen + (sourceLen >> 12) + (sourceLen >> 14) + + (sourceLen >> 25) + 13; +} diff --git a/lib/os/windows/zlib-1.2.13/contrib/minizip/crypt.h b/lib/os/windows/zlib-1.2.13/contrib/minizip/crypt.h new file mode 100644 index 000000000000..1cc41f19d78d --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/contrib/minizip/crypt.h @@ -0,0 +1,132 @@ +/* crypt.h -- base code for crypt/uncrypt ZIPfile + + + Version 1.01e, February 12th, 2005 + + Copyright (C) 1998-2005 Gilles Vollant + + This code is a modified version of crypting code in Infozip distribution + + The encryption/decryption parts of this source code (as opposed to the + non-echoing password parts) were originally written in Europe. The + whole source package can be freely distributed, including from the USA. + (Prior to January 2000, re-export from the US was a violation of US law.) + + This encryption code is a direct transcription of the algorithm from + Roger Schlafly, described by Phil Katz in the file appnote.txt. This + file (appnote.txt) is distributed with the PKZIP program (even in the + version without encryption capabilities). + + If you don't need crypting in your application, just define symbols + NOCRYPT and NOUNCRYPT. + + This code support the "Traditional PKWARE Encryption". + + The new AES encryption added on Zip format by Winzip (see the page + http://www.winzip.com/aes_info.htm ) and PKWare PKZip 5.x Strong + Encryption is not supported. +*/ + +#define CRC32(c, b) ((*(pcrc_32_tab+(((int)(c) ^ (b)) & 0xff))) ^ ((c) >> 8)) + +/*********************************************************************** + * Return the next byte in the pseudo-random sequence + */ +static int decrypt_byte(unsigned long* pkeys, const z_crc_t* pcrc_32_tab) +{ + unsigned temp; /* POTENTIAL BUG: temp*(temp^1) may overflow in an + * unpredictable manner on 16-bit systems; not a problem + * with any known compiler so far, though */ + + (void)pcrc_32_tab; + temp = ((unsigned)(*(pkeys+2)) & 0xffff) | 2; + return (int)(((temp * (temp ^ 1)) >> 8) & 0xff); +} + +/*********************************************************************** + * Update the encryption keys with the next byte of plain text + */ +static int update_keys(unsigned long* pkeys,const z_crc_t* pcrc_32_tab,int c) +{ + (*(pkeys+0)) = CRC32((*(pkeys+0)), c); + (*(pkeys+1)) += (*(pkeys+0)) & 0xff; + (*(pkeys+1)) = (*(pkeys+1)) * 134775813L + 1; + { + register int keyshift = (int)((*(pkeys+1)) >> 24); + (*(pkeys+2)) = CRC32((*(pkeys+2)), keyshift); + } + return c; +} + + +/*********************************************************************** + * Initialize the encryption keys and the random header according to + * the given password. + */ +static void init_keys(const char* passwd,unsigned long* pkeys,const z_crc_t* pcrc_32_tab) +{ + *(pkeys+0) = 305419896L; + *(pkeys+1) = 591751049L; + *(pkeys+2) = 878082192L; + while (*passwd != '\0') { + update_keys(pkeys,pcrc_32_tab,(int)*passwd); + passwd++; + } +} + +#define zdecode(pkeys,pcrc_32_tab,c) \ + (update_keys(pkeys,pcrc_32_tab,c ^= decrypt_byte(pkeys,pcrc_32_tab))) + +#define zencode(pkeys,pcrc_32_tab,c,t) \ + (t=decrypt_byte(pkeys,pcrc_32_tab), update_keys(pkeys,pcrc_32_tab,c), (Byte)t^(c)) + +#ifdef INCLUDECRYPTINGCODE_IFCRYPTALLOWED + +#define RAND_HEAD_LEN 12 + /* "last resort" source for second part of crypt seed pattern */ +# ifndef ZCR_SEED2 +# define ZCR_SEED2 3141592654UL /* use PI as default pattern */ +# endif + +static unsigned crypthead(const char* passwd, /* password string */ + unsigned char* buf, /* where to write header */ + int bufSize, + unsigned long* pkeys, + const z_crc_t* pcrc_32_tab, + unsigned long crcForCrypting) +{ + unsigned n; /* index in random header */ + int t; /* temporary */ + int c; /* random byte */ + unsigned char header[RAND_HEAD_LEN-2]; /* random header */ + static unsigned calls = 0; /* ensure different random header each time */ + + if (bufSize> 7) & 0xff; + header[n] = (unsigned char)zencode(pkeys, pcrc_32_tab, c, t); + } + /* Encrypt random header (last two bytes is high word of crc) */ + init_keys(passwd, pkeys, pcrc_32_tab); + for (n = 0; n < RAND_HEAD_LEN-2; n++) + { + buf[n] = (unsigned char)zencode(pkeys, pcrc_32_tab, header[n], t); + } + buf[n++] = (unsigned char)zencode(pkeys, pcrc_32_tab, (int)(crcForCrypting >> 16) & 0xff, t); + buf[n++] = (unsigned char)zencode(pkeys, pcrc_32_tab, (int)(crcForCrypting >> 24) & 0xff, t); + return n; +} + +#endif diff --git a/lib/os/windows/zlib-1.2.13/contrib/minizip/ioapi.c b/lib/os/windows/zlib-1.2.13/contrib/minizip/ioapi.c new file mode 100644 index 000000000000..814a6fd38c26 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/contrib/minizip/ioapi.c @@ -0,0 +1,257 @@ +/* ioapi.h -- IO base function header for compress/uncompress .zip + part of the MiniZip project - ( http://www.winimage.com/zLibDll/minizip.html ) + + Copyright (C) 1998-2010 Gilles Vollant (minizip) ( http://www.winimage.com/zLibDll/minizip.html ) + + Modifications for Zip64 support + Copyright (C) 2009-2010 Mathias Svensson ( http://result42.com ) + + For more info read MiniZip_info.txt + +*/ + +#if defined(_WIN32) && (!(defined(_CRT_SECURE_NO_WARNINGS))) + #define _CRT_SECURE_NO_WARNINGS +#endif + +#if defined(__APPLE__) || defined(IOAPI_NO_64) +// In darwin and perhaps other BSD variants off_t is a 64 bit value, hence no need for specific 64 bit functions +#define FOPEN_FUNC(filename, mode) fopen(filename, mode) +#define FTELLO_FUNC(stream) ftello(stream) +#define FSEEKO_FUNC(stream, offset, origin) fseeko(stream, offset, origin) +#else +#define FOPEN_FUNC(filename, mode) fopen64(filename, mode) +#define FTELLO_FUNC(stream) ftello64(stream) +#define FSEEKO_FUNC(stream, offset, origin) fseeko64(stream, offset, origin) +#endif + + +#include "ioapi.h" + +voidpf call_zopen64 (const zlib_filefunc64_32_def* pfilefunc,const void*filename,int mode) +{ + if (pfilefunc->zfile_func64.zopen64_file != NULL) + return (*(pfilefunc->zfile_func64.zopen64_file)) (pfilefunc->zfile_func64.opaque,filename,mode); + else + { + return (*(pfilefunc->zopen32_file))(pfilefunc->zfile_func64.opaque,(const char*)filename,mode); + } +} + +long call_zseek64 (const zlib_filefunc64_32_def* pfilefunc,voidpf filestream, ZPOS64_T offset, int origin) +{ + if (pfilefunc->zfile_func64.zseek64_file != NULL) + return (*(pfilefunc->zfile_func64.zseek64_file)) (pfilefunc->zfile_func64.opaque,filestream,offset,origin); + else + { + uLong offsetTruncated = (uLong)offset; + if (offsetTruncated != offset) + return -1; + else + return (*(pfilefunc->zseek32_file))(pfilefunc->zfile_func64.opaque,filestream,offsetTruncated,origin); + } +} + +ZPOS64_T call_ztell64 (const zlib_filefunc64_32_def* pfilefunc,voidpf filestream) +{ + if (pfilefunc->zfile_func64.zseek64_file != NULL) + return (*(pfilefunc->zfile_func64.ztell64_file)) (pfilefunc->zfile_func64.opaque,filestream); + else + { + uLong tell_uLong = (uLong)(*(pfilefunc->ztell32_file))(pfilefunc->zfile_func64.opaque,filestream); + if ((tell_uLong) == MAXU32) + return (ZPOS64_T)-1; + else + return tell_uLong; + } +} + +void fill_zlib_filefunc64_32_def_from_filefunc32(zlib_filefunc64_32_def* p_filefunc64_32,const zlib_filefunc_def* p_filefunc32) +{ + p_filefunc64_32->zfile_func64.zopen64_file = NULL; + p_filefunc64_32->zopen32_file = p_filefunc32->zopen_file; + p_filefunc64_32->zfile_func64.zerror_file = p_filefunc32->zerror_file; + p_filefunc64_32->zfile_func64.zread_file = p_filefunc32->zread_file; + p_filefunc64_32->zfile_func64.zwrite_file = p_filefunc32->zwrite_file; + p_filefunc64_32->zfile_func64.ztell64_file = NULL; + p_filefunc64_32->zfile_func64.zseek64_file = NULL; + p_filefunc64_32->zfile_func64.zclose_file = p_filefunc32->zclose_file; + p_filefunc64_32->zfile_func64.zerror_file = p_filefunc32->zerror_file; + p_filefunc64_32->zfile_func64.opaque = p_filefunc32->opaque; + p_filefunc64_32->zseek32_file = p_filefunc32->zseek_file; + p_filefunc64_32->ztell32_file = p_filefunc32->ztell_file; +} + + + +static voidpf ZCALLBACK fopen_file_func OF((voidpf opaque, const char* filename, int mode)); +static uLong ZCALLBACK fread_file_func OF((voidpf opaque, voidpf stream, void* buf, uLong size)); +static uLong ZCALLBACK fwrite_file_func OF((voidpf opaque, voidpf stream, const void* buf,uLong size)); +static ZPOS64_T ZCALLBACK ftell64_file_func OF((voidpf opaque, voidpf stream)); +static long ZCALLBACK fseek64_file_func OF((voidpf opaque, voidpf stream, ZPOS64_T offset, int origin)); +static int ZCALLBACK fclose_file_func OF((voidpf opaque, voidpf stream)); +static int ZCALLBACK ferror_file_func OF((voidpf opaque, voidpf stream)); + +static voidpf ZCALLBACK fopen_file_func (voidpf opaque, const char* filename, int mode) +{ + FILE* file = NULL; + const char* mode_fopen = NULL; + (void)opaque; + if ((mode & ZLIB_FILEFUNC_MODE_READWRITEFILTER)==ZLIB_FILEFUNC_MODE_READ) + mode_fopen = "rb"; + else + if (mode & ZLIB_FILEFUNC_MODE_EXISTING) + mode_fopen = "r+b"; + else + if (mode & ZLIB_FILEFUNC_MODE_CREATE) + mode_fopen = "wb"; + + if ((filename!=NULL) && (mode_fopen != NULL)) + file = fopen(filename, mode_fopen); + return file; +} + +static voidpf ZCALLBACK fopen64_file_func (voidpf opaque, const void* filename, int mode) +{ + FILE* file = NULL; + const char* mode_fopen = NULL; + (void)opaque; + if ((mode & ZLIB_FILEFUNC_MODE_READWRITEFILTER)==ZLIB_FILEFUNC_MODE_READ) + mode_fopen = "rb"; + else + if (mode & ZLIB_FILEFUNC_MODE_EXISTING) + mode_fopen = "r+b"; + else + if (mode & ZLIB_FILEFUNC_MODE_CREATE) + mode_fopen = "wb"; + + if ((filename!=NULL) && (mode_fopen != NULL)) + file = FOPEN_FUNC((const char*)filename, mode_fopen); + return file; +} + + +static uLong ZCALLBACK fread_file_func (voidpf opaque, voidpf stream, void* buf, uLong size) +{ + uLong ret; + (void)opaque; + ret = (uLong)fread(buf, 1, (size_t)size, (FILE *)stream); + return ret; +} + +static uLong ZCALLBACK fwrite_file_func (voidpf opaque, voidpf stream, const void* buf, uLong size) +{ + uLong ret; + (void)opaque; + ret = (uLong)fwrite(buf, 1, (size_t)size, (FILE *)stream); + return ret; +} + +static long ZCALLBACK ftell_file_func (voidpf opaque, voidpf stream) +{ + long ret; + (void)opaque; + ret = ftell((FILE *)stream); + return ret; +} + + +static ZPOS64_T ZCALLBACK ftell64_file_func (voidpf opaque, voidpf stream) +{ + ZPOS64_T ret; + (void)opaque; + ret = (ZPOS64_T)FTELLO_FUNC((FILE *)stream); + return ret; +} + +static long ZCALLBACK fseek_file_func (voidpf opaque, voidpf stream, uLong offset, int origin) +{ + int fseek_origin=0; + long ret; + (void)opaque; + switch (origin) + { + case ZLIB_FILEFUNC_SEEK_CUR : + fseek_origin = SEEK_CUR; + break; + case ZLIB_FILEFUNC_SEEK_END : + fseek_origin = SEEK_END; + break; + case ZLIB_FILEFUNC_SEEK_SET : + fseek_origin = SEEK_SET; + break; + default: return -1; + } + ret = 0; + if (fseek((FILE *)stream, (long)offset, fseek_origin) != 0) + ret = -1; + return ret; +} + +static long ZCALLBACK fseek64_file_func (voidpf opaque, voidpf stream, ZPOS64_T offset, int origin) +{ + int fseek_origin=0; + long ret; + (void)opaque; + switch (origin) + { + case ZLIB_FILEFUNC_SEEK_CUR : + fseek_origin = SEEK_CUR; + break; + case ZLIB_FILEFUNC_SEEK_END : + fseek_origin = SEEK_END; + break; + case ZLIB_FILEFUNC_SEEK_SET : + fseek_origin = SEEK_SET; + break; + default: return -1; + } + ret = 0; + + if(FSEEKO_FUNC((FILE *)stream, (z_off_t)offset, fseek_origin) != 0) + ret = -1; + + return ret; +} + + +static int ZCALLBACK fclose_file_func (voidpf opaque, voidpf stream) +{ + int ret; + (void)opaque; + ret = fclose((FILE *)stream); + return ret; +} + +static int ZCALLBACK ferror_file_func (voidpf opaque, voidpf stream) +{ + int ret; + (void)opaque; + ret = ferror((FILE *)stream); + return ret; +} + +void fill_fopen_filefunc (pzlib_filefunc_def) + zlib_filefunc_def* pzlib_filefunc_def; +{ + pzlib_filefunc_def->zopen_file = fopen_file_func; + pzlib_filefunc_def->zread_file = fread_file_func; + pzlib_filefunc_def->zwrite_file = fwrite_file_func; + pzlib_filefunc_def->ztell_file = ftell_file_func; + pzlib_filefunc_def->zseek_file = fseek_file_func; + pzlib_filefunc_def->zclose_file = fclose_file_func; + pzlib_filefunc_def->zerror_file = ferror_file_func; + pzlib_filefunc_def->opaque = NULL; +} + +void fill_fopen64_filefunc (zlib_filefunc64_def* pzlib_filefunc_def) +{ + pzlib_filefunc_def->zopen64_file = fopen64_file_func; + pzlib_filefunc_def->zread_file = fread_file_func; + pzlib_filefunc_def->zwrite_file = fwrite_file_func; + pzlib_filefunc_def->ztell64_file = ftell64_file_func; + pzlib_filefunc_def->zseek64_file = fseek64_file_func; + pzlib_filefunc_def->zclose_file = fclose_file_func; + pzlib_filefunc_def->zerror_file = ferror_file_func; + pzlib_filefunc_def->opaque = NULL; +} diff --git a/lib/os/windows/zlib-1.2.13/contrib/minizip/ioapi.h b/lib/os/windows/zlib-1.2.13/contrib/minizip/ioapi.h new file mode 100644 index 000000000000..ae9ca7e8337d --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/contrib/minizip/ioapi.h @@ -0,0 +1,210 @@ +/* ioapi.h -- IO base function header for compress/uncompress .zip + part of the MiniZip project - ( http://www.winimage.com/zLibDll/minizip.html ) + + Copyright (C) 1998-2010 Gilles Vollant (minizip) ( http://www.winimage.com/zLibDll/minizip.html ) + + Modifications for Zip64 support + Copyright (C) 2009-2010 Mathias Svensson ( http://result42.com ) + + For more info read MiniZip_info.txt + + Changes + + Oct-2009 - Defined ZPOS64_T to fpos_t on windows and u_int64_t on linux. (might need to find a better why for this) + Oct-2009 - Change to fseeko64, ftello64 and fopen64 so large files would work on linux. + More if/def section may be needed to support other platforms + Oct-2009 - Defined fxxxx64 calls to normal fopen/ftell/fseek so they would compile on windows. + (but you should use iowin32.c for windows instead) + +*/ + +#ifndef _ZLIBIOAPI64_H +#define _ZLIBIOAPI64_H + +#if (!defined(_WIN32)) && (!defined(WIN32)) && (!defined(__APPLE__)) + + // Linux needs this to support file operation on files larger then 4+GB + // But might need better if/def to select just the platforms that needs them. + + #ifndef __USE_FILE_OFFSET64 + #define __USE_FILE_OFFSET64 + #endif + #ifndef __USE_LARGEFILE64 + #define __USE_LARGEFILE64 + #endif + #ifndef _LARGEFILE64_SOURCE + #define _LARGEFILE64_SOURCE + #endif + #ifndef _FILE_OFFSET_BIT + #define _FILE_OFFSET_BIT 64 + #endif + +#endif + +#include +#include +#include "zlib.h" + +#if defined(USE_FILE32API) +#define fopen64 fopen +#define ftello64 ftell +#define fseeko64 fseek +#else +#if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__NetBSD__) +#define fopen64 fopen +#define ftello64 ftello +#define fseeko64 fseeko +#endif +#ifdef _MSC_VER + #define fopen64 fopen + #if (_MSC_VER >= 1400) && (!(defined(NO_MSCVER_FILE64_FUNC))) + #define ftello64 _ftelli64 + #define fseeko64 _fseeki64 + #else // old MSC + #define ftello64 ftell + #define fseeko64 fseek + #endif +#endif +#endif + +/* +#ifndef ZPOS64_T + #ifdef _WIN32 + #define ZPOS64_T fpos_t + #else + #include + #define ZPOS64_T uint64_t + #endif +#endif +*/ + +#ifdef HAVE_MINIZIP64_CONF_H +#include "mz64conf.h" +#endif + +/* a type choosen by DEFINE */ +#ifdef HAVE_64BIT_INT_CUSTOM +typedef 64BIT_INT_CUSTOM_TYPE ZPOS64_T; +#else +#ifdef HAS_STDINT_H +#include "stdint.h" +typedef uint64_t ZPOS64_T; +#else + + + +#if defined(_MSC_VER) || defined(__BORLANDC__) +typedef unsigned __int64 ZPOS64_T; +#else +typedef unsigned long long int ZPOS64_T; +#endif +#endif +#endif + +/* Maximum unsigned 32-bit value used as placeholder for zip64 */ +#ifndef MAXU32 +#define MAXU32 (0xffffffff) +#endif + +#ifdef __cplusplus +extern "C" { +#endif + + +#define ZLIB_FILEFUNC_SEEK_CUR (1) +#define ZLIB_FILEFUNC_SEEK_END (2) +#define ZLIB_FILEFUNC_SEEK_SET (0) + +#define ZLIB_FILEFUNC_MODE_READ (1) +#define ZLIB_FILEFUNC_MODE_WRITE (2) +#define ZLIB_FILEFUNC_MODE_READWRITEFILTER (3) + +#define ZLIB_FILEFUNC_MODE_EXISTING (4) +#define ZLIB_FILEFUNC_MODE_CREATE (8) + + +#ifndef ZCALLBACK + #if (defined(WIN32) || defined(_WIN32) || defined (WINDOWS) || defined (_WINDOWS)) && defined(CALLBACK) && defined (USEWINDOWS_CALLBACK) + #define ZCALLBACK CALLBACK + #else + #define ZCALLBACK + #endif +#endif + + + + +typedef voidpf (ZCALLBACK *open_file_func) OF((voidpf opaque, const char* filename, int mode)); +typedef uLong (ZCALLBACK *read_file_func) OF((voidpf opaque, voidpf stream, void* buf, uLong size)); +typedef uLong (ZCALLBACK *write_file_func) OF((voidpf opaque, voidpf stream, const void* buf, uLong size)); +typedef int (ZCALLBACK *close_file_func) OF((voidpf opaque, voidpf stream)); +typedef int (ZCALLBACK *testerror_file_func) OF((voidpf opaque, voidpf stream)); + +typedef long (ZCALLBACK *tell_file_func) OF((voidpf opaque, voidpf stream)); +typedef long (ZCALLBACK *seek_file_func) OF((voidpf opaque, voidpf stream, uLong offset, int origin)); + + +/* here is the "old" 32 bits structure structure */ +typedef struct zlib_filefunc_def_s +{ + open_file_func zopen_file; + read_file_func zread_file; + write_file_func zwrite_file; + tell_file_func ztell_file; + seek_file_func zseek_file; + close_file_func zclose_file; + testerror_file_func zerror_file; + voidpf opaque; +} zlib_filefunc_def; + +typedef ZPOS64_T (ZCALLBACK *tell64_file_func) OF((voidpf opaque, voidpf stream)); +typedef long (ZCALLBACK *seek64_file_func) OF((voidpf opaque, voidpf stream, ZPOS64_T offset, int origin)); +typedef voidpf (ZCALLBACK *open64_file_func) OF((voidpf opaque, const void* filename, int mode)); + +typedef struct zlib_filefunc64_def_s +{ + open64_file_func zopen64_file; + read_file_func zread_file; + write_file_func zwrite_file; + tell64_file_func ztell64_file; + seek64_file_func zseek64_file; + close_file_func zclose_file; + testerror_file_func zerror_file; + voidpf opaque; +} zlib_filefunc64_def; + +void fill_fopen64_filefunc OF((zlib_filefunc64_def* pzlib_filefunc_def)); +void fill_fopen_filefunc OF((zlib_filefunc_def* pzlib_filefunc_def)); + +/* now internal definition, only for zip.c and unzip.h */ +typedef struct zlib_filefunc64_32_def_s +{ + zlib_filefunc64_def zfile_func64; + open_file_func zopen32_file; + tell_file_func ztell32_file; + seek_file_func zseek32_file; +} zlib_filefunc64_32_def; + + +#define ZREAD64(filefunc,filestream,buf,size) ((*((filefunc).zfile_func64.zread_file)) ((filefunc).zfile_func64.opaque,filestream,buf,size)) +#define ZWRITE64(filefunc,filestream,buf,size) ((*((filefunc).zfile_func64.zwrite_file)) ((filefunc).zfile_func64.opaque,filestream,buf,size)) +//#define ZTELL64(filefunc,filestream) ((*((filefunc).ztell64_file)) ((filefunc).opaque,filestream)) +//#define ZSEEK64(filefunc,filestream,pos,mode) ((*((filefunc).zseek64_file)) ((filefunc).opaque,filestream,pos,mode)) +#define ZCLOSE64(filefunc,filestream) ((*((filefunc).zfile_func64.zclose_file)) ((filefunc).zfile_func64.opaque,filestream)) +#define ZERROR64(filefunc,filestream) ((*((filefunc).zfile_func64.zerror_file)) ((filefunc).zfile_func64.opaque,filestream)) + +voidpf call_zopen64 OF((const zlib_filefunc64_32_def* pfilefunc,const void*filename,int mode)); +long call_zseek64 OF((const zlib_filefunc64_32_def* pfilefunc,voidpf filestream, ZPOS64_T offset, int origin)); +ZPOS64_T call_ztell64 OF((const zlib_filefunc64_32_def* pfilefunc,voidpf filestream)); + +void fill_zlib_filefunc64_32_def_from_filefunc32(zlib_filefunc64_32_def* p_filefunc64_32,const zlib_filefunc_def* p_filefunc32); + +#define ZOPEN64(filefunc,filename,mode) (call_zopen64((&(filefunc)),(filename),(mode))) +#define ZTELL64(filefunc,filestream) (call_ztell64((&(filefunc)),(filestream))) +#define ZSEEK64(filefunc,filestream,pos,mode) (call_zseek64((&(filefunc)),(filestream),(pos),(mode))) + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/lib/os/windows/zlib-1.2.13/contrib/minizip/unzip.c b/lib/os/windows/zlib-1.2.13/contrib/minizip/unzip.c new file mode 100644 index 000000000000..3036b470b72b --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/contrib/minizip/unzip.c @@ -0,0 +1,2130 @@ +/* unzip.c -- IO for uncompress .zip files using zlib + Version 1.1, February 14h, 2010 + part of the MiniZip project - ( http://www.winimage.com/zLibDll/minizip.html ) + + Copyright (C) 1998-2010 Gilles Vollant (minizip) ( http://www.winimage.com/zLibDll/minizip.html ) + + Modifications of Unzip for Zip64 + Copyright (C) 2007-2008 Even Rouault + + Modifications for Zip64 support on both zip and unzip + Copyright (C) 2009-2010 Mathias Svensson ( http://result42.com ) + + For more info read MiniZip_info.txt + + + ------------------------------------------------------------------------------------ + Decryption code comes from crypt.c by Info-ZIP but has been greatly reduced in terms of + compatibility with older software. The following is from the original crypt.c. + Code woven in by Terry Thorsen 1/2003. + + Copyright (c) 1990-2000 Info-ZIP. All rights reserved. + + See the accompanying file LICENSE, version 2000-Apr-09 or later + (the contents of which are also included in zip.h) for terms of use. + If, for some reason, all these files are missing, the Info-ZIP license + also may be found at: ftp://ftp.info-zip.org/pub/infozip/license.html + + crypt.c (full version) by Info-ZIP. Last revised: [see crypt.h] + + The encryption/decryption parts of this source code (as opposed to the + non-echoing password parts) were originally written in Europe. The + whole source package can be freely distributed, including from the USA. + (Prior to January 2000, re-export from the US was a violation of US law.) + + This encryption code is a direct transcription of the algorithm from + Roger Schlafly, described by Phil Katz in the file appnote.txt. This + file (appnote.txt) is distributed with the PKZIP program (even in the + version without encryption capabilities). + + ------------------------------------------------------------------------------------ + + Changes in unzip.c + + 2007-2008 - Even Rouault - Addition of cpl_unzGetCurrentFileZStreamPos + 2007-2008 - Even Rouault - Decoration of symbol names unz* -> cpl_unz* + 2007-2008 - Even Rouault - Remove old C style function prototypes + 2007-2008 - Even Rouault - Add unzip support for ZIP64 + + Copyright (C) 2007-2008 Even Rouault + + + Oct-2009 - Mathias Svensson - Removed cpl_* from symbol names (Even Rouault added them but since this is now moved to a new project (minizip64) I renamed them again). + Oct-2009 - Mathias Svensson - Fixed problem if uncompressed size was > 4G and compressed size was <4G + should only read the compressed/uncompressed size from the Zip64 format if + the size from normal header was 0xFFFFFFFF + Oct-2009 - Mathias Svensson - Applied some bug fixes from paches recived from Gilles Vollant + Oct-2009 - Mathias Svensson - Applied support to unzip files with compression mathod BZIP2 (bzip2 lib is required) + Patch created by Daniel Borca + + Jan-2010 - back to unzip and minizip 1.0 name scheme, with compatibility layer + + Copyright (C) 1998 - 2010 Gilles Vollant, Even Rouault, Mathias Svensson + +*/ + + +#include +#include +#include + +#ifndef NOUNCRYPT + #define NOUNCRYPT +#endif + +#include "zlib.h" +#include "unzip.h" + +#ifdef STDC +# include +# include +# include +#endif +#ifdef NO_ERRNO_H + extern int errno; +#else +# include +#endif + + +#ifndef local +# define local static +#endif +/* compile with -Dlocal if your debugger can't find static symbols */ + + +#ifndef CASESENSITIVITYDEFAULT_NO +# if !defined(unix) && !defined(CASESENSITIVITYDEFAULT_YES) +# define CASESENSITIVITYDEFAULT_NO +# endif +#endif + + +#ifndef UNZ_BUFSIZE +#define UNZ_BUFSIZE (16384) +#endif + +#ifndef UNZ_MAXFILENAMEINZIP +#define UNZ_MAXFILENAMEINZIP (256) +#endif + +#ifndef ALLOC +# define ALLOC(size) (malloc(size)) +#endif +#ifndef TRYFREE +# define TRYFREE(p) { free(p);} +#endif + +#define SIZECENTRALDIRITEM (0x2e) +#define SIZEZIPLOCALHEADER (0x1e) + + +const char unz_copyright[] = + " unzip 1.01 Copyright 1998-2004 Gilles Vollant - http://www.winimage.com/zLibDll"; + +/* unz_file_info_interntal contain internal info about a file in zipfile*/ +typedef struct unz_file_info64_internal_s +{ + ZPOS64_T offset_curfile;/* relative offset of local header 8 bytes */ +} unz_file_info64_internal; + + +/* file_in_zip_read_info_s contain internal information about a file in zipfile, + when reading and decompress it */ +typedef struct +{ + char *read_buffer; /* internal buffer for compressed data */ + z_stream stream; /* zLib stream structure for inflate */ + +#ifdef HAVE_BZIP2 + bz_stream bstream; /* bzLib stream structure for bziped */ +#endif + + ZPOS64_T pos_in_zipfile; /* position in byte on the zipfile, for fseek*/ + uLong stream_initialised; /* flag set if stream structure is initialised*/ + + ZPOS64_T offset_local_extrafield;/* offset of the local extra field */ + uInt size_local_extrafield;/* size of the local extra field */ + ZPOS64_T pos_local_extrafield; /* position in the local extra field in read*/ + ZPOS64_T total_out_64; + + uLong crc32; /* crc32 of all data uncompressed */ + uLong crc32_wait; /* crc32 we must obtain after decompress all */ + ZPOS64_T rest_read_compressed; /* number of byte to be decompressed */ + ZPOS64_T rest_read_uncompressed;/*number of byte to be obtained after decomp*/ + zlib_filefunc64_32_def z_filefunc; + voidpf filestream; /* io structore of the zipfile */ + uLong compression_method; /* compression method (0==store) */ + ZPOS64_T byte_before_the_zipfile;/* byte before the zipfile, (>0 for sfx)*/ + int raw; +} file_in_zip64_read_info_s; + + +/* unz64_s contain internal information about the zipfile +*/ +typedef struct +{ + zlib_filefunc64_32_def z_filefunc; + int is64bitOpenFunction; + voidpf filestream; /* io structore of the zipfile */ + unz_global_info64 gi; /* public global information */ + ZPOS64_T byte_before_the_zipfile;/* byte before the zipfile, (>0 for sfx)*/ + ZPOS64_T num_file; /* number of the current file in the zipfile*/ + ZPOS64_T pos_in_central_dir; /* pos of the current file in the central dir*/ + ZPOS64_T current_file_ok; /* flag about the usability of the current file*/ + ZPOS64_T central_pos; /* position of the beginning of the central dir*/ + + ZPOS64_T size_central_dir; /* size of the central directory */ + ZPOS64_T offset_central_dir; /* offset of start of central directory with + respect to the starting disk number */ + + unz_file_info64 cur_file_info; /* public info about the current file in zip*/ + unz_file_info64_internal cur_file_info_internal; /* private info about it*/ + file_in_zip64_read_info_s* pfile_in_zip_read; /* structure about the current + file if we are decompressing it */ + int encrypted; + + int isZip64; + +# ifndef NOUNCRYPT + unsigned long keys[3]; /* keys defining the pseudo-random sequence */ + const z_crc_t* pcrc_32_tab; +# endif +} unz64_s; + + +#ifndef NOUNCRYPT +#include "crypt.h" +#endif + +/* =========================================================================== + Read a byte from a gz_stream; update next_in and avail_in. Return EOF + for end of file. + IN assertion: the stream s has been successfully opened for reading. +*/ + + +local int unz64local_getByte OF(( + const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream, + int *pi)); + +local int unz64local_getByte(const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, int *pi) +{ + unsigned char c; + int err = (int)ZREAD64(*pzlib_filefunc_def,filestream,&c,1); + if (err==1) + { + *pi = (int)c; + return UNZ_OK; + } + else + { + if (ZERROR64(*pzlib_filefunc_def,filestream)) + return UNZ_ERRNO; + else + return UNZ_EOF; + } +} + + +/* =========================================================================== + Reads a long in LSB order from the given gz_stream. Sets +*/ +local int unz64local_getShort OF(( + const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream, + uLong *pX)); + +local int unz64local_getShort (const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream, + uLong *pX) +{ + uLong x ; + int i = 0; + int err; + + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x = (uLong)i; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((uLong)i)<<8; + + if (err==UNZ_OK) + *pX = x; + else + *pX = 0; + return err; +} + +local int unz64local_getLong OF(( + const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream, + uLong *pX)); + +local int unz64local_getLong (const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream, + uLong *pX) +{ + uLong x ; + int i = 0; + int err; + + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x = (uLong)i; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((uLong)i)<<8; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((uLong)i)<<16; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((uLong)i)<<24; + + if (err==UNZ_OK) + *pX = x; + else + *pX = 0; + return err; +} + +local int unz64local_getLong64 OF(( + const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream, + ZPOS64_T *pX)); + + +local int unz64local_getLong64 (const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream, + ZPOS64_T *pX) +{ + ZPOS64_T x ; + int i = 0; + int err; + + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x = (ZPOS64_T)i; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((ZPOS64_T)i)<<8; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((ZPOS64_T)i)<<16; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((ZPOS64_T)i)<<24; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((ZPOS64_T)i)<<32; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((ZPOS64_T)i)<<40; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((ZPOS64_T)i)<<48; + + if (err==UNZ_OK) + err = unz64local_getByte(pzlib_filefunc_def,filestream,&i); + x |= ((ZPOS64_T)i)<<56; + + if (err==UNZ_OK) + *pX = x; + else + *pX = 0; + return err; +} + +/* My own strcmpi / strcasecmp */ +local int strcmpcasenosensitive_internal (const char* fileName1, const char* fileName2) +{ + for (;;) + { + char c1=*(fileName1++); + char c2=*(fileName2++); + if ((c1>='a') && (c1<='z')) + c1 -= 0x20; + if ((c2>='a') && (c2<='z')) + c2 -= 0x20; + if (c1=='\0') + return ((c2=='\0') ? 0 : -1); + if (c2=='\0') + return 1; + if (c1c2) + return 1; + } +} + + +#ifdef CASESENSITIVITYDEFAULT_NO +#define CASESENSITIVITYDEFAULTVALUE 2 +#else +#define CASESENSITIVITYDEFAULTVALUE 1 +#endif + +#ifndef STRCMPCASENOSENTIVEFUNCTION +#define STRCMPCASENOSENTIVEFUNCTION strcmpcasenosensitive_internal +#endif + +/* + Compare two filename (fileName1,fileName2). + If iCaseSenisivity = 1, comparision is case sensitivity (like strcmp) + If iCaseSenisivity = 2, comparision is not case sensitivity (like strcmpi + or strcasecmp) + If iCaseSenisivity = 0, case sensitivity is defaut of your operating system + (like 1 on Unix, 2 on Windows) + +*/ +extern int ZEXPORT unzStringFileNameCompare (const char* fileName1, + const char* fileName2, + int iCaseSensitivity) + +{ + if (iCaseSensitivity==0) + iCaseSensitivity=CASESENSITIVITYDEFAULTVALUE; + + if (iCaseSensitivity==1) + return strcmp(fileName1,fileName2); + + return STRCMPCASENOSENTIVEFUNCTION(fileName1,fileName2); +} + +#ifndef BUFREADCOMMENT +#define BUFREADCOMMENT (0x400) +#endif + +/* + Locate the Central directory of a zipfile (at the end, just before + the global comment) +*/ +local ZPOS64_T unz64local_SearchCentralDir OF((const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream)); +local ZPOS64_T unz64local_SearchCentralDir(const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream) +{ + unsigned char* buf; + ZPOS64_T uSizeFile; + ZPOS64_T uBackRead; + ZPOS64_T uMaxBack=0xffff; /* maximum size of global comment */ + ZPOS64_T uPosFound=0; + + if (ZSEEK64(*pzlib_filefunc_def,filestream,0,ZLIB_FILEFUNC_SEEK_END) != 0) + return 0; + + + uSizeFile = ZTELL64(*pzlib_filefunc_def,filestream); + + if (uMaxBack>uSizeFile) + uMaxBack = uSizeFile; + + buf = (unsigned char*)ALLOC(BUFREADCOMMENT+4); + if (buf==NULL) + return 0; + + uBackRead = 4; + while (uBackReaduMaxBack) + uBackRead = uMaxBack; + else + uBackRead+=BUFREADCOMMENT; + uReadPos = uSizeFile-uBackRead ; + + uReadSize = ((BUFREADCOMMENT+4) < (uSizeFile-uReadPos)) ? + (BUFREADCOMMENT+4) : (uLong)(uSizeFile-uReadPos); + if (ZSEEK64(*pzlib_filefunc_def,filestream,uReadPos,ZLIB_FILEFUNC_SEEK_SET)!=0) + break; + + if (ZREAD64(*pzlib_filefunc_def,filestream,buf,uReadSize)!=uReadSize) + break; + + for (i=(int)uReadSize-3; (i--)>0;) + if (((*(buf+i))==0x50) && ((*(buf+i+1))==0x4b) && + ((*(buf+i+2))==0x05) && ((*(buf+i+3))==0x06)) + { + uPosFound = uReadPos+(unsigned)i; + break; + } + + if (uPosFound!=0) + break; + } + TRYFREE(buf); + return uPosFound; +} + + +/* + Locate the Central directory 64 of a zipfile (at the end, just before + the global comment) +*/ +local ZPOS64_T unz64local_SearchCentralDir64 OF(( + const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream)); + +local ZPOS64_T unz64local_SearchCentralDir64(const zlib_filefunc64_32_def* pzlib_filefunc_def, + voidpf filestream) +{ + unsigned char* buf; + ZPOS64_T uSizeFile; + ZPOS64_T uBackRead; + ZPOS64_T uMaxBack=0xffff; /* maximum size of global comment */ + ZPOS64_T uPosFound=0; + uLong uL; + ZPOS64_T relativeOffset; + + if (ZSEEK64(*pzlib_filefunc_def,filestream,0,ZLIB_FILEFUNC_SEEK_END) != 0) + return 0; + + + uSizeFile = ZTELL64(*pzlib_filefunc_def,filestream); + + if (uMaxBack>uSizeFile) + uMaxBack = uSizeFile; + + buf = (unsigned char*)ALLOC(BUFREADCOMMENT+4); + if (buf==NULL) + return 0; + + uBackRead = 4; + while (uBackReaduMaxBack) + uBackRead = uMaxBack; + else + uBackRead+=BUFREADCOMMENT; + uReadPos = uSizeFile-uBackRead ; + + uReadSize = ((BUFREADCOMMENT+4) < (uSizeFile-uReadPos)) ? + (BUFREADCOMMENT+4) : (uLong)(uSizeFile-uReadPos); + if (ZSEEK64(*pzlib_filefunc_def,filestream,uReadPos,ZLIB_FILEFUNC_SEEK_SET)!=0) + break; + + if (ZREAD64(*pzlib_filefunc_def,filestream,buf,uReadSize)!=uReadSize) + break; + + for (i=(int)uReadSize-3; (i--)>0;) + if (((*(buf+i))==0x50) && ((*(buf+i+1))==0x4b) && + ((*(buf+i+2))==0x06) && ((*(buf+i+3))==0x07)) + { + uPosFound = uReadPos+(unsigned)i; + break; + } + + if (uPosFound!=0) + break; + } + TRYFREE(buf); + if (uPosFound == 0) + return 0; + + /* Zip64 end of central directory locator */ + if (ZSEEK64(*pzlib_filefunc_def,filestream, uPosFound,ZLIB_FILEFUNC_SEEK_SET)!=0) + return 0; + + /* the signature, already checked */ + if (unz64local_getLong(pzlib_filefunc_def,filestream,&uL)!=UNZ_OK) + return 0; + + /* number of the disk with the start of the zip64 end of central directory */ + if (unz64local_getLong(pzlib_filefunc_def,filestream,&uL)!=UNZ_OK) + return 0; + if (uL != 0) + return 0; + + /* relative offset of the zip64 end of central directory record */ + if (unz64local_getLong64(pzlib_filefunc_def,filestream,&relativeOffset)!=UNZ_OK) + return 0; + + /* total number of disks */ + if (unz64local_getLong(pzlib_filefunc_def,filestream,&uL)!=UNZ_OK) + return 0; + if (uL != 1) + return 0; + + /* Goto end of central directory record */ + if (ZSEEK64(*pzlib_filefunc_def,filestream, relativeOffset,ZLIB_FILEFUNC_SEEK_SET)!=0) + return 0; + + /* the signature */ + if (unz64local_getLong(pzlib_filefunc_def,filestream,&uL)!=UNZ_OK) + return 0; + + if (uL != 0x06064b50) + return 0; + + return relativeOffset; +} + +/* + Open a Zip file. path contain the full pathname (by example, + on a Windows NT computer "c:\\test\\zlib114.zip" or on an Unix computer + "zlib/zlib114.zip". + If the zipfile cannot be opened (file doesn't exist or in not valid), the + return value is NULL. + Else, the return value is a unzFile Handle, usable with other function + of this unzip package. +*/ +local unzFile unzOpenInternal (const void *path, + zlib_filefunc64_32_def* pzlib_filefunc64_32_def, + int is64bitOpenFunction) +{ + unz64_s us; + unz64_s *s; + ZPOS64_T central_pos; + uLong uL; + + uLong number_disk; /* number of the current dist, used for + spaning ZIP, unsupported, always 0*/ + uLong number_disk_with_CD; /* number the the disk with central dir, used + for spaning ZIP, unsupported, always 0*/ + ZPOS64_T number_entry_CD; /* total number of entries in + the central dir + (same than number_entry on nospan) */ + + int err=UNZ_OK; + + if (unz_copyright[0]!=' ') + return NULL; + + us.z_filefunc.zseek32_file = NULL; + us.z_filefunc.ztell32_file = NULL; + if (pzlib_filefunc64_32_def==NULL) + fill_fopen64_filefunc(&us.z_filefunc.zfile_func64); + else + us.z_filefunc = *pzlib_filefunc64_32_def; + us.is64bitOpenFunction = is64bitOpenFunction; + + + + us.filestream = ZOPEN64(us.z_filefunc, + path, + ZLIB_FILEFUNC_MODE_READ | + ZLIB_FILEFUNC_MODE_EXISTING); + if (us.filestream==NULL) + return NULL; + + central_pos = unz64local_SearchCentralDir64(&us.z_filefunc,us.filestream); + if (central_pos) + { + uLong uS; + ZPOS64_T uL64; + + us.isZip64 = 1; + + if (ZSEEK64(us.z_filefunc, us.filestream, + central_pos,ZLIB_FILEFUNC_SEEK_SET)!=0) + err=UNZ_ERRNO; + + /* the signature, already checked */ + if (unz64local_getLong(&us.z_filefunc, us.filestream,&uL)!=UNZ_OK) + err=UNZ_ERRNO; + + /* size of zip64 end of central directory record */ + if (unz64local_getLong64(&us.z_filefunc, us.filestream,&uL64)!=UNZ_OK) + err=UNZ_ERRNO; + + /* version made by */ + if (unz64local_getShort(&us.z_filefunc, us.filestream,&uS)!=UNZ_OK) + err=UNZ_ERRNO; + + /* version needed to extract */ + if (unz64local_getShort(&us.z_filefunc, us.filestream,&uS)!=UNZ_OK) + err=UNZ_ERRNO; + + /* number of this disk */ + if (unz64local_getLong(&us.z_filefunc, us.filestream,&number_disk)!=UNZ_OK) + err=UNZ_ERRNO; + + /* number of the disk with the start of the central directory */ + if (unz64local_getLong(&us.z_filefunc, us.filestream,&number_disk_with_CD)!=UNZ_OK) + err=UNZ_ERRNO; + + /* total number of entries in the central directory on this disk */ + if (unz64local_getLong64(&us.z_filefunc, us.filestream,&us.gi.number_entry)!=UNZ_OK) + err=UNZ_ERRNO; + + /* total number of entries in the central directory */ + if (unz64local_getLong64(&us.z_filefunc, us.filestream,&number_entry_CD)!=UNZ_OK) + err=UNZ_ERRNO; + + if ((number_entry_CD!=us.gi.number_entry) || + (number_disk_with_CD!=0) || + (number_disk!=0)) + err=UNZ_BADZIPFILE; + + /* size of the central directory */ + if (unz64local_getLong64(&us.z_filefunc, us.filestream,&us.size_central_dir)!=UNZ_OK) + err=UNZ_ERRNO; + + /* offset of start of central directory with respect to the + starting disk number */ + if (unz64local_getLong64(&us.z_filefunc, us.filestream,&us.offset_central_dir)!=UNZ_OK) + err=UNZ_ERRNO; + + us.gi.size_comment = 0; + } + else + { + central_pos = unz64local_SearchCentralDir(&us.z_filefunc,us.filestream); + if (central_pos==0) + err=UNZ_ERRNO; + + us.isZip64 = 0; + + if (ZSEEK64(us.z_filefunc, us.filestream, + central_pos,ZLIB_FILEFUNC_SEEK_SET)!=0) + err=UNZ_ERRNO; + + /* the signature, already checked */ + if (unz64local_getLong(&us.z_filefunc, us.filestream,&uL)!=UNZ_OK) + err=UNZ_ERRNO; + + /* number of this disk */ + if (unz64local_getShort(&us.z_filefunc, us.filestream,&number_disk)!=UNZ_OK) + err=UNZ_ERRNO; + + /* number of the disk with the start of the central directory */ + if (unz64local_getShort(&us.z_filefunc, us.filestream,&number_disk_with_CD)!=UNZ_OK) + err=UNZ_ERRNO; + + /* total number of entries in the central dir on this disk */ + if (unz64local_getShort(&us.z_filefunc, us.filestream,&uL)!=UNZ_OK) + err=UNZ_ERRNO; + us.gi.number_entry = uL; + + /* total number of entries in the central dir */ + if (unz64local_getShort(&us.z_filefunc, us.filestream,&uL)!=UNZ_OK) + err=UNZ_ERRNO; + number_entry_CD = uL; + + if ((number_entry_CD!=us.gi.number_entry) || + (number_disk_with_CD!=0) || + (number_disk!=0)) + err=UNZ_BADZIPFILE; + + /* size of the central directory */ + if (unz64local_getLong(&us.z_filefunc, us.filestream,&uL)!=UNZ_OK) + err=UNZ_ERRNO; + us.size_central_dir = uL; + + /* offset of start of central directory with respect to the + starting disk number */ + if (unz64local_getLong(&us.z_filefunc, us.filestream,&uL)!=UNZ_OK) + err=UNZ_ERRNO; + us.offset_central_dir = uL; + + /* zipfile comment length */ + if (unz64local_getShort(&us.z_filefunc, us.filestream,&us.gi.size_comment)!=UNZ_OK) + err=UNZ_ERRNO; + } + + if ((central_pospfile_in_zip_read!=NULL) + unzCloseCurrentFile(file); + + ZCLOSE64(s->z_filefunc, s->filestream); + TRYFREE(s); + return UNZ_OK; +} + + +/* + Write info about the ZipFile in the *pglobal_info structure. + No preparation of the structure is needed + return UNZ_OK if there is no problem. */ +extern int ZEXPORT unzGetGlobalInfo64 (unzFile file, unz_global_info64* pglobal_info) +{ + unz64_s* s; + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + *pglobal_info=s->gi; + return UNZ_OK; +} + +extern int ZEXPORT unzGetGlobalInfo (unzFile file, unz_global_info* pglobal_info32) +{ + unz64_s* s; + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + /* to do : check if number_entry is not truncated */ + pglobal_info32->number_entry = (uLong)s->gi.number_entry; + pglobal_info32->size_comment = s->gi.size_comment; + return UNZ_OK; +} +/* + Translate date/time from Dos format to tm_unz (readable more easilty) +*/ +local void unz64local_DosDateToTmuDate (ZPOS64_T ulDosDate, tm_unz* ptm) +{ + ZPOS64_T uDate; + uDate = (ZPOS64_T)(ulDosDate>>16); + ptm->tm_mday = (int)(uDate&0x1f) ; + ptm->tm_mon = (int)((((uDate)&0x1E0)/0x20)-1) ; + ptm->tm_year = (int)(((uDate&0x0FE00)/0x0200)+1980) ; + + ptm->tm_hour = (int) ((ulDosDate &0xF800)/0x800); + ptm->tm_min = (int) ((ulDosDate&0x7E0)/0x20) ; + ptm->tm_sec = (int) (2*(ulDosDate&0x1f)) ; +} + +/* + Get Info about the current file in the zipfile, with internal only info +*/ +local int unz64local_GetCurrentFileInfoInternal OF((unzFile file, + unz_file_info64 *pfile_info, + unz_file_info64_internal + *pfile_info_internal, + char *szFileName, + uLong fileNameBufferSize, + void *extraField, + uLong extraFieldBufferSize, + char *szComment, + uLong commentBufferSize)); + +local int unz64local_GetCurrentFileInfoInternal (unzFile file, + unz_file_info64 *pfile_info, + unz_file_info64_internal + *pfile_info_internal, + char *szFileName, + uLong fileNameBufferSize, + void *extraField, + uLong extraFieldBufferSize, + char *szComment, + uLong commentBufferSize) +{ + unz64_s* s; + unz_file_info64 file_info; + unz_file_info64_internal file_info_internal; + int err=UNZ_OK; + uLong uMagic; + long lSeek=0; + uLong uL; + + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + if (ZSEEK64(s->z_filefunc, s->filestream, + s->pos_in_central_dir+s->byte_before_the_zipfile, + ZLIB_FILEFUNC_SEEK_SET)!=0) + err=UNZ_ERRNO; + + + /* we check the magic */ + if (err==UNZ_OK) + { + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uMagic) != UNZ_OK) + err=UNZ_ERRNO; + else if (uMagic!=0x02014b50) + err=UNZ_BADZIPFILE; + } + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.version) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.version_needed) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.flag) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.compression_method) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&file_info.dosDate) != UNZ_OK) + err=UNZ_ERRNO; + + unz64local_DosDateToTmuDate(file_info.dosDate,&file_info.tmu_date); + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&file_info.crc) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uL) != UNZ_OK) + err=UNZ_ERRNO; + file_info.compressed_size = uL; + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uL) != UNZ_OK) + err=UNZ_ERRNO; + file_info.uncompressed_size = uL; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.size_filename) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.size_file_extra) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.size_file_comment) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.disk_num_start) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&file_info.internal_fa) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&file_info.external_fa) != UNZ_OK) + err=UNZ_ERRNO; + + // relative offset of local header + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uL) != UNZ_OK) + err=UNZ_ERRNO; + file_info_internal.offset_curfile = uL; + + lSeek+=file_info.size_filename; + if ((err==UNZ_OK) && (szFileName!=NULL)) + { + uLong uSizeRead ; + if (file_info.size_filename0) && (fileNameBufferSize>0)) + if (ZREAD64(s->z_filefunc, s->filestream,szFileName,uSizeRead)!=uSizeRead) + err=UNZ_ERRNO; + lSeek -= uSizeRead; + } + + // Read extrafield + if ((err==UNZ_OK) && (extraField!=NULL)) + { + ZPOS64_T uSizeRead ; + if (file_info.size_file_extraz_filefunc, s->filestream,(ZPOS64_T)lSeek,ZLIB_FILEFUNC_SEEK_CUR)==0) + lSeek=0; + else + err=UNZ_ERRNO; + } + + if ((file_info.size_file_extra>0) && (extraFieldBufferSize>0)) + if (ZREAD64(s->z_filefunc, s->filestream,extraField,(uLong)uSizeRead)!=uSizeRead) + err=UNZ_ERRNO; + + lSeek += file_info.size_file_extra - (uLong)uSizeRead; + } + else + lSeek += file_info.size_file_extra; + + + if ((err==UNZ_OK) && (file_info.size_file_extra != 0)) + { + uLong acc = 0; + + // since lSeek now points to after the extra field we need to move back + lSeek -= file_info.size_file_extra; + + if (lSeek!=0) + { + if (ZSEEK64(s->z_filefunc, s->filestream,(ZPOS64_T)lSeek,ZLIB_FILEFUNC_SEEK_CUR)==0) + lSeek=0; + else + err=UNZ_ERRNO; + } + + while(acc < file_info.size_file_extra) + { + uLong headerId; + uLong dataSize; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&headerId) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&dataSize) != UNZ_OK) + err=UNZ_ERRNO; + + /* ZIP64 extra fields */ + if (headerId == 0x0001) + { + uLong uL; + + if(file_info.uncompressed_size == MAXU32) + { + if (unz64local_getLong64(&s->z_filefunc, s->filestream,&file_info.uncompressed_size) != UNZ_OK) + err=UNZ_ERRNO; + } + + if(file_info.compressed_size == MAXU32) + { + if (unz64local_getLong64(&s->z_filefunc, s->filestream,&file_info.compressed_size) != UNZ_OK) + err=UNZ_ERRNO; + } + + if(file_info_internal.offset_curfile == MAXU32) + { + /* Relative Header offset */ + if (unz64local_getLong64(&s->z_filefunc, s->filestream,&file_info_internal.offset_curfile) != UNZ_OK) + err=UNZ_ERRNO; + } + + if(file_info.disk_num_start == MAXU32) + { + /* Disk Start Number */ + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uL) != UNZ_OK) + err=UNZ_ERRNO; + } + + } + else + { + if (ZSEEK64(s->z_filefunc, s->filestream,dataSize,ZLIB_FILEFUNC_SEEK_CUR)!=0) + err=UNZ_ERRNO; + } + + acc += 2 + 2 + dataSize; + } + } + + if ((err==UNZ_OK) && (szComment!=NULL)) + { + uLong uSizeRead ; + if (file_info.size_file_commentz_filefunc, s->filestream,(ZPOS64_T)lSeek,ZLIB_FILEFUNC_SEEK_CUR)==0) + lSeek=0; + else + err=UNZ_ERRNO; + } + + if ((file_info.size_file_comment>0) && (commentBufferSize>0)) + if (ZREAD64(s->z_filefunc, s->filestream,szComment,uSizeRead)!=uSizeRead) + err=UNZ_ERRNO; + lSeek+=file_info.size_file_comment - uSizeRead; + } + else + lSeek+=file_info.size_file_comment; + + + if ((err==UNZ_OK) && (pfile_info!=NULL)) + *pfile_info=file_info; + + if ((err==UNZ_OK) && (pfile_info_internal!=NULL)) + *pfile_info_internal=file_info_internal; + + return err; +} + + + +/* + Write info about the ZipFile in the *pglobal_info structure. + No preparation of the structure is needed + return UNZ_OK if there is no problem. +*/ +extern int ZEXPORT unzGetCurrentFileInfo64 (unzFile file, + unz_file_info64 * pfile_info, + char * szFileName, uLong fileNameBufferSize, + void *extraField, uLong extraFieldBufferSize, + char* szComment, uLong commentBufferSize) +{ + return unz64local_GetCurrentFileInfoInternal(file,pfile_info,NULL, + szFileName,fileNameBufferSize, + extraField,extraFieldBufferSize, + szComment,commentBufferSize); +} + +extern int ZEXPORT unzGetCurrentFileInfo (unzFile file, + unz_file_info * pfile_info, + char * szFileName, uLong fileNameBufferSize, + void *extraField, uLong extraFieldBufferSize, + char* szComment, uLong commentBufferSize) +{ + int err; + unz_file_info64 file_info64; + err = unz64local_GetCurrentFileInfoInternal(file,&file_info64,NULL, + szFileName,fileNameBufferSize, + extraField,extraFieldBufferSize, + szComment,commentBufferSize); + if ((err==UNZ_OK) && (pfile_info != NULL)) + { + pfile_info->version = file_info64.version; + pfile_info->version_needed = file_info64.version_needed; + pfile_info->flag = file_info64.flag; + pfile_info->compression_method = file_info64.compression_method; + pfile_info->dosDate = file_info64.dosDate; + pfile_info->crc = file_info64.crc; + + pfile_info->size_filename = file_info64.size_filename; + pfile_info->size_file_extra = file_info64.size_file_extra; + pfile_info->size_file_comment = file_info64.size_file_comment; + + pfile_info->disk_num_start = file_info64.disk_num_start; + pfile_info->internal_fa = file_info64.internal_fa; + pfile_info->external_fa = file_info64.external_fa; + + pfile_info->tmu_date = file_info64.tmu_date, + + + pfile_info->compressed_size = (uLong)file_info64.compressed_size; + pfile_info->uncompressed_size = (uLong)file_info64.uncompressed_size; + + } + return err; +} +/* + Set the current file of the zipfile to the first file. + return UNZ_OK if there is no problem +*/ +extern int ZEXPORT unzGoToFirstFile (unzFile file) +{ + int err=UNZ_OK; + unz64_s* s; + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + s->pos_in_central_dir=s->offset_central_dir; + s->num_file=0; + err=unz64local_GetCurrentFileInfoInternal(file,&s->cur_file_info, + &s->cur_file_info_internal, + NULL,0,NULL,0,NULL,0); + s->current_file_ok = (err == UNZ_OK); + return err; +} + +/* + Set the current file of the zipfile to the next file. + return UNZ_OK if there is no problem + return UNZ_END_OF_LIST_OF_FILE if the actual file was the latest. +*/ +extern int ZEXPORT unzGoToNextFile (unzFile file) +{ + unz64_s* s; + int err; + + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + if (!s->current_file_ok) + return UNZ_END_OF_LIST_OF_FILE; + if (s->gi.number_entry != 0xffff) /* 2^16 files overflow hack */ + if (s->num_file+1==s->gi.number_entry) + return UNZ_END_OF_LIST_OF_FILE; + + s->pos_in_central_dir += SIZECENTRALDIRITEM + s->cur_file_info.size_filename + + s->cur_file_info.size_file_extra + s->cur_file_info.size_file_comment ; + s->num_file++; + err = unz64local_GetCurrentFileInfoInternal(file,&s->cur_file_info, + &s->cur_file_info_internal, + NULL,0,NULL,0,NULL,0); + s->current_file_ok = (err == UNZ_OK); + return err; +} + + +/* + Try locate the file szFileName in the zipfile. + For the iCaseSensitivity signification, see unzStringFileNameCompare + + return value : + UNZ_OK if the file is found. It becomes the current file. + UNZ_END_OF_LIST_OF_FILE if the file is not found +*/ +extern int ZEXPORT unzLocateFile (unzFile file, const char *szFileName, int iCaseSensitivity) +{ + unz64_s* s; + int err; + + /* We remember the 'current' position in the file so that we can jump + * back there if we fail. + */ + unz_file_info64 cur_file_infoSaved; + unz_file_info64_internal cur_file_info_internalSaved; + ZPOS64_T num_fileSaved; + ZPOS64_T pos_in_central_dirSaved; + + + if (file==NULL) + return UNZ_PARAMERROR; + + if (strlen(szFileName)>=UNZ_MAXFILENAMEINZIP) + return UNZ_PARAMERROR; + + s=(unz64_s*)file; + if (!s->current_file_ok) + return UNZ_END_OF_LIST_OF_FILE; + + /* Save the current state */ + num_fileSaved = s->num_file; + pos_in_central_dirSaved = s->pos_in_central_dir; + cur_file_infoSaved = s->cur_file_info; + cur_file_info_internalSaved = s->cur_file_info_internal; + + err = unzGoToFirstFile(file); + + while (err == UNZ_OK) + { + char szCurrentFileName[UNZ_MAXFILENAMEINZIP+1]; + err = unzGetCurrentFileInfo64(file,NULL, + szCurrentFileName,sizeof(szCurrentFileName)-1, + NULL,0,NULL,0); + if (err == UNZ_OK) + { + if (unzStringFileNameCompare(szCurrentFileName, + szFileName,iCaseSensitivity)==0) + return UNZ_OK; + err = unzGoToNextFile(file); + } + } + + /* We failed, so restore the state of the 'current file' to where we + * were. + */ + s->num_file = num_fileSaved ; + s->pos_in_central_dir = pos_in_central_dirSaved ; + s->cur_file_info = cur_file_infoSaved; + s->cur_file_info_internal = cur_file_info_internalSaved; + return err; +} + + +/* +/////////////////////////////////////////// +// Contributed by Ryan Haksi (mailto://cryogen@infoserve.net) +// I need random access +// +// Further optimization could be realized by adding an ability +// to cache the directory in memory. The goal being a single +// comprehensive file read to put the file I need in a memory. +*/ + +/* +typedef struct unz_file_pos_s +{ + ZPOS64_T pos_in_zip_directory; // offset in file + ZPOS64_T num_of_file; // # of file +} unz_file_pos; +*/ + +extern int ZEXPORT unzGetFilePos64(unzFile file, unz64_file_pos* file_pos) +{ + unz64_s* s; + + if (file==NULL || file_pos==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + if (!s->current_file_ok) + return UNZ_END_OF_LIST_OF_FILE; + + file_pos->pos_in_zip_directory = s->pos_in_central_dir; + file_pos->num_of_file = s->num_file; + + return UNZ_OK; +} + +extern int ZEXPORT unzGetFilePos( + unzFile file, + unz_file_pos* file_pos) +{ + unz64_file_pos file_pos64; + int err = unzGetFilePos64(file,&file_pos64); + if (err==UNZ_OK) + { + file_pos->pos_in_zip_directory = (uLong)file_pos64.pos_in_zip_directory; + file_pos->num_of_file = (uLong)file_pos64.num_of_file; + } + return err; +} + +extern int ZEXPORT unzGoToFilePos64(unzFile file, const unz64_file_pos* file_pos) +{ + unz64_s* s; + int err; + + if (file==NULL || file_pos==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + + /* jump to the right spot */ + s->pos_in_central_dir = file_pos->pos_in_zip_directory; + s->num_file = file_pos->num_of_file; + + /* set the current file */ + err = unz64local_GetCurrentFileInfoInternal(file,&s->cur_file_info, + &s->cur_file_info_internal, + NULL,0,NULL,0,NULL,0); + /* return results */ + s->current_file_ok = (err == UNZ_OK); + return err; +} + +extern int ZEXPORT unzGoToFilePos( + unzFile file, + unz_file_pos* file_pos) +{ + unz64_file_pos file_pos64; + if (file_pos == NULL) + return UNZ_PARAMERROR; + + file_pos64.pos_in_zip_directory = file_pos->pos_in_zip_directory; + file_pos64.num_of_file = file_pos->num_of_file; + return unzGoToFilePos64(file,&file_pos64); +} + +/* +// Unzip Helper Functions - should be here? +/////////////////////////////////////////// +*/ + +/* + Read the local header of the current zipfile + Check the coherency of the local header and info in the end of central + directory about this file + store in *piSizeVar the size of extra info in local header + (filename and size of extra field data) +*/ +local int unz64local_CheckCurrentFileCoherencyHeader (unz64_s* s, uInt* piSizeVar, + ZPOS64_T * poffset_local_extrafield, + uInt * psize_local_extrafield) +{ + uLong uMagic,uData,uFlags; + uLong size_filename; + uLong size_extra_field; + int err=UNZ_OK; + + *piSizeVar = 0; + *poffset_local_extrafield = 0; + *psize_local_extrafield = 0; + + if (ZSEEK64(s->z_filefunc, s->filestream,s->cur_file_info_internal.offset_curfile + + s->byte_before_the_zipfile,ZLIB_FILEFUNC_SEEK_SET)!=0) + return UNZ_ERRNO; + + + if (err==UNZ_OK) + { + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uMagic) != UNZ_OK) + err=UNZ_ERRNO; + else if (uMagic!=0x04034b50) + err=UNZ_BADZIPFILE; + } + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&uData) != UNZ_OK) + err=UNZ_ERRNO; +/* + else if ((err==UNZ_OK) && (uData!=s->cur_file_info.wVersion)) + err=UNZ_BADZIPFILE; +*/ + if (unz64local_getShort(&s->z_filefunc, s->filestream,&uFlags) != UNZ_OK) + err=UNZ_ERRNO; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&uData) != UNZ_OK) + err=UNZ_ERRNO; + else if ((err==UNZ_OK) && (uData!=s->cur_file_info.compression_method)) + err=UNZ_BADZIPFILE; + + if ((err==UNZ_OK) && (s->cur_file_info.compression_method!=0) && +/* #ifdef HAVE_BZIP2 */ + (s->cur_file_info.compression_method!=Z_BZIP2ED) && +/* #endif */ + (s->cur_file_info.compression_method!=Z_DEFLATED)) + err=UNZ_BADZIPFILE; + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uData) != UNZ_OK) /* date/time */ + err=UNZ_ERRNO; + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uData) != UNZ_OK) /* crc */ + err=UNZ_ERRNO; + else if ((err==UNZ_OK) && (uData!=s->cur_file_info.crc) && ((uFlags & 8)==0)) + err=UNZ_BADZIPFILE; + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uData) != UNZ_OK) /* size compr */ + err=UNZ_ERRNO; + else if (uData != 0xFFFFFFFF && (err==UNZ_OK) && (uData!=s->cur_file_info.compressed_size) && ((uFlags & 8)==0)) + err=UNZ_BADZIPFILE; + + if (unz64local_getLong(&s->z_filefunc, s->filestream,&uData) != UNZ_OK) /* size uncompr */ + err=UNZ_ERRNO; + else if (uData != 0xFFFFFFFF && (err==UNZ_OK) && (uData!=s->cur_file_info.uncompressed_size) && ((uFlags & 8)==0)) + err=UNZ_BADZIPFILE; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&size_filename) != UNZ_OK) + err=UNZ_ERRNO; + else if ((err==UNZ_OK) && (size_filename!=s->cur_file_info.size_filename)) + err=UNZ_BADZIPFILE; + + *piSizeVar += (uInt)size_filename; + + if (unz64local_getShort(&s->z_filefunc, s->filestream,&size_extra_field) != UNZ_OK) + err=UNZ_ERRNO; + *poffset_local_extrafield= s->cur_file_info_internal.offset_curfile + + SIZEZIPLOCALHEADER + size_filename; + *psize_local_extrafield = (uInt)size_extra_field; + + *piSizeVar += (uInt)size_extra_field; + + return err; +} + +/* + Open for reading data the current file in the zipfile. + If there is no error and the file is opened, the return value is UNZ_OK. +*/ +extern int ZEXPORT unzOpenCurrentFile3 (unzFile file, int* method, + int* level, int raw, const char* password) +{ + int err=UNZ_OK; + uInt iSizeVar; + unz64_s* s; + file_in_zip64_read_info_s* pfile_in_zip_read_info; + ZPOS64_T offset_local_extrafield; /* offset of the local extra field */ + uInt size_local_extrafield; /* size of the local extra field */ +# ifndef NOUNCRYPT + char source[12]; +# else + if (password != NULL) + return UNZ_PARAMERROR; +# endif + + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + if (!s->current_file_ok) + return UNZ_PARAMERROR; + + if (s->pfile_in_zip_read != NULL) + unzCloseCurrentFile(file); + + if (unz64local_CheckCurrentFileCoherencyHeader(s,&iSizeVar, &offset_local_extrafield,&size_local_extrafield)!=UNZ_OK) + return UNZ_BADZIPFILE; + + pfile_in_zip_read_info = (file_in_zip64_read_info_s*)ALLOC(sizeof(file_in_zip64_read_info_s)); + if (pfile_in_zip_read_info==NULL) + return UNZ_INTERNALERROR; + + pfile_in_zip_read_info->read_buffer=(char*)ALLOC(UNZ_BUFSIZE); + pfile_in_zip_read_info->offset_local_extrafield = offset_local_extrafield; + pfile_in_zip_read_info->size_local_extrafield = size_local_extrafield; + pfile_in_zip_read_info->pos_local_extrafield=0; + pfile_in_zip_read_info->raw=raw; + + if (pfile_in_zip_read_info->read_buffer==NULL) + { + TRYFREE(pfile_in_zip_read_info); + return UNZ_INTERNALERROR; + } + + pfile_in_zip_read_info->stream_initialised=0; + + if (method!=NULL) + *method = (int)s->cur_file_info.compression_method; + + if (level!=NULL) + { + *level = 6; + switch (s->cur_file_info.flag & 0x06) + { + case 6 : *level = 1; break; + case 4 : *level = 2; break; + case 2 : *level = 9; break; + } + } + + if ((s->cur_file_info.compression_method!=0) && +/* #ifdef HAVE_BZIP2 */ + (s->cur_file_info.compression_method!=Z_BZIP2ED) && +/* #endif */ + (s->cur_file_info.compression_method!=Z_DEFLATED)) + + err=UNZ_BADZIPFILE; + + pfile_in_zip_read_info->crc32_wait=s->cur_file_info.crc; + pfile_in_zip_read_info->crc32=0; + pfile_in_zip_read_info->total_out_64=0; + pfile_in_zip_read_info->compression_method = s->cur_file_info.compression_method; + pfile_in_zip_read_info->filestream=s->filestream; + pfile_in_zip_read_info->z_filefunc=s->z_filefunc; + pfile_in_zip_read_info->byte_before_the_zipfile=s->byte_before_the_zipfile; + + pfile_in_zip_read_info->stream.total_out = 0; + + if ((s->cur_file_info.compression_method==Z_BZIP2ED) && (!raw)) + { +#ifdef HAVE_BZIP2 + pfile_in_zip_read_info->bstream.bzalloc = (void *(*) (void *, int, int))0; + pfile_in_zip_read_info->bstream.bzfree = (free_func)0; + pfile_in_zip_read_info->bstream.opaque = (voidpf)0; + pfile_in_zip_read_info->bstream.state = (voidpf)0; + + pfile_in_zip_read_info->stream.zalloc = (alloc_func)0; + pfile_in_zip_read_info->stream.zfree = (free_func)0; + pfile_in_zip_read_info->stream.opaque = (voidpf)0; + pfile_in_zip_read_info->stream.next_in = (voidpf)0; + pfile_in_zip_read_info->stream.avail_in = 0; + + err=BZ2_bzDecompressInit(&pfile_in_zip_read_info->bstream, 0, 0); + if (err == Z_OK) + pfile_in_zip_read_info->stream_initialised=Z_BZIP2ED; + else + { + TRYFREE(pfile_in_zip_read_info->read_buffer); + TRYFREE(pfile_in_zip_read_info); + return err; + } +#else + pfile_in_zip_read_info->raw=1; +#endif + } + else if ((s->cur_file_info.compression_method==Z_DEFLATED) && (!raw)) + { + pfile_in_zip_read_info->stream.zalloc = (alloc_func)0; + pfile_in_zip_read_info->stream.zfree = (free_func)0; + pfile_in_zip_read_info->stream.opaque = (voidpf)0; + pfile_in_zip_read_info->stream.next_in = 0; + pfile_in_zip_read_info->stream.avail_in = 0; + + err=inflateInit2(&pfile_in_zip_read_info->stream, -MAX_WBITS); + if (err == Z_OK) + pfile_in_zip_read_info->stream_initialised=Z_DEFLATED; + else + { + TRYFREE(pfile_in_zip_read_info->read_buffer); + TRYFREE(pfile_in_zip_read_info); + return err; + } + /* windowBits is passed < 0 to tell that there is no zlib header. + * Note that in this case inflate *requires* an extra "dummy" byte + * after the compressed stream in order to complete decompression and + * return Z_STREAM_END. + * In unzip, i don't wait absolutely Z_STREAM_END because I known the + * size of both compressed and uncompressed data + */ + } + pfile_in_zip_read_info->rest_read_compressed = + s->cur_file_info.compressed_size ; + pfile_in_zip_read_info->rest_read_uncompressed = + s->cur_file_info.uncompressed_size ; + + + pfile_in_zip_read_info->pos_in_zipfile = + s->cur_file_info_internal.offset_curfile + SIZEZIPLOCALHEADER + + iSizeVar; + + pfile_in_zip_read_info->stream.avail_in = (uInt)0; + + s->pfile_in_zip_read = pfile_in_zip_read_info; + s->encrypted = 0; + +# ifndef NOUNCRYPT + if (password != NULL) + { + int i; + s->pcrc_32_tab = get_crc_table(); + init_keys(password,s->keys,s->pcrc_32_tab); + if (ZSEEK64(s->z_filefunc, s->filestream, + s->pfile_in_zip_read->pos_in_zipfile + + s->pfile_in_zip_read->byte_before_the_zipfile, + SEEK_SET)!=0) + return UNZ_INTERNALERROR; + if(ZREAD64(s->z_filefunc, s->filestream,source, 12)<12) + return UNZ_INTERNALERROR; + + for (i = 0; i<12; i++) + zdecode(s->keys,s->pcrc_32_tab,source[i]); + + s->pfile_in_zip_read->pos_in_zipfile+=12; + s->encrypted=1; + } +# endif + + + return UNZ_OK; +} + +extern int ZEXPORT unzOpenCurrentFile (unzFile file) +{ + return unzOpenCurrentFile3(file, NULL, NULL, 0, NULL); +} + +extern int ZEXPORT unzOpenCurrentFilePassword (unzFile file, const char* password) +{ + return unzOpenCurrentFile3(file, NULL, NULL, 0, password); +} + +extern int ZEXPORT unzOpenCurrentFile2 (unzFile file, int* method, int* level, int raw) +{ + return unzOpenCurrentFile3(file, method, level, raw, NULL); +} + +/** Addition for GDAL : START */ + +extern ZPOS64_T ZEXPORT unzGetCurrentFileZStreamPos64( unzFile file) +{ + unz64_s* s; + file_in_zip64_read_info_s* pfile_in_zip_read_info; + s=(unz64_s*)file; + if (file==NULL) + return 0; //UNZ_PARAMERROR; + pfile_in_zip_read_info=s->pfile_in_zip_read; + if (pfile_in_zip_read_info==NULL) + return 0; //UNZ_PARAMERROR; + return pfile_in_zip_read_info->pos_in_zipfile + + pfile_in_zip_read_info->byte_before_the_zipfile; +} + +/** Addition for GDAL : END */ + +/* + Read bytes from the current file. + buf contain buffer where data must be copied + len the size of buf. + + return the number of byte copied if somes bytes are copied + return 0 if the end of file was reached + return <0 with error code if there is an error + (UNZ_ERRNO for IO error, or zLib error for uncompress error) +*/ +extern int ZEXPORT unzReadCurrentFile (unzFile file, voidp buf, unsigned len) +{ + int err=UNZ_OK; + uInt iRead = 0; + unz64_s* s; + file_in_zip64_read_info_s* pfile_in_zip_read_info; + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + pfile_in_zip_read_info=s->pfile_in_zip_read; + + if (pfile_in_zip_read_info==NULL) + return UNZ_PARAMERROR; + + + if (pfile_in_zip_read_info->read_buffer == NULL) + return UNZ_END_OF_LIST_OF_FILE; + if (len==0) + return 0; + + pfile_in_zip_read_info->stream.next_out = (Bytef*)buf; + + pfile_in_zip_read_info->stream.avail_out = (uInt)len; + + if ((len>pfile_in_zip_read_info->rest_read_uncompressed) && + (!(pfile_in_zip_read_info->raw))) + pfile_in_zip_read_info->stream.avail_out = + (uInt)pfile_in_zip_read_info->rest_read_uncompressed; + + if ((len>pfile_in_zip_read_info->rest_read_compressed+ + pfile_in_zip_read_info->stream.avail_in) && + (pfile_in_zip_read_info->raw)) + pfile_in_zip_read_info->stream.avail_out = + (uInt)pfile_in_zip_read_info->rest_read_compressed+ + pfile_in_zip_read_info->stream.avail_in; + + while (pfile_in_zip_read_info->stream.avail_out>0) + { + if ((pfile_in_zip_read_info->stream.avail_in==0) && + (pfile_in_zip_read_info->rest_read_compressed>0)) + { + uInt uReadThis = UNZ_BUFSIZE; + if (pfile_in_zip_read_info->rest_read_compressedrest_read_compressed; + if (uReadThis == 0) + return UNZ_EOF; + if (ZSEEK64(pfile_in_zip_read_info->z_filefunc, + pfile_in_zip_read_info->filestream, + pfile_in_zip_read_info->pos_in_zipfile + + pfile_in_zip_read_info->byte_before_the_zipfile, + ZLIB_FILEFUNC_SEEK_SET)!=0) + return UNZ_ERRNO; + if (ZREAD64(pfile_in_zip_read_info->z_filefunc, + pfile_in_zip_read_info->filestream, + pfile_in_zip_read_info->read_buffer, + uReadThis)!=uReadThis) + return UNZ_ERRNO; + + +# ifndef NOUNCRYPT + if(s->encrypted) + { + uInt i; + for(i=0;iread_buffer[i] = + zdecode(s->keys,s->pcrc_32_tab, + pfile_in_zip_read_info->read_buffer[i]); + } +# endif + + + pfile_in_zip_read_info->pos_in_zipfile += uReadThis; + + pfile_in_zip_read_info->rest_read_compressed-=uReadThis; + + pfile_in_zip_read_info->stream.next_in = + (Bytef*)pfile_in_zip_read_info->read_buffer; + pfile_in_zip_read_info->stream.avail_in = (uInt)uReadThis; + } + + if ((pfile_in_zip_read_info->compression_method==0) || (pfile_in_zip_read_info->raw)) + { + uInt uDoCopy,i ; + + if ((pfile_in_zip_read_info->stream.avail_in == 0) && + (pfile_in_zip_read_info->rest_read_compressed == 0)) + return (iRead==0) ? UNZ_EOF : (int)iRead; + + if (pfile_in_zip_read_info->stream.avail_out < + pfile_in_zip_read_info->stream.avail_in) + uDoCopy = pfile_in_zip_read_info->stream.avail_out ; + else + uDoCopy = pfile_in_zip_read_info->stream.avail_in ; + + for (i=0;istream.next_out+i) = + *(pfile_in_zip_read_info->stream.next_in+i); + + pfile_in_zip_read_info->total_out_64 = pfile_in_zip_read_info->total_out_64 + uDoCopy; + + pfile_in_zip_read_info->crc32 = crc32(pfile_in_zip_read_info->crc32, + pfile_in_zip_read_info->stream.next_out, + uDoCopy); + pfile_in_zip_read_info->rest_read_uncompressed-=uDoCopy; + pfile_in_zip_read_info->stream.avail_in -= uDoCopy; + pfile_in_zip_read_info->stream.avail_out -= uDoCopy; + pfile_in_zip_read_info->stream.next_out += uDoCopy; + pfile_in_zip_read_info->stream.next_in += uDoCopy; + pfile_in_zip_read_info->stream.total_out += uDoCopy; + iRead += uDoCopy; + } + else if (pfile_in_zip_read_info->compression_method==Z_BZIP2ED) + { +#ifdef HAVE_BZIP2 + uLong uTotalOutBefore,uTotalOutAfter; + const Bytef *bufBefore; + uLong uOutThis; + + pfile_in_zip_read_info->bstream.next_in = (char*)pfile_in_zip_read_info->stream.next_in; + pfile_in_zip_read_info->bstream.avail_in = pfile_in_zip_read_info->stream.avail_in; + pfile_in_zip_read_info->bstream.total_in_lo32 = pfile_in_zip_read_info->stream.total_in; + pfile_in_zip_read_info->bstream.total_in_hi32 = 0; + pfile_in_zip_read_info->bstream.next_out = (char*)pfile_in_zip_read_info->stream.next_out; + pfile_in_zip_read_info->bstream.avail_out = pfile_in_zip_read_info->stream.avail_out; + pfile_in_zip_read_info->bstream.total_out_lo32 = pfile_in_zip_read_info->stream.total_out; + pfile_in_zip_read_info->bstream.total_out_hi32 = 0; + + uTotalOutBefore = pfile_in_zip_read_info->bstream.total_out_lo32; + bufBefore = (const Bytef *)pfile_in_zip_read_info->bstream.next_out; + + err=BZ2_bzDecompress(&pfile_in_zip_read_info->bstream); + + uTotalOutAfter = pfile_in_zip_read_info->bstream.total_out_lo32; + uOutThis = uTotalOutAfter-uTotalOutBefore; + + pfile_in_zip_read_info->total_out_64 = pfile_in_zip_read_info->total_out_64 + uOutThis; + + pfile_in_zip_read_info->crc32 = crc32(pfile_in_zip_read_info->crc32,bufBefore, (uInt)(uOutThis)); + pfile_in_zip_read_info->rest_read_uncompressed -= uOutThis; + iRead += (uInt)(uTotalOutAfter - uTotalOutBefore); + + pfile_in_zip_read_info->stream.next_in = (Bytef*)pfile_in_zip_read_info->bstream.next_in; + pfile_in_zip_read_info->stream.avail_in = pfile_in_zip_read_info->bstream.avail_in; + pfile_in_zip_read_info->stream.total_in = pfile_in_zip_read_info->bstream.total_in_lo32; + pfile_in_zip_read_info->stream.next_out = (Bytef*)pfile_in_zip_read_info->bstream.next_out; + pfile_in_zip_read_info->stream.avail_out = pfile_in_zip_read_info->bstream.avail_out; + pfile_in_zip_read_info->stream.total_out = pfile_in_zip_read_info->bstream.total_out_lo32; + + if (err==BZ_STREAM_END) + return (iRead==0) ? UNZ_EOF : iRead; + if (err!=BZ_OK) + break; +#endif + } // end Z_BZIP2ED + else + { + ZPOS64_T uTotalOutBefore,uTotalOutAfter; + const Bytef *bufBefore; + ZPOS64_T uOutThis; + int flush=Z_SYNC_FLUSH; + + uTotalOutBefore = pfile_in_zip_read_info->stream.total_out; + bufBefore = pfile_in_zip_read_info->stream.next_out; + + /* + if ((pfile_in_zip_read_info->rest_read_uncompressed == + pfile_in_zip_read_info->stream.avail_out) && + (pfile_in_zip_read_info->rest_read_compressed == 0)) + flush = Z_FINISH; + */ + err=inflate(&pfile_in_zip_read_info->stream,flush); + + if ((err>=0) && (pfile_in_zip_read_info->stream.msg!=NULL)) + err = Z_DATA_ERROR; + + uTotalOutAfter = pfile_in_zip_read_info->stream.total_out; + /* Detect overflow, because z_stream.total_out is uLong (32 bits) */ + if (uTotalOutAftertotal_out_64 = pfile_in_zip_read_info->total_out_64 + uOutThis; + + pfile_in_zip_read_info->crc32 = + crc32(pfile_in_zip_read_info->crc32,bufBefore, + (uInt)(uOutThis)); + + pfile_in_zip_read_info->rest_read_uncompressed -= + uOutThis; + + iRead += (uInt)(uTotalOutAfter - uTotalOutBefore); + + if (err==Z_STREAM_END) + return (iRead==0) ? UNZ_EOF : (int)iRead; + if (err!=Z_OK) + break; + } + } + + if (err==Z_OK) + return (int)iRead; + return err; +} + + +/* + Give the current position in uncompressed data +*/ +extern z_off_t ZEXPORT unztell (unzFile file) +{ + unz64_s* s; + file_in_zip64_read_info_s* pfile_in_zip_read_info; + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + pfile_in_zip_read_info=s->pfile_in_zip_read; + + if (pfile_in_zip_read_info==NULL) + return UNZ_PARAMERROR; + + return (z_off_t)pfile_in_zip_read_info->stream.total_out; +} + +extern ZPOS64_T ZEXPORT unztell64 (unzFile file) +{ + + unz64_s* s; + file_in_zip64_read_info_s* pfile_in_zip_read_info; + if (file==NULL) + return (ZPOS64_T)-1; + s=(unz64_s*)file; + pfile_in_zip_read_info=s->pfile_in_zip_read; + + if (pfile_in_zip_read_info==NULL) + return (ZPOS64_T)-1; + + return pfile_in_zip_read_info->total_out_64; +} + + +/* + return 1 if the end of file was reached, 0 elsewhere +*/ +extern int ZEXPORT unzeof (unzFile file) +{ + unz64_s* s; + file_in_zip64_read_info_s* pfile_in_zip_read_info; + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + pfile_in_zip_read_info=s->pfile_in_zip_read; + + if (pfile_in_zip_read_info==NULL) + return UNZ_PARAMERROR; + + if (pfile_in_zip_read_info->rest_read_uncompressed == 0) + return 1; + else + return 0; +} + + + +/* +Read extra field from the current file (opened by unzOpenCurrentFile) +This is the local-header version of the extra field (sometimes, there is +more info in the local-header version than in the central-header) + + if buf==NULL, it return the size of the local extra field that can be read + + if buf!=NULL, len is the size of the buffer, the extra header is copied in + buf. + the return value is the number of bytes copied in buf, or (if <0) + the error code +*/ +extern int ZEXPORT unzGetLocalExtrafield (unzFile file, voidp buf, unsigned len) +{ + unz64_s* s; + file_in_zip64_read_info_s* pfile_in_zip_read_info; + uInt read_now; + ZPOS64_T size_to_read; + + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + pfile_in_zip_read_info=s->pfile_in_zip_read; + + if (pfile_in_zip_read_info==NULL) + return UNZ_PARAMERROR; + + size_to_read = (pfile_in_zip_read_info->size_local_extrafield - + pfile_in_zip_read_info->pos_local_extrafield); + + if (buf==NULL) + return (int)size_to_read; + + if (len>size_to_read) + read_now = (uInt)size_to_read; + else + read_now = (uInt)len ; + + if (read_now==0) + return 0; + + if (ZSEEK64(pfile_in_zip_read_info->z_filefunc, + pfile_in_zip_read_info->filestream, + pfile_in_zip_read_info->offset_local_extrafield + + pfile_in_zip_read_info->pos_local_extrafield, + ZLIB_FILEFUNC_SEEK_SET)!=0) + return UNZ_ERRNO; + + if (ZREAD64(pfile_in_zip_read_info->z_filefunc, + pfile_in_zip_read_info->filestream, + buf,read_now)!=read_now) + return UNZ_ERRNO; + + return (int)read_now; +} + +/* + Close the file in zip opened with unzOpenCurrentFile + Return UNZ_CRCERROR if all the file was read but the CRC is not good +*/ +extern int ZEXPORT unzCloseCurrentFile (unzFile file) +{ + int err=UNZ_OK; + + unz64_s* s; + file_in_zip64_read_info_s* pfile_in_zip_read_info; + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + pfile_in_zip_read_info=s->pfile_in_zip_read; + + if (pfile_in_zip_read_info==NULL) + return UNZ_PARAMERROR; + + + if ((pfile_in_zip_read_info->rest_read_uncompressed == 0) && + (!pfile_in_zip_read_info->raw)) + { + if (pfile_in_zip_read_info->crc32 != pfile_in_zip_read_info->crc32_wait) + err=UNZ_CRCERROR; + } + + + TRYFREE(pfile_in_zip_read_info->read_buffer); + pfile_in_zip_read_info->read_buffer = NULL; + if (pfile_in_zip_read_info->stream_initialised == Z_DEFLATED) + inflateEnd(&pfile_in_zip_read_info->stream); +#ifdef HAVE_BZIP2 + else if (pfile_in_zip_read_info->stream_initialised == Z_BZIP2ED) + BZ2_bzDecompressEnd(&pfile_in_zip_read_info->bstream); +#endif + + + pfile_in_zip_read_info->stream_initialised = 0; + TRYFREE(pfile_in_zip_read_info); + + s->pfile_in_zip_read=NULL; + + return err; +} + + +/* + Get the global comment string of the ZipFile, in the szComment buffer. + uSizeBuf is the size of the szComment buffer. + return the number of byte copied or an error code <0 +*/ +extern int ZEXPORT unzGetGlobalComment (unzFile file, char * szComment, uLong uSizeBuf) +{ + unz64_s* s; + uLong uReadThis ; + if (file==NULL) + return (int)UNZ_PARAMERROR; + s=(unz64_s*)file; + + uReadThis = uSizeBuf; + if (uReadThis>s->gi.size_comment) + uReadThis = s->gi.size_comment; + + if (ZSEEK64(s->z_filefunc,s->filestream,s->central_pos+22,ZLIB_FILEFUNC_SEEK_SET)!=0) + return UNZ_ERRNO; + + if (uReadThis>0) + { + *szComment='\0'; + if (ZREAD64(s->z_filefunc,s->filestream,szComment,uReadThis)!=uReadThis) + return UNZ_ERRNO; + } + + if ((szComment != NULL) && (uSizeBuf > s->gi.size_comment)) + *(szComment+s->gi.size_comment)='\0'; + return (int)uReadThis; +} + +/* Additions by RX '2004 */ +extern ZPOS64_T ZEXPORT unzGetOffset64(unzFile file) +{ + unz64_s* s; + + if (file==NULL) + return 0; //UNZ_PARAMERROR; + s=(unz64_s*)file; + if (!s->current_file_ok) + return 0; + if (s->gi.number_entry != 0 && s->gi.number_entry != 0xffff) + if (s->num_file==s->gi.number_entry) + return 0; + return s->pos_in_central_dir; +} + +extern uLong ZEXPORT unzGetOffset (unzFile file) +{ + ZPOS64_T offset64; + + if (file==NULL) + return 0; //UNZ_PARAMERROR; + offset64 = unzGetOffset64(file); + return (uLong)offset64; +} + +extern int ZEXPORT unzSetOffset64(unzFile file, ZPOS64_T pos) +{ + unz64_s* s; + int err; + + if (file==NULL) + return UNZ_PARAMERROR; + s=(unz64_s*)file; + + s->pos_in_central_dir = pos; + s->num_file = s->gi.number_entry; /* hack */ + err = unz64local_GetCurrentFileInfoInternal(file,&s->cur_file_info, + &s->cur_file_info_internal, + NULL,0,NULL,0,NULL,0); + s->current_file_ok = (err == UNZ_OK); + return err; +} + +extern int ZEXPORT unzSetOffset (unzFile file, uLong pos) +{ + return unzSetOffset64(file,pos); +} diff --git a/lib/os/windows/zlib-1.2.13/contrib/minizip/unzip.h b/lib/os/windows/zlib-1.2.13/contrib/minizip/unzip.h new file mode 100644 index 000000000000..6f95e94d7568 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/contrib/minizip/unzip.h @@ -0,0 +1,437 @@ +/* unzip.h -- IO for uncompress .zip files using zlib + Version 1.1, February 14h, 2010 + part of the MiniZip project - ( http://www.winimage.com/zLibDll/minizip.html ) + + Copyright (C) 1998-2010 Gilles Vollant (minizip) ( http://www.winimage.com/zLibDll/minizip.html ) + + Modifications of Unzip for Zip64 + Copyright (C) 2007-2008 Even Rouault + + Modifications for Zip64 support on both zip and unzip + Copyright (C) 2009-2010 Mathias Svensson ( http://result42.com ) + + For more info read MiniZip_info.txt + + --------------------------------------------------------------------------------- + + Condition of use and distribution are the same than zlib : + + This software is provided 'as-is', without any express or implied + warranty. In no event will the authors be held liable for any damages + arising from the use of this software. + + Permission is granted to anyone to use this software for any purpose, + including commercial applications, and to alter it and redistribute it + freely, subject to the following restrictions: + + 1. The origin of this software must not be misrepresented; you must not + claim that you wrote the original software. If you use this software + in a product, an acknowledgment in the product documentation would be + appreciated but is not required. + 2. Altered source versions must be plainly marked as such, and must not be + misrepresented as being the original software. + 3. This notice may not be removed or altered from any source distribution. + + --------------------------------------------------------------------------------- + + Changes + + See header of unzip64.c + +*/ + +#ifndef _unz64_H +#define _unz64_H + +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef _ZLIB_H +#include "zlib.h" +#endif + +#ifndef _ZLIBIOAPI_H +#include "ioapi.h" +#endif + +#ifdef HAVE_BZIP2 +#include "bzlib.h" +#endif + +#define Z_BZIP2ED 12 + +#if defined(STRICTUNZIP) || defined(STRICTZIPUNZIP) +/* like the STRICT of WIN32, we define a pointer that cannot be converted + from (void*) without cast */ +typedef struct TagunzFile__ { int unused; } unzFile__; +typedef unzFile__ *unzFile; +#else +typedef voidp unzFile; +#endif + + +#define UNZ_OK (0) +#define UNZ_END_OF_LIST_OF_FILE (-100) +#define UNZ_ERRNO (Z_ERRNO) +#define UNZ_EOF (0) +#define UNZ_PARAMERROR (-102) +#define UNZ_BADZIPFILE (-103) +#define UNZ_INTERNALERROR (-104) +#define UNZ_CRCERROR (-105) + +/* tm_unz contain date/time info */ +typedef struct tm_unz_s +{ + int tm_sec; /* seconds after the minute - [0,59] */ + int tm_min; /* minutes after the hour - [0,59] */ + int tm_hour; /* hours since midnight - [0,23] */ + int tm_mday; /* day of the month - [1,31] */ + int tm_mon; /* months since January - [0,11] */ + int tm_year; /* years - [1980..2044] */ +} tm_unz; + +/* unz_global_info structure contain global data about the ZIPfile + These data comes from the end of central dir */ +typedef struct unz_global_info64_s +{ + ZPOS64_T number_entry; /* total number of entries in + the central dir on this disk */ + uLong size_comment; /* size of the global comment of the zipfile */ +} unz_global_info64; + +typedef struct unz_global_info_s +{ + uLong number_entry; /* total number of entries in + the central dir on this disk */ + uLong size_comment; /* size of the global comment of the zipfile */ +} unz_global_info; + +/* unz_file_info contain information about a file in the zipfile */ +typedef struct unz_file_info64_s +{ + uLong version; /* version made by 2 bytes */ + uLong version_needed; /* version needed to extract 2 bytes */ + uLong flag; /* general purpose bit flag 2 bytes */ + uLong compression_method; /* compression method 2 bytes */ + uLong dosDate; /* last mod file date in Dos fmt 4 bytes */ + uLong crc; /* crc-32 4 bytes */ + ZPOS64_T compressed_size; /* compressed size 8 bytes */ + ZPOS64_T uncompressed_size; /* uncompressed size 8 bytes */ + uLong size_filename; /* filename length 2 bytes */ + uLong size_file_extra; /* extra field length 2 bytes */ + uLong size_file_comment; /* file comment length 2 bytes */ + + uLong disk_num_start; /* disk number start 2 bytes */ + uLong internal_fa; /* internal file attributes 2 bytes */ + uLong external_fa; /* external file attributes 4 bytes */ + + tm_unz tmu_date; +} unz_file_info64; + +typedef struct unz_file_info_s +{ + uLong version; /* version made by 2 bytes */ + uLong version_needed; /* version needed to extract 2 bytes */ + uLong flag; /* general purpose bit flag 2 bytes */ + uLong compression_method; /* compression method 2 bytes */ + uLong dosDate; /* last mod file date in Dos fmt 4 bytes */ + uLong crc; /* crc-32 4 bytes */ + uLong compressed_size; /* compressed size 4 bytes */ + uLong uncompressed_size; /* uncompressed size 4 bytes */ + uLong size_filename; /* filename length 2 bytes */ + uLong size_file_extra; /* extra field length 2 bytes */ + uLong size_file_comment; /* file comment length 2 bytes */ + + uLong disk_num_start; /* disk number start 2 bytes */ + uLong internal_fa; /* internal file attributes 2 bytes */ + uLong external_fa; /* external file attributes 4 bytes */ + + tm_unz tmu_date; +} unz_file_info; + +extern int ZEXPORT unzStringFileNameCompare OF ((const char* fileName1, + const char* fileName2, + int iCaseSensitivity)); +/* + Compare two filename (fileName1,fileName2). + If iCaseSenisivity = 1, comparision is case sensitivity (like strcmp) + If iCaseSenisivity = 2, comparision is not case sensitivity (like strcmpi + or strcasecmp) + If iCaseSenisivity = 0, case sensitivity is defaut of your operating system + (like 1 on Unix, 2 on Windows) +*/ + + +extern unzFile ZEXPORT unzOpen OF((const char *path)); +extern unzFile ZEXPORT unzOpen64 OF((const void *path)); +/* + Open a Zip file. path contain the full pathname (by example, + on a Windows XP computer "c:\\zlib\\zlib113.zip" or on an Unix computer + "zlib/zlib113.zip". + If the zipfile cannot be opened (file don't exist or in not valid), the + return value is NULL. + Else, the return value is a unzFile Handle, usable with other function + of this unzip package. + the "64" function take a const void* pointer, because the path is just the + value passed to the open64_file_func callback. + Under Windows, if UNICODE is defined, using fill_fopen64_filefunc, the path + is a pointer to a wide unicode string (LPCTSTR is LPCWSTR), so const char* + does not describe the reality +*/ + + +extern unzFile ZEXPORT unzOpen2 OF((const char *path, + zlib_filefunc_def* pzlib_filefunc_def)); +/* + Open a Zip file, like unzOpen, but provide a set of file low level API + for read/write the zip file (see ioapi.h) +*/ + +extern unzFile ZEXPORT unzOpen2_64 OF((const void *path, + zlib_filefunc64_def* pzlib_filefunc_def)); +/* + Open a Zip file, like unz64Open, but provide a set of file low level API + for read/write the zip file (see ioapi.h) +*/ + +extern int ZEXPORT unzClose OF((unzFile file)); +/* + Close a ZipFile opened with unzOpen. + If there is files inside the .Zip opened with unzOpenCurrentFile (see later), + these files MUST be closed with unzCloseCurrentFile before call unzClose. + return UNZ_OK if there is no problem. */ + +extern int ZEXPORT unzGetGlobalInfo OF((unzFile file, + unz_global_info *pglobal_info)); + +extern int ZEXPORT unzGetGlobalInfo64 OF((unzFile file, + unz_global_info64 *pglobal_info)); +/* + Write info about the ZipFile in the *pglobal_info structure. + No preparation of the structure is needed + return UNZ_OK if there is no problem. */ + + +extern int ZEXPORT unzGetGlobalComment OF((unzFile file, + char *szComment, + uLong uSizeBuf)); +/* + Get the global comment string of the ZipFile, in the szComment buffer. + uSizeBuf is the size of the szComment buffer. + return the number of byte copied or an error code <0 +*/ + + +/***************************************************************************/ +/* Unzip package allow you browse the directory of the zipfile */ + +extern int ZEXPORT unzGoToFirstFile OF((unzFile file)); +/* + Set the current file of the zipfile to the first file. + return UNZ_OK if there is no problem +*/ + +extern int ZEXPORT unzGoToNextFile OF((unzFile file)); +/* + Set the current file of the zipfile to the next file. + return UNZ_OK if there is no problem + return UNZ_END_OF_LIST_OF_FILE if the actual file was the latest. +*/ + +extern int ZEXPORT unzLocateFile OF((unzFile file, + const char *szFileName, + int iCaseSensitivity)); +/* + Try locate the file szFileName in the zipfile. + For the iCaseSensitivity signification, see unzStringFileNameCompare + + return value : + UNZ_OK if the file is found. It becomes the current file. + UNZ_END_OF_LIST_OF_FILE if the file is not found +*/ + + +/* ****************************************** */ +/* Ryan supplied functions */ +/* unz_file_info contain information about a file in the zipfile */ +typedef struct unz_file_pos_s +{ + uLong pos_in_zip_directory; /* offset in zip file directory */ + uLong num_of_file; /* # of file */ +} unz_file_pos; + +extern int ZEXPORT unzGetFilePos( + unzFile file, + unz_file_pos* file_pos); + +extern int ZEXPORT unzGoToFilePos( + unzFile file, + unz_file_pos* file_pos); + +typedef struct unz64_file_pos_s +{ + ZPOS64_T pos_in_zip_directory; /* offset in zip file directory */ + ZPOS64_T num_of_file; /* # of file */ +} unz64_file_pos; + +extern int ZEXPORT unzGetFilePos64( + unzFile file, + unz64_file_pos* file_pos); + +extern int ZEXPORT unzGoToFilePos64( + unzFile file, + const unz64_file_pos* file_pos); + +/* ****************************************** */ + +extern int ZEXPORT unzGetCurrentFileInfo64 OF((unzFile file, + unz_file_info64 *pfile_info, + char *szFileName, + uLong fileNameBufferSize, + void *extraField, + uLong extraFieldBufferSize, + char *szComment, + uLong commentBufferSize)); + +extern int ZEXPORT unzGetCurrentFileInfo OF((unzFile file, + unz_file_info *pfile_info, + char *szFileName, + uLong fileNameBufferSize, + void *extraField, + uLong extraFieldBufferSize, + char *szComment, + uLong commentBufferSize)); +/* + Get Info about the current file + if pfile_info!=NULL, the *pfile_info structure will contain somes info about + the current file + if szFileName!=NULL, the filemane string will be copied in szFileName + (fileNameBufferSize is the size of the buffer) + if extraField!=NULL, the extra field information will be copied in extraField + (extraFieldBufferSize is the size of the buffer). + This is the Central-header version of the extra field + if szComment!=NULL, the comment string of the file will be copied in szComment + (commentBufferSize is the size of the buffer) +*/ + + +/** Addition for GDAL : START */ + +extern ZPOS64_T ZEXPORT unzGetCurrentFileZStreamPos64 OF((unzFile file)); + +/** Addition for GDAL : END */ + + +/***************************************************************************/ +/* for reading the content of the current zipfile, you can open it, read data + from it, and close it (you can close it before reading all the file) + */ + +extern int ZEXPORT unzOpenCurrentFile OF((unzFile file)); +/* + Open for reading data the current file in the zipfile. + If there is no error, the return value is UNZ_OK. +*/ + +extern int ZEXPORT unzOpenCurrentFilePassword OF((unzFile file, + const char* password)); +/* + Open for reading data the current file in the zipfile. + password is a crypting password + If there is no error, the return value is UNZ_OK. +*/ + +extern int ZEXPORT unzOpenCurrentFile2 OF((unzFile file, + int* method, + int* level, + int raw)); +/* + Same than unzOpenCurrentFile, but open for read raw the file (not uncompress) + if raw==1 + *method will receive method of compression, *level will receive level of + compression + note : you can set level parameter as NULL (if you did not want known level, + but you CANNOT set method parameter as NULL +*/ + +extern int ZEXPORT unzOpenCurrentFile3 OF((unzFile file, + int* method, + int* level, + int raw, + const char* password)); +/* + Same than unzOpenCurrentFile, but open for read raw the file (not uncompress) + if raw==1 + *method will receive method of compression, *level will receive level of + compression + note : you can set level parameter as NULL (if you did not want known level, + but you CANNOT set method parameter as NULL +*/ + + +extern int ZEXPORT unzCloseCurrentFile OF((unzFile file)); +/* + Close the file in zip opened with unzOpenCurrentFile + Return UNZ_CRCERROR if all the file was read but the CRC is not good +*/ + +extern int ZEXPORT unzReadCurrentFile OF((unzFile file, + voidp buf, + unsigned len)); +/* + Read bytes from the current file (opened by unzOpenCurrentFile) + buf contain buffer where data must be copied + len the size of buf. + + return the number of byte copied if somes bytes are copied + return 0 if the end of file was reached + return <0 with error code if there is an error + (UNZ_ERRNO for IO error, or zLib error for uncompress error) +*/ + +extern z_off_t ZEXPORT unztell OF((unzFile file)); + +extern ZPOS64_T ZEXPORT unztell64 OF((unzFile file)); +/* + Give the current position in uncompressed data +*/ + +extern int ZEXPORT unzeof OF((unzFile file)); +/* + return 1 if the end of file was reached, 0 elsewhere +*/ + +extern int ZEXPORT unzGetLocalExtrafield OF((unzFile file, + voidp buf, + unsigned len)); +/* + Read extra field from the current file (opened by unzOpenCurrentFile) + This is the local-header version of the extra field (sometimes, there is + more info in the local-header version than in the central-header) + + if buf==NULL, it return the size of the local extra field + + if buf!=NULL, len is the size of the buffer, the extra header is copied in + buf. + the return value is the number of bytes copied in buf, or (if <0) + the error code +*/ + +/***************************************************************************/ + +/* Get the current file offset */ +extern ZPOS64_T ZEXPORT unzGetOffset64 (unzFile file); +extern uLong ZEXPORT unzGetOffset (unzFile file); + +/* Set the current file offset */ +extern int ZEXPORT unzSetOffset64 (unzFile file, ZPOS64_T pos); +extern int ZEXPORT unzSetOffset (unzFile file, uLong pos); + + + +#ifdef __cplusplus +} +#endif + +#endif /* _unz64_H */ diff --git a/lib/os/windows/zlib-1.2.13/contrib/minizip/zip.c b/lib/os/windows/zlib-1.2.13/contrib/minizip/zip.c new file mode 100644 index 000000000000..66d693f85a58 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/contrib/minizip/zip.c @@ -0,0 +1,2002 @@ +/* zip.c -- IO on .zip files using zlib + Version 1.1, February 14h, 2010 + part of the MiniZip project - ( http://www.winimage.com/zLibDll/minizip.html ) + + Copyright (C) 1998-2010 Gilles Vollant (minizip) ( http://www.winimage.com/zLibDll/minizip.html ) + + Modifications for Zip64 support + Copyright (C) 2009-2010 Mathias Svensson ( http://result42.com ) + + For more info read MiniZip_info.txt + + Changes + Oct-2009 - Mathias Svensson - Remove old C style function prototypes + Oct-2009 - Mathias Svensson - Added Zip64 Support when creating new file archives + Oct-2009 - Mathias Svensson - Did some code cleanup and refactoring to get better overview of some functions. + Oct-2009 - Mathias Svensson - Added zipRemoveExtraInfoBlock to strip extra field data from its ZIP64 data + It is used when recreting zip archive with RAW when deleting items from a zip. + ZIP64 data is automatically added to items that needs it, and existing ZIP64 data need to be removed. + Oct-2009 - Mathias Svensson - Added support for BZIP2 as compression mode (bzip2 lib is required) + Jan-2010 - back to unzip and minizip 1.0 name scheme, with compatibility layer + +*/ + + +#include +#include +#include +#include +#include "zlib.h" +#include "zip.h" + +#ifdef STDC +# include +# include +# include +#endif +#ifdef NO_ERRNO_H + extern int errno; +#else +# include +#endif + + +#ifndef local +# define local static +#endif +/* compile with -Dlocal if your debugger can't find static symbols */ + +#ifndef VERSIONMADEBY +# define VERSIONMADEBY (0x0) /* platform depedent */ +#endif + +#ifndef Z_BUFSIZE +#define Z_BUFSIZE (64*1024) //(16384) +#endif + +#ifndef Z_MAXFILENAMEINZIP +#define Z_MAXFILENAMEINZIP (256) +#endif + +#ifndef ALLOC +# define ALLOC(size) (malloc(size)) +#endif +#ifndef TRYFREE +# define TRYFREE(p) {if (p) free(p);} +#endif + +/* +#define SIZECENTRALDIRITEM (0x2e) +#define SIZEZIPLOCALHEADER (0x1e) +*/ + +/* I've found an old Unix (a SunOS 4.1.3_U1) without all SEEK_* defined.... */ + + +// NOT sure that this work on ALL platform +#define MAKEULONG64(a, b) ((ZPOS64_T)(((unsigned long)(a)) | ((ZPOS64_T)((unsigned long)(b))) << 32)) + +#ifndef SEEK_CUR +#define SEEK_CUR 1 +#endif + +#ifndef SEEK_END +#define SEEK_END 2 +#endif + +#ifndef SEEK_SET +#define SEEK_SET 0 +#endif + +#ifndef DEF_MEM_LEVEL +#if MAX_MEM_LEVEL >= 8 +# define DEF_MEM_LEVEL 8 +#else +# define DEF_MEM_LEVEL MAX_MEM_LEVEL +#endif +#endif +const char zip_copyright[] =" zip 1.01 Copyright 1998-2004 Gilles Vollant - http://www.winimage.com/zLibDll"; + + +#define SIZEDATA_INDATABLOCK (4096-(4*4)) + +#define LOCALHEADERMAGIC (0x04034b50) +#define CENTRALHEADERMAGIC (0x02014b50) +#define ENDHEADERMAGIC (0x06054b50) +#define ZIP64ENDHEADERMAGIC (0x6064b50) +#define ZIP64ENDLOCHEADERMAGIC (0x7064b50) + +#define FLAG_LOCALHEADER_OFFSET (0x06) +#define CRC_LOCALHEADER_OFFSET (0x0e) + +#define SIZECENTRALHEADER (0x2e) /* 46 */ + +typedef struct linkedlist_datablock_internal_s +{ + struct linkedlist_datablock_internal_s* next_datablock; + uLong avail_in_this_block; + uLong filled_in_this_block; + uLong unused; /* for future use and alignment */ + unsigned char data[SIZEDATA_INDATABLOCK]; +} linkedlist_datablock_internal; + +typedef struct linkedlist_data_s +{ + linkedlist_datablock_internal* first_block; + linkedlist_datablock_internal* last_block; +} linkedlist_data; + + +typedef struct +{ + z_stream stream; /* zLib stream structure for inflate */ +#ifdef HAVE_BZIP2 + bz_stream bstream; /* bzLib stream structure for bziped */ +#endif + + int stream_initialised; /* 1 is stream is initialised */ + uInt pos_in_buffered_data; /* last written byte in buffered_data */ + + ZPOS64_T pos_local_header; /* offset of the local header of the file + currenty writing */ + char* central_header; /* central header data for the current file */ + uLong size_centralExtra; + uLong size_centralheader; /* size of the central header for cur file */ + uLong size_centralExtraFree; /* Extra bytes allocated to the centralheader but that are not used */ + uLong flag; /* flag of the file currently writing */ + + int method; /* compression method of file currenty wr.*/ + int raw; /* 1 for directly writing raw data */ + Byte buffered_data[Z_BUFSIZE];/* buffer contain compressed data to be writ*/ + uLong dosDate; + uLong crc32; + int encrypt; + int zip64; /* Add ZIP64 extened information in the extra field */ + ZPOS64_T pos_zip64extrainfo; + ZPOS64_T totalCompressedData; + ZPOS64_T totalUncompressedData; +#ifndef NOCRYPT + unsigned long keys[3]; /* keys defining the pseudo-random sequence */ + const z_crc_t* pcrc_32_tab; + unsigned crypt_header_size; +#endif +} curfile64_info; + +typedef struct +{ + zlib_filefunc64_32_def z_filefunc; + voidpf filestream; /* io structore of the zipfile */ + linkedlist_data central_dir;/* datablock with central dir in construction*/ + int in_opened_file_inzip; /* 1 if a file in the zip is currently writ.*/ + curfile64_info ci; /* info on the file curretly writing */ + + ZPOS64_T begin_pos; /* position of the beginning of the zipfile */ + ZPOS64_T add_position_when_writing_offset; + ZPOS64_T number_entry; + +#ifndef NO_ADDFILEINEXISTINGZIP + char *globalcomment; +#endif + +} zip64_internal; + + +#ifndef NOCRYPT +#define INCLUDECRYPTINGCODE_IFCRYPTALLOWED +#include "crypt.h" +#endif + +local linkedlist_datablock_internal* allocate_new_datablock() +{ + linkedlist_datablock_internal* ldi; + ldi = (linkedlist_datablock_internal*) + ALLOC(sizeof(linkedlist_datablock_internal)); + if (ldi!=NULL) + { + ldi->next_datablock = NULL ; + ldi->filled_in_this_block = 0 ; + ldi->avail_in_this_block = SIZEDATA_INDATABLOCK ; + } + return ldi; +} + +local void free_datablock(linkedlist_datablock_internal* ldi) +{ + while (ldi!=NULL) + { + linkedlist_datablock_internal* ldinext = ldi->next_datablock; + TRYFREE(ldi); + ldi = ldinext; + } +} + +local void init_linkedlist(linkedlist_data* ll) +{ + ll->first_block = ll->last_block = NULL; +} + +local void free_linkedlist(linkedlist_data* ll) +{ + free_datablock(ll->first_block); + ll->first_block = ll->last_block = NULL; +} + + +local int add_data_in_datablock(linkedlist_data* ll, const void* buf, uLong len) +{ + linkedlist_datablock_internal* ldi; + const unsigned char* from_copy; + + if (ll==NULL) + return ZIP_INTERNALERROR; + + if (ll->last_block == NULL) + { + ll->first_block = ll->last_block = allocate_new_datablock(); + if (ll->first_block == NULL) + return ZIP_INTERNALERROR; + } + + ldi = ll->last_block; + from_copy = (unsigned char*)buf; + + while (len>0) + { + uInt copy_this; + uInt i; + unsigned char* to_copy; + + if (ldi->avail_in_this_block==0) + { + ldi->next_datablock = allocate_new_datablock(); + if (ldi->next_datablock == NULL) + return ZIP_INTERNALERROR; + ldi = ldi->next_datablock ; + ll->last_block = ldi; + } + + if (ldi->avail_in_this_block < len) + copy_this = (uInt)ldi->avail_in_this_block; + else + copy_this = (uInt)len; + + to_copy = &(ldi->data[ldi->filled_in_this_block]); + + for (i=0;ifilled_in_this_block += copy_this; + ldi->avail_in_this_block -= copy_this; + from_copy += copy_this ; + len -= copy_this; + } + return ZIP_OK; +} + + + +/****************************************************************************/ + +#ifndef NO_ADDFILEINEXISTINGZIP +/* =========================================================================== + Inputs a long in LSB order to the given file + nbByte == 1, 2 ,4 or 8 (byte, short or long, ZPOS64_T) +*/ + +local int zip64local_putValue OF((const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, ZPOS64_T x, int nbByte)); +local int zip64local_putValue (const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, ZPOS64_T x, int nbByte) +{ + unsigned char buf[8]; + int n; + for (n = 0; n < nbByte; n++) + { + buf[n] = (unsigned char)(x & 0xff); + x >>= 8; + } + if (x != 0) + { /* data overflow - hack for ZIP64 (X Roche) */ + for (n = 0; n < nbByte; n++) + { + buf[n] = 0xff; + } + } + + if (ZWRITE64(*pzlib_filefunc_def,filestream,buf,(uLong)nbByte)!=(uLong)nbByte) + return ZIP_ERRNO; + else + return ZIP_OK; +} + +local void zip64local_putValue_inmemory OF((void* dest, ZPOS64_T x, int nbByte)); +local void zip64local_putValue_inmemory (void* dest, ZPOS64_T x, int nbByte) +{ + unsigned char* buf=(unsigned char*)dest; + int n; + for (n = 0; n < nbByte; n++) { + buf[n] = (unsigned char)(x & 0xff); + x >>= 8; + } + + if (x != 0) + { /* data overflow - hack for ZIP64 */ + for (n = 0; n < nbByte; n++) + { + buf[n] = 0xff; + } + } +} + +/****************************************************************************/ + + +local uLong zip64local_TmzDateToDosDate(const tm_zip* ptm) +{ + uLong year = (uLong)ptm->tm_year; + if (year>=1980) + year-=1980; + else if (year>=80) + year-=80; + return + (uLong) (((uLong)(ptm->tm_mday) + (32 * (uLong)(ptm->tm_mon+1)) + (512 * year)) << 16) | + (((uLong)ptm->tm_sec/2) + (32 * (uLong)ptm->tm_min) + (2048 * (uLong)ptm->tm_hour)); +} + + +/****************************************************************************/ + +local int zip64local_getByte OF((const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, int *pi)); + +local int zip64local_getByte(const zlib_filefunc64_32_def* pzlib_filefunc_def,voidpf filestream,int* pi) +{ + unsigned char c; + int err = (int)ZREAD64(*pzlib_filefunc_def,filestream,&c,1); + if (err==1) + { + *pi = (int)c; + return ZIP_OK; + } + else + { + if (ZERROR64(*pzlib_filefunc_def,filestream)) + return ZIP_ERRNO; + else + return ZIP_EOF; + } +} + + +/* =========================================================================== + Reads a long in LSB order from the given gz_stream. Sets +*/ +local int zip64local_getShort OF((const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, uLong *pX)); + +local int zip64local_getShort (const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, uLong* pX) +{ + uLong x ; + int i = 0; + int err; + + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x = (uLong)i; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((uLong)i)<<8; + + if (err==ZIP_OK) + *pX = x; + else + *pX = 0; + return err; +} + +local int zip64local_getLong OF((const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, uLong *pX)); + +local int zip64local_getLong (const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, uLong* pX) +{ + uLong x ; + int i = 0; + int err; + + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x = (uLong)i; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((uLong)i)<<8; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((uLong)i)<<16; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((uLong)i)<<24; + + if (err==ZIP_OK) + *pX = x; + else + *pX = 0; + return err; +} + +local int zip64local_getLong64 OF((const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, ZPOS64_T *pX)); + + +local int zip64local_getLong64 (const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream, ZPOS64_T *pX) +{ + ZPOS64_T x; + int i = 0; + int err; + + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x = (ZPOS64_T)i; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((ZPOS64_T)i)<<8; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((ZPOS64_T)i)<<16; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((ZPOS64_T)i)<<24; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((ZPOS64_T)i)<<32; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((ZPOS64_T)i)<<40; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((ZPOS64_T)i)<<48; + + if (err==ZIP_OK) + err = zip64local_getByte(pzlib_filefunc_def,filestream,&i); + x += ((ZPOS64_T)i)<<56; + + if (err==ZIP_OK) + *pX = x; + else + *pX = 0; + + return err; +} + +#ifndef BUFREADCOMMENT +#define BUFREADCOMMENT (0x400) +#endif +/* + Locate the Central directory of a zipfile (at the end, just before + the global comment) +*/ +local ZPOS64_T zip64local_SearchCentralDir OF((const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream)); + +local ZPOS64_T zip64local_SearchCentralDir(const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream) +{ + unsigned char* buf; + ZPOS64_T uSizeFile; + ZPOS64_T uBackRead; + ZPOS64_T uMaxBack=0xffff; /* maximum size of global comment */ + ZPOS64_T uPosFound=0; + + if (ZSEEK64(*pzlib_filefunc_def,filestream,0,ZLIB_FILEFUNC_SEEK_END) != 0) + return 0; + + + uSizeFile = ZTELL64(*pzlib_filefunc_def,filestream); + + if (uMaxBack>uSizeFile) + uMaxBack = uSizeFile; + + buf = (unsigned char*)ALLOC(BUFREADCOMMENT+4); + if (buf==NULL) + return 0; + + uBackRead = 4; + while (uBackReaduMaxBack) + uBackRead = uMaxBack; + else + uBackRead+=BUFREADCOMMENT; + uReadPos = uSizeFile-uBackRead ; + + uReadSize = ((BUFREADCOMMENT+4) < (uSizeFile-uReadPos)) ? + (BUFREADCOMMENT+4) : (uLong)(uSizeFile-uReadPos); + if (ZSEEK64(*pzlib_filefunc_def,filestream,uReadPos,ZLIB_FILEFUNC_SEEK_SET)!=0) + break; + + if (ZREAD64(*pzlib_filefunc_def,filestream,buf,uReadSize)!=uReadSize) + break; + + for (i=(int)uReadSize-3; (i--)>0;) + if (((*(buf+i))==0x50) && ((*(buf+i+1))==0x4b) && + ((*(buf+i+2))==0x05) && ((*(buf+i+3))==0x06)) + { + uPosFound = uReadPos+(unsigned)i; + break; + } + + if (uPosFound!=0) + break; + } + TRYFREE(buf); + return uPosFound; +} + +/* +Locate the End of Zip64 Central directory locator and from there find the CD of a zipfile (at the end, just before +the global comment) +*/ +local ZPOS64_T zip64local_SearchCentralDir64 OF((const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream)); + +local ZPOS64_T zip64local_SearchCentralDir64(const zlib_filefunc64_32_def* pzlib_filefunc_def, voidpf filestream) +{ + unsigned char* buf; + ZPOS64_T uSizeFile; + ZPOS64_T uBackRead; + ZPOS64_T uMaxBack=0xffff; /* maximum size of global comment */ + ZPOS64_T uPosFound=0; + uLong uL; + ZPOS64_T relativeOffset; + + if (ZSEEK64(*pzlib_filefunc_def,filestream,0,ZLIB_FILEFUNC_SEEK_END) != 0) + return 0; + + uSizeFile = ZTELL64(*pzlib_filefunc_def,filestream); + + if (uMaxBack>uSizeFile) + uMaxBack = uSizeFile; + + buf = (unsigned char*)ALLOC(BUFREADCOMMENT+4); + if (buf==NULL) + return 0; + + uBackRead = 4; + while (uBackReaduMaxBack) + uBackRead = uMaxBack; + else + uBackRead+=BUFREADCOMMENT; + uReadPos = uSizeFile-uBackRead ; + + uReadSize = ((BUFREADCOMMENT+4) < (uSizeFile-uReadPos)) ? + (BUFREADCOMMENT+4) : (uLong)(uSizeFile-uReadPos); + if (ZSEEK64(*pzlib_filefunc_def,filestream,uReadPos,ZLIB_FILEFUNC_SEEK_SET)!=0) + break; + + if (ZREAD64(*pzlib_filefunc_def,filestream,buf,uReadSize)!=uReadSize) + break; + + for (i=(int)uReadSize-3; (i--)>0;) + { + // Signature "0x07064b50" Zip64 end of central directory locater + if (((*(buf+i))==0x50) && ((*(buf+i+1))==0x4b) && ((*(buf+i+2))==0x06) && ((*(buf+i+3))==0x07)) + { + uPosFound = uReadPos+(unsigned)i; + break; + } + } + + if (uPosFound!=0) + break; + } + + TRYFREE(buf); + if (uPosFound == 0) + return 0; + + /* Zip64 end of central directory locator */ + if (ZSEEK64(*pzlib_filefunc_def,filestream, uPosFound,ZLIB_FILEFUNC_SEEK_SET)!=0) + return 0; + + /* the signature, already checked */ + if (zip64local_getLong(pzlib_filefunc_def,filestream,&uL)!=ZIP_OK) + return 0; + + /* number of the disk with the start of the zip64 end of central directory */ + if (zip64local_getLong(pzlib_filefunc_def,filestream,&uL)!=ZIP_OK) + return 0; + if (uL != 0) + return 0; + + /* relative offset of the zip64 end of central directory record */ + if (zip64local_getLong64(pzlib_filefunc_def,filestream,&relativeOffset)!=ZIP_OK) + return 0; + + /* total number of disks */ + if (zip64local_getLong(pzlib_filefunc_def,filestream,&uL)!=ZIP_OK) + return 0; + if (uL != 1) + return 0; + + /* Goto Zip64 end of central directory record */ + if (ZSEEK64(*pzlib_filefunc_def,filestream, relativeOffset,ZLIB_FILEFUNC_SEEK_SET)!=0) + return 0; + + /* the signature */ + if (zip64local_getLong(pzlib_filefunc_def,filestream,&uL)!=ZIP_OK) + return 0; + + if (uL != 0x06064b50) // signature of 'Zip64 end of central directory' + return 0; + + return relativeOffset; +} + +local int LoadCentralDirectoryRecord(zip64_internal* pziinit) +{ + int err=ZIP_OK; + ZPOS64_T byte_before_the_zipfile;/* byte before the zipfile, (>0 for sfx)*/ + + ZPOS64_T size_central_dir; /* size of the central directory */ + ZPOS64_T offset_central_dir; /* offset of start of central directory */ + ZPOS64_T central_pos; + uLong uL; + + uLong number_disk; /* number of the current dist, used for + spaning ZIP, unsupported, always 0*/ + uLong number_disk_with_CD; /* number the the disk with central dir, used + for spaning ZIP, unsupported, always 0*/ + ZPOS64_T number_entry; + ZPOS64_T number_entry_CD; /* total number of entries in + the central dir + (same than number_entry on nospan) */ + uLong VersionMadeBy; + uLong VersionNeeded; + uLong size_comment; + + int hasZIP64Record = 0; + + // check first if we find a ZIP64 record + central_pos = zip64local_SearchCentralDir64(&pziinit->z_filefunc,pziinit->filestream); + if(central_pos > 0) + { + hasZIP64Record = 1; + } + else if(central_pos == 0) + { + central_pos = zip64local_SearchCentralDir(&pziinit->z_filefunc,pziinit->filestream); + } + +/* disable to allow appending to empty ZIP archive + if (central_pos==0) + err=ZIP_ERRNO; +*/ + + if(hasZIP64Record) + { + ZPOS64_T sizeEndOfCentralDirectory; + if (ZSEEK64(pziinit->z_filefunc, pziinit->filestream, central_pos, ZLIB_FILEFUNC_SEEK_SET) != 0) + err=ZIP_ERRNO; + + /* the signature, already checked */ + if (zip64local_getLong(&pziinit->z_filefunc, pziinit->filestream,&uL)!=ZIP_OK) + err=ZIP_ERRNO; + + /* size of zip64 end of central directory record */ + if (zip64local_getLong64(&pziinit->z_filefunc, pziinit->filestream, &sizeEndOfCentralDirectory)!=ZIP_OK) + err=ZIP_ERRNO; + + /* version made by */ + if (zip64local_getShort(&pziinit->z_filefunc, pziinit->filestream, &VersionMadeBy)!=ZIP_OK) + err=ZIP_ERRNO; + + /* version needed to extract */ + if (zip64local_getShort(&pziinit->z_filefunc, pziinit->filestream, &VersionNeeded)!=ZIP_OK) + err=ZIP_ERRNO; + + /* number of this disk */ + if (zip64local_getLong(&pziinit->z_filefunc, pziinit->filestream,&number_disk)!=ZIP_OK) + err=ZIP_ERRNO; + + /* number of the disk with the start of the central directory */ + if (zip64local_getLong(&pziinit->z_filefunc, pziinit->filestream,&number_disk_with_CD)!=ZIP_OK) + err=ZIP_ERRNO; + + /* total number of entries in the central directory on this disk */ + if (zip64local_getLong64(&pziinit->z_filefunc, pziinit->filestream, &number_entry)!=ZIP_OK) + err=ZIP_ERRNO; + + /* total number of entries in the central directory */ + if (zip64local_getLong64(&pziinit->z_filefunc, pziinit->filestream,&number_entry_CD)!=ZIP_OK) + err=ZIP_ERRNO; + + if ((number_entry_CD!=number_entry) || (number_disk_with_CD!=0) || (number_disk!=0)) + err=ZIP_BADZIPFILE; + + /* size of the central directory */ + if (zip64local_getLong64(&pziinit->z_filefunc, pziinit->filestream,&size_central_dir)!=ZIP_OK) + err=ZIP_ERRNO; + + /* offset of start of central directory with respect to the + starting disk number */ + if (zip64local_getLong64(&pziinit->z_filefunc, pziinit->filestream,&offset_central_dir)!=ZIP_OK) + err=ZIP_ERRNO; + + // TODO.. + // read the comment from the standard central header. + size_comment = 0; + } + else + { + // Read End of central Directory info + if (ZSEEK64(pziinit->z_filefunc, pziinit->filestream, central_pos,ZLIB_FILEFUNC_SEEK_SET)!=0) + err=ZIP_ERRNO; + + /* the signature, already checked */ + if (zip64local_getLong(&pziinit->z_filefunc, pziinit->filestream,&uL)!=ZIP_OK) + err=ZIP_ERRNO; + + /* number of this disk */ + if (zip64local_getShort(&pziinit->z_filefunc, pziinit->filestream,&number_disk)!=ZIP_OK) + err=ZIP_ERRNO; + + /* number of the disk with the start of the central directory */ + if (zip64local_getShort(&pziinit->z_filefunc, pziinit->filestream,&number_disk_with_CD)!=ZIP_OK) + err=ZIP_ERRNO; + + /* total number of entries in the central dir on this disk */ + number_entry = 0; + if (zip64local_getShort(&pziinit->z_filefunc, pziinit->filestream, &uL)!=ZIP_OK) + err=ZIP_ERRNO; + else + number_entry = uL; + + /* total number of entries in the central dir */ + number_entry_CD = 0; + if (zip64local_getShort(&pziinit->z_filefunc, pziinit->filestream, &uL)!=ZIP_OK) + err=ZIP_ERRNO; + else + number_entry_CD = uL; + + if ((number_entry_CD!=number_entry) || (number_disk_with_CD!=0) || (number_disk!=0)) + err=ZIP_BADZIPFILE; + + /* size of the central directory */ + size_central_dir = 0; + if (zip64local_getLong(&pziinit->z_filefunc, pziinit->filestream, &uL)!=ZIP_OK) + err=ZIP_ERRNO; + else + size_central_dir = uL; + + /* offset of start of central directory with respect to the starting disk number */ + offset_central_dir = 0; + if (zip64local_getLong(&pziinit->z_filefunc, pziinit->filestream, &uL)!=ZIP_OK) + err=ZIP_ERRNO; + else + offset_central_dir = uL; + + + /* zipfile global comment length */ + if (zip64local_getShort(&pziinit->z_filefunc, pziinit->filestream, &size_comment)!=ZIP_OK) + err=ZIP_ERRNO; + } + + if ((central_posz_filefunc, pziinit->filestream); + return ZIP_ERRNO; + } + + if (size_comment>0) + { + pziinit->globalcomment = (char*)ALLOC(size_comment+1); + if (pziinit->globalcomment) + { + size_comment = ZREAD64(pziinit->z_filefunc, pziinit->filestream, pziinit->globalcomment,size_comment); + pziinit->globalcomment[size_comment]=0; + } + } + + byte_before_the_zipfile = central_pos - (offset_central_dir+size_central_dir); + pziinit->add_position_when_writing_offset = byte_before_the_zipfile; + + { + ZPOS64_T size_central_dir_to_read = size_central_dir; + size_t buf_size = SIZEDATA_INDATABLOCK; + void* buf_read = (void*)ALLOC(buf_size); + if (ZSEEK64(pziinit->z_filefunc, pziinit->filestream, offset_central_dir + byte_before_the_zipfile, ZLIB_FILEFUNC_SEEK_SET) != 0) + err=ZIP_ERRNO; + + while ((size_central_dir_to_read>0) && (err==ZIP_OK)) + { + ZPOS64_T read_this = SIZEDATA_INDATABLOCK; + if (read_this > size_central_dir_to_read) + read_this = size_central_dir_to_read; + + if (ZREAD64(pziinit->z_filefunc, pziinit->filestream,buf_read,(uLong)read_this) != read_this) + err=ZIP_ERRNO; + + if (err==ZIP_OK) + err = add_data_in_datablock(&pziinit->central_dir,buf_read, (uLong)read_this); + + size_central_dir_to_read-=read_this; + } + TRYFREE(buf_read); + } + pziinit->begin_pos = byte_before_the_zipfile; + pziinit->number_entry = number_entry_CD; + + if (ZSEEK64(pziinit->z_filefunc, pziinit->filestream, offset_central_dir+byte_before_the_zipfile,ZLIB_FILEFUNC_SEEK_SET) != 0) + err=ZIP_ERRNO; + + return err; +} + + +#endif /* !NO_ADDFILEINEXISTINGZIP*/ + + +/************************************************************/ +extern zipFile ZEXPORT zipOpen3 (const void *pathname, int append, zipcharpc* globalcomment, zlib_filefunc64_32_def* pzlib_filefunc64_32_def) +{ + zip64_internal ziinit; + zip64_internal* zi; + int err=ZIP_OK; + + ziinit.z_filefunc.zseek32_file = NULL; + ziinit.z_filefunc.ztell32_file = NULL; + if (pzlib_filefunc64_32_def==NULL) + fill_fopen64_filefunc(&ziinit.z_filefunc.zfile_func64); + else + ziinit.z_filefunc = *pzlib_filefunc64_32_def; + + ziinit.filestream = ZOPEN64(ziinit.z_filefunc, + pathname, + (append == APPEND_STATUS_CREATE) ? + (ZLIB_FILEFUNC_MODE_READ | ZLIB_FILEFUNC_MODE_WRITE | ZLIB_FILEFUNC_MODE_CREATE) : + (ZLIB_FILEFUNC_MODE_READ | ZLIB_FILEFUNC_MODE_WRITE | ZLIB_FILEFUNC_MODE_EXISTING)); + + if (ziinit.filestream == NULL) + return NULL; + + if (append == APPEND_STATUS_CREATEAFTER) + ZSEEK64(ziinit.z_filefunc,ziinit.filestream,0,SEEK_END); + + ziinit.begin_pos = ZTELL64(ziinit.z_filefunc,ziinit.filestream); + ziinit.in_opened_file_inzip = 0; + ziinit.ci.stream_initialised = 0; + ziinit.number_entry = 0; + ziinit.add_position_when_writing_offset = 0; + init_linkedlist(&(ziinit.central_dir)); + + + + zi = (zip64_internal*)ALLOC(sizeof(zip64_internal)); + if (zi==NULL) + { + ZCLOSE64(ziinit.z_filefunc,ziinit.filestream); + return NULL; + } + + /* now we add file in a zipfile */ +# ifndef NO_ADDFILEINEXISTINGZIP + ziinit.globalcomment = NULL; + if (append == APPEND_STATUS_ADDINZIP) + { + // Read and Cache Central Directory Records + err = LoadCentralDirectoryRecord(&ziinit); + } + + if (globalcomment) + { + *globalcomment = ziinit.globalcomment; + } +# endif /* !NO_ADDFILEINEXISTINGZIP*/ + + if (err != ZIP_OK) + { +# ifndef NO_ADDFILEINEXISTINGZIP + TRYFREE(ziinit.globalcomment); +# endif /* !NO_ADDFILEINEXISTINGZIP*/ + TRYFREE(zi); + return NULL; + } + else + { + *zi = ziinit; + return (zipFile)zi; + } +} + +extern zipFile ZEXPORT zipOpen2 (const char *pathname, int append, zipcharpc* globalcomment, zlib_filefunc_def* pzlib_filefunc32_def) +{ + if (pzlib_filefunc32_def != NULL) + { + zlib_filefunc64_32_def zlib_filefunc64_32_def_fill; + fill_zlib_filefunc64_32_def_from_filefunc32(&zlib_filefunc64_32_def_fill,pzlib_filefunc32_def); + return zipOpen3(pathname, append, globalcomment, &zlib_filefunc64_32_def_fill); + } + else + return zipOpen3(pathname, append, globalcomment, NULL); +} + +extern zipFile ZEXPORT zipOpen2_64 (const void *pathname, int append, zipcharpc* globalcomment, zlib_filefunc64_def* pzlib_filefunc_def) +{ + if (pzlib_filefunc_def != NULL) + { + zlib_filefunc64_32_def zlib_filefunc64_32_def_fill; + zlib_filefunc64_32_def_fill.zfile_func64 = *pzlib_filefunc_def; + zlib_filefunc64_32_def_fill.ztell32_file = NULL; + zlib_filefunc64_32_def_fill.zseek32_file = NULL; + return zipOpen3(pathname, append, globalcomment, &zlib_filefunc64_32_def_fill); + } + else + return zipOpen3(pathname, append, globalcomment, NULL); +} + + + +extern zipFile ZEXPORT zipOpen (const char* pathname, int append) +{ + return zipOpen3((const void*)pathname,append,NULL,NULL); +} + +extern zipFile ZEXPORT zipOpen64 (const void* pathname, int append) +{ + return zipOpen3(pathname,append,NULL,NULL); +} + +local int Write_LocalFileHeader(zip64_internal* zi, const char* filename, uInt size_extrafield_local, const void* extrafield_local) +{ + /* write the local header */ + int err; + uInt size_filename = (uInt)strlen(filename); + uInt size_extrafield = size_extrafield_local; + + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)LOCALHEADERMAGIC, 4); + + if (err==ZIP_OK) + { + if(zi->ci.zip64) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)45,2);/* version needed to extract */ + else + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)20,2);/* version needed to extract */ + } + + if (err==ZIP_OK) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)zi->ci.flag,2); + + if (err==ZIP_OK) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)zi->ci.method,2); + + if (err==ZIP_OK) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)zi->ci.dosDate,4); + + // CRC / Compressed size / Uncompressed size will be filled in later and rewritten later + if (err==ZIP_OK) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0,4); /* crc 32, unknown */ + if (err==ZIP_OK) + { + if(zi->ci.zip64) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0xFFFFFFFF,4); /* compressed size, unknown */ + else + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0,4); /* compressed size, unknown */ + } + if (err==ZIP_OK) + { + if(zi->ci.zip64) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0xFFFFFFFF,4); /* uncompressed size, unknown */ + else + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0,4); /* uncompressed size, unknown */ + } + + if (err==ZIP_OK) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)size_filename,2); + + if(zi->ci.zip64) + { + size_extrafield += 20; + } + + if (err==ZIP_OK) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)size_extrafield,2); + + if ((err==ZIP_OK) && (size_filename > 0)) + { + if (ZWRITE64(zi->z_filefunc,zi->filestream,filename,size_filename)!=size_filename) + err = ZIP_ERRNO; + } + + if ((err==ZIP_OK) && (size_extrafield_local > 0)) + { + if (ZWRITE64(zi->z_filefunc, zi->filestream, extrafield_local, size_extrafield_local) != size_extrafield_local) + err = ZIP_ERRNO; + } + + + if ((err==ZIP_OK) && (zi->ci.zip64)) + { + // write the Zip64 extended info + short HeaderID = 1; + short DataSize = 16; + ZPOS64_T CompressedSize = 0; + ZPOS64_T UncompressedSize = 0; + + // Remember position of Zip64 extended info for the local file header. (needed when we update size after done with file) + zi->ci.pos_zip64extrainfo = ZTELL64(zi->z_filefunc,zi->filestream); + + err = zip64local_putValue(&zi->z_filefunc, zi->filestream, (ZPOS64_T)HeaderID,2); + err = zip64local_putValue(&zi->z_filefunc, zi->filestream, (ZPOS64_T)DataSize,2); + + err = zip64local_putValue(&zi->z_filefunc, zi->filestream, (ZPOS64_T)UncompressedSize,8); + err = zip64local_putValue(&zi->z_filefunc, zi->filestream, (ZPOS64_T)CompressedSize,8); + } + + return err; +} + +/* + NOTE. + When writing RAW the ZIP64 extended information in extrafield_local and extrafield_global needs to be stripped + before calling this function it can be done with zipRemoveExtraInfoBlock + + It is not done here because then we need to realloc a new buffer since parameters are 'const' and I want to minimize + unnecessary allocations. + */ +extern int ZEXPORT zipOpenNewFileInZip4_64 (zipFile file, const char* filename, const zip_fileinfo* zipfi, + const void* extrafield_local, uInt size_extrafield_local, + const void* extrafield_global, uInt size_extrafield_global, + const char* comment, int method, int level, int raw, + int windowBits,int memLevel, int strategy, + const char* password, uLong crcForCrypting, + uLong versionMadeBy, uLong flagBase, int zip64) +{ + zip64_internal* zi; + uInt size_filename; + uInt size_comment; + uInt i; + int err = ZIP_OK; + +# ifdef NOCRYPT + (crcForCrypting); + if (password != NULL) + return ZIP_PARAMERROR; +# endif + + if (file == NULL) + return ZIP_PARAMERROR; + +#ifdef HAVE_BZIP2 + if ((method!=0) && (method!=Z_DEFLATED) && (method!=Z_BZIP2ED)) + return ZIP_PARAMERROR; +#else + if ((method!=0) && (method!=Z_DEFLATED)) + return ZIP_PARAMERROR; +#endif + + zi = (zip64_internal*)file; + + if (zi->in_opened_file_inzip == 1) + { + err = zipCloseFileInZip (file); + if (err != ZIP_OK) + return err; + } + + if (filename==NULL) + filename="-"; + + if (comment==NULL) + size_comment = 0; + else + size_comment = (uInt)strlen(comment); + + size_filename = (uInt)strlen(filename); + + if (zipfi == NULL) + zi->ci.dosDate = 0; + else + { + if (zipfi->dosDate != 0) + zi->ci.dosDate = zipfi->dosDate; + else + zi->ci.dosDate = zip64local_TmzDateToDosDate(&zipfi->tmz_date); + } + + zi->ci.flag = flagBase; + if ((level==8) || (level==9)) + zi->ci.flag |= 2; + if (level==2) + zi->ci.flag |= 4; + if (level==1) + zi->ci.flag |= 6; + if (password != NULL) + zi->ci.flag |= 1; + + zi->ci.crc32 = 0; + zi->ci.method = method; + zi->ci.encrypt = 0; + zi->ci.stream_initialised = 0; + zi->ci.pos_in_buffered_data = 0; + zi->ci.raw = raw; + zi->ci.pos_local_header = ZTELL64(zi->z_filefunc,zi->filestream); + + zi->ci.size_centralheader = SIZECENTRALHEADER + size_filename + size_extrafield_global + size_comment; + zi->ci.size_centralExtraFree = 32; // Extra space we have reserved in case we need to add ZIP64 extra info data + + zi->ci.central_header = (char*)ALLOC((uInt)zi->ci.size_centralheader + zi->ci.size_centralExtraFree); + + zi->ci.size_centralExtra = size_extrafield_global; + zip64local_putValue_inmemory(zi->ci.central_header,(uLong)CENTRALHEADERMAGIC,4); + /* version info */ + zip64local_putValue_inmemory(zi->ci.central_header+4,(uLong)versionMadeBy,2); + zip64local_putValue_inmemory(zi->ci.central_header+6,(uLong)20,2); + zip64local_putValue_inmemory(zi->ci.central_header+8,(uLong)zi->ci.flag,2); + zip64local_putValue_inmemory(zi->ci.central_header+10,(uLong)zi->ci.method,2); + zip64local_putValue_inmemory(zi->ci.central_header+12,(uLong)zi->ci.dosDate,4); + zip64local_putValue_inmemory(zi->ci.central_header+16,(uLong)0,4); /*crc*/ + zip64local_putValue_inmemory(zi->ci.central_header+20,(uLong)0,4); /*compr size*/ + zip64local_putValue_inmemory(zi->ci.central_header+24,(uLong)0,4); /*uncompr size*/ + zip64local_putValue_inmemory(zi->ci.central_header+28,(uLong)size_filename,2); + zip64local_putValue_inmemory(zi->ci.central_header+30,(uLong)size_extrafield_global,2); + zip64local_putValue_inmemory(zi->ci.central_header+32,(uLong)size_comment,2); + zip64local_putValue_inmemory(zi->ci.central_header+34,(uLong)0,2); /*disk nm start*/ + + if (zipfi==NULL) + zip64local_putValue_inmemory(zi->ci.central_header+36,(uLong)0,2); + else + zip64local_putValue_inmemory(zi->ci.central_header+36,(uLong)zipfi->internal_fa,2); + + if (zipfi==NULL) + zip64local_putValue_inmemory(zi->ci.central_header+38,(uLong)0,4); + else + zip64local_putValue_inmemory(zi->ci.central_header+38,(uLong)zipfi->external_fa,4); + + if(zi->ci.pos_local_header >= 0xffffffff) + zip64local_putValue_inmemory(zi->ci.central_header+42,(uLong)0xffffffff,4); + else + zip64local_putValue_inmemory(zi->ci.central_header+42,(uLong)zi->ci.pos_local_header - zi->add_position_when_writing_offset,4); + + for (i=0;ici.central_header+SIZECENTRALHEADER+i) = *(filename+i); + + for (i=0;ici.central_header+SIZECENTRALHEADER+size_filename+i) = + *(((const char*)extrafield_global)+i); + + for (i=0;ici.central_header+SIZECENTRALHEADER+size_filename+ + size_extrafield_global+i) = *(comment+i); + if (zi->ci.central_header == NULL) + return ZIP_INTERNALERROR; + + zi->ci.zip64 = zip64; + zi->ci.totalCompressedData = 0; + zi->ci.totalUncompressedData = 0; + zi->ci.pos_zip64extrainfo = 0; + + err = Write_LocalFileHeader(zi, filename, size_extrafield_local, extrafield_local); + +#ifdef HAVE_BZIP2 + zi->ci.bstream.avail_in = (uInt)0; + zi->ci.bstream.avail_out = (uInt)Z_BUFSIZE; + zi->ci.bstream.next_out = (char*)zi->ci.buffered_data; + zi->ci.bstream.total_in_hi32 = 0; + zi->ci.bstream.total_in_lo32 = 0; + zi->ci.bstream.total_out_hi32 = 0; + zi->ci.bstream.total_out_lo32 = 0; +#endif + + zi->ci.stream.avail_in = (uInt)0; + zi->ci.stream.avail_out = (uInt)Z_BUFSIZE; + zi->ci.stream.next_out = zi->ci.buffered_data; + zi->ci.stream.total_in = 0; + zi->ci.stream.total_out = 0; + zi->ci.stream.data_type = Z_BINARY; + +#ifdef HAVE_BZIP2 + if ((err==ZIP_OK) && (zi->ci.method == Z_DEFLATED || zi->ci.method == Z_BZIP2ED) && (!zi->ci.raw)) +#else + if ((err==ZIP_OK) && (zi->ci.method == Z_DEFLATED) && (!zi->ci.raw)) +#endif + { + if(zi->ci.method == Z_DEFLATED) + { + zi->ci.stream.zalloc = (alloc_func)0; + zi->ci.stream.zfree = (free_func)0; + zi->ci.stream.opaque = (voidpf)0; + + if (windowBits>0) + windowBits = -windowBits; + + err = deflateInit2(&zi->ci.stream, level, Z_DEFLATED, windowBits, memLevel, strategy); + + if (err==Z_OK) + zi->ci.stream_initialised = Z_DEFLATED; + } + else if(zi->ci.method == Z_BZIP2ED) + { +#ifdef HAVE_BZIP2 + // Init BZip stuff here + zi->ci.bstream.bzalloc = 0; + zi->ci.bstream.bzfree = 0; + zi->ci.bstream.opaque = (voidpf)0; + + err = BZ2_bzCompressInit(&zi->ci.bstream, level, 0,35); + if(err == BZ_OK) + zi->ci.stream_initialised = Z_BZIP2ED; +#endif + } + + } + +# ifndef NOCRYPT + zi->ci.crypt_header_size = 0; + if ((err==Z_OK) && (password != NULL)) + { + unsigned char bufHead[RAND_HEAD_LEN]; + unsigned int sizeHead; + zi->ci.encrypt = 1; + zi->ci.pcrc_32_tab = get_crc_table(); + /*init_keys(password,zi->ci.keys,zi->ci.pcrc_32_tab);*/ + + sizeHead=crypthead(password,bufHead,RAND_HEAD_LEN,zi->ci.keys,zi->ci.pcrc_32_tab,crcForCrypting); + zi->ci.crypt_header_size = sizeHead; + + if (ZWRITE64(zi->z_filefunc,zi->filestream,bufHead,sizeHead) != sizeHead) + err = ZIP_ERRNO; + } +# endif + + if (err==Z_OK) + zi->in_opened_file_inzip = 1; + return err; +} + +extern int ZEXPORT zipOpenNewFileInZip4 (zipFile file, const char* filename, const zip_fileinfo* zipfi, + const void* extrafield_local, uInt size_extrafield_local, + const void* extrafield_global, uInt size_extrafield_global, + const char* comment, int method, int level, int raw, + int windowBits,int memLevel, int strategy, + const char* password, uLong crcForCrypting, + uLong versionMadeBy, uLong flagBase) +{ + return zipOpenNewFileInZip4_64 (file, filename, zipfi, + extrafield_local, size_extrafield_local, + extrafield_global, size_extrafield_global, + comment, method, level, raw, + windowBits, memLevel, strategy, + password, crcForCrypting, versionMadeBy, flagBase, 0); +} + +extern int ZEXPORT zipOpenNewFileInZip3 (zipFile file, const char* filename, const zip_fileinfo* zipfi, + const void* extrafield_local, uInt size_extrafield_local, + const void* extrafield_global, uInt size_extrafield_global, + const char* comment, int method, int level, int raw, + int windowBits,int memLevel, int strategy, + const char* password, uLong crcForCrypting) +{ + return zipOpenNewFileInZip4_64 (file, filename, zipfi, + extrafield_local, size_extrafield_local, + extrafield_global, size_extrafield_global, + comment, method, level, raw, + windowBits, memLevel, strategy, + password, crcForCrypting, VERSIONMADEBY, 0, 0); +} + +extern int ZEXPORT zipOpenNewFileInZip3_64(zipFile file, const char* filename, const zip_fileinfo* zipfi, + const void* extrafield_local, uInt size_extrafield_local, + const void* extrafield_global, uInt size_extrafield_global, + const char* comment, int method, int level, int raw, + int windowBits,int memLevel, int strategy, + const char* password, uLong crcForCrypting, int zip64) +{ + return zipOpenNewFileInZip4_64 (file, filename, zipfi, + extrafield_local, size_extrafield_local, + extrafield_global, size_extrafield_global, + comment, method, level, raw, + windowBits, memLevel, strategy, + password, crcForCrypting, VERSIONMADEBY, 0, zip64); +} + +extern int ZEXPORT zipOpenNewFileInZip2(zipFile file, const char* filename, const zip_fileinfo* zipfi, + const void* extrafield_local, uInt size_extrafield_local, + const void* extrafield_global, uInt size_extrafield_global, + const char* comment, int method, int level, int raw) +{ + return zipOpenNewFileInZip4_64 (file, filename, zipfi, + extrafield_local, size_extrafield_local, + extrafield_global, size_extrafield_global, + comment, method, level, raw, + -MAX_WBITS, DEF_MEM_LEVEL, Z_DEFAULT_STRATEGY, + NULL, 0, VERSIONMADEBY, 0, 0); +} + +extern int ZEXPORT zipOpenNewFileInZip2_64(zipFile file, const char* filename, const zip_fileinfo* zipfi, + const void* extrafield_local, uInt size_extrafield_local, + const void* extrafield_global, uInt size_extrafield_global, + const char* comment, int method, int level, int raw, int zip64) +{ + return zipOpenNewFileInZip4_64 (file, filename, zipfi, + extrafield_local, size_extrafield_local, + extrafield_global, size_extrafield_global, + comment, method, level, raw, + -MAX_WBITS, DEF_MEM_LEVEL, Z_DEFAULT_STRATEGY, + NULL, 0, VERSIONMADEBY, 0, zip64); +} + +extern int ZEXPORT zipOpenNewFileInZip64 (zipFile file, const char* filename, const zip_fileinfo* zipfi, + const void* extrafield_local, uInt size_extrafield_local, + const void*extrafield_global, uInt size_extrafield_global, + const char* comment, int method, int level, int zip64) +{ + return zipOpenNewFileInZip4_64 (file, filename, zipfi, + extrafield_local, size_extrafield_local, + extrafield_global, size_extrafield_global, + comment, method, level, 0, + -MAX_WBITS, DEF_MEM_LEVEL, Z_DEFAULT_STRATEGY, + NULL, 0, VERSIONMADEBY, 0, zip64); +} + +extern int ZEXPORT zipOpenNewFileInZip (zipFile file, const char* filename, const zip_fileinfo* zipfi, + const void* extrafield_local, uInt size_extrafield_local, + const void*extrafield_global, uInt size_extrafield_global, + const char* comment, int method, int level) +{ + return zipOpenNewFileInZip4_64 (file, filename, zipfi, + extrafield_local, size_extrafield_local, + extrafield_global, size_extrafield_global, + comment, method, level, 0, + -MAX_WBITS, DEF_MEM_LEVEL, Z_DEFAULT_STRATEGY, + NULL, 0, VERSIONMADEBY, 0, 0); +} + +local int zip64FlushWriteBuffer(zip64_internal* zi) +{ + int err=ZIP_OK; + + if (zi->ci.encrypt != 0) + { +#ifndef NOCRYPT + uInt i; + int t; + for (i=0;ici.pos_in_buffered_data;i++) + zi->ci.buffered_data[i] = zencode(zi->ci.keys, zi->ci.pcrc_32_tab, zi->ci.buffered_data[i],t); +#endif + } + + if (ZWRITE64(zi->z_filefunc,zi->filestream,zi->ci.buffered_data,zi->ci.pos_in_buffered_data) != zi->ci.pos_in_buffered_data) + err = ZIP_ERRNO; + + zi->ci.totalCompressedData += zi->ci.pos_in_buffered_data; + +#ifdef HAVE_BZIP2 + if(zi->ci.method == Z_BZIP2ED) + { + zi->ci.totalUncompressedData += zi->ci.bstream.total_in_lo32; + zi->ci.bstream.total_in_lo32 = 0; + zi->ci.bstream.total_in_hi32 = 0; + } + else +#endif + { + zi->ci.totalUncompressedData += zi->ci.stream.total_in; + zi->ci.stream.total_in = 0; + } + + + zi->ci.pos_in_buffered_data = 0; + + return err; +} + +extern int ZEXPORT zipWriteInFileInZip (zipFile file,const void* buf,unsigned int len) +{ + zip64_internal* zi; + int err=ZIP_OK; + + if (file == NULL) + return ZIP_PARAMERROR; + zi = (zip64_internal*)file; + + if (zi->in_opened_file_inzip == 0) + return ZIP_PARAMERROR; + + zi->ci.crc32 = crc32(zi->ci.crc32,buf,(uInt)len); + +#ifdef HAVE_BZIP2 + if(zi->ci.method == Z_BZIP2ED && (!zi->ci.raw)) + { + zi->ci.bstream.next_in = (void*)buf; + zi->ci.bstream.avail_in = len; + err = BZ_RUN_OK; + + while ((err==BZ_RUN_OK) && (zi->ci.bstream.avail_in>0)) + { + if (zi->ci.bstream.avail_out == 0) + { + if (zip64FlushWriteBuffer(zi) == ZIP_ERRNO) + err = ZIP_ERRNO; + zi->ci.bstream.avail_out = (uInt)Z_BUFSIZE; + zi->ci.bstream.next_out = (char*)zi->ci.buffered_data; + } + + + if(err != BZ_RUN_OK) + break; + + if ((zi->ci.method == Z_BZIP2ED) && (!zi->ci.raw)) + { + uLong uTotalOutBefore_lo = zi->ci.bstream.total_out_lo32; +// uLong uTotalOutBefore_hi = zi->ci.bstream.total_out_hi32; + err=BZ2_bzCompress(&zi->ci.bstream, BZ_RUN); + + zi->ci.pos_in_buffered_data += (uInt)(zi->ci.bstream.total_out_lo32 - uTotalOutBefore_lo) ; + } + } + + if(err == BZ_RUN_OK) + err = ZIP_OK; + } + else +#endif + { + zi->ci.stream.next_in = (Bytef*)buf; + zi->ci.stream.avail_in = len; + + while ((err==ZIP_OK) && (zi->ci.stream.avail_in>0)) + { + if (zi->ci.stream.avail_out == 0) + { + if (zip64FlushWriteBuffer(zi) == ZIP_ERRNO) + err = ZIP_ERRNO; + zi->ci.stream.avail_out = (uInt)Z_BUFSIZE; + zi->ci.stream.next_out = zi->ci.buffered_data; + } + + + if(err != ZIP_OK) + break; + + if ((zi->ci.method == Z_DEFLATED) && (!zi->ci.raw)) + { + uLong uTotalOutBefore = zi->ci.stream.total_out; + err=deflate(&zi->ci.stream, Z_NO_FLUSH); + + zi->ci.pos_in_buffered_data += (uInt)(zi->ci.stream.total_out - uTotalOutBefore) ; + } + else + { + uInt copy_this,i; + if (zi->ci.stream.avail_in < zi->ci.stream.avail_out) + copy_this = zi->ci.stream.avail_in; + else + copy_this = zi->ci.stream.avail_out; + + for (i = 0; i < copy_this; i++) + *(((char*)zi->ci.stream.next_out)+i) = + *(((const char*)zi->ci.stream.next_in)+i); + { + zi->ci.stream.avail_in -= copy_this; + zi->ci.stream.avail_out-= copy_this; + zi->ci.stream.next_in+= copy_this; + zi->ci.stream.next_out+= copy_this; + zi->ci.stream.total_in+= copy_this; + zi->ci.stream.total_out+= copy_this; + zi->ci.pos_in_buffered_data += copy_this; + } + } + }// while(...) + } + + return err; +} + +extern int ZEXPORT zipCloseFileInZipRaw (zipFile file, uLong uncompressed_size, uLong crc32) +{ + return zipCloseFileInZipRaw64 (file, uncompressed_size, crc32); +} + +extern int ZEXPORT zipCloseFileInZipRaw64 (zipFile file, ZPOS64_T uncompressed_size, uLong crc32) +{ + zip64_internal* zi; + ZPOS64_T compressed_size; + uLong invalidValue = 0xffffffff; + unsigned datasize = 0; + int err=ZIP_OK; + + if (file == NULL) + return ZIP_PARAMERROR; + zi = (zip64_internal*)file; + + if (zi->in_opened_file_inzip == 0) + return ZIP_PARAMERROR; + zi->ci.stream.avail_in = 0; + + if ((zi->ci.method == Z_DEFLATED) && (!zi->ci.raw)) + { + while (err==ZIP_OK) + { + uLong uTotalOutBefore; + if (zi->ci.stream.avail_out == 0) + { + if (zip64FlushWriteBuffer(zi) == ZIP_ERRNO) + err = ZIP_ERRNO; + zi->ci.stream.avail_out = (uInt)Z_BUFSIZE; + zi->ci.stream.next_out = zi->ci.buffered_data; + } + uTotalOutBefore = zi->ci.stream.total_out; + err=deflate(&zi->ci.stream, Z_FINISH); + zi->ci.pos_in_buffered_data += (uInt)(zi->ci.stream.total_out - uTotalOutBefore) ; + } + } + else if ((zi->ci.method == Z_BZIP2ED) && (!zi->ci.raw)) + { +#ifdef HAVE_BZIP2 + err = BZ_FINISH_OK; + while (err==BZ_FINISH_OK) + { + uLong uTotalOutBefore; + if (zi->ci.bstream.avail_out == 0) + { + if (zip64FlushWriteBuffer(zi) == ZIP_ERRNO) + err = ZIP_ERRNO; + zi->ci.bstream.avail_out = (uInt)Z_BUFSIZE; + zi->ci.bstream.next_out = (char*)zi->ci.buffered_data; + } + uTotalOutBefore = zi->ci.bstream.total_out_lo32; + err=BZ2_bzCompress(&zi->ci.bstream, BZ_FINISH); + if(err == BZ_STREAM_END) + err = Z_STREAM_END; + + zi->ci.pos_in_buffered_data += (uInt)(zi->ci.bstream.total_out_lo32 - uTotalOutBefore); + } + + if(err == BZ_FINISH_OK) + err = ZIP_OK; +#endif + } + + if (err==Z_STREAM_END) + err=ZIP_OK; /* this is normal */ + + if ((zi->ci.pos_in_buffered_data>0) && (err==ZIP_OK)) + { + if (zip64FlushWriteBuffer(zi)==ZIP_ERRNO) + err = ZIP_ERRNO; + } + + if ((zi->ci.method == Z_DEFLATED) && (!zi->ci.raw)) + { + int tmp_err = deflateEnd(&zi->ci.stream); + if (err == ZIP_OK) + err = tmp_err; + zi->ci.stream_initialised = 0; + } +#ifdef HAVE_BZIP2 + else if((zi->ci.method == Z_BZIP2ED) && (!zi->ci.raw)) + { + int tmperr = BZ2_bzCompressEnd(&zi->ci.bstream); + if (err==ZIP_OK) + err = tmperr; + zi->ci.stream_initialised = 0; + } +#endif + + if (!zi->ci.raw) + { + crc32 = (uLong)zi->ci.crc32; + uncompressed_size = zi->ci.totalUncompressedData; + } + compressed_size = zi->ci.totalCompressedData; + +# ifndef NOCRYPT + compressed_size += zi->ci.crypt_header_size; +# endif + + // update Current Item crc and sizes, + if(compressed_size >= 0xffffffff || uncompressed_size >= 0xffffffff || zi->ci.pos_local_header >= 0xffffffff) + { + /*version Made by*/ + zip64local_putValue_inmemory(zi->ci.central_header+4,(uLong)45,2); + /*version needed*/ + zip64local_putValue_inmemory(zi->ci.central_header+6,(uLong)45,2); + + } + + zip64local_putValue_inmemory(zi->ci.central_header+16,crc32,4); /*crc*/ + + + if(compressed_size >= 0xffffffff) + zip64local_putValue_inmemory(zi->ci.central_header+20, invalidValue,4); /*compr size*/ + else + zip64local_putValue_inmemory(zi->ci.central_header+20, compressed_size,4); /*compr size*/ + + /// set internal file attributes field + if (zi->ci.stream.data_type == Z_ASCII) + zip64local_putValue_inmemory(zi->ci.central_header+36,(uLong)Z_ASCII,2); + + if(uncompressed_size >= 0xffffffff) + zip64local_putValue_inmemory(zi->ci.central_header+24, invalidValue,4); /*uncompr size*/ + else + zip64local_putValue_inmemory(zi->ci.central_header+24, uncompressed_size,4); /*uncompr size*/ + + // Add ZIP64 extra info field for uncompressed size + if(uncompressed_size >= 0xffffffff) + datasize += 8; + + // Add ZIP64 extra info field for compressed size + if(compressed_size >= 0xffffffff) + datasize += 8; + + // Add ZIP64 extra info field for relative offset to local file header of current file + if(zi->ci.pos_local_header >= 0xffffffff) + datasize += 8; + + if(datasize > 0) + { + char* p = NULL; + + if((uLong)(datasize + 4) > zi->ci.size_centralExtraFree) + { + // we can not write more data to the buffer that we have room for. + return ZIP_BADZIPFILE; + } + + p = zi->ci.central_header + zi->ci.size_centralheader; + + // Add Extra Information Header for 'ZIP64 information' + zip64local_putValue_inmemory(p, 0x0001, 2); // HeaderID + p += 2; + zip64local_putValue_inmemory(p, datasize, 2); // DataSize + p += 2; + + if(uncompressed_size >= 0xffffffff) + { + zip64local_putValue_inmemory(p, uncompressed_size, 8); + p += 8; + } + + if(compressed_size >= 0xffffffff) + { + zip64local_putValue_inmemory(p, compressed_size, 8); + p += 8; + } + + if(zi->ci.pos_local_header >= 0xffffffff) + { + zip64local_putValue_inmemory(p, zi->ci.pos_local_header, 8); + p += 8; + } + + // Update how much extra free space we got in the memory buffer + // and increase the centralheader size so the new ZIP64 fields are included + // ( 4 below is the size of HeaderID and DataSize field ) + zi->ci.size_centralExtraFree -= datasize + 4; + zi->ci.size_centralheader += datasize + 4; + + // Update the extra info size field + zi->ci.size_centralExtra += datasize + 4; + zip64local_putValue_inmemory(zi->ci.central_header+30,(uLong)zi->ci.size_centralExtra,2); + } + + if (err==ZIP_OK) + err = add_data_in_datablock(&zi->central_dir, zi->ci.central_header, (uLong)zi->ci.size_centralheader); + + free(zi->ci.central_header); + + if (err==ZIP_OK) + { + // Update the LocalFileHeader with the new values. + + ZPOS64_T cur_pos_inzip = ZTELL64(zi->z_filefunc,zi->filestream); + + if (ZSEEK64(zi->z_filefunc,zi->filestream, zi->ci.pos_local_header + 14,ZLIB_FILEFUNC_SEEK_SET)!=0) + err = ZIP_ERRNO; + + if (err==ZIP_OK) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,crc32,4); /* crc 32, unknown */ + + if(uncompressed_size >= 0xffffffff || compressed_size >= 0xffffffff ) + { + if(zi->ci.pos_zip64extrainfo > 0) + { + // Update the size in the ZIP64 extended field. + if (ZSEEK64(zi->z_filefunc,zi->filestream, zi->ci.pos_zip64extrainfo + 4,ZLIB_FILEFUNC_SEEK_SET)!=0) + err = ZIP_ERRNO; + + if (err==ZIP_OK) /* compressed size, unknown */ + err = zip64local_putValue(&zi->z_filefunc, zi->filestream, uncompressed_size, 8); + + if (err==ZIP_OK) /* uncompressed size, unknown */ + err = zip64local_putValue(&zi->z_filefunc, zi->filestream, compressed_size, 8); + } + else + err = ZIP_BADZIPFILE; // Caller passed zip64 = 0, so no room for zip64 info -> fatal + } + else + { + if (err==ZIP_OK) /* compressed size, unknown */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,compressed_size,4); + + if (err==ZIP_OK) /* uncompressed size, unknown */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,uncompressed_size,4); + } + + if (ZSEEK64(zi->z_filefunc,zi->filestream, cur_pos_inzip,ZLIB_FILEFUNC_SEEK_SET)!=0) + err = ZIP_ERRNO; + } + + zi->number_entry ++; + zi->in_opened_file_inzip = 0; + + return err; +} + +extern int ZEXPORT zipCloseFileInZip (zipFile file) +{ + return zipCloseFileInZipRaw (file,0,0); +} + +local int Write_Zip64EndOfCentralDirectoryLocator(zip64_internal* zi, ZPOS64_T zip64eocd_pos_inzip) +{ + int err = ZIP_OK; + ZPOS64_T pos = zip64eocd_pos_inzip - zi->add_position_when_writing_offset; + + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)ZIP64ENDLOCHEADERMAGIC,4); + + /*num disks*/ + if (err==ZIP_OK) /* number of the disk with the start of the central directory */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0,4); + + /*relative offset*/ + if (err==ZIP_OK) /* Relative offset to the Zip64EndOfCentralDirectory */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream, pos,8); + + /*total disks*/ /* Do not support spawning of disk so always say 1 here*/ + if (err==ZIP_OK) /* number of the disk with the start of the central directory */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)1,4); + + return err; +} + +local int Write_Zip64EndOfCentralDirectoryRecord(zip64_internal* zi, uLong size_centraldir, ZPOS64_T centraldir_pos_inzip) +{ + int err = ZIP_OK; + + uLong Zip64DataSize = 44; + + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)ZIP64ENDHEADERMAGIC,4); + + if (err==ZIP_OK) /* size of this 'zip64 end of central directory' */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(ZPOS64_T)Zip64DataSize,8); // why ZPOS64_T of this ? + + if (err==ZIP_OK) /* version made by */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)45,2); + + if (err==ZIP_OK) /* version needed */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)45,2); + + if (err==ZIP_OK) /* number of this disk */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0,4); + + if (err==ZIP_OK) /* number of the disk with the start of the central directory */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0,4); + + if (err==ZIP_OK) /* total number of entries in the central dir on this disk */ + err = zip64local_putValue(&zi->z_filefunc, zi->filestream, zi->number_entry, 8); + + if (err==ZIP_OK) /* total number of entries in the central dir */ + err = zip64local_putValue(&zi->z_filefunc, zi->filestream, zi->number_entry, 8); + + if (err==ZIP_OK) /* size of the central directory */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(ZPOS64_T)size_centraldir,8); + + if (err==ZIP_OK) /* offset of start of central directory with respect to the starting disk number */ + { + ZPOS64_T pos = centraldir_pos_inzip - zi->add_position_when_writing_offset; + err = zip64local_putValue(&zi->z_filefunc,zi->filestream, (ZPOS64_T)pos,8); + } + return err; +} +local int Write_EndOfCentralDirectoryRecord(zip64_internal* zi, uLong size_centraldir, ZPOS64_T centraldir_pos_inzip) +{ + int err = ZIP_OK; + + /*signature*/ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)ENDHEADERMAGIC,4); + + if (err==ZIP_OK) /* number of this disk */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0,2); + + if (err==ZIP_OK) /* number of the disk with the start of the central directory */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0,2); + + if (err==ZIP_OK) /* total number of entries in the central dir on this disk */ + { + { + if(zi->number_entry >= 0xFFFF) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0xffff,2); // use value in ZIP64 record + else + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)zi->number_entry,2); + } + } + + if (err==ZIP_OK) /* total number of entries in the central dir */ + { + if(zi->number_entry >= 0xFFFF) + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)0xffff,2); // use value in ZIP64 record + else + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)zi->number_entry,2); + } + + if (err==ZIP_OK) /* size of the central directory */ + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)size_centraldir,4); + + if (err==ZIP_OK) /* offset of start of central directory with respect to the starting disk number */ + { + ZPOS64_T pos = centraldir_pos_inzip - zi->add_position_when_writing_offset; + if(pos >= 0xffffffff) + { + err = zip64local_putValue(&zi->z_filefunc,zi->filestream, (uLong)0xffffffff,4); + } + else + err = zip64local_putValue(&zi->z_filefunc,zi->filestream, (uLong)(centraldir_pos_inzip - zi->add_position_when_writing_offset),4); + } + + return err; +} + +local int Write_GlobalComment(zip64_internal* zi, const char* global_comment) +{ + int err = ZIP_OK; + uInt size_global_comment = 0; + + if(global_comment != NULL) + size_global_comment = (uInt)strlen(global_comment); + + err = zip64local_putValue(&zi->z_filefunc,zi->filestream,(uLong)size_global_comment,2); + + if (err == ZIP_OK && size_global_comment > 0) + { + if (ZWRITE64(zi->z_filefunc,zi->filestream, global_comment, size_global_comment) != size_global_comment) + err = ZIP_ERRNO; + } + return err; +} + +extern int ZEXPORT zipClose (zipFile file, const char* global_comment) +{ + zip64_internal* zi; + int err = 0; + uLong size_centraldir = 0; + ZPOS64_T centraldir_pos_inzip; + ZPOS64_T pos; + + if (file == NULL) + return ZIP_PARAMERROR; + + zi = (zip64_internal*)file; + + if (zi->in_opened_file_inzip == 1) + { + err = zipCloseFileInZip (file); + } + +#ifndef NO_ADDFILEINEXISTINGZIP + if (global_comment==NULL) + global_comment = zi->globalcomment; +#endif + + centraldir_pos_inzip = ZTELL64(zi->z_filefunc,zi->filestream); + + if (err==ZIP_OK) + { + linkedlist_datablock_internal* ldi = zi->central_dir.first_block; + while (ldi!=NULL) + { + if ((err==ZIP_OK) && (ldi->filled_in_this_block>0)) + { + if (ZWRITE64(zi->z_filefunc,zi->filestream, ldi->data, ldi->filled_in_this_block) != ldi->filled_in_this_block) + err = ZIP_ERRNO; + } + + size_centraldir += ldi->filled_in_this_block; + ldi = ldi->next_datablock; + } + } + free_linkedlist(&(zi->central_dir)); + + pos = centraldir_pos_inzip - zi->add_position_when_writing_offset; + if(pos >= 0xffffffff || zi->number_entry > 0xFFFF) + { + ZPOS64_T Zip64EOCDpos = ZTELL64(zi->z_filefunc,zi->filestream); + Write_Zip64EndOfCentralDirectoryRecord(zi, size_centraldir, centraldir_pos_inzip); + + Write_Zip64EndOfCentralDirectoryLocator(zi, Zip64EOCDpos); + } + + if (err==ZIP_OK) + err = Write_EndOfCentralDirectoryRecord(zi, size_centraldir, centraldir_pos_inzip); + + if(err == ZIP_OK) + err = Write_GlobalComment(zi, global_comment); + + if (ZCLOSE64(zi->z_filefunc,zi->filestream) != 0) + if (err == ZIP_OK) + err = ZIP_ERRNO; + +#ifndef NO_ADDFILEINEXISTINGZIP + TRYFREE(zi->globalcomment); +#endif + TRYFREE(zi); + + return err; +} + +extern int ZEXPORT zipRemoveExtraInfoBlock (char* pData, int* dataLen, short sHeader) +{ + char* p = pData; + int size = 0; + char* pNewHeader; + char* pTmp; + short header; + short dataSize; + + int retVal = ZIP_OK; + + if(pData == NULL || dataLen == NULL || *dataLen < 4) + return ZIP_PARAMERROR; + + pNewHeader = (char*)ALLOC((unsigned)*dataLen); + pTmp = pNewHeader; + + while(p < (pData + *dataLen)) + { + header = *(short*)p; + dataSize = *(((short*)p)+1); + + if( header == sHeader ) // Header found. + { + p += dataSize + 4; // skip it. do not copy to temp buffer + } + else + { + // Extra Info block should not be removed, So copy it to the temp buffer. + memcpy(pTmp, p, dataSize + 4); + p += dataSize + 4; + size += dataSize + 4; + } + + } + + if(size < *dataLen) + { + // clean old extra info block. + memset(pData,0, *dataLen); + + // copy the new extra info block over the old + if(size > 0) + memcpy(pData, pNewHeader, size); + + // set the new extra info size + *dataLen = size; + + retVal = ZIP_OK; + } + else + retVal = ZIP_ERRNO; + + TRYFREE(pNewHeader); + + return retVal; +} diff --git a/lib/os/windows/zlib-1.2.13/contrib/minizip/zip.h b/lib/os/windows/zlib-1.2.13/contrib/minizip/zip.h new file mode 100644 index 000000000000..7e4509d77bee --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/contrib/minizip/zip.h @@ -0,0 +1,367 @@ +/* zip.h -- IO on .zip files using zlib + Version 1.1, February 14h, 2010 + part of the MiniZip project - ( http://www.winimage.com/zLibDll/minizip.html ) + + Copyright (C) 1998-2010 Gilles Vollant (minizip) ( http://www.winimage.com/zLibDll/minizip.html ) + + Modifications for Zip64 support + Copyright (C) 2009-2010 Mathias Svensson ( http://result42.com ) + + For more info read MiniZip_info.txt + + --------------------------------------------------------------------------- + + Condition of use and distribution are the same than zlib : + + This software is provided 'as-is', without any express or implied + warranty. In no event will the authors be held liable for any damages + arising from the use of this software. + + Permission is granted to anyone to use this software for any purpose, + including commercial applications, and to alter it and redistribute it + freely, subject to the following restrictions: + + 1. The origin of this software must not be misrepresented; you must not + claim that you wrote the original software. If you use this software + in a product, an acknowledgment in the product documentation would be + appreciated but is not required. + 2. Altered source versions must be plainly marked as such, and must not be + misrepresented as being the original software. + 3. This notice may not be removed or altered from any source distribution. + + --------------------------------------------------------------------------- + + Changes + + See header of zip.h + +*/ + +#ifndef _zip12_H +#define _zip12_H + +#ifdef __cplusplus +extern "C" { +#endif + +//#define HAVE_BZIP2 + +#ifndef _ZLIB_H +#include "zlib.h" +#endif + +#ifndef _ZLIBIOAPI_H +#include "ioapi.h" +#endif + +#ifdef HAVE_BZIP2 +#include "bzlib.h" +#endif + +#define Z_BZIP2ED 12 + +#if defined(STRICTZIP) || defined(STRICTZIPUNZIP) +/* like the STRICT of WIN32, we define a pointer that cannot be converted + from (void*) without cast */ +typedef struct TagzipFile__ { int unused; } zipFile__; +typedef zipFile__ *zipFile; +#else +typedef voidp zipFile; +#endif + +#define ZIP_OK (0) +#define ZIP_EOF (0) +#define ZIP_ERRNO (Z_ERRNO) +#define ZIP_PARAMERROR (-102) +#define ZIP_BADZIPFILE (-103) +#define ZIP_INTERNALERROR (-104) + +#ifndef DEF_MEM_LEVEL +# if MAX_MEM_LEVEL >= 8 +# define DEF_MEM_LEVEL 8 +# else +# define DEF_MEM_LEVEL MAX_MEM_LEVEL +# endif +#endif +/* default memLevel */ + +/* tm_zip contain date/time info */ +typedef struct tm_zip_s +{ + int tm_sec; /* seconds after the minute - [0,59] */ + int tm_min; /* minutes after the hour - [0,59] */ + int tm_hour; /* hours since midnight - [0,23] */ + int tm_mday; /* day of the month - [1,31] */ + int tm_mon; /* months since January - [0,11] */ + int tm_year; /* years - [1980..2044] */ +} tm_zip; + +typedef struct +{ + tm_zip tmz_date; /* date in understandable format */ + uLong dosDate; /* if dos_date == 0, tmu_date is used */ +/* uLong flag; */ /* general purpose bit flag 2 bytes */ + + uLong internal_fa; /* internal file attributes 2 bytes */ + uLong external_fa; /* external file attributes 4 bytes */ +} zip_fileinfo; + +typedef const char* zipcharpc; + + +#define APPEND_STATUS_CREATE (0) +#define APPEND_STATUS_CREATEAFTER (1) +#define APPEND_STATUS_ADDINZIP (2) + +extern zipFile ZEXPORT zipOpen OF((const char *pathname, int append)); +extern zipFile ZEXPORT zipOpen64 OF((const void *pathname, int append)); +/* + Create a zipfile. + pathname contain on Windows XP a filename like "c:\\zlib\\zlib113.zip" or on + an Unix computer "zlib/zlib113.zip". + if the file pathname exist and append==APPEND_STATUS_CREATEAFTER, the zip + will be created at the end of the file. + (useful if the file contain a self extractor code) + if the file pathname exist and append==APPEND_STATUS_ADDINZIP, we will + add files in existing zip (be sure you don't add file that doesn't exist) + If the zipfile cannot be opened, the return value is NULL. + Else, the return value is a zipFile Handle, usable with other function + of this zip package. +*/ + +/* Note : there is no delete function into a zipfile. + If you want delete file into a zipfile, you must open a zipfile, and create another + Of couse, you can use RAW reading and writing to copy the file you did not want delte +*/ + +extern zipFile ZEXPORT zipOpen2 OF((const char *pathname, + int append, + zipcharpc* globalcomment, + zlib_filefunc_def* pzlib_filefunc_def)); + +extern zipFile ZEXPORT zipOpen2_64 OF((const void *pathname, + int append, + zipcharpc* globalcomment, + zlib_filefunc64_def* pzlib_filefunc_def)); + +extern zipFile ZEXPORT zipOpen3 OF((const void *pathname, + int append, + zipcharpc* globalcomment, + zlib_filefunc64_32_def* pzlib_filefunc64_32_def)); + +extern int ZEXPORT zipOpenNewFileInZip OF((zipFile file, + const char* filename, + const zip_fileinfo* zipfi, + const void* extrafield_local, + uInt size_extrafield_local, + const void* extrafield_global, + uInt size_extrafield_global, + const char* comment, + int method, + int level)); + +extern int ZEXPORT zipOpenNewFileInZip64 OF((zipFile file, + const char* filename, + const zip_fileinfo* zipfi, + const void* extrafield_local, + uInt size_extrafield_local, + const void* extrafield_global, + uInt size_extrafield_global, + const char* comment, + int method, + int level, + int zip64)); + +/* + Open a file in the ZIP for writing. + filename : the filename in zip (if NULL, '-' without quote will be used + *zipfi contain supplemental information + if extrafield_local!=NULL and size_extrafield_local>0, extrafield_local + contains the extrafield data the the local header + if extrafield_global!=NULL and size_extrafield_global>0, extrafield_global + contains the extrafield data the the local header + if comment != NULL, comment contain the comment string + method contain the compression method (0 for store, Z_DEFLATED for deflate) + level contain the level of compression (can be Z_DEFAULT_COMPRESSION) + zip64 is set to 1 if a zip64 extended information block should be added to the local file header. + this MUST be '1' if the uncompressed size is >= 0xffffffff. + +*/ + + +extern int ZEXPORT zipOpenNewFileInZip2 OF((zipFile file, + const char* filename, + const zip_fileinfo* zipfi, + const void* extrafield_local, + uInt size_extrafield_local, + const void* extrafield_global, + uInt size_extrafield_global, + const char* comment, + int method, + int level, + int raw)); + + +extern int ZEXPORT zipOpenNewFileInZip2_64 OF((zipFile file, + const char* filename, + const zip_fileinfo* zipfi, + const void* extrafield_local, + uInt size_extrafield_local, + const void* extrafield_global, + uInt size_extrafield_global, + const char* comment, + int method, + int level, + int raw, + int zip64)); +/* + Same than zipOpenNewFileInZip, except if raw=1, we write raw file + */ + +extern int ZEXPORT zipOpenNewFileInZip3 OF((zipFile file, + const char* filename, + const zip_fileinfo* zipfi, + const void* extrafield_local, + uInt size_extrafield_local, + const void* extrafield_global, + uInt size_extrafield_global, + const char* comment, + int method, + int level, + int raw, + int windowBits, + int memLevel, + int strategy, + const char* password, + uLong crcForCrypting)); + +extern int ZEXPORT zipOpenNewFileInZip3_64 OF((zipFile file, + const char* filename, + const zip_fileinfo* zipfi, + const void* extrafield_local, + uInt size_extrafield_local, + const void* extrafield_global, + uInt size_extrafield_global, + const char* comment, + int method, + int level, + int raw, + int windowBits, + int memLevel, + int strategy, + const char* password, + uLong crcForCrypting, + int zip64 + )); + +/* + Same than zipOpenNewFileInZip2, except + windowBits,memLevel,,strategy : see parameter strategy in deflateInit2 + password : crypting password (NULL for no crypting) + crcForCrypting : crc of file to compress (needed for crypting) + */ + +extern int ZEXPORT zipOpenNewFileInZip4 OF((zipFile file, + const char* filename, + const zip_fileinfo* zipfi, + const void* extrafield_local, + uInt size_extrafield_local, + const void* extrafield_global, + uInt size_extrafield_global, + const char* comment, + int method, + int level, + int raw, + int windowBits, + int memLevel, + int strategy, + const char* password, + uLong crcForCrypting, + uLong versionMadeBy, + uLong flagBase + )); + + +extern int ZEXPORT zipOpenNewFileInZip4_64 OF((zipFile file, + const char* filename, + const zip_fileinfo* zipfi, + const void* extrafield_local, + uInt size_extrafield_local, + const void* extrafield_global, + uInt size_extrafield_global, + const char* comment, + int method, + int level, + int raw, + int windowBits, + int memLevel, + int strategy, + const char* password, + uLong crcForCrypting, + uLong versionMadeBy, + uLong flagBase, + int zip64 + )); +/* + Same than zipOpenNewFileInZip4, except + versionMadeBy : value for Version made by field + flag : value for flag field (compression level info will be added) + */ + + +extern int ZEXPORT zipWriteInFileInZip OF((zipFile file, + const void* buf, + unsigned len)); +/* + Write data in the zipfile +*/ + +extern int ZEXPORT zipCloseFileInZip OF((zipFile file)); +/* + Close the current file in the zipfile +*/ + +extern int ZEXPORT zipCloseFileInZipRaw OF((zipFile file, + uLong uncompressed_size, + uLong crc32)); + +extern int ZEXPORT zipCloseFileInZipRaw64 OF((zipFile file, + ZPOS64_T uncompressed_size, + uLong crc32)); + +/* + Close the current file in the zipfile, for file opened with + parameter raw=1 in zipOpenNewFileInZip2 + uncompressed_size and crc32 are value for the uncompressed size +*/ + +extern int ZEXPORT zipClose OF((zipFile file, + const char* global_comment)); +/* + Close the zipfile +*/ + + +extern int ZEXPORT zipRemoveExtraInfoBlock OF((char* pData, int* dataLen, short sHeader)); +/* + zipRemoveExtraInfoBlock - Added by Mathias Svensson + + Remove extra information block from a extra information data for the local file header or central directory header + + It is needed to remove ZIP64 extra information blocks when before data is written if using RAW mode. + + 0x0001 is the signature header for the ZIP64 extra information blocks + + usage. + Remove ZIP64 Extra information from a central director extra field data + zipRemoveExtraInfoBlock(pCenDirExtraFieldData, &nCenDirExtraFieldDataLen, 0x0001); + + Remove ZIP64 Extra information from a Local File Header extra field data + zipRemoveExtraInfoBlock(pLocalHeaderExtraFieldData, &nLocalHeaderExtraFieldDataLen, 0x0001); +*/ + +#ifdef __cplusplus +} +#endif + +#endif /* _zip64_H */ diff --git a/lib/os/windows/zlib-1.2.13/crc32.c b/lib/os/windows/zlib-1.2.13/crc32.c new file mode 100644 index 000000000000..f8357b083f76 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/crc32.c @@ -0,0 +1,1125 @@ +/* crc32.c -- compute the CRC-32 of a data stream + * Copyright (C) 1995-2022 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + * + * This interleaved implementation of a CRC makes use of pipelined multiple + * arithmetic-logic units, commonly found in modern CPU cores. It is due to + * Kadatch and Jenkins (2010). See doc/crc-doc.1.0.pdf in this distribution. + */ + +/* @(#) $Id$ */ + +/* + Note on the use of DYNAMIC_CRC_TABLE: there is no mutex or semaphore + protection on the static variables used to control the first-use generation + of the crc tables. Therefore, if you #define DYNAMIC_CRC_TABLE, you should + first call get_crc_table() to initialize the tables before allowing more than + one thread to use crc32(). + + MAKECRCH can be #defined to write out crc32.h. A main() routine is also + produced, so that this one source file can be compiled to an executable. + */ + +#ifdef MAKECRCH +# include +# ifndef DYNAMIC_CRC_TABLE +# define DYNAMIC_CRC_TABLE +# endif /* !DYNAMIC_CRC_TABLE */ +#endif /* MAKECRCH */ + +#include "zutil.h" /* for Z_U4, Z_U8, z_crc_t, and FAR definitions */ + + /* + A CRC of a message is computed on N braids of words in the message, where + each word consists of W bytes (4 or 8). If N is 3, for example, then three + running sparse CRCs are calculated respectively on each braid, at these + indices in the array of words: 0, 3, 6, ..., 1, 4, 7, ..., and 2, 5, 8, ... + This is done starting at a word boundary, and continues until as many blocks + of N * W bytes as are available have been processed. The results are combined + into a single CRC at the end. For this code, N must be in the range 1..6 and + W must be 4 or 8. The upper limit on N can be increased if desired by adding + more #if blocks, extending the patterns apparent in the code. In addition, + crc32.h would need to be regenerated, if the maximum N value is increased. + + N and W are chosen empirically by benchmarking the execution time on a given + processor. The choices for N and W below were based on testing on Intel Kaby + Lake i7, AMD Ryzen 7, ARM Cortex-A57, Sparc64-VII, PowerPC POWER9, and MIPS64 + Octeon II processors. The Intel, AMD, and ARM processors were all fastest + with N=5, W=8. The Sparc, PowerPC, and MIPS64 were all fastest at N=5, W=4. + They were all tested with either gcc or clang, all using the -O3 optimization + level. Your mileage may vary. + */ + +/* Define N */ +#ifdef Z_TESTN +# define N Z_TESTN +#else +# define N 5 +#endif +#if N < 1 || N > 6 +# error N must be in 1..6 +#endif + +/* + z_crc_t must be at least 32 bits. z_word_t must be at least as long as + z_crc_t. It is assumed here that z_word_t is either 32 bits or 64 bits, and + that bytes are eight bits. + */ + +/* + Define W and the associated z_word_t type. If W is not defined, then a + braided calculation is not used, and the associated tables and code are not + compiled. + */ +#ifdef Z_TESTW +# if Z_TESTW-1 != -1 +# define W Z_TESTW +# endif +#else +# ifdef MAKECRCH +# define W 8 /* required for MAKECRCH */ +# else +# if defined(__x86_64__) || defined(__aarch64__) +# define W 8 +# else +# define W 4 +# endif +# endif +#endif +#ifdef W +# if W == 8 && defined(Z_U8) + typedef Z_U8 z_word_t; +# elif defined(Z_U4) +# undef W +# define W 4 + typedef Z_U4 z_word_t; +# else +# undef W +# endif +#endif + +/* If available, use the ARM processor CRC32 instruction. */ +#if defined(__aarch64__) && defined(__ARM_FEATURE_CRC32) && W == 8 +# define ARMCRC32 +#endif + +/* Local functions. */ +local z_crc_t multmodp OF((z_crc_t a, z_crc_t b)); +local z_crc_t x2nmodp OF((z_off64_t n, unsigned k)); + +#if defined(W) && (!defined(ARMCRC32) || defined(DYNAMIC_CRC_TABLE)) + local z_word_t byte_swap OF((z_word_t word)); +#endif + +#if defined(W) && !defined(ARMCRC32) + local z_crc_t crc_word OF((z_word_t data)); + local z_word_t crc_word_big OF((z_word_t data)); +#endif + +#if defined(W) && (!defined(ARMCRC32) || defined(DYNAMIC_CRC_TABLE)) +/* + Swap the bytes in a z_word_t to convert between little and big endian. Any + self-respecting compiler will optimize this to a single machine byte-swap + instruction, if one is available. This assumes that word_t is either 32 bits + or 64 bits. + */ +local z_word_t byte_swap(word) + z_word_t word; +{ +# if W == 8 + return + (word & 0xff00000000000000) >> 56 | + (word & 0xff000000000000) >> 40 | + (word & 0xff0000000000) >> 24 | + (word & 0xff00000000) >> 8 | + (word & 0xff000000) << 8 | + (word & 0xff0000) << 24 | + (word & 0xff00) << 40 | + (word & 0xff) << 56; +# else /* W == 4 */ + return + (word & 0xff000000) >> 24 | + (word & 0xff0000) >> 8 | + (word & 0xff00) << 8 | + (word & 0xff) << 24; +# endif +} +#endif + +/* CRC polynomial. */ +#define POLY 0xedb88320 /* p(x) reflected, with x^32 implied */ + +#ifdef DYNAMIC_CRC_TABLE + +local z_crc_t FAR crc_table[256]; +local z_crc_t FAR x2n_table[32]; +local void make_crc_table OF((void)); +#ifdef W + local z_word_t FAR crc_big_table[256]; + local z_crc_t FAR crc_braid_table[W][256]; + local z_word_t FAR crc_braid_big_table[W][256]; + local void braid OF((z_crc_t [][256], z_word_t [][256], int, int)); +#endif +#ifdef MAKECRCH + local void write_table OF((FILE *, const z_crc_t FAR *, int)); + local void write_table32hi OF((FILE *, const z_word_t FAR *, int)); + local void write_table64 OF((FILE *, const z_word_t FAR *, int)); +#endif /* MAKECRCH */ + +/* + Define a once() function depending on the availability of atomics. If this is + compiled with DYNAMIC_CRC_TABLE defined, and if CRCs will be computed in + multiple threads, and if atomics are not available, then get_crc_table() must + be called to initialize the tables and must return before any threads are + allowed to compute or combine CRCs. + */ + +/* Definition of once functionality. */ +typedef struct once_s once_t; +local void once OF((once_t *, void (*)(void))); + +/* Check for the availability of atomics. */ +#if defined(__STDC__) && __STDC_VERSION__ >= 201112L && \ + !defined(__STDC_NO_ATOMICS__) + +#include + +/* Structure for once(), which must be initialized with ONCE_INIT. */ +struct once_s { + atomic_flag begun; + atomic_int done; +}; +#define ONCE_INIT {ATOMIC_FLAG_INIT, 0} + +/* + Run the provided init() function exactly once, even if multiple threads + invoke once() at the same time. The state must be a once_t initialized with + ONCE_INIT. + */ +local void once(state, init) + once_t *state; + void (*init)(void); +{ + if (!atomic_load(&state->done)) { + if (atomic_flag_test_and_set(&state->begun)) + while (!atomic_load(&state->done)) + ; + else { + init(); + atomic_store(&state->done, 1); + } + } +} + +#else /* no atomics */ + +/* Structure for once(), which must be initialized with ONCE_INIT. */ +struct once_s { + volatile int begun; + volatile int done; +}; +#define ONCE_INIT {0, 0} + +/* Test and set. Alas, not atomic, but tries to minimize the period of + vulnerability. */ +local int test_and_set OF((int volatile *)); +local int test_and_set(flag) + int volatile *flag; +{ + int was; + + was = *flag; + *flag = 1; + return was; +} + +/* Run the provided init() function once. This is not thread-safe. */ +local void once(state, init) + once_t *state; + void (*init)(void); +{ + if (!state->done) { + if (test_and_set(&state->begun)) + while (!state->done) + ; + else { + init(); + state->done = 1; + } + } +} + +#endif + +/* State for once(). */ +local once_t made = ONCE_INIT; + +/* + Generate tables for a byte-wise 32-bit CRC calculation on the polynomial: + x^32+x^26+x^23+x^22+x^16+x^12+x^11+x^10+x^8+x^7+x^5+x^4+x^2+x+1. + + Polynomials over GF(2) are represented in binary, one bit per coefficient, + with the lowest powers in the most significant bit. Then adding polynomials + is just exclusive-or, and multiplying a polynomial by x is a right shift by + one. If we call the above polynomial p, and represent a byte as the + polynomial q, also with the lowest power in the most significant bit (so the + byte 0xb1 is the polynomial x^7+x^3+x^2+1), then the CRC is (q*x^32) mod p, + where a mod b means the remainder after dividing a by b. + + This calculation is done using the shift-register method of multiplying and + taking the remainder. The register is initialized to zero, and for each + incoming bit, x^32 is added mod p to the register if the bit is a one (where + x^32 mod p is p+x^32 = x^26+...+1), and the register is multiplied mod p by x + (which is shifting right by one and adding x^32 mod p if the bit shifted out + is a one). We start with the highest power (least significant bit) of q and + repeat for all eight bits of q. + + The table is simply the CRC of all possible eight bit values. This is all the + information needed to generate CRCs on data a byte at a time for all + combinations of CRC register values and incoming bytes. + */ + +local void make_crc_table() +{ + unsigned i, j, n; + z_crc_t p; + + /* initialize the CRC of bytes tables */ + for (i = 0; i < 256; i++) { + p = i; + for (j = 0; j < 8; j++) + p = p & 1 ? (p >> 1) ^ POLY : p >> 1; + crc_table[i] = p; +#ifdef W + crc_big_table[i] = byte_swap(p); +#endif + } + + /* initialize the x^2^n mod p(x) table */ + p = (z_crc_t)1 << 30; /* x^1 */ + x2n_table[0] = p; + for (n = 1; n < 32; n++) + x2n_table[n] = p = multmodp(p, p); + +#ifdef W + /* initialize the braiding tables -- needs x2n_table[] */ + braid(crc_braid_table, crc_braid_big_table, N, W); +#endif + +#ifdef MAKECRCH + { + /* + The crc32.h header file contains tables for both 32-bit and 64-bit + z_word_t's, and so requires a 64-bit type be available. In that case, + z_word_t must be defined to be 64-bits. This code then also generates + and writes out the tables for the case that z_word_t is 32 bits. + */ +#if !defined(W) || W != 8 +# error Need a 64-bit integer type in order to generate crc32.h. +#endif + FILE *out; + int k, n; + z_crc_t ltl[8][256]; + z_word_t big[8][256]; + + out = fopen("crc32.h", "w"); + if (out == NULL) return; + + /* write out little-endian CRC table to crc32.h */ + fprintf(out, + "/* crc32.h -- tables for rapid CRC calculation\n" + " * Generated automatically by crc32.c\n */\n" + "\n" + "local const z_crc_t FAR crc_table[] = {\n" + " "); + write_table(out, crc_table, 256); + fprintf(out, + "};\n"); + + /* write out big-endian CRC table for 64-bit z_word_t to crc32.h */ + fprintf(out, + "\n" + "#ifdef W\n" + "\n" + "#if W == 8\n" + "\n" + "local const z_word_t FAR crc_big_table[] = {\n" + " "); + write_table64(out, crc_big_table, 256); + fprintf(out, + "};\n"); + + /* write out big-endian CRC table for 32-bit z_word_t to crc32.h */ + fprintf(out, + "\n" + "#else /* W == 4 */\n" + "\n" + "local const z_word_t FAR crc_big_table[] = {\n" + " "); + write_table32hi(out, crc_big_table, 256); + fprintf(out, + "};\n" + "\n" + "#endif\n"); + + /* write out braid tables for each value of N */ + for (n = 1; n <= 6; n++) { + fprintf(out, + "\n" + "#if N == %d\n", n); + + /* compute braid tables for this N and 64-bit word_t */ + braid(ltl, big, n, 8); + + /* write out braid tables for 64-bit z_word_t to crc32.h */ + fprintf(out, + "\n" + "#if W == 8\n" + "\n" + "local const z_crc_t FAR crc_braid_table[][256] = {\n"); + for (k = 0; k < 8; k++) { + fprintf(out, " {"); + write_table(out, ltl[k], 256); + fprintf(out, "}%s", k < 7 ? ",\n" : ""); + } + fprintf(out, + "};\n" + "\n" + "local const z_word_t FAR crc_braid_big_table[][256] = {\n"); + for (k = 0; k < 8; k++) { + fprintf(out, " {"); + write_table64(out, big[k], 256); + fprintf(out, "}%s", k < 7 ? ",\n" : ""); + } + fprintf(out, + "};\n"); + + /* compute braid tables for this N and 32-bit word_t */ + braid(ltl, big, n, 4); + + /* write out braid tables for 32-bit z_word_t to crc32.h */ + fprintf(out, + "\n" + "#else /* W == 4 */\n" + "\n" + "local const z_crc_t FAR crc_braid_table[][256] = {\n"); + for (k = 0; k < 4; k++) { + fprintf(out, " {"); + write_table(out, ltl[k], 256); + fprintf(out, "}%s", k < 3 ? ",\n" : ""); + } + fprintf(out, + "};\n" + "\n" + "local const z_word_t FAR crc_braid_big_table[][256] = {\n"); + for (k = 0; k < 4; k++) { + fprintf(out, " {"); + write_table32hi(out, big[k], 256); + fprintf(out, "}%s", k < 3 ? ",\n" : ""); + } + fprintf(out, + "};\n" + "\n" + "#endif\n" + "\n" + "#endif\n"); + } + fprintf(out, + "\n" + "#endif\n"); + + /* write out zeros operator table to crc32.h */ + fprintf(out, + "\n" + "local const z_crc_t FAR x2n_table[] = {\n" + " "); + write_table(out, x2n_table, 32); + fprintf(out, + "};\n"); + fclose(out); + } +#endif /* MAKECRCH */ +} + +#ifdef MAKECRCH + +/* + Write the 32-bit values in table[0..k-1] to out, five per line in + hexadecimal separated by commas. + */ +local void write_table(out, table, k) + FILE *out; + const z_crc_t FAR *table; + int k; +{ + int n; + + for (n = 0; n < k; n++) + fprintf(out, "%s0x%08lx%s", n == 0 || n % 5 ? "" : " ", + (unsigned long)(table[n]), + n == k - 1 ? "" : (n % 5 == 4 ? ",\n" : ", ")); +} + +/* + Write the high 32-bits of each value in table[0..k-1] to out, five per line + in hexadecimal separated by commas. + */ +local void write_table32hi(out, table, k) +FILE *out; +const z_word_t FAR *table; +int k; +{ + int n; + + for (n = 0; n < k; n++) + fprintf(out, "%s0x%08lx%s", n == 0 || n % 5 ? "" : " ", + (unsigned long)(table[n] >> 32), + n == k - 1 ? "" : (n % 5 == 4 ? ",\n" : ", ")); +} + +/* + Write the 64-bit values in table[0..k-1] to out, three per line in + hexadecimal separated by commas. This assumes that if there is a 64-bit + type, then there is also a long long integer type, and it is at least 64 + bits. If not, then the type cast and format string can be adjusted + accordingly. + */ +local void write_table64(out, table, k) + FILE *out; + const z_word_t FAR *table; + int k; +{ + int n; + + for (n = 0; n < k; n++) + fprintf(out, "%s0x%016llx%s", n == 0 || n % 3 ? "" : " ", + (unsigned long long)(table[n]), + n == k - 1 ? "" : (n % 3 == 2 ? ",\n" : ", ")); +} + +/* Actually do the deed. */ +int main() +{ + make_crc_table(); + return 0; +} + +#endif /* MAKECRCH */ + +#ifdef W +/* + Generate the little and big-endian braid tables for the given n and z_word_t + size w. Each array must have room for w blocks of 256 elements. + */ +local void braid(ltl, big, n, w) + z_crc_t ltl[][256]; + z_word_t big[][256]; + int n; + int w; +{ + int k; + z_crc_t i, p, q; + for (k = 0; k < w; k++) { + p = x2nmodp((n * w + 3 - k) << 3, 0); + ltl[k][0] = 0; + big[w - 1 - k][0] = 0; + for (i = 1; i < 256; i++) { + ltl[k][i] = q = multmodp(i << 24, p); + big[w - 1 - k][i] = byte_swap(q); + } + } +} +#endif + +#else /* !DYNAMIC_CRC_TABLE */ +/* ======================================================================== + * Tables for byte-wise and braided CRC-32 calculations, and a table of powers + * of x for combining CRC-32s, all made by make_crc_table(). + */ +#include "crc32.h" +#endif /* DYNAMIC_CRC_TABLE */ + +/* ======================================================================== + * Routines used for CRC calculation. Some are also required for the table + * generation above. + */ + +/* + Return a(x) multiplied by b(x) modulo p(x), where p(x) is the CRC polynomial, + reflected. For speed, this requires that a not be zero. + */ +local z_crc_t multmodp(a, b) + z_crc_t a; + z_crc_t b; +{ + z_crc_t m, p; + + m = (z_crc_t)1 << 31; + p = 0; + for (;;) { + if (a & m) { + p ^= b; + if ((a & (m - 1)) == 0) + break; + } + m >>= 1; + b = b & 1 ? (b >> 1) ^ POLY : b >> 1; + } + return p; +} + +/* + Return x^(n * 2^k) modulo p(x). Requires that x2n_table[] has been + initialized. + */ +local z_crc_t x2nmodp(n, k) + z_off64_t n; + unsigned k; +{ + z_crc_t p; + + p = (z_crc_t)1 << 31; /* x^0 == 1 */ + while (n) { + if (n & 1) + p = multmodp(x2n_table[k & 31], p); + n >>= 1; + k++; + } + return p; +} + +/* ========================================================================= + * This function can be used by asm versions of crc32(), and to force the + * generation of the CRC tables in a threaded application. + */ +const z_crc_t FAR * ZEXPORT get_crc_table() +{ +#ifdef DYNAMIC_CRC_TABLE + once(&made, make_crc_table); +#endif /* DYNAMIC_CRC_TABLE */ + return (const z_crc_t FAR *)crc_table; +} + +/* ========================================================================= + * Use ARM machine instructions if available. This will compute the CRC about + * ten times faster than the braided calculation. This code does not check for + * the presence of the CRC instruction at run time. __ARM_FEATURE_CRC32 will + * only be defined if the compilation specifies an ARM processor architecture + * that has the instructions. For example, compiling with -march=armv8.1-a or + * -march=armv8-a+crc, or -march=native if the compile machine has the crc32 + * instructions. + */ +#ifdef ARMCRC32 + +/* + Constants empirically determined to maximize speed. These values are from + measurements on a Cortex-A57. Your mileage may vary. + */ +#define Z_BATCH 3990 /* number of words in a batch */ +#define Z_BATCH_ZEROS 0xa10d3d0c /* computed from Z_BATCH = 3990 */ +#define Z_BATCH_MIN 800 /* fewest words in a final batch */ + +unsigned long ZEXPORT crc32_z(crc, buf, len) + unsigned long crc; + const unsigned char FAR *buf; + z_size_t len; +{ + z_crc_t val; + z_word_t crc1, crc2; + const z_word_t *word; + z_word_t val0, val1, val2; + z_size_t last, last2, i; + z_size_t num; + + /* Return initial CRC, if requested. */ + if (buf == Z_NULL) return 0; + +#ifdef DYNAMIC_CRC_TABLE + once(&made, make_crc_table); +#endif /* DYNAMIC_CRC_TABLE */ + + /* Pre-condition the CRC */ + crc = (~crc) & 0xffffffff; + + /* Compute the CRC up to a word boundary. */ + while (len && ((z_size_t)buf & 7) != 0) { + len--; + val = *buf++; + __asm__ volatile("crc32b %w0, %w0, %w1" : "+r"(crc) : "r"(val)); + } + + /* Prepare to compute the CRC on full 64-bit words word[0..num-1]. */ + word = (z_word_t const *)buf; + num = len >> 3; + len &= 7; + + /* Do three interleaved CRCs to realize the throughput of one crc32x + instruction per cycle. Each CRC is calculated on Z_BATCH words. The + three CRCs are combined into a single CRC after each set of batches. */ + while (num >= 3 * Z_BATCH) { + crc1 = 0; + crc2 = 0; + for (i = 0; i < Z_BATCH; i++) { + val0 = word[i]; + val1 = word[i + Z_BATCH]; + val2 = word[i + 2 * Z_BATCH]; + __asm__ volatile("crc32x %w0, %w0, %x1" : "+r"(crc) : "r"(val0)); + __asm__ volatile("crc32x %w0, %w0, %x1" : "+r"(crc1) : "r"(val1)); + __asm__ volatile("crc32x %w0, %w0, %x1" : "+r"(crc2) : "r"(val2)); + } + word += 3 * Z_BATCH; + num -= 3 * Z_BATCH; + crc = multmodp(Z_BATCH_ZEROS, crc) ^ crc1; + crc = multmodp(Z_BATCH_ZEROS, crc) ^ crc2; + } + + /* Do one last smaller batch with the remaining words, if there are enough + to pay for the combination of CRCs. */ + last = num / 3; + if (last >= Z_BATCH_MIN) { + last2 = last << 1; + crc1 = 0; + crc2 = 0; + for (i = 0; i < last; i++) { + val0 = word[i]; + val1 = word[i + last]; + val2 = word[i + last2]; + __asm__ volatile("crc32x %w0, %w0, %x1" : "+r"(crc) : "r"(val0)); + __asm__ volatile("crc32x %w0, %w0, %x1" : "+r"(crc1) : "r"(val1)); + __asm__ volatile("crc32x %w0, %w0, %x1" : "+r"(crc2) : "r"(val2)); + } + word += 3 * last; + num -= 3 * last; + val = x2nmodp(last, 6); + crc = multmodp(val, crc) ^ crc1; + crc = multmodp(val, crc) ^ crc2; + } + + /* Compute the CRC on any remaining words. */ + for (i = 0; i < num; i++) { + val0 = word[i]; + __asm__ volatile("crc32x %w0, %w0, %x1" : "+r"(crc) : "r"(val0)); + } + word += num; + + /* Complete the CRC on any remaining bytes. */ + buf = (const unsigned char FAR *)word; + while (len) { + len--; + val = *buf++; + __asm__ volatile("crc32b %w0, %w0, %w1" : "+r"(crc) : "r"(val)); + } + + /* Return the CRC, post-conditioned. */ + return crc ^ 0xffffffff; +} + +#else + +#ifdef W + +/* + Return the CRC of the W bytes in the word_t data, taking the + least-significant byte of the word as the first byte of data, without any pre + or post conditioning. This is used to combine the CRCs of each braid. + */ +local z_crc_t crc_word(data) + z_word_t data; +{ + int k; + for (k = 0; k < W; k++) + data = (data >> 8) ^ crc_table[data & 0xff]; + return (z_crc_t)data; +} + +local z_word_t crc_word_big(data) + z_word_t data; +{ + int k; + for (k = 0; k < W; k++) + data = (data << 8) ^ + crc_big_table[(data >> ((W - 1) << 3)) & 0xff]; + return data; +} + +#endif + +/* ========================================================================= */ +unsigned long ZEXPORT crc32_z(crc, buf, len) + unsigned long crc; + const unsigned char FAR *buf; + z_size_t len; +{ + /* Return initial CRC, if requested. */ + if (buf == Z_NULL) return 0; + +#ifdef DYNAMIC_CRC_TABLE + once(&made, make_crc_table); +#endif /* DYNAMIC_CRC_TABLE */ + + /* Pre-condition the CRC */ + crc = (~crc) & 0xffffffff; + +#ifdef W + + /* If provided enough bytes, do a braided CRC calculation. */ + if (len >= N * W + W - 1) { + z_size_t blks; + z_word_t const *words; + unsigned endian; + int k; + + /* Compute the CRC up to a z_word_t boundary. */ + while (len && ((z_size_t)buf & (W - 1)) != 0) { + len--; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + } + + /* Compute the CRC on as many N z_word_t blocks as are available. */ + blks = len / (N * W); + len -= blks * N * W; + words = (z_word_t const *)buf; + + /* Do endian check at execution time instead of compile time, since ARM + processors can change the endianess at execution time. If the + compiler knows what the endianess will be, it can optimize out the + check and the unused branch. */ + endian = 1; + if (*(unsigned char *)&endian) { + /* Little endian. */ + + z_crc_t crc0; + z_word_t word0; +#if N > 1 + z_crc_t crc1; + z_word_t word1; +#if N > 2 + z_crc_t crc2; + z_word_t word2; +#if N > 3 + z_crc_t crc3; + z_word_t word3; +#if N > 4 + z_crc_t crc4; + z_word_t word4; +#if N > 5 + z_crc_t crc5; + z_word_t word5; +#endif +#endif +#endif +#endif +#endif + + /* Initialize the CRC for each braid. */ + crc0 = crc; +#if N > 1 + crc1 = 0; +#if N > 2 + crc2 = 0; +#if N > 3 + crc3 = 0; +#if N > 4 + crc4 = 0; +#if N > 5 + crc5 = 0; +#endif +#endif +#endif +#endif +#endif + + /* + Process the first blks-1 blocks, computing the CRCs on each braid + independently. + */ + while (--blks) { + /* Load the word for each braid into registers. */ + word0 = crc0 ^ words[0]; +#if N > 1 + word1 = crc1 ^ words[1]; +#if N > 2 + word2 = crc2 ^ words[2]; +#if N > 3 + word3 = crc3 ^ words[3]; +#if N > 4 + word4 = crc4 ^ words[4]; +#if N > 5 + word5 = crc5 ^ words[5]; +#endif +#endif +#endif +#endif +#endif + words += N; + + /* Compute and update the CRC for each word. The loop should + get unrolled. */ + crc0 = crc_braid_table[0][word0 & 0xff]; +#if N > 1 + crc1 = crc_braid_table[0][word1 & 0xff]; +#if N > 2 + crc2 = crc_braid_table[0][word2 & 0xff]; +#if N > 3 + crc3 = crc_braid_table[0][word3 & 0xff]; +#if N > 4 + crc4 = crc_braid_table[0][word4 & 0xff]; +#if N > 5 + crc5 = crc_braid_table[0][word5 & 0xff]; +#endif +#endif +#endif +#endif +#endif + for (k = 1; k < W; k++) { + crc0 ^= crc_braid_table[k][(word0 >> (k << 3)) & 0xff]; +#if N > 1 + crc1 ^= crc_braid_table[k][(word1 >> (k << 3)) & 0xff]; +#if N > 2 + crc2 ^= crc_braid_table[k][(word2 >> (k << 3)) & 0xff]; +#if N > 3 + crc3 ^= crc_braid_table[k][(word3 >> (k << 3)) & 0xff]; +#if N > 4 + crc4 ^= crc_braid_table[k][(word4 >> (k << 3)) & 0xff]; +#if N > 5 + crc5 ^= crc_braid_table[k][(word5 >> (k << 3)) & 0xff]; +#endif +#endif +#endif +#endif +#endif + } + } + + /* + Process the last block, combining the CRCs of the N braids at the + same time. + */ + crc = crc_word(crc0 ^ words[0]); +#if N > 1 + crc = crc_word(crc1 ^ words[1] ^ crc); +#if N > 2 + crc = crc_word(crc2 ^ words[2] ^ crc); +#if N > 3 + crc = crc_word(crc3 ^ words[3] ^ crc); +#if N > 4 + crc = crc_word(crc4 ^ words[4] ^ crc); +#if N > 5 + crc = crc_word(crc5 ^ words[5] ^ crc); +#endif +#endif +#endif +#endif +#endif + words += N; + } + else { + /* Big endian. */ + + z_word_t crc0, word0, comb; +#if N > 1 + z_word_t crc1, word1; +#if N > 2 + z_word_t crc2, word2; +#if N > 3 + z_word_t crc3, word3; +#if N > 4 + z_word_t crc4, word4; +#if N > 5 + z_word_t crc5, word5; +#endif +#endif +#endif +#endif +#endif + + /* Initialize the CRC for each braid. */ + crc0 = byte_swap(crc); +#if N > 1 + crc1 = 0; +#if N > 2 + crc2 = 0; +#if N > 3 + crc3 = 0; +#if N > 4 + crc4 = 0; +#if N > 5 + crc5 = 0; +#endif +#endif +#endif +#endif +#endif + + /* + Process the first blks-1 blocks, computing the CRCs on each braid + independently. + */ + while (--blks) { + /* Load the word for each braid into registers. */ + word0 = crc0 ^ words[0]; +#if N > 1 + word1 = crc1 ^ words[1]; +#if N > 2 + word2 = crc2 ^ words[2]; +#if N > 3 + word3 = crc3 ^ words[3]; +#if N > 4 + word4 = crc4 ^ words[4]; +#if N > 5 + word5 = crc5 ^ words[5]; +#endif +#endif +#endif +#endif +#endif + words += N; + + /* Compute and update the CRC for each word. The loop should + get unrolled. */ + crc0 = crc_braid_big_table[0][word0 & 0xff]; +#if N > 1 + crc1 = crc_braid_big_table[0][word1 & 0xff]; +#if N > 2 + crc2 = crc_braid_big_table[0][word2 & 0xff]; +#if N > 3 + crc3 = crc_braid_big_table[0][word3 & 0xff]; +#if N > 4 + crc4 = crc_braid_big_table[0][word4 & 0xff]; +#if N > 5 + crc5 = crc_braid_big_table[0][word5 & 0xff]; +#endif +#endif +#endif +#endif +#endif + for (k = 1; k < W; k++) { + crc0 ^= crc_braid_big_table[k][(word0 >> (k << 3)) & 0xff]; +#if N > 1 + crc1 ^= crc_braid_big_table[k][(word1 >> (k << 3)) & 0xff]; +#if N > 2 + crc2 ^= crc_braid_big_table[k][(word2 >> (k << 3)) & 0xff]; +#if N > 3 + crc3 ^= crc_braid_big_table[k][(word3 >> (k << 3)) & 0xff]; +#if N > 4 + crc4 ^= crc_braid_big_table[k][(word4 >> (k << 3)) & 0xff]; +#if N > 5 + crc5 ^= crc_braid_big_table[k][(word5 >> (k << 3)) & 0xff]; +#endif +#endif +#endif +#endif +#endif + } + } + + /* + Process the last block, combining the CRCs of the N braids at the + same time. + */ + comb = crc_word_big(crc0 ^ words[0]); +#if N > 1 + comb = crc_word_big(crc1 ^ words[1] ^ comb); +#if N > 2 + comb = crc_word_big(crc2 ^ words[2] ^ comb); +#if N > 3 + comb = crc_word_big(crc3 ^ words[3] ^ comb); +#if N > 4 + comb = crc_word_big(crc4 ^ words[4] ^ comb); +#if N > 5 + comb = crc_word_big(crc5 ^ words[5] ^ comb); +#endif +#endif +#endif +#endif +#endif + words += N; + crc = byte_swap(comb); + } + + /* + Update the pointer to the remaining bytes to process. + */ + buf = (unsigned char const *)words; + } + +#endif /* W */ + + /* Complete the computation of the CRC on any remaining bytes. */ + while (len >= 8) { + len -= 8; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + } + while (len) { + len--; + crc = (crc >> 8) ^ crc_table[(crc ^ *buf++) & 0xff]; + } + + /* Return the CRC, post-conditioned. */ + return crc ^ 0xffffffff; +} + +#endif + +/* ========================================================================= */ +unsigned long ZEXPORT crc32(crc, buf, len) + unsigned long crc; + const unsigned char FAR *buf; + uInt len; +{ + return crc32_z(crc, buf, len); +} + +/* ========================================================================= */ +uLong ZEXPORT crc32_combine64(crc1, crc2, len2) + uLong crc1; + uLong crc2; + z_off64_t len2; +{ +#ifdef DYNAMIC_CRC_TABLE + once(&made, make_crc_table); +#endif /* DYNAMIC_CRC_TABLE */ + return multmodp(x2nmodp(len2, 3), crc1) ^ (crc2 & 0xffffffff); +} + +/* ========================================================================= */ +uLong ZEXPORT crc32_combine(crc1, crc2, len2) + uLong crc1; + uLong crc2; + z_off_t len2; +{ + return crc32_combine64(crc1, crc2, (z_off64_t)len2); +} + +/* ========================================================================= */ +uLong ZEXPORT crc32_combine_gen64(len2) + z_off64_t len2; +{ +#ifdef DYNAMIC_CRC_TABLE + once(&made, make_crc_table); +#endif /* DYNAMIC_CRC_TABLE */ + return x2nmodp(len2, 3); +} + +/* ========================================================================= */ +uLong ZEXPORT crc32_combine_gen(len2) + z_off_t len2; +{ + return crc32_combine_gen64((z_off64_t)len2); +} + +/* ========================================================================= */ +uLong ZEXPORT crc32_combine_op(crc1, crc2, op) + uLong crc1; + uLong crc2; + uLong op; +{ + return multmodp(op, crc1) ^ (crc2 & 0xffffffff); +} diff --git a/lib/os/windows/zlib-1.2.13/crc32.h b/lib/os/windows/zlib-1.2.13/crc32.h new file mode 100644 index 000000000000..137df68d616c --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/crc32.h @@ -0,0 +1,9446 @@ +/* crc32.h -- tables for rapid CRC calculation + * Generated automatically by crc32.c + */ + +local const z_crc_t FAR crc_table[] = { + 0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, + 0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4, + 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, + 0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, + 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856, + 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, + 0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4, + 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, + 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, + 0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a, + 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, + 0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, + 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190, + 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, + 0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e, + 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, + 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, + 0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950, + 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, + 0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, + 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a, + 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, + 0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010, + 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, + 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, + 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6, + 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, + 0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, + 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344, + 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, + 0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a, + 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, + 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, + 0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c, + 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, + 0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, + 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe, + 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, + 0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c, + 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, + 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, + 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242, + 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, + 0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, + 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278, + 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, + 0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66, + 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, + 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, + 0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8, + 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, + 0x2d02ef8d}; + +#ifdef W + +#if W == 8 + +local const z_word_t FAR crc_big_table[] = { + 0x0000000000000000, 0x9630077700000000, 0x2c610eee00000000, + 0xba51099900000000, 0x19c46d0700000000, 0x8ff46a7000000000, + 0x35a563e900000000, 0xa395649e00000000, 0x3288db0e00000000, + 0xa4b8dc7900000000, 0x1ee9d5e000000000, 0x88d9d29700000000, + 0x2b4cb60900000000, 0xbd7cb17e00000000, 0x072db8e700000000, + 0x911dbf9000000000, 0x6410b71d00000000, 0xf220b06a00000000, + 0x4871b9f300000000, 0xde41be8400000000, 0x7dd4da1a00000000, + 0xebe4dd6d00000000, 0x51b5d4f400000000, 0xc785d38300000000, + 0x56986c1300000000, 0xc0a86b6400000000, 0x7af962fd00000000, + 0xecc9658a00000000, 0x4f5c011400000000, 0xd96c066300000000, + 0x633d0ffa00000000, 0xf50d088d00000000, 0xc8206e3b00000000, + 0x5e10694c00000000, 0xe44160d500000000, 0x727167a200000000, + 0xd1e4033c00000000, 0x47d4044b00000000, 0xfd850dd200000000, + 0x6bb50aa500000000, 0xfaa8b53500000000, 0x6c98b24200000000, + 0xd6c9bbdb00000000, 0x40f9bcac00000000, 0xe36cd83200000000, + 0x755cdf4500000000, 0xcf0dd6dc00000000, 0x593dd1ab00000000, + 0xac30d92600000000, 0x3a00de5100000000, 0x8051d7c800000000, + 0x1661d0bf00000000, 0xb5f4b42100000000, 0x23c4b35600000000, + 0x9995bacf00000000, 0x0fa5bdb800000000, 0x9eb8022800000000, + 0x0888055f00000000, 0xb2d90cc600000000, 0x24e90bb100000000, + 0x877c6f2f00000000, 0x114c685800000000, 0xab1d61c100000000, + 0x3d2d66b600000000, 0x9041dc7600000000, 0x0671db0100000000, + 0xbc20d29800000000, 0x2a10d5ef00000000, 0x8985b17100000000, + 0x1fb5b60600000000, 0xa5e4bf9f00000000, 0x33d4b8e800000000, + 0xa2c9077800000000, 0x34f9000f00000000, 0x8ea8099600000000, + 0x18980ee100000000, 0xbb0d6a7f00000000, 0x2d3d6d0800000000, + 0x976c649100000000, 0x015c63e600000000, 0xf4516b6b00000000, + 0x62616c1c00000000, 0xd830658500000000, 0x4e0062f200000000, + 0xed95066c00000000, 0x7ba5011b00000000, 0xc1f4088200000000, + 0x57c40ff500000000, 0xc6d9b06500000000, 0x50e9b71200000000, + 0xeab8be8b00000000, 0x7c88b9fc00000000, 0xdf1ddd6200000000, + 0x492dda1500000000, 0xf37cd38c00000000, 0x654cd4fb00000000, + 0x5861b24d00000000, 0xce51b53a00000000, 0x7400bca300000000, + 0xe230bbd400000000, 0x41a5df4a00000000, 0xd795d83d00000000, + 0x6dc4d1a400000000, 0xfbf4d6d300000000, 0x6ae9694300000000, + 0xfcd96e3400000000, 0x468867ad00000000, 0xd0b860da00000000, + 0x732d044400000000, 0xe51d033300000000, 0x5f4c0aaa00000000, + 0xc97c0ddd00000000, 0x3c71055000000000, 0xaa41022700000000, + 0x10100bbe00000000, 0x86200cc900000000, 0x25b5685700000000, + 0xb3856f2000000000, 0x09d466b900000000, 0x9fe461ce00000000, + 0x0ef9de5e00000000, 0x98c9d92900000000, 0x2298d0b000000000, + 0xb4a8d7c700000000, 0x173db35900000000, 0x810db42e00000000, + 0x3b5cbdb700000000, 0xad6cbac000000000, 0x2083b8ed00000000, + 0xb6b3bf9a00000000, 0x0ce2b60300000000, 0x9ad2b17400000000, + 0x3947d5ea00000000, 0xaf77d29d00000000, 0x1526db0400000000, + 0x8316dc7300000000, 0x120b63e300000000, 0x843b649400000000, + 0x3e6a6d0d00000000, 0xa85a6a7a00000000, 0x0bcf0ee400000000, + 0x9dff099300000000, 0x27ae000a00000000, 0xb19e077d00000000, + 0x44930ff000000000, 0xd2a3088700000000, 0x68f2011e00000000, + 0xfec2066900000000, 0x5d5762f700000000, 0xcb67658000000000, + 0x71366c1900000000, 0xe7066b6e00000000, 0x761bd4fe00000000, + 0xe02bd38900000000, 0x5a7ada1000000000, 0xcc4add6700000000, + 0x6fdfb9f900000000, 0xf9efbe8e00000000, 0x43beb71700000000, + 0xd58eb06000000000, 0xe8a3d6d600000000, 0x7e93d1a100000000, + 0xc4c2d83800000000, 0x52f2df4f00000000, 0xf167bbd100000000, + 0x6757bca600000000, 0xdd06b53f00000000, 0x4b36b24800000000, + 0xda2b0dd800000000, 0x4c1b0aaf00000000, 0xf64a033600000000, + 0x607a044100000000, 0xc3ef60df00000000, 0x55df67a800000000, + 0xef8e6e3100000000, 0x79be694600000000, 0x8cb361cb00000000, + 0x1a8366bc00000000, 0xa0d26f2500000000, 0x36e2685200000000, + 0x95770ccc00000000, 0x03470bbb00000000, 0xb916022200000000, + 0x2f26055500000000, 0xbe3bbac500000000, 0x280bbdb200000000, + 0x925ab42b00000000, 0x046ab35c00000000, 0xa7ffd7c200000000, + 0x31cfd0b500000000, 0x8b9ed92c00000000, 0x1daede5b00000000, + 0xb0c2649b00000000, 0x26f263ec00000000, 0x9ca36a7500000000, + 0x0a936d0200000000, 0xa906099c00000000, 0x3f360eeb00000000, + 0x8567077200000000, 0x1357000500000000, 0x824abf9500000000, + 0x147ab8e200000000, 0xae2bb17b00000000, 0x381bb60c00000000, + 0x9b8ed29200000000, 0x0dbed5e500000000, 0xb7efdc7c00000000, + 0x21dfdb0b00000000, 0xd4d2d38600000000, 0x42e2d4f100000000, + 0xf8b3dd6800000000, 0x6e83da1f00000000, 0xcd16be8100000000, + 0x5b26b9f600000000, 0xe177b06f00000000, 0x7747b71800000000, + 0xe65a088800000000, 0x706a0fff00000000, 0xca3b066600000000, + 0x5c0b011100000000, 0xff9e658f00000000, 0x69ae62f800000000, + 0xd3ff6b6100000000, 0x45cf6c1600000000, 0x78e20aa000000000, + 0xeed20dd700000000, 0x5483044e00000000, 0xc2b3033900000000, + 0x612667a700000000, 0xf71660d000000000, 0x4d47694900000000, + 0xdb776e3e00000000, 0x4a6ad1ae00000000, 0xdc5ad6d900000000, + 0x660bdf4000000000, 0xf03bd83700000000, 0x53aebca900000000, + 0xc59ebbde00000000, 0x7fcfb24700000000, 0xe9ffb53000000000, + 0x1cf2bdbd00000000, 0x8ac2baca00000000, 0x3093b35300000000, + 0xa6a3b42400000000, 0x0536d0ba00000000, 0x9306d7cd00000000, + 0x2957de5400000000, 0xbf67d92300000000, 0x2e7a66b300000000, + 0xb84a61c400000000, 0x021b685d00000000, 0x942b6f2a00000000, + 0x37be0bb400000000, 0xa18e0cc300000000, 0x1bdf055a00000000, + 0x8def022d00000000}; + +#else /* W == 4 */ + +local const z_word_t FAR crc_big_table[] = { + 0x00000000, 0x96300777, 0x2c610eee, 0xba510999, 0x19c46d07, + 0x8ff46a70, 0x35a563e9, 0xa395649e, 0x3288db0e, 0xa4b8dc79, + 0x1ee9d5e0, 0x88d9d297, 0x2b4cb609, 0xbd7cb17e, 0x072db8e7, + 0x911dbf90, 0x6410b71d, 0xf220b06a, 0x4871b9f3, 0xde41be84, + 0x7dd4da1a, 0xebe4dd6d, 0x51b5d4f4, 0xc785d383, 0x56986c13, + 0xc0a86b64, 0x7af962fd, 0xecc9658a, 0x4f5c0114, 0xd96c0663, + 0x633d0ffa, 0xf50d088d, 0xc8206e3b, 0x5e10694c, 0xe44160d5, + 0x727167a2, 0xd1e4033c, 0x47d4044b, 0xfd850dd2, 0x6bb50aa5, + 0xfaa8b535, 0x6c98b242, 0xd6c9bbdb, 0x40f9bcac, 0xe36cd832, + 0x755cdf45, 0xcf0dd6dc, 0x593dd1ab, 0xac30d926, 0x3a00de51, + 0x8051d7c8, 0x1661d0bf, 0xb5f4b421, 0x23c4b356, 0x9995bacf, + 0x0fa5bdb8, 0x9eb80228, 0x0888055f, 0xb2d90cc6, 0x24e90bb1, + 0x877c6f2f, 0x114c6858, 0xab1d61c1, 0x3d2d66b6, 0x9041dc76, + 0x0671db01, 0xbc20d298, 0x2a10d5ef, 0x8985b171, 0x1fb5b606, + 0xa5e4bf9f, 0x33d4b8e8, 0xa2c90778, 0x34f9000f, 0x8ea80996, + 0x18980ee1, 0xbb0d6a7f, 0x2d3d6d08, 0x976c6491, 0x015c63e6, + 0xf4516b6b, 0x62616c1c, 0xd8306585, 0x4e0062f2, 0xed95066c, + 0x7ba5011b, 0xc1f40882, 0x57c40ff5, 0xc6d9b065, 0x50e9b712, + 0xeab8be8b, 0x7c88b9fc, 0xdf1ddd62, 0x492dda15, 0xf37cd38c, + 0x654cd4fb, 0x5861b24d, 0xce51b53a, 0x7400bca3, 0xe230bbd4, + 0x41a5df4a, 0xd795d83d, 0x6dc4d1a4, 0xfbf4d6d3, 0x6ae96943, + 0xfcd96e34, 0x468867ad, 0xd0b860da, 0x732d0444, 0xe51d0333, + 0x5f4c0aaa, 0xc97c0ddd, 0x3c710550, 0xaa410227, 0x10100bbe, + 0x86200cc9, 0x25b56857, 0xb3856f20, 0x09d466b9, 0x9fe461ce, + 0x0ef9de5e, 0x98c9d929, 0x2298d0b0, 0xb4a8d7c7, 0x173db359, + 0x810db42e, 0x3b5cbdb7, 0xad6cbac0, 0x2083b8ed, 0xb6b3bf9a, + 0x0ce2b603, 0x9ad2b174, 0x3947d5ea, 0xaf77d29d, 0x1526db04, + 0x8316dc73, 0x120b63e3, 0x843b6494, 0x3e6a6d0d, 0xa85a6a7a, + 0x0bcf0ee4, 0x9dff0993, 0x27ae000a, 0xb19e077d, 0x44930ff0, + 0xd2a30887, 0x68f2011e, 0xfec20669, 0x5d5762f7, 0xcb676580, + 0x71366c19, 0xe7066b6e, 0x761bd4fe, 0xe02bd389, 0x5a7ada10, + 0xcc4add67, 0x6fdfb9f9, 0xf9efbe8e, 0x43beb717, 0xd58eb060, + 0xe8a3d6d6, 0x7e93d1a1, 0xc4c2d838, 0x52f2df4f, 0xf167bbd1, + 0x6757bca6, 0xdd06b53f, 0x4b36b248, 0xda2b0dd8, 0x4c1b0aaf, + 0xf64a0336, 0x607a0441, 0xc3ef60df, 0x55df67a8, 0xef8e6e31, + 0x79be6946, 0x8cb361cb, 0x1a8366bc, 0xa0d26f25, 0x36e26852, + 0x95770ccc, 0x03470bbb, 0xb9160222, 0x2f260555, 0xbe3bbac5, + 0x280bbdb2, 0x925ab42b, 0x046ab35c, 0xa7ffd7c2, 0x31cfd0b5, + 0x8b9ed92c, 0x1daede5b, 0xb0c2649b, 0x26f263ec, 0x9ca36a75, + 0x0a936d02, 0xa906099c, 0x3f360eeb, 0x85670772, 0x13570005, + 0x824abf95, 0x147ab8e2, 0xae2bb17b, 0x381bb60c, 0x9b8ed292, + 0x0dbed5e5, 0xb7efdc7c, 0x21dfdb0b, 0xd4d2d386, 0x42e2d4f1, + 0xf8b3dd68, 0x6e83da1f, 0xcd16be81, 0x5b26b9f6, 0xe177b06f, + 0x7747b718, 0xe65a0888, 0x706a0fff, 0xca3b0666, 0x5c0b0111, + 0xff9e658f, 0x69ae62f8, 0xd3ff6b61, 0x45cf6c16, 0x78e20aa0, + 0xeed20dd7, 0x5483044e, 0xc2b30339, 0x612667a7, 0xf71660d0, + 0x4d476949, 0xdb776e3e, 0x4a6ad1ae, 0xdc5ad6d9, 0x660bdf40, + 0xf03bd837, 0x53aebca9, 0xc59ebbde, 0x7fcfb247, 0xe9ffb530, + 0x1cf2bdbd, 0x8ac2baca, 0x3093b353, 0xa6a3b424, 0x0536d0ba, + 0x9306d7cd, 0x2957de54, 0xbf67d923, 0x2e7a66b3, 0xb84a61c4, + 0x021b685d, 0x942b6f2a, 0x37be0bb4, 0xa18e0cc3, 0x1bdf055a, + 0x8def022d}; + +#endif + +#if N == 1 + +#if W == 8 + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0xccaa009e, 0x4225077d, 0x8e8f07e3, 0x844a0efa, + 0x48e00e64, 0xc66f0987, 0x0ac50919, 0xd3e51bb5, 0x1f4f1b2b, + 0x91c01cc8, 0x5d6a1c56, 0x57af154f, 0x9b0515d1, 0x158a1232, + 0xd92012ac, 0x7cbb312b, 0xb01131b5, 0x3e9e3656, 0xf23436c8, + 0xf8f13fd1, 0x345b3f4f, 0xbad438ac, 0x767e3832, 0xaf5e2a9e, + 0x63f42a00, 0xed7b2de3, 0x21d12d7d, 0x2b142464, 0xe7be24fa, + 0x69312319, 0xa59b2387, 0xf9766256, 0x35dc62c8, 0xbb53652b, + 0x77f965b5, 0x7d3c6cac, 0xb1966c32, 0x3f196bd1, 0xf3b36b4f, + 0x2a9379e3, 0xe639797d, 0x68b67e9e, 0xa41c7e00, 0xaed97719, + 0x62737787, 0xecfc7064, 0x205670fa, 0x85cd537d, 0x496753e3, + 0xc7e85400, 0x0b42549e, 0x01875d87, 0xcd2d5d19, 0x43a25afa, + 0x8f085a64, 0x562848c8, 0x9a824856, 0x140d4fb5, 0xd8a74f2b, + 0xd2624632, 0x1ec846ac, 0x9047414f, 0x5ced41d1, 0x299dc2ed, + 0xe537c273, 0x6bb8c590, 0xa712c50e, 0xadd7cc17, 0x617dcc89, + 0xeff2cb6a, 0x2358cbf4, 0xfa78d958, 0x36d2d9c6, 0xb85dde25, + 0x74f7debb, 0x7e32d7a2, 0xb298d73c, 0x3c17d0df, 0xf0bdd041, + 0x5526f3c6, 0x998cf358, 0x1703f4bb, 0xdba9f425, 0xd16cfd3c, + 0x1dc6fda2, 0x9349fa41, 0x5fe3fadf, 0x86c3e873, 0x4a69e8ed, + 0xc4e6ef0e, 0x084cef90, 0x0289e689, 0xce23e617, 0x40ace1f4, + 0x8c06e16a, 0xd0eba0bb, 0x1c41a025, 0x92cea7c6, 0x5e64a758, + 0x54a1ae41, 0x980baedf, 0x1684a93c, 0xda2ea9a2, 0x030ebb0e, + 0xcfa4bb90, 0x412bbc73, 0x8d81bced, 0x8744b5f4, 0x4beeb56a, + 0xc561b289, 0x09cbb217, 0xac509190, 0x60fa910e, 0xee7596ed, + 0x22df9673, 0x281a9f6a, 0xe4b09ff4, 0x6a3f9817, 0xa6959889, + 0x7fb58a25, 0xb31f8abb, 0x3d908d58, 0xf13a8dc6, 0xfbff84df, + 0x37558441, 0xb9da83a2, 0x7570833c, 0x533b85da, 0x9f918544, + 0x111e82a7, 0xddb48239, 0xd7718b20, 0x1bdb8bbe, 0x95548c5d, + 0x59fe8cc3, 0x80de9e6f, 0x4c749ef1, 0xc2fb9912, 0x0e51998c, + 0x04949095, 0xc83e900b, 0x46b197e8, 0x8a1b9776, 0x2f80b4f1, + 0xe32ab46f, 0x6da5b38c, 0xa10fb312, 0xabcaba0b, 0x6760ba95, + 0xe9efbd76, 0x2545bde8, 0xfc65af44, 0x30cfafda, 0xbe40a839, + 0x72eaa8a7, 0x782fa1be, 0xb485a120, 0x3a0aa6c3, 0xf6a0a65d, + 0xaa4de78c, 0x66e7e712, 0xe868e0f1, 0x24c2e06f, 0x2e07e976, + 0xe2ade9e8, 0x6c22ee0b, 0xa088ee95, 0x79a8fc39, 0xb502fca7, + 0x3b8dfb44, 0xf727fbda, 0xfde2f2c3, 0x3148f25d, 0xbfc7f5be, + 0x736df520, 0xd6f6d6a7, 0x1a5cd639, 0x94d3d1da, 0x5879d144, + 0x52bcd85d, 0x9e16d8c3, 0x1099df20, 0xdc33dfbe, 0x0513cd12, + 0xc9b9cd8c, 0x4736ca6f, 0x8b9ccaf1, 0x8159c3e8, 0x4df3c376, + 0xc37cc495, 0x0fd6c40b, 0x7aa64737, 0xb60c47a9, 0x3883404a, + 0xf42940d4, 0xfeec49cd, 0x32464953, 0xbcc94eb0, 0x70634e2e, + 0xa9435c82, 0x65e95c1c, 0xeb665bff, 0x27cc5b61, 0x2d095278, + 0xe1a352e6, 0x6f2c5505, 0xa386559b, 0x061d761c, 0xcab77682, + 0x44387161, 0x889271ff, 0x825778e6, 0x4efd7878, 0xc0727f9b, + 0x0cd87f05, 0xd5f86da9, 0x19526d37, 0x97dd6ad4, 0x5b776a4a, + 0x51b26353, 0x9d1863cd, 0x1397642e, 0xdf3d64b0, 0x83d02561, + 0x4f7a25ff, 0xc1f5221c, 0x0d5f2282, 0x079a2b9b, 0xcb302b05, + 0x45bf2ce6, 0x89152c78, 0x50353ed4, 0x9c9f3e4a, 0x121039a9, + 0xdeba3937, 0xd47f302e, 0x18d530b0, 0x965a3753, 0x5af037cd, + 0xff6b144a, 0x33c114d4, 0xbd4e1337, 0x71e413a9, 0x7b211ab0, + 0xb78b1a2e, 0x39041dcd, 0xf5ae1d53, 0x2c8e0fff, 0xe0240f61, + 0x6eab0882, 0xa201081c, 0xa8c40105, 0x646e019b, 0xeae10678, + 0x264b06e6}, + {0x00000000, 0xa6770bb4, 0x979f1129, 0x31e81a9d, 0xf44f2413, + 0x52382fa7, 0x63d0353a, 0xc5a73e8e, 0x33ef4e67, 0x959845d3, + 0xa4705f4e, 0x020754fa, 0xc7a06a74, 0x61d761c0, 0x503f7b5d, + 0xf64870e9, 0x67de9cce, 0xc1a9977a, 0xf0418de7, 0x56368653, + 0x9391b8dd, 0x35e6b369, 0x040ea9f4, 0xa279a240, 0x5431d2a9, + 0xf246d91d, 0xc3aec380, 0x65d9c834, 0xa07ef6ba, 0x0609fd0e, + 0x37e1e793, 0x9196ec27, 0xcfbd399c, 0x69ca3228, 0x582228b5, + 0xfe552301, 0x3bf21d8f, 0x9d85163b, 0xac6d0ca6, 0x0a1a0712, + 0xfc5277fb, 0x5a257c4f, 0x6bcd66d2, 0xcdba6d66, 0x081d53e8, + 0xae6a585c, 0x9f8242c1, 0x39f54975, 0xa863a552, 0x0e14aee6, + 0x3ffcb47b, 0x998bbfcf, 0x5c2c8141, 0xfa5b8af5, 0xcbb39068, + 0x6dc49bdc, 0x9b8ceb35, 0x3dfbe081, 0x0c13fa1c, 0xaa64f1a8, + 0x6fc3cf26, 0xc9b4c492, 0xf85cde0f, 0x5e2bd5bb, 0x440b7579, + 0xe27c7ecd, 0xd3946450, 0x75e36fe4, 0xb044516a, 0x16335ade, + 0x27db4043, 0x81ac4bf7, 0x77e43b1e, 0xd19330aa, 0xe07b2a37, + 0x460c2183, 0x83ab1f0d, 0x25dc14b9, 0x14340e24, 0xb2430590, + 0x23d5e9b7, 0x85a2e203, 0xb44af89e, 0x123df32a, 0xd79acda4, + 0x71edc610, 0x4005dc8d, 0xe672d739, 0x103aa7d0, 0xb64dac64, + 0x87a5b6f9, 0x21d2bd4d, 0xe47583c3, 0x42028877, 0x73ea92ea, + 0xd59d995e, 0x8bb64ce5, 0x2dc14751, 0x1c295dcc, 0xba5e5678, + 0x7ff968f6, 0xd98e6342, 0xe86679df, 0x4e11726b, 0xb8590282, + 0x1e2e0936, 0x2fc613ab, 0x89b1181f, 0x4c162691, 0xea612d25, + 0xdb8937b8, 0x7dfe3c0c, 0xec68d02b, 0x4a1fdb9f, 0x7bf7c102, + 0xdd80cab6, 0x1827f438, 0xbe50ff8c, 0x8fb8e511, 0x29cfeea5, + 0xdf879e4c, 0x79f095f8, 0x48188f65, 0xee6f84d1, 0x2bc8ba5f, + 0x8dbfb1eb, 0xbc57ab76, 0x1a20a0c2, 0x8816eaf2, 0x2e61e146, + 0x1f89fbdb, 0xb9fef06f, 0x7c59cee1, 0xda2ec555, 0xebc6dfc8, + 0x4db1d47c, 0xbbf9a495, 0x1d8eaf21, 0x2c66b5bc, 0x8a11be08, + 0x4fb68086, 0xe9c18b32, 0xd82991af, 0x7e5e9a1b, 0xefc8763c, + 0x49bf7d88, 0x78576715, 0xde206ca1, 0x1b87522f, 0xbdf0599b, + 0x8c184306, 0x2a6f48b2, 0xdc27385b, 0x7a5033ef, 0x4bb82972, + 0xedcf22c6, 0x28681c48, 0x8e1f17fc, 0xbff70d61, 0x198006d5, + 0x47abd36e, 0xe1dcd8da, 0xd034c247, 0x7643c9f3, 0xb3e4f77d, + 0x1593fcc9, 0x247be654, 0x820cede0, 0x74449d09, 0xd23396bd, + 0xe3db8c20, 0x45ac8794, 0x800bb91a, 0x267cb2ae, 0x1794a833, + 0xb1e3a387, 0x20754fa0, 0x86024414, 0xb7ea5e89, 0x119d553d, + 0xd43a6bb3, 0x724d6007, 0x43a57a9a, 0xe5d2712e, 0x139a01c7, + 0xb5ed0a73, 0x840510ee, 0x22721b5a, 0xe7d525d4, 0x41a22e60, + 0x704a34fd, 0xd63d3f49, 0xcc1d9f8b, 0x6a6a943f, 0x5b828ea2, + 0xfdf58516, 0x3852bb98, 0x9e25b02c, 0xafcdaab1, 0x09baa105, + 0xfff2d1ec, 0x5985da58, 0x686dc0c5, 0xce1acb71, 0x0bbdf5ff, + 0xadcafe4b, 0x9c22e4d6, 0x3a55ef62, 0xabc30345, 0x0db408f1, + 0x3c5c126c, 0x9a2b19d8, 0x5f8c2756, 0xf9fb2ce2, 0xc813367f, + 0x6e643dcb, 0x982c4d22, 0x3e5b4696, 0x0fb35c0b, 0xa9c457bf, + 0x6c636931, 0xca146285, 0xfbfc7818, 0x5d8b73ac, 0x03a0a617, + 0xa5d7ada3, 0x943fb73e, 0x3248bc8a, 0xf7ef8204, 0x519889b0, + 0x6070932d, 0xc6079899, 0x304fe870, 0x9638e3c4, 0xa7d0f959, + 0x01a7f2ed, 0xc400cc63, 0x6277c7d7, 0x539fdd4a, 0xf5e8d6fe, + 0x647e3ad9, 0xc209316d, 0xf3e12bf0, 0x55962044, 0x90311eca, + 0x3646157e, 0x07ae0fe3, 0xa1d90457, 0x579174be, 0xf1e67f0a, + 0xc00e6597, 0x66796e23, 0xa3de50ad, 0x05a95b19, 0x34414184, + 0x92364a30}, + {0x00000000, 0xcb5cd3a5, 0x4dc8a10b, 0x869472ae, 0x9b914216, + 0x50cd91b3, 0xd659e31d, 0x1d0530b8, 0xec53826d, 0x270f51c8, + 0xa19b2366, 0x6ac7f0c3, 0x77c2c07b, 0xbc9e13de, 0x3a0a6170, + 0xf156b2d5, 0x03d6029b, 0xc88ad13e, 0x4e1ea390, 0x85427035, + 0x9847408d, 0x531b9328, 0xd58fe186, 0x1ed33223, 0xef8580f6, + 0x24d95353, 0xa24d21fd, 0x6911f258, 0x7414c2e0, 0xbf481145, + 0x39dc63eb, 0xf280b04e, 0x07ac0536, 0xccf0d693, 0x4a64a43d, + 0x81387798, 0x9c3d4720, 0x57619485, 0xd1f5e62b, 0x1aa9358e, + 0xebff875b, 0x20a354fe, 0xa6372650, 0x6d6bf5f5, 0x706ec54d, + 0xbb3216e8, 0x3da66446, 0xf6fab7e3, 0x047a07ad, 0xcf26d408, + 0x49b2a6a6, 0x82ee7503, 0x9feb45bb, 0x54b7961e, 0xd223e4b0, + 0x197f3715, 0xe82985c0, 0x23755665, 0xa5e124cb, 0x6ebdf76e, + 0x73b8c7d6, 0xb8e41473, 0x3e7066dd, 0xf52cb578, 0x0f580a6c, + 0xc404d9c9, 0x4290ab67, 0x89cc78c2, 0x94c9487a, 0x5f959bdf, + 0xd901e971, 0x125d3ad4, 0xe30b8801, 0x28575ba4, 0xaec3290a, + 0x659ffaaf, 0x789aca17, 0xb3c619b2, 0x35526b1c, 0xfe0eb8b9, + 0x0c8e08f7, 0xc7d2db52, 0x4146a9fc, 0x8a1a7a59, 0x971f4ae1, + 0x5c439944, 0xdad7ebea, 0x118b384f, 0xe0dd8a9a, 0x2b81593f, + 0xad152b91, 0x6649f834, 0x7b4cc88c, 0xb0101b29, 0x36846987, + 0xfdd8ba22, 0x08f40f5a, 0xc3a8dcff, 0x453cae51, 0x8e607df4, + 0x93654d4c, 0x58399ee9, 0xdeadec47, 0x15f13fe2, 0xe4a78d37, + 0x2ffb5e92, 0xa96f2c3c, 0x6233ff99, 0x7f36cf21, 0xb46a1c84, + 0x32fe6e2a, 0xf9a2bd8f, 0x0b220dc1, 0xc07ede64, 0x46eaacca, + 0x8db67f6f, 0x90b34fd7, 0x5bef9c72, 0xdd7beedc, 0x16273d79, + 0xe7718fac, 0x2c2d5c09, 0xaab92ea7, 0x61e5fd02, 0x7ce0cdba, + 0xb7bc1e1f, 0x31286cb1, 0xfa74bf14, 0x1eb014d8, 0xd5ecc77d, + 0x5378b5d3, 0x98246676, 0x852156ce, 0x4e7d856b, 0xc8e9f7c5, + 0x03b52460, 0xf2e396b5, 0x39bf4510, 0xbf2b37be, 0x7477e41b, + 0x6972d4a3, 0xa22e0706, 0x24ba75a8, 0xefe6a60d, 0x1d661643, + 0xd63ac5e6, 0x50aeb748, 0x9bf264ed, 0x86f75455, 0x4dab87f0, + 0xcb3ff55e, 0x006326fb, 0xf135942e, 0x3a69478b, 0xbcfd3525, + 0x77a1e680, 0x6aa4d638, 0xa1f8059d, 0x276c7733, 0xec30a496, + 0x191c11ee, 0xd240c24b, 0x54d4b0e5, 0x9f886340, 0x828d53f8, + 0x49d1805d, 0xcf45f2f3, 0x04192156, 0xf54f9383, 0x3e134026, + 0xb8873288, 0x73dbe12d, 0x6eded195, 0xa5820230, 0x2316709e, + 0xe84aa33b, 0x1aca1375, 0xd196c0d0, 0x5702b27e, 0x9c5e61db, + 0x815b5163, 0x4a0782c6, 0xcc93f068, 0x07cf23cd, 0xf6999118, + 0x3dc542bd, 0xbb513013, 0x700de3b6, 0x6d08d30e, 0xa65400ab, + 0x20c07205, 0xeb9ca1a0, 0x11e81eb4, 0xdab4cd11, 0x5c20bfbf, + 0x977c6c1a, 0x8a795ca2, 0x41258f07, 0xc7b1fda9, 0x0ced2e0c, + 0xfdbb9cd9, 0x36e74f7c, 0xb0733dd2, 0x7b2fee77, 0x662adecf, + 0xad760d6a, 0x2be27fc4, 0xe0beac61, 0x123e1c2f, 0xd962cf8a, + 0x5ff6bd24, 0x94aa6e81, 0x89af5e39, 0x42f38d9c, 0xc467ff32, + 0x0f3b2c97, 0xfe6d9e42, 0x35314de7, 0xb3a53f49, 0x78f9ecec, + 0x65fcdc54, 0xaea00ff1, 0x28347d5f, 0xe368aefa, 0x16441b82, + 0xdd18c827, 0x5b8cba89, 0x90d0692c, 0x8dd55994, 0x46898a31, + 0xc01df89f, 0x0b412b3a, 0xfa1799ef, 0x314b4a4a, 0xb7df38e4, + 0x7c83eb41, 0x6186dbf9, 0xaada085c, 0x2c4e7af2, 0xe712a957, + 0x15921919, 0xdececabc, 0x585ab812, 0x93066bb7, 0x8e035b0f, + 0x455f88aa, 0xc3cbfa04, 0x089729a1, 0xf9c19b74, 0x329d48d1, + 0xb4093a7f, 0x7f55e9da, 0x6250d962, 0xa90c0ac7, 0x2f987869, + 0xe4c4abcc}, + {0x00000000, 0x3d6029b0, 0x7ac05360, 0x47a07ad0, 0xf580a6c0, + 0xc8e08f70, 0x8f40f5a0, 0xb220dc10, 0x30704bc1, 0x0d106271, + 0x4ab018a1, 0x77d03111, 0xc5f0ed01, 0xf890c4b1, 0xbf30be61, + 0x825097d1, 0x60e09782, 0x5d80be32, 0x1a20c4e2, 0x2740ed52, + 0x95603142, 0xa80018f2, 0xefa06222, 0xd2c04b92, 0x5090dc43, + 0x6df0f5f3, 0x2a508f23, 0x1730a693, 0xa5107a83, 0x98705333, + 0xdfd029e3, 0xe2b00053, 0xc1c12f04, 0xfca106b4, 0xbb017c64, + 0x866155d4, 0x344189c4, 0x0921a074, 0x4e81daa4, 0x73e1f314, + 0xf1b164c5, 0xccd14d75, 0x8b7137a5, 0xb6111e15, 0x0431c205, + 0x3951ebb5, 0x7ef19165, 0x4391b8d5, 0xa121b886, 0x9c419136, + 0xdbe1ebe6, 0xe681c256, 0x54a11e46, 0x69c137f6, 0x2e614d26, + 0x13016496, 0x9151f347, 0xac31daf7, 0xeb91a027, 0xd6f18997, + 0x64d15587, 0x59b17c37, 0x1e1106e7, 0x23712f57, 0x58f35849, + 0x659371f9, 0x22330b29, 0x1f532299, 0xad73fe89, 0x9013d739, + 0xd7b3ade9, 0xead38459, 0x68831388, 0x55e33a38, 0x124340e8, + 0x2f236958, 0x9d03b548, 0xa0639cf8, 0xe7c3e628, 0xdaa3cf98, + 0x3813cfcb, 0x0573e67b, 0x42d39cab, 0x7fb3b51b, 0xcd93690b, + 0xf0f340bb, 0xb7533a6b, 0x8a3313db, 0x0863840a, 0x3503adba, + 0x72a3d76a, 0x4fc3feda, 0xfde322ca, 0xc0830b7a, 0x872371aa, + 0xba43581a, 0x9932774d, 0xa4525efd, 0xe3f2242d, 0xde920d9d, + 0x6cb2d18d, 0x51d2f83d, 0x167282ed, 0x2b12ab5d, 0xa9423c8c, + 0x9422153c, 0xd3826fec, 0xeee2465c, 0x5cc29a4c, 0x61a2b3fc, + 0x2602c92c, 0x1b62e09c, 0xf9d2e0cf, 0xc4b2c97f, 0x8312b3af, + 0xbe729a1f, 0x0c52460f, 0x31326fbf, 0x7692156f, 0x4bf23cdf, + 0xc9a2ab0e, 0xf4c282be, 0xb362f86e, 0x8e02d1de, 0x3c220dce, + 0x0142247e, 0x46e25eae, 0x7b82771e, 0xb1e6b092, 0x8c869922, + 0xcb26e3f2, 0xf646ca42, 0x44661652, 0x79063fe2, 0x3ea64532, + 0x03c66c82, 0x8196fb53, 0xbcf6d2e3, 0xfb56a833, 0xc6368183, + 0x74165d93, 0x49767423, 0x0ed60ef3, 0x33b62743, 0xd1062710, + 0xec660ea0, 0xabc67470, 0x96a65dc0, 0x248681d0, 0x19e6a860, + 0x5e46d2b0, 0x6326fb00, 0xe1766cd1, 0xdc164561, 0x9bb63fb1, + 0xa6d61601, 0x14f6ca11, 0x2996e3a1, 0x6e369971, 0x5356b0c1, + 0x70279f96, 0x4d47b626, 0x0ae7ccf6, 0x3787e546, 0x85a73956, + 0xb8c710e6, 0xff676a36, 0xc2074386, 0x4057d457, 0x7d37fde7, + 0x3a978737, 0x07f7ae87, 0xb5d77297, 0x88b75b27, 0xcf1721f7, + 0xf2770847, 0x10c70814, 0x2da721a4, 0x6a075b74, 0x576772c4, + 0xe547aed4, 0xd8278764, 0x9f87fdb4, 0xa2e7d404, 0x20b743d5, + 0x1dd76a65, 0x5a7710b5, 0x67173905, 0xd537e515, 0xe857cca5, + 0xaff7b675, 0x92979fc5, 0xe915e8db, 0xd475c16b, 0x93d5bbbb, + 0xaeb5920b, 0x1c954e1b, 0x21f567ab, 0x66551d7b, 0x5b3534cb, + 0xd965a31a, 0xe4058aaa, 0xa3a5f07a, 0x9ec5d9ca, 0x2ce505da, + 0x11852c6a, 0x562556ba, 0x6b457f0a, 0x89f57f59, 0xb49556e9, + 0xf3352c39, 0xce550589, 0x7c75d999, 0x4115f029, 0x06b58af9, + 0x3bd5a349, 0xb9853498, 0x84e51d28, 0xc34567f8, 0xfe254e48, + 0x4c059258, 0x7165bbe8, 0x36c5c138, 0x0ba5e888, 0x28d4c7df, + 0x15b4ee6f, 0x521494bf, 0x6f74bd0f, 0xdd54611f, 0xe03448af, + 0xa794327f, 0x9af41bcf, 0x18a48c1e, 0x25c4a5ae, 0x6264df7e, + 0x5f04f6ce, 0xed242ade, 0xd044036e, 0x97e479be, 0xaa84500e, + 0x4834505d, 0x755479ed, 0x32f4033d, 0x0f942a8d, 0xbdb4f69d, + 0x80d4df2d, 0xc774a5fd, 0xfa148c4d, 0x78441b9c, 0x4524322c, + 0x028448fc, 0x3fe4614c, 0x8dc4bd5c, 0xb0a494ec, 0xf704ee3c, + 0xca64c78c}, + {0x00000000, 0xb8bc6765, 0xaa09c88b, 0x12b5afee, 0x8f629757, + 0x37def032, 0x256b5fdc, 0x9dd738b9, 0xc5b428ef, 0x7d084f8a, + 0x6fbde064, 0xd7018701, 0x4ad6bfb8, 0xf26ad8dd, 0xe0df7733, + 0x58631056, 0x5019579f, 0xe8a530fa, 0xfa109f14, 0x42acf871, + 0xdf7bc0c8, 0x67c7a7ad, 0x75720843, 0xcdce6f26, 0x95ad7f70, + 0x2d111815, 0x3fa4b7fb, 0x8718d09e, 0x1acfe827, 0xa2738f42, + 0xb0c620ac, 0x087a47c9, 0xa032af3e, 0x188ec85b, 0x0a3b67b5, + 0xb28700d0, 0x2f503869, 0x97ec5f0c, 0x8559f0e2, 0x3de59787, + 0x658687d1, 0xdd3ae0b4, 0xcf8f4f5a, 0x7733283f, 0xeae41086, + 0x525877e3, 0x40edd80d, 0xf851bf68, 0xf02bf8a1, 0x48979fc4, + 0x5a22302a, 0xe29e574f, 0x7f496ff6, 0xc7f50893, 0xd540a77d, + 0x6dfcc018, 0x359fd04e, 0x8d23b72b, 0x9f9618c5, 0x272a7fa0, + 0xbafd4719, 0x0241207c, 0x10f48f92, 0xa848e8f7, 0x9b14583d, + 0x23a83f58, 0x311d90b6, 0x89a1f7d3, 0x1476cf6a, 0xaccaa80f, + 0xbe7f07e1, 0x06c36084, 0x5ea070d2, 0xe61c17b7, 0xf4a9b859, + 0x4c15df3c, 0xd1c2e785, 0x697e80e0, 0x7bcb2f0e, 0xc377486b, + 0xcb0d0fa2, 0x73b168c7, 0x6104c729, 0xd9b8a04c, 0x446f98f5, + 0xfcd3ff90, 0xee66507e, 0x56da371b, 0x0eb9274d, 0xb6054028, + 0xa4b0efc6, 0x1c0c88a3, 0x81dbb01a, 0x3967d77f, 0x2bd27891, + 0x936e1ff4, 0x3b26f703, 0x839a9066, 0x912f3f88, 0x299358ed, + 0xb4446054, 0x0cf80731, 0x1e4da8df, 0xa6f1cfba, 0xfe92dfec, + 0x462eb889, 0x549b1767, 0xec277002, 0x71f048bb, 0xc94c2fde, + 0xdbf98030, 0x6345e755, 0x6b3fa09c, 0xd383c7f9, 0xc1366817, + 0x798a0f72, 0xe45d37cb, 0x5ce150ae, 0x4e54ff40, 0xf6e89825, + 0xae8b8873, 0x1637ef16, 0x048240f8, 0xbc3e279d, 0x21e91f24, + 0x99557841, 0x8be0d7af, 0x335cb0ca, 0xed59b63b, 0x55e5d15e, + 0x47507eb0, 0xffec19d5, 0x623b216c, 0xda874609, 0xc832e9e7, + 0x708e8e82, 0x28ed9ed4, 0x9051f9b1, 0x82e4565f, 0x3a58313a, + 0xa78f0983, 0x1f336ee6, 0x0d86c108, 0xb53aa66d, 0xbd40e1a4, + 0x05fc86c1, 0x1749292f, 0xaff54e4a, 0x322276f3, 0x8a9e1196, + 0x982bbe78, 0x2097d91d, 0x78f4c94b, 0xc048ae2e, 0xd2fd01c0, + 0x6a4166a5, 0xf7965e1c, 0x4f2a3979, 0x5d9f9697, 0xe523f1f2, + 0x4d6b1905, 0xf5d77e60, 0xe762d18e, 0x5fdeb6eb, 0xc2098e52, + 0x7ab5e937, 0x680046d9, 0xd0bc21bc, 0x88df31ea, 0x3063568f, + 0x22d6f961, 0x9a6a9e04, 0x07bda6bd, 0xbf01c1d8, 0xadb46e36, + 0x15080953, 0x1d724e9a, 0xa5ce29ff, 0xb77b8611, 0x0fc7e174, + 0x9210d9cd, 0x2aacbea8, 0x38191146, 0x80a57623, 0xd8c66675, + 0x607a0110, 0x72cfaefe, 0xca73c99b, 0x57a4f122, 0xef189647, + 0xfdad39a9, 0x45115ecc, 0x764dee06, 0xcef18963, 0xdc44268d, + 0x64f841e8, 0xf92f7951, 0x41931e34, 0x5326b1da, 0xeb9ad6bf, + 0xb3f9c6e9, 0x0b45a18c, 0x19f00e62, 0xa14c6907, 0x3c9b51be, + 0x842736db, 0x96929935, 0x2e2efe50, 0x2654b999, 0x9ee8defc, + 0x8c5d7112, 0x34e11677, 0xa9362ece, 0x118a49ab, 0x033fe645, + 0xbb838120, 0xe3e09176, 0x5b5cf613, 0x49e959fd, 0xf1553e98, + 0x6c820621, 0xd43e6144, 0xc68bceaa, 0x7e37a9cf, 0xd67f4138, + 0x6ec3265d, 0x7c7689b3, 0xc4caeed6, 0x591dd66f, 0xe1a1b10a, + 0xf3141ee4, 0x4ba87981, 0x13cb69d7, 0xab770eb2, 0xb9c2a15c, + 0x017ec639, 0x9ca9fe80, 0x241599e5, 0x36a0360b, 0x8e1c516e, + 0x866616a7, 0x3eda71c2, 0x2c6fde2c, 0x94d3b949, 0x090481f0, + 0xb1b8e695, 0xa30d497b, 0x1bb12e1e, 0x43d23e48, 0xfb6e592d, + 0xe9dbf6c3, 0x516791a6, 0xccb0a91f, 0x740cce7a, 0x66b96194, + 0xde0506f1}, + {0x00000000, 0x01c26a37, 0x0384d46e, 0x0246be59, 0x0709a8dc, + 0x06cbc2eb, 0x048d7cb2, 0x054f1685, 0x0e1351b8, 0x0fd13b8f, + 0x0d9785d6, 0x0c55efe1, 0x091af964, 0x08d89353, 0x0a9e2d0a, + 0x0b5c473d, 0x1c26a370, 0x1de4c947, 0x1fa2771e, 0x1e601d29, + 0x1b2f0bac, 0x1aed619b, 0x18abdfc2, 0x1969b5f5, 0x1235f2c8, + 0x13f798ff, 0x11b126a6, 0x10734c91, 0x153c5a14, 0x14fe3023, + 0x16b88e7a, 0x177ae44d, 0x384d46e0, 0x398f2cd7, 0x3bc9928e, + 0x3a0bf8b9, 0x3f44ee3c, 0x3e86840b, 0x3cc03a52, 0x3d025065, + 0x365e1758, 0x379c7d6f, 0x35dac336, 0x3418a901, 0x3157bf84, + 0x3095d5b3, 0x32d36bea, 0x331101dd, 0x246be590, 0x25a98fa7, + 0x27ef31fe, 0x262d5bc9, 0x23624d4c, 0x22a0277b, 0x20e69922, + 0x2124f315, 0x2a78b428, 0x2bbade1f, 0x29fc6046, 0x283e0a71, + 0x2d711cf4, 0x2cb376c3, 0x2ef5c89a, 0x2f37a2ad, 0x709a8dc0, + 0x7158e7f7, 0x731e59ae, 0x72dc3399, 0x7793251c, 0x76514f2b, + 0x7417f172, 0x75d59b45, 0x7e89dc78, 0x7f4bb64f, 0x7d0d0816, + 0x7ccf6221, 0x798074a4, 0x78421e93, 0x7a04a0ca, 0x7bc6cafd, + 0x6cbc2eb0, 0x6d7e4487, 0x6f38fade, 0x6efa90e9, 0x6bb5866c, + 0x6a77ec5b, 0x68315202, 0x69f33835, 0x62af7f08, 0x636d153f, + 0x612bab66, 0x60e9c151, 0x65a6d7d4, 0x6464bde3, 0x662203ba, + 0x67e0698d, 0x48d7cb20, 0x4915a117, 0x4b531f4e, 0x4a917579, + 0x4fde63fc, 0x4e1c09cb, 0x4c5ab792, 0x4d98dda5, 0x46c49a98, + 0x4706f0af, 0x45404ef6, 0x448224c1, 0x41cd3244, 0x400f5873, + 0x4249e62a, 0x438b8c1d, 0x54f16850, 0x55330267, 0x5775bc3e, + 0x56b7d609, 0x53f8c08c, 0x523aaabb, 0x507c14e2, 0x51be7ed5, + 0x5ae239e8, 0x5b2053df, 0x5966ed86, 0x58a487b1, 0x5deb9134, + 0x5c29fb03, 0x5e6f455a, 0x5fad2f6d, 0xe1351b80, 0xe0f771b7, + 0xe2b1cfee, 0xe373a5d9, 0xe63cb35c, 0xe7fed96b, 0xe5b86732, + 0xe47a0d05, 0xef264a38, 0xeee4200f, 0xeca29e56, 0xed60f461, + 0xe82fe2e4, 0xe9ed88d3, 0xebab368a, 0xea695cbd, 0xfd13b8f0, + 0xfcd1d2c7, 0xfe976c9e, 0xff5506a9, 0xfa1a102c, 0xfbd87a1b, + 0xf99ec442, 0xf85cae75, 0xf300e948, 0xf2c2837f, 0xf0843d26, + 0xf1465711, 0xf4094194, 0xf5cb2ba3, 0xf78d95fa, 0xf64fffcd, + 0xd9785d60, 0xd8ba3757, 0xdafc890e, 0xdb3ee339, 0xde71f5bc, + 0xdfb39f8b, 0xddf521d2, 0xdc374be5, 0xd76b0cd8, 0xd6a966ef, + 0xd4efd8b6, 0xd52db281, 0xd062a404, 0xd1a0ce33, 0xd3e6706a, + 0xd2241a5d, 0xc55efe10, 0xc49c9427, 0xc6da2a7e, 0xc7184049, + 0xc25756cc, 0xc3953cfb, 0xc1d382a2, 0xc011e895, 0xcb4dafa8, + 0xca8fc59f, 0xc8c97bc6, 0xc90b11f1, 0xcc440774, 0xcd866d43, + 0xcfc0d31a, 0xce02b92d, 0x91af9640, 0x906dfc77, 0x922b422e, + 0x93e92819, 0x96a63e9c, 0x976454ab, 0x9522eaf2, 0x94e080c5, + 0x9fbcc7f8, 0x9e7eadcf, 0x9c381396, 0x9dfa79a1, 0x98b56f24, + 0x99770513, 0x9b31bb4a, 0x9af3d17d, 0x8d893530, 0x8c4b5f07, + 0x8e0de15e, 0x8fcf8b69, 0x8a809dec, 0x8b42f7db, 0x89044982, + 0x88c623b5, 0x839a6488, 0x82580ebf, 0x801eb0e6, 0x81dcdad1, + 0x8493cc54, 0x8551a663, 0x8717183a, 0x86d5720d, 0xa9e2d0a0, + 0xa820ba97, 0xaa6604ce, 0xaba46ef9, 0xaeeb787c, 0xaf29124b, + 0xad6fac12, 0xacadc625, 0xa7f18118, 0xa633eb2f, 0xa4755576, + 0xa5b73f41, 0xa0f829c4, 0xa13a43f3, 0xa37cfdaa, 0xa2be979d, + 0xb5c473d0, 0xb40619e7, 0xb640a7be, 0xb782cd89, 0xb2cddb0c, + 0xb30fb13b, 0xb1490f62, 0xb08b6555, 0xbbd72268, 0xba15485f, + 0xb853f606, 0xb9919c31, 0xbcde8ab4, 0xbd1ce083, 0xbf5a5eda, + 0xbe9834ed}, + {0x00000000, 0x191b3141, 0x32366282, 0x2b2d53c3, 0x646cc504, + 0x7d77f445, 0x565aa786, 0x4f4196c7, 0xc8d98a08, 0xd1c2bb49, + 0xfaefe88a, 0xe3f4d9cb, 0xacb54f0c, 0xb5ae7e4d, 0x9e832d8e, + 0x87981ccf, 0x4ac21251, 0x53d92310, 0x78f470d3, 0x61ef4192, + 0x2eaed755, 0x37b5e614, 0x1c98b5d7, 0x05838496, 0x821b9859, + 0x9b00a918, 0xb02dfadb, 0xa936cb9a, 0xe6775d5d, 0xff6c6c1c, + 0xd4413fdf, 0xcd5a0e9e, 0x958424a2, 0x8c9f15e3, 0xa7b24620, + 0xbea97761, 0xf1e8e1a6, 0xe8f3d0e7, 0xc3de8324, 0xdac5b265, + 0x5d5daeaa, 0x44469feb, 0x6f6bcc28, 0x7670fd69, 0x39316bae, + 0x202a5aef, 0x0b07092c, 0x121c386d, 0xdf4636f3, 0xc65d07b2, + 0xed705471, 0xf46b6530, 0xbb2af3f7, 0xa231c2b6, 0x891c9175, + 0x9007a034, 0x179fbcfb, 0x0e848dba, 0x25a9de79, 0x3cb2ef38, + 0x73f379ff, 0x6ae848be, 0x41c51b7d, 0x58de2a3c, 0xf0794f05, + 0xe9627e44, 0xc24f2d87, 0xdb541cc6, 0x94158a01, 0x8d0ebb40, + 0xa623e883, 0xbf38d9c2, 0x38a0c50d, 0x21bbf44c, 0x0a96a78f, + 0x138d96ce, 0x5ccc0009, 0x45d73148, 0x6efa628b, 0x77e153ca, + 0xbabb5d54, 0xa3a06c15, 0x888d3fd6, 0x91960e97, 0xded79850, + 0xc7cca911, 0xece1fad2, 0xf5facb93, 0x7262d75c, 0x6b79e61d, + 0x4054b5de, 0x594f849f, 0x160e1258, 0x0f152319, 0x243870da, + 0x3d23419b, 0x65fd6ba7, 0x7ce65ae6, 0x57cb0925, 0x4ed03864, + 0x0191aea3, 0x188a9fe2, 0x33a7cc21, 0x2abcfd60, 0xad24e1af, + 0xb43fd0ee, 0x9f12832d, 0x8609b26c, 0xc94824ab, 0xd05315ea, + 0xfb7e4629, 0xe2657768, 0x2f3f79f6, 0x362448b7, 0x1d091b74, + 0x04122a35, 0x4b53bcf2, 0x52488db3, 0x7965de70, 0x607eef31, + 0xe7e6f3fe, 0xfefdc2bf, 0xd5d0917c, 0xcccba03d, 0x838a36fa, + 0x9a9107bb, 0xb1bc5478, 0xa8a76539, 0x3b83984b, 0x2298a90a, + 0x09b5fac9, 0x10aecb88, 0x5fef5d4f, 0x46f46c0e, 0x6dd93fcd, + 0x74c20e8c, 0xf35a1243, 0xea412302, 0xc16c70c1, 0xd8774180, + 0x9736d747, 0x8e2de606, 0xa500b5c5, 0xbc1b8484, 0x71418a1a, + 0x685abb5b, 0x4377e898, 0x5a6cd9d9, 0x152d4f1e, 0x0c367e5f, + 0x271b2d9c, 0x3e001cdd, 0xb9980012, 0xa0833153, 0x8bae6290, + 0x92b553d1, 0xddf4c516, 0xc4eff457, 0xefc2a794, 0xf6d996d5, + 0xae07bce9, 0xb71c8da8, 0x9c31de6b, 0x852aef2a, 0xca6b79ed, + 0xd37048ac, 0xf85d1b6f, 0xe1462a2e, 0x66de36e1, 0x7fc507a0, + 0x54e85463, 0x4df36522, 0x02b2f3e5, 0x1ba9c2a4, 0x30849167, + 0x299fa026, 0xe4c5aeb8, 0xfdde9ff9, 0xd6f3cc3a, 0xcfe8fd7b, + 0x80a96bbc, 0x99b25afd, 0xb29f093e, 0xab84387f, 0x2c1c24b0, + 0x350715f1, 0x1e2a4632, 0x07317773, 0x4870e1b4, 0x516bd0f5, + 0x7a468336, 0x635db277, 0xcbfad74e, 0xd2e1e60f, 0xf9ccb5cc, + 0xe0d7848d, 0xaf96124a, 0xb68d230b, 0x9da070c8, 0x84bb4189, + 0x03235d46, 0x1a386c07, 0x31153fc4, 0x280e0e85, 0x674f9842, + 0x7e54a903, 0x5579fac0, 0x4c62cb81, 0x8138c51f, 0x9823f45e, + 0xb30ea79d, 0xaa1596dc, 0xe554001b, 0xfc4f315a, 0xd7626299, + 0xce7953d8, 0x49e14f17, 0x50fa7e56, 0x7bd72d95, 0x62cc1cd4, + 0x2d8d8a13, 0x3496bb52, 0x1fbbe891, 0x06a0d9d0, 0x5e7ef3ec, + 0x4765c2ad, 0x6c48916e, 0x7553a02f, 0x3a1236e8, 0x230907a9, + 0x0824546a, 0x113f652b, 0x96a779e4, 0x8fbc48a5, 0xa4911b66, + 0xbd8a2a27, 0xf2cbbce0, 0xebd08da1, 0xc0fdde62, 0xd9e6ef23, + 0x14bce1bd, 0x0da7d0fc, 0x268a833f, 0x3f91b27e, 0x70d024b9, + 0x69cb15f8, 0x42e6463b, 0x5bfd777a, 0xdc656bb5, 0xc57e5af4, + 0xee530937, 0xf7483876, 0xb809aeb1, 0xa1129ff0, 0x8a3fcc33, + 0x9324fd72}, + {0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, + 0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4, + 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, + 0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, + 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856, + 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, + 0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4, + 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, + 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, + 0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a, + 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, + 0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, + 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190, + 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, + 0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e, + 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, + 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, + 0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950, + 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, + 0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, + 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a, + 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, + 0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010, + 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, + 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, + 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6, + 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, + 0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, + 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344, + 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, + 0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a, + 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, + 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, + 0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c, + 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, + 0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, + 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe, + 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, + 0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c, + 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, + 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, + 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242, + 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, + 0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, + 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278, + 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, + 0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66, + 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, + 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, + 0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8, + 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, + 0x2d02ef8d}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x0000000000000000, 0x9630077700000000, 0x2c610eee00000000, + 0xba51099900000000, 0x19c46d0700000000, 0x8ff46a7000000000, + 0x35a563e900000000, 0xa395649e00000000, 0x3288db0e00000000, + 0xa4b8dc7900000000, 0x1ee9d5e000000000, 0x88d9d29700000000, + 0x2b4cb60900000000, 0xbd7cb17e00000000, 0x072db8e700000000, + 0x911dbf9000000000, 0x6410b71d00000000, 0xf220b06a00000000, + 0x4871b9f300000000, 0xde41be8400000000, 0x7dd4da1a00000000, + 0xebe4dd6d00000000, 0x51b5d4f400000000, 0xc785d38300000000, + 0x56986c1300000000, 0xc0a86b6400000000, 0x7af962fd00000000, + 0xecc9658a00000000, 0x4f5c011400000000, 0xd96c066300000000, + 0x633d0ffa00000000, 0xf50d088d00000000, 0xc8206e3b00000000, + 0x5e10694c00000000, 0xe44160d500000000, 0x727167a200000000, + 0xd1e4033c00000000, 0x47d4044b00000000, 0xfd850dd200000000, + 0x6bb50aa500000000, 0xfaa8b53500000000, 0x6c98b24200000000, + 0xd6c9bbdb00000000, 0x40f9bcac00000000, 0xe36cd83200000000, + 0x755cdf4500000000, 0xcf0dd6dc00000000, 0x593dd1ab00000000, + 0xac30d92600000000, 0x3a00de5100000000, 0x8051d7c800000000, + 0x1661d0bf00000000, 0xb5f4b42100000000, 0x23c4b35600000000, + 0x9995bacf00000000, 0x0fa5bdb800000000, 0x9eb8022800000000, + 0x0888055f00000000, 0xb2d90cc600000000, 0x24e90bb100000000, + 0x877c6f2f00000000, 0x114c685800000000, 0xab1d61c100000000, + 0x3d2d66b600000000, 0x9041dc7600000000, 0x0671db0100000000, + 0xbc20d29800000000, 0x2a10d5ef00000000, 0x8985b17100000000, + 0x1fb5b60600000000, 0xa5e4bf9f00000000, 0x33d4b8e800000000, + 0xa2c9077800000000, 0x34f9000f00000000, 0x8ea8099600000000, + 0x18980ee100000000, 0xbb0d6a7f00000000, 0x2d3d6d0800000000, + 0x976c649100000000, 0x015c63e600000000, 0xf4516b6b00000000, + 0x62616c1c00000000, 0xd830658500000000, 0x4e0062f200000000, + 0xed95066c00000000, 0x7ba5011b00000000, 0xc1f4088200000000, + 0x57c40ff500000000, 0xc6d9b06500000000, 0x50e9b71200000000, + 0xeab8be8b00000000, 0x7c88b9fc00000000, 0xdf1ddd6200000000, + 0x492dda1500000000, 0xf37cd38c00000000, 0x654cd4fb00000000, + 0x5861b24d00000000, 0xce51b53a00000000, 0x7400bca300000000, + 0xe230bbd400000000, 0x41a5df4a00000000, 0xd795d83d00000000, + 0x6dc4d1a400000000, 0xfbf4d6d300000000, 0x6ae9694300000000, + 0xfcd96e3400000000, 0x468867ad00000000, 0xd0b860da00000000, + 0x732d044400000000, 0xe51d033300000000, 0x5f4c0aaa00000000, + 0xc97c0ddd00000000, 0x3c71055000000000, 0xaa41022700000000, + 0x10100bbe00000000, 0x86200cc900000000, 0x25b5685700000000, + 0xb3856f2000000000, 0x09d466b900000000, 0x9fe461ce00000000, + 0x0ef9de5e00000000, 0x98c9d92900000000, 0x2298d0b000000000, + 0xb4a8d7c700000000, 0x173db35900000000, 0x810db42e00000000, + 0x3b5cbdb700000000, 0xad6cbac000000000, 0x2083b8ed00000000, + 0xb6b3bf9a00000000, 0x0ce2b60300000000, 0x9ad2b17400000000, + 0x3947d5ea00000000, 0xaf77d29d00000000, 0x1526db0400000000, + 0x8316dc7300000000, 0x120b63e300000000, 0x843b649400000000, + 0x3e6a6d0d00000000, 0xa85a6a7a00000000, 0x0bcf0ee400000000, + 0x9dff099300000000, 0x27ae000a00000000, 0xb19e077d00000000, + 0x44930ff000000000, 0xd2a3088700000000, 0x68f2011e00000000, + 0xfec2066900000000, 0x5d5762f700000000, 0xcb67658000000000, + 0x71366c1900000000, 0xe7066b6e00000000, 0x761bd4fe00000000, + 0xe02bd38900000000, 0x5a7ada1000000000, 0xcc4add6700000000, + 0x6fdfb9f900000000, 0xf9efbe8e00000000, 0x43beb71700000000, + 0xd58eb06000000000, 0xe8a3d6d600000000, 0x7e93d1a100000000, + 0xc4c2d83800000000, 0x52f2df4f00000000, 0xf167bbd100000000, + 0x6757bca600000000, 0xdd06b53f00000000, 0x4b36b24800000000, + 0xda2b0dd800000000, 0x4c1b0aaf00000000, 0xf64a033600000000, + 0x607a044100000000, 0xc3ef60df00000000, 0x55df67a800000000, + 0xef8e6e3100000000, 0x79be694600000000, 0x8cb361cb00000000, + 0x1a8366bc00000000, 0xa0d26f2500000000, 0x36e2685200000000, + 0x95770ccc00000000, 0x03470bbb00000000, 0xb916022200000000, + 0x2f26055500000000, 0xbe3bbac500000000, 0x280bbdb200000000, + 0x925ab42b00000000, 0x046ab35c00000000, 0xa7ffd7c200000000, + 0x31cfd0b500000000, 0x8b9ed92c00000000, 0x1daede5b00000000, + 0xb0c2649b00000000, 0x26f263ec00000000, 0x9ca36a7500000000, + 0x0a936d0200000000, 0xa906099c00000000, 0x3f360eeb00000000, + 0x8567077200000000, 0x1357000500000000, 0x824abf9500000000, + 0x147ab8e200000000, 0xae2bb17b00000000, 0x381bb60c00000000, + 0x9b8ed29200000000, 0x0dbed5e500000000, 0xb7efdc7c00000000, + 0x21dfdb0b00000000, 0xd4d2d38600000000, 0x42e2d4f100000000, + 0xf8b3dd6800000000, 0x6e83da1f00000000, 0xcd16be8100000000, + 0x5b26b9f600000000, 0xe177b06f00000000, 0x7747b71800000000, + 0xe65a088800000000, 0x706a0fff00000000, 0xca3b066600000000, + 0x5c0b011100000000, 0xff9e658f00000000, 0x69ae62f800000000, + 0xd3ff6b6100000000, 0x45cf6c1600000000, 0x78e20aa000000000, + 0xeed20dd700000000, 0x5483044e00000000, 0xc2b3033900000000, + 0x612667a700000000, 0xf71660d000000000, 0x4d47694900000000, + 0xdb776e3e00000000, 0x4a6ad1ae00000000, 0xdc5ad6d900000000, + 0x660bdf4000000000, 0xf03bd83700000000, 0x53aebca900000000, + 0xc59ebbde00000000, 0x7fcfb24700000000, 0xe9ffb53000000000, + 0x1cf2bdbd00000000, 0x8ac2baca00000000, 0x3093b35300000000, + 0xa6a3b42400000000, 0x0536d0ba00000000, 0x9306d7cd00000000, + 0x2957de5400000000, 0xbf67d92300000000, 0x2e7a66b300000000, + 0xb84a61c400000000, 0x021b685d00000000, 0x942b6f2a00000000, + 0x37be0bb400000000, 0xa18e0cc300000000, 0x1bdf055a00000000, + 0x8def022d00000000}, + {0x0000000000000000, 0x41311b1900000000, 0x8262363200000000, + 0xc3532d2b00000000, 0x04c56c6400000000, 0x45f4777d00000000, + 0x86a75a5600000000, 0xc796414f00000000, 0x088ad9c800000000, + 0x49bbc2d100000000, 0x8ae8effa00000000, 0xcbd9f4e300000000, + 0x0c4fb5ac00000000, 0x4d7eaeb500000000, 0x8e2d839e00000000, + 0xcf1c988700000000, 0x5112c24a00000000, 0x1023d95300000000, + 0xd370f47800000000, 0x9241ef6100000000, 0x55d7ae2e00000000, + 0x14e6b53700000000, 0xd7b5981c00000000, 0x9684830500000000, + 0x59981b8200000000, 0x18a9009b00000000, 0xdbfa2db000000000, + 0x9acb36a900000000, 0x5d5d77e600000000, 0x1c6c6cff00000000, + 0xdf3f41d400000000, 0x9e0e5acd00000000, 0xa224849500000000, + 0xe3159f8c00000000, 0x2046b2a700000000, 0x6177a9be00000000, + 0xa6e1e8f100000000, 0xe7d0f3e800000000, 0x2483dec300000000, + 0x65b2c5da00000000, 0xaaae5d5d00000000, 0xeb9f464400000000, + 0x28cc6b6f00000000, 0x69fd707600000000, 0xae6b313900000000, + 0xef5a2a2000000000, 0x2c09070b00000000, 0x6d381c1200000000, + 0xf33646df00000000, 0xb2075dc600000000, 0x715470ed00000000, + 0x30656bf400000000, 0xf7f32abb00000000, 0xb6c231a200000000, + 0x75911c8900000000, 0x34a0079000000000, 0xfbbc9f1700000000, + 0xba8d840e00000000, 0x79dea92500000000, 0x38efb23c00000000, + 0xff79f37300000000, 0xbe48e86a00000000, 0x7d1bc54100000000, + 0x3c2ade5800000000, 0x054f79f000000000, 0x447e62e900000000, + 0x872d4fc200000000, 0xc61c54db00000000, 0x018a159400000000, + 0x40bb0e8d00000000, 0x83e823a600000000, 0xc2d938bf00000000, + 0x0dc5a03800000000, 0x4cf4bb2100000000, 0x8fa7960a00000000, + 0xce968d1300000000, 0x0900cc5c00000000, 0x4831d74500000000, + 0x8b62fa6e00000000, 0xca53e17700000000, 0x545dbbba00000000, + 0x156ca0a300000000, 0xd63f8d8800000000, 0x970e969100000000, + 0x5098d7de00000000, 0x11a9ccc700000000, 0xd2fae1ec00000000, + 0x93cbfaf500000000, 0x5cd7627200000000, 0x1de6796b00000000, + 0xdeb5544000000000, 0x9f844f5900000000, 0x58120e1600000000, + 0x1923150f00000000, 0xda70382400000000, 0x9b41233d00000000, + 0xa76bfd6500000000, 0xe65ae67c00000000, 0x2509cb5700000000, + 0x6438d04e00000000, 0xa3ae910100000000, 0xe29f8a1800000000, + 0x21cca73300000000, 0x60fdbc2a00000000, 0xafe124ad00000000, + 0xeed03fb400000000, 0x2d83129f00000000, 0x6cb2098600000000, + 0xab2448c900000000, 0xea1553d000000000, 0x29467efb00000000, + 0x687765e200000000, 0xf6793f2f00000000, 0xb748243600000000, + 0x741b091d00000000, 0x352a120400000000, 0xf2bc534b00000000, + 0xb38d485200000000, 0x70de657900000000, 0x31ef7e6000000000, + 0xfef3e6e700000000, 0xbfc2fdfe00000000, 0x7c91d0d500000000, + 0x3da0cbcc00000000, 0xfa368a8300000000, 0xbb07919a00000000, + 0x7854bcb100000000, 0x3965a7a800000000, 0x4b98833b00000000, + 0x0aa9982200000000, 0xc9fab50900000000, 0x88cbae1000000000, + 0x4f5def5f00000000, 0x0e6cf44600000000, 0xcd3fd96d00000000, + 0x8c0ec27400000000, 0x43125af300000000, 0x022341ea00000000, + 0xc1706cc100000000, 0x804177d800000000, 0x47d7369700000000, + 0x06e62d8e00000000, 0xc5b500a500000000, 0x84841bbc00000000, + 0x1a8a417100000000, 0x5bbb5a6800000000, 0x98e8774300000000, + 0xd9d96c5a00000000, 0x1e4f2d1500000000, 0x5f7e360c00000000, + 0x9c2d1b2700000000, 0xdd1c003e00000000, 0x120098b900000000, + 0x533183a000000000, 0x9062ae8b00000000, 0xd153b59200000000, + 0x16c5f4dd00000000, 0x57f4efc400000000, 0x94a7c2ef00000000, + 0xd596d9f600000000, 0xe9bc07ae00000000, 0xa88d1cb700000000, + 0x6bde319c00000000, 0x2aef2a8500000000, 0xed796bca00000000, + 0xac4870d300000000, 0x6f1b5df800000000, 0x2e2a46e100000000, + 0xe136de6600000000, 0xa007c57f00000000, 0x6354e85400000000, + 0x2265f34d00000000, 0xe5f3b20200000000, 0xa4c2a91b00000000, + 0x6791843000000000, 0x26a09f2900000000, 0xb8aec5e400000000, + 0xf99fdefd00000000, 0x3accf3d600000000, 0x7bfde8cf00000000, + 0xbc6ba98000000000, 0xfd5ab29900000000, 0x3e099fb200000000, + 0x7f3884ab00000000, 0xb0241c2c00000000, 0xf115073500000000, + 0x32462a1e00000000, 0x7377310700000000, 0xb4e1704800000000, + 0xf5d06b5100000000, 0x3683467a00000000, 0x77b25d6300000000, + 0x4ed7facb00000000, 0x0fe6e1d200000000, 0xccb5ccf900000000, + 0x8d84d7e000000000, 0x4a1296af00000000, 0x0b238db600000000, + 0xc870a09d00000000, 0x8941bb8400000000, 0x465d230300000000, + 0x076c381a00000000, 0xc43f153100000000, 0x850e0e2800000000, + 0x42984f6700000000, 0x03a9547e00000000, 0xc0fa795500000000, + 0x81cb624c00000000, 0x1fc5388100000000, 0x5ef4239800000000, + 0x9da70eb300000000, 0xdc9615aa00000000, 0x1b0054e500000000, + 0x5a314ffc00000000, 0x996262d700000000, 0xd85379ce00000000, + 0x174fe14900000000, 0x567efa5000000000, 0x952dd77b00000000, + 0xd41ccc6200000000, 0x138a8d2d00000000, 0x52bb963400000000, + 0x91e8bb1f00000000, 0xd0d9a00600000000, 0xecf37e5e00000000, + 0xadc2654700000000, 0x6e91486c00000000, 0x2fa0537500000000, + 0xe836123a00000000, 0xa907092300000000, 0x6a54240800000000, + 0x2b653f1100000000, 0xe479a79600000000, 0xa548bc8f00000000, + 0x661b91a400000000, 0x272a8abd00000000, 0xe0bccbf200000000, + 0xa18dd0eb00000000, 0x62defdc000000000, 0x23efe6d900000000, + 0xbde1bc1400000000, 0xfcd0a70d00000000, 0x3f838a2600000000, + 0x7eb2913f00000000, 0xb924d07000000000, 0xf815cb6900000000, + 0x3b46e64200000000, 0x7a77fd5b00000000, 0xb56b65dc00000000, + 0xf45a7ec500000000, 0x370953ee00000000, 0x763848f700000000, + 0xb1ae09b800000000, 0xf09f12a100000000, 0x33cc3f8a00000000, + 0x72fd249300000000}, + {0x0000000000000000, 0x376ac20100000000, 0x6ed4840300000000, + 0x59be460200000000, 0xdca8090700000000, 0xebc2cb0600000000, + 0xb27c8d0400000000, 0x85164f0500000000, 0xb851130e00000000, + 0x8f3bd10f00000000, 0xd685970d00000000, 0xe1ef550c00000000, + 0x64f91a0900000000, 0x5393d80800000000, 0x0a2d9e0a00000000, + 0x3d475c0b00000000, 0x70a3261c00000000, 0x47c9e41d00000000, + 0x1e77a21f00000000, 0x291d601e00000000, 0xac0b2f1b00000000, + 0x9b61ed1a00000000, 0xc2dfab1800000000, 0xf5b5691900000000, + 0xc8f2351200000000, 0xff98f71300000000, 0xa626b11100000000, + 0x914c731000000000, 0x145a3c1500000000, 0x2330fe1400000000, + 0x7a8eb81600000000, 0x4de47a1700000000, 0xe0464d3800000000, + 0xd72c8f3900000000, 0x8e92c93b00000000, 0xb9f80b3a00000000, + 0x3cee443f00000000, 0x0b84863e00000000, 0x523ac03c00000000, + 0x6550023d00000000, 0x58175e3600000000, 0x6f7d9c3700000000, + 0x36c3da3500000000, 0x01a9183400000000, 0x84bf573100000000, + 0xb3d5953000000000, 0xea6bd33200000000, 0xdd01113300000000, + 0x90e56b2400000000, 0xa78fa92500000000, 0xfe31ef2700000000, + 0xc95b2d2600000000, 0x4c4d622300000000, 0x7b27a02200000000, + 0x2299e62000000000, 0x15f3242100000000, 0x28b4782a00000000, + 0x1fdeba2b00000000, 0x4660fc2900000000, 0x710a3e2800000000, + 0xf41c712d00000000, 0xc376b32c00000000, 0x9ac8f52e00000000, + 0xada2372f00000000, 0xc08d9a7000000000, 0xf7e7587100000000, + 0xae591e7300000000, 0x9933dc7200000000, 0x1c25937700000000, + 0x2b4f517600000000, 0x72f1177400000000, 0x459bd57500000000, + 0x78dc897e00000000, 0x4fb64b7f00000000, 0x16080d7d00000000, + 0x2162cf7c00000000, 0xa474807900000000, 0x931e427800000000, + 0xcaa0047a00000000, 0xfdcac67b00000000, 0xb02ebc6c00000000, + 0x87447e6d00000000, 0xdefa386f00000000, 0xe990fa6e00000000, + 0x6c86b56b00000000, 0x5bec776a00000000, 0x0252316800000000, + 0x3538f36900000000, 0x087faf6200000000, 0x3f156d6300000000, + 0x66ab2b6100000000, 0x51c1e96000000000, 0xd4d7a66500000000, + 0xe3bd646400000000, 0xba03226600000000, 0x8d69e06700000000, + 0x20cbd74800000000, 0x17a1154900000000, 0x4e1f534b00000000, + 0x7975914a00000000, 0xfc63de4f00000000, 0xcb091c4e00000000, + 0x92b75a4c00000000, 0xa5dd984d00000000, 0x989ac44600000000, + 0xaff0064700000000, 0xf64e404500000000, 0xc124824400000000, + 0x4432cd4100000000, 0x73580f4000000000, 0x2ae6494200000000, + 0x1d8c8b4300000000, 0x5068f15400000000, 0x6702335500000000, + 0x3ebc755700000000, 0x09d6b75600000000, 0x8cc0f85300000000, + 0xbbaa3a5200000000, 0xe2147c5000000000, 0xd57ebe5100000000, + 0xe839e25a00000000, 0xdf53205b00000000, 0x86ed665900000000, + 0xb187a45800000000, 0x3491eb5d00000000, 0x03fb295c00000000, + 0x5a456f5e00000000, 0x6d2fad5f00000000, 0x801b35e100000000, + 0xb771f7e000000000, 0xeecfb1e200000000, 0xd9a573e300000000, + 0x5cb33ce600000000, 0x6bd9fee700000000, 0x3267b8e500000000, + 0x050d7ae400000000, 0x384a26ef00000000, 0x0f20e4ee00000000, + 0x569ea2ec00000000, 0x61f460ed00000000, 0xe4e22fe800000000, + 0xd388ede900000000, 0x8a36abeb00000000, 0xbd5c69ea00000000, + 0xf0b813fd00000000, 0xc7d2d1fc00000000, 0x9e6c97fe00000000, + 0xa90655ff00000000, 0x2c101afa00000000, 0x1b7ad8fb00000000, + 0x42c49ef900000000, 0x75ae5cf800000000, 0x48e900f300000000, + 0x7f83c2f200000000, 0x263d84f000000000, 0x115746f100000000, + 0x944109f400000000, 0xa32bcbf500000000, 0xfa958df700000000, + 0xcdff4ff600000000, 0x605d78d900000000, 0x5737bad800000000, + 0x0e89fcda00000000, 0x39e33edb00000000, 0xbcf571de00000000, + 0x8b9fb3df00000000, 0xd221f5dd00000000, 0xe54b37dc00000000, + 0xd80c6bd700000000, 0xef66a9d600000000, 0xb6d8efd400000000, + 0x81b22dd500000000, 0x04a462d000000000, 0x33cea0d100000000, + 0x6a70e6d300000000, 0x5d1a24d200000000, 0x10fe5ec500000000, + 0x27949cc400000000, 0x7e2adac600000000, 0x494018c700000000, + 0xcc5657c200000000, 0xfb3c95c300000000, 0xa282d3c100000000, + 0x95e811c000000000, 0xa8af4dcb00000000, 0x9fc58fca00000000, + 0xc67bc9c800000000, 0xf1110bc900000000, 0x740744cc00000000, + 0x436d86cd00000000, 0x1ad3c0cf00000000, 0x2db902ce00000000, + 0x4096af9100000000, 0x77fc6d9000000000, 0x2e422b9200000000, + 0x1928e99300000000, 0x9c3ea69600000000, 0xab54649700000000, + 0xf2ea229500000000, 0xc580e09400000000, 0xf8c7bc9f00000000, + 0xcfad7e9e00000000, 0x9613389c00000000, 0xa179fa9d00000000, + 0x246fb59800000000, 0x1305779900000000, 0x4abb319b00000000, + 0x7dd1f39a00000000, 0x3035898d00000000, 0x075f4b8c00000000, + 0x5ee10d8e00000000, 0x698bcf8f00000000, 0xec9d808a00000000, + 0xdbf7428b00000000, 0x8249048900000000, 0xb523c68800000000, + 0x88649a8300000000, 0xbf0e588200000000, 0xe6b01e8000000000, + 0xd1dadc8100000000, 0x54cc938400000000, 0x63a6518500000000, + 0x3a18178700000000, 0x0d72d58600000000, 0xa0d0e2a900000000, + 0x97ba20a800000000, 0xce0466aa00000000, 0xf96ea4ab00000000, + 0x7c78ebae00000000, 0x4b1229af00000000, 0x12ac6fad00000000, + 0x25c6adac00000000, 0x1881f1a700000000, 0x2feb33a600000000, + 0x765575a400000000, 0x413fb7a500000000, 0xc429f8a000000000, + 0xf3433aa100000000, 0xaafd7ca300000000, 0x9d97bea200000000, + 0xd073c4b500000000, 0xe71906b400000000, 0xbea740b600000000, + 0x89cd82b700000000, 0x0cdbcdb200000000, 0x3bb10fb300000000, + 0x620f49b100000000, 0x55658bb000000000, 0x6822d7bb00000000, + 0x5f4815ba00000000, 0x06f653b800000000, 0x319c91b900000000, + 0xb48adebc00000000, 0x83e01cbd00000000, 0xda5e5abf00000000, + 0xed3498be00000000}, + {0x0000000000000000, 0x6567bcb800000000, 0x8bc809aa00000000, + 0xeeafb51200000000, 0x5797628f00000000, 0x32f0de3700000000, + 0xdc5f6b2500000000, 0xb938d79d00000000, 0xef28b4c500000000, + 0x8a4f087d00000000, 0x64e0bd6f00000000, 0x018701d700000000, + 0xb8bfd64a00000000, 0xddd86af200000000, 0x3377dfe000000000, + 0x5610635800000000, 0x9f57195000000000, 0xfa30a5e800000000, + 0x149f10fa00000000, 0x71f8ac4200000000, 0xc8c07bdf00000000, + 0xada7c76700000000, 0x4308727500000000, 0x266fcecd00000000, + 0x707fad9500000000, 0x1518112d00000000, 0xfbb7a43f00000000, + 0x9ed0188700000000, 0x27e8cf1a00000000, 0x428f73a200000000, + 0xac20c6b000000000, 0xc9477a0800000000, 0x3eaf32a000000000, + 0x5bc88e1800000000, 0xb5673b0a00000000, 0xd00087b200000000, + 0x6938502f00000000, 0x0c5fec9700000000, 0xe2f0598500000000, + 0x8797e53d00000000, 0xd187866500000000, 0xb4e03add00000000, + 0x5a4f8fcf00000000, 0x3f28337700000000, 0x8610e4ea00000000, + 0xe377585200000000, 0x0dd8ed4000000000, 0x68bf51f800000000, + 0xa1f82bf000000000, 0xc49f974800000000, 0x2a30225a00000000, + 0x4f579ee200000000, 0xf66f497f00000000, 0x9308f5c700000000, + 0x7da740d500000000, 0x18c0fc6d00000000, 0x4ed09f3500000000, + 0x2bb7238d00000000, 0xc518969f00000000, 0xa07f2a2700000000, + 0x1947fdba00000000, 0x7c20410200000000, 0x928ff41000000000, + 0xf7e848a800000000, 0x3d58149b00000000, 0x583fa82300000000, + 0xb6901d3100000000, 0xd3f7a18900000000, 0x6acf761400000000, + 0x0fa8caac00000000, 0xe1077fbe00000000, 0x8460c30600000000, + 0xd270a05e00000000, 0xb7171ce600000000, 0x59b8a9f400000000, + 0x3cdf154c00000000, 0x85e7c2d100000000, 0xe0807e6900000000, + 0x0e2fcb7b00000000, 0x6b4877c300000000, 0xa20f0dcb00000000, + 0xc768b17300000000, 0x29c7046100000000, 0x4ca0b8d900000000, + 0xf5986f4400000000, 0x90ffd3fc00000000, 0x7e5066ee00000000, + 0x1b37da5600000000, 0x4d27b90e00000000, 0x284005b600000000, + 0xc6efb0a400000000, 0xa3880c1c00000000, 0x1ab0db8100000000, + 0x7fd7673900000000, 0x9178d22b00000000, 0xf41f6e9300000000, + 0x03f7263b00000000, 0x66909a8300000000, 0x883f2f9100000000, + 0xed58932900000000, 0x546044b400000000, 0x3107f80c00000000, + 0xdfa84d1e00000000, 0xbacff1a600000000, 0xecdf92fe00000000, + 0x89b82e4600000000, 0x67179b5400000000, 0x027027ec00000000, + 0xbb48f07100000000, 0xde2f4cc900000000, 0x3080f9db00000000, + 0x55e7456300000000, 0x9ca03f6b00000000, 0xf9c783d300000000, + 0x176836c100000000, 0x720f8a7900000000, 0xcb375de400000000, + 0xae50e15c00000000, 0x40ff544e00000000, 0x2598e8f600000000, + 0x73888bae00000000, 0x16ef371600000000, 0xf840820400000000, + 0x9d273ebc00000000, 0x241fe92100000000, 0x4178559900000000, + 0xafd7e08b00000000, 0xcab05c3300000000, 0x3bb659ed00000000, + 0x5ed1e55500000000, 0xb07e504700000000, 0xd519ecff00000000, + 0x6c213b6200000000, 0x094687da00000000, 0xe7e932c800000000, + 0x828e8e7000000000, 0xd49eed2800000000, 0xb1f9519000000000, + 0x5f56e48200000000, 0x3a31583a00000000, 0x83098fa700000000, + 0xe66e331f00000000, 0x08c1860d00000000, 0x6da63ab500000000, + 0xa4e140bd00000000, 0xc186fc0500000000, 0x2f29491700000000, + 0x4a4ef5af00000000, 0xf376223200000000, 0x96119e8a00000000, + 0x78be2b9800000000, 0x1dd9972000000000, 0x4bc9f47800000000, + 0x2eae48c000000000, 0xc001fdd200000000, 0xa566416a00000000, + 0x1c5e96f700000000, 0x79392a4f00000000, 0x97969f5d00000000, + 0xf2f123e500000000, 0x05196b4d00000000, 0x607ed7f500000000, + 0x8ed162e700000000, 0xebb6de5f00000000, 0x528e09c200000000, + 0x37e9b57a00000000, 0xd946006800000000, 0xbc21bcd000000000, + 0xea31df8800000000, 0x8f56633000000000, 0x61f9d62200000000, + 0x049e6a9a00000000, 0xbda6bd0700000000, 0xd8c101bf00000000, + 0x366eb4ad00000000, 0x5309081500000000, 0x9a4e721d00000000, + 0xff29cea500000000, 0x11867bb700000000, 0x74e1c70f00000000, + 0xcdd9109200000000, 0xa8beac2a00000000, 0x4611193800000000, + 0x2376a58000000000, 0x7566c6d800000000, 0x10017a6000000000, + 0xfeaecf7200000000, 0x9bc973ca00000000, 0x22f1a45700000000, + 0x479618ef00000000, 0xa939adfd00000000, 0xcc5e114500000000, + 0x06ee4d7600000000, 0x6389f1ce00000000, 0x8d2644dc00000000, + 0xe841f86400000000, 0x51792ff900000000, 0x341e934100000000, + 0xdab1265300000000, 0xbfd69aeb00000000, 0xe9c6f9b300000000, + 0x8ca1450b00000000, 0x620ef01900000000, 0x07694ca100000000, + 0xbe519b3c00000000, 0xdb36278400000000, 0x3599929600000000, + 0x50fe2e2e00000000, 0x99b9542600000000, 0xfcdee89e00000000, + 0x12715d8c00000000, 0x7716e13400000000, 0xce2e36a900000000, + 0xab498a1100000000, 0x45e63f0300000000, 0x208183bb00000000, + 0x7691e0e300000000, 0x13f65c5b00000000, 0xfd59e94900000000, + 0x983e55f100000000, 0x2106826c00000000, 0x44613ed400000000, + 0xaace8bc600000000, 0xcfa9377e00000000, 0x38417fd600000000, + 0x5d26c36e00000000, 0xb389767c00000000, 0xd6eecac400000000, + 0x6fd61d5900000000, 0x0ab1a1e100000000, 0xe41e14f300000000, + 0x8179a84b00000000, 0xd769cb1300000000, 0xb20e77ab00000000, + 0x5ca1c2b900000000, 0x39c67e0100000000, 0x80fea99c00000000, + 0xe599152400000000, 0x0b36a03600000000, 0x6e511c8e00000000, + 0xa716668600000000, 0xc271da3e00000000, 0x2cde6f2c00000000, + 0x49b9d39400000000, 0xf081040900000000, 0x95e6b8b100000000, + 0x7b490da300000000, 0x1e2eb11b00000000, 0x483ed24300000000, + 0x2d596efb00000000, 0xc3f6dbe900000000, 0xa691675100000000, + 0x1fa9b0cc00000000, 0x7ace0c7400000000, 0x9461b96600000000, + 0xf10605de00000000}, + {0x0000000000000000, 0xb029603d00000000, 0x6053c07a00000000, + 0xd07aa04700000000, 0xc0a680f500000000, 0x708fe0c800000000, + 0xa0f5408f00000000, 0x10dc20b200000000, 0xc14b703000000000, + 0x7162100d00000000, 0xa118b04a00000000, 0x1131d07700000000, + 0x01edf0c500000000, 0xb1c490f800000000, 0x61be30bf00000000, + 0xd197508200000000, 0x8297e06000000000, 0x32be805d00000000, + 0xe2c4201a00000000, 0x52ed402700000000, 0x4231609500000000, + 0xf21800a800000000, 0x2262a0ef00000000, 0x924bc0d200000000, + 0x43dc905000000000, 0xf3f5f06d00000000, 0x238f502a00000000, + 0x93a6301700000000, 0x837a10a500000000, 0x3353709800000000, + 0xe329d0df00000000, 0x5300b0e200000000, 0x042fc1c100000000, + 0xb406a1fc00000000, 0x647c01bb00000000, 0xd455618600000000, + 0xc489413400000000, 0x74a0210900000000, 0xa4da814e00000000, + 0x14f3e17300000000, 0xc564b1f100000000, 0x754dd1cc00000000, + 0xa537718b00000000, 0x151e11b600000000, 0x05c2310400000000, + 0xb5eb513900000000, 0x6591f17e00000000, 0xd5b8914300000000, + 0x86b821a100000000, 0x3691419c00000000, 0xe6ebe1db00000000, + 0x56c281e600000000, 0x461ea15400000000, 0xf637c16900000000, + 0x264d612e00000000, 0x9664011300000000, 0x47f3519100000000, + 0xf7da31ac00000000, 0x27a091eb00000000, 0x9789f1d600000000, + 0x8755d16400000000, 0x377cb15900000000, 0xe706111e00000000, + 0x572f712300000000, 0x4958f35800000000, 0xf971936500000000, + 0x290b332200000000, 0x9922531f00000000, 0x89fe73ad00000000, + 0x39d7139000000000, 0xe9adb3d700000000, 0x5984d3ea00000000, + 0x8813836800000000, 0x383ae35500000000, 0xe840431200000000, + 0x5869232f00000000, 0x48b5039d00000000, 0xf89c63a000000000, + 0x28e6c3e700000000, 0x98cfa3da00000000, 0xcbcf133800000000, + 0x7be6730500000000, 0xab9cd34200000000, 0x1bb5b37f00000000, + 0x0b6993cd00000000, 0xbb40f3f000000000, 0x6b3a53b700000000, + 0xdb13338a00000000, 0x0a84630800000000, 0xbaad033500000000, + 0x6ad7a37200000000, 0xdafec34f00000000, 0xca22e3fd00000000, + 0x7a0b83c000000000, 0xaa71238700000000, 0x1a5843ba00000000, + 0x4d77329900000000, 0xfd5e52a400000000, 0x2d24f2e300000000, + 0x9d0d92de00000000, 0x8dd1b26c00000000, 0x3df8d25100000000, + 0xed82721600000000, 0x5dab122b00000000, 0x8c3c42a900000000, + 0x3c15229400000000, 0xec6f82d300000000, 0x5c46e2ee00000000, + 0x4c9ac25c00000000, 0xfcb3a26100000000, 0x2cc9022600000000, + 0x9ce0621b00000000, 0xcfe0d2f900000000, 0x7fc9b2c400000000, + 0xafb3128300000000, 0x1f9a72be00000000, 0x0f46520c00000000, + 0xbf6f323100000000, 0x6f15927600000000, 0xdf3cf24b00000000, + 0x0eaba2c900000000, 0xbe82c2f400000000, 0x6ef862b300000000, + 0xded1028e00000000, 0xce0d223c00000000, 0x7e24420100000000, + 0xae5ee24600000000, 0x1e77827b00000000, 0x92b0e6b100000000, + 0x2299868c00000000, 0xf2e326cb00000000, 0x42ca46f600000000, + 0x5216664400000000, 0xe23f067900000000, 0x3245a63e00000000, + 0x826cc60300000000, 0x53fb968100000000, 0xe3d2f6bc00000000, + 0x33a856fb00000000, 0x838136c600000000, 0x935d167400000000, + 0x2374764900000000, 0xf30ed60e00000000, 0x4327b63300000000, + 0x102706d100000000, 0xa00e66ec00000000, 0x7074c6ab00000000, + 0xc05da69600000000, 0xd081862400000000, 0x60a8e61900000000, + 0xb0d2465e00000000, 0x00fb266300000000, 0xd16c76e100000000, + 0x614516dc00000000, 0xb13fb69b00000000, 0x0116d6a600000000, + 0x11caf61400000000, 0xa1e3962900000000, 0x7199366e00000000, + 0xc1b0565300000000, 0x969f277000000000, 0x26b6474d00000000, + 0xf6cce70a00000000, 0x46e5873700000000, 0x5639a78500000000, + 0xe610c7b800000000, 0x366a67ff00000000, 0x864307c200000000, + 0x57d4574000000000, 0xe7fd377d00000000, 0x3787973a00000000, + 0x87aef70700000000, 0x9772d7b500000000, 0x275bb78800000000, + 0xf72117cf00000000, 0x470877f200000000, 0x1408c71000000000, + 0xa421a72d00000000, 0x745b076a00000000, 0xc472675700000000, + 0xd4ae47e500000000, 0x648727d800000000, 0xb4fd879f00000000, + 0x04d4e7a200000000, 0xd543b72000000000, 0x656ad71d00000000, + 0xb510775a00000000, 0x0539176700000000, 0x15e537d500000000, + 0xa5cc57e800000000, 0x75b6f7af00000000, 0xc59f979200000000, + 0xdbe815e900000000, 0x6bc175d400000000, 0xbbbbd59300000000, + 0x0b92b5ae00000000, 0x1b4e951c00000000, 0xab67f52100000000, + 0x7b1d556600000000, 0xcb34355b00000000, 0x1aa365d900000000, + 0xaa8a05e400000000, 0x7af0a5a300000000, 0xcad9c59e00000000, + 0xda05e52c00000000, 0x6a2c851100000000, 0xba56255600000000, + 0x0a7f456b00000000, 0x597ff58900000000, 0xe95695b400000000, + 0x392c35f300000000, 0x890555ce00000000, 0x99d9757c00000000, + 0x29f0154100000000, 0xf98ab50600000000, 0x49a3d53b00000000, + 0x983485b900000000, 0x281de58400000000, 0xf86745c300000000, + 0x484e25fe00000000, 0x5892054c00000000, 0xe8bb657100000000, + 0x38c1c53600000000, 0x88e8a50b00000000, 0xdfc7d42800000000, + 0x6feeb41500000000, 0xbf94145200000000, 0x0fbd746f00000000, + 0x1f6154dd00000000, 0xaf4834e000000000, 0x7f3294a700000000, + 0xcf1bf49a00000000, 0x1e8ca41800000000, 0xaea5c42500000000, + 0x7edf646200000000, 0xcef6045f00000000, 0xde2a24ed00000000, + 0x6e0344d000000000, 0xbe79e49700000000, 0x0e5084aa00000000, + 0x5d50344800000000, 0xed79547500000000, 0x3d03f43200000000, + 0x8d2a940f00000000, 0x9df6b4bd00000000, 0x2ddfd48000000000, + 0xfda574c700000000, 0x4d8c14fa00000000, 0x9c1b447800000000, + 0x2c32244500000000, 0xfc48840200000000, 0x4c61e43f00000000, + 0x5cbdc48d00000000, 0xec94a4b000000000, 0x3cee04f700000000, + 0x8cc764ca00000000}, + {0x0000000000000000, 0xa5d35ccb00000000, 0x0ba1c84d00000000, + 0xae72948600000000, 0x1642919b00000000, 0xb391cd5000000000, + 0x1de359d600000000, 0xb830051d00000000, 0x6d8253ec00000000, + 0xc8510f2700000000, 0x66239ba100000000, 0xc3f0c76a00000000, + 0x7bc0c27700000000, 0xde139ebc00000000, 0x70610a3a00000000, + 0xd5b256f100000000, 0x9b02d60300000000, 0x3ed18ac800000000, + 0x90a31e4e00000000, 0x3570428500000000, 0x8d40479800000000, + 0x28931b5300000000, 0x86e18fd500000000, 0x2332d31e00000000, + 0xf68085ef00000000, 0x5353d92400000000, 0xfd214da200000000, + 0x58f2116900000000, 0xe0c2147400000000, 0x451148bf00000000, + 0xeb63dc3900000000, 0x4eb080f200000000, 0x3605ac0700000000, + 0x93d6f0cc00000000, 0x3da4644a00000000, 0x9877388100000000, + 0x20473d9c00000000, 0x8594615700000000, 0x2be6f5d100000000, + 0x8e35a91a00000000, 0x5b87ffeb00000000, 0xfe54a32000000000, + 0x502637a600000000, 0xf5f56b6d00000000, 0x4dc56e7000000000, + 0xe81632bb00000000, 0x4664a63d00000000, 0xe3b7faf600000000, + 0xad077a0400000000, 0x08d426cf00000000, 0xa6a6b24900000000, + 0x0375ee8200000000, 0xbb45eb9f00000000, 0x1e96b75400000000, + 0xb0e423d200000000, 0x15377f1900000000, 0xc08529e800000000, + 0x6556752300000000, 0xcb24e1a500000000, 0x6ef7bd6e00000000, + 0xd6c7b87300000000, 0x7314e4b800000000, 0xdd66703e00000000, + 0x78b52cf500000000, 0x6c0a580f00000000, 0xc9d904c400000000, + 0x67ab904200000000, 0xc278cc8900000000, 0x7a48c99400000000, + 0xdf9b955f00000000, 0x71e901d900000000, 0xd43a5d1200000000, + 0x01880be300000000, 0xa45b572800000000, 0x0a29c3ae00000000, + 0xaffa9f6500000000, 0x17ca9a7800000000, 0xb219c6b300000000, + 0x1c6b523500000000, 0xb9b80efe00000000, 0xf7088e0c00000000, + 0x52dbd2c700000000, 0xfca9464100000000, 0x597a1a8a00000000, + 0xe14a1f9700000000, 0x4499435c00000000, 0xeaebd7da00000000, + 0x4f388b1100000000, 0x9a8adde000000000, 0x3f59812b00000000, + 0x912b15ad00000000, 0x34f8496600000000, 0x8cc84c7b00000000, + 0x291b10b000000000, 0x8769843600000000, 0x22bad8fd00000000, + 0x5a0ff40800000000, 0xffdca8c300000000, 0x51ae3c4500000000, + 0xf47d608e00000000, 0x4c4d659300000000, 0xe99e395800000000, + 0x47ecadde00000000, 0xe23ff11500000000, 0x378da7e400000000, + 0x925efb2f00000000, 0x3c2c6fa900000000, 0x99ff336200000000, + 0x21cf367f00000000, 0x841c6ab400000000, 0x2a6efe3200000000, + 0x8fbda2f900000000, 0xc10d220b00000000, 0x64de7ec000000000, + 0xcaacea4600000000, 0x6f7fb68d00000000, 0xd74fb39000000000, + 0x729cef5b00000000, 0xdcee7bdd00000000, 0x793d271600000000, + 0xac8f71e700000000, 0x095c2d2c00000000, 0xa72eb9aa00000000, + 0x02fde56100000000, 0xbacde07c00000000, 0x1f1ebcb700000000, + 0xb16c283100000000, 0x14bf74fa00000000, 0xd814b01e00000000, + 0x7dc7ecd500000000, 0xd3b5785300000000, 0x7666249800000000, + 0xce56218500000000, 0x6b857d4e00000000, 0xc5f7e9c800000000, + 0x6024b50300000000, 0xb596e3f200000000, 0x1045bf3900000000, + 0xbe372bbf00000000, 0x1be4777400000000, 0xa3d4726900000000, + 0x06072ea200000000, 0xa875ba2400000000, 0x0da6e6ef00000000, + 0x4316661d00000000, 0xe6c53ad600000000, 0x48b7ae5000000000, + 0xed64f29b00000000, 0x5554f78600000000, 0xf087ab4d00000000, + 0x5ef53fcb00000000, 0xfb26630000000000, 0x2e9435f100000000, + 0x8b47693a00000000, 0x2535fdbc00000000, 0x80e6a17700000000, + 0x38d6a46a00000000, 0x9d05f8a100000000, 0x33776c2700000000, + 0x96a430ec00000000, 0xee111c1900000000, 0x4bc240d200000000, + 0xe5b0d45400000000, 0x4063889f00000000, 0xf8538d8200000000, + 0x5d80d14900000000, 0xf3f245cf00000000, 0x5621190400000000, + 0x83934ff500000000, 0x2640133e00000000, 0x883287b800000000, + 0x2de1db7300000000, 0x95d1de6e00000000, 0x300282a500000000, + 0x9e70162300000000, 0x3ba34ae800000000, 0x7513ca1a00000000, + 0xd0c096d100000000, 0x7eb2025700000000, 0xdb615e9c00000000, + 0x63515b8100000000, 0xc682074a00000000, 0x68f093cc00000000, + 0xcd23cf0700000000, 0x189199f600000000, 0xbd42c53d00000000, + 0x133051bb00000000, 0xb6e30d7000000000, 0x0ed3086d00000000, + 0xab0054a600000000, 0x0572c02000000000, 0xa0a19ceb00000000, + 0xb41ee81100000000, 0x11cdb4da00000000, 0xbfbf205c00000000, + 0x1a6c7c9700000000, 0xa25c798a00000000, 0x078f254100000000, + 0xa9fdb1c700000000, 0x0c2eed0c00000000, 0xd99cbbfd00000000, + 0x7c4fe73600000000, 0xd23d73b000000000, 0x77ee2f7b00000000, + 0xcfde2a6600000000, 0x6a0d76ad00000000, 0xc47fe22b00000000, + 0x61acbee000000000, 0x2f1c3e1200000000, 0x8acf62d900000000, + 0x24bdf65f00000000, 0x816eaa9400000000, 0x395eaf8900000000, + 0x9c8df34200000000, 0x32ff67c400000000, 0x972c3b0f00000000, + 0x429e6dfe00000000, 0xe74d313500000000, 0x493fa5b300000000, + 0xececf97800000000, 0x54dcfc6500000000, 0xf10fa0ae00000000, + 0x5f7d342800000000, 0xfaae68e300000000, 0x821b441600000000, + 0x27c818dd00000000, 0x89ba8c5b00000000, 0x2c69d09000000000, + 0x9459d58d00000000, 0x318a894600000000, 0x9ff81dc000000000, + 0x3a2b410b00000000, 0xef9917fa00000000, 0x4a4a4b3100000000, + 0xe438dfb700000000, 0x41eb837c00000000, 0xf9db866100000000, + 0x5c08daaa00000000, 0xf27a4e2c00000000, 0x57a912e700000000, + 0x1919921500000000, 0xbccacede00000000, 0x12b85a5800000000, + 0xb76b069300000000, 0x0f5b038e00000000, 0xaa885f4500000000, + 0x04facbc300000000, 0xa129970800000000, 0x749bc1f900000000, + 0xd1489d3200000000, 0x7f3a09b400000000, 0xdae9557f00000000, + 0x62d9506200000000, 0xc70a0ca900000000, 0x6978982f00000000, + 0xccabc4e400000000}, + {0x0000000000000000, 0xb40b77a600000000, 0x29119f9700000000, + 0x9d1ae83100000000, 0x13244ff400000000, 0xa72f385200000000, + 0x3a35d06300000000, 0x8e3ea7c500000000, 0x674eef3300000000, + 0xd345989500000000, 0x4e5f70a400000000, 0xfa54070200000000, + 0x746aa0c700000000, 0xc061d76100000000, 0x5d7b3f5000000000, + 0xe97048f600000000, 0xce9cde6700000000, 0x7a97a9c100000000, + 0xe78d41f000000000, 0x5386365600000000, 0xddb8919300000000, + 0x69b3e63500000000, 0xf4a90e0400000000, 0x40a279a200000000, + 0xa9d2315400000000, 0x1dd946f200000000, 0x80c3aec300000000, + 0x34c8d96500000000, 0xbaf67ea000000000, 0x0efd090600000000, + 0x93e7e13700000000, 0x27ec969100000000, 0x9c39bdcf00000000, + 0x2832ca6900000000, 0xb528225800000000, 0x012355fe00000000, + 0x8f1df23b00000000, 0x3b16859d00000000, 0xa60c6dac00000000, + 0x12071a0a00000000, 0xfb7752fc00000000, 0x4f7c255a00000000, + 0xd266cd6b00000000, 0x666dbacd00000000, 0xe8531d0800000000, + 0x5c586aae00000000, 0xc142829f00000000, 0x7549f53900000000, + 0x52a563a800000000, 0xe6ae140e00000000, 0x7bb4fc3f00000000, + 0xcfbf8b9900000000, 0x41812c5c00000000, 0xf58a5bfa00000000, + 0x6890b3cb00000000, 0xdc9bc46d00000000, 0x35eb8c9b00000000, + 0x81e0fb3d00000000, 0x1cfa130c00000000, 0xa8f164aa00000000, + 0x26cfc36f00000000, 0x92c4b4c900000000, 0x0fde5cf800000000, + 0xbbd52b5e00000000, 0x79750b4400000000, 0xcd7e7ce200000000, + 0x506494d300000000, 0xe46fe37500000000, 0x6a5144b000000000, + 0xde5a331600000000, 0x4340db2700000000, 0xf74bac8100000000, + 0x1e3be47700000000, 0xaa3093d100000000, 0x372a7be000000000, + 0x83210c4600000000, 0x0d1fab8300000000, 0xb914dc2500000000, + 0x240e341400000000, 0x900543b200000000, 0xb7e9d52300000000, + 0x03e2a28500000000, 0x9ef84ab400000000, 0x2af33d1200000000, + 0xa4cd9ad700000000, 0x10c6ed7100000000, 0x8ddc054000000000, + 0x39d772e600000000, 0xd0a73a1000000000, 0x64ac4db600000000, + 0xf9b6a58700000000, 0x4dbdd22100000000, 0xc38375e400000000, + 0x7788024200000000, 0xea92ea7300000000, 0x5e999dd500000000, + 0xe54cb68b00000000, 0x5147c12d00000000, 0xcc5d291c00000000, + 0x78565eba00000000, 0xf668f97f00000000, 0x42638ed900000000, + 0xdf7966e800000000, 0x6b72114e00000000, 0x820259b800000000, + 0x36092e1e00000000, 0xab13c62f00000000, 0x1f18b18900000000, + 0x9126164c00000000, 0x252d61ea00000000, 0xb83789db00000000, + 0x0c3cfe7d00000000, 0x2bd068ec00000000, 0x9fdb1f4a00000000, + 0x02c1f77b00000000, 0xb6ca80dd00000000, 0x38f4271800000000, + 0x8cff50be00000000, 0x11e5b88f00000000, 0xa5eecf2900000000, + 0x4c9e87df00000000, 0xf895f07900000000, 0x658f184800000000, + 0xd1846fee00000000, 0x5fbac82b00000000, 0xebb1bf8d00000000, + 0x76ab57bc00000000, 0xc2a0201a00000000, 0xf2ea168800000000, + 0x46e1612e00000000, 0xdbfb891f00000000, 0x6ff0feb900000000, + 0xe1ce597c00000000, 0x55c52eda00000000, 0xc8dfc6eb00000000, + 0x7cd4b14d00000000, 0x95a4f9bb00000000, 0x21af8e1d00000000, + 0xbcb5662c00000000, 0x08be118a00000000, 0x8680b64f00000000, + 0x328bc1e900000000, 0xaf9129d800000000, 0x1b9a5e7e00000000, + 0x3c76c8ef00000000, 0x887dbf4900000000, 0x1567577800000000, + 0xa16c20de00000000, 0x2f52871b00000000, 0x9b59f0bd00000000, + 0x0643188c00000000, 0xb2486f2a00000000, 0x5b3827dc00000000, + 0xef33507a00000000, 0x7229b84b00000000, 0xc622cfed00000000, + 0x481c682800000000, 0xfc171f8e00000000, 0x610df7bf00000000, + 0xd506801900000000, 0x6ed3ab4700000000, 0xdad8dce100000000, + 0x47c234d000000000, 0xf3c9437600000000, 0x7df7e4b300000000, + 0xc9fc931500000000, 0x54e67b2400000000, 0xe0ed0c8200000000, + 0x099d447400000000, 0xbd9633d200000000, 0x208cdbe300000000, + 0x9487ac4500000000, 0x1ab90b8000000000, 0xaeb27c2600000000, + 0x33a8941700000000, 0x87a3e3b100000000, 0xa04f752000000000, + 0x1444028600000000, 0x895eeab700000000, 0x3d559d1100000000, + 0xb36b3ad400000000, 0x07604d7200000000, 0x9a7aa54300000000, + 0x2e71d2e500000000, 0xc7019a1300000000, 0x730aedb500000000, + 0xee10058400000000, 0x5a1b722200000000, 0xd425d5e700000000, + 0x602ea24100000000, 0xfd344a7000000000, 0x493f3dd600000000, + 0x8b9f1dcc00000000, 0x3f946a6a00000000, 0xa28e825b00000000, + 0x1685f5fd00000000, 0x98bb523800000000, 0x2cb0259e00000000, + 0xb1aacdaf00000000, 0x05a1ba0900000000, 0xecd1f2ff00000000, + 0x58da855900000000, 0xc5c06d6800000000, 0x71cb1ace00000000, + 0xfff5bd0b00000000, 0x4bfecaad00000000, 0xd6e4229c00000000, + 0x62ef553a00000000, 0x4503c3ab00000000, 0xf108b40d00000000, + 0x6c125c3c00000000, 0xd8192b9a00000000, 0x56278c5f00000000, + 0xe22cfbf900000000, 0x7f3613c800000000, 0xcb3d646e00000000, + 0x224d2c9800000000, 0x96465b3e00000000, 0x0b5cb30f00000000, + 0xbf57c4a900000000, 0x3169636c00000000, 0x856214ca00000000, + 0x1878fcfb00000000, 0xac738b5d00000000, 0x17a6a00300000000, + 0xa3add7a500000000, 0x3eb73f9400000000, 0x8abc483200000000, + 0x0482eff700000000, 0xb089985100000000, 0x2d93706000000000, + 0x999807c600000000, 0x70e84f3000000000, 0xc4e3389600000000, + 0x59f9d0a700000000, 0xedf2a70100000000, 0x63cc00c400000000, + 0xd7c7776200000000, 0x4add9f5300000000, 0xfed6e8f500000000, + 0xd93a7e6400000000, 0x6d3109c200000000, 0xf02be1f300000000, + 0x4420965500000000, 0xca1e319000000000, 0x7e15463600000000, + 0xe30fae0700000000, 0x5704d9a100000000, 0xbe74915700000000, + 0x0a7fe6f100000000, 0x97650ec000000000, 0x236e796600000000, + 0xad50dea300000000, 0x195ba90500000000, 0x8441413400000000, + 0x304a369200000000}, + {0x0000000000000000, 0x9e00aacc00000000, 0x7d07254200000000, + 0xe3078f8e00000000, 0xfa0e4a8400000000, 0x640ee04800000000, + 0x87096fc600000000, 0x1909c50a00000000, 0xb51be5d300000000, + 0x2b1b4f1f00000000, 0xc81cc09100000000, 0x561c6a5d00000000, + 0x4f15af5700000000, 0xd115059b00000000, 0x32128a1500000000, + 0xac1220d900000000, 0x2b31bb7c00000000, 0xb53111b000000000, + 0x56369e3e00000000, 0xc83634f200000000, 0xd13ff1f800000000, + 0x4f3f5b3400000000, 0xac38d4ba00000000, 0x32387e7600000000, + 0x9e2a5eaf00000000, 0x002af46300000000, 0xe32d7bed00000000, + 0x7d2dd12100000000, 0x6424142b00000000, 0xfa24bee700000000, + 0x1923316900000000, 0x87239ba500000000, 0x566276f900000000, + 0xc862dc3500000000, 0x2b6553bb00000000, 0xb565f97700000000, + 0xac6c3c7d00000000, 0x326c96b100000000, 0xd16b193f00000000, + 0x4f6bb3f300000000, 0xe379932a00000000, 0x7d7939e600000000, + 0x9e7eb66800000000, 0x007e1ca400000000, 0x1977d9ae00000000, + 0x8777736200000000, 0x6470fcec00000000, 0xfa70562000000000, + 0x7d53cd8500000000, 0xe353674900000000, 0x0054e8c700000000, + 0x9e54420b00000000, 0x875d870100000000, 0x195d2dcd00000000, + 0xfa5aa24300000000, 0x645a088f00000000, 0xc848285600000000, + 0x5648829a00000000, 0xb54f0d1400000000, 0x2b4fa7d800000000, + 0x324662d200000000, 0xac46c81e00000000, 0x4f41479000000000, + 0xd141ed5c00000000, 0xedc29d2900000000, 0x73c237e500000000, + 0x90c5b86b00000000, 0x0ec512a700000000, 0x17ccd7ad00000000, + 0x89cc7d6100000000, 0x6acbf2ef00000000, 0xf4cb582300000000, + 0x58d978fa00000000, 0xc6d9d23600000000, 0x25de5db800000000, + 0xbbdef77400000000, 0xa2d7327e00000000, 0x3cd798b200000000, + 0xdfd0173c00000000, 0x41d0bdf000000000, 0xc6f3265500000000, + 0x58f38c9900000000, 0xbbf4031700000000, 0x25f4a9db00000000, + 0x3cfd6cd100000000, 0xa2fdc61d00000000, 0x41fa499300000000, + 0xdffae35f00000000, 0x73e8c38600000000, 0xede8694a00000000, + 0x0eefe6c400000000, 0x90ef4c0800000000, 0x89e6890200000000, + 0x17e623ce00000000, 0xf4e1ac4000000000, 0x6ae1068c00000000, + 0xbba0ebd000000000, 0x25a0411c00000000, 0xc6a7ce9200000000, + 0x58a7645e00000000, 0x41aea15400000000, 0xdfae0b9800000000, + 0x3ca9841600000000, 0xa2a92eda00000000, 0x0ebb0e0300000000, + 0x90bba4cf00000000, 0x73bc2b4100000000, 0xedbc818d00000000, + 0xf4b5448700000000, 0x6ab5ee4b00000000, 0x89b261c500000000, + 0x17b2cb0900000000, 0x909150ac00000000, 0x0e91fa6000000000, + 0xed9675ee00000000, 0x7396df2200000000, 0x6a9f1a2800000000, + 0xf49fb0e400000000, 0x17983f6a00000000, 0x899895a600000000, + 0x258ab57f00000000, 0xbb8a1fb300000000, 0x588d903d00000000, + 0xc68d3af100000000, 0xdf84fffb00000000, 0x4184553700000000, + 0xa283dab900000000, 0x3c83707500000000, 0xda853b5300000000, + 0x4485919f00000000, 0xa7821e1100000000, 0x3982b4dd00000000, + 0x208b71d700000000, 0xbe8bdb1b00000000, 0x5d8c549500000000, + 0xc38cfe5900000000, 0x6f9ede8000000000, 0xf19e744c00000000, + 0x1299fbc200000000, 0x8c99510e00000000, 0x9590940400000000, + 0x0b903ec800000000, 0xe897b14600000000, 0x76971b8a00000000, + 0xf1b4802f00000000, 0x6fb42ae300000000, 0x8cb3a56d00000000, + 0x12b30fa100000000, 0x0bbacaab00000000, 0x95ba606700000000, + 0x76bdefe900000000, 0xe8bd452500000000, 0x44af65fc00000000, + 0xdaafcf3000000000, 0x39a840be00000000, 0xa7a8ea7200000000, + 0xbea12f7800000000, 0x20a185b400000000, 0xc3a60a3a00000000, + 0x5da6a0f600000000, 0x8ce74daa00000000, 0x12e7e76600000000, + 0xf1e068e800000000, 0x6fe0c22400000000, 0x76e9072e00000000, + 0xe8e9ade200000000, 0x0bee226c00000000, 0x95ee88a000000000, + 0x39fca87900000000, 0xa7fc02b500000000, 0x44fb8d3b00000000, + 0xdafb27f700000000, 0xc3f2e2fd00000000, 0x5df2483100000000, + 0xbef5c7bf00000000, 0x20f56d7300000000, 0xa7d6f6d600000000, + 0x39d65c1a00000000, 0xdad1d39400000000, 0x44d1795800000000, + 0x5dd8bc5200000000, 0xc3d8169e00000000, 0x20df991000000000, + 0xbedf33dc00000000, 0x12cd130500000000, 0x8ccdb9c900000000, + 0x6fca364700000000, 0xf1ca9c8b00000000, 0xe8c3598100000000, + 0x76c3f34d00000000, 0x95c47cc300000000, 0x0bc4d60f00000000, + 0x3747a67a00000000, 0xa9470cb600000000, 0x4a40833800000000, + 0xd44029f400000000, 0xcd49ecfe00000000, 0x5349463200000000, + 0xb04ec9bc00000000, 0x2e4e637000000000, 0x825c43a900000000, + 0x1c5ce96500000000, 0xff5b66eb00000000, 0x615bcc2700000000, + 0x7852092d00000000, 0xe652a3e100000000, 0x05552c6f00000000, + 0x9b5586a300000000, 0x1c761d0600000000, 0x8276b7ca00000000, + 0x6171384400000000, 0xff71928800000000, 0xe678578200000000, + 0x7878fd4e00000000, 0x9b7f72c000000000, 0x057fd80c00000000, + 0xa96df8d500000000, 0x376d521900000000, 0xd46add9700000000, + 0x4a6a775b00000000, 0x5363b25100000000, 0xcd63189d00000000, + 0x2e64971300000000, 0xb0643ddf00000000, 0x6125d08300000000, + 0xff257a4f00000000, 0x1c22f5c100000000, 0x82225f0d00000000, + 0x9b2b9a0700000000, 0x052b30cb00000000, 0xe62cbf4500000000, + 0x782c158900000000, 0xd43e355000000000, 0x4a3e9f9c00000000, + 0xa939101200000000, 0x3739bade00000000, 0x2e307fd400000000, + 0xb030d51800000000, 0x53375a9600000000, 0xcd37f05a00000000, + 0x4a146bff00000000, 0xd414c13300000000, 0x37134ebd00000000, + 0xa913e47100000000, 0xb01a217b00000000, 0x2e1a8bb700000000, + 0xcd1d043900000000, 0x531daef500000000, 0xff0f8e2c00000000, + 0x610f24e000000000, 0x8208ab6e00000000, 0x1c0801a200000000, + 0x0501c4a800000000, 0x9b016e6400000000, 0x7806e1ea00000000, + 0xe6064b2600000000}}; + +#else /* W == 4 */ + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0xb8bc6765, 0xaa09c88b, 0x12b5afee, 0x8f629757, + 0x37def032, 0x256b5fdc, 0x9dd738b9, 0xc5b428ef, 0x7d084f8a, + 0x6fbde064, 0xd7018701, 0x4ad6bfb8, 0xf26ad8dd, 0xe0df7733, + 0x58631056, 0x5019579f, 0xe8a530fa, 0xfa109f14, 0x42acf871, + 0xdf7bc0c8, 0x67c7a7ad, 0x75720843, 0xcdce6f26, 0x95ad7f70, + 0x2d111815, 0x3fa4b7fb, 0x8718d09e, 0x1acfe827, 0xa2738f42, + 0xb0c620ac, 0x087a47c9, 0xa032af3e, 0x188ec85b, 0x0a3b67b5, + 0xb28700d0, 0x2f503869, 0x97ec5f0c, 0x8559f0e2, 0x3de59787, + 0x658687d1, 0xdd3ae0b4, 0xcf8f4f5a, 0x7733283f, 0xeae41086, + 0x525877e3, 0x40edd80d, 0xf851bf68, 0xf02bf8a1, 0x48979fc4, + 0x5a22302a, 0xe29e574f, 0x7f496ff6, 0xc7f50893, 0xd540a77d, + 0x6dfcc018, 0x359fd04e, 0x8d23b72b, 0x9f9618c5, 0x272a7fa0, + 0xbafd4719, 0x0241207c, 0x10f48f92, 0xa848e8f7, 0x9b14583d, + 0x23a83f58, 0x311d90b6, 0x89a1f7d3, 0x1476cf6a, 0xaccaa80f, + 0xbe7f07e1, 0x06c36084, 0x5ea070d2, 0xe61c17b7, 0xf4a9b859, + 0x4c15df3c, 0xd1c2e785, 0x697e80e0, 0x7bcb2f0e, 0xc377486b, + 0xcb0d0fa2, 0x73b168c7, 0x6104c729, 0xd9b8a04c, 0x446f98f5, + 0xfcd3ff90, 0xee66507e, 0x56da371b, 0x0eb9274d, 0xb6054028, + 0xa4b0efc6, 0x1c0c88a3, 0x81dbb01a, 0x3967d77f, 0x2bd27891, + 0x936e1ff4, 0x3b26f703, 0x839a9066, 0x912f3f88, 0x299358ed, + 0xb4446054, 0x0cf80731, 0x1e4da8df, 0xa6f1cfba, 0xfe92dfec, + 0x462eb889, 0x549b1767, 0xec277002, 0x71f048bb, 0xc94c2fde, + 0xdbf98030, 0x6345e755, 0x6b3fa09c, 0xd383c7f9, 0xc1366817, + 0x798a0f72, 0xe45d37cb, 0x5ce150ae, 0x4e54ff40, 0xf6e89825, + 0xae8b8873, 0x1637ef16, 0x048240f8, 0xbc3e279d, 0x21e91f24, + 0x99557841, 0x8be0d7af, 0x335cb0ca, 0xed59b63b, 0x55e5d15e, + 0x47507eb0, 0xffec19d5, 0x623b216c, 0xda874609, 0xc832e9e7, + 0x708e8e82, 0x28ed9ed4, 0x9051f9b1, 0x82e4565f, 0x3a58313a, + 0xa78f0983, 0x1f336ee6, 0x0d86c108, 0xb53aa66d, 0xbd40e1a4, + 0x05fc86c1, 0x1749292f, 0xaff54e4a, 0x322276f3, 0x8a9e1196, + 0x982bbe78, 0x2097d91d, 0x78f4c94b, 0xc048ae2e, 0xd2fd01c0, + 0x6a4166a5, 0xf7965e1c, 0x4f2a3979, 0x5d9f9697, 0xe523f1f2, + 0x4d6b1905, 0xf5d77e60, 0xe762d18e, 0x5fdeb6eb, 0xc2098e52, + 0x7ab5e937, 0x680046d9, 0xd0bc21bc, 0x88df31ea, 0x3063568f, + 0x22d6f961, 0x9a6a9e04, 0x07bda6bd, 0xbf01c1d8, 0xadb46e36, + 0x15080953, 0x1d724e9a, 0xa5ce29ff, 0xb77b8611, 0x0fc7e174, + 0x9210d9cd, 0x2aacbea8, 0x38191146, 0x80a57623, 0xd8c66675, + 0x607a0110, 0x72cfaefe, 0xca73c99b, 0x57a4f122, 0xef189647, + 0xfdad39a9, 0x45115ecc, 0x764dee06, 0xcef18963, 0xdc44268d, + 0x64f841e8, 0xf92f7951, 0x41931e34, 0x5326b1da, 0xeb9ad6bf, + 0xb3f9c6e9, 0x0b45a18c, 0x19f00e62, 0xa14c6907, 0x3c9b51be, + 0x842736db, 0x96929935, 0x2e2efe50, 0x2654b999, 0x9ee8defc, + 0x8c5d7112, 0x34e11677, 0xa9362ece, 0x118a49ab, 0x033fe645, + 0xbb838120, 0xe3e09176, 0x5b5cf613, 0x49e959fd, 0xf1553e98, + 0x6c820621, 0xd43e6144, 0xc68bceaa, 0x7e37a9cf, 0xd67f4138, + 0x6ec3265d, 0x7c7689b3, 0xc4caeed6, 0x591dd66f, 0xe1a1b10a, + 0xf3141ee4, 0x4ba87981, 0x13cb69d7, 0xab770eb2, 0xb9c2a15c, + 0x017ec639, 0x9ca9fe80, 0x241599e5, 0x36a0360b, 0x8e1c516e, + 0x866616a7, 0x3eda71c2, 0x2c6fde2c, 0x94d3b949, 0x090481f0, + 0xb1b8e695, 0xa30d497b, 0x1bb12e1e, 0x43d23e48, 0xfb6e592d, + 0xe9dbf6c3, 0x516791a6, 0xccb0a91f, 0x740cce7a, 0x66b96194, + 0xde0506f1}, + {0x00000000, 0x01c26a37, 0x0384d46e, 0x0246be59, 0x0709a8dc, + 0x06cbc2eb, 0x048d7cb2, 0x054f1685, 0x0e1351b8, 0x0fd13b8f, + 0x0d9785d6, 0x0c55efe1, 0x091af964, 0x08d89353, 0x0a9e2d0a, + 0x0b5c473d, 0x1c26a370, 0x1de4c947, 0x1fa2771e, 0x1e601d29, + 0x1b2f0bac, 0x1aed619b, 0x18abdfc2, 0x1969b5f5, 0x1235f2c8, + 0x13f798ff, 0x11b126a6, 0x10734c91, 0x153c5a14, 0x14fe3023, + 0x16b88e7a, 0x177ae44d, 0x384d46e0, 0x398f2cd7, 0x3bc9928e, + 0x3a0bf8b9, 0x3f44ee3c, 0x3e86840b, 0x3cc03a52, 0x3d025065, + 0x365e1758, 0x379c7d6f, 0x35dac336, 0x3418a901, 0x3157bf84, + 0x3095d5b3, 0x32d36bea, 0x331101dd, 0x246be590, 0x25a98fa7, + 0x27ef31fe, 0x262d5bc9, 0x23624d4c, 0x22a0277b, 0x20e69922, + 0x2124f315, 0x2a78b428, 0x2bbade1f, 0x29fc6046, 0x283e0a71, + 0x2d711cf4, 0x2cb376c3, 0x2ef5c89a, 0x2f37a2ad, 0x709a8dc0, + 0x7158e7f7, 0x731e59ae, 0x72dc3399, 0x7793251c, 0x76514f2b, + 0x7417f172, 0x75d59b45, 0x7e89dc78, 0x7f4bb64f, 0x7d0d0816, + 0x7ccf6221, 0x798074a4, 0x78421e93, 0x7a04a0ca, 0x7bc6cafd, + 0x6cbc2eb0, 0x6d7e4487, 0x6f38fade, 0x6efa90e9, 0x6bb5866c, + 0x6a77ec5b, 0x68315202, 0x69f33835, 0x62af7f08, 0x636d153f, + 0x612bab66, 0x60e9c151, 0x65a6d7d4, 0x6464bde3, 0x662203ba, + 0x67e0698d, 0x48d7cb20, 0x4915a117, 0x4b531f4e, 0x4a917579, + 0x4fde63fc, 0x4e1c09cb, 0x4c5ab792, 0x4d98dda5, 0x46c49a98, + 0x4706f0af, 0x45404ef6, 0x448224c1, 0x41cd3244, 0x400f5873, + 0x4249e62a, 0x438b8c1d, 0x54f16850, 0x55330267, 0x5775bc3e, + 0x56b7d609, 0x53f8c08c, 0x523aaabb, 0x507c14e2, 0x51be7ed5, + 0x5ae239e8, 0x5b2053df, 0x5966ed86, 0x58a487b1, 0x5deb9134, + 0x5c29fb03, 0x5e6f455a, 0x5fad2f6d, 0xe1351b80, 0xe0f771b7, + 0xe2b1cfee, 0xe373a5d9, 0xe63cb35c, 0xe7fed96b, 0xe5b86732, + 0xe47a0d05, 0xef264a38, 0xeee4200f, 0xeca29e56, 0xed60f461, + 0xe82fe2e4, 0xe9ed88d3, 0xebab368a, 0xea695cbd, 0xfd13b8f0, + 0xfcd1d2c7, 0xfe976c9e, 0xff5506a9, 0xfa1a102c, 0xfbd87a1b, + 0xf99ec442, 0xf85cae75, 0xf300e948, 0xf2c2837f, 0xf0843d26, + 0xf1465711, 0xf4094194, 0xf5cb2ba3, 0xf78d95fa, 0xf64fffcd, + 0xd9785d60, 0xd8ba3757, 0xdafc890e, 0xdb3ee339, 0xde71f5bc, + 0xdfb39f8b, 0xddf521d2, 0xdc374be5, 0xd76b0cd8, 0xd6a966ef, + 0xd4efd8b6, 0xd52db281, 0xd062a404, 0xd1a0ce33, 0xd3e6706a, + 0xd2241a5d, 0xc55efe10, 0xc49c9427, 0xc6da2a7e, 0xc7184049, + 0xc25756cc, 0xc3953cfb, 0xc1d382a2, 0xc011e895, 0xcb4dafa8, + 0xca8fc59f, 0xc8c97bc6, 0xc90b11f1, 0xcc440774, 0xcd866d43, + 0xcfc0d31a, 0xce02b92d, 0x91af9640, 0x906dfc77, 0x922b422e, + 0x93e92819, 0x96a63e9c, 0x976454ab, 0x9522eaf2, 0x94e080c5, + 0x9fbcc7f8, 0x9e7eadcf, 0x9c381396, 0x9dfa79a1, 0x98b56f24, + 0x99770513, 0x9b31bb4a, 0x9af3d17d, 0x8d893530, 0x8c4b5f07, + 0x8e0de15e, 0x8fcf8b69, 0x8a809dec, 0x8b42f7db, 0x89044982, + 0x88c623b5, 0x839a6488, 0x82580ebf, 0x801eb0e6, 0x81dcdad1, + 0x8493cc54, 0x8551a663, 0x8717183a, 0x86d5720d, 0xa9e2d0a0, + 0xa820ba97, 0xaa6604ce, 0xaba46ef9, 0xaeeb787c, 0xaf29124b, + 0xad6fac12, 0xacadc625, 0xa7f18118, 0xa633eb2f, 0xa4755576, + 0xa5b73f41, 0xa0f829c4, 0xa13a43f3, 0xa37cfdaa, 0xa2be979d, + 0xb5c473d0, 0xb40619e7, 0xb640a7be, 0xb782cd89, 0xb2cddb0c, + 0xb30fb13b, 0xb1490f62, 0xb08b6555, 0xbbd72268, 0xba15485f, + 0xb853f606, 0xb9919c31, 0xbcde8ab4, 0xbd1ce083, 0xbf5a5eda, + 0xbe9834ed}, + {0x00000000, 0x191b3141, 0x32366282, 0x2b2d53c3, 0x646cc504, + 0x7d77f445, 0x565aa786, 0x4f4196c7, 0xc8d98a08, 0xd1c2bb49, + 0xfaefe88a, 0xe3f4d9cb, 0xacb54f0c, 0xb5ae7e4d, 0x9e832d8e, + 0x87981ccf, 0x4ac21251, 0x53d92310, 0x78f470d3, 0x61ef4192, + 0x2eaed755, 0x37b5e614, 0x1c98b5d7, 0x05838496, 0x821b9859, + 0x9b00a918, 0xb02dfadb, 0xa936cb9a, 0xe6775d5d, 0xff6c6c1c, + 0xd4413fdf, 0xcd5a0e9e, 0x958424a2, 0x8c9f15e3, 0xa7b24620, + 0xbea97761, 0xf1e8e1a6, 0xe8f3d0e7, 0xc3de8324, 0xdac5b265, + 0x5d5daeaa, 0x44469feb, 0x6f6bcc28, 0x7670fd69, 0x39316bae, + 0x202a5aef, 0x0b07092c, 0x121c386d, 0xdf4636f3, 0xc65d07b2, + 0xed705471, 0xf46b6530, 0xbb2af3f7, 0xa231c2b6, 0x891c9175, + 0x9007a034, 0x179fbcfb, 0x0e848dba, 0x25a9de79, 0x3cb2ef38, + 0x73f379ff, 0x6ae848be, 0x41c51b7d, 0x58de2a3c, 0xf0794f05, + 0xe9627e44, 0xc24f2d87, 0xdb541cc6, 0x94158a01, 0x8d0ebb40, + 0xa623e883, 0xbf38d9c2, 0x38a0c50d, 0x21bbf44c, 0x0a96a78f, + 0x138d96ce, 0x5ccc0009, 0x45d73148, 0x6efa628b, 0x77e153ca, + 0xbabb5d54, 0xa3a06c15, 0x888d3fd6, 0x91960e97, 0xded79850, + 0xc7cca911, 0xece1fad2, 0xf5facb93, 0x7262d75c, 0x6b79e61d, + 0x4054b5de, 0x594f849f, 0x160e1258, 0x0f152319, 0x243870da, + 0x3d23419b, 0x65fd6ba7, 0x7ce65ae6, 0x57cb0925, 0x4ed03864, + 0x0191aea3, 0x188a9fe2, 0x33a7cc21, 0x2abcfd60, 0xad24e1af, + 0xb43fd0ee, 0x9f12832d, 0x8609b26c, 0xc94824ab, 0xd05315ea, + 0xfb7e4629, 0xe2657768, 0x2f3f79f6, 0x362448b7, 0x1d091b74, + 0x04122a35, 0x4b53bcf2, 0x52488db3, 0x7965de70, 0x607eef31, + 0xe7e6f3fe, 0xfefdc2bf, 0xd5d0917c, 0xcccba03d, 0x838a36fa, + 0x9a9107bb, 0xb1bc5478, 0xa8a76539, 0x3b83984b, 0x2298a90a, + 0x09b5fac9, 0x10aecb88, 0x5fef5d4f, 0x46f46c0e, 0x6dd93fcd, + 0x74c20e8c, 0xf35a1243, 0xea412302, 0xc16c70c1, 0xd8774180, + 0x9736d747, 0x8e2de606, 0xa500b5c5, 0xbc1b8484, 0x71418a1a, + 0x685abb5b, 0x4377e898, 0x5a6cd9d9, 0x152d4f1e, 0x0c367e5f, + 0x271b2d9c, 0x3e001cdd, 0xb9980012, 0xa0833153, 0x8bae6290, + 0x92b553d1, 0xddf4c516, 0xc4eff457, 0xefc2a794, 0xf6d996d5, + 0xae07bce9, 0xb71c8da8, 0x9c31de6b, 0x852aef2a, 0xca6b79ed, + 0xd37048ac, 0xf85d1b6f, 0xe1462a2e, 0x66de36e1, 0x7fc507a0, + 0x54e85463, 0x4df36522, 0x02b2f3e5, 0x1ba9c2a4, 0x30849167, + 0x299fa026, 0xe4c5aeb8, 0xfdde9ff9, 0xd6f3cc3a, 0xcfe8fd7b, + 0x80a96bbc, 0x99b25afd, 0xb29f093e, 0xab84387f, 0x2c1c24b0, + 0x350715f1, 0x1e2a4632, 0x07317773, 0x4870e1b4, 0x516bd0f5, + 0x7a468336, 0x635db277, 0xcbfad74e, 0xd2e1e60f, 0xf9ccb5cc, + 0xe0d7848d, 0xaf96124a, 0xb68d230b, 0x9da070c8, 0x84bb4189, + 0x03235d46, 0x1a386c07, 0x31153fc4, 0x280e0e85, 0x674f9842, + 0x7e54a903, 0x5579fac0, 0x4c62cb81, 0x8138c51f, 0x9823f45e, + 0xb30ea79d, 0xaa1596dc, 0xe554001b, 0xfc4f315a, 0xd7626299, + 0xce7953d8, 0x49e14f17, 0x50fa7e56, 0x7bd72d95, 0x62cc1cd4, + 0x2d8d8a13, 0x3496bb52, 0x1fbbe891, 0x06a0d9d0, 0x5e7ef3ec, + 0x4765c2ad, 0x6c48916e, 0x7553a02f, 0x3a1236e8, 0x230907a9, + 0x0824546a, 0x113f652b, 0x96a779e4, 0x8fbc48a5, 0xa4911b66, + 0xbd8a2a27, 0xf2cbbce0, 0xebd08da1, 0xc0fdde62, 0xd9e6ef23, + 0x14bce1bd, 0x0da7d0fc, 0x268a833f, 0x3f91b27e, 0x70d024b9, + 0x69cb15f8, 0x42e6463b, 0x5bfd777a, 0xdc656bb5, 0xc57e5af4, + 0xee530937, 0xf7483876, 0xb809aeb1, 0xa1129ff0, 0x8a3fcc33, + 0x9324fd72}, + {0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, + 0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4, + 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, + 0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, + 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856, + 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, + 0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4, + 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, + 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, + 0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a, + 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, + 0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, + 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190, + 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, + 0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e, + 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, + 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, + 0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950, + 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, + 0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, + 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a, + 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, + 0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010, + 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, + 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, + 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6, + 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, + 0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, + 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344, + 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, + 0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a, + 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, + 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, + 0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c, + 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, + 0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, + 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe, + 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, + 0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c, + 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, + 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, + 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242, + 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, + 0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, + 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278, + 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, + 0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66, + 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, + 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, + 0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8, + 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, + 0x2d02ef8d}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x00000000, 0x96300777, 0x2c610eee, 0xba510999, 0x19c46d07, + 0x8ff46a70, 0x35a563e9, 0xa395649e, 0x3288db0e, 0xa4b8dc79, + 0x1ee9d5e0, 0x88d9d297, 0x2b4cb609, 0xbd7cb17e, 0x072db8e7, + 0x911dbf90, 0x6410b71d, 0xf220b06a, 0x4871b9f3, 0xde41be84, + 0x7dd4da1a, 0xebe4dd6d, 0x51b5d4f4, 0xc785d383, 0x56986c13, + 0xc0a86b64, 0x7af962fd, 0xecc9658a, 0x4f5c0114, 0xd96c0663, + 0x633d0ffa, 0xf50d088d, 0xc8206e3b, 0x5e10694c, 0xe44160d5, + 0x727167a2, 0xd1e4033c, 0x47d4044b, 0xfd850dd2, 0x6bb50aa5, + 0xfaa8b535, 0x6c98b242, 0xd6c9bbdb, 0x40f9bcac, 0xe36cd832, + 0x755cdf45, 0xcf0dd6dc, 0x593dd1ab, 0xac30d926, 0x3a00de51, + 0x8051d7c8, 0x1661d0bf, 0xb5f4b421, 0x23c4b356, 0x9995bacf, + 0x0fa5bdb8, 0x9eb80228, 0x0888055f, 0xb2d90cc6, 0x24e90bb1, + 0x877c6f2f, 0x114c6858, 0xab1d61c1, 0x3d2d66b6, 0x9041dc76, + 0x0671db01, 0xbc20d298, 0x2a10d5ef, 0x8985b171, 0x1fb5b606, + 0xa5e4bf9f, 0x33d4b8e8, 0xa2c90778, 0x34f9000f, 0x8ea80996, + 0x18980ee1, 0xbb0d6a7f, 0x2d3d6d08, 0x976c6491, 0x015c63e6, + 0xf4516b6b, 0x62616c1c, 0xd8306585, 0x4e0062f2, 0xed95066c, + 0x7ba5011b, 0xc1f40882, 0x57c40ff5, 0xc6d9b065, 0x50e9b712, + 0xeab8be8b, 0x7c88b9fc, 0xdf1ddd62, 0x492dda15, 0xf37cd38c, + 0x654cd4fb, 0x5861b24d, 0xce51b53a, 0x7400bca3, 0xe230bbd4, + 0x41a5df4a, 0xd795d83d, 0x6dc4d1a4, 0xfbf4d6d3, 0x6ae96943, + 0xfcd96e34, 0x468867ad, 0xd0b860da, 0x732d0444, 0xe51d0333, + 0x5f4c0aaa, 0xc97c0ddd, 0x3c710550, 0xaa410227, 0x10100bbe, + 0x86200cc9, 0x25b56857, 0xb3856f20, 0x09d466b9, 0x9fe461ce, + 0x0ef9de5e, 0x98c9d929, 0x2298d0b0, 0xb4a8d7c7, 0x173db359, + 0x810db42e, 0x3b5cbdb7, 0xad6cbac0, 0x2083b8ed, 0xb6b3bf9a, + 0x0ce2b603, 0x9ad2b174, 0x3947d5ea, 0xaf77d29d, 0x1526db04, + 0x8316dc73, 0x120b63e3, 0x843b6494, 0x3e6a6d0d, 0xa85a6a7a, + 0x0bcf0ee4, 0x9dff0993, 0x27ae000a, 0xb19e077d, 0x44930ff0, + 0xd2a30887, 0x68f2011e, 0xfec20669, 0x5d5762f7, 0xcb676580, + 0x71366c19, 0xe7066b6e, 0x761bd4fe, 0xe02bd389, 0x5a7ada10, + 0xcc4add67, 0x6fdfb9f9, 0xf9efbe8e, 0x43beb717, 0xd58eb060, + 0xe8a3d6d6, 0x7e93d1a1, 0xc4c2d838, 0x52f2df4f, 0xf167bbd1, + 0x6757bca6, 0xdd06b53f, 0x4b36b248, 0xda2b0dd8, 0x4c1b0aaf, + 0xf64a0336, 0x607a0441, 0xc3ef60df, 0x55df67a8, 0xef8e6e31, + 0x79be6946, 0x8cb361cb, 0x1a8366bc, 0xa0d26f25, 0x36e26852, + 0x95770ccc, 0x03470bbb, 0xb9160222, 0x2f260555, 0xbe3bbac5, + 0x280bbdb2, 0x925ab42b, 0x046ab35c, 0xa7ffd7c2, 0x31cfd0b5, + 0x8b9ed92c, 0x1daede5b, 0xb0c2649b, 0x26f263ec, 0x9ca36a75, + 0x0a936d02, 0xa906099c, 0x3f360eeb, 0x85670772, 0x13570005, + 0x824abf95, 0x147ab8e2, 0xae2bb17b, 0x381bb60c, 0x9b8ed292, + 0x0dbed5e5, 0xb7efdc7c, 0x21dfdb0b, 0xd4d2d386, 0x42e2d4f1, + 0xf8b3dd68, 0x6e83da1f, 0xcd16be81, 0x5b26b9f6, 0xe177b06f, + 0x7747b718, 0xe65a0888, 0x706a0fff, 0xca3b0666, 0x5c0b0111, + 0xff9e658f, 0x69ae62f8, 0xd3ff6b61, 0x45cf6c16, 0x78e20aa0, + 0xeed20dd7, 0x5483044e, 0xc2b30339, 0x612667a7, 0xf71660d0, + 0x4d476949, 0xdb776e3e, 0x4a6ad1ae, 0xdc5ad6d9, 0x660bdf40, + 0xf03bd837, 0x53aebca9, 0xc59ebbde, 0x7fcfb247, 0xe9ffb530, + 0x1cf2bdbd, 0x8ac2baca, 0x3093b353, 0xa6a3b424, 0x0536d0ba, + 0x9306d7cd, 0x2957de54, 0xbf67d923, 0x2e7a66b3, 0xb84a61c4, + 0x021b685d, 0x942b6f2a, 0x37be0bb4, 0xa18e0cc3, 0x1bdf055a, + 0x8def022d}, + {0x00000000, 0x41311b19, 0x82623632, 0xc3532d2b, 0x04c56c64, + 0x45f4777d, 0x86a75a56, 0xc796414f, 0x088ad9c8, 0x49bbc2d1, + 0x8ae8effa, 0xcbd9f4e3, 0x0c4fb5ac, 0x4d7eaeb5, 0x8e2d839e, + 0xcf1c9887, 0x5112c24a, 0x1023d953, 0xd370f478, 0x9241ef61, + 0x55d7ae2e, 0x14e6b537, 0xd7b5981c, 0x96848305, 0x59981b82, + 0x18a9009b, 0xdbfa2db0, 0x9acb36a9, 0x5d5d77e6, 0x1c6c6cff, + 0xdf3f41d4, 0x9e0e5acd, 0xa2248495, 0xe3159f8c, 0x2046b2a7, + 0x6177a9be, 0xa6e1e8f1, 0xe7d0f3e8, 0x2483dec3, 0x65b2c5da, + 0xaaae5d5d, 0xeb9f4644, 0x28cc6b6f, 0x69fd7076, 0xae6b3139, + 0xef5a2a20, 0x2c09070b, 0x6d381c12, 0xf33646df, 0xb2075dc6, + 0x715470ed, 0x30656bf4, 0xf7f32abb, 0xb6c231a2, 0x75911c89, + 0x34a00790, 0xfbbc9f17, 0xba8d840e, 0x79dea925, 0x38efb23c, + 0xff79f373, 0xbe48e86a, 0x7d1bc541, 0x3c2ade58, 0x054f79f0, + 0x447e62e9, 0x872d4fc2, 0xc61c54db, 0x018a1594, 0x40bb0e8d, + 0x83e823a6, 0xc2d938bf, 0x0dc5a038, 0x4cf4bb21, 0x8fa7960a, + 0xce968d13, 0x0900cc5c, 0x4831d745, 0x8b62fa6e, 0xca53e177, + 0x545dbbba, 0x156ca0a3, 0xd63f8d88, 0x970e9691, 0x5098d7de, + 0x11a9ccc7, 0xd2fae1ec, 0x93cbfaf5, 0x5cd76272, 0x1de6796b, + 0xdeb55440, 0x9f844f59, 0x58120e16, 0x1923150f, 0xda703824, + 0x9b41233d, 0xa76bfd65, 0xe65ae67c, 0x2509cb57, 0x6438d04e, + 0xa3ae9101, 0xe29f8a18, 0x21cca733, 0x60fdbc2a, 0xafe124ad, + 0xeed03fb4, 0x2d83129f, 0x6cb20986, 0xab2448c9, 0xea1553d0, + 0x29467efb, 0x687765e2, 0xf6793f2f, 0xb7482436, 0x741b091d, + 0x352a1204, 0xf2bc534b, 0xb38d4852, 0x70de6579, 0x31ef7e60, + 0xfef3e6e7, 0xbfc2fdfe, 0x7c91d0d5, 0x3da0cbcc, 0xfa368a83, + 0xbb07919a, 0x7854bcb1, 0x3965a7a8, 0x4b98833b, 0x0aa99822, + 0xc9fab509, 0x88cbae10, 0x4f5def5f, 0x0e6cf446, 0xcd3fd96d, + 0x8c0ec274, 0x43125af3, 0x022341ea, 0xc1706cc1, 0x804177d8, + 0x47d73697, 0x06e62d8e, 0xc5b500a5, 0x84841bbc, 0x1a8a4171, + 0x5bbb5a68, 0x98e87743, 0xd9d96c5a, 0x1e4f2d15, 0x5f7e360c, + 0x9c2d1b27, 0xdd1c003e, 0x120098b9, 0x533183a0, 0x9062ae8b, + 0xd153b592, 0x16c5f4dd, 0x57f4efc4, 0x94a7c2ef, 0xd596d9f6, + 0xe9bc07ae, 0xa88d1cb7, 0x6bde319c, 0x2aef2a85, 0xed796bca, + 0xac4870d3, 0x6f1b5df8, 0x2e2a46e1, 0xe136de66, 0xa007c57f, + 0x6354e854, 0x2265f34d, 0xe5f3b202, 0xa4c2a91b, 0x67918430, + 0x26a09f29, 0xb8aec5e4, 0xf99fdefd, 0x3accf3d6, 0x7bfde8cf, + 0xbc6ba980, 0xfd5ab299, 0x3e099fb2, 0x7f3884ab, 0xb0241c2c, + 0xf1150735, 0x32462a1e, 0x73773107, 0xb4e17048, 0xf5d06b51, + 0x3683467a, 0x77b25d63, 0x4ed7facb, 0x0fe6e1d2, 0xccb5ccf9, + 0x8d84d7e0, 0x4a1296af, 0x0b238db6, 0xc870a09d, 0x8941bb84, + 0x465d2303, 0x076c381a, 0xc43f1531, 0x850e0e28, 0x42984f67, + 0x03a9547e, 0xc0fa7955, 0x81cb624c, 0x1fc53881, 0x5ef42398, + 0x9da70eb3, 0xdc9615aa, 0x1b0054e5, 0x5a314ffc, 0x996262d7, + 0xd85379ce, 0x174fe149, 0x567efa50, 0x952dd77b, 0xd41ccc62, + 0x138a8d2d, 0x52bb9634, 0x91e8bb1f, 0xd0d9a006, 0xecf37e5e, + 0xadc26547, 0x6e91486c, 0x2fa05375, 0xe836123a, 0xa9070923, + 0x6a542408, 0x2b653f11, 0xe479a796, 0xa548bc8f, 0x661b91a4, + 0x272a8abd, 0xe0bccbf2, 0xa18dd0eb, 0x62defdc0, 0x23efe6d9, + 0xbde1bc14, 0xfcd0a70d, 0x3f838a26, 0x7eb2913f, 0xb924d070, + 0xf815cb69, 0x3b46e642, 0x7a77fd5b, 0xb56b65dc, 0xf45a7ec5, + 0x370953ee, 0x763848f7, 0xb1ae09b8, 0xf09f12a1, 0x33cc3f8a, + 0x72fd2493}, + {0x00000000, 0x376ac201, 0x6ed48403, 0x59be4602, 0xdca80907, + 0xebc2cb06, 0xb27c8d04, 0x85164f05, 0xb851130e, 0x8f3bd10f, + 0xd685970d, 0xe1ef550c, 0x64f91a09, 0x5393d808, 0x0a2d9e0a, + 0x3d475c0b, 0x70a3261c, 0x47c9e41d, 0x1e77a21f, 0x291d601e, + 0xac0b2f1b, 0x9b61ed1a, 0xc2dfab18, 0xf5b56919, 0xc8f23512, + 0xff98f713, 0xa626b111, 0x914c7310, 0x145a3c15, 0x2330fe14, + 0x7a8eb816, 0x4de47a17, 0xe0464d38, 0xd72c8f39, 0x8e92c93b, + 0xb9f80b3a, 0x3cee443f, 0x0b84863e, 0x523ac03c, 0x6550023d, + 0x58175e36, 0x6f7d9c37, 0x36c3da35, 0x01a91834, 0x84bf5731, + 0xb3d59530, 0xea6bd332, 0xdd011133, 0x90e56b24, 0xa78fa925, + 0xfe31ef27, 0xc95b2d26, 0x4c4d6223, 0x7b27a022, 0x2299e620, + 0x15f32421, 0x28b4782a, 0x1fdeba2b, 0x4660fc29, 0x710a3e28, + 0xf41c712d, 0xc376b32c, 0x9ac8f52e, 0xada2372f, 0xc08d9a70, + 0xf7e75871, 0xae591e73, 0x9933dc72, 0x1c259377, 0x2b4f5176, + 0x72f11774, 0x459bd575, 0x78dc897e, 0x4fb64b7f, 0x16080d7d, + 0x2162cf7c, 0xa4748079, 0x931e4278, 0xcaa0047a, 0xfdcac67b, + 0xb02ebc6c, 0x87447e6d, 0xdefa386f, 0xe990fa6e, 0x6c86b56b, + 0x5bec776a, 0x02523168, 0x3538f369, 0x087faf62, 0x3f156d63, + 0x66ab2b61, 0x51c1e960, 0xd4d7a665, 0xe3bd6464, 0xba032266, + 0x8d69e067, 0x20cbd748, 0x17a11549, 0x4e1f534b, 0x7975914a, + 0xfc63de4f, 0xcb091c4e, 0x92b75a4c, 0xa5dd984d, 0x989ac446, + 0xaff00647, 0xf64e4045, 0xc1248244, 0x4432cd41, 0x73580f40, + 0x2ae64942, 0x1d8c8b43, 0x5068f154, 0x67023355, 0x3ebc7557, + 0x09d6b756, 0x8cc0f853, 0xbbaa3a52, 0xe2147c50, 0xd57ebe51, + 0xe839e25a, 0xdf53205b, 0x86ed6659, 0xb187a458, 0x3491eb5d, + 0x03fb295c, 0x5a456f5e, 0x6d2fad5f, 0x801b35e1, 0xb771f7e0, + 0xeecfb1e2, 0xd9a573e3, 0x5cb33ce6, 0x6bd9fee7, 0x3267b8e5, + 0x050d7ae4, 0x384a26ef, 0x0f20e4ee, 0x569ea2ec, 0x61f460ed, + 0xe4e22fe8, 0xd388ede9, 0x8a36abeb, 0xbd5c69ea, 0xf0b813fd, + 0xc7d2d1fc, 0x9e6c97fe, 0xa90655ff, 0x2c101afa, 0x1b7ad8fb, + 0x42c49ef9, 0x75ae5cf8, 0x48e900f3, 0x7f83c2f2, 0x263d84f0, + 0x115746f1, 0x944109f4, 0xa32bcbf5, 0xfa958df7, 0xcdff4ff6, + 0x605d78d9, 0x5737bad8, 0x0e89fcda, 0x39e33edb, 0xbcf571de, + 0x8b9fb3df, 0xd221f5dd, 0xe54b37dc, 0xd80c6bd7, 0xef66a9d6, + 0xb6d8efd4, 0x81b22dd5, 0x04a462d0, 0x33cea0d1, 0x6a70e6d3, + 0x5d1a24d2, 0x10fe5ec5, 0x27949cc4, 0x7e2adac6, 0x494018c7, + 0xcc5657c2, 0xfb3c95c3, 0xa282d3c1, 0x95e811c0, 0xa8af4dcb, + 0x9fc58fca, 0xc67bc9c8, 0xf1110bc9, 0x740744cc, 0x436d86cd, + 0x1ad3c0cf, 0x2db902ce, 0x4096af91, 0x77fc6d90, 0x2e422b92, + 0x1928e993, 0x9c3ea696, 0xab546497, 0xf2ea2295, 0xc580e094, + 0xf8c7bc9f, 0xcfad7e9e, 0x9613389c, 0xa179fa9d, 0x246fb598, + 0x13057799, 0x4abb319b, 0x7dd1f39a, 0x3035898d, 0x075f4b8c, + 0x5ee10d8e, 0x698bcf8f, 0xec9d808a, 0xdbf7428b, 0x82490489, + 0xb523c688, 0x88649a83, 0xbf0e5882, 0xe6b01e80, 0xd1dadc81, + 0x54cc9384, 0x63a65185, 0x3a181787, 0x0d72d586, 0xa0d0e2a9, + 0x97ba20a8, 0xce0466aa, 0xf96ea4ab, 0x7c78ebae, 0x4b1229af, + 0x12ac6fad, 0x25c6adac, 0x1881f1a7, 0x2feb33a6, 0x765575a4, + 0x413fb7a5, 0xc429f8a0, 0xf3433aa1, 0xaafd7ca3, 0x9d97bea2, + 0xd073c4b5, 0xe71906b4, 0xbea740b6, 0x89cd82b7, 0x0cdbcdb2, + 0x3bb10fb3, 0x620f49b1, 0x55658bb0, 0x6822d7bb, 0x5f4815ba, + 0x06f653b8, 0x319c91b9, 0xb48adebc, 0x83e01cbd, 0xda5e5abf, + 0xed3498be}, + {0x00000000, 0x6567bcb8, 0x8bc809aa, 0xeeafb512, 0x5797628f, + 0x32f0de37, 0xdc5f6b25, 0xb938d79d, 0xef28b4c5, 0x8a4f087d, + 0x64e0bd6f, 0x018701d7, 0xb8bfd64a, 0xddd86af2, 0x3377dfe0, + 0x56106358, 0x9f571950, 0xfa30a5e8, 0x149f10fa, 0x71f8ac42, + 0xc8c07bdf, 0xada7c767, 0x43087275, 0x266fcecd, 0x707fad95, + 0x1518112d, 0xfbb7a43f, 0x9ed01887, 0x27e8cf1a, 0x428f73a2, + 0xac20c6b0, 0xc9477a08, 0x3eaf32a0, 0x5bc88e18, 0xb5673b0a, + 0xd00087b2, 0x6938502f, 0x0c5fec97, 0xe2f05985, 0x8797e53d, + 0xd1878665, 0xb4e03add, 0x5a4f8fcf, 0x3f283377, 0x8610e4ea, + 0xe3775852, 0x0dd8ed40, 0x68bf51f8, 0xa1f82bf0, 0xc49f9748, + 0x2a30225a, 0x4f579ee2, 0xf66f497f, 0x9308f5c7, 0x7da740d5, + 0x18c0fc6d, 0x4ed09f35, 0x2bb7238d, 0xc518969f, 0xa07f2a27, + 0x1947fdba, 0x7c204102, 0x928ff410, 0xf7e848a8, 0x3d58149b, + 0x583fa823, 0xb6901d31, 0xd3f7a189, 0x6acf7614, 0x0fa8caac, + 0xe1077fbe, 0x8460c306, 0xd270a05e, 0xb7171ce6, 0x59b8a9f4, + 0x3cdf154c, 0x85e7c2d1, 0xe0807e69, 0x0e2fcb7b, 0x6b4877c3, + 0xa20f0dcb, 0xc768b173, 0x29c70461, 0x4ca0b8d9, 0xf5986f44, + 0x90ffd3fc, 0x7e5066ee, 0x1b37da56, 0x4d27b90e, 0x284005b6, + 0xc6efb0a4, 0xa3880c1c, 0x1ab0db81, 0x7fd76739, 0x9178d22b, + 0xf41f6e93, 0x03f7263b, 0x66909a83, 0x883f2f91, 0xed589329, + 0x546044b4, 0x3107f80c, 0xdfa84d1e, 0xbacff1a6, 0xecdf92fe, + 0x89b82e46, 0x67179b54, 0x027027ec, 0xbb48f071, 0xde2f4cc9, + 0x3080f9db, 0x55e74563, 0x9ca03f6b, 0xf9c783d3, 0x176836c1, + 0x720f8a79, 0xcb375de4, 0xae50e15c, 0x40ff544e, 0x2598e8f6, + 0x73888bae, 0x16ef3716, 0xf8408204, 0x9d273ebc, 0x241fe921, + 0x41785599, 0xafd7e08b, 0xcab05c33, 0x3bb659ed, 0x5ed1e555, + 0xb07e5047, 0xd519ecff, 0x6c213b62, 0x094687da, 0xe7e932c8, + 0x828e8e70, 0xd49eed28, 0xb1f95190, 0x5f56e482, 0x3a31583a, + 0x83098fa7, 0xe66e331f, 0x08c1860d, 0x6da63ab5, 0xa4e140bd, + 0xc186fc05, 0x2f294917, 0x4a4ef5af, 0xf3762232, 0x96119e8a, + 0x78be2b98, 0x1dd99720, 0x4bc9f478, 0x2eae48c0, 0xc001fdd2, + 0xa566416a, 0x1c5e96f7, 0x79392a4f, 0x97969f5d, 0xf2f123e5, + 0x05196b4d, 0x607ed7f5, 0x8ed162e7, 0xebb6de5f, 0x528e09c2, + 0x37e9b57a, 0xd9460068, 0xbc21bcd0, 0xea31df88, 0x8f566330, + 0x61f9d622, 0x049e6a9a, 0xbda6bd07, 0xd8c101bf, 0x366eb4ad, + 0x53090815, 0x9a4e721d, 0xff29cea5, 0x11867bb7, 0x74e1c70f, + 0xcdd91092, 0xa8beac2a, 0x46111938, 0x2376a580, 0x7566c6d8, + 0x10017a60, 0xfeaecf72, 0x9bc973ca, 0x22f1a457, 0x479618ef, + 0xa939adfd, 0xcc5e1145, 0x06ee4d76, 0x6389f1ce, 0x8d2644dc, + 0xe841f864, 0x51792ff9, 0x341e9341, 0xdab12653, 0xbfd69aeb, + 0xe9c6f9b3, 0x8ca1450b, 0x620ef019, 0x07694ca1, 0xbe519b3c, + 0xdb362784, 0x35999296, 0x50fe2e2e, 0x99b95426, 0xfcdee89e, + 0x12715d8c, 0x7716e134, 0xce2e36a9, 0xab498a11, 0x45e63f03, + 0x208183bb, 0x7691e0e3, 0x13f65c5b, 0xfd59e949, 0x983e55f1, + 0x2106826c, 0x44613ed4, 0xaace8bc6, 0xcfa9377e, 0x38417fd6, + 0x5d26c36e, 0xb389767c, 0xd6eecac4, 0x6fd61d59, 0x0ab1a1e1, + 0xe41e14f3, 0x8179a84b, 0xd769cb13, 0xb20e77ab, 0x5ca1c2b9, + 0x39c67e01, 0x80fea99c, 0xe5991524, 0x0b36a036, 0x6e511c8e, + 0xa7166686, 0xc271da3e, 0x2cde6f2c, 0x49b9d394, 0xf0810409, + 0x95e6b8b1, 0x7b490da3, 0x1e2eb11b, 0x483ed243, 0x2d596efb, + 0xc3f6dbe9, 0xa6916751, 0x1fa9b0cc, 0x7ace0c74, 0x9461b966, + 0xf10605de}}; + +#endif + +#endif + +#if N == 2 + +#if W == 8 + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0xae689191, 0x87a02563, 0x29c8b4f2, 0xd4314c87, + 0x7a59dd16, 0x539169e4, 0xfdf9f875, 0x73139f4f, 0xdd7b0ede, + 0xf4b3ba2c, 0x5adb2bbd, 0xa722d3c8, 0x094a4259, 0x2082f6ab, + 0x8eea673a, 0xe6273e9e, 0x484faf0f, 0x61871bfd, 0xcfef8a6c, + 0x32167219, 0x9c7ee388, 0xb5b6577a, 0x1bdec6eb, 0x9534a1d1, + 0x3b5c3040, 0x129484b2, 0xbcfc1523, 0x4105ed56, 0xef6d7cc7, + 0xc6a5c835, 0x68cd59a4, 0x173f7b7d, 0xb957eaec, 0x909f5e1e, + 0x3ef7cf8f, 0xc30e37fa, 0x6d66a66b, 0x44ae1299, 0xeac68308, + 0x642ce432, 0xca4475a3, 0xe38cc151, 0x4de450c0, 0xb01da8b5, + 0x1e753924, 0x37bd8dd6, 0x99d51c47, 0xf11845e3, 0x5f70d472, + 0x76b86080, 0xd8d0f111, 0x25290964, 0x8b4198f5, 0xa2892c07, + 0x0ce1bd96, 0x820bdaac, 0x2c634b3d, 0x05abffcf, 0xabc36e5e, + 0x563a962b, 0xf85207ba, 0xd19ab348, 0x7ff222d9, 0x2e7ef6fa, + 0x8016676b, 0xa9ded399, 0x07b64208, 0xfa4fba7d, 0x54272bec, + 0x7def9f1e, 0xd3870e8f, 0x5d6d69b5, 0xf305f824, 0xdacd4cd6, + 0x74a5dd47, 0x895c2532, 0x2734b4a3, 0x0efc0051, 0xa09491c0, + 0xc859c864, 0x663159f5, 0x4ff9ed07, 0xe1917c96, 0x1c6884e3, + 0xb2001572, 0x9bc8a180, 0x35a03011, 0xbb4a572b, 0x1522c6ba, + 0x3cea7248, 0x9282e3d9, 0x6f7b1bac, 0xc1138a3d, 0xe8db3ecf, + 0x46b3af5e, 0x39418d87, 0x97291c16, 0xbee1a8e4, 0x10893975, + 0xed70c100, 0x43185091, 0x6ad0e463, 0xc4b875f2, 0x4a5212c8, + 0xe43a8359, 0xcdf237ab, 0x639aa63a, 0x9e635e4f, 0x300bcfde, + 0x19c37b2c, 0xb7abeabd, 0xdf66b319, 0x710e2288, 0x58c6967a, + 0xf6ae07eb, 0x0b57ff9e, 0xa53f6e0f, 0x8cf7dafd, 0x229f4b6c, + 0xac752c56, 0x021dbdc7, 0x2bd50935, 0x85bd98a4, 0x784460d1, + 0xd62cf140, 0xffe445b2, 0x518cd423, 0x5cfdedf4, 0xf2957c65, + 0xdb5dc897, 0x75355906, 0x88cca173, 0x26a430e2, 0x0f6c8410, + 0xa1041581, 0x2fee72bb, 0x8186e32a, 0xa84e57d8, 0x0626c649, + 0xfbdf3e3c, 0x55b7afad, 0x7c7f1b5f, 0xd2178ace, 0xbadad36a, + 0x14b242fb, 0x3d7af609, 0x93126798, 0x6eeb9fed, 0xc0830e7c, + 0xe94bba8e, 0x47232b1f, 0xc9c94c25, 0x67a1ddb4, 0x4e696946, + 0xe001f8d7, 0x1df800a2, 0xb3909133, 0x9a5825c1, 0x3430b450, + 0x4bc29689, 0xe5aa0718, 0xcc62b3ea, 0x620a227b, 0x9ff3da0e, + 0x319b4b9f, 0x1853ff6d, 0xb63b6efc, 0x38d109c6, 0x96b99857, + 0xbf712ca5, 0x1119bd34, 0xece04541, 0x4288d4d0, 0x6b406022, + 0xc528f1b3, 0xade5a817, 0x038d3986, 0x2a458d74, 0x842d1ce5, + 0x79d4e490, 0xd7bc7501, 0xfe74c1f3, 0x501c5062, 0xdef63758, + 0x709ea6c9, 0x5956123b, 0xf73e83aa, 0x0ac77bdf, 0xa4afea4e, + 0x8d675ebc, 0x230fcf2d, 0x72831b0e, 0xdceb8a9f, 0xf5233e6d, + 0x5b4baffc, 0xa6b25789, 0x08dac618, 0x211272ea, 0x8f7ae37b, + 0x01908441, 0xaff815d0, 0x8630a122, 0x285830b3, 0xd5a1c8c6, + 0x7bc95957, 0x5201eda5, 0xfc697c34, 0x94a42590, 0x3accb401, + 0x130400f3, 0xbd6c9162, 0x40956917, 0xeefdf886, 0xc7354c74, + 0x695ddde5, 0xe7b7badf, 0x49df2b4e, 0x60179fbc, 0xce7f0e2d, + 0x3386f658, 0x9dee67c9, 0xb426d33b, 0x1a4e42aa, 0x65bc6073, + 0xcbd4f1e2, 0xe21c4510, 0x4c74d481, 0xb18d2cf4, 0x1fe5bd65, + 0x362d0997, 0x98459806, 0x16afff3c, 0xb8c76ead, 0x910fda5f, + 0x3f674bce, 0xc29eb3bb, 0x6cf6222a, 0x453e96d8, 0xeb560749, + 0x839b5eed, 0x2df3cf7c, 0x043b7b8e, 0xaa53ea1f, 0x57aa126a, + 0xf9c283fb, 0xd00a3709, 0x7e62a698, 0xf088c1a2, 0x5ee05033, + 0x7728e4c1, 0xd9407550, 0x24b98d25, 0x8ad11cb4, 0xa319a846, + 0x0d7139d7}, + {0x00000000, 0xb9fbdbe8, 0xa886b191, 0x117d6a79, 0x8a7c6563, + 0x3387be8b, 0x22fad4f2, 0x9b010f1a, 0xcf89cc87, 0x7672176f, + 0x670f7d16, 0xdef4a6fe, 0x45f5a9e4, 0xfc0e720c, 0xed731875, + 0x5488c39d, 0x44629f4f, 0xfd9944a7, 0xece42ede, 0x551ff536, + 0xce1efa2c, 0x77e521c4, 0x66984bbd, 0xdf639055, 0x8beb53c8, + 0x32108820, 0x236de259, 0x9a9639b1, 0x019736ab, 0xb86ced43, + 0xa911873a, 0x10ea5cd2, 0x88c53e9e, 0x313ee576, 0x20438f0f, + 0x99b854e7, 0x02b95bfd, 0xbb428015, 0xaa3fea6c, 0x13c43184, + 0x474cf219, 0xfeb729f1, 0xefca4388, 0x56319860, 0xcd30977a, + 0x74cb4c92, 0x65b626eb, 0xdc4dfd03, 0xcca7a1d1, 0x755c7a39, + 0x64211040, 0xdddacba8, 0x46dbc4b2, 0xff201f5a, 0xee5d7523, + 0x57a6aecb, 0x032e6d56, 0xbad5b6be, 0xaba8dcc7, 0x1253072f, + 0x89520835, 0x30a9d3dd, 0x21d4b9a4, 0x982f624c, 0xcafb7b7d, + 0x7300a095, 0x627dcaec, 0xdb861104, 0x40871e1e, 0xf97cc5f6, + 0xe801af8f, 0x51fa7467, 0x0572b7fa, 0xbc896c12, 0xadf4066b, + 0x140fdd83, 0x8f0ed299, 0x36f50971, 0x27886308, 0x9e73b8e0, + 0x8e99e432, 0x37623fda, 0x261f55a3, 0x9fe48e4b, 0x04e58151, + 0xbd1e5ab9, 0xac6330c0, 0x1598eb28, 0x411028b5, 0xf8ebf35d, + 0xe9969924, 0x506d42cc, 0xcb6c4dd6, 0x7297963e, 0x63eafc47, + 0xda1127af, 0x423e45e3, 0xfbc59e0b, 0xeab8f472, 0x53432f9a, + 0xc8422080, 0x71b9fb68, 0x60c49111, 0xd93f4af9, 0x8db78964, + 0x344c528c, 0x253138f5, 0x9ccae31d, 0x07cbec07, 0xbe3037ef, + 0xaf4d5d96, 0x16b6867e, 0x065cdaac, 0xbfa70144, 0xaeda6b3d, + 0x1721b0d5, 0x8c20bfcf, 0x35db6427, 0x24a60e5e, 0x9d5dd5b6, + 0xc9d5162b, 0x702ecdc3, 0x6153a7ba, 0xd8a87c52, 0x43a97348, + 0xfa52a8a0, 0xeb2fc2d9, 0x52d41931, 0x4e87f0bb, 0xf77c2b53, + 0xe601412a, 0x5ffa9ac2, 0xc4fb95d8, 0x7d004e30, 0x6c7d2449, + 0xd586ffa1, 0x810e3c3c, 0x38f5e7d4, 0x29888dad, 0x90735645, + 0x0b72595f, 0xb28982b7, 0xa3f4e8ce, 0x1a0f3326, 0x0ae56ff4, + 0xb31eb41c, 0xa263de65, 0x1b98058d, 0x80990a97, 0x3962d17f, + 0x281fbb06, 0x91e460ee, 0xc56ca373, 0x7c97789b, 0x6dea12e2, + 0xd411c90a, 0x4f10c610, 0xf6eb1df8, 0xe7967781, 0x5e6dac69, + 0xc642ce25, 0x7fb915cd, 0x6ec47fb4, 0xd73fa45c, 0x4c3eab46, + 0xf5c570ae, 0xe4b81ad7, 0x5d43c13f, 0x09cb02a2, 0xb030d94a, + 0xa14db333, 0x18b668db, 0x83b767c1, 0x3a4cbc29, 0x2b31d650, + 0x92ca0db8, 0x8220516a, 0x3bdb8a82, 0x2aa6e0fb, 0x935d3b13, + 0x085c3409, 0xb1a7efe1, 0xa0da8598, 0x19215e70, 0x4da99ded, + 0xf4524605, 0xe52f2c7c, 0x5cd4f794, 0xc7d5f88e, 0x7e2e2366, + 0x6f53491f, 0xd6a892f7, 0x847c8bc6, 0x3d87502e, 0x2cfa3a57, + 0x9501e1bf, 0x0e00eea5, 0xb7fb354d, 0xa6865f34, 0x1f7d84dc, + 0x4bf54741, 0xf20e9ca9, 0xe373f6d0, 0x5a882d38, 0xc1892222, + 0x7872f9ca, 0x690f93b3, 0xd0f4485b, 0xc01e1489, 0x79e5cf61, + 0x6898a518, 0xd1637ef0, 0x4a6271ea, 0xf399aa02, 0xe2e4c07b, + 0x5b1f1b93, 0x0f97d80e, 0xb66c03e6, 0xa711699f, 0x1eeab277, + 0x85ebbd6d, 0x3c106685, 0x2d6d0cfc, 0x9496d714, 0x0cb9b558, + 0xb5426eb0, 0xa43f04c9, 0x1dc4df21, 0x86c5d03b, 0x3f3e0bd3, + 0x2e4361aa, 0x97b8ba42, 0xc33079df, 0x7acba237, 0x6bb6c84e, + 0xd24d13a6, 0x494c1cbc, 0xf0b7c754, 0xe1caad2d, 0x583176c5, + 0x48db2a17, 0xf120f1ff, 0xe05d9b86, 0x59a6406e, 0xc2a74f74, + 0x7b5c949c, 0x6a21fee5, 0xd3da250d, 0x8752e690, 0x3ea93d78, + 0x2fd45701, 0x962f8ce9, 0x0d2e83f3, 0xb4d5581b, 0xa5a83262, + 0x1c53e98a}, + {0x00000000, 0x9d0fe176, 0xe16ec4ad, 0x7c6125db, 0x19ac8f1b, + 0x84a36e6d, 0xf8c24bb6, 0x65cdaac0, 0x33591e36, 0xae56ff40, + 0xd237da9b, 0x4f383bed, 0x2af5912d, 0xb7fa705b, 0xcb9b5580, + 0x5694b4f6, 0x66b23c6c, 0xfbbddd1a, 0x87dcf8c1, 0x1ad319b7, + 0x7f1eb377, 0xe2115201, 0x9e7077da, 0x037f96ac, 0x55eb225a, + 0xc8e4c32c, 0xb485e6f7, 0x298a0781, 0x4c47ad41, 0xd1484c37, + 0xad2969ec, 0x3026889a, 0xcd6478d8, 0x506b99ae, 0x2c0abc75, + 0xb1055d03, 0xd4c8f7c3, 0x49c716b5, 0x35a6336e, 0xa8a9d218, + 0xfe3d66ee, 0x63328798, 0x1f53a243, 0x825c4335, 0xe791e9f5, + 0x7a9e0883, 0x06ff2d58, 0x9bf0cc2e, 0xabd644b4, 0x36d9a5c2, + 0x4ab88019, 0xd7b7616f, 0xb27acbaf, 0x2f752ad9, 0x53140f02, + 0xce1bee74, 0x988f5a82, 0x0580bbf4, 0x79e19e2f, 0xe4ee7f59, + 0x8123d599, 0x1c2c34ef, 0x604d1134, 0xfd42f042, 0x41b9f7f1, + 0xdcb61687, 0xa0d7335c, 0x3dd8d22a, 0x581578ea, 0xc51a999c, + 0xb97bbc47, 0x24745d31, 0x72e0e9c7, 0xefef08b1, 0x938e2d6a, + 0x0e81cc1c, 0x6b4c66dc, 0xf64387aa, 0x8a22a271, 0x172d4307, + 0x270bcb9d, 0xba042aeb, 0xc6650f30, 0x5b6aee46, 0x3ea74486, + 0xa3a8a5f0, 0xdfc9802b, 0x42c6615d, 0x1452d5ab, 0x895d34dd, + 0xf53c1106, 0x6833f070, 0x0dfe5ab0, 0x90f1bbc6, 0xec909e1d, + 0x719f7f6b, 0x8cdd8f29, 0x11d26e5f, 0x6db34b84, 0xf0bcaaf2, + 0x95710032, 0x087ee144, 0x741fc49f, 0xe91025e9, 0xbf84911f, + 0x228b7069, 0x5eea55b2, 0xc3e5b4c4, 0xa6281e04, 0x3b27ff72, + 0x4746daa9, 0xda493bdf, 0xea6fb345, 0x77605233, 0x0b0177e8, + 0x960e969e, 0xf3c33c5e, 0x6eccdd28, 0x12adf8f3, 0x8fa21985, + 0xd936ad73, 0x44394c05, 0x385869de, 0xa55788a8, 0xc09a2268, + 0x5d95c31e, 0x21f4e6c5, 0xbcfb07b3, 0x8373efe2, 0x1e7c0e94, + 0x621d2b4f, 0xff12ca39, 0x9adf60f9, 0x07d0818f, 0x7bb1a454, + 0xe6be4522, 0xb02af1d4, 0x2d2510a2, 0x51443579, 0xcc4bd40f, + 0xa9867ecf, 0x34899fb9, 0x48e8ba62, 0xd5e75b14, 0xe5c1d38e, + 0x78ce32f8, 0x04af1723, 0x99a0f655, 0xfc6d5c95, 0x6162bde3, + 0x1d039838, 0x800c794e, 0xd698cdb8, 0x4b972cce, 0x37f60915, + 0xaaf9e863, 0xcf3442a3, 0x523ba3d5, 0x2e5a860e, 0xb3556778, + 0x4e17973a, 0xd318764c, 0xaf795397, 0x3276b2e1, 0x57bb1821, + 0xcab4f957, 0xb6d5dc8c, 0x2bda3dfa, 0x7d4e890c, 0xe041687a, + 0x9c204da1, 0x012facd7, 0x64e20617, 0xf9ede761, 0x858cc2ba, + 0x188323cc, 0x28a5ab56, 0xb5aa4a20, 0xc9cb6ffb, 0x54c48e8d, + 0x3109244d, 0xac06c53b, 0xd067e0e0, 0x4d680196, 0x1bfcb560, + 0x86f35416, 0xfa9271cd, 0x679d90bb, 0x02503a7b, 0x9f5fdb0d, + 0xe33efed6, 0x7e311fa0, 0xc2ca1813, 0x5fc5f965, 0x23a4dcbe, + 0xbeab3dc8, 0xdb669708, 0x4669767e, 0x3a0853a5, 0xa707b2d3, + 0xf1930625, 0x6c9ce753, 0x10fdc288, 0x8df223fe, 0xe83f893e, + 0x75306848, 0x09514d93, 0x945eace5, 0xa478247f, 0x3977c509, + 0x4516e0d2, 0xd81901a4, 0xbdd4ab64, 0x20db4a12, 0x5cba6fc9, + 0xc1b58ebf, 0x97213a49, 0x0a2edb3f, 0x764ffee4, 0xeb401f92, + 0x8e8db552, 0x13825424, 0x6fe371ff, 0xf2ec9089, 0x0fae60cb, + 0x92a181bd, 0xeec0a466, 0x73cf4510, 0x1602efd0, 0x8b0d0ea6, + 0xf76c2b7d, 0x6a63ca0b, 0x3cf77efd, 0xa1f89f8b, 0xdd99ba50, + 0x40965b26, 0x255bf1e6, 0xb8541090, 0xc435354b, 0x593ad43d, + 0x691c5ca7, 0xf413bdd1, 0x8872980a, 0x157d797c, 0x70b0d3bc, + 0xedbf32ca, 0x91de1711, 0x0cd1f667, 0x5a454291, 0xc74aa3e7, + 0xbb2b863c, 0x2624674a, 0x43e9cd8a, 0xdee62cfc, 0xa2870927, + 0x3f88e851}, + {0x00000000, 0xdd96d985, 0x605cb54b, 0xbdca6cce, 0xc0b96a96, + 0x1d2fb313, 0xa0e5dfdd, 0x7d730658, 0x5a03d36d, 0x87950ae8, + 0x3a5f6626, 0xe7c9bfa3, 0x9abab9fb, 0x472c607e, 0xfae60cb0, + 0x2770d535, 0xb407a6da, 0x69917f5f, 0xd45b1391, 0x09cdca14, + 0x74becc4c, 0xa92815c9, 0x14e27907, 0xc974a082, 0xee0475b7, + 0x3392ac32, 0x8e58c0fc, 0x53ce1979, 0x2ebd1f21, 0xf32bc6a4, + 0x4ee1aa6a, 0x937773ef, 0xb37e4bf5, 0x6ee89270, 0xd322febe, + 0x0eb4273b, 0x73c72163, 0xae51f8e6, 0x139b9428, 0xce0d4dad, + 0xe97d9898, 0x34eb411d, 0x89212dd3, 0x54b7f456, 0x29c4f20e, + 0xf4522b8b, 0x49984745, 0x940e9ec0, 0x0779ed2f, 0xdaef34aa, + 0x67255864, 0xbab381e1, 0xc7c087b9, 0x1a565e3c, 0xa79c32f2, + 0x7a0aeb77, 0x5d7a3e42, 0x80ece7c7, 0x3d268b09, 0xe0b0528c, + 0x9dc354d4, 0x40558d51, 0xfd9fe19f, 0x2009381a, 0xbd8d91ab, + 0x601b482e, 0xddd124e0, 0x0047fd65, 0x7d34fb3d, 0xa0a222b8, + 0x1d684e76, 0xc0fe97f3, 0xe78e42c6, 0x3a189b43, 0x87d2f78d, + 0x5a442e08, 0x27372850, 0xfaa1f1d5, 0x476b9d1b, 0x9afd449e, + 0x098a3771, 0xd41ceef4, 0x69d6823a, 0xb4405bbf, 0xc9335de7, + 0x14a58462, 0xa96fe8ac, 0x74f93129, 0x5389e41c, 0x8e1f3d99, + 0x33d55157, 0xee4388d2, 0x93308e8a, 0x4ea6570f, 0xf36c3bc1, + 0x2efae244, 0x0ef3da5e, 0xd36503db, 0x6eaf6f15, 0xb339b690, + 0xce4ab0c8, 0x13dc694d, 0xae160583, 0x7380dc06, 0x54f00933, + 0x8966d0b6, 0x34acbc78, 0xe93a65fd, 0x944963a5, 0x49dfba20, + 0xf415d6ee, 0x29830f6b, 0xbaf47c84, 0x6762a501, 0xdaa8c9cf, + 0x073e104a, 0x7a4d1612, 0xa7dbcf97, 0x1a11a359, 0xc7877adc, + 0xe0f7afe9, 0x3d61766c, 0x80ab1aa2, 0x5d3dc327, 0x204ec57f, + 0xfdd81cfa, 0x40127034, 0x9d84a9b1, 0xa06a2517, 0x7dfcfc92, + 0xc036905c, 0x1da049d9, 0x60d34f81, 0xbd459604, 0x008ffaca, + 0xdd19234f, 0xfa69f67a, 0x27ff2fff, 0x9a354331, 0x47a39ab4, + 0x3ad09cec, 0xe7464569, 0x5a8c29a7, 0x871af022, 0x146d83cd, + 0xc9fb5a48, 0x74313686, 0xa9a7ef03, 0xd4d4e95b, 0x094230de, + 0xb4885c10, 0x691e8595, 0x4e6e50a0, 0x93f88925, 0x2e32e5eb, + 0xf3a43c6e, 0x8ed73a36, 0x5341e3b3, 0xee8b8f7d, 0x331d56f8, + 0x13146ee2, 0xce82b767, 0x7348dba9, 0xaede022c, 0xd3ad0474, + 0x0e3bddf1, 0xb3f1b13f, 0x6e6768ba, 0x4917bd8f, 0x9481640a, + 0x294b08c4, 0xf4ddd141, 0x89aed719, 0x54380e9c, 0xe9f26252, + 0x3464bbd7, 0xa713c838, 0x7a8511bd, 0xc74f7d73, 0x1ad9a4f6, + 0x67aaa2ae, 0xba3c7b2b, 0x07f617e5, 0xda60ce60, 0xfd101b55, + 0x2086c2d0, 0x9d4cae1e, 0x40da779b, 0x3da971c3, 0xe03fa846, + 0x5df5c488, 0x80631d0d, 0x1de7b4bc, 0xc0716d39, 0x7dbb01f7, + 0xa02dd872, 0xdd5ede2a, 0x00c807af, 0xbd026b61, 0x6094b2e4, + 0x47e467d1, 0x9a72be54, 0x27b8d29a, 0xfa2e0b1f, 0x875d0d47, + 0x5acbd4c2, 0xe701b80c, 0x3a976189, 0xa9e01266, 0x7476cbe3, + 0xc9bca72d, 0x142a7ea8, 0x695978f0, 0xb4cfa175, 0x0905cdbb, + 0xd493143e, 0xf3e3c10b, 0x2e75188e, 0x93bf7440, 0x4e29adc5, + 0x335aab9d, 0xeecc7218, 0x53061ed6, 0x8e90c753, 0xae99ff49, + 0x730f26cc, 0xcec54a02, 0x13539387, 0x6e2095df, 0xb3b64c5a, + 0x0e7c2094, 0xd3eaf911, 0xf49a2c24, 0x290cf5a1, 0x94c6996f, + 0x495040ea, 0x342346b2, 0xe9b59f37, 0x547ff3f9, 0x89e92a7c, + 0x1a9e5993, 0xc7088016, 0x7ac2ecd8, 0xa754355d, 0xda273305, + 0x07b1ea80, 0xba7b864e, 0x67ed5fcb, 0x409d8afe, 0x9d0b537b, + 0x20c13fb5, 0xfd57e630, 0x8024e068, 0x5db239ed, 0xe0785523, + 0x3dee8ca6}, + {0x00000000, 0x9ba54c6f, 0xec3b9e9f, 0x779ed2f0, 0x03063b7f, + 0x98a37710, 0xef3da5e0, 0x7498e98f, 0x060c76fe, 0x9da93a91, + 0xea37e861, 0x7192a40e, 0x050a4d81, 0x9eaf01ee, 0xe931d31e, + 0x72949f71, 0x0c18edfc, 0x97bda193, 0xe0237363, 0x7b863f0c, + 0x0f1ed683, 0x94bb9aec, 0xe325481c, 0x78800473, 0x0a149b02, + 0x91b1d76d, 0xe62f059d, 0x7d8a49f2, 0x0912a07d, 0x92b7ec12, + 0xe5293ee2, 0x7e8c728d, 0x1831dbf8, 0x83949797, 0xf40a4567, + 0x6faf0908, 0x1b37e087, 0x8092ace8, 0xf70c7e18, 0x6ca93277, + 0x1e3dad06, 0x8598e169, 0xf2063399, 0x69a37ff6, 0x1d3b9679, + 0x869eda16, 0xf10008e6, 0x6aa54489, 0x14293604, 0x8f8c7a6b, + 0xf812a89b, 0x63b7e4f4, 0x172f0d7b, 0x8c8a4114, 0xfb1493e4, + 0x60b1df8b, 0x122540fa, 0x89800c95, 0xfe1ede65, 0x65bb920a, + 0x11237b85, 0x8a8637ea, 0xfd18e51a, 0x66bda975, 0x3063b7f0, + 0xabc6fb9f, 0xdc58296f, 0x47fd6500, 0x33658c8f, 0xa8c0c0e0, + 0xdf5e1210, 0x44fb5e7f, 0x366fc10e, 0xadca8d61, 0xda545f91, + 0x41f113fe, 0x3569fa71, 0xaeccb61e, 0xd95264ee, 0x42f72881, + 0x3c7b5a0c, 0xa7de1663, 0xd040c493, 0x4be588fc, 0x3f7d6173, + 0xa4d82d1c, 0xd346ffec, 0x48e3b383, 0x3a772cf2, 0xa1d2609d, + 0xd64cb26d, 0x4de9fe02, 0x3971178d, 0xa2d45be2, 0xd54a8912, + 0x4eefc57d, 0x28526c08, 0xb3f72067, 0xc469f297, 0x5fccbef8, + 0x2b545777, 0xb0f11b18, 0xc76fc9e8, 0x5cca8587, 0x2e5e1af6, + 0xb5fb5699, 0xc2658469, 0x59c0c806, 0x2d582189, 0xb6fd6de6, + 0xc163bf16, 0x5ac6f379, 0x244a81f4, 0xbfefcd9b, 0xc8711f6b, + 0x53d45304, 0x274cba8b, 0xbce9f6e4, 0xcb772414, 0x50d2687b, + 0x2246f70a, 0xb9e3bb65, 0xce7d6995, 0x55d825fa, 0x2140cc75, + 0xbae5801a, 0xcd7b52ea, 0x56de1e85, 0x60c76fe0, 0xfb62238f, + 0x8cfcf17f, 0x1759bd10, 0x63c1549f, 0xf86418f0, 0x8ffaca00, + 0x145f866f, 0x66cb191e, 0xfd6e5571, 0x8af08781, 0x1155cbee, + 0x65cd2261, 0xfe686e0e, 0x89f6bcfe, 0x1253f091, 0x6cdf821c, + 0xf77ace73, 0x80e41c83, 0x1b4150ec, 0x6fd9b963, 0xf47cf50c, + 0x83e227fc, 0x18476b93, 0x6ad3f4e2, 0xf176b88d, 0x86e86a7d, + 0x1d4d2612, 0x69d5cf9d, 0xf27083f2, 0x85ee5102, 0x1e4b1d6d, + 0x78f6b418, 0xe353f877, 0x94cd2a87, 0x0f6866e8, 0x7bf08f67, + 0xe055c308, 0x97cb11f8, 0x0c6e5d97, 0x7efac2e6, 0xe55f8e89, + 0x92c15c79, 0x09641016, 0x7dfcf999, 0xe659b5f6, 0x91c76706, + 0x0a622b69, 0x74ee59e4, 0xef4b158b, 0x98d5c77b, 0x03708b14, + 0x77e8629b, 0xec4d2ef4, 0x9bd3fc04, 0x0076b06b, 0x72e22f1a, + 0xe9476375, 0x9ed9b185, 0x057cfdea, 0x71e41465, 0xea41580a, + 0x9ddf8afa, 0x067ac695, 0x50a4d810, 0xcb01947f, 0xbc9f468f, + 0x273a0ae0, 0x53a2e36f, 0xc807af00, 0xbf997df0, 0x243c319f, + 0x56a8aeee, 0xcd0de281, 0xba933071, 0x21367c1e, 0x55ae9591, + 0xce0bd9fe, 0xb9950b0e, 0x22304761, 0x5cbc35ec, 0xc7197983, + 0xb087ab73, 0x2b22e71c, 0x5fba0e93, 0xc41f42fc, 0xb381900c, + 0x2824dc63, 0x5ab04312, 0xc1150f7d, 0xb68bdd8d, 0x2d2e91e2, + 0x59b6786d, 0xc2133402, 0xb58de6f2, 0x2e28aa9d, 0x489503e8, + 0xd3304f87, 0xa4ae9d77, 0x3f0bd118, 0x4b933897, 0xd03674f8, + 0xa7a8a608, 0x3c0dea67, 0x4e997516, 0xd53c3979, 0xa2a2eb89, + 0x3907a7e6, 0x4d9f4e69, 0xd63a0206, 0xa1a4d0f6, 0x3a019c99, + 0x448dee14, 0xdf28a27b, 0xa8b6708b, 0x33133ce4, 0x478bd56b, + 0xdc2e9904, 0xabb04bf4, 0x3015079b, 0x428198ea, 0xd924d485, + 0xaeba0675, 0x351f4a1a, 0x4187a395, 0xda22effa, 0xadbc3d0a, + 0x36197165}, + {0x00000000, 0xc18edfc0, 0x586cb9c1, 0x99e26601, 0xb0d97382, + 0x7157ac42, 0xe8b5ca43, 0x293b1583, 0xbac3e145, 0x7b4d3e85, + 0xe2af5884, 0x23218744, 0x0a1a92c7, 0xcb944d07, 0x52762b06, + 0x93f8f4c6, 0xaef6c4cb, 0x6f781b0b, 0xf69a7d0a, 0x3714a2ca, + 0x1e2fb749, 0xdfa16889, 0x46430e88, 0x87cdd148, 0x1435258e, + 0xd5bbfa4e, 0x4c599c4f, 0x8dd7438f, 0xa4ec560c, 0x656289cc, + 0xfc80efcd, 0x3d0e300d, 0x869c8fd7, 0x47125017, 0xdef03616, + 0x1f7ee9d6, 0x3645fc55, 0xf7cb2395, 0x6e294594, 0xafa79a54, + 0x3c5f6e92, 0xfdd1b152, 0x6433d753, 0xa5bd0893, 0x8c861d10, + 0x4d08c2d0, 0xd4eaa4d1, 0x15647b11, 0x286a4b1c, 0xe9e494dc, + 0x7006f2dd, 0xb1882d1d, 0x98b3389e, 0x593de75e, 0xc0df815f, + 0x01515e9f, 0x92a9aa59, 0x53277599, 0xcac51398, 0x0b4bcc58, + 0x2270d9db, 0xe3fe061b, 0x7a1c601a, 0xbb92bfda, 0xd64819ef, + 0x17c6c62f, 0x8e24a02e, 0x4faa7fee, 0x66916a6d, 0xa71fb5ad, + 0x3efdd3ac, 0xff730c6c, 0x6c8bf8aa, 0xad05276a, 0x34e7416b, + 0xf5699eab, 0xdc528b28, 0x1ddc54e8, 0x843e32e9, 0x45b0ed29, + 0x78bedd24, 0xb93002e4, 0x20d264e5, 0xe15cbb25, 0xc867aea6, + 0x09e97166, 0x900b1767, 0x5185c8a7, 0xc27d3c61, 0x03f3e3a1, + 0x9a1185a0, 0x5b9f5a60, 0x72a44fe3, 0xb32a9023, 0x2ac8f622, + 0xeb4629e2, 0x50d49638, 0x915a49f8, 0x08b82ff9, 0xc936f039, + 0xe00de5ba, 0x21833a7a, 0xb8615c7b, 0x79ef83bb, 0xea17777d, + 0x2b99a8bd, 0xb27bcebc, 0x73f5117c, 0x5ace04ff, 0x9b40db3f, + 0x02a2bd3e, 0xc32c62fe, 0xfe2252f3, 0x3fac8d33, 0xa64eeb32, + 0x67c034f2, 0x4efb2171, 0x8f75feb1, 0x169798b0, 0xd7194770, + 0x44e1b3b6, 0x856f6c76, 0x1c8d0a77, 0xdd03d5b7, 0xf438c034, + 0x35b61ff4, 0xac5479f5, 0x6ddaa635, 0x77e1359f, 0xb66fea5f, + 0x2f8d8c5e, 0xee03539e, 0xc738461d, 0x06b699dd, 0x9f54ffdc, + 0x5eda201c, 0xcd22d4da, 0x0cac0b1a, 0x954e6d1b, 0x54c0b2db, + 0x7dfba758, 0xbc757898, 0x25971e99, 0xe419c159, 0xd917f154, + 0x18992e94, 0x817b4895, 0x40f59755, 0x69ce82d6, 0xa8405d16, + 0x31a23b17, 0xf02ce4d7, 0x63d41011, 0xa25acfd1, 0x3bb8a9d0, + 0xfa367610, 0xd30d6393, 0x1283bc53, 0x8b61da52, 0x4aef0592, + 0xf17dba48, 0x30f36588, 0xa9110389, 0x689fdc49, 0x41a4c9ca, + 0x802a160a, 0x19c8700b, 0xd846afcb, 0x4bbe5b0d, 0x8a3084cd, + 0x13d2e2cc, 0xd25c3d0c, 0xfb67288f, 0x3ae9f74f, 0xa30b914e, + 0x62854e8e, 0x5f8b7e83, 0x9e05a143, 0x07e7c742, 0xc6691882, + 0xef520d01, 0x2edcd2c1, 0xb73eb4c0, 0x76b06b00, 0xe5489fc6, + 0x24c64006, 0xbd242607, 0x7caaf9c7, 0x5591ec44, 0x941f3384, + 0x0dfd5585, 0xcc738a45, 0xa1a92c70, 0x6027f3b0, 0xf9c595b1, + 0x384b4a71, 0x11705ff2, 0xd0fe8032, 0x491ce633, 0x889239f3, + 0x1b6acd35, 0xdae412f5, 0x430674f4, 0x8288ab34, 0xabb3beb7, + 0x6a3d6177, 0xf3df0776, 0x3251d8b6, 0x0f5fe8bb, 0xced1377b, + 0x5733517a, 0x96bd8eba, 0xbf869b39, 0x7e0844f9, 0xe7ea22f8, + 0x2664fd38, 0xb59c09fe, 0x7412d63e, 0xedf0b03f, 0x2c7e6fff, + 0x05457a7c, 0xc4cba5bc, 0x5d29c3bd, 0x9ca71c7d, 0x2735a3a7, + 0xe6bb7c67, 0x7f591a66, 0xbed7c5a6, 0x97ecd025, 0x56620fe5, + 0xcf8069e4, 0x0e0eb624, 0x9df642e2, 0x5c789d22, 0xc59afb23, + 0x041424e3, 0x2d2f3160, 0xeca1eea0, 0x754388a1, 0xb4cd5761, + 0x89c3676c, 0x484db8ac, 0xd1afdead, 0x1021016d, 0x391a14ee, + 0xf894cb2e, 0x6176ad2f, 0xa0f872ef, 0x33008629, 0xf28e59e9, + 0x6b6c3fe8, 0xaae2e028, 0x83d9f5ab, 0x42572a6b, 0xdbb54c6a, + 0x1a3b93aa}, + {0x00000000, 0xefc26b3e, 0x04f5d03d, 0xeb37bb03, 0x09eba07a, + 0xe629cb44, 0x0d1e7047, 0xe2dc1b79, 0x13d740f4, 0xfc152bca, + 0x172290c9, 0xf8e0fbf7, 0x1a3ce08e, 0xf5fe8bb0, 0x1ec930b3, + 0xf10b5b8d, 0x27ae81e8, 0xc86cead6, 0x235b51d5, 0xcc993aeb, + 0x2e452192, 0xc1874aac, 0x2ab0f1af, 0xc5729a91, 0x3479c11c, + 0xdbbbaa22, 0x308c1121, 0xdf4e7a1f, 0x3d926166, 0xd2500a58, + 0x3967b15b, 0xd6a5da65, 0x4f5d03d0, 0xa09f68ee, 0x4ba8d3ed, + 0xa46ab8d3, 0x46b6a3aa, 0xa974c894, 0x42437397, 0xad8118a9, + 0x5c8a4324, 0xb348281a, 0x587f9319, 0xb7bdf827, 0x5561e35e, + 0xbaa38860, 0x51943363, 0xbe56585d, 0x68f38238, 0x8731e906, + 0x6c065205, 0x83c4393b, 0x61182242, 0x8eda497c, 0x65edf27f, + 0x8a2f9941, 0x7b24c2cc, 0x94e6a9f2, 0x7fd112f1, 0x901379cf, + 0x72cf62b6, 0x9d0d0988, 0x763ab28b, 0x99f8d9b5, 0x9eba07a0, + 0x71786c9e, 0x9a4fd79d, 0x758dbca3, 0x9751a7da, 0x7893cce4, + 0x93a477e7, 0x7c661cd9, 0x8d6d4754, 0x62af2c6a, 0x89989769, + 0x665afc57, 0x8486e72e, 0x6b448c10, 0x80733713, 0x6fb15c2d, + 0xb9148648, 0x56d6ed76, 0xbde15675, 0x52233d4b, 0xb0ff2632, + 0x5f3d4d0c, 0xb40af60f, 0x5bc89d31, 0xaac3c6bc, 0x4501ad82, + 0xae361681, 0x41f47dbf, 0xa32866c6, 0x4cea0df8, 0xa7ddb6fb, + 0x481fddc5, 0xd1e70470, 0x3e256f4e, 0xd512d44d, 0x3ad0bf73, + 0xd80ca40a, 0x37cecf34, 0xdcf97437, 0x333b1f09, 0xc2304484, + 0x2df22fba, 0xc6c594b9, 0x2907ff87, 0xcbdbe4fe, 0x24198fc0, + 0xcf2e34c3, 0x20ec5ffd, 0xf6498598, 0x198beea6, 0xf2bc55a5, + 0x1d7e3e9b, 0xffa225e2, 0x10604edc, 0xfb57f5df, 0x14959ee1, + 0xe59ec56c, 0x0a5cae52, 0xe16b1551, 0x0ea97e6f, 0xec756516, + 0x03b70e28, 0xe880b52b, 0x0742de15, 0xe6050901, 0x09c7623f, + 0xe2f0d93c, 0x0d32b202, 0xefeea97b, 0x002cc245, 0xeb1b7946, + 0x04d91278, 0xf5d249f5, 0x1a1022cb, 0xf12799c8, 0x1ee5f2f6, + 0xfc39e98f, 0x13fb82b1, 0xf8cc39b2, 0x170e528c, 0xc1ab88e9, + 0x2e69e3d7, 0xc55e58d4, 0x2a9c33ea, 0xc8402893, 0x278243ad, + 0xccb5f8ae, 0x23779390, 0xd27cc81d, 0x3dbea323, 0xd6891820, + 0x394b731e, 0xdb976867, 0x34550359, 0xdf62b85a, 0x30a0d364, + 0xa9580ad1, 0x469a61ef, 0xadaddaec, 0x426fb1d2, 0xa0b3aaab, + 0x4f71c195, 0xa4467a96, 0x4b8411a8, 0xba8f4a25, 0x554d211b, + 0xbe7a9a18, 0x51b8f126, 0xb364ea5f, 0x5ca68161, 0xb7913a62, + 0x5853515c, 0x8ef68b39, 0x6134e007, 0x8a035b04, 0x65c1303a, + 0x871d2b43, 0x68df407d, 0x83e8fb7e, 0x6c2a9040, 0x9d21cbcd, + 0x72e3a0f3, 0x99d41bf0, 0x761670ce, 0x94ca6bb7, 0x7b080089, + 0x903fbb8a, 0x7ffdd0b4, 0x78bf0ea1, 0x977d659f, 0x7c4ade9c, + 0x9388b5a2, 0x7154aedb, 0x9e96c5e5, 0x75a17ee6, 0x9a6315d8, + 0x6b684e55, 0x84aa256b, 0x6f9d9e68, 0x805ff556, 0x6283ee2f, + 0x8d418511, 0x66763e12, 0x89b4552c, 0x5f118f49, 0xb0d3e477, + 0x5be45f74, 0xb426344a, 0x56fa2f33, 0xb938440d, 0x520fff0e, + 0xbdcd9430, 0x4cc6cfbd, 0xa304a483, 0x48331f80, 0xa7f174be, + 0x452d6fc7, 0xaaef04f9, 0x41d8bffa, 0xae1ad4c4, 0x37e20d71, + 0xd820664f, 0x3317dd4c, 0xdcd5b672, 0x3e09ad0b, 0xd1cbc635, + 0x3afc7d36, 0xd53e1608, 0x24354d85, 0xcbf726bb, 0x20c09db8, + 0xcf02f686, 0x2ddeedff, 0xc21c86c1, 0x292b3dc2, 0xc6e956fc, + 0x104c8c99, 0xff8ee7a7, 0x14b95ca4, 0xfb7b379a, 0x19a72ce3, + 0xf66547dd, 0x1d52fcde, 0xf29097e0, 0x039bcc6d, 0xec59a753, + 0x076e1c50, 0xe8ac776e, 0x0a706c17, 0xe5b20729, 0x0e85bc2a, + 0xe147d714}, + {0x00000000, 0x177b1443, 0x2ef62886, 0x398d3cc5, 0x5dec510c, + 0x4a97454f, 0x731a798a, 0x64616dc9, 0xbbd8a218, 0xaca3b65b, + 0x952e8a9e, 0x82559edd, 0xe634f314, 0xf14fe757, 0xc8c2db92, + 0xdfb9cfd1, 0xacc04271, 0xbbbb5632, 0x82366af7, 0x954d7eb4, + 0xf12c137d, 0xe657073e, 0xdfda3bfb, 0xc8a12fb8, 0x1718e069, + 0x0063f42a, 0x39eec8ef, 0x2e95dcac, 0x4af4b165, 0x5d8fa526, + 0x640299e3, 0x73798da0, 0x82f182a3, 0x958a96e0, 0xac07aa25, + 0xbb7cbe66, 0xdf1dd3af, 0xc866c7ec, 0xf1ebfb29, 0xe690ef6a, + 0x392920bb, 0x2e5234f8, 0x17df083d, 0x00a41c7e, 0x64c571b7, + 0x73be65f4, 0x4a335931, 0x5d484d72, 0x2e31c0d2, 0x394ad491, + 0x00c7e854, 0x17bcfc17, 0x73dd91de, 0x64a6859d, 0x5d2bb958, + 0x4a50ad1b, 0x95e962ca, 0x82927689, 0xbb1f4a4c, 0xac645e0f, + 0xc80533c6, 0xdf7e2785, 0xe6f31b40, 0xf1880f03, 0xde920307, + 0xc9e91744, 0xf0642b81, 0xe71f3fc2, 0x837e520b, 0x94054648, + 0xad887a8d, 0xbaf36ece, 0x654aa11f, 0x7231b55c, 0x4bbc8999, + 0x5cc79dda, 0x38a6f013, 0x2fdde450, 0x1650d895, 0x012bccd6, + 0x72524176, 0x65295535, 0x5ca469f0, 0x4bdf7db3, 0x2fbe107a, + 0x38c50439, 0x014838fc, 0x16332cbf, 0xc98ae36e, 0xdef1f72d, + 0xe77ccbe8, 0xf007dfab, 0x9466b262, 0x831da621, 0xba909ae4, + 0xadeb8ea7, 0x5c6381a4, 0x4b1895e7, 0x7295a922, 0x65eebd61, + 0x018fd0a8, 0x16f4c4eb, 0x2f79f82e, 0x3802ec6d, 0xe7bb23bc, + 0xf0c037ff, 0xc94d0b3a, 0xde361f79, 0xba5772b0, 0xad2c66f3, + 0x94a15a36, 0x83da4e75, 0xf0a3c3d5, 0xe7d8d796, 0xde55eb53, + 0xc92eff10, 0xad4f92d9, 0xba34869a, 0x83b9ba5f, 0x94c2ae1c, + 0x4b7b61cd, 0x5c00758e, 0x658d494b, 0x72f65d08, 0x169730c1, + 0x01ec2482, 0x38611847, 0x2f1a0c04, 0x6655004f, 0x712e140c, + 0x48a328c9, 0x5fd83c8a, 0x3bb95143, 0x2cc24500, 0x154f79c5, + 0x02346d86, 0xdd8da257, 0xcaf6b614, 0xf37b8ad1, 0xe4009e92, + 0x8061f35b, 0x971ae718, 0xae97dbdd, 0xb9eccf9e, 0xca95423e, + 0xddee567d, 0xe4636ab8, 0xf3187efb, 0x97791332, 0x80020771, + 0xb98f3bb4, 0xaef42ff7, 0x714de026, 0x6636f465, 0x5fbbc8a0, + 0x48c0dce3, 0x2ca1b12a, 0x3bdaa569, 0x025799ac, 0x152c8def, + 0xe4a482ec, 0xf3df96af, 0xca52aa6a, 0xdd29be29, 0xb948d3e0, + 0xae33c7a3, 0x97befb66, 0x80c5ef25, 0x5f7c20f4, 0x480734b7, + 0x718a0872, 0x66f11c31, 0x029071f8, 0x15eb65bb, 0x2c66597e, + 0x3b1d4d3d, 0x4864c09d, 0x5f1fd4de, 0x6692e81b, 0x71e9fc58, + 0x15889191, 0x02f385d2, 0x3b7eb917, 0x2c05ad54, 0xf3bc6285, + 0xe4c776c6, 0xdd4a4a03, 0xca315e40, 0xae503389, 0xb92b27ca, + 0x80a61b0f, 0x97dd0f4c, 0xb8c70348, 0xafbc170b, 0x96312bce, + 0x814a3f8d, 0xe52b5244, 0xf2504607, 0xcbdd7ac2, 0xdca66e81, + 0x031fa150, 0x1464b513, 0x2de989d6, 0x3a929d95, 0x5ef3f05c, + 0x4988e41f, 0x7005d8da, 0x677ecc99, 0x14074139, 0x037c557a, + 0x3af169bf, 0x2d8a7dfc, 0x49eb1035, 0x5e900476, 0x671d38b3, + 0x70662cf0, 0xafdfe321, 0xb8a4f762, 0x8129cba7, 0x9652dfe4, + 0xf233b22d, 0xe548a66e, 0xdcc59aab, 0xcbbe8ee8, 0x3a3681eb, + 0x2d4d95a8, 0x14c0a96d, 0x03bbbd2e, 0x67dad0e7, 0x70a1c4a4, + 0x492cf861, 0x5e57ec22, 0x81ee23f3, 0x969537b0, 0xaf180b75, + 0xb8631f36, 0xdc0272ff, 0xcb7966bc, 0xf2f45a79, 0xe58f4e3a, + 0x96f6c39a, 0x818dd7d9, 0xb800eb1c, 0xaf7bff5f, 0xcb1a9296, + 0xdc6186d5, 0xe5ecba10, 0xf297ae53, 0x2d2e6182, 0x3a5575c1, + 0x03d84904, 0x14a35d47, 0x70c2308e, 0x67b924cd, 0x5e341808, + 0x494f0c4b}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x0000000000000000, 0x43147b1700000000, 0x8628f62e00000000, + 0xc53c8d3900000000, 0x0c51ec5d00000000, 0x4f45974a00000000, + 0x8a791a7300000000, 0xc96d616400000000, 0x18a2d8bb00000000, + 0x5bb6a3ac00000000, 0x9e8a2e9500000000, 0xdd9e558200000000, + 0x14f334e600000000, 0x57e74ff100000000, 0x92dbc2c800000000, + 0xd1cfb9df00000000, 0x7142c0ac00000000, 0x3256bbbb00000000, + 0xf76a368200000000, 0xb47e4d9500000000, 0x7d132cf100000000, + 0x3e0757e600000000, 0xfb3bdadf00000000, 0xb82fa1c800000000, + 0x69e0181700000000, 0x2af4630000000000, 0xefc8ee3900000000, + 0xacdc952e00000000, 0x65b1f44a00000000, 0x26a58f5d00000000, + 0xe399026400000000, 0xa08d797300000000, 0xa382f18200000000, + 0xe0968a9500000000, 0x25aa07ac00000000, 0x66be7cbb00000000, + 0xafd31ddf00000000, 0xecc766c800000000, 0x29fbebf100000000, + 0x6aef90e600000000, 0xbb20293900000000, 0xf834522e00000000, + 0x3d08df1700000000, 0x7e1ca40000000000, 0xb771c56400000000, + 0xf465be7300000000, 0x3159334a00000000, 0x724d485d00000000, + 0xd2c0312e00000000, 0x91d44a3900000000, 0x54e8c70000000000, + 0x17fcbc1700000000, 0xde91dd7300000000, 0x9d85a66400000000, + 0x58b92b5d00000000, 0x1bad504a00000000, 0xca62e99500000000, + 0x8976928200000000, 0x4c4a1fbb00000000, 0x0f5e64ac00000000, + 0xc63305c800000000, 0x85277edf00000000, 0x401bf3e600000000, + 0x030f88f100000000, 0x070392de00000000, 0x4417e9c900000000, + 0x812b64f000000000, 0xc23f1fe700000000, 0x0b527e8300000000, + 0x4846059400000000, 0x8d7a88ad00000000, 0xce6ef3ba00000000, + 0x1fa14a6500000000, 0x5cb5317200000000, 0x9989bc4b00000000, + 0xda9dc75c00000000, 0x13f0a63800000000, 0x50e4dd2f00000000, + 0x95d8501600000000, 0xd6cc2b0100000000, 0x7641527200000000, + 0x3555296500000000, 0xf069a45c00000000, 0xb37ddf4b00000000, + 0x7a10be2f00000000, 0x3904c53800000000, 0xfc38480100000000, + 0xbf2c331600000000, 0x6ee38ac900000000, 0x2df7f1de00000000, + 0xe8cb7ce700000000, 0xabdf07f000000000, 0x62b2669400000000, + 0x21a61d8300000000, 0xe49a90ba00000000, 0xa78eebad00000000, + 0xa481635c00000000, 0xe795184b00000000, 0x22a9957200000000, + 0x61bdee6500000000, 0xa8d08f0100000000, 0xebc4f41600000000, + 0x2ef8792f00000000, 0x6dec023800000000, 0xbc23bbe700000000, + 0xff37c0f000000000, 0x3a0b4dc900000000, 0x791f36de00000000, + 0xb07257ba00000000, 0xf3662cad00000000, 0x365aa19400000000, + 0x754eda8300000000, 0xd5c3a3f000000000, 0x96d7d8e700000000, + 0x53eb55de00000000, 0x10ff2ec900000000, 0xd9924fad00000000, + 0x9a8634ba00000000, 0x5fbab98300000000, 0x1caec29400000000, + 0xcd617b4b00000000, 0x8e75005c00000000, 0x4b498d6500000000, + 0x085df67200000000, 0xc130971600000000, 0x8224ec0100000000, + 0x4718613800000000, 0x040c1a2f00000000, 0x4f00556600000000, + 0x0c142e7100000000, 0xc928a34800000000, 0x8a3cd85f00000000, + 0x4351b93b00000000, 0x0045c22c00000000, 0xc5794f1500000000, + 0x866d340200000000, 0x57a28ddd00000000, 0x14b6f6ca00000000, + 0xd18a7bf300000000, 0x929e00e400000000, 0x5bf3618000000000, + 0x18e71a9700000000, 0xdddb97ae00000000, 0x9ecfecb900000000, + 0x3e4295ca00000000, 0x7d56eedd00000000, 0xb86a63e400000000, + 0xfb7e18f300000000, 0x3213799700000000, 0x7107028000000000, + 0xb43b8fb900000000, 0xf72ff4ae00000000, 0x26e04d7100000000, + 0x65f4366600000000, 0xa0c8bb5f00000000, 0xe3dcc04800000000, + 0x2ab1a12c00000000, 0x69a5da3b00000000, 0xac99570200000000, + 0xef8d2c1500000000, 0xec82a4e400000000, 0xaf96dff300000000, + 0x6aaa52ca00000000, 0x29be29dd00000000, 0xe0d348b900000000, + 0xa3c733ae00000000, 0x66fbbe9700000000, 0x25efc58000000000, + 0xf4207c5f00000000, 0xb734074800000000, 0x72088a7100000000, + 0x311cf16600000000, 0xf871900200000000, 0xbb65eb1500000000, + 0x7e59662c00000000, 0x3d4d1d3b00000000, 0x9dc0644800000000, + 0xded41f5f00000000, 0x1be8926600000000, 0x58fce97100000000, + 0x9191881500000000, 0xd285f30200000000, 0x17b97e3b00000000, + 0x54ad052c00000000, 0x8562bcf300000000, 0xc676c7e400000000, + 0x034a4add00000000, 0x405e31ca00000000, 0x893350ae00000000, + 0xca272bb900000000, 0x0f1ba68000000000, 0x4c0fdd9700000000, + 0x4803c7b800000000, 0x0b17bcaf00000000, 0xce2b319600000000, + 0x8d3f4a8100000000, 0x44522be500000000, 0x074650f200000000, + 0xc27addcb00000000, 0x816ea6dc00000000, 0x50a11f0300000000, + 0x13b5641400000000, 0xd689e92d00000000, 0x959d923a00000000, + 0x5cf0f35e00000000, 0x1fe4884900000000, 0xdad8057000000000, + 0x99cc7e6700000000, 0x3941071400000000, 0x7a557c0300000000, + 0xbf69f13a00000000, 0xfc7d8a2d00000000, 0x3510eb4900000000, + 0x7604905e00000000, 0xb3381d6700000000, 0xf02c667000000000, + 0x21e3dfaf00000000, 0x62f7a4b800000000, 0xa7cb298100000000, + 0xe4df529600000000, 0x2db233f200000000, 0x6ea648e500000000, + 0xab9ac5dc00000000, 0xe88ebecb00000000, 0xeb81363a00000000, + 0xa8954d2d00000000, 0x6da9c01400000000, 0x2ebdbb0300000000, + 0xe7d0da6700000000, 0xa4c4a17000000000, 0x61f82c4900000000, + 0x22ec575e00000000, 0xf323ee8100000000, 0xb037959600000000, + 0x750b18af00000000, 0x361f63b800000000, 0xff7202dc00000000, + 0xbc6679cb00000000, 0x795af4f200000000, 0x3a4e8fe500000000, + 0x9ac3f69600000000, 0xd9d78d8100000000, 0x1ceb00b800000000, + 0x5fff7baf00000000, 0x96921acb00000000, 0xd58661dc00000000, + 0x10baece500000000, 0x53ae97f200000000, 0x82612e2d00000000, + 0xc175553a00000000, 0x0449d80300000000, 0x475da31400000000, + 0x8e30c27000000000, 0xcd24b96700000000, 0x0818345e00000000, + 0x4b0c4f4900000000}, + {0x0000000000000000, 0x3e6bc2ef00000000, 0x3dd0f50400000000, + 0x03bb37eb00000000, 0x7aa0eb0900000000, 0x44cb29e600000000, + 0x47701e0d00000000, 0x791bdce200000000, 0xf440d71300000000, + 0xca2b15fc00000000, 0xc990221700000000, 0xf7fbe0f800000000, + 0x8ee03c1a00000000, 0xb08bfef500000000, 0xb330c91e00000000, + 0x8d5b0bf100000000, 0xe881ae2700000000, 0xd6ea6cc800000000, + 0xd5515b2300000000, 0xeb3a99cc00000000, 0x9221452e00000000, + 0xac4a87c100000000, 0xaff1b02a00000000, 0x919a72c500000000, + 0x1cc1793400000000, 0x22aabbdb00000000, 0x21118c3000000000, + 0x1f7a4edf00000000, 0x6661923d00000000, 0x580a50d200000000, + 0x5bb1673900000000, 0x65daa5d600000000, 0xd0035d4f00000000, + 0xee689fa000000000, 0xedd3a84b00000000, 0xd3b86aa400000000, + 0xaaa3b64600000000, 0x94c874a900000000, 0x9773434200000000, + 0xa91881ad00000000, 0x24438a5c00000000, 0x1a2848b300000000, + 0x19937f5800000000, 0x27f8bdb700000000, 0x5ee3615500000000, + 0x6088a3ba00000000, 0x6333945100000000, 0x5d5856be00000000, + 0x3882f36800000000, 0x06e9318700000000, 0x0552066c00000000, + 0x3b39c48300000000, 0x4222186100000000, 0x7c49da8e00000000, + 0x7ff2ed6500000000, 0x41992f8a00000000, 0xccc2247b00000000, + 0xf2a9e69400000000, 0xf112d17f00000000, 0xcf79139000000000, + 0xb662cf7200000000, 0x88090d9d00000000, 0x8bb23a7600000000, + 0xb5d9f89900000000, 0xa007ba9e00000000, 0x9e6c787100000000, + 0x9dd74f9a00000000, 0xa3bc8d7500000000, 0xdaa7519700000000, + 0xe4cc937800000000, 0xe777a49300000000, 0xd91c667c00000000, + 0x54476d8d00000000, 0x6a2caf6200000000, 0x6997988900000000, + 0x57fc5a6600000000, 0x2ee7868400000000, 0x108c446b00000000, + 0x1337738000000000, 0x2d5cb16f00000000, 0x488614b900000000, + 0x76edd65600000000, 0x7556e1bd00000000, 0x4b3d235200000000, + 0x3226ffb000000000, 0x0c4d3d5f00000000, 0x0ff60ab400000000, + 0x319dc85b00000000, 0xbcc6c3aa00000000, 0x82ad014500000000, + 0x811636ae00000000, 0xbf7df44100000000, 0xc66628a300000000, + 0xf80dea4c00000000, 0xfbb6dda700000000, 0xc5dd1f4800000000, + 0x7004e7d100000000, 0x4e6f253e00000000, 0x4dd412d500000000, + 0x73bfd03a00000000, 0x0aa40cd800000000, 0x34cfce3700000000, + 0x3774f9dc00000000, 0x091f3b3300000000, 0x844430c200000000, + 0xba2ff22d00000000, 0xb994c5c600000000, 0x87ff072900000000, + 0xfee4dbcb00000000, 0xc08f192400000000, 0xc3342ecf00000000, + 0xfd5fec2000000000, 0x988549f600000000, 0xa6ee8b1900000000, + 0xa555bcf200000000, 0x9b3e7e1d00000000, 0xe225a2ff00000000, + 0xdc4e601000000000, 0xdff557fb00000000, 0xe19e951400000000, + 0x6cc59ee500000000, 0x52ae5c0a00000000, 0x51156be100000000, + 0x6f7ea90e00000000, 0x166575ec00000000, 0x280eb70300000000, + 0x2bb580e800000000, 0x15de420700000000, 0x010905e600000000, + 0x3f62c70900000000, 0x3cd9f0e200000000, 0x02b2320d00000000, + 0x7ba9eeef00000000, 0x45c22c0000000000, 0x46791beb00000000, + 0x7812d90400000000, 0xf549d2f500000000, 0xcb22101a00000000, + 0xc89927f100000000, 0xf6f2e51e00000000, 0x8fe939fc00000000, + 0xb182fb1300000000, 0xb239ccf800000000, 0x8c520e1700000000, + 0xe988abc100000000, 0xd7e3692e00000000, 0xd4585ec500000000, + 0xea339c2a00000000, 0x932840c800000000, 0xad43822700000000, + 0xaef8b5cc00000000, 0x9093772300000000, 0x1dc87cd200000000, + 0x23a3be3d00000000, 0x201889d600000000, 0x1e734b3900000000, + 0x676897db00000000, 0x5903553400000000, 0x5ab862df00000000, + 0x64d3a03000000000, 0xd10a58a900000000, 0xef619a4600000000, + 0xecdaadad00000000, 0xd2b16f4200000000, 0xabaab3a000000000, + 0x95c1714f00000000, 0x967a46a400000000, 0xa811844b00000000, + 0x254a8fba00000000, 0x1b214d5500000000, 0x189a7abe00000000, + 0x26f1b85100000000, 0x5fea64b300000000, 0x6181a65c00000000, + 0x623a91b700000000, 0x5c51535800000000, 0x398bf68e00000000, + 0x07e0346100000000, 0x045b038a00000000, 0x3a30c16500000000, + 0x432b1d8700000000, 0x7d40df6800000000, 0x7efbe88300000000, + 0x40902a6c00000000, 0xcdcb219d00000000, 0xf3a0e37200000000, + 0xf01bd49900000000, 0xce70167600000000, 0xb76bca9400000000, + 0x8900087b00000000, 0x8abb3f9000000000, 0xb4d0fd7f00000000, + 0xa10ebf7800000000, 0x9f657d9700000000, 0x9cde4a7c00000000, + 0xa2b5889300000000, 0xdbae547100000000, 0xe5c5969e00000000, + 0xe67ea17500000000, 0xd815639a00000000, 0x554e686b00000000, + 0x6b25aa8400000000, 0x689e9d6f00000000, 0x56f55f8000000000, + 0x2fee836200000000, 0x1185418d00000000, 0x123e766600000000, + 0x2c55b48900000000, 0x498f115f00000000, 0x77e4d3b000000000, + 0x745fe45b00000000, 0x4a3426b400000000, 0x332ffa5600000000, + 0x0d4438b900000000, 0x0eff0f5200000000, 0x3094cdbd00000000, + 0xbdcfc64c00000000, 0x83a404a300000000, 0x801f334800000000, + 0xbe74f1a700000000, 0xc76f2d4500000000, 0xf904efaa00000000, + 0xfabfd84100000000, 0xc4d41aae00000000, 0x710de23700000000, + 0x4f6620d800000000, 0x4cdd173300000000, 0x72b6d5dc00000000, + 0x0bad093e00000000, 0x35c6cbd100000000, 0x367dfc3a00000000, + 0x08163ed500000000, 0x854d352400000000, 0xbb26f7cb00000000, + 0xb89dc02000000000, 0x86f602cf00000000, 0xffedde2d00000000, + 0xc1861cc200000000, 0xc23d2b2900000000, 0xfc56e9c600000000, + 0x998c4c1000000000, 0xa7e78eff00000000, 0xa45cb91400000000, + 0x9a377bfb00000000, 0xe32ca71900000000, 0xdd4765f600000000, + 0xdefc521d00000000, 0xe09790f200000000, 0x6dcc9b0300000000, + 0x53a759ec00000000, 0x501c6e0700000000, 0x6e77ace800000000, + 0x176c700a00000000, 0x2907b2e500000000, 0x2abc850e00000000, + 0x14d747e100000000}, + {0x0000000000000000, 0xc0df8ec100000000, 0xc1b96c5800000000, + 0x0166e29900000000, 0x8273d9b000000000, 0x42ac577100000000, + 0x43cab5e800000000, 0x83153b2900000000, 0x45e1c3ba00000000, + 0x853e4d7b00000000, 0x8458afe200000000, 0x4487212300000000, + 0xc7921a0a00000000, 0x074d94cb00000000, 0x062b765200000000, + 0xc6f4f89300000000, 0xcbc4f6ae00000000, 0x0b1b786f00000000, + 0x0a7d9af600000000, 0xcaa2143700000000, 0x49b72f1e00000000, + 0x8968a1df00000000, 0x880e434600000000, 0x48d1cd8700000000, + 0x8e25351400000000, 0x4efabbd500000000, 0x4f9c594c00000000, + 0x8f43d78d00000000, 0x0c56eca400000000, 0xcc89626500000000, + 0xcdef80fc00000000, 0x0d300e3d00000000, 0xd78f9c8600000000, + 0x1750124700000000, 0x1636f0de00000000, 0xd6e97e1f00000000, + 0x55fc453600000000, 0x9523cbf700000000, 0x9445296e00000000, + 0x549aa7af00000000, 0x926e5f3c00000000, 0x52b1d1fd00000000, + 0x53d7336400000000, 0x9308bda500000000, 0x101d868c00000000, + 0xd0c2084d00000000, 0xd1a4ead400000000, 0x117b641500000000, + 0x1c4b6a2800000000, 0xdc94e4e900000000, 0xddf2067000000000, + 0x1d2d88b100000000, 0x9e38b39800000000, 0x5ee73d5900000000, + 0x5f81dfc000000000, 0x9f5e510100000000, 0x59aaa99200000000, + 0x9975275300000000, 0x9813c5ca00000000, 0x58cc4b0b00000000, + 0xdbd9702200000000, 0x1b06fee300000000, 0x1a601c7a00000000, + 0xdabf92bb00000000, 0xef1948d600000000, 0x2fc6c61700000000, + 0x2ea0248e00000000, 0xee7faa4f00000000, 0x6d6a916600000000, + 0xadb51fa700000000, 0xacd3fd3e00000000, 0x6c0c73ff00000000, + 0xaaf88b6c00000000, 0x6a2705ad00000000, 0x6b41e73400000000, + 0xab9e69f500000000, 0x288b52dc00000000, 0xe854dc1d00000000, + 0xe9323e8400000000, 0x29edb04500000000, 0x24ddbe7800000000, + 0xe40230b900000000, 0xe564d22000000000, 0x25bb5ce100000000, + 0xa6ae67c800000000, 0x6671e90900000000, 0x67170b9000000000, + 0xa7c8855100000000, 0x613c7dc200000000, 0xa1e3f30300000000, + 0xa085119a00000000, 0x605a9f5b00000000, 0xe34fa47200000000, + 0x23902ab300000000, 0x22f6c82a00000000, 0xe22946eb00000000, + 0x3896d45000000000, 0xf8495a9100000000, 0xf92fb80800000000, + 0x39f036c900000000, 0xbae50de000000000, 0x7a3a832100000000, + 0x7b5c61b800000000, 0xbb83ef7900000000, 0x7d7717ea00000000, + 0xbda8992b00000000, 0xbcce7bb200000000, 0x7c11f57300000000, + 0xff04ce5a00000000, 0x3fdb409b00000000, 0x3ebda20200000000, + 0xfe622cc300000000, 0xf35222fe00000000, 0x338dac3f00000000, + 0x32eb4ea600000000, 0xf234c06700000000, 0x7121fb4e00000000, + 0xb1fe758f00000000, 0xb098971600000000, 0x704719d700000000, + 0xb6b3e14400000000, 0x766c6f8500000000, 0x770a8d1c00000000, + 0xb7d503dd00000000, 0x34c038f400000000, 0xf41fb63500000000, + 0xf57954ac00000000, 0x35a6da6d00000000, 0x9f35e17700000000, + 0x5fea6fb600000000, 0x5e8c8d2f00000000, 0x9e5303ee00000000, + 0x1d4638c700000000, 0xdd99b60600000000, 0xdcff549f00000000, + 0x1c20da5e00000000, 0xdad422cd00000000, 0x1a0bac0c00000000, + 0x1b6d4e9500000000, 0xdbb2c05400000000, 0x58a7fb7d00000000, + 0x987875bc00000000, 0x991e972500000000, 0x59c119e400000000, + 0x54f117d900000000, 0x942e991800000000, 0x95487b8100000000, + 0x5597f54000000000, 0xd682ce6900000000, 0x165d40a800000000, + 0x173ba23100000000, 0xd7e42cf000000000, 0x1110d46300000000, + 0xd1cf5aa200000000, 0xd0a9b83b00000000, 0x107636fa00000000, + 0x93630dd300000000, 0x53bc831200000000, 0x52da618b00000000, + 0x9205ef4a00000000, 0x48ba7df100000000, 0x8865f33000000000, + 0x890311a900000000, 0x49dc9f6800000000, 0xcac9a44100000000, + 0x0a162a8000000000, 0x0b70c81900000000, 0xcbaf46d800000000, + 0x0d5bbe4b00000000, 0xcd84308a00000000, 0xcce2d21300000000, + 0x0c3d5cd200000000, 0x8f2867fb00000000, 0x4ff7e93a00000000, + 0x4e910ba300000000, 0x8e4e856200000000, 0x837e8b5f00000000, + 0x43a1059e00000000, 0x42c7e70700000000, 0x821869c600000000, + 0x010d52ef00000000, 0xc1d2dc2e00000000, 0xc0b43eb700000000, + 0x006bb07600000000, 0xc69f48e500000000, 0x0640c62400000000, + 0x072624bd00000000, 0xc7f9aa7c00000000, 0x44ec915500000000, + 0x84331f9400000000, 0x8555fd0d00000000, 0x458a73cc00000000, + 0x702ca9a100000000, 0xb0f3276000000000, 0xb195c5f900000000, + 0x714a4b3800000000, 0xf25f701100000000, 0x3280fed000000000, + 0x33e61c4900000000, 0xf339928800000000, 0x35cd6a1b00000000, + 0xf512e4da00000000, 0xf474064300000000, 0x34ab888200000000, + 0xb7beb3ab00000000, 0x77613d6a00000000, 0x7607dff300000000, + 0xb6d8513200000000, 0xbbe85f0f00000000, 0x7b37d1ce00000000, + 0x7a51335700000000, 0xba8ebd9600000000, 0x399b86bf00000000, + 0xf944087e00000000, 0xf822eae700000000, 0x38fd642600000000, + 0xfe099cb500000000, 0x3ed6127400000000, 0x3fb0f0ed00000000, + 0xff6f7e2c00000000, 0x7c7a450500000000, 0xbca5cbc400000000, + 0xbdc3295d00000000, 0x7d1ca79c00000000, 0xa7a3352700000000, + 0x677cbbe600000000, 0x661a597f00000000, 0xa6c5d7be00000000, + 0x25d0ec9700000000, 0xe50f625600000000, 0xe46980cf00000000, + 0x24b60e0e00000000, 0xe242f69d00000000, 0x229d785c00000000, + 0x23fb9ac500000000, 0xe324140400000000, 0x60312f2d00000000, + 0xa0eea1ec00000000, 0xa188437500000000, 0x6157cdb400000000, + 0x6c67c38900000000, 0xacb84d4800000000, 0xaddeafd100000000, + 0x6d01211000000000, 0xee141a3900000000, 0x2ecb94f800000000, + 0x2fad766100000000, 0xef72f8a000000000, 0x2986003300000000, + 0xe9598ef200000000, 0xe83f6c6b00000000, 0x28e0e2aa00000000, + 0xabf5d98300000000, 0x6b2a574200000000, 0x6a4cb5db00000000, + 0xaa933b1a00000000}, + {0x0000000000000000, 0x6f4ca59b00000000, 0x9f9e3bec00000000, + 0xf0d29e7700000000, 0x7f3b060300000000, 0x1077a39800000000, + 0xe0a53def00000000, 0x8fe9987400000000, 0xfe760c0600000000, + 0x913aa99d00000000, 0x61e837ea00000000, 0x0ea4927100000000, + 0x814d0a0500000000, 0xee01af9e00000000, 0x1ed331e900000000, + 0x719f947200000000, 0xfced180c00000000, 0x93a1bd9700000000, + 0x637323e000000000, 0x0c3f867b00000000, 0x83d61e0f00000000, + 0xec9abb9400000000, 0x1c4825e300000000, 0x7304807800000000, + 0x029b140a00000000, 0x6dd7b19100000000, 0x9d052fe600000000, + 0xf2498a7d00000000, 0x7da0120900000000, 0x12ecb79200000000, + 0xe23e29e500000000, 0x8d728c7e00000000, 0xf8db311800000000, + 0x9797948300000000, 0x67450af400000000, 0x0809af6f00000000, + 0x87e0371b00000000, 0xe8ac928000000000, 0x187e0cf700000000, + 0x7732a96c00000000, 0x06ad3d1e00000000, 0x69e1988500000000, + 0x993306f200000000, 0xf67fa36900000000, 0x79963b1d00000000, + 0x16da9e8600000000, 0xe60800f100000000, 0x8944a56a00000000, + 0x0436291400000000, 0x6b7a8c8f00000000, 0x9ba812f800000000, + 0xf4e4b76300000000, 0x7b0d2f1700000000, 0x14418a8c00000000, + 0xe49314fb00000000, 0x8bdfb16000000000, 0xfa40251200000000, + 0x950c808900000000, 0x65de1efe00000000, 0x0a92bb6500000000, + 0x857b231100000000, 0xea37868a00000000, 0x1ae518fd00000000, + 0x75a9bd6600000000, 0xf0b7633000000000, 0x9ffbc6ab00000000, + 0x6f2958dc00000000, 0x0065fd4700000000, 0x8f8c653300000000, + 0xe0c0c0a800000000, 0x10125edf00000000, 0x7f5efb4400000000, + 0x0ec16f3600000000, 0x618dcaad00000000, 0x915f54da00000000, + 0xfe13f14100000000, 0x71fa693500000000, 0x1eb6ccae00000000, + 0xee6452d900000000, 0x8128f74200000000, 0x0c5a7b3c00000000, + 0x6316dea700000000, 0x93c440d000000000, 0xfc88e54b00000000, + 0x73617d3f00000000, 0x1c2dd8a400000000, 0xecff46d300000000, + 0x83b3e34800000000, 0xf22c773a00000000, 0x9d60d2a100000000, + 0x6db24cd600000000, 0x02fee94d00000000, 0x8d17713900000000, + 0xe25bd4a200000000, 0x12894ad500000000, 0x7dc5ef4e00000000, + 0x086c522800000000, 0x6720f7b300000000, 0x97f269c400000000, + 0xf8becc5f00000000, 0x7757542b00000000, 0x181bf1b000000000, + 0xe8c96fc700000000, 0x8785ca5c00000000, 0xf61a5e2e00000000, + 0x9956fbb500000000, 0x698465c200000000, 0x06c8c05900000000, + 0x8921582d00000000, 0xe66dfdb600000000, 0x16bf63c100000000, + 0x79f3c65a00000000, 0xf4814a2400000000, 0x9bcdefbf00000000, + 0x6b1f71c800000000, 0x0453d45300000000, 0x8bba4c2700000000, + 0xe4f6e9bc00000000, 0x142477cb00000000, 0x7b68d25000000000, + 0x0af7462200000000, 0x65bbe3b900000000, 0x95697dce00000000, + 0xfa25d85500000000, 0x75cc402100000000, 0x1a80e5ba00000000, + 0xea527bcd00000000, 0x851ede5600000000, 0xe06fc76000000000, + 0x8f2362fb00000000, 0x7ff1fc8c00000000, 0x10bd591700000000, + 0x9f54c16300000000, 0xf01864f800000000, 0x00cafa8f00000000, + 0x6f865f1400000000, 0x1e19cb6600000000, 0x71556efd00000000, + 0x8187f08a00000000, 0xeecb551100000000, 0x6122cd6500000000, + 0x0e6e68fe00000000, 0xfebcf68900000000, 0x91f0531200000000, + 0x1c82df6c00000000, 0x73ce7af700000000, 0x831ce48000000000, + 0xec50411b00000000, 0x63b9d96f00000000, 0x0cf57cf400000000, + 0xfc27e28300000000, 0x936b471800000000, 0xe2f4d36a00000000, + 0x8db876f100000000, 0x7d6ae88600000000, 0x12264d1d00000000, + 0x9dcfd56900000000, 0xf28370f200000000, 0x0251ee8500000000, + 0x6d1d4b1e00000000, 0x18b4f67800000000, 0x77f853e300000000, + 0x872acd9400000000, 0xe866680f00000000, 0x678ff07b00000000, + 0x08c355e000000000, 0xf811cb9700000000, 0x975d6e0c00000000, + 0xe6c2fa7e00000000, 0x898e5fe500000000, 0x795cc19200000000, + 0x1610640900000000, 0x99f9fc7d00000000, 0xf6b559e600000000, + 0x0667c79100000000, 0x692b620a00000000, 0xe459ee7400000000, + 0x8b154bef00000000, 0x7bc7d59800000000, 0x148b700300000000, + 0x9b62e87700000000, 0xf42e4dec00000000, 0x04fcd39b00000000, + 0x6bb0760000000000, 0x1a2fe27200000000, 0x756347e900000000, + 0x85b1d99e00000000, 0xeafd7c0500000000, 0x6514e47100000000, + 0x0a5841ea00000000, 0xfa8adf9d00000000, 0x95c67a0600000000, + 0x10d8a45000000000, 0x7f9401cb00000000, 0x8f469fbc00000000, + 0xe00a3a2700000000, 0x6fe3a25300000000, 0x00af07c800000000, + 0xf07d99bf00000000, 0x9f313c2400000000, 0xeeaea85600000000, + 0x81e20dcd00000000, 0x713093ba00000000, 0x1e7c362100000000, + 0x9195ae5500000000, 0xfed90bce00000000, 0x0e0b95b900000000, + 0x6147302200000000, 0xec35bc5c00000000, 0x837919c700000000, + 0x73ab87b000000000, 0x1ce7222b00000000, 0x930eba5f00000000, + 0xfc421fc400000000, 0x0c9081b300000000, 0x63dc242800000000, + 0x1243b05a00000000, 0x7d0f15c100000000, 0x8ddd8bb600000000, + 0xe2912e2d00000000, 0x6d78b65900000000, 0x023413c200000000, + 0xf2e68db500000000, 0x9daa282e00000000, 0xe803954800000000, + 0x874f30d300000000, 0x779daea400000000, 0x18d10b3f00000000, + 0x9738934b00000000, 0xf87436d000000000, 0x08a6a8a700000000, + 0x67ea0d3c00000000, 0x1675994e00000000, 0x79393cd500000000, + 0x89eba2a200000000, 0xe6a7073900000000, 0x694e9f4d00000000, + 0x06023ad600000000, 0xf6d0a4a100000000, 0x999c013a00000000, + 0x14ee8d4400000000, 0x7ba228df00000000, 0x8b70b6a800000000, + 0xe43c133300000000, 0x6bd58b4700000000, 0x04992edc00000000, + 0xf44bb0ab00000000, 0x9b07153000000000, 0xea98814200000000, + 0x85d424d900000000, 0x7506baae00000000, 0x1a4a1f3500000000, + 0x95a3874100000000, 0xfaef22da00000000, 0x0a3dbcad00000000, + 0x6571193600000000}, + {0x0000000000000000, 0x85d996dd00000000, 0x4bb55c6000000000, + 0xce6ccabd00000000, 0x966ab9c000000000, 0x13b32f1d00000000, + 0xdddfe5a000000000, 0x5806737d00000000, 0x6dd3035a00000000, + 0xe80a958700000000, 0x26665f3a00000000, 0xa3bfc9e700000000, + 0xfbb9ba9a00000000, 0x7e602c4700000000, 0xb00ce6fa00000000, + 0x35d5702700000000, 0xdaa607b400000000, 0x5f7f916900000000, + 0x91135bd400000000, 0x14cacd0900000000, 0x4cccbe7400000000, + 0xc91528a900000000, 0x0779e21400000000, 0x82a074c900000000, + 0xb77504ee00000000, 0x32ac923300000000, 0xfcc0588e00000000, + 0x7919ce5300000000, 0x211fbd2e00000000, 0xa4c62bf300000000, + 0x6aaae14e00000000, 0xef73779300000000, 0xf54b7eb300000000, + 0x7092e86e00000000, 0xbefe22d300000000, 0x3b27b40e00000000, + 0x6321c77300000000, 0xe6f851ae00000000, 0x28949b1300000000, + 0xad4d0dce00000000, 0x98987de900000000, 0x1d41eb3400000000, + 0xd32d218900000000, 0x56f4b75400000000, 0x0ef2c42900000000, + 0x8b2b52f400000000, 0x4547984900000000, 0xc09e0e9400000000, + 0x2fed790700000000, 0xaa34efda00000000, 0x6458256700000000, + 0xe181b3ba00000000, 0xb987c0c700000000, 0x3c5e561a00000000, + 0xf2329ca700000000, 0x77eb0a7a00000000, 0x423e7a5d00000000, + 0xc7e7ec8000000000, 0x098b263d00000000, 0x8c52b0e000000000, + 0xd454c39d00000000, 0x518d554000000000, 0x9fe19ffd00000000, + 0x1a38092000000000, 0xab918dbd00000000, 0x2e481b6000000000, + 0xe024d1dd00000000, 0x65fd470000000000, 0x3dfb347d00000000, + 0xb822a2a000000000, 0x764e681d00000000, 0xf397fec000000000, + 0xc6428ee700000000, 0x439b183a00000000, 0x8df7d28700000000, + 0x082e445a00000000, 0x5028372700000000, 0xd5f1a1fa00000000, + 0x1b9d6b4700000000, 0x9e44fd9a00000000, 0x71378a0900000000, + 0xf4ee1cd400000000, 0x3a82d66900000000, 0xbf5b40b400000000, + 0xe75d33c900000000, 0x6284a51400000000, 0xace86fa900000000, + 0x2931f97400000000, 0x1ce4895300000000, 0x993d1f8e00000000, + 0x5751d53300000000, 0xd28843ee00000000, 0x8a8e309300000000, + 0x0f57a64e00000000, 0xc13b6cf300000000, 0x44e2fa2e00000000, + 0x5edaf30e00000000, 0xdb0365d300000000, 0x156faf6e00000000, + 0x90b639b300000000, 0xc8b04ace00000000, 0x4d69dc1300000000, + 0x830516ae00000000, 0x06dc807300000000, 0x3309f05400000000, + 0xb6d0668900000000, 0x78bcac3400000000, 0xfd653ae900000000, + 0xa563499400000000, 0x20badf4900000000, 0xeed615f400000000, + 0x6b0f832900000000, 0x847cf4ba00000000, 0x01a5626700000000, + 0xcfc9a8da00000000, 0x4a103e0700000000, 0x12164d7a00000000, + 0x97cfdba700000000, 0x59a3111a00000000, 0xdc7a87c700000000, + 0xe9aff7e000000000, 0x6c76613d00000000, 0xa21aab8000000000, + 0x27c33d5d00000000, 0x7fc54e2000000000, 0xfa1cd8fd00000000, + 0x3470124000000000, 0xb1a9849d00000000, 0x17256aa000000000, + 0x92fcfc7d00000000, 0x5c9036c000000000, 0xd949a01d00000000, + 0x814fd36000000000, 0x049645bd00000000, 0xcafa8f0000000000, + 0x4f2319dd00000000, 0x7af669fa00000000, 0xff2fff2700000000, + 0x3143359a00000000, 0xb49aa34700000000, 0xec9cd03a00000000, + 0x694546e700000000, 0xa7298c5a00000000, 0x22f01a8700000000, + 0xcd836d1400000000, 0x485afbc900000000, 0x8636317400000000, + 0x03efa7a900000000, 0x5be9d4d400000000, 0xde30420900000000, + 0x105c88b400000000, 0x95851e6900000000, 0xa0506e4e00000000, + 0x2589f89300000000, 0xebe5322e00000000, 0x6e3ca4f300000000, + 0x363ad78e00000000, 0xb3e3415300000000, 0x7d8f8bee00000000, + 0xf8561d3300000000, 0xe26e141300000000, 0x67b782ce00000000, + 0xa9db487300000000, 0x2c02deae00000000, 0x7404add300000000, + 0xf1dd3b0e00000000, 0x3fb1f1b300000000, 0xba68676e00000000, + 0x8fbd174900000000, 0x0a64819400000000, 0xc4084b2900000000, + 0x41d1ddf400000000, 0x19d7ae8900000000, 0x9c0e385400000000, + 0x5262f2e900000000, 0xd7bb643400000000, 0x38c813a700000000, + 0xbd11857a00000000, 0x737d4fc700000000, 0xf6a4d91a00000000, + 0xaea2aa6700000000, 0x2b7b3cba00000000, 0xe517f60700000000, + 0x60ce60da00000000, 0x551b10fd00000000, 0xd0c2862000000000, + 0x1eae4c9d00000000, 0x9b77da4000000000, 0xc371a93d00000000, + 0x46a83fe000000000, 0x88c4f55d00000000, 0x0d1d638000000000, + 0xbcb4e71d00000000, 0x396d71c000000000, 0xf701bb7d00000000, + 0x72d82da000000000, 0x2ade5edd00000000, 0xaf07c80000000000, + 0x616b02bd00000000, 0xe4b2946000000000, 0xd167e44700000000, + 0x54be729a00000000, 0x9ad2b82700000000, 0x1f0b2efa00000000, + 0x470d5d8700000000, 0xc2d4cb5a00000000, 0x0cb801e700000000, + 0x8961973a00000000, 0x6612e0a900000000, 0xe3cb767400000000, + 0x2da7bcc900000000, 0xa87e2a1400000000, 0xf078596900000000, + 0x75a1cfb400000000, 0xbbcd050900000000, 0x3e1493d400000000, + 0x0bc1e3f300000000, 0x8e18752e00000000, 0x4074bf9300000000, + 0xc5ad294e00000000, 0x9dab5a3300000000, 0x1872ccee00000000, + 0xd61e065300000000, 0x53c7908e00000000, 0x49ff99ae00000000, + 0xcc260f7300000000, 0x024ac5ce00000000, 0x8793531300000000, + 0xdf95206e00000000, 0x5a4cb6b300000000, 0x94207c0e00000000, + 0x11f9ead300000000, 0x242c9af400000000, 0xa1f50c2900000000, + 0x6f99c69400000000, 0xea40504900000000, 0xb246233400000000, + 0x379fb5e900000000, 0xf9f37f5400000000, 0x7c2ae98900000000, + 0x93599e1a00000000, 0x168008c700000000, 0xd8ecc27a00000000, + 0x5d3554a700000000, 0x053327da00000000, 0x80eab10700000000, + 0x4e867bba00000000, 0xcb5fed6700000000, 0xfe8a9d4000000000, + 0x7b530b9d00000000, 0xb53fc12000000000, 0x30e657fd00000000, + 0x68e0248000000000, 0xed39b25d00000000, 0x235578e000000000, + 0xa68cee3d00000000}, + {0x0000000000000000, 0x76e10f9d00000000, 0xadc46ee100000000, + 0xdb25617c00000000, 0x1b8fac1900000000, 0x6d6ea38400000000, + 0xb64bc2f800000000, 0xc0aacd6500000000, 0x361e593300000000, + 0x40ff56ae00000000, 0x9bda37d200000000, 0xed3b384f00000000, + 0x2d91f52a00000000, 0x5b70fab700000000, 0x80559bcb00000000, + 0xf6b4945600000000, 0x6c3cb26600000000, 0x1addbdfb00000000, + 0xc1f8dc8700000000, 0xb719d31a00000000, 0x77b31e7f00000000, + 0x015211e200000000, 0xda77709e00000000, 0xac967f0300000000, + 0x5a22eb5500000000, 0x2cc3e4c800000000, 0xf7e685b400000000, + 0x81078a2900000000, 0x41ad474c00000000, 0x374c48d100000000, + 0xec6929ad00000000, 0x9a88263000000000, 0xd87864cd00000000, + 0xae996b5000000000, 0x75bc0a2c00000000, 0x035d05b100000000, + 0xc3f7c8d400000000, 0xb516c74900000000, 0x6e33a63500000000, + 0x18d2a9a800000000, 0xee663dfe00000000, 0x9887326300000000, + 0x43a2531f00000000, 0x35435c8200000000, 0xf5e991e700000000, + 0x83089e7a00000000, 0x582dff0600000000, 0x2eccf09b00000000, + 0xb444d6ab00000000, 0xc2a5d93600000000, 0x1980b84a00000000, + 0x6f61b7d700000000, 0xafcb7ab200000000, 0xd92a752f00000000, + 0x020f145300000000, 0x74ee1bce00000000, 0x825a8f9800000000, + 0xf4bb800500000000, 0x2f9ee17900000000, 0x597feee400000000, + 0x99d5238100000000, 0xef342c1c00000000, 0x34114d6000000000, + 0x42f042fd00000000, 0xf1f7b94100000000, 0x8716b6dc00000000, + 0x5c33d7a000000000, 0x2ad2d83d00000000, 0xea78155800000000, + 0x9c991ac500000000, 0x47bc7bb900000000, 0x315d742400000000, + 0xc7e9e07200000000, 0xb108efef00000000, 0x6a2d8e9300000000, + 0x1ccc810e00000000, 0xdc664c6b00000000, 0xaa8743f600000000, + 0x71a2228a00000000, 0x07432d1700000000, 0x9dcb0b2700000000, + 0xeb2a04ba00000000, 0x300f65c600000000, 0x46ee6a5b00000000, + 0x8644a73e00000000, 0xf0a5a8a300000000, 0x2b80c9df00000000, + 0x5d61c64200000000, 0xabd5521400000000, 0xdd345d8900000000, + 0x06113cf500000000, 0x70f0336800000000, 0xb05afe0d00000000, + 0xc6bbf19000000000, 0x1d9e90ec00000000, 0x6b7f9f7100000000, + 0x298fdd8c00000000, 0x5f6ed21100000000, 0x844bb36d00000000, + 0xf2aabcf000000000, 0x3200719500000000, 0x44e17e0800000000, + 0x9fc41f7400000000, 0xe92510e900000000, 0x1f9184bf00000000, + 0x69708b2200000000, 0xb255ea5e00000000, 0xc4b4e5c300000000, + 0x041e28a600000000, 0x72ff273b00000000, 0xa9da464700000000, + 0xdf3b49da00000000, 0x45b36fea00000000, 0x3352607700000000, + 0xe877010b00000000, 0x9e960e9600000000, 0x5e3cc3f300000000, + 0x28ddcc6e00000000, 0xf3f8ad1200000000, 0x8519a28f00000000, + 0x73ad36d900000000, 0x054c394400000000, 0xde69583800000000, + 0xa88857a500000000, 0x68229ac000000000, 0x1ec3955d00000000, + 0xc5e6f42100000000, 0xb307fbbc00000000, 0xe2ef738300000000, + 0x940e7c1e00000000, 0x4f2b1d6200000000, 0x39ca12ff00000000, + 0xf960df9a00000000, 0x8f81d00700000000, 0x54a4b17b00000000, + 0x2245bee600000000, 0xd4f12ab000000000, 0xa210252d00000000, + 0x7935445100000000, 0x0fd44bcc00000000, 0xcf7e86a900000000, + 0xb99f893400000000, 0x62bae84800000000, 0x145be7d500000000, + 0x8ed3c1e500000000, 0xf832ce7800000000, 0x2317af0400000000, + 0x55f6a09900000000, 0x955c6dfc00000000, 0xe3bd626100000000, + 0x3898031d00000000, 0x4e790c8000000000, 0xb8cd98d600000000, + 0xce2c974b00000000, 0x1509f63700000000, 0x63e8f9aa00000000, + 0xa34234cf00000000, 0xd5a33b5200000000, 0x0e865a2e00000000, + 0x786755b300000000, 0x3a97174e00000000, 0x4c7618d300000000, + 0x975379af00000000, 0xe1b2763200000000, 0x2118bb5700000000, + 0x57f9b4ca00000000, 0x8cdcd5b600000000, 0xfa3dda2b00000000, + 0x0c894e7d00000000, 0x7a6841e000000000, 0xa14d209c00000000, + 0xd7ac2f0100000000, 0x1706e26400000000, 0x61e7edf900000000, + 0xbac28c8500000000, 0xcc23831800000000, 0x56aba52800000000, + 0x204aaab500000000, 0xfb6fcbc900000000, 0x8d8ec45400000000, + 0x4d24093100000000, 0x3bc506ac00000000, 0xe0e067d000000000, + 0x9601684d00000000, 0x60b5fc1b00000000, 0x1654f38600000000, + 0xcd7192fa00000000, 0xbb909d6700000000, 0x7b3a500200000000, + 0x0ddb5f9f00000000, 0xd6fe3ee300000000, 0xa01f317e00000000, + 0x1318cac200000000, 0x65f9c55f00000000, 0xbedca42300000000, + 0xc83dabbe00000000, 0x089766db00000000, 0x7e76694600000000, + 0xa553083a00000000, 0xd3b207a700000000, 0x250693f100000000, + 0x53e79c6c00000000, 0x88c2fd1000000000, 0xfe23f28d00000000, + 0x3e893fe800000000, 0x4868307500000000, 0x934d510900000000, + 0xe5ac5e9400000000, 0x7f2478a400000000, 0x09c5773900000000, + 0xd2e0164500000000, 0xa40119d800000000, 0x64abd4bd00000000, + 0x124adb2000000000, 0xc96fba5c00000000, 0xbf8eb5c100000000, + 0x493a219700000000, 0x3fdb2e0a00000000, 0xe4fe4f7600000000, + 0x921f40eb00000000, 0x52b58d8e00000000, 0x2454821300000000, + 0xff71e36f00000000, 0x8990ecf200000000, 0xcb60ae0f00000000, + 0xbd81a19200000000, 0x66a4c0ee00000000, 0x1045cf7300000000, + 0xd0ef021600000000, 0xa60e0d8b00000000, 0x7d2b6cf700000000, + 0x0bca636a00000000, 0xfd7ef73c00000000, 0x8b9ff8a100000000, + 0x50ba99dd00000000, 0x265b964000000000, 0xe6f15b2500000000, + 0x901054b800000000, 0x4b3535c400000000, 0x3dd43a5900000000, + 0xa75c1c6900000000, 0xd1bd13f400000000, 0x0a98728800000000, + 0x7c797d1500000000, 0xbcd3b07000000000, 0xca32bfed00000000, + 0x1117de9100000000, 0x67f6d10c00000000, 0x9142455a00000000, + 0xe7a34ac700000000, 0x3c862bbb00000000, 0x4a67242600000000, + 0x8acde94300000000, 0xfc2ce6de00000000, 0x270987a200000000, + 0x51e8883f00000000}, + {0x0000000000000000, 0xe8dbfbb900000000, 0x91b186a800000000, + 0x796a7d1100000000, 0x63657c8a00000000, 0x8bbe873300000000, + 0xf2d4fa2200000000, 0x1a0f019b00000000, 0x87cc89cf00000000, + 0x6f17727600000000, 0x167d0f6700000000, 0xfea6f4de00000000, + 0xe4a9f54500000000, 0x0c720efc00000000, 0x751873ed00000000, + 0x9dc3885400000000, 0x4f9f624400000000, 0xa74499fd00000000, + 0xde2ee4ec00000000, 0x36f51f5500000000, 0x2cfa1ece00000000, + 0xc421e57700000000, 0xbd4b986600000000, 0x559063df00000000, + 0xc853eb8b00000000, 0x2088103200000000, 0x59e26d2300000000, + 0xb139969a00000000, 0xab36970100000000, 0x43ed6cb800000000, + 0x3a8711a900000000, 0xd25cea1000000000, 0x9e3ec58800000000, + 0x76e53e3100000000, 0x0f8f432000000000, 0xe754b89900000000, + 0xfd5bb90200000000, 0x158042bb00000000, 0x6cea3faa00000000, + 0x8431c41300000000, 0x19f24c4700000000, 0xf129b7fe00000000, + 0x8843caef00000000, 0x6098315600000000, 0x7a9730cd00000000, + 0x924ccb7400000000, 0xeb26b66500000000, 0x03fd4ddc00000000, + 0xd1a1a7cc00000000, 0x397a5c7500000000, 0x4010216400000000, + 0xa8cbdadd00000000, 0xb2c4db4600000000, 0x5a1f20ff00000000, + 0x23755dee00000000, 0xcbaea65700000000, 0x566d2e0300000000, + 0xbeb6d5ba00000000, 0xc7dca8ab00000000, 0x2f07531200000000, + 0x3508528900000000, 0xddd3a93000000000, 0xa4b9d42100000000, + 0x4c622f9800000000, 0x7d7bfbca00000000, 0x95a0007300000000, + 0xecca7d6200000000, 0x041186db00000000, 0x1e1e874000000000, + 0xf6c57cf900000000, 0x8faf01e800000000, 0x6774fa5100000000, + 0xfab7720500000000, 0x126c89bc00000000, 0x6b06f4ad00000000, + 0x83dd0f1400000000, 0x99d20e8f00000000, 0x7109f53600000000, + 0x0863882700000000, 0xe0b8739e00000000, 0x32e4998e00000000, + 0xda3f623700000000, 0xa3551f2600000000, 0x4b8ee49f00000000, + 0x5181e50400000000, 0xb95a1ebd00000000, 0xc03063ac00000000, + 0x28eb981500000000, 0xb528104100000000, 0x5df3ebf800000000, + 0x249996e900000000, 0xcc426d5000000000, 0xd64d6ccb00000000, + 0x3e96977200000000, 0x47fcea6300000000, 0xaf2711da00000000, + 0xe3453e4200000000, 0x0b9ec5fb00000000, 0x72f4b8ea00000000, + 0x9a2f435300000000, 0x802042c800000000, 0x68fbb97100000000, + 0x1191c46000000000, 0xf94a3fd900000000, 0x6489b78d00000000, + 0x8c524c3400000000, 0xf538312500000000, 0x1de3ca9c00000000, + 0x07eccb0700000000, 0xef3730be00000000, 0x965d4daf00000000, + 0x7e86b61600000000, 0xacda5c0600000000, 0x4401a7bf00000000, + 0x3d6bdaae00000000, 0xd5b0211700000000, 0xcfbf208c00000000, + 0x2764db3500000000, 0x5e0ea62400000000, 0xb6d55d9d00000000, + 0x2b16d5c900000000, 0xc3cd2e7000000000, 0xbaa7536100000000, + 0x527ca8d800000000, 0x4873a94300000000, 0xa0a852fa00000000, + 0xd9c22feb00000000, 0x3119d45200000000, 0xbbf0874e00000000, + 0x532b7cf700000000, 0x2a4101e600000000, 0xc29afa5f00000000, + 0xd895fbc400000000, 0x304e007d00000000, 0x49247d6c00000000, + 0xa1ff86d500000000, 0x3c3c0e8100000000, 0xd4e7f53800000000, + 0xad8d882900000000, 0x4556739000000000, 0x5f59720b00000000, + 0xb78289b200000000, 0xcee8f4a300000000, 0x26330f1a00000000, + 0xf46fe50a00000000, 0x1cb41eb300000000, 0x65de63a200000000, + 0x8d05981b00000000, 0x970a998000000000, 0x7fd1623900000000, + 0x06bb1f2800000000, 0xee60e49100000000, 0x73a36cc500000000, + 0x9b78977c00000000, 0xe212ea6d00000000, 0x0ac911d400000000, + 0x10c6104f00000000, 0xf81debf600000000, 0x817796e700000000, + 0x69ac6d5e00000000, 0x25ce42c600000000, 0xcd15b97f00000000, + 0xb47fc46e00000000, 0x5ca43fd700000000, 0x46ab3e4c00000000, + 0xae70c5f500000000, 0xd71ab8e400000000, 0x3fc1435d00000000, + 0xa202cb0900000000, 0x4ad930b000000000, 0x33b34da100000000, + 0xdb68b61800000000, 0xc167b78300000000, 0x29bc4c3a00000000, + 0x50d6312b00000000, 0xb80dca9200000000, 0x6a51208200000000, + 0x828adb3b00000000, 0xfbe0a62a00000000, 0x133b5d9300000000, + 0x09345c0800000000, 0xe1efa7b100000000, 0x9885daa000000000, + 0x705e211900000000, 0xed9da94d00000000, 0x054652f400000000, + 0x7c2c2fe500000000, 0x94f7d45c00000000, 0x8ef8d5c700000000, + 0x66232e7e00000000, 0x1f49536f00000000, 0xf792a8d600000000, + 0xc68b7c8400000000, 0x2e50873d00000000, 0x573afa2c00000000, + 0xbfe1019500000000, 0xa5ee000e00000000, 0x4d35fbb700000000, + 0x345f86a600000000, 0xdc847d1f00000000, 0x4147f54b00000000, + 0xa99c0ef200000000, 0xd0f673e300000000, 0x382d885a00000000, + 0x222289c100000000, 0xcaf9727800000000, 0xb3930f6900000000, + 0x5b48f4d000000000, 0x89141ec000000000, 0x61cfe57900000000, + 0x18a5986800000000, 0xf07e63d100000000, 0xea71624a00000000, + 0x02aa99f300000000, 0x7bc0e4e200000000, 0x931b1f5b00000000, + 0x0ed8970f00000000, 0xe6036cb600000000, 0x9f6911a700000000, + 0x77b2ea1e00000000, 0x6dbdeb8500000000, 0x8566103c00000000, + 0xfc0c6d2d00000000, 0x14d7969400000000, 0x58b5b90c00000000, + 0xb06e42b500000000, 0xc9043fa400000000, 0x21dfc41d00000000, + 0x3bd0c58600000000, 0xd30b3e3f00000000, 0xaa61432e00000000, + 0x42bab89700000000, 0xdf7930c300000000, 0x37a2cb7a00000000, + 0x4ec8b66b00000000, 0xa6134dd200000000, 0xbc1c4c4900000000, + 0x54c7b7f000000000, 0x2dadcae100000000, 0xc576315800000000, + 0x172adb4800000000, 0xfff120f100000000, 0x869b5de000000000, + 0x6e40a65900000000, 0x744fa7c200000000, 0x9c945c7b00000000, + 0xe5fe216a00000000, 0x0d25dad300000000, 0x90e6528700000000, + 0x783da93e00000000, 0x0157d42f00000000, 0xe98c2f9600000000, + 0xf3832e0d00000000, 0x1b58d5b400000000, 0x6232a8a500000000, + 0x8ae9531c00000000}, + {0x0000000000000000, 0x919168ae00000000, 0x6325a08700000000, + 0xf2b4c82900000000, 0x874c31d400000000, 0x16dd597a00000000, + 0xe469915300000000, 0x75f8f9fd00000000, 0x4f9f137300000000, + 0xde0e7bdd00000000, 0x2cbab3f400000000, 0xbd2bdb5a00000000, + 0xc8d322a700000000, 0x59424a0900000000, 0xabf6822000000000, + 0x3a67ea8e00000000, 0x9e3e27e600000000, 0x0faf4f4800000000, + 0xfd1b876100000000, 0x6c8aefcf00000000, 0x1972163200000000, + 0x88e37e9c00000000, 0x7a57b6b500000000, 0xebc6de1b00000000, + 0xd1a1349500000000, 0x40305c3b00000000, 0xb284941200000000, + 0x2315fcbc00000000, 0x56ed054100000000, 0xc77c6def00000000, + 0x35c8a5c600000000, 0xa459cd6800000000, 0x7d7b3f1700000000, + 0xecea57b900000000, 0x1e5e9f9000000000, 0x8fcff73e00000000, + 0xfa370ec300000000, 0x6ba6666d00000000, 0x9912ae4400000000, + 0x0883c6ea00000000, 0x32e42c6400000000, 0xa37544ca00000000, + 0x51c18ce300000000, 0xc050e44d00000000, 0xb5a81db000000000, + 0x2439751e00000000, 0xd68dbd3700000000, 0x471cd59900000000, + 0xe34518f100000000, 0x72d4705f00000000, 0x8060b87600000000, + 0x11f1d0d800000000, 0x6409292500000000, 0xf598418b00000000, + 0x072c89a200000000, 0x96bde10c00000000, 0xacda0b8200000000, + 0x3d4b632c00000000, 0xcfffab0500000000, 0x5e6ec3ab00000000, + 0x2b963a5600000000, 0xba0752f800000000, 0x48b39ad100000000, + 0xd922f27f00000000, 0xfaf67e2e00000000, 0x6b67168000000000, + 0x99d3dea900000000, 0x0842b60700000000, 0x7dba4ffa00000000, + 0xec2b275400000000, 0x1e9fef7d00000000, 0x8f0e87d300000000, + 0xb5696d5d00000000, 0x24f805f300000000, 0xd64ccdda00000000, + 0x47dda57400000000, 0x32255c8900000000, 0xa3b4342700000000, + 0x5100fc0e00000000, 0xc09194a000000000, 0x64c859c800000000, + 0xf559316600000000, 0x07edf94f00000000, 0x967c91e100000000, + 0xe384681c00000000, 0x721500b200000000, 0x80a1c89b00000000, + 0x1130a03500000000, 0x2b574abb00000000, 0xbac6221500000000, + 0x4872ea3c00000000, 0xd9e3829200000000, 0xac1b7b6f00000000, + 0x3d8a13c100000000, 0xcf3edbe800000000, 0x5eafb34600000000, + 0x878d413900000000, 0x161c299700000000, 0xe4a8e1be00000000, + 0x7539891000000000, 0x00c170ed00000000, 0x9150184300000000, + 0x63e4d06a00000000, 0xf275b8c400000000, 0xc812524a00000000, + 0x59833ae400000000, 0xab37f2cd00000000, 0x3aa69a6300000000, + 0x4f5e639e00000000, 0xdecf0b3000000000, 0x2c7bc31900000000, + 0xbdeaabb700000000, 0x19b366df00000000, 0x88220e7100000000, + 0x7a96c65800000000, 0xeb07aef600000000, 0x9eff570b00000000, + 0x0f6e3fa500000000, 0xfddaf78c00000000, 0x6c4b9f2200000000, + 0x562c75ac00000000, 0xc7bd1d0200000000, 0x3509d52b00000000, + 0xa498bd8500000000, 0xd160447800000000, 0x40f12cd600000000, + 0xb245e4ff00000000, 0x23d48c5100000000, 0xf4edfd5c00000000, + 0x657c95f200000000, 0x97c85ddb00000000, 0x0659357500000000, + 0x73a1cc8800000000, 0xe230a42600000000, 0x10846c0f00000000, + 0x811504a100000000, 0xbb72ee2f00000000, 0x2ae3868100000000, + 0xd8574ea800000000, 0x49c6260600000000, 0x3c3edffb00000000, + 0xadafb75500000000, 0x5f1b7f7c00000000, 0xce8a17d200000000, + 0x6ad3daba00000000, 0xfb42b21400000000, 0x09f67a3d00000000, + 0x9867129300000000, 0xed9feb6e00000000, 0x7c0e83c000000000, + 0x8eba4be900000000, 0x1f2b234700000000, 0x254cc9c900000000, + 0xb4dda16700000000, 0x4669694e00000000, 0xd7f801e000000000, + 0xa200f81d00000000, 0x339190b300000000, 0xc125589a00000000, + 0x50b4303400000000, 0x8996c24b00000000, 0x1807aae500000000, + 0xeab362cc00000000, 0x7b220a6200000000, 0x0edaf39f00000000, + 0x9f4b9b3100000000, 0x6dff531800000000, 0xfc6e3bb600000000, + 0xc609d13800000000, 0x5798b99600000000, 0xa52c71bf00000000, + 0x34bd191100000000, 0x4145e0ec00000000, 0xd0d4884200000000, + 0x2260406b00000000, 0xb3f128c500000000, 0x17a8e5ad00000000, + 0x86398d0300000000, 0x748d452a00000000, 0xe51c2d8400000000, + 0x90e4d47900000000, 0x0175bcd700000000, 0xf3c174fe00000000, + 0x62501c5000000000, 0x5837f6de00000000, 0xc9a69e7000000000, + 0x3b12565900000000, 0xaa833ef700000000, 0xdf7bc70a00000000, + 0x4eeaafa400000000, 0xbc5e678d00000000, 0x2dcf0f2300000000, + 0x0e1b837200000000, 0x9f8aebdc00000000, 0x6d3e23f500000000, + 0xfcaf4b5b00000000, 0x8957b2a600000000, 0x18c6da0800000000, + 0xea72122100000000, 0x7be37a8f00000000, 0x4184900100000000, + 0xd015f8af00000000, 0x22a1308600000000, 0xb330582800000000, + 0xc6c8a1d500000000, 0x5759c97b00000000, 0xa5ed015200000000, + 0x347c69fc00000000, 0x9025a49400000000, 0x01b4cc3a00000000, + 0xf300041300000000, 0x62916cbd00000000, 0x1769954000000000, + 0x86f8fdee00000000, 0x744c35c700000000, 0xe5dd5d6900000000, + 0xdfbab7e700000000, 0x4e2bdf4900000000, 0xbc9f176000000000, + 0x2d0e7fce00000000, 0x58f6863300000000, 0xc967ee9d00000000, + 0x3bd326b400000000, 0xaa424e1a00000000, 0x7360bc6500000000, + 0xe2f1d4cb00000000, 0x10451ce200000000, 0x81d4744c00000000, + 0xf42c8db100000000, 0x65bde51f00000000, 0x97092d3600000000, + 0x0698459800000000, 0x3cffaf1600000000, 0xad6ec7b800000000, + 0x5fda0f9100000000, 0xce4b673f00000000, 0xbbb39ec200000000, + 0x2a22f66c00000000, 0xd8963e4500000000, 0x490756eb00000000, + 0xed5e9b8300000000, 0x7ccff32d00000000, 0x8e7b3b0400000000, + 0x1fea53aa00000000, 0x6a12aa5700000000, 0xfb83c2f900000000, + 0x09370ad000000000, 0x98a6627e00000000, 0xa2c188f000000000, + 0x3350e05e00000000, 0xc1e4287700000000, 0x507540d900000000, + 0x258db92400000000, 0xb41cd18a00000000, 0x46a819a300000000, + 0xd739710d00000000}}; + +#else /* W == 4 */ + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0xccaa009e, 0x4225077d, 0x8e8f07e3, 0x844a0efa, + 0x48e00e64, 0xc66f0987, 0x0ac50919, 0xd3e51bb5, 0x1f4f1b2b, + 0x91c01cc8, 0x5d6a1c56, 0x57af154f, 0x9b0515d1, 0x158a1232, + 0xd92012ac, 0x7cbb312b, 0xb01131b5, 0x3e9e3656, 0xf23436c8, + 0xf8f13fd1, 0x345b3f4f, 0xbad438ac, 0x767e3832, 0xaf5e2a9e, + 0x63f42a00, 0xed7b2de3, 0x21d12d7d, 0x2b142464, 0xe7be24fa, + 0x69312319, 0xa59b2387, 0xf9766256, 0x35dc62c8, 0xbb53652b, + 0x77f965b5, 0x7d3c6cac, 0xb1966c32, 0x3f196bd1, 0xf3b36b4f, + 0x2a9379e3, 0xe639797d, 0x68b67e9e, 0xa41c7e00, 0xaed97719, + 0x62737787, 0xecfc7064, 0x205670fa, 0x85cd537d, 0x496753e3, + 0xc7e85400, 0x0b42549e, 0x01875d87, 0xcd2d5d19, 0x43a25afa, + 0x8f085a64, 0x562848c8, 0x9a824856, 0x140d4fb5, 0xd8a74f2b, + 0xd2624632, 0x1ec846ac, 0x9047414f, 0x5ced41d1, 0x299dc2ed, + 0xe537c273, 0x6bb8c590, 0xa712c50e, 0xadd7cc17, 0x617dcc89, + 0xeff2cb6a, 0x2358cbf4, 0xfa78d958, 0x36d2d9c6, 0xb85dde25, + 0x74f7debb, 0x7e32d7a2, 0xb298d73c, 0x3c17d0df, 0xf0bdd041, + 0x5526f3c6, 0x998cf358, 0x1703f4bb, 0xdba9f425, 0xd16cfd3c, + 0x1dc6fda2, 0x9349fa41, 0x5fe3fadf, 0x86c3e873, 0x4a69e8ed, + 0xc4e6ef0e, 0x084cef90, 0x0289e689, 0xce23e617, 0x40ace1f4, + 0x8c06e16a, 0xd0eba0bb, 0x1c41a025, 0x92cea7c6, 0x5e64a758, + 0x54a1ae41, 0x980baedf, 0x1684a93c, 0xda2ea9a2, 0x030ebb0e, + 0xcfa4bb90, 0x412bbc73, 0x8d81bced, 0x8744b5f4, 0x4beeb56a, + 0xc561b289, 0x09cbb217, 0xac509190, 0x60fa910e, 0xee7596ed, + 0x22df9673, 0x281a9f6a, 0xe4b09ff4, 0x6a3f9817, 0xa6959889, + 0x7fb58a25, 0xb31f8abb, 0x3d908d58, 0xf13a8dc6, 0xfbff84df, + 0x37558441, 0xb9da83a2, 0x7570833c, 0x533b85da, 0x9f918544, + 0x111e82a7, 0xddb48239, 0xd7718b20, 0x1bdb8bbe, 0x95548c5d, + 0x59fe8cc3, 0x80de9e6f, 0x4c749ef1, 0xc2fb9912, 0x0e51998c, + 0x04949095, 0xc83e900b, 0x46b197e8, 0x8a1b9776, 0x2f80b4f1, + 0xe32ab46f, 0x6da5b38c, 0xa10fb312, 0xabcaba0b, 0x6760ba95, + 0xe9efbd76, 0x2545bde8, 0xfc65af44, 0x30cfafda, 0xbe40a839, + 0x72eaa8a7, 0x782fa1be, 0xb485a120, 0x3a0aa6c3, 0xf6a0a65d, + 0xaa4de78c, 0x66e7e712, 0xe868e0f1, 0x24c2e06f, 0x2e07e976, + 0xe2ade9e8, 0x6c22ee0b, 0xa088ee95, 0x79a8fc39, 0xb502fca7, + 0x3b8dfb44, 0xf727fbda, 0xfde2f2c3, 0x3148f25d, 0xbfc7f5be, + 0x736df520, 0xd6f6d6a7, 0x1a5cd639, 0x94d3d1da, 0x5879d144, + 0x52bcd85d, 0x9e16d8c3, 0x1099df20, 0xdc33dfbe, 0x0513cd12, + 0xc9b9cd8c, 0x4736ca6f, 0x8b9ccaf1, 0x8159c3e8, 0x4df3c376, + 0xc37cc495, 0x0fd6c40b, 0x7aa64737, 0xb60c47a9, 0x3883404a, + 0xf42940d4, 0xfeec49cd, 0x32464953, 0xbcc94eb0, 0x70634e2e, + 0xa9435c82, 0x65e95c1c, 0xeb665bff, 0x27cc5b61, 0x2d095278, + 0xe1a352e6, 0x6f2c5505, 0xa386559b, 0x061d761c, 0xcab77682, + 0x44387161, 0x889271ff, 0x825778e6, 0x4efd7878, 0xc0727f9b, + 0x0cd87f05, 0xd5f86da9, 0x19526d37, 0x97dd6ad4, 0x5b776a4a, + 0x51b26353, 0x9d1863cd, 0x1397642e, 0xdf3d64b0, 0x83d02561, + 0x4f7a25ff, 0xc1f5221c, 0x0d5f2282, 0x079a2b9b, 0xcb302b05, + 0x45bf2ce6, 0x89152c78, 0x50353ed4, 0x9c9f3e4a, 0x121039a9, + 0xdeba3937, 0xd47f302e, 0x18d530b0, 0x965a3753, 0x5af037cd, + 0xff6b144a, 0x33c114d4, 0xbd4e1337, 0x71e413a9, 0x7b211ab0, + 0xb78b1a2e, 0x39041dcd, 0xf5ae1d53, 0x2c8e0fff, 0xe0240f61, + 0x6eab0882, 0xa201081c, 0xa8c40105, 0x646e019b, 0xeae10678, + 0x264b06e6}, + {0x00000000, 0xa6770bb4, 0x979f1129, 0x31e81a9d, 0xf44f2413, + 0x52382fa7, 0x63d0353a, 0xc5a73e8e, 0x33ef4e67, 0x959845d3, + 0xa4705f4e, 0x020754fa, 0xc7a06a74, 0x61d761c0, 0x503f7b5d, + 0xf64870e9, 0x67de9cce, 0xc1a9977a, 0xf0418de7, 0x56368653, + 0x9391b8dd, 0x35e6b369, 0x040ea9f4, 0xa279a240, 0x5431d2a9, + 0xf246d91d, 0xc3aec380, 0x65d9c834, 0xa07ef6ba, 0x0609fd0e, + 0x37e1e793, 0x9196ec27, 0xcfbd399c, 0x69ca3228, 0x582228b5, + 0xfe552301, 0x3bf21d8f, 0x9d85163b, 0xac6d0ca6, 0x0a1a0712, + 0xfc5277fb, 0x5a257c4f, 0x6bcd66d2, 0xcdba6d66, 0x081d53e8, + 0xae6a585c, 0x9f8242c1, 0x39f54975, 0xa863a552, 0x0e14aee6, + 0x3ffcb47b, 0x998bbfcf, 0x5c2c8141, 0xfa5b8af5, 0xcbb39068, + 0x6dc49bdc, 0x9b8ceb35, 0x3dfbe081, 0x0c13fa1c, 0xaa64f1a8, + 0x6fc3cf26, 0xc9b4c492, 0xf85cde0f, 0x5e2bd5bb, 0x440b7579, + 0xe27c7ecd, 0xd3946450, 0x75e36fe4, 0xb044516a, 0x16335ade, + 0x27db4043, 0x81ac4bf7, 0x77e43b1e, 0xd19330aa, 0xe07b2a37, + 0x460c2183, 0x83ab1f0d, 0x25dc14b9, 0x14340e24, 0xb2430590, + 0x23d5e9b7, 0x85a2e203, 0xb44af89e, 0x123df32a, 0xd79acda4, + 0x71edc610, 0x4005dc8d, 0xe672d739, 0x103aa7d0, 0xb64dac64, + 0x87a5b6f9, 0x21d2bd4d, 0xe47583c3, 0x42028877, 0x73ea92ea, + 0xd59d995e, 0x8bb64ce5, 0x2dc14751, 0x1c295dcc, 0xba5e5678, + 0x7ff968f6, 0xd98e6342, 0xe86679df, 0x4e11726b, 0xb8590282, + 0x1e2e0936, 0x2fc613ab, 0x89b1181f, 0x4c162691, 0xea612d25, + 0xdb8937b8, 0x7dfe3c0c, 0xec68d02b, 0x4a1fdb9f, 0x7bf7c102, + 0xdd80cab6, 0x1827f438, 0xbe50ff8c, 0x8fb8e511, 0x29cfeea5, + 0xdf879e4c, 0x79f095f8, 0x48188f65, 0xee6f84d1, 0x2bc8ba5f, + 0x8dbfb1eb, 0xbc57ab76, 0x1a20a0c2, 0x8816eaf2, 0x2e61e146, + 0x1f89fbdb, 0xb9fef06f, 0x7c59cee1, 0xda2ec555, 0xebc6dfc8, + 0x4db1d47c, 0xbbf9a495, 0x1d8eaf21, 0x2c66b5bc, 0x8a11be08, + 0x4fb68086, 0xe9c18b32, 0xd82991af, 0x7e5e9a1b, 0xefc8763c, + 0x49bf7d88, 0x78576715, 0xde206ca1, 0x1b87522f, 0xbdf0599b, + 0x8c184306, 0x2a6f48b2, 0xdc27385b, 0x7a5033ef, 0x4bb82972, + 0xedcf22c6, 0x28681c48, 0x8e1f17fc, 0xbff70d61, 0x198006d5, + 0x47abd36e, 0xe1dcd8da, 0xd034c247, 0x7643c9f3, 0xb3e4f77d, + 0x1593fcc9, 0x247be654, 0x820cede0, 0x74449d09, 0xd23396bd, + 0xe3db8c20, 0x45ac8794, 0x800bb91a, 0x267cb2ae, 0x1794a833, + 0xb1e3a387, 0x20754fa0, 0x86024414, 0xb7ea5e89, 0x119d553d, + 0xd43a6bb3, 0x724d6007, 0x43a57a9a, 0xe5d2712e, 0x139a01c7, + 0xb5ed0a73, 0x840510ee, 0x22721b5a, 0xe7d525d4, 0x41a22e60, + 0x704a34fd, 0xd63d3f49, 0xcc1d9f8b, 0x6a6a943f, 0x5b828ea2, + 0xfdf58516, 0x3852bb98, 0x9e25b02c, 0xafcdaab1, 0x09baa105, + 0xfff2d1ec, 0x5985da58, 0x686dc0c5, 0xce1acb71, 0x0bbdf5ff, + 0xadcafe4b, 0x9c22e4d6, 0x3a55ef62, 0xabc30345, 0x0db408f1, + 0x3c5c126c, 0x9a2b19d8, 0x5f8c2756, 0xf9fb2ce2, 0xc813367f, + 0x6e643dcb, 0x982c4d22, 0x3e5b4696, 0x0fb35c0b, 0xa9c457bf, + 0x6c636931, 0xca146285, 0xfbfc7818, 0x5d8b73ac, 0x03a0a617, + 0xa5d7ada3, 0x943fb73e, 0x3248bc8a, 0xf7ef8204, 0x519889b0, + 0x6070932d, 0xc6079899, 0x304fe870, 0x9638e3c4, 0xa7d0f959, + 0x01a7f2ed, 0xc400cc63, 0x6277c7d7, 0x539fdd4a, 0xf5e8d6fe, + 0x647e3ad9, 0xc209316d, 0xf3e12bf0, 0x55962044, 0x90311eca, + 0x3646157e, 0x07ae0fe3, 0xa1d90457, 0x579174be, 0xf1e67f0a, + 0xc00e6597, 0x66796e23, 0xa3de50ad, 0x05a95b19, 0x34414184, + 0x92364a30}, + {0x00000000, 0xcb5cd3a5, 0x4dc8a10b, 0x869472ae, 0x9b914216, + 0x50cd91b3, 0xd659e31d, 0x1d0530b8, 0xec53826d, 0x270f51c8, + 0xa19b2366, 0x6ac7f0c3, 0x77c2c07b, 0xbc9e13de, 0x3a0a6170, + 0xf156b2d5, 0x03d6029b, 0xc88ad13e, 0x4e1ea390, 0x85427035, + 0x9847408d, 0x531b9328, 0xd58fe186, 0x1ed33223, 0xef8580f6, + 0x24d95353, 0xa24d21fd, 0x6911f258, 0x7414c2e0, 0xbf481145, + 0x39dc63eb, 0xf280b04e, 0x07ac0536, 0xccf0d693, 0x4a64a43d, + 0x81387798, 0x9c3d4720, 0x57619485, 0xd1f5e62b, 0x1aa9358e, + 0xebff875b, 0x20a354fe, 0xa6372650, 0x6d6bf5f5, 0x706ec54d, + 0xbb3216e8, 0x3da66446, 0xf6fab7e3, 0x047a07ad, 0xcf26d408, + 0x49b2a6a6, 0x82ee7503, 0x9feb45bb, 0x54b7961e, 0xd223e4b0, + 0x197f3715, 0xe82985c0, 0x23755665, 0xa5e124cb, 0x6ebdf76e, + 0x73b8c7d6, 0xb8e41473, 0x3e7066dd, 0xf52cb578, 0x0f580a6c, + 0xc404d9c9, 0x4290ab67, 0x89cc78c2, 0x94c9487a, 0x5f959bdf, + 0xd901e971, 0x125d3ad4, 0xe30b8801, 0x28575ba4, 0xaec3290a, + 0x659ffaaf, 0x789aca17, 0xb3c619b2, 0x35526b1c, 0xfe0eb8b9, + 0x0c8e08f7, 0xc7d2db52, 0x4146a9fc, 0x8a1a7a59, 0x971f4ae1, + 0x5c439944, 0xdad7ebea, 0x118b384f, 0xe0dd8a9a, 0x2b81593f, + 0xad152b91, 0x6649f834, 0x7b4cc88c, 0xb0101b29, 0x36846987, + 0xfdd8ba22, 0x08f40f5a, 0xc3a8dcff, 0x453cae51, 0x8e607df4, + 0x93654d4c, 0x58399ee9, 0xdeadec47, 0x15f13fe2, 0xe4a78d37, + 0x2ffb5e92, 0xa96f2c3c, 0x6233ff99, 0x7f36cf21, 0xb46a1c84, + 0x32fe6e2a, 0xf9a2bd8f, 0x0b220dc1, 0xc07ede64, 0x46eaacca, + 0x8db67f6f, 0x90b34fd7, 0x5bef9c72, 0xdd7beedc, 0x16273d79, + 0xe7718fac, 0x2c2d5c09, 0xaab92ea7, 0x61e5fd02, 0x7ce0cdba, + 0xb7bc1e1f, 0x31286cb1, 0xfa74bf14, 0x1eb014d8, 0xd5ecc77d, + 0x5378b5d3, 0x98246676, 0x852156ce, 0x4e7d856b, 0xc8e9f7c5, + 0x03b52460, 0xf2e396b5, 0x39bf4510, 0xbf2b37be, 0x7477e41b, + 0x6972d4a3, 0xa22e0706, 0x24ba75a8, 0xefe6a60d, 0x1d661643, + 0xd63ac5e6, 0x50aeb748, 0x9bf264ed, 0x86f75455, 0x4dab87f0, + 0xcb3ff55e, 0x006326fb, 0xf135942e, 0x3a69478b, 0xbcfd3525, + 0x77a1e680, 0x6aa4d638, 0xa1f8059d, 0x276c7733, 0xec30a496, + 0x191c11ee, 0xd240c24b, 0x54d4b0e5, 0x9f886340, 0x828d53f8, + 0x49d1805d, 0xcf45f2f3, 0x04192156, 0xf54f9383, 0x3e134026, + 0xb8873288, 0x73dbe12d, 0x6eded195, 0xa5820230, 0x2316709e, + 0xe84aa33b, 0x1aca1375, 0xd196c0d0, 0x5702b27e, 0x9c5e61db, + 0x815b5163, 0x4a0782c6, 0xcc93f068, 0x07cf23cd, 0xf6999118, + 0x3dc542bd, 0xbb513013, 0x700de3b6, 0x6d08d30e, 0xa65400ab, + 0x20c07205, 0xeb9ca1a0, 0x11e81eb4, 0xdab4cd11, 0x5c20bfbf, + 0x977c6c1a, 0x8a795ca2, 0x41258f07, 0xc7b1fda9, 0x0ced2e0c, + 0xfdbb9cd9, 0x36e74f7c, 0xb0733dd2, 0x7b2fee77, 0x662adecf, + 0xad760d6a, 0x2be27fc4, 0xe0beac61, 0x123e1c2f, 0xd962cf8a, + 0x5ff6bd24, 0x94aa6e81, 0x89af5e39, 0x42f38d9c, 0xc467ff32, + 0x0f3b2c97, 0xfe6d9e42, 0x35314de7, 0xb3a53f49, 0x78f9ecec, + 0x65fcdc54, 0xaea00ff1, 0x28347d5f, 0xe368aefa, 0x16441b82, + 0xdd18c827, 0x5b8cba89, 0x90d0692c, 0x8dd55994, 0x46898a31, + 0xc01df89f, 0x0b412b3a, 0xfa1799ef, 0x314b4a4a, 0xb7df38e4, + 0x7c83eb41, 0x6186dbf9, 0xaada085c, 0x2c4e7af2, 0xe712a957, + 0x15921919, 0xdececabc, 0x585ab812, 0x93066bb7, 0x8e035b0f, + 0x455f88aa, 0xc3cbfa04, 0x089729a1, 0xf9c19b74, 0x329d48d1, + 0xb4093a7f, 0x7f55e9da, 0x6250d962, 0xa90c0ac7, 0x2f987869, + 0xe4c4abcc}, + {0x00000000, 0x3d6029b0, 0x7ac05360, 0x47a07ad0, 0xf580a6c0, + 0xc8e08f70, 0x8f40f5a0, 0xb220dc10, 0x30704bc1, 0x0d106271, + 0x4ab018a1, 0x77d03111, 0xc5f0ed01, 0xf890c4b1, 0xbf30be61, + 0x825097d1, 0x60e09782, 0x5d80be32, 0x1a20c4e2, 0x2740ed52, + 0x95603142, 0xa80018f2, 0xefa06222, 0xd2c04b92, 0x5090dc43, + 0x6df0f5f3, 0x2a508f23, 0x1730a693, 0xa5107a83, 0x98705333, + 0xdfd029e3, 0xe2b00053, 0xc1c12f04, 0xfca106b4, 0xbb017c64, + 0x866155d4, 0x344189c4, 0x0921a074, 0x4e81daa4, 0x73e1f314, + 0xf1b164c5, 0xccd14d75, 0x8b7137a5, 0xb6111e15, 0x0431c205, + 0x3951ebb5, 0x7ef19165, 0x4391b8d5, 0xa121b886, 0x9c419136, + 0xdbe1ebe6, 0xe681c256, 0x54a11e46, 0x69c137f6, 0x2e614d26, + 0x13016496, 0x9151f347, 0xac31daf7, 0xeb91a027, 0xd6f18997, + 0x64d15587, 0x59b17c37, 0x1e1106e7, 0x23712f57, 0x58f35849, + 0x659371f9, 0x22330b29, 0x1f532299, 0xad73fe89, 0x9013d739, + 0xd7b3ade9, 0xead38459, 0x68831388, 0x55e33a38, 0x124340e8, + 0x2f236958, 0x9d03b548, 0xa0639cf8, 0xe7c3e628, 0xdaa3cf98, + 0x3813cfcb, 0x0573e67b, 0x42d39cab, 0x7fb3b51b, 0xcd93690b, + 0xf0f340bb, 0xb7533a6b, 0x8a3313db, 0x0863840a, 0x3503adba, + 0x72a3d76a, 0x4fc3feda, 0xfde322ca, 0xc0830b7a, 0x872371aa, + 0xba43581a, 0x9932774d, 0xa4525efd, 0xe3f2242d, 0xde920d9d, + 0x6cb2d18d, 0x51d2f83d, 0x167282ed, 0x2b12ab5d, 0xa9423c8c, + 0x9422153c, 0xd3826fec, 0xeee2465c, 0x5cc29a4c, 0x61a2b3fc, + 0x2602c92c, 0x1b62e09c, 0xf9d2e0cf, 0xc4b2c97f, 0x8312b3af, + 0xbe729a1f, 0x0c52460f, 0x31326fbf, 0x7692156f, 0x4bf23cdf, + 0xc9a2ab0e, 0xf4c282be, 0xb362f86e, 0x8e02d1de, 0x3c220dce, + 0x0142247e, 0x46e25eae, 0x7b82771e, 0xb1e6b092, 0x8c869922, + 0xcb26e3f2, 0xf646ca42, 0x44661652, 0x79063fe2, 0x3ea64532, + 0x03c66c82, 0x8196fb53, 0xbcf6d2e3, 0xfb56a833, 0xc6368183, + 0x74165d93, 0x49767423, 0x0ed60ef3, 0x33b62743, 0xd1062710, + 0xec660ea0, 0xabc67470, 0x96a65dc0, 0x248681d0, 0x19e6a860, + 0x5e46d2b0, 0x6326fb00, 0xe1766cd1, 0xdc164561, 0x9bb63fb1, + 0xa6d61601, 0x14f6ca11, 0x2996e3a1, 0x6e369971, 0x5356b0c1, + 0x70279f96, 0x4d47b626, 0x0ae7ccf6, 0x3787e546, 0x85a73956, + 0xb8c710e6, 0xff676a36, 0xc2074386, 0x4057d457, 0x7d37fde7, + 0x3a978737, 0x07f7ae87, 0xb5d77297, 0x88b75b27, 0xcf1721f7, + 0xf2770847, 0x10c70814, 0x2da721a4, 0x6a075b74, 0x576772c4, + 0xe547aed4, 0xd8278764, 0x9f87fdb4, 0xa2e7d404, 0x20b743d5, + 0x1dd76a65, 0x5a7710b5, 0x67173905, 0xd537e515, 0xe857cca5, + 0xaff7b675, 0x92979fc5, 0xe915e8db, 0xd475c16b, 0x93d5bbbb, + 0xaeb5920b, 0x1c954e1b, 0x21f567ab, 0x66551d7b, 0x5b3534cb, + 0xd965a31a, 0xe4058aaa, 0xa3a5f07a, 0x9ec5d9ca, 0x2ce505da, + 0x11852c6a, 0x562556ba, 0x6b457f0a, 0x89f57f59, 0xb49556e9, + 0xf3352c39, 0xce550589, 0x7c75d999, 0x4115f029, 0x06b58af9, + 0x3bd5a349, 0xb9853498, 0x84e51d28, 0xc34567f8, 0xfe254e48, + 0x4c059258, 0x7165bbe8, 0x36c5c138, 0x0ba5e888, 0x28d4c7df, + 0x15b4ee6f, 0x521494bf, 0x6f74bd0f, 0xdd54611f, 0xe03448af, + 0xa794327f, 0x9af41bcf, 0x18a48c1e, 0x25c4a5ae, 0x6264df7e, + 0x5f04f6ce, 0xed242ade, 0xd044036e, 0x97e479be, 0xaa84500e, + 0x4834505d, 0x755479ed, 0x32f4033d, 0x0f942a8d, 0xbdb4f69d, + 0x80d4df2d, 0xc774a5fd, 0xfa148c4d, 0x78441b9c, 0x4524322c, + 0x028448fc, 0x3fe4614c, 0x8dc4bd5c, 0xb0a494ec, 0xf704ee3c, + 0xca64c78c}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x00000000, 0xb029603d, 0x6053c07a, 0xd07aa047, 0xc0a680f5, + 0x708fe0c8, 0xa0f5408f, 0x10dc20b2, 0xc14b7030, 0x7162100d, + 0xa118b04a, 0x1131d077, 0x01edf0c5, 0xb1c490f8, 0x61be30bf, + 0xd1975082, 0x8297e060, 0x32be805d, 0xe2c4201a, 0x52ed4027, + 0x42316095, 0xf21800a8, 0x2262a0ef, 0x924bc0d2, 0x43dc9050, + 0xf3f5f06d, 0x238f502a, 0x93a63017, 0x837a10a5, 0x33537098, + 0xe329d0df, 0x5300b0e2, 0x042fc1c1, 0xb406a1fc, 0x647c01bb, + 0xd4556186, 0xc4894134, 0x74a02109, 0xa4da814e, 0x14f3e173, + 0xc564b1f1, 0x754dd1cc, 0xa537718b, 0x151e11b6, 0x05c23104, + 0xb5eb5139, 0x6591f17e, 0xd5b89143, 0x86b821a1, 0x3691419c, + 0xe6ebe1db, 0x56c281e6, 0x461ea154, 0xf637c169, 0x264d612e, + 0x96640113, 0x47f35191, 0xf7da31ac, 0x27a091eb, 0x9789f1d6, + 0x8755d164, 0x377cb159, 0xe706111e, 0x572f7123, 0x4958f358, + 0xf9719365, 0x290b3322, 0x9922531f, 0x89fe73ad, 0x39d71390, + 0xe9adb3d7, 0x5984d3ea, 0x88138368, 0x383ae355, 0xe8404312, + 0x5869232f, 0x48b5039d, 0xf89c63a0, 0x28e6c3e7, 0x98cfa3da, + 0xcbcf1338, 0x7be67305, 0xab9cd342, 0x1bb5b37f, 0x0b6993cd, + 0xbb40f3f0, 0x6b3a53b7, 0xdb13338a, 0x0a846308, 0xbaad0335, + 0x6ad7a372, 0xdafec34f, 0xca22e3fd, 0x7a0b83c0, 0xaa712387, + 0x1a5843ba, 0x4d773299, 0xfd5e52a4, 0x2d24f2e3, 0x9d0d92de, + 0x8dd1b26c, 0x3df8d251, 0xed827216, 0x5dab122b, 0x8c3c42a9, + 0x3c152294, 0xec6f82d3, 0x5c46e2ee, 0x4c9ac25c, 0xfcb3a261, + 0x2cc90226, 0x9ce0621b, 0xcfe0d2f9, 0x7fc9b2c4, 0xafb31283, + 0x1f9a72be, 0x0f46520c, 0xbf6f3231, 0x6f159276, 0xdf3cf24b, + 0x0eaba2c9, 0xbe82c2f4, 0x6ef862b3, 0xded1028e, 0xce0d223c, + 0x7e244201, 0xae5ee246, 0x1e77827b, 0x92b0e6b1, 0x2299868c, + 0xf2e326cb, 0x42ca46f6, 0x52166644, 0xe23f0679, 0x3245a63e, + 0x826cc603, 0x53fb9681, 0xe3d2f6bc, 0x33a856fb, 0x838136c6, + 0x935d1674, 0x23747649, 0xf30ed60e, 0x4327b633, 0x102706d1, + 0xa00e66ec, 0x7074c6ab, 0xc05da696, 0xd0818624, 0x60a8e619, + 0xb0d2465e, 0x00fb2663, 0xd16c76e1, 0x614516dc, 0xb13fb69b, + 0x0116d6a6, 0x11caf614, 0xa1e39629, 0x7199366e, 0xc1b05653, + 0x969f2770, 0x26b6474d, 0xf6cce70a, 0x46e58737, 0x5639a785, + 0xe610c7b8, 0x366a67ff, 0x864307c2, 0x57d45740, 0xe7fd377d, + 0x3787973a, 0x87aef707, 0x9772d7b5, 0x275bb788, 0xf72117cf, + 0x470877f2, 0x1408c710, 0xa421a72d, 0x745b076a, 0xc4726757, + 0xd4ae47e5, 0x648727d8, 0xb4fd879f, 0x04d4e7a2, 0xd543b720, + 0x656ad71d, 0xb510775a, 0x05391767, 0x15e537d5, 0xa5cc57e8, + 0x75b6f7af, 0xc59f9792, 0xdbe815e9, 0x6bc175d4, 0xbbbbd593, + 0x0b92b5ae, 0x1b4e951c, 0xab67f521, 0x7b1d5566, 0xcb34355b, + 0x1aa365d9, 0xaa8a05e4, 0x7af0a5a3, 0xcad9c59e, 0xda05e52c, + 0x6a2c8511, 0xba562556, 0x0a7f456b, 0x597ff589, 0xe95695b4, + 0x392c35f3, 0x890555ce, 0x99d9757c, 0x29f01541, 0xf98ab506, + 0x49a3d53b, 0x983485b9, 0x281de584, 0xf86745c3, 0x484e25fe, + 0x5892054c, 0xe8bb6571, 0x38c1c536, 0x88e8a50b, 0xdfc7d428, + 0x6feeb415, 0xbf941452, 0x0fbd746f, 0x1f6154dd, 0xaf4834e0, + 0x7f3294a7, 0xcf1bf49a, 0x1e8ca418, 0xaea5c425, 0x7edf6462, + 0xcef6045f, 0xde2a24ed, 0x6e0344d0, 0xbe79e497, 0x0e5084aa, + 0x5d503448, 0xed795475, 0x3d03f432, 0x8d2a940f, 0x9df6b4bd, + 0x2ddfd480, 0xfda574c7, 0x4d8c14fa, 0x9c1b4478, 0x2c322445, + 0xfc488402, 0x4c61e43f, 0x5cbdc48d, 0xec94a4b0, 0x3cee04f7, + 0x8cc764ca}, + {0x00000000, 0xa5d35ccb, 0x0ba1c84d, 0xae729486, 0x1642919b, + 0xb391cd50, 0x1de359d6, 0xb830051d, 0x6d8253ec, 0xc8510f27, + 0x66239ba1, 0xc3f0c76a, 0x7bc0c277, 0xde139ebc, 0x70610a3a, + 0xd5b256f1, 0x9b02d603, 0x3ed18ac8, 0x90a31e4e, 0x35704285, + 0x8d404798, 0x28931b53, 0x86e18fd5, 0x2332d31e, 0xf68085ef, + 0x5353d924, 0xfd214da2, 0x58f21169, 0xe0c21474, 0x451148bf, + 0xeb63dc39, 0x4eb080f2, 0x3605ac07, 0x93d6f0cc, 0x3da4644a, + 0x98773881, 0x20473d9c, 0x85946157, 0x2be6f5d1, 0x8e35a91a, + 0x5b87ffeb, 0xfe54a320, 0x502637a6, 0xf5f56b6d, 0x4dc56e70, + 0xe81632bb, 0x4664a63d, 0xe3b7faf6, 0xad077a04, 0x08d426cf, + 0xa6a6b249, 0x0375ee82, 0xbb45eb9f, 0x1e96b754, 0xb0e423d2, + 0x15377f19, 0xc08529e8, 0x65567523, 0xcb24e1a5, 0x6ef7bd6e, + 0xd6c7b873, 0x7314e4b8, 0xdd66703e, 0x78b52cf5, 0x6c0a580f, + 0xc9d904c4, 0x67ab9042, 0xc278cc89, 0x7a48c994, 0xdf9b955f, + 0x71e901d9, 0xd43a5d12, 0x01880be3, 0xa45b5728, 0x0a29c3ae, + 0xaffa9f65, 0x17ca9a78, 0xb219c6b3, 0x1c6b5235, 0xb9b80efe, + 0xf7088e0c, 0x52dbd2c7, 0xfca94641, 0x597a1a8a, 0xe14a1f97, + 0x4499435c, 0xeaebd7da, 0x4f388b11, 0x9a8adde0, 0x3f59812b, + 0x912b15ad, 0x34f84966, 0x8cc84c7b, 0x291b10b0, 0x87698436, + 0x22bad8fd, 0x5a0ff408, 0xffdca8c3, 0x51ae3c45, 0xf47d608e, + 0x4c4d6593, 0xe99e3958, 0x47ecadde, 0xe23ff115, 0x378da7e4, + 0x925efb2f, 0x3c2c6fa9, 0x99ff3362, 0x21cf367f, 0x841c6ab4, + 0x2a6efe32, 0x8fbda2f9, 0xc10d220b, 0x64de7ec0, 0xcaacea46, + 0x6f7fb68d, 0xd74fb390, 0x729cef5b, 0xdcee7bdd, 0x793d2716, + 0xac8f71e7, 0x095c2d2c, 0xa72eb9aa, 0x02fde561, 0xbacde07c, + 0x1f1ebcb7, 0xb16c2831, 0x14bf74fa, 0xd814b01e, 0x7dc7ecd5, + 0xd3b57853, 0x76662498, 0xce562185, 0x6b857d4e, 0xc5f7e9c8, + 0x6024b503, 0xb596e3f2, 0x1045bf39, 0xbe372bbf, 0x1be47774, + 0xa3d47269, 0x06072ea2, 0xa875ba24, 0x0da6e6ef, 0x4316661d, + 0xe6c53ad6, 0x48b7ae50, 0xed64f29b, 0x5554f786, 0xf087ab4d, + 0x5ef53fcb, 0xfb266300, 0x2e9435f1, 0x8b47693a, 0x2535fdbc, + 0x80e6a177, 0x38d6a46a, 0x9d05f8a1, 0x33776c27, 0x96a430ec, + 0xee111c19, 0x4bc240d2, 0xe5b0d454, 0x4063889f, 0xf8538d82, + 0x5d80d149, 0xf3f245cf, 0x56211904, 0x83934ff5, 0x2640133e, + 0x883287b8, 0x2de1db73, 0x95d1de6e, 0x300282a5, 0x9e701623, + 0x3ba34ae8, 0x7513ca1a, 0xd0c096d1, 0x7eb20257, 0xdb615e9c, + 0x63515b81, 0xc682074a, 0x68f093cc, 0xcd23cf07, 0x189199f6, + 0xbd42c53d, 0x133051bb, 0xb6e30d70, 0x0ed3086d, 0xab0054a6, + 0x0572c020, 0xa0a19ceb, 0xb41ee811, 0x11cdb4da, 0xbfbf205c, + 0x1a6c7c97, 0xa25c798a, 0x078f2541, 0xa9fdb1c7, 0x0c2eed0c, + 0xd99cbbfd, 0x7c4fe736, 0xd23d73b0, 0x77ee2f7b, 0xcfde2a66, + 0x6a0d76ad, 0xc47fe22b, 0x61acbee0, 0x2f1c3e12, 0x8acf62d9, + 0x24bdf65f, 0x816eaa94, 0x395eaf89, 0x9c8df342, 0x32ff67c4, + 0x972c3b0f, 0x429e6dfe, 0xe74d3135, 0x493fa5b3, 0xececf978, + 0x54dcfc65, 0xf10fa0ae, 0x5f7d3428, 0xfaae68e3, 0x821b4416, + 0x27c818dd, 0x89ba8c5b, 0x2c69d090, 0x9459d58d, 0x318a8946, + 0x9ff81dc0, 0x3a2b410b, 0xef9917fa, 0x4a4a4b31, 0xe438dfb7, + 0x41eb837c, 0xf9db8661, 0x5c08daaa, 0xf27a4e2c, 0x57a912e7, + 0x19199215, 0xbccacede, 0x12b85a58, 0xb76b0693, 0x0f5b038e, + 0xaa885f45, 0x04facbc3, 0xa1299708, 0x749bc1f9, 0xd1489d32, + 0x7f3a09b4, 0xdae9557f, 0x62d95062, 0xc70a0ca9, 0x6978982f, + 0xccabc4e4}, + {0x00000000, 0xb40b77a6, 0x29119f97, 0x9d1ae831, 0x13244ff4, + 0xa72f3852, 0x3a35d063, 0x8e3ea7c5, 0x674eef33, 0xd3459895, + 0x4e5f70a4, 0xfa540702, 0x746aa0c7, 0xc061d761, 0x5d7b3f50, + 0xe97048f6, 0xce9cde67, 0x7a97a9c1, 0xe78d41f0, 0x53863656, + 0xddb89193, 0x69b3e635, 0xf4a90e04, 0x40a279a2, 0xa9d23154, + 0x1dd946f2, 0x80c3aec3, 0x34c8d965, 0xbaf67ea0, 0x0efd0906, + 0x93e7e137, 0x27ec9691, 0x9c39bdcf, 0x2832ca69, 0xb5282258, + 0x012355fe, 0x8f1df23b, 0x3b16859d, 0xa60c6dac, 0x12071a0a, + 0xfb7752fc, 0x4f7c255a, 0xd266cd6b, 0x666dbacd, 0xe8531d08, + 0x5c586aae, 0xc142829f, 0x7549f539, 0x52a563a8, 0xe6ae140e, + 0x7bb4fc3f, 0xcfbf8b99, 0x41812c5c, 0xf58a5bfa, 0x6890b3cb, + 0xdc9bc46d, 0x35eb8c9b, 0x81e0fb3d, 0x1cfa130c, 0xa8f164aa, + 0x26cfc36f, 0x92c4b4c9, 0x0fde5cf8, 0xbbd52b5e, 0x79750b44, + 0xcd7e7ce2, 0x506494d3, 0xe46fe375, 0x6a5144b0, 0xde5a3316, + 0x4340db27, 0xf74bac81, 0x1e3be477, 0xaa3093d1, 0x372a7be0, + 0x83210c46, 0x0d1fab83, 0xb914dc25, 0x240e3414, 0x900543b2, + 0xb7e9d523, 0x03e2a285, 0x9ef84ab4, 0x2af33d12, 0xa4cd9ad7, + 0x10c6ed71, 0x8ddc0540, 0x39d772e6, 0xd0a73a10, 0x64ac4db6, + 0xf9b6a587, 0x4dbdd221, 0xc38375e4, 0x77880242, 0xea92ea73, + 0x5e999dd5, 0xe54cb68b, 0x5147c12d, 0xcc5d291c, 0x78565eba, + 0xf668f97f, 0x42638ed9, 0xdf7966e8, 0x6b72114e, 0x820259b8, + 0x36092e1e, 0xab13c62f, 0x1f18b189, 0x9126164c, 0x252d61ea, + 0xb83789db, 0x0c3cfe7d, 0x2bd068ec, 0x9fdb1f4a, 0x02c1f77b, + 0xb6ca80dd, 0x38f42718, 0x8cff50be, 0x11e5b88f, 0xa5eecf29, + 0x4c9e87df, 0xf895f079, 0x658f1848, 0xd1846fee, 0x5fbac82b, + 0xebb1bf8d, 0x76ab57bc, 0xc2a0201a, 0xf2ea1688, 0x46e1612e, + 0xdbfb891f, 0x6ff0feb9, 0xe1ce597c, 0x55c52eda, 0xc8dfc6eb, + 0x7cd4b14d, 0x95a4f9bb, 0x21af8e1d, 0xbcb5662c, 0x08be118a, + 0x8680b64f, 0x328bc1e9, 0xaf9129d8, 0x1b9a5e7e, 0x3c76c8ef, + 0x887dbf49, 0x15675778, 0xa16c20de, 0x2f52871b, 0x9b59f0bd, + 0x0643188c, 0xb2486f2a, 0x5b3827dc, 0xef33507a, 0x7229b84b, + 0xc622cfed, 0x481c6828, 0xfc171f8e, 0x610df7bf, 0xd5068019, + 0x6ed3ab47, 0xdad8dce1, 0x47c234d0, 0xf3c94376, 0x7df7e4b3, + 0xc9fc9315, 0x54e67b24, 0xe0ed0c82, 0x099d4474, 0xbd9633d2, + 0x208cdbe3, 0x9487ac45, 0x1ab90b80, 0xaeb27c26, 0x33a89417, + 0x87a3e3b1, 0xa04f7520, 0x14440286, 0x895eeab7, 0x3d559d11, + 0xb36b3ad4, 0x07604d72, 0x9a7aa543, 0x2e71d2e5, 0xc7019a13, + 0x730aedb5, 0xee100584, 0x5a1b7222, 0xd425d5e7, 0x602ea241, + 0xfd344a70, 0x493f3dd6, 0x8b9f1dcc, 0x3f946a6a, 0xa28e825b, + 0x1685f5fd, 0x98bb5238, 0x2cb0259e, 0xb1aacdaf, 0x05a1ba09, + 0xecd1f2ff, 0x58da8559, 0xc5c06d68, 0x71cb1ace, 0xfff5bd0b, + 0x4bfecaad, 0xd6e4229c, 0x62ef553a, 0x4503c3ab, 0xf108b40d, + 0x6c125c3c, 0xd8192b9a, 0x56278c5f, 0xe22cfbf9, 0x7f3613c8, + 0xcb3d646e, 0x224d2c98, 0x96465b3e, 0x0b5cb30f, 0xbf57c4a9, + 0x3169636c, 0x856214ca, 0x1878fcfb, 0xac738b5d, 0x17a6a003, + 0xa3add7a5, 0x3eb73f94, 0x8abc4832, 0x0482eff7, 0xb0899851, + 0x2d937060, 0x999807c6, 0x70e84f30, 0xc4e33896, 0x59f9d0a7, + 0xedf2a701, 0x63cc00c4, 0xd7c77762, 0x4add9f53, 0xfed6e8f5, + 0xd93a7e64, 0x6d3109c2, 0xf02be1f3, 0x44209655, 0xca1e3190, + 0x7e154636, 0xe30fae07, 0x5704d9a1, 0xbe749157, 0x0a7fe6f1, + 0x97650ec0, 0x236e7966, 0xad50dea3, 0x195ba905, 0x84414134, + 0x304a3692}, + {0x00000000, 0x9e00aacc, 0x7d072542, 0xe3078f8e, 0xfa0e4a84, + 0x640ee048, 0x87096fc6, 0x1909c50a, 0xb51be5d3, 0x2b1b4f1f, + 0xc81cc091, 0x561c6a5d, 0x4f15af57, 0xd115059b, 0x32128a15, + 0xac1220d9, 0x2b31bb7c, 0xb53111b0, 0x56369e3e, 0xc83634f2, + 0xd13ff1f8, 0x4f3f5b34, 0xac38d4ba, 0x32387e76, 0x9e2a5eaf, + 0x002af463, 0xe32d7bed, 0x7d2dd121, 0x6424142b, 0xfa24bee7, + 0x19233169, 0x87239ba5, 0x566276f9, 0xc862dc35, 0x2b6553bb, + 0xb565f977, 0xac6c3c7d, 0x326c96b1, 0xd16b193f, 0x4f6bb3f3, + 0xe379932a, 0x7d7939e6, 0x9e7eb668, 0x007e1ca4, 0x1977d9ae, + 0x87777362, 0x6470fcec, 0xfa705620, 0x7d53cd85, 0xe3536749, + 0x0054e8c7, 0x9e54420b, 0x875d8701, 0x195d2dcd, 0xfa5aa243, + 0x645a088f, 0xc8482856, 0x5648829a, 0xb54f0d14, 0x2b4fa7d8, + 0x324662d2, 0xac46c81e, 0x4f414790, 0xd141ed5c, 0xedc29d29, + 0x73c237e5, 0x90c5b86b, 0x0ec512a7, 0x17ccd7ad, 0x89cc7d61, + 0x6acbf2ef, 0xf4cb5823, 0x58d978fa, 0xc6d9d236, 0x25de5db8, + 0xbbdef774, 0xa2d7327e, 0x3cd798b2, 0xdfd0173c, 0x41d0bdf0, + 0xc6f32655, 0x58f38c99, 0xbbf40317, 0x25f4a9db, 0x3cfd6cd1, + 0xa2fdc61d, 0x41fa4993, 0xdffae35f, 0x73e8c386, 0xede8694a, + 0x0eefe6c4, 0x90ef4c08, 0x89e68902, 0x17e623ce, 0xf4e1ac40, + 0x6ae1068c, 0xbba0ebd0, 0x25a0411c, 0xc6a7ce92, 0x58a7645e, + 0x41aea154, 0xdfae0b98, 0x3ca98416, 0xa2a92eda, 0x0ebb0e03, + 0x90bba4cf, 0x73bc2b41, 0xedbc818d, 0xf4b54487, 0x6ab5ee4b, + 0x89b261c5, 0x17b2cb09, 0x909150ac, 0x0e91fa60, 0xed9675ee, + 0x7396df22, 0x6a9f1a28, 0xf49fb0e4, 0x17983f6a, 0x899895a6, + 0x258ab57f, 0xbb8a1fb3, 0x588d903d, 0xc68d3af1, 0xdf84fffb, + 0x41845537, 0xa283dab9, 0x3c837075, 0xda853b53, 0x4485919f, + 0xa7821e11, 0x3982b4dd, 0x208b71d7, 0xbe8bdb1b, 0x5d8c5495, + 0xc38cfe59, 0x6f9ede80, 0xf19e744c, 0x1299fbc2, 0x8c99510e, + 0x95909404, 0x0b903ec8, 0xe897b146, 0x76971b8a, 0xf1b4802f, + 0x6fb42ae3, 0x8cb3a56d, 0x12b30fa1, 0x0bbacaab, 0x95ba6067, + 0x76bdefe9, 0xe8bd4525, 0x44af65fc, 0xdaafcf30, 0x39a840be, + 0xa7a8ea72, 0xbea12f78, 0x20a185b4, 0xc3a60a3a, 0x5da6a0f6, + 0x8ce74daa, 0x12e7e766, 0xf1e068e8, 0x6fe0c224, 0x76e9072e, + 0xe8e9ade2, 0x0bee226c, 0x95ee88a0, 0x39fca879, 0xa7fc02b5, + 0x44fb8d3b, 0xdafb27f7, 0xc3f2e2fd, 0x5df24831, 0xbef5c7bf, + 0x20f56d73, 0xa7d6f6d6, 0x39d65c1a, 0xdad1d394, 0x44d17958, + 0x5dd8bc52, 0xc3d8169e, 0x20df9910, 0xbedf33dc, 0x12cd1305, + 0x8ccdb9c9, 0x6fca3647, 0xf1ca9c8b, 0xe8c35981, 0x76c3f34d, + 0x95c47cc3, 0x0bc4d60f, 0x3747a67a, 0xa9470cb6, 0x4a408338, + 0xd44029f4, 0xcd49ecfe, 0x53494632, 0xb04ec9bc, 0x2e4e6370, + 0x825c43a9, 0x1c5ce965, 0xff5b66eb, 0x615bcc27, 0x7852092d, + 0xe652a3e1, 0x05552c6f, 0x9b5586a3, 0x1c761d06, 0x8276b7ca, + 0x61713844, 0xff719288, 0xe6785782, 0x7878fd4e, 0x9b7f72c0, + 0x057fd80c, 0xa96df8d5, 0x376d5219, 0xd46add97, 0x4a6a775b, + 0x5363b251, 0xcd63189d, 0x2e649713, 0xb0643ddf, 0x6125d083, + 0xff257a4f, 0x1c22f5c1, 0x82225f0d, 0x9b2b9a07, 0x052b30cb, + 0xe62cbf45, 0x782c1589, 0xd43e3550, 0x4a3e9f9c, 0xa9391012, + 0x3739bade, 0x2e307fd4, 0xb030d518, 0x53375a96, 0xcd37f05a, + 0x4a146bff, 0xd414c133, 0x37134ebd, 0xa913e471, 0xb01a217b, + 0x2e1a8bb7, 0xcd1d0439, 0x531daef5, 0xff0f8e2c, 0x610f24e0, + 0x8208ab6e, 0x1c0801a2, 0x0501c4a8, 0x9b016e64, 0x7806e1ea, + 0xe6064b26}}; + +#endif + +#endif + +#if N == 3 + +#if W == 8 + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0x81256527, 0xd93bcc0f, 0x581ea928, 0x69069e5f, + 0xe823fb78, 0xb03d5250, 0x31183777, 0xd20d3cbe, 0x53285999, + 0x0b36f0b1, 0x8a139596, 0xbb0ba2e1, 0x3a2ec7c6, 0x62306eee, + 0xe3150bc9, 0x7f6b7f3d, 0xfe4e1a1a, 0xa650b332, 0x2775d615, + 0x166de162, 0x97488445, 0xcf562d6d, 0x4e73484a, 0xad664383, + 0x2c4326a4, 0x745d8f8c, 0xf578eaab, 0xc460dddc, 0x4545b8fb, + 0x1d5b11d3, 0x9c7e74f4, 0xfed6fe7a, 0x7ff39b5d, 0x27ed3275, + 0xa6c85752, 0x97d06025, 0x16f50502, 0x4eebac2a, 0xcfcec90d, + 0x2cdbc2c4, 0xadfea7e3, 0xf5e00ecb, 0x74c56bec, 0x45dd5c9b, + 0xc4f839bc, 0x9ce69094, 0x1dc3f5b3, 0x81bd8147, 0x0098e460, + 0x58864d48, 0xd9a3286f, 0xe8bb1f18, 0x699e7a3f, 0x3180d317, + 0xb0a5b630, 0x53b0bdf9, 0xd295d8de, 0x8a8b71f6, 0x0bae14d1, + 0x3ab623a6, 0xbb934681, 0xe38defa9, 0x62a88a8e, 0x26dcfab5, + 0xa7f99f92, 0xffe736ba, 0x7ec2539d, 0x4fda64ea, 0xceff01cd, + 0x96e1a8e5, 0x17c4cdc2, 0xf4d1c60b, 0x75f4a32c, 0x2dea0a04, + 0xaccf6f23, 0x9dd75854, 0x1cf23d73, 0x44ec945b, 0xc5c9f17c, + 0x59b78588, 0xd892e0af, 0x808c4987, 0x01a92ca0, 0x30b11bd7, + 0xb1947ef0, 0xe98ad7d8, 0x68afb2ff, 0x8bbab936, 0x0a9fdc11, + 0x52817539, 0xd3a4101e, 0xe2bc2769, 0x6399424e, 0x3b87eb66, + 0xbaa28e41, 0xd80a04cf, 0x592f61e8, 0x0131c8c0, 0x8014ade7, + 0xb10c9a90, 0x3029ffb7, 0x6837569f, 0xe91233b8, 0x0a073871, + 0x8b225d56, 0xd33cf47e, 0x52199159, 0x6301a62e, 0xe224c309, + 0xba3a6a21, 0x3b1f0f06, 0xa7617bf2, 0x26441ed5, 0x7e5ab7fd, + 0xff7fd2da, 0xce67e5ad, 0x4f42808a, 0x175c29a2, 0x96794c85, + 0x756c474c, 0xf449226b, 0xac578b43, 0x2d72ee64, 0x1c6ad913, + 0x9d4fbc34, 0xc551151c, 0x4474703b, 0x4db9f56a, 0xcc9c904d, + 0x94823965, 0x15a75c42, 0x24bf6b35, 0xa59a0e12, 0xfd84a73a, + 0x7ca1c21d, 0x9fb4c9d4, 0x1e91acf3, 0x468f05db, 0xc7aa60fc, + 0xf6b2578b, 0x779732ac, 0x2f899b84, 0xaeacfea3, 0x32d28a57, + 0xb3f7ef70, 0xebe94658, 0x6acc237f, 0x5bd41408, 0xdaf1712f, + 0x82efd807, 0x03cabd20, 0xe0dfb6e9, 0x61fad3ce, 0x39e47ae6, + 0xb8c11fc1, 0x89d928b6, 0x08fc4d91, 0x50e2e4b9, 0xd1c7819e, + 0xb36f0b10, 0x324a6e37, 0x6a54c71f, 0xeb71a238, 0xda69954f, + 0x5b4cf068, 0x03525940, 0x82773c67, 0x616237ae, 0xe0475289, + 0xb859fba1, 0x397c9e86, 0x0864a9f1, 0x8941ccd6, 0xd15f65fe, + 0x507a00d9, 0xcc04742d, 0x4d21110a, 0x153fb822, 0x941add05, + 0xa502ea72, 0x24278f55, 0x7c39267d, 0xfd1c435a, 0x1e094893, + 0x9f2c2db4, 0xc732849c, 0x4617e1bb, 0x770fd6cc, 0xf62ab3eb, + 0xae341ac3, 0x2f117fe4, 0x6b650fdf, 0xea406af8, 0xb25ec3d0, + 0x337ba6f7, 0x02639180, 0x8346f4a7, 0xdb585d8f, 0x5a7d38a8, + 0xb9683361, 0x384d5646, 0x6053ff6e, 0xe1769a49, 0xd06ead3e, + 0x514bc819, 0x09556131, 0x88700416, 0x140e70e2, 0x952b15c5, + 0xcd35bced, 0x4c10d9ca, 0x7d08eebd, 0xfc2d8b9a, 0xa43322b2, + 0x25164795, 0xc6034c5c, 0x4726297b, 0x1f388053, 0x9e1de574, + 0xaf05d203, 0x2e20b724, 0x763e1e0c, 0xf71b7b2b, 0x95b3f1a5, + 0x14969482, 0x4c883daa, 0xcdad588d, 0xfcb56ffa, 0x7d900add, + 0x258ea3f5, 0xa4abc6d2, 0x47becd1b, 0xc69ba83c, 0x9e850114, + 0x1fa06433, 0x2eb85344, 0xaf9d3663, 0xf7839f4b, 0x76a6fa6c, + 0xead88e98, 0x6bfdebbf, 0x33e34297, 0xb2c627b0, 0x83de10c7, + 0x02fb75e0, 0x5ae5dcc8, 0xdbc0b9ef, 0x38d5b226, 0xb9f0d701, + 0xe1ee7e29, 0x60cb1b0e, 0x51d32c79, 0xd0f6495e, 0x88e8e076, + 0x09cd8551}, + {0x00000000, 0x9b73ead4, 0xed96d3e9, 0x76e5393d, 0x005ca193, + 0x9b2f4b47, 0xedca727a, 0x76b998ae, 0x00b94326, 0x9bcaa9f2, + 0xed2f90cf, 0x765c7a1b, 0x00e5e2b5, 0x9b960861, 0xed73315c, + 0x7600db88, 0x0172864c, 0x9a016c98, 0xece455a5, 0x7797bf71, + 0x012e27df, 0x9a5dcd0b, 0xecb8f436, 0x77cb1ee2, 0x01cbc56a, + 0x9ab82fbe, 0xec5d1683, 0x772efc57, 0x019764f9, 0x9ae48e2d, + 0xec01b710, 0x77725dc4, 0x02e50c98, 0x9996e64c, 0xef73df71, + 0x740035a5, 0x02b9ad0b, 0x99ca47df, 0xef2f7ee2, 0x745c9436, + 0x025c4fbe, 0x992fa56a, 0xefca9c57, 0x74b97683, 0x0200ee2d, + 0x997304f9, 0xef963dc4, 0x74e5d710, 0x03978ad4, 0x98e46000, + 0xee01593d, 0x7572b3e9, 0x03cb2b47, 0x98b8c193, 0xee5df8ae, + 0x752e127a, 0x032ec9f2, 0x985d2326, 0xeeb81a1b, 0x75cbf0cf, + 0x03726861, 0x980182b5, 0xeee4bb88, 0x7597515c, 0x05ca1930, + 0x9eb9f3e4, 0xe85ccad9, 0x732f200d, 0x0596b8a3, 0x9ee55277, + 0xe8006b4a, 0x7373819e, 0x05735a16, 0x9e00b0c2, 0xe8e589ff, + 0x7396632b, 0x052ffb85, 0x9e5c1151, 0xe8b9286c, 0x73cac2b8, + 0x04b89f7c, 0x9fcb75a8, 0xe92e4c95, 0x725da641, 0x04e43eef, + 0x9f97d43b, 0xe972ed06, 0x720107d2, 0x0401dc5a, 0x9f72368e, + 0xe9970fb3, 0x72e4e567, 0x045d7dc9, 0x9f2e971d, 0xe9cbae20, + 0x72b844f4, 0x072f15a8, 0x9c5cff7c, 0xeab9c641, 0x71ca2c95, + 0x0773b43b, 0x9c005eef, 0xeae567d2, 0x71968d06, 0x0796568e, + 0x9ce5bc5a, 0xea008567, 0x71736fb3, 0x07caf71d, 0x9cb91dc9, + 0xea5c24f4, 0x712fce20, 0x065d93e4, 0x9d2e7930, 0xebcb400d, + 0x70b8aad9, 0x06013277, 0x9d72d8a3, 0xeb97e19e, 0x70e40b4a, + 0x06e4d0c2, 0x9d973a16, 0xeb72032b, 0x7001e9ff, 0x06b87151, + 0x9dcb9b85, 0xeb2ea2b8, 0x705d486c, 0x0b943260, 0x90e7d8b4, + 0xe602e189, 0x7d710b5d, 0x0bc893f3, 0x90bb7927, 0xe65e401a, + 0x7d2daace, 0x0b2d7146, 0x905e9b92, 0xe6bba2af, 0x7dc8487b, + 0x0b71d0d5, 0x90023a01, 0xe6e7033c, 0x7d94e9e8, 0x0ae6b42c, + 0x91955ef8, 0xe77067c5, 0x7c038d11, 0x0aba15bf, 0x91c9ff6b, + 0xe72cc656, 0x7c5f2c82, 0x0a5ff70a, 0x912c1dde, 0xe7c924e3, + 0x7cbace37, 0x0a035699, 0x9170bc4d, 0xe7958570, 0x7ce66fa4, + 0x09713ef8, 0x9202d42c, 0xe4e7ed11, 0x7f9407c5, 0x092d9f6b, + 0x925e75bf, 0xe4bb4c82, 0x7fc8a656, 0x09c87dde, 0x92bb970a, + 0xe45eae37, 0x7f2d44e3, 0x0994dc4d, 0x92e73699, 0xe4020fa4, + 0x7f71e570, 0x0803b8b4, 0x93705260, 0xe5956b5d, 0x7ee68189, + 0x085f1927, 0x932cf3f3, 0xe5c9cace, 0x7eba201a, 0x08bafb92, + 0x93c91146, 0xe52c287b, 0x7e5fc2af, 0x08e65a01, 0x9395b0d5, + 0xe57089e8, 0x7e03633c, 0x0e5e2b50, 0x952dc184, 0xe3c8f8b9, + 0x78bb126d, 0x0e028ac3, 0x95716017, 0xe394592a, 0x78e7b3fe, + 0x0ee76876, 0x959482a2, 0xe371bb9f, 0x7802514b, 0x0ebbc9e5, + 0x95c82331, 0xe32d1a0c, 0x785ef0d8, 0x0f2cad1c, 0x945f47c8, + 0xe2ba7ef5, 0x79c99421, 0x0f700c8f, 0x9403e65b, 0xe2e6df66, + 0x799535b2, 0x0f95ee3a, 0x94e604ee, 0xe2033dd3, 0x7970d707, + 0x0fc94fa9, 0x94baa57d, 0xe25f9c40, 0x792c7694, 0x0cbb27c8, + 0x97c8cd1c, 0xe12df421, 0x7a5e1ef5, 0x0ce7865b, 0x97946c8f, + 0xe17155b2, 0x7a02bf66, 0x0c0264ee, 0x97718e3a, 0xe194b707, + 0x7ae75dd3, 0x0c5ec57d, 0x972d2fa9, 0xe1c81694, 0x7abbfc40, + 0x0dc9a184, 0x96ba4b50, 0xe05f726d, 0x7b2c98b9, 0x0d950017, + 0x96e6eac3, 0xe003d3fe, 0x7b70392a, 0x0d70e2a2, 0x96030876, + 0xe0e6314b, 0x7b95db9f, 0x0d2c4331, 0x965fa9e5, 0xe0ba90d8, + 0x7bc97a0c}, + {0x00000000, 0x172864c0, 0x2e50c980, 0x3978ad40, 0x5ca19300, + 0x4b89f7c0, 0x72f15a80, 0x65d93e40, 0xb9432600, 0xae6b42c0, + 0x9713ef80, 0x803b8b40, 0xe5e2b500, 0xf2cad1c0, 0xcbb27c80, + 0xdc9a1840, 0xa9f74a41, 0xbedf2e81, 0x87a783c1, 0x908fe701, + 0xf556d941, 0xe27ebd81, 0xdb0610c1, 0xcc2e7401, 0x10b46c41, + 0x079c0881, 0x3ee4a5c1, 0x29ccc101, 0x4c15ff41, 0x5b3d9b81, + 0x624536c1, 0x756d5201, 0x889f92c3, 0x9fb7f603, 0xa6cf5b43, + 0xb1e73f83, 0xd43e01c3, 0xc3166503, 0xfa6ec843, 0xed46ac83, + 0x31dcb4c3, 0x26f4d003, 0x1f8c7d43, 0x08a41983, 0x6d7d27c3, + 0x7a554303, 0x432dee43, 0x54058a83, 0x2168d882, 0x3640bc42, + 0x0f381102, 0x181075c2, 0x7dc94b82, 0x6ae12f42, 0x53998202, + 0x44b1e6c2, 0x982bfe82, 0x8f039a42, 0xb67b3702, 0xa15353c2, + 0xc48a6d82, 0xd3a20942, 0xeadaa402, 0xfdf2c0c2, 0xca4e23c7, + 0xdd664707, 0xe41eea47, 0xf3368e87, 0x96efb0c7, 0x81c7d407, + 0xb8bf7947, 0xaf971d87, 0x730d05c7, 0x64256107, 0x5d5dcc47, + 0x4a75a887, 0x2fac96c7, 0x3884f207, 0x01fc5f47, 0x16d43b87, + 0x63b96986, 0x74910d46, 0x4de9a006, 0x5ac1c4c6, 0x3f18fa86, + 0x28309e46, 0x11483306, 0x066057c6, 0xdafa4f86, 0xcdd22b46, + 0xf4aa8606, 0xe382e2c6, 0x865bdc86, 0x9173b846, 0xa80b1506, + 0xbf2371c6, 0x42d1b104, 0x55f9d5c4, 0x6c817884, 0x7ba91c44, + 0x1e702204, 0x095846c4, 0x3020eb84, 0x27088f44, 0xfb929704, + 0xecbaf3c4, 0xd5c25e84, 0xc2ea3a44, 0xa7330404, 0xb01b60c4, + 0x8963cd84, 0x9e4ba944, 0xeb26fb45, 0xfc0e9f85, 0xc57632c5, + 0xd25e5605, 0xb7876845, 0xa0af0c85, 0x99d7a1c5, 0x8effc505, + 0x5265dd45, 0x454db985, 0x7c3514c5, 0x6b1d7005, 0x0ec44e45, + 0x19ec2a85, 0x209487c5, 0x37bce305, 0x4fed41cf, 0x58c5250f, + 0x61bd884f, 0x7695ec8f, 0x134cd2cf, 0x0464b60f, 0x3d1c1b4f, + 0x2a347f8f, 0xf6ae67cf, 0xe186030f, 0xd8feae4f, 0xcfd6ca8f, + 0xaa0ff4cf, 0xbd27900f, 0x845f3d4f, 0x9377598f, 0xe61a0b8e, + 0xf1326f4e, 0xc84ac20e, 0xdf62a6ce, 0xbabb988e, 0xad93fc4e, + 0x94eb510e, 0x83c335ce, 0x5f592d8e, 0x4871494e, 0x7109e40e, + 0x662180ce, 0x03f8be8e, 0x14d0da4e, 0x2da8770e, 0x3a8013ce, + 0xc772d30c, 0xd05ab7cc, 0xe9221a8c, 0xfe0a7e4c, 0x9bd3400c, + 0x8cfb24cc, 0xb583898c, 0xa2abed4c, 0x7e31f50c, 0x691991cc, + 0x50613c8c, 0x4749584c, 0x2290660c, 0x35b802cc, 0x0cc0af8c, + 0x1be8cb4c, 0x6e85994d, 0x79adfd8d, 0x40d550cd, 0x57fd340d, + 0x32240a4d, 0x250c6e8d, 0x1c74c3cd, 0x0b5ca70d, 0xd7c6bf4d, + 0xc0eedb8d, 0xf99676cd, 0xeebe120d, 0x8b672c4d, 0x9c4f488d, + 0xa537e5cd, 0xb21f810d, 0x85a36208, 0x928b06c8, 0xabf3ab88, + 0xbcdbcf48, 0xd902f108, 0xce2a95c8, 0xf7523888, 0xe07a5c48, + 0x3ce04408, 0x2bc820c8, 0x12b08d88, 0x0598e948, 0x6041d708, + 0x7769b3c8, 0x4e111e88, 0x59397a48, 0x2c542849, 0x3b7c4c89, + 0x0204e1c9, 0x152c8509, 0x70f5bb49, 0x67dddf89, 0x5ea572c9, + 0x498d1609, 0x95170e49, 0x823f6a89, 0xbb47c7c9, 0xac6fa309, + 0xc9b69d49, 0xde9ef989, 0xe7e654c9, 0xf0ce3009, 0x0d3cf0cb, + 0x1a14940b, 0x236c394b, 0x34445d8b, 0x519d63cb, 0x46b5070b, + 0x7fcdaa4b, 0x68e5ce8b, 0xb47fd6cb, 0xa357b20b, 0x9a2f1f4b, + 0x8d077b8b, 0xe8de45cb, 0xfff6210b, 0xc68e8c4b, 0xd1a6e88b, + 0xa4cbba8a, 0xb3e3de4a, 0x8a9b730a, 0x9db317ca, 0xf86a298a, + 0xef424d4a, 0xd63ae00a, 0xc11284ca, 0x1d889c8a, 0x0aa0f84a, + 0x33d8550a, 0x24f031ca, 0x41290f8a, 0x56016b4a, 0x6f79c60a, + 0x7851a2ca}, + {0x00000000, 0x9fda839e, 0xe4c4017d, 0x7b1e82e3, 0x12f904bb, + 0x8d238725, 0xf63d05c6, 0x69e78658, 0x25f20976, 0xba288ae8, + 0xc136080b, 0x5eec8b95, 0x370b0dcd, 0xa8d18e53, 0xd3cf0cb0, + 0x4c158f2e, 0x4be412ec, 0xd43e9172, 0xaf201391, 0x30fa900f, + 0x591d1657, 0xc6c795c9, 0xbdd9172a, 0x220394b4, 0x6e161b9a, + 0xf1cc9804, 0x8ad21ae7, 0x15089979, 0x7cef1f21, 0xe3359cbf, + 0x982b1e5c, 0x07f19dc2, 0x97c825d8, 0x0812a646, 0x730c24a5, + 0xecd6a73b, 0x85312163, 0x1aeba2fd, 0x61f5201e, 0xfe2fa380, + 0xb23a2cae, 0x2de0af30, 0x56fe2dd3, 0xc924ae4d, 0xa0c32815, + 0x3f19ab8b, 0x44072968, 0xdbddaaf6, 0xdc2c3734, 0x43f6b4aa, + 0x38e83649, 0xa732b5d7, 0xced5338f, 0x510fb011, 0x2a1132f2, + 0xb5cbb16c, 0xf9de3e42, 0x6604bddc, 0x1d1a3f3f, 0x82c0bca1, + 0xeb273af9, 0x74fdb967, 0x0fe33b84, 0x9039b81a, 0xf4e14df1, + 0x6b3bce6f, 0x10254c8c, 0x8fffcf12, 0xe618494a, 0x79c2cad4, + 0x02dc4837, 0x9d06cba9, 0xd1134487, 0x4ec9c719, 0x35d745fa, + 0xaa0dc664, 0xc3ea403c, 0x5c30c3a2, 0x272e4141, 0xb8f4c2df, + 0xbf055f1d, 0x20dfdc83, 0x5bc15e60, 0xc41bddfe, 0xadfc5ba6, + 0x3226d838, 0x49385adb, 0xd6e2d945, 0x9af7566b, 0x052dd5f5, + 0x7e335716, 0xe1e9d488, 0x880e52d0, 0x17d4d14e, 0x6cca53ad, + 0xf310d033, 0x63296829, 0xfcf3ebb7, 0x87ed6954, 0x1837eaca, + 0x71d06c92, 0xee0aef0c, 0x95146def, 0x0aceee71, 0x46db615f, + 0xd901e2c1, 0xa21f6022, 0x3dc5e3bc, 0x542265e4, 0xcbf8e67a, + 0xb0e66499, 0x2f3ce707, 0x28cd7ac5, 0xb717f95b, 0xcc097bb8, + 0x53d3f826, 0x3a347e7e, 0xa5eefde0, 0xdef07f03, 0x412afc9d, + 0x0d3f73b3, 0x92e5f02d, 0xe9fb72ce, 0x7621f150, 0x1fc67708, + 0x801cf496, 0xfb027675, 0x64d8f5eb, 0x32b39da3, 0xad691e3d, + 0xd6779cde, 0x49ad1f40, 0x204a9918, 0xbf901a86, 0xc48e9865, + 0x5b541bfb, 0x174194d5, 0x889b174b, 0xf38595a8, 0x6c5f1636, + 0x05b8906e, 0x9a6213f0, 0xe17c9113, 0x7ea6128d, 0x79578f4f, + 0xe68d0cd1, 0x9d938e32, 0x02490dac, 0x6bae8bf4, 0xf474086a, + 0x8f6a8a89, 0x10b00917, 0x5ca58639, 0xc37f05a7, 0xb8618744, + 0x27bb04da, 0x4e5c8282, 0xd186011c, 0xaa9883ff, 0x35420061, + 0xa57bb87b, 0x3aa13be5, 0x41bfb906, 0xde653a98, 0xb782bcc0, + 0x28583f5e, 0x5346bdbd, 0xcc9c3e23, 0x8089b10d, 0x1f533293, + 0x644db070, 0xfb9733ee, 0x9270b5b6, 0x0daa3628, 0x76b4b4cb, + 0xe96e3755, 0xee9faa97, 0x71452909, 0x0a5babea, 0x95812874, + 0xfc66ae2c, 0x63bc2db2, 0x18a2af51, 0x87782ccf, 0xcb6da3e1, + 0x54b7207f, 0x2fa9a29c, 0xb0732102, 0xd994a75a, 0x464e24c4, + 0x3d50a627, 0xa28a25b9, 0xc652d052, 0x598853cc, 0x2296d12f, + 0xbd4c52b1, 0xd4abd4e9, 0x4b715777, 0x306fd594, 0xafb5560a, + 0xe3a0d924, 0x7c7a5aba, 0x0764d859, 0x98be5bc7, 0xf159dd9f, + 0x6e835e01, 0x159ddce2, 0x8a475f7c, 0x8db6c2be, 0x126c4120, + 0x6972c3c3, 0xf6a8405d, 0x9f4fc605, 0x0095459b, 0x7b8bc778, + 0xe45144e6, 0xa844cbc8, 0x379e4856, 0x4c80cab5, 0xd35a492b, + 0xbabdcf73, 0x25674ced, 0x5e79ce0e, 0xc1a34d90, 0x519af58a, + 0xce407614, 0xb55ef4f7, 0x2a847769, 0x4363f131, 0xdcb972af, + 0xa7a7f04c, 0x387d73d2, 0x7468fcfc, 0xebb27f62, 0x90acfd81, + 0x0f767e1f, 0x6691f847, 0xf94b7bd9, 0x8255f93a, 0x1d8f7aa4, + 0x1a7ee766, 0x85a464f8, 0xfebae61b, 0x61606585, 0x0887e3dd, + 0x975d6043, 0xec43e2a0, 0x7399613e, 0x3f8cee10, 0xa0566d8e, + 0xdb48ef6d, 0x44926cf3, 0x2d75eaab, 0xb2af6935, 0xc9b1ebd6, + 0x566b6848}, + {0x00000000, 0x65673b46, 0xcace768c, 0xafa94dca, 0x4eedeb59, + 0x2b8ad01f, 0x84239dd5, 0xe144a693, 0x9ddbd6b2, 0xf8bcedf4, + 0x5715a03e, 0x32729b78, 0xd3363deb, 0xb65106ad, 0x19f84b67, + 0x7c9f7021, 0xe0c6ab25, 0x85a19063, 0x2a08dda9, 0x4f6fe6ef, + 0xae2b407c, 0xcb4c7b3a, 0x64e536f0, 0x01820db6, 0x7d1d7d97, + 0x187a46d1, 0xb7d30b1b, 0xd2b4305d, 0x33f096ce, 0x5697ad88, + 0xf93ee042, 0x9c59db04, 0x1afc500b, 0x7f9b6b4d, 0xd0322687, + 0xb5551dc1, 0x5411bb52, 0x31768014, 0x9edfcdde, 0xfbb8f698, + 0x872786b9, 0xe240bdff, 0x4de9f035, 0x288ecb73, 0xc9ca6de0, + 0xacad56a6, 0x03041b6c, 0x6663202a, 0xfa3afb2e, 0x9f5dc068, + 0x30f48da2, 0x5593b6e4, 0xb4d71077, 0xd1b02b31, 0x7e1966fb, + 0x1b7e5dbd, 0x67e12d9c, 0x028616da, 0xad2f5b10, 0xc8486056, + 0x290cc6c5, 0x4c6bfd83, 0xe3c2b049, 0x86a58b0f, 0x35f8a016, + 0x509f9b50, 0xff36d69a, 0x9a51eddc, 0x7b154b4f, 0x1e727009, + 0xb1db3dc3, 0xd4bc0685, 0xa82376a4, 0xcd444de2, 0x62ed0028, + 0x078a3b6e, 0xe6ce9dfd, 0x83a9a6bb, 0x2c00eb71, 0x4967d037, + 0xd53e0b33, 0xb0593075, 0x1ff07dbf, 0x7a9746f9, 0x9bd3e06a, + 0xfeb4db2c, 0x511d96e6, 0x347aada0, 0x48e5dd81, 0x2d82e6c7, + 0x822bab0d, 0xe74c904b, 0x060836d8, 0x636f0d9e, 0xccc64054, + 0xa9a17b12, 0x2f04f01d, 0x4a63cb5b, 0xe5ca8691, 0x80adbdd7, + 0x61e91b44, 0x048e2002, 0xab276dc8, 0xce40568e, 0xb2df26af, + 0xd7b81de9, 0x78115023, 0x1d766b65, 0xfc32cdf6, 0x9955f6b0, + 0x36fcbb7a, 0x539b803c, 0xcfc25b38, 0xaaa5607e, 0x050c2db4, + 0x606b16f2, 0x812fb061, 0xe4488b27, 0x4be1c6ed, 0x2e86fdab, + 0x52198d8a, 0x377eb6cc, 0x98d7fb06, 0xfdb0c040, 0x1cf466d3, + 0x79935d95, 0xd63a105f, 0xb35d2b19, 0x6bf1402c, 0x0e967b6a, + 0xa13f36a0, 0xc4580de6, 0x251cab75, 0x407b9033, 0xefd2ddf9, + 0x8ab5e6bf, 0xf62a969e, 0x934dadd8, 0x3ce4e012, 0x5983db54, + 0xb8c77dc7, 0xdda04681, 0x72090b4b, 0x176e300d, 0x8b37eb09, + 0xee50d04f, 0x41f99d85, 0x249ea6c3, 0xc5da0050, 0xa0bd3b16, + 0x0f1476dc, 0x6a734d9a, 0x16ec3dbb, 0x738b06fd, 0xdc224b37, + 0xb9457071, 0x5801d6e2, 0x3d66eda4, 0x92cfa06e, 0xf7a89b28, + 0x710d1027, 0x146a2b61, 0xbbc366ab, 0xdea45ded, 0x3fe0fb7e, + 0x5a87c038, 0xf52e8df2, 0x9049b6b4, 0xecd6c695, 0x89b1fdd3, + 0x2618b019, 0x437f8b5f, 0xa23b2dcc, 0xc75c168a, 0x68f55b40, + 0x0d926006, 0x91cbbb02, 0xf4ac8044, 0x5b05cd8e, 0x3e62f6c8, + 0xdf26505b, 0xba416b1d, 0x15e826d7, 0x708f1d91, 0x0c106db0, + 0x697756f6, 0xc6de1b3c, 0xa3b9207a, 0x42fd86e9, 0x279abdaf, + 0x8833f065, 0xed54cb23, 0x5e09e03a, 0x3b6edb7c, 0x94c796b6, + 0xf1a0adf0, 0x10e40b63, 0x75833025, 0xda2a7def, 0xbf4d46a9, + 0xc3d23688, 0xa6b50dce, 0x091c4004, 0x6c7b7b42, 0x8d3fddd1, + 0xe858e697, 0x47f1ab5d, 0x2296901b, 0xbecf4b1f, 0xdba87059, + 0x74013d93, 0x116606d5, 0xf022a046, 0x95459b00, 0x3aecd6ca, + 0x5f8bed8c, 0x23149dad, 0x4673a6eb, 0xe9daeb21, 0x8cbdd067, + 0x6df976f4, 0x089e4db2, 0xa7370078, 0xc2503b3e, 0x44f5b031, + 0x21928b77, 0x8e3bc6bd, 0xeb5cfdfb, 0x0a185b68, 0x6f7f602e, + 0xc0d62de4, 0xa5b116a2, 0xd92e6683, 0xbc495dc5, 0x13e0100f, + 0x76872b49, 0x97c38dda, 0xf2a4b69c, 0x5d0dfb56, 0x386ac010, + 0xa4331b14, 0xc1542052, 0x6efd6d98, 0x0b9a56de, 0xeadef04d, + 0x8fb9cb0b, 0x201086c1, 0x4577bd87, 0x39e8cda6, 0x5c8ff6e0, + 0xf326bb2a, 0x9641806c, 0x770526ff, 0x12621db9, 0xbdcb5073, + 0xd8ac6b35}, + {0x00000000, 0xd7e28058, 0x74b406f1, 0xa35686a9, 0xe9680de2, + 0x3e8a8dba, 0x9ddc0b13, 0x4a3e8b4b, 0x09a11d85, 0xde439ddd, + 0x7d151b74, 0xaaf79b2c, 0xe0c91067, 0x372b903f, 0x947d1696, + 0x439f96ce, 0x13423b0a, 0xc4a0bb52, 0x67f63dfb, 0xb014bda3, + 0xfa2a36e8, 0x2dc8b6b0, 0x8e9e3019, 0x597cb041, 0x1ae3268f, + 0xcd01a6d7, 0x6e57207e, 0xb9b5a026, 0xf38b2b6d, 0x2469ab35, + 0x873f2d9c, 0x50ddadc4, 0x26847614, 0xf166f64c, 0x523070e5, + 0x85d2f0bd, 0xcfec7bf6, 0x180efbae, 0xbb587d07, 0x6cbafd5f, + 0x2f256b91, 0xf8c7ebc9, 0x5b916d60, 0x8c73ed38, 0xc64d6673, + 0x11afe62b, 0xb2f96082, 0x651be0da, 0x35c64d1e, 0xe224cd46, + 0x41724bef, 0x9690cbb7, 0xdcae40fc, 0x0b4cc0a4, 0xa81a460d, + 0x7ff8c655, 0x3c67509b, 0xeb85d0c3, 0x48d3566a, 0x9f31d632, + 0xd50f5d79, 0x02eddd21, 0xa1bb5b88, 0x7659dbd0, 0x4d08ec28, + 0x9aea6c70, 0x39bcead9, 0xee5e6a81, 0xa460e1ca, 0x73826192, + 0xd0d4e73b, 0x07366763, 0x44a9f1ad, 0x934b71f5, 0x301df75c, + 0xe7ff7704, 0xadc1fc4f, 0x7a237c17, 0xd975fabe, 0x0e977ae6, + 0x5e4ad722, 0x89a8577a, 0x2afed1d3, 0xfd1c518b, 0xb722dac0, + 0x60c05a98, 0xc396dc31, 0x14745c69, 0x57ebcaa7, 0x80094aff, + 0x235fcc56, 0xf4bd4c0e, 0xbe83c745, 0x6961471d, 0xca37c1b4, + 0x1dd541ec, 0x6b8c9a3c, 0xbc6e1a64, 0x1f389ccd, 0xc8da1c95, + 0x82e497de, 0x55061786, 0xf650912f, 0x21b21177, 0x622d87b9, + 0xb5cf07e1, 0x16998148, 0xc17b0110, 0x8b458a5b, 0x5ca70a03, + 0xfff18caa, 0x28130cf2, 0x78cea136, 0xaf2c216e, 0x0c7aa7c7, + 0xdb98279f, 0x91a6acd4, 0x46442c8c, 0xe512aa25, 0x32f02a7d, + 0x716fbcb3, 0xa68d3ceb, 0x05dbba42, 0xd2393a1a, 0x9807b151, + 0x4fe53109, 0xecb3b7a0, 0x3b5137f8, 0x9a11d850, 0x4df35808, + 0xeea5dea1, 0x39475ef9, 0x7379d5b2, 0xa49b55ea, 0x07cdd343, + 0xd02f531b, 0x93b0c5d5, 0x4452458d, 0xe704c324, 0x30e6437c, + 0x7ad8c837, 0xad3a486f, 0x0e6ccec6, 0xd98e4e9e, 0x8953e35a, + 0x5eb16302, 0xfde7e5ab, 0x2a0565f3, 0x603beeb8, 0xb7d96ee0, + 0x148fe849, 0xc36d6811, 0x80f2fedf, 0x57107e87, 0xf446f82e, + 0x23a47876, 0x699af33d, 0xbe787365, 0x1d2ef5cc, 0xcacc7594, + 0xbc95ae44, 0x6b772e1c, 0xc821a8b5, 0x1fc328ed, 0x55fda3a6, + 0x821f23fe, 0x2149a557, 0xf6ab250f, 0xb534b3c1, 0x62d63399, + 0xc180b530, 0x16623568, 0x5c5cbe23, 0x8bbe3e7b, 0x28e8b8d2, + 0xff0a388a, 0xafd7954e, 0x78351516, 0xdb6393bf, 0x0c8113e7, + 0x46bf98ac, 0x915d18f4, 0x320b9e5d, 0xe5e91e05, 0xa67688cb, + 0x71940893, 0xd2c28e3a, 0x05200e62, 0x4f1e8529, 0x98fc0571, + 0x3baa83d8, 0xec480380, 0xd7193478, 0x00fbb420, 0xa3ad3289, + 0x744fb2d1, 0x3e71399a, 0xe993b9c2, 0x4ac53f6b, 0x9d27bf33, + 0xdeb829fd, 0x095aa9a5, 0xaa0c2f0c, 0x7deeaf54, 0x37d0241f, + 0xe032a447, 0x436422ee, 0x9486a2b6, 0xc45b0f72, 0x13b98f2a, + 0xb0ef0983, 0x670d89db, 0x2d330290, 0xfad182c8, 0x59870461, + 0x8e658439, 0xcdfa12f7, 0x1a1892af, 0xb94e1406, 0x6eac945e, + 0x24921f15, 0xf3709f4d, 0x502619e4, 0x87c499bc, 0xf19d426c, + 0x267fc234, 0x8529449d, 0x52cbc4c5, 0x18f54f8e, 0xcf17cfd6, + 0x6c41497f, 0xbba3c927, 0xf83c5fe9, 0x2fdedfb1, 0x8c885918, + 0x5b6ad940, 0x1154520b, 0xc6b6d253, 0x65e054fa, 0xb202d4a2, + 0xe2df7966, 0x353df93e, 0x966b7f97, 0x4189ffcf, 0x0bb77484, + 0xdc55f4dc, 0x7f037275, 0xa8e1f22d, 0xeb7e64e3, 0x3c9ce4bb, + 0x9fca6212, 0x4828e24a, 0x02166901, 0xd5f4e959, 0x76a26ff0, + 0xa140efa8}, + {0x00000000, 0xef52b6e1, 0x05d46b83, 0xea86dd62, 0x0ba8d706, + 0xe4fa61e7, 0x0e7cbc85, 0xe12e0a64, 0x1751ae0c, 0xf80318ed, + 0x1285c58f, 0xfdd7736e, 0x1cf9790a, 0xf3abcfeb, 0x192d1289, + 0xf67fa468, 0x2ea35c18, 0xc1f1eaf9, 0x2b77379b, 0xc425817a, + 0x250b8b1e, 0xca593dff, 0x20dfe09d, 0xcf8d567c, 0x39f2f214, + 0xd6a044f5, 0x3c269997, 0xd3742f76, 0x325a2512, 0xdd0893f3, + 0x378e4e91, 0xd8dcf870, 0x5d46b830, 0xb2140ed1, 0x5892d3b3, + 0xb7c06552, 0x56ee6f36, 0xb9bcd9d7, 0x533a04b5, 0xbc68b254, + 0x4a17163c, 0xa545a0dd, 0x4fc37dbf, 0xa091cb5e, 0x41bfc13a, + 0xaeed77db, 0x446baab9, 0xab391c58, 0x73e5e428, 0x9cb752c9, + 0x76318fab, 0x9963394a, 0x784d332e, 0x971f85cf, 0x7d9958ad, + 0x92cbee4c, 0x64b44a24, 0x8be6fcc5, 0x616021a7, 0x8e329746, + 0x6f1c9d22, 0x804e2bc3, 0x6ac8f6a1, 0x859a4040, 0xba8d7060, + 0x55dfc681, 0xbf591be3, 0x500bad02, 0xb125a766, 0x5e771187, + 0xb4f1cce5, 0x5ba37a04, 0xaddcde6c, 0x428e688d, 0xa808b5ef, + 0x475a030e, 0xa674096a, 0x4926bf8b, 0xa3a062e9, 0x4cf2d408, + 0x942e2c78, 0x7b7c9a99, 0x91fa47fb, 0x7ea8f11a, 0x9f86fb7e, + 0x70d44d9f, 0x9a5290fd, 0x7500261c, 0x837f8274, 0x6c2d3495, + 0x86abe9f7, 0x69f95f16, 0x88d75572, 0x6785e393, 0x8d033ef1, + 0x62518810, 0xe7cbc850, 0x08997eb1, 0xe21fa3d3, 0x0d4d1532, + 0xec631f56, 0x0331a9b7, 0xe9b774d5, 0x06e5c234, 0xf09a665c, + 0x1fc8d0bd, 0xf54e0ddf, 0x1a1cbb3e, 0xfb32b15a, 0x146007bb, + 0xfee6dad9, 0x11b46c38, 0xc9689448, 0x263a22a9, 0xccbcffcb, + 0x23ee492a, 0xc2c0434e, 0x2d92f5af, 0xc71428cd, 0x28469e2c, + 0xde393a44, 0x316b8ca5, 0xdbed51c7, 0x34bfe726, 0xd591ed42, + 0x3ac35ba3, 0xd04586c1, 0x3f173020, 0xae6be681, 0x41395060, + 0xabbf8d02, 0x44ed3be3, 0xa5c33187, 0x4a918766, 0xa0175a04, + 0x4f45ece5, 0xb93a488d, 0x5668fe6c, 0xbcee230e, 0x53bc95ef, + 0xb2929f8b, 0x5dc0296a, 0xb746f408, 0x581442e9, 0x80c8ba99, + 0x6f9a0c78, 0x851cd11a, 0x6a4e67fb, 0x8b606d9f, 0x6432db7e, + 0x8eb4061c, 0x61e6b0fd, 0x97991495, 0x78cba274, 0x924d7f16, + 0x7d1fc9f7, 0x9c31c393, 0x73637572, 0x99e5a810, 0x76b71ef1, + 0xf32d5eb1, 0x1c7fe850, 0xf6f93532, 0x19ab83d3, 0xf88589b7, + 0x17d73f56, 0xfd51e234, 0x120354d5, 0xe47cf0bd, 0x0b2e465c, + 0xe1a89b3e, 0x0efa2ddf, 0xefd427bb, 0x0086915a, 0xea004c38, + 0x0552fad9, 0xdd8e02a9, 0x32dcb448, 0xd85a692a, 0x3708dfcb, + 0xd626d5af, 0x3974634e, 0xd3f2be2c, 0x3ca008cd, 0xcadfaca5, + 0x258d1a44, 0xcf0bc726, 0x205971c7, 0xc1777ba3, 0x2e25cd42, + 0xc4a31020, 0x2bf1a6c1, 0x14e696e1, 0xfbb42000, 0x1132fd62, + 0xfe604b83, 0x1f4e41e7, 0xf01cf706, 0x1a9a2a64, 0xf5c89c85, + 0x03b738ed, 0xece58e0c, 0x0663536e, 0xe931e58f, 0x081fefeb, + 0xe74d590a, 0x0dcb8468, 0xe2993289, 0x3a45caf9, 0xd5177c18, + 0x3f91a17a, 0xd0c3179b, 0x31ed1dff, 0xdebfab1e, 0x3439767c, + 0xdb6bc09d, 0x2d1464f5, 0xc246d214, 0x28c00f76, 0xc792b997, + 0x26bcb3f3, 0xc9ee0512, 0x2368d870, 0xcc3a6e91, 0x49a02ed1, + 0xa6f29830, 0x4c744552, 0xa326f3b3, 0x4208f9d7, 0xad5a4f36, + 0x47dc9254, 0xa88e24b5, 0x5ef180dd, 0xb1a3363c, 0x5b25eb5e, + 0xb4775dbf, 0x555957db, 0xba0be13a, 0x508d3c58, 0xbfdf8ab9, + 0x670372c9, 0x8851c428, 0x62d7194a, 0x8d85afab, 0x6caba5cf, + 0x83f9132e, 0x697fce4c, 0x862d78ad, 0x7052dcc5, 0x9f006a24, + 0x7586b746, 0x9ad401a7, 0x7bfa0bc3, 0x94a8bd22, 0x7e2e6040, + 0x917cd6a1}, + {0x00000000, 0x87a6cb43, 0xd43c90c7, 0x539a5b84, 0x730827cf, + 0xf4aeec8c, 0xa734b708, 0x20927c4b, 0xe6104f9e, 0x61b684dd, + 0x322cdf59, 0xb58a141a, 0x95186851, 0x12bea312, 0x4124f896, + 0xc68233d5, 0x1751997d, 0x90f7523e, 0xc36d09ba, 0x44cbc2f9, + 0x6459beb2, 0xe3ff75f1, 0xb0652e75, 0x37c3e536, 0xf141d6e3, + 0x76e71da0, 0x257d4624, 0xa2db8d67, 0x8249f12c, 0x05ef3a6f, + 0x567561eb, 0xd1d3aaa8, 0x2ea332fa, 0xa905f9b9, 0xfa9fa23d, + 0x7d39697e, 0x5dab1535, 0xda0dde76, 0x899785f2, 0x0e314eb1, + 0xc8b37d64, 0x4f15b627, 0x1c8feda3, 0x9b2926e0, 0xbbbb5aab, + 0x3c1d91e8, 0x6f87ca6c, 0xe821012f, 0x39f2ab87, 0xbe5460c4, + 0xedce3b40, 0x6a68f003, 0x4afa8c48, 0xcd5c470b, 0x9ec61c8f, + 0x1960d7cc, 0xdfe2e419, 0x58442f5a, 0x0bde74de, 0x8c78bf9d, + 0xaceac3d6, 0x2b4c0895, 0x78d65311, 0xff709852, 0x5d4665f4, + 0xdae0aeb7, 0x897af533, 0x0edc3e70, 0x2e4e423b, 0xa9e88978, + 0xfa72d2fc, 0x7dd419bf, 0xbb562a6a, 0x3cf0e129, 0x6f6abaad, + 0xe8cc71ee, 0xc85e0da5, 0x4ff8c6e6, 0x1c629d62, 0x9bc45621, + 0x4a17fc89, 0xcdb137ca, 0x9e2b6c4e, 0x198da70d, 0x391fdb46, + 0xbeb91005, 0xed234b81, 0x6a8580c2, 0xac07b317, 0x2ba17854, + 0x783b23d0, 0xff9de893, 0xdf0f94d8, 0x58a95f9b, 0x0b33041f, + 0x8c95cf5c, 0x73e5570e, 0xf4439c4d, 0xa7d9c7c9, 0x207f0c8a, + 0x00ed70c1, 0x874bbb82, 0xd4d1e006, 0x53772b45, 0x95f51890, + 0x1253d3d3, 0x41c98857, 0xc66f4314, 0xe6fd3f5f, 0x615bf41c, + 0x32c1af98, 0xb56764db, 0x64b4ce73, 0xe3120530, 0xb0885eb4, + 0x372e95f7, 0x17bce9bc, 0x901a22ff, 0xc380797b, 0x4426b238, + 0x82a481ed, 0x05024aae, 0x5698112a, 0xd13eda69, 0xf1aca622, + 0x760a6d61, 0x259036e5, 0xa236fda6, 0xba8ccbe8, 0x3d2a00ab, + 0x6eb05b2f, 0xe916906c, 0xc984ec27, 0x4e222764, 0x1db87ce0, + 0x9a1eb7a3, 0x5c9c8476, 0xdb3a4f35, 0x88a014b1, 0x0f06dff2, + 0x2f94a3b9, 0xa83268fa, 0xfba8337e, 0x7c0ef83d, 0xaddd5295, + 0x2a7b99d6, 0x79e1c252, 0xfe470911, 0xded5755a, 0x5973be19, + 0x0ae9e59d, 0x8d4f2ede, 0x4bcd1d0b, 0xcc6bd648, 0x9ff18dcc, + 0x1857468f, 0x38c53ac4, 0xbf63f187, 0xecf9aa03, 0x6b5f6140, + 0x942ff912, 0x13893251, 0x401369d5, 0xc7b5a296, 0xe727dedd, + 0x6081159e, 0x331b4e1a, 0xb4bd8559, 0x723fb68c, 0xf5997dcf, + 0xa603264b, 0x21a5ed08, 0x01379143, 0x86915a00, 0xd50b0184, + 0x52adcac7, 0x837e606f, 0x04d8ab2c, 0x5742f0a8, 0xd0e43beb, + 0xf07647a0, 0x77d08ce3, 0x244ad767, 0xa3ec1c24, 0x656e2ff1, + 0xe2c8e4b2, 0xb152bf36, 0x36f47475, 0x1666083e, 0x91c0c37d, + 0xc25a98f9, 0x45fc53ba, 0xe7caae1c, 0x606c655f, 0x33f63edb, + 0xb450f598, 0x94c289d3, 0x13644290, 0x40fe1914, 0xc758d257, + 0x01dae182, 0x867c2ac1, 0xd5e67145, 0x5240ba06, 0x72d2c64d, + 0xf5740d0e, 0xa6ee568a, 0x21489dc9, 0xf09b3761, 0x773dfc22, + 0x24a7a7a6, 0xa3016ce5, 0x839310ae, 0x0435dbed, 0x57af8069, + 0xd0094b2a, 0x168b78ff, 0x912db3bc, 0xc2b7e838, 0x4511237b, + 0x65835f30, 0xe2259473, 0xb1bfcff7, 0x361904b4, 0xc9699ce6, + 0x4ecf57a5, 0x1d550c21, 0x9af3c762, 0xba61bb29, 0x3dc7706a, + 0x6e5d2bee, 0xe9fbe0ad, 0x2f79d378, 0xa8df183b, 0xfb4543bf, + 0x7ce388fc, 0x5c71f4b7, 0xdbd73ff4, 0x884d6470, 0x0febaf33, + 0xde38059b, 0x599eced8, 0x0a04955c, 0x8da25e1f, 0xad302254, + 0x2a96e917, 0x790cb293, 0xfeaa79d0, 0x38284a05, 0xbf8e8146, + 0xec14dac2, 0x6bb21181, 0x4b206dca, 0xcc86a689, 0x9f1cfd0d, + 0x18ba364e}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x0000000000000000, 0x43cba68700000000, 0xc7903cd400000000, + 0x845b9a5300000000, 0xcf27087300000000, 0x8cecaef400000000, + 0x08b734a700000000, 0x4b7c922000000000, 0x9e4f10e600000000, + 0xdd84b66100000000, 0x59df2c3200000000, 0x1a148ab500000000, + 0x5168189500000000, 0x12a3be1200000000, 0x96f8244100000000, + 0xd53382c600000000, 0x7d99511700000000, 0x3e52f79000000000, + 0xba096dc300000000, 0xf9c2cb4400000000, 0xb2be596400000000, + 0xf175ffe300000000, 0x752e65b000000000, 0x36e5c33700000000, + 0xe3d641f100000000, 0xa01de77600000000, 0x24467d2500000000, + 0x678ddba200000000, 0x2cf1498200000000, 0x6f3aef0500000000, + 0xeb61755600000000, 0xa8aad3d100000000, 0xfa32a32e00000000, + 0xb9f905a900000000, 0x3da29ffa00000000, 0x7e69397d00000000, + 0x3515ab5d00000000, 0x76de0dda00000000, 0xf285978900000000, + 0xb14e310e00000000, 0x647db3c800000000, 0x27b6154f00000000, + 0xa3ed8f1c00000000, 0xe026299b00000000, 0xab5abbbb00000000, + 0xe8911d3c00000000, 0x6cca876f00000000, 0x2f0121e800000000, + 0x87abf23900000000, 0xc46054be00000000, 0x403bceed00000000, + 0x03f0686a00000000, 0x488cfa4a00000000, 0x0b475ccd00000000, + 0x8f1cc69e00000000, 0xccd7601900000000, 0x19e4e2df00000000, + 0x5a2f445800000000, 0xde74de0b00000000, 0x9dbf788c00000000, + 0xd6c3eaac00000000, 0x95084c2b00000000, 0x1153d67800000000, + 0x529870ff00000000, 0xf465465d00000000, 0xb7aee0da00000000, + 0x33f57a8900000000, 0x703edc0e00000000, 0x3b424e2e00000000, + 0x7889e8a900000000, 0xfcd272fa00000000, 0xbf19d47d00000000, + 0x6a2a56bb00000000, 0x29e1f03c00000000, 0xadba6a6f00000000, + 0xee71cce800000000, 0xa50d5ec800000000, 0xe6c6f84f00000000, + 0x629d621c00000000, 0x2156c49b00000000, 0x89fc174a00000000, + 0xca37b1cd00000000, 0x4e6c2b9e00000000, 0x0da78d1900000000, + 0x46db1f3900000000, 0x0510b9be00000000, 0x814b23ed00000000, + 0xc280856a00000000, 0x17b307ac00000000, 0x5478a12b00000000, + 0xd0233b7800000000, 0x93e89dff00000000, 0xd8940fdf00000000, + 0x9b5fa95800000000, 0x1f04330b00000000, 0x5ccf958c00000000, + 0x0e57e57300000000, 0x4d9c43f400000000, 0xc9c7d9a700000000, + 0x8a0c7f2000000000, 0xc170ed0000000000, 0x82bb4b8700000000, + 0x06e0d1d400000000, 0x452b775300000000, 0x9018f59500000000, + 0xd3d3531200000000, 0x5788c94100000000, 0x14436fc600000000, + 0x5f3ffde600000000, 0x1cf45b6100000000, 0x98afc13200000000, + 0xdb6467b500000000, 0x73ceb46400000000, 0x300512e300000000, + 0xb45e88b000000000, 0xf7952e3700000000, 0xbce9bc1700000000, + 0xff221a9000000000, 0x7b7980c300000000, 0x38b2264400000000, + 0xed81a48200000000, 0xae4a020500000000, 0x2a11985600000000, + 0x69da3ed100000000, 0x22a6acf100000000, 0x616d0a7600000000, + 0xe536902500000000, 0xa6fd36a200000000, 0xe8cb8cba00000000, + 0xab002a3d00000000, 0x2f5bb06e00000000, 0x6c9016e900000000, + 0x27ec84c900000000, 0x6427224e00000000, 0xe07cb81d00000000, + 0xa3b71e9a00000000, 0x76849c5c00000000, 0x354f3adb00000000, + 0xb114a08800000000, 0xf2df060f00000000, 0xb9a3942f00000000, + 0xfa6832a800000000, 0x7e33a8fb00000000, 0x3df80e7c00000000, + 0x9552ddad00000000, 0xd6997b2a00000000, 0x52c2e17900000000, + 0x110947fe00000000, 0x5a75d5de00000000, 0x19be735900000000, + 0x9de5e90a00000000, 0xde2e4f8d00000000, 0x0b1dcd4b00000000, + 0x48d66bcc00000000, 0xcc8df19f00000000, 0x8f46571800000000, + 0xc43ac53800000000, 0x87f163bf00000000, 0x03aaf9ec00000000, + 0x40615f6b00000000, 0x12f92f9400000000, 0x5132891300000000, + 0xd569134000000000, 0x96a2b5c700000000, 0xddde27e700000000, + 0x9e15816000000000, 0x1a4e1b3300000000, 0x5985bdb400000000, + 0x8cb63f7200000000, 0xcf7d99f500000000, 0x4b2603a600000000, + 0x08eda52100000000, 0x4391370100000000, 0x005a918600000000, + 0x84010bd500000000, 0xc7caad5200000000, 0x6f607e8300000000, + 0x2cabd80400000000, 0xa8f0425700000000, 0xeb3be4d000000000, + 0xa04776f000000000, 0xe38cd07700000000, 0x67d74a2400000000, + 0x241ceca300000000, 0xf12f6e6500000000, 0xb2e4c8e200000000, + 0x36bf52b100000000, 0x7574f43600000000, 0x3e08661600000000, + 0x7dc3c09100000000, 0xf9985ac200000000, 0xba53fc4500000000, + 0x1caecae700000000, 0x5f656c6000000000, 0xdb3ef63300000000, + 0x98f550b400000000, 0xd389c29400000000, 0x9042641300000000, + 0x1419fe4000000000, 0x57d258c700000000, 0x82e1da0100000000, + 0xc12a7c8600000000, 0x4571e6d500000000, 0x06ba405200000000, + 0x4dc6d27200000000, 0x0e0d74f500000000, 0x8a56eea600000000, + 0xc99d482100000000, 0x61379bf000000000, 0x22fc3d7700000000, + 0xa6a7a72400000000, 0xe56c01a300000000, 0xae10938300000000, + 0xeddb350400000000, 0x6980af5700000000, 0x2a4b09d000000000, + 0xff788b1600000000, 0xbcb32d9100000000, 0x38e8b7c200000000, + 0x7b23114500000000, 0x305f836500000000, 0x739425e200000000, + 0xf7cfbfb100000000, 0xb404193600000000, 0xe69c69c900000000, + 0xa557cf4e00000000, 0x210c551d00000000, 0x62c7f39a00000000, + 0x29bb61ba00000000, 0x6a70c73d00000000, 0xee2b5d6e00000000, + 0xade0fbe900000000, 0x78d3792f00000000, 0x3b18dfa800000000, + 0xbf4345fb00000000, 0xfc88e37c00000000, 0xb7f4715c00000000, + 0xf43fd7db00000000, 0x70644d8800000000, 0x33afeb0f00000000, + 0x9b0538de00000000, 0xd8ce9e5900000000, 0x5c95040a00000000, + 0x1f5ea28d00000000, 0x542230ad00000000, 0x17e9962a00000000, + 0x93b20c7900000000, 0xd079aafe00000000, 0x054a283800000000, + 0x46818ebf00000000, 0xc2da14ec00000000, 0x8111b26b00000000, + 0xca6d204b00000000, 0x89a686cc00000000, 0x0dfd1c9f00000000, + 0x4e36ba1800000000}, + {0x0000000000000000, 0xe1b652ef00000000, 0x836bd40500000000, + 0x62dd86ea00000000, 0x06d7a80b00000000, 0xe761fae400000000, + 0x85bc7c0e00000000, 0x640a2ee100000000, 0x0cae511700000000, + 0xed1803f800000000, 0x8fc5851200000000, 0x6e73d7fd00000000, + 0x0a79f91c00000000, 0xebcfabf300000000, 0x89122d1900000000, + 0x68a47ff600000000, 0x185ca32e00000000, 0xf9eaf1c100000000, + 0x9b37772b00000000, 0x7a8125c400000000, 0x1e8b0b2500000000, + 0xff3d59ca00000000, 0x9de0df2000000000, 0x7c568dcf00000000, + 0x14f2f23900000000, 0xf544a0d600000000, 0x9799263c00000000, + 0x762f74d300000000, 0x12255a3200000000, 0xf39308dd00000000, + 0x914e8e3700000000, 0x70f8dcd800000000, 0x30b8465d00000000, + 0xd10e14b200000000, 0xb3d3925800000000, 0x5265c0b700000000, + 0x366fee5600000000, 0xd7d9bcb900000000, 0xb5043a5300000000, + 0x54b268bc00000000, 0x3c16174a00000000, 0xdda045a500000000, + 0xbf7dc34f00000000, 0x5ecb91a000000000, 0x3ac1bf4100000000, + 0xdb77edae00000000, 0xb9aa6b4400000000, 0x581c39ab00000000, + 0x28e4e57300000000, 0xc952b79c00000000, 0xab8f317600000000, + 0x4a39639900000000, 0x2e334d7800000000, 0xcf851f9700000000, + 0xad58997d00000000, 0x4ceecb9200000000, 0x244ab46400000000, + 0xc5fce68b00000000, 0xa721606100000000, 0x4697328e00000000, + 0x229d1c6f00000000, 0xc32b4e8000000000, 0xa1f6c86a00000000, + 0x40409a8500000000, 0x60708dba00000000, 0x81c6df5500000000, + 0xe31b59bf00000000, 0x02ad0b5000000000, 0x66a725b100000000, + 0x8711775e00000000, 0xe5ccf1b400000000, 0x047aa35b00000000, + 0x6cdedcad00000000, 0x8d688e4200000000, 0xefb508a800000000, + 0x0e035a4700000000, 0x6a0974a600000000, 0x8bbf264900000000, + 0xe962a0a300000000, 0x08d4f24c00000000, 0x782c2e9400000000, + 0x999a7c7b00000000, 0xfb47fa9100000000, 0x1af1a87e00000000, + 0x7efb869f00000000, 0x9f4dd47000000000, 0xfd90529a00000000, + 0x1c26007500000000, 0x74827f8300000000, 0x95342d6c00000000, + 0xf7e9ab8600000000, 0x165ff96900000000, 0x7255d78800000000, + 0x93e3856700000000, 0xf13e038d00000000, 0x1088516200000000, + 0x50c8cbe700000000, 0xb17e990800000000, 0xd3a31fe200000000, + 0x32154d0d00000000, 0x561f63ec00000000, 0xb7a9310300000000, + 0xd574b7e900000000, 0x34c2e50600000000, 0x5c669af000000000, + 0xbdd0c81f00000000, 0xdf0d4ef500000000, 0x3ebb1c1a00000000, + 0x5ab132fb00000000, 0xbb07601400000000, 0xd9dae6fe00000000, + 0x386cb41100000000, 0x489468c900000000, 0xa9223a2600000000, + 0xcbffbccc00000000, 0x2a49ee2300000000, 0x4e43c0c200000000, + 0xaff5922d00000000, 0xcd2814c700000000, 0x2c9e462800000000, + 0x443a39de00000000, 0xa58c6b3100000000, 0xc751eddb00000000, + 0x26e7bf3400000000, 0x42ed91d500000000, 0xa35bc33a00000000, + 0xc18645d000000000, 0x2030173f00000000, 0x81e66bae00000000, + 0x6050394100000000, 0x028dbfab00000000, 0xe33bed4400000000, + 0x8731c3a500000000, 0x6687914a00000000, 0x045a17a000000000, + 0xe5ec454f00000000, 0x8d483ab900000000, 0x6cfe685600000000, + 0x0e23eebc00000000, 0xef95bc5300000000, 0x8b9f92b200000000, + 0x6a29c05d00000000, 0x08f446b700000000, 0xe942145800000000, + 0x99bac88000000000, 0x780c9a6f00000000, 0x1ad11c8500000000, + 0xfb674e6a00000000, 0x9f6d608b00000000, 0x7edb326400000000, + 0x1c06b48e00000000, 0xfdb0e66100000000, 0x9514999700000000, + 0x74a2cb7800000000, 0x167f4d9200000000, 0xf7c91f7d00000000, + 0x93c3319c00000000, 0x7275637300000000, 0x10a8e59900000000, + 0xf11eb77600000000, 0xb15e2df300000000, 0x50e87f1c00000000, + 0x3235f9f600000000, 0xd383ab1900000000, 0xb78985f800000000, + 0x563fd71700000000, 0x34e251fd00000000, 0xd554031200000000, + 0xbdf07ce400000000, 0x5c462e0b00000000, 0x3e9ba8e100000000, + 0xdf2dfa0e00000000, 0xbb27d4ef00000000, 0x5a91860000000000, + 0x384c00ea00000000, 0xd9fa520500000000, 0xa9028edd00000000, + 0x48b4dc3200000000, 0x2a695ad800000000, 0xcbdf083700000000, + 0xafd526d600000000, 0x4e63743900000000, 0x2cbef2d300000000, + 0xcd08a03c00000000, 0xa5acdfca00000000, 0x441a8d2500000000, + 0x26c70bcf00000000, 0xc771592000000000, 0xa37b77c100000000, + 0x42cd252e00000000, 0x2010a3c400000000, 0xc1a6f12b00000000, + 0xe196e61400000000, 0x0020b4fb00000000, 0x62fd321100000000, + 0x834b60fe00000000, 0xe7414e1f00000000, 0x06f71cf000000000, + 0x642a9a1a00000000, 0x859cc8f500000000, 0xed38b70300000000, + 0x0c8ee5ec00000000, 0x6e53630600000000, 0x8fe531e900000000, + 0xebef1f0800000000, 0x0a594de700000000, 0x6884cb0d00000000, + 0x893299e200000000, 0xf9ca453a00000000, 0x187c17d500000000, + 0x7aa1913f00000000, 0x9b17c3d000000000, 0xff1ded3100000000, + 0x1eabbfde00000000, 0x7c76393400000000, 0x9dc06bdb00000000, + 0xf564142d00000000, 0x14d246c200000000, 0x760fc02800000000, + 0x97b992c700000000, 0xf3b3bc2600000000, 0x1205eec900000000, + 0x70d8682300000000, 0x916e3acc00000000, 0xd12ea04900000000, + 0x3098f2a600000000, 0x5245744c00000000, 0xb3f326a300000000, + 0xd7f9084200000000, 0x364f5aad00000000, 0x5492dc4700000000, + 0xb5248ea800000000, 0xdd80f15e00000000, 0x3c36a3b100000000, + 0x5eeb255b00000000, 0xbf5d77b400000000, 0xdb57595500000000, + 0x3ae10bba00000000, 0x583c8d5000000000, 0xb98adfbf00000000, + 0xc972036700000000, 0x28c4518800000000, 0x4a19d76200000000, + 0xabaf858d00000000, 0xcfa5ab6c00000000, 0x2e13f98300000000, + 0x4cce7f6900000000, 0xad782d8600000000, 0xc5dc527000000000, + 0x246a009f00000000, 0x46b7867500000000, 0xa701d49a00000000, + 0xc30bfa7b00000000, 0x22bda89400000000, 0x40602e7e00000000, + 0xa1d67c9100000000}, + {0x0000000000000000, 0x5880e2d700000000, 0xf106b47400000000, + 0xa98656a300000000, 0xe20d68e900000000, 0xba8d8a3e00000000, + 0x130bdc9d00000000, 0x4b8b3e4a00000000, 0x851da10900000000, + 0xdd9d43de00000000, 0x741b157d00000000, 0x2c9bf7aa00000000, + 0x6710c9e000000000, 0x3f902b3700000000, 0x96167d9400000000, + 0xce969f4300000000, 0x0a3b421300000000, 0x52bba0c400000000, + 0xfb3df66700000000, 0xa3bd14b000000000, 0xe8362afa00000000, + 0xb0b6c82d00000000, 0x19309e8e00000000, 0x41b07c5900000000, + 0x8f26e31a00000000, 0xd7a601cd00000000, 0x7e20576e00000000, + 0x26a0b5b900000000, 0x6d2b8bf300000000, 0x35ab692400000000, + 0x9c2d3f8700000000, 0xc4addd5000000000, 0x1476842600000000, + 0x4cf666f100000000, 0xe570305200000000, 0xbdf0d28500000000, + 0xf67beccf00000000, 0xaefb0e1800000000, 0x077d58bb00000000, + 0x5ffdba6c00000000, 0x916b252f00000000, 0xc9ebc7f800000000, + 0x606d915b00000000, 0x38ed738c00000000, 0x73664dc600000000, + 0x2be6af1100000000, 0x8260f9b200000000, 0xdae01b6500000000, + 0x1e4dc63500000000, 0x46cd24e200000000, 0xef4b724100000000, + 0xb7cb909600000000, 0xfc40aedc00000000, 0xa4c04c0b00000000, + 0x0d461aa800000000, 0x55c6f87f00000000, 0x9b50673c00000000, + 0xc3d085eb00000000, 0x6a56d34800000000, 0x32d6319f00000000, + 0x795d0fd500000000, 0x21dded0200000000, 0x885bbba100000000, + 0xd0db597600000000, 0x28ec084d00000000, 0x706cea9a00000000, + 0xd9eabc3900000000, 0x816a5eee00000000, 0xcae160a400000000, + 0x9261827300000000, 0x3be7d4d000000000, 0x6367360700000000, + 0xadf1a94400000000, 0xf5714b9300000000, 0x5cf71d3000000000, + 0x0477ffe700000000, 0x4ffcc1ad00000000, 0x177c237a00000000, + 0xbefa75d900000000, 0xe67a970e00000000, 0x22d74a5e00000000, + 0x7a57a88900000000, 0xd3d1fe2a00000000, 0x8b511cfd00000000, + 0xc0da22b700000000, 0x985ac06000000000, 0x31dc96c300000000, + 0x695c741400000000, 0xa7caeb5700000000, 0xff4a098000000000, + 0x56cc5f2300000000, 0x0e4cbdf400000000, 0x45c783be00000000, + 0x1d47616900000000, 0xb4c137ca00000000, 0xec41d51d00000000, + 0x3c9a8c6b00000000, 0x641a6ebc00000000, 0xcd9c381f00000000, + 0x951cdac800000000, 0xde97e48200000000, 0x8617065500000000, + 0x2f9150f600000000, 0x7711b22100000000, 0xb9872d6200000000, + 0xe107cfb500000000, 0x4881991600000000, 0x10017bc100000000, + 0x5b8a458b00000000, 0x030aa75c00000000, 0xaa8cf1ff00000000, + 0xf20c132800000000, 0x36a1ce7800000000, 0x6e212caf00000000, + 0xc7a77a0c00000000, 0x9f2798db00000000, 0xd4aca69100000000, + 0x8c2c444600000000, 0x25aa12e500000000, 0x7d2af03200000000, + 0xb3bc6f7100000000, 0xeb3c8da600000000, 0x42badb0500000000, + 0x1a3a39d200000000, 0x51b1079800000000, 0x0931e54f00000000, + 0xa0b7b3ec00000000, 0xf837513b00000000, 0x50d8119a00000000, + 0x0858f34d00000000, 0xa1dea5ee00000000, 0xf95e473900000000, + 0xb2d5797300000000, 0xea559ba400000000, 0x43d3cd0700000000, + 0x1b532fd000000000, 0xd5c5b09300000000, 0x8d45524400000000, + 0x24c304e700000000, 0x7c43e63000000000, 0x37c8d87a00000000, + 0x6f483aad00000000, 0xc6ce6c0e00000000, 0x9e4e8ed900000000, + 0x5ae3538900000000, 0x0263b15e00000000, 0xabe5e7fd00000000, + 0xf365052a00000000, 0xb8ee3b6000000000, 0xe06ed9b700000000, + 0x49e88f1400000000, 0x11686dc300000000, 0xdffef28000000000, + 0x877e105700000000, 0x2ef846f400000000, 0x7678a42300000000, + 0x3df39a6900000000, 0x657378be00000000, 0xccf52e1d00000000, + 0x9475ccca00000000, 0x44ae95bc00000000, 0x1c2e776b00000000, + 0xb5a821c800000000, 0xed28c31f00000000, 0xa6a3fd5500000000, + 0xfe231f8200000000, 0x57a5492100000000, 0x0f25abf600000000, + 0xc1b334b500000000, 0x9933d66200000000, 0x30b580c100000000, + 0x6835621600000000, 0x23be5c5c00000000, 0x7b3ebe8b00000000, + 0xd2b8e82800000000, 0x8a380aff00000000, 0x4e95d7af00000000, + 0x1615357800000000, 0xbf9363db00000000, 0xe713810c00000000, + 0xac98bf4600000000, 0xf4185d9100000000, 0x5d9e0b3200000000, + 0x051ee9e500000000, 0xcb8876a600000000, 0x9308947100000000, + 0x3a8ec2d200000000, 0x620e200500000000, 0x29851e4f00000000, + 0x7105fc9800000000, 0xd883aa3b00000000, 0x800348ec00000000, + 0x783419d700000000, 0x20b4fb0000000000, 0x8932ada300000000, + 0xd1b24f7400000000, 0x9a39713e00000000, 0xc2b993e900000000, + 0x6b3fc54a00000000, 0x33bf279d00000000, 0xfd29b8de00000000, + 0xa5a95a0900000000, 0x0c2f0caa00000000, 0x54afee7d00000000, + 0x1f24d03700000000, 0x47a432e000000000, 0xee22644300000000, + 0xb6a2869400000000, 0x720f5bc400000000, 0x2a8fb91300000000, + 0x8309efb000000000, 0xdb890d6700000000, 0x9002332d00000000, + 0xc882d1fa00000000, 0x6104875900000000, 0x3984658e00000000, + 0xf712facd00000000, 0xaf92181a00000000, 0x06144eb900000000, + 0x5e94ac6e00000000, 0x151f922400000000, 0x4d9f70f300000000, + 0xe419265000000000, 0xbc99c48700000000, 0x6c429df100000000, + 0x34c27f2600000000, 0x9d44298500000000, 0xc5c4cb5200000000, + 0x8e4ff51800000000, 0xd6cf17cf00000000, 0x7f49416c00000000, + 0x27c9a3bb00000000, 0xe95f3cf800000000, 0xb1dfde2f00000000, + 0x1859888c00000000, 0x40d96a5b00000000, 0x0b52541100000000, + 0x53d2b6c600000000, 0xfa54e06500000000, 0xa2d402b200000000, + 0x6679dfe200000000, 0x3ef93d3500000000, 0x977f6b9600000000, + 0xcfff894100000000, 0x8474b70b00000000, 0xdcf455dc00000000, + 0x7572037f00000000, 0x2df2e1a800000000, 0xe3647eeb00000000, + 0xbbe49c3c00000000, 0x1262ca9f00000000, 0x4ae2284800000000, + 0x0169160200000000, 0x59e9f4d500000000, 0xf06fa27600000000, + 0xa8ef40a100000000}, + {0x0000000000000000, 0x463b676500000000, 0x8c76ceca00000000, + 0xca4da9af00000000, 0x59ebed4e00000000, 0x1fd08a2b00000000, + 0xd59d238400000000, 0x93a644e100000000, 0xb2d6db9d00000000, + 0xf4edbcf800000000, 0x3ea0155700000000, 0x789b723200000000, + 0xeb3d36d300000000, 0xad0651b600000000, 0x674bf81900000000, + 0x21709f7c00000000, 0x25abc6e000000000, 0x6390a18500000000, + 0xa9dd082a00000000, 0xefe66f4f00000000, 0x7c402bae00000000, + 0x3a7b4ccb00000000, 0xf036e56400000000, 0xb60d820100000000, + 0x977d1d7d00000000, 0xd1467a1800000000, 0x1b0bd3b700000000, + 0x5d30b4d200000000, 0xce96f03300000000, 0x88ad975600000000, + 0x42e03ef900000000, 0x04db599c00000000, 0x0b50fc1a00000000, + 0x4d6b9b7f00000000, 0x872632d000000000, 0xc11d55b500000000, + 0x52bb115400000000, 0x1480763100000000, 0xdecddf9e00000000, + 0x98f6b8fb00000000, 0xb986278700000000, 0xffbd40e200000000, + 0x35f0e94d00000000, 0x73cb8e2800000000, 0xe06dcac900000000, + 0xa656adac00000000, 0x6c1b040300000000, 0x2a20636600000000, + 0x2efb3afa00000000, 0x68c05d9f00000000, 0xa28df43000000000, + 0xe4b6935500000000, 0x7710d7b400000000, 0x312bb0d100000000, + 0xfb66197e00000000, 0xbd5d7e1b00000000, 0x9c2de16700000000, + 0xda16860200000000, 0x105b2fad00000000, 0x566048c800000000, + 0xc5c60c2900000000, 0x83fd6b4c00000000, 0x49b0c2e300000000, + 0x0f8ba58600000000, 0x16a0f83500000000, 0x509b9f5000000000, + 0x9ad636ff00000000, 0xdced519a00000000, 0x4f4b157b00000000, + 0x0970721e00000000, 0xc33ddbb100000000, 0x8506bcd400000000, + 0xa47623a800000000, 0xe24d44cd00000000, 0x2800ed6200000000, + 0x6e3b8a0700000000, 0xfd9dcee600000000, 0xbba6a98300000000, + 0x71eb002c00000000, 0x37d0674900000000, 0x330b3ed500000000, + 0x753059b000000000, 0xbf7df01f00000000, 0xf946977a00000000, + 0x6ae0d39b00000000, 0x2cdbb4fe00000000, 0xe6961d5100000000, + 0xa0ad7a3400000000, 0x81dde54800000000, 0xc7e6822d00000000, + 0x0dab2b8200000000, 0x4b904ce700000000, 0xd836080600000000, + 0x9e0d6f6300000000, 0x5440c6cc00000000, 0x127ba1a900000000, + 0x1df0042f00000000, 0x5bcb634a00000000, 0x9186cae500000000, + 0xd7bdad8000000000, 0x441be96100000000, 0x02208e0400000000, + 0xc86d27ab00000000, 0x8e5640ce00000000, 0xaf26dfb200000000, + 0xe91db8d700000000, 0x2350117800000000, 0x656b761d00000000, + 0xf6cd32fc00000000, 0xb0f6559900000000, 0x7abbfc3600000000, + 0x3c809b5300000000, 0x385bc2cf00000000, 0x7e60a5aa00000000, + 0xb42d0c0500000000, 0xf2166b6000000000, 0x61b02f8100000000, + 0x278b48e400000000, 0xedc6e14b00000000, 0xabfd862e00000000, + 0x8a8d195200000000, 0xccb67e3700000000, 0x06fbd79800000000, + 0x40c0b0fd00000000, 0xd366f41c00000000, 0x955d937900000000, + 0x5f103ad600000000, 0x192b5db300000000, 0x2c40f16b00000000, + 0x6a7b960e00000000, 0xa0363fa100000000, 0xe60d58c400000000, + 0x75ab1c2500000000, 0x33907b4000000000, 0xf9ddd2ef00000000, + 0xbfe6b58a00000000, 0x9e962af600000000, 0xd8ad4d9300000000, + 0x12e0e43c00000000, 0x54db835900000000, 0xc77dc7b800000000, + 0x8146a0dd00000000, 0x4b0b097200000000, 0x0d306e1700000000, + 0x09eb378b00000000, 0x4fd050ee00000000, 0x859df94100000000, + 0xc3a69e2400000000, 0x5000dac500000000, 0x163bbda000000000, + 0xdc76140f00000000, 0x9a4d736a00000000, 0xbb3dec1600000000, + 0xfd068b7300000000, 0x374b22dc00000000, 0x717045b900000000, + 0xe2d6015800000000, 0xa4ed663d00000000, 0x6ea0cf9200000000, + 0x289ba8f700000000, 0x27100d7100000000, 0x612b6a1400000000, + 0xab66c3bb00000000, 0xed5da4de00000000, 0x7efbe03f00000000, + 0x38c0875a00000000, 0xf28d2ef500000000, 0xb4b6499000000000, + 0x95c6d6ec00000000, 0xd3fdb18900000000, 0x19b0182600000000, + 0x5f8b7f4300000000, 0xcc2d3ba200000000, 0x8a165cc700000000, + 0x405bf56800000000, 0x0660920d00000000, 0x02bbcb9100000000, + 0x4480acf400000000, 0x8ecd055b00000000, 0xc8f6623e00000000, + 0x5b5026df00000000, 0x1d6b41ba00000000, 0xd726e81500000000, + 0x911d8f7000000000, 0xb06d100c00000000, 0xf656776900000000, + 0x3c1bdec600000000, 0x7a20b9a300000000, 0xe986fd4200000000, + 0xafbd9a2700000000, 0x65f0338800000000, 0x23cb54ed00000000, + 0x3ae0095e00000000, 0x7cdb6e3b00000000, 0xb696c79400000000, + 0xf0ada0f100000000, 0x630be41000000000, 0x2530837500000000, + 0xef7d2ada00000000, 0xa9464dbf00000000, 0x8836d2c300000000, + 0xce0db5a600000000, 0x04401c0900000000, 0x427b7b6c00000000, + 0xd1dd3f8d00000000, 0x97e658e800000000, 0x5dabf14700000000, + 0x1b90962200000000, 0x1f4bcfbe00000000, 0x5970a8db00000000, + 0x933d017400000000, 0xd506661100000000, 0x46a022f000000000, + 0x009b459500000000, 0xcad6ec3a00000000, 0x8ced8b5f00000000, + 0xad9d142300000000, 0xeba6734600000000, 0x21ebdae900000000, + 0x67d0bd8c00000000, 0xf476f96d00000000, 0xb24d9e0800000000, + 0x780037a700000000, 0x3e3b50c200000000, 0x31b0f54400000000, + 0x778b922100000000, 0xbdc63b8e00000000, 0xfbfd5ceb00000000, + 0x685b180a00000000, 0x2e607f6f00000000, 0xe42dd6c000000000, + 0xa216b1a500000000, 0x83662ed900000000, 0xc55d49bc00000000, + 0x0f10e01300000000, 0x492b877600000000, 0xda8dc39700000000, + 0x9cb6a4f200000000, 0x56fb0d5d00000000, 0x10c06a3800000000, + 0x141b33a400000000, 0x522054c100000000, 0x986dfd6e00000000, + 0xde569a0b00000000, 0x4df0deea00000000, 0x0bcbb98f00000000, + 0xc186102000000000, 0x87bd774500000000, 0xa6cde83900000000, + 0xe0f68f5c00000000, 0x2abb26f300000000, 0x6c80419600000000, + 0xff26057700000000, 0xb91d621200000000, 0x7350cbbd00000000, + 0x356bacd800000000}, + {0x0000000000000000, 0x9e83da9f00000000, 0x7d01c4e400000000, + 0xe3821e7b00000000, 0xbb04f91200000000, 0x2587238d00000000, + 0xc6053df600000000, 0x5886e76900000000, 0x7609f22500000000, + 0xe88a28ba00000000, 0x0b0836c100000000, 0x958bec5e00000000, + 0xcd0d0b3700000000, 0x538ed1a800000000, 0xb00ccfd300000000, + 0x2e8f154c00000000, 0xec12e44b00000000, 0x72913ed400000000, + 0x911320af00000000, 0x0f90fa3000000000, 0x57161d5900000000, + 0xc995c7c600000000, 0x2a17d9bd00000000, 0xb494032200000000, + 0x9a1b166e00000000, 0x0498ccf100000000, 0xe71ad28a00000000, + 0x7999081500000000, 0x211fef7c00000000, 0xbf9c35e300000000, + 0x5c1e2b9800000000, 0xc29df10700000000, 0xd825c89700000000, + 0x46a6120800000000, 0xa5240c7300000000, 0x3ba7d6ec00000000, + 0x6321318500000000, 0xfda2eb1a00000000, 0x1e20f56100000000, + 0x80a32ffe00000000, 0xae2c3ab200000000, 0x30afe02d00000000, + 0xd32dfe5600000000, 0x4dae24c900000000, 0x1528c3a000000000, + 0x8bab193f00000000, 0x6829074400000000, 0xf6aadddb00000000, + 0x34372cdc00000000, 0xaab4f64300000000, 0x4936e83800000000, + 0xd7b532a700000000, 0x8f33d5ce00000000, 0x11b00f5100000000, + 0xf232112a00000000, 0x6cb1cbb500000000, 0x423edef900000000, + 0xdcbd046600000000, 0x3f3f1a1d00000000, 0xa1bcc08200000000, + 0xf93a27eb00000000, 0x67b9fd7400000000, 0x843be30f00000000, + 0x1ab8399000000000, 0xf14de1f400000000, 0x6fce3b6b00000000, + 0x8c4c251000000000, 0x12cfff8f00000000, 0x4a4918e600000000, + 0xd4cac27900000000, 0x3748dc0200000000, 0xa9cb069d00000000, + 0x874413d100000000, 0x19c7c94e00000000, 0xfa45d73500000000, + 0x64c60daa00000000, 0x3c40eac300000000, 0xa2c3305c00000000, + 0x41412e2700000000, 0xdfc2f4b800000000, 0x1d5f05bf00000000, + 0x83dcdf2000000000, 0x605ec15b00000000, 0xfedd1bc400000000, + 0xa65bfcad00000000, 0x38d8263200000000, 0xdb5a384900000000, + 0x45d9e2d600000000, 0x6b56f79a00000000, 0xf5d52d0500000000, + 0x1657337e00000000, 0x88d4e9e100000000, 0xd0520e8800000000, + 0x4ed1d41700000000, 0xad53ca6c00000000, 0x33d010f300000000, + 0x2968296300000000, 0xb7ebf3fc00000000, 0x5469ed8700000000, + 0xcaea371800000000, 0x926cd07100000000, 0x0cef0aee00000000, + 0xef6d149500000000, 0x71eece0a00000000, 0x5f61db4600000000, + 0xc1e201d900000000, 0x22601fa200000000, 0xbce3c53d00000000, + 0xe465225400000000, 0x7ae6f8cb00000000, 0x9964e6b000000000, + 0x07e73c2f00000000, 0xc57acd2800000000, 0x5bf917b700000000, + 0xb87b09cc00000000, 0x26f8d35300000000, 0x7e7e343a00000000, + 0xe0fdeea500000000, 0x037ff0de00000000, 0x9dfc2a4100000000, + 0xb3733f0d00000000, 0x2df0e59200000000, 0xce72fbe900000000, + 0x50f1217600000000, 0x0877c61f00000000, 0x96f41c8000000000, + 0x757602fb00000000, 0xebf5d86400000000, 0xa39db33200000000, + 0x3d1e69ad00000000, 0xde9c77d600000000, 0x401fad4900000000, + 0x18994a2000000000, 0x861a90bf00000000, 0x65988ec400000000, + 0xfb1b545b00000000, 0xd594411700000000, 0x4b179b8800000000, + 0xa89585f300000000, 0x36165f6c00000000, 0x6e90b80500000000, + 0xf013629a00000000, 0x13917ce100000000, 0x8d12a67e00000000, + 0x4f8f577900000000, 0xd10c8de600000000, 0x328e939d00000000, + 0xac0d490200000000, 0xf48bae6b00000000, 0x6a0874f400000000, + 0x898a6a8f00000000, 0x1709b01000000000, 0x3986a55c00000000, + 0xa7057fc300000000, 0x448761b800000000, 0xda04bb2700000000, + 0x82825c4e00000000, 0x1c0186d100000000, 0xff8398aa00000000, + 0x6100423500000000, 0x7bb87ba500000000, 0xe53ba13a00000000, + 0x06b9bf4100000000, 0x983a65de00000000, 0xc0bc82b700000000, + 0x5e3f582800000000, 0xbdbd465300000000, 0x233e9ccc00000000, + 0x0db1898000000000, 0x9332531f00000000, 0x70b04d6400000000, + 0xee3397fb00000000, 0xb6b5709200000000, 0x2836aa0d00000000, + 0xcbb4b47600000000, 0x55376ee900000000, 0x97aa9fee00000000, + 0x0929457100000000, 0xeaab5b0a00000000, 0x7428819500000000, + 0x2cae66fc00000000, 0xb22dbc6300000000, 0x51afa21800000000, + 0xcf2c788700000000, 0xe1a36dcb00000000, 0x7f20b75400000000, + 0x9ca2a92f00000000, 0x022173b000000000, 0x5aa794d900000000, + 0xc4244e4600000000, 0x27a6503d00000000, 0xb9258aa200000000, + 0x52d052c600000000, 0xcc53885900000000, 0x2fd1962200000000, + 0xb1524cbd00000000, 0xe9d4abd400000000, 0x7757714b00000000, + 0x94d56f3000000000, 0x0a56b5af00000000, 0x24d9a0e300000000, + 0xba5a7a7c00000000, 0x59d8640700000000, 0xc75bbe9800000000, + 0x9fdd59f100000000, 0x015e836e00000000, 0xe2dc9d1500000000, + 0x7c5f478a00000000, 0xbec2b68d00000000, 0x20416c1200000000, + 0xc3c3726900000000, 0x5d40a8f600000000, 0x05c64f9f00000000, + 0x9b45950000000000, 0x78c78b7b00000000, 0xe64451e400000000, + 0xc8cb44a800000000, 0x56489e3700000000, 0xb5ca804c00000000, + 0x2b495ad300000000, 0x73cfbdba00000000, 0xed4c672500000000, + 0x0ece795e00000000, 0x904da3c100000000, 0x8af59a5100000000, + 0x147640ce00000000, 0xf7f45eb500000000, 0x6977842a00000000, + 0x31f1634300000000, 0xaf72b9dc00000000, 0x4cf0a7a700000000, + 0xd2737d3800000000, 0xfcfc687400000000, 0x627fb2eb00000000, + 0x81fdac9000000000, 0x1f7e760f00000000, 0x47f8916600000000, + 0xd97b4bf900000000, 0x3af9558200000000, 0xa47a8f1d00000000, + 0x66e77e1a00000000, 0xf864a48500000000, 0x1be6bafe00000000, + 0x8565606100000000, 0xdde3870800000000, 0x43605d9700000000, + 0xa0e243ec00000000, 0x3e61997300000000, 0x10ee8c3f00000000, + 0x8e6d56a000000000, 0x6def48db00000000, 0xf36c924400000000, + 0xabea752d00000000, 0x3569afb200000000, 0xd6ebb1c900000000, + 0x48686b5600000000}, + {0x0000000000000000, 0xc064281700000000, 0x80c9502e00000000, + 0x40ad783900000000, 0x0093a15c00000000, 0xc0f7894b00000000, + 0x805af17200000000, 0x403ed96500000000, 0x002643b900000000, + 0xc0426bae00000000, 0x80ef139700000000, 0x408b3b8000000000, + 0x00b5e2e500000000, 0xc0d1caf200000000, 0x807cb2cb00000000, + 0x40189adc00000000, 0x414af7a900000000, 0x812edfbe00000000, + 0xc183a78700000000, 0x01e78f9000000000, 0x41d956f500000000, + 0x81bd7ee200000000, 0xc11006db00000000, 0x01742ecc00000000, + 0x416cb41000000000, 0x81089c0700000000, 0xc1a5e43e00000000, + 0x01c1cc2900000000, 0x41ff154c00000000, 0x819b3d5b00000000, + 0xc136456200000000, 0x01526d7500000000, 0xc3929f8800000000, + 0x03f6b79f00000000, 0x435bcfa600000000, 0x833fe7b100000000, + 0xc3013ed400000000, 0x036516c300000000, 0x43c86efa00000000, + 0x83ac46ed00000000, 0xc3b4dc3100000000, 0x03d0f42600000000, + 0x437d8c1f00000000, 0x8319a40800000000, 0xc3277d6d00000000, + 0x0343557a00000000, 0x43ee2d4300000000, 0x838a055400000000, + 0x82d8682100000000, 0x42bc403600000000, 0x0211380f00000000, + 0xc275101800000000, 0x824bc97d00000000, 0x422fe16a00000000, + 0x0282995300000000, 0xc2e6b14400000000, 0x82fe2b9800000000, + 0x429a038f00000000, 0x02377bb600000000, 0xc25353a100000000, + 0x826d8ac400000000, 0x4209a2d300000000, 0x02a4daea00000000, + 0xc2c0f2fd00000000, 0xc7234eca00000000, 0x074766dd00000000, + 0x47ea1ee400000000, 0x878e36f300000000, 0xc7b0ef9600000000, + 0x07d4c78100000000, 0x4779bfb800000000, 0x871d97af00000000, + 0xc7050d7300000000, 0x0761256400000000, 0x47cc5d5d00000000, + 0x87a8754a00000000, 0xc796ac2f00000000, 0x07f2843800000000, + 0x475ffc0100000000, 0x873bd41600000000, 0x8669b96300000000, + 0x460d917400000000, 0x06a0e94d00000000, 0xc6c4c15a00000000, + 0x86fa183f00000000, 0x469e302800000000, 0x0633481100000000, + 0xc657600600000000, 0x864ffada00000000, 0x462bd2cd00000000, + 0x0686aaf400000000, 0xc6e282e300000000, 0x86dc5b8600000000, + 0x46b8739100000000, 0x06150ba800000000, 0xc67123bf00000000, + 0x04b1d14200000000, 0xc4d5f95500000000, 0x8478816c00000000, + 0x441ca97b00000000, 0x0422701e00000000, 0xc446580900000000, + 0x84eb203000000000, 0x448f082700000000, 0x049792fb00000000, + 0xc4f3baec00000000, 0x845ec2d500000000, 0x443aeac200000000, + 0x040433a700000000, 0xc4601bb000000000, 0x84cd638900000000, + 0x44a94b9e00000000, 0x45fb26eb00000000, 0x859f0efc00000000, + 0xc53276c500000000, 0x05565ed200000000, 0x456887b700000000, + 0x850cafa000000000, 0xc5a1d79900000000, 0x05c5ff8e00000000, + 0x45dd655200000000, 0x85b94d4500000000, 0xc514357c00000000, + 0x05701d6b00000000, 0x454ec40e00000000, 0x852aec1900000000, + 0xc587942000000000, 0x05e3bc3700000000, 0xcf41ed4f00000000, + 0x0f25c55800000000, 0x4f88bd6100000000, 0x8fec957600000000, + 0xcfd24c1300000000, 0x0fb6640400000000, 0x4f1b1c3d00000000, + 0x8f7f342a00000000, 0xcf67aef600000000, 0x0f0386e100000000, + 0x4faefed800000000, 0x8fcad6cf00000000, 0xcff40faa00000000, + 0x0f9027bd00000000, 0x4f3d5f8400000000, 0x8f59779300000000, + 0x8e0b1ae600000000, 0x4e6f32f100000000, 0x0ec24ac800000000, + 0xcea662df00000000, 0x8e98bbba00000000, 0x4efc93ad00000000, + 0x0e51eb9400000000, 0xce35c38300000000, 0x8e2d595f00000000, + 0x4e49714800000000, 0x0ee4097100000000, 0xce80216600000000, + 0x8ebef80300000000, 0x4edad01400000000, 0x0e77a82d00000000, + 0xce13803a00000000, 0x0cd372c700000000, 0xccb75ad000000000, + 0x8c1a22e900000000, 0x4c7e0afe00000000, 0x0c40d39b00000000, + 0xcc24fb8c00000000, 0x8c8983b500000000, 0x4cedaba200000000, + 0x0cf5317e00000000, 0xcc91196900000000, 0x8c3c615000000000, + 0x4c58494700000000, 0x0c66902200000000, 0xcc02b83500000000, + 0x8cafc00c00000000, 0x4ccbe81b00000000, 0x4d99856e00000000, + 0x8dfdad7900000000, 0xcd50d54000000000, 0x0d34fd5700000000, + 0x4d0a243200000000, 0x8d6e0c2500000000, 0xcdc3741c00000000, + 0x0da75c0b00000000, 0x4dbfc6d700000000, 0x8ddbeec000000000, + 0xcd7696f900000000, 0x0d12beee00000000, 0x4d2c678b00000000, + 0x8d484f9c00000000, 0xcde537a500000000, 0x0d811fb200000000, + 0x0862a38500000000, 0xc8068b9200000000, 0x88abf3ab00000000, + 0x48cfdbbc00000000, 0x08f102d900000000, 0xc8952ace00000000, + 0x883852f700000000, 0x485c7ae000000000, 0x0844e03c00000000, + 0xc820c82b00000000, 0x888db01200000000, 0x48e9980500000000, + 0x08d7416000000000, 0xc8b3697700000000, 0x881e114e00000000, + 0x487a395900000000, 0x4928542c00000000, 0x894c7c3b00000000, + 0xc9e1040200000000, 0x09852c1500000000, 0x49bbf57000000000, + 0x89dfdd6700000000, 0xc972a55e00000000, 0x09168d4900000000, + 0x490e179500000000, 0x896a3f8200000000, 0xc9c747bb00000000, + 0x09a36fac00000000, 0x499db6c900000000, 0x89f99ede00000000, + 0xc954e6e700000000, 0x0930cef000000000, 0xcbf03c0d00000000, + 0x0b94141a00000000, 0x4b396c2300000000, 0x8b5d443400000000, + 0xcb639d5100000000, 0x0b07b54600000000, 0x4baacd7f00000000, + 0x8bcee56800000000, 0xcbd67fb400000000, 0x0bb257a300000000, + 0x4b1f2f9a00000000, 0x8b7b078d00000000, 0xcb45dee800000000, + 0x0b21f6ff00000000, 0x4b8c8ec600000000, 0x8be8a6d100000000, + 0x8abacba400000000, 0x4adee3b300000000, 0x0a739b8a00000000, + 0xca17b39d00000000, 0x8a296af800000000, 0x4a4d42ef00000000, + 0x0ae03ad600000000, 0xca8412c100000000, 0x8a9c881d00000000, + 0x4af8a00a00000000, 0x0a55d83300000000, 0xca31f02400000000, + 0x8a0f294100000000, 0x4a6b015600000000, 0x0ac6796f00000000, + 0xcaa2517800000000}, + {0x0000000000000000, 0xd4ea739b00000000, 0xe9d396ed00000000, + 0x3d39e57600000000, 0x93a15c0000000000, 0x474b2f9b00000000, + 0x7a72caed00000000, 0xae98b97600000000, 0x2643b90000000000, + 0xf2a9ca9b00000000, 0xcf902fed00000000, 0x1b7a5c7600000000, + 0xb5e2e50000000000, 0x6108969b00000000, 0x5c3173ed00000000, + 0x88db007600000000, 0x4c86720100000000, 0x986c019a00000000, + 0xa555e4ec00000000, 0x71bf977700000000, 0xdf272e0100000000, + 0x0bcd5d9a00000000, 0x36f4b8ec00000000, 0xe21ecb7700000000, + 0x6ac5cb0100000000, 0xbe2fb89a00000000, 0x83165dec00000000, + 0x57fc2e7700000000, 0xf964970100000000, 0x2d8ee49a00000000, + 0x10b701ec00000000, 0xc45d727700000000, 0x980ce50200000000, + 0x4ce6969900000000, 0x71df73ef00000000, 0xa535007400000000, + 0x0badb90200000000, 0xdf47ca9900000000, 0xe27e2fef00000000, + 0x36945c7400000000, 0xbe4f5c0200000000, 0x6aa52f9900000000, + 0x579ccaef00000000, 0x8376b97400000000, 0x2dee000200000000, + 0xf904739900000000, 0xc43d96ef00000000, 0x10d7e57400000000, + 0xd48a970300000000, 0x0060e49800000000, 0x3d5901ee00000000, + 0xe9b3727500000000, 0x472bcb0300000000, 0x93c1b89800000000, + 0xaef85dee00000000, 0x7a122e7500000000, 0xf2c92e0300000000, + 0x26235d9800000000, 0x1b1ab8ee00000000, 0xcff0cb7500000000, + 0x6168720300000000, 0xb582019800000000, 0x88bbe4ee00000000, + 0x5c51977500000000, 0x3019ca0500000000, 0xe4f3b99e00000000, + 0xd9ca5ce800000000, 0x0d202f7300000000, 0xa3b8960500000000, + 0x7752e59e00000000, 0x4a6b00e800000000, 0x9e81737300000000, + 0x165a730500000000, 0xc2b0009e00000000, 0xff89e5e800000000, + 0x2b63967300000000, 0x85fb2f0500000000, 0x51115c9e00000000, + 0x6c28b9e800000000, 0xb8c2ca7300000000, 0x7c9fb80400000000, + 0xa875cb9f00000000, 0x954c2ee900000000, 0x41a65d7200000000, + 0xef3ee40400000000, 0x3bd4979f00000000, 0x06ed72e900000000, + 0xd207017200000000, 0x5adc010400000000, 0x8e36729f00000000, + 0xb30f97e900000000, 0x67e5e47200000000, 0xc97d5d0400000000, + 0x1d972e9f00000000, 0x20aecbe900000000, 0xf444b87200000000, + 0xa8152f0700000000, 0x7cff5c9c00000000, 0x41c6b9ea00000000, + 0x952cca7100000000, 0x3bb4730700000000, 0xef5e009c00000000, + 0xd267e5ea00000000, 0x068d967100000000, 0x8e56960700000000, + 0x5abce59c00000000, 0x678500ea00000000, 0xb36f737100000000, + 0x1df7ca0700000000, 0xc91db99c00000000, 0xf4245cea00000000, + 0x20ce2f7100000000, 0xe4935d0600000000, 0x30792e9d00000000, + 0x0d40cbeb00000000, 0xd9aab87000000000, 0x7732010600000000, + 0xa3d8729d00000000, 0x9ee197eb00000000, 0x4a0be47000000000, + 0xc2d0e40600000000, 0x163a979d00000000, 0x2b0372eb00000000, + 0xffe9017000000000, 0x5171b80600000000, 0x859bcb9d00000000, + 0xb8a22eeb00000000, 0x6c485d7000000000, 0x6032940b00000000, + 0xb4d8e79000000000, 0x89e102e600000000, 0x5d0b717d00000000, + 0xf393c80b00000000, 0x2779bb9000000000, 0x1a405ee600000000, + 0xceaa2d7d00000000, 0x46712d0b00000000, 0x929b5e9000000000, + 0xafa2bbe600000000, 0x7b48c87d00000000, 0xd5d0710b00000000, + 0x013a029000000000, 0x3c03e7e600000000, 0xe8e9947d00000000, + 0x2cb4e60a00000000, 0xf85e959100000000, 0xc56770e700000000, + 0x118d037c00000000, 0xbf15ba0a00000000, 0x6bffc99100000000, + 0x56c62ce700000000, 0x822c5f7c00000000, 0x0af75f0a00000000, + 0xde1d2c9100000000, 0xe324c9e700000000, 0x37ceba7c00000000, + 0x9956030a00000000, 0x4dbc709100000000, 0x708595e700000000, + 0xa46fe67c00000000, 0xf83e710900000000, 0x2cd4029200000000, + 0x11ede7e400000000, 0xc507947f00000000, 0x6b9f2d0900000000, + 0xbf755e9200000000, 0x824cbbe400000000, 0x56a6c87f00000000, + 0xde7dc80900000000, 0x0a97bb9200000000, 0x37ae5ee400000000, + 0xe3442d7f00000000, 0x4ddc940900000000, 0x9936e79200000000, + 0xa40f02e400000000, 0x70e5717f00000000, 0xb4b8030800000000, + 0x6052709300000000, 0x5d6b95e500000000, 0x8981e67e00000000, + 0x27195f0800000000, 0xf3f32c9300000000, 0xcecac9e500000000, + 0x1a20ba7e00000000, 0x92fbba0800000000, 0x4611c99300000000, + 0x7b282ce500000000, 0xafc25f7e00000000, 0x015ae60800000000, + 0xd5b0959300000000, 0xe88970e500000000, 0x3c63037e00000000, + 0x502b5e0e00000000, 0x84c12d9500000000, 0xb9f8c8e300000000, + 0x6d12bb7800000000, 0xc38a020e00000000, 0x1760719500000000, + 0x2a5994e300000000, 0xfeb3e77800000000, 0x7668e70e00000000, + 0xa282949500000000, 0x9fbb71e300000000, 0x4b51027800000000, + 0xe5c9bb0e00000000, 0x3123c89500000000, 0x0c1a2de300000000, + 0xd8f05e7800000000, 0x1cad2c0f00000000, 0xc8475f9400000000, + 0xf57ebae200000000, 0x2194c97900000000, 0x8f0c700f00000000, + 0x5be6039400000000, 0x66dfe6e200000000, 0xb235957900000000, + 0x3aee950f00000000, 0xee04e69400000000, 0xd33d03e200000000, + 0x07d7707900000000, 0xa94fc90f00000000, 0x7da5ba9400000000, + 0x409c5fe200000000, 0x94762c7900000000, 0xc827bb0c00000000, + 0x1ccdc89700000000, 0x21f42de100000000, 0xf51e5e7a00000000, + 0x5b86e70c00000000, 0x8f6c949700000000, 0xb25571e100000000, + 0x66bf027a00000000, 0xee64020c00000000, 0x3a8e719700000000, + 0x07b794e100000000, 0xd35de77a00000000, 0x7dc55e0c00000000, + 0xa92f2d9700000000, 0x9416c8e100000000, 0x40fcbb7a00000000, + 0x84a1c90d00000000, 0x504bba9600000000, 0x6d725fe000000000, + 0xb9982c7b00000000, 0x1700950d00000000, 0xc3eae69600000000, + 0xfed303e000000000, 0x2a39707b00000000, 0xa2e2700d00000000, + 0x7608039600000000, 0x4b31e6e000000000, 0x9fdb957b00000000, + 0x31432c0d00000000, 0xe5a95f9600000000, 0xd890bae000000000, + 0x0c7ac97b00000000}, + {0x0000000000000000, 0x2765258100000000, 0x0fcc3bd900000000, + 0x28a91e5800000000, 0x5f9e066900000000, 0x78fb23e800000000, + 0x50523db000000000, 0x7737183100000000, 0xbe3c0dd200000000, + 0x9959285300000000, 0xb1f0360b00000000, 0x9695138a00000000, + 0xe1a20bbb00000000, 0xc6c72e3a00000000, 0xee6e306200000000, + 0xc90b15e300000000, 0x3d7f6b7f00000000, 0x1a1a4efe00000000, + 0x32b350a600000000, 0x15d6752700000000, 0x62e16d1600000000, + 0x4584489700000000, 0x6d2d56cf00000000, 0x4a48734e00000000, + 0x834366ad00000000, 0xa426432c00000000, 0x8c8f5d7400000000, + 0xabea78f500000000, 0xdcdd60c400000000, 0xfbb8454500000000, + 0xd3115b1d00000000, 0xf4747e9c00000000, 0x7afed6fe00000000, + 0x5d9bf37f00000000, 0x7532ed2700000000, 0x5257c8a600000000, + 0x2560d09700000000, 0x0205f51600000000, 0x2aaceb4e00000000, + 0x0dc9cecf00000000, 0xc4c2db2c00000000, 0xe3a7fead00000000, + 0xcb0ee0f500000000, 0xec6bc57400000000, 0x9b5cdd4500000000, + 0xbc39f8c400000000, 0x9490e69c00000000, 0xb3f5c31d00000000, + 0x4781bd8100000000, 0x60e4980000000000, 0x484d865800000000, + 0x6f28a3d900000000, 0x181fbbe800000000, 0x3f7a9e6900000000, + 0x17d3803100000000, 0x30b6a5b000000000, 0xf9bdb05300000000, + 0xded895d200000000, 0xf6718b8a00000000, 0xd114ae0b00000000, + 0xa623b63a00000000, 0x814693bb00000000, 0xa9ef8de300000000, + 0x8e8aa86200000000, 0xb5fadc2600000000, 0x929ff9a700000000, + 0xba36e7ff00000000, 0x9d53c27e00000000, 0xea64da4f00000000, + 0xcd01ffce00000000, 0xe5a8e19600000000, 0xc2cdc41700000000, + 0x0bc6d1f400000000, 0x2ca3f47500000000, 0x040aea2d00000000, + 0x236fcfac00000000, 0x5458d79d00000000, 0x733df21c00000000, + 0x5b94ec4400000000, 0x7cf1c9c500000000, 0x8885b75900000000, + 0xafe092d800000000, 0x87498c8000000000, 0xa02ca90100000000, + 0xd71bb13000000000, 0xf07e94b100000000, 0xd8d78ae900000000, + 0xffb2af6800000000, 0x36b9ba8b00000000, 0x11dc9f0a00000000, + 0x3975815200000000, 0x1e10a4d300000000, 0x6927bce200000000, + 0x4e42996300000000, 0x66eb873b00000000, 0x418ea2ba00000000, + 0xcf040ad800000000, 0xe8612f5900000000, 0xc0c8310100000000, + 0xe7ad148000000000, 0x909a0cb100000000, 0xb7ff293000000000, + 0x9f56376800000000, 0xb83312e900000000, 0x7138070a00000000, + 0x565d228b00000000, 0x7ef43cd300000000, 0x5991195200000000, + 0x2ea6016300000000, 0x09c324e200000000, 0x216a3aba00000000, + 0x060f1f3b00000000, 0xf27b61a700000000, 0xd51e442600000000, + 0xfdb75a7e00000000, 0xdad27fff00000000, 0xade567ce00000000, + 0x8a80424f00000000, 0xa2295c1700000000, 0x854c799600000000, + 0x4c476c7500000000, 0x6b2249f400000000, 0x438b57ac00000000, + 0x64ee722d00000000, 0x13d96a1c00000000, 0x34bc4f9d00000000, + 0x1c1551c500000000, 0x3b70744400000000, 0x6af5b94d00000000, + 0x4d909ccc00000000, 0x6539829400000000, 0x425ca71500000000, + 0x356bbf2400000000, 0x120e9aa500000000, 0x3aa784fd00000000, + 0x1dc2a17c00000000, 0xd4c9b49f00000000, 0xf3ac911e00000000, + 0xdb058f4600000000, 0xfc60aac700000000, 0x8b57b2f600000000, + 0xac32977700000000, 0x849b892f00000000, 0xa3feacae00000000, + 0x578ad23200000000, 0x70eff7b300000000, 0x5846e9eb00000000, + 0x7f23cc6a00000000, 0x0814d45b00000000, 0x2f71f1da00000000, + 0x07d8ef8200000000, 0x20bdca0300000000, 0xe9b6dfe000000000, + 0xced3fa6100000000, 0xe67ae43900000000, 0xc11fc1b800000000, + 0xb628d98900000000, 0x914dfc0800000000, 0xb9e4e25000000000, + 0x9e81c7d100000000, 0x100b6fb300000000, 0x376e4a3200000000, + 0x1fc7546a00000000, 0x38a271eb00000000, 0x4f9569da00000000, + 0x68f04c5b00000000, 0x4059520300000000, 0x673c778200000000, + 0xae37626100000000, 0x895247e000000000, 0xa1fb59b800000000, + 0x869e7c3900000000, 0xf1a9640800000000, 0xd6cc418900000000, + 0xfe655fd100000000, 0xd9007a5000000000, 0x2d7404cc00000000, + 0x0a11214d00000000, 0x22b83f1500000000, 0x05dd1a9400000000, + 0x72ea02a500000000, 0x558f272400000000, 0x7d26397c00000000, + 0x5a431cfd00000000, 0x9348091e00000000, 0xb42d2c9f00000000, + 0x9c8432c700000000, 0xbbe1174600000000, 0xccd60f7700000000, + 0xebb32af600000000, 0xc31a34ae00000000, 0xe47f112f00000000, + 0xdf0f656b00000000, 0xf86a40ea00000000, 0xd0c35eb200000000, + 0xf7a67b3300000000, 0x8091630200000000, 0xa7f4468300000000, + 0x8f5d58db00000000, 0xa8387d5a00000000, 0x613368b900000000, + 0x46564d3800000000, 0x6eff536000000000, 0x499a76e100000000, + 0x3ead6ed000000000, 0x19c84b5100000000, 0x3161550900000000, + 0x1604708800000000, 0xe2700e1400000000, 0xc5152b9500000000, + 0xedbc35cd00000000, 0xcad9104c00000000, 0xbdee087d00000000, + 0x9a8b2dfc00000000, 0xb22233a400000000, 0x9547162500000000, + 0x5c4c03c600000000, 0x7b29264700000000, 0x5380381f00000000, + 0x74e51d9e00000000, 0x03d205af00000000, 0x24b7202e00000000, + 0x0c1e3e7600000000, 0x2b7b1bf700000000, 0xa5f1b39500000000, + 0x8294961400000000, 0xaa3d884c00000000, 0x8d58adcd00000000, + 0xfa6fb5fc00000000, 0xdd0a907d00000000, 0xf5a38e2500000000, + 0xd2c6aba400000000, 0x1bcdbe4700000000, 0x3ca89bc600000000, + 0x1401859e00000000, 0x3364a01f00000000, 0x4453b82e00000000, + 0x63369daf00000000, 0x4b9f83f700000000, 0x6cfaa67600000000, + 0x988ed8ea00000000, 0xbfebfd6b00000000, 0x9742e33300000000, + 0xb027c6b200000000, 0xc710de8300000000, 0xe075fb0200000000, + 0xc8dce55a00000000, 0xefb9c0db00000000, 0x26b2d53800000000, + 0x01d7f0b900000000, 0x297eeee100000000, 0x0e1bcb6000000000, + 0x792cd35100000000, 0x5e49f6d000000000, 0x76e0e88800000000, + 0x5185cd0900000000}}; + +#else /* W == 4 */ + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0x9ba54c6f, 0xec3b9e9f, 0x779ed2f0, 0x03063b7f, + 0x98a37710, 0xef3da5e0, 0x7498e98f, 0x060c76fe, 0x9da93a91, + 0xea37e861, 0x7192a40e, 0x050a4d81, 0x9eaf01ee, 0xe931d31e, + 0x72949f71, 0x0c18edfc, 0x97bda193, 0xe0237363, 0x7b863f0c, + 0x0f1ed683, 0x94bb9aec, 0xe325481c, 0x78800473, 0x0a149b02, + 0x91b1d76d, 0xe62f059d, 0x7d8a49f2, 0x0912a07d, 0x92b7ec12, + 0xe5293ee2, 0x7e8c728d, 0x1831dbf8, 0x83949797, 0xf40a4567, + 0x6faf0908, 0x1b37e087, 0x8092ace8, 0xf70c7e18, 0x6ca93277, + 0x1e3dad06, 0x8598e169, 0xf2063399, 0x69a37ff6, 0x1d3b9679, + 0x869eda16, 0xf10008e6, 0x6aa54489, 0x14293604, 0x8f8c7a6b, + 0xf812a89b, 0x63b7e4f4, 0x172f0d7b, 0x8c8a4114, 0xfb1493e4, + 0x60b1df8b, 0x122540fa, 0x89800c95, 0xfe1ede65, 0x65bb920a, + 0x11237b85, 0x8a8637ea, 0xfd18e51a, 0x66bda975, 0x3063b7f0, + 0xabc6fb9f, 0xdc58296f, 0x47fd6500, 0x33658c8f, 0xa8c0c0e0, + 0xdf5e1210, 0x44fb5e7f, 0x366fc10e, 0xadca8d61, 0xda545f91, + 0x41f113fe, 0x3569fa71, 0xaeccb61e, 0xd95264ee, 0x42f72881, + 0x3c7b5a0c, 0xa7de1663, 0xd040c493, 0x4be588fc, 0x3f7d6173, + 0xa4d82d1c, 0xd346ffec, 0x48e3b383, 0x3a772cf2, 0xa1d2609d, + 0xd64cb26d, 0x4de9fe02, 0x3971178d, 0xa2d45be2, 0xd54a8912, + 0x4eefc57d, 0x28526c08, 0xb3f72067, 0xc469f297, 0x5fccbef8, + 0x2b545777, 0xb0f11b18, 0xc76fc9e8, 0x5cca8587, 0x2e5e1af6, + 0xb5fb5699, 0xc2658469, 0x59c0c806, 0x2d582189, 0xb6fd6de6, + 0xc163bf16, 0x5ac6f379, 0x244a81f4, 0xbfefcd9b, 0xc8711f6b, + 0x53d45304, 0x274cba8b, 0xbce9f6e4, 0xcb772414, 0x50d2687b, + 0x2246f70a, 0xb9e3bb65, 0xce7d6995, 0x55d825fa, 0x2140cc75, + 0xbae5801a, 0xcd7b52ea, 0x56de1e85, 0x60c76fe0, 0xfb62238f, + 0x8cfcf17f, 0x1759bd10, 0x63c1549f, 0xf86418f0, 0x8ffaca00, + 0x145f866f, 0x66cb191e, 0xfd6e5571, 0x8af08781, 0x1155cbee, + 0x65cd2261, 0xfe686e0e, 0x89f6bcfe, 0x1253f091, 0x6cdf821c, + 0xf77ace73, 0x80e41c83, 0x1b4150ec, 0x6fd9b963, 0xf47cf50c, + 0x83e227fc, 0x18476b93, 0x6ad3f4e2, 0xf176b88d, 0x86e86a7d, + 0x1d4d2612, 0x69d5cf9d, 0xf27083f2, 0x85ee5102, 0x1e4b1d6d, + 0x78f6b418, 0xe353f877, 0x94cd2a87, 0x0f6866e8, 0x7bf08f67, + 0xe055c308, 0x97cb11f8, 0x0c6e5d97, 0x7efac2e6, 0xe55f8e89, + 0x92c15c79, 0x09641016, 0x7dfcf999, 0xe659b5f6, 0x91c76706, + 0x0a622b69, 0x74ee59e4, 0xef4b158b, 0x98d5c77b, 0x03708b14, + 0x77e8629b, 0xec4d2ef4, 0x9bd3fc04, 0x0076b06b, 0x72e22f1a, + 0xe9476375, 0x9ed9b185, 0x057cfdea, 0x71e41465, 0xea41580a, + 0x9ddf8afa, 0x067ac695, 0x50a4d810, 0xcb01947f, 0xbc9f468f, + 0x273a0ae0, 0x53a2e36f, 0xc807af00, 0xbf997df0, 0x243c319f, + 0x56a8aeee, 0xcd0de281, 0xba933071, 0x21367c1e, 0x55ae9591, + 0xce0bd9fe, 0xb9950b0e, 0x22304761, 0x5cbc35ec, 0xc7197983, + 0xb087ab73, 0x2b22e71c, 0x5fba0e93, 0xc41f42fc, 0xb381900c, + 0x2824dc63, 0x5ab04312, 0xc1150f7d, 0xb68bdd8d, 0x2d2e91e2, + 0x59b6786d, 0xc2133402, 0xb58de6f2, 0x2e28aa9d, 0x489503e8, + 0xd3304f87, 0xa4ae9d77, 0x3f0bd118, 0x4b933897, 0xd03674f8, + 0xa7a8a608, 0x3c0dea67, 0x4e997516, 0xd53c3979, 0xa2a2eb89, + 0x3907a7e6, 0x4d9f4e69, 0xd63a0206, 0xa1a4d0f6, 0x3a019c99, + 0x448dee14, 0xdf28a27b, 0xa8b6708b, 0x33133ce4, 0x478bd56b, + 0xdc2e9904, 0xabb04bf4, 0x3015079b, 0x428198ea, 0xd924d485, + 0xaeba0675, 0x351f4a1a, 0x4187a395, 0xda22effa, 0xadbc3d0a, + 0x36197165}, + {0x00000000, 0xc18edfc0, 0x586cb9c1, 0x99e26601, 0xb0d97382, + 0x7157ac42, 0xe8b5ca43, 0x293b1583, 0xbac3e145, 0x7b4d3e85, + 0xe2af5884, 0x23218744, 0x0a1a92c7, 0xcb944d07, 0x52762b06, + 0x93f8f4c6, 0xaef6c4cb, 0x6f781b0b, 0xf69a7d0a, 0x3714a2ca, + 0x1e2fb749, 0xdfa16889, 0x46430e88, 0x87cdd148, 0x1435258e, + 0xd5bbfa4e, 0x4c599c4f, 0x8dd7438f, 0xa4ec560c, 0x656289cc, + 0xfc80efcd, 0x3d0e300d, 0x869c8fd7, 0x47125017, 0xdef03616, + 0x1f7ee9d6, 0x3645fc55, 0xf7cb2395, 0x6e294594, 0xafa79a54, + 0x3c5f6e92, 0xfdd1b152, 0x6433d753, 0xa5bd0893, 0x8c861d10, + 0x4d08c2d0, 0xd4eaa4d1, 0x15647b11, 0x286a4b1c, 0xe9e494dc, + 0x7006f2dd, 0xb1882d1d, 0x98b3389e, 0x593de75e, 0xc0df815f, + 0x01515e9f, 0x92a9aa59, 0x53277599, 0xcac51398, 0x0b4bcc58, + 0x2270d9db, 0xe3fe061b, 0x7a1c601a, 0xbb92bfda, 0xd64819ef, + 0x17c6c62f, 0x8e24a02e, 0x4faa7fee, 0x66916a6d, 0xa71fb5ad, + 0x3efdd3ac, 0xff730c6c, 0x6c8bf8aa, 0xad05276a, 0x34e7416b, + 0xf5699eab, 0xdc528b28, 0x1ddc54e8, 0x843e32e9, 0x45b0ed29, + 0x78bedd24, 0xb93002e4, 0x20d264e5, 0xe15cbb25, 0xc867aea6, + 0x09e97166, 0x900b1767, 0x5185c8a7, 0xc27d3c61, 0x03f3e3a1, + 0x9a1185a0, 0x5b9f5a60, 0x72a44fe3, 0xb32a9023, 0x2ac8f622, + 0xeb4629e2, 0x50d49638, 0x915a49f8, 0x08b82ff9, 0xc936f039, + 0xe00de5ba, 0x21833a7a, 0xb8615c7b, 0x79ef83bb, 0xea17777d, + 0x2b99a8bd, 0xb27bcebc, 0x73f5117c, 0x5ace04ff, 0x9b40db3f, + 0x02a2bd3e, 0xc32c62fe, 0xfe2252f3, 0x3fac8d33, 0xa64eeb32, + 0x67c034f2, 0x4efb2171, 0x8f75feb1, 0x169798b0, 0xd7194770, + 0x44e1b3b6, 0x856f6c76, 0x1c8d0a77, 0xdd03d5b7, 0xf438c034, + 0x35b61ff4, 0xac5479f5, 0x6ddaa635, 0x77e1359f, 0xb66fea5f, + 0x2f8d8c5e, 0xee03539e, 0xc738461d, 0x06b699dd, 0x9f54ffdc, + 0x5eda201c, 0xcd22d4da, 0x0cac0b1a, 0x954e6d1b, 0x54c0b2db, + 0x7dfba758, 0xbc757898, 0x25971e99, 0xe419c159, 0xd917f154, + 0x18992e94, 0x817b4895, 0x40f59755, 0x69ce82d6, 0xa8405d16, + 0x31a23b17, 0xf02ce4d7, 0x63d41011, 0xa25acfd1, 0x3bb8a9d0, + 0xfa367610, 0xd30d6393, 0x1283bc53, 0x8b61da52, 0x4aef0592, + 0xf17dba48, 0x30f36588, 0xa9110389, 0x689fdc49, 0x41a4c9ca, + 0x802a160a, 0x19c8700b, 0xd846afcb, 0x4bbe5b0d, 0x8a3084cd, + 0x13d2e2cc, 0xd25c3d0c, 0xfb67288f, 0x3ae9f74f, 0xa30b914e, + 0x62854e8e, 0x5f8b7e83, 0x9e05a143, 0x07e7c742, 0xc6691882, + 0xef520d01, 0x2edcd2c1, 0xb73eb4c0, 0x76b06b00, 0xe5489fc6, + 0x24c64006, 0xbd242607, 0x7caaf9c7, 0x5591ec44, 0x941f3384, + 0x0dfd5585, 0xcc738a45, 0xa1a92c70, 0x6027f3b0, 0xf9c595b1, + 0x384b4a71, 0x11705ff2, 0xd0fe8032, 0x491ce633, 0x889239f3, + 0x1b6acd35, 0xdae412f5, 0x430674f4, 0x8288ab34, 0xabb3beb7, + 0x6a3d6177, 0xf3df0776, 0x3251d8b6, 0x0f5fe8bb, 0xced1377b, + 0x5733517a, 0x96bd8eba, 0xbf869b39, 0x7e0844f9, 0xe7ea22f8, + 0x2664fd38, 0xb59c09fe, 0x7412d63e, 0xedf0b03f, 0x2c7e6fff, + 0x05457a7c, 0xc4cba5bc, 0x5d29c3bd, 0x9ca71c7d, 0x2735a3a7, + 0xe6bb7c67, 0x7f591a66, 0xbed7c5a6, 0x97ecd025, 0x56620fe5, + 0xcf8069e4, 0x0e0eb624, 0x9df642e2, 0x5c789d22, 0xc59afb23, + 0x041424e3, 0x2d2f3160, 0xeca1eea0, 0x754388a1, 0xb4cd5761, + 0x89c3676c, 0x484db8ac, 0xd1afdead, 0x1021016d, 0x391a14ee, + 0xf894cb2e, 0x6176ad2f, 0xa0f872ef, 0x33008629, 0xf28e59e9, + 0x6b6c3fe8, 0xaae2e028, 0x83d9f5ab, 0x42572a6b, 0xdbb54c6a, + 0x1a3b93aa}, + {0x00000000, 0xefc26b3e, 0x04f5d03d, 0xeb37bb03, 0x09eba07a, + 0xe629cb44, 0x0d1e7047, 0xe2dc1b79, 0x13d740f4, 0xfc152bca, + 0x172290c9, 0xf8e0fbf7, 0x1a3ce08e, 0xf5fe8bb0, 0x1ec930b3, + 0xf10b5b8d, 0x27ae81e8, 0xc86cead6, 0x235b51d5, 0xcc993aeb, + 0x2e452192, 0xc1874aac, 0x2ab0f1af, 0xc5729a91, 0x3479c11c, + 0xdbbbaa22, 0x308c1121, 0xdf4e7a1f, 0x3d926166, 0xd2500a58, + 0x3967b15b, 0xd6a5da65, 0x4f5d03d0, 0xa09f68ee, 0x4ba8d3ed, + 0xa46ab8d3, 0x46b6a3aa, 0xa974c894, 0x42437397, 0xad8118a9, + 0x5c8a4324, 0xb348281a, 0x587f9319, 0xb7bdf827, 0x5561e35e, + 0xbaa38860, 0x51943363, 0xbe56585d, 0x68f38238, 0x8731e906, + 0x6c065205, 0x83c4393b, 0x61182242, 0x8eda497c, 0x65edf27f, + 0x8a2f9941, 0x7b24c2cc, 0x94e6a9f2, 0x7fd112f1, 0x901379cf, + 0x72cf62b6, 0x9d0d0988, 0x763ab28b, 0x99f8d9b5, 0x9eba07a0, + 0x71786c9e, 0x9a4fd79d, 0x758dbca3, 0x9751a7da, 0x7893cce4, + 0x93a477e7, 0x7c661cd9, 0x8d6d4754, 0x62af2c6a, 0x89989769, + 0x665afc57, 0x8486e72e, 0x6b448c10, 0x80733713, 0x6fb15c2d, + 0xb9148648, 0x56d6ed76, 0xbde15675, 0x52233d4b, 0xb0ff2632, + 0x5f3d4d0c, 0xb40af60f, 0x5bc89d31, 0xaac3c6bc, 0x4501ad82, + 0xae361681, 0x41f47dbf, 0xa32866c6, 0x4cea0df8, 0xa7ddb6fb, + 0x481fddc5, 0xd1e70470, 0x3e256f4e, 0xd512d44d, 0x3ad0bf73, + 0xd80ca40a, 0x37cecf34, 0xdcf97437, 0x333b1f09, 0xc2304484, + 0x2df22fba, 0xc6c594b9, 0x2907ff87, 0xcbdbe4fe, 0x24198fc0, + 0xcf2e34c3, 0x20ec5ffd, 0xf6498598, 0x198beea6, 0xf2bc55a5, + 0x1d7e3e9b, 0xffa225e2, 0x10604edc, 0xfb57f5df, 0x14959ee1, + 0xe59ec56c, 0x0a5cae52, 0xe16b1551, 0x0ea97e6f, 0xec756516, + 0x03b70e28, 0xe880b52b, 0x0742de15, 0xe6050901, 0x09c7623f, + 0xe2f0d93c, 0x0d32b202, 0xefeea97b, 0x002cc245, 0xeb1b7946, + 0x04d91278, 0xf5d249f5, 0x1a1022cb, 0xf12799c8, 0x1ee5f2f6, + 0xfc39e98f, 0x13fb82b1, 0xf8cc39b2, 0x170e528c, 0xc1ab88e9, + 0x2e69e3d7, 0xc55e58d4, 0x2a9c33ea, 0xc8402893, 0x278243ad, + 0xccb5f8ae, 0x23779390, 0xd27cc81d, 0x3dbea323, 0xd6891820, + 0x394b731e, 0xdb976867, 0x34550359, 0xdf62b85a, 0x30a0d364, + 0xa9580ad1, 0x469a61ef, 0xadaddaec, 0x426fb1d2, 0xa0b3aaab, + 0x4f71c195, 0xa4467a96, 0x4b8411a8, 0xba8f4a25, 0x554d211b, + 0xbe7a9a18, 0x51b8f126, 0xb364ea5f, 0x5ca68161, 0xb7913a62, + 0x5853515c, 0x8ef68b39, 0x6134e007, 0x8a035b04, 0x65c1303a, + 0x871d2b43, 0x68df407d, 0x83e8fb7e, 0x6c2a9040, 0x9d21cbcd, + 0x72e3a0f3, 0x99d41bf0, 0x761670ce, 0x94ca6bb7, 0x7b080089, + 0x903fbb8a, 0x7ffdd0b4, 0x78bf0ea1, 0x977d659f, 0x7c4ade9c, + 0x9388b5a2, 0x7154aedb, 0x9e96c5e5, 0x75a17ee6, 0x9a6315d8, + 0x6b684e55, 0x84aa256b, 0x6f9d9e68, 0x805ff556, 0x6283ee2f, + 0x8d418511, 0x66763e12, 0x89b4552c, 0x5f118f49, 0xb0d3e477, + 0x5be45f74, 0xb426344a, 0x56fa2f33, 0xb938440d, 0x520fff0e, + 0xbdcd9430, 0x4cc6cfbd, 0xa304a483, 0x48331f80, 0xa7f174be, + 0x452d6fc7, 0xaaef04f9, 0x41d8bffa, 0xae1ad4c4, 0x37e20d71, + 0xd820664f, 0x3317dd4c, 0xdcd5b672, 0x3e09ad0b, 0xd1cbc635, + 0x3afc7d36, 0xd53e1608, 0x24354d85, 0xcbf726bb, 0x20c09db8, + 0xcf02f686, 0x2ddeedff, 0xc21c86c1, 0x292b3dc2, 0xc6e956fc, + 0x104c8c99, 0xff8ee7a7, 0x14b95ca4, 0xfb7b379a, 0x19a72ce3, + 0xf66547dd, 0x1d52fcde, 0xf29097e0, 0x039bcc6d, 0xec59a753, + 0x076e1c50, 0xe8ac776e, 0x0a706c17, 0xe5b20729, 0x0e85bc2a, + 0xe147d714}, + {0x00000000, 0x177b1443, 0x2ef62886, 0x398d3cc5, 0x5dec510c, + 0x4a97454f, 0x731a798a, 0x64616dc9, 0xbbd8a218, 0xaca3b65b, + 0x952e8a9e, 0x82559edd, 0xe634f314, 0xf14fe757, 0xc8c2db92, + 0xdfb9cfd1, 0xacc04271, 0xbbbb5632, 0x82366af7, 0x954d7eb4, + 0xf12c137d, 0xe657073e, 0xdfda3bfb, 0xc8a12fb8, 0x1718e069, + 0x0063f42a, 0x39eec8ef, 0x2e95dcac, 0x4af4b165, 0x5d8fa526, + 0x640299e3, 0x73798da0, 0x82f182a3, 0x958a96e0, 0xac07aa25, + 0xbb7cbe66, 0xdf1dd3af, 0xc866c7ec, 0xf1ebfb29, 0xe690ef6a, + 0x392920bb, 0x2e5234f8, 0x17df083d, 0x00a41c7e, 0x64c571b7, + 0x73be65f4, 0x4a335931, 0x5d484d72, 0x2e31c0d2, 0x394ad491, + 0x00c7e854, 0x17bcfc17, 0x73dd91de, 0x64a6859d, 0x5d2bb958, + 0x4a50ad1b, 0x95e962ca, 0x82927689, 0xbb1f4a4c, 0xac645e0f, + 0xc80533c6, 0xdf7e2785, 0xe6f31b40, 0xf1880f03, 0xde920307, + 0xc9e91744, 0xf0642b81, 0xe71f3fc2, 0x837e520b, 0x94054648, + 0xad887a8d, 0xbaf36ece, 0x654aa11f, 0x7231b55c, 0x4bbc8999, + 0x5cc79dda, 0x38a6f013, 0x2fdde450, 0x1650d895, 0x012bccd6, + 0x72524176, 0x65295535, 0x5ca469f0, 0x4bdf7db3, 0x2fbe107a, + 0x38c50439, 0x014838fc, 0x16332cbf, 0xc98ae36e, 0xdef1f72d, + 0xe77ccbe8, 0xf007dfab, 0x9466b262, 0x831da621, 0xba909ae4, + 0xadeb8ea7, 0x5c6381a4, 0x4b1895e7, 0x7295a922, 0x65eebd61, + 0x018fd0a8, 0x16f4c4eb, 0x2f79f82e, 0x3802ec6d, 0xe7bb23bc, + 0xf0c037ff, 0xc94d0b3a, 0xde361f79, 0xba5772b0, 0xad2c66f3, + 0x94a15a36, 0x83da4e75, 0xf0a3c3d5, 0xe7d8d796, 0xde55eb53, + 0xc92eff10, 0xad4f92d9, 0xba34869a, 0x83b9ba5f, 0x94c2ae1c, + 0x4b7b61cd, 0x5c00758e, 0x658d494b, 0x72f65d08, 0x169730c1, + 0x01ec2482, 0x38611847, 0x2f1a0c04, 0x6655004f, 0x712e140c, + 0x48a328c9, 0x5fd83c8a, 0x3bb95143, 0x2cc24500, 0x154f79c5, + 0x02346d86, 0xdd8da257, 0xcaf6b614, 0xf37b8ad1, 0xe4009e92, + 0x8061f35b, 0x971ae718, 0xae97dbdd, 0xb9eccf9e, 0xca95423e, + 0xddee567d, 0xe4636ab8, 0xf3187efb, 0x97791332, 0x80020771, + 0xb98f3bb4, 0xaef42ff7, 0x714de026, 0x6636f465, 0x5fbbc8a0, + 0x48c0dce3, 0x2ca1b12a, 0x3bdaa569, 0x025799ac, 0x152c8def, + 0xe4a482ec, 0xf3df96af, 0xca52aa6a, 0xdd29be29, 0xb948d3e0, + 0xae33c7a3, 0x97befb66, 0x80c5ef25, 0x5f7c20f4, 0x480734b7, + 0x718a0872, 0x66f11c31, 0x029071f8, 0x15eb65bb, 0x2c66597e, + 0x3b1d4d3d, 0x4864c09d, 0x5f1fd4de, 0x6692e81b, 0x71e9fc58, + 0x15889191, 0x02f385d2, 0x3b7eb917, 0x2c05ad54, 0xf3bc6285, + 0xe4c776c6, 0xdd4a4a03, 0xca315e40, 0xae503389, 0xb92b27ca, + 0x80a61b0f, 0x97dd0f4c, 0xb8c70348, 0xafbc170b, 0x96312bce, + 0x814a3f8d, 0xe52b5244, 0xf2504607, 0xcbdd7ac2, 0xdca66e81, + 0x031fa150, 0x1464b513, 0x2de989d6, 0x3a929d95, 0x5ef3f05c, + 0x4988e41f, 0x7005d8da, 0x677ecc99, 0x14074139, 0x037c557a, + 0x3af169bf, 0x2d8a7dfc, 0x49eb1035, 0x5e900476, 0x671d38b3, + 0x70662cf0, 0xafdfe321, 0xb8a4f762, 0x8129cba7, 0x9652dfe4, + 0xf233b22d, 0xe548a66e, 0xdcc59aab, 0xcbbe8ee8, 0x3a3681eb, + 0x2d4d95a8, 0x14c0a96d, 0x03bbbd2e, 0x67dad0e7, 0x70a1c4a4, + 0x492cf861, 0x5e57ec22, 0x81ee23f3, 0x969537b0, 0xaf180b75, + 0xb8631f36, 0xdc0272ff, 0xcb7966bc, 0xf2f45a79, 0xe58f4e3a, + 0x96f6c39a, 0x818dd7d9, 0xb800eb1c, 0xaf7bff5f, 0xcb1a9296, + 0xdc6186d5, 0xe5ecba10, 0xf297ae53, 0x2d2e6182, 0x3a5575c1, + 0x03d84904, 0x14a35d47, 0x70c2308e, 0x67b924cd, 0x5e341808, + 0x494f0c4b}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x00000000, 0x43147b17, 0x8628f62e, 0xc53c8d39, 0x0c51ec5d, + 0x4f45974a, 0x8a791a73, 0xc96d6164, 0x18a2d8bb, 0x5bb6a3ac, + 0x9e8a2e95, 0xdd9e5582, 0x14f334e6, 0x57e74ff1, 0x92dbc2c8, + 0xd1cfb9df, 0x7142c0ac, 0x3256bbbb, 0xf76a3682, 0xb47e4d95, + 0x7d132cf1, 0x3e0757e6, 0xfb3bdadf, 0xb82fa1c8, 0x69e01817, + 0x2af46300, 0xefc8ee39, 0xacdc952e, 0x65b1f44a, 0x26a58f5d, + 0xe3990264, 0xa08d7973, 0xa382f182, 0xe0968a95, 0x25aa07ac, + 0x66be7cbb, 0xafd31ddf, 0xecc766c8, 0x29fbebf1, 0x6aef90e6, + 0xbb202939, 0xf834522e, 0x3d08df17, 0x7e1ca400, 0xb771c564, + 0xf465be73, 0x3159334a, 0x724d485d, 0xd2c0312e, 0x91d44a39, + 0x54e8c700, 0x17fcbc17, 0xde91dd73, 0x9d85a664, 0x58b92b5d, + 0x1bad504a, 0xca62e995, 0x89769282, 0x4c4a1fbb, 0x0f5e64ac, + 0xc63305c8, 0x85277edf, 0x401bf3e6, 0x030f88f1, 0x070392de, + 0x4417e9c9, 0x812b64f0, 0xc23f1fe7, 0x0b527e83, 0x48460594, + 0x8d7a88ad, 0xce6ef3ba, 0x1fa14a65, 0x5cb53172, 0x9989bc4b, + 0xda9dc75c, 0x13f0a638, 0x50e4dd2f, 0x95d85016, 0xd6cc2b01, + 0x76415272, 0x35552965, 0xf069a45c, 0xb37ddf4b, 0x7a10be2f, + 0x3904c538, 0xfc384801, 0xbf2c3316, 0x6ee38ac9, 0x2df7f1de, + 0xe8cb7ce7, 0xabdf07f0, 0x62b26694, 0x21a61d83, 0xe49a90ba, + 0xa78eebad, 0xa481635c, 0xe795184b, 0x22a99572, 0x61bdee65, + 0xa8d08f01, 0xebc4f416, 0x2ef8792f, 0x6dec0238, 0xbc23bbe7, + 0xff37c0f0, 0x3a0b4dc9, 0x791f36de, 0xb07257ba, 0xf3662cad, + 0x365aa194, 0x754eda83, 0xd5c3a3f0, 0x96d7d8e7, 0x53eb55de, + 0x10ff2ec9, 0xd9924fad, 0x9a8634ba, 0x5fbab983, 0x1caec294, + 0xcd617b4b, 0x8e75005c, 0x4b498d65, 0x085df672, 0xc1309716, + 0x8224ec01, 0x47186138, 0x040c1a2f, 0x4f005566, 0x0c142e71, + 0xc928a348, 0x8a3cd85f, 0x4351b93b, 0x0045c22c, 0xc5794f15, + 0x866d3402, 0x57a28ddd, 0x14b6f6ca, 0xd18a7bf3, 0x929e00e4, + 0x5bf36180, 0x18e71a97, 0xdddb97ae, 0x9ecfecb9, 0x3e4295ca, + 0x7d56eedd, 0xb86a63e4, 0xfb7e18f3, 0x32137997, 0x71070280, + 0xb43b8fb9, 0xf72ff4ae, 0x26e04d71, 0x65f43666, 0xa0c8bb5f, + 0xe3dcc048, 0x2ab1a12c, 0x69a5da3b, 0xac995702, 0xef8d2c15, + 0xec82a4e4, 0xaf96dff3, 0x6aaa52ca, 0x29be29dd, 0xe0d348b9, + 0xa3c733ae, 0x66fbbe97, 0x25efc580, 0xf4207c5f, 0xb7340748, + 0x72088a71, 0x311cf166, 0xf8719002, 0xbb65eb15, 0x7e59662c, + 0x3d4d1d3b, 0x9dc06448, 0xded41f5f, 0x1be89266, 0x58fce971, + 0x91918815, 0xd285f302, 0x17b97e3b, 0x54ad052c, 0x8562bcf3, + 0xc676c7e4, 0x034a4add, 0x405e31ca, 0x893350ae, 0xca272bb9, + 0x0f1ba680, 0x4c0fdd97, 0x4803c7b8, 0x0b17bcaf, 0xce2b3196, + 0x8d3f4a81, 0x44522be5, 0x074650f2, 0xc27addcb, 0x816ea6dc, + 0x50a11f03, 0x13b56414, 0xd689e92d, 0x959d923a, 0x5cf0f35e, + 0x1fe48849, 0xdad80570, 0x99cc7e67, 0x39410714, 0x7a557c03, + 0xbf69f13a, 0xfc7d8a2d, 0x3510eb49, 0x7604905e, 0xb3381d67, + 0xf02c6670, 0x21e3dfaf, 0x62f7a4b8, 0xa7cb2981, 0xe4df5296, + 0x2db233f2, 0x6ea648e5, 0xab9ac5dc, 0xe88ebecb, 0xeb81363a, + 0xa8954d2d, 0x6da9c014, 0x2ebdbb03, 0xe7d0da67, 0xa4c4a170, + 0x61f82c49, 0x22ec575e, 0xf323ee81, 0xb0379596, 0x750b18af, + 0x361f63b8, 0xff7202dc, 0xbc6679cb, 0x795af4f2, 0x3a4e8fe5, + 0x9ac3f696, 0xd9d78d81, 0x1ceb00b8, 0x5fff7baf, 0x96921acb, + 0xd58661dc, 0x10baece5, 0x53ae97f2, 0x82612e2d, 0xc175553a, + 0x0449d803, 0x475da314, 0x8e30c270, 0xcd24b967, 0x0818345e, + 0x4b0c4f49}, + {0x00000000, 0x3e6bc2ef, 0x3dd0f504, 0x03bb37eb, 0x7aa0eb09, + 0x44cb29e6, 0x47701e0d, 0x791bdce2, 0xf440d713, 0xca2b15fc, + 0xc9902217, 0xf7fbe0f8, 0x8ee03c1a, 0xb08bfef5, 0xb330c91e, + 0x8d5b0bf1, 0xe881ae27, 0xd6ea6cc8, 0xd5515b23, 0xeb3a99cc, + 0x9221452e, 0xac4a87c1, 0xaff1b02a, 0x919a72c5, 0x1cc17934, + 0x22aabbdb, 0x21118c30, 0x1f7a4edf, 0x6661923d, 0x580a50d2, + 0x5bb16739, 0x65daa5d6, 0xd0035d4f, 0xee689fa0, 0xedd3a84b, + 0xd3b86aa4, 0xaaa3b646, 0x94c874a9, 0x97734342, 0xa91881ad, + 0x24438a5c, 0x1a2848b3, 0x19937f58, 0x27f8bdb7, 0x5ee36155, + 0x6088a3ba, 0x63339451, 0x5d5856be, 0x3882f368, 0x06e93187, + 0x0552066c, 0x3b39c483, 0x42221861, 0x7c49da8e, 0x7ff2ed65, + 0x41992f8a, 0xccc2247b, 0xf2a9e694, 0xf112d17f, 0xcf791390, + 0xb662cf72, 0x88090d9d, 0x8bb23a76, 0xb5d9f899, 0xa007ba9e, + 0x9e6c7871, 0x9dd74f9a, 0xa3bc8d75, 0xdaa75197, 0xe4cc9378, + 0xe777a493, 0xd91c667c, 0x54476d8d, 0x6a2caf62, 0x69979889, + 0x57fc5a66, 0x2ee78684, 0x108c446b, 0x13377380, 0x2d5cb16f, + 0x488614b9, 0x76edd656, 0x7556e1bd, 0x4b3d2352, 0x3226ffb0, + 0x0c4d3d5f, 0x0ff60ab4, 0x319dc85b, 0xbcc6c3aa, 0x82ad0145, + 0x811636ae, 0xbf7df441, 0xc66628a3, 0xf80dea4c, 0xfbb6dda7, + 0xc5dd1f48, 0x7004e7d1, 0x4e6f253e, 0x4dd412d5, 0x73bfd03a, + 0x0aa40cd8, 0x34cfce37, 0x3774f9dc, 0x091f3b33, 0x844430c2, + 0xba2ff22d, 0xb994c5c6, 0x87ff0729, 0xfee4dbcb, 0xc08f1924, + 0xc3342ecf, 0xfd5fec20, 0x988549f6, 0xa6ee8b19, 0xa555bcf2, + 0x9b3e7e1d, 0xe225a2ff, 0xdc4e6010, 0xdff557fb, 0xe19e9514, + 0x6cc59ee5, 0x52ae5c0a, 0x51156be1, 0x6f7ea90e, 0x166575ec, + 0x280eb703, 0x2bb580e8, 0x15de4207, 0x010905e6, 0x3f62c709, + 0x3cd9f0e2, 0x02b2320d, 0x7ba9eeef, 0x45c22c00, 0x46791beb, + 0x7812d904, 0xf549d2f5, 0xcb22101a, 0xc89927f1, 0xf6f2e51e, + 0x8fe939fc, 0xb182fb13, 0xb239ccf8, 0x8c520e17, 0xe988abc1, + 0xd7e3692e, 0xd4585ec5, 0xea339c2a, 0x932840c8, 0xad438227, + 0xaef8b5cc, 0x90937723, 0x1dc87cd2, 0x23a3be3d, 0x201889d6, + 0x1e734b39, 0x676897db, 0x59035534, 0x5ab862df, 0x64d3a030, + 0xd10a58a9, 0xef619a46, 0xecdaadad, 0xd2b16f42, 0xabaab3a0, + 0x95c1714f, 0x967a46a4, 0xa811844b, 0x254a8fba, 0x1b214d55, + 0x189a7abe, 0x26f1b851, 0x5fea64b3, 0x6181a65c, 0x623a91b7, + 0x5c515358, 0x398bf68e, 0x07e03461, 0x045b038a, 0x3a30c165, + 0x432b1d87, 0x7d40df68, 0x7efbe883, 0x40902a6c, 0xcdcb219d, + 0xf3a0e372, 0xf01bd499, 0xce701676, 0xb76bca94, 0x8900087b, + 0x8abb3f90, 0xb4d0fd7f, 0xa10ebf78, 0x9f657d97, 0x9cde4a7c, + 0xa2b58893, 0xdbae5471, 0xe5c5969e, 0xe67ea175, 0xd815639a, + 0x554e686b, 0x6b25aa84, 0x689e9d6f, 0x56f55f80, 0x2fee8362, + 0x1185418d, 0x123e7666, 0x2c55b489, 0x498f115f, 0x77e4d3b0, + 0x745fe45b, 0x4a3426b4, 0x332ffa56, 0x0d4438b9, 0x0eff0f52, + 0x3094cdbd, 0xbdcfc64c, 0x83a404a3, 0x801f3348, 0xbe74f1a7, + 0xc76f2d45, 0xf904efaa, 0xfabfd841, 0xc4d41aae, 0x710de237, + 0x4f6620d8, 0x4cdd1733, 0x72b6d5dc, 0x0bad093e, 0x35c6cbd1, + 0x367dfc3a, 0x08163ed5, 0x854d3524, 0xbb26f7cb, 0xb89dc020, + 0x86f602cf, 0xffedde2d, 0xc1861cc2, 0xc23d2b29, 0xfc56e9c6, + 0x998c4c10, 0xa7e78eff, 0xa45cb914, 0x9a377bfb, 0xe32ca719, + 0xdd4765f6, 0xdefc521d, 0xe09790f2, 0x6dcc9b03, 0x53a759ec, + 0x501c6e07, 0x6e77ace8, 0x176c700a, 0x2907b2e5, 0x2abc850e, + 0x14d747e1}, + {0x00000000, 0xc0df8ec1, 0xc1b96c58, 0x0166e299, 0x8273d9b0, + 0x42ac5771, 0x43cab5e8, 0x83153b29, 0x45e1c3ba, 0x853e4d7b, + 0x8458afe2, 0x44872123, 0xc7921a0a, 0x074d94cb, 0x062b7652, + 0xc6f4f893, 0xcbc4f6ae, 0x0b1b786f, 0x0a7d9af6, 0xcaa21437, + 0x49b72f1e, 0x8968a1df, 0x880e4346, 0x48d1cd87, 0x8e253514, + 0x4efabbd5, 0x4f9c594c, 0x8f43d78d, 0x0c56eca4, 0xcc896265, + 0xcdef80fc, 0x0d300e3d, 0xd78f9c86, 0x17501247, 0x1636f0de, + 0xd6e97e1f, 0x55fc4536, 0x9523cbf7, 0x9445296e, 0x549aa7af, + 0x926e5f3c, 0x52b1d1fd, 0x53d73364, 0x9308bda5, 0x101d868c, + 0xd0c2084d, 0xd1a4ead4, 0x117b6415, 0x1c4b6a28, 0xdc94e4e9, + 0xddf20670, 0x1d2d88b1, 0x9e38b398, 0x5ee73d59, 0x5f81dfc0, + 0x9f5e5101, 0x59aaa992, 0x99752753, 0x9813c5ca, 0x58cc4b0b, + 0xdbd97022, 0x1b06fee3, 0x1a601c7a, 0xdabf92bb, 0xef1948d6, + 0x2fc6c617, 0x2ea0248e, 0xee7faa4f, 0x6d6a9166, 0xadb51fa7, + 0xacd3fd3e, 0x6c0c73ff, 0xaaf88b6c, 0x6a2705ad, 0x6b41e734, + 0xab9e69f5, 0x288b52dc, 0xe854dc1d, 0xe9323e84, 0x29edb045, + 0x24ddbe78, 0xe40230b9, 0xe564d220, 0x25bb5ce1, 0xa6ae67c8, + 0x6671e909, 0x67170b90, 0xa7c88551, 0x613c7dc2, 0xa1e3f303, + 0xa085119a, 0x605a9f5b, 0xe34fa472, 0x23902ab3, 0x22f6c82a, + 0xe22946eb, 0x3896d450, 0xf8495a91, 0xf92fb808, 0x39f036c9, + 0xbae50de0, 0x7a3a8321, 0x7b5c61b8, 0xbb83ef79, 0x7d7717ea, + 0xbda8992b, 0xbcce7bb2, 0x7c11f573, 0xff04ce5a, 0x3fdb409b, + 0x3ebda202, 0xfe622cc3, 0xf35222fe, 0x338dac3f, 0x32eb4ea6, + 0xf234c067, 0x7121fb4e, 0xb1fe758f, 0xb0989716, 0x704719d7, + 0xb6b3e144, 0x766c6f85, 0x770a8d1c, 0xb7d503dd, 0x34c038f4, + 0xf41fb635, 0xf57954ac, 0x35a6da6d, 0x9f35e177, 0x5fea6fb6, + 0x5e8c8d2f, 0x9e5303ee, 0x1d4638c7, 0xdd99b606, 0xdcff549f, + 0x1c20da5e, 0xdad422cd, 0x1a0bac0c, 0x1b6d4e95, 0xdbb2c054, + 0x58a7fb7d, 0x987875bc, 0x991e9725, 0x59c119e4, 0x54f117d9, + 0x942e9918, 0x95487b81, 0x5597f540, 0xd682ce69, 0x165d40a8, + 0x173ba231, 0xd7e42cf0, 0x1110d463, 0xd1cf5aa2, 0xd0a9b83b, + 0x107636fa, 0x93630dd3, 0x53bc8312, 0x52da618b, 0x9205ef4a, + 0x48ba7df1, 0x8865f330, 0x890311a9, 0x49dc9f68, 0xcac9a441, + 0x0a162a80, 0x0b70c819, 0xcbaf46d8, 0x0d5bbe4b, 0xcd84308a, + 0xcce2d213, 0x0c3d5cd2, 0x8f2867fb, 0x4ff7e93a, 0x4e910ba3, + 0x8e4e8562, 0x837e8b5f, 0x43a1059e, 0x42c7e707, 0x821869c6, + 0x010d52ef, 0xc1d2dc2e, 0xc0b43eb7, 0x006bb076, 0xc69f48e5, + 0x0640c624, 0x072624bd, 0xc7f9aa7c, 0x44ec9155, 0x84331f94, + 0x8555fd0d, 0x458a73cc, 0x702ca9a1, 0xb0f32760, 0xb195c5f9, + 0x714a4b38, 0xf25f7011, 0x3280fed0, 0x33e61c49, 0xf3399288, + 0x35cd6a1b, 0xf512e4da, 0xf4740643, 0x34ab8882, 0xb7beb3ab, + 0x77613d6a, 0x7607dff3, 0xb6d85132, 0xbbe85f0f, 0x7b37d1ce, + 0x7a513357, 0xba8ebd96, 0x399b86bf, 0xf944087e, 0xf822eae7, + 0x38fd6426, 0xfe099cb5, 0x3ed61274, 0x3fb0f0ed, 0xff6f7e2c, + 0x7c7a4505, 0xbca5cbc4, 0xbdc3295d, 0x7d1ca79c, 0xa7a33527, + 0x677cbbe6, 0x661a597f, 0xa6c5d7be, 0x25d0ec97, 0xe50f6256, + 0xe46980cf, 0x24b60e0e, 0xe242f69d, 0x229d785c, 0x23fb9ac5, + 0xe3241404, 0x60312f2d, 0xa0eea1ec, 0xa1884375, 0x6157cdb4, + 0x6c67c389, 0xacb84d48, 0xaddeafd1, 0x6d012110, 0xee141a39, + 0x2ecb94f8, 0x2fad7661, 0xef72f8a0, 0x29860033, 0xe9598ef2, + 0xe83f6c6b, 0x28e0e2aa, 0xabf5d983, 0x6b2a5742, 0x6a4cb5db, + 0xaa933b1a}, + {0x00000000, 0x6f4ca59b, 0x9f9e3bec, 0xf0d29e77, 0x7f3b0603, + 0x1077a398, 0xe0a53def, 0x8fe99874, 0xfe760c06, 0x913aa99d, + 0x61e837ea, 0x0ea49271, 0x814d0a05, 0xee01af9e, 0x1ed331e9, + 0x719f9472, 0xfced180c, 0x93a1bd97, 0x637323e0, 0x0c3f867b, + 0x83d61e0f, 0xec9abb94, 0x1c4825e3, 0x73048078, 0x029b140a, + 0x6dd7b191, 0x9d052fe6, 0xf2498a7d, 0x7da01209, 0x12ecb792, + 0xe23e29e5, 0x8d728c7e, 0xf8db3118, 0x97979483, 0x67450af4, + 0x0809af6f, 0x87e0371b, 0xe8ac9280, 0x187e0cf7, 0x7732a96c, + 0x06ad3d1e, 0x69e19885, 0x993306f2, 0xf67fa369, 0x79963b1d, + 0x16da9e86, 0xe60800f1, 0x8944a56a, 0x04362914, 0x6b7a8c8f, + 0x9ba812f8, 0xf4e4b763, 0x7b0d2f17, 0x14418a8c, 0xe49314fb, + 0x8bdfb160, 0xfa402512, 0x950c8089, 0x65de1efe, 0x0a92bb65, + 0x857b2311, 0xea37868a, 0x1ae518fd, 0x75a9bd66, 0xf0b76330, + 0x9ffbc6ab, 0x6f2958dc, 0x0065fd47, 0x8f8c6533, 0xe0c0c0a8, + 0x10125edf, 0x7f5efb44, 0x0ec16f36, 0x618dcaad, 0x915f54da, + 0xfe13f141, 0x71fa6935, 0x1eb6ccae, 0xee6452d9, 0x8128f742, + 0x0c5a7b3c, 0x6316dea7, 0x93c440d0, 0xfc88e54b, 0x73617d3f, + 0x1c2dd8a4, 0xecff46d3, 0x83b3e348, 0xf22c773a, 0x9d60d2a1, + 0x6db24cd6, 0x02fee94d, 0x8d177139, 0xe25bd4a2, 0x12894ad5, + 0x7dc5ef4e, 0x086c5228, 0x6720f7b3, 0x97f269c4, 0xf8becc5f, + 0x7757542b, 0x181bf1b0, 0xe8c96fc7, 0x8785ca5c, 0xf61a5e2e, + 0x9956fbb5, 0x698465c2, 0x06c8c059, 0x8921582d, 0xe66dfdb6, + 0x16bf63c1, 0x79f3c65a, 0xf4814a24, 0x9bcdefbf, 0x6b1f71c8, + 0x0453d453, 0x8bba4c27, 0xe4f6e9bc, 0x142477cb, 0x7b68d250, + 0x0af74622, 0x65bbe3b9, 0x95697dce, 0xfa25d855, 0x75cc4021, + 0x1a80e5ba, 0xea527bcd, 0x851ede56, 0xe06fc760, 0x8f2362fb, + 0x7ff1fc8c, 0x10bd5917, 0x9f54c163, 0xf01864f8, 0x00cafa8f, + 0x6f865f14, 0x1e19cb66, 0x71556efd, 0x8187f08a, 0xeecb5511, + 0x6122cd65, 0x0e6e68fe, 0xfebcf689, 0x91f05312, 0x1c82df6c, + 0x73ce7af7, 0x831ce480, 0xec50411b, 0x63b9d96f, 0x0cf57cf4, + 0xfc27e283, 0x936b4718, 0xe2f4d36a, 0x8db876f1, 0x7d6ae886, + 0x12264d1d, 0x9dcfd569, 0xf28370f2, 0x0251ee85, 0x6d1d4b1e, + 0x18b4f678, 0x77f853e3, 0x872acd94, 0xe866680f, 0x678ff07b, + 0x08c355e0, 0xf811cb97, 0x975d6e0c, 0xe6c2fa7e, 0x898e5fe5, + 0x795cc192, 0x16106409, 0x99f9fc7d, 0xf6b559e6, 0x0667c791, + 0x692b620a, 0xe459ee74, 0x8b154bef, 0x7bc7d598, 0x148b7003, + 0x9b62e877, 0xf42e4dec, 0x04fcd39b, 0x6bb07600, 0x1a2fe272, + 0x756347e9, 0x85b1d99e, 0xeafd7c05, 0x6514e471, 0x0a5841ea, + 0xfa8adf9d, 0x95c67a06, 0x10d8a450, 0x7f9401cb, 0x8f469fbc, + 0xe00a3a27, 0x6fe3a253, 0x00af07c8, 0xf07d99bf, 0x9f313c24, + 0xeeaea856, 0x81e20dcd, 0x713093ba, 0x1e7c3621, 0x9195ae55, + 0xfed90bce, 0x0e0b95b9, 0x61473022, 0xec35bc5c, 0x837919c7, + 0x73ab87b0, 0x1ce7222b, 0x930eba5f, 0xfc421fc4, 0x0c9081b3, + 0x63dc2428, 0x1243b05a, 0x7d0f15c1, 0x8ddd8bb6, 0xe2912e2d, + 0x6d78b659, 0x023413c2, 0xf2e68db5, 0x9daa282e, 0xe8039548, + 0x874f30d3, 0x779daea4, 0x18d10b3f, 0x9738934b, 0xf87436d0, + 0x08a6a8a7, 0x67ea0d3c, 0x1675994e, 0x79393cd5, 0x89eba2a2, + 0xe6a70739, 0x694e9f4d, 0x06023ad6, 0xf6d0a4a1, 0x999c013a, + 0x14ee8d44, 0x7ba228df, 0x8b70b6a8, 0xe43c1333, 0x6bd58b47, + 0x04992edc, 0xf44bb0ab, 0x9b071530, 0xea988142, 0x85d424d9, + 0x7506baae, 0x1a4a1f35, 0x95a38741, 0xfaef22da, 0x0a3dbcad, + 0x65711936}}; + +#endif + +#endif + +#if N == 4 + +#if W == 8 + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0xf1da05aa, 0x38c50d15, 0xc91f08bf, 0x718a1a2a, + 0x80501f80, 0x494f173f, 0xb8951295, 0xe3143454, 0x12ce31fe, + 0xdbd13941, 0x2a0b3ceb, 0x929e2e7e, 0x63442bd4, 0xaa5b236b, + 0x5b8126c1, 0x1d596ee9, 0xec836b43, 0x259c63fc, 0xd4466656, + 0x6cd374c3, 0x9d097169, 0x541679d6, 0xa5cc7c7c, 0xfe4d5abd, + 0x0f975f17, 0xc68857a8, 0x37525202, 0x8fc74097, 0x7e1d453d, + 0xb7024d82, 0x46d84828, 0x3ab2ddd2, 0xcb68d878, 0x0277d0c7, + 0xf3add56d, 0x4b38c7f8, 0xbae2c252, 0x73fdcaed, 0x8227cf47, + 0xd9a6e986, 0x287cec2c, 0xe163e493, 0x10b9e139, 0xa82cf3ac, + 0x59f6f606, 0x90e9feb9, 0x6133fb13, 0x27ebb33b, 0xd631b691, + 0x1f2ebe2e, 0xeef4bb84, 0x5661a911, 0xa7bbacbb, 0x6ea4a404, + 0x9f7ea1ae, 0xc4ff876f, 0x352582c5, 0xfc3a8a7a, 0x0de08fd0, + 0xb5759d45, 0x44af98ef, 0x8db09050, 0x7c6a95fa, 0x7565bba4, + 0x84bfbe0e, 0x4da0b6b1, 0xbc7ab31b, 0x04efa18e, 0xf535a424, + 0x3c2aac9b, 0xcdf0a931, 0x96718ff0, 0x67ab8a5a, 0xaeb482e5, + 0x5f6e874f, 0xe7fb95da, 0x16219070, 0xdf3e98cf, 0x2ee49d65, + 0x683cd54d, 0x99e6d0e7, 0x50f9d858, 0xa123ddf2, 0x19b6cf67, + 0xe86ccacd, 0x2173c272, 0xd0a9c7d8, 0x8b28e119, 0x7af2e4b3, + 0xb3edec0c, 0x4237e9a6, 0xfaa2fb33, 0x0b78fe99, 0xc267f626, + 0x33bdf38c, 0x4fd76676, 0xbe0d63dc, 0x77126b63, 0x86c86ec9, + 0x3e5d7c5c, 0xcf8779f6, 0x06987149, 0xf74274e3, 0xacc35222, + 0x5d195788, 0x94065f37, 0x65dc5a9d, 0xdd494808, 0x2c934da2, + 0xe58c451d, 0x145640b7, 0x528e089f, 0xa3540d35, 0x6a4b058a, + 0x9b910020, 0x230412b5, 0xd2de171f, 0x1bc11fa0, 0xea1b1a0a, + 0xb19a3ccb, 0x40403961, 0x895f31de, 0x78853474, 0xc01026e1, + 0x31ca234b, 0xf8d52bf4, 0x090f2e5e, 0xeacb7748, 0x1b1172e2, + 0xd20e7a5d, 0x23d47ff7, 0x9b416d62, 0x6a9b68c8, 0xa3846077, + 0x525e65dd, 0x09df431c, 0xf80546b6, 0x311a4e09, 0xc0c04ba3, + 0x78555936, 0x898f5c9c, 0x40905423, 0xb14a5189, 0xf79219a1, + 0x06481c0b, 0xcf5714b4, 0x3e8d111e, 0x8618038b, 0x77c20621, + 0xbedd0e9e, 0x4f070b34, 0x14862df5, 0xe55c285f, 0x2c4320e0, + 0xdd99254a, 0x650c37df, 0x94d63275, 0x5dc93aca, 0xac133f60, + 0xd079aa9a, 0x21a3af30, 0xe8bca78f, 0x1966a225, 0xa1f3b0b0, + 0x5029b51a, 0x9936bda5, 0x68ecb80f, 0x336d9ece, 0xc2b79b64, + 0x0ba893db, 0xfa729671, 0x42e784e4, 0xb33d814e, 0x7a2289f1, + 0x8bf88c5b, 0xcd20c473, 0x3cfac1d9, 0xf5e5c966, 0x043fcccc, + 0xbcaade59, 0x4d70dbf3, 0x846fd34c, 0x75b5d6e6, 0x2e34f027, + 0xdfeef58d, 0x16f1fd32, 0xe72bf898, 0x5fbeea0d, 0xae64efa7, + 0x677be718, 0x96a1e2b2, 0x9faeccec, 0x6e74c946, 0xa76bc1f9, + 0x56b1c453, 0xee24d6c6, 0x1ffed36c, 0xd6e1dbd3, 0x273bde79, + 0x7cbaf8b8, 0x8d60fd12, 0x447ff5ad, 0xb5a5f007, 0x0d30e292, + 0xfceae738, 0x35f5ef87, 0xc42fea2d, 0x82f7a205, 0x732da7af, + 0xba32af10, 0x4be8aaba, 0xf37db82f, 0x02a7bd85, 0xcbb8b53a, + 0x3a62b090, 0x61e39651, 0x903993fb, 0x59269b44, 0xa8fc9eee, + 0x10698c7b, 0xe1b389d1, 0x28ac816e, 0xd97684c4, 0xa51c113e, + 0x54c61494, 0x9dd91c2b, 0x6c031981, 0xd4960b14, 0x254c0ebe, + 0xec530601, 0x1d8903ab, 0x4608256a, 0xb7d220c0, 0x7ecd287f, + 0x8f172dd5, 0x37823f40, 0xc6583aea, 0x0f473255, 0xfe9d37ff, + 0xb8457fd7, 0x499f7a7d, 0x808072c2, 0x715a7768, 0xc9cf65fd, + 0x38156057, 0xf10a68e8, 0x00d06d42, 0x5b514b83, 0xaa8b4e29, + 0x63944696, 0x924e433c, 0x2adb51a9, 0xdb015403, 0x121e5cbc, + 0xe3c45916}, + {0x00000000, 0x0ee7e8d1, 0x1dcfd1a2, 0x13283973, 0x3b9fa344, + 0x35784b95, 0x265072e6, 0x28b79a37, 0x773f4688, 0x79d8ae59, + 0x6af0972a, 0x64177ffb, 0x4ca0e5cc, 0x42470d1d, 0x516f346e, + 0x5f88dcbf, 0xee7e8d10, 0xe09965c1, 0xf3b15cb2, 0xfd56b463, + 0xd5e12e54, 0xdb06c685, 0xc82efff6, 0xc6c91727, 0x9941cb98, + 0x97a62349, 0x848e1a3a, 0x8a69f2eb, 0xa2de68dc, 0xac39800d, + 0xbf11b97e, 0xb1f651af, 0x078c1c61, 0x096bf4b0, 0x1a43cdc3, + 0x14a42512, 0x3c13bf25, 0x32f457f4, 0x21dc6e87, 0x2f3b8656, + 0x70b35ae9, 0x7e54b238, 0x6d7c8b4b, 0x639b639a, 0x4b2cf9ad, + 0x45cb117c, 0x56e3280f, 0x5804c0de, 0xe9f29171, 0xe71579a0, + 0xf43d40d3, 0xfadaa802, 0xd26d3235, 0xdc8adae4, 0xcfa2e397, + 0xc1450b46, 0x9ecdd7f9, 0x902a3f28, 0x8302065b, 0x8de5ee8a, + 0xa55274bd, 0xabb59c6c, 0xb89da51f, 0xb67a4dce, 0x0f1838c2, + 0x01ffd013, 0x12d7e960, 0x1c3001b1, 0x34879b86, 0x3a607357, + 0x29484a24, 0x27afa2f5, 0x78277e4a, 0x76c0969b, 0x65e8afe8, + 0x6b0f4739, 0x43b8dd0e, 0x4d5f35df, 0x5e770cac, 0x5090e47d, + 0xe166b5d2, 0xef815d03, 0xfca96470, 0xf24e8ca1, 0xdaf91696, + 0xd41efe47, 0xc736c734, 0xc9d12fe5, 0x9659f35a, 0x98be1b8b, + 0x8b9622f8, 0x8571ca29, 0xadc6501e, 0xa321b8cf, 0xb00981bc, + 0xbeee696d, 0x089424a3, 0x0673cc72, 0x155bf501, 0x1bbc1dd0, + 0x330b87e7, 0x3dec6f36, 0x2ec45645, 0x2023be94, 0x7fab622b, + 0x714c8afa, 0x6264b389, 0x6c835b58, 0x4434c16f, 0x4ad329be, + 0x59fb10cd, 0x571cf81c, 0xe6eaa9b3, 0xe80d4162, 0xfb257811, + 0xf5c290c0, 0xdd750af7, 0xd392e226, 0xc0badb55, 0xce5d3384, + 0x91d5ef3b, 0x9f3207ea, 0x8c1a3e99, 0x82fdd648, 0xaa4a4c7f, + 0xa4ada4ae, 0xb7859ddd, 0xb962750c, 0x1e307184, 0x10d79955, + 0x03ffa026, 0x0d1848f7, 0x25afd2c0, 0x2b483a11, 0x38600362, + 0x3687ebb3, 0x690f370c, 0x67e8dfdd, 0x74c0e6ae, 0x7a270e7f, + 0x52909448, 0x5c777c99, 0x4f5f45ea, 0x41b8ad3b, 0xf04efc94, + 0xfea91445, 0xed812d36, 0xe366c5e7, 0xcbd15fd0, 0xc536b701, + 0xd61e8e72, 0xd8f966a3, 0x8771ba1c, 0x899652cd, 0x9abe6bbe, + 0x9459836f, 0xbcee1958, 0xb209f189, 0xa121c8fa, 0xafc6202b, + 0x19bc6de5, 0x175b8534, 0x0473bc47, 0x0a945496, 0x2223cea1, + 0x2cc42670, 0x3fec1f03, 0x310bf7d2, 0x6e832b6d, 0x6064c3bc, + 0x734cfacf, 0x7dab121e, 0x551c8829, 0x5bfb60f8, 0x48d3598b, + 0x4634b15a, 0xf7c2e0f5, 0xf9250824, 0xea0d3157, 0xe4ead986, + 0xcc5d43b1, 0xc2baab60, 0xd1929213, 0xdf757ac2, 0x80fda67d, + 0x8e1a4eac, 0x9d3277df, 0x93d59f0e, 0xbb620539, 0xb585ede8, + 0xa6add49b, 0xa84a3c4a, 0x11284946, 0x1fcfa197, 0x0ce798e4, + 0x02007035, 0x2ab7ea02, 0x245002d3, 0x37783ba0, 0x399fd371, + 0x66170fce, 0x68f0e71f, 0x7bd8de6c, 0x753f36bd, 0x5d88ac8a, + 0x536f445b, 0x40477d28, 0x4ea095f9, 0xff56c456, 0xf1b12c87, + 0xe29915f4, 0xec7efd25, 0xc4c96712, 0xca2e8fc3, 0xd906b6b0, + 0xd7e15e61, 0x886982de, 0x868e6a0f, 0x95a6537c, 0x9b41bbad, + 0xb3f6219a, 0xbd11c94b, 0xae39f038, 0xa0de18e9, 0x16a45527, + 0x1843bdf6, 0x0b6b8485, 0x058c6c54, 0x2d3bf663, 0x23dc1eb2, + 0x30f427c1, 0x3e13cf10, 0x619b13af, 0x6f7cfb7e, 0x7c54c20d, + 0x72b32adc, 0x5a04b0eb, 0x54e3583a, 0x47cb6149, 0x492c8998, + 0xf8dad837, 0xf63d30e6, 0xe5150995, 0xebf2e144, 0xc3457b73, + 0xcda293a2, 0xde8aaad1, 0xd06d4200, 0x8fe59ebf, 0x8102766e, + 0x922a4f1d, 0x9ccda7cc, 0xb47a3dfb, 0xba9dd52a, 0xa9b5ec59, + 0xa7520488}, + {0x00000000, 0x3c60e308, 0x78c1c610, 0x44a12518, 0xf1838c20, + 0xcde36f28, 0x89424a30, 0xb522a938, 0x38761e01, 0x0416fd09, + 0x40b7d811, 0x7cd73b19, 0xc9f59221, 0xf5957129, 0xb1345431, + 0x8d54b739, 0x70ec3c02, 0x4c8cdf0a, 0x082dfa12, 0x344d191a, + 0x816fb022, 0xbd0f532a, 0xf9ae7632, 0xc5ce953a, 0x489a2203, + 0x74fac10b, 0x305be413, 0x0c3b071b, 0xb919ae23, 0x85794d2b, + 0xc1d86833, 0xfdb88b3b, 0xe1d87804, 0xddb89b0c, 0x9919be14, + 0xa5795d1c, 0x105bf424, 0x2c3b172c, 0x689a3234, 0x54fad13c, + 0xd9ae6605, 0xe5ce850d, 0xa16fa015, 0x9d0f431d, 0x282dea25, + 0x144d092d, 0x50ec2c35, 0x6c8ccf3d, 0x91344406, 0xad54a70e, + 0xe9f58216, 0xd595611e, 0x60b7c826, 0x5cd72b2e, 0x18760e36, + 0x2416ed3e, 0xa9425a07, 0x9522b90f, 0xd1839c17, 0xede37f1f, + 0x58c1d627, 0x64a1352f, 0x20001037, 0x1c60f33f, 0x18c1f649, + 0x24a11541, 0x60003059, 0x5c60d351, 0xe9427a69, 0xd5229961, + 0x9183bc79, 0xade35f71, 0x20b7e848, 0x1cd70b40, 0x58762e58, + 0x6416cd50, 0xd1346468, 0xed548760, 0xa9f5a278, 0x95954170, + 0x682dca4b, 0x544d2943, 0x10ec0c5b, 0x2c8cef53, 0x99ae466b, + 0xa5cea563, 0xe16f807b, 0xdd0f6373, 0x505bd44a, 0x6c3b3742, + 0x289a125a, 0x14faf152, 0xa1d8586a, 0x9db8bb62, 0xd9199e7a, + 0xe5797d72, 0xf9198e4d, 0xc5796d45, 0x81d8485d, 0xbdb8ab55, + 0x089a026d, 0x34fae165, 0x705bc47d, 0x4c3b2775, 0xc16f904c, + 0xfd0f7344, 0xb9ae565c, 0x85ceb554, 0x30ec1c6c, 0x0c8cff64, + 0x482dda7c, 0x744d3974, 0x89f5b24f, 0xb5955147, 0xf134745f, + 0xcd549757, 0x78763e6f, 0x4416dd67, 0x00b7f87f, 0x3cd71b77, + 0xb183ac4e, 0x8de34f46, 0xc9426a5e, 0xf5228956, 0x4000206e, + 0x7c60c366, 0x38c1e67e, 0x04a10576, 0x3183ec92, 0x0de30f9a, + 0x49422a82, 0x7522c98a, 0xc00060b2, 0xfc6083ba, 0xb8c1a6a2, + 0x84a145aa, 0x09f5f293, 0x3595119b, 0x71343483, 0x4d54d78b, + 0xf8767eb3, 0xc4169dbb, 0x80b7b8a3, 0xbcd75bab, 0x416fd090, + 0x7d0f3398, 0x39ae1680, 0x05cef588, 0xb0ec5cb0, 0x8c8cbfb8, + 0xc82d9aa0, 0xf44d79a8, 0x7919ce91, 0x45792d99, 0x01d80881, + 0x3db8eb89, 0x889a42b1, 0xb4faa1b9, 0xf05b84a1, 0xcc3b67a9, + 0xd05b9496, 0xec3b779e, 0xa89a5286, 0x94fab18e, 0x21d818b6, + 0x1db8fbbe, 0x5919dea6, 0x65793dae, 0xe82d8a97, 0xd44d699f, + 0x90ec4c87, 0xac8caf8f, 0x19ae06b7, 0x25cee5bf, 0x616fc0a7, + 0x5d0f23af, 0xa0b7a894, 0x9cd74b9c, 0xd8766e84, 0xe4168d8c, + 0x513424b4, 0x6d54c7bc, 0x29f5e2a4, 0x159501ac, 0x98c1b695, + 0xa4a1559d, 0xe0007085, 0xdc60938d, 0x69423ab5, 0x5522d9bd, + 0x1183fca5, 0x2de31fad, 0x29421adb, 0x1522f9d3, 0x5183dccb, + 0x6de33fc3, 0xd8c196fb, 0xe4a175f3, 0xa00050eb, 0x9c60b3e3, + 0x113404da, 0x2d54e7d2, 0x69f5c2ca, 0x559521c2, 0xe0b788fa, + 0xdcd76bf2, 0x98764eea, 0xa416ade2, 0x59ae26d9, 0x65cec5d1, + 0x216fe0c9, 0x1d0f03c1, 0xa82daaf9, 0x944d49f1, 0xd0ec6ce9, + 0xec8c8fe1, 0x61d838d8, 0x5db8dbd0, 0x1919fec8, 0x25791dc0, + 0x905bb4f8, 0xac3b57f0, 0xe89a72e8, 0xd4fa91e0, 0xc89a62df, + 0xf4fa81d7, 0xb05ba4cf, 0x8c3b47c7, 0x3919eeff, 0x05790df7, + 0x41d828ef, 0x7db8cbe7, 0xf0ec7cde, 0xcc8c9fd6, 0x882dbace, + 0xb44d59c6, 0x016ff0fe, 0x3d0f13f6, 0x79ae36ee, 0x45ced5e6, + 0xb8765edd, 0x8416bdd5, 0xc0b798cd, 0xfcd77bc5, 0x49f5d2fd, + 0x759531f5, 0x313414ed, 0x0d54f7e5, 0x800040dc, 0xbc60a3d4, + 0xf8c186cc, 0xc4a165c4, 0x7183ccfc, 0x4de32ff4, 0x09420aec, + 0x3522e9e4}, + {0x00000000, 0x6307d924, 0xc60fb248, 0xa5086b6c, 0x576e62d1, + 0x3469bbf5, 0x9161d099, 0xf26609bd, 0xaedcc5a2, 0xcddb1c86, + 0x68d377ea, 0x0bd4aece, 0xf9b2a773, 0x9ab57e57, 0x3fbd153b, + 0x5cbacc1f, 0x86c88d05, 0xe5cf5421, 0x40c73f4d, 0x23c0e669, + 0xd1a6efd4, 0xb2a136f0, 0x17a95d9c, 0x74ae84b8, 0x281448a7, + 0x4b139183, 0xee1bfaef, 0x8d1c23cb, 0x7f7a2a76, 0x1c7df352, + 0xb975983e, 0xda72411a, 0xd6e01c4b, 0xb5e7c56f, 0x10efae03, + 0x73e87727, 0x818e7e9a, 0xe289a7be, 0x4781ccd2, 0x248615f6, + 0x783cd9e9, 0x1b3b00cd, 0xbe336ba1, 0xdd34b285, 0x2f52bb38, + 0x4c55621c, 0xe95d0970, 0x8a5ad054, 0x5028914e, 0x332f486a, + 0x96272306, 0xf520fa22, 0x0746f39f, 0x64412abb, 0xc14941d7, + 0xa24e98f3, 0xfef454ec, 0x9df38dc8, 0x38fbe6a4, 0x5bfc3f80, + 0xa99a363d, 0xca9def19, 0x6f958475, 0x0c925d51, 0x76b13ed7, + 0x15b6e7f3, 0xb0be8c9f, 0xd3b955bb, 0x21df5c06, 0x42d88522, + 0xe7d0ee4e, 0x84d7376a, 0xd86dfb75, 0xbb6a2251, 0x1e62493d, + 0x7d659019, 0x8f0399a4, 0xec044080, 0x490c2bec, 0x2a0bf2c8, + 0xf079b3d2, 0x937e6af6, 0x3676019a, 0x5571d8be, 0xa717d103, + 0xc4100827, 0x6118634b, 0x021fba6f, 0x5ea57670, 0x3da2af54, + 0x98aac438, 0xfbad1d1c, 0x09cb14a1, 0x6acccd85, 0xcfc4a6e9, + 0xacc37fcd, 0xa051229c, 0xc356fbb8, 0x665e90d4, 0x055949f0, + 0xf73f404d, 0x94389969, 0x3130f205, 0x52372b21, 0x0e8de73e, + 0x6d8a3e1a, 0xc8825576, 0xab858c52, 0x59e385ef, 0x3ae45ccb, + 0x9fec37a7, 0xfcebee83, 0x2699af99, 0x459e76bd, 0xe0961dd1, + 0x8391c4f5, 0x71f7cd48, 0x12f0146c, 0xb7f87f00, 0xd4ffa624, + 0x88456a3b, 0xeb42b31f, 0x4e4ad873, 0x2d4d0157, 0xdf2b08ea, + 0xbc2cd1ce, 0x1924baa2, 0x7a236386, 0xed627dae, 0x8e65a48a, + 0x2b6dcfe6, 0x486a16c2, 0xba0c1f7f, 0xd90bc65b, 0x7c03ad37, + 0x1f047413, 0x43beb80c, 0x20b96128, 0x85b10a44, 0xe6b6d360, + 0x14d0dadd, 0x77d703f9, 0xd2df6895, 0xb1d8b1b1, 0x6baaf0ab, + 0x08ad298f, 0xada542e3, 0xcea29bc7, 0x3cc4927a, 0x5fc34b5e, + 0xfacb2032, 0x99ccf916, 0xc5763509, 0xa671ec2d, 0x03798741, + 0x607e5e65, 0x921857d8, 0xf11f8efc, 0x5417e590, 0x37103cb4, + 0x3b8261e5, 0x5885b8c1, 0xfd8dd3ad, 0x9e8a0a89, 0x6cec0334, + 0x0febda10, 0xaae3b17c, 0xc9e46858, 0x955ea447, 0xf6597d63, + 0x5351160f, 0x3056cf2b, 0xc230c696, 0xa1371fb2, 0x043f74de, + 0x6738adfa, 0xbd4aece0, 0xde4d35c4, 0x7b455ea8, 0x1842878c, + 0xea248e31, 0x89235715, 0x2c2b3c79, 0x4f2ce55d, 0x13962942, + 0x7091f066, 0xd5999b0a, 0xb69e422e, 0x44f84b93, 0x27ff92b7, + 0x82f7f9db, 0xe1f020ff, 0x9bd34379, 0xf8d49a5d, 0x5ddcf131, + 0x3edb2815, 0xccbd21a8, 0xafbaf88c, 0x0ab293e0, 0x69b54ac4, + 0x350f86db, 0x56085fff, 0xf3003493, 0x9007edb7, 0x6261e40a, + 0x01663d2e, 0xa46e5642, 0xc7698f66, 0x1d1bce7c, 0x7e1c1758, + 0xdb147c34, 0xb813a510, 0x4a75acad, 0x29727589, 0x8c7a1ee5, + 0xef7dc7c1, 0xb3c70bde, 0xd0c0d2fa, 0x75c8b996, 0x16cf60b2, + 0xe4a9690f, 0x87aeb02b, 0x22a6db47, 0x41a10263, 0x4d335f32, + 0x2e348616, 0x8b3ced7a, 0xe83b345e, 0x1a5d3de3, 0x795ae4c7, + 0xdc528fab, 0xbf55568f, 0xe3ef9a90, 0x80e843b4, 0x25e028d8, + 0x46e7f1fc, 0xb481f841, 0xd7862165, 0x728e4a09, 0x1189932d, + 0xcbfbd237, 0xa8fc0b13, 0x0df4607f, 0x6ef3b95b, 0x9c95b0e6, + 0xff9269c2, 0x5a9a02ae, 0x399ddb8a, 0x65271795, 0x0620ceb1, + 0xa328a5dd, 0xc02f7cf9, 0x32497544, 0x514eac60, 0xf446c70c, + 0x97411e28}, + {0x00000000, 0x01b5fd1d, 0x036bfa3a, 0x02de0727, 0x06d7f474, + 0x07620969, 0x05bc0e4e, 0x0409f353, 0x0dafe8e8, 0x0c1a15f5, + 0x0ec412d2, 0x0f71efcf, 0x0b781c9c, 0x0acde181, 0x0813e6a6, + 0x09a61bbb, 0x1b5fd1d0, 0x1aea2ccd, 0x18342bea, 0x1981d6f7, + 0x1d8825a4, 0x1c3dd8b9, 0x1ee3df9e, 0x1f562283, 0x16f03938, + 0x1745c425, 0x159bc302, 0x142e3e1f, 0x1027cd4c, 0x11923051, + 0x134c3776, 0x12f9ca6b, 0x36bfa3a0, 0x370a5ebd, 0x35d4599a, + 0x3461a487, 0x306857d4, 0x31ddaac9, 0x3303adee, 0x32b650f3, + 0x3b104b48, 0x3aa5b655, 0x387bb172, 0x39ce4c6f, 0x3dc7bf3c, + 0x3c724221, 0x3eac4506, 0x3f19b81b, 0x2de07270, 0x2c558f6d, + 0x2e8b884a, 0x2f3e7557, 0x2b378604, 0x2a827b19, 0x285c7c3e, + 0x29e98123, 0x204f9a98, 0x21fa6785, 0x232460a2, 0x22919dbf, + 0x26986eec, 0x272d93f1, 0x25f394d6, 0x244669cb, 0x6d7f4740, + 0x6ccaba5d, 0x6e14bd7a, 0x6fa14067, 0x6ba8b334, 0x6a1d4e29, + 0x68c3490e, 0x6976b413, 0x60d0afa8, 0x616552b5, 0x63bb5592, + 0x620ea88f, 0x66075bdc, 0x67b2a6c1, 0x656ca1e6, 0x64d95cfb, + 0x76209690, 0x77956b8d, 0x754b6caa, 0x74fe91b7, 0x70f762e4, + 0x71429ff9, 0x739c98de, 0x722965c3, 0x7b8f7e78, 0x7a3a8365, + 0x78e48442, 0x7951795f, 0x7d588a0c, 0x7ced7711, 0x7e337036, + 0x7f868d2b, 0x5bc0e4e0, 0x5a7519fd, 0x58ab1eda, 0x591ee3c7, + 0x5d171094, 0x5ca2ed89, 0x5e7ceaae, 0x5fc917b3, 0x566f0c08, + 0x57daf115, 0x5504f632, 0x54b10b2f, 0x50b8f87c, 0x510d0561, + 0x53d30246, 0x5266ff5b, 0x409f3530, 0x412ac82d, 0x43f4cf0a, + 0x42413217, 0x4648c144, 0x47fd3c59, 0x45233b7e, 0x4496c663, + 0x4d30ddd8, 0x4c8520c5, 0x4e5b27e2, 0x4feedaff, 0x4be729ac, + 0x4a52d4b1, 0x488cd396, 0x49392e8b, 0xdafe8e80, 0xdb4b739d, + 0xd99574ba, 0xd82089a7, 0xdc297af4, 0xdd9c87e9, 0xdf4280ce, + 0xdef77dd3, 0xd7516668, 0xd6e49b75, 0xd43a9c52, 0xd58f614f, + 0xd186921c, 0xd0336f01, 0xd2ed6826, 0xd358953b, 0xc1a15f50, + 0xc014a24d, 0xc2caa56a, 0xc37f5877, 0xc776ab24, 0xc6c35639, + 0xc41d511e, 0xc5a8ac03, 0xcc0eb7b8, 0xcdbb4aa5, 0xcf654d82, + 0xced0b09f, 0xcad943cc, 0xcb6cbed1, 0xc9b2b9f6, 0xc80744eb, + 0xec412d20, 0xedf4d03d, 0xef2ad71a, 0xee9f2a07, 0xea96d954, + 0xeb232449, 0xe9fd236e, 0xe848de73, 0xe1eec5c8, 0xe05b38d5, + 0xe2853ff2, 0xe330c2ef, 0xe73931bc, 0xe68ccca1, 0xe452cb86, + 0xe5e7369b, 0xf71efcf0, 0xf6ab01ed, 0xf47506ca, 0xf5c0fbd7, + 0xf1c90884, 0xf07cf599, 0xf2a2f2be, 0xf3170fa3, 0xfab11418, + 0xfb04e905, 0xf9daee22, 0xf86f133f, 0xfc66e06c, 0xfdd31d71, + 0xff0d1a56, 0xfeb8e74b, 0xb781c9c0, 0xb63434dd, 0xb4ea33fa, + 0xb55fcee7, 0xb1563db4, 0xb0e3c0a9, 0xb23dc78e, 0xb3883a93, + 0xba2e2128, 0xbb9bdc35, 0xb945db12, 0xb8f0260f, 0xbcf9d55c, + 0xbd4c2841, 0xbf922f66, 0xbe27d27b, 0xacde1810, 0xad6be50d, + 0xafb5e22a, 0xae001f37, 0xaa09ec64, 0xabbc1179, 0xa962165e, + 0xa8d7eb43, 0xa171f0f8, 0xa0c40de5, 0xa21a0ac2, 0xa3aff7df, + 0xa7a6048c, 0xa613f991, 0xa4cdfeb6, 0xa57803ab, 0x813e6a60, + 0x808b977d, 0x8255905a, 0x83e06d47, 0x87e99e14, 0x865c6309, + 0x8482642e, 0x85379933, 0x8c918288, 0x8d247f95, 0x8ffa78b2, + 0x8e4f85af, 0x8a4676fc, 0x8bf38be1, 0x892d8cc6, 0x889871db, + 0x9a61bbb0, 0x9bd446ad, 0x990a418a, 0x98bfbc97, 0x9cb64fc4, + 0x9d03b2d9, 0x9fddb5fe, 0x9e6848e3, 0x97ce5358, 0x967bae45, + 0x94a5a962, 0x9510547f, 0x9119a72c, 0x90ac5a31, 0x92725d16, + 0x93c7a00b}, + {0x00000000, 0x6e8c1b41, 0xdd183682, 0xb3942dc3, 0x61416b45, + 0x0fcd7004, 0xbc595dc7, 0xd2d54686, 0xc282d68a, 0xac0ecdcb, + 0x1f9ae008, 0x7116fb49, 0xa3c3bdcf, 0xcd4fa68e, 0x7edb8b4d, + 0x1057900c, 0x5e74ab55, 0x30f8b014, 0x836c9dd7, 0xede08696, + 0x3f35c010, 0x51b9db51, 0xe22df692, 0x8ca1edd3, 0x9cf67ddf, + 0xf27a669e, 0x41ee4b5d, 0x2f62501c, 0xfdb7169a, 0x933b0ddb, + 0x20af2018, 0x4e233b59, 0xbce956aa, 0xd2654deb, 0x61f16028, + 0x0f7d7b69, 0xdda83def, 0xb32426ae, 0x00b00b6d, 0x6e3c102c, + 0x7e6b8020, 0x10e79b61, 0xa373b6a2, 0xcdffade3, 0x1f2aeb65, + 0x71a6f024, 0xc232dde7, 0xacbec6a6, 0xe29dfdff, 0x8c11e6be, + 0x3f85cb7d, 0x5109d03c, 0x83dc96ba, 0xed508dfb, 0x5ec4a038, + 0x3048bb79, 0x201f2b75, 0x4e933034, 0xfd071df7, 0x938b06b6, + 0x415e4030, 0x2fd25b71, 0x9c4676b2, 0xf2ca6df3, 0xa2a3ab15, + 0xcc2fb054, 0x7fbb9d97, 0x113786d6, 0xc3e2c050, 0xad6edb11, + 0x1efaf6d2, 0x7076ed93, 0x60217d9f, 0x0ead66de, 0xbd394b1d, + 0xd3b5505c, 0x016016da, 0x6fec0d9b, 0xdc782058, 0xb2f43b19, + 0xfcd70040, 0x925b1b01, 0x21cf36c2, 0x4f432d83, 0x9d966b05, + 0xf31a7044, 0x408e5d87, 0x2e0246c6, 0x3e55d6ca, 0x50d9cd8b, + 0xe34de048, 0x8dc1fb09, 0x5f14bd8f, 0x3198a6ce, 0x820c8b0d, + 0xec80904c, 0x1e4afdbf, 0x70c6e6fe, 0xc352cb3d, 0xadded07c, + 0x7f0b96fa, 0x11878dbb, 0xa213a078, 0xcc9fbb39, 0xdcc82b35, + 0xb2443074, 0x01d01db7, 0x6f5c06f6, 0xbd894070, 0xd3055b31, + 0x609176f2, 0x0e1d6db3, 0x403e56ea, 0x2eb24dab, 0x9d266068, + 0xf3aa7b29, 0x217f3daf, 0x4ff326ee, 0xfc670b2d, 0x92eb106c, + 0x82bc8060, 0xec309b21, 0x5fa4b6e2, 0x3128ada3, 0xe3fdeb25, + 0x8d71f064, 0x3ee5dda7, 0x5069c6e6, 0x9e36506b, 0xf0ba4b2a, + 0x432e66e9, 0x2da27da8, 0xff773b2e, 0x91fb206f, 0x226f0dac, + 0x4ce316ed, 0x5cb486e1, 0x32389da0, 0x81acb063, 0xef20ab22, + 0x3df5eda4, 0x5379f6e5, 0xe0eddb26, 0x8e61c067, 0xc042fb3e, + 0xaecee07f, 0x1d5acdbc, 0x73d6d6fd, 0xa103907b, 0xcf8f8b3a, + 0x7c1ba6f9, 0x1297bdb8, 0x02c02db4, 0x6c4c36f5, 0xdfd81b36, + 0xb1540077, 0x638146f1, 0x0d0d5db0, 0xbe997073, 0xd0156b32, + 0x22df06c1, 0x4c531d80, 0xffc73043, 0x914b2b02, 0x439e6d84, + 0x2d1276c5, 0x9e865b06, 0xf00a4047, 0xe05dd04b, 0x8ed1cb0a, + 0x3d45e6c9, 0x53c9fd88, 0x811cbb0e, 0xef90a04f, 0x5c048d8c, + 0x328896cd, 0x7cabad94, 0x1227b6d5, 0xa1b39b16, 0xcf3f8057, + 0x1deac6d1, 0x7366dd90, 0xc0f2f053, 0xae7eeb12, 0xbe297b1e, + 0xd0a5605f, 0x63314d9c, 0x0dbd56dd, 0xdf68105b, 0xb1e40b1a, + 0x027026d9, 0x6cfc3d98, 0x3c95fb7e, 0x5219e03f, 0xe18dcdfc, + 0x8f01d6bd, 0x5dd4903b, 0x33588b7a, 0x80cca6b9, 0xee40bdf8, + 0xfe172df4, 0x909b36b5, 0x230f1b76, 0x4d830037, 0x9f5646b1, + 0xf1da5df0, 0x424e7033, 0x2cc26b72, 0x62e1502b, 0x0c6d4b6a, + 0xbff966a9, 0xd1757de8, 0x03a03b6e, 0x6d2c202f, 0xdeb80dec, + 0xb03416ad, 0xa06386a1, 0xceef9de0, 0x7d7bb023, 0x13f7ab62, + 0xc122ede4, 0xafaef6a5, 0x1c3adb66, 0x72b6c027, 0x807cadd4, + 0xeef0b695, 0x5d649b56, 0x33e88017, 0xe13dc691, 0x8fb1ddd0, + 0x3c25f013, 0x52a9eb52, 0x42fe7b5e, 0x2c72601f, 0x9fe64ddc, + 0xf16a569d, 0x23bf101b, 0x4d330b5a, 0xfea72699, 0x902b3dd8, + 0xde080681, 0xb0841dc0, 0x03103003, 0x6d9c2b42, 0xbf496dc4, + 0xd1c57685, 0x62515b46, 0x0cdd4007, 0x1c8ad00b, 0x7206cb4a, + 0xc192e689, 0xaf1efdc8, 0x7dcbbb4e, 0x1347a00f, 0xa0d38dcc, + 0xce5f968d}, + {0x00000000, 0xe71da697, 0x154a4b6f, 0xf257edf8, 0x2a9496de, + 0xcd893049, 0x3fdeddb1, 0xd8c37b26, 0x55292dbc, 0xb2348b2b, + 0x406366d3, 0xa77ec044, 0x7fbdbb62, 0x98a01df5, 0x6af7f00d, + 0x8dea569a, 0xaa525b78, 0x4d4ffdef, 0xbf181017, 0x5805b680, + 0x80c6cda6, 0x67db6b31, 0x958c86c9, 0x7291205e, 0xff7b76c4, + 0x1866d053, 0xea313dab, 0x0d2c9b3c, 0xd5efe01a, 0x32f2468d, + 0xc0a5ab75, 0x27b80de2, 0x8fd5b0b1, 0x68c81626, 0x9a9ffbde, + 0x7d825d49, 0xa541266f, 0x425c80f8, 0xb00b6d00, 0x5716cb97, + 0xdafc9d0d, 0x3de13b9a, 0xcfb6d662, 0x28ab70f5, 0xf0680bd3, + 0x1775ad44, 0xe52240bc, 0x023fe62b, 0x2587ebc9, 0xc29a4d5e, + 0x30cda0a6, 0xd7d00631, 0x0f137d17, 0xe80edb80, 0x1a593678, + 0xfd4490ef, 0x70aec675, 0x97b360e2, 0x65e48d1a, 0x82f92b8d, + 0x5a3a50ab, 0xbd27f63c, 0x4f701bc4, 0xa86dbd53, 0xc4da6723, + 0x23c7c1b4, 0xd1902c4c, 0x368d8adb, 0xee4ef1fd, 0x0953576a, + 0xfb04ba92, 0x1c191c05, 0x91f34a9f, 0x76eeec08, 0x84b901f0, + 0x63a4a767, 0xbb67dc41, 0x5c7a7ad6, 0xae2d972e, 0x493031b9, + 0x6e883c5b, 0x89959acc, 0x7bc27734, 0x9cdfd1a3, 0x441caa85, + 0xa3010c12, 0x5156e1ea, 0xb64b477d, 0x3ba111e7, 0xdcbcb770, + 0x2eeb5a88, 0xc9f6fc1f, 0x11358739, 0xf62821ae, 0x047fcc56, + 0xe3626ac1, 0x4b0fd792, 0xac127105, 0x5e459cfd, 0xb9583a6a, + 0x619b414c, 0x8686e7db, 0x74d10a23, 0x93ccacb4, 0x1e26fa2e, + 0xf93b5cb9, 0x0b6cb141, 0xec7117d6, 0x34b26cf0, 0xd3afca67, + 0x21f8279f, 0xc6e58108, 0xe15d8cea, 0x06402a7d, 0xf417c785, + 0x130a6112, 0xcbc91a34, 0x2cd4bca3, 0xde83515b, 0x399ef7cc, + 0xb474a156, 0x536907c1, 0xa13eea39, 0x46234cae, 0x9ee03788, + 0x79fd911f, 0x8baa7ce7, 0x6cb7da70, 0x52c5c807, 0xb5d86e90, + 0x478f8368, 0xa09225ff, 0x78515ed9, 0x9f4cf84e, 0x6d1b15b6, + 0x8a06b321, 0x07ece5bb, 0xe0f1432c, 0x12a6aed4, 0xf5bb0843, + 0x2d787365, 0xca65d5f2, 0x3832380a, 0xdf2f9e9d, 0xf897937f, + 0x1f8a35e8, 0xedddd810, 0x0ac07e87, 0xd20305a1, 0x351ea336, + 0xc7494ece, 0x2054e859, 0xadbebec3, 0x4aa31854, 0xb8f4f5ac, + 0x5fe9533b, 0x872a281d, 0x60378e8a, 0x92606372, 0x757dc5e5, + 0xdd1078b6, 0x3a0dde21, 0xc85a33d9, 0x2f47954e, 0xf784ee68, + 0x109948ff, 0xe2cea507, 0x05d30390, 0x8839550a, 0x6f24f39d, + 0x9d731e65, 0x7a6eb8f2, 0xa2adc3d4, 0x45b06543, 0xb7e788bb, + 0x50fa2e2c, 0x774223ce, 0x905f8559, 0x620868a1, 0x8515ce36, + 0x5dd6b510, 0xbacb1387, 0x489cfe7f, 0xaf8158e8, 0x226b0e72, + 0xc576a8e5, 0x3721451d, 0xd03ce38a, 0x08ff98ac, 0xefe23e3b, + 0x1db5d3c3, 0xfaa87554, 0x961faf24, 0x710209b3, 0x8355e44b, + 0x644842dc, 0xbc8b39fa, 0x5b969f6d, 0xa9c17295, 0x4edcd402, + 0xc3368298, 0x242b240f, 0xd67cc9f7, 0x31616f60, 0xe9a21446, + 0x0ebfb2d1, 0xfce85f29, 0x1bf5f9be, 0x3c4df45c, 0xdb5052cb, + 0x2907bf33, 0xce1a19a4, 0x16d96282, 0xf1c4c415, 0x039329ed, + 0xe48e8f7a, 0x6964d9e0, 0x8e797f77, 0x7c2e928f, 0x9b333418, + 0x43f04f3e, 0xa4ede9a9, 0x56ba0451, 0xb1a7a2c6, 0x19ca1f95, + 0xfed7b902, 0x0c8054fa, 0xeb9df26d, 0x335e894b, 0xd4432fdc, + 0x2614c224, 0xc10964b3, 0x4ce33229, 0xabfe94be, 0x59a97946, + 0xbeb4dfd1, 0x6677a4f7, 0x816a0260, 0x733def98, 0x9420490f, + 0xb39844ed, 0x5485e27a, 0xa6d20f82, 0x41cfa915, 0x990cd233, + 0x7e1174a4, 0x8c46995c, 0x6b5b3fcb, 0xe6b16951, 0x01accfc6, + 0xf3fb223e, 0x14e684a9, 0xcc25ff8f, 0x2b385918, 0xd96fb4e0, + 0x3e721277}, + {0x00000000, 0xa58b900e, 0x9066265d, 0x35edb653, 0xfbbd4afb, + 0x5e36daf5, 0x6bdb6ca6, 0xce50fca8, 0x2c0b93b7, 0x898003b9, + 0xbc6db5ea, 0x19e625e4, 0xd7b6d94c, 0x723d4942, 0x47d0ff11, + 0xe25b6f1f, 0x5817276e, 0xfd9cb760, 0xc8710133, 0x6dfa913d, + 0xa3aa6d95, 0x0621fd9b, 0x33cc4bc8, 0x9647dbc6, 0x741cb4d9, + 0xd19724d7, 0xe47a9284, 0x41f1028a, 0x8fa1fe22, 0x2a2a6e2c, + 0x1fc7d87f, 0xba4c4871, 0xb02e4edc, 0x15a5ded2, 0x20486881, + 0x85c3f88f, 0x4b930427, 0xee189429, 0xdbf5227a, 0x7e7eb274, + 0x9c25dd6b, 0x39ae4d65, 0x0c43fb36, 0xa9c86b38, 0x67989790, + 0xc213079e, 0xf7feb1cd, 0x527521c3, 0xe83969b2, 0x4db2f9bc, + 0x785f4fef, 0xddd4dfe1, 0x13842349, 0xb60fb347, 0x83e20514, + 0x2669951a, 0xc432fa05, 0x61b96a0b, 0x5454dc58, 0xf1df4c56, + 0x3f8fb0fe, 0x9a0420f0, 0xafe996a3, 0x0a6206ad, 0xbb2d9bf9, + 0x1ea60bf7, 0x2b4bbda4, 0x8ec02daa, 0x4090d102, 0xe51b410c, + 0xd0f6f75f, 0x757d6751, 0x9726084e, 0x32ad9840, 0x07402e13, + 0xa2cbbe1d, 0x6c9b42b5, 0xc910d2bb, 0xfcfd64e8, 0x5976f4e6, + 0xe33abc97, 0x46b12c99, 0x735c9aca, 0xd6d70ac4, 0x1887f66c, + 0xbd0c6662, 0x88e1d031, 0x2d6a403f, 0xcf312f20, 0x6ababf2e, + 0x5f57097d, 0xfadc9973, 0x348c65db, 0x9107f5d5, 0xa4ea4386, + 0x0161d388, 0x0b03d525, 0xae88452b, 0x9b65f378, 0x3eee6376, + 0xf0be9fde, 0x55350fd0, 0x60d8b983, 0xc553298d, 0x27084692, + 0x8283d69c, 0xb76e60cf, 0x12e5f0c1, 0xdcb50c69, 0x793e9c67, + 0x4cd32a34, 0xe958ba3a, 0x5314f24b, 0xf69f6245, 0xc372d416, + 0x66f94418, 0xa8a9b8b0, 0x0d2228be, 0x38cf9eed, 0x9d440ee3, + 0x7f1f61fc, 0xda94f1f2, 0xef7947a1, 0x4af2d7af, 0x84a22b07, + 0x2129bb09, 0x14c40d5a, 0xb14f9d54, 0xad2a31b3, 0x08a1a1bd, + 0x3d4c17ee, 0x98c787e0, 0x56977b48, 0xf31ceb46, 0xc6f15d15, + 0x637acd1b, 0x8121a204, 0x24aa320a, 0x11478459, 0xb4cc1457, + 0x7a9ce8ff, 0xdf1778f1, 0xeafacea2, 0x4f715eac, 0xf53d16dd, + 0x50b686d3, 0x655b3080, 0xc0d0a08e, 0x0e805c26, 0xab0bcc28, + 0x9ee67a7b, 0x3b6dea75, 0xd936856a, 0x7cbd1564, 0x4950a337, + 0xecdb3339, 0x228bcf91, 0x87005f9f, 0xb2ede9cc, 0x176679c2, + 0x1d047f6f, 0xb88fef61, 0x8d625932, 0x28e9c93c, 0xe6b93594, + 0x4332a59a, 0x76df13c9, 0xd35483c7, 0x310fecd8, 0x94847cd6, + 0xa169ca85, 0x04e25a8b, 0xcab2a623, 0x6f39362d, 0x5ad4807e, + 0xff5f1070, 0x45135801, 0xe098c80f, 0xd5757e5c, 0x70feee52, + 0xbeae12fa, 0x1b2582f4, 0x2ec834a7, 0x8b43a4a9, 0x6918cbb6, + 0xcc935bb8, 0xf97eedeb, 0x5cf57de5, 0x92a5814d, 0x372e1143, + 0x02c3a710, 0xa748371e, 0x1607aa4a, 0xb38c3a44, 0x86618c17, + 0x23ea1c19, 0xedbae0b1, 0x483170bf, 0x7ddcc6ec, 0xd85756e2, + 0x3a0c39fd, 0x9f87a9f3, 0xaa6a1fa0, 0x0fe18fae, 0xc1b17306, + 0x643ae308, 0x51d7555b, 0xf45cc555, 0x4e108d24, 0xeb9b1d2a, + 0xde76ab79, 0x7bfd3b77, 0xb5adc7df, 0x102657d1, 0x25cbe182, + 0x8040718c, 0x621b1e93, 0xc7908e9d, 0xf27d38ce, 0x57f6a8c0, + 0x99a65468, 0x3c2dc466, 0x09c07235, 0xac4be23b, 0xa629e496, + 0x03a27498, 0x364fc2cb, 0x93c452c5, 0x5d94ae6d, 0xf81f3e63, + 0xcdf28830, 0x6879183e, 0x8a227721, 0x2fa9e72f, 0x1a44517c, + 0xbfcfc172, 0x719f3dda, 0xd414add4, 0xe1f91b87, 0x44728b89, + 0xfe3ec3f8, 0x5bb553f6, 0x6e58e5a5, 0xcbd375ab, 0x05838903, + 0xa008190d, 0x95e5af5e, 0x306e3f50, 0xd235504f, 0x77bec041, + 0x42537612, 0xe7d8e61c, 0x29881ab4, 0x8c038aba, 0xb9ee3ce9, + 0x1c65ace7}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x0000000000000000, 0x0e908ba500000000, 0x5d26669000000000, + 0x53b6ed3500000000, 0xfb4abdfb00000000, 0xf5da365e00000000, + 0xa66cdb6b00000000, 0xa8fc50ce00000000, 0xb7930b2c00000000, + 0xb903808900000000, 0xeab56dbc00000000, 0xe425e61900000000, + 0x4cd9b6d700000000, 0x42493d7200000000, 0x11ffd04700000000, + 0x1f6f5be200000000, 0x6e27175800000000, 0x60b79cfd00000000, + 0x330171c800000000, 0x3d91fa6d00000000, 0x956daaa300000000, + 0x9bfd210600000000, 0xc84bcc3300000000, 0xc6db479600000000, + 0xd9b41c7400000000, 0xd72497d100000000, 0x84927ae400000000, + 0x8a02f14100000000, 0x22fea18f00000000, 0x2c6e2a2a00000000, + 0x7fd8c71f00000000, 0x71484cba00000000, 0xdc4e2eb000000000, + 0xd2dea51500000000, 0x8168482000000000, 0x8ff8c38500000000, + 0x2704934b00000000, 0x299418ee00000000, 0x7a22f5db00000000, + 0x74b27e7e00000000, 0x6bdd259c00000000, 0x654dae3900000000, + 0x36fb430c00000000, 0x386bc8a900000000, 0x9097986700000000, + 0x9e0713c200000000, 0xcdb1fef700000000, 0xc321755200000000, + 0xb26939e800000000, 0xbcf9b24d00000000, 0xef4f5f7800000000, + 0xe1dfd4dd00000000, 0x4923841300000000, 0x47b30fb600000000, + 0x1405e28300000000, 0x1a95692600000000, 0x05fa32c400000000, + 0x0b6ab96100000000, 0x58dc545400000000, 0x564cdff100000000, + 0xfeb08f3f00000000, 0xf020049a00000000, 0xa396e9af00000000, + 0xad06620a00000000, 0xf99b2dbb00000000, 0xf70ba61e00000000, + 0xa4bd4b2b00000000, 0xaa2dc08e00000000, 0x02d1904000000000, + 0x0c411be500000000, 0x5ff7f6d000000000, 0x51677d7500000000, + 0x4e08269700000000, 0x4098ad3200000000, 0x132e400700000000, + 0x1dbecba200000000, 0xb5429b6c00000000, 0xbbd210c900000000, + 0xe864fdfc00000000, 0xe6f4765900000000, 0x97bc3ae300000000, + 0x992cb14600000000, 0xca9a5c7300000000, 0xc40ad7d600000000, + 0x6cf6871800000000, 0x62660cbd00000000, 0x31d0e18800000000, + 0x3f406a2d00000000, 0x202f31cf00000000, 0x2ebfba6a00000000, + 0x7d09575f00000000, 0x7399dcfa00000000, 0xdb658c3400000000, + 0xd5f5079100000000, 0x8643eaa400000000, 0x88d3610100000000, + 0x25d5030b00000000, 0x2b4588ae00000000, 0x78f3659b00000000, + 0x7663ee3e00000000, 0xde9fbef000000000, 0xd00f355500000000, + 0x83b9d86000000000, 0x8d2953c500000000, 0x9246082700000000, + 0x9cd6838200000000, 0xcf606eb700000000, 0xc1f0e51200000000, + 0x690cb5dc00000000, 0x679c3e7900000000, 0x342ad34c00000000, + 0x3aba58e900000000, 0x4bf2145300000000, 0x45629ff600000000, + 0x16d472c300000000, 0x1844f96600000000, 0xb0b8a9a800000000, + 0xbe28220d00000000, 0xed9ecf3800000000, 0xe30e449d00000000, + 0xfc611f7f00000000, 0xf2f194da00000000, 0xa14779ef00000000, + 0xafd7f24a00000000, 0x072ba28400000000, 0x09bb292100000000, + 0x5a0dc41400000000, 0x549d4fb100000000, 0xb3312aad00000000, + 0xbda1a10800000000, 0xee174c3d00000000, 0xe087c79800000000, + 0x487b975600000000, 0x46eb1cf300000000, 0x155df1c600000000, + 0x1bcd7a6300000000, 0x04a2218100000000, 0x0a32aa2400000000, + 0x5984471100000000, 0x5714ccb400000000, 0xffe89c7a00000000, + 0xf17817df00000000, 0xa2cefaea00000000, 0xac5e714f00000000, + 0xdd163df500000000, 0xd386b65000000000, 0x80305b6500000000, + 0x8ea0d0c000000000, 0x265c800e00000000, 0x28cc0bab00000000, + 0x7b7ae69e00000000, 0x75ea6d3b00000000, 0x6a8536d900000000, + 0x6415bd7c00000000, 0x37a3504900000000, 0x3933dbec00000000, + 0x91cf8b2200000000, 0x9f5f008700000000, 0xcce9edb200000000, + 0xc279661700000000, 0x6f7f041d00000000, 0x61ef8fb800000000, + 0x3259628d00000000, 0x3cc9e92800000000, 0x9435b9e600000000, + 0x9aa5324300000000, 0xc913df7600000000, 0xc78354d300000000, + 0xd8ec0f3100000000, 0xd67c849400000000, 0x85ca69a100000000, + 0x8b5ae20400000000, 0x23a6b2ca00000000, 0x2d36396f00000000, + 0x7e80d45a00000000, 0x70105fff00000000, 0x0158134500000000, + 0x0fc898e000000000, 0x5c7e75d500000000, 0x52eefe7000000000, + 0xfa12aebe00000000, 0xf482251b00000000, 0xa734c82e00000000, + 0xa9a4438b00000000, 0xb6cb186900000000, 0xb85b93cc00000000, + 0xebed7ef900000000, 0xe57df55c00000000, 0x4d81a59200000000, + 0x43112e3700000000, 0x10a7c30200000000, 0x1e3748a700000000, + 0x4aaa071600000000, 0x443a8cb300000000, 0x178c618600000000, + 0x191cea2300000000, 0xb1e0baed00000000, 0xbf70314800000000, + 0xecc6dc7d00000000, 0xe25657d800000000, 0xfd390c3a00000000, + 0xf3a9879f00000000, 0xa01f6aaa00000000, 0xae8fe10f00000000, + 0x0673b1c100000000, 0x08e33a6400000000, 0x5b55d75100000000, + 0x55c55cf400000000, 0x248d104e00000000, 0x2a1d9beb00000000, + 0x79ab76de00000000, 0x773bfd7b00000000, 0xdfc7adb500000000, + 0xd157261000000000, 0x82e1cb2500000000, 0x8c71408000000000, + 0x931e1b6200000000, 0x9d8e90c700000000, 0xce387df200000000, + 0xc0a8f65700000000, 0x6854a69900000000, 0x66c42d3c00000000, + 0x3572c00900000000, 0x3be24bac00000000, 0x96e429a600000000, + 0x9874a20300000000, 0xcbc24f3600000000, 0xc552c49300000000, + 0x6dae945d00000000, 0x633e1ff800000000, 0x3088f2cd00000000, + 0x3e18796800000000, 0x2177228a00000000, 0x2fe7a92f00000000, + 0x7c51441a00000000, 0x72c1cfbf00000000, 0xda3d9f7100000000, + 0xd4ad14d400000000, 0x871bf9e100000000, 0x898b724400000000, + 0xf8c33efe00000000, 0xf653b55b00000000, 0xa5e5586e00000000, + 0xab75d3cb00000000, 0x0389830500000000, 0x0d1908a000000000, + 0x5eafe59500000000, 0x503f6e3000000000, 0x4f5035d200000000, + 0x41c0be7700000000, 0x1276534200000000, 0x1ce6d8e700000000, + 0xb41a882900000000, 0xba8a038c00000000, 0xe93ceeb900000000, + 0xe7ac651c00000000}, + {0x0000000000000000, 0x97a61de700000000, 0x6f4b4a1500000000, + 0xf8ed57f200000000, 0xde96942a00000000, 0x493089cd00000000, + 0xb1ddde3f00000000, 0x267bc3d800000000, 0xbc2d295500000000, + 0x2b8b34b200000000, 0xd366634000000000, 0x44c07ea700000000, + 0x62bbbd7f00000000, 0xf51da09800000000, 0x0df0f76a00000000, + 0x9a56ea8d00000000, 0x785b52aa00000000, 0xeffd4f4d00000000, + 0x171018bf00000000, 0x80b6055800000000, 0xa6cdc68000000000, + 0x316bdb6700000000, 0xc9868c9500000000, 0x5e20917200000000, + 0xc4767bff00000000, 0x53d0661800000000, 0xab3d31ea00000000, + 0x3c9b2c0d00000000, 0x1ae0efd500000000, 0x8d46f23200000000, + 0x75aba5c000000000, 0xe20db82700000000, 0xb1b0d58f00000000, + 0x2616c86800000000, 0xdefb9f9a00000000, 0x495d827d00000000, + 0x6f2641a500000000, 0xf8805c4200000000, 0x006d0bb000000000, + 0x97cb165700000000, 0x0d9dfcda00000000, 0x9a3be13d00000000, + 0x62d6b6cf00000000, 0xf570ab2800000000, 0xd30b68f000000000, + 0x44ad751700000000, 0xbc4022e500000000, 0x2be63f0200000000, + 0xc9eb872500000000, 0x5e4d9ac200000000, 0xa6a0cd3000000000, + 0x3106d0d700000000, 0x177d130f00000000, 0x80db0ee800000000, + 0x7836591a00000000, 0xef9044fd00000000, 0x75c6ae7000000000, + 0xe260b39700000000, 0x1a8de46500000000, 0x8d2bf98200000000, + 0xab503a5a00000000, 0x3cf627bd00000000, 0xc41b704f00000000, + 0x53bd6da800000000, 0x2367dac400000000, 0xb4c1c72300000000, + 0x4c2c90d100000000, 0xdb8a8d3600000000, 0xfdf14eee00000000, + 0x6a57530900000000, 0x92ba04fb00000000, 0x051c191c00000000, + 0x9f4af39100000000, 0x08ecee7600000000, 0xf001b98400000000, + 0x67a7a46300000000, 0x41dc67bb00000000, 0xd67a7a5c00000000, + 0x2e972dae00000000, 0xb931304900000000, 0x5b3c886e00000000, + 0xcc9a958900000000, 0x3477c27b00000000, 0xa3d1df9c00000000, + 0x85aa1c4400000000, 0x120c01a300000000, 0xeae1565100000000, + 0x7d474bb600000000, 0xe711a13b00000000, 0x70b7bcdc00000000, + 0x885aeb2e00000000, 0x1ffcf6c900000000, 0x3987351100000000, + 0xae2128f600000000, 0x56cc7f0400000000, 0xc16a62e300000000, + 0x92d70f4b00000000, 0x057112ac00000000, 0xfd9c455e00000000, + 0x6a3a58b900000000, 0x4c419b6100000000, 0xdbe7868600000000, + 0x230ad17400000000, 0xb4accc9300000000, 0x2efa261e00000000, + 0xb95c3bf900000000, 0x41b16c0b00000000, 0xd61771ec00000000, + 0xf06cb23400000000, 0x67caafd300000000, 0x9f27f82100000000, + 0x0881e5c600000000, 0xea8c5de100000000, 0x7d2a400600000000, + 0x85c717f400000000, 0x12610a1300000000, 0x341ac9cb00000000, + 0xa3bcd42c00000000, 0x5b5183de00000000, 0xccf79e3900000000, + 0x56a174b400000000, 0xc107695300000000, 0x39ea3ea100000000, + 0xae4c234600000000, 0x8837e09e00000000, 0x1f91fd7900000000, + 0xe77caa8b00000000, 0x70dab76c00000000, 0x07c8c55200000000, + 0x906ed8b500000000, 0x68838f4700000000, 0xff2592a000000000, + 0xd95e517800000000, 0x4ef84c9f00000000, 0xb6151b6d00000000, + 0x21b3068a00000000, 0xbbe5ec0700000000, 0x2c43f1e000000000, + 0xd4aea61200000000, 0x4308bbf500000000, 0x6573782d00000000, + 0xf2d565ca00000000, 0x0a38323800000000, 0x9d9e2fdf00000000, + 0x7f9397f800000000, 0xe8358a1f00000000, 0x10d8dded00000000, + 0x877ec00a00000000, 0xa10503d200000000, 0x36a31e3500000000, + 0xce4e49c700000000, 0x59e8542000000000, 0xc3bebead00000000, + 0x5418a34a00000000, 0xacf5f4b800000000, 0x3b53e95f00000000, + 0x1d282a8700000000, 0x8a8e376000000000, 0x7263609200000000, + 0xe5c57d7500000000, 0xb67810dd00000000, 0x21de0d3a00000000, + 0xd9335ac800000000, 0x4e95472f00000000, 0x68ee84f700000000, + 0xff48991000000000, 0x07a5cee200000000, 0x9003d30500000000, + 0x0a55398800000000, 0x9df3246f00000000, 0x651e739d00000000, + 0xf2b86e7a00000000, 0xd4c3ada200000000, 0x4365b04500000000, + 0xbb88e7b700000000, 0x2c2efa5000000000, 0xce23427700000000, + 0x59855f9000000000, 0xa168086200000000, 0x36ce158500000000, + 0x10b5d65d00000000, 0x8713cbba00000000, 0x7ffe9c4800000000, + 0xe85881af00000000, 0x720e6b2200000000, 0xe5a876c500000000, + 0x1d45213700000000, 0x8ae33cd000000000, 0xac98ff0800000000, + 0x3b3ee2ef00000000, 0xc3d3b51d00000000, 0x5475a8fa00000000, + 0x24af1f9600000000, 0xb309027100000000, 0x4be4558300000000, + 0xdc42486400000000, 0xfa398bbc00000000, 0x6d9f965b00000000, + 0x9572c1a900000000, 0x02d4dc4e00000000, 0x988236c300000000, + 0x0f242b2400000000, 0xf7c97cd600000000, 0x606f613100000000, + 0x4614a2e900000000, 0xd1b2bf0e00000000, 0x295fe8fc00000000, + 0xbef9f51b00000000, 0x5cf44d3c00000000, 0xcb5250db00000000, + 0x33bf072900000000, 0xa4191ace00000000, 0x8262d91600000000, + 0x15c4c4f100000000, 0xed29930300000000, 0x7a8f8ee400000000, + 0xe0d9646900000000, 0x777f798e00000000, 0x8f922e7c00000000, + 0x1834339b00000000, 0x3e4ff04300000000, 0xa9e9eda400000000, + 0x5104ba5600000000, 0xc6a2a7b100000000, 0x951fca1900000000, + 0x02b9d7fe00000000, 0xfa54800c00000000, 0x6df29deb00000000, + 0x4b895e3300000000, 0xdc2f43d400000000, 0x24c2142600000000, + 0xb36409c100000000, 0x2932e34c00000000, 0xbe94feab00000000, + 0x4679a95900000000, 0xd1dfb4be00000000, 0xf7a4776600000000, + 0x60026a8100000000, 0x98ef3d7300000000, 0x0f49209400000000, + 0xed4498b300000000, 0x7ae2855400000000, 0x820fd2a600000000, + 0x15a9cf4100000000, 0x33d20c9900000000, 0xa474117e00000000, + 0x5c99468c00000000, 0xcb3f5b6b00000000, 0x5169b1e600000000, + 0xc6cfac0100000000, 0x3e22fbf300000000, 0xa984e61400000000, + 0x8fff25cc00000000, 0x1859382b00000000, 0xe0b46fd900000000, + 0x7712723e00000000}, + {0x0000000000000000, 0x411b8c6e00000000, 0x823618dd00000000, + 0xc32d94b300000000, 0x456b416100000000, 0x0470cd0f00000000, + 0xc75d59bc00000000, 0x8646d5d200000000, 0x8ad682c200000000, + 0xcbcd0eac00000000, 0x08e09a1f00000000, 0x49fb167100000000, + 0xcfbdc3a300000000, 0x8ea64fcd00000000, 0x4d8bdb7e00000000, + 0x0c90571000000000, 0x55ab745e00000000, 0x14b0f83000000000, + 0xd79d6c8300000000, 0x9686e0ed00000000, 0x10c0353f00000000, + 0x51dbb95100000000, 0x92f62de200000000, 0xd3eda18c00000000, + 0xdf7df69c00000000, 0x9e667af200000000, 0x5d4bee4100000000, + 0x1c50622f00000000, 0x9a16b7fd00000000, 0xdb0d3b9300000000, + 0x1820af2000000000, 0x593b234e00000000, 0xaa56e9bc00000000, + 0xeb4d65d200000000, 0x2860f16100000000, 0x697b7d0f00000000, + 0xef3da8dd00000000, 0xae2624b300000000, 0x6d0bb00000000000, + 0x2c103c6e00000000, 0x20806b7e00000000, 0x619be71000000000, + 0xa2b673a300000000, 0xe3adffcd00000000, 0x65eb2a1f00000000, + 0x24f0a67100000000, 0xe7dd32c200000000, 0xa6c6beac00000000, + 0xfffd9de200000000, 0xbee6118c00000000, 0x7dcb853f00000000, + 0x3cd0095100000000, 0xba96dc8300000000, 0xfb8d50ed00000000, + 0x38a0c45e00000000, 0x79bb483000000000, 0x752b1f2000000000, + 0x3430934e00000000, 0xf71d07fd00000000, 0xb6068b9300000000, + 0x30405e4100000000, 0x715bd22f00000000, 0xb276469c00000000, + 0xf36dcaf200000000, 0x15aba3a200000000, 0x54b02fcc00000000, + 0x979dbb7f00000000, 0xd686371100000000, 0x50c0e2c300000000, + 0x11db6ead00000000, 0xd2f6fa1e00000000, 0x93ed767000000000, + 0x9f7d216000000000, 0xde66ad0e00000000, 0x1d4b39bd00000000, + 0x5c50b5d300000000, 0xda16600100000000, 0x9b0dec6f00000000, + 0x582078dc00000000, 0x193bf4b200000000, 0x4000d7fc00000000, + 0x011b5b9200000000, 0xc236cf2100000000, 0x832d434f00000000, + 0x056b969d00000000, 0x44701af300000000, 0x875d8e4000000000, + 0xc646022e00000000, 0xcad6553e00000000, 0x8bcdd95000000000, + 0x48e04de300000000, 0x09fbc18d00000000, 0x8fbd145f00000000, + 0xcea6983100000000, 0x0d8b0c8200000000, 0x4c9080ec00000000, + 0xbffd4a1e00000000, 0xfee6c67000000000, 0x3dcb52c300000000, + 0x7cd0dead00000000, 0xfa960b7f00000000, 0xbb8d871100000000, + 0x78a013a200000000, 0x39bb9fcc00000000, 0x352bc8dc00000000, + 0x743044b200000000, 0xb71dd00100000000, 0xf6065c6f00000000, + 0x704089bd00000000, 0x315b05d300000000, 0xf276916000000000, + 0xb36d1d0e00000000, 0xea563e4000000000, 0xab4db22e00000000, + 0x6860269d00000000, 0x297baaf300000000, 0xaf3d7f2100000000, + 0xee26f34f00000000, 0x2d0b67fc00000000, 0x6c10eb9200000000, + 0x6080bc8200000000, 0x219b30ec00000000, 0xe2b6a45f00000000, + 0xa3ad283100000000, 0x25ebfde300000000, 0x64f0718d00000000, + 0xa7dde53e00000000, 0xe6c6695000000000, 0x6b50369e00000000, + 0x2a4bbaf000000000, 0xe9662e4300000000, 0xa87da22d00000000, + 0x2e3b77ff00000000, 0x6f20fb9100000000, 0xac0d6f2200000000, + 0xed16e34c00000000, 0xe186b45c00000000, 0xa09d383200000000, + 0x63b0ac8100000000, 0x22ab20ef00000000, 0xa4edf53d00000000, + 0xe5f6795300000000, 0x26dbede000000000, 0x67c0618e00000000, + 0x3efb42c000000000, 0x7fe0ceae00000000, 0xbccd5a1d00000000, + 0xfdd6d67300000000, 0x7b9003a100000000, 0x3a8b8fcf00000000, + 0xf9a61b7c00000000, 0xb8bd971200000000, 0xb42dc00200000000, + 0xf5364c6c00000000, 0x361bd8df00000000, 0x770054b100000000, + 0xf146816300000000, 0xb05d0d0d00000000, 0x737099be00000000, + 0x326b15d000000000, 0xc106df2200000000, 0x801d534c00000000, + 0x4330c7ff00000000, 0x022b4b9100000000, 0x846d9e4300000000, + 0xc576122d00000000, 0x065b869e00000000, 0x47400af000000000, + 0x4bd05de000000000, 0x0acbd18e00000000, 0xc9e6453d00000000, + 0x88fdc95300000000, 0x0ebb1c8100000000, 0x4fa090ef00000000, + 0x8c8d045c00000000, 0xcd96883200000000, 0x94adab7c00000000, + 0xd5b6271200000000, 0x169bb3a100000000, 0x57803fcf00000000, + 0xd1c6ea1d00000000, 0x90dd667300000000, 0x53f0f2c000000000, + 0x12eb7eae00000000, 0x1e7b29be00000000, 0x5f60a5d000000000, + 0x9c4d316300000000, 0xdd56bd0d00000000, 0x5b1068df00000000, + 0x1a0be4b100000000, 0xd926700200000000, 0x983dfc6c00000000, + 0x7efb953c00000000, 0x3fe0195200000000, 0xfccd8de100000000, + 0xbdd6018f00000000, 0x3b90d45d00000000, 0x7a8b583300000000, + 0xb9a6cc8000000000, 0xf8bd40ee00000000, 0xf42d17fe00000000, + 0xb5369b9000000000, 0x761b0f2300000000, 0x3700834d00000000, + 0xb146569f00000000, 0xf05ddaf100000000, 0x33704e4200000000, + 0x726bc22c00000000, 0x2b50e16200000000, 0x6a4b6d0c00000000, + 0xa966f9bf00000000, 0xe87d75d100000000, 0x6e3ba00300000000, + 0x2f202c6d00000000, 0xec0db8de00000000, 0xad1634b000000000, + 0xa18663a000000000, 0xe09defce00000000, 0x23b07b7d00000000, + 0x62abf71300000000, 0xe4ed22c100000000, 0xa5f6aeaf00000000, + 0x66db3a1c00000000, 0x27c0b67200000000, 0xd4ad7c8000000000, + 0x95b6f0ee00000000, 0x569b645d00000000, 0x1780e83300000000, + 0x91c63de100000000, 0xd0ddb18f00000000, 0x13f0253c00000000, + 0x52eba95200000000, 0x5e7bfe4200000000, 0x1f60722c00000000, + 0xdc4de69f00000000, 0x9d566af100000000, 0x1b10bf2300000000, + 0x5a0b334d00000000, 0x9926a7fe00000000, 0xd83d2b9000000000, + 0x810608de00000000, 0xc01d84b000000000, 0x0330100300000000, + 0x422b9c6d00000000, 0xc46d49bf00000000, 0x8576c5d100000000, + 0x465b516200000000, 0x0740dd0c00000000, 0x0bd08a1c00000000, + 0x4acb067200000000, 0x89e692c100000000, 0xc8fd1eaf00000000, + 0x4ebbcb7d00000000, 0x0fa0471300000000, 0xcc8dd3a000000000, + 0x8d965fce00000000}, + {0x0000000000000000, 0x1dfdb50100000000, 0x3afa6b0300000000, + 0x2707de0200000000, 0x74f4d70600000000, 0x6909620700000000, + 0x4e0ebc0500000000, 0x53f3090400000000, 0xe8e8af0d00000000, + 0xf5151a0c00000000, 0xd212c40e00000000, 0xcfef710f00000000, + 0x9c1c780b00000000, 0x81e1cd0a00000000, 0xa6e6130800000000, + 0xbb1ba60900000000, 0xd0d15f1b00000000, 0xcd2cea1a00000000, + 0xea2b341800000000, 0xf7d6811900000000, 0xa425881d00000000, + 0xb9d83d1c00000000, 0x9edfe31e00000000, 0x8322561f00000000, + 0x3839f01600000000, 0x25c4451700000000, 0x02c39b1500000000, + 0x1f3e2e1400000000, 0x4ccd271000000000, 0x5130921100000000, + 0x76374c1300000000, 0x6bcaf91200000000, 0xa0a3bf3600000000, + 0xbd5e0a3700000000, 0x9a59d43500000000, 0x87a4613400000000, + 0xd457683000000000, 0xc9aadd3100000000, 0xeead033300000000, + 0xf350b63200000000, 0x484b103b00000000, 0x55b6a53a00000000, + 0x72b17b3800000000, 0x6f4cce3900000000, 0x3cbfc73d00000000, + 0x2142723c00000000, 0x0645ac3e00000000, 0x1bb8193f00000000, + 0x7072e02d00000000, 0x6d8f552c00000000, 0x4a888b2e00000000, + 0x57753e2f00000000, 0x0486372b00000000, 0x197b822a00000000, + 0x3e7c5c2800000000, 0x2381e92900000000, 0x989a4f2000000000, + 0x8567fa2100000000, 0xa260242300000000, 0xbf9d912200000000, + 0xec6e982600000000, 0xf1932d2700000000, 0xd694f32500000000, + 0xcb69462400000000, 0x40477f6d00000000, 0x5dbaca6c00000000, + 0x7abd146e00000000, 0x6740a16f00000000, 0x34b3a86b00000000, + 0x294e1d6a00000000, 0x0e49c36800000000, 0x13b4766900000000, + 0xa8afd06000000000, 0xb552656100000000, 0x9255bb6300000000, + 0x8fa80e6200000000, 0xdc5b076600000000, 0xc1a6b26700000000, + 0xe6a16c6500000000, 0xfb5cd96400000000, 0x9096207600000000, + 0x8d6b957700000000, 0xaa6c4b7500000000, 0xb791fe7400000000, + 0xe462f77000000000, 0xf99f427100000000, 0xde989c7300000000, + 0xc365297200000000, 0x787e8f7b00000000, 0x65833a7a00000000, + 0x4284e47800000000, 0x5f79517900000000, 0x0c8a587d00000000, + 0x1177ed7c00000000, 0x3670337e00000000, 0x2b8d867f00000000, + 0xe0e4c05b00000000, 0xfd19755a00000000, 0xda1eab5800000000, + 0xc7e31e5900000000, 0x9410175d00000000, 0x89eda25c00000000, + 0xaeea7c5e00000000, 0xb317c95f00000000, 0x080c6f5600000000, + 0x15f1da5700000000, 0x32f6045500000000, 0x2f0bb15400000000, + 0x7cf8b85000000000, 0x61050d5100000000, 0x4602d35300000000, + 0x5bff665200000000, 0x30359f4000000000, 0x2dc82a4100000000, + 0x0acff44300000000, 0x1732414200000000, 0x44c1484600000000, + 0x593cfd4700000000, 0x7e3b234500000000, 0x63c6964400000000, + 0xd8dd304d00000000, 0xc520854c00000000, 0xe2275b4e00000000, + 0xffdaee4f00000000, 0xac29e74b00000000, 0xb1d4524a00000000, + 0x96d38c4800000000, 0x8b2e394900000000, 0x808efeda00000000, + 0x9d734bdb00000000, 0xba7495d900000000, 0xa78920d800000000, + 0xf47a29dc00000000, 0xe9879cdd00000000, 0xce8042df00000000, + 0xd37df7de00000000, 0x686651d700000000, 0x759be4d600000000, + 0x529c3ad400000000, 0x4f618fd500000000, 0x1c9286d100000000, + 0x016f33d000000000, 0x2668edd200000000, 0x3b9558d300000000, + 0x505fa1c100000000, 0x4da214c000000000, 0x6aa5cac200000000, + 0x77587fc300000000, 0x24ab76c700000000, 0x3956c3c600000000, + 0x1e511dc400000000, 0x03aca8c500000000, 0xb8b70ecc00000000, + 0xa54abbcd00000000, 0x824d65cf00000000, 0x9fb0d0ce00000000, + 0xcc43d9ca00000000, 0xd1be6ccb00000000, 0xf6b9b2c900000000, + 0xeb4407c800000000, 0x202d41ec00000000, 0x3dd0f4ed00000000, + 0x1ad72aef00000000, 0x072a9fee00000000, 0x54d996ea00000000, + 0x492423eb00000000, 0x6e23fde900000000, 0x73de48e800000000, + 0xc8c5eee100000000, 0xd5385be000000000, 0xf23f85e200000000, + 0xefc230e300000000, 0xbc3139e700000000, 0xa1cc8ce600000000, + 0x86cb52e400000000, 0x9b36e7e500000000, 0xf0fc1ef700000000, + 0xed01abf600000000, 0xca0675f400000000, 0xd7fbc0f500000000, + 0x8408c9f100000000, 0x99f57cf000000000, 0xbef2a2f200000000, + 0xa30f17f300000000, 0x1814b1fa00000000, 0x05e904fb00000000, + 0x22eedaf900000000, 0x3f136ff800000000, 0x6ce066fc00000000, + 0x711dd3fd00000000, 0x561a0dff00000000, 0x4be7b8fe00000000, + 0xc0c981b700000000, 0xdd3434b600000000, 0xfa33eab400000000, + 0xe7ce5fb500000000, 0xb43d56b100000000, 0xa9c0e3b000000000, + 0x8ec73db200000000, 0x933a88b300000000, 0x28212eba00000000, + 0x35dc9bbb00000000, 0x12db45b900000000, 0x0f26f0b800000000, + 0x5cd5f9bc00000000, 0x41284cbd00000000, 0x662f92bf00000000, + 0x7bd227be00000000, 0x1018deac00000000, 0x0de56bad00000000, + 0x2ae2b5af00000000, 0x371f00ae00000000, 0x64ec09aa00000000, + 0x7911bcab00000000, 0x5e1662a900000000, 0x43ebd7a800000000, + 0xf8f071a100000000, 0xe50dc4a000000000, 0xc20a1aa200000000, + 0xdff7afa300000000, 0x8c04a6a700000000, 0x91f913a600000000, + 0xb6fecda400000000, 0xab0378a500000000, 0x606a3e8100000000, + 0x7d978b8000000000, 0x5a90558200000000, 0x476de08300000000, + 0x149ee98700000000, 0x09635c8600000000, 0x2e64828400000000, + 0x3399378500000000, 0x8882918c00000000, 0x957f248d00000000, + 0xb278fa8f00000000, 0xaf854f8e00000000, 0xfc76468a00000000, + 0xe18bf38b00000000, 0xc68c2d8900000000, 0xdb71988800000000, + 0xb0bb619a00000000, 0xad46d49b00000000, 0x8a410a9900000000, + 0x97bcbf9800000000, 0xc44fb69c00000000, 0xd9b2039d00000000, + 0xfeb5dd9f00000000, 0xe348689e00000000, 0x5853ce9700000000, + 0x45ae7b9600000000, 0x62a9a59400000000, 0x7f54109500000000, + 0x2ca7199100000000, 0x315aac9000000000, 0x165d729200000000, + 0x0ba0c79300000000}, + {0x0000000000000000, 0x24d9076300000000, 0x48b20fc600000000, + 0x6c6b08a500000000, 0xd1626e5700000000, 0xf5bb693400000000, + 0x99d0619100000000, 0xbd0966f200000000, 0xa2c5dcae00000000, + 0x861cdbcd00000000, 0xea77d36800000000, 0xceaed40b00000000, + 0x73a7b2f900000000, 0x577eb59a00000000, 0x3b15bd3f00000000, + 0x1fccba5c00000000, 0x058dc88600000000, 0x2154cfe500000000, + 0x4d3fc74000000000, 0x69e6c02300000000, 0xd4efa6d100000000, + 0xf036a1b200000000, 0x9c5da91700000000, 0xb884ae7400000000, + 0xa748142800000000, 0x8391134b00000000, 0xeffa1bee00000000, + 0xcb231c8d00000000, 0x762a7a7f00000000, 0x52f37d1c00000000, + 0x3e9875b900000000, 0x1a4172da00000000, 0x4b1ce0d600000000, + 0x6fc5e7b500000000, 0x03aeef1000000000, 0x2777e87300000000, + 0x9a7e8e8100000000, 0xbea789e200000000, 0xd2cc814700000000, + 0xf615862400000000, 0xe9d93c7800000000, 0xcd003b1b00000000, + 0xa16b33be00000000, 0x85b234dd00000000, 0x38bb522f00000000, + 0x1c62554c00000000, 0x70095de900000000, 0x54d05a8a00000000, + 0x4e91285000000000, 0x6a482f3300000000, 0x0623279600000000, + 0x22fa20f500000000, 0x9ff3460700000000, 0xbb2a416400000000, + 0xd74149c100000000, 0xf3984ea200000000, 0xec54f4fe00000000, + 0xc88df39d00000000, 0xa4e6fb3800000000, 0x803ffc5b00000000, + 0x3d369aa900000000, 0x19ef9dca00000000, 0x7584956f00000000, + 0x515d920c00000000, 0xd73eb17600000000, 0xf3e7b61500000000, + 0x9f8cbeb000000000, 0xbb55b9d300000000, 0x065cdf2100000000, + 0x2285d84200000000, 0x4eeed0e700000000, 0x6a37d78400000000, + 0x75fb6dd800000000, 0x51226abb00000000, 0x3d49621e00000000, + 0x1990657d00000000, 0xa499038f00000000, 0x804004ec00000000, + 0xec2b0c4900000000, 0xc8f20b2a00000000, 0xd2b379f000000000, + 0xf66a7e9300000000, 0x9a01763600000000, 0xbed8715500000000, + 0x03d117a700000000, 0x270810c400000000, 0x4b63186100000000, + 0x6fba1f0200000000, 0x7076a55e00000000, 0x54afa23d00000000, + 0x38c4aa9800000000, 0x1c1dadfb00000000, 0xa114cb0900000000, + 0x85cdcc6a00000000, 0xe9a6c4cf00000000, 0xcd7fc3ac00000000, + 0x9c2251a000000000, 0xb8fb56c300000000, 0xd4905e6600000000, + 0xf049590500000000, 0x4d403ff700000000, 0x6999389400000000, + 0x05f2303100000000, 0x212b375200000000, 0x3ee78d0e00000000, + 0x1a3e8a6d00000000, 0x765582c800000000, 0x528c85ab00000000, + 0xef85e35900000000, 0xcb5ce43a00000000, 0xa737ec9f00000000, + 0x83eeebfc00000000, 0x99af992600000000, 0xbd769e4500000000, + 0xd11d96e000000000, 0xf5c4918300000000, 0x48cdf77100000000, + 0x6c14f01200000000, 0x007ff8b700000000, 0x24a6ffd400000000, + 0x3b6a458800000000, 0x1fb342eb00000000, 0x73d84a4e00000000, + 0x57014d2d00000000, 0xea082bdf00000000, 0xced12cbc00000000, + 0xa2ba241900000000, 0x8663237a00000000, 0xae7d62ed00000000, + 0x8aa4658e00000000, 0xe6cf6d2b00000000, 0xc2166a4800000000, + 0x7f1f0cba00000000, 0x5bc60bd900000000, 0x37ad037c00000000, + 0x1374041f00000000, 0x0cb8be4300000000, 0x2861b92000000000, + 0x440ab18500000000, 0x60d3b6e600000000, 0xdddad01400000000, + 0xf903d77700000000, 0x9568dfd200000000, 0xb1b1d8b100000000, + 0xabf0aa6b00000000, 0x8f29ad0800000000, 0xe342a5ad00000000, + 0xc79ba2ce00000000, 0x7a92c43c00000000, 0x5e4bc35f00000000, + 0x3220cbfa00000000, 0x16f9cc9900000000, 0x093576c500000000, + 0x2dec71a600000000, 0x4187790300000000, 0x655e7e6000000000, + 0xd857189200000000, 0xfc8e1ff100000000, 0x90e5175400000000, + 0xb43c103700000000, 0xe561823b00000000, 0xc1b8855800000000, + 0xadd38dfd00000000, 0x890a8a9e00000000, 0x3403ec6c00000000, + 0x10daeb0f00000000, 0x7cb1e3aa00000000, 0x5868e4c900000000, + 0x47a45e9500000000, 0x637d59f600000000, 0x0f16515300000000, + 0x2bcf563000000000, 0x96c630c200000000, 0xb21f37a100000000, + 0xde743f0400000000, 0xfaad386700000000, 0xe0ec4abd00000000, + 0xc4354dde00000000, 0xa85e457b00000000, 0x8c87421800000000, + 0x318e24ea00000000, 0x1557238900000000, 0x793c2b2c00000000, + 0x5de52c4f00000000, 0x4229961300000000, 0x66f0917000000000, + 0x0a9b99d500000000, 0x2e429eb600000000, 0x934bf84400000000, + 0xb792ff2700000000, 0xdbf9f78200000000, 0xff20f0e100000000, + 0x7943d39b00000000, 0x5d9ad4f800000000, 0x31f1dc5d00000000, + 0x1528db3e00000000, 0xa821bdcc00000000, 0x8cf8baaf00000000, + 0xe093b20a00000000, 0xc44ab56900000000, 0xdb860f3500000000, + 0xff5f085600000000, 0x933400f300000000, 0xb7ed079000000000, + 0x0ae4616200000000, 0x2e3d660100000000, 0x42566ea400000000, + 0x668f69c700000000, 0x7cce1b1d00000000, 0x58171c7e00000000, + 0x347c14db00000000, 0x10a513b800000000, 0xadac754a00000000, + 0x8975722900000000, 0xe51e7a8c00000000, 0xc1c77def00000000, + 0xde0bc7b300000000, 0xfad2c0d000000000, 0x96b9c87500000000, + 0xb260cf1600000000, 0x0f69a9e400000000, 0x2bb0ae8700000000, + 0x47dba62200000000, 0x6302a14100000000, 0x325f334d00000000, + 0x1686342e00000000, 0x7aed3c8b00000000, 0x5e343be800000000, + 0xe33d5d1a00000000, 0xc7e45a7900000000, 0xab8f52dc00000000, + 0x8f5655bf00000000, 0x909aefe300000000, 0xb443e88000000000, + 0xd828e02500000000, 0xfcf1e74600000000, 0x41f881b400000000, + 0x652186d700000000, 0x094a8e7200000000, 0x2d93891100000000, + 0x37d2fbcb00000000, 0x130bfca800000000, 0x7f60f40d00000000, + 0x5bb9f36e00000000, 0xe6b0959c00000000, 0xc26992ff00000000, + 0xae029a5a00000000, 0x8adb9d3900000000, 0x9517276500000000, + 0xb1ce200600000000, 0xdda528a300000000, 0xf97c2fc000000000, + 0x4475493200000000, 0x60ac4e5100000000, 0x0cc746f400000000, + 0x281e419700000000}, + {0x0000000000000000, 0x08e3603c00000000, 0x10c6c17800000000, + 0x1825a14400000000, 0x208c83f100000000, 0x286fe3cd00000000, + 0x304a428900000000, 0x38a922b500000000, 0x011e763800000000, + 0x09fd160400000000, 0x11d8b74000000000, 0x193bd77c00000000, + 0x2192f5c900000000, 0x297195f500000000, 0x315434b100000000, + 0x39b7548d00000000, 0x023cec7000000000, 0x0adf8c4c00000000, + 0x12fa2d0800000000, 0x1a194d3400000000, 0x22b06f8100000000, + 0x2a530fbd00000000, 0x3276aef900000000, 0x3a95cec500000000, + 0x03229a4800000000, 0x0bc1fa7400000000, 0x13e45b3000000000, + 0x1b073b0c00000000, 0x23ae19b900000000, 0x2b4d798500000000, + 0x3368d8c100000000, 0x3b8bb8fd00000000, 0x0478d8e100000000, + 0x0c9bb8dd00000000, 0x14be199900000000, 0x1c5d79a500000000, + 0x24f45b1000000000, 0x2c173b2c00000000, 0x34329a6800000000, + 0x3cd1fa5400000000, 0x0566aed900000000, 0x0d85cee500000000, + 0x15a06fa100000000, 0x1d430f9d00000000, 0x25ea2d2800000000, + 0x2d094d1400000000, 0x352cec5000000000, 0x3dcf8c6c00000000, + 0x0644349100000000, 0x0ea754ad00000000, 0x1682f5e900000000, + 0x1e6195d500000000, 0x26c8b76000000000, 0x2e2bd75c00000000, + 0x360e761800000000, 0x3eed162400000000, 0x075a42a900000000, + 0x0fb9229500000000, 0x179c83d100000000, 0x1f7fe3ed00000000, + 0x27d6c15800000000, 0x2f35a16400000000, 0x3710002000000000, + 0x3ff3601c00000000, 0x49f6c11800000000, 0x4115a12400000000, + 0x5930006000000000, 0x51d3605c00000000, 0x697a42e900000000, + 0x619922d500000000, 0x79bc839100000000, 0x715fe3ad00000000, + 0x48e8b72000000000, 0x400bd71c00000000, 0x582e765800000000, + 0x50cd166400000000, 0x686434d100000000, 0x608754ed00000000, + 0x78a2f5a900000000, 0x7041959500000000, 0x4bca2d6800000000, + 0x43294d5400000000, 0x5b0cec1000000000, 0x53ef8c2c00000000, + 0x6b46ae9900000000, 0x63a5cea500000000, 0x7b806fe100000000, + 0x73630fdd00000000, 0x4ad45b5000000000, 0x42373b6c00000000, + 0x5a129a2800000000, 0x52f1fa1400000000, 0x6a58d8a100000000, + 0x62bbb89d00000000, 0x7a9e19d900000000, 0x727d79e500000000, + 0x4d8e19f900000000, 0x456d79c500000000, 0x5d48d88100000000, + 0x55abb8bd00000000, 0x6d029a0800000000, 0x65e1fa3400000000, + 0x7dc45b7000000000, 0x75273b4c00000000, 0x4c906fc100000000, + 0x44730ffd00000000, 0x5c56aeb900000000, 0x54b5ce8500000000, + 0x6c1cec3000000000, 0x64ff8c0c00000000, 0x7cda2d4800000000, + 0x74394d7400000000, 0x4fb2f58900000000, 0x475195b500000000, + 0x5f7434f100000000, 0x579754cd00000000, 0x6f3e767800000000, + 0x67dd164400000000, 0x7ff8b70000000000, 0x771bd73c00000000, + 0x4eac83b100000000, 0x464fe38d00000000, 0x5e6a42c900000000, + 0x568922f500000000, 0x6e20004000000000, 0x66c3607c00000000, + 0x7ee6c13800000000, 0x7605a10400000000, 0x92ec833100000000, + 0x9a0fe30d00000000, 0x822a424900000000, 0x8ac9227500000000, + 0xb26000c000000000, 0xba8360fc00000000, 0xa2a6c1b800000000, + 0xaa45a18400000000, 0x93f2f50900000000, 0x9b11953500000000, + 0x8334347100000000, 0x8bd7544d00000000, 0xb37e76f800000000, + 0xbb9d16c400000000, 0xa3b8b78000000000, 0xab5bd7bc00000000, + 0x90d06f4100000000, 0x98330f7d00000000, 0x8016ae3900000000, + 0x88f5ce0500000000, 0xb05cecb000000000, 0xb8bf8c8c00000000, + 0xa09a2dc800000000, 0xa8794df400000000, 0x91ce197900000000, + 0x992d794500000000, 0x8108d80100000000, 0x89ebb83d00000000, + 0xb1429a8800000000, 0xb9a1fab400000000, 0xa1845bf000000000, + 0xa9673bcc00000000, 0x96945bd000000000, 0x9e773bec00000000, + 0x86529aa800000000, 0x8eb1fa9400000000, 0xb618d82100000000, + 0xbefbb81d00000000, 0xa6de195900000000, 0xae3d796500000000, + 0x978a2de800000000, 0x9f694dd400000000, 0x874cec9000000000, + 0x8faf8cac00000000, 0xb706ae1900000000, 0xbfe5ce2500000000, + 0xa7c06f6100000000, 0xaf230f5d00000000, 0x94a8b7a000000000, + 0x9c4bd79c00000000, 0x846e76d800000000, 0x8c8d16e400000000, + 0xb424345100000000, 0xbcc7546d00000000, 0xa4e2f52900000000, + 0xac01951500000000, 0x95b6c19800000000, 0x9d55a1a400000000, + 0x857000e000000000, 0x8d9360dc00000000, 0xb53a426900000000, + 0xbdd9225500000000, 0xa5fc831100000000, 0xad1fe32d00000000, + 0xdb1a422900000000, 0xd3f9221500000000, 0xcbdc835100000000, + 0xc33fe36d00000000, 0xfb96c1d800000000, 0xf375a1e400000000, + 0xeb5000a000000000, 0xe3b3609c00000000, 0xda04341100000000, + 0xd2e7542d00000000, 0xcac2f56900000000, 0xc221955500000000, + 0xfa88b7e000000000, 0xf26bd7dc00000000, 0xea4e769800000000, + 0xe2ad16a400000000, 0xd926ae5900000000, 0xd1c5ce6500000000, + 0xc9e06f2100000000, 0xc1030f1d00000000, 0xf9aa2da800000000, + 0xf1494d9400000000, 0xe96cecd000000000, 0xe18f8cec00000000, + 0xd838d86100000000, 0xd0dbb85d00000000, 0xc8fe191900000000, + 0xc01d792500000000, 0xf8b45b9000000000, 0xf0573bac00000000, + 0xe8729ae800000000, 0xe091fad400000000, 0xdf629ac800000000, + 0xd781faf400000000, 0xcfa45bb000000000, 0xc7473b8c00000000, + 0xffee193900000000, 0xf70d790500000000, 0xef28d84100000000, + 0xe7cbb87d00000000, 0xde7cecf000000000, 0xd69f8ccc00000000, + 0xceba2d8800000000, 0xc6594db400000000, 0xfef06f0100000000, + 0xf6130f3d00000000, 0xee36ae7900000000, 0xe6d5ce4500000000, + 0xdd5e76b800000000, 0xd5bd168400000000, 0xcd98b7c000000000, + 0xc57bd7fc00000000, 0xfdd2f54900000000, 0xf531957500000000, + 0xed14343100000000, 0xe5f7540d00000000, 0xdc40008000000000, + 0xd4a360bc00000000, 0xcc86c1f800000000, 0xc465a1c400000000, + 0xfccc837100000000, 0xf42fe34d00000000, 0xec0a420900000000, + 0xe4e9223500000000}, + {0x0000000000000000, 0xd1e8e70e00000000, 0xa2d1cf1d00000000, + 0x7339281300000000, 0x44a39f3b00000000, 0x954b783500000000, + 0xe672502600000000, 0x379ab72800000000, 0x88463f7700000000, + 0x59aed87900000000, 0x2a97f06a00000000, 0xfb7f176400000000, + 0xcce5a04c00000000, 0x1d0d474200000000, 0x6e346f5100000000, + 0xbfdc885f00000000, 0x108d7eee00000000, 0xc16599e000000000, + 0xb25cb1f300000000, 0x63b456fd00000000, 0x542ee1d500000000, + 0x85c606db00000000, 0xf6ff2ec800000000, 0x2717c9c600000000, + 0x98cb419900000000, 0x4923a69700000000, 0x3a1a8e8400000000, + 0xebf2698a00000000, 0xdc68dea200000000, 0x0d8039ac00000000, + 0x7eb911bf00000000, 0xaf51f6b100000000, 0x611c8c0700000000, + 0xb0f46b0900000000, 0xc3cd431a00000000, 0x1225a41400000000, + 0x25bf133c00000000, 0xf457f43200000000, 0x876edc2100000000, + 0x56863b2f00000000, 0xe95ab37000000000, 0x38b2547e00000000, + 0x4b8b7c6d00000000, 0x9a639b6300000000, 0xadf92c4b00000000, + 0x7c11cb4500000000, 0x0f28e35600000000, 0xdec0045800000000, + 0x7191f2e900000000, 0xa07915e700000000, 0xd3403df400000000, + 0x02a8dafa00000000, 0x35326dd200000000, 0xe4da8adc00000000, + 0x97e3a2cf00000000, 0x460b45c100000000, 0xf9d7cd9e00000000, + 0x283f2a9000000000, 0x5b06028300000000, 0x8aeee58d00000000, + 0xbd7452a500000000, 0x6c9cb5ab00000000, 0x1fa59db800000000, + 0xce4d7ab600000000, 0xc238180f00000000, 0x13d0ff0100000000, + 0x60e9d71200000000, 0xb101301c00000000, 0x869b873400000000, + 0x5773603a00000000, 0x244a482900000000, 0xf5a2af2700000000, + 0x4a7e277800000000, 0x9b96c07600000000, 0xe8afe86500000000, + 0x39470f6b00000000, 0x0eddb84300000000, 0xdf355f4d00000000, + 0xac0c775e00000000, 0x7de4905000000000, 0xd2b566e100000000, + 0x035d81ef00000000, 0x7064a9fc00000000, 0xa18c4ef200000000, + 0x9616f9da00000000, 0x47fe1ed400000000, 0x34c736c700000000, + 0xe52fd1c900000000, 0x5af3599600000000, 0x8b1bbe9800000000, + 0xf822968b00000000, 0x29ca718500000000, 0x1e50c6ad00000000, + 0xcfb821a300000000, 0xbc8109b000000000, 0x6d69eebe00000000, + 0xa324940800000000, 0x72cc730600000000, 0x01f55b1500000000, + 0xd01dbc1b00000000, 0xe7870b3300000000, 0x366fec3d00000000, + 0x4556c42e00000000, 0x94be232000000000, 0x2b62ab7f00000000, + 0xfa8a4c7100000000, 0x89b3646200000000, 0x585b836c00000000, + 0x6fc1344400000000, 0xbe29d34a00000000, 0xcd10fb5900000000, + 0x1cf81c5700000000, 0xb3a9eae600000000, 0x62410de800000000, + 0x117825fb00000000, 0xc090c2f500000000, 0xf70a75dd00000000, + 0x26e292d300000000, 0x55dbbac000000000, 0x84335dce00000000, + 0x3befd59100000000, 0xea07329f00000000, 0x993e1a8c00000000, + 0x48d6fd8200000000, 0x7f4c4aaa00000000, 0xaea4ada400000000, + 0xdd9d85b700000000, 0x0c7562b900000000, 0x8471301e00000000, + 0x5599d71000000000, 0x26a0ff0300000000, 0xf748180d00000000, + 0xc0d2af2500000000, 0x113a482b00000000, 0x6203603800000000, + 0xb3eb873600000000, 0x0c370f6900000000, 0xdddfe86700000000, + 0xaee6c07400000000, 0x7f0e277a00000000, 0x4894905200000000, + 0x997c775c00000000, 0xea455f4f00000000, 0x3badb84100000000, + 0x94fc4ef000000000, 0x4514a9fe00000000, 0x362d81ed00000000, + 0xe7c566e300000000, 0xd05fd1cb00000000, 0x01b736c500000000, + 0x728e1ed600000000, 0xa366f9d800000000, 0x1cba718700000000, + 0xcd52968900000000, 0xbe6bbe9a00000000, 0x6f83599400000000, + 0x5819eebc00000000, 0x89f109b200000000, 0xfac821a100000000, + 0x2b20c6af00000000, 0xe56dbc1900000000, 0x34855b1700000000, + 0x47bc730400000000, 0x9654940a00000000, 0xa1ce232200000000, + 0x7026c42c00000000, 0x031fec3f00000000, 0xd2f70b3100000000, + 0x6d2b836e00000000, 0xbcc3646000000000, 0xcffa4c7300000000, + 0x1e12ab7d00000000, 0x29881c5500000000, 0xf860fb5b00000000, + 0x8b59d34800000000, 0x5ab1344600000000, 0xf5e0c2f700000000, + 0x240825f900000000, 0x57310dea00000000, 0x86d9eae400000000, + 0xb1435dcc00000000, 0x60abbac200000000, 0x139292d100000000, + 0xc27a75df00000000, 0x7da6fd8000000000, 0xac4e1a8e00000000, + 0xdf77329d00000000, 0x0e9fd59300000000, 0x390562bb00000000, + 0xe8ed85b500000000, 0x9bd4ada600000000, 0x4a3c4aa800000000, + 0x4649281100000000, 0x97a1cf1f00000000, 0xe498e70c00000000, + 0x3570000200000000, 0x02eab72a00000000, 0xd302502400000000, + 0xa03b783700000000, 0x71d39f3900000000, 0xce0f176600000000, + 0x1fe7f06800000000, 0x6cded87b00000000, 0xbd363f7500000000, + 0x8aac885d00000000, 0x5b446f5300000000, 0x287d474000000000, + 0xf995a04e00000000, 0x56c456ff00000000, 0x872cb1f100000000, + 0xf41599e200000000, 0x25fd7eec00000000, 0x1267c9c400000000, + 0xc38f2eca00000000, 0xb0b606d900000000, 0x615ee1d700000000, + 0xde82698800000000, 0x0f6a8e8600000000, 0x7c53a69500000000, + 0xadbb419b00000000, 0x9a21f6b300000000, 0x4bc911bd00000000, + 0x38f039ae00000000, 0xe918dea000000000, 0x2755a41600000000, + 0xf6bd431800000000, 0x85846b0b00000000, 0x546c8c0500000000, + 0x63f63b2d00000000, 0xb21edc2300000000, 0xc127f43000000000, + 0x10cf133e00000000, 0xaf139b6100000000, 0x7efb7c6f00000000, + 0x0dc2547c00000000, 0xdc2ab37200000000, 0xebb0045a00000000, + 0x3a58e35400000000, 0x4961cb4700000000, 0x98892c4900000000, + 0x37d8daf800000000, 0xe6303df600000000, 0x950915e500000000, + 0x44e1f2eb00000000, 0x737b45c300000000, 0xa293a2cd00000000, + 0xd1aa8ade00000000, 0x00426dd000000000, 0xbf9ee58f00000000, + 0x6e76028100000000, 0x1d4f2a9200000000, 0xcca7cd9c00000000, + 0xfb3d7ab400000000, 0x2ad59dba00000000, 0x59ecb5a900000000, + 0x880452a700000000}, + {0x0000000000000000, 0xaa05daf100000000, 0x150dc53800000000, + 0xbf081fc900000000, 0x2a1a8a7100000000, 0x801f508000000000, + 0x3f174f4900000000, 0x951295b800000000, 0x543414e300000000, + 0xfe31ce1200000000, 0x4139d1db00000000, 0xeb3c0b2a00000000, + 0x7e2e9e9200000000, 0xd42b446300000000, 0x6b235baa00000000, + 0xc126815b00000000, 0xe96e591d00000000, 0x436b83ec00000000, + 0xfc639c2500000000, 0x566646d400000000, 0xc374d36c00000000, + 0x6971099d00000000, 0xd679165400000000, 0x7c7ccca500000000, + 0xbd5a4dfe00000000, 0x175f970f00000000, 0xa85788c600000000, + 0x0252523700000000, 0x9740c78f00000000, 0x3d451d7e00000000, + 0x824d02b700000000, 0x2848d84600000000, 0xd2ddb23a00000000, + 0x78d868cb00000000, 0xc7d0770200000000, 0x6dd5adf300000000, + 0xf8c7384b00000000, 0x52c2e2ba00000000, 0xedcafd7300000000, + 0x47cf278200000000, 0x86e9a6d900000000, 0x2cec7c2800000000, + 0x93e463e100000000, 0x39e1b91000000000, 0xacf32ca800000000, + 0x06f6f65900000000, 0xb9fee99000000000, 0x13fb336100000000, + 0x3bb3eb2700000000, 0x91b631d600000000, 0x2ebe2e1f00000000, + 0x84bbf4ee00000000, 0x11a9615600000000, 0xbbacbba700000000, + 0x04a4a46e00000000, 0xaea17e9f00000000, 0x6f87ffc400000000, + 0xc582253500000000, 0x7a8a3afc00000000, 0xd08fe00d00000000, + 0x459d75b500000000, 0xef98af4400000000, 0x5090b08d00000000, + 0xfa956a7c00000000, 0xa4bb657500000000, 0x0ebebf8400000000, + 0xb1b6a04d00000000, 0x1bb37abc00000000, 0x8ea1ef0400000000, + 0x24a435f500000000, 0x9bac2a3c00000000, 0x31a9f0cd00000000, + 0xf08f719600000000, 0x5a8aab6700000000, 0xe582b4ae00000000, + 0x4f876e5f00000000, 0xda95fbe700000000, 0x7090211600000000, + 0xcf983edf00000000, 0x659de42e00000000, 0x4dd53c6800000000, + 0xe7d0e69900000000, 0x58d8f95000000000, 0xf2dd23a100000000, + 0x67cfb61900000000, 0xcdca6ce800000000, 0x72c2732100000000, + 0xd8c7a9d000000000, 0x19e1288b00000000, 0xb3e4f27a00000000, + 0x0cecedb300000000, 0xa6e9374200000000, 0x33fba2fa00000000, + 0x99fe780b00000000, 0x26f667c200000000, 0x8cf3bd3300000000, + 0x7666d74f00000000, 0xdc630dbe00000000, 0x636b127700000000, + 0xc96ec88600000000, 0x5c7c5d3e00000000, 0xf67987cf00000000, + 0x4971980600000000, 0xe37442f700000000, 0x2252c3ac00000000, + 0x8857195d00000000, 0x375f069400000000, 0x9d5adc6500000000, + 0x084849dd00000000, 0xa24d932c00000000, 0x1d458ce500000000, + 0xb740561400000000, 0x9f088e5200000000, 0x350d54a300000000, + 0x8a054b6a00000000, 0x2000919b00000000, 0xb512042300000000, + 0x1f17ded200000000, 0xa01fc11b00000000, 0x0a1a1bea00000000, + 0xcb3c9ab100000000, 0x6139404000000000, 0xde315f8900000000, + 0x7434857800000000, 0xe12610c000000000, 0x4b23ca3100000000, + 0xf42bd5f800000000, 0x5e2e0f0900000000, 0x4877cbea00000000, + 0xe272111b00000000, 0x5d7a0ed200000000, 0xf77fd42300000000, + 0x626d419b00000000, 0xc8689b6a00000000, 0x776084a300000000, + 0xdd655e5200000000, 0x1c43df0900000000, 0xb64605f800000000, + 0x094e1a3100000000, 0xa34bc0c000000000, 0x3659557800000000, + 0x9c5c8f8900000000, 0x2354904000000000, 0x89514ab100000000, + 0xa11992f700000000, 0x0b1c480600000000, 0xb41457cf00000000, + 0x1e118d3e00000000, 0x8b03188600000000, 0x2106c27700000000, + 0x9e0eddbe00000000, 0x340b074f00000000, 0xf52d861400000000, + 0x5f285ce500000000, 0xe020432c00000000, 0x4a2599dd00000000, + 0xdf370c6500000000, 0x7532d69400000000, 0xca3ac95d00000000, + 0x603f13ac00000000, 0x9aaa79d000000000, 0x30afa32100000000, + 0x8fa7bce800000000, 0x25a2661900000000, 0xb0b0f3a100000000, + 0x1ab5295000000000, 0xa5bd369900000000, 0x0fb8ec6800000000, + 0xce9e6d3300000000, 0x649bb7c200000000, 0xdb93a80b00000000, + 0x719672fa00000000, 0xe484e74200000000, 0x4e813db300000000, + 0xf189227a00000000, 0x5b8cf88b00000000, 0x73c420cd00000000, + 0xd9c1fa3c00000000, 0x66c9e5f500000000, 0xcccc3f0400000000, + 0x59deaabc00000000, 0xf3db704d00000000, 0x4cd36f8400000000, + 0xe6d6b57500000000, 0x27f0342e00000000, 0x8df5eedf00000000, + 0x32fdf11600000000, 0x98f82be700000000, 0x0deabe5f00000000, + 0xa7ef64ae00000000, 0x18e77b6700000000, 0xb2e2a19600000000, + 0xecccae9f00000000, 0x46c9746e00000000, 0xf9c16ba700000000, + 0x53c4b15600000000, 0xc6d624ee00000000, 0x6cd3fe1f00000000, + 0xd3dbe1d600000000, 0x79de3b2700000000, 0xb8f8ba7c00000000, + 0x12fd608d00000000, 0xadf57f4400000000, 0x07f0a5b500000000, + 0x92e2300d00000000, 0x38e7eafc00000000, 0x87eff53500000000, + 0x2dea2fc400000000, 0x05a2f78200000000, 0xafa72d7300000000, + 0x10af32ba00000000, 0xbaaae84b00000000, 0x2fb87df300000000, + 0x85bda70200000000, 0x3ab5b8cb00000000, 0x90b0623a00000000, + 0x5196e36100000000, 0xfb93399000000000, 0x449b265900000000, + 0xee9efca800000000, 0x7b8c691000000000, 0xd189b3e100000000, + 0x6e81ac2800000000, 0xc48476d900000000, 0x3e111ca500000000, + 0x9414c65400000000, 0x2b1cd99d00000000, 0x8119036c00000000, + 0x140b96d400000000, 0xbe0e4c2500000000, 0x010653ec00000000, + 0xab03891d00000000, 0x6a25084600000000, 0xc020d2b700000000, + 0x7f28cd7e00000000, 0xd52d178f00000000, 0x403f823700000000, + 0xea3a58c600000000, 0x5532470f00000000, 0xff379dfe00000000, + 0xd77f45b800000000, 0x7d7a9f4900000000, 0xc272808000000000, + 0x68775a7100000000, 0xfd65cfc900000000, 0x5760153800000000, + 0xe8680af100000000, 0x426dd00000000000, 0x834b515b00000000, + 0x294e8baa00000000, 0x9646946300000000, 0x3c434e9200000000, + 0xa951db2a00000000, 0x035401db00000000, 0xbc5c1e1200000000, + 0x1659c4e300000000}}; + +#else /* W == 4 */ + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0xae689191, 0x87a02563, 0x29c8b4f2, 0xd4314c87, + 0x7a59dd16, 0x539169e4, 0xfdf9f875, 0x73139f4f, 0xdd7b0ede, + 0xf4b3ba2c, 0x5adb2bbd, 0xa722d3c8, 0x094a4259, 0x2082f6ab, + 0x8eea673a, 0xe6273e9e, 0x484faf0f, 0x61871bfd, 0xcfef8a6c, + 0x32167219, 0x9c7ee388, 0xb5b6577a, 0x1bdec6eb, 0x9534a1d1, + 0x3b5c3040, 0x129484b2, 0xbcfc1523, 0x4105ed56, 0xef6d7cc7, + 0xc6a5c835, 0x68cd59a4, 0x173f7b7d, 0xb957eaec, 0x909f5e1e, + 0x3ef7cf8f, 0xc30e37fa, 0x6d66a66b, 0x44ae1299, 0xeac68308, + 0x642ce432, 0xca4475a3, 0xe38cc151, 0x4de450c0, 0xb01da8b5, + 0x1e753924, 0x37bd8dd6, 0x99d51c47, 0xf11845e3, 0x5f70d472, + 0x76b86080, 0xd8d0f111, 0x25290964, 0x8b4198f5, 0xa2892c07, + 0x0ce1bd96, 0x820bdaac, 0x2c634b3d, 0x05abffcf, 0xabc36e5e, + 0x563a962b, 0xf85207ba, 0xd19ab348, 0x7ff222d9, 0x2e7ef6fa, + 0x8016676b, 0xa9ded399, 0x07b64208, 0xfa4fba7d, 0x54272bec, + 0x7def9f1e, 0xd3870e8f, 0x5d6d69b5, 0xf305f824, 0xdacd4cd6, + 0x74a5dd47, 0x895c2532, 0x2734b4a3, 0x0efc0051, 0xa09491c0, + 0xc859c864, 0x663159f5, 0x4ff9ed07, 0xe1917c96, 0x1c6884e3, + 0xb2001572, 0x9bc8a180, 0x35a03011, 0xbb4a572b, 0x1522c6ba, + 0x3cea7248, 0x9282e3d9, 0x6f7b1bac, 0xc1138a3d, 0xe8db3ecf, + 0x46b3af5e, 0x39418d87, 0x97291c16, 0xbee1a8e4, 0x10893975, + 0xed70c100, 0x43185091, 0x6ad0e463, 0xc4b875f2, 0x4a5212c8, + 0xe43a8359, 0xcdf237ab, 0x639aa63a, 0x9e635e4f, 0x300bcfde, + 0x19c37b2c, 0xb7abeabd, 0xdf66b319, 0x710e2288, 0x58c6967a, + 0xf6ae07eb, 0x0b57ff9e, 0xa53f6e0f, 0x8cf7dafd, 0x229f4b6c, + 0xac752c56, 0x021dbdc7, 0x2bd50935, 0x85bd98a4, 0x784460d1, + 0xd62cf140, 0xffe445b2, 0x518cd423, 0x5cfdedf4, 0xf2957c65, + 0xdb5dc897, 0x75355906, 0x88cca173, 0x26a430e2, 0x0f6c8410, + 0xa1041581, 0x2fee72bb, 0x8186e32a, 0xa84e57d8, 0x0626c649, + 0xfbdf3e3c, 0x55b7afad, 0x7c7f1b5f, 0xd2178ace, 0xbadad36a, + 0x14b242fb, 0x3d7af609, 0x93126798, 0x6eeb9fed, 0xc0830e7c, + 0xe94bba8e, 0x47232b1f, 0xc9c94c25, 0x67a1ddb4, 0x4e696946, + 0xe001f8d7, 0x1df800a2, 0xb3909133, 0x9a5825c1, 0x3430b450, + 0x4bc29689, 0xe5aa0718, 0xcc62b3ea, 0x620a227b, 0x9ff3da0e, + 0x319b4b9f, 0x1853ff6d, 0xb63b6efc, 0x38d109c6, 0x96b99857, + 0xbf712ca5, 0x1119bd34, 0xece04541, 0x4288d4d0, 0x6b406022, + 0xc528f1b3, 0xade5a817, 0x038d3986, 0x2a458d74, 0x842d1ce5, + 0x79d4e490, 0xd7bc7501, 0xfe74c1f3, 0x501c5062, 0xdef63758, + 0x709ea6c9, 0x5956123b, 0xf73e83aa, 0x0ac77bdf, 0xa4afea4e, + 0x8d675ebc, 0x230fcf2d, 0x72831b0e, 0xdceb8a9f, 0xf5233e6d, + 0x5b4baffc, 0xa6b25789, 0x08dac618, 0x211272ea, 0x8f7ae37b, + 0x01908441, 0xaff815d0, 0x8630a122, 0x285830b3, 0xd5a1c8c6, + 0x7bc95957, 0x5201eda5, 0xfc697c34, 0x94a42590, 0x3accb401, + 0x130400f3, 0xbd6c9162, 0x40956917, 0xeefdf886, 0xc7354c74, + 0x695ddde5, 0xe7b7badf, 0x49df2b4e, 0x60179fbc, 0xce7f0e2d, + 0x3386f658, 0x9dee67c9, 0xb426d33b, 0x1a4e42aa, 0x65bc6073, + 0xcbd4f1e2, 0xe21c4510, 0x4c74d481, 0xb18d2cf4, 0x1fe5bd65, + 0x362d0997, 0x98459806, 0x16afff3c, 0xb8c76ead, 0x910fda5f, + 0x3f674bce, 0xc29eb3bb, 0x6cf6222a, 0x453e96d8, 0xeb560749, + 0x839b5eed, 0x2df3cf7c, 0x043b7b8e, 0xaa53ea1f, 0x57aa126a, + 0xf9c283fb, 0xd00a3709, 0x7e62a698, 0xf088c1a2, 0x5ee05033, + 0x7728e4c1, 0xd9407550, 0x24b98d25, 0x8ad11cb4, 0xa319a846, + 0x0d7139d7}, + {0x00000000, 0xb9fbdbe8, 0xa886b191, 0x117d6a79, 0x8a7c6563, + 0x3387be8b, 0x22fad4f2, 0x9b010f1a, 0xcf89cc87, 0x7672176f, + 0x670f7d16, 0xdef4a6fe, 0x45f5a9e4, 0xfc0e720c, 0xed731875, + 0x5488c39d, 0x44629f4f, 0xfd9944a7, 0xece42ede, 0x551ff536, + 0xce1efa2c, 0x77e521c4, 0x66984bbd, 0xdf639055, 0x8beb53c8, + 0x32108820, 0x236de259, 0x9a9639b1, 0x019736ab, 0xb86ced43, + 0xa911873a, 0x10ea5cd2, 0x88c53e9e, 0x313ee576, 0x20438f0f, + 0x99b854e7, 0x02b95bfd, 0xbb428015, 0xaa3fea6c, 0x13c43184, + 0x474cf219, 0xfeb729f1, 0xefca4388, 0x56319860, 0xcd30977a, + 0x74cb4c92, 0x65b626eb, 0xdc4dfd03, 0xcca7a1d1, 0x755c7a39, + 0x64211040, 0xdddacba8, 0x46dbc4b2, 0xff201f5a, 0xee5d7523, + 0x57a6aecb, 0x032e6d56, 0xbad5b6be, 0xaba8dcc7, 0x1253072f, + 0x89520835, 0x30a9d3dd, 0x21d4b9a4, 0x982f624c, 0xcafb7b7d, + 0x7300a095, 0x627dcaec, 0xdb861104, 0x40871e1e, 0xf97cc5f6, + 0xe801af8f, 0x51fa7467, 0x0572b7fa, 0xbc896c12, 0xadf4066b, + 0x140fdd83, 0x8f0ed299, 0x36f50971, 0x27886308, 0x9e73b8e0, + 0x8e99e432, 0x37623fda, 0x261f55a3, 0x9fe48e4b, 0x04e58151, + 0xbd1e5ab9, 0xac6330c0, 0x1598eb28, 0x411028b5, 0xf8ebf35d, + 0xe9969924, 0x506d42cc, 0xcb6c4dd6, 0x7297963e, 0x63eafc47, + 0xda1127af, 0x423e45e3, 0xfbc59e0b, 0xeab8f472, 0x53432f9a, + 0xc8422080, 0x71b9fb68, 0x60c49111, 0xd93f4af9, 0x8db78964, + 0x344c528c, 0x253138f5, 0x9ccae31d, 0x07cbec07, 0xbe3037ef, + 0xaf4d5d96, 0x16b6867e, 0x065cdaac, 0xbfa70144, 0xaeda6b3d, + 0x1721b0d5, 0x8c20bfcf, 0x35db6427, 0x24a60e5e, 0x9d5dd5b6, + 0xc9d5162b, 0x702ecdc3, 0x6153a7ba, 0xd8a87c52, 0x43a97348, + 0xfa52a8a0, 0xeb2fc2d9, 0x52d41931, 0x4e87f0bb, 0xf77c2b53, + 0xe601412a, 0x5ffa9ac2, 0xc4fb95d8, 0x7d004e30, 0x6c7d2449, + 0xd586ffa1, 0x810e3c3c, 0x38f5e7d4, 0x29888dad, 0x90735645, + 0x0b72595f, 0xb28982b7, 0xa3f4e8ce, 0x1a0f3326, 0x0ae56ff4, + 0xb31eb41c, 0xa263de65, 0x1b98058d, 0x80990a97, 0x3962d17f, + 0x281fbb06, 0x91e460ee, 0xc56ca373, 0x7c97789b, 0x6dea12e2, + 0xd411c90a, 0x4f10c610, 0xf6eb1df8, 0xe7967781, 0x5e6dac69, + 0xc642ce25, 0x7fb915cd, 0x6ec47fb4, 0xd73fa45c, 0x4c3eab46, + 0xf5c570ae, 0xe4b81ad7, 0x5d43c13f, 0x09cb02a2, 0xb030d94a, + 0xa14db333, 0x18b668db, 0x83b767c1, 0x3a4cbc29, 0x2b31d650, + 0x92ca0db8, 0x8220516a, 0x3bdb8a82, 0x2aa6e0fb, 0x935d3b13, + 0x085c3409, 0xb1a7efe1, 0xa0da8598, 0x19215e70, 0x4da99ded, + 0xf4524605, 0xe52f2c7c, 0x5cd4f794, 0xc7d5f88e, 0x7e2e2366, + 0x6f53491f, 0xd6a892f7, 0x847c8bc6, 0x3d87502e, 0x2cfa3a57, + 0x9501e1bf, 0x0e00eea5, 0xb7fb354d, 0xa6865f34, 0x1f7d84dc, + 0x4bf54741, 0xf20e9ca9, 0xe373f6d0, 0x5a882d38, 0xc1892222, + 0x7872f9ca, 0x690f93b3, 0xd0f4485b, 0xc01e1489, 0x79e5cf61, + 0x6898a518, 0xd1637ef0, 0x4a6271ea, 0xf399aa02, 0xe2e4c07b, + 0x5b1f1b93, 0x0f97d80e, 0xb66c03e6, 0xa711699f, 0x1eeab277, + 0x85ebbd6d, 0x3c106685, 0x2d6d0cfc, 0x9496d714, 0x0cb9b558, + 0xb5426eb0, 0xa43f04c9, 0x1dc4df21, 0x86c5d03b, 0x3f3e0bd3, + 0x2e4361aa, 0x97b8ba42, 0xc33079df, 0x7acba237, 0x6bb6c84e, + 0xd24d13a6, 0x494c1cbc, 0xf0b7c754, 0xe1caad2d, 0x583176c5, + 0x48db2a17, 0xf120f1ff, 0xe05d9b86, 0x59a6406e, 0xc2a74f74, + 0x7b5c949c, 0x6a21fee5, 0xd3da250d, 0x8752e690, 0x3ea93d78, + 0x2fd45701, 0x962f8ce9, 0x0d2e83f3, 0xb4d5581b, 0xa5a83262, + 0x1c53e98a}, + {0x00000000, 0x9d0fe176, 0xe16ec4ad, 0x7c6125db, 0x19ac8f1b, + 0x84a36e6d, 0xf8c24bb6, 0x65cdaac0, 0x33591e36, 0xae56ff40, + 0xd237da9b, 0x4f383bed, 0x2af5912d, 0xb7fa705b, 0xcb9b5580, + 0x5694b4f6, 0x66b23c6c, 0xfbbddd1a, 0x87dcf8c1, 0x1ad319b7, + 0x7f1eb377, 0xe2115201, 0x9e7077da, 0x037f96ac, 0x55eb225a, + 0xc8e4c32c, 0xb485e6f7, 0x298a0781, 0x4c47ad41, 0xd1484c37, + 0xad2969ec, 0x3026889a, 0xcd6478d8, 0x506b99ae, 0x2c0abc75, + 0xb1055d03, 0xd4c8f7c3, 0x49c716b5, 0x35a6336e, 0xa8a9d218, + 0xfe3d66ee, 0x63328798, 0x1f53a243, 0x825c4335, 0xe791e9f5, + 0x7a9e0883, 0x06ff2d58, 0x9bf0cc2e, 0xabd644b4, 0x36d9a5c2, + 0x4ab88019, 0xd7b7616f, 0xb27acbaf, 0x2f752ad9, 0x53140f02, + 0xce1bee74, 0x988f5a82, 0x0580bbf4, 0x79e19e2f, 0xe4ee7f59, + 0x8123d599, 0x1c2c34ef, 0x604d1134, 0xfd42f042, 0x41b9f7f1, + 0xdcb61687, 0xa0d7335c, 0x3dd8d22a, 0x581578ea, 0xc51a999c, + 0xb97bbc47, 0x24745d31, 0x72e0e9c7, 0xefef08b1, 0x938e2d6a, + 0x0e81cc1c, 0x6b4c66dc, 0xf64387aa, 0x8a22a271, 0x172d4307, + 0x270bcb9d, 0xba042aeb, 0xc6650f30, 0x5b6aee46, 0x3ea74486, + 0xa3a8a5f0, 0xdfc9802b, 0x42c6615d, 0x1452d5ab, 0x895d34dd, + 0xf53c1106, 0x6833f070, 0x0dfe5ab0, 0x90f1bbc6, 0xec909e1d, + 0x719f7f6b, 0x8cdd8f29, 0x11d26e5f, 0x6db34b84, 0xf0bcaaf2, + 0x95710032, 0x087ee144, 0x741fc49f, 0xe91025e9, 0xbf84911f, + 0x228b7069, 0x5eea55b2, 0xc3e5b4c4, 0xa6281e04, 0x3b27ff72, + 0x4746daa9, 0xda493bdf, 0xea6fb345, 0x77605233, 0x0b0177e8, + 0x960e969e, 0xf3c33c5e, 0x6eccdd28, 0x12adf8f3, 0x8fa21985, + 0xd936ad73, 0x44394c05, 0x385869de, 0xa55788a8, 0xc09a2268, + 0x5d95c31e, 0x21f4e6c5, 0xbcfb07b3, 0x8373efe2, 0x1e7c0e94, + 0x621d2b4f, 0xff12ca39, 0x9adf60f9, 0x07d0818f, 0x7bb1a454, + 0xe6be4522, 0xb02af1d4, 0x2d2510a2, 0x51443579, 0xcc4bd40f, + 0xa9867ecf, 0x34899fb9, 0x48e8ba62, 0xd5e75b14, 0xe5c1d38e, + 0x78ce32f8, 0x04af1723, 0x99a0f655, 0xfc6d5c95, 0x6162bde3, + 0x1d039838, 0x800c794e, 0xd698cdb8, 0x4b972cce, 0x37f60915, + 0xaaf9e863, 0xcf3442a3, 0x523ba3d5, 0x2e5a860e, 0xb3556778, + 0x4e17973a, 0xd318764c, 0xaf795397, 0x3276b2e1, 0x57bb1821, + 0xcab4f957, 0xb6d5dc8c, 0x2bda3dfa, 0x7d4e890c, 0xe041687a, + 0x9c204da1, 0x012facd7, 0x64e20617, 0xf9ede761, 0x858cc2ba, + 0x188323cc, 0x28a5ab56, 0xb5aa4a20, 0xc9cb6ffb, 0x54c48e8d, + 0x3109244d, 0xac06c53b, 0xd067e0e0, 0x4d680196, 0x1bfcb560, + 0x86f35416, 0xfa9271cd, 0x679d90bb, 0x02503a7b, 0x9f5fdb0d, + 0xe33efed6, 0x7e311fa0, 0xc2ca1813, 0x5fc5f965, 0x23a4dcbe, + 0xbeab3dc8, 0xdb669708, 0x4669767e, 0x3a0853a5, 0xa707b2d3, + 0xf1930625, 0x6c9ce753, 0x10fdc288, 0x8df223fe, 0xe83f893e, + 0x75306848, 0x09514d93, 0x945eace5, 0xa478247f, 0x3977c509, + 0x4516e0d2, 0xd81901a4, 0xbdd4ab64, 0x20db4a12, 0x5cba6fc9, + 0xc1b58ebf, 0x97213a49, 0x0a2edb3f, 0x764ffee4, 0xeb401f92, + 0x8e8db552, 0x13825424, 0x6fe371ff, 0xf2ec9089, 0x0fae60cb, + 0x92a181bd, 0xeec0a466, 0x73cf4510, 0x1602efd0, 0x8b0d0ea6, + 0xf76c2b7d, 0x6a63ca0b, 0x3cf77efd, 0xa1f89f8b, 0xdd99ba50, + 0x40965b26, 0x255bf1e6, 0xb8541090, 0xc435354b, 0x593ad43d, + 0x691c5ca7, 0xf413bdd1, 0x8872980a, 0x157d797c, 0x70b0d3bc, + 0xedbf32ca, 0x91de1711, 0x0cd1f667, 0x5a454291, 0xc74aa3e7, + 0xbb2b863c, 0x2624674a, 0x43e9cd8a, 0xdee62cfc, 0xa2870927, + 0x3f88e851}, + {0x00000000, 0xdd96d985, 0x605cb54b, 0xbdca6cce, 0xc0b96a96, + 0x1d2fb313, 0xa0e5dfdd, 0x7d730658, 0x5a03d36d, 0x87950ae8, + 0x3a5f6626, 0xe7c9bfa3, 0x9abab9fb, 0x472c607e, 0xfae60cb0, + 0x2770d535, 0xb407a6da, 0x69917f5f, 0xd45b1391, 0x09cdca14, + 0x74becc4c, 0xa92815c9, 0x14e27907, 0xc974a082, 0xee0475b7, + 0x3392ac32, 0x8e58c0fc, 0x53ce1979, 0x2ebd1f21, 0xf32bc6a4, + 0x4ee1aa6a, 0x937773ef, 0xb37e4bf5, 0x6ee89270, 0xd322febe, + 0x0eb4273b, 0x73c72163, 0xae51f8e6, 0x139b9428, 0xce0d4dad, + 0xe97d9898, 0x34eb411d, 0x89212dd3, 0x54b7f456, 0x29c4f20e, + 0xf4522b8b, 0x49984745, 0x940e9ec0, 0x0779ed2f, 0xdaef34aa, + 0x67255864, 0xbab381e1, 0xc7c087b9, 0x1a565e3c, 0xa79c32f2, + 0x7a0aeb77, 0x5d7a3e42, 0x80ece7c7, 0x3d268b09, 0xe0b0528c, + 0x9dc354d4, 0x40558d51, 0xfd9fe19f, 0x2009381a, 0xbd8d91ab, + 0x601b482e, 0xddd124e0, 0x0047fd65, 0x7d34fb3d, 0xa0a222b8, + 0x1d684e76, 0xc0fe97f3, 0xe78e42c6, 0x3a189b43, 0x87d2f78d, + 0x5a442e08, 0x27372850, 0xfaa1f1d5, 0x476b9d1b, 0x9afd449e, + 0x098a3771, 0xd41ceef4, 0x69d6823a, 0xb4405bbf, 0xc9335de7, + 0x14a58462, 0xa96fe8ac, 0x74f93129, 0x5389e41c, 0x8e1f3d99, + 0x33d55157, 0xee4388d2, 0x93308e8a, 0x4ea6570f, 0xf36c3bc1, + 0x2efae244, 0x0ef3da5e, 0xd36503db, 0x6eaf6f15, 0xb339b690, + 0xce4ab0c8, 0x13dc694d, 0xae160583, 0x7380dc06, 0x54f00933, + 0x8966d0b6, 0x34acbc78, 0xe93a65fd, 0x944963a5, 0x49dfba20, + 0xf415d6ee, 0x29830f6b, 0xbaf47c84, 0x6762a501, 0xdaa8c9cf, + 0x073e104a, 0x7a4d1612, 0xa7dbcf97, 0x1a11a359, 0xc7877adc, + 0xe0f7afe9, 0x3d61766c, 0x80ab1aa2, 0x5d3dc327, 0x204ec57f, + 0xfdd81cfa, 0x40127034, 0x9d84a9b1, 0xa06a2517, 0x7dfcfc92, + 0xc036905c, 0x1da049d9, 0x60d34f81, 0xbd459604, 0x008ffaca, + 0xdd19234f, 0xfa69f67a, 0x27ff2fff, 0x9a354331, 0x47a39ab4, + 0x3ad09cec, 0xe7464569, 0x5a8c29a7, 0x871af022, 0x146d83cd, + 0xc9fb5a48, 0x74313686, 0xa9a7ef03, 0xd4d4e95b, 0x094230de, + 0xb4885c10, 0x691e8595, 0x4e6e50a0, 0x93f88925, 0x2e32e5eb, + 0xf3a43c6e, 0x8ed73a36, 0x5341e3b3, 0xee8b8f7d, 0x331d56f8, + 0x13146ee2, 0xce82b767, 0x7348dba9, 0xaede022c, 0xd3ad0474, + 0x0e3bddf1, 0xb3f1b13f, 0x6e6768ba, 0x4917bd8f, 0x9481640a, + 0x294b08c4, 0xf4ddd141, 0x89aed719, 0x54380e9c, 0xe9f26252, + 0x3464bbd7, 0xa713c838, 0x7a8511bd, 0xc74f7d73, 0x1ad9a4f6, + 0x67aaa2ae, 0xba3c7b2b, 0x07f617e5, 0xda60ce60, 0xfd101b55, + 0x2086c2d0, 0x9d4cae1e, 0x40da779b, 0x3da971c3, 0xe03fa846, + 0x5df5c488, 0x80631d0d, 0x1de7b4bc, 0xc0716d39, 0x7dbb01f7, + 0xa02dd872, 0xdd5ede2a, 0x00c807af, 0xbd026b61, 0x6094b2e4, + 0x47e467d1, 0x9a72be54, 0x27b8d29a, 0xfa2e0b1f, 0x875d0d47, + 0x5acbd4c2, 0xe701b80c, 0x3a976189, 0xa9e01266, 0x7476cbe3, + 0xc9bca72d, 0x142a7ea8, 0x695978f0, 0xb4cfa175, 0x0905cdbb, + 0xd493143e, 0xf3e3c10b, 0x2e75188e, 0x93bf7440, 0x4e29adc5, + 0x335aab9d, 0xeecc7218, 0x53061ed6, 0x8e90c753, 0xae99ff49, + 0x730f26cc, 0xcec54a02, 0x13539387, 0x6e2095df, 0xb3b64c5a, + 0x0e7c2094, 0xd3eaf911, 0xf49a2c24, 0x290cf5a1, 0x94c6996f, + 0x495040ea, 0x342346b2, 0xe9b59f37, 0x547ff3f9, 0x89e92a7c, + 0x1a9e5993, 0xc7088016, 0x7ac2ecd8, 0xa754355d, 0xda273305, + 0x07b1ea80, 0xba7b864e, 0x67ed5fcb, 0x409d8afe, 0x9d0b537b, + 0x20c13fb5, 0xfd57e630, 0x8024e068, 0x5db239ed, 0xe0785523, + 0x3dee8ca6}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x00000000, 0x85d996dd, 0x4bb55c60, 0xce6ccabd, 0x966ab9c0, + 0x13b32f1d, 0xdddfe5a0, 0x5806737d, 0x6dd3035a, 0xe80a9587, + 0x26665f3a, 0xa3bfc9e7, 0xfbb9ba9a, 0x7e602c47, 0xb00ce6fa, + 0x35d57027, 0xdaa607b4, 0x5f7f9169, 0x91135bd4, 0x14cacd09, + 0x4cccbe74, 0xc91528a9, 0x0779e214, 0x82a074c9, 0xb77504ee, + 0x32ac9233, 0xfcc0588e, 0x7919ce53, 0x211fbd2e, 0xa4c62bf3, + 0x6aaae14e, 0xef737793, 0xf54b7eb3, 0x7092e86e, 0xbefe22d3, + 0x3b27b40e, 0x6321c773, 0xe6f851ae, 0x28949b13, 0xad4d0dce, + 0x98987de9, 0x1d41eb34, 0xd32d2189, 0x56f4b754, 0x0ef2c429, + 0x8b2b52f4, 0x45479849, 0xc09e0e94, 0x2fed7907, 0xaa34efda, + 0x64582567, 0xe181b3ba, 0xb987c0c7, 0x3c5e561a, 0xf2329ca7, + 0x77eb0a7a, 0x423e7a5d, 0xc7e7ec80, 0x098b263d, 0x8c52b0e0, + 0xd454c39d, 0x518d5540, 0x9fe19ffd, 0x1a380920, 0xab918dbd, + 0x2e481b60, 0xe024d1dd, 0x65fd4700, 0x3dfb347d, 0xb822a2a0, + 0x764e681d, 0xf397fec0, 0xc6428ee7, 0x439b183a, 0x8df7d287, + 0x082e445a, 0x50283727, 0xd5f1a1fa, 0x1b9d6b47, 0x9e44fd9a, + 0x71378a09, 0xf4ee1cd4, 0x3a82d669, 0xbf5b40b4, 0xe75d33c9, + 0x6284a514, 0xace86fa9, 0x2931f974, 0x1ce48953, 0x993d1f8e, + 0x5751d533, 0xd28843ee, 0x8a8e3093, 0x0f57a64e, 0xc13b6cf3, + 0x44e2fa2e, 0x5edaf30e, 0xdb0365d3, 0x156faf6e, 0x90b639b3, + 0xc8b04ace, 0x4d69dc13, 0x830516ae, 0x06dc8073, 0x3309f054, + 0xb6d06689, 0x78bcac34, 0xfd653ae9, 0xa5634994, 0x20badf49, + 0xeed615f4, 0x6b0f8329, 0x847cf4ba, 0x01a56267, 0xcfc9a8da, + 0x4a103e07, 0x12164d7a, 0x97cfdba7, 0x59a3111a, 0xdc7a87c7, + 0xe9aff7e0, 0x6c76613d, 0xa21aab80, 0x27c33d5d, 0x7fc54e20, + 0xfa1cd8fd, 0x34701240, 0xb1a9849d, 0x17256aa0, 0x92fcfc7d, + 0x5c9036c0, 0xd949a01d, 0x814fd360, 0x049645bd, 0xcafa8f00, + 0x4f2319dd, 0x7af669fa, 0xff2fff27, 0x3143359a, 0xb49aa347, + 0xec9cd03a, 0x694546e7, 0xa7298c5a, 0x22f01a87, 0xcd836d14, + 0x485afbc9, 0x86363174, 0x03efa7a9, 0x5be9d4d4, 0xde304209, + 0x105c88b4, 0x95851e69, 0xa0506e4e, 0x2589f893, 0xebe5322e, + 0x6e3ca4f3, 0x363ad78e, 0xb3e34153, 0x7d8f8bee, 0xf8561d33, + 0xe26e1413, 0x67b782ce, 0xa9db4873, 0x2c02deae, 0x7404add3, + 0xf1dd3b0e, 0x3fb1f1b3, 0xba68676e, 0x8fbd1749, 0x0a648194, + 0xc4084b29, 0x41d1ddf4, 0x19d7ae89, 0x9c0e3854, 0x5262f2e9, + 0xd7bb6434, 0x38c813a7, 0xbd11857a, 0x737d4fc7, 0xf6a4d91a, + 0xaea2aa67, 0x2b7b3cba, 0xe517f607, 0x60ce60da, 0x551b10fd, + 0xd0c28620, 0x1eae4c9d, 0x9b77da40, 0xc371a93d, 0x46a83fe0, + 0x88c4f55d, 0x0d1d6380, 0xbcb4e71d, 0x396d71c0, 0xf701bb7d, + 0x72d82da0, 0x2ade5edd, 0xaf07c800, 0x616b02bd, 0xe4b29460, + 0xd167e447, 0x54be729a, 0x9ad2b827, 0x1f0b2efa, 0x470d5d87, + 0xc2d4cb5a, 0x0cb801e7, 0x8961973a, 0x6612e0a9, 0xe3cb7674, + 0x2da7bcc9, 0xa87e2a14, 0xf0785969, 0x75a1cfb4, 0xbbcd0509, + 0x3e1493d4, 0x0bc1e3f3, 0x8e18752e, 0x4074bf93, 0xc5ad294e, + 0x9dab5a33, 0x1872ccee, 0xd61e0653, 0x53c7908e, 0x49ff99ae, + 0xcc260f73, 0x024ac5ce, 0x87935313, 0xdf95206e, 0x5a4cb6b3, + 0x94207c0e, 0x11f9ead3, 0x242c9af4, 0xa1f50c29, 0x6f99c694, + 0xea405049, 0xb2462334, 0x379fb5e9, 0xf9f37f54, 0x7c2ae989, + 0x93599e1a, 0x168008c7, 0xd8ecc27a, 0x5d3554a7, 0x053327da, + 0x80eab107, 0x4e867bba, 0xcb5fed67, 0xfe8a9d40, 0x7b530b9d, + 0xb53fc120, 0x30e657fd, 0x68e02480, 0xed39b25d, 0x235578e0, + 0xa68cee3d}, + {0x00000000, 0x76e10f9d, 0xadc46ee1, 0xdb25617c, 0x1b8fac19, + 0x6d6ea384, 0xb64bc2f8, 0xc0aacd65, 0x361e5933, 0x40ff56ae, + 0x9bda37d2, 0xed3b384f, 0x2d91f52a, 0x5b70fab7, 0x80559bcb, + 0xf6b49456, 0x6c3cb266, 0x1addbdfb, 0xc1f8dc87, 0xb719d31a, + 0x77b31e7f, 0x015211e2, 0xda77709e, 0xac967f03, 0x5a22eb55, + 0x2cc3e4c8, 0xf7e685b4, 0x81078a29, 0x41ad474c, 0x374c48d1, + 0xec6929ad, 0x9a882630, 0xd87864cd, 0xae996b50, 0x75bc0a2c, + 0x035d05b1, 0xc3f7c8d4, 0xb516c749, 0x6e33a635, 0x18d2a9a8, + 0xee663dfe, 0x98873263, 0x43a2531f, 0x35435c82, 0xf5e991e7, + 0x83089e7a, 0x582dff06, 0x2eccf09b, 0xb444d6ab, 0xc2a5d936, + 0x1980b84a, 0x6f61b7d7, 0xafcb7ab2, 0xd92a752f, 0x020f1453, + 0x74ee1bce, 0x825a8f98, 0xf4bb8005, 0x2f9ee179, 0x597feee4, + 0x99d52381, 0xef342c1c, 0x34114d60, 0x42f042fd, 0xf1f7b941, + 0x8716b6dc, 0x5c33d7a0, 0x2ad2d83d, 0xea781558, 0x9c991ac5, + 0x47bc7bb9, 0x315d7424, 0xc7e9e072, 0xb108efef, 0x6a2d8e93, + 0x1ccc810e, 0xdc664c6b, 0xaa8743f6, 0x71a2228a, 0x07432d17, + 0x9dcb0b27, 0xeb2a04ba, 0x300f65c6, 0x46ee6a5b, 0x8644a73e, + 0xf0a5a8a3, 0x2b80c9df, 0x5d61c642, 0xabd55214, 0xdd345d89, + 0x06113cf5, 0x70f03368, 0xb05afe0d, 0xc6bbf190, 0x1d9e90ec, + 0x6b7f9f71, 0x298fdd8c, 0x5f6ed211, 0x844bb36d, 0xf2aabcf0, + 0x32007195, 0x44e17e08, 0x9fc41f74, 0xe92510e9, 0x1f9184bf, + 0x69708b22, 0xb255ea5e, 0xc4b4e5c3, 0x041e28a6, 0x72ff273b, + 0xa9da4647, 0xdf3b49da, 0x45b36fea, 0x33526077, 0xe877010b, + 0x9e960e96, 0x5e3cc3f3, 0x28ddcc6e, 0xf3f8ad12, 0x8519a28f, + 0x73ad36d9, 0x054c3944, 0xde695838, 0xa88857a5, 0x68229ac0, + 0x1ec3955d, 0xc5e6f421, 0xb307fbbc, 0xe2ef7383, 0x940e7c1e, + 0x4f2b1d62, 0x39ca12ff, 0xf960df9a, 0x8f81d007, 0x54a4b17b, + 0x2245bee6, 0xd4f12ab0, 0xa210252d, 0x79354451, 0x0fd44bcc, + 0xcf7e86a9, 0xb99f8934, 0x62bae848, 0x145be7d5, 0x8ed3c1e5, + 0xf832ce78, 0x2317af04, 0x55f6a099, 0x955c6dfc, 0xe3bd6261, + 0x3898031d, 0x4e790c80, 0xb8cd98d6, 0xce2c974b, 0x1509f637, + 0x63e8f9aa, 0xa34234cf, 0xd5a33b52, 0x0e865a2e, 0x786755b3, + 0x3a97174e, 0x4c7618d3, 0x975379af, 0xe1b27632, 0x2118bb57, + 0x57f9b4ca, 0x8cdcd5b6, 0xfa3dda2b, 0x0c894e7d, 0x7a6841e0, + 0xa14d209c, 0xd7ac2f01, 0x1706e264, 0x61e7edf9, 0xbac28c85, + 0xcc238318, 0x56aba528, 0x204aaab5, 0xfb6fcbc9, 0x8d8ec454, + 0x4d240931, 0x3bc506ac, 0xe0e067d0, 0x9601684d, 0x60b5fc1b, + 0x1654f386, 0xcd7192fa, 0xbb909d67, 0x7b3a5002, 0x0ddb5f9f, + 0xd6fe3ee3, 0xa01f317e, 0x1318cac2, 0x65f9c55f, 0xbedca423, + 0xc83dabbe, 0x089766db, 0x7e766946, 0xa553083a, 0xd3b207a7, + 0x250693f1, 0x53e79c6c, 0x88c2fd10, 0xfe23f28d, 0x3e893fe8, + 0x48683075, 0x934d5109, 0xe5ac5e94, 0x7f2478a4, 0x09c57739, + 0xd2e01645, 0xa40119d8, 0x64abd4bd, 0x124adb20, 0xc96fba5c, + 0xbf8eb5c1, 0x493a2197, 0x3fdb2e0a, 0xe4fe4f76, 0x921f40eb, + 0x52b58d8e, 0x24548213, 0xff71e36f, 0x8990ecf2, 0xcb60ae0f, + 0xbd81a192, 0x66a4c0ee, 0x1045cf73, 0xd0ef0216, 0xa60e0d8b, + 0x7d2b6cf7, 0x0bca636a, 0xfd7ef73c, 0x8b9ff8a1, 0x50ba99dd, + 0x265b9640, 0xe6f15b25, 0x901054b8, 0x4b3535c4, 0x3dd43a59, + 0xa75c1c69, 0xd1bd13f4, 0x0a987288, 0x7c797d15, 0xbcd3b070, + 0xca32bfed, 0x1117de91, 0x67f6d10c, 0x9142455a, 0xe7a34ac7, + 0x3c862bbb, 0x4a672426, 0x8acde943, 0xfc2ce6de, 0x270987a2, + 0x51e8883f}, + {0x00000000, 0xe8dbfbb9, 0x91b186a8, 0x796a7d11, 0x63657c8a, + 0x8bbe8733, 0xf2d4fa22, 0x1a0f019b, 0x87cc89cf, 0x6f177276, + 0x167d0f67, 0xfea6f4de, 0xe4a9f545, 0x0c720efc, 0x751873ed, + 0x9dc38854, 0x4f9f6244, 0xa74499fd, 0xde2ee4ec, 0x36f51f55, + 0x2cfa1ece, 0xc421e577, 0xbd4b9866, 0x559063df, 0xc853eb8b, + 0x20881032, 0x59e26d23, 0xb139969a, 0xab369701, 0x43ed6cb8, + 0x3a8711a9, 0xd25cea10, 0x9e3ec588, 0x76e53e31, 0x0f8f4320, + 0xe754b899, 0xfd5bb902, 0x158042bb, 0x6cea3faa, 0x8431c413, + 0x19f24c47, 0xf129b7fe, 0x8843caef, 0x60983156, 0x7a9730cd, + 0x924ccb74, 0xeb26b665, 0x03fd4ddc, 0xd1a1a7cc, 0x397a5c75, + 0x40102164, 0xa8cbdadd, 0xb2c4db46, 0x5a1f20ff, 0x23755dee, + 0xcbaea657, 0x566d2e03, 0xbeb6d5ba, 0xc7dca8ab, 0x2f075312, + 0x35085289, 0xddd3a930, 0xa4b9d421, 0x4c622f98, 0x7d7bfbca, + 0x95a00073, 0xecca7d62, 0x041186db, 0x1e1e8740, 0xf6c57cf9, + 0x8faf01e8, 0x6774fa51, 0xfab77205, 0x126c89bc, 0x6b06f4ad, + 0x83dd0f14, 0x99d20e8f, 0x7109f536, 0x08638827, 0xe0b8739e, + 0x32e4998e, 0xda3f6237, 0xa3551f26, 0x4b8ee49f, 0x5181e504, + 0xb95a1ebd, 0xc03063ac, 0x28eb9815, 0xb5281041, 0x5df3ebf8, + 0x249996e9, 0xcc426d50, 0xd64d6ccb, 0x3e969772, 0x47fcea63, + 0xaf2711da, 0xe3453e42, 0x0b9ec5fb, 0x72f4b8ea, 0x9a2f4353, + 0x802042c8, 0x68fbb971, 0x1191c460, 0xf94a3fd9, 0x6489b78d, + 0x8c524c34, 0xf5383125, 0x1de3ca9c, 0x07eccb07, 0xef3730be, + 0x965d4daf, 0x7e86b616, 0xacda5c06, 0x4401a7bf, 0x3d6bdaae, + 0xd5b02117, 0xcfbf208c, 0x2764db35, 0x5e0ea624, 0xb6d55d9d, + 0x2b16d5c9, 0xc3cd2e70, 0xbaa75361, 0x527ca8d8, 0x4873a943, + 0xa0a852fa, 0xd9c22feb, 0x3119d452, 0xbbf0874e, 0x532b7cf7, + 0x2a4101e6, 0xc29afa5f, 0xd895fbc4, 0x304e007d, 0x49247d6c, + 0xa1ff86d5, 0x3c3c0e81, 0xd4e7f538, 0xad8d8829, 0x45567390, + 0x5f59720b, 0xb78289b2, 0xcee8f4a3, 0x26330f1a, 0xf46fe50a, + 0x1cb41eb3, 0x65de63a2, 0x8d05981b, 0x970a9980, 0x7fd16239, + 0x06bb1f28, 0xee60e491, 0x73a36cc5, 0x9b78977c, 0xe212ea6d, + 0x0ac911d4, 0x10c6104f, 0xf81debf6, 0x817796e7, 0x69ac6d5e, + 0x25ce42c6, 0xcd15b97f, 0xb47fc46e, 0x5ca43fd7, 0x46ab3e4c, + 0xae70c5f5, 0xd71ab8e4, 0x3fc1435d, 0xa202cb09, 0x4ad930b0, + 0x33b34da1, 0xdb68b618, 0xc167b783, 0x29bc4c3a, 0x50d6312b, + 0xb80dca92, 0x6a512082, 0x828adb3b, 0xfbe0a62a, 0x133b5d93, + 0x09345c08, 0xe1efa7b1, 0x9885daa0, 0x705e2119, 0xed9da94d, + 0x054652f4, 0x7c2c2fe5, 0x94f7d45c, 0x8ef8d5c7, 0x66232e7e, + 0x1f49536f, 0xf792a8d6, 0xc68b7c84, 0x2e50873d, 0x573afa2c, + 0xbfe10195, 0xa5ee000e, 0x4d35fbb7, 0x345f86a6, 0xdc847d1f, + 0x4147f54b, 0xa99c0ef2, 0xd0f673e3, 0x382d885a, 0x222289c1, + 0xcaf97278, 0xb3930f69, 0x5b48f4d0, 0x89141ec0, 0x61cfe579, + 0x18a59868, 0xf07e63d1, 0xea71624a, 0x02aa99f3, 0x7bc0e4e2, + 0x931b1f5b, 0x0ed8970f, 0xe6036cb6, 0x9f6911a7, 0x77b2ea1e, + 0x6dbdeb85, 0x8566103c, 0xfc0c6d2d, 0x14d79694, 0x58b5b90c, + 0xb06e42b5, 0xc9043fa4, 0x21dfc41d, 0x3bd0c586, 0xd30b3e3f, + 0xaa61432e, 0x42bab897, 0xdf7930c3, 0x37a2cb7a, 0x4ec8b66b, + 0xa6134dd2, 0xbc1c4c49, 0x54c7b7f0, 0x2dadcae1, 0xc5763158, + 0x172adb48, 0xfff120f1, 0x869b5de0, 0x6e40a659, 0x744fa7c2, + 0x9c945c7b, 0xe5fe216a, 0x0d25dad3, 0x90e65287, 0x783da93e, + 0x0157d42f, 0xe98c2f96, 0xf3832e0d, 0x1b58d5b4, 0x6232a8a5, + 0x8ae9531c}, + {0x00000000, 0x919168ae, 0x6325a087, 0xf2b4c829, 0x874c31d4, + 0x16dd597a, 0xe4699153, 0x75f8f9fd, 0x4f9f1373, 0xde0e7bdd, + 0x2cbab3f4, 0xbd2bdb5a, 0xc8d322a7, 0x59424a09, 0xabf68220, + 0x3a67ea8e, 0x9e3e27e6, 0x0faf4f48, 0xfd1b8761, 0x6c8aefcf, + 0x19721632, 0x88e37e9c, 0x7a57b6b5, 0xebc6de1b, 0xd1a13495, + 0x40305c3b, 0xb2849412, 0x2315fcbc, 0x56ed0541, 0xc77c6def, + 0x35c8a5c6, 0xa459cd68, 0x7d7b3f17, 0xecea57b9, 0x1e5e9f90, + 0x8fcff73e, 0xfa370ec3, 0x6ba6666d, 0x9912ae44, 0x0883c6ea, + 0x32e42c64, 0xa37544ca, 0x51c18ce3, 0xc050e44d, 0xb5a81db0, + 0x2439751e, 0xd68dbd37, 0x471cd599, 0xe34518f1, 0x72d4705f, + 0x8060b876, 0x11f1d0d8, 0x64092925, 0xf598418b, 0x072c89a2, + 0x96bde10c, 0xacda0b82, 0x3d4b632c, 0xcfffab05, 0x5e6ec3ab, + 0x2b963a56, 0xba0752f8, 0x48b39ad1, 0xd922f27f, 0xfaf67e2e, + 0x6b671680, 0x99d3dea9, 0x0842b607, 0x7dba4ffa, 0xec2b2754, + 0x1e9fef7d, 0x8f0e87d3, 0xb5696d5d, 0x24f805f3, 0xd64ccdda, + 0x47dda574, 0x32255c89, 0xa3b43427, 0x5100fc0e, 0xc09194a0, + 0x64c859c8, 0xf5593166, 0x07edf94f, 0x967c91e1, 0xe384681c, + 0x721500b2, 0x80a1c89b, 0x1130a035, 0x2b574abb, 0xbac62215, + 0x4872ea3c, 0xd9e38292, 0xac1b7b6f, 0x3d8a13c1, 0xcf3edbe8, + 0x5eafb346, 0x878d4139, 0x161c2997, 0xe4a8e1be, 0x75398910, + 0x00c170ed, 0x91501843, 0x63e4d06a, 0xf275b8c4, 0xc812524a, + 0x59833ae4, 0xab37f2cd, 0x3aa69a63, 0x4f5e639e, 0xdecf0b30, + 0x2c7bc319, 0xbdeaabb7, 0x19b366df, 0x88220e71, 0x7a96c658, + 0xeb07aef6, 0x9eff570b, 0x0f6e3fa5, 0xfddaf78c, 0x6c4b9f22, + 0x562c75ac, 0xc7bd1d02, 0x3509d52b, 0xa498bd85, 0xd1604478, + 0x40f12cd6, 0xb245e4ff, 0x23d48c51, 0xf4edfd5c, 0x657c95f2, + 0x97c85ddb, 0x06593575, 0x73a1cc88, 0xe230a426, 0x10846c0f, + 0x811504a1, 0xbb72ee2f, 0x2ae38681, 0xd8574ea8, 0x49c62606, + 0x3c3edffb, 0xadafb755, 0x5f1b7f7c, 0xce8a17d2, 0x6ad3daba, + 0xfb42b214, 0x09f67a3d, 0x98671293, 0xed9feb6e, 0x7c0e83c0, + 0x8eba4be9, 0x1f2b2347, 0x254cc9c9, 0xb4dda167, 0x4669694e, + 0xd7f801e0, 0xa200f81d, 0x339190b3, 0xc125589a, 0x50b43034, + 0x8996c24b, 0x1807aae5, 0xeab362cc, 0x7b220a62, 0x0edaf39f, + 0x9f4b9b31, 0x6dff5318, 0xfc6e3bb6, 0xc609d138, 0x5798b996, + 0xa52c71bf, 0x34bd1911, 0x4145e0ec, 0xd0d48842, 0x2260406b, + 0xb3f128c5, 0x17a8e5ad, 0x86398d03, 0x748d452a, 0xe51c2d84, + 0x90e4d479, 0x0175bcd7, 0xf3c174fe, 0x62501c50, 0x5837f6de, + 0xc9a69e70, 0x3b125659, 0xaa833ef7, 0xdf7bc70a, 0x4eeaafa4, + 0xbc5e678d, 0x2dcf0f23, 0x0e1b8372, 0x9f8aebdc, 0x6d3e23f5, + 0xfcaf4b5b, 0x8957b2a6, 0x18c6da08, 0xea721221, 0x7be37a8f, + 0x41849001, 0xd015f8af, 0x22a13086, 0xb3305828, 0xc6c8a1d5, + 0x5759c97b, 0xa5ed0152, 0x347c69fc, 0x9025a494, 0x01b4cc3a, + 0xf3000413, 0x62916cbd, 0x17699540, 0x86f8fdee, 0x744c35c7, + 0xe5dd5d69, 0xdfbab7e7, 0x4e2bdf49, 0xbc9f1760, 0x2d0e7fce, + 0x58f68633, 0xc967ee9d, 0x3bd326b4, 0xaa424e1a, 0x7360bc65, + 0xe2f1d4cb, 0x10451ce2, 0x81d4744c, 0xf42c8db1, 0x65bde51f, + 0x97092d36, 0x06984598, 0x3cffaf16, 0xad6ec7b8, 0x5fda0f91, + 0xce4b673f, 0xbbb39ec2, 0x2a22f66c, 0xd8963e45, 0x490756eb, + 0xed5e9b83, 0x7ccff32d, 0x8e7b3b04, 0x1fea53aa, 0x6a12aa57, + 0xfb83c2f9, 0x09370ad0, 0x98a6627e, 0xa2c188f0, 0x3350e05e, + 0xc1e42877, 0x507540d9, 0x258db924, 0xb41cd18a, 0x46a819a3, + 0xd739710d}}; + +#endif + +#endif + +#if N == 5 + +#if W == 8 + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0xaf449247, 0x85f822cf, 0x2abcb088, 0xd08143df, + 0x7fc5d198, 0x55796110, 0xfa3df357, 0x7a7381ff, 0xd53713b8, + 0xff8ba330, 0x50cf3177, 0xaaf2c220, 0x05b65067, 0x2f0ae0ef, + 0x804e72a8, 0xf4e703fe, 0x5ba391b9, 0x711f2131, 0xde5bb376, + 0x24664021, 0x8b22d266, 0xa19e62ee, 0x0edaf0a9, 0x8e948201, + 0x21d01046, 0x0b6ca0ce, 0xa4283289, 0x5e15c1de, 0xf1515399, + 0xdbede311, 0x74a97156, 0x32bf01bd, 0x9dfb93fa, 0xb7472372, + 0x1803b135, 0xe23e4262, 0x4d7ad025, 0x67c660ad, 0xc882f2ea, + 0x48cc8042, 0xe7881205, 0xcd34a28d, 0x627030ca, 0x984dc39d, + 0x370951da, 0x1db5e152, 0xb2f17315, 0xc6580243, 0x691c9004, + 0x43a0208c, 0xece4b2cb, 0x16d9419c, 0xb99dd3db, 0x93216353, + 0x3c65f114, 0xbc2b83bc, 0x136f11fb, 0x39d3a173, 0x96973334, + 0x6caac063, 0xc3ee5224, 0xe952e2ac, 0x461670eb, 0x657e037a, + 0xca3a913d, 0xe08621b5, 0x4fc2b3f2, 0xb5ff40a5, 0x1abbd2e2, + 0x3007626a, 0x9f43f02d, 0x1f0d8285, 0xb04910c2, 0x9af5a04a, + 0x35b1320d, 0xcf8cc15a, 0x60c8531d, 0x4a74e395, 0xe53071d2, + 0x91990084, 0x3edd92c3, 0x1461224b, 0xbb25b00c, 0x4118435b, + 0xee5cd11c, 0xc4e06194, 0x6ba4f3d3, 0xebea817b, 0x44ae133c, + 0x6e12a3b4, 0xc15631f3, 0x3b6bc2a4, 0x942f50e3, 0xbe93e06b, + 0x11d7722c, 0x57c102c7, 0xf8859080, 0xd2392008, 0x7d7db24f, + 0x87404118, 0x2804d35f, 0x02b863d7, 0xadfcf190, 0x2db28338, + 0x82f6117f, 0xa84aa1f7, 0x070e33b0, 0xfd33c0e7, 0x527752a0, + 0x78cbe228, 0xd78f706f, 0xa3260139, 0x0c62937e, 0x26de23f6, + 0x899ab1b1, 0x73a742e6, 0xdce3d0a1, 0xf65f6029, 0x591bf26e, + 0xd95580c6, 0x76111281, 0x5cada209, 0xf3e9304e, 0x09d4c319, + 0xa690515e, 0x8c2ce1d6, 0x23687391, 0xcafc06f4, 0x65b894b3, + 0x4f04243b, 0xe040b67c, 0x1a7d452b, 0xb539d76c, 0x9f8567e4, + 0x30c1f5a3, 0xb08f870b, 0x1fcb154c, 0x3577a5c4, 0x9a333783, + 0x600ec4d4, 0xcf4a5693, 0xe5f6e61b, 0x4ab2745c, 0x3e1b050a, + 0x915f974d, 0xbbe327c5, 0x14a7b582, 0xee9a46d5, 0x41ded492, + 0x6b62641a, 0xc426f65d, 0x446884f5, 0xeb2c16b2, 0xc190a63a, + 0x6ed4347d, 0x94e9c72a, 0x3bad556d, 0x1111e5e5, 0xbe5577a2, + 0xf8430749, 0x5707950e, 0x7dbb2586, 0xd2ffb7c1, 0x28c24496, + 0x8786d6d1, 0xad3a6659, 0x027ef41e, 0x823086b6, 0x2d7414f1, + 0x07c8a479, 0xa88c363e, 0x52b1c569, 0xfdf5572e, 0xd749e7a6, + 0x780d75e1, 0x0ca404b7, 0xa3e096f0, 0x895c2678, 0x2618b43f, + 0xdc254768, 0x7361d52f, 0x59dd65a7, 0xf699f7e0, 0x76d78548, + 0xd993170f, 0xf32fa787, 0x5c6b35c0, 0xa656c697, 0x091254d0, + 0x23aee458, 0x8cea761f, 0xaf82058e, 0x00c697c9, 0x2a7a2741, + 0x853eb506, 0x7f034651, 0xd047d416, 0xfafb649e, 0x55bff6d9, + 0xd5f18471, 0x7ab51636, 0x5009a6be, 0xff4d34f9, 0x0570c7ae, + 0xaa3455e9, 0x8088e561, 0x2fcc7726, 0x5b650670, 0xf4219437, + 0xde9d24bf, 0x71d9b6f8, 0x8be445af, 0x24a0d7e8, 0x0e1c6760, + 0xa158f527, 0x2116878f, 0x8e5215c8, 0xa4eea540, 0x0baa3707, + 0xf197c450, 0x5ed35617, 0x746fe69f, 0xdb2b74d8, 0x9d3d0433, + 0x32799674, 0x18c526fc, 0xb781b4bb, 0x4dbc47ec, 0xe2f8d5ab, + 0xc8446523, 0x6700f764, 0xe74e85cc, 0x480a178b, 0x62b6a703, + 0xcdf23544, 0x37cfc613, 0x988b5454, 0xb237e4dc, 0x1d73769b, + 0x69da07cd, 0xc69e958a, 0xec222502, 0x4366b745, 0xb95b4412, + 0x161fd655, 0x3ca366dd, 0x93e7f49a, 0x13a98632, 0xbced1475, + 0x9651a4fd, 0x391536ba, 0xc328c5ed, 0x6c6c57aa, 0x46d0e722, + 0xe9947565}, + {0x00000000, 0x4e890ba9, 0x9d121752, 0xd39b1cfb, 0xe15528e5, + 0xafdc234c, 0x7c473fb7, 0x32ce341e, 0x19db578b, 0x57525c22, + 0x84c940d9, 0xca404b70, 0xf88e7f6e, 0xb60774c7, 0x659c683c, + 0x2b156395, 0x33b6af16, 0x7d3fa4bf, 0xaea4b844, 0xe02db3ed, + 0xd2e387f3, 0x9c6a8c5a, 0x4ff190a1, 0x01789b08, 0x2a6df89d, + 0x64e4f334, 0xb77fefcf, 0xf9f6e466, 0xcb38d078, 0x85b1dbd1, + 0x562ac72a, 0x18a3cc83, 0x676d5e2c, 0x29e45585, 0xfa7f497e, + 0xb4f642d7, 0x863876c9, 0xc8b17d60, 0x1b2a619b, 0x55a36a32, + 0x7eb609a7, 0x303f020e, 0xe3a41ef5, 0xad2d155c, 0x9fe32142, + 0xd16a2aeb, 0x02f13610, 0x4c783db9, 0x54dbf13a, 0x1a52fa93, + 0xc9c9e668, 0x8740edc1, 0xb58ed9df, 0xfb07d276, 0x289cce8d, + 0x6615c524, 0x4d00a6b1, 0x0389ad18, 0xd012b1e3, 0x9e9bba4a, + 0xac558e54, 0xe2dc85fd, 0x31479906, 0x7fce92af, 0xcedabc58, + 0x8053b7f1, 0x53c8ab0a, 0x1d41a0a3, 0x2f8f94bd, 0x61069f14, + 0xb29d83ef, 0xfc148846, 0xd701ebd3, 0x9988e07a, 0x4a13fc81, + 0x049af728, 0x3654c336, 0x78ddc89f, 0xab46d464, 0xe5cfdfcd, + 0xfd6c134e, 0xb3e518e7, 0x607e041c, 0x2ef70fb5, 0x1c393bab, + 0x52b03002, 0x812b2cf9, 0xcfa22750, 0xe4b744c5, 0xaa3e4f6c, + 0x79a55397, 0x372c583e, 0x05e26c20, 0x4b6b6789, 0x98f07b72, + 0xd67970db, 0xa9b7e274, 0xe73ee9dd, 0x34a5f526, 0x7a2cfe8f, + 0x48e2ca91, 0x066bc138, 0xd5f0ddc3, 0x9b79d66a, 0xb06cb5ff, + 0xfee5be56, 0x2d7ea2ad, 0x63f7a904, 0x51399d1a, 0x1fb096b3, + 0xcc2b8a48, 0x82a281e1, 0x9a014d62, 0xd48846cb, 0x07135a30, + 0x499a5199, 0x7b546587, 0x35dd6e2e, 0xe64672d5, 0xa8cf797c, + 0x83da1ae9, 0xcd531140, 0x1ec80dbb, 0x50410612, 0x628f320c, + 0x2c0639a5, 0xff9d255e, 0xb1142ef7, 0x46c47ef1, 0x084d7558, + 0xdbd669a3, 0x955f620a, 0xa7915614, 0xe9185dbd, 0x3a834146, + 0x740a4aef, 0x5f1f297a, 0x119622d3, 0xc20d3e28, 0x8c843581, + 0xbe4a019f, 0xf0c30a36, 0x235816cd, 0x6dd11d64, 0x7572d1e7, + 0x3bfbda4e, 0xe860c6b5, 0xa6e9cd1c, 0x9427f902, 0xdaaef2ab, + 0x0935ee50, 0x47bce5f9, 0x6ca9866c, 0x22208dc5, 0xf1bb913e, + 0xbf329a97, 0x8dfcae89, 0xc375a520, 0x10eeb9db, 0x5e67b272, + 0x21a920dd, 0x6f202b74, 0xbcbb378f, 0xf2323c26, 0xc0fc0838, + 0x8e750391, 0x5dee1f6a, 0x136714c3, 0x38727756, 0x76fb7cff, + 0xa5606004, 0xebe96bad, 0xd9275fb3, 0x97ae541a, 0x443548e1, + 0x0abc4348, 0x121f8fcb, 0x5c968462, 0x8f0d9899, 0xc1849330, + 0xf34aa72e, 0xbdc3ac87, 0x6e58b07c, 0x20d1bbd5, 0x0bc4d840, + 0x454dd3e9, 0x96d6cf12, 0xd85fc4bb, 0xea91f0a5, 0xa418fb0c, + 0x7783e7f7, 0x390aec5e, 0x881ec2a9, 0xc697c900, 0x150cd5fb, + 0x5b85de52, 0x694bea4c, 0x27c2e1e5, 0xf459fd1e, 0xbad0f6b7, + 0x91c59522, 0xdf4c9e8b, 0x0cd78270, 0x425e89d9, 0x7090bdc7, + 0x3e19b66e, 0xed82aa95, 0xa30ba13c, 0xbba86dbf, 0xf5216616, + 0x26ba7aed, 0x68337144, 0x5afd455a, 0x14744ef3, 0xc7ef5208, + 0x896659a1, 0xa2733a34, 0xecfa319d, 0x3f612d66, 0x71e826cf, + 0x432612d1, 0x0daf1978, 0xde340583, 0x90bd0e2a, 0xef739c85, + 0xa1fa972c, 0x72618bd7, 0x3ce8807e, 0x0e26b460, 0x40afbfc9, + 0x9334a332, 0xddbda89b, 0xf6a8cb0e, 0xb821c0a7, 0x6bbadc5c, + 0x2533d7f5, 0x17fde3eb, 0x5974e842, 0x8aeff4b9, 0xc466ff10, + 0xdcc53393, 0x924c383a, 0x41d724c1, 0x0f5e2f68, 0x3d901b76, + 0x731910df, 0xa0820c24, 0xee0b078d, 0xc51e6418, 0x8b976fb1, + 0x580c734a, 0x168578e3, 0x244b4cfd, 0x6ac24754, 0xb9595baf, + 0xf7d05006}, + {0x00000000, 0x8d88fde2, 0xc060fd85, 0x4de80067, 0x5bb0fd4b, + 0xd63800a9, 0x9bd000ce, 0x1658fd2c, 0xb761fa96, 0x3ae90774, + 0x77010713, 0xfa89faf1, 0xecd107dd, 0x6159fa3f, 0x2cb1fa58, + 0xa13907ba, 0xb5b2f36d, 0x383a0e8f, 0x75d20ee8, 0xf85af30a, + 0xee020e26, 0x638af3c4, 0x2e62f3a3, 0xa3ea0e41, 0x02d309fb, + 0x8f5bf419, 0xc2b3f47e, 0x4f3b099c, 0x5963f4b0, 0xd4eb0952, + 0x99030935, 0x148bf4d7, 0xb014e09b, 0x3d9c1d79, 0x70741d1e, + 0xfdfce0fc, 0xeba41dd0, 0x662ce032, 0x2bc4e055, 0xa64c1db7, + 0x07751a0d, 0x8afde7ef, 0xc715e788, 0x4a9d1a6a, 0x5cc5e746, + 0xd14d1aa4, 0x9ca51ac3, 0x112de721, 0x05a613f6, 0x882eee14, + 0xc5c6ee73, 0x484e1391, 0x5e16eebd, 0xd39e135f, 0x9e761338, + 0x13feeeda, 0xb2c7e960, 0x3f4f1482, 0x72a714e5, 0xff2fe907, + 0xe977142b, 0x64ffe9c9, 0x2917e9ae, 0xa49f144c, 0xbb58c777, + 0x36d03a95, 0x7b383af2, 0xf6b0c710, 0xe0e83a3c, 0x6d60c7de, + 0x2088c7b9, 0xad003a5b, 0x0c393de1, 0x81b1c003, 0xcc59c064, + 0x41d13d86, 0x5789c0aa, 0xda013d48, 0x97e93d2f, 0x1a61c0cd, + 0x0eea341a, 0x8362c9f8, 0xce8ac99f, 0x4302347d, 0x555ac951, + 0xd8d234b3, 0x953a34d4, 0x18b2c936, 0xb98bce8c, 0x3403336e, + 0x79eb3309, 0xf463ceeb, 0xe23b33c7, 0x6fb3ce25, 0x225bce42, + 0xafd333a0, 0x0b4c27ec, 0x86c4da0e, 0xcb2cda69, 0x46a4278b, + 0x50fcdaa7, 0xdd742745, 0x909c2722, 0x1d14dac0, 0xbc2ddd7a, + 0x31a52098, 0x7c4d20ff, 0xf1c5dd1d, 0xe79d2031, 0x6a15ddd3, + 0x27fdddb4, 0xaa752056, 0xbefed481, 0x33762963, 0x7e9e2904, + 0xf316d4e6, 0xe54e29ca, 0x68c6d428, 0x252ed44f, 0xa8a629ad, + 0x099f2e17, 0x8417d3f5, 0xc9ffd392, 0x44772e70, 0x522fd35c, + 0xdfa72ebe, 0x924f2ed9, 0x1fc7d33b, 0xadc088af, 0x2048754d, + 0x6da0752a, 0xe02888c8, 0xf67075e4, 0x7bf88806, 0x36108861, + 0xbb987583, 0x1aa17239, 0x97298fdb, 0xdac18fbc, 0x5749725e, + 0x41118f72, 0xcc997290, 0x817172f7, 0x0cf98f15, 0x18727bc2, + 0x95fa8620, 0xd8128647, 0x559a7ba5, 0x43c28689, 0xce4a7b6b, + 0x83a27b0c, 0x0e2a86ee, 0xaf138154, 0x229b7cb6, 0x6f737cd1, + 0xe2fb8133, 0xf4a37c1f, 0x792b81fd, 0x34c3819a, 0xb94b7c78, + 0x1dd46834, 0x905c95d6, 0xddb495b1, 0x503c6853, 0x4664957f, + 0xcbec689d, 0x860468fa, 0x0b8c9518, 0xaab592a2, 0x273d6f40, + 0x6ad56f27, 0xe75d92c5, 0xf1056fe9, 0x7c8d920b, 0x3165926c, + 0xbced6f8e, 0xa8669b59, 0x25ee66bb, 0x680666dc, 0xe58e9b3e, + 0xf3d66612, 0x7e5e9bf0, 0x33b69b97, 0xbe3e6675, 0x1f0761cf, + 0x928f9c2d, 0xdf679c4a, 0x52ef61a8, 0x44b79c84, 0xc93f6166, + 0x84d76101, 0x095f9ce3, 0x16984fd8, 0x9b10b23a, 0xd6f8b25d, + 0x5b704fbf, 0x4d28b293, 0xc0a04f71, 0x8d484f16, 0x00c0b2f4, + 0xa1f9b54e, 0x2c7148ac, 0x619948cb, 0xec11b529, 0xfa494805, + 0x77c1b5e7, 0x3a29b580, 0xb7a14862, 0xa32abcb5, 0x2ea24157, + 0x634a4130, 0xeec2bcd2, 0xf89a41fe, 0x7512bc1c, 0x38fabc7b, + 0xb5724199, 0x144b4623, 0x99c3bbc1, 0xd42bbba6, 0x59a34644, + 0x4ffbbb68, 0xc273468a, 0x8f9b46ed, 0x0213bb0f, 0xa68caf43, + 0x2b0452a1, 0x66ec52c6, 0xeb64af24, 0xfd3c5208, 0x70b4afea, + 0x3d5caf8d, 0xb0d4526f, 0x11ed55d5, 0x9c65a837, 0xd18da850, + 0x5c0555b2, 0x4a5da89e, 0xc7d5557c, 0x8a3d551b, 0x07b5a8f9, + 0x133e5c2e, 0x9eb6a1cc, 0xd35ea1ab, 0x5ed65c49, 0x488ea165, + 0xc5065c87, 0x88ee5ce0, 0x0566a102, 0xa45fa6b8, 0x29d75b5a, + 0x643f5b3d, 0xe9b7a6df, 0xffef5bf3, 0x7267a611, 0x3f8fa676, + 0xb2075b94}, + {0x00000000, 0x80f0171f, 0xda91287f, 0x5a613f60, 0x6e5356bf, + 0xeea341a0, 0xb4c27ec0, 0x343269df, 0xdca6ad7e, 0x5c56ba61, + 0x06378501, 0x86c7921e, 0xb2f5fbc1, 0x3205ecde, 0x6864d3be, + 0xe894c4a1, 0x623c5cbd, 0xe2cc4ba2, 0xb8ad74c2, 0x385d63dd, + 0x0c6f0a02, 0x8c9f1d1d, 0xd6fe227d, 0x560e3562, 0xbe9af1c3, + 0x3e6ae6dc, 0x640bd9bc, 0xe4fbcea3, 0xd0c9a77c, 0x5039b063, + 0x0a588f03, 0x8aa8981c, 0xc478b97a, 0x4488ae65, 0x1ee99105, + 0x9e19861a, 0xaa2befc5, 0x2adbf8da, 0x70bac7ba, 0xf04ad0a5, + 0x18de1404, 0x982e031b, 0xc24f3c7b, 0x42bf2b64, 0x768d42bb, + 0xf67d55a4, 0xac1c6ac4, 0x2cec7ddb, 0xa644e5c7, 0x26b4f2d8, + 0x7cd5cdb8, 0xfc25daa7, 0xc817b378, 0x48e7a467, 0x12869b07, + 0x92768c18, 0x7ae248b9, 0xfa125fa6, 0xa07360c6, 0x208377d9, + 0x14b11e06, 0x94410919, 0xce203679, 0x4ed02166, 0x538074b5, + 0xd37063aa, 0x89115cca, 0x09e14bd5, 0x3dd3220a, 0xbd233515, + 0xe7420a75, 0x67b21d6a, 0x8f26d9cb, 0x0fd6ced4, 0x55b7f1b4, + 0xd547e6ab, 0xe1758f74, 0x6185986b, 0x3be4a70b, 0xbb14b014, + 0x31bc2808, 0xb14c3f17, 0xeb2d0077, 0x6bdd1768, 0x5fef7eb7, + 0xdf1f69a8, 0x857e56c8, 0x058e41d7, 0xed1a8576, 0x6dea9269, + 0x378bad09, 0xb77bba16, 0x8349d3c9, 0x03b9c4d6, 0x59d8fbb6, + 0xd928eca9, 0x97f8cdcf, 0x1708dad0, 0x4d69e5b0, 0xcd99f2af, + 0xf9ab9b70, 0x795b8c6f, 0x233ab30f, 0xa3caa410, 0x4b5e60b1, + 0xcbae77ae, 0x91cf48ce, 0x113f5fd1, 0x250d360e, 0xa5fd2111, + 0xff9c1e71, 0x7f6c096e, 0xf5c49172, 0x7534866d, 0x2f55b90d, + 0xafa5ae12, 0x9b97c7cd, 0x1b67d0d2, 0x4106efb2, 0xc1f6f8ad, + 0x29623c0c, 0xa9922b13, 0xf3f31473, 0x7303036c, 0x47316ab3, + 0xc7c17dac, 0x9da042cc, 0x1d5055d3, 0xa700e96a, 0x27f0fe75, + 0x7d91c115, 0xfd61d60a, 0xc953bfd5, 0x49a3a8ca, 0x13c297aa, + 0x933280b5, 0x7ba64414, 0xfb56530b, 0xa1376c6b, 0x21c77b74, + 0x15f512ab, 0x950505b4, 0xcf643ad4, 0x4f942dcb, 0xc53cb5d7, + 0x45cca2c8, 0x1fad9da8, 0x9f5d8ab7, 0xab6fe368, 0x2b9ff477, + 0x71fecb17, 0xf10edc08, 0x199a18a9, 0x996a0fb6, 0xc30b30d6, + 0x43fb27c9, 0x77c94e16, 0xf7395909, 0xad586669, 0x2da87176, + 0x63785010, 0xe388470f, 0xb9e9786f, 0x39196f70, 0x0d2b06af, + 0x8ddb11b0, 0xd7ba2ed0, 0x574a39cf, 0xbfdefd6e, 0x3f2eea71, + 0x654fd511, 0xe5bfc20e, 0xd18dabd1, 0x517dbcce, 0x0b1c83ae, + 0x8bec94b1, 0x01440cad, 0x81b41bb2, 0xdbd524d2, 0x5b2533cd, + 0x6f175a12, 0xefe74d0d, 0xb586726d, 0x35766572, 0xdde2a1d3, + 0x5d12b6cc, 0x077389ac, 0x87839eb3, 0xb3b1f76c, 0x3341e073, + 0x6920df13, 0xe9d0c80c, 0xf4809ddf, 0x74708ac0, 0x2e11b5a0, + 0xaee1a2bf, 0x9ad3cb60, 0x1a23dc7f, 0x4042e31f, 0xc0b2f400, + 0x282630a1, 0xa8d627be, 0xf2b718de, 0x72470fc1, 0x4675661e, + 0xc6857101, 0x9ce44e61, 0x1c14597e, 0x96bcc162, 0x164cd67d, + 0x4c2de91d, 0xccddfe02, 0xf8ef97dd, 0x781f80c2, 0x227ebfa2, + 0xa28ea8bd, 0x4a1a6c1c, 0xcaea7b03, 0x908b4463, 0x107b537c, + 0x24493aa3, 0xa4b92dbc, 0xfed812dc, 0x7e2805c3, 0x30f824a5, + 0xb00833ba, 0xea690cda, 0x6a991bc5, 0x5eab721a, 0xde5b6505, + 0x843a5a65, 0x04ca4d7a, 0xec5e89db, 0x6cae9ec4, 0x36cfa1a4, + 0xb63fb6bb, 0x820ddf64, 0x02fdc87b, 0x589cf71b, 0xd86ce004, + 0x52c47818, 0xd2346f07, 0x88555067, 0x08a54778, 0x3c972ea7, + 0xbc6739b8, 0xe60606d8, 0x66f611c7, 0x8e62d566, 0x0e92c279, + 0x54f3fd19, 0xd403ea06, 0xe03183d9, 0x60c194c6, 0x3aa0aba6, + 0xba50bcb9}, + {0x00000000, 0x9570d495, 0xf190af6b, 0x64e07bfe, 0x38505897, + 0xad208c02, 0xc9c0f7fc, 0x5cb02369, 0x70a0b12e, 0xe5d065bb, + 0x81301e45, 0x1440cad0, 0x48f0e9b9, 0xdd803d2c, 0xb96046d2, + 0x2c109247, 0xe141625c, 0x7431b6c9, 0x10d1cd37, 0x85a119a2, + 0xd9113acb, 0x4c61ee5e, 0x288195a0, 0xbdf14135, 0x91e1d372, + 0x049107e7, 0x60717c19, 0xf501a88c, 0xa9b18be5, 0x3cc15f70, + 0x5821248e, 0xcd51f01b, 0x19f3c2f9, 0x8c83166c, 0xe8636d92, + 0x7d13b907, 0x21a39a6e, 0xb4d34efb, 0xd0333505, 0x4543e190, + 0x695373d7, 0xfc23a742, 0x98c3dcbc, 0x0db30829, 0x51032b40, + 0xc473ffd5, 0xa093842b, 0x35e350be, 0xf8b2a0a5, 0x6dc27430, + 0x09220fce, 0x9c52db5b, 0xc0e2f832, 0x55922ca7, 0x31725759, + 0xa40283cc, 0x8812118b, 0x1d62c51e, 0x7982bee0, 0xecf26a75, + 0xb042491c, 0x25329d89, 0x41d2e677, 0xd4a232e2, 0x33e785f2, + 0xa6975167, 0xc2772a99, 0x5707fe0c, 0x0bb7dd65, 0x9ec709f0, + 0xfa27720e, 0x6f57a69b, 0x434734dc, 0xd637e049, 0xb2d79bb7, + 0x27a74f22, 0x7b176c4b, 0xee67b8de, 0x8a87c320, 0x1ff717b5, + 0xd2a6e7ae, 0x47d6333b, 0x233648c5, 0xb6469c50, 0xeaf6bf39, + 0x7f866bac, 0x1b661052, 0x8e16c4c7, 0xa2065680, 0x37768215, + 0x5396f9eb, 0xc6e62d7e, 0x9a560e17, 0x0f26da82, 0x6bc6a17c, + 0xfeb675e9, 0x2a14470b, 0xbf64939e, 0xdb84e860, 0x4ef43cf5, + 0x12441f9c, 0x8734cb09, 0xe3d4b0f7, 0x76a46462, 0x5ab4f625, + 0xcfc422b0, 0xab24594e, 0x3e548ddb, 0x62e4aeb2, 0xf7947a27, + 0x937401d9, 0x0604d54c, 0xcb552557, 0x5e25f1c2, 0x3ac58a3c, + 0xafb55ea9, 0xf3057dc0, 0x6675a955, 0x0295d2ab, 0x97e5063e, + 0xbbf59479, 0x2e8540ec, 0x4a653b12, 0xdf15ef87, 0x83a5ccee, + 0x16d5187b, 0x72356385, 0xe745b710, 0x67cf0be4, 0xf2bfdf71, + 0x965fa48f, 0x032f701a, 0x5f9f5373, 0xcaef87e6, 0xae0ffc18, + 0x3b7f288d, 0x176fbaca, 0x821f6e5f, 0xe6ff15a1, 0x738fc134, + 0x2f3fe25d, 0xba4f36c8, 0xdeaf4d36, 0x4bdf99a3, 0x868e69b8, + 0x13febd2d, 0x771ec6d3, 0xe26e1246, 0xbede312f, 0x2baee5ba, + 0x4f4e9e44, 0xda3e4ad1, 0xf62ed896, 0x635e0c03, 0x07be77fd, + 0x92cea368, 0xce7e8001, 0x5b0e5494, 0x3fee2f6a, 0xaa9efbff, + 0x7e3cc91d, 0xeb4c1d88, 0x8fac6676, 0x1adcb2e3, 0x466c918a, + 0xd31c451f, 0xb7fc3ee1, 0x228cea74, 0x0e9c7833, 0x9becaca6, + 0xff0cd758, 0x6a7c03cd, 0x36cc20a4, 0xa3bcf431, 0xc75c8fcf, + 0x522c5b5a, 0x9f7dab41, 0x0a0d7fd4, 0x6eed042a, 0xfb9dd0bf, + 0xa72df3d6, 0x325d2743, 0x56bd5cbd, 0xc3cd8828, 0xefdd1a6f, + 0x7aadcefa, 0x1e4db504, 0x8b3d6191, 0xd78d42f8, 0x42fd966d, + 0x261ded93, 0xb36d3906, 0x54288e16, 0xc1585a83, 0xa5b8217d, + 0x30c8f5e8, 0x6c78d681, 0xf9080214, 0x9de879ea, 0x0898ad7f, + 0x24883f38, 0xb1f8ebad, 0xd5189053, 0x406844c6, 0x1cd867af, + 0x89a8b33a, 0xed48c8c4, 0x78381c51, 0xb569ec4a, 0x201938df, + 0x44f94321, 0xd18997b4, 0x8d39b4dd, 0x18496048, 0x7ca91bb6, + 0xe9d9cf23, 0xc5c95d64, 0x50b989f1, 0x3459f20f, 0xa129269a, + 0xfd9905f3, 0x68e9d166, 0x0c09aa98, 0x99797e0d, 0x4ddb4cef, + 0xd8ab987a, 0xbc4be384, 0x293b3711, 0x758b1478, 0xe0fbc0ed, + 0x841bbb13, 0x116b6f86, 0x3d7bfdc1, 0xa80b2954, 0xcceb52aa, + 0x599b863f, 0x052ba556, 0x905b71c3, 0xf4bb0a3d, 0x61cbdea8, + 0xac9a2eb3, 0x39eafa26, 0x5d0a81d8, 0xc87a554d, 0x94ca7624, + 0x01baa2b1, 0x655ad94f, 0xf02a0dda, 0xdc3a9f9d, 0x494a4b08, + 0x2daa30f6, 0xb8dae463, 0xe46ac70a, 0x711a139f, 0x15fa6861, + 0x808abcf4}, + {0x00000000, 0xcf9e17c8, 0x444d29d1, 0x8bd33e19, 0x889a53a2, + 0x4704446a, 0xccd77a73, 0x03496dbb, 0xca45a105, 0x05dbb6cd, + 0x8e0888d4, 0x41969f1c, 0x42dff2a7, 0x8d41e56f, 0x0692db76, + 0xc90cccbe, 0x4ffa444b, 0x80645383, 0x0bb76d9a, 0xc4297a52, + 0xc76017e9, 0x08fe0021, 0x832d3e38, 0x4cb329f0, 0x85bfe54e, + 0x4a21f286, 0xc1f2cc9f, 0x0e6cdb57, 0x0d25b6ec, 0xc2bba124, + 0x49689f3d, 0x86f688f5, 0x9ff48896, 0x506a9f5e, 0xdbb9a147, + 0x1427b68f, 0x176edb34, 0xd8f0ccfc, 0x5323f2e5, 0x9cbde52d, + 0x55b12993, 0x9a2f3e5b, 0x11fc0042, 0xde62178a, 0xdd2b7a31, + 0x12b56df9, 0x996653e0, 0x56f84428, 0xd00eccdd, 0x1f90db15, + 0x9443e50c, 0x5bddf2c4, 0x58949f7f, 0x970a88b7, 0x1cd9b6ae, + 0xd347a166, 0x1a4b6dd8, 0xd5d57a10, 0x5e064409, 0x919853c1, + 0x92d13e7a, 0x5d4f29b2, 0xd69c17ab, 0x19020063, 0xe498176d, + 0x2b0600a5, 0xa0d53ebc, 0x6f4b2974, 0x6c0244cf, 0xa39c5307, + 0x284f6d1e, 0xe7d17ad6, 0x2eddb668, 0xe143a1a0, 0x6a909fb9, + 0xa50e8871, 0xa647e5ca, 0x69d9f202, 0xe20acc1b, 0x2d94dbd3, + 0xab625326, 0x64fc44ee, 0xef2f7af7, 0x20b16d3f, 0x23f80084, + 0xec66174c, 0x67b52955, 0xa82b3e9d, 0x6127f223, 0xaeb9e5eb, + 0x256adbf2, 0xeaf4cc3a, 0xe9bda181, 0x2623b649, 0xadf08850, + 0x626e9f98, 0x7b6c9ffb, 0xb4f28833, 0x3f21b62a, 0xf0bfa1e2, + 0xf3f6cc59, 0x3c68db91, 0xb7bbe588, 0x7825f240, 0xb1293efe, + 0x7eb72936, 0xf564172f, 0x3afa00e7, 0x39b36d5c, 0xf62d7a94, + 0x7dfe448d, 0xb2605345, 0x3496dbb0, 0xfb08cc78, 0x70dbf261, + 0xbf45e5a9, 0xbc0c8812, 0x73929fda, 0xf841a1c3, 0x37dfb60b, + 0xfed37ab5, 0x314d6d7d, 0xba9e5364, 0x750044ac, 0x76492917, + 0xb9d73edf, 0x320400c6, 0xfd9a170e, 0x1241289b, 0xdddf3f53, + 0x560c014a, 0x99921682, 0x9adb7b39, 0x55456cf1, 0xde9652e8, + 0x11084520, 0xd804899e, 0x179a9e56, 0x9c49a04f, 0x53d7b787, + 0x509eda3c, 0x9f00cdf4, 0x14d3f3ed, 0xdb4de425, 0x5dbb6cd0, + 0x92257b18, 0x19f64501, 0xd66852c9, 0xd5213f72, 0x1abf28ba, + 0x916c16a3, 0x5ef2016b, 0x97fecdd5, 0x5860da1d, 0xd3b3e404, + 0x1c2df3cc, 0x1f649e77, 0xd0fa89bf, 0x5b29b7a6, 0x94b7a06e, + 0x8db5a00d, 0x422bb7c5, 0xc9f889dc, 0x06669e14, 0x052ff3af, + 0xcab1e467, 0x4162da7e, 0x8efccdb6, 0x47f00108, 0x886e16c0, + 0x03bd28d9, 0xcc233f11, 0xcf6a52aa, 0x00f44562, 0x8b277b7b, + 0x44b96cb3, 0xc24fe446, 0x0dd1f38e, 0x8602cd97, 0x499cda5f, + 0x4ad5b7e4, 0x854ba02c, 0x0e989e35, 0xc10689fd, 0x080a4543, + 0xc794528b, 0x4c476c92, 0x83d97b5a, 0x809016e1, 0x4f0e0129, + 0xc4dd3f30, 0x0b4328f8, 0xf6d93ff6, 0x3947283e, 0xb2941627, + 0x7d0a01ef, 0x7e436c54, 0xb1dd7b9c, 0x3a0e4585, 0xf590524d, + 0x3c9c9ef3, 0xf302893b, 0x78d1b722, 0xb74fa0ea, 0xb406cd51, + 0x7b98da99, 0xf04be480, 0x3fd5f348, 0xb9237bbd, 0x76bd6c75, + 0xfd6e526c, 0x32f045a4, 0x31b9281f, 0xfe273fd7, 0x75f401ce, + 0xba6a1606, 0x7366dab8, 0xbcf8cd70, 0x372bf369, 0xf8b5e4a1, + 0xfbfc891a, 0x34629ed2, 0xbfb1a0cb, 0x702fb703, 0x692db760, + 0xa6b3a0a8, 0x2d609eb1, 0xe2fe8979, 0xe1b7e4c2, 0x2e29f30a, + 0xa5facd13, 0x6a64dadb, 0xa3681665, 0x6cf601ad, 0xe7253fb4, + 0x28bb287c, 0x2bf245c7, 0xe46c520f, 0x6fbf6c16, 0xa0217bde, + 0x26d7f32b, 0xe949e4e3, 0x629adafa, 0xad04cd32, 0xae4da089, + 0x61d3b741, 0xea008958, 0x259e9e90, 0xec92522e, 0x230c45e6, + 0xa8df7bff, 0x67416c37, 0x6408018c, 0xab961644, 0x2045285d, + 0xefdb3f95}, + {0x00000000, 0x24825136, 0x4904a26c, 0x6d86f35a, 0x920944d8, + 0xb68b15ee, 0xdb0de6b4, 0xff8fb782, 0xff638ff1, 0xdbe1dec7, + 0xb6672d9d, 0x92e57cab, 0x6d6acb29, 0x49e89a1f, 0x246e6945, + 0x00ec3873, 0x25b619a3, 0x01344895, 0x6cb2bbcf, 0x4830eaf9, + 0xb7bf5d7b, 0x933d0c4d, 0xfebbff17, 0xda39ae21, 0xdad59652, + 0xfe57c764, 0x93d1343e, 0xb7536508, 0x48dcd28a, 0x6c5e83bc, + 0x01d870e6, 0x255a21d0, 0x4b6c3346, 0x6fee6270, 0x0268912a, + 0x26eac01c, 0xd965779e, 0xfde726a8, 0x9061d5f2, 0xb4e384c4, + 0xb40fbcb7, 0x908ded81, 0xfd0b1edb, 0xd9894fed, 0x2606f86f, + 0x0284a959, 0x6f025a03, 0x4b800b35, 0x6eda2ae5, 0x4a587bd3, + 0x27de8889, 0x035cd9bf, 0xfcd36e3d, 0xd8513f0b, 0xb5d7cc51, + 0x91559d67, 0x91b9a514, 0xb53bf422, 0xd8bd0778, 0xfc3f564e, + 0x03b0e1cc, 0x2732b0fa, 0x4ab443a0, 0x6e361296, 0x96d8668c, + 0xb25a37ba, 0xdfdcc4e0, 0xfb5e95d6, 0x04d12254, 0x20537362, + 0x4dd58038, 0x6957d10e, 0x69bbe97d, 0x4d39b84b, 0x20bf4b11, + 0x043d1a27, 0xfbb2ada5, 0xdf30fc93, 0xb2b60fc9, 0x96345eff, + 0xb36e7f2f, 0x97ec2e19, 0xfa6add43, 0xdee88c75, 0x21673bf7, + 0x05e56ac1, 0x6863999b, 0x4ce1c8ad, 0x4c0df0de, 0x688fa1e8, + 0x050952b2, 0x218b0384, 0xde04b406, 0xfa86e530, 0x9700166a, + 0xb382475c, 0xddb455ca, 0xf93604fc, 0x94b0f7a6, 0xb032a690, + 0x4fbd1112, 0x6b3f4024, 0x06b9b37e, 0x223be248, 0x22d7da3b, + 0x06558b0d, 0x6bd37857, 0x4f512961, 0xb0de9ee3, 0x945ccfd5, + 0xf9da3c8f, 0xdd586db9, 0xf8024c69, 0xdc801d5f, 0xb106ee05, + 0x9584bf33, 0x6a0b08b1, 0x4e895987, 0x230faadd, 0x078dfbeb, + 0x0761c398, 0x23e392ae, 0x4e6561f4, 0x6ae730c2, 0x95688740, + 0xb1ead676, 0xdc6c252c, 0xf8ee741a, 0xf6c1cb59, 0xd2439a6f, + 0xbfc56935, 0x9b473803, 0x64c88f81, 0x404adeb7, 0x2dcc2ded, + 0x094e7cdb, 0x09a244a8, 0x2d20159e, 0x40a6e6c4, 0x6424b7f2, + 0x9bab0070, 0xbf295146, 0xd2afa21c, 0xf62df32a, 0xd377d2fa, + 0xf7f583cc, 0x9a737096, 0xbef121a0, 0x417e9622, 0x65fcc714, + 0x087a344e, 0x2cf86578, 0x2c145d0b, 0x08960c3d, 0x6510ff67, + 0x4192ae51, 0xbe1d19d3, 0x9a9f48e5, 0xf719bbbf, 0xd39bea89, + 0xbdadf81f, 0x992fa929, 0xf4a95a73, 0xd02b0b45, 0x2fa4bcc7, + 0x0b26edf1, 0x66a01eab, 0x42224f9d, 0x42ce77ee, 0x664c26d8, + 0x0bcad582, 0x2f4884b4, 0xd0c73336, 0xf4456200, 0x99c3915a, + 0xbd41c06c, 0x981be1bc, 0xbc99b08a, 0xd11f43d0, 0xf59d12e6, + 0x0a12a564, 0x2e90f452, 0x43160708, 0x6794563e, 0x67786e4d, + 0x43fa3f7b, 0x2e7ccc21, 0x0afe9d17, 0xf5712a95, 0xd1f37ba3, + 0xbc7588f9, 0x98f7d9cf, 0x6019add5, 0x449bfce3, 0x291d0fb9, + 0x0d9f5e8f, 0xf210e90d, 0xd692b83b, 0xbb144b61, 0x9f961a57, + 0x9f7a2224, 0xbbf87312, 0xd67e8048, 0xf2fcd17e, 0x0d7366fc, + 0x29f137ca, 0x4477c490, 0x60f595a6, 0x45afb476, 0x612de540, + 0x0cab161a, 0x2829472c, 0xd7a6f0ae, 0xf324a198, 0x9ea252c2, + 0xba2003f4, 0xbacc3b87, 0x9e4e6ab1, 0xf3c899eb, 0xd74ac8dd, + 0x28c57f5f, 0x0c472e69, 0x61c1dd33, 0x45438c05, 0x2b759e93, + 0x0ff7cfa5, 0x62713cff, 0x46f36dc9, 0xb97cda4b, 0x9dfe8b7d, + 0xf0787827, 0xd4fa2911, 0xd4161162, 0xf0944054, 0x9d12b30e, + 0xb990e238, 0x461f55ba, 0x629d048c, 0x0f1bf7d6, 0x2b99a6e0, + 0x0ec38730, 0x2a41d606, 0x47c7255c, 0x6345746a, 0x9ccac3e8, + 0xb84892de, 0xd5ce6184, 0xf14c30b2, 0xf1a008c1, 0xd52259f7, + 0xb8a4aaad, 0x9c26fb9b, 0x63a94c19, 0x472b1d2f, 0x2aadee75, + 0x0e2fbf43}, + {0x00000000, 0x36f290f3, 0x6de521e6, 0x5b17b115, 0xdbca43cc, + 0xed38d33f, 0xb62f622a, 0x80ddf2d9, 0x6ce581d9, 0x5a17112a, + 0x0100a03f, 0x37f230cc, 0xb72fc215, 0x81dd52e6, 0xdacae3f3, + 0xec387300, 0xd9cb03b2, 0xef399341, 0xb42e2254, 0x82dcb2a7, + 0x0201407e, 0x34f3d08d, 0x6fe46198, 0x5916f16b, 0xb52e826b, + 0x83dc1298, 0xd8cba38d, 0xee39337e, 0x6ee4c1a7, 0x58165154, + 0x0301e041, 0x35f370b2, 0x68e70125, 0x5e1591d6, 0x050220c3, + 0x33f0b030, 0xb32d42e9, 0x85dfd21a, 0xdec8630f, 0xe83af3fc, + 0x040280fc, 0x32f0100f, 0x69e7a11a, 0x5f1531e9, 0xdfc8c330, + 0xe93a53c3, 0xb22de2d6, 0x84df7225, 0xb12c0297, 0x87de9264, + 0xdcc92371, 0xea3bb382, 0x6ae6415b, 0x5c14d1a8, 0x070360bd, + 0x31f1f04e, 0xddc9834e, 0xeb3b13bd, 0xb02ca2a8, 0x86de325b, + 0x0603c082, 0x30f15071, 0x6be6e164, 0x5d147197, 0xd1ce024a, + 0xe73c92b9, 0xbc2b23ac, 0x8ad9b35f, 0x0a044186, 0x3cf6d175, + 0x67e16060, 0x5113f093, 0xbd2b8393, 0x8bd91360, 0xd0cea275, + 0xe63c3286, 0x66e1c05f, 0x501350ac, 0x0b04e1b9, 0x3df6714a, + 0x080501f8, 0x3ef7910b, 0x65e0201e, 0x5312b0ed, 0xd3cf4234, + 0xe53dd2c7, 0xbe2a63d2, 0x88d8f321, 0x64e08021, 0x521210d2, + 0x0905a1c7, 0x3ff73134, 0xbf2ac3ed, 0x89d8531e, 0xd2cfe20b, + 0xe43d72f8, 0xb929036f, 0x8fdb939c, 0xd4cc2289, 0xe23eb27a, + 0x62e340a3, 0x5411d050, 0x0f066145, 0x39f4f1b6, 0xd5cc82b6, + 0xe33e1245, 0xb829a350, 0x8edb33a3, 0x0e06c17a, 0x38f45189, + 0x63e3e09c, 0x5511706f, 0x60e200dd, 0x5610902e, 0x0d07213b, + 0x3bf5b1c8, 0xbb284311, 0x8ddad3e2, 0xd6cd62f7, 0xe03ff204, + 0x0c078104, 0x3af511f7, 0x61e2a0e2, 0x57103011, 0xd7cdc2c8, + 0xe13f523b, 0xba28e32e, 0x8cda73dd, 0x78ed02d5, 0x4e1f9226, + 0x15082333, 0x23fab3c0, 0xa3274119, 0x95d5d1ea, 0xcec260ff, + 0xf830f00c, 0x1408830c, 0x22fa13ff, 0x79eda2ea, 0x4f1f3219, + 0xcfc2c0c0, 0xf9305033, 0xa227e126, 0x94d571d5, 0xa1260167, + 0x97d49194, 0xccc32081, 0xfa31b072, 0x7aec42ab, 0x4c1ed258, + 0x1709634d, 0x21fbf3be, 0xcdc380be, 0xfb31104d, 0xa026a158, + 0x96d431ab, 0x1609c372, 0x20fb5381, 0x7bece294, 0x4d1e7267, + 0x100a03f0, 0x26f89303, 0x7def2216, 0x4b1db2e5, 0xcbc0403c, + 0xfd32d0cf, 0xa62561da, 0x90d7f129, 0x7cef8229, 0x4a1d12da, + 0x110aa3cf, 0x27f8333c, 0xa725c1e5, 0x91d75116, 0xcac0e003, + 0xfc3270f0, 0xc9c10042, 0xff3390b1, 0xa42421a4, 0x92d6b157, + 0x120b438e, 0x24f9d37d, 0x7fee6268, 0x491cf29b, 0xa524819b, + 0x93d61168, 0xc8c1a07d, 0xfe33308e, 0x7eeec257, 0x481c52a4, + 0x130be3b1, 0x25f97342, 0xa923009f, 0x9fd1906c, 0xc4c62179, + 0xf234b18a, 0x72e94353, 0x441bd3a0, 0x1f0c62b5, 0x29fef246, + 0xc5c68146, 0xf33411b5, 0xa823a0a0, 0x9ed13053, 0x1e0cc28a, + 0x28fe5279, 0x73e9e36c, 0x451b739f, 0x70e8032d, 0x461a93de, + 0x1d0d22cb, 0x2bffb238, 0xab2240e1, 0x9dd0d012, 0xc6c76107, + 0xf035f1f4, 0x1c0d82f4, 0x2aff1207, 0x71e8a312, 0x471a33e1, + 0xc7c7c138, 0xf13551cb, 0xaa22e0de, 0x9cd0702d, 0xc1c401ba, + 0xf7369149, 0xac21205c, 0x9ad3b0af, 0x1a0e4276, 0x2cfcd285, + 0x77eb6390, 0x4119f363, 0xad218063, 0x9bd31090, 0xc0c4a185, + 0xf6363176, 0x76ebc3af, 0x4019535c, 0x1b0ee249, 0x2dfc72ba, + 0x180f0208, 0x2efd92fb, 0x75ea23ee, 0x4318b31d, 0xc3c541c4, + 0xf537d137, 0xae206022, 0x98d2f0d1, 0x74ea83d1, 0x42181322, + 0x190fa237, 0x2ffd32c4, 0xaf20c01d, 0x99d250ee, 0xc2c5e1fb, + 0xf4377108}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x0000000000000000, 0xf390f23600000000, 0xe621e56d00000000, + 0x15b1175b00000000, 0xcc43cadb00000000, 0x3fd338ed00000000, + 0x2a622fb600000000, 0xd9f2dd8000000000, 0xd981e56c00000000, + 0x2a11175a00000000, 0x3fa0000100000000, 0xcc30f23700000000, + 0x15c22fb700000000, 0xe652dd8100000000, 0xf3e3cada00000000, + 0x007338ec00000000, 0xb203cbd900000000, 0x419339ef00000000, + 0x54222eb400000000, 0xa7b2dc8200000000, 0x7e40010200000000, + 0x8dd0f33400000000, 0x9861e46f00000000, 0x6bf1165900000000, + 0x6b822eb500000000, 0x9812dc8300000000, 0x8da3cbd800000000, + 0x7e3339ee00000000, 0xa7c1e46e00000000, 0x5451165800000000, + 0x41e0010300000000, 0xb270f33500000000, 0x2501e76800000000, + 0xd691155e00000000, 0xc320020500000000, 0x30b0f03300000000, + 0xe9422db300000000, 0x1ad2df8500000000, 0x0f63c8de00000000, + 0xfcf33ae800000000, 0xfc80020400000000, 0x0f10f03200000000, + 0x1aa1e76900000000, 0xe931155f00000000, 0x30c3c8df00000000, + 0xc3533ae900000000, 0xd6e22db200000000, 0x2572df8400000000, + 0x97022cb100000000, 0x6492de8700000000, 0x7123c9dc00000000, + 0x82b33bea00000000, 0x5b41e66a00000000, 0xa8d1145c00000000, + 0xbd60030700000000, 0x4ef0f13100000000, 0x4e83c9dd00000000, + 0xbd133beb00000000, 0xa8a22cb000000000, 0x5b32de8600000000, + 0x82c0030600000000, 0x7150f13000000000, 0x64e1e66b00000000, + 0x9771145d00000000, 0x4a02ced100000000, 0xb9923ce700000000, + 0xac232bbc00000000, 0x5fb3d98a00000000, 0x8641040a00000000, + 0x75d1f63c00000000, 0x6060e16700000000, 0x93f0135100000000, + 0x93832bbd00000000, 0x6013d98b00000000, 0x75a2ced000000000, + 0x86323ce600000000, 0x5fc0e16600000000, 0xac50135000000000, + 0xb9e1040b00000000, 0x4a71f63d00000000, 0xf801050800000000, + 0x0b91f73e00000000, 0x1e20e06500000000, 0xedb0125300000000, + 0x3442cfd300000000, 0xc7d23de500000000, 0xd2632abe00000000, + 0x21f3d88800000000, 0x2180e06400000000, 0xd210125200000000, + 0xc7a1050900000000, 0x3431f73f00000000, 0xedc32abf00000000, + 0x1e53d88900000000, 0x0be2cfd200000000, 0xf8723de400000000, + 0x6f0329b900000000, 0x9c93db8f00000000, 0x8922ccd400000000, + 0x7ab23ee200000000, 0xa340e36200000000, 0x50d0115400000000, + 0x4561060f00000000, 0xb6f1f43900000000, 0xb682ccd500000000, + 0x45123ee300000000, 0x50a329b800000000, 0xa333db8e00000000, + 0x7ac1060e00000000, 0x8951f43800000000, 0x9ce0e36300000000, + 0x6f70115500000000, 0xdd00e26000000000, 0x2e90105600000000, + 0x3b21070d00000000, 0xc8b1f53b00000000, 0x114328bb00000000, + 0xe2d3da8d00000000, 0xf762cdd600000000, 0x04f23fe000000000, + 0x0481070c00000000, 0xf711f53a00000000, 0xe2a0e26100000000, + 0x1130105700000000, 0xc8c2cdd700000000, 0x3b523fe100000000, + 0x2ee328ba00000000, 0xdd73da8c00000000, 0xd502ed7800000000, + 0x26921f4e00000000, 0x3323081500000000, 0xc0b3fa2300000000, + 0x194127a300000000, 0xead1d59500000000, 0xff60c2ce00000000, + 0x0cf030f800000000, 0x0c83081400000000, 0xff13fa2200000000, + 0xeaa2ed7900000000, 0x19321f4f00000000, 0xc0c0c2cf00000000, + 0x335030f900000000, 0x26e127a200000000, 0xd571d59400000000, + 0x670126a100000000, 0x9491d49700000000, 0x8120c3cc00000000, + 0x72b031fa00000000, 0xab42ec7a00000000, 0x58d21e4c00000000, + 0x4d63091700000000, 0xbef3fb2100000000, 0xbe80c3cd00000000, + 0x4d1031fb00000000, 0x58a126a000000000, 0xab31d49600000000, + 0x72c3091600000000, 0x8153fb2000000000, 0x94e2ec7b00000000, + 0x67721e4d00000000, 0xf0030a1000000000, 0x0393f82600000000, + 0x1622ef7d00000000, 0xe5b21d4b00000000, 0x3c40c0cb00000000, + 0xcfd032fd00000000, 0xda6125a600000000, 0x29f1d79000000000, + 0x2982ef7c00000000, 0xda121d4a00000000, 0xcfa30a1100000000, + 0x3c33f82700000000, 0xe5c125a700000000, 0x1651d79100000000, + 0x03e0c0ca00000000, 0xf07032fc00000000, 0x4200c1c900000000, + 0xb19033ff00000000, 0xa42124a400000000, 0x57b1d69200000000, + 0x8e430b1200000000, 0x7dd3f92400000000, 0x6862ee7f00000000, + 0x9bf21c4900000000, 0x9b8124a500000000, 0x6811d69300000000, + 0x7da0c1c800000000, 0x8e3033fe00000000, 0x57c2ee7e00000000, + 0xa4521c4800000000, 0xb1e30b1300000000, 0x4273f92500000000, + 0x9f0023a900000000, 0x6c90d19f00000000, 0x7921c6c400000000, + 0x8ab134f200000000, 0x5343e97200000000, 0xa0d31b4400000000, + 0xb5620c1f00000000, 0x46f2fe2900000000, 0x4681c6c500000000, + 0xb51134f300000000, 0xa0a023a800000000, 0x5330d19e00000000, + 0x8ac20c1e00000000, 0x7952fe2800000000, 0x6ce3e97300000000, + 0x9f731b4500000000, 0x2d03e87000000000, 0xde931a4600000000, + 0xcb220d1d00000000, 0x38b2ff2b00000000, 0xe14022ab00000000, + 0x12d0d09d00000000, 0x0761c7c600000000, 0xf4f135f000000000, + 0xf4820d1c00000000, 0x0712ff2a00000000, 0x12a3e87100000000, + 0xe1331a4700000000, 0x38c1c7c700000000, 0xcb5135f100000000, + 0xdee022aa00000000, 0x2d70d09c00000000, 0xba01c4c100000000, + 0x499136f700000000, 0x5c2021ac00000000, 0xafb0d39a00000000, + 0x76420e1a00000000, 0x85d2fc2c00000000, 0x9063eb7700000000, + 0x63f3194100000000, 0x638021ad00000000, 0x9010d39b00000000, + 0x85a1c4c000000000, 0x763136f600000000, 0xafc3eb7600000000, + 0x5c53194000000000, 0x49e20e1b00000000, 0xba72fc2d00000000, + 0x08020f1800000000, 0xfb92fd2e00000000, 0xee23ea7500000000, + 0x1db3184300000000, 0xc441c5c300000000, 0x37d137f500000000, + 0x226020ae00000000, 0xd1f0d29800000000, 0xd183ea7400000000, + 0x2213184200000000, 0x37a20f1900000000, 0xc432fd2f00000000, + 0x1dc020af00000000, 0xee50d29900000000, 0xfbe1c5c200000000, + 0x087137f400000000}, + {0x0000000000000000, 0x3651822400000000, 0x6ca2044900000000, + 0x5af3866d00000000, 0xd844099200000000, 0xee158bb600000000, + 0xb4e60ddb00000000, 0x82b78fff00000000, 0xf18f63ff00000000, + 0xc7dee1db00000000, 0x9d2d67b600000000, 0xab7ce59200000000, + 0x29cb6a6d00000000, 0x1f9ae84900000000, 0x45696e2400000000, + 0x7338ec0000000000, 0xa319b62500000000, 0x9548340100000000, + 0xcfbbb26c00000000, 0xf9ea304800000000, 0x7b5dbfb700000000, + 0x4d0c3d9300000000, 0x17ffbbfe00000000, 0x21ae39da00000000, + 0x5296d5da00000000, 0x64c757fe00000000, 0x3e34d19300000000, + 0x086553b700000000, 0x8ad2dc4800000000, 0xbc835e6c00000000, + 0xe670d80100000000, 0xd0215a2500000000, 0x46336c4b00000000, + 0x7062ee6f00000000, 0x2a91680200000000, 0x1cc0ea2600000000, + 0x9e7765d900000000, 0xa826e7fd00000000, 0xf2d5619000000000, + 0xc484e3b400000000, 0xb7bc0fb400000000, 0x81ed8d9000000000, + 0xdb1e0bfd00000000, 0xed4f89d900000000, 0x6ff8062600000000, + 0x59a9840200000000, 0x035a026f00000000, 0x350b804b00000000, + 0xe52ada6e00000000, 0xd37b584a00000000, 0x8988de2700000000, + 0xbfd95c0300000000, 0x3d6ed3fc00000000, 0x0b3f51d800000000, + 0x51ccd7b500000000, 0x679d559100000000, 0x14a5b99100000000, + 0x22f43bb500000000, 0x7807bdd800000000, 0x4e563ffc00000000, + 0xcce1b00300000000, 0xfab0322700000000, 0xa043b44a00000000, + 0x9612366e00000000, 0x8c66d89600000000, 0xba375ab200000000, + 0xe0c4dcdf00000000, 0xd6955efb00000000, 0x5422d10400000000, + 0x6273532000000000, 0x3880d54d00000000, 0x0ed1576900000000, + 0x7de9bb6900000000, 0x4bb8394d00000000, 0x114bbf2000000000, + 0x271a3d0400000000, 0xa5adb2fb00000000, 0x93fc30df00000000, + 0xc90fb6b200000000, 0xff5e349600000000, 0x2f7f6eb300000000, + 0x192eec9700000000, 0x43dd6afa00000000, 0x758ce8de00000000, + 0xf73b672100000000, 0xc16ae50500000000, 0x9b99636800000000, + 0xadc8e14c00000000, 0xdef00d4c00000000, 0xe8a18f6800000000, + 0xb252090500000000, 0x84038b2100000000, 0x06b404de00000000, + 0x30e586fa00000000, 0x6a16009700000000, 0x5c4782b300000000, + 0xca55b4dd00000000, 0xfc0436f900000000, 0xa6f7b09400000000, + 0x90a632b000000000, 0x1211bd4f00000000, 0x24403f6b00000000, + 0x7eb3b90600000000, 0x48e23b2200000000, 0x3bdad72200000000, + 0x0d8b550600000000, 0x5778d36b00000000, 0x6129514f00000000, + 0xe39edeb000000000, 0xd5cf5c9400000000, 0x8f3cdaf900000000, + 0xb96d58dd00000000, 0x694c02f800000000, 0x5f1d80dc00000000, + 0x05ee06b100000000, 0x33bf849500000000, 0xb1080b6a00000000, + 0x8759894e00000000, 0xddaa0f2300000000, 0xebfb8d0700000000, + 0x98c3610700000000, 0xae92e32300000000, 0xf461654e00000000, + 0xc230e76a00000000, 0x4087689500000000, 0x76d6eab100000000, + 0x2c256cdc00000000, 0x1a74eef800000000, 0x59cbc1f600000000, + 0x6f9a43d200000000, 0x3569c5bf00000000, 0x0338479b00000000, + 0x818fc86400000000, 0xb7de4a4000000000, 0xed2dcc2d00000000, + 0xdb7c4e0900000000, 0xa844a20900000000, 0x9e15202d00000000, + 0xc4e6a64000000000, 0xf2b7246400000000, 0x7000ab9b00000000, + 0x465129bf00000000, 0x1ca2afd200000000, 0x2af32df600000000, + 0xfad277d300000000, 0xcc83f5f700000000, 0x9670739a00000000, + 0xa021f1be00000000, 0x22967e4100000000, 0x14c7fc6500000000, + 0x4e347a0800000000, 0x7865f82c00000000, 0x0b5d142c00000000, + 0x3d0c960800000000, 0x67ff106500000000, 0x51ae924100000000, + 0xd3191dbe00000000, 0xe5489f9a00000000, 0xbfbb19f700000000, + 0x89ea9bd300000000, 0x1ff8adbd00000000, 0x29a92f9900000000, + 0x735aa9f400000000, 0x450b2bd000000000, 0xc7bca42f00000000, + 0xf1ed260b00000000, 0xab1ea06600000000, 0x9d4f224200000000, + 0xee77ce4200000000, 0xd8264c6600000000, 0x82d5ca0b00000000, + 0xb484482f00000000, 0x3633c7d000000000, 0x006245f400000000, + 0x5a91c39900000000, 0x6cc041bd00000000, 0xbce11b9800000000, + 0x8ab099bc00000000, 0xd0431fd100000000, 0xe6129df500000000, + 0x64a5120a00000000, 0x52f4902e00000000, 0x0807164300000000, + 0x3e56946700000000, 0x4d6e786700000000, 0x7b3ffa4300000000, + 0x21cc7c2e00000000, 0x179dfe0a00000000, 0x952a71f500000000, + 0xa37bf3d100000000, 0xf98875bc00000000, 0xcfd9f79800000000, + 0xd5ad196000000000, 0xe3fc9b4400000000, 0xb90f1d2900000000, + 0x8f5e9f0d00000000, 0x0de910f200000000, 0x3bb892d600000000, + 0x614b14bb00000000, 0x571a969f00000000, 0x24227a9f00000000, + 0x1273f8bb00000000, 0x48807ed600000000, 0x7ed1fcf200000000, + 0xfc66730d00000000, 0xca37f12900000000, 0x90c4774400000000, + 0xa695f56000000000, 0x76b4af4500000000, 0x40e52d6100000000, + 0x1a16ab0c00000000, 0x2c47292800000000, 0xaef0a6d700000000, + 0x98a124f300000000, 0xc252a29e00000000, 0xf40320ba00000000, + 0x873bccba00000000, 0xb16a4e9e00000000, 0xeb99c8f300000000, + 0xddc84ad700000000, 0x5f7fc52800000000, 0x692e470c00000000, + 0x33ddc16100000000, 0x058c434500000000, 0x939e752b00000000, + 0xa5cff70f00000000, 0xff3c716200000000, 0xc96df34600000000, + 0x4bda7cb900000000, 0x7d8bfe9d00000000, 0x277878f000000000, + 0x1129fad400000000, 0x621116d400000000, 0x544094f000000000, + 0x0eb3129d00000000, 0x38e290b900000000, 0xba551f4600000000, + 0x8c049d6200000000, 0xd6f71b0f00000000, 0xe0a6992b00000000, + 0x3087c30e00000000, 0x06d6412a00000000, 0x5c25c74700000000, + 0x6a74456300000000, 0xe8c3ca9c00000000, 0xde9248b800000000, + 0x8461ced500000000, 0xb2304cf100000000, 0xc108a0f100000000, + 0xf75922d500000000, 0xadaaa4b800000000, 0x9bfb269c00000000, + 0x194ca96300000000, 0x2f1d2b4700000000, 0x75eead2a00000000, + 0x43bf2f0e00000000}, + {0x0000000000000000, 0xc8179ecf00000000, 0xd1294d4400000000, + 0x193ed38b00000000, 0xa2539a8800000000, 0x6a44044700000000, + 0x737ad7cc00000000, 0xbb6d490300000000, 0x05a145ca00000000, + 0xcdb6db0500000000, 0xd488088e00000000, 0x1c9f964100000000, + 0xa7f2df4200000000, 0x6fe5418d00000000, 0x76db920600000000, + 0xbecc0cc900000000, 0x4b44fa4f00000000, 0x8353648000000000, + 0x9a6db70b00000000, 0x527a29c400000000, 0xe91760c700000000, + 0x2100fe0800000000, 0x383e2d8300000000, 0xf029b34c00000000, + 0x4ee5bf8500000000, 0x86f2214a00000000, 0x9fccf2c100000000, + 0x57db6c0e00000000, 0xecb6250d00000000, 0x24a1bbc200000000, + 0x3d9f684900000000, 0xf588f68600000000, 0x9688f49f00000000, + 0x5e9f6a5000000000, 0x47a1b9db00000000, 0x8fb6271400000000, + 0x34db6e1700000000, 0xfcccf0d800000000, 0xe5f2235300000000, + 0x2de5bd9c00000000, 0x9329b15500000000, 0x5b3e2f9a00000000, + 0x4200fc1100000000, 0x8a1762de00000000, 0x317a2bdd00000000, + 0xf96db51200000000, 0xe053669900000000, 0x2844f85600000000, + 0xddcc0ed000000000, 0x15db901f00000000, 0x0ce5439400000000, + 0xc4f2dd5b00000000, 0x7f9f945800000000, 0xb7880a9700000000, + 0xaeb6d91c00000000, 0x66a147d300000000, 0xd86d4b1a00000000, + 0x107ad5d500000000, 0x0944065e00000000, 0xc153989100000000, + 0x7a3ed19200000000, 0xb2294f5d00000000, 0xab179cd600000000, + 0x6300021900000000, 0x6d1798e400000000, 0xa500062b00000000, + 0xbc3ed5a000000000, 0x74294b6f00000000, 0xcf44026c00000000, + 0x07539ca300000000, 0x1e6d4f2800000000, 0xd67ad1e700000000, + 0x68b6dd2e00000000, 0xa0a143e100000000, 0xb99f906a00000000, + 0x71880ea500000000, 0xcae547a600000000, 0x02f2d96900000000, + 0x1bcc0ae200000000, 0xd3db942d00000000, 0x265362ab00000000, + 0xee44fc6400000000, 0xf77a2fef00000000, 0x3f6db12000000000, + 0x8400f82300000000, 0x4c1766ec00000000, 0x5529b56700000000, + 0x9d3e2ba800000000, 0x23f2276100000000, 0xebe5b9ae00000000, + 0xf2db6a2500000000, 0x3accf4ea00000000, 0x81a1bde900000000, + 0x49b6232600000000, 0x5088f0ad00000000, 0x989f6e6200000000, + 0xfb9f6c7b00000000, 0x3388f2b400000000, 0x2ab6213f00000000, + 0xe2a1bff000000000, 0x59ccf6f300000000, 0x91db683c00000000, + 0x88e5bbb700000000, 0x40f2257800000000, 0xfe3e29b100000000, + 0x3629b77e00000000, 0x2f1764f500000000, 0xe700fa3a00000000, + 0x5c6db33900000000, 0x947a2df600000000, 0x8d44fe7d00000000, + 0x455360b200000000, 0xb0db963400000000, 0x78cc08fb00000000, + 0x61f2db7000000000, 0xa9e545bf00000000, 0x12880cbc00000000, + 0xda9f927300000000, 0xc3a141f800000000, 0x0bb6df3700000000, + 0xb57ad3fe00000000, 0x7d6d4d3100000000, 0x64539eba00000000, + 0xac44007500000000, 0x1729497600000000, 0xdf3ed7b900000000, + 0xc600043200000000, 0x0e179afd00000000, 0x9b28411200000000, + 0x533fdfdd00000000, 0x4a010c5600000000, 0x8216929900000000, + 0x397bdb9a00000000, 0xf16c455500000000, 0xe85296de00000000, + 0x2045081100000000, 0x9e8904d800000000, 0x569e9a1700000000, + 0x4fa0499c00000000, 0x87b7d75300000000, 0x3cda9e5000000000, + 0xf4cd009f00000000, 0xedf3d31400000000, 0x25e44ddb00000000, + 0xd06cbb5d00000000, 0x187b259200000000, 0x0145f61900000000, + 0xc95268d600000000, 0x723f21d500000000, 0xba28bf1a00000000, + 0xa3166c9100000000, 0x6b01f25e00000000, 0xd5cdfe9700000000, + 0x1dda605800000000, 0x04e4b3d300000000, 0xccf32d1c00000000, + 0x779e641f00000000, 0xbf89fad000000000, 0xa6b7295b00000000, + 0x6ea0b79400000000, 0x0da0b58d00000000, 0xc5b72b4200000000, + 0xdc89f8c900000000, 0x149e660600000000, 0xaff32f0500000000, + 0x67e4b1ca00000000, 0x7eda624100000000, 0xb6cdfc8e00000000, + 0x0801f04700000000, 0xc0166e8800000000, 0xd928bd0300000000, + 0x113f23cc00000000, 0xaa526acf00000000, 0x6245f40000000000, + 0x7b7b278b00000000, 0xb36cb94400000000, 0x46e44fc200000000, + 0x8ef3d10d00000000, 0x97cd028600000000, 0x5fda9c4900000000, + 0xe4b7d54a00000000, 0x2ca04b8500000000, 0x359e980e00000000, + 0xfd8906c100000000, 0x43450a0800000000, 0x8b5294c700000000, + 0x926c474c00000000, 0x5a7bd98300000000, 0xe116908000000000, + 0x29010e4f00000000, 0x303fddc400000000, 0xf828430b00000000, + 0xf63fd9f600000000, 0x3e28473900000000, 0x271694b200000000, + 0xef010a7d00000000, 0x546c437e00000000, 0x9c7bddb100000000, + 0x85450e3a00000000, 0x4d5290f500000000, 0xf39e9c3c00000000, + 0x3b8902f300000000, 0x22b7d17800000000, 0xeaa04fb700000000, + 0x51cd06b400000000, 0x99da987b00000000, 0x80e44bf000000000, + 0x48f3d53f00000000, 0xbd7b23b900000000, 0x756cbd7600000000, + 0x6c526efd00000000, 0xa445f03200000000, 0x1f28b93100000000, + 0xd73f27fe00000000, 0xce01f47500000000, 0x06166aba00000000, + 0xb8da667300000000, 0x70cdf8bc00000000, 0x69f32b3700000000, + 0xa1e4b5f800000000, 0x1a89fcfb00000000, 0xd29e623400000000, + 0xcba0b1bf00000000, 0x03b72f7000000000, 0x60b72d6900000000, + 0xa8a0b3a600000000, 0xb19e602d00000000, 0x7989fee200000000, + 0xc2e4b7e100000000, 0x0af3292e00000000, 0x13cdfaa500000000, + 0xdbda646a00000000, 0x651668a300000000, 0xad01f66c00000000, + 0xb43f25e700000000, 0x7c28bb2800000000, 0xc745f22b00000000, + 0x0f526ce400000000, 0x166cbf6f00000000, 0xde7b21a000000000, + 0x2bf3d72600000000, 0xe3e449e900000000, 0xfada9a6200000000, + 0x32cd04ad00000000, 0x89a04dae00000000, 0x41b7d36100000000, + 0x588900ea00000000, 0x909e9e2500000000, 0x2e5292ec00000000, + 0xe6450c2300000000, 0xff7bdfa800000000, 0x376c416700000000, + 0x8c01086400000000, 0x441696ab00000000, 0x5d28452000000000, + 0x953fdbef00000000}, + {0x0000000000000000, 0x95d4709500000000, 0x6baf90f100000000, + 0xfe7be06400000000, 0x9758503800000000, 0x028c20ad00000000, + 0xfcf7c0c900000000, 0x6923b05c00000000, 0x2eb1a07000000000, + 0xbb65d0e500000000, 0x451e308100000000, 0xd0ca401400000000, + 0xb9e9f04800000000, 0x2c3d80dd00000000, 0xd24660b900000000, + 0x4792102c00000000, 0x5c6241e100000000, 0xc9b6317400000000, + 0x37cdd11000000000, 0xa219a18500000000, 0xcb3a11d900000000, + 0x5eee614c00000000, 0xa095812800000000, 0x3541f1bd00000000, + 0x72d3e19100000000, 0xe707910400000000, 0x197c716000000000, + 0x8ca801f500000000, 0xe58bb1a900000000, 0x705fc13c00000000, + 0x8e24215800000000, 0x1bf051cd00000000, 0xf9c2f31900000000, + 0x6c16838c00000000, 0x926d63e800000000, 0x07b9137d00000000, + 0x6e9aa32100000000, 0xfb4ed3b400000000, 0x053533d000000000, + 0x90e1434500000000, 0xd773536900000000, 0x42a723fc00000000, + 0xbcdcc39800000000, 0x2908b30d00000000, 0x402b035100000000, + 0xd5ff73c400000000, 0x2b8493a000000000, 0xbe50e33500000000, + 0xa5a0b2f800000000, 0x3074c26d00000000, 0xce0f220900000000, + 0x5bdb529c00000000, 0x32f8e2c000000000, 0xa72c925500000000, + 0x5957723100000000, 0xcc8302a400000000, 0x8b11128800000000, + 0x1ec5621d00000000, 0xe0be827900000000, 0x756af2ec00000000, + 0x1c4942b000000000, 0x899d322500000000, 0x77e6d24100000000, + 0xe232a2d400000000, 0xf285e73300000000, 0x675197a600000000, + 0x992a77c200000000, 0x0cfe075700000000, 0x65ddb70b00000000, + 0xf009c79e00000000, 0x0e7227fa00000000, 0x9ba6576f00000000, + 0xdc34474300000000, 0x49e037d600000000, 0xb79bd7b200000000, + 0x224fa72700000000, 0x4b6c177b00000000, 0xdeb867ee00000000, + 0x20c3878a00000000, 0xb517f71f00000000, 0xaee7a6d200000000, + 0x3b33d64700000000, 0xc548362300000000, 0x509c46b600000000, + 0x39bff6ea00000000, 0xac6b867f00000000, 0x5210661b00000000, + 0xc7c4168e00000000, 0x805606a200000000, 0x1582763700000000, + 0xebf9965300000000, 0x7e2de6c600000000, 0x170e569a00000000, + 0x82da260f00000000, 0x7ca1c66b00000000, 0xe975b6fe00000000, + 0x0b47142a00000000, 0x9e9364bf00000000, 0x60e884db00000000, + 0xf53cf44e00000000, 0x9c1f441200000000, 0x09cb348700000000, + 0xf7b0d4e300000000, 0x6264a47600000000, 0x25f6b45a00000000, + 0xb022c4cf00000000, 0x4e5924ab00000000, 0xdb8d543e00000000, + 0xb2aee46200000000, 0x277a94f700000000, 0xd901749300000000, + 0x4cd5040600000000, 0x572555cb00000000, 0xc2f1255e00000000, + 0x3c8ac53a00000000, 0xa95eb5af00000000, 0xc07d05f300000000, + 0x55a9756600000000, 0xabd2950200000000, 0x3e06e59700000000, + 0x7994f5bb00000000, 0xec40852e00000000, 0x123b654a00000000, + 0x87ef15df00000000, 0xeecca58300000000, 0x7b18d51600000000, + 0x8563357200000000, 0x10b745e700000000, 0xe40bcf6700000000, + 0x71dfbff200000000, 0x8fa45f9600000000, 0x1a702f0300000000, + 0x73539f5f00000000, 0xe687efca00000000, 0x18fc0fae00000000, + 0x8d287f3b00000000, 0xcaba6f1700000000, 0x5f6e1f8200000000, + 0xa115ffe600000000, 0x34c18f7300000000, 0x5de23f2f00000000, + 0xc8364fba00000000, 0x364dafde00000000, 0xa399df4b00000000, + 0xb8698e8600000000, 0x2dbdfe1300000000, 0xd3c61e7700000000, + 0x46126ee200000000, 0x2f31debe00000000, 0xbae5ae2b00000000, + 0x449e4e4f00000000, 0xd14a3eda00000000, 0x96d82ef600000000, + 0x030c5e6300000000, 0xfd77be0700000000, 0x68a3ce9200000000, + 0x01807ece00000000, 0x94540e5b00000000, 0x6a2fee3f00000000, + 0xfffb9eaa00000000, 0x1dc93c7e00000000, 0x881d4ceb00000000, + 0x7666ac8f00000000, 0xe3b2dc1a00000000, 0x8a916c4600000000, + 0x1f451cd300000000, 0xe13efcb700000000, 0x74ea8c2200000000, + 0x33789c0e00000000, 0xa6acec9b00000000, 0x58d70cff00000000, + 0xcd037c6a00000000, 0xa420cc3600000000, 0x31f4bca300000000, + 0xcf8f5cc700000000, 0x5a5b2c5200000000, 0x41ab7d9f00000000, + 0xd47f0d0a00000000, 0x2a04ed6e00000000, 0xbfd09dfb00000000, + 0xd6f32da700000000, 0x43275d3200000000, 0xbd5cbd5600000000, + 0x2888cdc300000000, 0x6f1addef00000000, 0xfacead7a00000000, + 0x04b54d1e00000000, 0x91613d8b00000000, 0xf8428dd700000000, + 0x6d96fd4200000000, 0x93ed1d2600000000, 0x06396db300000000, + 0x168e285400000000, 0x835a58c100000000, 0x7d21b8a500000000, + 0xe8f5c83000000000, 0x81d6786c00000000, 0x140208f900000000, + 0xea79e89d00000000, 0x7fad980800000000, 0x383f882400000000, + 0xadebf8b100000000, 0x539018d500000000, 0xc644684000000000, + 0xaf67d81c00000000, 0x3ab3a88900000000, 0xc4c848ed00000000, + 0x511c387800000000, 0x4aec69b500000000, 0xdf38192000000000, + 0x2143f94400000000, 0xb49789d100000000, 0xddb4398d00000000, + 0x4860491800000000, 0xb61ba97c00000000, 0x23cfd9e900000000, + 0x645dc9c500000000, 0xf189b95000000000, 0x0ff2593400000000, + 0x9a2629a100000000, 0xf30599fd00000000, 0x66d1e96800000000, + 0x98aa090c00000000, 0x0d7e799900000000, 0xef4cdb4d00000000, + 0x7a98abd800000000, 0x84e34bbc00000000, 0x11373b2900000000, + 0x78148b7500000000, 0xedc0fbe000000000, 0x13bb1b8400000000, + 0x866f6b1100000000, 0xc1fd7b3d00000000, 0x54290ba800000000, + 0xaa52ebcc00000000, 0x3f869b5900000000, 0x56a52b0500000000, + 0xc3715b9000000000, 0x3d0abbf400000000, 0xa8decb6100000000, + 0xb32e9aac00000000, 0x26faea3900000000, 0xd8810a5d00000000, + 0x4d557ac800000000, 0x2476ca9400000000, 0xb1a2ba0100000000, + 0x4fd95a6500000000, 0xda0d2af000000000, 0x9d9f3adc00000000, + 0x084b4a4900000000, 0xf630aa2d00000000, 0x63e4dab800000000, + 0x0ac76ae400000000, 0x9f131a7100000000, 0x6168fa1500000000, + 0xf4bc8a8000000000}, + {0x0000000000000000, 0x1f17f08000000000, 0x7f2891da00000000, + 0x603f615a00000000, 0xbf56536e00000000, 0xa041a3ee00000000, + 0xc07ec2b400000000, 0xdf69323400000000, 0x7eada6dc00000000, + 0x61ba565c00000000, 0x0185370600000000, 0x1e92c78600000000, + 0xc1fbf5b200000000, 0xdeec053200000000, 0xbed3646800000000, + 0xa1c494e800000000, 0xbd5c3c6200000000, 0xa24bcce200000000, + 0xc274adb800000000, 0xdd635d3800000000, 0x020a6f0c00000000, + 0x1d1d9f8c00000000, 0x7d22fed600000000, 0x62350e5600000000, + 0xc3f19abe00000000, 0xdce66a3e00000000, 0xbcd90b6400000000, + 0xa3cefbe400000000, 0x7ca7c9d000000000, 0x63b0395000000000, + 0x038f580a00000000, 0x1c98a88a00000000, 0x7ab978c400000000, + 0x65ae884400000000, 0x0591e91e00000000, 0x1a86199e00000000, + 0xc5ef2baa00000000, 0xdaf8db2a00000000, 0xbac7ba7000000000, + 0xa5d04af000000000, 0x0414de1800000000, 0x1b032e9800000000, + 0x7b3c4fc200000000, 0x642bbf4200000000, 0xbb428d7600000000, + 0xa4557df600000000, 0xc46a1cac00000000, 0xdb7dec2c00000000, + 0xc7e544a600000000, 0xd8f2b42600000000, 0xb8cdd57c00000000, + 0xa7da25fc00000000, 0x78b317c800000000, 0x67a4e74800000000, + 0x079b861200000000, 0x188c769200000000, 0xb948e27a00000000, + 0xa65f12fa00000000, 0xc66073a000000000, 0xd977832000000000, + 0x061eb11400000000, 0x1909419400000000, 0x793620ce00000000, + 0x6621d04e00000000, 0xb574805300000000, 0xaa6370d300000000, + 0xca5c118900000000, 0xd54be10900000000, 0x0a22d33d00000000, + 0x153523bd00000000, 0x750a42e700000000, 0x6a1db26700000000, + 0xcbd9268f00000000, 0xd4ced60f00000000, 0xb4f1b75500000000, + 0xabe647d500000000, 0x748f75e100000000, 0x6b98856100000000, + 0x0ba7e43b00000000, 0x14b014bb00000000, 0x0828bc3100000000, + 0x173f4cb100000000, 0x77002deb00000000, 0x6817dd6b00000000, + 0xb77eef5f00000000, 0xa8691fdf00000000, 0xc8567e8500000000, + 0xd7418e0500000000, 0x76851aed00000000, 0x6992ea6d00000000, + 0x09ad8b3700000000, 0x16ba7bb700000000, 0xc9d3498300000000, + 0xd6c4b90300000000, 0xb6fbd85900000000, 0xa9ec28d900000000, + 0xcfcdf89700000000, 0xd0da081700000000, 0xb0e5694d00000000, + 0xaff299cd00000000, 0x709babf900000000, 0x6f8c5b7900000000, + 0x0fb33a2300000000, 0x10a4caa300000000, 0xb1605e4b00000000, + 0xae77aecb00000000, 0xce48cf9100000000, 0xd15f3f1100000000, + 0x0e360d2500000000, 0x1121fda500000000, 0x711e9cff00000000, + 0x6e096c7f00000000, 0x7291c4f500000000, 0x6d86347500000000, + 0x0db9552f00000000, 0x12aea5af00000000, 0xcdc7979b00000000, + 0xd2d0671b00000000, 0xb2ef064100000000, 0xadf8f6c100000000, + 0x0c3c622900000000, 0x132b92a900000000, 0x7314f3f300000000, + 0x6c03037300000000, 0xb36a314700000000, 0xac7dc1c700000000, + 0xcc42a09d00000000, 0xd355501d00000000, 0x6ae900a700000000, + 0x75fef02700000000, 0x15c1917d00000000, 0x0ad661fd00000000, + 0xd5bf53c900000000, 0xcaa8a34900000000, 0xaa97c21300000000, + 0xb580329300000000, 0x1444a67b00000000, 0x0b5356fb00000000, + 0x6b6c37a100000000, 0x747bc72100000000, 0xab12f51500000000, + 0xb405059500000000, 0xd43a64cf00000000, 0xcb2d944f00000000, + 0xd7b53cc500000000, 0xc8a2cc4500000000, 0xa89dad1f00000000, + 0xb78a5d9f00000000, 0x68e36fab00000000, 0x77f49f2b00000000, + 0x17cbfe7100000000, 0x08dc0ef100000000, 0xa9189a1900000000, + 0xb60f6a9900000000, 0xd6300bc300000000, 0xc927fb4300000000, + 0x164ec97700000000, 0x095939f700000000, 0x696658ad00000000, + 0x7671a82d00000000, 0x1050786300000000, 0x0f4788e300000000, + 0x6f78e9b900000000, 0x706f193900000000, 0xaf062b0d00000000, + 0xb011db8d00000000, 0xd02ebad700000000, 0xcf394a5700000000, + 0x6efddebf00000000, 0x71ea2e3f00000000, 0x11d54f6500000000, + 0x0ec2bfe500000000, 0xd1ab8dd100000000, 0xcebc7d5100000000, + 0xae831c0b00000000, 0xb194ec8b00000000, 0xad0c440100000000, + 0xb21bb48100000000, 0xd224d5db00000000, 0xcd33255b00000000, + 0x125a176f00000000, 0x0d4de7ef00000000, 0x6d7286b500000000, + 0x7265763500000000, 0xd3a1e2dd00000000, 0xccb6125d00000000, + 0xac89730700000000, 0xb39e838700000000, 0x6cf7b1b300000000, + 0x73e0413300000000, 0x13df206900000000, 0x0cc8d0e900000000, + 0xdf9d80f400000000, 0xc08a707400000000, 0xa0b5112e00000000, + 0xbfa2e1ae00000000, 0x60cbd39a00000000, 0x7fdc231a00000000, + 0x1fe3424000000000, 0x00f4b2c000000000, 0xa130262800000000, + 0xbe27d6a800000000, 0xde18b7f200000000, 0xc10f477200000000, + 0x1e66754600000000, 0x017185c600000000, 0x614ee49c00000000, + 0x7e59141c00000000, 0x62c1bc9600000000, 0x7dd64c1600000000, + 0x1de92d4c00000000, 0x02feddcc00000000, 0xdd97eff800000000, + 0xc2801f7800000000, 0xa2bf7e2200000000, 0xbda88ea200000000, + 0x1c6c1a4a00000000, 0x037beaca00000000, 0x63448b9000000000, + 0x7c537b1000000000, 0xa33a492400000000, 0xbc2db9a400000000, + 0xdc12d8fe00000000, 0xc305287e00000000, 0xa524f83000000000, + 0xba3308b000000000, 0xda0c69ea00000000, 0xc51b996a00000000, + 0x1a72ab5e00000000, 0x05655bde00000000, 0x655a3a8400000000, + 0x7a4dca0400000000, 0xdb895eec00000000, 0xc49eae6c00000000, + 0xa4a1cf3600000000, 0xbbb63fb600000000, 0x64df0d8200000000, + 0x7bc8fd0200000000, 0x1bf79c5800000000, 0x04e06cd800000000, + 0x1878c45200000000, 0x076f34d200000000, 0x6750558800000000, + 0x7847a50800000000, 0xa72e973c00000000, 0xb83967bc00000000, + 0xd80606e600000000, 0xc711f66600000000, 0x66d5628e00000000, + 0x79c2920e00000000, 0x19fdf35400000000, 0x06ea03d400000000, + 0xd98331e000000000, 0xc694c16000000000, 0xa6aba03a00000000, + 0xb9bc50ba00000000}, + {0x0000000000000000, 0xe2fd888d00000000, 0x85fd60c000000000, + 0x6700e84d00000000, 0x4bfdb05b00000000, 0xa90038d600000000, + 0xce00d09b00000000, 0x2cfd581600000000, 0x96fa61b700000000, + 0x7407e93a00000000, 0x1307017700000000, 0xf1fa89fa00000000, + 0xdd07d1ec00000000, 0x3ffa596100000000, 0x58fab12c00000000, + 0xba0739a100000000, 0x6df3b2b500000000, 0x8f0e3a3800000000, + 0xe80ed27500000000, 0x0af35af800000000, 0x260e02ee00000000, + 0xc4f38a6300000000, 0xa3f3622e00000000, 0x410eeaa300000000, + 0xfb09d30200000000, 0x19f45b8f00000000, 0x7ef4b3c200000000, + 0x9c093b4f00000000, 0xb0f4635900000000, 0x5209ebd400000000, + 0x3509039900000000, 0xd7f48b1400000000, 0x9be014b000000000, + 0x791d9c3d00000000, 0x1e1d747000000000, 0xfce0fcfd00000000, + 0xd01da4eb00000000, 0x32e02c6600000000, 0x55e0c42b00000000, + 0xb71d4ca600000000, 0x0d1a750700000000, 0xefe7fd8a00000000, + 0x88e715c700000000, 0x6a1a9d4a00000000, 0x46e7c55c00000000, + 0xa41a4dd100000000, 0xc31aa59c00000000, 0x21e72d1100000000, + 0xf613a60500000000, 0x14ee2e8800000000, 0x73eec6c500000000, + 0x91134e4800000000, 0xbdee165e00000000, 0x5f139ed300000000, + 0x3813769e00000000, 0xdaeefe1300000000, 0x60e9c7b200000000, + 0x82144f3f00000000, 0xe514a77200000000, 0x07e92fff00000000, + 0x2b1477e900000000, 0xc9e9ff6400000000, 0xaee9172900000000, + 0x4c149fa400000000, 0x77c758bb00000000, 0x953ad03600000000, + 0xf23a387b00000000, 0x10c7b0f600000000, 0x3c3ae8e000000000, + 0xdec7606d00000000, 0xb9c7882000000000, 0x5b3a00ad00000000, + 0xe13d390c00000000, 0x03c0b18100000000, 0x64c059cc00000000, + 0x863dd14100000000, 0xaac0895700000000, 0x483d01da00000000, + 0x2f3de99700000000, 0xcdc0611a00000000, 0x1a34ea0e00000000, + 0xf8c9628300000000, 0x9fc98ace00000000, 0x7d34024300000000, + 0x51c95a5500000000, 0xb334d2d800000000, 0xd4343a9500000000, + 0x36c9b21800000000, 0x8cce8bb900000000, 0x6e33033400000000, + 0x0933eb7900000000, 0xebce63f400000000, 0xc7333be200000000, + 0x25ceb36f00000000, 0x42ce5b2200000000, 0xa033d3af00000000, + 0xec274c0b00000000, 0x0edac48600000000, 0x69da2ccb00000000, + 0x8b27a44600000000, 0xa7dafc5000000000, 0x452774dd00000000, + 0x22279c9000000000, 0xc0da141d00000000, 0x7add2dbc00000000, + 0x9820a53100000000, 0xff204d7c00000000, 0x1dddc5f100000000, + 0x31209de700000000, 0xd3dd156a00000000, 0xb4ddfd2700000000, + 0x562075aa00000000, 0x81d4febe00000000, 0x6329763300000000, + 0x04299e7e00000000, 0xe6d416f300000000, 0xca294ee500000000, + 0x28d4c66800000000, 0x4fd42e2500000000, 0xad29a6a800000000, + 0x172e9f0900000000, 0xf5d3178400000000, 0x92d3ffc900000000, + 0x702e774400000000, 0x5cd32f5200000000, 0xbe2ea7df00000000, + 0xd92e4f9200000000, 0x3bd3c71f00000000, 0xaf88c0ad00000000, + 0x4d75482000000000, 0x2a75a06d00000000, 0xc88828e000000000, + 0xe47570f600000000, 0x0688f87b00000000, 0x6188103600000000, + 0x837598bb00000000, 0x3972a11a00000000, 0xdb8f299700000000, + 0xbc8fc1da00000000, 0x5e72495700000000, 0x728f114100000000, + 0x907299cc00000000, 0xf772718100000000, 0x158ff90c00000000, + 0xc27b721800000000, 0x2086fa9500000000, 0x478612d800000000, + 0xa57b9a5500000000, 0x8986c24300000000, 0x6b7b4ace00000000, + 0x0c7ba28300000000, 0xee862a0e00000000, 0x548113af00000000, + 0xb67c9b2200000000, 0xd17c736f00000000, 0x3381fbe200000000, + 0x1f7ca3f400000000, 0xfd812b7900000000, 0x9a81c33400000000, + 0x787c4bb900000000, 0x3468d41d00000000, 0xd6955c9000000000, + 0xb195b4dd00000000, 0x53683c5000000000, 0x7f95644600000000, + 0x9d68eccb00000000, 0xfa68048600000000, 0x18958c0b00000000, + 0xa292b5aa00000000, 0x406f3d2700000000, 0x276fd56a00000000, + 0xc5925de700000000, 0xe96f05f100000000, 0x0b928d7c00000000, + 0x6c92653100000000, 0x8e6fedbc00000000, 0x599b66a800000000, + 0xbb66ee2500000000, 0xdc66066800000000, 0x3e9b8ee500000000, + 0x1266d6f300000000, 0xf09b5e7e00000000, 0x979bb63300000000, + 0x75663ebe00000000, 0xcf61071f00000000, 0x2d9c8f9200000000, + 0x4a9c67df00000000, 0xa861ef5200000000, 0x849cb74400000000, + 0x66613fc900000000, 0x0161d78400000000, 0xe39c5f0900000000, + 0xd84f981600000000, 0x3ab2109b00000000, 0x5db2f8d600000000, + 0xbf4f705b00000000, 0x93b2284d00000000, 0x714fa0c000000000, + 0x164f488d00000000, 0xf4b2c00000000000, 0x4eb5f9a100000000, + 0xac48712c00000000, 0xcb48996100000000, 0x29b511ec00000000, + 0x054849fa00000000, 0xe7b5c17700000000, 0x80b5293a00000000, + 0x6248a1b700000000, 0xb5bc2aa300000000, 0x5741a22e00000000, + 0x30414a6300000000, 0xd2bcc2ee00000000, 0xfe419af800000000, + 0x1cbc127500000000, 0x7bbcfa3800000000, 0x994172b500000000, + 0x23464b1400000000, 0xc1bbc39900000000, 0xa6bb2bd400000000, + 0x4446a35900000000, 0x68bbfb4f00000000, 0x8a4673c200000000, + 0xed469b8f00000000, 0x0fbb130200000000, 0x43af8ca600000000, + 0xa152042b00000000, 0xc652ec6600000000, 0x24af64eb00000000, + 0x08523cfd00000000, 0xeaafb47000000000, 0x8daf5c3d00000000, + 0x6f52d4b000000000, 0xd555ed1100000000, 0x37a8659c00000000, + 0x50a88dd100000000, 0xb255055c00000000, 0x9ea85d4a00000000, + 0x7c55d5c700000000, 0x1b553d8a00000000, 0xf9a8b50700000000, + 0x2e5c3e1300000000, 0xcca1b69e00000000, 0xaba15ed300000000, + 0x495cd65e00000000, 0x65a18e4800000000, 0x875c06c500000000, + 0xe05cee8800000000, 0x02a1660500000000, 0xb8a65fa400000000, + 0x5a5bd72900000000, 0x3d5b3f6400000000, 0xdfa6b7e900000000, + 0xf35befff00000000, 0x11a6677200000000, 0x76a68f3f00000000, + 0x945b07b200000000}, + {0x0000000000000000, 0xa90b894e00000000, 0x5217129d00000000, + 0xfb1c9bd300000000, 0xe52855e100000000, 0x4c23dcaf00000000, + 0xb73f477c00000000, 0x1e34ce3200000000, 0x8b57db1900000000, + 0x225c525700000000, 0xd940c98400000000, 0x704b40ca00000000, + 0x6e7f8ef800000000, 0xc77407b600000000, 0x3c689c6500000000, + 0x9563152b00000000, 0x16afb63300000000, 0xbfa43f7d00000000, + 0x44b8a4ae00000000, 0xedb32de000000000, 0xf387e3d200000000, + 0x5a8c6a9c00000000, 0xa190f14f00000000, 0x089b780100000000, + 0x9df86d2a00000000, 0x34f3e46400000000, 0xcfef7fb700000000, + 0x66e4f6f900000000, 0x78d038cb00000000, 0xd1dbb18500000000, + 0x2ac72a5600000000, 0x83cca31800000000, 0x2c5e6d6700000000, + 0x8555e42900000000, 0x7e497ffa00000000, 0xd742f6b400000000, + 0xc976388600000000, 0x607db1c800000000, 0x9b612a1b00000000, + 0x326aa35500000000, 0xa709b67e00000000, 0x0e023f3000000000, + 0xf51ea4e300000000, 0x5c152dad00000000, 0x4221e39f00000000, + 0xeb2a6ad100000000, 0x1036f10200000000, 0xb93d784c00000000, + 0x3af1db5400000000, 0x93fa521a00000000, 0x68e6c9c900000000, + 0xc1ed408700000000, 0xdfd98eb500000000, 0x76d207fb00000000, + 0x8dce9c2800000000, 0x24c5156600000000, 0xb1a6004d00000000, + 0x18ad890300000000, 0xe3b112d000000000, 0x4aba9b9e00000000, + 0x548e55ac00000000, 0xfd85dce200000000, 0x0699473100000000, + 0xaf92ce7f00000000, 0x58bcdace00000000, 0xf1b7538000000000, + 0x0aabc85300000000, 0xa3a0411d00000000, 0xbd948f2f00000000, + 0x149f066100000000, 0xef839db200000000, 0x468814fc00000000, + 0xd3eb01d700000000, 0x7ae0889900000000, 0x81fc134a00000000, + 0x28f79a0400000000, 0x36c3543600000000, 0x9fc8dd7800000000, + 0x64d446ab00000000, 0xcddfcfe500000000, 0x4e136cfd00000000, + 0xe718e5b300000000, 0x1c047e6000000000, 0xb50ff72e00000000, + 0xab3b391c00000000, 0x0230b05200000000, 0xf92c2b8100000000, + 0x5027a2cf00000000, 0xc544b7e400000000, 0x6c4f3eaa00000000, + 0x9753a57900000000, 0x3e582c3700000000, 0x206ce20500000000, + 0x89676b4b00000000, 0x727bf09800000000, 0xdb7079d600000000, + 0x74e2b7a900000000, 0xdde93ee700000000, 0x26f5a53400000000, + 0x8ffe2c7a00000000, 0x91cae24800000000, 0x38c16b0600000000, + 0xc3ddf0d500000000, 0x6ad6799b00000000, 0xffb56cb000000000, + 0x56bee5fe00000000, 0xada27e2d00000000, 0x04a9f76300000000, + 0x1a9d395100000000, 0xb396b01f00000000, 0x488a2bcc00000000, + 0xe181a28200000000, 0x624d019a00000000, 0xcb4688d400000000, + 0x305a130700000000, 0x99519a4900000000, 0x8765547b00000000, + 0x2e6edd3500000000, 0xd57246e600000000, 0x7c79cfa800000000, + 0xe91ada8300000000, 0x401153cd00000000, 0xbb0dc81e00000000, + 0x1206415000000000, 0x0c328f6200000000, 0xa539062c00000000, + 0x5e259dff00000000, 0xf72e14b100000000, 0xf17ec44600000000, + 0x58754d0800000000, 0xa369d6db00000000, 0x0a625f9500000000, + 0x145691a700000000, 0xbd5d18e900000000, 0x4641833a00000000, + 0xef4a0a7400000000, 0x7a291f5f00000000, 0xd322961100000000, + 0x283e0dc200000000, 0x8135848c00000000, 0x9f014abe00000000, + 0x360ac3f000000000, 0xcd16582300000000, 0x641dd16d00000000, + 0xe7d1727500000000, 0x4edafb3b00000000, 0xb5c660e800000000, + 0x1ccde9a600000000, 0x02f9279400000000, 0xabf2aeda00000000, + 0x50ee350900000000, 0xf9e5bc4700000000, 0x6c86a96c00000000, + 0xc58d202200000000, 0x3e91bbf100000000, 0x979a32bf00000000, + 0x89aefc8d00000000, 0x20a575c300000000, 0xdbb9ee1000000000, + 0x72b2675e00000000, 0xdd20a92100000000, 0x742b206f00000000, + 0x8f37bbbc00000000, 0x263c32f200000000, 0x3808fcc000000000, + 0x9103758e00000000, 0x6a1fee5d00000000, 0xc314671300000000, + 0x5677723800000000, 0xff7cfb7600000000, 0x046060a500000000, + 0xad6be9eb00000000, 0xb35f27d900000000, 0x1a54ae9700000000, + 0xe148354400000000, 0x4843bc0a00000000, 0xcb8f1f1200000000, + 0x6284965c00000000, 0x99980d8f00000000, 0x309384c100000000, + 0x2ea74af300000000, 0x87acc3bd00000000, 0x7cb0586e00000000, + 0xd5bbd12000000000, 0x40d8c40b00000000, 0xe9d34d4500000000, + 0x12cfd69600000000, 0xbbc45fd800000000, 0xa5f091ea00000000, + 0x0cfb18a400000000, 0xf7e7837700000000, 0x5eec0a3900000000, + 0xa9c21e8800000000, 0x00c997c600000000, 0xfbd50c1500000000, + 0x52de855b00000000, 0x4cea4b6900000000, 0xe5e1c22700000000, + 0x1efd59f400000000, 0xb7f6d0ba00000000, 0x2295c59100000000, + 0x8b9e4cdf00000000, 0x7082d70c00000000, 0xd9895e4200000000, + 0xc7bd907000000000, 0x6eb6193e00000000, 0x95aa82ed00000000, + 0x3ca10ba300000000, 0xbf6da8bb00000000, 0x166621f500000000, + 0xed7aba2600000000, 0x4471336800000000, 0x5a45fd5a00000000, + 0xf34e741400000000, 0x0852efc700000000, 0xa159668900000000, + 0x343a73a200000000, 0x9d31faec00000000, 0x662d613f00000000, + 0xcf26e87100000000, 0xd112264300000000, 0x7819af0d00000000, + 0x830534de00000000, 0x2a0ebd9000000000, 0x859c73ef00000000, + 0x2c97faa100000000, 0xd78b617200000000, 0x7e80e83c00000000, + 0x60b4260e00000000, 0xc9bfaf4000000000, 0x32a3349300000000, + 0x9ba8bddd00000000, 0x0ecba8f600000000, 0xa7c021b800000000, + 0x5cdcba6b00000000, 0xf5d7332500000000, 0xebe3fd1700000000, + 0x42e8745900000000, 0xb9f4ef8a00000000, 0x10ff66c400000000, + 0x9333c5dc00000000, 0x3a384c9200000000, 0xc124d74100000000, + 0x682f5e0f00000000, 0x761b903d00000000, 0xdf10197300000000, + 0x240c82a000000000, 0x8d070bee00000000, 0x18641ec500000000, + 0xb16f978b00000000, 0x4a730c5800000000, 0xe378851600000000, + 0xfd4c4b2400000000, 0x5447c26a00000000, 0xaf5b59b900000000, + 0x0650d0f700000000}, + {0x0000000000000000, 0x479244af00000000, 0xcf22f88500000000, + 0x88b0bc2a00000000, 0xdf4381d000000000, 0x98d1c57f00000000, + 0x1061795500000000, 0x57f33dfa00000000, 0xff81737a00000000, + 0xb81337d500000000, 0x30a38bff00000000, 0x7731cf5000000000, + 0x20c2f2aa00000000, 0x6750b60500000000, 0xefe00a2f00000000, + 0xa8724e8000000000, 0xfe03e7f400000000, 0xb991a35b00000000, + 0x31211f7100000000, 0x76b35bde00000000, 0x2140662400000000, + 0x66d2228b00000000, 0xee629ea100000000, 0xa9f0da0e00000000, + 0x0182948e00000000, 0x4610d02100000000, 0xcea06c0b00000000, + 0x893228a400000000, 0xdec1155e00000000, 0x995351f100000000, + 0x11e3eddb00000000, 0x5671a97400000000, 0xbd01bf3200000000, + 0xfa93fb9d00000000, 0x722347b700000000, 0x35b1031800000000, + 0x62423ee200000000, 0x25d07a4d00000000, 0xad60c66700000000, + 0xeaf282c800000000, 0x4280cc4800000000, 0x051288e700000000, + 0x8da234cd00000000, 0xca30706200000000, 0x9dc34d9800000000, + 0xda51093700000000, 0x52e1b51d00000000, 0x1573f1b200000000, + 0x430258c600000000, 0x04901c6900000000, 0x8c20a04300000000, + 0xcbb2e4ec00000000, 0x9c41d91600000000, 0xdbd39db900000000, + 0x5363219300000000, 0x14f1653c00000000, 0xbc832bbc00000000, + 0xfb116f1300000000, 0x73a1d33900000000, 0x3433979600000000, + 0x63c0aa6c00000000, 0x2452eec300000000, 0xace252e900000000, + 0xeb70164600000000, 0x7a037e6500000000, 0x3d913aca00000000, + 0xb52186e000000000, 0xf2b3c24f00000000, 0xa540ffb500000000, + 0xe2d2bb1a00000000, 0x6a62073000000000, 0x2df0439f00000000, + 0x85820d1f00000000, 0xc21049b000000000, 0x4aa0f59a00000000, + 0x0d32b13500000000, 0x5ac18ccf00000000, 0x1d53c86000000000, + 0x95e3744a00000000, 0xd27130e500000000, 0x8400999100000000, + 0xc392dd3e00000000, 0x4b22611400000000, 0x0cb025bb00000000, + 0x5b43184100000000, 0x1cd15cee00000000, 0x9461e0c400000000, + 0xd3f3a46b00000000, 0x7b81eaeb00000000, 0x3c13ae4400000000, + 0xb4a3126e00000000, 0xf33156c100000000, 0xa4c26b3b00000000, + 0xe3502f9400000000, 0x6be093be00000000, 0x2c72d71100000000, + 0xc702c15700000000, 0x809085f800000000, 0x082039d200000000, + 0x4fb27d7d00000000, 0x1841408700000000, 0x5fd3042800000000, + 0xd763b80200000000, 0x90f1fcad00000000, 0x3883b22d00000000, + 0x7f11f68200000000, 0xf7a14aa800000000, 0xb0330e0700000000, + 0xe7c033fd00000000, 0xa052775200000000, 0x28e2cb7800000000, + 0x6f708fd700000000, 0x390126a300000000, 0x7e93620c00000000, + 0xf623de2600000000, 0xb1b19a8900000000, 0xe642a77300000000, + 0xa1d0e3dc00000000, 0x29605ff600000000, 0x6ef21b5900000000, + 0xc68055d900000000, 0x8112117600000000, 0x09a2ad5c00000000, + 0x4e30e9f300000000, 0x19c3d40900000000, 0x5e5190a600000000, + 0xd6e12c8c00000000, 0x9173682300000000, 0xf406fcca00000000, + 0xb394b86500000000, 0x3b24044f00000000, 0x7cb640e000000000, + 0x2b457d1a00000000, 0x6cd739b500000000, 0xe467859f00000000, + 0xa3f5c13000000000, 0x0b878fb000000000, 0x4c15cb1f00000000, + 0xc4a5773500000000, 0x8337339a00000000, 0xd4c40e6000000000, + 0x93564acf00000000, 0x1be6f6e500000000, 0x5c74b24a00000000, + 0x0a051b3e00000000, 0x4d975f9100000000, 0xc527e3bb00000000, + 0x82b5a71400000000, 0xd5469aee00000000, 0x92d4de4100000000, + 0x1a64626b00000000, 0x5df626c400000000, 0xf584684400000000, + 0xb2162ceb00000000, 0x3aa690c100000000, 0x7d34d46e00000000, + 0x2ac7e99400000000, 0x6d55ad3b00000000, 0xe5e5111100000000, + 0xa27755be00000000, 0x490743f800000000, 0x0e95075700000000, + 0x8625bb7d00000000, 0xc1b7ffd200000000, 0x9644c22800000000, + 0xd1d6868700000000, 0x59663aad00000000, 0x1ef47e0200000000, + 0xb686308200000000, 0xf114742d00000000, 0x79a4c80700000000, + 0x3e368ca800000000, 0x69c5b15200000000, 0x2e57f5fd00000000, + 0xa6e749d700000000, 0xe1750d7800000000, 0xb704a40c00000000, + 0xf096e0a300000000, 0x78265c8900000000, 0x3fb4182600000000, + 0x684725dc00000000, 0x2fd5617300000000, 0xa765dd5900000000, + 0xe0f799f600000000, 0x4885d77600000000, 0x0f1793d900000000, + 0x87a72ff300000000, 0xc0356b5c00000000, 0x97c656a600000000, + 0xd054120900000000, 0x58e4ae2300000000, 0x1f76ea8c00000000, + 0x8e0582af00000000, 0xc997c60000000000, 0x41277a2a00000000, + 0x06b53e8500000000, 0x5146037f00000000, 0x16d447d000000000, + 0x9e64fbfa00000000, 0xd9f6bf5500000000, 0x7184f1d500000000, + 0x3616b57a00000000, 0xbea6095000000000, 0xf9344dff00000000, + 0xaec7700500000000, 0xe95534aa00000000, 0x61e5888000000000, + 0x2677cc2f00000000, 0x7006655b00000000, 0x379421f400000000, + 0xbf249dde00000000, 0xf8b6d97100000000, 0xaf45e48b00000000, + 0xe8d7a02400000000, 0x60671c0e00000000, 0x27f558a100000000, + 0x8f87162100000000, 0xc815528e00000000, 0x40a5eea400000000, + 0x0737aa0b00000000, 0x50c497f100000000, 0x1756d35e00000000, + 0x9fe66f7400000000, 0xd8742bdb00000000, 0x33043d9d00000000, + 0x7496793200000000, 0xfc26c51800000000, 0xbbb481b700000000, + 0xec47bc4d00000000, 0xabd5f8e200000000, 0x236544c800000000, + 0x64f7006700000000, 0xcc854ee700000000, 0x8b170a4800000000, + 0x03a7b66200000000, 0x4435f2cd00000000, 0x13c6cf3700000000, + 0x54548b9800000000, 0xdce437b200000000, 0x9b76731d00000000, + 0xcd07da6900000000, 0x8a959ec600000000, 0x022522ec00000000, + 0x45b7664300000000, 0x12445bb900000000, 0x55d61f1600000000, + 0xdd66a33c00000000, 0x9af4e79300000000, 0x3286a91300000000, + 0x7514edbc00000000, 0xfda4519600000000, 0xba36153900000000, + 0xedc528c300000000, 0xaa576c6c00000000, 0x22e7d04600000000, + 0x657594e900000000}}; + +#else /* W == 4 */ + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0x65673b46, 0xcace768c, 0xafa94dca, 0x4eedeb59, + 0x2b8ad01f, 0x84239dd5, 0xe144a693, 0x9ddbd6b2, 0xf8bcedf4, + 0x5715a03e, 0x32729b78, 0xd3363deb, 0xb65106ad, 0x19f84b67, + 0x7c9f7021, 0xe0c6ab25, 0x85a19063, 0x2a08dda9, 0x4f6fe6ef, + 0xae2b407c, 0xcb4c7b3a, 0x64e536f0, 0x01820db6, 0x7d1d7d97, + 0x187a46d1, 0xb7d30b1b, 0xd2b4305d, 0x33f096ce, 0x5697ad88, + 0xf93ee042, 0x9c59db04, 0x1afc500b, 0x7f9b6b4d, 0xd0322687, + 0xb5551dc1, 0x5411bb52, 0x31768014, 0x9edfcdde, 0xfbb8f698, + 0x872786b9, 0xe240bdff, 0x4de9f035, 0x288ecb73, 0xc9ca6de0, + 0xacad56a6, 0x03041b6c, 0x6663202a, 0xfa3afb2e, 0x9f5dc068, + 0x30f48da2, 0x5593b6e4, 0xb4d71077, 0xd1b02b31, 0x7e1966fb, + 0x1b7e5dbd, 0x67e12d9c, 0x028616da, 0xad2f5b10, 0xc8486056, + 0x290cc6c5, 0x4c6bfd83, 0xe3c2b049, 0x86a58b0f, 0x35f8a016, + 0x509f9b50, 0xff36d69a, 0x9a51eddc, 0x7b154b4f, 0x1e727009, + 0xb1db3dc3, 0xd4bc0685, 0xa82376a4, 0xcd444de2, 0x62ed0028, + 0x078a3b6e, 0xe6ce9dfd, 0x83a9a6bb, 0x2c00eb71, 0x4967d037, + 0xd53e0b33, 0xb0593075, 0x1ff07dbf, 0x7a9746f9, 0x9bd3e06a, + 0xfeb4db2c, 0x511d96e6, 0x347aada0, 0x48e5dd81, 0x2d82e6c7, + 0x822bab0d, 0xe74c904b, 0x060836d8, 0x636f0d9e, 0xccc64054, + 0xa9a17b12, 0x2f04f01d, 0x4a63cb5b, 0xe5ca8691, 0x80adbdd7, + 0x61e91b44, 0x048e2002, 0xab276dc8, 0xce40568e, 0xb2df26af, + 0xd7b81de9, 0x78115023, 0x1d766b65, 0xfc32cdf6, 0x9955f6b0, + 0x36fcbb7a, 0x539b803c, 0xcfc25b38, 0xaaa5607e, 0x050c2db4, + 0x606b16f2, 0x812fb061, 0xe4488b27, 0x4be1c6ed, 0x2e86fdab, + 0x52198d8a, 0x377eb6cc, 0x98d7fb06, 0xfdb0c040, 0x1cf466d3, + 0x79935d95, 0xd63a105f, 0xb35d2b19, 0x6bf1402c, 0x0e967b6a, + 0xa13f36a0, 0xc4580de6, 0x251cab75, 0x407b9033, 0xefd2ddf9, + 0x8ab5e6bf, 0xf62a969e, 0x934dadd8, 0x3ce4e012, 0x5983db54, + 0xb8c77dc7, 0xdda04681, 0x72090b4b, 0x176e300d, 0x8b37eb09, + 0xee50d04f, 0x41f99d85, 0x249ea6c3, 0xc5da0050, 0xa0bd3b16, + 0x0f1476dc, 0x6a734d9a, 0x16ec3dbb, 0x738b06fd, 0xdc224b37, + 0xb9457071, 0x5801d6e2, 0x3d66eda4, 0x92cfa06e, 0xf7a89b28, + 0x710d1027, 0x146a2b61, 0xbbc366ab, 0xdea45ded, 0x3fe0fb7e, + 0x5a87c038, 0xf52e8df2, 0x9049b6b4, 0xecd6c695, 0x89b1fdd3, + 0x2618b019, 0x437f8b5f, 0xa23b2dcc, 0xc75c168a, 0x68f55b40, + 0x0d926006, 0x91cbbb02, 0xf4ac8044, 0x5b05cd8e, 0x3e62f6c8, + 0xdf26505b, 0xba416b1d, 0x15e826d7, 0x708f1d91, 0x0c106db0, + 0x697756f6, 0xc6de1b3c, 0xa3b9207a, 0x42fd86e9, 0x279abdaf, + 0x8833f065, 0xed54cb23, 0x5e09e03a, 0x3b6edb7c, 0x94c796b6, + 0xf1a0adf0, 0x10e40b63, 0x75833025, 0xda2a7def, 0xbf4d46a9, + 0xc3d23688, 0xa6b50dce, 0x091c4004, 0x6c7b7b42, 0x8d3fddd1, + 0xe858e697, 0x47f1ab5d, 0x2296901b, 0xbecf4b1f, 0xdba87059, + 0x74013d93, 0x116606d5, 0xf022a046, 0x95459b00, 0x3aecd6ca, + 0x5f8bed8c, 0x23149dad, 0x4673a6eb, 0xe9daeb21, 0x8cbdd067, + 0x6df976f4, 0x089e4db2, 0xa7370078, 0xc2503b3e, 0x44f5b031, + 0x21928b77, 0x8e3bc6bd, 0xeb5cfdfb, 0x0a185b68, 0x6f7f602e, + 0xc0d62de4, 0xa5b116a2, 0xd92e6683, 0xbc495dc5, 0x13e0100f, + 0x76872b49, 0x97c38dda, 0xf2a4b69c, 0x5d0dfb56, 0x386ac010, + 0xa4331b14, 0xc1542052, 0x6efd6d98, 0x0b9a56de, 0xeadef04d, + 0x8fb9cb0b, 0x201086c1, 0x4577bd87, 0x39e8cda6, 0x5c8ff6e0, + 0xf326bb2a, 0x9641806c, 0x770526ff, 0x12621db9, 0xbdcb5073, + 0xd8ac6b35}, + {0x00000000, 0xd7e28058, 0x74b406f1, 0xa35686a9, 0xe9680de2, + 0x3e8a8dba, 0x9ddc0b13, 0x4a3e8b4b, 0x09a11d85, 0xde439ddd, + 0x7d151b74, 0xaaf79b2c, 0xe0c91067, 0x372b903f, 0x947d1696, + 0x439f96ce, 0x13423b0a, 0xc4a0bb52, 0x67f63dfb, 0xb014bda3, + 0xfa2a36e8, 0x2dc8b6b0, 0x8e9e3019, 0x597cb041, 0x1ae3268f, + 0xcd01a6d7, 0x6e57207e, 0xb9b5a026, 0xf38b2b6d, 0x2469ab35, + 0x873f2d9c, 0x50ddadc4, 0x26847614, 0xf166f64c, 0x523070e5, + 0x85d2f0bd, 0xcfec7bf6, 0x180efbae, 0xbb587d07, 0x6cbafd5f, + 0x2f256b91, 0xf8c7ebc9, 0x5b916d60, 0x8c73ed38, 0xc64d6673, + 0x11afe62b, 0xb2f96082, 0x651be0da, 0x35c64d1e, 0xe224cd46, + 0x41724bef, 0x9690cbb7, 0xdcae40fc, 0x0b4cc0a4, 0xa81a460d, + 0x7ff8c655, 0x3c67509b, 0xeb85d0c3, 0x48d3566a, 0x9f31d632, + 0xd50f5d79, 0x02eddd21, 0xa1bb5b88, 0x7659dbd0, 0x4d08ec28, + 0x9aea6c70, 0x39bcead9, 0xee5e6a81, 0xa460e1ca, 0x73826192, + 0xd0d4e73b, 0x07366763, 0x44a9f1ad, 0x934b71f5, 0x301df75c, + 0xe7ff7704, 0xadc1fc4f, 0x7a237c17, 0xd975fabe, 0x0e977ae6, + 0x5e4ad722, 0x89a8577a, 0x2afed1d3, 0xfd1c518b, 0xb722dac0, + 0x60c05a98, 0xc396dc31, 0x14745c69, 0x57ebcaa7, 0x80094aff, + 0x235fcc56, 0xf4bd4c0e, 0xbe83c745, 0x6961471d, 0xca37c1b4, + 0x1dd541ec, 0x6b8c9a3c, 0xbc6e1a64, 0x1f389ccd, 0xc8da1c95, + 0x82e497de, 0x55061786, 0xf650912f, 0x21b21177, 0x622d87b9, + 0xb5cf07e1, 0x16998148, 0xc17b0110, 0x8b458a5b, 0x5ca70a03, + 0xfff18caa, 0x28130cf2, 0x78cea136, 0xaf2c216e, 0x0c7aa7c7, + 0xdb98279f, 0x91a6acd4, 0x46442c8c, 0xe512aa25, 0x32f02a7d, + 0x716fbcb3, 0xa68d3ceb, 0x05dbba42, 0xd2393a1a, 0x9807b151, + 0x4fe53109, 0xecb3b7a0, 0x3b5137f8, 0x9a11d850, 0x4df35808, + 0xeea5dea1, 0x39475ef9, 0x7379d5b2, 0xa49b55ea, 0x07cdd343, + 0xd02f531b, 0x93b0c5d5, 0x4452458d, 0xe704c324, 0x30e6437c, + 0x7ad8c837, 0xad3a486f, 0x0e6ccec6, 0xd98e4e9e, 0x8953e35a, + 0x5eb16302, 0xfde7e5ab, 0x2a0565f3, 0x603beeb8, 0xb7d96ee0, + 0x148fe849, 0xc36d6811, 0x80f2fedf, 0x57107e87, 0xf446f82e, + 0x23a47876, 0x699af33d, 0xbe787365, 0x1d2ef5cc, 0xcacc7594, + 0xbc95ae44, 0x6b772e1c, 0xc821a8b5, 0x1fc328ed, 0x55fda3a6, + 0x821f23fe, 0x2149a557, 0xf6ab250f, 0xb534b3c1, 0x62d63399, + 0xc180b530, 0x16623568, 0x5c5cbe23, 0x8bbe3e7b, 0x28e8b8d2, + 0xff0a388a, 0xafd7954e, 0x78351516, 0xdb6393bf, 0x0c8113e7, + 0x46bf98ac, 0x915d18f4, 0x320b9e5d, 0xe5e91e05, 0xa67688cb, + 0x71940893, 0xd2c28e3a, 0x05200e62, 0x4f1e8529, 0x98fc0571, + 0x3baa83d8, 0xec480380, 0xd7193478, 0x00fbb420, 0xa3ad3289, + 0x744fb2d1, 0x3e71399a, 0xe993b9c2, 0x4ac53f6b, 0x9d27bf33, + 0xdeb829fd, 0x095aa9a5, 0xaa0c2f0c, 0x7deeaf54, 0x37d0241f, + 0xe032a447, 0x436422ee, 0x9486a2b6, 0xc45b0f72, 0x13b98f2a, + 0xb0ef0983, 0x670d89db, 0x2d330290, 0xfad182c8, 0x59870461, + 0x8e658439, 0xcdfa12f7, 0x1a1892af, 0xb94e1406, 0x6eac945e, + 0x24921f15, 0xf3709f4d, 0x502619e4, 0x87c499bc, 0xf19d426c, + 0x267fc234, 0x8529449d, 0x52cbc4c5, 0x18f54f8e, 0xcf17cfd6, + 0x6c41497f, 0xbba3c927, 0xf83c5fe9, 0x2fdedfb1, 0x8c885918, + 0x5b6ad940, 0x1154520b, 0xc6b6d253, 0x65e054fa, 0xb202d4a2, + 0xe2df7966, 0x353df93e, 0x966b7f97, 0x4189ffcf, 0x0bb77484, + 0xdc55f4dc, 0x7f037275, 0xa8e1f22d, 0xeb7e64e3, 0x3c9ce4bb, + 0x9fca6212, 0x4828e24a, 0x02166901, 0xd5f4e959, 0x76a26ff0, + 0xa140efa8}, + {0x00000000, 0xef52b6e1, 0x05d46b83, 0xea86dd62, 0x0ba8d706, + 0xe4fa61e7, 0x0e7cbc85, 0xe12e0a64, 0x1751ae0c, 0xf80318ed, + 0x1285c58f, 0xfdd7736e, 0x1cf9790a, 0xf3abcfeb, 0x192d1289, + 0xf67fa468, 0x2ea35c18, 0xc1f1eaf9, 0x2b77379b, 0xc425817a, + 0x250b8b1e, 0xca593dff, 0x20dfe09d, 0xcf8d567c, 0x39f2f214, + 0xd6a044f5, 0x3c269997, 0xd3742f76, 0x325a2512, 0xdd0893f3, + 0x378e4e91, 0xd8dcf870, 0x5d46b830, 0xb2140ed1, 0x5892d3b3, + 0xb7c06552, 0x56ee6f36, 0xb9bcd9d7, 0x533a04b5, 0xbc68b254, + 0x4a17163c, 0xa545a0dd, 0x4fc37dbf, 0xa091cb5e, 0x41bfc13a, + 0xaeed77db, 0x446baab9, 0xab391c58, 0x73e5e428, 0x9cb752c9, + 0x76318fab, 0x9963394a, 0x784d332e, 0x971f85cf, 0x7d9958ad, + 0x92cbee4c, 0x64b44a24, 0x8be6fcc5, 0x616021a7, 0x8e329746, + 0x6f1c9d22, 0x804e2bc3, 0x6ac8f6a1, 0x859a4040, 0xba8d7060, + 0x55dfc681, 0xbf591be3, 0x500bad02, 0xb125a766, 0x5e771187, + 0xb4f1cce5, 0x5ba37a04, 0xaddcde6c, 0x428e688d, 0xa808b5ef, + 0x475a030e, 0xa674096a, 0x4926bf8b, 0xa3a062e9, 0x4cf2d408, + 0x942e2c78, 0x7b7c9a99, 0x91fa47fb, 0x7ea8f11a, 0x9f86fb7e, + 0x70d44d9f, 0x9a5290fd, 0x7500261c, 0x837f8274, 0x6c2d3495, + 0x86abe9f7, 0x69f95f16, 0x88d75572, 0x6785e393, 0x8d033ef1, + 0x62518810, 0xe7cbc850, 0x08997eb1, 0xe21fa3d3, 0x0d4d1532, + 0xec631f56, 0x0331a9b7, 0xe9b774d5, 0x06e5c234, 0xf09a665c, + 0x1fc8d0bd, 0xf54e0ddf, 0x1a1cbb3e, 0xfb32b15a, 0x146007bb, + 0xfee6dad9, 0x11b46c38, 0xc9689448, 0x263a22a9, 0xccbcffcb, + 0x23ee492a, 0xc2c0434e, 0x2d92f5af, 0xc71428cd, 0x28469e2c, + 0xde393a44, 0x316b8ca5, 0xdbed51c7, 0x34bfe726, 0xd591ed42, + 0x3ac35ba3, 0xd04586c1, 0x3f173020, 0xae6be681, 0x41395060, + 0xabbf8d02, 0x44ed3be3, 0xa5c33187, 0x4a918766, 0xa0175a04, + 0x4f45ece5, 0xb93a488d, 0x5668fe6c, 0xbcee230e, 0x53bc95ef, + 0xb2929f8b, 0x5dc0296a, 0xb746f408, 0x581442e9, 0x80c8ba99, + 0x6f9a0c78, 0x851cd11a, 0x6a4e67fb, 0x8b606d9f, 0x6432db7e, + 0x8eb4061c, 0x61e6b0fd, 0x97991495, 0x78cba274, 0x924d7f16, + 0x7d1fc9f7, 0x9c31c393, 0x73637572, 0x99e5a810, 0x76b71ef1, + 0xf32d5eb1, 0x1c7fe850, 0xf6f93532, 0x19ab83d3, 0xf88589b7, + 0x17d73f56, 0xfd51e234, 0x120354d5, 0xe47cf0bd, 0x0b2e465c, + 0xe1a89b3e, 0x0efa2ddf, 0xefd427bb, 0x0086915a, 0xea004c38, + 0x0552fad9, 0xdd8e02a9, 0x32dcb448, 0xd85a692a, 0x3708dfcb, + 0xd626d5af, 0x3974634e, 0xd3f2be2c, 0x3ca008cd, 0xcadfaca5, + 0x258d1a44, 0xcf0bc726, 0x205971c7, 0xc1777ba3, 0x2e25cd42, + 0xc4a31020, 0x2bf1a6c1, 0x14e696e1, 0xfbb42000, 0x1132fd62, + 0xfe604b83, 0x1f4e41e7, 0xf01cf706, 0x1a9a2a64, 0xf5c89c85, + 0x03b738ed, 0xece58e0c, 0x0663536e, 0xe931e58f, 0x081fefeb, + 0xe74d590a, 0x0dcb8468, 0xe2993289, 0x3a45caf9, 0xd5177c18, + 0x3f91a17a, 0xd0c3179b, 0x31ed1dff, 0xdebfab1e, 0x3439767c, + 0xdb6bc09d, 0x2d1464f5, 0xc246d214, 0x28c00f76, 0xc792b997, + 0x26bcb3f3, 0xc9ee0512, 0x2368d870, 0xcc3a6e91, 0x49a02ed1, + 0xa6f29830, 0x4c744552, 0xa326f3b3, 0x4208f9d7, 0xad5a4f36, + 0x47dc9254, 0xa88e24b5, 0x5ef180dd, 0xb1a3363c, 0x5b25eb5e, + 0xb4775dbf, 0x555957db, 0xba0be13a, 0x508d3c58, 0xbfdf8ab9, + 0x670372c9, 0x8851c428, 0x62d7194a, 0x8d85afab, 0x6caba5cf, + 0x83f9132e, 0x697fce4c, 0x862d78ad, 0x7052dcc5, 0x9f006a24, + 0x7586b746, 0x9ad401a7, 0x7bfa0bc3, 0x94a8bd22, 0x7e2e6040, + 0x917cd6a1}, + {0x00000000, 0x87a6cb43, 0xd43c90c7, 0x539a5b84, 0x730827cf, + 0xf4aeec8c, 0xa734b708, 0x20927c4b, 0xe6104f9e, 0x61b684dd, + 0x322cdf59, 0xb58a141a, 0x95186851, 0x12bea312, 0x4124f896, + 0xc68233d5, 0x1751997d, 0x90f7523e, 0xc36d09ba, 0x44cbc2f9, + 0x6459beb2, 0xe3ff75f1, 0xb0652e75, 0x37c3e536, 0xf141d6e3, + 0x76e71da0, 0x257d4624, 0xa2db8d67, 0x8249f12c, 0x05ef3a6f, + 0x567561eb, 0xd1d3aaa8, 0x2ea332fa, 0xa905f9b9, 0xfa9fa23d, + 0x7d39697e, 0x5dab1535, 0xda0dde76, 0x899785f2, 0x0e314eb1, + 0xc8b37d64, 0x4f15b627, 0x1c8feda3, 0x9b2926e0, 0xbbbb5aab, + 0x3c1d91e8, 0x6f87ca6c, 0xe821012f, 0x39f2ab87, 0xbe5460c4, + 0xedce3b40, 0x6a68f003, 0x4afa8c48, 0xcd5c470b, 0x9ec61c8f, + 0x1960d7cc, 0xdfe2e419, 0x58442f5a, 0x0bde74de, 0x8c78bf9d, + 0xaceac3d6, 0x2b4c0895, 0x78d65311, 0xff709852, 0x5d4665f4, + 0xdae0aeb7, 0x897af533, 0x0edc3e70, 0x2e4e423b, 0xa9e88978, + 0xfa72d2fc, 0x7dd419bf, 0xbb562a6a, 0x3cf0e129, 0x6f6abaad, + 0xe8cc71ee, 0xc85e0da5, 0x4ff8c6e6, 0x1c629d62, 0x9bc45621, + 0x4a17fc89, 0xcdb137ca, 0x9e2b6c4e, 0x198da70d, 0x391fdb46, + 0xbeb91005, 0xed234b81, 0x6a8580c2, 0xac07b317, 0x2ba17854, + 0x783b23d0, 0xff9de893, 0xdf0f94d8, 0x58a95f9b, 0x0b33041f, + 0x8c95cf5c, 0x73e5570e, 0xf4439c4d, 0xa7d9c7c9, 0x207f0c8a, + 0x00ed70c1, 0x874bbb82, 0xd4d1e006, 0x53772b45, 0x95f51890, + 0x1253d3d3, 0x41c98857, 0xc66f4314, 0xe6fd3f5f, 0x615bf41c, + 0x32c1af98, 0xb56764db, 0x64b4ce73, 0xe3120530, 0xb0885eb4, + 0x372e95f7, 0x17bce9bc, 0x901a22ff, 0xc380797b, 0x4426b238, + 0x82a481ed, 0x05024aae, 0x5698112a, 0xd13eda69, 0xf1aca622, + 0x760a6d61, 0x259036e5, 0xa236fda6, 0xba8ccbe8, 0x3d2a00ab, + 0x6eb05b2f, 0xe916906c, 0xc984ec27, 0x4e222764, 0x1db87ce0, + 0x9a1eb7a3, 0x5c9c8476, 0xdb3a4f35, 0x88a014b1, 0x0f06dff2, + 0x2f94a3b9, 0xa83268fa, 0xfba8337e, 0x7c0ef83d, 0xaddd5295, + 0x2a7b99d6, 0x79e1c252, 0xfe470911, 0xded5755a, 0x5973be19, + 0x0ae9e59d, 0x8d4f2ede, 0x4bcd1d0b, 0xcc6bd648, 0x9ff18dcc, + 0x1857468f, 0x38c53ac4, 0xbf63f187, 0xecf9aa03, 0x6b5f6140, + 0x942ff912, 0x13893251, 0x401369d5, 0xc7b5a296, 0xe727dedd, + 0x6081159e, 0x331b4e1a, 0xb4bd8559, 0x723fb68c, 0xf5997dcf, + 0xa603264b, 0x21a5ed08, 0x01379143, 0x86915a00, 0xd50b0184, + 0x52adcac7, 0x837e606f, 0x04d8ab2c, 0x5742f0a8, 0xd0e43beb, + 0xf07647a0, 0x77d08ce3, 0x244ad767, 0xa3ec1c24, 0x656e2ff1, + 0xe2c8e4b2, 0xb152bf36, 0x36f47475, 0x1666083e, 0x91c0c37d, + 0xc25a98f9, 0x45fc53ba, 0xe7caae1c, 0x606c655f, 0x33f63edb, + 0xb450f598, 0x94c289d3, 0x13644290, 0x40fe1914, 0xc758d257, + 0x01dae182, 0x867c2ac1, 0xd5e67145, 0x5240ba06, 0x72d2c64d, + 0xf5740d0e, 0xa6ee568a, 0x21489dc9, 0xf09b3761, 0x773dfc22, + 0x24a7a7a6, 0xa3016ce5, 0x839310ae, 0x0435dbed, 0x57af8069, + 0xd0094b2a, 0x168b78ff, 0x912db3bc, 0xc2b7e838, 0x4511237b, + 0x65835f30, 0xe2259473, 0xb1bfcff7, 0x361904b4, 0xc9699ce6, + 0x4ecf57a5, 0x1d550c21, 0x9af3c762, 0xba61bb29, 0x3dc7706a, + 0x6e5d2bee, 0xe9fbe0ad, 0x2f79d378, 0xa8df183b, 0xfb4543bf, + 0x7ce388fc, 0x5c71f4b7, 0xdbd73ff4, 0x884d6470, 0x0febaf33, + 0xde38059b, 0x599eced8, 0x0a04955c, 0x8da25e1f, 0xad302254, + 0x2a96e917, 0x790cb293, 0xfeaa79d0, 0x38284a05, 0xbf8e8146, + 0xec14dac2, 0x6bb21181, 0x4b206dca, 0xcc86a689, 0x9f1cfd0d, + 0x18ba364e}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x00000000, 0x43cba687, 0xc7903cd4, 0x845b9a53, 0xcf270873, + 0x8cecaef4, 0x08b734a7, 0x4b7c9220, 0x9e4f10e6, 0xdd84b661, + 0x59df2c32, 0x1a148ab5, 0x51681895, 0x12a3be12, 0x96f82441, + 0xd53382c6, 0x7d995117, 0x3e52f790, 0xba096dc3, 0xf9c2cb44, + 0xb2be5964, 0xf175ffe3, 0x752e65b0, 0x36e5c337, 0xe3d641f1, + 0xa01de776, 0x24467d25, 0x678ddba2, 0x2cf14982, 0x6f3aef05, + 0xeb617556, 0xa8aad3d1, 0xfa32a32e, 0xb9f905a9, 0x3da29ffa, + 0x7e69397d, 0x3515ab5d, 0x76de0dda, 0xf2859789, 0xb14e310e, + 0x647db3c8, 0x27b6154f, 0xa3ed8f1c, 0xe026299b, 0xab5abbbb, + 0xe8911d3c, 0x6cca876f, 0x2f0121e8, 0x87abf239, 0xc46054be, + 0x403bceed, 0x03f0686a, 0x488cfa4a, 0x0b475ccd, 0x8f1cc69e, + 0xccd76019, 0x19e4e2df, 0x5a2f4458, 0xde74de0b, 0x9dbf788c, + 0xd6c3eaac, 0x95084c2b, 0x1153d678, 0x529870ff, 0xf465465d, + 0xb7aee0da, 0x33f57a89, 0x703edc0e, 0x3b424e2e, 0x7889e8a9, + 0xfcd272fa, 0xbf19d47d, 0x6a2a56bb, 0x29e1f03c, 0xadba6a6f, + 0xee71cce8, 0xa50d5ec8, 0xe6c6f84f, 0x629d621c, 0x2156c49b, + 0x89fc174a, 0xca37b1cd, 0x4e6c2b9e, 0x0da78d19, 0x46db1f39, + 0x0510b9be, 0x814b23ed, 0xc280856a, 0x17b307ac, 0x5478a12b, + 0xd0233b78, 0x93e89dff, 0xd8940fdf, 0x9b5fa958, 0x1f04330b, + 0x5ccf958c, 0x0e57e573, 0x4d9c43f4, 0xc9c7d9a7, 0x8a0c7f20, + 0xc170ed00, 0x82bb4b87, 0x06e0d1d4, 0x452b7753, 0x9018f595, + 0xd3d35312, 0x5788c941, 0x14436fc6, 0x5f3ffde6, 0x1cf45b61, + 0x98afc132, 0xdb6467b5, 0x73ceb464, 0x300512e3, 0xb45e88b0, + 0xf7952e37, 0xbce9bc17, 0xff221a90, 0x7b7980c3, 0x38b22644, + 0xed81a482, 0xae4a0205, 0x2a119856, 0x69da3ed1, 0x22a6acf1, + 0x616d0a76, 0xe5369025, 0xa6fd36a2, 0xe8cb8cba, 0xab002a3d, + 0x2f5bb06e, 0x6c9016e9, 0x27ec84c9, 0x6427224e, 0xe07cb81d, + 0xa3b71e9a, 0x76849c5c, 0x354f3adb, 0xb114a088, 0xf2df060f, + 0xb9a3942f, 0xfa6832a8, 0x7e33a8fb, 0x3df80e7c, 0x9552ddad, + 0xd6997b2a, 0x52c2e179, 0x110947fe, 0x5a75d5de, 0x19be7359, + 0x9de5e90a, 0xde2e4f8d, 0x0b1dcd4b, 0x48d66bcc, 0xcc8df19f, + 0x8f465718, 0xc43ac538, 0x87f163bf, 0x03aaf9ec, 0x40615f6b, + 0x12f92f94, 0x51328913, 0xd5691340, 0x96a2b5c7, 0xddde27e7, + 0x9e158160, 0x1a4e1b33, 0x5985bdb4, 0x8cb63f72, 0xcf7d99f5, + 0x4b2603a6, 0x08eda521, 0x43913701, 0x005a9186, 0x84010bd5, + 0xc7caad52, 0x6f607e83, 0x2cabd804, 0xa8f04257, 0xeb3be4d0, + 0xa04776f0, 0xe38cd077, 0x67d74a24, 0x241ceca3, 0xf12f6e65, + 0xb2e4c8e2, 0x36bf52b1, 0x7574f436, 0x3e086616, 0x7dc3c091, + 0xf9985ac2, 0xba53fc45, 0x1caecae7, 0x5f656c60, 0xdb3ef633, + 0x98f550b4, 0xd389c294, 0x90426413, 0x1419fe40, 0x57d258c7, + 0x82e1da01, 0xc12a7c86, 0x4571e6d5, 0x06ba4052, 0x4dc6d272, + 0x0e0d74f5, 0x8a56eea6, 0xc99d4821, 0x61379bf0, 0x22fc3d77, + 0xa6a7a724, 0xe56c01a3, 0xae109383, 0xeddb3504, 0x6980af57, + 0x2a4b09d0, 0xff788b16, 0xbcb32d91, 0x38e8b7c2, 0x7b231145, + 0x305f8365, 0x739425e2, 0xf7cfbfb1, 0xb4041936, 0xe69c69c9, + 0xa557cf4e, 0x210c551d, 0x62c7f39a, 0x29bb61ba, 0x6a70c73d, + 0xee2b5d6e, 0xade0fbe9, 0x78d3792f, 0x3b18dfa8, 0xbf4345fb, + 0xfc88e37c, 0xb7f4715c, 0xf43fd7db, 0x70644d88, 0x33afeb0f, + 0x9b0538de, 0xd8ce9e59, 0x5c95040a, 0x1f5ea28d, 0x542230ad, + 0x17e9962a, 0x93b20c79, 0xd079aafe, 0x054a2838, 0x46818ebf, + 0xc2da14ec, 0x8111b26b, 0xca6d204b, 0x89a686cc, 0x0dfd1c9f, + 0x4e36ba18}, + {0x00000000, 0xe1b652ef, 0x836bd405, 0x62dd86ea, 0x06d7a80b, + 0xe761fae4, 0x85bc7c0e, 0x640a2ee1, 0x0cae5117, 0xed1803f8, + 0x8fc58512, 0x6e73d7fd, 0x0a79f91c, 0xebcfabf3, 0x89122d19, + 0x68a47ff6, 0x185ca32e, 0xf9eaf1c1, 0x9b37772b, 0x7a8125c4, + 0x1e8b0b25, 0xff3d59ca, 0x9de0df20, 0x7c568dcf, 0x14f2f239, + 0xf544a0d6, 0x9799263c, 0x762f74d3, 0x12255a32, 0xf39308dd, + 0x914e8e37, 0x70f8dcd8, 0x30b8465d, 0xd10e14b2, 0xb3d39258, + 0x5265c0b7, 0x366fee56, 0xd7d9bcb9, 0xb5043a53, 0x54b268bc, + 0x3c16174a, 0xdda045a5, 0xbf7dc34f, 0x5ecb91a0, 0x3ac1bf41, + 0xdb77edae, 0xb9aa6b44, 0x581c39ab, 0x28e4e573, 0xc952b79c, + 0xab8f3176, 0x4a396399, 0x2e334d78, 0xcf851f97, 0xad58997d, + 0x4ceecb92, 0x244ab464, 0xc5fce68b, 0xa7216061, 0x4697328e, + 0x229d1c6f, 0xc32b4e80, 0xa1f6c86a, 0x40409a85, 0x60708dba, + 0x81c6df55, 0xe31b59bf, 0x02ad0b50, 0x66a725b1, 0x8711775e, + 0xe5ccf1b4, 0x047aa35b, 0x6cdedcad, 0x8d688e42, 0xefb508a8, + 0x0e035a47, 0x6a0974a6, 0x8bbf2649, 0xe962a0a3, 0x08d4f24c, + 0x782c2e94, 0x999a7c7b, 0xfb47fa91, 0x1af1a87e, 0x7efb869f, + 0x9f4dd470, 0xfd90529a, 0x1c260075, 0x74827f83, 0x95342d6c, + 0xf7e9ab86, 0x165ff969, 0x7255d788, 0x93e38567, 0xf13e038d, + 0x10885162, 0x50c8cbe7, 0xb17e9908, 0xd3a31fe2, 0x32154d0d, + 0x561f63ec, 0xb7a93103, 0xd574b7e9, 0x34c2e506, 0x5c669af0, + 0xbdd0c81f, 0xdf0d4ef5, 0x3ebb1c1a, 0x5ab132fb, 0xbb076014, + 0xd9dae6fe, 0x386cb411, 0x489468c9, 0xa9223a26, 0xcbffbccc, + 0x2a49ee23, 0x4e43c0c2, 0xaff5922d, 0xcd2814c7, 0x2c9e4628, + 0x443a39de, 0xa58c6b31, 0xc751eddb, 0x26e7bf34, 0x42ed91d5, + 0xa35bc33a, 0xc18645d0, 0x2030173f, 0x81e66bae, 0x60503941, + 0x028dbfab, 0xe33bed44, 0x8731c3a5, 0x6687914a, 0x045a17a0, + 0xe5ec454f, 0x8d483ab9, 0x6cfe6856, 0x0e23eebc, 0xef95bc53, + 0x8b9f92b2, 0x6a29c05d, 0x08f446b7, 0xe9421458, 0x99bac880, + 0x780c9a6f, 0x1ad11c85, 0xfb674e6a, 0x9f6d608b, 0x7edb3264, + 0x1c06b48e, 0xfdb0e661, 0x95149997, 0x74a2cb78, 0x167f4d92, + 0xf7c91f7d, 0x93c3319c, 0x72756373, 0x10a8e599, 0xf11eb776, + 0xb15e2df3, 0x50e87f1c, 0x3235f9f6, 0xd383ab19, 0xb78985f8, + 0x563fd717, 0x34e251fd, 0xd5540312, 0xbdf07ce4, 0x5c462e0b, + 0x3e9ba8e1, 0xdf2dfa0e, 0xbb27d4ef, 0x5a918600, 0x384c00ea, + 0xd9fa5205, 0xa9028edd, 0x48b4dc32, 0x2a695ad8, 0xcbdf0837, + 0xafd526d6, 0x4e637439, 0x2cbef2d3, 0xcd08a03c, 0xa5acdfca, + 0x441a8d25, 0x26c70bcf, 0xc7715920, 0xa37b77c1, 0x42cd252e, + 0x2010a3c4, 0xc1a6f12b, 0xe196e614, 0x0020b4fb, 0x62fd3211, + 0x834b60fe, 0xe7414e1f, 0x06f71cf0, 0x642a9a1a, 0x859cc8f5, + 0xed38b703, 0x0c8ee5ec, 0x6e536306, 0x8fe531e9, 0xebef1f08, + 0x0a594de7, 0x6884cb0d, 0x893299e2, 0xf9ca453a, 0x187c17d5, + 0x7aa1913f, 0x9b17c3d0, 0xff1ded31, 0x1eabbfde, 0x7c763934, + 0x9dc06bdb, 0xf564142d, 0x14d246c2, 0x760fc028, 0x97b992c7, + 0xf3b3bc26, 0x1205eec9, 0x70d86823, 0x916e3acc, 0xd12ea049, + 0x3098f2a6, 0x5245744c, 0xb3f326a3, 0xd7f90842, 0x364f5aad, + 0x5492dc47, 0xb5248ea8, 0xdd80f15e, 0x3c36a3b1, 0x5eeb255b, + 0xbf5d77b4, 0xdb575955, 0x3ae10bba, 0x583c8d50, 0xb98adfbf, + 0xc9720367, 0x28c45188, 0x4a19d762, 0xabaf858d, 0xcfa5ab6c, + 0x2e13f983, 0x4cce7f69, 0xad782d86, 0xc5dc5270, 0x246a009f, + 0x46b78675, 0xa701d49a, 0xc30bfa7b, 0x22bda894, 0x40602e7e, + 0xa1d67c91}, + {0x00000000, 0x5880e2d7, 0xf106b474, 0xa98656a3, 0xe20d68e9, + 0xba8d8a3e, 0x130bdc9d, 0x4b8b3e4a, 0x851da109, 0xdd9d43de, + 0x741b157d, 0x2c9bf7aa, 0x6710c9e0, 0x3f902b37, 0x96167d94, + 0xce969f43, 0x0a3b4213, 0x52bba0c4, 0xfb3df667, 0xa3bd14b0, + 0xe8362afa, 0xb0b6c82d, 0x19309e8e, 0x41b07c59, 0x8f26e31a, + 0xd7a601cd, 0x7e20576e, 0x26a0b5b9, 0x6d2b8bf3, 0x35ab6924, + 0x9c2d3f87, 0xc4addd50, 0x14768426, 0x4cf666f1, 0xe5703052, + 0xbdf0d285, 0xf67beccf, 0xaefb0e18, 0x077d58bb, 0x5ffdba6c, + 0x916b252f, 0xc9ebc7f8, 0x606d915b, 0x38ed738c, 0x73664dc6, + 0x2be6af11, 0x8260f9b2, 0xdae01b65, 0x1e4dc635, 0x46cd24e2, + 0xef4b7241, 0xb7cb9096, 0xfc40aedc, 0xa4c04c0b, 0x0d461aa8, + 0x55c6f87f, 0x9b50673c, 0xc3d085eb, 0x6a56d348, 0x32d6319f, + 0x795d0fd5, 0x21dded02, 0x885bbba1, 0xd0db5976, 0x28ec084d, + 0x706cea9a, 0xd9eabc39, 0x816a5eee, 0xcae160a4, 0x92618273, + 0x3be7d4d0, 0x63673607, 0xadf1a944, 0xf5714b93, 0x5cf71d30, + 0x0477ffe7, 0x4ffcc1ad, 0x177c237a, 0xbefa75d9, 0xe67a970e, + 0x22d74a5e, 0x7a57a889, 0xd3d1fe2a, 0x8b511cfd, 0xc0da22b7, + 0x985ac060, 0x31dc96c3, 0x695c7414, 0xa7caeb57, 0xff4a0980, + 0x56cc5f23, 0x0e4cbdf4, 0x45c783be, 0x1d476169, 0xb4c137ca, + 0xec41d51d, 0x3c9a8c6b, 0x641a6ebc, 0xcd9c381f, 0x951cdac8, + 0xde97e482, 0x86170655, 0x2f9150f6, 0x7711b221, 0xb9872d62, + 0xe107cfb5, 0x48819916, 0x10017bc1, 0x5b8a458b, 0x030aa75c, + 0xaa8cf1ff, 0xf20c1328, 0x36a1ce78, 0x6e212caf, 0xc7a77a0c, + 0x9f2798db, 0xd4aca691, 0x8c2c4446, 0x25aa12e5, 0x7d2af032, + 0xb3bc6f71, 0xeb3c8da6, 0x42badb05, 0x1a3a39d2, 0x51b10798, + 0x0931e54f, 0xa0b7b3ec, 0xf837513b, 0x50d8119a, 0x0858f34d, + 0xa1dea5ee, 0xf95e4739, 0xb2d57973, 0xea559ba4, 0x43d3cd07, + 0x1b532fd0, 0xd5c5b093, 0x8d455244, 0x24c304e7, 0x7c43e630, + 0x37c8d87a, 0x6f483aad, 0xc6ce6c0e, 0x9e4e8ed9, 0x5ae35389, + 0x0263b15e, 0xabe5e7fd, 0xf365052a, 0xb8ee3b60, 0xe06ed9b7, + 0x49e88f14, 0x11686dc3, 0xdffef280, 0x877e1057, 0x2ef846f4, + 0x7678a423, 0x3df39a69, 0x657378be, 0xccf52e1d, 0x9475ccca, + 0x44ae95bc, 0x1c2e776b, 0xb5a821c8, 0xed28c31f, 0xa6a3fd55, + 0xfe231f82, 0x57a54921, 0x0f25abf6, 0xc1b334b5, 0x9933d662, + 0x30b580c1, 0x68356216, 0x23be5c5c, 0x7b3ebe8b, 0xd2b8e828, + 0x8a380aff, 0x4e95d7af, 0x16153578, 0xbf9363db, 0xe713810c, + 0xac98bf46, 0xf4185d91, 0x5d9e0b32, 0x051ee9e5, 0xcb8876a6, + 0x93089471, 0x3a8ec2d2, 0x620e2005, 0x29851e4f, 0x7105fc98, + 0xd883aa3b, 0x800348ec, 0x783419d7, 0x20b4fb00, 0x8932ada3, + 0xd1b24f74, 0x9a39713e, 0xc2b993e9, 0x6b3fc54a, 0x33bf279d, + 0xfd29b8de, 0xa5a95a09, 0x0c2f0caa, 0x54afee7d, 0x1f24d037, + 0x47a432e0, 0xee226443, 0xb6a28694, 0x720f5bc4, 0x2a8fb913, + 0x8309efb0, 0xdb890d67, 0x9002332d, 0xc882d1fa, 0x61048759, + 0x3984658e, 0xf712facd, 0xaf92181a, 0x06144eb9, 0x5e94ac6e, + 0x151f9224, 0x4d9f70f3, 0xe4192650, 0xbc99c487, 0x6c429df1, + 0x34c27f26, 0x9d442985, 0xc5c4cb52, 0x8e4ff518, 0xd6cf17cf, + 0x7f49416c, 0x27c9a3bb, 0xe95f3cf8, 0xb1dfde2f, 0x1859888c, + 0x40d96a5b, 0x0b525411, 0x53d2b6c6, 0xfa54e065, 0xa2d402b2, + 0x6679dfe2, 0x3ef93d35, 0x977f6b96, 0xcfff8941, 0x8474b70b, + 0xdcf455dc, 0x7572037f, 0x2df2e1a8, 0xe3647eeb, 0xbbe49c3c, + 0x1262ca9f, 0x4ae22848, 0x01691602, 0x59e9f4d5, 0xf06fa276, + 0xa8ef40a1}, + {0x00000000, 0x463b6765, 0x8c76ceca, 0xca4da9af, 0x59ebed4e, + 0x1fd08a2b, 0xd59d2384, 0x93a644e1, 0xb2d6db9d, 0xf4edbcf8, + 0x3ea01557, 0x789b7232, 0xeb3d36d3, 0xad0651b6, 0x674bf819, + 0x21709f7c, 0x25abc6e0, 0x6390a185, 0xa9dd082a, 0xefe66f4f, + 0x7c402bae, 0x3a7b4ccb, 0xf036e564, 0xb60d8201, 0x977d1d7d, + 0xd1467a18, 0x1b0bd3b7, 0x5d30b4d2, 0xce96f033, 0x88ad9756, + 0x42e03ef9, 0x04db599c, 0x0b50fc1a, 0x4d6b9b7f, 0x872632d0, + 0xc11d55b5, 0x52bb1154, 0x14807631, 0xdecddf9e, 0x98f6b8fb, + 0xb9862787, 0xffbd40e2, 0x35f0e94d, 0x73cb8e28, 0xe06dcac9, + 0xa656adac, 0x6c1b0403, 0x2a206366, 0x2efb3afa, 0x68c05d9f, + 0xa28df430, 0xe4b69355, 0x7710d7b4, 0x312bb0d1, 0xfb66197e, + 0xbd5d7e1b, 0x9c2de167, 0xda168602, 0x105b2fad, 0x566048c8, + 0xc5c60c29, 0x83fd6b4c, 0x49b0c2e3, 0x0f8ba586, 0x16a0f835, + 0x509b9f50, 0x9ad636ff, 0xdced519a, 0x4f4b157b, 0x0970721e, + 0xc33ddbb1, 0x8506bcd4, 0xa47623a8, 0xe24d44cd, 0x2800ed62, + 0x6e3b8a07, 0xfd9dcee6, 0xbba6a983, 0x71eb002c, 0x37d06749, + 0x330b3ed5, 0x753059b0, 0xbf7df01f, 0xf946977a, 0x6ae0d39b, + 0x2cdbb4fe, 0xe6961d51, 0xa0ad7a34, 0x81dde548, 0xc7e6822d, + 0x0dab2b82, 0x4b904ce7, 0xd8360806, 0x9e0d6f63, 0x5440c6cc, + 0x127ba1a9, 0x1df0042f, 0x5bcb634a, 0x9186cae5, 0xd7bdad80, + 0x441be961, 0x02208e04, 0xc86d27ab, 0x8e5640ce, 0xaf26dfb2, + 0xe91db8d7, 0x23501178, 0x656b761d, 0xf6cd32fc, 0xb0f65599, + 0x7abbfc36, 0x3c809b53, 0x385bc2cf, 0x7e60a5aa, 0xb42d0c05, + 0xf2166b60, 0x61b02f81, 0x278b48e4, 0xedc6e14b, 0xabfd862e, + 0x8a8d1952, 0xccb67e37, 0x06fbd798, 0x40c0b0fd, 0xd366f41c, + 0x955d9379, 0x5f103ad6, 0x192b5db3, 0x2c40f16b, 0x6a7b960e, + 0xa0363fa1, 0xe60d58c4, 0x75ab1c25, 0x33907b40, 0xf9ddd2ef, + 0xbfe6b58a, 0x9e962af6, 0xd8ad4d93, 0x12e0e43c, 0x54db8359, + 0xc77dc7b8, 0x8146a0dd, 0x4b0b0972, 0x0d306e17, 0x09eb378b, + 0x4fd050ee, 0x859df941, 0xc3a69e24, 0x5000dac5, 0x163bbda0, + 0xdc76140f, 0x9a4d736a, 0xbb3dec16, 0xfd068b73, 0x374b22dc, + 0x717045b9, 0xe2d60158, 0xa4ed663d, 0x6ea0cf92, 0x289ba8f7, + 0x27100d71, 0x612b6a14, 0xab66c3bb, 0xed5da4de, 0x7efbe03f, + 0x38c0875a, 0xf28d2ef5, 0xb4b64990, 0x95c6d6ec, 0xd3fdb189, + 0x19b01826, 0x5f8b7f43, 0xcc2d3ba2, 0x8a165cc7, 0x405bf568, + 0x0660920d, 0x02bbcb91, 0x4480acf4, 0x8ecd055b, 0xc8f6623e, + 0x5b5026df, 0x1d6b41ba, 0xd726e815, 0x911d8f70, 0xb06d100c, + 0xf6567769, 0x3c1bdec6, 0x7a20b9a3, 0xe986fd42, 0xafbd9a27, + 0x65f03388, 0x23cb54ed, 0x3ae0095e, 0x7cdb6e3b, 0xb696c794, + 0xf0ada0f1, 0x630be410, 0x25308375, 0xef7d2ada, 0xa9464dbf, + 0x8836d2c3, 0xce0db5a6, 0x04401c09, 0x427b7b6c, 0xd1dd3f8d, + 0x97e658e8, 0x5dabf147, 0x1b909622, 0x1f4bcfbe, 0x5970a8db, + 0x933d0174, 0xd5066611, 0x46a022f0, 0x009b4595, 0xcad6ec3a, + 0x8ced8b5f, 0xad9d1423, 0xeba67346, 0x21ebdae9, 0x67d0bd8c, + 0xf476f96d, 0xb24d9e08, 0x780037a7, 0x3e3b50c2, 0x31b0f544, + 0x778b9221, 0xbdc63b8e, 0xfbfd5ceb, 0x685b180a, 0x2e607f6f, + 0xe42dd6c0, 0xa216b1a5, 0x83662ed9, 0xc55d49bc, 0x0f10e013, + 0x492b8776, 0xda8dc397, 0x9cb6a4f2, 0x56fb0d5d, 0x10c06a38, + 0x141b33a4, 0x522054c1, 0x986dfd6e, 0xde569a0b, 0x4df0deea, + 0x0bcbb98f, 0xc1861020, 0x87bd7745, 0xa6cde839, 0xe0f68f5c, + 0x2abb26f3, 0x6c804196, 0xff260577, 0xb91d6212, 0x7350cbbd, + 0x356bacd8}}; + +#endif + +#endif + +#if N == 6 + +#if W == 8 + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0x3db1ecdc, 0x7b63d9b8, 0x46d23564, 0xf6c7b370, + 0xcb765fac, 0x8da46ac8, 0xb0158614, 0x36fe60a1, 0x0b4f8c7d, + 0x4d9db919, 0x702c55c5, 0xc039d3d1, 0xfd883f0d, 0xbb5a0a69, + 0x86ebe6b5, 0x6dfcc142, 0x504d2d9e, 0x169f18fa, 0x2b2ef426, + 0x9b3b7232, 0xa68a9eee, 0xe058ab8a, 0xdde94756, 0x5b02a1e3, + 0x66b34d3f, 0x2061785b, 0x1dd09487, 0xadc51293, 0x9074fe4f, + 0xd6a6cb2b, 0xeb1727f7, 0xdbf98284, 0xe6486e58, 0xa09a5b3c, + 0x9d2bb7e0, 0x2d3e31f4, 0x108fdd28, 0x565de84c, 0x6bec0490, + 0xed07e225, 0xd0b60ef9, 0x96643b9d, 0xabd5d741, 0x1bc05155, + 0x2671bd89, 0x60a388ed, 0x5d126431, 0xb60543c6, 0x8bb4af1a, + 0xcd669a7e, 0xf0d776a2, 0x40c2f0b6, 0x7d731c6a, 0x3ba1290e, + 0x0610c5d2, 0x80fb2367, 0xbd4acfbb, 0xfb98fadf, 0xc6291603, + 0x763c9017, 0x4b8d7ccb, 0x0d5f49af, 0x30eea573, 0x6c820349, + 0x5133ef95, 0x17e1daf1, 0x2a50362d, 0x9a45b039, 0xa7f45ce5, + 0xe1266981, 0xdc97855d, 0x5a7c63e8, 0x67cd8f34, 0x211fba50, + 0x1cae568c, 0xacbbd098, 0x910a3c44, 0xd7d80920, 0xea69e5fc, + 0x017ec20b, 0x3ccf2ed7, 0x7a1d1bb3, 0x47acf76f, 0xf7b9717b, + 0xca089da7, 0x8cdaa8c3, 0xb16b441f, 0x3780a2aa, 0x0a314e76, + 0x4ce37b12, 0x715297ce, 0xc14711da, 0xfcf6fd06, 0xba24c862, + 0x879524be, 0xb77b81cd, 0x8aca6d11, 0xcc185875, 0xf1a9b4a9, + 0x41bc32bd, 0x7c0dde61, 0x3adfeb05, 0x076e07d9, 0x8185e16c, + 0xbc340db0, 0xfae638d4, 0xc757d408, 0x7742521c, 0x4af3bec0, + 0x0c218ba4, 0x31906778, 0xda87408f, 0xe736ac53, 0xa1e49937, + 0x9c5575eb, 0x2c40f3ff, 0x11f11f23, 0x57232a47, 0x6a92c69b, + 0xec79202e, 0xd1c8ccf2, 0x971af996, 0xaaab154a, 0x1abe935e, + 0x270f7f82, 0x61dd4ae6, 0x5c6ca63a, 0xd9040692, 0xe4b5ea4e, + 0xa267df2a, 0x9fd633f6, 0x2fc3b5e2, 0x1272593e, 0x54a06c5a, + 0x69118086, 0xeffa6633, 0xd24b8aef, 0x9499bf8b, 0xa9285357, + 0x193dd543, 0x248c399f, 0x625e0cfb, 0x5fefe027, 0xb4f8c7d0, + 0x89492b0c, 0xcf9b1e68, 0xf22af2b4, 0x423f74a0, 0x7f8e987c, + 0x395cad18, 0x04ed41c4, 0x8206a771, 0xbfb74bad, 0xf9657ec9, + 0xc4d49215, 0x74c11401, 0x4970f8dd, 0x0fa2cdb9, 0x32132165, + 0x02fd8416, 0x3f4c68ca, 0x799e5dae, 0x442fb172, 0xf43a3766, + 0xc98bdbba, 0x8f59eede, 0xb2e80202, 0x3403e4b7, 0x09b2086b, + 0x4f603d0f, 0x72d1d1d3, 0xc2c457c7, 0xff75bb1b, 0xb9a78e7f, + 0x841662a3, 0x6f014554, 0x52b0a988, 0x14629cec, 0x29d37030, + 0x99c6f624, 0xa4771af8, 0xe2a52f9c, 0xdf14c340, 0x59ff25f5, + 0x644ec929, 0x229cfc4d, 0x1f2d1091, 0xaf389685, 0x92897a59, + 0xd45b4f3d, 0xe9eaa3e1, 0xb58605db, 0x8837e907, 0xcee5dc63, + 0xf35430bf, 0x4341b6ab, 0x7ef05a77, 0x38226f13, 0x059383cf, + 0x8378657a, 0xbec989a6, 0xf81bbcc2, 0xc5aa501e, 0x75bfd60a, + 0x480e3ad6, 0x0edc0fb2, 0x336de36e, 0xd87ac499, 0xe5cb2845, + 0xa3191d21, 0x9ea8f1fd, 0x2ebd77e9, 0x130c9b35, 0x55deae51, + 0x686f428d, 0xee84a438, 0xd33548e4, 0x95e77d80, 0xa856915c, + 0x18431748, 0x25f2fb94, 0x6320cef0, 0x5e91222c, 0x6e7f875f, + 0x53ce6b83, 0x151c5ee7, 0x28adb23b, 0x98b8342f, 0xa509d8f3, + 0xe3dbed97, 0xde6a014b, 0x5881e7fe, 0x65300b22, 0x23e23e46, + 0x1e53d29a, 0xae46548e, 0x93f7b852, 0xd5258d36, 0xe89461ea, + 0x0383461d, 0x3e32aac1, 0x78e09fa5, 0x45517379, 0xf544f56d, + 0xc8f519b1, 0x8e272cd5, 0xb396c009, 0x357d26bc, 0x08ccca60, + 0x4e1eff04, 0x73af13d8, 0xc3ba95cc, 0xfe0b7910, 0xb8d94c74, + 0x8568a0a8}, + {0x00000000, 0x69790b65, 0xd2f216ca, 0xbb8b1daf, 0x7e952bd5, + 0x17ec20b0, 0xac673d1f, 0xc51e367a, 0xfd2a57aa, 0x94535ccf, + 0x2fd84160, 0x46a14a05, 0x83bf7c7f, 0xeac6771a, 0x514d6ab5, + 0x383461d0, 0x2125a915, 0x485ca270, 0xf3d7bfdf, 0x9aaeb4ba, + 0x5fb082c0, 0x36c989a5, 0x8d42940a, 0xe43b9f6f, 0xdc0ffebf, + 0xb576f5da, 0x0efde875, 0x6784e310, 0xa29ad56a, 0xcbe3de0f, + 0x7068c3a0, 0x1911c8c5, 0x424b522a, 0x2b32594f, 0x90b944e0, + 0xf9c04f85, 0x3cde79ff, 0x55a7729a, 0xee2c6f35, 0x87556450, + 0xbf610580, 0xd6180ee5, 0x6d93134a, 0x04ea182f, 0xc1f42e55, + 0xa88d2530, 0x1306389f, 0x7a7f33fa, 0x636efb3f, 0x0a17f05a, + 0xb19cedf5, 0xd8e5e690, 0x1dfbd0ea, 0x7482db8f, 0xcf09c620, + 0xa670cd45, 0x9e44ac95, 0xf73da7f0, 0x4cb6ba5f, 0x25cfb13a, + 0xe0d18740, 0x89a88c25, 0x3223918a, 0x5b5a9aef, 0x8496a454, + 0xedefaf31, 0x5664b29e, 0x3f1db9fb, 0xfa038f81, 0x937a84e4, + 0x28f1994b, 0x4188922e, 0x79bcf3fe, 0x10c5f89b, 0xab4ee534, + 0xc237ee51, 0x0729d82b, 0x6e50d34e, 0xd5dbcee1, 0xbca2c584, + 0xa5b30d41, 0xccca0624, 0x77411b8b, 0x1e3810ee, 0xdb262694, + 0xb25f2df1, 0x09d4305e, 0x60ad3b3b, 0x58995aeb, 0x31e0518e, + 0x8a6b4c21, 0xe3124744, 0x260c713e, 0x4f757a5b, 0xf4fe67f4, + 0x9d876c91, 0xc6ddf67e, 0xafa4fd1b, 0x142fe0b4, 0x7d56ebd1, + 0xb848ddab, 0xd131d6ce, 0x6abacb61, 0x03c3c004, 0x3bf7a1d4, + 0x528eaab1, 0xe905b71e, 0x807cbc7b, 0x45628a01, 0x2c1b8164, + 0x97909ccb, 0xfee997ae, 0xe7f85f6b, 0x8e81540e, 0x350a49a1, + 0x5c7342c4, 0x996d74be, 0xf0147fdb, 0x4b9f6274, 0x22e66911, + 0x1ad208c1, 0x73ab03a4, 0xc8201e0b, 0xa159156e, 0x64472314, + 0x0d3e2871, 0xb6b535de, 0xdfcc3ebb, 0xd25c4ee9, 0xbb25458c, + 0x00ae5823, 0x69d75346, 0xacc9653c, 0xc5b06e59, 0x7e3b73f6, + 0x17427893, 0x2f761943, 0x460f1226, 0xfd840f89, 0x94fd04ec, + 0x51e33296, 0x389a39f3, 0x8311245c, 0xea682f39, 0xf379e7fc, + 0x9a00ec99, 0x218bf136, 0x48f2fa53, 0x8deccc29, 0xe495c74c, + 0x5f1edae3, 0x3667d186, 0x0e53b056, 0x672abb33, 0xdca1a69c, + 0xb5d8adf9, 0x70c69b83, 0x19bf90e6, 0xa2348d49, 0xcb4d862c, + 0x90171cc3, 0xf96e17a6, 0x42e50a09, 0x2b9c016c, 0xee823716, + 0x87fb3c73, 0x3c7021dc, 0x55092ab9, 0x6d3d4b69, 0x0444400c, + 0xbfcf5da3, 0xd6b656c6, 0x13a860bc, 0x7ad16bd9, 0xc15a7676, + 0xa8237d13, 0xb132b5d6, 0xd84bbeb3, 0x63c0a31c, 0x0ab9a879, + 0xcfa79e03, 0xa6de9566, 0x1d5588c9, 0x742c83ac, 0x4c18e27c, + 0x2561e919, 0x9eeaf4b6, 0xf793ffd3, 0x328dc9a9, 0x5bf4c2cc, + 0xe07fdf63, 0x8906d406, 0x56caeabd, 0x3fb3e1d8, 0x8438fc77, + 0xed41f712, 0x285fc168, 0x4126ca0d, 0xfaadd7a2, 0x93d4dcc7, + 0xabe0bd17, 0xc299b672, 0x7912abdd, 0x106ba0b8, 0xd57596c2, + 0xbc0c9da7, 0x07878008, 0x6efe8b6d, 0x77ef43a8, 0x1e9648cd, + 0xa51d5562, 0xcc645e07, 0x097a687d, 0x60036318, 0xdb887eb7, + 0xb2f175d2, 0x8ac51402, 0xe3bc1f67, 0x583702c8, 0x314e09ad, + 0xf4503fd7, 0x9d2934b2, 0x26a2291d, 0x4fdb2278, 0x1481b897, + 0x7df8b3f2, 0xc673ae5d, 0xaf0aa538, 0x6a149342, 0x036d9827, + 0xb8e68588, 0xd19f8eed, 0xe9abef3d, 0x80d2e458, 0x3b59f9f7, + 0x5220f292, 0x973ec4e8, 0xfe47cf8d, 0x45ccd222, 0x2cb5d947, + 0x35a41182, 0x5cdd1ae7, 0xe7560748, 0x8e2f0c2d, 0x4b313a57, + 0x22483132, 0x99c32c9d, 0xf0ba27f8, 0xc88e4628, 0xa1f74d4d, + 0x1a7c50e2, 0x73055b87, 0xb61b6dfd, 0xdf626698, 0x64e97b37, + 0x0d907052}, + {0x00000000, 0x7fc99b93, 0xff933726, 0x805aacb5, 0x2457680d, + 0x5b9ef39e, 0xdbc45f2b, 0xa40dc4b8, 0x48aed01a, 0x37674b89, + 0xb73de73c, 0xc8f47caf, 0x6cf9b817, 0x13302384, 0x936a8f31, + 0xeca314a2, 0x915da034, 0xee943ba7, 0x6ece9712, 0x11070c81, + 0xb50ac839, 0xcac353aa, 0x4a99ff1f, 0x3550648c, 0xd9f3702e, + 0xa63aebbd, 0x26604708, 0x59a9dc9b, 0xfda41823, 0x826d83b0, + 0x02372f05, 0x7dfeb496, 0xf9ca4629, 0x8603ddba, 0x0659710f, + 0x7990ea9c, 0xdd9d2e24, 0xa254b5b7, 0x220e1902, 0x5dc78291, + 0xb1649633, 0xcead0da0, 0x4ef7a115, 0x313e3a86, 0x9533fe3e, + 0xeafa65ad, 0x6aa0c918, 0x1569528b, 0x6897e61d, 0x175e7d8e, + 0x9704d13b, 0xe8cd4aa8, 0x4cc08e10, 0x33091583, 0xb353b936, + 0xcc9a22a5, 0x20393607, 0x5ff0ad94, 0xdfaa0121, 0xa0639ab2, + 0x046e5e0a, 0x7ba7c599, 0xfbfd692c, 0x8434f2bf, 0x28e58a13, + 0x572c1180, 0xd776bd35, 0xa8bf26a6, 0x0cb2e21e, 0x737b798d, + 0xf321d538, 0x8ce84eab, 0x604b5a09, 0x1f82c19a, 0x9fd86d2f, + 0xe011f6bc, 0x441c3204, 0x3bd5a997, 0xbb8f0522, 0xc4469eb1, + 0xb9b82a27, 0xc671b1b4, 0x462b1d01, 0x39e28692, 0x9def422a, + 0xe226d9b9, 0x627c750c, 0x1db5ee9f, 0xf116fa3d, 0x8edf61ae, + 0x0e85cd1b, 0x714c5688, 0xd5419230, 0xaa8809a3, 0x2ad2a516, + 0x551b3e85, 0xd12fcc3a, 0xaee657a9, 0x2ebcfb1c, 0x5175608f, + 0xf578a437, 0x8ab13fa4, 0x0aeb9311, 0x75220882, 0x99811c20, + 0xe64887b3, 0x66122b06, 0x19dbb095, 0xbdd6742d, 0xc21fefbe, + 0x4245430b, 0x3d8cd898, 0x40726c0e, 0x3fbbf79d, 0xbfe15b28, + 0xc028c0bb, 0x64250403, 0x1bec9f90, 0x9bb63325, 0xe47fa8b6, + 0x08dcbc14, 0x77152787, 0xf74f8b32, 0x888610a1, 0x2c8bd419, + 0x53424f8a, 0xd318e33f, 0xacd178ac, 0x51cb1426, 0x2e028fb5, + 0xae582300, 0xd191b893, 0x759c7c2b, 0x0a55e7b8, 0x8a0f4b0d, + 0xf5c6d09e, 0x1965c43c, 0x66ac5faf, 0xe6f6f31a, 0x993f6889, + 0x3d32ac31, 0x42fb37a2, 0xc2a19b17, 0xbd680084, 0xc096b412, + 0xbf5f2f81, 0x3f058334, 0x40cc18a7, 0xe4c1dc1f, 0x9b08478c, + 0x1b52eb39, 0x649b70aa, 0x88386408, 0xf7f1ff9b, 0x77ab532e, + 0x0862c8bd, 0xac6f0c05, 0xd3a69796, 0x53fc3b23, 0x2c35a0b0, + 0xa801520f, 0xd7c8c99c, 0x57926529, 0x285bfeba, 0x8c563a02, + 0xf39fa191, 0x73c50d24, 0x0c0c96b7, 0xe0af8215, 0x9f661986, + 0x1f3cb533, 0x60f52ea0, 0xc4f8ea18, 0xbb31718b, 0x3b6bdd3e, + 0x44a246ad, 0x395cf23b, 0x469569a8, 0xc6cfc51d, 0xb9065e8e, + 0x1d0b9a36, 0x62c201a5, 0xe298ad10, 0x9d513683, 0x71f22221, + 0x0e3bb9b2, 0x8e611507, 0xf1a88e94, 0x55a54a2c, 0x2a6cd1bf, + 0xaa367d0a, 0xd5ffe699, 0x792e9e35, 0x06e705a6, 0x86bda913, + 0xf9743280, 0x5d79f638, 0x22b06dab, 0xa2eac11e, 0xdd235a8d, + 0x31804e2f, 0x4e49d5bc, 0xce137909, 0xb1dae29a, 0x15d72622, + 0x6a1ebdb1, 0xea441104, 0x958d8a97, 0xe8733e01, 0x97baa592, + 0x17e00927, 0x682992b4, 0xcc24560c, 0xb3edcd9f, 0x33b7612a, + 0x4c7efab9, 0xa0ddee1b, 0xdf147588, 0x5f4ed93d, 0x208742ae, + 0x848a8616, 0xfb431d85, 0x7b19b130, 0x04d02aa3, 0x80e4d81c, + 0xff2d438f, 0x7f77ef3a, 0x00be74a9, 0xa4b3b011, 0xdb7a2b82, + 0x5b208737, 0x24e91ca4, 0xc84a0806, 0xb7839395, 0x37d93f20, + 0x4810a4b3, 0xec1d600b, 0x93d4fb98, 0x138e572d, 0x6c47ccbe, + 0x11b97828, 0x6e70e3bb, 0xee2a4f0e, 0x91e3d49d, 0x35ee1025, + 0x4a278bb6, 0xca7d2703, 0xb5b4bc90, 0x5917a832, 0x26de33a1, + 0xa6849f14, 0xd94d0487, 0x7d40c03f, 0x02895bac, 0x82d3f719, + 0xfd1a6c8a}, + {0x00000000, 0xa396284c, 0x9c5d56d9, 0x3fcb7e95, 0xe3cbabf3, + 0x405d83bf, 0x7f96fd2a, 0xdc00d566, 0x1ce651a7, 0xbf7079eb, + 0x80bb077e, 0x232d2f32, 0xff2dfa54, 0x5cbbd218, 0x6370ac8d, + 0xc0e684c1, 0x39cca34e, 0x9a5a8b02, 0xa591f597, 0x0607dddb, + 0xda0708bd, 0x799120f1, 0x465a5e64, 0xe5cc7628, 0x252af2e9, + 0x86bcdaa5, 0xb977a430, 0x1ae18c7c, 0xc6e1591a, 0x65777156, + 0x5abc0fc3, 0xf92a278f, 0x7399469c, 0xd00f6ed0, 0xefc41045, + 0x4c523809, 0x9052ed6f, 0x33c4c523, 0x0c0fbbb6, 0xaf9993fa, + 0x6f7f173b, 0xcce93f77, 0xf32241e2, 0x50b469ae, 0x8cb4bcc8, + 0x2f229484, 0x10e9ea11, 0xb37fc25d, 0x4a55e5d2, 0xe9c3cd9e, + 0xd608b30b, 0x759e9b47, 0xa99e4e21, 0x0a08666d, 0x35c318f8, + 0x965530b4, 0x56b3b475, 0xf5259c39, 0xcaeee2ac, 0x6978cae0, + 0xb5781f86, 0x16ee37ca, 0x2925495f, 0x8ab36113, 0xe7328d38, + 0x44a4a574, 0x7b6fdbe1, 0xd8f9f3ad, 0x04f926cb, 0xa76f0e87, + 0x98a47012, 0x3b32585e, 0xfbd4dc9f, 0x5842f4d3, 0x67898a46, + 0xc41fa20a, 0x181f776c, 0xbb895f20, 0x844221b5, 0x27d409f9, + 0xdefe2e76, 0x7d68063a, 0x42a378af, 0xe13550e3, 0x3d358585, + 0x9ea3adc9, 0xa168d35c, 0x02fefb10, 0xc2187fd1, 0x618e579d, + 0x5e452908, 0xfdd30144, 0x21d3d422, 0x8245fc6e, 0xbd8e82fb, + 0x1e18aab7, 0x94abcba4, 0x373de3e8, 0x08f69d7d, 0xab60b531, + 0x77606057, 0xd4f6481b, 0xeb3d368e, 0x48ab1ec2, 0x884d9a03, + 0x2bdbb24f, 0x1410ccda, 0xb786e496, 0x6b8631f0, 0xc81019bc, + 0xf7db6729, 0x544d4f65, 0xad6768ea, 0x0ef140a6, 0x313a3e33, + 0x92ac167f, 0x4eacc319, 0xed3aeb55, 0xd2f195c0, 0x7167bd8c, + 0xb181394d, 0x12171101, 0x2ddc6f94, 0x8e4a47d8, 0x524a92be, + 0xf1dcbaf2, 0xce17c467, 0x6d81ec2b, 0x15141c31, 0xb682347d, + 0x89494ae8, 0x2adf62a4, 0xf6dfb7c2, 0x55499f8e, 0x6a82e11b, + 0xc914c957, 0x09f24d96, 0xaa6465da, 0x95af1b4f, 0x36393303, + 0xea39e665, 0x49afce29, 0x7664b0bc, 0xd5f298f0, 0x2cd8bf7f, + 0x8f4e9733, 0xb085e9a6, 0x1313c1ea, 0xcf13148c, 0x6c853cc0, + 0x534e4255, 0xf0d86a19, 0x303eeed8, 0x93a8c694, 0xac63b801, + 0x0ff5904d, 0xd3f5452b, 0x70636d67, 0x4fa813f2, 0xec3e3bbe, + 0x668d5aad, 0xc51b72e1, 0xfad00c74, 0x59462438, 0x8546f15e, + 0x26d0d912, 0x191ba787, 0xba8d8fcb, 0x7a6b0b0a, 0xd9fd2346, + 0xe6365dd3, 0x45a0759f, 0x99a0a0f9, 0x3a3688b5, 0x05fdf620, + 0xa66bde6c, 0x5f41f9e3, 0xfcd7d1af, 0xc31caf3a, 0x608a8776, + 0xbc8a5210, 0x1f1c7a5c, 0x20d704c9, 0x83412c85, 0x43a7a844, + 0xe0318008, 0xdffafe9d, 0x7c6cd6d1, 0xa06c03b7, 0x03fa2bfb, + 0x3c31556e, 0x9fa77d22, 0xf2269109, 0x51b0b945, 0x6e7bc7d0, + 0xcdedef9c, 0x11ed3afa, 0xb27b12b6, 0x8db06c23, 0x2e26446f, + 0xeec0c0ae, 0x4d56e8e2, 0x729d9677, 0xd10bbe3b, 0x0d0b6b5d, + 0xae9d4311, 0x91563d84, 0x32c015c8, 0xcbea3247, 0x687c1a0b, + 0x57b7649e, 0xf4214cd2, 0x282199b4, 0x8bb7b1f8, 0xb47ccf6d, + 0x17eae721, 0xd70c63e0, 0x749a4bac, 0x4b513539, 0xe8c71d75, + 0x34c7c813, 0x9751e05f, 0xa89a9eca, 0x0b0cb686, 0x81bfd795, + 0x2229ffd9, 0x1de2814c, 0xbe74a900, 0x62747c66, 0xc1e2542a, + 0xfe292abf, 0x5dbf02f3, 0x9d598632, 0x3ecfae7e, 0x0104d0eb, + 0xa292f8a7, 0x7e922dc1, 0xdd04058d, 0xe2cf7b18, 0x41595354, + 0xb87374db, 0x1be55c97, 0x242e2202, 0x87b80a4e, 0x5bb8df28, + 0xf82ef764, 0xc7e589f1, 0x6473a1bd, 0xa495257c, 0x07030d30, + 0x38c873a5, 0x9b5e5be9, 0x475e8e8f, 0xe4c8a6c3, 0xdb03d856, + 0x7895f01a}, + {0x00000000, 0x2a283862, 0x545070c4, 0x7e7848a6, 0xa8a0e188, + 0x8288d9ea, 0xfcf0914c, 0xd6d8a92e, 0x8a30c551, 0xa018fd33, + 0xde60b595, 0xf4488df7, 0x229024d9, 0x08b81cbb, 0x76c0541d, + 0x5ce86c7f, 0xcf108ce3, 0xe538b481, 0x9b40fc27, 0xb168c445, + 0x67b06d6b, 0x4d985509, 0x33e01daf, 0x19c825cd, 0x452049b2, + 0x6f0871d0, 0x11703976, 0x3b580114, 0xed80a83a, 0xc7a89058, + 0xb9d0d8fe, 0x93f8e09c, 0x45501f87, 0x6f7827e5, 0x11006f43, + 0x3b285721, 0xedf0fe0f, 0xc7d8c66d, 0xb9a08ecb, 0x9388b6a9, + 0xcf60dad6, 0xe548e2b4, 0x9b30aa12, 0xb1189270, 0x67c03b5e, + 0x4de8033c, 0x33904b9a, 0x19b873f8, 0x8a409364, 0xa068ab06, + 0xde10e3a0, 0xf438dbc2, 0x22e072ec, 0x08c84a8e, 0x76b00228, + 0x5c983a4a, 0x00705635, 0x2a586e57, 0x542026f1, 0x7e081e93, + 0xa8d0b7bd, 0x82f88fdf, 0xfc80c779, 0xd6a8ff1b, 0x8aa03f0e, + 0xa088076c, 0xdef04fca, 0xf4d877a8, 0x2200de86, 0x0828e6e4, + 0x7650ae42, 0x5c789620, 0x0090fa5f, 0x2ab8c23d, 0x54c08a9b, + 0x7ee8b2f9, 0xa8301bd7, 0x821823b5, 0xfc606b13, 0xd6485371, + 0x45b0b3ed, 0x6f988b8f, 0x11e0c329, 0x3bc8fb4b, 0xed105265, + 0xc7386a07, 0xb94022a1, 0x93681ac3, 0xcf8076bc, 0xe5a84ede, + 0x9bd00678, 0xb1f83e1a, 0x67209734, 0x4d08af56, 0x3370e7f0, + 0x1958df92, 0xcff02089, 0xe5d818eb, 0x9ba0504d, 0xb188682f, + 0x6750c101, 0x4d78f963, 0x3300b1c5, 0x192889a7, 0x45c0e5d8, + 0x6fe8ddba, 0x1190951c, 0x3bb8ad7e, 0xed600450, 0xc7483c32, + 0xb9307494, 0x93184cf6, 0x00e0ac6a, 0x2ac89408, 0x54b0dcae, + 0x7e98e4cc, 0xa8404de2, 0x82687580, 0xfc103d26, 0xd6380544, + 0x8ad0693b, 0xa0f85159, 0xde8019ff, 0xf4a8219d, 0x227088b3, + 0x0858b0d1, 0x7620f877, 0x5c08c015, 0xce31785d, 0xe419403f, + 0x9a610899, 0xb04930fb, 0x669199d5, 0x4cb9a1b7, 0x32c1e911, + 0x18e9d173, 0x4401bd0c, 0x6e29856e, 0x1051cdc8, 0x3a79f5aa, + 0xeca15c84, 0xc68964e6, 0xb8f12c40, 0x92d91422, 0x0121f4be, + 0x2b09ccdc, 0x5571847a, 0x7f59bc18, 0xa9811536, 0x83a92d54, + 0xfdd165f2, 0xd7f95d90, 0x8b1131ef, 0xa139098d, 0xdf41412b, + 0xf5697949, 0x23b1d067, 0x0999e805, 0x77e1a0a3, 0x5dc998c1, + 0x8b6167da, 0xa1495fb8, 0xdf31171e, 0xf5192f7c, 0x23c18652, + 0x09e9be30, 0x7791f696, 0x5db9cef4, 0x0151a28b, 0x2b799ae9, + 0x5501d24f, 0x7f29ea2d, 0xa9f14303, 0x83d97b61, 0xfda133c7, + 0xd7890ba5, 0x4471eb39, 0x6e59d35b, 0x10219bfd, 0x3a09a39f, + 0xecd10ab1, 0xc6f932d3, 0xb8817a75, 0x92a94217, 0xce412e68, + 0xe469160a, 0x9a115eac, 0xb03966ce, 0x66e1cfe0, 0x4cc9f782, + 0x32b1bf24, 0x18998746, 0x44914753, 0x6eb97f31, 0x10c13797, + 0x3ae90ff5, 0xec31a6db, 0xc6199eb9, 0xb861d61f, 0x9249ee7d, + 0xcea18202, 0xe489ba60, 0x9af1f2c6, 0xb0d9caa4, 0x6601638a, + 0x4c295be8, 0x3251134e, 0x18792b2c, 0x8b81cbb0, 0xa1a9f3d2, + 0xdfd1bb74, 0xf5f98316, 0x23212a38, 0x0909125a, 0x77715afc, + 0x5d59629e, 0x01b10ee1, 0x2b993683, 0x55e17e25, 0x7fc94647, + 0xa911ef69, 0x8339d70b, 0xfd419fad, 0xd769a7cf, 0x01c158d4, + 0x2be960b6, 0x55912810, 0x7fb91072, 0xa961b95c, 0x8349813e, + 0xfd31c998, 0xd719f1fa, 0x8bf19d85, 0xa1d9a5e7, 0xdfa1ed41, + 0xf589d523, 0x23517c0d, 0x0979446f, 0x77010cc9, 0x5d2934ab, + 0xced1d437, 0xe4f9ec55, 0x9a81a4f3, 0xb0a99c91, 0x667135bf, + 0x4c590ddd, 0x3221457b, 0x18097d19, 0x44e11166, 0x6ec92904, + 0x10b161a2, 0x3a9959c0, 0xec41f0ee, 0xc669c88c, 0xb811802a, + 0x9239b848}, + {0x00000000, 0x4713f6fb, 0x8e27edf6, 0xc9341b0d, 0xc73eddad, + 0x802d2b56, 0x4919305b, 0x0e0ac6a0, 0x550cbd1b, 0x121f4be0, + 0xdb2b50ed, 0x9c38a616, 0x923260b6, 0xd521964d, 0x1c158d40, + 0x5b067bbb, 0xaa197a36, 0xed0a8ccd, 0x243e97c0, 0x632d613b, + 0x6d27a79b, 0x2a345160, 0xe3004a6d, 0xa413bc96, 0xff15c72d, + 0xb80631d6, 0x71322adb, 0x3621dc20, 0x382b1a80, 0x7f38ec7b, + 0xb60cf776, 0xf11f018d, 0x8f43f22d, 0xc85004d6, 0x01641fdb, + 0x4677e920, 0x487d2f80, 0x0f6ed97b, 0xc65ac276, 0x8149348d, + 0xda4f4f36, 0x9d5cb9cd, 0x5468a2c0, 0x137b543b, 0x1d71929b, + 0x5a626460, 0x93567f6d, 0xd4458996, 0x255a881b, 0x62497ee0, + 0xab7d65ed, 0xec6e9316, 0xe26455b6, 0xa577a34d, 0x6c43b840, + 0x2b504ebb, 0x70563500, 0x3745c3fb, 0xfe71d8f6, 0xb9622e0d, + 0xb768e8ad, 0xf07b1e56, 0x394f055b, 0x7e5cf3a0, 0xc5f6e21b, + 0x82e514e0, 0x4bd10fed, 0x0cc2f916, 0x02c83fb6, 0x45dbc94d, + 0x8cefd240, 0xcbfc24bb, 0x90fa5f00, 0xd7e9a9fb, 0x1eddb2f6, + 0x59ce440d, 0x57c482ad, 0x10d77456, 0xd9e36f5b, 0x9ef099a0, + 0x6fef982d, 0x28fc6ed6, 0xe1c875db, 0xa6db8320, 0xa8d14580, + 0xefc2b37b, 0x26f6a876, 0x61e55e8d, 0x3ae32536, 0x7df0d3cd, + 0xb4c4c8c0, 0xf3d73e3b, 0xfdddf89b, 0xbace0e60, 0x73fa156d, + 0x34e9e396, 0x4ab51036, 0x0da6e6cd, 0xc492fdc0, 0x83810b3b, + 0x8d8bcd9b, 0xca983b60, 0x03ac206d, 0x44bfd696, 0x1fb9ad2d, + 0x58aa5bd6, 0x919e40db, 0xd68db620, 0xd8877080, 0x9f94867b, + 0x56a09d76, 0x11b36b8d, 0xe0ac6a00, 0xa7bf9cfb, 0x6e8b87f6, + 0x2998710d, 0x2792b7ad, 0x60814156, 0xa9b55a5b, 0xeea6aca0, + 0xb5a0d71b, 0xf2b321e0, 0x3b873aed, 0x7c94cc16, 0x729e0ab6, + 0x358dfc4d, 0xfcb9e740, 0xbbaa11bb, 0x509cc277, 0x178f348c, + 0xdebb2f81, 0x99a8d97a, 0x97a21fda, 0xd0b1e921, 0x1985f22c, + 0x5e9604d7, 0x05907f6c, 0x42838997, 0x8bb7929a, 0xcca46461, + 0xc2aea2c1, 0x85bd543a, 0x4c894f37, 0x0b9ab9cc, 0xfa85b841, + 0xbd964eba, 0x74a255b7, 0x33b1a34c, 0x3dbb65ec, 0x7aa89317, + 0xb39c881a, 0xf48f7ee1, 0xaf89055a, 0xe89af3a1, 0x21aee8ac, + 0x66bd1e57, 0x68b7d8f7, 0x2fa42e0c, 0xe6903501, 0xa183c3fa, + 0xdfdf305a, 0x98ccc6a1, 0x51f8ddac, 0x16eb2b57, 0x18e1edf7, + 0x5ff21b0c, 0x96c60001, 0xd1d5f6fa, 0x8ad38d41, 0xcdc07bba, + 0x04f460b7, 0x43e7964c, 0x4ded50ec, 0x0afea617, 0xc3cabd1a, + 0x84d94be1, 0x75c64a6c, 0x32d5bc97, 0xfbe1a79a, 0xbcf25161, + 0xb2f897c1, 0xf5eb613a, 0x3cdf7a37, 0x7bcc8ccc, 0x20caf777, + 0x67d9018c, 0xaeed1a81, 0xe9feec7a, 0xe7f42ada, 0xa0e7dc21, + 0x69d3c72c, 0x2ec031d7, 0x956a206c, 0xd279d697, 0x1b4dcd9a, + 0x5c5e3b61, 0x5254fdc1, 0x15470b3a, 0xdc731037, 0x9b60e6cc, + 0xc0669d77, 0x87756b8c, 0x4e417081, 0x0952867a, 0x075840da, + 0x404bb621, 0x897fad2c, 0xce6c5bd7, 0x3f735a5a, 0x7860aca1, + 0xb154b7ac, 0xf6474157, 0xf84d87f7, 0xbf5e710c, 0x766a6a01, + 0x31799cfa, 0x6a7fe741, 0x2d6c11ba, 0xe4580ab7, 0xa34bfc4c, + 0xad413aec, 0xea52cc17, 0x2366d71a, 0x647521e1, 0x1a29d241, + 0x5d3a24ba, 0x940e3fb7, 0xd31dc94c, 0xdd170fec, 0x9a04f917, + 0x5330e21a, 0x142314e1, 0x4f256f5a, 0x083699a1, 0xc10282ac, + 0x86117457, 0x881bb2f7, 0xcf08440c, 0x063c5f01, 0x412fa9fa, + 0xb030a877, 0xf7235e8c, 0x3e174581, 0x7904b37a, 0x770e75da, + 0x301d8321, 0xf929982c, 0xbe3a6ed7, 0xe53c156c, 0xa22fe397, + 0x6b1bf89a, 0x2c080e61, 0x2202c8c1, 0x65113e3a, 0xac252537, + 0xeb36d3cc}, + {0x00000000, 0xa13984ee, 0x99020f9d, 0x383b8b73, 0xe975197b, + 0x484c9d95, 0x707716e6, 0xd14e9208, 0x099b34b7, 0xa8a2b059, + 0x90993b2a, 0x31a0bfc4, 0xe0ee2dcc, 0x41d7a922, 0x79ec2251, + 0xd8d5a6bf, 0x1336696e, 0xb20fed80, 0x8a3466f3, 0x2b0de21d, + 0xfa437015, 0x5b7af4fb, 0x63417f88, 0xc278fb66, 0x1aad5dd9, + 0xbb94d937, 0x83af5244, 0x2296d6aa, 0xf3d844a2, 0x52e1c04c, + 0x6ada4b3f, 0xcbe3cfd1, 0x266cd2dc, 0x87555632, 0xbf6edd41, + 0x1e5759af, 0xcf19cba7, 0x6e204f49, 0x561bc43a, 0xf72240d4, + 0x2ff7e66b, 0x8ece6285, 0xb6f5e9f6, 0x17cc6d18, 0xc682ff10, + 0x67bb7bfe, 0x5f80f08d, 0xfeb97463, 0x355abbb2, 0x94633f5c, + 0xac58b42f, 0x0d6130c1, 0xdc2fa2c9, 0x7d162627, 0x452dad54, + 0xe41429ba, 0x3cc18f05, 0x9df80beb, 0xa5c38098, 0x04fa0476, + 0xd5b4967e, 0x748d1290, 0x4cb699e3, 0xed8f1d0d, 0x4cd9a5b8, + 0xede02156, 0xd5dbaa25, 0x74e22ecb, 0xa5acbcc3, 0x0495382d, + 0x3caeb35e, 0x9d9737b0, 0x4542910f, 0xe47b15e1, 0xdc409e92, + 0x7d791a7c, 0xac378874, 0x0d0e0c9a, 0x353587e9, 0x940c0307, + 0x5fefccd6, 0xfed64838, 0xc6edc34b, 0x67d447a5, 0xb69ad5ad, + 0x17a35143, 0x2f98da30, 0x8ea15ede, 0x5674f861, 0xf74d7c8f, + 0xcf76f7fc, 0x6e4f7312, 0xbf01e11a, 0x1e3865f4, 0x2603ee87, + 0x873a6a69, 0x6ab57764, 0xcb8cf38a, 0xf3b778f9, 0x528efc17, + 0x83c06e1f, 0x22f9eaf1, 0x1ac26182, 0xbbfbe56c, 0x632e43d3, + 0xc217c73d, 0xfa2c4c4e, 0x5b15c8a0, 0x8a5b5aa8, 0x2b62de46, + 0x13595535, 0xb260d1db, 0x79831e0a, 0xd8ba9ae4, 0xe0811197, + 0x41b89579, 0x90f60771, 0x31cf839f, 0x09f408ec, 0xa8cd8c02, + 0x70182abd, 0xd121ae53, 0xe91a2520, 0x4823a1ce, 0x996d33c6, + 0x3854b728, 0x006f3c5b, 0xa156b8b5, 0x99b34b70, 0x388acf9e, + 0x00b144ed, 0xa188c003, 0x70c6520b, 0xd1ffd6e5, 0xe9c45d96, + 0x48fdd978, 0x90287fc7, 0x3111fb29, 0x092a705a, 0xa813f4b4, + 0x795d66bc, 0xd864e252, 0xe05f6921, 0x4166edcf, 0x8a85221e, + 0x2bbca6f0, 0x13872d83, 0xb2bea96d, 0x63f03b65, 0xc2c9bf8b, + 0xfaf234f8, 0x5bcbb016, 0x831e16a9, 0x22279247, 0x1a1c1934, + 0xbb259dda, 0x6a6b0fd2, 0xcb528b3c, 0xf369004f, 0x525084a1, + 0xbfdf99ac, 0x1ee61d42, 0x26dd9631, 0x87e412df, 0x56aa80d7, + 0xf7930439, 0xcfa88f4a, 0x6e910ba4, 0xb644ad1b, 0x177d29f5, + 0x2f46a286, 0x8e7f2668, 0x5f31b460, 0xfe08308e, 0xc633bbfd, + 0x670a3f13, 0xace9f0c2, 0x0dd0742c, 0x35ebff5f, 0x94d27bb1, + 0x459ce9b9, 0xe4a56d57, 0xdc9ee624, 0x7da762ca, 0xa572c475, + 0x044b409b, 0x3c70cbe8, 0x9d494f06, 0x4c07dd0e, 0xed3e59e0, + 0xd505d293, 0x743c567d, 0xd56aeec8, 0x74536a26, 0x4c68e155, + 0xed5165bb, 0x3c1ff7b3, 0x9d26735d, 0xa51df82e, 0x04247cc0, + 0xdcf1da7f, 0x7dc85e91, 0x45f3d5e2, 0xe4ca510c, 0x3584c304, + 0x94bd47ea, 0xac86cc99, 0x0dbf4877, 0xc65c87a6, 0x67650348, + 0x5f5e883b, 0xfe670cd5, 0x2f299edd, 0x8e101a33, 0xb62b9140, + 0x171215ae, 0xcfc7b311, 0x6efe37ff, 0x56c5bc8c, 0xf7fc3862, + 0x26b2aa6a, 0x878b2e84, 0xbfb0a5f7, 0x1e892119, 0xf3063c14, + 0x523fb8fa, 0x6a043389, 0xcb3db767, 0x1a73256f, 0xbb4aa181, + 0x83712af2, 0x2248ae1c, 0xfa9d08a3, 0x5ba48c4d, 0x639f073e, + 0xc2a683d0, 0x13e811d8, 0xb2d19536, 0x8aea1e45, 0x2bd39aab, + 0xe030557a, 0x4109d194, 0x79325ae7, 0xd80bde09, 0x09454c01, + 0xa87cc8ef, 0x9047439c, 0x317ec772, 0xe9ab61cd, 0x4892e523, + 0x70a96e50, 0xd190eabe, 0x00de78b6, 0xa1e7fc58, 0x99dc772b, + 0x38e5f3c5}, + {0x00000000, 0xe81790a1, 0x0b5e2703, 0xe349b7a2, 0x16bc4e06, + 0xfeabdea7, 0x1de26905, 0xf5f5f9a4, 0x2d789c0c, 0xc56f0cad, + 0x2626bb0f, 0xce312bae, 0x3bc4d20a, 0xd3d342ab, 0x309af509, + 0xd88d65a8, 0x5af13818, 0xb2e6a8b9, 0x51af1f1b, 0xb9b88fba, + 0x4c4d761e, 0xa45ae6bf, 0x4713511d, 0xaf04c1bc, 0x7789a414, + 0x9f9e34b5, 0x7cd78317, 0x94c013b6, 0x6135ea12, 0x89227ab3, + 0x6a6bcd11, 0x827c5db0, 0xb5e27030, 0x5df5e091, 0xbebc5733, + 0x56abc792, 0xa35e3e36, 0x4b49ae97, 0xa8001935, 0x40178994, + 0x989aec3c, 0x708d7c9d, 0x93c4cb3f, 0x7bd35b9e, 0x8e26a23a, + 0x6631329b, 0x85788539, 0x6d6f1598, 0xef134828, 0x0704d889, + 0xe44d6f2b, 0x0c5aff8a, 0xf9af062e, 0x11b8968f, 0xf2f1212d, + 0x1ae6b18c, 0xc26bd424, 0x2a7c4485, 0xc935f327, 0x21226386, + 0xd4d79a22, 0x3cc00a83, 0xdf89bd21, 0x379e2d80, 0xb0b5e621, + 0x58a27680, 0xbbebc122, 0x53fc5183, 0xa609a827, 0x4e1e3886, + 0xad578f24, 0x45401f85, 0x9dcd7a2d, 0x75daea8c, 0x96935d2e, + 0x7e84cd8f, 0x8b71342b, 0x6366a48a, 0x802f1328, 0x68388389, + 0xea44de39, 0x02534e98, 0xe11af93a, 0x090d699b, 0xfcf8903f, + 0x14ef009e, 0xf7a6b73c, 0x1fb1279d, 0xc73c4235, 0x2f2bd294, + 0xcc626536, 0x2475f597, 0xd1800c33, 0x39979c92, 0xdade2b30, + 0x32c9bb91, 0x05579611, 0xed4006b0, 0x0e09b112, 0xe61e21b3, + 0x13ebd817, 0xfbfc48b6, 0x18b5ff14, 0xf0a26fb5, 0x282f0a1d, + 0xc0389abc, 0x23712d1e, 0xcb66bdbf, 0x3e93441b, 0xd684d4ba, + 0x35cd6318, 0xdddaf3b9, 0x5fa6ae09, 0xb7b13ea8, 0x54f8890a, + 0xbcef19ab, 0x491ae00f, 0xa10d70ae, 0x4244c70c, 0xaa5357ad, + 0x72de3205, 0x9ac9a2a4, 0x79801506, 0x919785a7, 0x64627c03, + 0x8c75eca2, 0x6f3c5b00, 0x872bcba1, 0xba1aca03, 0x520d5aa2, + 0xb144ed00, 0x59537da1, 0xaca68405, 0x44b114a4, 0xa7f8a306, + 0x4fef33a7, 0x9762560f, 0x7f75c6ae, 0x9c3c710c, 0x742be1ad, + 0x81de1809, 0x69c988a8, 0x8a803f0a, 0x6297afab, 0xe0ebf21b, + 0x08fc62ba, 0xebb5d518, 0x03a245b9, 0xf657bc1d, 0x1e402cbc, + 0xfd099b1e, 0x151e0bbf, 0xcd936e17, 0x2584feb6, 0xc6cd4914, + 0x2edad9b5, 0xdb2f2011, 0x3338b0b0, 0xd0710712, 0x386697b3, + 0x0ff8ba33, 0xe7ef2a92, 0x04a69d30, 0xecb10d91, 0x1944f435, + 0xf1536494, 0x121ad336, 0xfa0d4397, 0x2280263f, 0xca97b69e, + 0x29de013c, 0xc1c9919d, 0x343c6839, 0xdc2bf898, 0x3f624f3a, + 0xd775df9b, 0x5509822b, 0xbd1e128a, 0x5e57a528, 0xb6403589, + 0x43b5cc2d, 0xaba25c8c, 0x48ebeb2e, 0xa0fc7b8f, 0x78711e27, + 0x90668e86, 0x732f3924, 0x9b38a985, 0x6ecd5021, 0x86dac080, + 0x65937722, 0x8d84e783, 0x0aaf2c22, 0xe2b8bc83, 0x01f10b21, + 0xe9e69b80, 0x1c136224, 0xf404f285, 0x174d4527, 0xff5ad586, + 0x27d7b02e, 0xcfc0208f, 0x2c89972d, 0xc49e078c, 0x316bfe28, + 0xd97c6e89, 0x3a35d92b, 0xd222498a, 0x505e143a, 0xb849849b, + 0x5b003339, 0xb317a398, 0x46e25a3c, 0xaef5ca9d, 0x4dbc7d3f, + 0xa5abed9e, 0x7d268836, 0x95311897, 0x7678af35, 0x9e6f3f94, + 0x6b9ac630, 0x838d5691, 0x60c4e133, 0x88d37192, 0xbf4d5c12, + 0x575accb3, 0xb4137b11, 0x5c04ebb0, 0xa9f11214, 0x41e682b5, + 0xa2af3517, 0x4ab8a5b6, 0x9235c01e, 0x7a2250bf, 0x996be71d, + 0x717c77bc, 0x84898e18, 0x6c9e1eb9, 0x8fd7a91b, 0x67c039ba, + 0xe5bc640a, 0x0dabf4ab, 0xeee24309, 0x06f5d3a8, 0xf3002a0c, + 0x1b17baad, 0xf85e0d0f, 0x10499dae, 0xc8c4f806, 0x20d368a7, + 0xc39adf05, 0x2b8d4fa4, 0xde78b600, 0x366f26a1, 0xd5269103, + 0x3d3101a2}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x0000000000000000, 0xa19017e800000000, 0x03275e0b00000000, + 0xa2b749e300000000, 0x064ebc1600000000, 0xa7deabfe00000000, + 0x0569e21d00000000, 0xa4f9f5f500000000, 0x0c9c782d00000000, + 0xad0c6fc500000000, 0x0fbb262600000000, 0xae2b31ce00000000, + 0x0ad2c43b00000000, 0xab42d3d300000000, 0x09f59a3000000000, + 0xa8658dd800000000, 0x1838f15a00000000, 0xb9a8e6b200000000, + 0x1b1faf5100000000, 0xba8fb8b900000000, 0x1e764d4c00000000, + 0xbfe65aa400000000, 0x1d51134700000000, 0xbcc104af00000000, + 0x14a4897700000000, 0xb5349e9f00000000, 0x1783d77c00000000, + 0xb613c09400000000, 0x12ea356100000000, 0xb37a228900000000, + 0x11cd6b6a00000000, 0xb05d7c8200000000, 0x3070e2b500000000, + 0x91e0f55d00000000, 0x3357bcbe00000000, 0x92c7ab5600000000, + 0x363e5ea300000000, 0x97ae494b00000000, 0x351900a800000000, + 0x9489174000000000, 0x3cec9a9800000000, 0x9d7c8d7000000000, + 0x3fcbc49300000000, 0x9e5bd37b00000000, 0x3aa2268e00000000, + 0x9b32316600000000, 0x3985788500000000, 0x98156f6d00000000, + 0x284813ef00000000, 0x89d8040700000000, 0x2b6f4de400000000, + 0x8aff5a0c00000000, 0x2e06aff900000000, 0x8f96b81100000000, + 0x2d21f1f200000000, 0x8cb1e61a00000000, 0x24d46bc200000000, + 0x85447c2a00000000, 0x27f335c900000000, 0x8663222100000000, + 0x229ad7d400000000, 0x830ac03c00000000, 0x21bd89df00000000, + 0x802d9e3700000000, 0x21e6b5b000000000, 0x8076a25800000000, + 0x22c1ebbb00000000, 0x8351fc5300000000, 0x27a809a600000000, + 0x86381e4e00000000, 0x248f57ad00000000, 0x851f404500000000, + 0x2d7acd9d00000000, 0x8ceada7500000000, 0x2e5d939600000000, + 0x8fcd847e00000000, 0x2b34718b00000000, 0x8aa4666300000000, + 0x28132f8000000000, 0x8983386800000000, 0x39de44ea00000000, + 0x984e530200000000, 0x3af91ae100000000, 0x9b690d0900000000, + 0x3f90f8fc00000000, 0x9e00ef1400000000, 0x3cb7a6f700000000, + 0x9d27b11f00000000, 0x35423cc700000000, 0x94d22b2f00000000, + 0x366562cc00000000, 0x97f5752400000000, 0x330c80d100000000, + 0x929c973900000000, 0x302bdeda00000000, 0x91bbc93200000000, + 0x1196570500000000, 0xb00640ed00000000, 0x12b1090e00000000, + 0xb3211ee600000000, 0x17d8eb1300000000, 0xb648fcfb00000000, + 0x14ffb51800000000, 0xb56fa2f000000000, 0x1d0a2f2800000000, + 0xbc9a38c000000000, 0x1e2d712300000000, 0xbfbd66cb00000000, + 0x1b44933e00000000, 0xbad484d600000000, 0x1863cd3500000000, + 0xb9f3dadd00000000, 0x09aea65f00000000, 0xa83eb1b700000000, + 0x0a89f85400000000, 0xab19efbc00000000, 0x0fe01a4900000000, + 0xae700da100000000, 0x0cc7444200000000, 0xad5753aa00000000, + 0x0532de7200000000, 0xa4a2c99a00000000, 0x0615807900000000, + 0xa785979100000000, 0x037c626400000000, 0xa2ec758c00000000, + 0x005b3c6f00000000, 0xa1cb2b8700000000, 0x03ca1aba00000000, + 0xa25a0d5200000000, 0x00ed44b100000000, 0xa17d535900000000, + 0x0584a6ac00000000, 0xa414b14400000000, 0x06a3f8a700000000, + 0xa733ef4f00000000, 0x0f56629700000000, 0xaec6757f00000000, + 0x0c713c9c00000000, 0xade12b7400000000, 0x0918de8100000000, + 0xa888c96900000000, 0x0a3f808a00000000, 0xabaf976200000000, + 0x1bf2ebe000000000, 0xba62fc0800000000, 0x18d5b5eb00000000, + 0xb945a20300000000, 0x1dbc57f600000000, 0xbc2c401e00000000, + 0x1e9b09fd00000000, 0xbf0b1e1500000000, 0x176e93cd00000000, + 0xb6fe842500000000, 0x1449cdc600000000, 0xb5d9da2e00000000, + 0x11202fdb00000000, 0xb0b0383300000000, 0x120771d000000000, + 0xb397663800000000, 0x33baf80f00000000, 0x922aefe700000000, + 0x309da60400000000, 0x910db1ec00000000, 0x35f4441900000000, + 0x946453f100000000, 0x36d31a1200000000, 0x97430dfa00000000, + 0x3f26802200000000, 0x9eb697ca00000000, 0x3c01de2900000000, + 0x9d91c9c100000000, 0x39683c3400000000, 0x98f82bdc00000000, + 0x3a4f623f00000000, 0x9bdf75d700000000, 0x2b82095500000000, + 0x8a121ebd00000000, 0x28a5575e00000000, 0x893540b600000000, + 0x2dccb54300000000, 0x8c5ca2ab00000000, 0x2eebeb4800000000, + 0x8f7bfca000000000, 0x271e717800000000, 0x868e669000000000, + 0x24392f7300000000, 0x85a9389b00000000, 0x2150cd6e00000000, + 0x80c0da8600000000, 0x2277936500000000, 0x83e7848d00000000, + 0x222caf0a00000000, 0x83bcb8e200000000, 0x210bf10100000000, + 0x809be6e900000000, 0x2462131c00000000, 0x85f204f400000000, + 0x27454d1700000000, 0x86d55aff00000000, 0x2eb0d72700000000, + 0x8f20c0cf00000000, 0x2d97892c00000000, 0x8c079ec400000000, + 0x28fe6b3100000000, 0x896e7cd900000000, 0x2bd9353a00000000, + 0x8a4922d200000000, 0x3a145e5000000000, 0x9b8449b800000000, + 0x3933005b00000000, 0x98a317b300000000, 0x3c5ae24600000000, + 0x9dcaf5ae00000000, 0x3f7dbc4d00000000, 0x9eedaba500000000, + 0x3688267d00000000, 0x9718319500000000, 0x35af787600000000, + 0x943f6f9e00000000, 0x30c69a6b00000000, 0x91568d8300000000, + 0x33e1c46000000000, 0x9271d38800000000, 0x125c4dbf00000000, + 0xb3cc5a5700000000, 0x117b13b400000000, 0xb0eb045c00000000, + 0x1412f1a900000000, 0xb582e64100000000, 0x1735afa200000000, + 0xb6a5b84a00000000, 0x1ec0359200000000, 0xbf50227a00000000, + 0x1de76b9900000000, 0xbc777c7100000000, 0x188e898400000000, + 0xb91e9e6c00000000, 0x1ba9d78f00000000, 0xba39c06700000000, + 0x0a64bce500000000, 0xabf4ab0d00000000, 0x0943e2ee00000000, + 0xa8d3f50600000000, 0x0c2a00f300000000, 0xadba171b00000000, + 0x0f0d5ef800000000, 0xae9d491000000000, 0x06f8c4c800000000, + 0xa768d32000000000, 0x05df9ac300000000, 0xa44f8d2b00000000, + 0x00b678de00000000, 0xa1266f3600000000, 0x039126d500000000, + 0xa201313d00000000}, + {0x0000000000000000, 0xee8439a100000000, 0x9d0f029900000000, + 0x738b3b3800000000, 0x7b1975e900000000, 0x959d4c4800000000, + 0xe616777000000000, 0x08924ed100000000, 0xb7349b0900000000, + 0x59b0a2a800000000, 0x2a3b999000000000, 0xc4bfa03100000000, + 0xcc2deee000000000, 0x22a9d74100000000, 0x5122ec7900000000, + 0xbfa6d5d800000000, 0x6e69361300000000, 0x80ed0fb200000000, + 0xf366348a00000000, 0x1de20d2b00000000, 0x157043fa00000000, + 0xfbf47a5b00000000, 0x887f416300000000, 0x66fb78c200000000, + 0xd95dad1a00000000, 0x37d994bb00000000, 0x4452af8300000000, + 0xaad6962200000000, 0xa244d8f300000000, 0x4cc0e15200000000, + 0x3f4bda6a00000000, 0xd1cfe3cb00000000, 0xdcd26c2600000000, + 0x3256558700000000, 0x41dd6ebf00000000, 0xaf59571e00000000, + 0xa7cb19cf00000000, 0x494f206e00000000, 0x3ac41b5600000000, + 0xd44022f700000000, 0x6be6f72f00000000, 0x8562ce8e00000000, + 0xf6e9f5b600000000, 0x186dcc1700000000, 0x10ff82c600000000, + 0xfe7bbb6700000000, 0x8df0805f00000000, 0x6374b9fe00000000, + 0xb2bb5a3500000000, 0x5c3f639400000000, 0x2fb458ac00000000, + 0xc130610d00000000, 0xc9a22fdc00000000, 0x2726167d00000000, + 0x54ad2d4500000000, 0xba2914e400000000, 0x058fc13c00000000, + 0xeb0bf89d00000000, 0x9880c3a500000000, 0x7604fa0400000000, + 0x7e96b4d500000000, 0x90128d7400000000, 0xe399b64c00000000, + 0x0d1d8fed00000000, 0xb8a5d94c00000000, 0x5621e0ed00000000, + 0x25aadbd500000000, 0xcb2ee27400000000, 0xc3bcaca500000000, + 0x2d38950400000000, 0x5eb3ae3c00000000, 0xb037979d00000000, + 0x0f91424500000000, 0xe1157be400000000, 0x929e40dc00000000, + 0x7c1a797d00000000, 0x748837ac00000000, 0x9a0c0e0d00000000, + 0xe987353500000000, 0x07030c9400000000, 0xd6ccef5f00000000, + 0x3848d6fe00000000, 0x4bc3edc600000000, 0xa547d46700000000, + 0xadd59ab600000000, 0x4351a31700000000, 0x30da982f00000000, + 0xde5ea18e00000000, 0x61f8745600000000, 0x8f7c4df700000000, + 0xfcf776cf00000000, 0x12734f6e00000000, 0x1ae101bf00000000, + 0xf465381e00000000, 0x87ee032600000000, 0x696a3a8700000000, + 0x6477b56a00000000, 0x8af38ccb00000000, 0xf978b7f300000000, + 0x17fc8e5200000000, 0x1f6ec08300000000, 0xf1eaf92200000000, + 0x8261c21a00000000, 0x6ce5fbbb00000000, 0xd3432e6300000000, + 0x3dc717c200000000, 0x4e4c2cfa00000000, 0xa0c8155b00000000, + 0xa85a5b8a00000000, 0x46de622b00000000, 0x3555591300000000, + 0xdbd160b200000000, 0x0a1e837900000000, 0xe49abad800000000, + 0x971181e000000000, 0x7995b84100000000, 0x7107f69000000000, + 0x9f83cf3100000000, 0xec08f40900000000, 0x028ccda800000000, + 0xbd2a187000000000, 0x53ae21d100000000, 0x20251ae900000000, + 0xcea1234800000000, 0xc6336d9900000000, 0x28b7543800000000, + 0x5b3c6f0000000000, 0xb5b856a100000000, 0x704bb39900000000, + 0x9ecf8a3800000000, 0xed44b10000000000, 0x03c088a100000000, + 0x0b52c67000000000, 0xe5d6ffd100000000, 0x965dc4e900000000, + 0x78d9fd4800000000, 0xc77f289000000000, 0x29fb113100000000, + 0x5a702a0900000000, 0xb4f413a800000000, 0xbc665d7900000000, + 0x52e264d800000000, 0x21695fe000000000, 0xcfed664100000000, + 0x1e22858a00000000, 0xf0a6bc2b00000000, 0x832d871300000000, + 0x6da9beb200000000, 0x653bf06300000000, 0x8bbfc9c200000000, + 0xf834f2fa00000000, 0x16b0cb5b00000000, 0xa9161e8300000000, + 0x4792272200000000, 0x34191c1a00000000, 0xda9d25bb00000000, + 0xd20f6b6a00000000, 0x3c8b52cb00000000, 0x4f0069f300000000, + 0xa184505200000000, 0xac99dfbf00000000, 0x421de61e00000000, + 0x3196dd2600000000, 0xdf12e48700000000, 0xd780aa5600000000, + 0x390493f700000000, 0x4a8fa8cf00000000, 0xa40b916e00000000, + 0x1bad44b600000000, 0xf5297d1700000000, 0x86a2462f00000000, + 0x68267f8e00000000, 0x60b4315f00000000, 0x8e3008fe00000000, + 0xfdbb33c600000000, 0x133f0a6700000000, 0xc2f0e9ac00000000, + 0x2c74d00d00000000, 0x5fffeb3500000000, 0xb17bd29400000000, + 0xb9e99c4500000000, 0x576da5e400000000, 0x24e69edc00000000, + 0xca62a77d00000000, 0x75c472a500000000, 0x9b404b0400000000, + 0xe8cb703c00000000, 0x064f499d00000000, 0x0edd074c00000000, + 0xe0593eed00000000, 0x93d205d500000000, 0x7d563c7400000000, + 0xc8ee6ad500000000, 0x266a537400000000, 0x55e1684c00000000, + 0xbb6551ed00000000, 0xb3f71f3c00000000, 0x5d73269d00000000, + 0x2ef81da500000000, 0xc07c240400000000, 0x7fdaf1dc00000000, + 0x915ec87d00000000, 0xe2d5f34500000000, 0x0c51cae400000000, + 0x04c3843500000000, 0xea47bd9400000000, 0x99cc86ac00000000, + 0x7748bf0d00000000, 0xa6875cc600000000, 0x4803656700000000, + 0x3b885e5f00000000, 0xd50c67fe00000000, 0xdd9e292f00000000, + 0x331a108e00000000, 0x40912bb600000000, 0xae15121700000000, + 0x11b3c7cf00000000, 0xff37fe6e00000000, 0x8cbcc55600000000, + 0x6238fcf700000000, 0x6aaab22600000000, 0x842e8b8700000000, + 0xf7a5b0bf00000000, 0x1921891e00000000, 0x143c06f300000000, + 0xfab83f5200000000, 0x8933046a00000000, 0x67b73dcb00000000, + 0x6f25731a00000000, 0x81a14abb00000000, 0xf22a718300000000, + 0x1cae482200000000, 0xa3089dfa00000000, 0x4d8ca45b00000000, + 0x3e079f6300000000, 0xd083a6c200000000, 0xd811e81300000000, + 0x3695d1b200000000, 0x451eea8a00000000, 0xab9ad32b00000000, + 0x7a5530e000000000, 0x94d1094100000000, 0xe75a327900000000, + 0x09de0bd800000000, 0x014c450900000000, 0xefc87ca800000000, + 0x9c43479000000000, 0x72c77e3100000000, 0xcd61abe900000000, + 0x23e5924800000000, 0x506ea97000000000, 0xbeea90d100000000, + 0xb678de0000000000, 0x58fce7a100000000, 0x2b77dc9900000000, + 0xc5f3e53800000000}, + {0x0000000000000000, 0xfbf6134700000000, 0xf6ed278e00000000, + 0x0d1b34c900000000, 0xaddd3ec700000000, 0x562b2d8000000000, + 0x5b30194900000000, 0xa0c60a0e00000000, 0x1bbd0c5500000000, + 0xe04b1f1200000000, 0xed502bdb00000000, 0x16a6389c00000000, + 0xb660329200000000, 0x4d9621d500000000, 0x408d151c00000000, + 0xbb7b065b00000000, 0x367a19aa00000000, 0xcd8c0aed00000000, + 0xc0973e2400000000, 0x3b612d6300000000, 0x9ba7276d00000000, + 0x6051342a00000000, 0x6d4a00e300000000, 0x96bc13a400000000, + 0x2dc715ff00000000, 0xd63106b800000000, 0xdb2a327100000000, + 0x20dc213600000000, 0x801a2b3800000000, 0x7bec387f00000000, + 0x76f70cb600000000, 0x8d011ff100000000, 0x2df2438f00000000, + 0xd60450c800000000, 0xdb1f640100000000, 0x20e9774600000000, + 0x802f7d4800000000, 0x7bd96e0f00000000, 0x76c25ac600000000, + 0x8d34498100000000, 0x364f4fda00000000, 0xcdb95c9d00000000, + 0xc0a2685400000000, 0x3b547b1300000000, 0x9b92711d00000000, + 0x6064625a00000000, 0x6d7f569300000000, 0x968945d400000000, + 0x1b885a2500000000, 0xe07e496200000000, 0xed657dab00000000, + 0x16936eec00000000, 0xb65564e200000000, 0x4da377a500000000, + 0x40b8436c00000000, 0xbb4e502b00000000, 0x0035567000000000, + 0xfbc3453700000000, 0xf6d871fe00000000, 0x0d2e62b900000000, + 0xade868b700000000, 0x561e7bf000000000, 0x5b054f3900000000, + 0xa0f35c7e00000000, 0x1be2f6c500000000, 0xe014e58200000000, + 0xed0fd14b00000000, 0x16f9c20c00000000, 0xb63fc80200000000, + 0x4dc9db4500000000, 0x40d2ef8c00000000, 0xbb24fccb00000000, + 0x005ffa9000000000, 0xfba9e9d700000000, 0xf6b2dd1e00000000, + 0x0d44ce5900000000, 0xad82c45700000000, 0x5674d71000000000, + 0x5b6fe3d900000000, 0xa099f09e00000000, 0x2d98ef6f00000000, + 0xd66efc2800000000, 0xdb75c8e100000000, 0x2083dba600000000, + 0x8045d1a800000000, 0x7bb3c2ef00000000, 0x76a8f62600000000, + 0x8d5ee56100000000, 0x3625e33a00000000, 0xcdd3f07d00000000, + 0xc0c8c4b400000000, 0x3b3ed7f300000000, 0x9bf8ddfd00000000, + 0x600eceba00000000, 0x6d15fa7300000000, 0x96e3e93400000000, + 0x3610b54a00000000, 0xcde6a60d00000000, 0xc0fd92c400000000, + 0x3b0b818300000000, 0x9bcd8b8d00000000, 0x603b98ca00000000, + 0x6d20ac0300000000, 0x96d6bf4400000000, 0x2dadb91f00000000, + 0xd65baa5800000000, 0xdb409e9100000000, 0x20b68dd600000000, + 0x807087d800000000, 0x7b86949f00000000, 0x769da05600000000, + 0x8d6bb31100000000, 0x006aace000000000, 0xfb9cbfa700000000, + 0xf6878b6e00000000, 0x0d71982900000000, 0xadb7922700000000, + 0x5641816000000000, 0x5b5ab5a900000000, 0xa0aca6ee00000000, + 0x1bd7a0b500000000, 0xe021b3f200000000, 0xed3a873b00000000, + 0x16cc947c00000000, 0xb60a9e7200000000, 0x4dfc8d3500000000, + 0x40e7b9fc00000000, 0xbb11aabb00000000, 0x77c29c5000000000, + 0x8c348f1700000000, 0x812fbbde00000000, 0x7ad9a89900000000, + 0xda1fa29700000000, 0x21e9b1d000000000, 0x2cf2851900000000, + 0xd704965e00000000, 0x6c7f900500000000, 0x9789834200000000, + 0x9a92b78b00000000, 0x6164a4cc00000000, 0xc1a2aec200000000, + 0x3a54bd8500000000, 0x374f894c00000000, 0xccb99a0b00000000, + 0x41b885fa00000000, 0xba4e96bd00000000, 0xb755a27400000000, + 0x4ca3b13300000000, 0xec65bb3d00000000, 0x1793a87a00000000, + 0x1a889cb300000000, 0xe17e8ff400000000, 0x5a0589af00000000, + 0xa1f39ae800000000, 0xace8ae2100000000, 0x571ebd6600000000, + 0xf7d8b76800000000, 0x0c2ea42f00000000, 0x013590e600000000, + 0xfac383a100000000, 0x5a30dfdf00000000, 0xa1c6cc9800000000, + 0xacddf85100000000, 0x572beb1600000000, 0xf7ede11800000000, + 0x0c1bf25f00000000, 0x0100c69600000000, 0xfaf6d5d100000000, + 0x418dd38a00000000, 0xba7bc0cd00000000, 0xb760f40400000000, + 0x4c96e74300000000, 0xec50ed4d00000000, 0x17a6fe0a00000000, + 0x1abdcac300000000, 0xe14bd98400000000, 0x6c4ac67500000000, + 0x97bcd53200000000, 0x9aa7e1fb00000000, 0x6151f2bc00000000, + 0xc197f8b200000000, 0x3a61ebf500000000, 0x377adf3c00000000, + 0xcc8ccc7b00000000, 0x77f7ca2000000000, 0x8c01d96700000000, + 0x811aedae00000000, 0x7aecfee900000000, 0xda2af4e700000000, + 0x21dce7a000000000, 0x2cc7d36900000000, 0xd731c02e00000000, + 0x6c206a9500000000, 0x97d679d200000000, 0x9acd4d1b00000000, + 0x613b5e5c00000000, 0xc1fd545200000000, 0x3a0b471500000000, + 0x371073dc00000000, 0xcce6609b00000000, 0x779d66c000000000, + 0x8c6b758700000000, 0x8170414e00000000, 0x7a86520900000000, + 0xda40580700000000, 0x21b64b4000000000, 0x2cad7f8900000000, + 0xd75b6cce00000000, 0x5a5a733f00000000, 0xa1ac607800000000, + 0xacb754b100000000, 0x574147f600000000, 0xf7874df800000000, + 0x0c715ebf00000000, 0x016a6a7600000000, 0xfa9c793100000000, + 0x41e77f6a00000000, 0xba116c2d00000000, 0xb70a58e400000000, + 0x4cfc4ba300000000, 0xec3a41ad00000000, 0x17cc52ea00000000, + 0x1ad7662300000000, 0xe121756400000000, 0x41d2291a00000000, + 0xba243a5d00000000, 0xb73f0e9400000000, 0x4cc91dd300000000, + 0xec0f17dd00000000, 0x17f9049a00000000, 0x1ae2305300000000, + 0xe114231400000000, 0x5a6f254f00000000, 0xa199360800000000, + 0xac8202c100000000, 0x5774118600000000, 0xf7b21b8800000000, + 0x0c4408cf00000000, 0x015f3c0600000000, 0xfaa92f4100000000, + 0x77a830b000000000, 0x8c5e23f700000000, 0x8145173e00000000, + 0x7ab3047900000000, 0xda750e7700000000, 0x21831d3000000000, + 0x2c9829f900000000, 0xd76e3abe00000000, 0x6c153ce500000000, + 0x97e32fa200000000, 0x9af81b6b00000000, 0x610e082c00000000, + 0xc1c8022200000000, 0x3a3e116500000000, 0x372525ac00000000, + 0xccd336eb00000000}, + {0x0000000000000000, 0x6238282a00000000, 0xc470505400000000, + 0xa648787e00000000, 0x88e1a0a800000000, 0xead9888200000000, + 0x4c91f0fc00000000, 0x2ea9d8d600000000, 0x51c5308a00000000, + 0x33fd18a000000000, 0x95b560de00000000, 0xf78d48f400000000, + 0xd924902200000000, 0xbb1cb80800000000, 0x1d54c07600000000, + 0x7f6ce85c00000000, 0xe38c10cf00000000, 0x81b438e500000000, + 0x27fc409b00000000, 0x45c468b100000000, 0x6b6db06700000000, + 0x0955984d00000000, 0xaf1de03300000000, 0xcd25c81900000000, + 0xb249204500000000, 0xd071086f00000000, 0x7639701100000000, + 0x1401583b00000000, 0x3aa880ed00000000, 0x5890a8c700000000, + 0xfed8d0b900000000, 0x9ce0f89300000000, 0x871f504500000000, + 0xe527786f00000000, 0x436f001100000000, 0x2157283b00000000, + 0x0ffef0ed00000000, 0x6dc6d8c700000000, 0xcb8ea0b900000000, + 0xa9b6889300000000, 0xd6da60cf00000000, 0xb4e248e500000000, + 0x12aa309b00000000, 0x709218b100000000, 0x5e3bc06700000000, + 0x3c03e84d00000000, 0x9a4b903300000000, 0xf873b81900000000, + 0x6493408a00000000, 0x06ab68a000000000, 0xa0e310de00000000, + 0xc2db38f400000000, 0xec72e02200000000, 0x8e4ac80800000000, + 0x2802b07600000000, 0x4a3a985c00000000, 0x3556700000000000, + 0x576e582a00000000, 0xf126205400000000, 0x931e087e00000000, + 0xbdb7d0a800000000, 0xdf8ff88200000000, 0x79c780fc00000000, + 0x1bffa8d600000000, 0x0e3fa08a00000000, 0x6c0788a000000000, + 0xca4ff0de00000000, 0xa877d8f400000000, 0x86de002200000000, + 0xe4e6280800000000, 0x42ae507600000000, 0x2096785c00000000, + 0x5ffa900000000000, 0x3dc2b82a00000000, 0x9b8ac05400000000, + 0xf9b2e87e00000000, 0xd71b30a800000000, 0xb523188200000000, + 0x136b60fc00000000, 0x715348d600000000, 0xedb3b04500000000, + 0x8f8b986f00000000, 0x29c3e01100000000, 0x4bfbc83b00000000, + 0x655210ed00000000, 0x076a38c700000000, 0xa12240b900000000, + 0xc31a689300000000, 0xbc7680cf00000000, 0xde4ea8e500000000, + 0x7806d09b00000000, 0x1a3ef8b100000000, 0x3497206700000000, + 0x56af084d00000000, 0xf0e7703300000000, 0x92df581900000000, + 0x8920f0cf00000000, 0xeb18d8e500000000, 0x4d50a09b00000000, + 0x2f6888b100000000, 0x01c1506700000000, 0x63f9784d00000000, + 0xc5b1003300000000, 0xa789281900000000, 0xd8e5c04500000000, + 0xbadde86f00000000, 0x1c95901100000000, 0x7eadb83b00000000, + 0x500460ed00000000, 0x323c48c700000000, 0x947430b900000000, + 0xf64c189300000000, 0x6aace00000000000, 0x0894c82a00000000, + 0xaedcb05400000000, 0xcce4987e00000000, 0xe24d40a800000000, + 0x8075688200000000, 0x263d10fc00000000, 0x440538d600000000, + 0x3b69d08a00000000, 0x5951f8a000000000, 0xff1980de00000000, + 0x9d21a8f400000000, 0xb388702200000000, 0xd1b0580800000000, + 0x77f8207600000000, 0x15c0085c00000000, 0x5d7831ce00000000, + 0x3f4019e400000000, 0x9908619a00000000, 0xfb3049b000000000, + 0xd599916600000000, 0xb7a1b94c00000000, 0x11e9c13200000000, + 0x73d1e91800000000, 0x0cbd014400000000, 0x6e85296e00000000, + 0xc8cd511000000000, 0xaaf5793a00000000, 0x845ca1ec00000000, + 0xe66489c600000000, 0x402cf1b800000000, 0x2214d99200000000, + 0xbef4210100000000, 0xdccc092b00000000, 0x7a84715500000000, + 0x18bc597f00000000, 0x361581a900000000, 0x542da98300000000, + 0xf265d1fd00000000, 0x905df9d700000000, 0xef31118b00000000, + 0x8d0939a100000000, 0x2b4141df00000000, 0x497969f500000000, + 0x67d0b12300000000, 0x05e8990900000000, 0xa3a0e17700000000, + 0xc198c95d00000000, 0xda67618b00000000, 0xb85f49a100000000, + 0x1e1731df00000000, 0x7c2f19f500000000, 0x5286c12300000000, + 0x30bee90900000000, 0x96f6917700000000, 0xf4ceb95d00000000, + 0x8ba2510100000000, 0xe99a792b00000000, 0x4fd2015500000000, + 0x2dea297f00000000, 0x0343f1a900000000, 0x617bd98300000000, + 0xc733a1fd00000000, 0xa50b89d700000000, 0x39eb714400000000, + 0x5bd3596e00000000, 0xfd9b211000000000, 0x9fa3093a00000000, + 0xb10ad1ec00000000, 0xd332f9c600000000, 0x757a81b800000000, + 0x1742a99200000000, 0x682e41ce00000000, 0x0a1669e400000000, + 0xac5e119a00000000, 0xce6639b000000000, 0xe0cfe16600000000, + 0x82f7c94c00000000, 0x24bfb13200000000, 0x4687991800000000, + 0x5347914400000000, 0x317fb96e00000000, 0x9737c11000000000, + 0xf50fe93a00000000, 0xdba631ec00000000, 0xb99e19c600000000, + 0x1fd661b800000000, 0x7dee499200000000, 0x0282a1ce00000000, + 0x60ba89e400000000, 0xc6f2f19a00000000, 0xa4cad9b000000000, + 0x8a63016600000000, 0xe85b294c00000000, 0x4e13513200000000, + 0x2c2b791800000000, 0xb0cb818b00000000, 0xd2f3a9a100000000, + 0x74bbd1df00000000, 0x1683f9f500000000, 0x382a212300000000, + 0x5a12090900000000, 0xfc5a717700000000, 0x9e62595d00000000, + 0xe10eb10100000000, 0x8336992b00000000, 0x257ee15500000000, + 0x4746c97f00000000, 0x69ef11a900000000, 0x0bd7398300000000, + 0xad9f41fd00000000, 0xcfa769d700000000, 0xd458c10100000000, + 0xb660e92b00000000, 0x1028915500000000, 0x7210b97f00000000, + 0x5cb961a900000000, 0x3e81498300000000, 0x98c931fd00000000, + 0xfaf119d700000000, 0x859df18b00000000, 0xe7a5d9a100000000, + 0x41eda1df00000000, 0x23d589f500000000, 0x0d7c512300000000, + 0x6f44790900000000, 0xc90c017700000000, 0xab34295d00000000, + 0x37d4d1ce00000000, 0x55ecf9e400000000, 0xf3a4819a00000000, + 0x919ca9b000000000, 0xbf35716600000000, 0xdd0d594c00000000, + 0x7b45213200000000, 0x197d091800000000, 0x6611e14400000000, + 0x0429c96e00000000, 0xa261b11000000000, 0xc059993a00000000, + 0xeef041ec00000000, 0x8cc869c600000000, 0x2a8011b800000000, + 0x48b8399200000000}, + {0x0000000000000000, 0x4c2896a300000000, 0xd9565d9c00000000, + 0x957ecb3f00000000, 0xf3abcbe300000000, 0xbf835d4000000000, + 0x2afd967f00000000, 0x66d500dc00000000, 0xa751e61c00000000, + 0xeb7970bf00000000, 0x7e07bb8000000000, 0x322f2d2300000000, + 0x54fa2dff00000000, 0x18d2bb5c00000000, 0x8dac706300000000, + 0xc184e6c000000000, 0x4ea3cc3900000000, 0x028b5a9a00000000, + 0x97f591a500000000, 0xdbdd070600000000, 0xbd0807da00000000, + 0xf120917900000000, 0x645e5a4600000000, 0x2876cce500000000, + 0xe9f22a2500000000, 0xa5dabc8600000000, 0x30a477b900000000, + 0x7c8ce11a00000000, 0x1a59e1c600000000, 0x5671776500000000, + 0xc30fbc5a00000000, 0x8f272af900000000, 0x9c46997300000000, + 0xd06e0fd000000000, 0x4510c4ef00000000, 0x0938524c00000000, + 0x6fed529000000000, 0x23c5c43300000000, 0xb6bb0f0c00000000, + 0xfa9399af00000000, 0x3b177f6f00000000, 0x773fe9cc00000000, + 0xe24122f300000000, 0xae69b45000000000, 0xc8bcb48c00000000, + 0x8494222f00000000, 0x11eae91000000000, 0x5dc27fb300000000, + 0xd2e5554a00000000, 0x9ecdc3e900000000, 0x0bb308d600000000, + 0x479b9e7500000000, 0x214e9ea900000000, 0x6d66080a00000000, + 0xf818c33500000000, 0xb430559600000000, 0x75b4b35600000000, + 0x399c25f500000000, 0xace2eeca00000000, 0xe0ca786900000000, + 0x861f78b500000000, 0xca37ee1600000000, 0x5f49252900000000, + 0x1361b38a00000000, 0x388d32e700000000, 0x74a5a44400000000, + 0xe1db6f7b00000000, 0xadf3f9d800000000, 0xcb26f90400000000, + 0x870e6fa700000000, 0x1270a49800000000, 0x5e58323b00000000, + 0x9fdcd4fb00000000, 0xd3f4425800000000, 0x468a896700000000, + 0x0aa21fc400000000, 0x6c771f1800000000, 0x205f89bb00000000, + 0xb521428400000000, 0xf909d42700000000, 0x762efede00000000, + 0x3a06687d00000000, 0xaf78a34200000000, 0xe35035e100000000, + 0x8585353d00000000, 0xc9ada39e00000000, 0x5cd368a100000000, + 0x10fbfe0200000000, 0xd17f18c200000000, 0x9d578e6100000000, + 0x0829455e00000000, 0x4401d3fd00000000, 0x22d4d32100000000, + 0x6efc458200000000, 0xfb828ebd00000000, 0xb7aa181e00000000, + 0xa4cbab9400000000, 0xe8e33d3700000000, 0x7d9df60800000000, + 0x31b560ab00000000, 0x5760607700000000, 0x1b48f6d400000000, + 0x8e363deb00000000, 0xc21eab4800000000, 0x039a4d8800000000, + 0x4fb2db2b00000000, 0xdacc101400000000, 0x96e486b700000000, + 0xf031866b00000000, 0xbc1910c800000000, 0x2967dbf700000000, + 0x654f4d5400000000, 0xea6867ad00000000, 0xa640f10e00000000, + 0x333e3a3100000000, 0x7f16ac9200000000, 0x19c3ac4e00000000, + 0x55eb3aed00000000, 0xc095f1d200000000, 0x8cbd677100000000, + 0x4d3981b100000000, 0x0111171200000000, 0x946fdc2d00000000, + 0xd8474a8e00000000, 0xbe924a5200000000, 0xf2badcf100000000, + 0x67c417ce00000000, 0x2bec816d00000000, 0x311c141500000000, + 0x7d3482b600000000, 0xe84a498900000000, 0xa462df2a00000000, + 0xc2b7dff600000000, 0x8e9f495500000000, 0x1be1826a00000000, + 0x57c914c900000000, 0x964df20900000000, 0xda6564aa00000000, + 0x4f1baf9500000000, 0x0333393600000000, 0x65e639ea00000000, + 0x29ceaf4900000000, 0xbcb0647600000000, 0xf098f2d500000000, + 0x7fbfd82c00000000, 0x33974e8f00000000, 0xa6e985b000000000, + 0xeac1131300000000, 0x8c1413cf00000000, 0xc03c856c00000000, + 0x55424e5300000000, 0x196ad8f000000000, 0xd8ee3e3000000000, + 0x94c6a89300000000, 0x01b863ac00000000, 0x4d90f50f00000000, + 0x2b45f5d300000000, 0x676d637000000000, 0xf213a84f00000000, + 0xbe3b3eec00000000, 0xad5a8d6600000000, 0xe1721bc500000000, + 0x740cd0fa00000000, 0x3824465900000000, 0x5ef1468500000000, + 0x12d9d02600000000, 0x87a71b1900000000, 0xcb8f8dba00000000, + 0x0a0b6b7a00000000, 0x4623fdd900000000, 0xd35d36e600000000, + 0x9f75a04500000000, 0xf9a0a09900000000, 0xb588363a00000000, + 0x20f6fd0500000000, 0x6cde6ba600000000, 0xe3f9415f00000000, + 0xafd1d7fc00000000, 0x3aaf1cc300000000, 0x76878a6000000000, + 0x10528abc00000000, 0x5c7a1c1f00000000, 0xc904d72000000000, + 0x852c418300000000, 0x44a8a74300000000, 0x088031e000000000, + 0x9dfefadf00000000, 0xd1d66c7c00000000, 0xb7036ca000000000, + 0xfb2bfa0300000000, 0x6e55313c00000000, 0x227da79f00000000, + 0x099126f200000000, 0x45b9b05100000000, 0xd0c77b6e00000000, + 0x9cefedcd00000000, 0xfa3aed1100000000, 0xb6127bb200000000, + 0x236cb08d00000000, 0x6f44262e00000000, 0xaec0c0ee00000000, + 0xe2e8564d00000000, 0x77969d7200000000, 0x3bbe0bd100000000, + 0x5d6b0b0d00000000, 0x11439dae00000000, 0x843d569100000000, + 0xc815c03200000000, 0x4732eacb00000000, 0x0b1a7c6800000000, + 0x9e64b75700000000, 0xd24c21f400000000, 0xb499212800000000, + 0xf8b1b78b00000000, 0x6dcf7cb400000000, 0x21e7ea1700000000, + 0xe0630cd700000000, 0xac4b9a7400000000, 0x3935514b00000000, + 0x751dc7e800000000, 0x13c8c73400000000, 0x5fe0519700000000, + 0xca9e9aa800000000, 0x86b60c0b00000000, 0x95d7bf8100000000, + 0xd9ff292200000000, 0x4c81e21d00000000, 0x00a974be00000000, + 0x667c746200000000, 0x2a54e2c100000000, 0xbf2a29fe00000000, + 0xf302bf5d00000000, 0x3286599d00000000, 0x7eaecf3e00000000, + 0xebd0040100000000, 0xa7f892a200000000, 0xc12d927e00000000, + 0x8d0504dd00000000, 0x187bcfe200000000, 0x5453594100000000, + 0xdb7473b800000000, 0x975ce51b00000000, 0x02222e2400000000, + 0x4e0ab88700000000, 0x28dfb85b00000000, 0x64f72ef800000000, + 0xf189e5c700000000, 0xbda1736400000000, 0x7c2595a400000000, + 0x300d030700000000, 0xa573c83800000000, 0xe95b5e9b00000000, + 0x8f8e5e4700000000, 0xc3a6c8e400000000, 0x56d803db00000000, + 0x1af0957800000000}, + {0x0000000000000000, 0x939bc97f00000000, 0x263793ff00000000, + 0xb5ac5a8000000000, 0x0d68572400000000, 0x9ef39e5b00000000, + 0x2b5fc4db00000000, 0xb8c40da400000000, 0x1ad0ae4800000000, + 0x894b673700000000, 0x3ce73db700000000, 0xaf7cf4c800000000, + 0x17b8f96c00000000, 0x8423301300000000, 0x318f6a9300000000, + 0xa214a3ec00000000, 0x34a05d9100000000, 0xa73b94ee00000000, + 0x1297ce6e00000000, 0x810c071100000000, 0x39c80ab500000000, + 0xaa53c3ca00000000, 0x1fff994a00000000, 0x8c64503500000000, + 0x2e70f3d900000000, 0xbdeb3aa600000000, 0x0847602600000000, + 0x9bdca95900000000, 0x2318a4fd00000000, 0xb0836d8200000000, + 0x052f370200000000, 0x96b4fe7d00000000, 0x2946caf900000000, + 0xbadd038600000000, 0x0f71590600000000, 0x9cea907900000000, + 0x242e9ddd00000000, 0xb7b554a200000000, 0x02190e2200000000, + 0x9182c75d00000000, 0x339664b100000000, 0xa00dadce00000000, + 0x15a1f74e00000000, 0x863a3e3100000000, 0x3efe339500000000, + 0xad65faea00000000, 0x18c9a06a00000000, 0x8b52691500000000, + 0x1de6976800000000, 0x8e7d5e1700000000, 0x3bd1049700000000, + 0xa84acde800000000, 0x108ec04c00000000, 0x8315093300000000, + 0x36b953b300000000, 0xa5229acc00000000, 0x0736392000000000, + 0x94adf05f00000000, 0x2101aadf00000000, 0xb29a63a000000000, + 0x0a5e6e0400000000, 0x99c5a77b00000000, 0x2c69fdfb00000000, + 0xbff2348400000000, 0x138ae52800000000, 0x80112c5700000000, + 0x35bd76d700000000, 0xa626bfa800000000, 0x1ee2b20c00000000, + 0x8d797b7300000000, 0x38d521f300000000, 0xab4ee88c00000000, + 0x095a4b6000000000, 0x9ac1821f00000000, 0x2f6dd89f00000000, + 0xbcf611e000000000, 0x04321c4400000000, 0x97a9d53b00000000, + 0x22058fbb00000000, 0xb19e46c400000000, 0x272ab8b900000000, + 0xb4b171c600000000, 0x011d2b4600000000, 0x9286e23900000000, + 0x2a42ef9d00000000, 0xb9d926e200000000, 0x0c757c6200000000, + 0x9feeb51d00000000, 0x3dfa16f100000000, 0xae61df8e00000000, + 0x1bcd850e00000000, 0x88564c7100000000, 0x309241d500000000, + 0xa30988aa00000000, 0x16a5d22a00000000, 0x853e1b5500000000, + 0x3acc2fd100000000, 0xa957e6ae00000000, 0x1cfbbc2e00000000, + 0x8f60755100000000, 0x37a478f500000000, 0xa43fb18a00000000, + 0x1193eb0a00000000, 0x8208227500000000, 0x201c819900000000, + 0xb38748e600000000, 0x062b126600000000, 0x95b0db1900000000, + 0x2d74d6bd00000000, 0xbeef1fc200000000, 0x0b43454200000000, + 0x98d88c3d00000000, 0x0e6c724000000000, 0x9df7bb3f00000000, + 0x285be1bf00000000, 0xbbc028c000000000, 0x0304256400000000, + 0x909fec1b00000000, 0x2533b69b00000000, 0xb6a87fe400000000, + 0x14bcdc0800000000, 0x8727157700000000, 0x328b4ff700000000, + 0xa110868800000000, 0x19d48b2c00000000, 0x8a4f425300000000, + 0x3fe318d300000000, 0xac78d1ac00000000, 0x2614cb5100000000, + 0xb58f022e00000000, 0x002358ae00000000, 0x93b891d100000000, + 0x2b7c9c7500000000, 0xb8e7550a00000000, 0x0d4b0f8a00000000, + 0x9ed0c6f500000000, 0x3cc4651900000000, 0xaf5fac6600000000, + 0x1af3f6e600000000, 0x89683f9900000000, 0x31ac323d00000000, + 0xa237fb4200000000, 0x179ba1c200000000, 0x840068bd00000000, + 0x12b496c000000000, 0x812f5fbf00000000, 0x3483053f00000000, + 0xa718cc4000000000, 0x1fdcc1e400000000, 0x8c47089b00000000, + 0x39eb521b00000000, 0xaa709b6400000000, 0x0864388800000000, + 0x9bfff1f700000000, 0x2e53ab7700000000, 0xbdc8620800000000, + 0x050c6fac00000000, 0x9697a6d300000000, 0x233bfc5300000000, + 0xb0a0352c00000000, 0x0f5201a800000000, 0x9cc9c8d700000000, + 0x2965925700000000, 0xbafe5b2800000000, 0x023a568c00000000, + 0x91a19ff300000000, 0x240dc57300000000, 0xb7960c0c00000000, + 0x1582afe000000000, 0x8619669f00000000, 0x33b53c1f00000000, + 0xa02ef56000000000, 0x18eaf8c400000000, 0x8b7131bb00000000, + 0x3edd6b3b00000000, 0xad46a24400000000, 0x3bf25c3900000000, + 0xa869954600000000, 0x1dc5cfc600000000, 0x8e5e06b900000000, + 0x369a0b1d00000000, 0xa501c26200000000, 0x10ad98e200000000, + 0x8336519d00000000, 0x2122f27100000000, 0xb2b93b0e00000000, + 0x0715618e00000000, 0x948ea8f100000000, 0x2c4aa55500000000, + 0xbfd16c2a00000000, 0x0a7d36aa00000000, 0x99e6ffd500000000, + 0x359e2e7900000000, 0xa605e70600000000, 0x13a9bd8600000000, + 0x803274f900000000, 0x38f6795d00000000, 0xab6db02200000000, + 0x1ec1eaa200000000, 0x8d5a23dd00000000, 0x2f4e803100000000, + 0xbcd5494e00000000, 0x097913ce00000000, 0x9ae2dab100000000, + 0x2226d71500000000, 0xb1bd1e6a00000000, 0x041144ea00000000, + 0x978a8d9500000000, 0x013e73e800000000, 0x92a5ba9700000000, + 0x2709e01700000000, 0xb492296800000000, 0x0c5624cc00000000, + 0x9fcdedb300000000, 0x2a61b73300000000, 0xb9fa7e4c00000000, + 0x1beedda000000000, 0x887514df00000000, 0x3dd94e5f00000000, + 0xae42872000000000, 0x16868a8400000000, 0x851d43fb00000000, + 0x30b1197b00000000, 0xa32ad00400000000, 0x1cd8e48000000000, + 0x8f432dff00000000, 0x3aef777f00000000, 0xa974be0000000000, + 0x11b0b3a400000000, 0x822b7adb00000000, 0x3787205b00000000, + 0xa41ce92400000000, 0x06084ac800000000, 0x959383b700000000, + 0x203fd93700000000, 0xb3a4104800000000, 0x0b601dec00000000, + 0x98fbd49300000000, 0x2d578e1300000000, 0xbecc476c00000000, + 0x2878b91100000000, 0xbbe3706e00000000, 0x0e4f2aee00000000, + 0x9dd4e39100000000, 0x2510ee3500000000, 0xb68b274a00000000, + 0x03277dca00000000, 0x90bcb4b500000000, 0x32a8175900000000, + 0xa133de2600000000, 0x149f84a600000000, 0x87044dd900000000, + 0x3fc0407d00000000, 0xac5b890200000000, 0x19f7d38200000000, + 0x8a6c1afd00000000}, + {0x0000000000000000, 0x650b796900000000, 0xca16f2d200000000, + 0xaf1d8bbb00000000, 0xd52b957e00000000, 0xb020ec1700000000, + 0x1f3d67ac00000000, 0x7a361ec500000000, 0xaa572afd00000000, + 0xcf5c539400000000, 0x6041d82f00000000, 0x054aa14600000000, + 0x7f7cbf8300000000, 0x1a77c6ea00000000, 0xb56a4d5100000000, + 0xd061343800000000, 0x15a9252100000000, 0x70a25c4800000000, + 0xdfbfd7f300000000, 0xbab4ae9a00000000, 0xc082b05f00000000, + 0xa589c93600000000, 0x0a94428d00000000, 0x6f9f3be400000000, + 0xbffe0fdc00000000, 0xdaf576b500000000, 0x75e8fd0e00000000, + 0x10e3846700000000, 0x6ad59aa200000000, 0x0fdee3cb00000000, + 0xa0c3687000000000, 0xc5c8111900000000, 0x2a524b4200000000, + 0x4f59322b00000000, 0xe044b99000000000, 0x854fc0f900000000, + 0xff79de3c00000000, 0x9a72a75500000000, 0x356f2cee00000000, + 0x5064558700000000, 0x800561bf00000000, 0xe50e18d600000000, + 0x4a13936d00000000, 0x2f18ea0400000000, 0x552ef4c100000000, + 0x30258da800000000, 0x9f38061300000000, 0xfa337f7a00000000, + 0x3ffb6e6300000000, 0x5af0170a00000000, 0xf5ed9cb100000000, + 0x90e6e5d800000000, 0xead0fb1d00000000, 0x8fdb827400000000, + 0x20c609cf00000000, 0x45cd70a600000000, 0x95ac449e00000000, + 0xf0a73df700000000, 0x5fbab64c00000000, 0x3ab1cf2500000000, + 0x4087d1e000000000, 0x258ca88900000000, 0x8a91233200000000, + 0xef9a5a5b00000000, 0x54a4968400000000, 0x31afefed00000000, + 0x9eb2645600000000, 0xfbb91d3f00000000, 0x818f03fa00000000, + 0xe4847a9300000000, 0x4b99f12800000000, 0x2e92884100000000, + 0xfef3bc7900000000, 0x9bf8c51000000000, 0x34e54eab00000000, + 0x51ee37c200000000, 0x2bd8290700000000, 0x4ed3506e00000000, + 0xe1cedbd500000000, 0x84c5a2bc00000000, 0x410db3a500000000, + 0x2406cacc00000000, 0x8b1b417700000000, 0xee10381e00000000, + 0x942626db00000000, 0xf12d5fb200000000, 0x5e30d40900000000, + 0x3b3bad6000000000, 0xeb5a995800000000, 0x8e51e03100000000, + 0x214c6b8a00000000, 0x444712e300000000, 0x3e710c2600000000, + 0x5b7a754f00000000, 0xf467fef400000000, 0x916c879d00000000, + 0x7ef6ddc600000000, 0x1bfda4af00000000, 0xb4e02f1400000000, + 0xd1eb567d00000000, 0xabdd48b800000000, 0xced631d100000000, + 0x61cbba6a00000000, 0x04c0c30300000000, 0xd4a1f73b00000000, + 0xb1aa8e5200000000, 0x1eb705e900000000, 0x7bbc7c8000000000, + 0x018a624500000000, 0x64811b2c00000000, 0xcb9c909700000000, + 0xae97e9fe00000000, 0x6b5ff8e700000000, 0x0e54818e00000000, + 0xa1490a3500000000, 0xc442735c00000000, 0xbe746d9900000000, + 0xdb7f14f000000000, 0x74629f4b00000000, 0x1169e62200000000, + 0xc108d21a00000000, 0xa403ab7300000000, 0x0b1e20c800000000, + 0x6e1559a100000000, 0x1423476400000000, 0x71283e0d00000000, + 0xde35b5b600000000, 0xbb3eccdf00000000, 0xe94e5cd200000000, + 0x8c4525bb00000000, 0x2358ae0000000000, 0x4653d76900000000, + 0x3c65c9ac00000000, 0x596eb0c500000000, 0xf6733b7e00000000, + 0x9378421700000000, 0x4319762f00000000, 0x26120f4600000000, + 0x890f84fd00000000, 0xec04fd9400000000, 0x9632e35100000000, + 0xf3399a3800000000, 0x5c24118300000000, 0x392f68ea00000000, + 0xfce779f300000000, 0x99ec009a00000000, 0x36f18b2100000000, + 0x53faf24800000000, 0x29ccec8d00000000, 0x4cc795e400000000, + 0xe3da1e5f00000000, 0x86d1673600000000, 0x56b0530e00000000, + 0x33bb2a6700000000, 0x9ca6a1dc00000000, 0xf9add8b500000000, + 0x839bc67000000000, 0xe690bf1900000000, 0x498d34a200000000, + 0x2c864dcb00000000, 0xc31c179000000000, 0xa6176ef900000000, + 0x090ae54200000000, 0x6c019c2b00000000, 0x163782ee00000000, + 0x733cfb8700000000, 0xdc21703c00000000, 0xb92a095500000000, + 0x694b3d6d00000000, 0x0c40440400000000, 0xa35dcfbf00000000, + 0xc656b6d600000000, 0xbc60a81300000000, 0xd96bd17a00000000, + 0x76765ac100000000, 0x137d23a800000000, 0xd6b532b100000000, + 0xb3be4bd800000000, 0x1ca3c06300000000, 0x79a8b90a00000000, + 0x039ea7cf00000000, 0x6695dea600000000, 0xc988551d00000000, + 0xac832c7400000000, 0x7ce2184c00000000, 0x19e9612500000000, + 0xb6f4ea9e00000000, 0xd3ff93f700000000, 0xa9c98d3200000000, + 0xccc2f45b00000000, 0x63df7fe000000000, 0x06d4068900000000, + 0xbdeaca5600000000, 0xd8e1b33f00000000, 0x77fc388400000000, + 0x12f741ed00000000, 0x68c15f2800000000, 0x0dca264100000000, + 0xa2d7adfa00000000, 0xc7dcd49300000000, 0x17bde0ab00000000, + 0x72b699c200000000, 0xddab127900000000, 0xb8a06b1000000000, + 0xc29675d500000000, 0xa79d0cbc00000000, 0x0880870700000000, + 0x6d8bfe6e00000000, 0xa843ef7700000000, 0xcd48961e00000000, + 0x62551da500000000, 0x075e64cc00000000, 0x7d687a0900000000, + 0x1863036000000000, 0xb77e88db00000000, 0xd275f1b200000000, + 0x0214c58a00000000, 0x671fbce300000000, 0xc802375800000000, + 0xad094e3100000000, 0xd73f50f400000000, 0xb234299d00000000, + 0x1d29a22600000000, 0x7822db4f00000000, 0x97b8811400000000, + 0xf2b3f87d00000000, 0x5dae73c600000000, 0x38a50aaf00000000, + 0x4293146a00000000, 0x27986d0300000000, 0x8885e6b800000000, + 0xed8e9fd100000000, 0x3defabe900000000, 0x58e4d28000000000, + 0xf7f9593b00000000, 0x92f2205200000000, 0xe8c43e9700000000, + 0x8dcf47fe00000000, 0x22d2cc4500000000, 0x47d9b52c00000000, + 0x8211a43500000000, 0xe71add5c00000000, 0x480756e700000000, + 0x2d0c2f8e00000000, 0x573a314b00000000, 0x3231482200000000, + 0x9d2cc39900000000, 0xf827baf000000000, 0x28468ec800000000, + 0x4d4df7a100000000, 0xe2507c1a00000000, 0x875b057300000000, + 0xfd6d1bb600000000, 0x986662df00000000, 0x377be96400000000, + 0x5270900d00000000}, + {0x0000000000000000, 0xdcecb13d00000000, 0xb8d9637b00000000, + 0x6435d24600000000, 0x70b3c7f600000000, 0xac5f76cb00000000, + 0xc86aa48d00000000, 0x148615b000000000, 0xa160fe3600000000, + 0x7d8c4f0b00000000, 0x19b99d4d00000000, 0xc5552c7000000000, + 0xd1d339c000000000, 0x0d3f88fd00000000, 0x690a5abb00000000, + 0xb5e6eb8600000000, 0x42c1fc6d00000000, 0x9e2d4d5000000000, + 0xfa189f1600000000, 0x26f42e2b00000000, 0x32723b9b00000000, + 0xee9e8aa600000000, 0x8aab58e000000000, 0x5647e9dd00000000, + 0xe3a1025b00000000, 0x3f4db36600000000, 0x5b78612000000000, + 0x8794d01d00000000, 0x9312c5ad00000000, 0x4ffe749000000000, + 0x2bcba6d600000000, 0xf72717eb00000000, 0x8482f9db00000000, + 0x586e48e600000000, 0x3c5b9aa000000000, 0xe0b72b9d00000000, + 0xf4313e2d00000000, 0x28dd8f1000000000, 0x4ce85d5600000000, + 0x9004ec6b00000000, 0x25e207ed00000000, 0xf90eb6d000000000, + 0x9d3b649600000000, 0x41d7d5ab00000000, 0x5551c01b00000000, + 0x89bd712600000000, 0xed88a36000000000, 0x3164125d00000000, + 0xc64305b600000000, 0x1aafb48b00000000, 0x7e9a66cd00000000, + 0xa276d7f000000000, 0xb6f0c24000000000, 0x6a1c737d00000000, + 0x0e29a13b00000000, 0xd2c5100600000000, 0x6723fb8000000000, + 0xbbcf4abd00000000, 0xdffa98fb00000000, 0x031629c600000000, + 0x17903c7600000000, 0xcb7c8d4b00000000, 0xaf495f0d00000000, + 0x73a5ee3000000000, 0x4903826c00000000, 0x95ef335100000000, + 0xf1dae11700000000, 0x2d36502a00000000, 0x39b0459a00000000, + 0xe55cf4a700000000, 0x816926e100000000, 0x5d8597dc00000000, + 0xe8637c5a00000000, 0x348fcd6700000000, 0x50ba1f2100000000, + 0x8c56ae1c00000000, 0x98d0bbac00000000, 0x443c0a9100000000, + 0x2009d8d700000000, 0xfce569ea00000000, 0x0bc27e0100000000, + 0xd72ecf3c00000000, 0xb31b1d7a00000000, 0x6ff7ac4700000000, + 0x7b71b9f700000000, 0xa79d08ca00000000, 0xc3a8da8c00000000, + 0x1f446bb100000000, 0xaaa2803700000000, 0x764e310a00000000, + 0x127be34c00000000, 0xce97527100000000, 0xda1147c100000000, + 0x06fdf6fc00000000, 0x62c824ba00000000, 0xbe24958700000000, + 0xcd817bb700000000, 0x116dca8a00000000, 0x755818cc00000000, + 0xa9b4a9f100000000, 0xbd32bc4100000000, 0x61de0d7c00000000, + 0x05ebdf3a00000000, 0xd9076e0700000000, 0x6ce1858100000000, + 0xb00d34bc00000000, 0xd438e6fa00000000, 0x08d457c700000000, + 0x1c52427700000000, 0xc0bef34a00000000, 0xa48b210c00000000, + 0x7867903100000000, 0x8f4087da00000000, 0x53ac36e700000000, + 0x3799e4a100000000, 0xeb75559c00000000, 0xfff3402c00000000, + 0x231ff11100000000, 0x472a235700000000, 0x9bc6926a00000000, + 0x2e2079ec00000000, 0xf2ccc8d100000000, 0x96f91a9700000000, + 0x4a15abaa00000000, 0x5e93be1a00000000, 0x827f0f2700000000, + 0xe64add6100000000, 0x3aa66c5c00000000, 0x920604d900000000, + 0x4eeab5e400000000, 0x2adf67a200000000, 0xf633d69f00000000, + 0xe2b5c32f00000000, 0x3e59721200000000, 0x5a6ca05400000000, + 0x8680116900000000, 0x3366faef00000000, 0xef8a4bd200000000, + 0x8bbf999400000000, 0x575328a900000000, 0x43d53d1900000000, + 0x9f398c2400000000, 0xfb0c5e6200000000, 0x27e0ef5f00000000, + 0xd0c7f8b400000000, 0x0c2b498900000000, 0x681e9bcf00000000, + 0xb4f22af200000000, 0xa0743f4200000000, 0x7c988e7f00000000, + 0x18ad5c3900000000, 0xc441ed0400000000, 0x71a7068200000000, + 0xad4bb7bf00000000, 0xc97e65f900000000, 0x1592d4c400000000, + 0x0114c17400000000, 0xddf8704900000000, 0xb9cda20f00000000, + 0x6521133200000000, 0x1684fd0200000000, 0xca684c3f00000000, + 0xae5d9e7900000000, 0x72b12f4400000000, 0x66373af400000000, + 0xbadb8bc900000000, 0xdeee598f00000000, 0x0202e8b200000000, + 0xb7e4033400000000, 0x6b08b20900000000, 0x0f3d604f00000000, + 0xd3d1d17200000000, 0xc757c4c200000000, 0x1bbb75ff00000000, + 0x7f8ea7b900000000, 0xa362168400000000, 0x5445016f00000000, + 0x88a9b05200000000, 0xec9c621400000000, 0x3070d32900000000, + 0x24f6c69900000000, 0xf81a77a400000000, 0x9c2fa5e200000000, + 0x40c314df00000000, 0xf525ff5900000000, 0x29c94e6400000000, + 0x4dfc9c2200000000, 0x91102d1f00000000, 0x859638af00000000, + 0x597a899200000000, 0x3d4f5bd400000000, 0xe1a3eae900000000, + 0xdb0586b500000000, 0x07e9378800000000, 0x63dce5ce00000000, + 0xbf3054f300000000, 0xabb6414300000000, 0x775af07e00000000, + 0x136f223800000000, 0xcf83930500000000, 0x7a65788300000000, + 0xa689c9be00000000, 0xc2bc1bf800000000, 0x1e50aac500000000, + 0x0ad6bf7500000000, 0xd63a0e4800000000, 0xb20fdc0e00000000, + 0x6ee36d3300000000, 0x99c47ad800000000, 0x4528cbe500000000, + 0x211d19a300000000, 0xfdf1a89e00000000, 0xe977bd2e00000000, + 0x359b0c1300000000, 0x51aede5500000000, 0x8d426f6800000000, + 0x38a484ee00000000, 0xe44835d300000000, 0x807de79500000000, + 0x5c9156a800000000, 0x4817431800000000, 0x94fbf22500000000, + 0xf0ce206300000000, 0x2c22915e00000000, 0x5f877f6e00000000, + 0x836bce5300000000, 0xe75e1c1500000000, 0x3bb2ad2800000000, + 0x2f34b89800000000, 0xf3d809a500000000, 0x97eddbe300000000, + 0x4b016ade00000000, 0xfee7815800000000, 0x220b306500000000, + 0x463ee22300000000, 0x9ad2531e00000000, 0x8e5446ae00000000, + 0x52b8f79300000000, 0x368d25d500000000, 0xea6194e800000000, + 0x1d46830300000000, 0xc1aa323e00000000, 0xa59fe07800000000, + 0x7973514500000000, 0x6df544f500000000, 0xb119f5c800000000, + 0xd52c278e00000000, 0x09c096b300000000, 0xbc267d3500000000, + 0x60cacc0800000000, 0x04ff1e4e00000000, 0xd813af7300000000, + 0xcc95bac300000000, 0x10790bfe00000000, 0x744cd9b800000000, + 0xa8a0688500000000}}; + +#else /* W == 4 */ + +local const z_crc_t FAR crc_braid_table[][256] = { + {0x00000000, 0x81256527, 0xd93bcc0f, 0x581ea928, 0x69069e5f, + 0xe823fb78, 0xb03d5250, 0x31183777, 0xd20d3cbe, 0x53285999, + 0x0b36f0b1, 0x8a139596, 0xbb0ba2e1, 0x3a2ec7c6, 0x62306eee, + 0xe3150bc9, 0x7f6b7f3d, 0xfe4e1a1a, 0xa650b332, 0x2775d615, + 0x166de162, 0x97488445, 0xcf562d6d, 0x4e73484a, 0xad664383, + 0x2c4326a4, 0x745d8f8c, 0xf578eaab, 0xc460dddc, 0x4545b8fb, + 0x1d5b11d3, 0x9c7e74f4, 0xfed6fe7a, 0x7ff39b5d, 0x27ed3275, + 0xa6c85752, 0x97d06025, 0x16f50502, 0x4eebac2a, 0xcfcec90d, + 0x2cdbc2c4, 0xadfea7e3, 0xf5e00ecb, 0x74c56bec, 0x45dd5c9b, + 0xc4f839bc, 0x9ce69094, 0x1dc3f5b3, 0x81bd8147, 0x0098e460, + 0x58864d48, 0xd9a3286f, 0xe8bb1f18, 0x699e7a3f, 0x3180d317, + 0xb0a5b630, 0x53b0bdf9, 0xd295d8de, 0x8a8b71f6, 0x0bae14d1, + 0x3ab623a6, 0xbb934681, 0xe38defa9, 0x62a88a8e, 0x26dcfab5, + 0xa7f99f92, 0xffe736ba, 0x7ec2539d, 0x4fda64ea, 0xceff01cd, + 0x96e1a8e5, 0x17c4cdc2, 0xf4d1c60b, 0x75f4a32c, 0x2dea0a04, + 0xaccf6f23, 0x9dd75854, 0x1cf23d73, 0x44ec945b, 0xc5c9f17c, + 0x59b78588, 0xd892e0af, 0x808c4987, 0x01a92ca0, 0x30b11bd7, + 0xb1947ef0, 0xe98ad7d8, 0x68afb2ff, 0x8bbab936, 0x0a9fdc11, + 0x52817539, 0xd3a4101e, 0xe2bc2769, 0x6399424e, 0x3b87eb66, + 0xbaa28e41, 0xd80a04cf, 0x592f61e8, 0x0131c8c0, 0x8014ade7, + 0xb10c9a90, 0x3029ffb7, 0x6837569f, 0xe91233b8, 0x0a073871, + 0x8b225d56, 0xd33cf47e, 0x52199159, 0x6301a62e, 0xe224c309, + 0xba3a6a21, 0x3b1f0f06, 0xa7617bf2, 0x26441ed5, 0x7e5ab7fd, + 0xff7fd2da, 0xce67e5ad, 0x4f42808a, 0x175c29a2, 0x96794c85, + 0x756c474c, 0xf449226b, 0xac578b43, 0x2d72ee64, 0x1c6ad913, + 0x9d4fbc34, 0xc551151c, 0x4474703b, 0x4db9f56a, 0xcc9c904d, + 0x94823965, 0x15a75c42, 0x24bf6b35, 0xa59a0e12, 0xfd84a73a, + 0x7ca1c21d, 0x9fb4c9d4, 0x1e91acf3, 0x468f05db, 0xc7aa60fc, + 0xf6b2578b, 0x779732ac, 0x2f899b84, 0xaeacfea3, 0x32d28a57, + 0xb3f7ef70, 0xebe94658, 0x6acc237f, 0x5bd41408, 0xdaf1712f, + 0x82efd807, 0x03cabd20, 0xe0dfb6e9, 0x61fad3ce, 0x39e47ae6, + 0xb8c11fc1, 0x89d928b6, 0x08fc4d91, 0x50e2e4b9, 0xd1c7819e, + 0xb36f0b10, 0x324a6e37, 0x6a54c71f, 0xeb71a238, 0xda69954f, + 0x5b4cf068, 0x03525940, 0x82773c67, 0x616237ae, 0xe0475289, + 0xb859fba1, 0x397c9e86, 0x0864a9f1, 0x8941ccd6, 0xd15f65fe, + 0x507a00d9, 0xcc04742d, 0x4d21110a, 0x153fb822, 0x941add05, + 0xa502ea72, 0x24278f55, 0x7c39267d, 0xfd1c435a, 0x1e094893, + 0x9f2c2db4, 0xc732849c, 0x4617e1bb, 0x770fd6cc, 0xf62ab3eb, + 0xae341ac3, 0x2f117fe4, 0x6b650fdf, 0xea406af8, 0xb25ec3d0, + 0x337ba6f7, 0x02639180, 0x8346f4a7, 0xdb585d8f, 0x5a7d38a8, + 0xb9683361, 0x384d5646, 0x6053ff6e, 0xe1769a49, 0xd06ead3e, + 0x514bc819, 0x09556131, 0x88700416, 0x140e70e2, 0x952b15c5, + 0xcd35bced, 0x4c10d9ca, 0x7d08eebd, 0xfc2d8b9a, 0xa43322b2, + 0x25164795, 0xc6034c5c, 0x4726297b, 0x1f388053, 0x9e1de574, + 0xaf05d203, 0x2e20b724, 0x763e1e0c, 0xf71b7b2b, 0x95b3f1a5, + 0x14969482, 0x4c883daa, 0xcdad588d, 0xfcb56ffa, 0x7d900add, + 0x258ea3f5, 0xa4abc6d2, 0x47becd1b, 0xc69ba83c, 0x9e850114, + 0x1fa06433, 0x2eb85344, 0xaf9d3663, 0xf7839f4b, 0x76a6fa6c, + 0xead88e98, 0x6bfdebbf, 0x33e34297, 0xb2c627b0, 0x83de10c7, + 0x02fb75e0, 0x5ae5dcc8, 0xdbc0b9ef, 0x38d5b226, 0xb9f0d701, + 0xe1ee7e29, 0x60cb1b0e, 0x51d32c79, 0xd0f6495e, 0x88e8e076, + 0x09cd8551}, + {0x00000000, 0x9b73ead4, 0xed96d3e9, 0x76e5393d, 0x005ca193, + 0x9b2f4b47, 0xedca727a, 0x76b998ae, 0x00b94326, 0x9bcaa9f2, + 0xed2f90cf, 0x765c7a1b, 0x00e5e2b5, 0x9b960861, 0xed73315c, + 0x7600db88, 0x0172864c, 0x9a016c98, 0xece455a5, 0x7797bf71, + 0x012e27df, 0x9a5dcd0b, 0xecb8f436, 0x77cb1ee2, 0x01cbc56a, + 0x9ab82fbe, 0xec5d1683, 0x772efc57, 0x019764f9, 0x9ae48e2d, + 0xec01b710, 0x77725dc4, 0x02e50c98, 0x9996e64c, 0xef73df71, + 0x740035a5, 0x02b9ad0b, 0x99ca47df, 0xef2f7ee2, 0x745c9436, + 0x025c4fbe, 0x992fa56a, 0xefca9c57, 0x74b97683, 0x0200ee2d, + 0x997304f9, 0xef963dc4, 0x74e5d710, 0x03978ad4, 0x98e46000, + 0xee01593d, 0x7572b3e9, 0x03cb2b47, 0x98b8c193, 0xee5df8ae, + 0x752e127a, 0x032ec9f2, 0x985d2326, 0xeeb81a1b, 0x75cbf0cf, + 0x03726861, 0x980182b5, 0xeee4bb88, 0x7597515c, 0x05ca1930, + 0x9eb9f3e4, 0xe85ccad9, 0x732f200d, 0x0596b8a3, 0x9ee55277, + 0xe8006b4a, 0x7373819e, 0x05735a16, 0x9e00b0c2, 0xe8e589ff, + 0x7396632b, 0x052ffb85, 0x9e5c1151, 0xe8b9286c, 0x73cac2b8, + 0x04b89f7c, 0x9fcb75a8, 0xe92e4c95, 0x725da641, 0x04e43eef, + 0x9f97d43b, 0xe972ed06, 0x720107d2, 0x0401dc5a, 0x9f72368e, + 0xe9970fb3, 0x72e4e567, 0x045d7dc9, 0x9f2e971d, 0xe9cbae20, + 0x72b844f4, 0x072f15a8, 0x9c5cff7c, 0xeab9c641, 0x71ca2c95, + 0x0773b43b, 0x9c005eef, 0xeae567d2, 0x71968d06, 0x0796568e, + 0x9ce5bc5a, 0xea008567, 0x71736fb3, 0x07caf71d, 0x9cb91dc9, + 0xea5c24f4, 0x712fce20, 0x065d93e4, 0x9d2e7930, 0xebcb400d, + 0x70b8aad9, 0x06013277, 0x9d72d8a3, 0xeb97e19e, 0x70e40b4a, + 0x06e4d0c2, 0x9d973a16, 0xeb72032b, 0x7001e9ff, 0x06b87151, + 0x9dcb9b85, 0xeb2ea2b8, 0x705d486c, 0x0b943260, 0x90e7d8b4, + 0xe602e189, 0x7d710b5d, 0x0bc893f3, 0x90bb7927, 0xe65e401a, + 0x7d2daace, 0x0b2d7146, 0x905e9b92, 0xe6bba2af, 0x7dc8487b, + 0x0b71d0d5, 0x90023a01, 0xe6e7033c, 0x7d94e9e8, 0x0ae6b42c, + 0x91955ef8, 0xe77067c5, 0x7c038d11, 0x0aba15bf, 0x91c9ff6b, + 0xe72cc656, 0x7c5f2c82, 0x0a5ff70a, 0x912c1dde, 0xe7c924e3, + 0x7cbace37, 0x0a035699, 0x9170bc4d, 0xe7958570, 0x7ce66fa4, + 0x09713ef8, 0x9202d42c, 0xe4e7ed11, 0x7f9407c5, 0x092d9f6b, + 0x925e75bf, 0xe4bb4c82, 0x7fc8a656, 0x09c87dde, 0x92bb970a, + 0xe45eae37, 0x7f2d44e3, 0x0994dc4d, 0x92e73699, 0xe4020fa4, + 0x7f71e570, 0x0803b8b4, 0x93705260, 0xe5956b5d, 0x7ee68189, + 0x085f1927, 0x932cf3f3, 0xe5c9cace, 0x7eba201a, 0x08bafb92, + 0x93c91146, 0xe52c287b, 0x7e5fc2af, 0x08e65a01, 0x9395b0d5, + 0xe57089e8, 0x7e03633c, 0x0e5e2b50, 0x952dc184, 0xe3c8f8b9, + 0x78bb126d, 0x0e028ac3, 0x95716017, 0xe394592a, 0x78e7b3fe, + 0x0ee76876, 0x959482a2, 0xe371bb9f, 0x7802514b, 0x0ebbc9e5, + 0x95c82331, 0xe32d1a0c, 0x785ef0d8, 0x0f2cad1c, 0x945f47c8, + 0xe2ba7ef5, 0x79c99421, 0x0f700c8f, 0x9403e65b, 0xe2e6df66, + 0x799535b2, 0x0f95ee3a, 0x94e604ee, 0xe2033dd3, 0x7970d707, + 0x0fc94fa9, 0x94baa57d, 0xe25f9c40, 0x792c7694, 0x0cbb27c8, + 0x97c8cd1c, 0xe12df421, 0x7a5e1ef5, 0x0ce7865b, 0x97946c8f, + 0xe17155b2, 0x7a02bf66, 0x0c0264ee, 0x97718e3a, 0xe194b707, + 0x7ae75dd3, 0x0c5ec57d, 0x972d2fa9, 0xe1c81694, 0x7abbfc40, + 0x0dc9a184, 0x96ba4b50, 0xe05f726d, 0x7b2c98b9, 0x0d950017, + 0x96e6eac3, 0xe003d3fe, 0x7b70392a, 0x0d70e2a2, 0x96030876, + 0xe0e6314b, 0x7b95db9f, 0x0d2c4331, 0x965fa9e5, 0xe0ba90d8, + 0x7bc97a0c}, + {0x00000000, 0x172864c0, 0x2e50c980, 0x3978ad40, 0x5ca19300, + 0x4b89f7c0, 0x72f15a80, 0x65d93e40, 0xb9432600, 0xae6b42c0, + 0x9713ef80, 0x803b8b40, 0xe5e2b500, 0xf2cad1c0, 0xcbb27c80, + 0xdc9a1840, 0xa9f74a41, 0xbedf2e81, 0x87a783c1, 0x908fe701, + 0xf556d941, 0xe27ebd81, 0xdb0610c1, 0xcc2e7401, 0x10b46c41, + 0x079c0881, 0x3ee4a5c1, 0x29ccc101, 0x4c15ff41, 0x5b3d9b81, + 0x624536c1, 0x756d5201, 0x889f92c3, 0x9fb7f603, 0xa6cf5b43, + 0xb1e73f83, 0xd43e01c3, 0xc3166503, 0xfa6ec843, 0xed46ac83, + 0x31dcb4c3, 0x26f4d003, 0x1f8c7d43, 0x08a41983, 0x6d7d27c3, + 0x7a554303, 0x432dee43, 0x54058a83, 0x2168d882, 0x3640bc42, + 0x0f381102, 0x181075c2, 0x7dc94b82, 0x6ae12f42, 0x53998202, + 0x44b1e6c2, 0x982bfe82, 0x8f039a42, 0xb67b3702, 0xa15353c2, + 0xc48a6d82, 0xd3a20942, 0xeadaa402, 0xfdf2c0c2, 0xca4e23c7, + 0xdd664707, 0xe41eea47, 0xf3368e87, 0x96efb0c7, 0x81c7d407, + 0xb8bf7947, 0xaf971d87, 0x730d05c7, 0x64256107, 0x5d5dcc47, + 0x4a75a887, 0x2fac96c7, 0x3884f207, 0x01fc5f47, 0x16d43b87, + 0x63b96986, 0x74910d46, 0x4de9a006, 0x5ac1c4c6, 0x3f18fa86, + 0x28309e46, 0x11483306, 0x066057c6, 0xdafa4f86, 0xcdd22b46, + 0xf4aa8606, 0xe382e2c6, 0x865bdc86, 0x9173b846, 0xa80b1506, + 0xbf2371c6, 0x42d1b104, 0x55f9d5c4, 0x6c817884, 0x7ba91c44, + 0x1e702204, 0x095846c4, 0x3020eb84, 0x27088f44, 0xfb929704, + 0xecbaf3c4, 0xd5c25e84, 0xc2ea3a44, 0xa7330404, 0xb01b60c4, + 0x8963cd84, 0x9e4ba944, 0xeb26fb45, 0xfc0e9f85, 0xc57632c5, + 0xd25e5605, 0xb7876845, 0xa0af0c85, 0x99d7a1c5, 0x8effc505, + 0x5265dd45, 0x454db985, 0x7c3514c5, 0x6b1d7005, 0x0ec44e45, + 0x19ec2a85, 0x209487c5, 0x37bce305, 0x4fed41cf, 0x58c5250f, + 0x61bd884f, 0x7695ec8f, 0x134cd2cf, 0x0464b60f, 0x3d1c1b4f, + 0x2a347f8f, 0xf6ae67cf, 0xe186030f, 0xd8feae4f, 0xcfd6ca8f, + 0xaa0ff4cf, 0xbd27900f, 0x845f3d4f, 0x9377598f, 0xe61a0b8e, + 0xf1326f4e, 0xc84ac20e, 0xdf62a6ce, 0xbabb988e, 0xad93fc4e, + 0x94eb510e, 0x83c335ce, 0x5f592d8e, 0x4871494e, 0x7109e40e, + 0x662180ce, 0x03f8be8e, 0x14d0da4e, 0x2da8770e, 0x3a8013ce, + 0xc772d30c, 0xd05ab7cc, 0xe9221a8c, 0xfe0a7e4c, 0x9bd3400c, + 0x8cfb24cc, 0xb583898c, 0xa2abed4c, 0x7e31f50c, 0x691991cc, + 0x50613c8c, 0x4749584c, 0x2290660c, 0x35b802cc, 0x0cc0af8c, + 0x1be8cb4c, 0x6e85994d, 0x79adfd8d, 0x40d550cd, 0x57fd340d, + 0x32240a4d, 0x250c6e8d, 0x1c74c3cd, 0x0b5ca70d, 0xd7c6bf4d, + 0xc0eedb8d, 0xf99676cd, 0xeebe120d, 0x8b672c4d, 0x9c4f488d, + 0xa537e5cd, 0xb21f810d, 0x85a36208, 0x928b06c8, 0xabf3ab88, + 0xbcdbcf48, 0xd902f108, 0xce2a95c8, 0xf7523888, 0xe07a5c48, + 0x3ce04408, 0x2bc820c8, 0x12b08d88, 0x0598e948, 0x6041d708, + 0x7769b3c8, 0x4e111e88, 0x59397a48, 0x2c542849, 0x3b7c4c89, + 0x0204e1c9, 0x152c8509, 0x70f5bb49, 0x67dddf89, 0x5ea572c9, + 0x498d1609, 0x95170e49, 0x823f6a89, 0xbb47c7c9, 0xac6fa309, + 0xc9b69d49, 0xde9ef989, 0xe7e654c9, 0xf0ce3009, 0x0d3cf0cb, + 0x1a14940b, 0x236c394b, 0x34445d8b, 0x519d63cb, 0x46b5070b, + 0x7fcdaa4b, 0x68e5ce8b, 0xb47fd6cb, 0xa357b20b, 0x9a2f1f4b, + 0x8d077b8b, 0xe8de45cb, 0xfff6210b, 0xc68e8c4b, 0xd1a6e88b, + 0xa4cbba8a, 0xb3e3de4a, 0x8a9b730a, 0x9db317ca, 0xf86a298a, + 0xef424d4a, 0xd63ae00a, 0xc11284ca, 0x1d889c8a, 0x0aa0f84a, + 0x33d8550a, 0x24f031ca, 0x41290f8a, 0x56016b4a, 0x6f79c60a, + 0x7851a2ca}, + {0x00000000, 0x9fda839e, 0xe4c4017d, 0x7b1e82e3, 0x12f904bb, + 0x8d238725, 0xf63d05c6, 0x69e78658, 0x25f20976, 0xba288ae8, + 0xc136080b, 0x5eec8b95, 0x370b0dcd, 0xa8d18e53, 0xd3cf0cb0, + 0x4c158f2e, 0x4be412ec, 0xd43e9172, 0xaf201391, 0x30fa900f, + 0x591d1657, 0xc6c795c9, 0xbdd9172a, 0x220394b4, 0x6e161b9a, + 0xf1cc9804, 0x8ad21ae7, 0x15089979, 0x7cef1f21, 0xe3359cbf, + 0x982b1e5c, 0x07f19dc2, 0x97c825d8, 0x0812a646, 0x730c24a5, + 0xecd6a73b, 0x85312163, 0x1aeba2fd, 0x61f5201e, 0xfe2fa380, + 0xb23a2cae, 0x2de0af30, 0x56fe2dd3, 0xc924ae4d, 0xa0c32815, + 0x3f19ab8b, 0x44072968, 0xdbddaaf6, 0xdc2c3734, 0x43f6b4aa, + 0x38e83649, 0xa732b5d7, 0xced5338f, 0x510fb011, 0x2a1132f2, + 0xb5cbb16c, 0xf9de3e42, 0x6604bddc, 0x1d1a3f3f, 0x82c0bca1, + 0xeb273af9, 0x74fdb967, 0x0fe33b84, 0x9039b81a, 0xf4e14df1, + 0x6b3bce6f, 0x10254c8c, 0x8fffcf12, 0xe618494a, 0x79c2cad4, + 0x02dc4837, 0x9d06cba9, 0xd1134487, 0x4ec9c719, 0x35d745fa, + 0xaa0dc664, 0xc3ea403c, 0x5c30c3a2, 0x272e4141, 0xb8f4c2df, + 0xbf055f1d, 0x20dfdc83, 0x5bc15e60, 0xc41bddfe, 0xadfc5ba6, + 0x3226d838, 0x49385adb, 0xd6e2d945, 0x9af7566b, 0x052dd5f5, + 0x7e335716, 0xe1e9d488, 0x880e52d0, 0x17d4d14e, 0x6cca53ad, + 0xf310d033, 0x63296829, 0xfcf3ebb7, 0x87ed6954, 0x1837eaca, + 0x71d06c92, 0xee0aef0c, 0x95146def, 0x0aceee71, 0x46db615f, + 0xd901e2c1, 0xa21f6022, 0x3dc5e3bc, 0x542265e4, 0xcbf8e67a, + 0xb0e66499, 0x2f3ce707, 0x28cd7ac5, 0xb717f95b, 0xcc097bb8, + 0x53d3f826, 0x3a347e7e, 0xa5eefde0, 0xdef07f03, 0x412afc9d, + 0x0d3f73b3, 0x92e5f02d, 0xe9fb72ce, 0x7621f150, 0x1fc67708, + 0x801cf496, 0xfb027675, 0x64d8f5eb, 0x32b39da3, 0xad691e3d, + 0xd6779cde, 0x49ad1f40, 0x204a9918, 0xbf901a86, 0xc48e9865, + 0x5b541bfb, 0x174194d5, 0x889b174b, 0xf38595a8, 0x6c5f1636, + 0x05b8906e, 0x9a6213f0, 0xe17c9113, 0x7ea6128d, 0x79578f4f, + 0xe68d0cd1, 0x9d938e32, 0x02490dac, 0x6bae8bf4, 0xf474086a, + 0x8f6a8a89, 0x10b00917, 0x5ca58639, 0xc37f05a7, 0xb8618744, + 0x27bb04da, 0x4e5c8282, 0xd186011c, 0xaa9883ff, 0x35420061, + 0xa57bb87b, 0x3aa13be5, 0x41bfb906, 0xde653a98, 0xb782bcc0, + 0x28583f5e, 0x5346bdbd, 0xcc9c3e23, 0x8089b10d, 0x1f533293, + 0x644db070, 0xfb9733ee, 0x9270b5b6, 0x0daa3628, 0x76b4b4cb, + 0xe96e3755, 0xee9faa97, 0x71452909, 0x0a5babea, 0x95812874, + 0xfc66ae2c, 0x63bc2db2, 0x18a2af51, 0x87782ccf, 0xcb6da3e1, + 0x54b7207f, 0x2fa9a29c, 0xb0732102, 0xd994a75a, 0x464e24c4, + 0x3d50a627, 0xa28a25b9, 0xc652d052, 0x598853cc, 0x2296d12f, + 0xbd4c52b1, 0xd4abd4e9, 0x4b715777, 0x306fd594, 0xafb5560a, + 0xe3a0d924, 0x7c7a5aba, 0x0764d859, 0x98be5bc7, 0xf159dd9f, + 0x6e835e01, 0x159ddce2, 0x8a475f7c, 0x8db6c2be, 0x126c4120, + 0x6972c3c3, 0xf6a8405d, 0x9f4fc605, 0x0095459b, 0x7b8bc778, + 0xe45144e6, 0xa844cbc8, 0x379e4856, 0x4c80cab5, 0xd35a492b, + 0xbabdcf73, 0x25674ced, 0x5e79ce0e, 0xc1a34d90, 0x519af58a, + 0xce407614, 0xb55ef4f7, 0x2a847769, 0x4363f131, 0xdcb972af, + 0xa7a7f04c, 0x387d73d2, 0x7468fcfc, 0xebb27f62, 0x90acfd81, + 0x0f767e1f, 0x6691f847, 0xf94b7bd9, 0x8255f93a, 0x1d8f7aa4, + 0x1a7ee766, 0x85a464f8, 0xfebae61b, 0x61606585, 0x0887e3dd, + 0x975d6043, 0xec43e2a0, 0x7399613e, 0x3f8cee10, 0xa0566d8e, + 0xdb48ef6d, 0x44926cf3, 0x2d75eaab, 0xb2af6935, 0xc9b1ebd6, + 0x566b6848}}; + +local const z_word_t FAR crc_braid_big_table[][256] = { + {0x00000000, 0x9e83da9f, 0x7d01c4e4, 0xe3821e7b, 0xbb04f912, + 0x2587238d, 0xc6053df6, 0x5886e769, 0x7609f225, 0xe88a28ba, + 0x0b0836c1, 0x958bec5e, 0xcd0d0b37, 0x538ed1a8, 0xb00ccfd3, + 0x2e8f154c, 0xec12e44b, 0x72913ed4, 0x911320af, 0x0f90fa30, + 0x57161d59, 0xc995c7c6, 0x2a17d9bd, 0xb4940322, 0x9a1b166e, + 0x0498ccf1, 0xe71ad28a, 0x79990815, 0x211fef7c, 0xbf9c35e3, + 0x5c1e2b98, 0xc29df107, 0xd825c897, 0x46a61208, 0xa5240c73, + 0x3ba7d6ec, 0x63213185, 0xfda2eb1a, 0x1e20f561, 0x80a32ffe, + 0xae2c3ab2, 0x30afe02d, 0xd32dfe56, 0x4dae24c9, 0x1528c3a0, + 0x8bab193f, 0x68290744, 0xf6aadddb, 0x34372cdc, 0xaab4f643, + 0x4936e838, 0xd7b532a7, 0x8f33d5ce, 0x11b00f51, 0xf232112a, + 0x6cb1cbb5, 0x423edef9, 0xdcbd0466, 0x3f3f1a1d, 0xa1bcc082, + 0xf93a27eb, 0x67b9fd74, 0x843be30f, 0x1ab83990, 0xf14de1f4, + 0x6fce3b6b, 0x8c4c2510, 0x12cfff8f, 0x4a4918e6, 0xd4cac279, + 0x3748dc02, 0xa9cb069d, 0x874413d1, 0x19c7c94e, 0xfa45d735, + 0x64c60daa, 0x3c40eac3, 0xa2c3305c, 0x41412e27, 0xdfc2f4b8, + 0x1d5f05bf, 0x83dcdf20, 0x605ec15b, 0xfedd1bc4, 0xa65bfcad, + 0x38d82632, 0xdb5a3849, 0x45d9e2d6, 0x6b56f79a, 0xf5d52d05, + 0x1657337e, 0x88d4e9e1, 0xd0520e88, 0x4ed1d417, 0xad53ca6c, + 0x33d010f3, 0x29682963, 0xb7ebf3fc, 0x5469ed87, 0xcaea3718, + 0x926cd071, 0x0cef0aee, 0xef6d1495, 0x71eece0a, 0x5f61db46, + 0xc1e201d9, 0x22601fa2, 0xbce3c53d, 0xe4652254, 0x7ae6f8cb, + 0x9964e6b0, 0x07e73c2f, 0xc57acd28, 0x5bf917b7, 0xb87b09cc, + 0x26f8d353, 0x7e7e343a, 0xe0fdeea5, 0x037ff0de, 0x9dfc2a41, + 0xb3733f0d, 0x2df0e592, 0xce72fbe9, 0x50f12176, 0x0877c61f, + 0x96f41c80, 0x757602fb, 0xebf5d864, 0xa39db332, 0x3d1e69ad, + 0xde9c77d6, 0x401fad49, 0x18994a20, 0x861a90bf, 0x65988ec4, + 0xfb1b545b, 0xd5944117, 0x4b179b88, 0xa89585f3, 0x36165f6c, + 0x6e90b805, 0xf013629a, 0x13917ce1, 0x8d12a67e, 0x4f8f5779, + 0xd10c8de6, 0x328e939d, 0xac0d4902, 0xf48bae6b, 0x6a0874f4, + 0x898a6a8f, 0x1709b010, 0x3986a55c, 0xa7057fc3, 0x448761b8, + 0xda04bb27, 0x82825c4e, 0x1c0186d1, 0xff8398aa, 0x61004235, + 0x7bb87ba5, 0xe53ba13a, 0x06b9bf41, 0x983a65de, 0xc0bc82b7, + 0x5e3f5828, 0xbdbd4653, 0x233e9ccc, 0x0db18980, 0x9332531f, + 0x70b04d64, 0xee3397fb, 0xb6b57092, 0x2836aa0d, 0xcbb4b476, + 0x55376ee9, 0x97aa9fee, 0x09294571, 0xeaab5b0a, 0x74288195, + 0x2cae66fc, 0xb22dbc63, 0x51afa218, 0xcf2c7887, 0xe1a36dcb, + 0x7f20b754, 0x9ca2a92f, 0x022173b0, 0x5aa794d9, 0xc4244e46, + 0x27a6503d, 0xb9258aa2, 0x52d052c6, 0xcc538859, 0x2fd19622, + 0xb1524cbd, 0xe9d4abd4, 0x7757714b, 0x94d56f30, 0x0a56b5af, + 0x24d9a0e3, 0xba5a7a7c, 0x59d86407, 0xc75bbe98, 0x9fdd59f1, + 0x015e836e, 0xe2dc9d15, 0x7c5f478a, 0xbec2b68d, 0x20416c12, + 0xc3c37269, 0x5d40a8f6, 0x05c64f9f, 0x9b459500, 0x78c78b7b, + 0xe64451e4, 0xc8cb44a8, 0x56489e37, 0xb5ca804c, 0x2b495ad3, + 0x73cfbdba, 0xed4c6725, 0x0ece795e, 0x904da3c1, 0x8af59a51, + 0x147640ce, 0xf7f45eb5, 0x6977842a, 0x31f16343, 0xaf72b9dc, + 0x4cf0a7a7, 0xd2737d38, 0xfcfc6874, 0x627fb2eb, 0x81fdac90, + 0x1f7e760f, 0x47f89166, 0xd97b4bf9, 0x3af95582, 0xa47a8f1d, + 0x66e77e1a, 0xf864a485, 0x1be6bafe, 0x85656061, 0xdde38708, + 0x43605d97, 0xa0e243ec, 0x3e619973, 0x10ee8c3f, 0x8e6d56a0, + 0x6def48db, 0xf36c9244, 0xabea752d, 0x3569afb2, 0xd6ebb1c9, + 0x48686b56}, + {0x00000000, 0xc0642817, 0x80c9502e, 0x40ad7839, 0x0093a15c, + 0xc0f7894b, 0x805af172, 0x403ed965, 0x002643b9, 0xc0426bae, + 0x80ef1397, 0x408b3b80, 0x00b5e2e5, 0xc0d1caf2, 0x807cb2cb, + 0x40189adc, 0x414af7a9, 0x812edfbe, 0xc183a787, 0x01e78f90, + 0x41d956f5, 0x81bd7ee2, 0xc11006db, 0x01742ecc, 0x416cb410, + 0x81089c07, 0xc1a5e43e, 0x01c1cc29, 0x41ff154c, 0x819b3d5b, + 0xc1364562, 0x01526d75, 0xc3929f88, 0x03f6b79f, 0x435bcfa6, + 0x833fe7b1, 0xc3013ed4, 0x036516c3, 0x43c86efa, 0x83ac46ed, + 0xc3b4dc31, 0x03d0f426, 0x437d8c1f, 0x8319a408, 0xc3277d6d, + 0x0343557a, 0x43ee2d43, 0x838a0554, 0x82d86821, 0x42bc4036, + 0x0211380f, 0xc2751018, 0x824bc97d, 0x422fe16a, 0x02829953, + 0xc2e6b144, 0x82fe2b98, 0x429a038f, 0x02377bb6, 0xc25353a1, + 0x826d8ac4, 0x4209a2d3, 0x02a4daea, 0xc2c0f2fd, 0xc7234eca, + 0x074766dd, 0x47ea1ee4, 0x878e36f3, 0xc7b0ef96, 0x07d4c781, + 0x4779bfb8, 0x871d97af, 0xc7050d73, 0x07612564, 0x47cc5d5d, + 0x87a8754a, 0xc796ac2f, 0x07f28438, 0x475ffc01, 0x873bd416, + 0x8669b963, 0x460d9174, 0x06a0e94d, 0xc6c4c15a, 0x86fa183f, + 0x469e3028, 0x06334811, 0xc6576006, 0x864ffada, 0x462bd2cd, + 0x0686aaf4, 0xc6e282e3, 0x86dc5b86, 0x46b87391, 0x06150ba8, + 0xc67123bf, 0x04b1d142, 0xc4d5f955, 0x8478816c, 0x441ca97b, + 0x0422701e, 0xc4465809, 0x84eb2030, 0x448f0827, 0x049792fb, + 0xc4f3baec, 0x845ec2d5, 0x443aeac2, 0x040433a7, 0xc4601bb0, + 0x84cd6389, 0x44a94b9e, 0x45fb26eb, 0x859f0efc, 0xc53276c5, + 0x05565ed2, 0x456887b7, 0x850cafa0, 0xc5a1d799, 0x05c5ff8e, + 0x45dd6552, 0x85b94d45, 0xc514357c, 0x05701d6b, 0x454ec40e, + 0x852aec19, 0xc5879420, 0x05e3bc37, 0xcf41ed4f, 0x0f25c558, + 0x4f88bd61, 0x8fec9576, 0xcfd24c13, 0x0fb66404, 0x4f1b1c3d, + 0x8f7f342a, 0xcf67aef6, 0x0f0386e1, 0x4faefed8, 0x8fcad6cf, + 0xcff40faa, 0x0f9027bd, 0x4f3d5f84, 0x8f597793, 0x8e0b1ae6, + 0x4e6f32f1, 0x0ec24ac8, 0xcea662df, 0x8e98bbba, 0x4efc93ad, + 0x0e51eb94, 0xce35c383, 0x8e2d595f, 0x4e497148, 0x0ee40971, + 0xce802166, 0x8ebef803, 0x4edad014, 0x0e77a82d, 0xce13803a, + 0x0cd372c7, 0xccb75ad0, 0x8c1a22e9, 0x4c7e0afe, 0x0c40d39b, + 0xcc24fb8c, 0x8c8983b5, 0x4cedaba2, 0x0cf5317e, 0xcc911969, + 0x8c3c6150, 0x4c584947, 0x0c669022, 0xcc02b835, 0x8cafc00c, + 0x4ccbe81b, 0x4d99856e, 0x8dfdad79, 0xcd50d540, 0x0d34fd57, + 0x4d0a2432, 0x8d6e0c25, 0xcdc3741c, 0x0da75c0b, 0x4dbfc6d7, + 0x8ddbeec0, 0xcd7696f9, 0x0d12beee, 0x4d2c678b, 0x8d484f9c, + 0xcde537a5, 0x0d811fb2, 0x0862a385, 0xc8068b92, 0x88abf3ab, + 0x48cfdbbc, 0x08f102d9, 0xc8952ace, 0x883852f7, 0x485c7ae0, + 0x0844e03c, 0xc820c82b, 0x888db012, 0x48e99805, 0x08d74160, + 0xc8b36977, 0x881e114e, 0x487a3959, 0x4928542c, 0x894c7c3b, + 0xc9e10402, 0x09852c15, 0x49bbf570, 0x89dfdd67, 0xc972a55e, + 0x09168d49, 0x490e1795, 0x896a3f82, 0xc9c747bb, 0x09a36fac, + 0x499db6c9, 0x89f99ede, 0xc954e6e7, 0x0930cef0, 0xcbf03c0d, + 0x0b94141a, 0x4b396c23, 0x8b5d4434, 0xcb639d51, 0x0b07b546, + 0x4baacd7f, 0x8bcee568, 0xcbd67fb4, 0x0bb257a3, 0x4b1f2f9a, + 0x8b7b078d, 0xcb45dee8, 0x0b21f6ff, 0x4b8c8ec6, 0x8be8a6d1, + 0x8abacba4, 0x4adee3b3, 0x0a739b8a, 0xca17b39d, 0x8a296af8, + 0x4a4d42ef, 0x0ae03ad6, 0xca8412c1, 0x8a9c881d, 0x4af8a00a, + 0x0a55d833, 0xca31f024, 0x8a0f2941, 0x4a6b0156, 0x0ac6796f, + 0xcaa25178}, + {0x00000000, 0xd4ea739b, 0xe9d396ed, 0x3d39e576, 0x93a15c00, + 0x474b2f9b, 0x7a72caed, 0xae98b976, 0x2643b900, 0xf2a9ca9b, + 0xcf902fed, 0x1b7a5c76, 0xb5e2e500, 0x6108969b, 0x5c3173ed, + 0x88db0076, 0x4c867201, 0x986c019a, 0xa555e4ec, 0x71bf9777, + 0xdf272e01, 0x0bcd5d9a, 0x36f4b8ec, 0xe21ecb77, 0x6ac5cb01, + 0xbe2fb89a, 0x83165dec, 0x57fc2e77, 0xf9649701, 0x2d8ee49a, + 0x10b701ec, 0xc45d7277, 0x980ce502, 0x4ce69699, 0x71df73ef, + 0xa5350074, 0x0badb902, 0xdf47ca99, 0xe27e2fef, 0x36945c74, + 0xbe4f5c02, 0x6aa52f99, 0x579ccaef, 0x8376b974, 0x2dee0002, + 0xf9047399, 0xc43d96ef, 0x10d7e574, 0xd48a9703, 0x0060e498, + 0x3d5901ee, 0xe9b37275, 0x472bcb03, 0x93c1b898, 0xaef85dee, + 0x7a122e75, 0xf2c92e03, 0x26235d98, 0x1b1ab8ee, 0xcff0cb75, + 0x61687203, 0xb5820198, 0x88bbe4ee, 0x5c519775, 0x3019ca05, + 0xe4f3b99e, 0xd9ca5ce8, 0x0d202f73, 0xa3b89605, 0x7752e59e, + 0x4a6b00e8, 0x9e817373, 0x165a7305, 0xc2b0009e, 0xff89e5e8, + 0x2b639673, 0x85fb2f05, 0x51115c9e, 0x6c28b9e8, 0xb8c2ca73, + 0x7c9fb804, 0xa875cb9f, 0x954c2ee9, 0x41a65d72, 0xef3ee404, + 0x3bd4979f, 0x06ed72e9, 0xd2070172, 0x5adc0104, 0x8e36729f, + 0xb30f97e9, 0x67e5e472, 0xc97d5d04, 0x1d972e9f, 0x20aecbe9, + 0xf444b872, 0xa8152f07, 0x7cff5c9c, 0x41c6b9ea, 0x952cca71, + 0x3bb47307, 0xef5e009c, 0xd267e5ea, 0x068d9671, 0x8e569607, + 0x5abce59c, 0x678500ea, 0xb36f7371, 0x1df7ca07, 0xc91db99c, + 0xf4245cea, 0x20ce2f71, 0xe4935d06, 0x30792e9d, 0x0d40cbeb, + 0xd9aab870, 0x77320106, 0xa3d8729d, 0x9ee197eb, 0x4a0be470, + 0xc2d0e406, 0x163a979d, 0x2b0372eb, 0xffe90170, 0x5171b806, + 0x859bcb9d, 0xb8a22eeb, 0x6c485d70, 0x6032940b, 0xb4d8e790, + 0x89e102e6, 0x5d0b717d, 0xf393c80b, 0x2779bb90, 0x1a405ee6, + 0xceaa2d7d, 0x46712d0b, 0x929b5e90, 0xafa2bbe6, 0x7b48c87d, + 0xd5d0710b, 0x013a0290, 0x3c03e7e6, 0xe8e9947d, 0x2cb4e60a, + 0xf85e9591, 0xc56770e7, 0x118d037c, 0xbf15ba0a, 0x6bffc991, + 0x56c62ce7, 0x822c5f7c, 0x0af75f0a, 0xde1d2c91, 0xe324c9e7, + 0x37ceba7c, 0x9956030a, 0x4dbc7091, 0x708595e7, 0xa46fe67c, + 0xf83e7109, 0x2cd40292, 0x11ede7e4, 0xc507947f, 0x6b9f2d09, + 0xbf755e92, 0x824cbbe4, 0x56a6c87f, 0xde7dc809, 0x0a97bb92, + 0x37ae5ee4, 0xe3442d7f, 0x4ddc9409, 0x9936e792, 0xa40f02e4, + 0x70e5717f, 0xb4b80308, 0x60527093, 0x5d6b95e5, 0x8981e67e, + 0x27195f08, 0xf3f32c93, 0xcecac9e5, 0x1a20ba7e, 0x92fbba08, + 0x4611c993, 0x7b282ce5, 0xafc25f7e, 0x015ae608, 0xd5b09593, + 0xe88970e5, 0x3c63037e, 0x502b5e0e, 0x84c12d95, 0xb9f8c8e3, + 0x6d12bb78, 0xc38a020e, 0x17607195, 0x2a5994e3, 0xfeb3e778, + 0x7668e70e, 0xa2829495, 0x9fbb71e3, 0x4b510278, 0xe5c9bb0e, + 0x3123c895, 0x0c1a2de3, 0xd8f05e78, 0x1cad2c0f, 0xc8475f94, + 0xf57ebae2, 0x2194c979, 0x8f0c700f, 0x5be60394, 0x66dfe6e2, + 0xb2359579, 0x3aee950f, 0xee04e694, 0xd33d03e2, 0x07d77079, + 0xa94fc90f, 0x7da5ba94, 0x409c5fe2, 0x94762c79, 0xc827bb0c, + 0x1ccdc897, 0x21f42de1, 0xf51e5e7a, 0x5b86e70c, 0x8f6c9497, + 0xb25571e1, 0x66bf027a, 0xee64020c, 0x3a8e7197, 0x07b794e1, + 0xd35de77a, 0x7dc55e0c, 0xa92f2d97, 0x9416c8e1, 0x40fcbb7a, + 0x84a1c90d, 0x504bba96, 0x6d725fe0, 0xb9982c7b, 0x1700950d, + 0xc3eae696, 0xfed303e0, 0x2a39707b, 0xa2e2700d, 0x76080396, + 0x4b31e6e0, 0x9fdb957b, 0x31432c0d, 0xe5a95f96, 0xd890bae0, + 0x0c7ac97b}, + {0x00000000, 0x27652581, 0x0fcc3bd9, 0x28a91e58, 0x5f9e0669, + 0x78fb23e8, 0x50523db0, 0x77371831, 0xbe3c0dd2, 0x99592853, + 0xb1f0360b, 0x9695138a, 0xe1a20bbb, 0xc6c72e3a, 0xee6e3062, + 0xc90b15e3, 0x3d7f6b7f, 0x1a1a4efe, 0x32b350a6, 0x15d67527, + 0x62e16d16, 0x45844897, 0x6d2d56cf, 0x4a48734e, 0x834366ad, + 0xa426432c, 0x8c8f5d74, 0xabea78f5, 0xdcdd60c4, 0xfbb84545, + 0xd3115b1d, 0xf4747e9c, 0x7afed6fe, 0x5d9bf37f, 0x7532ed27, + 0x5257c8a6, 0x2560d097, 0x0205f516, 0x2aaceb4e, 0x0dc9cecf, + 0xc4c2db2c, 0xe3a7fead, 0xcb0ee0f5, 0xec6bc574, 0x9b5cdd45, + 0xbc39f8c4, 0x9490e69c, 0xb3f5c31d, 0x4781bd81, 0x60e49800, + 0x484d8658, 0x6f28a3d9, 0x181fbbe8, 0x3f7a9e69, 0x17d38031, + 0x30b6a5b0, 0xf9bdb053, 0xded895d2, 0xf6718b8a, 0xd114ae0b, + 0xa623b63a, 0x814693bb, 0xa9ef8de3, 0x8e8aa862, 0xb5fadc26, + 0x929ff9a7, 0xba36e7ff, 0x9d53c27e, 0xea64da4f, 0xcd01ffce, + 0xe5a8e196, 0xc2cdc417, 0x0bc6d1f4, 0x2ca3f475, 0x040aea2d, + 0x236fcfac, 0x5458d79d, 0x733df21c, 0x5b94ec44, 0x7cf1c9c5, + 0x8885b759, 0xafe092d8, 0x87498c80, 0xa02ca901, 0xd71bb130, + 0xf07e94b1, 0xd8d78ae9, 0xffb2af68, 0x36b9ba8b, 0x11dc9f0a, + 0x39758152, 0x1e10a4d3, 0x6927bce2, 0x4e429963, 0x66eb873b, + 0x418ea2ba, 0xcf040ad8, 0xe8612f59, 0xc0c83101, 0xe7ad1480, + 0x909a0cb1, 0xb7ff2930, 0x9f563768, 0xb83312e9, 0x7138070a, + 0x565d228b, 0x7ef43cd3, 0x59911952, 0x2ea60163, 0x09c324e2, + 0x216a3aba, 0x060f1f3b, 0xf27b61a7, 0xd51e4426, 0xfdb75a7e, + 0xdad27fff, 0xade567ce, 0x8a80424f, 0xa2295c17, 0x854c7996, + 0x4c476c75, 0x6b2249f4, 0x438b57ac, 0x64ee722d, 0x13d96a1c, + 0x34bc4f9d, 0x1c1551c5, 0x3b707444, 0x6af5b94d, 0x4d909ccc, + 0x65398294, 0x425ca715, 0x356bbf24, 0x120e9aa5, 0x3aa784fd, + 0x1dc2a17c, 0xd4c9b49f, 0xf3ac911e, 0xdb058f46, 0xfc60aac7, + 0x8b57b2f6, 0xac329777, 0x849b892f, 0xa3feacae, 0x578ad232, + 0x70eff7b3, 0x5846e9eb, 0x7f23cc6a, 0x0814d45b, 0x2f71f1da, + 0x07d8ef82, 0x20bdca03, 0xe9b6dfe0, 0xced3fa61, 0xe67ae439, + 0xc11fc1b8, 0xb628d989, 0x914dfc08, 0xb9e4e250, 0x9e81c7d1, + 0x100b6fb3, 0x376e4a32, 0x1fc7546a, 0x38a271eb, 0x4f9569da, + 0x68f04c5b, 0x40595203, 0x673c7782, 0xae376261, 0x895247e0, + 0xa1fb59b8, 0x869e7c39, 0xf1a96408, 0xd6cc4189, 0xfe655fd1, + 0xd9007a50, 0x2d7404cc, 0x0a11214d, 0x22b83f15, 0x05dd1a94, + 0x72ea02a5, 0x558f2724, 0x7d26397c, 0x5a431cfd, 0x9348091e, + 0xb42d2c9f, 0x9c8432c7, 0xbbe11746, 0xccd60f77, 0xebb32af6, + 0xc31a34ae, 0xe47f112f, 0xdf0f656b, 0xf86a40ea, 0xd0c35eb2, + 0xf7a67b33, 0x80916302, 0xa7f44683, 0x8f5d58db, 0xa8387d5a, + 0x613368b9, 0x46564d38, 0x6eff5360, 0x499a76e1, 0x3ead6ed0, + 0x19c84b51, 0x31615509, 0x16047088, 0xe2700e14, 0xc5152b95, + 0xedbc35cd, 0xcad9104c, 0xbdee087d, 0x9a8b2dfc, 0xb22233a4, + 0x95471625, 0x5c4c03c6, 0x7b292647, 0x5380381f, 0x74e51d9e, + 0x03d205af, 0x24b7202e, 0x0c1e3e76, 0x2b7b1bf7, 0xa5f1b395, + 0x82949614, 0xaa3d884c, 0x8d58adcd, 0xfa6fb5fc, 0xdd0a907d, + 0xf5a38e25, 0xd2c6aba4, 0x1bcdbe47, 0x3ca89bc6, 0x1401859e, + 0x3364a01f, 0x4453b82e, 0x63369daf, 0x4b9f83f7, 0x6cfaa676, + 0x988ed8ea, 0xbfebfd6b, 0x9742e333, 0xb027c6b2, 0xc710de83, + 0xe075fb02, 0xc8dce55a, 0xefb9c0db, 0x26b2d538, 0x01d7f0b9, + 0x297eeee1, 0x0e1bcb60, 0x792cd351, 0x5e49f6d0, 0x76e0e888, + 0x5185cd09}}; + +#endif + +#endif + +#endif + +local const z_crc_t FAR x2n_table[] = { + 0x40000000, 0x20000000, 0x08000000, 0x00800000, 0x00008000, + 0xedb88320, 0xb1e6b092, 0xa06a2517, 0xed627dae, 0x88d14467, + 0xd7bbfe6a, 0xec447f11, 0x8e7ea170, 0x6427800e, 0x4d47bae0, + 0x09fe548f, 0x83852d0f, 0x30362f1a, 0x7b5a9cc3, 0x31fec169, + 0x9fec022a, 0x6c8dedc4, 0x15d6874d, 0x5fde7a4e, 0xbad90e37, + 0x2e4e5eef, 0x4eaba214, 0xa8a472c0, 0x429a969e, 0x148d302a, + 0xc40ba6d0, 0xc4e22c3c}; diff --git a/lib/os/windows/zlib-1.2.13/deflate.c b/lib/os/windows/zlib-1.2.13/deflate.c new file mode 100644 index 000000000000..4a689db35989 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/deflate.c @@ -0,0 +1,2217 @@ +/* deflate.c -- compress data using the deflation algorithm + * Copyright (C) 1995-2022 Jean-loup Gailly and Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* + * ALGORITHM + * + * The "deflation" process depends on being able to identify portions + * of the input text which are identical to earlier input (within a + * sliding window trailing behind the input currently being processed). + * + * The most straightforward technique turns out to be the fastest for + * most input files: try all possible matches and select the longest. + * The key feature of this algorithm is that insertions into the string + * dictionary are very simple and thus fast, and deletions are avoided + * completely. Insertions are performed at each input character, whereas + * string matches are performed only when the previous match ends. So it + * is preferable to spend more time in matches to allow very fast string + * insertions and avoid deletions. The matching algorithm for small + * strings is inspired from that of Rabin & Karp. A brute force approach + * is used to find longer strings when a small match has been found. + * A similar algorithm is used in comic (by Jan-Mark Wams) and freeze + * (by Leonid Broukhis). + * A previous version of this file used a more sophisticated algorithm + * (by Fiala and Greene) which is guaranteed to run in linear amortized + * time, but has a larger average cost, uses more memory and is patented. + * However the F&G algorithm may be faster for some highly redundant + * files if the parameter max_chain_length (described below) is too large. + * + * ACKNOWLEDGEMENTS + * + * The idea of lazy evaluation of matches is due to Jan-Mark Wams, and + * I found it in 'freeze' written by Leonid Broukhis. + * Thanks to many people for bug reports and testing. + * + * REFERENCES + * + * Deutsch, L.P.,"DEFLATE Compressed Data Format Specification". + * Available in http://tools.ietf.org/html/rfc1951 + * + * A description of the Rabin and Karp algorithm is given in the book + * "Algorithms" by R. Sedgewick, Addison-Wesley, p252. + * + * Fiala,E.R., and Greene,D.H. + * Data Compression with Finite Windows, Comm.ACM, 32,4 (1989) 490-595 + * + */ + +/* @(#) $Id$ */ + +#include "deflate.h" + +const char deflate_copyright[] = + " deflate 1.2.13 Copyright 1995-2022 Jean-loup Gailly and Mark Adler "; +/* + If you use the zlib library in a product, an acknowledgment is welcome + in the documentation of your product. If for some reason you cannot + include such an acknowledgment, I would appreciate that you keep this + copyright string in the executable of your product. + */ + +/* =========================================================================== + * Function prototypes. + */ +typedef enum { + need_more, /* block not completed, need more input or more output */ + block_done, /* block flush performed */ + finish_started, /* finish started, need only more output at next deflate */ + finish_done /* finish done, accept no more input or output */ +} block_state; + +typedef block_state (*compress_func) OF((deflate_state *s, int flush)); +/* Compression function. Returns the block state after the call. */ + +local int deflateStateCheck OF((z_streamp strm)); +local void slide_hash OF((deflate_state *s)); +local void fill_window OF((deflate_state *s)); +local block_state deflate_stored OF((deflate_state *s, int flush)); +local block_state deflate_fast OF((deflate_state *s, int flush)); +#ifndef FASTEST +local block_state deflate_slow OF((deflate_state *s, int flush)); +#endif +local block_state deflate_rle OF((deflate_state *s, int flush)); +local block_state deflate_huff OF((deflate_state *s, int flush)); +local void lm_init OF((deflate_state *s)); +local void putShortMSB OF((deflate_state *s, uInt b)); +local void flush_pending OF((z_streamp strm)); +local unsigned read_buf OF((z_streamp strm, Bytef *buf, unsigned size)); +local uInt longest_match OF((deflate_state *s, IPos cur_match)); + +#ifdef ZLIB_DEBUG +local void check_match OF((deflate_state *s, IPos start, IPos match, + int length)); +#endif + +/* =========================================================================== + * Local data + */ + +#define NIL 0 +/* Tail of hash chains */ + +#ifndef TOO_FAR +# define TOO_FAR 4096 +#endif +/* Matches of length 3 are discarded if their distance exceeds TOO_FAR */ + +/* Values for max_lazy_match, good_match and max_chain_length, depending on + * the desired pack level (0..9). The values given below have been tuned to + * exclude worst case performance for pathological files. Better values may be + * found for specific files. + */ +typedef struct config_s { + ush good_length; /* reduce lazy search above this match length */ + ush max_lazy; /* do not perform lazy search above this match length */ + ush nice_length; /* quit search above this match length */ + ush max_chain; + compress_func func; +} config; + +#ifdef FASTEST +local const config configuration_table[2] = { +/* good lazy nice chain */ +/* 0 */ {0, 0, 0, 0, deflate_stored}, /* store only */ +/* 1 */ {4, 4, 8, 4, deflate_fast}}; /* max speed, no lazy matches */ +#else +local const config configuration_table[10] = { +/* good lazy nice chain */ +/* 0 */ {0, 0, 0, 0, deflate_stored}, /* store only */ +/* 1 */ {4, 4, 8, 4, deflate_fast}, /* max speed, no lazy matches */ +/* 2 */ {4, 5, 16, 8, deflate_fast}, +/* 3 */ {4, 6, 32, 32, deflate_fast}, + +/* 4 */ {4, 4, 16, 16, deflate_slow}, /* lazy matches */ +/* 5 */ {8, 16, 32, 32, deflate_slow}, +/* 6 */ {8, 16, 128, 128, deflate_slow}, +/* 7 */ {8, 32, 128, 256, deflate_slow}, +/* 8 */ {32, 128, 258, 1024, deflate_slow}, +/* 9 */ {32, 258, 258, 4096, deflate_slow}}; /* max compression */ +#endif + +/* Note: the deflate() code requires max_lazy >= MIN_MATCH and max_chain >= 4 + * For deflate_fast() (levels <= 3) good is ignored and lazy has a different + * meaning. + */ + +/* rank Z_BLOCK between Z_NO_FLUSH and Z_PARTIAL_FLUSH */ +#define RANK(f) (((f) * 2) - ((f) > 4 ? 9 : 0)) + +/* =========================================================================== + * Update a hash value with the given input byte + * IN assertion: all calls to UPDATE_HASH are made with consecutive input + * characters, so that a running hash key can be computed from the previous + * key instead of complete recalculation each time. + */ +#define UPDATE_HASH(s,h,c) (h = (((h) << s->hash_shift) ^ (c)) & s->hash_mask) + + +/* =========================================================================== + * Insert string str in the dictionary and set match_head to the previous head + * of the hash chain (the most recent string with same hash key). Return + * the previous length of the hash chain. + * If this file is compiled with -DFASTEST, the compression level is forced + * to 1, and no hash chains are maintained. + * IN assertion: all calls to INSERT_STRING are made with consecutive input + * characters and the first MIN_MATCH bytes of str are valid (except for + * the last MIN_MATCH-1 bytes of the input file). + */ +#ifdef FASTEST +#define INSERT_STRING(s, str, match_head) \ + (UPDATE_HASH(s, s->ins_h, s->window[(str) + (MIN_MATCH-1)]), \ + match_head = s->head[s->ins_h], \ + s->head[s->ins_h] = (Pos)(str)) +#else +#define INSERT_STRING(s, str, match_head) \ + (UPDATE_HASH(s, s->ins_h, s->window[(str) + (MIN_MATCH-1)]), \ + match_head = s->prev[(str) & s->w_mask] = s->head[s->ins_h], \ + s->head[s->ins_h] = (Pos)(str)) +#endif + +/* =========================================================================== + * Initialize the hash table (avoiding 64K overflow for 16 bit systems). + * prev[] will be initialized on the fly. + */ +#define CLEAR_HASH(s) \ + do { \ + s->head[s->hash_size - 1] = NIL; \ + zmemzero((Bytef *)s->head, \ + (unsigned)(s->hash_size - 1)*sizeof(*s->head)); \ + } while (0) + +/* =========================================================================== + * Slide the hash table when sliding the window down (could be avoided with 32 + * bit values at the expense of memory usage). We slide even when level == 0 to + * keep the hash table consistent if we switch back to level > 0 later. + */ +local void slide_hash(s) + deflate_state *s; +{ + unsigned n, m; + Posf *p; + uInt wsize = s->w_size; + + n = s->hash_size; + p = &s->head[n]; + do { + m = *--p; + *p = (Pos)(m >= wsize ? m - wsize : NIL); + } while (--n); + n = wsize; +#ifndef FASTEST + p = &s->prev[n]; + do { + m = *--p; + *p = (Pos)(m >= wsize ? m - wsize : NIL); + /* If n is not on any hash chain, prev[n] is garbage but + * its value will never be used. + */ + } while (--n); +#endif +} + +/* ========================================================================= */ +int ZEXPORT deflateInit_(strm, level, version, stream_size) + z_streamp strm; + int level; + const char *version; + int stream_size; +{ + return deflateInit2_(strm, level, Z_DEFLATED, MAX_WBITS, DEF_MEM_LEVEL, + Z_DEFAULT_STRATEGY, version, stream_size); + /* To do: ignore strm->next_in if we use it as window */ +} + +/* ========================================================================= */ +int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy, + version, stream_size) + z_streamp strm; + int level; + int method; + int windowBits; + int memLevel; + int strategy; + const char *version; + int stream_size; +{ + deflate_state *s; + int wrap = 1; + static const char my_version[] = ZLIB_VERSION; + + if (version == Z_NULL || version[0] != my_version[0] || + stream_size != sizeof(z_stream)) { + return Z_VERSION_ERROR; + } + if (strm == Z_NULL) return Z_STREAM_ERROR; + + strm->msg = Z_NULL; + if (strm->zalloc == (alloc_func)0) { +#ifdef Z_SOLO + return Z_STREAM_ERROR; +#else + strm->zalloc = zcalloc; + strm->opaque = (voidpf)0; +#endif + } + if (strm->zfree == (free_func)0) +#ifdef Z_SOLO + return Z_STREAM_ERROR; +#else + strm->zfree = zcfree; +#endif + +#ifdef FASTEST + if (level != 0) level = 1; +#else + if (level == Z_DEFAULT_COMPRESSION) level = 6; +#endif + + if (windowBits < 0) { /* suppress zlib wrapper */ + wrap = 0; + if (windowBits < -15) + return Z_STREAM_ERROR; + windowBits = -windowBits; + } +#ifdef GZIP + else if (windowBits > 15) { + wrap = 2; /* write gzip wrapper instead */ + windowBits -= 16; + } +#endif + if (memLevel < 1 || memLevel > MAX_MEM_LEVEL || method != Z_DEFLATED || + windowBits < 8 || windowBits > 15 || level < 0 || level > 9 || + strategy < 0 || strategy > Z_FIXED || (windowBits == 8 && wrap != 1)) { + return Z_STREAM_ERROR; + } + if (windowBits == 8) windowBits = 9; /* until 256-byte window bug fixed */ + s = (deflate_state *) ZALLOC(strm, 1, sizeof(deflate_state)); + if (s == Z_NULL) return Z_MEM_ERROR; + strm->state = (struct internal_state FAR *)s; + s->strm = strm; + s->status = INIT_STATE; /* to pass state test in deflateReset() */ + + s->wrap = wrap; + s->gzhead = Z_NULL; + s->w_bits = (uInt)windowBits; + s->w_size = 1 << s->w_bits; + s->w_mask = s->w_size - 1; + + s->hash_bits = (uInt)memLevel + 7; + s->hash_size = 1 << s->hash_bits; + s->hash_mask = s->hash_size - 1; + s->hash_shift = ((s->hash_bits + MIN_MATCH-1) / MIN_MATCH); + + s->window = (Bytef *) ZALLOC(strm, s->w_size, 2*sizeof(Byte)); + s->prev = (Posf *) ZALLOC(strm, s->w_size, sizeof(Pos)); + s->head = (Posf *) ZALLOC(strm, s->hash_size, sizeof(Pos)); + + s->high_water = 0; /* nothing written to s->window yet */ + + s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by default */ + + /* We overlay pending_buf and sym_buf. This works since the average size + * for length/distance pairs over any compressed block is assured to be 31 + * bits or less. + * + * Analysis: The longest fixed codes are a length code of 8 bits plus 5 + * extra bits, for lengths 131 to 257. The longest fixed distance codes are + * 5 bits plus 13 extra bits, for distances 16385 to 32768. The longest + * possible fixed-codes length/distance pair is then 31 bits total. + * + * sym_buf starts one-fourth of the way into pending_buf. So there are + * three bytes in sym_buf for every four bytes in pending_buf. Each symbol + * in sym_buf is three bytes -- two for the distance and one for the + * literal/length. As each symbol is consumed, the pointer to the next + * sym_buf value to read moves forward three bytes. From that symbol, up to + * 31 bits are written to pending_buf. The closest the written pending_buf + * bits gets to the next sym_buf symbol to read is just before the last + * code is written. At that time, 31*(n - 2) bits have been written, just + * after 24*(n - 2) bits have been consumed from sym_buf. sym_buf starts at + * 8*n bits into pending_buf. (Note that the symbol buffer fills when n - 1 + * symbols are written.) The closest the writing gets to what is unread is + * then n + 14 bits. Here n is lit_bufsize, which is 16384 by default, and + * can range from 128 to 32768. + * + * Therefore, at a minimum, there are 142 bits of space between what is + * written and what is read in the overlain buffers, so the symbols cannot + * be overwritten by the compressed data. That space is actually 139 bits, + * due to the three-bit fixed-code block header. + * + * That covers the case where either Z_FIXED is specified, forcing fixed + * codes, or when the use of fixed codes is chosen, because that choice + * results in a smaller compressed block than dynamic codes. That latter + * condition then assures that the above analysis also covers all dynamic + * blocks. A dynamic-code block will only be chosen to be emitted if it has + * fewer bits than a fixed-code block would for the same set of symbols. + * Therefore its average symbol length is assured to be less than 31. So + * the compressed data for a dynamic block also cannot overwrite the + * symbols from which it is being constructed. + */ + + s->pending_buf = (uchf *) ZALLOC(strm, s->lit_bufsize, 4); + s->pending_buf_size = (ulg)s->lit_bufsize * 4; + + if (s->window == Z_NULL || s->prev == Z_NULL || s->head == Z_NULL || + s->pending_buf == Z_NULL) { + s->status = FINISH_STATE; + strm->msg = ERR_MSG(Z_MEM_ERROR); + deflateEnd (strm); + return Z_MEM_ERROR; + } + s->sym_buf = s->pending_buf + s->lit_bufsize; + s->sym_end = (s->lit_bufsize - 1) * 3; + /* We avoid equality with lit_bufsize*3 because of wraparound at 64K + * on 16 bit machines and because stored blocks are restricted to + * 64K-1 bytes. + */ + + s->level = level; + s->strategy = strategy; + s->method = (Byte)method; + + return deflateReset(strm); +} + +/* ========================================================================= + * Check for a valid deflate stream state. Return 0 if ok, 1 if not. + */ +local int deflateStateCheck(strm) + z_streamp strm; +{ + deflate_state *s; + if (strm == Z_NULL || + strm->zalloc == (alloc_func)0 || strm->zfree == (free_func)0) + return 1; + s = strm->state; + if (s == Z_NULL || s->strm != strm || (s->status != INIT_STATE && +#ifdef GZIP + s->status != GZIP_STATE && +#endif + s->status != EXTRA_STATE && + s->status != NAME_STATE && + s->status != COMMENT_STATE && + s->status != HCRC_STATE && + s->status != BUSY_STATE && + s->status != FINISH_STATE)) + return 1; + return 0; +} + +/* ========================================================================= */ +int ZEXPORT deflateSetDictionary(strm, dictionary, dictLength) + z_streamp strm; + const Bytef *dictionary; + uInt dictLength; +{ + deflate_state *s; + uInt str, n; + int wrap; + unsigned avail; + z_const unsigned char *next; + + if (deflateStateCheck(strm) || dictionary == Z_NULL) + return Z_STREAM_ERROR; + s = strm->state; + wrap = s->wrap; + if (wrap == 2 || (wrap == 1 && s->status != INIT_STATE) || s->lookahead) + return Z_STREAM_ERROR; + + /* when using zlib wrappers, compute Adler-32 for provided dictionary */ + if (wrap == 1) + strm->adler = adler32(strm->adler, dictionary, dictLength); + s->wrap = 0; /* avoid computing Adler-32 in read_buf */ + + /* if dictionary would fill window, just replace the history */ + if (dictLength >= s->w_size) { + if (wrap == 0) { /* already empty otherwise */ + CLEAR_HASH(s); + s->strstart = 0; + s->block_start = 0L; + s->insert = 0; + } + dictionary += dictLength - s->w_size; /* use the tail */ + dictLength = s->w_size; + } + + /* insert dictionary into window and hash */ + avail = strm->avail_in; + next = strm->next_in; + strm->avail_in = dictLength; + strm->next_in = (z_const Bytef *)dictionary; + fill_window(s); + while (s->lookahead >= MIN_MATCH) { + str = s->strstart; + n = s->lookahead - (MIN_MATCH-1); + do { + UPDATE_HASH(s, s->ins_h, s->window[str + MIN_MATCH-1]); +#ifndef FASTEST + s->prev[str & s->w_mask] = s->head[s->ins_h]; +#endif + s->head[s->ins_h] = (Pos)str; + str++; + } while (--n); + s->strstart = str; + s->lookahead = MIN_MATCH-1; + fill_window(s); + } + s->strstart += s->lookahead; + s->block_start = (long)s->strstart; + s->insert = s->lookahead; + s->lookahead = 0; + s->match_length = s->prev_length = MIN_MATCH-1; + s->match_available = 0; + strm->next_in = next; + strm->avail_in = avail; + s->wrap = wrap; + return Z_OK; +} + +/* ========================================================================= */ +int ZEXPORT deflateGetDictionary(strm, dictionary, dictLength) + z_streamp strm; + Bytef *dictionary; + uInt *dictLength; +{ + deflate_state *s; + uInt len; + + if (deflateStateCheck(strm)) + return Z_STREAM_ERROR; + s = strm->state; + len = s->strstart + s->lookahead; + if (len > s->w_size) + len = s->w_size; + if (dictionary != Z_NULL && len) + zmemcpy(dictionary, s->window + s->strstart + s->lookahead - len, len); + if (dictLength != Z_NULL) + *dictLength = len; + return Z_OK; +} + +/* ========================================================================= */ +int ZEXPORT deflateResetKeep(strm) + z_streamp strm; +{ + deflate_state *s; + + if (deflateStateCheck(strm)) { + return Z_STREAM_ERROR; + } + + strm->total_in = strm->total_out = 0; + strm->msg = Z_NULL; /* use zfree if we ever allocate msg dynamically */ + strm->data_type = Z_UNKNOWN; + + s = (deflate_state *)strm->state; + s->pending = 0; + s->pending_out = s->pending_buf; + + if (s->wrap < 0) { + s->wrap = -s->wrap; /* was made negative by deflate(..., Z_FINISH); */ + } + s->status = +#ifdef GZIP + s->wrap == 2 ? GZIP_STATE : +#endif + INIT_STATE; + strm->adler = +#ifdef GZIP + s->wrap == 2 ? crc32(0L, Z_NULL, 0) : +#endif + adler32(0L, Z_NULL, 0); + s->last_flush = -2; + + _tr_init(s); + + return Z_OK; +} + +/* ========================================================================= */ +int ZEXPORT deflateReset(strm) + z_streamp strm; +{ + int ret; + + ret = deflateResetKeep(strm); + if (ret == Z_OK) + lm_init(strm->state); + return ret; +} + +/* ========================================================================= */ +int ZEXPORT deflateSetHeader(strm, head) + z_streamp strm; + gz_headerp head; +{ + if (deflateStateCheck(strm) || strm->state->wrap != 2) + return Z_STREAM_ERROR; + strm->state->gzhead = head; + return Z_OK; +} + +/* ========================================================================= */ +int ZEXPORT deflatePending(strm, pending, bits) + unsigned *pending; + int *bits; + z_streamp strm; +{ + if (deflateStateCheck(strm)) return Z_STREAM_ERROR; + if (pending != Z_NULL) + *pending = strm->state->pending; + if (bits != Z_NULL) + *bits = strm->state->bi_valid; + return Z_OK; +} + +/* ========================================================================= */ +int ZEXPORT deflatePrime(strm, bits, value) + z_streamp strm; + int bits; + int value; +{ + deflate_state *s; + int put; + + if (deflateStateCheck(strm)) return Z_STREAM_ERROR; + s = strm->state; + if (bits < 0 || bits > 16 || + s->sym_buf < s->pending_out + ((Buf_size + 7) >> 3)) + return Z_BUF_ERROR; + do { + put = Buf_size - s->bi_valid; + if (put > bits) + put = bits; + s->bi_buf |= (ush)((value & ((1 << put) - 1)) << s->bi_valid); + s->bi_valid += put; + _tr_flush_bits(s); + value >>= put; + bits -= put; + } while (bits); + return Z_OK; +} + +/* ========================================================================= */ +int ZEXPORT deflateParams(strm, level, strategy) + z_streamp strm; + int level; + int strategy; +{ + deflate_state *s; + compress_func func; + + if (deflateStateCheck(strm)) return Z_STREAM_ERROR; + s = strm->state; + +#ifdef FASTEST + if (level != 0) level = 1; +#else + if (level == Z_DEFAULT_COMPRESSION) level = 6; +#endif + if (level < 0 || level > 9 || strategy < 0 || strategy > Z_FIXED) { + return Z_STREAM_ERROR; + } + func = configuration_table[s->level].func; + + if ((strategy != s->strategy || func != configuration_table[level].func) && + s->last_flush != -2) { + /* Flush the last buffer: */ + int err = deflate(strm, Z_BLOCK); + if (err == Z_STREAM_ERROR) + return err; + if (strm->avail_in || (s->strstart - s->block_start) + s->lookahead) + return Z_BUF_ERROR; + } + if (s->level != level) { + if (s->level == 0 && s->matches != 0) { + if (s->matches == 1) + slide_hash(s); + else + CLEAR_HASH(s); + s->matches = 0; + } + s->level = level; + s->max_lazy_match = configuration_table[level].max_lazy; + s->good_match = configuration_table[level].good_length; + s->nice_match = configuration_table[level].nice_length; + s->max_chain_length = configuration_table[level].max_chain; + } + s->strategy = strategy; + return Z_OK; +} + +/* ========================================================================= */ +int ZEXPORT deflateTune(strm, good_length, max_lazy, nice_length, max_chain) + z_streamp strm; + int good_length; + int max_lazy; + int nice_length; + int max_chain; +{ + deflate_state *s; + + if (deflateStateCheck(strm)) return Z_STREAM_ERROR; + s = strm->state; + s->good_match = (uInt)good_length; + s->max_lazy_match = (uInt)max_lazy; + s->nice_match = nice_length; + s->max_chain_length = (uInt)max_chain; + return Z_OK; +} + +/* ========================================================================= + * For the default windowBits of 15 and memLevel of 8, this function returns a + * close to exact, as well as small, upper bound on the compressed size. This + * is an expansion of ~0.03%, plus a small constant. + * + * For any setting other than those defaults for windowBits and memLevel, one + * of two worst case bounds is returned. This is at most an expansion of ~4% or + * ~13%, plus a small constant. + * + * Both the 0.03% and 4% derive from the overhead of stored blocks. The first + * one is for stored blocks of 16383 bytes (memLevel == 8), whereas the second + * is for stored blocks of 127 bytes (the worst case memLevel == 1). The + * expansion results from five bytes of header for each stored block. + * + * The larger expansion of 13% results from a window size less than or equal to + * the symbols buffer size (windowBits <= memLevel + 7). In that case some of + * the data being compressed may have slid out of the sliding window, impeding + * a stored block from being emitted. Then the only choice is a fixed or + * dynamic block, where a fixed block limits the maximum expansion to 9 bits + * per 8-bit byte, plus 10 bits for every block. The smallest block size for + * which this can occur is 255 (memLevel == 2). + * + * Shifts are used to approximate divisions, for speed. + */ +uLong ZEXPORT deflateBound(strm, sourceLen) + z_streamp strm; + uLong sourceLen; +{ + deflate_state *s; + uLong fixedlen, storelen, wraplen; + + /* upper bound for fixed blocks with 9-bit literals and length 255 + (memLevel == 2, which is the lowest that may not use stored blocks) -- + ~13% overhead plus a small constant */ + fixedlen = sourceLen + (sourceLen >> 3) + (sourceLen >> 8) + + (sourceLen >> 9) + 4; + + /* upper bound for stored blocks with length 127 (memLevel == 1) -- + ~4% overhead plus a small constant */ + storelen = sourceLen + (sourceLen >> 5) + (sourceLen >> 7) + + (sourceLen >> 11) + 7; + + /* if can't get parameters, return larger bound plus a zlib wrapper */ + if (deflateStateCheck(strm)) + return (fixedlen > storelen ? fixedlen : storelen) + 6; + + /* compute wrapper length */ + s = strm->state; + switch (s->wrap) { + case 0: /* raw deflate */ + wraplen = 0; + break; + case 1: /* zlib wrapper */ + wraplen = 6 + (s->strstart ? 4 : 0); + break; +#ifdef GZIP + case 2: /* gzip wrapper */ + wraplen = 18; + if (s->gzhead != Z_NULL) { /* user-supplied gzip header */ + Bytef *str; + if (s->gzhead->extra != Z_NULL) + wraplen += 2 + s->gzhead->extra_len; + str = s->gzhead->name; + if (str != Z_NULL) + do { + wraplen++; + } while (*str++); + str = s->gzhead->comment; + if (str != Z_NULL) + do { + wraplen++; + } while (*str++); + if (s->gzhead->hcrc) + wraplen += 2; + } + break; +#endif + default: /* for compiler happiness */ + wraplen = 6; + } + + /* if not default parameters, return one of the conservative bounds */ + if (s->w_bits != 15 || s->hash_bits != 8 + 7) + return (s->w_bits <= s->hash_bits ? fixedlen : storelen) + wraplen; + + /* default settings: return tight bound for that case -- ~0.03% overhead + plus a small constant */ + return sourceLen + (sourceLen >> 12) + (sourceLen >> 14) + + (sourceLen >> 25) + 13 - 6 + wraplen; +} + +/* ========================================================================= + * Put a short in the pending buffer. The 16-bit value is put in MSB order. + * IN assertion: the stream state is correct and there is enough room in + * pending_buf. + */ +local void putShortMSB(s, b) + deflate_state *s; + uInt b; +{ + put_byte(s, (Byte)(b >> 8)); + put_byte(s, (Byte)(b & 0xff)); +} + +/* ========================================================================= + * Flush as much pending output as possible. All deflate() output, except for + * some deflate_stored() output, goes through this function so some + * applications may wish to modify it to avoid allocating a large + * strm->next_out buffer and copying into it. (See also read_buf()). + */ +local void flush_pending(strm) + z_streamp strm; +{ + unsigned len; + deflate_state *s = strm->state; + + _tr_flush_bits(s); + len = s->pending; + if (len > strm->avail_out) len = strm->avail_out; + if (len == 0) return; + + zmemcpy(strm->next_out, s->pending_out, len); + strm->next_out += len; + s->pending_out += len; + strm->total_out += len; + strm->avail_out -= len; + s->pending -= len; + if (s->pending == 0) { + s->pending_out = s->pending_buf; + } +} + +/* =========================================================================== + * Update the header CRC with the bytes s->pending_buf[beg..s->pending - 1]. + */ +#define HCRC_UPDATE(beg) \ + do { \ + if (s->gzhead->hcrc && s->pending > (beg)) \ + strm->adler = crc32(strm->adler, s->pending_buf + (beg), \ + s->pending - (beg)); \ + } while (0) + +/* ========================================================================= */ +int ZEXPORT deflate(strm, flush) + z_streamp strm; + int flush; +{ + int old_flush; /* value of flush param for previous deflate call */ + deflate_state *s; + + if (deflateStateCheck(strm) || flush > Z_BLOCK || flush < 0) { + return Z_STREAM_ERROR; + } + s = strm->state; + + if (strm->next_out == Z_NULL || + (strm->avail_in != 0 && strm->next_in == Z_NULL) || + (s->status == FINISH_STATE && flush != Z_FINISH)) { + ERR_RETURN(strm, Z_STREAM_ERROR); + } + if (strm->avail_out == 0) ERR_RETURN(strm, Z_BUF_ERROR); + + old_flush = s->last_flush; + s->last_flush = flush; + + /* Flush as much pending output as possible */ + if (s->pending != 0) { + flush_pending(strm); + if (strm->avail_out == 0) { + /* Since avail_out is 0, deflate will be called again with + * more output space, but possibly with both pending and + * avail_in equal to zero. There won't be anything to do, + * but this is not an error situation so make sure we + * return OK instead of BUF_ERROR at next call of deflate: + */ + s->last_flush = -1; + return Z_OK; + } + + /* Make sure there is something to do and avoid duplicate consecutive + * flushes. For repeated and useless calls with Z_FINISH, we keep + * returning Z_STREAM_END instead of Z_BUF_ERROR. + */ + } else if (strm->avail_in == 0 && RANK(flush) <= RANK(old_flush) && + flush != Z_FINISH) { + ERR_RETURN(strm, Z_BUF_ERROR); + } + + /* User must not provide more input after the first FINISH: */ + if (s->status == FINISH_STATE && strm->avail_in != 0) { + ERR_RETURN(strm, Z_BUF_ERROR); + } + + /* Write the header */ + if (s->status == INIT_STATE && s->wrap == 0) + s->status = BUSY_STATE; + if (s->status == INIT_STATE) { + /* zlib header */ + uInt header = (Z_DEFLATED + ((s->w_bits - 8) << 4)) << 8; + uInt level_flags; + + if (s->strategy >= Z_HUFFMAN_ONLY || s->level < 2) + level_flags = 0; + else if (s->level < 6) + level_flags = 1; + else if (s->level == 6) + level_flags = 2; + else + level_flags = 3; + header |= (level_flags << 6); + if (s->strstart != 0) header |= PRESET_DICT; + header += 31 - (header % 31); + + putShortMSB(s, header); + + /* Save the adler32 of the preset dictionary: */ + if (s->strstart != 0) { + putShortMSB(s, (uInt)(strm->adler >> 16)); + putShortMSB(s, (uInt)(strm->adler & 0xffff)); + } + strm->adler = adler32(0L, Z_NULL, 0); + s->status = BUSY_STATE; + + /* Compression must start with an empty pending buffer */ + flush_pending(strm); + if (s->pending != 0) { + s->last_flush = -1; + return Z_OK; + } + } +#ifdef GZIP + if (s->status == GZIP_STATE) { + /* gzip header */ + strm->adler = crc32(0L, Z_NULL, 0); + put_byte(s, 31); + put_byte(s, 139); + put_byte(s, 8); + if (s->gzhead == Z_NULL) { + put_byte(s, 0); + put_byte(s, 0); + put_byte(s, 0); + put_byte(s, 0); + put_byte(s, 0); + put_byte(s, s->level == 9 ? 2 : + (s->strategy >= Z_HUFFMAN_ONLY || s->level < 2 ? + 4 : 0)); + put_byte(s, OS_CODE); + s->status = BUSY_STATE; + + /* Compression must start with an empty pending buffer */ + flush_pending(strm); + if (s->pending != 0) { + s->last_flush = -1; + return Z_OK; + } + } + else { + put_byte(s, (s->gzhead->text ? 1 : 0) + + (s->gzhead->hcrc ? 2 : 0) + + (s->gzhead->extra == Z_NULL ? 0 : 4) + + (s->gzhead->name == Z_NULL ? 0 : 8) + + (s->gzhead->comment == Z_NULL ? 0 : 16) + ); + put_byte(s, (Byte)(s->gzhead->time & 0xff)); + put_byte(s, (Byte)((s->gzhead->time >> 8) & 0xff)); + put_byte(s, (Byte)((s->gzhead->time >> 16) & 0xff)); + put_byte(s, (Byte)((s->gzhead->time >> 24) & 0xff)); + put_byte(s, s->level == 9 ? 2 : + (s->strategy >= Z_HUFFMAN_ONLY || s->level < 2 ? + 4 : 0)); + put_byte(s, s->gzhead->os & 0xff); + if (s->gzhead->extra != Z_NULL) { + put_byte(s, s->gzhead->extra_len & 0xff); + put_byte(s, (s->gzhead->extra_len >> 8) & 0xff); + } + if (s->gzhead->hcrc) + strm->adler = crc32(strm->adler, s->pending_buf, + s->pending); + s->gzindex = 0; + s->status = EXTRA_STATE; + } + } + if (s->status == EXTRA_STATE) { + if (s->gzhead->extra != Z_NULL) { + ulg beg = s->pending; /* start of bytes to update crc */ + uInt left = (s->gzhead->extra_len & 0xffff) - s->gzindex; + while (s->pending + left > s->pending_buf_size) { + uInt copy = s->pending_buf_size - s->pending; + zmemcpy(s->pending_buf + s->pending, + s->gzhead->extra + s->gzindex, copy); + s->pending = s->pending_buf_size; + HCRC_UPDATE(beg); + s->gzindex += copy; + flush_pending(strm); + if (s->pending != 0) { + s->last_flush = -1; + return Z_OK; + } + beg = 0; + left -= copy; + } + zmemcpy(s->pending_buf + s->pending, + s->gzhead->extra + s->gzindex, left); + s->pending += left; + HCRC_UPDATE(beg); + s->gzindex = 0; + } + s->status = NAME_STATE; + } + if (s->status == NAME_STATE) { + if (s->gzhead->name != Z_NULL) { + ulg beg = s->pending; /* start of bytes to update crc */ + int val; + do { + if (s->pending == s->pending_buf_size) { + HCRC_UPDATE(beg); + flush_pending(strm); + if (s->pending != 0) { + s->last_flush = -1; + return Z_OK; + } + beg = 0; + } + val = s->gzhead->name[s->gzindex++]; + put_byte(s, val); + } while (val != 0); + HCRC_UPDATE(beg); + s->gzindex = 0; + } + s->status = COMMENT_STATE; + } + if (s->status == COMMENT_STATE) { + if (s->gzhead->comment != Z_NULL) { + ulg beg = s->pending; /* start of bytes to update crc */ + int val; + do { + if (s->pending == s->pending_buf_size) { + HCRC_UPDATE(beg); + flush_pending(strm); + if (s->pending != 0) { + s->last_flush = -1; + return Z_OK; + } + beg = 0; + } + val = s->gzhead->comment[s->gzindex++]; + put_byte(s, val); + } while (val != 0); + HCRC_UPDATE(beg); + } + s->status = HCRC_STATE; + } + if (s->status == HCRC_STATE) { + if (s->gzhead->hcrc) { + if (s->pending + 2 > s->pending_buf_size) { + flush_pending(strm); + if (s->pending != 0) { + s->last_flush = -1; + return Z_OK; + } + } + put_byte(s, (Byte)(strm->adler & 0xff)); + put_byte(s, (Byte)((strm->adler >> 8) & 0xff)); + strm->adler = crc32(0L, Z_NULL, 0); + } + s->status = BUSY_STATE; + + /* Compression must start with an empty pending buffer */ + flush_pending(strm); + if (s->pending != 0) { + s->last_flush = -1; + return Z_OK; + } + } +#endif + + /* Start a new block or continue the current one. + */ + if (strm->avail_in != 0 || s->lookahead != 0 || + (flush != Z_NO_FLUSH && s->status != FINISH_STATE)) { + block_state bstate; + + bstate = s->level == 0 ? deflate_stored(s, flush) : + s->strategy == Z_HUFFMAN_ONLY ? deflate_huff(s, flush) : + s->strategy == Z_RLE ? deflate_rle(s, flush) : + (*(configuration_table[s->level].func))(s, flush); + + if (bstate == finish_started || bstate == finish_done) { + s->status = FINISH_STATE; + } + if (bstate == need_more || bstate == finish_started) { + if (strm->avail_out == 0) { + s->last_flush = -1; /* avoid BUF_ERROR next call, see above */ + } + return Z_OK; + /* If flush != Z_NO_FLUSH && avail_out == 0, the next call + * of deflate should use the same flush parameter to make sure + * that the flush is complete. So we don't have to output an + * empty block here, this will be done at next call. This also + * ensures that for a very small output buffer, we emit at most + * one empty block. + */ + } + if (bstate == block_done) { + if (flush == Z_PARTIAL_FLUSH) { + _tr_align(s); + } else if (flush != Z_BLOCK) { /* FULL_FLUSH or SYNC_FLUSH */ + _tr_stored_block(s, (char*)0, 0L, 0); + /* For a full flush, this empty block will be recognized + * as a special marker by inflate_sync(). + */ + if (flush == Z_FULL_FLUSH) { + CLEAR_HASH(s); /* forget history */ + if (s->lookahead == 0) { + s->strstart = 0; + s->block_start = 0L; + s->insert = 0; + } + } + } + flush_pending(strm); + if (strm->avail_out == 0) { + s->last_flush = -1; /* avoid BUF_ERROR at next call, see above */ + return Z_OK; + } + } + } + + if (flush != Z_FINISH) return Z_OK; + if (s->wrap <= 0) return Z_STREAM_END; + + /* Write the trailer */ +#ifdef GZIP + if (s->wrap == 2) { + put_byte(s, (Byte)(strm->adler & 0xff)); + put_byte(s, (Byte)((strm->adler >> 8) & 0xff)); + put_byte(s, (Byte)((strm->adler >> 16) & 0xff)); + put_byte(s, (Byte)((strm->adler >> 24) & 0xff)); + put_byte(s, (Byte)(strm->total_in & 0xff)); + put_byte(s, (Byte)((strm->total_in >> 8) & 0xff)); + put_byte(s, (Byte)((strm->total_in >> 16) & 0xff)); + put_byte(s, (Byte)((strm->total_in >> 24) & 0xff)); + } + else +#endif + { + putShortMSB(s, (uInt)(strm->adler >> 16)); + putShortMSB(s, (uInt)(strm->adler & 0xffff)); + } + flush_pending(strm); + /* If avail_out is zero, the application will call deflate again + * to flush the rest. + */ + if (s->wrap > 0) s->wrap = -s->wrap; /* write the trailer only once! */ + return s->pending != 0 ? Z_OK : Z_STREAM_END; +} + +/* ========================================================================= */ +int ZEXPORT deflateEnd(strm) + z_streamp strm; +{ + int status; + + if (deflateStateCheck(strm)) return Z_STREAM_ERROR; + + status = strm->state->status; + + /* Deallocate in reverse order of allocations: */ + TRY_FREE(strm, strm->state->pending_buf); + TRY_FREE(strm, strm->state->head); + TRY_FREE(strm, strm->state->prev); + TRY_FREE(strm, strm->state->window); + + ZFREE(strm, strm->state); + strm->state = Z_NULL; + + return status == BUSY_STATE ? Z_DATA_ERROR : Z_OK; +} + +/* ========================================================================= + * Copy the source state to the destination state. + * To simplify the source, this is not supported for 16-bit MSDOS (which + * doesn't have enough memory anyway to duplicate compression states). + */ +int ZEXPORT deflateCopy(dest, source) + z_streamp dest; + z_streamp source; +{ +#ifdef MAXSEG_64K + return Z_STREAM_ERROR; +#else + deflate_state *ds; + deflate_state *ss; + + + if (deflateStateCheck(source) || dest == Z_NULL) { + return Z_STREAM_ERROR; + } + + ss = source->state; + + zmemcpy((voidpf)dest, (voidpf)source, sizeof(z_stream)); + + ds = (deflate_state *) ZALLOC(dest, 1, sizeof(deflate_state)); + if (ds == Z_NULL) return Z_MEM_ERROR; + dest->state = (struct internal_state FAR *) ds; + zmemcpy((voidpf)ds, (voidpf)ss, sizeof(deflate_state)); + ds->strm = dest; + + ds->window = (Bytef *) ZALLOC(dest, ds->w_size, 2*sizeof(Byte)); + ds->prev = (Posf *) ZALLOC(dest, ds->w_size, sizeof(Pos)); + ds->head = (Posf *) ZALLOC(dest, ds->hash_size, sizeof(Pos)); + ds->pending_buf = (uchf *) ZALLOC(dest, ds->lit_bufsize, 4); + + if (ds->window == Z_NULL || ds->prev == Z_NULL || ds->head == Z_NULL || + ds->pending_buf == Z_NULL) { + deflateEnd (dest); + return Z_MEM_ERROR; + } + /* following zmemcpy do not work for 16-bit MSDOS */ + zmemcpy(ds->window, ss->window, ds->w_size * 2 * sizeof(Byte)); + zmemcpy((voidpf)ds->prev, (voidpf)ss->prev, ds->w_size * sizeof(Pos)); + zmemcpy((voidpf)ds->head, (voidpf)ss->head, ds->hash_size * sizeof(Pos)); + zmemcpy(ds->pending_buf, ss->pending_buf, (uInt)ds->pending_buf_size); + + ds->pending_out = ds->pending_buf + (ss->pending_out - ss->pending_buf); + ds->sym_buf = ds->pending_buf + ds->lit_bufsize; + + ds->l_desc.dyn_tree = ds->dyn_ltree; + ds->d_desc.dyn_tree = ds->dyn_dtree; + ds->bl_desc.dyn_tree = ds->bl_tree; + + return Z_OK; +#endif /* MAXSEG_64K */ +} + +/* =========================================================================== + * Read a new buffer from the current input stream, update the adler32 + * and total number of bytes read. All deflate() input goes through + * this function so some applications may wish to modify it to avoid + * allocating a large strm->next_in buffer and copying from it. + * (See also flush_pending()). + */ +local unsigned read_buf(strm, buf, size) + z_streamp strm; + Bytef *buf; + unsigned size; +{ + unsigned len = strm->avail_in; + + if (len > size) len = size; + if (len == 0) return 0; + + strm->avail_in -= len; + + zmemcpy(buf, strm->next_in, len); + if (strm->state->wrap == 1) { + strm->adler = adler32(strm->adler, buf, len); + } +#ifdef GZIP + else if (strm->state->wrap == 2) { + strm->adler = crc32(strm->adler, buf, len); + } +#endif + strm->next_in += len; + strm->total_in += len; + + return len; +} + +/* =========================================================================== + * Initialize the "longest match" routines for a new zlib stream + */ +local void lm_init(s) + deflate_state *s; +{ + s->window_size = (ulg)2L*s->w_size; + + CLEAR_HASH(s); + + /* Set the default configuration parameters: + */ + s->max_lazy_match = configuration_table[s->level].max_lazy; + s->good_match = configuration_table[s->level].good_length; + s->nice_match = configuration_table[s->level].nice_length; + s->max_chain_length = configuration_table[s->level].max_chain; + + s->strstart = 0; + s->block_start = 0L; + s->lookahead = 0; + s->insert = 0; + s->match_length = s->prev_length = MIN_MATCH-1; + s->match_available = 0; + s->ins_h = 0; +} + +#ifndef FASTEST +/* =========================================================================== + * Set match_start to the longest match starting at the given string and + * return its length. Matches shorter or equal to prev_length are discarded, + * in which case the result is equal to prev_length and match_start is + * garbage. + * IN assertions: cur_match is the head of the hash chain for the current + * string (strstart) and its distance is <= MAX_DIST, and prev_length >= 1 + * OUT assertion: the match length is not greater than s->lookahead. + */ +local uInt longest_match(s, cur_match) + deflate_state *s; + IPos cur_match; /* current match */ +{ + unsigned chain_length = s->max_chain_length;/* max hash chain length */ + register Bytef *scan = s->window + s->strstart; /* current string */ + register Bytef *match; /* matched string */ + register int len; /* length of current match */ + int best_len = (int)s->prev_length; /* best match length so far */ + int nice_match = s->nice_match; /* stop if match long enough */ + IPos limit = s->strstart > (IPos)MAX_DIST(s) ? + s->strstart - (IPos)MAX_DIST(s) : NIL; + /* Stop when cur_match becomes <= limit. To simplify the code, + * we prevent matches with the string of window index 0. + */ + Posf *prev = s->prev; + uInt wmask = s->w_mask; + +#ifdef UNALIGNED_OK + /* Compare two bytes at a time. Note: this is not always beneficial. + * Try with and without -DUNALIGNED_OK to check. + */ + register Bytef *strend = s->window + s->strstart + MAX_MATCH - 1; + register ush scan_start = *(ushf*)scan; + register ush scan_end = *(ushf*)(scan + best_len - 1); +#else + register Bytef *strend = s->window + s->strstart + MAX_MATCH; + register Byte scan_end1 = scan[best_len - 1]; + register Byte scan_end = scan[best_len]; +#endif + + /* The code is optimized for HASH_BITS >= 8 and MAX_MATCH-2 multiple of 16. + * It is easy to get rid of this optimization if necessary. + */ + Assert(s->hash_bits >= 8 && MAX_MATCH == 258, "Code too clever"); + + /* Do not waste too much time if we already have a good match: */ + if (s->prev_length >= s->good_match) { + chain_length >>= 2; + } + /* Do not look for matches beyond the end of the input. This is necessary + * to make deflate deterministic. + */ + if ((uInt)nice_match > s->lookahead) nice_match = (int)s->lookahead; + + Assert((ulg)s->strstart <= s->window_size - MIN_LOOKAHEAD, + "need lookahead"); + + do { + Assert(cur_match < s->strstart, "no future"); + match = s->window + cur_match; + + /* Skip to next match if the match length cannot increase + * or if the match length is less than 2. Note that the checks below + * for insufficient lookahead only occur occasionally for performance + * reasons. Therefore uninitialized memory will be accessed, and + * conditional jumps will be made that depend on those values. + * However the length of the match is limited to the lookahead, so + * the output of deflate is not affected by the uninitialized values. + */ +#if (defined(UNALIGNED_OK) && MAX_MATCH == 258) + /* This code assumes sizeof(unsigned short) == 2. Do not use + * UNALIGNED_OK if your compiler uses a different size. + */ + if (*(ushf*)(match + best_len - 1) != scan_end || + *(ushf*)match != scan_start) continue; + + /* It is not necessary to compare scan[2] and match[2] since they are + * always equal when the other bytes match, given that the hash keys + * are equal and that HASH_BITS >= 8. Compare 2 bytes at a time at + * strstart + 3, + 5, up to strstart + 257. We check for insufficient + * lookahead only every 4th comparison; the 128th check will be made + * at strstart + 257. If MAX_MATCH-2 is not a multiple of 8, it is + * necessary to put more guard bytes at the end of the window, or + * to check more often for insufficient lookahead. + */ + Assert(scan[2] == match[2], "scan[2]?"); + scan++, match++; + do { + } while (*(ushf*)(scan += 2) == *(ushf*)(match += 2) && + *(ushf*)(scan += 2) == *(ushf*)(match += 2) && + *(ushf*)(scan += 2) == *(ushf*)(match += 2) && + *(ushf*)(scan += 2) == *(ushf*)(match += 2) && + scan < strend); + /* The funny "do {}" generates better code on most compilers */ + + /* Here, scan <= window + strstart + 257 */ + Assert(scan <= s->window + (unsigned)(s->window_size - 1), + "wild scan"); + if (*scan == *match) scan++; + + len = (MAX_MATCH - 1) - (int)(strend - scan); + scan = strend - (MAX_MATCH-1); + +#else /* UNALIGNED_OK */ + + if (match[best_len] != scan_end || + match[best_len - 1] != scan_end1 || + *match != *scan || + *++match != scan[1]) continue; + + /* The check at best_len - 1 can be removed because it will be made + * again later. (This heuristic is not always a win.) + * It is not necessary to compare scan[2] and match[2] since they + * are always equal when the other bytes match, given that + * the hash keys are equal and that HASH_BITS >= 8. + */ + scan += 2, match++; + Assert(*scan == *match, "match[2]?"); + + /* We check for insufficient lookahead only every 8th comparison; + * the 256th check will be made at strstart + 258. + */ + do { + } while (*++scan == *++match && *++scan == *++match && + *++scan == *++match && *++scan == *++match && + *++scan == *++match && *++scan == *++match && + *++scan == *++match && *++scan == *++match && + scan < strend); + + Assert(scan <= s->window + (unsigned)(s->window_size - 1), + "wild scan"); + + len = MAX_MATCH - (int)(strend - scan); + scan = strend - MAX_MATCH; + +#endif /* UNALIGNED_OK */ + + if (len > best_len) { + s->match_start = cur_match; + best_len = len; + if (len >= nice_match) break; +#ifdef UNALIGNED_OK + scan_end = *(ushf*)(scan + best_len - 1); +#else + scan_end1 = scan[best_len - 1]; + scan_end = scan[best_len]; +#endif + } + } while ((cur_match = prev[cur_match & wmask]) > limit + && --chain_length != 0); + + if ((uInt)best_len <= s->lookahead) return (uInt)best_len; + return s->lookahead; +} + +#else /* FASTEST */ + +/* --------------------------------------------------------------------------- + * Optimized version for FASTEST only + */ +local uInt longest_match(s, cur_match) + deflate_state *s; + IPos cur_match; /* current match */ +{ + register Bytef *scan = s->window + s->strstart; /* current string */ + register Bytef *match; /* matched string */ + register int len; /* length of current match */ + register Bytef *strend = s->window + s->strstart + MAX_MATCH; + + /* The code is optimized for HASH_BITS >= 8 and MAX_MATCH-2 multiple of 16. + * It is easy to get rid of this optimization if necessary. + */ + Assert(s->hash_bits >= 8 && MAX_MATCH == 258, "Code too clever"); + + Assert((ulg)s->strstart <= s->window_size - MIN_LOOKAHEAD, + "need lookahead"); + + Assert(cur_match < s->strstart, "no future"); + + match = s->window + cur_match; + + /* Return failure if the match length is less than 2: + */ + if (match[0] != scan[0] || match[1] != scan[1]) return MIN_MATCH-1; + + /* The check at best_len - 1 can be removed because it will be made + * again later. (This heuristic is not always a win.) + * It is not necessary to compare scan[2] and match[2] since they + * are always equal when the other bytes match, given that + * the hash keys are equal and that HASH_BITS >= 8. + */ + scan += 2, match += 2; + Assert(*scan == *match, "match[2]?"); + + /* We check for insufficient lookahead only every 8th comparison; + * the 256th check will be made at strstart + 258. + */ + do { + } while (*++scan == *++match && *++scan == *++match && + *++scan == *++match && *++scan == *++match && + *++scan == *++match && *++scan == *++match && + *++scan == *++match && *++scan == *++match && + scan < strend); + + Assert(scan <= s->window + (unsigned)(s->window_size - 1), "wild scan"); + + len = MAX_MATCH - (int)(strend - scan); + + if (len < MIN_MATCH) return MIN_MATCH - 1; + + s->match_start = cur_match; + return (uInt)len <= s->lookahead ? (uInt)len : s->lookahead; +} + +#endif /* FASTEST */ + +#ifdef ZLIB_DEBUG + +#define EQUAL 0 +/* result of memcmp for equal strings */ + +/* =========================================================================== + * Check that the match at match_start is indeed a match. + */ +local void check_match(s, start, match, length) + deflate_state *s; + IPos start, match; + int length; +{ + /* check that the match is indeed a match */ + if (zmemcmp(s->window + match, + s->window + start, length) != EQUAL) { + fprintf(stderr, " start %u, match %u, length %d\n", + start, match, length); + do { + fprintf(stderr, "%c%c", s->window[match++], s->window[start++]); + } while (--length != 0); + z_error("invalid match"); + } + if (z_verbose > 1) { + fprintf(stderr,"\\[%d,%d]", start - match, length); + do { putc(s->window[start++], stderr); } while (--length != 0); + } +} +#else +# define check_match(s, start, match, length) +#endif /* ZLIB_DEBUG */ + +/* =========================================================================== + * Fill the window when the lookahead becomes insufficient. + * Updates strstart and lookahead. + * + * IN assertion: lookahead < MIN_LOOKAHEAD + * OUT assertions: strstart <= window_size-MIN_LOOKAHEAD + * At least one byte has been read, or avail_in == 0; reads are + * performed for at least two bytes (required for the zip translate_eol + * option -- not supported here). + */ +local void fill_window(s) + deflate_state *s; +{ + unsigned n; + unsigned more; /* Amount of free space at the end of the window. */ + uInt wsize = s->w_size; + + Assert(s->lookahead < MIN_LOOKAHEAD, "already enough lookahead"); + + do { + more = (unsigned)(s->window_size -(ulg)s->lookahead -(ulg)s->strstart); + + /* Deal with !@#$% 64K limit: */ + if (sizeof(int) <= 2) { + if (more == 0 && s->strstart == 0 && s->lookahead == 0) { + more = wsize; + + } else if (more == (unsigned)(-1)) { + /* Very unlikely, but possible on 16 bit machine if + * strstart == 0 && lookahead == 1 (input done a byte at time) + */ + more--; + } + } + + /* If the window is almost full and there is insufficient lookahead, + * move the upper half to the lower one to make room in the upper half. + */ + if (s->strstart >= wsize + MAX_DIST(s)) { + + zmemcpy(s->window, s->window + wsize, (unsigned)wsize - more); + s->match_start -= wsize; + s->strstart -= wsize; /* we now have strstart >= MAX_DIST */ + s->block_start -= (long) wsize; + if (s->insert > s->strstart) + s->insert = s->strstart; + slide_hash(s); + more += wsize; + } + if (s->strm->avail_in == 0) break; + + /* If there was no sliding: + * strstart <= WSIZE+MAX_DIST-1 && lookahead <= MIN_LOOKAHEAD - 1 && + * more == window_size - lookahead - strstart + * => more >= window_size - (MIN_LOOKAHEAD-1 + WSIZE + MAX_DIST-1) + * => more >= window_size - 2*WSIZE + 2 + * In the BIG_MEM or MMAP case (not yet supported), + * window_size == input_size + MIN_LOOKAHEAD && + * strstart + s->lookahead <= input_size => more >= MIN_LOOKAHEAD. + * Otherwise, window_size == 2*WSIZE so more >= 2. + * If there was sliding, more >= WSIZE. So in all cases, more >= 2. + */ + Assert(more >= 2, "more < 2"); + + n = read_buf(s->strm, s->window + s->strstart + s->lookahead, more); + s->lookahead += n; + + /* Initialize the hash value now that we have some input: */ + if (s->lookahead + s->insert >= MIN_MATCH) { + uInt str = s->strstart - s->insert; + s->ins_h = s->window[str]; + UPDATE_HASH(s, s->ins_h, s->window[str + 1]); +#if MIN_MATCH != 3 + Call UPDATE_HASH() MIN_MATCH-3 more times +#endif + while (s->insert) { + UPDATE_HASH(s, s->ins_h, s->window[str + MIN_MATCH-1]); +#ifndef FASTEST + s->prev[str & s->w_mask] = s->head[s->ins_h]; +#endif + s->head[s->ins_h] = (Pos)str; + str++; + s->insert--; + if (s->lookahead + s->insert < MIN_MATCH) + break; + } + } + /* If the whole input has less than MIN_MATCH bytes, ins_h is garbage, + * but this is not important since only literal bytes will be emitted. + */ + + } while (s->lookahead < MIN_LOOKAHEAD && s->strm->avail_in != 0); + + /* If the WIN_INIT bytes after the end of the current data have never been + * written, then zero those bytes in order to avoid memory check reports of + * the use of uninitialized (or uninitialised as Julian writes) bytes by + * the longest match routines. Update the high water mark for the next + * time through here. WIN_INIT is set to MAX_MATCH since the longest match + * routines allow scanning to strstart + MAX_MATCH, ignoring lookahead. + */ + if (s->high_water < s->window_size) { + ulg curr = s->strstart + (ulg)(s->lookahead); + ulg init; + + if (s->high_water < curr) { + /* Previous high water mark below current data -- zero WIN_INIT + * bytes or up to end of window, whichever is less. + */ + init = s->window_size - curr; + if (init > WIN_INIT) + init = WIN_INIT; + zmemzero(s->window + curr, (unsigned)init); + s->high_water = curr + init; + } + else if (s->high_water < (ulg)curr + WIN_INIT) { + /* High water mark at or above current data, but below current data + * plus WIN_INIT -- zero out to current data plus WIN_INIT, or up + * to end of window, whichever is less. + */ + init = (ulg)curr + WIN_INIT - s->high_water; + if (init > s->window_size - s->high_water) + init = s->window_size - s->high_water; + zmemzero(s->window + s->high_water, (unsigned)init); + s->high_water += init; + } + } + + Assert((ulg)s->strstart <= s->window_size - MIN_LOOKAHEAD, + "not enough room for search"); +} + +/* =========================================================================== + * Flush the current block, with given end-of-file flag. + * IN assertion: strstart is set to the end of the current match. + */ +#define FLUSH_BLOCK_ONLY(s, last) { \ + _tr_flush_block(s, (s->block_start >= 0L ? \ + (charf *)&s->window[(unsigned)s->block_start] : \ + (charf *)Z_NULL), \ + (ulg)((long)s->strstart - s->block_start), \ + (last)); \ + s->block_start = s->strstart; \ + flush_pending(s->strm); \ + Tracev((stderr,"[FLUSH]")); \ +} + +/* Same but force premature exit if necessary. */ +#define FLUSH_BLOCK(s, last) { \ + FLUSH_BLOCK_ONLY(s, last); \ + if (s->strm->avail_out == 0) return (last) ? finish_started : need_more; \ +} + +/* Maximum stored block length in deflate format (not including header). */ +#define MAX_STORED 65535 + +/* Minimum of a and b. */ +#define MIN(a, b) ((a) > (b) ? (b) : (a)) + +/* =========================================================================== + * Copy without compression as much as possible from the input stream, return + * the current block state. + * + * In case deflateParams() is used to later switch to a non-zero compression + * level, s->matches (otherwise unused when storing) keeps track of the number + * of hash table slides to perform. If s->matches is 1, then one hash table + * slide will be done when switching. If s->matches is 2, the maximum value + * allowed here, then the hash table will be cleared, since two or more slides + * is the same as a clear. + * + * deflate_stored() is written to minimize the number of times an input byte is + * copied. It is most efficient with large input and output buffers, which + * maximizes the opportunities to have a single copy from next_in to next_out. + */ +local block_state deflate_stored(s, flush) + deflate_state *s; + int flush; +{ + /* Smallest worthy block size when not flushing or finishing. By default + * this is 32K. This can be as small as 507 bytes for memLevel == 1. For + * large input and output buffers, the stored block size will be larger. + */ + unsigned min_block = MIN(s->pending_buf_size - 5, s->w_size); + + /* Copy as many min_block or larger stored blocks directly to next_out as + * possible. If flushing, copy the remaining available input to next_out as + * stored blocks, if there is enough space. + */ + unsigned len, left, have, last = 0; + unsigned used = s->strm->avail_in; + do { + /* Set len to the maximum size block that we can copy directly with the + * available input data and output space. Set left to how much of that + * would be copied from what's left in the window. + */ + len = MAX_STORED; /* maximum deflate stored block length */ + have = (s->bi_valid + 42) >> 3; /* number of header bytes */ + if (s->strm->avail_out < have) /* need room for header */ + break; + /* maximum stored block length that will fit in avail_out: */ + have = s->strm->avail_out - have; + left = s->strstart - s->block_start; /* bytes left in window */ + if (len > (ulg)left + s->strm->avail_in) + len = left + s->strm->avail_in; /* limit len to the input */ + if (len > have) + len = have; /* limit len to the output */ + + /* If the stored block would be less than min_block in length, or if + * unable to copy all of the available input when flushing, then try + * copying to the window and the pending buffer instead. Also don't + * write an empty block when flushing -- deflate() does that. + */ + if (len < min_block && ((len == 0 && flush != Z_FINISH) || + flush == Z_NO_FLUSH || + len != left + s->strm->avail_in)) + break; + + /* Make a dummy stored block in pending to get the header bytes, + * including any pending bits. This also updates the debugging counts. + */ + last = flush == Z_FINISH && len == left + s->strm->avail_in ? 1 : 0; + _tr_stored_block(s, (char *)0, 0L, last); + + /* Replace the lengths in the dummy stored block with len. */ + s->pending_buf[s->pending - 4] = len; + s->pending_buf[s->pending - 3] = len >> 8; + s->pending_buf[s->pending - 2] = ~len; + s->pending_buf[s->pending - 1] = ~len >> 8; + + /* Write the stored block header bytes. */ + flush_pending(s->strm); + +#ifdef ZLIB_DEBUG + /* Update debugging counts for the data about to be copied. */ + s->compressed_len += len << 3; + s->bits_sent += len << 3; +#endif + + /* Copy uncompressed bytes from the window to next_out. */ + if (left) { + if (left > len) + left = len; + zmemcpy(s->strm->next_out, s->window + s->block_start, left); + s->strm->next_out += left; + s->strm->avail_out -= left; + s->strm->total_out += left; + s->block_start += left; + len -= left; + } + + /* Copy uncompressed bytes directly from next_in to next_out, updating + * the check value. + */ + if (len) { + read_buf(s->strm, s->strm->next_out, len); + s->strm->next_out += len; + s->strm->avail_out -= len; + s->strm->total_out += len; + } + } while (last == 0); + + /* Update the sliding window with the last s->w_size bytes of the copied + * data, or append all of the copied data to the existing window if less + * than s->w_size bytes were copied. Also update the number of bytes to + * insert in the hash tables, in the event that deflateParams() switches to + * a non-zero compression level. + */ + used -= s->strm->avail_in; /* number of input bytes directly copied */ + if (used) { + /* If any input was used, then no unused input remains in the window, + * therefore s->block_start == s->strstart. + */ + if (used >= s->w_size) { /* supplant the previous history */ + s->matches = 2; /* clear hash */ + zmemcpy(s->window, s->strm->next_in - s->w_size, s->w_size); + s->strstart = s->w_size; + s->insert = s->strstart; + } + else { + if (s->window_size - s->strstart <= used) { + /* Slide the window down. */ + s->strstart -= s->w_size; + zmemcpy(s->window, s->window + s->w_size, s->strstart); + if (s->matches < 2) + s->matches++; /* add a pending slide_hash() */ + if (s->insert > s->strstart) + s->insert = s->strstart; + } + zmemcpy(s->window + s->strstart, s->strm->next_in - used, used); + s->strstart += used; + s->insert += MIN(used, s->w_size - s->insert); + } + s->block_start = s->strstart; + } + if (s->high_water < s->strstart) + s->high_water = s->strstart; + + /* If the last block was written to next_out, then done. */ + if (last) + return finish_done; + + /* If flushing and all input has been consumed, then done. */ + if (flush != Z_NO_FLUSH && flush != Z_FINISH && + s->strm->avail_in == 0 && (long)s->strstart == s->block_start) + return block_done; + + /* Fill the window with any remaining input. */ + have = s->window_size - s->strstart; + if (s->strm->avail_in > have && s->block_start >= (long)s->w_size) { + /* Slide the window down. */ + s->block_start -= s->w_size; + s->strstart -= s->w_size; + zmemcpy(s->window, s->window + s->w_size, s->strstart); + if (s->matches < 2) + s->matches++; /* add a pending slide_hash() */ + have += s->w_size; /* more space now */ + if (s->insert > s->strstart) + s->insert = s->strstart; + } + if (have > s->strm->avail_in) + have = s->strm->avail_in; + if (have) { + read_buf(s->strm, s->window + s->strstart, have); + s->strstart += have; + s->insert += MIN(have, s->w_size - s->insert); + } + if (s->high_water < s->strstart) + s->high_water = s->strstart; + + /* There was not enough avail_out to write a complete worthy or flushed + * stored block to next_out. Write a stored block to pending instead, if we + * have enough input for a worthy block, or if flushing and there is enough + * room for the remaining input as a stored block in the pending buffer. + */ + have = (s->bi_valid + 42) >> 3; /* number of header bytes */ + /* maximum stored block length that will fit in pending: */ + have = MIN(s->pending_buf_size - have, MAX_STORED); + min_block = MIN(have, s->w_size); + left = s->strstart - s->block_start; + if (left >= min_block || + ((left || flush == Z_FINISH) && flush != Z_NO_FLUSH && + s->strm->avail_in == 0 && left <= have)) { + len = MIN(left, have); + last = flush == Z_FINISH && s->strm->avail_in == 0 && + len == left ? 1 : 0; + _tr_stored_block(s, (charf *)s->window + s->block_start, len, last); + s->block_start += len; + flush_pending(s->strm); + } + + /* We've done all we can with the available input and output. */ + return last ? finish_started : need_more; +} + +/* =========================================================================== + * Compress as much as possible from the input stream, return the current + * block state. + * This function does not perform lazy evaluation of matches and inserts + * new strings in the dictionary only for unmatched strings or for short + * matches. It is used only for the fast compression options. + */ +local block_state deflate_fast(s, flush) + deflate_state *s; + int flush; +{ + IPos hash_head; /* head of the hash chain */ + int bflush; /* set if current block must be flushed */ + + for (;;) { + /* Make sure that we always have enough lookahead, except + * at the end of the input file. We need MAX_MATCH bytes + * for the next match, plus MIN_MATCH bytes to insert the + * string following the next match. + */ + if (s->lookahead < MIN_LOOKAHEAD) { + fill_window(s); + if (s->lookahead < MIN_LOOKAHEAD && flush == Z_NO_FLUSH) { + return need_more; + } + if (s->lookahead == 0) break; /* flush the current block */ + } + + /* Insert the string window[strstart .. strstart + 2] in the + * dictionary, and set hash_head to the head of the hash chain: + */ + hash_head = NIL; + if (s->lookahead >= MIN_MATCH) { + INSERT_STRING(s, s->strstart, hash_head); + } + + /* Find the longest match, discarding those <= prev_length. + * At this point we have always match_length < MIN_MATCH + */ + if (hash_head != NIL && s->strstart - hash_head <= MAX_DIST(s)) { + /* To simplify the code, we prevent matches with the string + * of window index 0 (in particular we have to avoid a match + * of the string with itself at the start of the input file). + */ + s->match_length = longest_match (s, hash_head); + /* longest_match() sets match_start */ + } + if (s->match_length >= MIN_MATCH) { + check_match(s, s->strstart, s->match_start, s->match_length); + + _tr_tally_dist(s, s->strstart - s->match_start, + s->match_length - MIN_MATCH, bflush); + + s->lookahead -= s->match_length; + + /* Insert new strings in the hash table only if the match length + * is not too large. This saves time but degrades compression. + */ +#ifndef FASTEST + if (s->match_length <= s->max_insert_length && + s->lookahead >= MIN_MATCH) { + s->match_length--; /* string at strstart already in table */ + do { + s->strstart++; + INSERT_STRING(s, s->strstart, hash_head); + /* strstart never exceeds WSIZE-MAX_MATCH, so there are + * always MIN_MATCH bytes ahead. + */ + } while (--s->match_length != 0); + s->strstart++; + } else +#endif + { + s->strstart += s->match_length; + s->match_length = 0; + s->ins_h = s->window[s->strstart]; + UPDATE_HASH(s, s->ins_h, s->window[s->strstart + 1]); +#if MIN_MATCH != 3 + Call UPDATE_HASH() MIN_MATCH-3 more times +#endif + /* If lookahead < MIN_MATCH, ins_h is garbage, but it does not + * matter since it will be recomputed at next deflate call. + */ + } + } else { + /* No match, output a literal byte */ + Tracevv((stderr,"%c", s->window[s->strstart])); + _tr_tally_lit(s, s->window[s->strstart], bflush); + s->lookahead--; + s->strstart++; + } + if (bflush) FLUSH_BLOCK(s, 0); + } + s->insert = s->strstart < MIN_MATCH-1 ? s->strstart : MIN_MATCH-1; + if (flush == Z_FINISH) { + FLUSH_BLOCK(s, 1); + return finish_done; + } + if (s->sym_next) + FLUSH_BLOCK(s, 0); + return block_done; +} + +#ifndef FASTEST +/* =========================================================================== + * Same as above, but achieves better compression. We use a lazy + * evaluation for matches: a match is finally adopted only if there is + * no better match at the next window position. + */ +local block_state deflate_slow(s, flush) + deflate_state *s; + int flush; +{ + IPos hash_head; /* head of hash chain */ + int bflush; /* set if current block must be flushed */ + + /* Process the input block. */ + for (;;) { + /* Make sure that we always have enough lookahead, except + * at the end of the input file. We need MAX_MATCH bytes + * for the next match, plus MIN_MATCH bytes to insert the + * string following the next match. + */ + if (s->lookahead < MIN_LOOKAHEAD) { + fill_window(s); + if (s->lookahead < MIN_LOOKAHEAD && flush == Z_NO_FLUSH) { + return need_more; + } + if (s->lookahead == 0) break; /* flush the current block */ + } + + /* Insert the string window[strstart .. strstart + 2] in the + * dictionary, and set hash_head to the head of the hash chain: + */ + hash_head = NIL; + if (s->lookahead >= MIN_MATCH) { + INSERT_STRING(s, s->strstart, hash_head); + } + + /* Find the longest match, discarding those <= prev_length. + */ + s->prev_length = s->match_length, s->prev_match = s->match_start; + s->match_length = MIN_MATCH-1; + + if (hash_head != NIL && s->prev_length < s->max_lazy_match && + s->strstart - hash_head <= MAX_DIST(s)) { + /* To simplify the code, we prevent matches with the string + * of window index 0 (in particular we have to avoid a match + * of the string with itself at the start of the input file). + */ + s->match_length = longest_match (s, hash_head); + /* longest_match() sets match_start */ + + if (s->match_length <= 5 && (s->strategy == Z_FILTERED +#if TOO_FAR <= 32767 + || (s->match_length == MIN_MATCH && + s->strstart - s->match_start > TOO_FAR) +#endif + )) { + + /* If prev_match is also MIN_MATCH, match_start is garbage + * but we will ignore the current match anyway. + */ + s->match_length = MIN_MATCH-1; + } + } + /* If there was a match at the previous step and the current + * match is not better, output the previous match: + */ + if (s->prev_length >= MIN_MATCH && s->match_length <= s->prev_length) { + uInt max_insert = s->strstart + s->lookahead - MIN_MATCH; + /* Do not insert strings in hash table beyond this. */ + + check_match(s, s->strstart - 1, s->prev_match, s->prev_length); + + _tr_tally_dist(s, s->strstart - 1 - s->prev_match, + s->prev_length - MIN_MATCH, bflush); + + /* Insert in hash table all strings up to the end of the match. + * strstart - 1 and strstart are already inserted. If there is not + * enough lookahead, the last two strings are not inserted in + * the hash table. + */ + s->lookahead -= s->prev_length - 1; + s->prev_length -= 2; + do { + if (++s->strstart <= max_insert) { + INSERT_STRING(s, s->strstart, hash_head); + } + } while (--s->prev_length != 0); + s->match_available = 0; + s->match_length = MIN_MATCH-1; + s->strstart++; + + if (bflush) FLUSH_BLOCK(s, 0); + + } else if (s->match_available) { + /* If there was no match at the previous position, output a + * single literal. If there was a match but the current match + * is longer, truncate the previous match to a single literal. + */ + Tracevv((stderr,"%c", s->window[s->strstart - 1])); + _tr_tally_lit(s, s->window[s->strstart - 1], bflush); + if (bflush) { + FLUSH_BLOCK_ONLY(s, 0); + } + s->strstart++; + s->lookahead--; + if (s->strm->avail_out == 0) return need_more; + } else { + /* There is no previous match to compare with, wait for + * the next step to decide. + */ + s->match_available = 1; + s->strstart++; + s->lookahead--; + } + } + Assert (flush != Z_NO_FLUSH, "no flush?"); + if (s->match_available) { + Tracevv((stderr,"%c", s->window[s->strstart - 1])); + _tr_tally_lit(s, s->window[s->strstart - 1], bflush); + s->match_available = 0; + } + s->insert = s->strstart < MIN_MATCH-1 ? s->strstart : MIN_MATCH-1; + if (flush == Z_FINISH) { + FLUSH_BLOCK(s, 1); + return finish_done; + } + if (s->sym_next) + FLUSH_BLOCK(s, 0); + return block_done; +} +#endif /* FASTEST */ + +/* =========================================================================== + * For Z_RLE, simply look for runs of bytes, generate matches only of distance + * one. Do not maintain a hash table. (It will be regenerated if this run of + * deflate switches away from Z_RLE.) + */ +local block_state deflate_rle(s, flush) + deflate_state *s; + int flush; +{ + int bflush; /* set if current block must be flushed */ + uInt prev; /* byte at distance one to match */ + Bytef *scan, *strend; /* scan goes up to strend for length of run */ + + for (;;) { + /* Make sure that we always have enough lookahead, except + * at the end of the input file. We need MAX_MATCH bytes + * for the longest run, plus one for the unrolled loop. + */ + if (s->lookahead <= MAX_MATCH) { + fill_window(s); + if (s->lookahead <= MAX_MATCH && flush == Z_NO_FLUSH) { + return need_more; + } + if (s->lookahead == 0) break; /* flush the current block */ + } + + /* See how many times the previous byte repeats */ + s->match_length = 0; + if (s->lookahead >= MIN_MATCH && s->strstart > 0) { + scan = s->window + s->strstart - 1; + prev = *scan; + if (prev == *++scan && prev == *++scan && prev == *++scan) { + strend = s->window + s->strstart + MAX_MATCH; + do { + } while (prev == *++scan && prev == *++scan && + prev == *++scan && prev == *++scan && + prev == *++scan && prev == *++scan && + prev == *++scan && prev == *++scan && + scan < strend); + s->match_length = MAX_MATCH - (uInt)(strend - scan); + if (s->match_length > s->lookahead) + s->match_length = s->lookahead; + } + Assert(scan <= s->window + (uInt)(s->window_size - 1), + "wild scan"); + } + + /* Emit match if have run of MIN_MATCH or longer, else emit literal */ + if (s->match_length >= MIN_MATCH) { + check_match(s, s->strstart, s->strstart - 1, s->match_length); + + _tr_tally_dist(s, 1, s->match_length - MIN_MATCH, bflush); + + s->lookahead -= s->match_length; + s->strstart += s->match_length; + s->match_length = 0; + } else { + /* No match, output a literal byte */ + Tracevv((stderr,"%c", s->window[s->strstart])); + _tr_tally_lit(s, s->window[s->strstart], bflush); + s->lookahead--; + s->strstart++; + } + if (bflush) FLUSH_BLOCK(s, 0); + } + s->insert = 0; + if (flush == Z_FINISH) { + FLUSH_BLOCK(s, 1); + return finish_done; + } + if (s->sym_next) + FLUSH_BLOCK(s, 0); + return block_done; +} + +/* =========================================================================== + * For Z_HUFFMAN_ONLY, do not look for matches. Do not maintain a hash table. + * (It will be regenerated if this run of deflate switches away from Huffman.) + */ +local block_state deflate_huff(s, flush) + deflate_state *s; + int flush; +{ + int bflush; /* set if current block must be flushed */ + + for (;;) { + /* Make sure that we have a literal to write. */ + if (s->lookahead == 0) { + fill_window(s); + if (s->lookahead == 0) { + if (flush == Z_NO_FLUSH) + return need_more; + break; /* flush the current block */ + } + } + + /* Output a literal byte */ + s->match_length = 0; + Tracevv((stderr,"%c", s->window[s->strstart])); + _tr_tally_lit(s, s->window[s->strstart], bflush); + s->lookahead--; + s->strstart++; + if (bflush) FLUSH_BLOCK(s, 0); + } + s->insert = 0; + if (flush == Z_FINISH) { + FLUSH_BLOCK(s, 1); + return finish_done; + } + if (s->sym_next) + FLUSH_BLOCK(s, 0); + return block_done; +} diff --git a/lib/os/windows/zlib-1.2.13/deflate.h b/lib/os/windows/zlib-1.2.13/deflate.h new file mode 100644 index 000000000000..1a06cd5f25d1 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/deflate.h @@ -0,0 +1,346 @@ +/* deflate.h -- internal compression state + * Copyright (C) 1995-2018 Jean-loup Gailly + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* WARNING: this file should *not* be used by applications. It is + part of the implementation of the compression library and is + subject to change. Applications should only use zlib.h. + */ + +/* @(#) $Id$ */ + +#ifndef DEFLATE_H +#define DEFLATE_H + +#include "zutil.h" + +/* define NO_GZIP when compiling if you want to disable gzip header and + trailer creation by deflate(). NO_GZIP would be used to avoid linking in + the crc code when it is not needed. For shared libraries, gzip encoding + should be left enabled. */ +#ifndef NO_GZIP +# define GZIP +#endif + +/* =========================================================================== + * Internal compression state. + */ + +#define LENGTH_CODES 29 +/* number of length codes, not counting the special END_BLOCK code */ + +#define LITERALS 256 +/* number of literal bytes 0..255 */ + +#define L_CODES (LITERALS+1+LENGTH_CODES) +/* number of Literal or Length codes, including the END_BLOCK code */ + +#define D_CODES 30 +/* number of distance codes */ + +#define BL_CODES 19 +/* number of codes used to transfer the bit lengths */ + +#define HEAP_SIZE (2*L_CODES+1) +/* maximum heap size */ + +#define MAX_BITS 15 +/* All codes must not exceed MAX_BITS bits */ + +#define Buf_size 16 +/* size of bit buffer in bi_buf */ + +#define INIT_STATE 42 /* zlib header -> BUSY_STATE */ +#ifdef GZIP +# define GZIP_STATE 57 /* gzip header -> BUSY_STATE | EXTRA_STATE */ +#endif +#define EXTRA_STATE 69 /* gzip extra block -> NAME_STATE */ +#define NAME_STATE 73 /* gzip file name -> COMMENT_STATE */ +#define COMMENT_STATE 91 /* gzip comment -> HCRC_STATE */ +#define HCRC_STATE 103 /* gzip header CRC -> BUSY_STATE */ +#define BUSY_STATE 113 /* deflate -> FINISH_STATE */ +#define FINISH_STATE 666 /* stream complete */ +/* Stream status */ + + +/* Data structure describing a single value and its code string. */ +typedef struct ct_data_s { + union { + ush freq; /* frequency count */ + ush code; /* bit string */ + } fc; + union { + ush dad; /* father node in Huffman tree */ + ush len; /* length of bit string */ + } dl; +} FAR ct_data; + +#define Freq fc.freq +#define Code fc.code +#define Dad dl.dad +#define Len dl.len + +typedef struct static_tree_desc_s static_tree_desc; + +typedef struct tree_desc_s { + ct_data *dyn_tree; /* the dynamic tree */ + int max_code; /* largest code with non zero frequency */ + const static_tree_desc *stat_desc; /* the corresponding static tree */ +} FAR tree_desc; + +typedef ush Pos; +typedef Pos FAR Posf; +typedef unsigned IPos; + +/* A Pos is an index in the character window. We use short instead of int to + * save space in the various tables. IPos is used only for parameter passing. + */ + +typedef struct internal_state { + z_streamp strm; /* pointer back to this zlib stream */ + int status; /* as the name implies */ + Bytef *pending_buf; /* output still pending */ + ulg pending_buf_size; /* size of pending_buf */ + Bytef *pending_out; /* next pending byte to output to the stream */ + ulg pending; /* nb of bytes in the pending buffer */ + int wrap; /* bit 0 true for zlib, bit 1 true for gzip */ + gz_headerp gzhead; /* gzip header information to write */ + ulg gzindex; /* where in extra, name, or comment */ + Byte method; /* can only be DEFLATED */ + int last_flush; /* value of flush param for previous deflate call */ + + /* used by deflate.c: */ + + uInt w_size; /* LZ77 window size (32K by default) */ + uInt w_bits; /* log2(w_size) (8..16) */ + uInt w_mask; /* w_size - 1 */ + + Bytef *window; + /* Sliding window. Input bytes are read into the second half of the window, + * and move to the first half later to keep a dictionary of at least wSize + * bytes. With this organization, matches are limited to a distance of + * wSize-MAX_MATCH bytes, but this ensures that IO is always + * performed with a length multiple of the block size. Also, it limits + * the window size to 64K, which is quite useful on MSDOS. + * To do: use the user input buffer as sliding window. + */ + + ulg window_size; + /* Actual size of window: 2*wSize, except when the user input buffer + * is directly used as sliding window. + */ + + Posf *prev; + /* Link to older string with same hash index. To limit the size of this + * array to 64K, this link is maintained only for the last 32K strings. + * An index in this array is thus a window index modulo 32K. + */ + + Posf *head; /* Heads of the hash chains or NIL. */ + + uInt ins_h; /* hash index of string to be inserted */ + uInt hash_size; /* number of elements in hash table */ + uInt hash_bits; /* log2(hash_size) */ + uInt hash_mask; /* hash_size-1 */ + + uInt hash_shift; + /* Number of bits by which ins_h must be shifted at each input + * step. It must be such that after MIN_MATCH steps, the oldest + * byte no longer takes part in the hash key, that is: + * hash_shift * MIN_MATCH >= hash_bits + */ + + long block_start; + /* Window position at the beginning of the current output block. Gets + * negative when the window is moved backwards. + */ + + uInt match_length; /* length of best match */ + IPos prev_match; /* previous match */ + int match_available; /* set if previous match exists */ + uInt strstart; /* start of string to insert */ + uInt match_start; /* start of matching string */ + uInt lookahead; /* number of valid bytes ahead in window */ + + uInt prev_length; + /* Length of the best match at previous step. Matches not greater than this + * are discarded. This is used in the lazy match evaluation. + */ + + uInt max_chain_length; + /* To speed up deflation, hash chains are never searched beyond this + * length. A higher limit improves compression ratio but degrades the + * speed. + */ + + uInt max_lazy_match; + /* Attempt to find a better match only when the current match is strictly + * smaller than this value. This mechanism is used only for compression + * levels >= 4. + */ +# define max_insert_length max_lazy_match + /* Insert new strings in the hash table only if the match length is not + * greater than this length. This saves time but degrades compression. + * max_insert_length is used only for compression levels <= 3. + */ + + int level; /* compression level (1..9) */ + int strategy; /* favor or force Huffman coding*/ + + uInt good_match; + /* Use a faster search when the previous match is longer than this */ + + int nice_match; /* Stop searching when current match exceeds this */ + + /* used by trees.c: */ + /* Didn't use ct_data typedef below to suppress compiler warning */ + struct ct_data_s dyn_ltree[HEAP_SIZE]; /* literal and length tree */ + struct ct_data_s dyn_dtree[2*D_CODES+1]; /* distance tree */ + struct ct_data_s bl_tree[2*BL_CODES+1]; /* Huffman tree for bit lengths */ + + struct tree_desc_s l_desc; /* desc. for literal tree */ + struct tree_desc_s d_desc; /* desc. for distance tree */ + struct tree_desc_s bl_desc; /* desc. for bit length tree */ + + ush bl_count[MAX_BITS+1]; + /* number of codes at each bit length for an optimal tree */ + + int heap[2*L_CODES+1]; /* heap used to build the Huffman trees */ + int heap_len; /* number of elements in the heap */ + int heap_max; /* element of largest frequency */ + /* The sons of heap[n] are heap[2*n] and heap[2*n+1]. heap[0] is not used. + * The same heap array is used to build all trees. + */ + + uch depth[2*L_CODES+1]; + /* Depth of each subtree used as tie breaker for trees of equal frequency + */ + + uchf *sym_buf; /* buffer for distances and literals/lengths */ + + uInt lit_bufsize; + /* Size of match buffer for literals/lengths. There are 4 reasons for + * limiting lit_bufsize to 64K: + * - frequencies can be kept in 16 bit counters + * - if compression is not successful for the first block, all input + * data is still in the window so we can still emit a stored block even + * when input comes from standard input. (This can also be done for + * all blocks if lit_bufsize is not greater than 32K.) + * - if compression is not successful for a file smaller than 64K, we can + * even emit a stored file instead of a stored block (saving 5 bytes). + * This is applicable only for zip (not gzip or zlib). + * - creating new Huffman trees less frequently may not provide fast + * adaptation to changes in the input data statistics. (Take for + * example a binary file with poorly compressible code followed by + * a highly compressible string table.) Smaller buffer sizes give + * fast adaptation but have of course the overhead of transmitting + * trees more frequently. + * - I can't count above 4 + */ + + uInt sym_next; /* running index in sym_buf */ + uInt sym_end; /* symbol table full when sym_next reaches this */ + + ulg opt_len; /* bit length of current block with optimal trees */ + ulg static_len; /* bit length of current block with static trees */ + uInt matches; /* number of string matches in current block */ + uInt insert; /* bytes at end of window left to insert */ + +#ifdef ZLIB_DEBUG + ulg compressed_len; /* total bit length of compressed file mod 2^32 */ + ulg bits_sent; /* bit length of compressed data sent mod 2^32 */ +#endif + + ush bi_buf; + /* Output buffer. bits are inserted starting at the bottom (least + * significant bits). + */ + int bi_valid; + /* Number of valid bits in bi_buf. All bits above the last valid bit + * are always zero. + */ + + ulg high_water; + /* High water mark offset in window for initialized bytes -- bytes above + * this are set to zero in order to avoid memory check warnings when + * longest match routines access bytes past the input. This is then + * updated to the new high water mark. + */ + +} FAR deflate_state; + +/* Output a byte on the stream. + * IN assertion: there is enough room in pending_buf. + */ +#define put_byte(s, c) {s->pending_buf[s->pending++] = (Bytef)(c);} + + +#define MIN_LOOKAHEAD (MAX_MATCH+MIN_MATCH+1) +/* Minimum amount of lookahead, except at the end of the input file. + * See deflate.c for comments about the MIN_MATCH+1. + */ + +#define MAX_DIST(s) ((s)->w_size-MIN_LOOKAHEAD) +/* In order to simplify the code, particularly on 16 bit machines, match + * distances are limited to MAX_DIST instead of WSIZE. + */ + +#define WIN_INIT MAX_MATCH +/* Number of bytes after end of data in window to initialize in order to avoid + memory checker errors from longest match routines */ + + /* in trees.c */ +void ZLIB_INTERNAL _tr_init OF((deflate_state *s)); +int ZLIB_INTERNAL _tr_tally OF((deflate_state *s, unsigned dist, unsigned lc)); +void ZLIB_INTERNAL _tr_flush_block OF((deflate_state *s, charf *buf, + ulg stored_len, int last)); +void ZLIB_INTERNAL _tr_flush_bits OF((deflate_state *s)); +void ZLIB_INTERNAL _tr_align OF((deflate_state *s)); +void ZLIB_INTERNAL _tr_stored_block OF((deflate_state *s, charf *buf, + ulg stored_len, int last)); + +#define d_code(dist) \ + ((dist) < 256 ? _dist_code[dist] : _dist_code[256+((dist)>>7)]) +/* Mapping from a distance to a distance code. dist is the distance - 1 and + * must not have side effects. _dist_code[256] and _dist_code[257] are never + * used. + */ + +#ifndef ZLIB_DEBUG +/* Inline versions of _tr_tally for speed: */ + +#if defined(GEN_TREES_H) || !defined(STDC) + extern uch ZLIB_INTERNAL _length_code[]; + extern uch ZLIB_INTERNAL _dist_code[]; +#else + extern const uch ZLIB_INTERNAL _length_code[]; + extern const uch ZLIB_INTERNAL _dist_code[]; +#endif + +# define _tr_tally_lit(s, c, flush) \ + { uch cc = (c); \ + s->sym_buf[s->sym_next++] = 0; \ + s->sym_buf[s->sym_next++] = 0; \ + s->sym_buf[s->sym_next++] = cc; \ + s->dyn_ltree[cc].Freq++; \ + flush = (s->sym_next == s->sym_end); \ + } +# define _tr_tally_dist(s, distance, length, flush) \ + { uch len = (uch)(length); \ + ush dist = (ush)(distance); \ + s->sym_buf[s->sym_next++] = (uch)dist; \ + s->sym_buf[s->sym_next++] = (uch)(dist >> 8); \ + s->sym_buf[s->sym_next++] = len; \ + dist--; \ + s->dyn_ltree[_length_code[len]+LITERALS+1].Freq++; \ + s->dyn_dtree[d_code(dist)].Freq++; \ + flush = (s->sym_next == s->sym_end); \ + } +#else +# define _tr_tally_lit(s, c, flush) flush = _tr_tally(s, 0, c) +# define _tr_tally_dist(s, distance, length, flush) \ + flush = _tr_tally(s, distance, length) +#endif + +#endif /* DEFLATE_H */ diff --git a/lib/os/windows/zlib-1.2.13/doc/algorithm.txt b/lib/os/windows/zlib-1.2.13/doc/algorithm.txt new file mode 100644 index 000000000000..c97f495020b4 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/doc/algorithm.txt @@ -0,0 +1,209 @@ +1. Compression algorithm (deflate) + +The deflation algorithm used by gzip (also zip and zlib) is a variation of +LZ77 (Lempel-Ziv 1977, see reference below). It finds duplicated strings in +the input data. The second occurrence of a string is replaced by a +pointer to the previous string, in the form of a pair (distance, +length). Distances are limited to 32K bytes, and lengths are limited +to 258 bytes. When a string does not occur anywhere in the previous +32K bytes, it is emitted as a sequence of literal bytes. (In this +description, `string' must be taken as an arbitrary sequence of bytes, +and is not restricted to printable characters.) + +Literals or match lengths are compressed with one Huffman tree, and +match distances are compressed with another tree. The trees are stored +in a compact form at the start of each block. The blocks can have any +size (except that the compressed data for one block must fit in +available memory). A block is terminated when deflate() determines that +it would be useful to start another block with fresh trees. (This is +somewhat similar to the behavior of LZW-based _compress_.) + +Duplicated strings are found using a hash table. All input strings of +length 3 are inserted in the hash table. A hash index is computed for +the next 3 bytes. If the hash chain for this index is not empty, all +strings in the chain are compared with the current input string, and +the longest match is selected. + +The hash chains are searched starting with the most recent strings, to +favor small distances and thus take advantage of the Huffman encoding. +The hash chains are singly linked. There are no deletions from the +hash chains, the algorithm simply discards matches that are too old. + +To avoid a worst-case situation, very long hash chains are arbitrarily +truncated at a certain length, determined by a runtime option (level +parameter of deflateInit). So deflate() does not always find the longest +possible match but generally finds a match which is long enough. + +deflate() also defers the selection of matches with a lazy evaluation +mechanism. After a match of length N has been found, deflate() searches for +a longer match at the next input byte. If a longer match is found, the +previous match is truncated to a length of one (thus producing a single +literal byte) and the process of lazy evaluation begins again. Otherwise, +the original match is kept, and the next match search is attempted only N +steps later. + +The lazy match evaluation is also subject to a runtime parameter. If +the current match is long enough, deflate() reduces the search for a longer +match, thus speeding up the whole process. If compression ratio is more +important than speed, deflate() attempts a complete second search even if +the first match is already long enough. + +The lazy match evaluation is not performed for the fastest compression +modes (level parameter 1 to 3). For these fast modes, new strings +are inserted in the hash table only when no match was found, or +when the match is not too long. This degrades the compression ratio +but saves time since there are both fewer insertions and fewer searches. + + +2. Decompression algorithm (inflate) + +2.1 Introduction + +The key question is how to represent a Huffman code (or any prefix code) so +that you can decode fast. The most important characteristic is that shorter +codes are much more common than longer codes, so pay attention to decoding the +short codes fast, and let the long codes take longer to decode. + +inflate() sets up a first level table that covers some number of bits of +input less than the length of longest code. It gets that many bits from the +stream, and looks it up in the table. The table will tell if the next +code is that many bits or less and how many, and if it is, it will tell +the value, else it will point to the next level table for which inflate() +grabs more bits and tries to decode a longer code. + +How many bits to make the first lookup is a tradeoff between the time it +takes to decode and the time it takes to build the table. If building the +table took no time (and if you had infinite memory), then there would only +be a first level table to cover all the way to the longest code. However, +building the table ends up taking a lot longer for more bits since short +codes are replicated many times in such a table. What inflate() does is +simply to make the number of bits in the first table a variable, and then +to set that variable for the maximum speed. + +For inflate, which has 286 possible codes for the literal/length tree, the size +of the first table is nine bits. Also the distance trees have 30 possible +values, and the size of the first table is six bits. Note that for each of +those cases, the table ended up one bit longer than the ``average'' code +length, i.e. the code length of an approximately flat code which would be a +little more than eight bits for 286 symbols and a little less than five bits +for 30 symbols. + + +2.2 More details on the inflate table lookup + +Ok, you want to know what this cleverly obfuscated inflate tree actually +looks like. You are correct that it's not a Huffman tree. It is simply a +lookup table for the first, let's say, nine bits of a Huffman symbol. The +symbol could be as short as one bit or as long as 15 bits. If a particular +symbol is shorter than nine bits, then that symbol's translation is duplicated +in all those entries that start with that symbol's bits. For example, if the +symbol is four bits, then it's duplicated 32 times in a nine-bit table. If a +symbol is nine bits long, it appears in the table once. + +If the symbol is longer than nine bits, then that entry in the table points +to another similar table for the remaining bits. Again, there are duplicated +entries as needed. The idea is that most of the time the symbol will be short +and there will only be one table look up. (That's whole idea behind data +compression in the first place.) For the less frequent long symbols, there +will be two lookups. If you had a compression method with really long +symbols, you could have as many levels of lookups as is efficient. For +inflate, two is enough. + +So a table entry either points to another table (in which case nine bits in +the above example are gobbled), or it contains the translation for the symbol +and the number of bits to gobble. Then you start again with the next +ungobbled bit. + +You may wonder: why not just have one lookup table for how ever many bits the +longest symbol is? The reason is that if you do that, you end up spending +more time filling in duplicate symbol entries than you do actually decoding. +At least for deflate's output that generates new trees every several 10's of +kbytes. You can imagine that filling in a 2^15 entry table for a 15-bit code +would take too long if you're only decoding several thousand symbols. At the +other extreme, you could make a new table for every bit in the code. In fact, +that's essentially a Huffman tree. But then you spend too much time +traversing the tree while decoding, even for short symbols. + +So the number of bits for the first lookup table is a trade of the time to +fill out the table vs. the time spent looking at the second level and above of +the table. + +Here is an example, scaled down: + +The code being decoded, with 10 symbols, from 1 to 6 bits long: + +A: 0 +B: 10 +C: 1100 +D: 11010 +E: 11011 +F: 11100 +G: 11101 +H: 11110 +I: 111110 +J: 111111 + +Let's make the first table three bits long (eight entries): + +000: A,1 +001: A,1 +010: A,1 +011: A,1 +100: B,2 +101: B,2 +110: -> table X (gobble 3 bits) +111: -> table Y (gobble 3 bits) + +Each entry is what the bits decode as and how many bits that is, i.e. how +many bits to gobble. Or the entry points to another table, with the number of +bits to gobble implicit in the size of the table. + +Table X is two bits long since the longest code starting with 110 is five bits +long: + +00: C,1 +01: C,1 +10: D,2 +11: E,2 + +Table Y is three bits long since the longest code starting with 111 is six +bits long: + +000: F,2 +001: F,2 +010: G,2 +011: G,2 +100: H,2 +101: H,2 +110: I,3 +111: J,3 + +So what we have here are three tables with a total of 20 entries that had to +be constructed. That's compared to 64 entries for a single table. Or +compared to 16 entries for a Huffman tree (six two entry tables and one four +entry table). Assuming that the code ideally represents the probability of +the symbols, it takes on the average 1.25 lookups per symbol. That's compared +to one lookup for the single table, or 1.66 lookups per symbol for the +Huffman tree. + +There, I think that gives you a picture of what's going on. For inflate, the +meaning of a particular symbol is often more than just a letter. It can be a +byte (a "literal"), or it can be either a length or a distance which +indicates a base value and a number of bits to fetch after the code that is +added to the base value. Or it might be the special end-of-block code. The +data structures created in inftrees.c try to encode all that information +compactly in the tables. + + +Jean-loup Gailly Mark Adler +jloup@gzip.org madler@alumni.caltech.edu + + +References: + +[LZ77] Ziv J., Lempel A., ``A Universal Algorithm for Sequential Data +Compression,'' IEEE Transactions on Information Theory, Vol. 23, No. 3, +pp. 337-343. + +``DEFLATE Compressed Data Format Specification'' available in +http://tools.ietf.org/html/rfc1951 diff --git a/lib/os/windows/zlib-1.2.13/gzclose.c b/lib/os/windows/zlib-1.2.13/gzclose.c new file mode 100644 index 000000000000..caeb99a3177f --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/gzclose.c @@ -0,0 +1,25 @@ +/* gzclose.c -- zlib gzclose() function + * Copyright (C) 2004, 2010 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +#include "gzguts.h" + +/* gzclose() is in a separate file so that it is linked in only if it is used. + That way the other gzclose functions can be used instead to avoid linking in + unneeded compression or decompression routines. */ +int ZEXPORT gzclose(file) + gzFile file; +{ +#ifndef NO_GZCOMPRESS + gz_statep state; + + if (file == NULL) + return Z_STREAM_ERROR; + state = (gz_statep)file; + + return state->mode == GZ_READ ? gzclose_r(file) : gzclose_w(file); +#else + return gzclose_r(file); +#endif +} diff --git a/lib/os/windows/zlib-1.2.13/gzguts.h b/lib/os/windows/zlib-1.2.13/gzguts.h new file mode 100644 index 000000000000..57faf37165a3 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/gzguts.h @@ -0,0 +1,219 @@ +/* gzguts.h -- zlib internal header definitions for gz* operations + * Copyright (C) 2004-2019 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +#ifdef _LARGEFILE64_SOURCE +# ifndef _LARGEFILE_SOURCE +# define _LARGEFILE_SOURCE 1 +# endif +# ifdef _FILE_OFFSET_BITS +# undef _FILE_OFFSET_BITS +# endif +#endif + +#ifdef HAVE_HIDDEN +# define ZLIB_INTERNAL __attribute__((visibility ("hidden"))) +#else +# define ZLIB_INTERNAL +#endif + +#include +#include "zlib.h" +#ifdef STDC +# include +# include +# include +#endif + +#ifndef _POSIX_SOURCE +# define _POSIX_SOURCE +#endif +#include + +#ifdef _WIN32 +# include +#endif + +#if defined(__TURBOC__) || defined(_MSC_VER) || defined(_WIN32) +# include +#endif + +#if defined(_WIN32) +# define WIDECHAR +#endif + +#ifdef WINAPI_FAMILY +# define open _open +# define read _read +# define write _write +# define close _close +#endif + +#ifdef NO_DEFLATE /* for compatibility with old definition */ +# define NO_GZCOMPRESS +#endif + +#if defined(STDC99) || (defined(__TURBOC__) && __TURBOC__ >= 0x550) +# ifndef HAVE_VSNPRINTF +# define HAVE_VSNPRINTF +# endif +#endif + +#if defined(__CYGWIN__) +# ifndef HAVE_VSNPRINTF +# define HAVE_VSNPRINTF +# endif +#endif + +#if defined(MSDOS) && defined(__BORLANDC__) && (BORLANDC > 0x410) +# ifndef HAVE_VSNPRINTF +# define HAVE_VSNPRINTF +# endif +#endif + +#ifndef HAVE_VSNPRINTF +# ifdef MSDOS +/* vsnprintf may exist on some MS-DOS compilers (DJGPP?), + but for now we just assume it doesn't. */ +# define NO_vsnprintf +# endif +# ifdef __TURBOC__ +# define NO_vsnprintf +# endif +# ifdef WIN32 +/* In Win32, vsnprintf is available as the "non-ANSI" _vsnprintf. */ +# if !defined(vsnprintf) && !defined(NO_vsnprintf) +# if !defined(_MSC_VER) || ( defined(_MSC_VER) && _MSC_VER < 1500 ) +# define vsnprintf _vsnprintf +# endif +# endif +# endif +# ifdef __SASC +# define NO_vsnprintf +# endif +# ifdef VMS +# define NO_vsnprintf +# endif +# ifdef __OS400__ +# define NO_vsnprintf +# endif +# ifdef __MVS__ +# define NO_vsnprintf +# endif +#endif + +/* unlike snprintf (which is required in C99), _snprintf does not guarantee + null termination of the result -- however this is only used in gzlib.c where + the result is assured to fit in the space provided */ +#if defined(_MSC_VER) && _MSC_VER < 1900 +# define snprintf _snprintf +#endif + +#ifndef local +# define local static +#endif +/* since "static" is used to mean two completely different things in C, we + define "local" for the non-static meaning of "static", for readability + (compile with -Dlocal if your debugger can't find static symbols) */ + +/* gz* functions always use library allocation functions */ +#ifndef STDC + extern voidp malloc OF((uInt size)); + extern void free OF((voidpf ptr)); +#endif + +/* get errno and strerror definition */ +#if defined UNDER_CE +# include +# define zstrerror() gz_strwinerror((DWORD)GetLastError()) +#else +# ifndef NO_STRERROR +# include +# define zstrerror() strerror(errno) +# else +# define zstrerror() "stdio error (consult errno)" +# endif +#endif + +/* provide prototypes for these when building zlib without LFS */ +#if !defined(_LARGEFILE64_SOURCE) || _LFS64_LARGEFILE-0 == 0 + ZEXTERN gzFile ZEXPORT gzopen64 OF((const char *, const char *)); + ZEXTERN z_off64_t ZEXPORT gzseek64 OF((gzFile, z_off64_t, int)); + ZEXTERN z_off64_t ZEXPORT gztell64 OF((gzFile)); + ZEXTERN z_off64_t ZEXPORT gzoffset64 OF((gzFile)); +#endif + +/* default memLevel */ +#if MAX_MEM_LEVEL >= 8 +# define DEF_MEM_LEVEL 8 +#else +# define DEF_MEM_LEVEL MAX_MEM_LEVEL +#endif + +/* default i/o buffer size -- double this for output when reading (this and + twice this must be able to fit in an unsigned type) */ +#define GZBUFSIZE 8192 + +/* gzip modes, also provide a little integrity check on the passed structure */ +#define GZ_NONE 0 +#define GZ_READ 7247 +#define GZ_WRITE 31153 +#define GZ_APPEND 1 /* mode set to GZ_WRITE after the file is opened */ + +/* values for gz_state how */ +#define LOOK 0 /* look for a gzip header */ +#define COPY 1 /* copy input directly */ +#define GZIP 2 /* decompress a gzip stream */ + +/* internal gzip file state data structure */ +typedef struct { + /* exposed contents for gzgetc() macro */ + struct gzFile_s x; /* "x" for exposed */ + /* x.have: number of bytes available at x.next */ + /* x.next: next output data to deliver or write */ + /* x.pos: current position in uncompressed data */ + /* used for both reading and writing */ + int mode; /* see gzip modes above */ + int fd; /* file descriptor */ + char *path; /* path or fd for error messages */ + unsigned size; /* buffer size, zero if not allocated yet */ + unsigned want; /* requested buffer size, default is GZBUFSIZE */ + unsigned char *in; /* input buffer (double-sized when writing) */ + unsigned char *out; /* output buffer (double-sized when reading) */ + int direct; /* 0 if processing gzip, 1 if transparent */ + /* just for reading */ + int how; /* 0: get header, 1: copy, 2: decompress */ + z_off64_t start; /* where the gzip data started, for rewinding */ + int eof; /* true if end of input file reached */ + int past; /* true if read requested past end */ + /* just for writing */ + int level; /* compression level */ + int strategy; /* compression strategy */ + int reset; /* true if a reset is pending after a Z_FINISH */ + /* seek request */ + z_off64_t skip; /* amount to skip (already rewound if backwards) */ + int seek; /* true if seek request pending */ + /* error information */ + int err; /* error code */ + char *msg; /* error message */ + /* zlib inflate or deflate stream */ + z_stream strm; /* stream structure in-place (not a pointer) */ +} gz_state; +typedef gz_state FAR *gz_statep; + +/* shared functions */ +void ZLIB_INTERNAL gz_error OF((gz_statep, int, const char *)); +#if defined UNDER_CE +char ZLIB_INTERNAL *gz_strwinerror OF((DWORD error)); +#endif + +/* GT_OFF(x), where x is an unsigned value, is true if x > maximum z_off64_t + value -- needed when comparing unsigned to z_off64_t, which is signed + (possible z_off64_t types off_t, off64_t, and long are all signed) */ +#ifdef INT_MAX +# define GT_OFF(x) (sizeof(int) == sizeof(z_off64_t) && (x) > INT_MAX) +#else +unsigned ZLIB_INTERNAL gz_intmax OF((void)); +# define GT_OFF(x) (sizeof(int) == sizeof(z_off64_t) && (x) > gz_intmax()) +#endif diff --git a/lib/os/windows/zlib-1.2.13/gzlib.c b/lib/os/windows/zlib-1.2.13/gzlib.c new file mode 100644 index 000000000000..55da46a453fd --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/gzlib.c @@ -0,0 +1,639 @@ +/* gzlib.c -- zlib functions common to reading and writing gzip files + * Copyright (C) 2004-2019 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +#include "gzguts.h" + +#if defined(_WIN32) && !defined(__BORLANDC__) +# define LSEEK _lseeki64 +#else +#if defined(_LARGEFILE64_SOURCE) && _LFS64_LARGEFILE-0 +# define LSEEK lseek64 +#else +# define LSEEK lseek +#endif +#endif + +/* Local functions */ +local void gz_reset OF((gz_statep)); +local gzFile gz_open OF((const void *, int, const char *)); + +#if defined UNDER_CE + +/* Map the Windows error number in ERROR to a locale-dependent error message + string and return a pointer to it. Typically, the values for ERROR come + from GetLastError. + + The string pointed to shall not be modified by the application, but may be + overwritten by a subsequent call to gz_strwinerror + + The gz_strwinerror function does not change the current setting of + GetLastError. */ +char ZLIB_INTERNAL *gz_strwinerror(error) + DWORD error; +{ + static char buf[1024]; + + wchar_t *msgbuf; + DWORD lasterr = GetLastError(); + DWORD chars = FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM + | FORMAT_MESSAGE_ALLOCATE_BUFFER, + NULL, + error, + 0, /* Default language */ + (LPVOID)&msgbuf, + 0, + NULL); + if (chars != 0) { + /* If there is an \r\n appended, zap it. */ + if (chars >= 2 + && msgbuf[chars - 2] == '\r' && msgbuf[chars - 1] == '\n') { + chars -= 2; + msgbuf[chars] = 0; + } + + if (chars > sizeof (buf) - 1) { + chars = sizeof (buf) - 1; + msgbuf[chars] = 0; + } + + wcstombs(buf, msgbuf, chars + 1); + LocalFree(msgbuf); + } + else { + sprintf(buf, "unknown win32 error (%ld)", error); + } + + SetLastError(lasterr); + return buf; +} + +#endif /* UNDER_CE */ + +/* Reset gzip file state */ +local void gz_reset(state) + gz_statep state; +{ + state->x.have = 0; /* no output data available */ + if (state->mode == GZ_READ) { /* for reading ... */ + state->eof = 0; /* not at end of file */ + state->past = 0; /* have not read past end yet */ + state->how = LOOK; /* look for gzip header */ + } + else /* for writing ... */ + state->reset = 0; /* no deflateReset pending */ + state->seek = 0; /* no seek request pending */ + gz_error(state, Z_OK, NULL); /* clear error */ + state->x.pos = 0; /* no uncompressed data yet */ + state->strm.avail_in = 0; /* no input data yet */ +} + +/* Open a gzip file either by name or file descriptor. */ +local gzFile gz_open(path, fd, mode) + const void *path; + int fd; + const char *mode; +{ + gz_statep state; + z_size_t len; + int oflag; +#ifdef O_CLOEXEC + int cloexec = 0; +#endif +#ifdef O_EXCL + int exclusive = 0; +#endif + + /* check input */ + if (path == NULL) + return NULL; + + /* allocate gzFile structure to return */ + state = (gz_statep)malloc(sizeof(gz_state)); + if (state == NULL) + return NULL; + state->size = 0; /* no buffers allocated yet */ + state->want = GZBUFSIZE; /* requested buffer size */ + state->msg = NULL; /* no error message yet */ + + /* interpret mode */ + state->mode = GZ_NONE; + state->level = Z_DEFAULT_COMPRESSION; + state->strategy = Z_DEFAULT_STRATEGY; + state->direct = 0; + while (*mode) { + if (*mode >= '0' && *mode <= '9') + state->level = *mode - '0'; + else + switch (*mode) { + case 'r': + state->mode = GZ_READ; + break; +#ifndef NO_GZCOMPRESS + case 'w': + state->mode = GZ_WRITE; + break; + case 'a': + state->mode = GZ_APPEND; + break; +#endif + case '+': /* can't read and write at the same time */ + free(state); + return NULL; + case 'b': /* ignore -- will request binary anyway */ + break; +#ifdef O_CLOEXEC + case 'e': + cloexec = 1; + break; +#endif +#ifdef O_EXCL + case 'x': + exclusive = 1; + break; +#endif + case 'f': + state->strategy = Z_FILTERED; + break; + case 'h': + state->strategy = Z_HUFFMAN_ONLY; + break; + case 'R': + state->strategy = Z_RLE; + break; + case 'F': + state->strategy = Z_FIXED; + break; + case 'T': + state->direct = 1; + break; + default: /* could consider as an error, but just ignore */ + ; + } + mode++; + } + + /* must provide an "r", "w", or "a" */ + if (state->mode == GZ_NONE) { + free(state); + return NULL; + } + + /* can't force transparent read */ + if (state->mode == GZ_READ) { + if (state->direct) { + free(state); + return NULL; + } + state->direct = 1; /* for empty file */ + } + + /* save the path name for error messages */ +#ifdef WIDECHAR + if (fd == -2) { + len = wcstombs(NULL, path, 0); + if (len == (z_size_t)-1) + len = 0; + } + else +#endif + len = strlen((const char *)path); + state->path = (char *)malloc(len + 1); + if (state->path == NULL) { + free(state); + return NULL; + } +#ifdef WIDECHAR + if (fd == -2) + if (len) + wcstombs(state->path, path, len + 1); + else + *(state->path) = 0; + else +#endif +#if !defined(NO_snprintf) && !defined(NO_vsnprintf) + (void)snprintf(state->path, len + 1, "%s", (const char *)path); +#else + strcpy(state->path, path); +#endif + + /* compute the flags for open() */ + oflag = +#ifdef O_LARGEFILE + O_LARGEFILE | +#endif +#ifdef O_BINARY + O_BINARY | +#endif +#ifdef O_CLOEXEC + (cloexec ? O_CLOEXEC : 0) | +#endif + (state->mode == GZ_READ ? + O_RDONLY : + (O_WRONLY | O_CREAT | +#ifdef O_EXCL + (exclusive ? O_EXCL : 0) | +#endif + (state->mode == GZ_WRITE ? + O_TRUNC : + O_APPEND))); + + /* open the file with the appropriate flags (or just use fd) */ + state->fd = fd > -1 ? fd : ( +#ifdef WIDECHAR + fd == -2 ? _wopen(path, oflag, 0666) : +#endif + open((const char *)path, oflag, 0666)); + if (state->fd == -1) { + free(state->path); + free(state); + return NULL; + } + if (state->mode == GZ_APPEND) { + LSEEK(state->fd, 0, SEEK_END); /* so gzoffset() is correct */ + state->mode = GZ_WRITE; /* simplify later checks */ + } + + /* save the current position for rewinding (only if reading) */ + if (state->mode == GZ_READ) { + state->start = LSEEK(state->fd, 0, SEEK_CUR); + if (state->start == -1) state->start = 0; + } + + /* initialize stream */ + gz_reset(state); + + /* return stream */ + return (gzFile)state; +} + +/* -- see zlib.h -- */ +gzFile ZEXPORT gzopen(path, mode) + const char *path; + const char *mode; +{ + return gz_open(path, -1, mode); +} + +/* -- see zlib.h -- */ +gzFile ZEXPORT gzopen64(path, mode) + const char *path; + const char *mode; +{ + return gz_open(path, -1, mode); +} + +/* -- see zlib.h -- */ +gzFile ZEXPORT gzdopen(fd, mode) + int fd; + const char *mode; +{ + char *path; /* identifier for error messages */ + gzFile gz; + + if (fd == -1 || (path = (char *)malloc(7 + 3 * sizeof(int))) == NULL) + return NULL; +#if !defined(NO_snprintf) && !defined(NO_vsnprintf) + (void)snprintf(path, 7 + 3 * sizeof(int), "", fd); +#else + sprintf(path, "", fd); /* for debugging */ +#endif + gz = gz_open(path, fd, mode); + free(path); + return gz; +} + +/* -- see zlib.h -- */ +#ifdef WIDECHAR +gzFile ZEXPORT gzopen_w(path, mode) + const wchar_t *path; + const char *mode; +{ + return gz_open(path, -2, mode); +} +#endif + +/* -- see zlib.h -- */ +int ZEXPORT gzbuffer(file, size) + gzFile file; + unsigned size; +{ + gz_statep state; + + /* get internal structure and check integrity */ + if (file == NULL) + return -1; + state = (gz_statep)file; + if (state->mode != GZ_READ && state->mode != GZ_WRITE) + return -1; + + /* make sure we haven't already allocated memory */ + if (state->size != 0) + return -1; + + /* check and set requested size */ + if ((size << 1) < size) + return -1; /* need to be able to double it */ + if (size < 2) + size = 2; /* need two bytes to check magic header */ + state->want = size; + return 0; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzrewind(file) + gzFile file; +{ + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return -1; + state = (gz_statep)file; + + /* check that we're reading and that there's no error */ + if (state->mode != GZ_READ || + (state->err != Z_OK && state->err != Z_BUF_ERROR)) + return -1; + + /* back up and start over */ + if (LSEEK(state->fd, state->start, SEEK_SET) == -1) + return -1; + gz_reset(state); + return 0; +} + +/* -- see zlib.h -- */ +z_off64_t ZEXPORT gzseek64(file, offset, whence) + gzFile file; + z_off64_t offset; + int whence; +{ + unsigned n; + z_off64_t ret; + gz_statep state; + + /* get internal structure and check integrity */ + if (file == NULL) + return -1; + state = (gz_statep)file; + if (state->mode != GZ_READ && state->mode != GZ_WRITE) + return -1; + + /* check that there's no error */ + if (state->err != Z_OK && state->err != Z_BUF_ERROR) + return -1; + + /* can only seek from start or relative to current position */ + if (whence != SEEK_SET && whence != SEEK_CUR) + return -1; + + /* normalize offset to a SEEK_CUR specification */ + if (whence == SEEK_SET) + offset -= state->x.pos; + else if (state->seek) + offset += state->skip; + state->seek = 0; + + /* if within raw area while reading, just go there */ + if (state->mode == GZ_READ && state->how == COPY && + state->x.pos + offset >= 0) { + ret = LSEEK(state->fd, offset - (z_off64_t)state->x.have, SEEK_CUR); + if (ret == -1) + return -1; + state->x.have = 0; + state->eof = 0; + state->past = 0; + state->seek = 0; + gz_error(state, Z_OK, NULL); + state->strm.avail_in = 0; + state->x.pos += offset; + return state->x.pos; + } + + /* calculate skip amount, rewinding if needed for back seek when reading */ + if (offset < 0) { + if (state->mode != GZ_READ) /* writing -- can't go backwards */ + return -1; + offset += state->x.pos; + if (offset < 0) /* before start of file! */ + return -1; + if (gzrewind(file) == -1) /* rewind, then skip to offset */ + return -1; + } + + /* if reading, skip what's in output buffer (one less gzgetc() check) */ + if (state->mode == GZ_READ) { + n = GT_OFF(state->x.have) || (z_off64_t)state->x.have > offset ? + (unsigned)offset : state->x.have; + state->x.have -= n; + state->x.next += n; + state->x.pos += n; + offset -= n; + } + + /* request skip (if not zero) */ + if (offset) { + state->seek = 1; + state->skip = offset; + } + return state->x.pos + offset; +} + +/* -- see zlib.h -- */ +z_off_t ZEXPORT gzseek(file, offset, whence) + gzFile file; + z_off_t offset; + int whence; +{ + z_off64_t ret; + + ret = gzseek64(file, (z_off64_t)offset, whence); + return ret == (z_off_t)ret ? (z_off_t)ret : -1; +} + +/* -- see zlib.h -- */ +z_off64_t ZEXPORT gztell64(file) + gzFile file; +{ + gz_statep state; + + /* get internal structure and check integrity */ + if (file == NULL) + return -1; + state = (gz_statep)file; + if (state->mode != GZ_READ && state->mode != GZ_WRITE) + return -1; + + /* return position */ + return state->x.pos + (state->seek ? state->skip : 0); +} + +/* -- see zlib.h -- */ +z_off_t ZEXPORT gztell(file) + gzFile file; +{ + z_off64_t ret; + + ret = gztell64(file); + return ret == (z_off_t)ret ? (z_off_t)ret : -1; +} + +/* -- see zlib.h -- */ +z_off64_t ZEXPORT gzoffset64(file) + gzFile file; +{ + z_off64_t offset; + gz_statep state; + + /* get internal structure and check integrity */ + if (file == NULL) + return -1; + state = (gz_statep)file; + if (state->mode != GZ_READ && state->mode != GZ_WRITE) + return -1; + + /* compute and return effective offset in file */ + offset = LSEEK(state->fd, 0, SEEK_CUR); + if (offset == -1) + return -1; + if (state->mode == GZ_READ) /* reading */ + offset -= state->strm.avail_in; /* don't count buffered input */ + return offset; +} + +/* -- see zlib.h -- */ +z_off_t ZEXPORT gzoffset(file) + gzFile file; +{ + z_off64_t ret; + + ret = gzoffset64(file); + return ret == (z_off_t)ret ? (z_off_t)ret : -1; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzeof(file) + gzFile file; +{ + gz_statep state; + + /* get internal structure and check integrity */ + if (file == NULL) + return 0; + state = (gz_statep)file; + if (state->mode != GZ_READ && state->mode != GZ_WRITE) + return 0; + + /* return end-of-file state */ + return state->mode == GZ_READ ? state->past : 0; +} + +/* -- see zlib.h -- */ +const char * ZEXPORT gzerror(file, errnum) + gzFile file; + int *errnum; +{ + gz_statep state; + + /* get internal structure and check integrity */ + if (file == NULL) + return NULL; + state = (gz_statep)file; + if (state->mode != GZ_READ && state->mode != GZ_WRITE) + return NULL; + + /* return error information */ + if (errnum != NULL) + *errnum = state->err; + return state->err == Z_MEM_ERROR ? "out of memory" : + (state->msg == NULL ? "" : state->msg); +} + +/* -- see zlib.h -- */ +void ZEXPORT gzclearerr(file) + gzFile file; +{ + gz_statep state; + + /* get internal structure and check integrity */ + if (file == NULL) + return; + state = (gz_statep)file; + if (state->mode != GZ_READ && state->mode != GZ_WRITE) + return; + + /* clear error and end-of-file */ + if (state->mode == GZ_READ) { + state->eof = 0; + state->past = 0; + } + gz_error(state, Z_OK, NULL); +} + +/* Create an error message in allocated memory and set state->err and + state->msg accordingly. Free any previous error message already there. Do + not try to free or allocate space if the error is Z_MEM_ERROR (out of + memory). Simply save the error message as a static string. If there is an + allocation failure constructing the error message, then convert the error to + out of memory. */ +void ZLIB_INTERNAL gz_error(state, err, msg) + gz_statep state; + int err; + const char *msg; +{ + /* free previously allocated message and clear */ + if (state->msg != NULL) { + if (state->err != Z_MEM_ERROR) + free(state->msg); + state->msg = NULL; + } + + /* if fatal, set state->x.have to 0 so that the gzgetc() macro fails */ + if (err != Z_OK && err != Z_BUF_ERROR) + state->x.have = 0; + + /* set error code, and if no message, then done */ + state->err = err; + if (msg == NULL) + return; + + /* for an out of memory error, return literal string when requested */ + if (err == Z_MEM_ERROR) + return; + + /* construct error message with path */ + if ((state->msg = (char *)malloc(strlen(state->path) + strlen(msg) + 3)) == + NULL) { + state->err = Z_MEM_ERROR; + return; + } +#if !defined(NO_snprintf) && !defined(NO_vsnprintf) + (void)snprintf(state->msg, strlen(state->path) + strlen(msg) + 3, + "%s%s%s", state->path, ": ", msg); +#else + strcpy(state->msg, state->path); + strcat(state->msg, ": "); + strcat(state->msg, msg); +#endif +} + +#ifndef INT_MAX +/* portably return maximum value for an int (when limits.h presumed not + available) -- we need to do this to cover cases where 2's complement not + used, since C standard permits 1's complement and sign-bit representations, + otherwise we could just use ((unsigned)-1) >> 1 */ +unsigned ZLIB_INTERNAL gz_intmax() +{ + unsigned p, q; + + p = 1; + do { + q = p; + p <<= 1; + p++; + } while (p > q); + return q >> 1; +} +#endif diff --git a/lib/os/windows/zlib-1.2.13/gzread.c b/lib/os/windows/zlib-1.2.13/gzread.c new file mode 100644 index 000000000000..dd77381596cb --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/gzread.c @@ -0,0 +1,650 @@ +/* gzread.c -- zlib functions for reading gzip files + * Copyright (C) 2004-2017 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +#include "gzguts.h" + +/* Local functions */ +local int gz_load OF((gz_statep, unsigned char *, unsigned, unsigned *)); +local int gz_avail OF((gz_statep)); +local int gz_look OF((gz_statep)); +local int gz_decomp OF((gz_statep)); +local int gz_fetch OF((gz_statep)); +local int gz_skip OF((gz_statep, z_off64_t)); +local z_size_t gz_read OF((gz_statep, voidp, z_size_t)); + +/* Use read() to load a buffer -- return -1 on error, otherwise 0. Read from + state->fd, and update state->eof, state->err, and state->msg as appropriate. + This function needs to loop on read(), since read() is not guaranteed to + read the number of bytes requested, depending on the type of descriptor. */ +local int gz_load(state, buf, len, have) + gz_statep state; + unsigned char *buf; + unsigned len; + unsigned *have; +{ + int ret; + unsigned get, max = ((unsigned)-1 >> 2) + 1; + + *have = 0; + do { + get = len - *have; + if (get > max) + get = max; + ret = read(state->fd, buf + *have, get); + if (ret <= 0) + break; + *have += (unsigned)ret; + } while (*have < len); + if (ret < 0) { + gz_error(state, Z_ERRNO, zstrerror()); + return -1; + } + if (ret == 0) + state->eof = 1; + return 0; +} + +/* Load up input buffer and set eof flag if last data loaded -- return -1 on + error, 0 otherwise. Note that the eof flag is set when the end of the input + file is reached, even though there may be unused data in the buffer. Once + that data has been used, no more attempts will be made to read the file. + If strm->avail_in != 0, then the current data is moved to the beginning of + the input buffer, and then the remainder of the buffer is loaded with the + available data from the input file. */ +local int gz_avail(state) + gz_statep state; +{ + unsigned got; + z_streamp strm = &(state->strm); + + if (state->err != Z_OK && state->err != Z_BUF_ERROR) + return -1; + if (state->eof == 0) { + if (strm->avail_in) { /* copy what's there to the start */ + unsigned char *p = state->in; + unsigned const char *q = strm->next_in; + unsigned n = strm->avail_in; + do { + *p++ = *q++; + } while (--n); + } + if (gz_load(state, state->in + strm->avail_in, + state->size - strm->avail_in, &got) == -1) + return -1; + strm->avail_in += got; + strm->next_in = state->in; + } + return 0; +} + +/* Look for gzip header, set up for inflate or copy. state->x.have must be 0. + If this is the first time in, allocate required memory. state->how will be + left unchanged if there is no more input data available, will be set to COPY + if there is no gzip header and direct copying will be performed, or it will + be set to GZIP for decompression. If direct copying, then leftover input + data from the input buffer will be copied to the output buffer. In that + case, all further file reads will be directly to either the output buffer or + a user buffer. If decompressing, the inflate state will be initialized. + gz_look() will return 0 on success or -1 on failure. */ +local int gz_look(state) + gz_statep state; +{ + z_streamp strm = &(state->strm); + + /* allocate read buffers and inflate memory */ + if (state->size == 0) { + /* allocate buffers */ + state->in = (unsigned char *)malloc(state->want); + state->out = (unsigned char *)malloc(state->want << 1); + if (state->in == NULL || state->out == NULL) { + free(state->out); + free(state->in); + gz_error(state, Z_MEM_ERROR, "out of memory"); + return -1; + } + state->size = state->want; + + /* allocate inflate memory */ + state->strm.zalloc = Z_NULL; + state->strm.zfree = Z_NULL; + state->strm.opaque = Z_NULL; + state->strm.avail_in = 0; + state->strm.next_in = Z_NULL; + if (inflateInit2(&(state->strm), 15 + 16) != Z_OK) { /* gunzip */ + free(state->out); + free(state->in); + state->size = 0; + gz_error(state, Z_MEM_ERROR, "out of memory"); + return -1; + } + } + + /* get at least the magic bytes in the input buffer */ + if (strm->avail_in < 2) { + if (gz_avail(state) == -1) + return -1; + if (strm->avail_in == 0) + return 0; + } + + /* look for gzip magic bytes -- if there, do gzip decoding (note: there is + a logical dilemma here when considering the case of a partially written + gzip file, to wit, if a single 31 byte is written, then we cannot tell + whether this is a single-byte file, or just a partially written gzip + file -- for here we assume that if a gzip file is being written, then + the header will be written in a single operation, so that reading a + single byte is sufficient indication that it is not a gzip file) */ + if (strm->avail_in > 1 && + strm->next_in[0] == 31 && strm->next_in[1] == 139) { + inflateReset(strm); + state->how = GZIP; + state->direct = 0; + return 0; + } + + /* no gzip header -- if we were decoding gzip before, then this is trailing + garbage. Ignore the trailing garbage and finish. */ + if (state->direct == 0) { + strm->avail_in = 0; + state->eof = 1; + state->x.have = 0; + return 0; + } + + /* doing raw i/o, copy any leftover input to output -- this assumes that + the output buffer is larger than the input buffer, which also assures + space for gzungetc() */ + state->x.next = state->out; + memcpy(state->x.next, strm->next_in, strm->avail_in); + state->x.have = strm->avail_in; + strm->avail_in = 0; + state->how = COPY; + state->direct = 1; + return 0; +} + +/* Decompress from input to the provided next_out and avail_out in the state. + On return, state->x.have and state->x.next point to the just decompressed + data. If the gzip stream completes, state->how is reset to LOOK to look for + the next gzip stream or raw data, once state->x.have is depleted. Returns 0 + on success, -1 on failure. */ +local int gz_decomp(state) + gz_statep state; +{ + int ret = Z_OK; + unsigned had; + z_streamp strm = &(state->strm); + + /* fill output buffer up to end of deflate stream */ + had = strm->avail_out; + do { + /* get more input for inflate() */ + if (strm->avail_in == 0 && gz_avail(state) == -1) + return -1; + if (strm->avail_in == 0) { + gz_error(state, Z_BUF_ERROR, "unexpected end of file"); + break; + } + + /* decompress and handle errors */ + ret = inflate(strm, Z_NO_FLUSH); + if (ret == Z_STREAM_ERROR || ret == Z_NEED_DICT) { + gz_error(state, Z_STREAM_ERROR, + "internal error: inflate stream corrupt"); + return -1; + } + if (ret == Z_MEM_ERROR) { + gz_error(state, Z_MEM_ERROR, "out of memory"); + return -1; + } + if (ret == Z_DATA_ERROR) { /* deflate stream invalid */ + gz_error(state, Z_DATA_ERROR, + strm->msg == NULL ? "compressed data error" : strm->msg); + return -1; + } + } while (strm->avail_out && ret != Z_STREAM_END); + + /* update available output */ + state->x.have = had - strm->avail_out; + state->x.next = strm->next_out - state->x.have; + + /* if the gzip stream completed successfully, look for another */ + if (ret == Z_STREAM_END) + state->how = LOOK; + + /* good decompression */ + return 0; +} + +/* Fetch data and put it in the output buffer. Assumes state->x.have is 0. + Data is either copied from the input file or decompressed from the input + file depending on state->how. If state->how is LOOK, then a gzip header is + looked for to determine whether to copy or decompress. Returns -1 on error, + otherwise 0. gz_fetch() will leave state->how as COPY or GZIP unless the + end of the input file has been reached and all data has been processed. */ +local int gz_fetch(state) + gz_statep state; +{ + z_streamp strm = &(state->strm); + + do { + switch(state->how) { + case LOOK: /* -> LOOK, COPY (only if never GZIP), or GZIP */ + if (gz_look(state) == -1) + return -1; + if (state->how == LOOK) + return 0; + break; + case COPY: /* -> COPY */ + if (gz_load(state, state->out, state->size << 1, &(state->x.have)) + == -1) + return -1; + state->x.next = state->out; + return 0; + case GZIP: /* -> GZIP or LOOK (if end of gzip stream) */ + strm->avail_out = state->size << 1; + strm->next_out = state->out; + if (gz_decomp(state) == -1) + return -1; + } + } while (state->x.have == 0 && (!state->eof || strm->avail_in)); + return 0; +} + +/* Skip len uncompressed bytes of output. Return -1 on error, 0 on success. */ +local int gz_skip(state, len) + gz_statep state; + z_off64_t len; +{ + unsigned n; + + /* skip over len bytes or reach end-of-file, whichever comes first */ + while (len) + /* skip over whatever is in output buffer */ + if (state->x.have) { + n = GT_OFF(state->x.have) || (z_off64_t)state->x.have > len ? + (unsigned)len : state->x.have; + state->x.have -= n; + state->x.next += n; + state->x.pos += n; + len -= n; + } + + /* output buffer empty -- return if we're at the end of the input */ + else if (state->eof && state->strm.avail_in == 0) + break; + + /* need more data to skip -- load up output buffer */ + else { + /* get more output, looking for header if required */ + if (gz_fetch(state) == -1) + return -1; + } + return 0; +} + +/* Read len bytes into buf from file, or less than len up to the end of the + input. Return the number of bytes read. If zero is returned, either the + end of file was reached, or there was an error. state->err must be + consulted in that case to determine which. */ +local z_size_t gz_read(state, buf, len) + gz_statep state; + voidp buf; + z_size_t len; +{ + z_size_t got; + unsigned n; + + /* if len is zero, avoid unnecessary operations */ + if (len == 0) + return 0; + + /* process a skip request */ + if (state->seek) { + state->seek = 0; + if (gz_skip(state, state->skip) == -1) + return 0; + } + + /* get len bytes to buf, or less than len if at the end */ + got = 0; + do { + /* set n to the maximum amount of len that fits in an unsigned int */ + n = (unsigned)-1; + if (n > len) + n = (unsigned)len; + + /* first just try copying data from the output buffer */ + if (state->x.have) { + if (state->x.have < n) + n = state->x.have; + memcpy(buf, state->x.next, n); + state->x.next += n; + state->x.have -= n; + } + + /* output buffer empty -- return if we're at the end of the input */ + else if (state->eof && state->strm.avail_in == 0) { + state->past = 1; /* tried to read past end */ + break; + } + + /* need output data -- for small len or new stream load up our output + buffer */ + else if (state->how == LOOK || n < (state->size << 1)) { + /* get more output, looking for header if required */ + if (gz_fetch(state) == -1) + return 0; + continue; /* no progress yet -- go back to copy above */ + /* the copy above assures that we will leave with space in the + output buffer, allowing at least one gzungetc() to succeed */ + } + + /* large len -- read directly into user buffer */ + else if (state->how == COPY) { /* read directly */ + if (gz_load(state, (unsigned char *)buf, n, &n) == -1) + return 0; + } + + /* large len -- decompress directly into user buffer */ + else { /* state->how == GZIP */ + state->strm.avail_out = n; + state->strm.next_out = (unsigned char *)buf; + if (gz_decomp(state) == -1) + return 0; + n = state->x.have; + state->x.have = 0; + } + + /* update progress */ + len -= n; + buf = (char *)buf + n; + got += n; + state->x.pos += n; + } while (len); + + /* return number of bytes read into user buffer */ + return got; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzread(file, buf, len) + gzFile file; + voidp buf; + unsigned len; +{ + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return -1; + state = (gz_statep)file; + + /* check that we're reading and that there's no (serious) error */ + if (state->mode != GZ_READ || + (state->err != Z_OK && state->err != Z_BUF_ERROR)) + return -1; + + /* since an int is returned, make sure len fits in one, otherwise return + with an error (this avoids a flaw in the interface) */ + if ((int)len < 0) { + gz_error(state, Z_STREAM_ERROR, "request does not fit in an int"); + return -1; + } + + /* read len or fewer bytes to buf */ + len = (unsigned)gz_read(state, buf, len); + + /* check for an error */ + if (len == 0 && state->err != Z_OK && state->err != Z_BUF_ERROR) + return -1; + + /* return the number of bytes read (this is assured to fit in an int) */ + return (int)len; +} + +/* -- see zlib.h -- */ +z_size_t ZEXPORT gzfread(buf, size, nitems, file) + voidp buf; + z_size_t size; + z_size_t nitems; + gzFile file; +{ + z_size_t len; + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return 0; + state = (gz_statep)file; + + /* check that we're reading and that there's no (serious) error */ + if (state->mode != GZ_READ || + (state->err != Z_OK && state->err != Z_BUF_ERROR)) + return 0; + + /* compute bytes to read -- error on overflow */ + len = nitems * size; + if (size && len / size != nitems) { + gz_error(state, Z_STREAM_ERROR, "request does not fit in a size_t"); + return 0; + } + + /* read len or fewer bytes to buf, return the number of full items read */ + return len ? gz_read(state, buf, len) / size : 0; +} + +/* -- see zlib.h -- */ +#ifdef Z_PREFIX_SET +# undef z_gzgetc +#else +# undef gzgetc +#endif +int ZEXPORT gzgetc(file) + gzFile file; +{ + unsigned char buf[1]; + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return -1; + state = (gz_statep)file; + + /* check that we're reading and that there's no (serious) error */ + if (state->mode != GZ_READ || + (state->err != Z_OK && state->err != Z_BUF_ERROR)) + return -1; + + /* try output buffer (no need to check for skip request) */ + if (state->x.have) { + state->x.have--; + state->x.pos++; + return *(state->x.next)++; + } + + /* nothing there -- try gz_read() */ + return gz_read(state, buf, 1) < 1 ? -1 : buf[0]; +} + +int ZEXPORT gzgetc_(file) +gzFile file; +{ + return gzgetc(file); +} + +/* -- see zlib.h -- */ +int ZEXPORT gzungetc(c, file) + int c; + gzFile file; +{ + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return -1; + state = (gz_statep)file; + + /* check that we're reading and that there's no (serious) error */ + if (state->mode != GZ_READ || + (state->err != Z_OK && state->err != Z_BUF_ERROR)) + return -1; + + /* process a skip request */ + if (state->seek) { + state->seek = 0; + if (gz_skip(state, state->skip) == -1) + return -1; + } + + /* can't push EOF */ + if (c < 0) + return -1; + + /* if output buffer empty, put byte at end (allows more pushing) */ + if (state->x.have == 0) { + state->x.have = 1; + state->x.next = state->out + (state->size << 1) - 1; + state->x.next[0] = (unsigned char)c; + state->x.pos--; + state->past = 0; + return c; + } + + /* if no room, give up (must have already done a gzungetc()) */ + if (state->x.have == (state->size << 1)) { + gz_error(state, Z_DATA_ERROR, "out of room to push characters"); + return -1; + } + + /* slide output data if needed and insert byte before existing data */ + if (state->x.next == state->out) { + unsigned char *src = state->out + state->x.have; + unsigned char *dest = state->out + (state->size << 1); + while (src > state->out) + *--dest = *--src; + state->x.next = dest; + } + state->x.have++; + state->x.next--; + state->x.next[0] = (unsigned char)c; + state->x.pos--; + state->past = 0; + return c; +} + +/* -- see zlib.h -- */ +char * ZEXPORT gzgets(file, buf, len) + gzFile file; + char *buf; + int len; +{ + unsigned left, n; + char *str; + unsigned char *eol; + gz_statep state; + + /* check parameters and get internal structure */ + if (file == NULL || buf == NULL || len < 1) + return NULL; + state = (gz_statep)file; + + /* check that we're reading and that there's no (serious) error */ + if (state->mode != GZ_READ || + (state->err != Z_OK && state->err != Z_BUF_ERROR)) + return NULL; + + /* process a skip request */ + if (state->seek) { + state->seek = 0; + if (gz_skip(state, state->skip) == -1) + return NULL; + } + + /* copy output bytes up to new line or len - 1, whichever comes first -- + append a terminating zero to the string (we don't check for a zero in + the contents, let the user worry about that) */ + str = buf; + left = (unsigned)len - 1; + if (left) do { + /* assure that something is in the output buffer */ + if (state->x.have == 0 && gz_fetch(state) == -1) + return NULL; /* error */ + if (state->x.have == 0) { /* end of file */ + state->past = 1; /* read past end */ + break; /* return what we have */ + } + + /* look for end-of-line in current output buffer */ + n = state->x.have > left ? left : state->x.have; + eol = (unsigned char *)memchr(state->x.next, '\n', n); + if (eol != NULL) + n = (unsigned)(eol - state->x.next) + 1; + + /* copy through end-of-line, or remainder if not found */ + memcpy(buf, state->x.next, n); + state->x.have -= n; + state->x.next += n; + state->x.pos += n; + left -= n; + buf += n; + } while (left && eol == NULL); + + /* return terminated string, or if nothing, end of file */ + if (buf == str) + return NULL; + buf[0] = 0; + return str; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzdirect(file) + gzFile file; +{ + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return 0; + state = (gz_statep)file; + + /* if the state is not known, but we can find out, then do so (this is + mainly for right after a gzopen() or gzdopen()) */ + if (state->mode == GZ_READ && state->how == LOOK && state->x.have == 0) + (void)gz_look(state); + + /* return 1 if transparent, 0 if processing a gzip stream */ + return state->direct; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzclose_r(file) + gzFile file; +{ + int ret, err; + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return Z_STREAM_ERROR; + state = (gz_statep)file; + + /* check that we're reading */ + if (state->mode != GZ_READ) + return Z_STREAM_ERROR; + + /* free memory and close file */ + if (state->size) { + inflateEnd(&(state->strm)); + free(state->out); + free(state->in); + } + err = state->err == Z_BUF_ERROR ? Z_BUF_ERROR : Z_OK; + gz_error(state, Z_OK, NULL); + free(state->path); + ret = close(state->fd); + free(state); + return ret ? Z_ERRNO : err; +} diff --git a/lib/os/windows/zlib-1.2.13/gzwrite.c b/lib/os/windows/zlib-1.2.13/gzwrite.c new file mode 100644 index 000000000000..eb8a0e5893ff --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/gzwrite.c @@ -0,0 +1,677 @@ +/* gzwrite.c -- zlib functions for writing gzip files + * Copyright (C) 2004-2019 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +#include "gzguts.h" + +/* Local functions */ +local int gz_init OF((gz_statep)); +local int gz_comp OF((gz_statep, int)); +local int gz_zero OF((gz_statep, z_off64_t)); +local z_size_t gz_write OF((gz_statep, voidpc, z_size_t)); + +/* Initialize state for writing a gzip file. Mark initialization by setting + state->size to non-zero. Return -1 on a memory allocation failure, or 0 on + success. */ +local int gz_init(state) + gz_statep state; +{ + int ret; + z_streamp strm = &(state->strm); + + /* allocate input buffer (double size for gzprintf) */ + state->in = (unsigned char *)malloc(state->want << 1); + if (state->in == NULL) { + gz_error(state, Z_MEM_ERROR, "out of memory"); + return -1; + } + + /* only need output buffer and deflate state if compressing */ + if (!state->direct) { + /* allocate output buffer */ + state->out = (unsigned char *)malloc(state->want); + if (state->out == NULL) { + free(state->in); + gz_error(state, Z_MEM_ERROR, "out of memory"); + return -1; + } + + /* allocate deflate memory, set up for gzip compression */ + strm->zalloc = Z_NULL; + strm->zfree = Z_NULL; + strm->opaque = Z_NULL; + ret = deflateInit2(strm, state->level, Z_DEFLATED, + MAX_WBITS + 16, DEF_MEM_LEVEL, state->strategy); + if (ret != Z_OK) { + free(state->out); + free(state->in); + gz_error(state, Z_MEM_ERROR, "out of memory"); + return -1; + } + strm->next_in = NULL; + } + + /* mark state as initialized */ + state->size = state->want; + + /* initialize write buffer if compressing */ + if (!state->direct) { + strm->avail_out = state->size; + strm->next_out = state->out; + state->x.next = strm->next_out; + } + return 0; +} + +/* Compress whatever is at avail_in and next_in and write to the output file. + Return -1 if there is an error writing to the output file or if gz_init() + fails to allocate memory, otherwise 0. flush is assumed to be a valid + deflate() flush value. If flush is Z_FINISH, then the deflate() state is + reset to start a new gzip stream. If gz->direct is true, then simply write + to the output file without compressing, and ignore flush. */ +local int gz_comp(state, flush) + gz_statep state; + int flush; +{ + int ret, writ; + unsigned have, put, max = ((unsigned)-1 >> 2) + 1; + z_streamp strm = &(state->strm); + + /* allocate memory if this is the first time through */ + if (state->size == 0 && gz_init(state) == -1) + return -1; + + /* write directly if requested */ + if (state->direct) { + while (strm->avail_in) { + put = strm->avail_in > max ? max : strm->avail_in; + writ = write(state->fd, strm->next_in, put); + if (writ < 0) { + gz_error(state, Z_ERRNO, zstrerror()); + return -1; + } + strm->avail_in -= (unsigned)writ; + strm->next_in += writ; + } + return 0; + } + + /* check for a pending reset */ + if (state->reset) { + /* don't start a new gzip member unless there is data to write */ + if (strm->avail_in == 0) + return 0; + deflateReset(strm); + state->reset = 0; + } + + /* run deflate() on provided input until it produces no more output */ + ret = Z_OK; + do { + /* write out current buffer contents if full, or if flushing, but if + doing Z_FINISH then don't write until we get to Z_STREAM_END */ + if (strm->avail_out == 0 || (flush != Z_NO_FLUSH && + (flush != Z_FINISH || ret == Z_STREAM_END))) { + while (strm->next_out > state->x.next) { + put = strm->next_out - state->x.next > (int)max ? max : + (unsigned)(strm->next_out - state->x.next); + writ = write(state->fd, state->x.next, put); + if (writ < 0) { + gz_error(state, Z_ERRNO, zstrerror()); + return -1; + } + state->x.next += writ; + } + if (strm->avail_out == 0) { + strm->avail_out = state->size; + strm->next_out = state->out; + state->x.next = state->out; + } + } + + /* compress */ + have = strm->avail_out; + ret = deflate(strm, flush); + if (ret == Z_STREAM_ERROR) { + gz_error(state, Z_STREAM_ERROR, + "internal error: deflate stream corrupt"); + return -1; + } + have -= strm->avail_out; + } while (have); + + /* if that completed a deflate stream, allow another to start */ + if (flush == Z_FINISH) + state->reset = 1; + + /* all done, no errors */ + return 0; +} + +/* Compress len zeros to output. Return -1 on a write error or memory + allocation failure by gz_comp(), or 0 on success. */ +local int gz_zero(state, len) + gz_statep state; + z_off64_t len; +{ + int first; + unsigned n; + z_streamp strm = &(state->strm); + + /* consume whatever's left in the input buffer */ + if (strm->avail_in && gz_comp(state, Z_NO_FLUSH) == -1) + return -1; + + /* compress len zeros (len guaranteed > 0) */ + first = 1; + while (len) { + n = GT_OFF(state->size) || (z_off64_t)state->size > len ? + (unsigned)len : state->size; + if (first) { + memset(state->in, 0, n); + first = 0; + } + strm->avail_in = n; + strm->next_in = state->in; + state->x.pos += n; + if (gz_comp(state, Z_NO_FLUSH) == -1) + return -1; + len -= n; + } + return 0; +} + +/* Write len bytes from buf to file. Return the number of bytes written. If + the returned value is less than len, then there was an error. */ +local z_size_t gz_write(state, buf, len) + gz_statep state; + voidpc buf; + z_size_t len; +{ + z_size_t put = len; + + /* if len is zero, avoid unnecessary operations */ + if (len == 0) + return 0; + + /* allocate memory if this is the first time through */ + if (state->size == 0 && gz_init(state) == -1) + return 0; + + /* check for seek request */ + if (state->seek) { + state->seek = 0; + if (gz_zero(state, state->skip) == -1) + return 0; + } + + /* for small len, copy to input buffer, otherwise compress directly */ + if (len < state->size) { + /* copy to input buffer, compress when full */ + do { + unsigned have, copy; + + if (state->strm.avail_in == 0) + state->strm.next_in = state->in; + have = (unsigned)((state->strm.next_in + state->strm.avail_in) - + state->in); + copy = state->size - have; + if (copy > len) + copy = (unsigned)len; + memcpy(state->in + have, buf, copy); + state->strm.avail_in += copy; + state->x.pos += copy; + buf = (const char *)buf + copy; + len -= copy; + if (len && gz_comp(state, Z_NO_FLUSH) == -1) + return 0; + } while (len); + } + else { + /* consume whatever's left in the input buffer */ + if (state->strm.avail_in && gz_comp(state, Z_NO_FLUSH) == -1) + return 0; + + /* directly compress user buffer to file */ + state->strm.next_in = (z_const Bytef *)buf; + do { + unsigned n = (unsigned)-1; + if (n > len) + n = (unsigned)len; + state->strm.avail_in = n; + state->x.pos += n; + if (gz_comp(state, Z_NO_FLUSH) == -1) + return 0; + len -= n; + } while (len); + } + + /* input was all buffered or compressed */ + return put; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzwrite(file, buf, len) + gzFile file; + voidpc buf; + unsigned len; +{ + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return 0; + state = (gz_statep)file; + + /* check that we're writing and that there's no error */ + if (state->mode != GZ_WRITE || state->err != Z_OK) + return 0; + + /* since an int is returned, make sure len fits in one, otherwise return + with an error (this avoids a flaw in the interface) */ + if ((int)len < 0) { + gz_error(state, Z_DATA_ERROR, "requested length does not fit in int"); + return 0; + } + + /* write len bytes from buf (the return value will fit in an int) */ + return (int)gz_write(state, buf, len); +} + +/* -- see zlib.h -- */ +z_size_t ZEXPORT gzfwrite(buf, size, nitems, file) + voidpc buf; + z_size_t size; + z_size_t nitems; + gzFile file; +{ + z_size_t len; + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return 0; + state = (gz_statep)file; + + /* check that we're writing and that there's no error */ + if (state->mode != GZ_WRITE || state->err != Z_OK) + return 0; + + /* compute bytes to read -- error on overflow */ + len = nitems * size; + if (size && len / size != nitems) { + gz_error(state, Z_STREAM_ERROR, "request does not fit in a size_t"); + return 0; + } + + /* write len bytes to buf, return the number of full items written */ + return len ? gz_write(state, buf, len) / size : 0; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzputc(file, c) + gzFile file; + int c; +{ + unsigned have; + unsigned char buf[1]; + gz_statep state; + z_streamp strm; + + /* get internal structure */ + if (file == NULL) + return -1; + state = (gz_statep)file; + strm = &(state->strm); + + /* check that we're writing and that there's no error */ + if (state->mode != GZ_WRITE || state->err != Z_OK) + return -1; + + /* check for seek request */ + if (state->seek) { + state->seek = 0; + if (gz_zero(state, state->skip) == -1) + return -1; + } + + /* try writing to input buffer for speed (state->size == 0 if buffer not + initialized) */ + if (state->size) { + if (strm->avail_in == 0) + strm->next_in = state->in; + have = (unsigned)((strm->next_in + strm->avail_in) - state->in); + if (have < state->size) { + state->in[have] = (unsigned char)c; + strm->avail_in++; + state->x.pos++; + return c & 0xff; + } + } + + /* no room in buffer or not initialized, use gz_write() */ + buf[0] = (unsigned char)c; + if (gz_write(state, buf, 1) != 1) + return -1; + return c & 0xff; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzputs(file, s) + gzFile file; + const char *s; +{ + z_size_t len, put; + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return -1; + state = (gz_statep)file; + + /* check that we're writing and that there's no error */ + if (state->mode != GZ_WRITE || state->err != Z_OK) + return -1; + + /* write string */ + len = strlen(s); + if ((int)len < 0 || (unsigned)len != len) { + gz_error(state, Z_STREAM_ERROR, "string length does not fit in int"); + return -1; + } + put = gz_write(state, s, len); + return put < len ? -1 : (int)len; +} + +#if defined(STDC) || defined(Z_HAVE_STDARG_H) +#include + +/* -- see zlib.h -- */ +int ZEXPORTVA gzvprintf(gzFile file, const char *format, va_list va) +{ + int len; + unsigned left; + char *next; + gz_statep state; + z_streamp strm; + + /* get internal structure */ + if (file == NULL) + return Z_STREAM_ERROR; + state = (gz_statep)file; + strm = &(state->strm); + + /* check that we're writing and that there's no error */ + if (state->mode != GZ_WRITE || state->err != Z_OK) + return Z_STREAM_ERROR; + + /* make sure we have some buffer space */ + if (state->size == 0 && gz_init(state) == -1) + return state->err; + + /* check for seek request */ + if (state->seek) { + state->seek = 0; + if (gz_zero(state, state->skip) == -1) + return state->err; + } + + /* do the printf() into the input buffer, put length in len -- the input + buffer is double-sized just for this function, so there is guaranteed to + be state->size bytes available after the current contents */ + if (strm->avail_in == 0) + strm->next_in = state->in; + next = (char *)(state->in + (strm->next_in - state->in) + strm->avail_in); + next[state->size - 1] = 0; +#ifdef NO_vsnprintf +# ifdef HAS_vsprintf_void + (void)vsprintf(next, format, va); + for (len = 0; len < state->size; len++) + if (next[len] == 0) break; +# else + len = vsprintf(next, format, va); +# endif +#else +# ifdef HAS_vsnprintf_void + (void)vsnprintf(next, state->size, format, va); + len = strlen(next); +# else + len = vsnprintf(next, state->size, format, va); +# endif +#endif + + /* check that printf() results fit in buffer */ + if (len == 0 || (unsigned)len >= state->size || next[state->size - 1] != 0) + return 0; + + /* update buffer and position, compress first half if past that */ + strm->avail_in += (unsigned)len; + state->x.pos += len; + if (strm->avail_in >= state->size) { + left = strm->avail_in - state->size; + strm->avail_in = state->size; + if (gz_comp(state, Z_NO_FLUSH) == -1) + return state->err; + memmove(state->in, state->in + state->size, left); + strm->next_in = state->in; + strm->avail_in = left; + } + return len; +} + +int ZEXPORTVA gzprintf(gzFile file, const char *format, ...) +{ + va_list va; + int ret; + + va_start(va, format); + ret = gzvprintf(file, format, va); + va_end(va); + return ret; +} + +#else /* !STDC && !Z_HAVE_STDARG_H */ + +/* -- see zlib.h -- */ +int ZEXPORTVA gzprintf(file, format, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, + a11, a12, a13, a14, a15, a16, a17, a18, a19, a20) + gzFile file; + const char *format; + int a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, + a11, a12, a13, a14, a15, a16, a17, a18, a19, a20; +{ + unsigned len, left; + char *next; + gz_statep state; + z_streamp strm; + + /* get internal structure */ + if (file == NULL) + return Z_STREAM_ERROR; + state = (gz_statep)file; + strm = &(state->strm); + + /* check that can really pass pointer in ints */ + if (sizeof(int) != sizeof(void *)) + return Z_STREAM_ERROR; + + /* check that we're writing and that there's no error */ + if (state->mode != GZ_WRITE || state->err != Z_OK) + return Z_STREAM_ERROR; + + /* make sure we have some buffer space */ + if (state->size == 0 && gz_init(state) == -1) + return state->error; + + /* check for seek request */ + if (state->seek) { + state->seek = 0; + if (gz_zero(state, state->skip) == -1) + return state->error; + } + + /* do the printf() into the input buffer, put length in len -- the input + buffer is double-sized just for this function, so there is guaranteed to + be state->size bytes available after the current contents */ + if (strm->avail_in == 0) + strm->next_in = state->in; + next = (char *)(strm->next_in + strm->avail_in); + next[state->size - 1] = 0; +#ifdef NO_snprintf +# ifdef HAS_sprintf_void + sprintf(next, format, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, + a13, a14, a15, a16, a17, a18, a19, a20); + for (len = 0; len < size; len++) + if (next[len] == 0) + break; +# else + len = sprintf(next, format, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, + a12, a13, a14, a15, a16, a17, a18, a19, a20); +# endif +#else +# ifdef HAS_snprintf_void + snprintf(next, state->size, format, a1, a2, a3, a4, a5, a6, a7, a8, a9, + a10, a11, a12, a13, a14, a15, a16, a17, a18, a19, a20); + len = strlen(next); +# else + len = snprintf(next, state->size, format, a1, a2, a3, a4, a5, a6, a7, a8, + a9, a10, a11, a12, a13, a14, a15, a16, a17, a18, a19, a20); +# endif +#endif + + /* check that printf() results fit in buffer */ + if (len == 0 || len >= state->size || next[state->size - 1] != 0) + return 0; + + /* update buffer and position, compress first half if past that */ + strm->avail_in += len; + state->x.pos += len; + if (strm->avail_in >= state->size) { + left = strm->avail_in - state->size; + strm->avail_in = state->size; + if (gz_comp(state, Z_NO_FLUSH) == -1) + return state->err; + memmove(state->in, state->in + state->size, left); + strm->next_in = state->in; + strm->avail_in = left; + } + return (int)len; +} + +#endif + +/* -- see zlib.h -- */ +int ZEXPORT gzflush(file, flush) + gzFile file; + int flush; +{ + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return Z_STREAM_ERROR; + state = (gz_statep)file; + + /* check that we're writing and that there's no error */ + if (state->mode != GZ_WRITE || state->err != Z_OK) + return Z_STREAM_ERROR; + + /* check flush parameter */ + if (flush < 0 || flush > Z_FINISH) + return Z_STREAM_ERROR; + + /* check for seek request */ + if (state->seek) { + state->seek = 0; + if (gz_zero(state, state->skip) == -1) + return state->err; + } + + /* compress remaining data with requested flush */ + (void)gz_comp(state, flush); + return state->err; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzsetparams(file, level, strategy) + gzFile file; + int level; + int strategy; +{ + gz_statep state; + z_streamp strm; + + /* get internal structure */ + if (file == NULL) + return Z_STREAM_ERROR; + state = (gz_statep)file; + strm = &(state->strm); + + /* check that we're writing and that there's no error */ + if (state->mode != GZ_WRITE || state->err != Z_OK) + return Z_STREAM_ERROR; + + /* if no change is requested, then do nothing */ + if (level == state->level && strategy == state->strategy) + return Z_OK; + + /* check for seek request */ + if (state->seek) { + state->seek = 0; + if (gz_zero(state, state->skip) == -1) + return state->err; + } + + /* change compression parameters for subsequent input */ + if (state->size) { + /* flush previous input with previous parameters before changing */ + if (strm->avail_in && gz_comp(state, Z_BLOCK) == -1) + return state->err; + deflateParams(strm, level, strategy); + } + state->level = level; + state->strategy = strategy; + return Z_OK; +} + +/* -- see zlib.h -- */ +int ZEXPORT gzclose_w(file) + gzFile file; +{ + int ret = Z_OK; + gz_statep state; + + /* get internal structure */ + if (file == NULL) + return Z_STREAM_ERROR; + state = (gz_statep)file; + + /* check that we're writing */ + if (state->mode != GZ_WRITE) + return Z_STREAM_ERROR; + + /* check for seek request */ + if (state->seek) { + state->seek = 0; + if (gz_zero(state, state->skip) == -1) + ret = state->err; + } + + /* flush, free memory, and close file */ + if (gz_comp(state, Z_FINISH) == -1) + ret = state->err; + if (state->size) { + if (!state->direct) { + (void)deflateEnd(&(state->strm)); + free(state->out); + } + free(state->in); + } + gz_error(state, Z_OK, NULL); + free(state->path); + if (close(state->fd) == -1) + ret = Z_ERRNO; + free(state); + return ret; +} diff --git a/lib/os/windows/zlib-1.2.13/infback.c b/lib/os/windows/zlib-1.2.13/infback.c new file mode 100644 index 000000000000..babeaf1806f9 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/infback.c @@ -0,0 +1,644 @@ +/* infback.c -- inflate using a call-back interface + * Copyright (C) 1995-2022 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* + This code is largely copied from inflate.c. Normally either infback.o or + inflate.o would be linked into an application--not both. The interface + with inffast.c is retained so that optimized assembler-coded versions of + inflate_fast() can be used with either inflate.c or infback.c. + */ + +#include "zutil.h" +#include "inftrees.h" +#include "inflate.h" +#include "inffast.h" + +/* function prototypes */ +local void fixedtables OF((struct inflate_state FAR *state)); + +/* + strm provides memory allocation functions in zalloc and zfree, or + Z_NULL to use the library memory allocation functions. + + windowBits is in the range 8..15, and window is a user-supplied + window and output buffer that is 2**windowBits bytes. + */ +int ZEXPORT inflateBackInit_(strm, windowBits, window, version, stream_size) +z_streamp strm; +int windowBits; +unsigned char FAR *window; +const char *version; +int stream_size; +{ + struct inflate_state FAR *state; + + if (version == Z_NULL || version[0] != ZLIB_VERSION[0] || + stream_size != (int)(sizeof(z_stream))) + return Z_VERSION_ERROR; + if (strm == Z_NULL || window == Z_NULL || + windowBits < 8 || windowBits > 15) + return Z_STREAM_ERROR; + strm->msg = Z_NULL; /* in case we return an error */ + if (strm->zalloc == (alloc_func)0) { +#ifdef Z_SOLO + return Z_STREAM_ERROR; +#else + strm->zalloc = zcalloc; + strm->opaque = (voidpf)0; +#endif + } + if (strm->zfree == (free_func)0) +#ifdef Z_SOLO + return Z_STREAM_ERROR; +#else + strm->zfree = zcfree; +#endif + state = (struct inflate_state FAR *)ZALLOC(strm, 1, + sizeof(struct inflate_state)); + if (state == Z_NULL) return Z_MEM_ERROR; + Tracev((stderr, "inflate: allocated\n")); + strm->state = (struct internal_state FAR *)state; + state->dmax = 32768U; + state->wbits = (uInt)windowBits; + state->wsize = 1U << windowBits; + state->window = window; + state->wnext = 0; + state->whave = 0; + state->sane = 1; + return Z_OK; +} + +/* + Return state with length and distance decoding tables and index sizes set to + fixed code decoding. Normally this returns fixed tables from inffixed.h. + If BUILDFIXED is defined, then instead this routine builds the tables the + first time it's called, and returns those tables the first time and + thereafter. This reduces the size of the code by about 2K bytes, in + exchange for a little execution time. However, BUILDFIXED should not be + used for threaded applications, since the rewriting of the tables and virgin + may not be thread-safe. + */ +local void fixedtables(state) +struct inflate_state FAR *state; +{ +#ifdef BUILDFIXED + static int virgin = 1; + static code *lenfix, *distfix; + static code fixed[544]; + + /* build fixed huffman tables if first call (may not be thread safe) */ + if (virgin) { + unsigned sym, bits; + static code *next; + + /* literal/length table */ + sym = 0; + while (sym < 144) state->lens[sym++] = 8; + while (sym < 256) state->lens[sym++] = 9; + while (sym < 280) state->lens[sym++] = 7; + while (sym < 288) state->lens[sym++] = 8; + next = fixed; + lenfix = next; + bits = 9; + inflate_table(LENS, state->lens, 288, &(next), &(bits), state->work); + + /* distance table */ + sym = 0; + while (sym < 32) state->lens[sym++] = 5; + distfix = next; + bits = 5; + inflate_table(DISTS, state->lens, 32, &(next), &(bits), state->work); + + /* do this just once */ + virgin = 0; + } +#else /* !BUILDFIXED */ +# include "inffixed.h" +#endif /* BUILDFIXED */ + state->lencode = lenfix; + state->lenbits = 9; + state->distcode = distfix; + state->distbits = 5; +} + +/* Macros for inflateBack(): */ + +/* Load returned state from inflate_fast() */ +#define LOAD() \ + do { \ + put = strm->next_out; \ + left = strm->avail_out; \ + next = strm->next_in; \ + have = strm->avail_in; \ + hold = state->hold; \ + bits = state->bits; \ + } while (0) + +/* Set state from registers for inflate_fast() */ +#define RESTORE() \ + do { \ + strm->next_out = put; \ + strm->avail_out = left; \ + strm->next_in = next; \ + strm->avail_in = have; \ + state->hold = hold; \ + state->bits = bits; \ + } while (0) + +/* Clear the input bit accumulator */ +#define INITBITS() \ + do { \ + hold = 0; \ + bits = 0; \ + } while (0) + +/* Assure that some input is available. If input is requested, but denied, + then return a Z_BUF_ERROR from inflateBack(). */ +#define PULL() \ + do { \ + if (have == 0) { \ + have = in(in_desc, &next); \ + if (have == 0) { \ + next = Z_NULL; \ + ret = Z_BUF_ERROR; \ + goto inf_leave; \ + } \ + } \ + } while (0) + +/* Get a byte of input into the bit accumulator, or return from inflateBack() + with an error if there is no input available. */ +#define PULLBYTE() \ + do { \ + PULL(); \ + have--; \ + hold += (unsigned long)(*next++) << bits; \ + bits += 8; \ + } while (0) + +/* Assure that there are at least n bits in the bit accumulator. If there is + not enough available input to do that, then return from inflateBack() with + an error. */ +#define NEEDBITS(n) \ + do { \ + while (bits < (unsigned)(n)) \ + PULLBYTE(); \ + } while (0) + +/* Return the low n bits of the bit accumulator (n < 16) */ +#define BITS(n) \ + ((unsigned)hold & ((1U << (n)) - 1)) + +/* Remove n bits from the bit accumulator */ +#define DROPBITS(n) \ + do { \ + hold >>= (n); \ + bits -= (unsigned)(n); \ + } while (0) + +/* Remove zero to seven bits as needed to go to a byte boundary */ +#define BYTEBITS() \ + do { \ + hold >>= bits & 7; \ + bits -= bits & 7; \ + } while (0) + +/* Assure that some output space is available, by writing out the window + if it's full. If the write fails, return from inflateBack() with a + Z_BUF_ERROR. */ +#define ROOM() \ + do { \ + if (left == 0) { \ + put = state->window; \ + left = state->wsize; \ + state->whave = left; \ + if (out(out_desc, put, left)) { \ + ret = Z_BUF_ERROR; \ + goto inf_leave; \ + } \ + } \ + } while (0) + +/* + strm provides the memory allocation functions and window buffer on input, + and provides information on the unused input on return. For Z_DATA_ERROR + returns, strm will also provide an error message. + + in() and out() are the call-back input and output functions. When + inflateBack() needs more input, it calls in(). When inflateBack() has + filled the window with output, or when it completes with data in the + window, it calls out() to write out the data. The application must not + change the provided input until in() is called again or inflateBack() + returns. The application must not change the window/output buffer until + inflateBack() returns. + + in() and out() are called with a descriptor parameter provided in the + inflateBack() call. This parameter can be a structure that provides the + information required to do the read or write, as well as accumulated + information on the input and output such as totals and check values. + + in() should return zero on failure. out() should return non-zero on + failure. If either in() or out() fails, than inflateBack() returns a + Z_BUF_ERROR. strm->next_in can be checked for Z_NULL to see whether it + was in() or out() that caused in the error. Otherwise, inflateBack() + returns Z_STREAM_END on success, Z_DATA_ERROR for an deflate format + error, or Z_MEM_ERROR if it could not allocate memory for the state. + inflateBack() can also return Z_STREAM_ERROR if the input parameters + are not correct, i.e. strm is Z_NULL or the state was not initialized. + */ +int ZEXPORT inflateBack(strm, in, in_desc, out, out_desc) +z_streamp strm; +in_func in; +void FAR *in_desc; +out_func out; +void FAR *out_desc; +{ + struct inflate_state FAR *state; + z_const unsigned char FAR *next; /* next input */ + unsigned char FAR *put; /* next output */ + unsigned have, left; /* available input and output */ + unsigned long hold; /* bit buffer */ + unsigned bits; /* bits in bit buffer */ + unsigned copy; /* number of stored or match bytes to copy */ + unsigned char FAR *from; /* where to copy match bytes from */ + code here; /* current decoding table entry */ + code last; /* parent table entry */ + unsigned len; /* length to copy for repeats, bits to drop */ + int ret; /* return code */ + static const unsigned short order[19] = /* permutation of code lengths */ + {16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; + + /* Check that the strm exists and that the state was initialized */ + if (strm == Z_NULL || strm->state == Z_NULL) + return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + + /* Reset the state */ + strm->msg = Z_NULL; + state->mode = TYPE; + state->last = 0; + state->whave = 0; + next = strm->next_in; + have = next != Z_NULL ? strm->avail_in : 0; + hold = 0; + bits = 0; + put = state->window; + left = state->wsize; + + /* Inflate until end of block marked as last */ + for (;;) + switch (state->mode) { + case TYPE: + /* determine and dispatch block type */ + if (state->last) { + BYTEBITS(); + state->mode = DONE; + break; + } + NEEDBITS(3); + state->last = BITS(1); + DROPBITS(1); + switch (BITS(2)) { + case 0: /* stored block */ + Tracev((stderr, "inflate: stored block%s\n", + state->last ? " (last)" : "")); + state->mode = STORED; + break; + case 1: /* fixed block */ + fixedtables(state); + Tracev((stderr, "inflate: fixed codes block%s\n", + state->last ? " (last)" : "")); + state->mode = LEN; /* decode codes */ + break; + case 2: /* dynamic block */ + Tracev((stderr, "inflate: dynamic codes block%s\n", + state->last ? " (last)" : "")); + state->mode = TABLE; + break; + case 3: + strm->msg = (char *)"invalid block type"; + state->mode = BAD; + } + DROPBITS(2); + break; + + case STORED: + /* get and verify stored block length */ + BYTEBITS(); /* go to byte boundary */ + NEEDBITS(32); + if ((hold & 0xffff) != ((hold >> 16) ^ 0xffff)) { + strm->msg = (char *)"invalid stored block lengths"; + state->mode = BAD; + break; + } + state->length = (unsigned)hold & 0xffff; + Tracev((stderr, "inflate: stored length %u\n", + state->length)); + INITBITS(); + + /* copy stored block from input to output */ + while (state->length != 0) { + copy = state->length; + PULL(); + ROOM(); + if (copy > have) copy = have; + if (copy > left) copy = left; + zmemcpy(put, next, copy); + have -= copy; + next += copy; + left -= copy; + put += copy; + state->length -= copy; + } + Tracev((stderr, "inflate: stored end\n")); + state->mode = TYPE; + break; + + case TABLE: + /* get dynamic table entries descriptor */ + NEEDBITS(14); + state->nlen = BITS(5) + 257; + DROPBITS(5); + state->ndist = BITS(5) + 1; + DROPBITS(5); + state->ncode = BITS(4) + 4; + DROPBITS(4); +#ifndef PKZIP_BUG_WORKAROUND + if (state->nlen > 286 || state->ndist > 30) { + strm->msg = (char *)"too many length or distance symbols"; + state->mode = BAD; + break; + } +#endif + Tracev((stderr, "inflate: table sizes ok\n")); + + /* get code length code lengths (not a typo) */ + state->have = 0; + while (state->have < state->ncode) { + NEEDBITS(3); + state->lens[order[state->have++]] = (unsigned short)BITS(3); + DROPBITS(3); + } + while (state->have < 19) + state->lens[order[state->have++]] = 0; + state->next = state->codes; + state->lencode = (code const FAR *)(state->next); + state->lenbits = 7; + ret = inflate_table(CODES, state->lens, 19, &(state->next), + &(state->lenbits), state->work); + if (ret) { + strm->msg = (char *)"invalid code lengths set"; + state->mode = BAD; + break; + } + Tracev((stderr, "inflate: code lengths ok\n")); + + /* get length and distance code code lengths */ + state->have = 0; + while (state->have < state->nlen + state->ndist) { + for (;;) { + here = state->lencode[BITS(state->lenbits)]; + if ((unsigned)(here.bits) <= bits) break; + PULLBYTE(); + } + if (here.val < 16) { + DROPBITS(here.bits); + state->lens[state->have++] = here.val; + } + else { + if (here.val == 16) { + NEEDBITS(here.bits + 2); + DROPBITS(here.bits); + if (state->have == 0) { + strm->msg = (char *)"invalid bit length repeat"; + state->mode = BAD; + break; + } + len = (unsigned)(state->lens[state->have - 1]); + copy = 3 + BITS(2); + DROPBITS(2); + } + else if (here.val == 17) { + NEEDBITS(here.bits + 3); + DROPBITS(here.bits); + len = 0; + copy = 3 + BITS(3); + DROPBITS(3); + } + else { + NEEDBITS(here.bits + 7); + DROPBITS(here.bits); + len = 0; + copy = 11 + BITS(7); + DROPBITS(7); + } + if (state->have + copy > state->nlen + state->ndist) { + strm->msg = (char *)"invalid bit length repeat"; + state->mode = BAD; + break; + } + while (copy--) + state->lens[state->have++] = (unsigned short)len; + } + } + + /* handle error breaks in while */ + if (state->mode == BAD) break; + + /* check for end-of-block code (better have one) */ + if (state->lens[256] == 0) { + strm->msg = (char *)"invalid code -- missing end-of-block"; + state->mode = BAD; + break; + } + + /* build code tables -- note: do not change the lenbits or distbits + values here (9 and 6) without reading the comments in inftrees.h + concerning the ENOUGH constants, which depend on those values */ + state->next = state->codes; + state->lencode = (code const FAR *)(state->next); + state->lenbits = 9; + ret = inflate_table(LENS, state->lens, state->nlen, &(state->next), + &(state->lenbits), state->work); + if (ret) { + strm->msg = (char *)"invalid literal/lengths set"; + state->mode = BAD; + break; + } + state->distcode = (code const FAR *)(state->next); + state->distbits = 6; + ret = inflate_table(DISTS, state->lens + state->nlen, state->ndist, + &(state->next), &(state->distbits), state->work); + if (ret) { + strm->msg = (char *)"invalid distances set"; + state->mode = BAD; + break; + } + Tracev((stderr, "inflate: codes ok\n")); + state->mode = LEN; + /* fallthrough */ + + case LEN: + /* use inflate_fast() if we have enough input and output */ + if (have >= 6 && left >= 258) { + RESTORE(); + if (state->whave < state->wsize) + state->whave = state->wsize - left; + inflate_fast(strm, state->wsize); + LOAD(); + break; + } + + /* get a literal, length, or end-of-block code */ + for (;;) { + here = state->lencode[BITS(state->lenbits)]; + if ((unsigned)(here.bits) <= bits) break; + PULLBYTE(); + } + if (here.op && (here.op & 0xf0) == 0) { + last = here; + for (;;) { + here = state->lencode[last.val + + (BITS(last.bits + last.op) >> last.bits)]; + if ((unsigned)(last.bits + here.bits) <= bits) break; + PULLBYTE(); + } + DROPBITS(last.bits); + } + DROPBITS(here.bits); + state->length = (unsigned)here.val; + + /* process literal */ + if (here.op == 0) { + Tracevv((stderr, here.val >= 0x20 && here.val < 0x7f ? + "inflate: literal '%c'\n" : + "inflate: literal 0x%02x\n", here.val)); + ROOM(); + *put++ = (unsigned char)(state->length); + left--; + state->mode = LEN; + break; + } + + /* process end of block */ + if (here.op & 32) { + Tracevv((stderr, "inflate: end of block\n")); + state->mode = TYPE; + break; + } + + /* invalid code */ + if (here.op & 64) { + strm->msg = (char *)"invalid literal/length code"; + state->mode = BAD; + break; + } + + /* length code -- get extra bits, if any */ + state->extra = (unsigned)(here.op) & 15; + if (state->extra != 0) { + NEEDBITS(state->extra); + state->length += BITS(state->extra); + DROPBITS(state->extra); + } + Tracevv((stderr, "inflate: length %u\n", state->length)); + + /* get distance code */ + for (;;) { + here = state->distcode[BITS(state->distbits)]; + if ((unsigned)(here.bits) <= bits) break; + PULLBYTE(); + } + if ((here.op & 0xf0) == 0) { + last = here; + for (;;) { + here = state->distcode[last.val + + (BITS(last.bits + last.op) >> last.bits)]; + if ((unsigned)(last.bits + here.bits) <= bits) break; + PULLBYTE(); + } + DROPBITS(last.bits); + } + DROPBITS(here.bits); + if (here.op & 64) { + strm->msg = (char *)"invalid distance code"; + state->mode = BAD; + break; + } + state->offset = (unsigned)here.val; + + /* get distance extra bits, if any */ + state->extra = (unsigned)(here.op) & 15; + if (state->extra != 0) { + NEEDBITS(state->extra); + state->offset += BITS(state->extra); + DROPBITS(state->extra); + } + if (state->offset > state->wsize - (state->whave < state->wsize ? + left : 0)) { + strm->msg = (char *)"invalid distance too far back"; + state->mode = BAD; + break; + } + Tracevv((stderr, "inflate: distance %u\n", state->offset)); + + /* copy match from window to output */ + do { + ROOM(); + copy = state->wsize - state->offset; + if (copy < left) { + from = put + copy; + copy = left - copy; + } + else { + from = put - state->offset; + copy = left; + } + if (copy > state->length) copy = state->length; + state->length -= copy; + left -= copy; + do { + *put++ = *from++; + } while (--copy); + } while (state->length != 0); + break; + + case DONE: + /* inflate stream terminated properly */ + ret = Z_STREAM_END; + goto inf_leave; + + case BAD: + ret = Z_DATA_ERROR; + goto inf_leave; + + default: + /* can't happen, but makes compilers happy */ + ret = Z_STREAM_ERROR; + goto inf_leave; + } + + /* Write leftover output and return unused input */ + inf_leave: + if (left < state->wsize) { + if (out(out_desc, state->window, state->wsize - left) && + ret == Z_STREAM_END) + ret = Z_BUF_ERROR; + } + strm->next_in = next; + strm->avail_in = have; + return ret; +} + +int ZEXPORT inflateBackEnd(strm) +z_streamp strm; +{ + if (strm == Z_NULL || strm->state == Z_NULL || strm->zfree == (free_func)0) + return Z_STREAM_ERROR; + ZFREE(strm, strm->state); + strm->state = Z_NULL; + Tracev((stderr, "inflate: end\n")); + return Z_OK; +} diff --git a/lib/os/windows/zlib-1.2.13/inffast.c b/lib/os/windows/zlib-1.2.13/inffast.c new file mode 100644 index 000000000000..1fec7f363fa6 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/inffast.c @@ -0,0 +1,323 @@ +/* inffast.c -- fast decoding + * Copyright (C) 1995-2017 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +#include "zutil.h" +#include "inftrees.h" +#include "inflate.h" +#include "inffast.h" + +#ifdef ASMINF +# pragma message("Assembler code may have bugs -- use at your own risk") +#else + +/* + Decode literal, length, and distance codes and write out the resulting + literal and match bytes until either not enough input or output is + available, an end-of-block is encountered, or a data error is encountered. + When large enough input and output buffers are supplied to inflate(), for + example, a 16K input buffer and a 64K output buffer, more than 95% of the + inflate execution time is spent in this routine. + + Entry assumptions: + + state->mode == LEN + strm->avail_in >= 6 + strm->avail_out >= 258 + start >= strm->avail_out + state->bits < 8 + + On return, state->mode is one of: + + LEN -- ran out of enough output space or enough available input + TYPE -- reached end of block code, inflate() to interpret next block + BAD -- error in block data + + Notes: + + - The maximum input bits used by a length/distance pair is 15 bits for the + length code, 5 bits for the length extra, 15 bits for the distance code, + and 13 bits for the distance extra. This totals 48 bits, or six bytes. + Therefore if strm->avail_in >= 6, then there is enough input to avoid + checking for available input while decoding. + + - The maximum bytes that a single length/distance pair can output is 258 + bytes, which is the maximum length that can be coded. inflate_fast() + requires strm->avail_out >= 258 for each loop to avoid checking for + output space. + */ +void ZLIB_INTERNAL inflate_fast(strm, start) +z_streamp strm; +unsigned start; /* inflate()'s starting value for strm->avail_out */ +{ + struct inflate_state FAR *state; + z_const unsigned char FAR *in; /* local strm->next_in */ + z_const unsigned char FAR *last; /* have enough input while in < last */ + unsigned char FAR *out; /* local strm->next_out */ + unsigned char FAR *beg; /* inflate()'s initial strm->next_out */ + unsigned char FAR *end; /* while out < end, enough space available */ +#ifdef INFLATE_STRICT + unsigned dmax; /* maximum distance from zlib header */ +#endif + unsigned wsize; /* window size or zero if not using window */ + unsigned whave; /* valid bytes in the window */ + unsigned wnext; /* window write index */ + unsigned char FAR *window; /* allocated sliding window, if wsize != 0 */ + unsigned long hold; /* local strm->hold */ + unsigned bits; /* local strm->bits */ + code const FAR *lcode; /* local strm->lencode */ + code const FAR *dcode; /* local strm->distcode */ + unsigned lmask; /* mask for first level of length codes */ + unsigned dmask; /* mask for first level of distance codes */ + code const *here; /* retrieved table entry */ + unsigned op; /* code bits, operation, extra bits, or */ + /* window position, window bytes to copy */ + unsigned len; /* match length, unused bytes */ + unsigned dist; /* match distance */ + unsigned char FAR *from; /* where to copy match from */ + + /* copy state to local variables */ + state = (struct inflate_state FAR *)strm->state; + in = strm->next_in; + last = in + (strm->avail_in - 5); + out = strm->next_out; + beg = out - (start - strm->avail_out); + end = out + (strm->avail_out - 257); +#ifdef INFLATE_STRICT + dmax = state->dmax; +#endif + wsize = state->wsize; + whave = state->whave; + wnext = state->wnext; + window = state->window; + hold = state->hold; + bits = state->bits; + lcode = state->lencode; + dcode = state->distcode; + lmask = (1U << state->lenbits) - 1; + dmask = (1U << state->distbits) - 1; + + /* decode literals and length/distances until end-of-block or not enough + input data or output space */ + do { + if (bits < 15) { + hold += (unsigned long)(*in++) << bits; + bits += 8; + hold += (unsigned long)(*in++) << bits; + bits += 8; + } + here = lcode + (hold & lmask); + dolen: + op = (unsigned)(here->bits); + hold >>= op; + bits -= op; + op = (unsigned)(here->op); + if (op == 0) { /* literal */ + Tracevv((stderr, here->val >= 0x20 && here->val < 0x7f ? + "inflate: literal '%c'\n" : + "inflate: literal 0x%02x\n", here->val)); + *out++ = (unsigned char)(here->val); + } + else if (op & 16) { /* length base */ + len = (unsigned)(here->val); + op &= 15; /* number of extra bits */ + if (op) { + if (bits < op) { + hold += (unsigned long)(*in++) << bits; + bits += 8; + } + len += (unsigned)hold & ((1U << op) - 1); + hold >>= op; + bits -= op; + } + Tracevv((stderr, "inflate: length %u\n", len)); + if (bits < 15) { + hold += (unsigned long)(*in++) << bits; + bits += 8; + hold += (unsigned long)(*in++) << bits; + bits += 8; + } + here = dcode + (hold & dmask); + dodist: + op = (unsigned)(here->bits); + hold >>= op; + bits -= op; + op = (unsigned)(here->op); + if (op & 16) { /* distance base */ + dist = (unsigned)(here->val); + op &= 15; /* number of extra bits */ + if (bits < op) { + hold += (unsigned long)(*in++) << bits; + bits += 8; + if (bits < op) { + hold += (unsigned long)(*in++) << bits; + bits += 8; + } + } + dist += (unsigned)hold & ((1U << op) - 1); +#ifdef INFLATE_STRICT + if (dist > dmax) { + strm->msg = (char *)"invalid distance too far back"; + state->mode = BAD; + break; + } +#endif + hold >>= op; + bits -= op; + Tracevv((stderr, "inflate: distance %u\n", dist)); + op = (unsigned)(out - beg); /* max distance in output */ + if (dist > op) { /* see if copy from window */ + op = dist - op; /* distance back in window */ + if (op > whave) { + if (state->sane) { + strm->msg = + (char *)"invalid distance too far back"; + state->mode = BAD; + break; + } +#ifdef INFLATE_ALLOW_INVALID_DISTANCE_TOOFAR_ARRR + if (len <= op - whave) { + do { + *out++ = 0; + } while (--len); + continue; + } + len -= op - whave; + do { + *out++ = 0; + } while (--op > whave); + if (op == 0) { + from = out - dist; + do { + *out++ = *from++; + } while (--len); + continue; + } +#endif + } + from = window; + if (wnext == 0) { /* very common case */ + from += wsize - op; + if (op < len) { /* some from window */ + len -= op; + do { + *out++ = *from++; + } while (--op); + from = out - dist; /* rest from output */ + } + } + else if (wnext < op) { /* wrap around window */ + from += wsize + wnext - op; + op -= wnext; + if (op < len) { /* some from end of window */ + len -= op; + do { + *out++ = *from++; + } while (--op); + from = window; + if (wnext < len) { /* some from start of window */ + op = wnext; + len -= op; + do { + *out++ = *from++; + } while (--op); + from = out - dist; /* rest from output */ + } + } + } + else { /* contiguous in window */ + from += wnext - op; + if (op < len) { /* some from window */ + len -= op; + do { + *out++ = *from++; + } while (--op); + from = out - dist; /* rest from output */ + } + } + while (len > 2) { + *out++ = *from++; + *out++ = *from++; + *out++ = *from++; + len -= 3; + } + if (len) { + *out++ = *from++; + if (len > 1) + *out++ = *from++; + } + } + else { + from = out - dist; /* copy direct from output */ + do { /* minimum length is three */ + *out++ = *from++; + *out++ = *from++; + *out++ = *from++; + len -= 3; + } while (len > 2); + if (len) { + *out++ = *from++; + if (len > 1) + *out++ = *from++; + } + } + } + else if ((op & 64) == 0) { /* 2nd level distance code */ + here = dcode + here->val + (hold & ((1U << op) - 1)); + goto dodist; + } + else { + strm->msg = (char *)"invalid distance code"; + state->mode = BAD; + break; + } + } + else if ((op & 64) == 0) { /* 2nd level length code */ + here = lcode + here->val + (hold & ((1U << op) - 1)); + goto dolen; + } + else if (op & 32) { /* end-of-block */ + Tracevv((stderr, "inflate: end of block\n")); + state->mode = TYPE; + break; + } + else { + strm->msg = (char *)"invalid literal/length code"; + state->mode = BAD; + break; + } + } while (in < last && out < end); + + /* return unused bytes (on entry, bits < 8, so in won't go too far back) */ + len = bits >> 3; + in -= len; + bits -= len << 3; + hold &= (1U << bits) - 1; + + /* update state and return */ + strm->next_in = in; + strm->next_out = out; + strm->avail_in = (unsigned)(in < last ? 5 + (last - in) : 5 - (in - last)); + strm->avail_out = (unsigned)(out < end ? + 257 + (end - out) : 257 - (out - end)); + state->hold = hold; + state->bits = bits; + return; +} + +/* + inflate_fast() speedups that turned out slower (on a PowerPC G3 750CXe): + - Using bit fields for code structure + - Different op definition to avoid & for extra bits (do & for table bits) + - Three separate decoding do-loops for direct, window, and wnext == 0 + - Special case for distance > 1 copies to do overlapped load and store copy + - Explicit branch predictions (based on measured branch probabilities) + - Deferring match copy and interspersed it with decoding subsequent codes + - Swapping literal/length else + - Swapping window/direct else + - Larger unrolled copy loops (three is about right) + - Moving len -= 3 statement into middle of loop + */ + +#endif /* !ASMINF */ diff --git a/lib/os/windows/zlib-1.2.13/inffast.h b/lib/os/windows/zlib-1.2.13/inffast.h new file mode 100644 index 000000000000..e5c1aa4ca8cd --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/inffast.h @@ -0,0 +1,11 @@ +/* inffast.h -- header to use inffast.c + * Copyright (C) 1995-2003, 2010 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* WARNING: this file should *not* be used by applications. It is + part of the implementation of the compression library and is + subject to change. Applications should only use zlib.h. + */ + +void ZLIB_INTERNAL inflate_fast OF((z_streamp strm, unsigned start)); diff --git a/lib/os/windows/zlib-1.2.13/inffixed.h b/lib/os/windows/zlib-1.2.13/inffixed.h new file mode 100644 index 000000000000..d62832776948 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/inffixed.h @@ -0,0 +1,94 @@ + /* inffixed.h -- table for decoding fixed codes + * Generated automatically by makefixed(). + */ + + /* WARNING: this file should *not* be used by applications. + It is part of the implementation of this library and is + subject to change. Applications should only use zlib.h. + */ + + static const code lenfix[512] = { + {96,7,0},{0,8,80},{0,8,16},{20,8,115},{18,7,31},{0,8,112},{0,8,48}, + {0,9,192},{16,7,10},{0,8,96},{0,8,32},{0,9,160},{0,8,0},{0,8,128}, + {0,8,64},{0,9,224},{16,7,6},{0,8,88},{0,8,24},{0,9,144},{19,7,59}, + {0,8,120},{0,8,56},{0,9,208},{17,7,17},{0,8,104},{0,8,40},{0,9,176}, + {0,8,8},{0,8,136},{0,8,72},{0,9,240},{16,7,4},{0,8,84},{0,8,20}, + {21,8,227},{19,7,43},{0,8,116},{0,8,52},{0,9,200},{17,7,13},{0,8,100}, + {0,8,36},{0,9,168},{0,8,4},{0,8,132},{0,8,68},{0,9,232},{16,7,8}, + {0,8,92},{0,8,28},{0,9,152},{20,7,83},{0,8,124},{0,8,60},{0,9,216}, + {18,7,23},{0,8,108},{0,8,44},{0,9,184},{0,8,12},{0,8,140},{0,8,76}, + {0,9,248},{16,7,3},{0,8,82},{0,8,18},{21,8,163},{19,7,35},{0,8,114}, + {0,8,50},{0,9,196},{17,7,11},{0,8,98},{0,8,34},{0,9,164},{0,8,2}, + {0,8,130},{0,8,66},{0,9,228},{16,7,7},{0,8,90},{0,8,26},{0,9,148}, + {20,7,67},{0,8,122},{0,8,58},{0,9,212},{18,7,19},{0,8,106},{0,8,42}, + {0,9,180},{0,8,10},{0,8,138},{0,8,74},{0,9,244},{16,7,5},{0,8,86}, + {0,8,22},{64,8,0},{19,7,51},{0,8,118},{0,8,54},{0,9,204},{17,7,15}, + {0,8,102},{0,8,38},{0,9,172},{0,8,6},{0,8,134},{0,8,70},{0,9,236}, + {16,7,9},{0,8,94},{0,8,30},{0,9,156},{20,7,99},{0,8,126},{0,8,62}, + {0,9,220},{18,7,27},{0,8,110},{0,8,46},{0,9,188},{0,8,14},{0,8,142}, + {0,8,78},{0,9,252},{96,7,0},{0,8,81},{0,8,17},{21,8,131},{18,7,31}, + {0,8,113},{0,8,49},{0,9,194},{16,7,10},{0,8,97},{0,8,33},{0,9,162}, + {0,8,1},{0,8,129},{0,8,65},{0,9,226},{16,7,6},{0,8,89},{0,8,25}, + {0,9,146},{19,7,59},{0,8,121},{0,8,57},{0,9,210},{17,7,17},{0,8,105}, + {0,8,41},{0,9,178},{0,8,9},{0,8,137},{0,8,73},{0,9,242},{16,7,4}, + {0,8,85},{0,8,21},{16,8,258},{19,7,43},{0,8,117},{0,8,53},{0,9,202}, + {17,7,13},{0,8,101},{0,8,37},{0,9,170},{0,8,5},{0,8,133},{0,8,69}, + {0,9,234},{16,7,8},{0,8,93},{0,8,29},{0,9,154},{20,7,83},{0,8,125}, + {0,8,61},{0,9,218},{18,7,23},{0,8,109},{0,8,45},{0,9,186},{0,8,13}, + {0,8,141},{0,8,77},{0,9,250},{16,7,3},{0,8,83},{0,8,19},{21,8,195}, + {19,7,35},{0,8,115},{0,8,51},{0,9,198},{17,7,11},{0,8,99},{0,8,35}, + {0,9,166},{0,8,3},{0,8,131},{0,8,67},{0,9,230},{16,7,7},{0,8,91}, + {0,8,27},{0,9,150},{20,7,67},{0,8,123},{0,8,59},{0,9,214},{18,7,19}, + {0,8,107},{0,8,43},{0,9,182},{0,8,11},{0,8,139},{0,8,75},{0,9,246}, + {16,7,5},{0,8,87},{0,8,23},{64,8,0},{19,7,51},{0,8,119},{0,8,55}, + {0,9,206},{17,7,15},{0,8,103},{0,8,39},{0,9,174},{0,8,7},{0,8,135}, + {0,8,71},{0,9,238},{16,7,9},{0,8,95},{0,8,31},{0,9,158},{20,7,99}, + {0,8,127},{0,8,63},{0,9,222},{18,7,27},{0,8,111},{0,8,47},{0,9,190}, + {0,8,15},{0,8,143},{0,8,79},{0,9,254},{96,7,0},{0,8,80},{0,8,16}, + {20,8,115},{18,7,31},{0,8,112},{0,8,48},{0,9,193},{16,7,10},{0,8,96}, + {0,8,32},{0,9,161},{0,8,0},{0,8,128},{0,8,64},{0,9,225},{16,7,6}, + {0,8,88},{0,8,24},{0,9,145},{19,7,59},{0,8,120},{0,8,56},{0,9,209}, + {17,7,17},{0,8,104},{0,8,40},{0,9,177},{0,8,8},{0,8,136},{0,8,72}, + {0,9,241},{16,7,4},{0,8,84},{0,8,20},{21,8,227},{19,7,43},{0,8,116}, + {0,8,52},{0,9,201},{17,7,13},{0,8,100},{0,8,36},{0,9,169},{0,8,4}, + {0,8,132},{0,8,68},{0,9,233},{16,7,8},{0,8,92},{0,8,28},{0,9,153}, + {20,7,83},{0,8,124},{0,8,60},{0,9,217},{18,7,23},{0,8,108},{0,8,44}, + {0,9,185},{0,8,12},{0,8,140},{0,8,76},{0,9,249},{16,7,3},{0,8,82}, + {0,8,18},{21,8,163},{19,7,35},{0,8,114},{0,8,50},{0,9,197},{17,7,11}, + {0,8,98},{0,8,34},{0,9,165},{0,8,2},{0,8,130},{0,8,66},{0,9,229}, + {16,7,7},{0,8,90},{0,8,26},{0,9,149},{20,7,67},{0,8,122},{0,8,58}, + {0,9,213},{18,7,19},{0,8,106},{0,8,42},{0,9,181},{0,8,10},{0,8,138}, + {0,8,74},{0,9,245},{16,7,5},{0,8,86},{0,8,22},{64,8,0},{19,7,51}, + {0,8,118},{0,8,54},{0,9,205},{17,7,15},{0,8,102},{0,8,38},{0,9,173}, + {0,8,6},{0,8,134},{0,8,70},{0,9,237},{16,7,9},{0,8,94},{0,8,30}, + {0,9,157},{20,7,99},{0,8,126},{0,8,62},{0,9,221},{18,7,27},{0,8,110}, + {0,8,46},{0,9,189},{0,8,14},{0,8,142},{0,8,78},{0,9,253},{96,7,0}, + {0,8,81},{0,8,17},{21,8,131},{18,7,31},{0,8,113},{0,8,49},{0,9,195}, + {16,7,10},{0,8,97},{0,8,33},{0,9,163},{0,8,1},{0,8,129},{0,8,65}, + {0,9,227},{16,7,6},{0,8,89},{0,8,25},{0,9,147},{19,7,59},{0,8,121}, + {0,8,57},{0,9,211},{17,7,17},{0,8,105},{0,8,41},{0,9,179},{0,8,9}, + {0,8,137},{0,8,73},{0,9,243},{16,7,4},{0,8,85},{0,8,21},{16,8,258}, + {19,7,43},{0,8,117},{0,8,53},{0,9,203},{17,7,13},{0,8,101},{0,8,37}, + {0,9,171},{0,8,5},{0,8,133},{0,8,69},{0,9,235},{16,7,8},{0,8,93}, + {0,8,29},{0,9,155},{20,7,83},{0,8,125},{0,8,61},{0,9,219},{18,7,23}, + {0,8,109},{0,8,45},{0,9,187},{0,8,13},{0,8,141},{0,8,77},{0,9,251}, + {16,7,3},{0,8,83},{0,8,19},{21,8,195},{19,7,35},{0,8,115},{0,8,51}, + {0,9,199},{17,7,11},{0,8,99},{0,8,35},{0,9,167},{0,8,3},{0,8,131}, + {0,8,67},{0,9,231},{16,7,7},{0,8,91},{0,8,27},{0,9,151},{20,7,67}, + {0,8,123},{0,8,59},{0,9,215},{18,7,19},{0,8,107},{0,8,43},{0,9,183}, + {0,8,11},{0,8,139},{0,8,75},{0,9,247},{16,7,5},{0,8,87},{0,8,23}, + {64,8,0},{19,7,51},{0,8,119},{0,8,55},{0,9,207},{17,7,15},{0,8,103}, + {0,8,39},{0,9,175},{0,8,7},{0,8,135},{0,8,71},{0,9,239},{16,7,9}, + {0,8,95},{0,8,31},{0,9,159},{20,7,99},{0,8,127},{0,8,63},{0,9,223}, + {18,7,27},{0,8,111},{0,8,47},{0,9,191},{0,8,15},{0,8,143},{0,8,79}, + {0,9,255} + }; + + static const code distfix[32] = { + {16,5,1},{23,5,257},{19,5,17},{27,5,4097},{17,5,5},{25,5,1025}, + {21,5,65},{29,5,16385},{16,5,3},{24,5,513},{20,5,33},{28,5,8193}, + {18,5,9},{26,5,2049},{22,5,129},{64,5,0},{16,5,2},{23,5,385}, + {19,5,25},{27,5,6145},{17,5,7},{25,5,1537},{21,5,97},{29,5,24577}, + {16,5,4},{24,5,769},{20,5,49},{28,5,12289},{18,5,13},{26,5,3073}, + {22,5,193},{64,5,0} + }; diff --git a/lib/os/windows/zlib-1.2.13/inflate.c b/lib/os/windows/zlib-1.2.13/inflate.c new file mode 100644 index 000000000000..8acbef44e993 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/inflate.c @@ -0,0 +1,1595 @@ +/* inflate.c -- zlib decompression + * Copyright (C) 1995-2022 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* + * Change history: + * + * 1.2.beta0 24 Nov 2002 + * - First version -- complete rewrite of inflate to simplify code, avoid + * creation of window when not needed, minimize use of window when it is + * needed, make inffast.c even faster, implement gzip decoding, and to + * improve code readability and style over the previous zlib inflate code + * + * 1.2.beta1 25 Nov 2002 + * - Use pointers for available input and output checking in inffast.c + * - Remove input and output counters in inffast.c + * - Change inffast.c entry and loop from avail_in >= 7 to >= 6 + * - Remove unnecessary second byte pull from length extra in inffast.c + * - Unroll direct copy to three copies per loop in inffast.c + * + * 1.2.beta2 4 Dec 2002 + * - Change external routine names to reduce potential conflicts + * - Correct filename to inffixed.h for fixed tables in inflate.c + * - Make hbuf[] unsigned char to match parameter type in inflate.c + * - Change strm->next_out[-state->offset] to *(strm->next_out - state->offset) + * to avoid negation problem on Alphas (64 bit) in inflate.c + * + * 1.2.beta3 22 Dec 2002 + * - Add comments on state->bits assertion in inffast.c + * - Add comments on op field in inftrees.h + * - Fix bug in reuse of allocated window after inflateReset() + * - Remove bit fields--back to byte structure for speed + * - Remove distance extra == 0 check in inflate_fast()--only helps for lengths + * - Change post-increments to pre-increments in inflate_fast(), PPC biased? + * - Add compile time option, POSTINC, to use post-increments instead (Intel?) + * - Make MATCH copy in inflate() much faster for when inflate_fast() not used + * - Use local copies of stream next and avail values, as well as local bit + * buffer and bit count in inflate()--for speed when inflate_fast() not used + * + * 1.2.beta4 1 Jan 2003 + * - Split ptr - 257 statements in inflate_table() to avoid compiler warnings + * - Move a comment on output buffer sizes from inffast.c to inflate.c + * - Add comments in inffast.c to introduce the inflate_fast() routine + * - Rearrange window copies in inflate_fast() for speed and simplification + * - Unroll last copy for window match in inflate_fast() + * - Use local copies of window variables in inflate_fast() for speed + * - Pull out common wnext == 0 case for speed in inflate_fast() + * - Make op and len in inflate_fast() unsigned for consistency + * - Add FAR to lcode and dcode declarations in inflate_fast() + * - Simplified bad distance check in inflate_fast() + * - Added inflateBackInit(), inflateBack(), and inflateBackEnd() in new + * source file infback.c to provide a call-back interface to inflate for + * programs like gzip and unzip -- uses window as output buffer to avoid + * window copying + * + * 1.2.beta5 1 Jan 2003 + * - Improved inflateBack() interface to allow the caller to provide initial + * input in strm. + * - Fixed stored blocks bug in inflateBack() + * + * 1.2.beta6 4 Jan 2003 + * - Added comments in inffast.c on effectiveness of POSTINC + * - Typecasting all around to reduce compiler warnings + * - Changed loops from while (1) or do {} while (1) to for (;;), again to + * make compilers happy + * - Changed type of window in inflateBackInit() to unsigned char * + * + * 1.2.beta7 27 Jan 2003 + * - Changed many types to unsigned or unsigned short to avoid warnings + * - Added inflateCopy() function + * + * 1.2.0 9 Mar 2003 + * - Changed inflateBack() interface to provide separate opaque descriptors + * for the in() and out() functions + * - Changed inflateBack() argument and in_func typedef to swap the length + * and buffer address return values for the input function + * - Check next_in and next_out for Z_NULL on entry to inflate() + * + * The history for versions after 1.2.0 are in ChangeLog in zlib distribution. + */ + +#include "zutil.h" +#include "inftrees.h" +#include "inflate.h" +#include "inffast.h" + +#ifdef MAKEFIXED +# ifndef BUILDFIXED +# define BUILDFIXED +# endif +#endif + +/* function prototypes */ +local int inflateStateCheck OF((z_streamp strm)); +local void fixedtables OF((struct inflate_state FAR *state)); +local int updatewindow OF((z_streamp strm, const unsigned char FAR *end, + unsigned copy)); +#ifdef BUILDFIXED + void makefixed OF((void)); +#endif +local unsigned syncsearch OF((unsigned FAR *have, const unsigned char FAR *buf, + unsigned len)); + +local int inflateStateCheck(strm) +z_streamp strm; +{ + struct inflate_state FAR *state; + if (strm == Z_NULL || + strm->zalloc == (alloc_func)0 || strm->zfree == (free_func)0) + return 1; + state = (struct inflate_state FAR *)strm->state; + if (state == Z_NULL || state->strm != strm || + state->mode < HEAD || state->mode > SYNC) + return 1; + return 0; +} + +int ZEXPORT inflateResetKeep(strm) +z_streamp strm; +{ + struct inflate_state FAR *state; + + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + strm->total_in = strm->total_out = state->total = 0; + strm->msg = Z_NULL; + if (state->wrap) /* to support ill-conceived Java test suite */ + strm->adler = state->wrap & 1; + state->mode = HEAD; + state->last = 0; + state->havedict = 0; + state->flags = -1; + state->dmax = 32768U; + state->head = Z_NULL; + state->hold = 0; + state->bits = 0; + state->lencode = state->distcode = state->next = state->codes; + state->sane = 1; + state->back = -1; + Tracev((stderr, "inflate: reset\n")); + return Z_OK; +} + +int ZEXPORT inflateReset(strm) +z_streamp strm; +{ + struct inflate_state FAR *state; + + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + state->wsize = 0; + state->whave = 0; + state->wnext = 0; + return inflateResetKeep(strm); +} + +int ZEXPORT inflateReset2(strm, windowBits) +z_streamp strm; +int windowBits; +{ + int wrap; + struct inflate_state FAR *state; + + /* get the state */ + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + + /* extract wrap request from windowBits parameter */ + if (windowBits < 0) { + if (windowBits < -15) + return Z_STREAM_ERROR; + wrap = 0; + windowBits = -windowBits; + } + else { + wrap = (windowBits >> 4) + 5; +#ifdef GUNZIP + if (windowBits < 48) + windowBits &= 15; +#endif + } + + /* set number of window bits, free window if different */ + if (windowBits && (windowBits < 8 || windowBits > 15)) + return Z_STREAM_ERROR; + if (state->window != Z_NULL && state->wbits != (unsigned)windowBits) { + ZFREE(strm, state->window); + state->window = Z_NULL; + } + + /* update state and reset the rest of it */ + state->wrap = wrap; + state->wbits = (unsigned)windowBits; + return inflateReset(strm); +} + +int ZEXPORT inflateInit2_(strm, windowBits, version, stream_size) +z_streamp strm; +int windowBits; +const char *version; +int stream_size; +{ + int ret; + struct inflate_state FAR *state; + + if (version == Z_NULL || version[0] != ZLIB_VERSION[0] || + stream_size != (int)(sizeof(z_stream))) + return Z_VERSION_ERROR; + if (strm == Z_NULL) return Z_STREAM_ERROR; + strm->msg = Z_NULL; /* in case we return an error */ + if (strm->zalloc == (alloc_func)0) { +#ifdef Z_SOLO + return Z_STREAM_ERROR; +#else + strm->zalloc = zcalloc; + strm->opaque = (voidpf)0; +#endif + } + if (strm->zfree == (free_func)0) +#ifdef Z_SOLO + return Z_STREAM_ERROR; +#else + strm->zfree = zcfree; +#endif + state = (struct inflate_state FAR *) + ZALLOC(strm, 1, sizeof(struct inflate_state)); + if (state == Z_NULL) return Z_MEM_ERROR; + Tracev((stderr, "inflate: allocated\n")); + strm->state = (struct internal_state FAR *)state; + state->strm = strm; + state->window = Z_NULL; + state->mode = HEAD; /* to pass state test in inflateReset2() */ + ret = inflateReset2(strm, windowBits); + if (ret != Z_OK) { + ZFREE(strm, state); + strm->state = Z_NULL; + } + return ret; +} + +int ZEXPORT inflateInit_(strm, version, stream_size) +z_streamp strm; +const char *version; +int stream_size; +{ + return inflateInit2_(strm, DEF_WBITS, version, stream_size); +} + +int ZEXPORT inflatePrime(strm, bits, value) +z_streamp strm; +int bits; +int value; +{ + struct inflate_state FAR *state; + + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + if (bits < 0) { + state->hold = 0; + state->bits = 0; + return Z_OK; + } + if (bits > 16 || state->bits + (uInt)bits > 32) return Z_STREAM_ERROR; + value &= (1L << bits) - 1; + state->hold += (unsigned)value << state->bits; + state->bits += (uInt)bits; + return Z_OK; +} + +/* + Return state with length and distance decoding tables and index sizes set to + fixed code decoding. Normally this returns fixed tables from inffixed.h. + If BUILDFIXED is defined, then instead this routine builds the tables the + first time it's called, and returns those tables the first time and + thereafter. This reduces the size of the code by about 2K bytes, in + exchange for a little execution time. However, BUILDFIXED should not be + used for threaded applications, since the rewriting of the tables and virgin + may not be thread-safe. + */ +local void fixedtables(state) +struct inflate_state FAR *state; +{ +#ifdef BUILDFIXED + static int virgin = 1; + static code *lenfix, *distfix; + static code fixed[544]; + + /* build fixed huffman tables if first call (may not be thread safe) */ + if (virgin) { + unsigned sym, bits; + static code *next; + + /* literal/length table */ + sym = 0; + while (sym < 144) state->lens[sym++] = 8; + while (sym < 256) state->lens[sym++] = 9; + while (sym < 280) state->lens[sym++] = 7; + while (sym < 288) state->lens[sym++] = 8; + next = fixed; + lenfix = next; + bits = 9; + inflate_table(LENS, state->lens, 288, &(next), &(bits), state->work); + + /* distance table */ + sym = 0; + while (sym < 32) state->lens[sym++] = 5; + distfix = next; + bits = 5; + inflate_table(DISTS, state->lens, 32, &(next), &(bits), state->work); + + /* do this just once */ + virgin = 0; + } +#else /* !BUILDFIXED */ +# include "inffixed.h" +#endif /* BUILDFIXED */ + state->lencode = lenfix; + state->lenbits = 9; + state->distcode = distfix; + state->distbits = 5; +} + +#ifdef MAKEFIXED +#include + +/* + Write out the inffixed.h that is #include'd above. Defining MAKEFIXED also + defines BUILDFIXED, so the tables are built on the fly. makefixed() writes + those tables to stdout, which would be piped to inffixed.h. A small program + can simply call makefixed to do this: + + void makefixed(void); + + int main(void) + { + makefixed(); + return 0; + } + + Then that can be linked with zlib built with MAKEFIXED defined and run: + + a.out > inffixed.h + */ +void makefixed() +{ + unsigned low, size; + struct inflate_state state; + + fixedtables(&state); + puts(" /* inffixed.h -- table for decoding fixed codes"); + puts(" * Generated automatically by makefixed()."); + puts(" */"); + puts(""); + puts(" /* WARNING: this file should *not* be used by applications."); + puts(" It is part of the implementation of this library and is"); + puts(" subject to change. Applications should only use zlib.h."); + puts(" */"); + puts(""); + size = 1U << 9; + printf(" static const code lenfix[%u] = {", size); + low = 0; + for (;;) { + if ((low % 7) == 0) printf("\n "); + printf("{%u,%u,%d}", (low & 127) == 99 ? 64 : state.lencode[low].op, + state.lencode[low].bits, state.lencode[low].val); + if (++low == size) break; + putchar(','); + } + puts("\n };"); + size = 1U << 5; + printf("\n static const code distfix[%u] = {", size); + low = 0; + for (;;) { + if ((low % 6) == 0) printf("\n "); + printf("{%u,%u,%d}", state.distcode[low].op, state.distcode[low].bits, + state.distcode[low].val); + if (++low == size) break; + putchar(','); + } + puts("\n };"); +} +#endif /* MAKEFIXED */ + +/* + Update the window with the last wsize (normally 32K) bytes written before + returning. If window does not exist yet, create it. This is only called + when a window is already in use, or when output has been written during this + inflate call, but the end of the deflate stream has not been reached yet. + It is also called to create a window for dictionary data when a dictionary + is loaded. + + Providing output buffers larger than 32K to inflate() should provide a speed + advantage, since only the last 32K of output is copied to the sliding window + upon return from inflate(), and since all distances after the first 32K of + output will fall in the output data, making match copies simpler and faster. + The advantage may be dependent on the size of the processor's data caches. + */ +local int updatewindow(strm, end, copy) +z_streamp strm; +const Bytef *end; +unsigned copy; +{ + struct inflate_state FAR *state; + unsigned dist; + + state = (struct inflate_state FAR *)strm->state; + + /* if it hasn't been done already, allocate space for the window */ + if (state->window == Z_NULL) { + state->window = (unsigned char FAR *) + ZALLOC(strm, 1U << state->wbits, + sizeof(unsigned char)); + if (state->window == Z_NULL) return 1; + } + + /* if window not in use yet, initialize */ + if (state->wsize == 0) { + state->wsize = 1U << state->wbits; + state->wnext = 0; + state->whave = 0; + } + + /* copy state->wsize or less output bytes into the circular window */ + if (copy >= state->wsize) { + zmemcpy(state->window, end - state->wsize, state->wsize); + state->wnext = 0; + state->whave = state->wsize; + } + else { + dist = state->wsize - state->wnext; + if (dist > copy) dist = copy; + zmemcpy(state->window + state->wnext, end - copy, dist); + copy -= dist; + if (copy) { + zmemcpy(state->window, end - copy, copy); + state->wnext = copy; + state->whave = state->wsize; + } + else { + state->wnext += dist; + if (state->wnext == state->wsize) state->wnext = 0; + if (state->whave < state->wsize) state->whave += dist; + } + } + return 0; +} + +/* Macros for inflate(): */ + +/* check function to use adler32() for zlib or crc32() for gzip */ +#ifdef GUNZIP +# define UPDATE_CHECK(check, buf, len) \ + (state->flags ? crc32(check, buf, len) : adler32(check, buf, len)) +#else +# define UPDATE_CHECK(check, buf, len) adler32(check, buf, len) +#endif + +/* check macros for header crc */ +#ifdef GUNZIP +# define CRC2(check, word) \ + do { \ + hbuf[0] = (unsigned char)(word); \ + hbuf[1] = (unsigned char)((word) >> 8); \ + check = crc32(check, hbuf, 2); \ + } while (0) + +# define CRC4(check, word) \ + do { \ + hbuf[0] = (unsigned char)(word); \ + hbuf[1] = (unsigned char)((word) >> 8); \ + hbuf[2] = (unsigned char)((word) >> 16); \ + hbuf[3] = (unsigned char)((word) >> 24); \ + check = crc32(check, hbuf, 4); \ + } while (0) +#endif + +/* Load registers with state in inflate() for speed */ +#define LOAD() \ + do { \ + put = strm->next_out; \ + left = strm->avail_out; \ + next = strm->next_in; \ + have = strm->avail_in; \ + hold = state->hold; \ + bits = state->bits; \ + } while (0) + +/* Restore state from registers in inflate() */ +#define RESTORE() \ + do { \ + strm->next_out = put; \ + strm->avail_out = left; \ + strm->next_in = next; \ + strm->avail_in = have; \ + state->hold = hold; \ + state->bits = bits; \ + } while (0) + +/* Clear the input bit accumulator */ +#define INITBITS() \ + do { \ + hold = 0; \ + bits = 0; \ + } while (0) + +/* Get a byte of input into the bit accumulator, or return from inflate() + if there is no input available. */ +#define PULLBYTE() \ + do { \ + if (have == 0) goto inf_leave; \ + have--; \ + hold += (unsigned long)(*next++) << bits; \ + bits += 8; \ + } while (0) + +/* Assure that there are at least n bits in the bit accumulator. If there is + not enough available input to do that, then return from inflate(). */ +#define NEEDBITS(n) \ + do { \ + while (bits < (unsigned)(n)) \ + PULLBYTE(); \ + } while (0) + +/* Return the low n bits of the bit accumulator (n < 16) */ +#define BITS(n) \ + ((unsigned)hold & ((1U << (n)) - 1)) + +/* Remove n bits from the bit accumulator */ +#define DROPBITS(n) \ + do { \ + hold >>= (n); \ + bits -= (unsigned)(n); \ + } while (0) + +/* Remove zero to seven bits as needed to go to a byte boundary */ +#define BYTEBITS() \ + do { \ + hold >>= bits & 7; \ + bits -= bits & 7; \ + } while (0) + +/* + inflate() uses a state machine to process as much input data and generate as + much output data as possible before returning. The state machine is + structured roughly as follows: + + for (;;) switch (state) { + ... + case STATEn: + if (not enough input data or output space to make progress) + return; + ... make progress ... + state = STATEm; + break; + ... + } + + so when inflate() is called again, the same case is attempted again, and + if the appropriate resources are provided, the machine proceeds to the + next state. The NEEDBITS() macro is usually the way the state evaluates + whether it can proceed or should return. NEEDBITS() does the return if + the requested bits are not available. The typical use of the BITS macros + is: + + NEEDBITS(n); + ... do something with BITS(n) ... + DROPBITS(n); + + where NEEDBITS(n) either returns from inflate() if there isn't enough + input left to load n bits into the accumulator, or it continues. BITS(n) + gives the low n bits in the accumulator. When done, DROPBITS(n) drops + the low n bits off the accumulator. INITBITS() clears the accumulator + and sets the number of available bits to zero. BYTEBITS() discards just + enough bits to put the accumulator on a byte boundary. After BYTEBITS() + and a NEEDBITS(8), then BITS(8) would return the next byte in the stream. + + NEEDBITS(n) uses PULLBYTE() to get an available byte of input, or to return + if there is no input available. The decoding of variable length codes uses + PULLBYTE() directly in order to pull just enough bytes to decode the next + code, and no more. + + Some states loop until they get enough input, making sure that enough + state information is maintained to continue the loop where it left off + if NEEDBITS() returns in the loop. For example, want, need, and keep + would all have to actually be part of the saved state in case NEEDBITS() + returns: + + case STATEw: + while (want < need) { + NEEDBITS(n); + keep[want++] = BITS(n); + DROPBITS(n); + } + state = STATEx; + case STATEx: + + As shown above, if the next state is also the next case, then the break + is omitted. + + A state may also return if there is not enough output space available to + complete that state. Those states are copying stored data, writing a + literal byte, and copying a matching string. + + When returning, a "goto inf_leave" is used to update the total counters, + update the check value, and determine whether any progress has been made + during that inflate() call in order to return the proper return code. + Progress is defined as a change in either strm->avail_in or strm->avail_out. + When there is a window, goto inf_leave will update the window with the last + output written. If a goto inf_leave occurs in the middle of decompression + and there is no window currently, goto inf_leave will create one and copy + output to the window for the next call of inflate(). + + In this implementation, the flush parameter of inflate() only affects the + return code (per zlib.h). inflate() always writes as much as possible to + strm->next_out, given the space available and the provided input--the effect + documented in zlib.h of Z_SYNC_FLUSH. Furthermore, inflate() always defers + the allocation of and copying into a sliding window until necessary, which + provides the effect documented in zlib.h for Z_FINISH when the entire input + stream available. So the only thing the flush parameter actually does is: + when flush is set to Z_FINISH, inflate() cannot return Z_OK. Instead it + will return Z_BUF_ERROR if it has not reached the end of the stream. + */ + +int ZEXPORT inflate(strm, flush) +z_streamp strm; +int flush; +{ + struct inflate_state FAR *state; + z_const unsigned char FAR *next; /* next input */ + unsigned char FAR *put; /* next output */ + unsigned have, left; /* available input and output */ + unsigned long hold; /* bit buffer */ + unsigned bits; /* bits in bit buffer */ + unsigned in, out; /* save starting available input and output */ + unsigned copy; /* number of stored or match bytes to copy */ + unsigned char FAR *from; /* where to copy match bytes from */ + code here; /* current decoding table entry */ + code last; /* parent table entry */ + unsigned len; /* length to copy for repeats, bits to drop */ + int ret; /* return code */ +#ifdef GUNZIP + unsigned char hbuf[4]; /* buffer for gzip header crc calculation */ +#endif + static const unsigned short order[19] = /* permutation of code lengths */ + {16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; + + if (inflateStateCheck(strm) || strm->next_out == Z_NULL || + (strm->next_in == Z_NULL && strm->avail_in != 0)) + return Z_STREAM_ERROR; + + state = (struct inflate_state FAR *)strm->state; + if (state->mode == TYPE) state->mode = TYPEDO; /* skip check */ + LOAD(); + in = have; + out = left; + ret = Z_OK; + for (;;) + switch (state->mode) { + case HEAD: + if (state->wrap == 0) { + state->mode = TYPEDO; + break; + } + NEEDBITS(16); +#ifdef GUNZIP + if ((state->wrap & 2) && hold == 0x8b1f) { /* gzip header */ + if (state->wbits == 0) + state->wbits = 15; + state->check = crc32(0L, Z_NULL, 0); + CRC2(state->check, hold); + INITBITS(); + state->mode = FLAGS; + break; + } + if (state->head != Z_NULL) + state->head->done = -1; + if (!(state->wrap & 1) || /* check if zlib header allowed */ +#else + if ( +#endif + ((BITS(8) << 8) + (hold >> 8)) % 31) { + strm->msg = (char *)"incorrect header check"; + state->mode = BAD; + break; + } + if (BITS(4) != Z_DEFLATED) { + strm->msg = (char *)"unknown compression method"; + state->mode = BAD; + break; + } + DROPBITS(4); + len = BITS(4) + 8; + if (state->wbits == 0) + state->wbits = len; + if (len > 15 || len > state->wbits) { + strm->msg = (char *)"invalid window size"; + state->mode = BAD; + break; + } + state->dmax = 1U << len; + state->flags = 0; /* indicate zlib header */ + Tracev((stderr, "inflate: zlib header ok\n")); + strm->adler = state->check = adler32(0L, Z_NULL, 0); + state->mode = hold & 0x200 ? DICTID : TYPE; + INITBITS(); + break; +#ifdef GUNZIP + case FLAGS: + NEEDBITS(16); + state->flags = (int)(hold); + if ((state->flags & 0xff) != Z_DEFLATED) { + strm->msg = (char *)"unknown compression method"; + state->mode = BAD; + break; + } + if (state->flags & 0xe000) { + strm->msg = (char *)"unknown header flags set"; + state->mode = BAD; + break; + } + if (state->head != Z_NULL) + state->head->text = (int)((hold >> 8) & 1); + if ((state->flags & 0x0200) && (state->wrap & 4)) + CRC2(state->check, hold); + INITBITS(); + state->mode = TIME; + /* fallthrough */ + case TIME: + NEEDBITS(32); + if (state->head != Z_NULL) + state->head->time = hold; + if ((state->flags & 0x0200) && (state->wrap & 4)) + CRC4(state->check, hold); + INITBITS(); + state->mode = OS; + /* fallthrough */ + case OS: + NEEDBITS(16); + if (state->head != Z_NULL) { + state->head->xflags = (int)(hold & 0xff); + state->head->os = (int)(hold >> 8); + } + if ((state->flags & 0x0200) && (state->wrap & 4)) + CRC2(state->check, hold); + INITBITS(); + state->mode = EXLEN; + /* fallthrough */ + case EXLEN: + if (state->flags & 0x0400) { + NEEDBITS(16); + state->length = (unsigned)(hold); + if (state->head != Z_NULL) + state->head->extra_len = (unsigned)hold; + if ((state->flags & 0x0200) && (state->wrap & 4)) + CRC2(state->check, hold); + INITBITS(); + } + else if (state->head != Z_NULL) + state->head->extra = Z_NULL; + state->mode = EXTRA; + /* fallthrough */ + case EXTRA: + if (state->flags & 0x0400) { + copy = state->length; + if (copy > have) copy = have; + if (copy) { + if (state->head != Z_NULL && + state->head->extra != Z_NULL && + (len = state->head->extra_len - state->length) < + state->head->extra_max) { + zmemcpy(state->head->extra + len, next, + len + copy > state->head->extra_max ? + state->head->extra_max - len : copy); + } + if ((state->flags & 0x0200) && (state->wrap & 4)) + state->check = crc32(state->check, next, copy); + have -= copy; + next += copy; + state->length -= copy; + } + if (state->length) goto inf_leave; + } + state->length = 0; + state->mode = NAME; + /* fallthrough */ + case NAME: + if (state->flags & 0x0800) { + if (have == 0) goto inf_leave; + copy = 0; + do { + len = (unsigned)(next[copy++]); + if (state->head != Z_NULL && + state->head->name != Z_NULL && + state->length < state->head->name_max) + state->head->name[state->length++] = (Bytef)len; + } while (len && copy < have); + if ((state->flags & 0x0200) && (state->wrap & 4)) + state->check = crc32(state->check, next, copy); + have -= copy; + next += copy; + if (len) goto inf_leave; + } + else if (state->head != Z_NULL) + state->head->name = Z_NULL; + state->length = 0; + state->mode = COMMENT; + /* fallthrough */ + case COMMENT: + if (state->flags & 0x1000) { + if (have == 0) goto inf_leave; + copy = 0; + do { + len = (unsigned)(next[copy++]); + if (state->head != Z_NULL && + state->head->comment != Z_NULL && + state->length < state->head->comm_max) + state->head->comment[state->length++] = (Bytef)len; + } while (len && copy < have); + if ((state->flags & 0x0200) && (state->wrap & 4)) + state->check = crc32(state->check, next, copy); + have -= copy; + next += copy; + if (len) goto inf_leave; + } + else if (state->head != Z_NULL) + state->head->comment = Z_NULL; + state->mode = HCRC; + /* fallthrough */ + case HCRC: + if (state->flags & 0x0200) { + NEEDBITS(16); + if ((state->wrap & 4) && hold != (state->check & 0xffff)) { + strm->msg = (char *)"header crc mismatch"; + state->mode = BAD; + break; + } + INITBITS(); + } + if (state->head != Z_NULL) { + state->head->hcrc = (int)((state->flags >> 9) & 1); + state->head->done = 1; + } + strm->adler = state->check = crc32(0L, Z_NULL, 0); + state->mode = TYPE; + break; +#endif + case DICTID: + NEEDBITS(32); + strm->adler = state->check = ZSWAP32(hold); + INITBITS(); + state->mode = DICT; + /* fallthrough */ + case DICT: + if (state->havedict == 0) { + RESTORE(); + return Z_NEED_DICT; + } + strm->adler = state->check = adler32(0L, Z_NULL, 0); + state->mode = TYPE; + /* fallthrough */ + case TYPE: + if (flush == Z_BLOCK || flush == Z_TREES) goto inf_leave; + /* fallthrough */ + case TYPEDO: + if (state->last) { + BYTEBITS(); + state->mode = CHECK; + break; + } + NEEDBITS(3); + state->last = BITS(1); + DROPBITS(1); + switch (BITS(2)) { + case 0: /* stored block */ + Tracev((stderr, "inflate: stored block%s\n", + state->last ? " (last)" : "")); + state->mode = STORED; + break; + case 1: /* fixed block */ + fixedtables(state); + Tracev((stderr, "inflate: fixed codes block%s\n", + state->last ? " (last)" : "")); + state->mode = LEN_; /* decode codes */ + if (flush == Z_TREES) { + DROPBITS(2); + goto inf_leave; + } + break; + case 2: /* dynamic block */ + Tracev((stderr, "inflate: dynamic codes block%s\n", + state->last ? " (last)" : "")); + state->mode = TABLE; + break; + case 3: + strm->msg = (char *)"invalid block type"; + state->mode = BAD; + } + DROPBITS(2); + break; + case STORED: + BYTEBITS(); /* go to byte boundary */ + NEEDBITS(32); + if ((hold & 0xffff) != ((hold >> 16) ^ 0xffff)) { + strm->msg = (char *)"invalid stored block lengths"; + state->mode = BAD; + break; + } + state->length = (unsigned)hold & 0xffff; + Tracev((stderr, "inflate: stored length %u\n", + state->length)); + INITBITS(); + state->mode = COPY_; + if (flush == Z_TREES) goto inf_leave; + /* fallthrough */ + case COPY_: + state->mode = COPY; + /* fallthrough */ + case COPY: + copy = state->length; + if (copy) { + if (copy > have) copy = have; + if (copy > left) copy = left; + if (copy == 0) goto inf_leave; + zmemcpy(put, next, copy); + have -= copy; + next += copy; + left -= copy; + put += copy; + state->length -= copy; + break; + } + Tracev((stderr, "inflate: stored end\n")); + state->mode = TYPE; + break; + case TABLE: + NEEDBITS(14); + state->nlen = BITS(5) + 257; + DROPBITS(5); + state->ndist = BITS(5) + 1; + DROPBITS(5); + state->ncode = BITS(4) + 4; + DROPBITS(4); +#ifndef PKZIP_BUG_WORKAROUND + if (state->nlen > 286 || state->ndist > 30) { + strm->msg = (char *)"too many length or distance symbols"; + state->mode = BAD; + break; + } +#endif + Tracev((stderr, "inflate: table sizes ok\n")); + state->have = 0; + state->mode = LENLENS; + /* fallthrough */ + case LENLENS: + while (state->have < state->ncode) { + NEEDBITS(3); + state->lens[order[state->have++]] = (unsigned short)BITS(3); + DROPBITS(3); + } + while (state->have < 19) + state->lens[order[state->have++]] = 0; + state->next = state->codes; + state->lencode = (const code FAR *)(state->next); + state->lenbits = 7; + ret = inflate_table(CODES, state->lens, 19, &(state->next), + &(state->lenbits), state->work); + if (ret) { + strm->msg = (char *)"invalid code lengths set"; + state->mode = BAD; + break; + } + Tracev((stderr, "inflate: code lengths ok\n")); + state->have = 0; + state->mode = CODELENS; + /* fallthrough */ + case CODELENS: + while (state->have < state->nlen + state->ndist) { + for (;;) { + here = state->lencode[BITS(state->lenbits)]; + if ((unsigned)(here.bits) <= bits) break; + PULLBYTE(); + } + if (here.val < 16) { + DROPBITS(here.bits); + state->lens[state->have++] = here.val; + } + else { + if (here.val == 16) { + NEEDBITS(here.bits + 2); + DROPBITS(here.bits); + if (state->have == 0) { + strm->msg = (char *)"invalid bit length repeat"; + state->mode = BAD; + break; + } + len = state->lens[state->have - 1]; + copy = 3 + BITS(2); + DROPBITS(2); + } + else if (here.val == 17) { + NEEDBITS(here.bits + 3); + DROPBITS(here.bits); + len = 0; + copy = 3 + BITS(3); + DROPBITS(3); + } + else { + NEEDBITS(here.bits + 7); + DROPBITS(here.bits); + len = 0; + copy = 11 + BITS(7); + DROPBITS(7); + } + if (state->have + copy > state->nlen + state->ndist) { + strm->msg = (char *)"invalid bit length repeat"; + state->mode = BAD; + break; + } + while (copy--) + state->lens[state->have++] = (unsigned short)len; + } + } + + /* handle error breaks in while */ + if (state->mode == BAD) break; + + /* check for end-of-block code (better have one) */ + if (state->lens[256] == 0) { + strm->msg = (char *)"invalid code -- missing end-of-block"; + state->mode = BAD; + break; + } + + /* build code tables -- note: do not change the lenbits or distbits + values here (9 and 6) without reading the comments in inftrees.h + concerning the ENOUGH constants, which depend on those values */ + state->next = state->codes; + state->lencode = (const code FAR *)(state->next); + state->lenbits = 9; + ret = inflate_table(LENS, state->lens, state->nlen, &(state->next), + &(state->lenbits), state->work); + if (ret) { + strm->msg = (char *)"invalid literal/lengths set"; + state->mode = BAD; + break; + } + state->distcode = (const code FAR *)(state->next); + state->distbits = 6; + ret = inflate_table(DISTS, state->lens + state->nlen, state->ndist, + &(state->next), &(state->distbits), state->work); + if (ret) { + strm->msg = (char *)"invalid distances set"; + state->mode = BAD; + break; + } + Tracev((stderr, "inflate: codes ok\n")); + state->mode = LEN_; + if (flush == Z_TREES) goto inf_leave; + /* fallthrough */ + case LEN_: + state->mode = LEN; + /* fallthrough */ + case LEN: + if (have >= 6 && left >= 258) { + RESTORE(); + inflate_fast(strm, out); + LOAD(); + if (state->mode == TYPE) + state->back = -1; + break; + } + state->back = 0; + for (;;) { + here = state->lencode[BITS(state->lenbits)]; + if ((unsigned)(here.bits) <= bits) break; + PULLBYTE(); + } + if (here.op && (here.op & 0xf0) == 0) { + last = here; + for (;;) { + here = state->lencode[last.val + + (BITS(last.bits + last.op) >> last.bits)]; + if ((unsigned)(last.bits + here.bits) <= bits) break; + PULLBYTE(); + } + DROPBITS(last.bits); + state->back += last.bits; + } + DROPBITS(here.bits); + state->back += here.bits; + state->length = (unsigned)here.val; + if ((int)(here.op) == 0) { + Tracevv((stderr, here.val >= 0x20 && here.val < 0x7f ? + "inflate: literal '%c'\n" : + "inflate: literal 0x%02x\n", here.val)); + state->mode = LIT; + break; + } + if (here.op & 32) { + Tracevv((stderr, "inflate: end of block\n")); + state->back = -1; + state->mode = TYPE; + break; + } + if (here.op & 64) { + strm->msg = (char *)"invalid literal/length code"; + state->mode = BAD; + break; + } + state->extra = (unsigned)(here.op) & 15; + state->mode = LENEXT; + /* fallthrough */ + case LENEXT: + if (state->extra) { + NEEDBITS(state->extra); + state->length += BITS(state->extra); + DROPBITS(state->extra); + state->back += state->extra; + } + Tracevv((stderr, "inflate: length %u\n", state->length)); + state->was = state->length; + state->mode = DIST; + /* fallthrough */ + case DIST: + for (;;) { + here = state->distcode[BITS(state->distbits)]; + if ((unsigned)(here.bits) <= bits) break; + PULLBYTE(); + } + if ((here.op & 0xf0) == 0) { + last = here; + for (;;) { + here = state->distcode[last.val + + (BITS(last.bits + last.op) >> last.bits)]; + if ((unsigned)(last.bits + here.bits) <= bits) break; + PULLBYTE(); + } + DROPBITS(last.bits); + state->back += last.bits; + } + DROPBITS(here.bits); + state->back += here.bits; + if (here.op & 64) { + strm->msg = (char *)"invalid distance code"; + state->mode = BAD; + break; + } + state->offset = (unsigned)here.val; + state->extra = (unsigned)(here.op) & 15; + state->mode = DISTEXT; + /* fallthrough */ + case DISTEXT: + if (state->extra) { + NEEDBITS(state->extra); + state->offset += BITS(state->extra); + DROPBITS(state->extra); + state->back += state->extra; + } +#ifdef INFLATE_STRICT + if (state->offset > state->dmax) { + strm->msg = (char *)"invalid distance too far back"; + state->mode = BAD; + break; + } +#endif + Tracevv((stderr, "inflate: distance %u\n", state->offset)); + state->mode = MATCH; + /* fallthrough */ + case MATCH: + if (left == 0) goto inf_leave; + copy = out - left; + if (state->offset > copy) { /* copy from window */ + copy = state->offset - copy; + if (copy > state->whave) { + if (state->sane) { + strm->msg = (char *)"invalid distance too far back"; + state->mode = BAD; + break; + } +#ifdef INFLATE_ALLOW_INVALID_DISTANCE_TOOFAR_ARRR + Trace((stderr, "inflate.c too far\n")); + copy -= state->whave; + if (copy > state->length) copy = state->length; + if (copy > left) copy = left; + left -= copy; + state->length -= copy; + do { + *put++ = 0; + } while (--copy); + if (state->length == 0) state->mode = LEN; + break; +#endif + } + if (copy > state->wnext) { + copy -= state->wnext; + from = state->window + (state->wsize - copy); + } + else + from = state->window + (state->wnext - copy); + if (copy > state->length) copy = state->length; + } + else { /* copy from output */ + from = put - state->offset; + copy = state->length; + } + if (copy > left) copy = left; + left -= copy; + state->length -= copy; + do { + *put++ = *from++; + } while (--copy); + if (state->length == 0) state->mode = LEN; + break; + case LIT: + if (left == 0) goto inf_leave; + *put++ = (unsigned char)(state->length); + left--; + state->mode = LEN; + break; + case CHECK: + if (state->wrap) { + NEEDBITS(32); + out -= left; + strm->total_out += out; + state->total += out; + if ((state->wrap & 4) && out) + strm->adler = state->check = + UPDATE_CHECK(state->check, put - out, out); + out = left; + if ((state->wrap & 4) && ( +#ifdef GUNZIP + state->flags ? hold : +#endif + ZSWAP32(hold)) != state->check) { + strm->msg = (char *)"incorrect data check"; + state->mode = BAD; + break; + } + INITBITS(); + Tracev((stderr, "inflate: check matches trailer\n")); + } +#ifdef GUNZIP + state->mode = LENGTH; + /* fallthrough */ + case LENGTH: + if (state->wrap && state->flags) { + NEEDBITS(32); + if ((state->wrap & 4) && hold != (state->total & 0xffffffff)) { + strm->msg = (char *)"incorrect length check"; + state->mode = BAD; + break; + } + INITBITS(); + Tracev((stderr, "inflate: length matches trailer\n")); + } +#endif + state->mode = DONE; + /* fallthrough */ + case DONE: + ret = Z_STREAM_END; + goto inf_leave; + case BAD: + ret = Z_DATA_ERROR; + goto inf_leave; + case MEM: + return Z_MEM_ERROR; + case SYNC: + /* fallthrough */ + default: + return Z_STREAM_ERROR; + } + + /* + Return from inflate(), updating the total counts and the check value. + If there was no progress during the inflate() call, return a buffer + error. Call updatewindow() to create and/or update the window state. + Note: a memory error from inflate() is non-recoverable. + */ + inf_leave: + RESTORE(); + if (state->wsize || (out != strm->avail_out && state->mode < BAD && + (state->mode < CHECK || flush != Z_FINISH))) + if (updatewindow(strm, strm->next_out, out - strm->avail_out)) { + state->mode = MEM; + return Z_MEM_ERROR; + } + in -= strm->avail_in; + out -= strm->avail_out; + strm->total_in += in; + strm->total_out += out; + state->total += out; + if ((state->wrap & 4) && out) + strm->adler = state->check = + UPDATE_CHECK(state->check, strm->next_out - out, out); + strm->data_type = (int)state->bits + (state->last ? 64 : 0) + + (state->mode == TYPE ? 128 : 0) + + (state->mode == LEN_ || state->mode == COPY_ ? 256 : 0); + if (((in == 0 && out == 0) || flush == Z_FINISH) && ret == Z_OK) + ret = Z_BUF_ERROR; + return ret; +} + +int ZEXPORT inflateEnd(strm) +z_streamp strm; +{ + struct inflate_state FAR *state; + if (inflateStateCheck(strm)) + return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + if (state->window != Z_NULL) ZFREE(strm, state->window); + ZFREE(strm, strm->state); + strm->state = Z_NULL; + Tracev((stderr, "inflate: end\n")); + return Z_OK; +} + +int ZEXPORT inflateGetDictionary(strm, dictionary, dictLength) +z_streamp strm; +Bytef *dictionary; +uInt *dictLength; +{ + struct inflate_state FAR *state; + + /* check state */ + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + + /* copy dictionary */ + if (state->whave && dictionary != Z_NULL) { + zmemcpy(dictionary, state->window + state->wnext, + state->whave - state->wnext); + zmemcpy(dictionary + state->whave - state->wnext, + state->window, state->wnext); + } + if (dictLength != Z_NULL) + *dictLength = state->whave; + return Z_OK; +} + +int ZEXPORT inflateSetDictionary(strm, dictionary, dictLength) +z_streamp strm; +const Bytef *dictionary; +uInt dictLength; +{ + struct inflate_state FAR *state; + unsigned long dictid; + int ret; + + /* check state */ + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + if (state->wrap != 0 && state->mode != DICT) + return Z_STREAM_ERROR; + + /* check for correct dictionary identifier */ + if (state->mode == DICT) { + dictid = adler32(0L, Z_NULL, 0); + dictid = adler32(dictid, dictionary, dictLength); + if (dictid != state->check) + return Z_DATA_ERROR; + } + + /* copy dictionary to window using updatewindow(), which will amend the + existing dictionary if appropriate */ + ret = updatewindow(strm, dictionary + dictLength, dictLength); + if (ret) { + state->mode = MEM; + return Z_MEM_ERROR; + } + state->havedict = 1; + Tracev((stderr, "inflate: dictionary set\n")); + return Z_OK; +} + +int ZEXPORT inflateGetHeader(strm, head) +z_streamp strm; +gz_headerp head; +{ + struct inflate_state FAR *state; + + /* check state */ + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + if ((state->wrap & 2) == 0) return Z_STREAM_ERROR; + + /* save header structure */ + state->head = head; + head->done = 0; + return Z_OK; +} + +/* + Search buf[0..len-1] for the pattern: 0, 0, 0xff, 0xff. Return when found + or when out of input. When called, *have is the number of pattern bytes + found in order so far, in 0..3. On return *have is updated to the new + state. If on return *have equals four, then the pattern was found and the + return value is how many bytes were read including the last byte of the + pattern. If *have is less than four, then the pattern has not been found + yet and the return value is len. In the latter case, syncsearch() can be + called again with more data and the *have state. *have is initialized to + zero for the first call. + */ +local unsigned syncsearch(have, buf, len) +unsigned FAR *have; +const unsigned char FAR *buf; +unsigned len; +{ + unsigned got; + unsigned next; + + got = *have; + next = 0; + while (next < len && got < 4) { + if ((int)(buf[next]) == (got < 2 ? 0 : 0xff)) + got++; + else if (buf[next]) + got = 0; + else + got = 4 - got; + next++; + } + *have = got; + return next; +} + +int ZEXPORT inflateSync(strm) +z_streamp strm; +{ + unsigned len; /* number of bytes to look at or looked at */ + int flags; /* temporary to save header status */ + unsigned long in, out; /* temporary to save total_in and total_out */ + unsigned char buf[4]; /* to restore bit buffer to byte string */ + struct inflate_state FAR *state; + + /* check parameters */ + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + if (strm->avail_in == 0 && state->bits < 8) return Z_BUF_ERROR; + + /* if first time, start search in bit buffer */ + if (state->mode != SYNC) { + state->mode = SYNC; + state->hold <<= state->bits & 7; + state->bits -= state->bits & 7; + len = 0; + while (state->bits >= 8) { + buf[len++] = (unsigned char)(state->hold); + state->hold >>= 8; + state->bits -= 8; + } + state->have = 0; + syncsearch(&(state->have), buf, len); + } + + /* search available input */ + len = syncsearch(&(state->have), strm->next_in, strm->avail_in); + strm->avail_in -= len; + strm->next_in += len; + strm->total_in += len; + + /* return no joy or set up to restart inflate() on a new block */ + if (state->have != 4) return Z_DATA_ERROR; + if (state->flags == -1) + state->wrap = 0; /* if no header yet, treat as raw */ + else + state->wrap &= ~4; /* no point in computing a check value now */ + flags = state->flags; + in = strm->total_in; out = strm->total_out; + inflateReset(strm); + strm->total_in = in; strm->total_out = out; + state->flags = flags; + state->mode = TYPE; + return Z_OK; +} + +/* + Returns true if inflate is currently at the end of a block generated by + Z_SYNC_FLUSH or Z_FULL_FLUSH. This function is used by one PPP + implementation to provide an additional safety check. PPP uses + Z_SYNC_FLUSH but removes the length bytes of the resulting empty stored + block. When decompressing, PPP checks that at the end of input packet, + inflate is waiting for these length bytes. + */ +int ZEXPORT inflateSyncPoint(strm) +z_streamp strm; +{ + struct inflate_state FAR *state; + + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + return state->mode == STORED && state->bits == 0; +} + +int ZEXPORT inflateCopy(dest, source) +z_streamp dest; +z_streamp source; +{ + struct inflate_state FAR *state; + struct inflate_state FAR *copy; + unsigned char FAR *window; + unsigned wsize; + + /* check input */ + if (inflateStateCheck(source) || dest == Z_NULL) + return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)source->state; + + /* allocate space */ + copy = (struct inflate_state FAR *) + ZALLOC(source, 1, sizeof(struct inflate_state)); + if (copy == Z_NULL) return Z_MEM_ERROR; + window = Z_NULL; + if (state->window != Z_NULL) { + window = (unsigned char FAR *) + ZALLOC(source, 1U << state->wbits, sizeof(unsigned char)); + if (window == Z_NULL) { + ZFREE(source, copy); + return Z_MEM_ERROR; + } + } + + /* copy state */ + zmemcpy((voidpf)dest, (voidpf)source, sizeof(z_stream)); + zmemcpy((voidpf)copy, (voidpf)state, sizeof(struct inflate_state)); + copy->strm = dest; + if (state->lencode >= state->codes && + state->lencode <= state->codes + ENOUGH - 1) { + copy->lencode = copy->codes + (state->lencode - state->codes); + copy->distcode = copy->codes + (state->distcode - state->codes); + } + copy->next = copy->codes + (state->next - state->codes); + if (window != Z_NULL) { + wsize = 1U << state->wbits; + zmemcpy(window, state->window, wsize); + } + copy->window = window; + dest->state = (struct internal_state FAR *)copy; + return Z_OK; +} + +int ZEXPORT inflateUndermine(strm, subvert) +z_streamp strm; +int subvert; +{ + struct inflate_state FAR *state; + + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; +#ifdef INFLATE_ALLOW_INVALID_DISTANCE_TOOFAR_ARRR + state->sane = !subvert; + return Z_OK; +#else + (void)subvert; + state->sane = 1; + return Z_DATA_ERROR; +#endif +} + +int ZEXPORT inflateValidate(strm, check) +z_streamp strm; +int check; +{ + struct inflate_state FAR *state; + + if (inflateStateCheck(strm)) return Z_STREAM_ERROR; + state = (struct inflate_state FAR *)strm->state; + if (check && state->wrap) + state->wrap |= 4; + else + state->wrap &= ~4; + return Z_OK; +} + +long ZEXPORT inflateMark(strm) +z_streamp strm; +{ + struct inflate_state FAR *state; + + if (inflateStateCheck(strm)) + return -(1L << 16); + state = (struct inflate_state FAR *)strm->state; + return (long)(((unsigned long)((long)state->back)) << 16) + + (state->mode == COPY ? state->length : + (state->mode == MATCH ? state->was - state->length : 0)); +} + +unsigned long ZEXPORT inflateCodesUsed(strm) +z_streamp strm; +{ + struct inflate_state FAR *state; + if (inflateStateCheck(strm)) return (unsigned long)-1; + state = (struct inflate_state FAR *)strm->state; + return (unsigned long)(state->next - state->codes); +} diff --git a/lib/os/windows/zlib-1.2.13/inflate.h b/lib/os/windows/zlib-1.2.13/inflate.h new file mode 100644 index 000000000000..f127b6b1fa5f --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/inflate.h @@ -0,0 +1,126 @@ +/* inflate.h -- internal inflate state definition + * Copyright (C) 1995-2019 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* WARNING: this file should *not* be used by applications. It is + part of the implementation of the compression library and is + subject to change. Applications should only use zlib.h. + */ + +/* define NO_GZIP when compiling if you want to disable gzip header and + trailer decoding by inflate(). NO_GZIP would be used to avoid linking in + the crc code when it is not needed. For shared libraries, gzip decoding + should be left enabled. */ +#ifndef NO_GZIP +# define GUNZIP +#endif + +/* Possible inflate modes between inflate() calls */ +typedef enum { + HEAD = 16180, /* i: waiting for magic header */ + FLAGS, /* i: waiting for method and flags (gzip) */ + TIME, /* i: waiting for modification time (gzip) */ + OS, /* i: waiting for extra flags and operating system (gzip) */ + EXLEN, /* i: waiting for extra length (gzip) */ + EXTRA, /* i: waiting for extra bytes (gzip) */ + NAME, /* i: waiting for end of file name (gzip) */ + COMMENT, /* i: waiting for end of comment (gzip) */ + HCRC, /* i: waiting for header crc (gzip) */ + DICTID, /* i: waiting for dictionary check value */ + DICT, /* waiting for inflateSetDictionary() call */ + TYPE, /* i: waiting for type bits, including last-flag bit */ + TYPEDO, /* i: same, but skip check to exit inflate on new block */ + STORED, /* i: waiting for stored size (length and complement) */ + COPY_, /* i/o: same as COPY below, but only first time in */ + COPY, /* i/o: waiting for input or output to copy stored block */ + TABLE, /* i: waiting for dynamic block table lengths */ + LENLENS, /* i: waiting for code length code lengths */ + CODELENS, /* i: waiting for length/lit and distance code lengths */ + LEN_, /* i: same as LEN below, but only first time in */ + LEN, /* i: waiting for length/lit/eob code */ + LENEXT, /* i: waiting for length extra bits */ + DIST, /* i: waiting for distance code */ + DISTEXT, /* i: waiting for distance extra bits */ + MATCH, /* o: waiting for output space to copy string */ + LIT, /* o: waiting for output space to write literal */ + CHECK, /* i: waiting for 32-bit check value */ + LENGTH, /* i: waiting for 32-bit length (gzip) */ + DONE, /* finished check, done -- remain here until reset */ + BAD, /* got a data error -- remain here until reset */ + MEM, /* got an inflate() memory error -- remain here until reset */ + SYNC /* looking for synchronization bytes to restart inflate() */ +} inflate_mode; + +/* + State transitions between above modes - + + (most modes can go to BAD or MEM on error -- not shown for clarity) + + Process header: + HEAD -> (gzip) or (zlib) or (raw) + (gzip) -> FLAGS -> TIME -> OS -> EXLEN -> EXTRA -> NAME -> COMMENT -> + HCRC -> TYPE + (zlib) -> DICTID or TYPE + DICTID -> DICT -> TYPE + (raw) -> TYPEDO + Read deflate blocks: + TYPE -> TYPEDO -> STORED or TABLE or LEN_ or CHECK + STORED -> COPY_ -> COPY -> TYPE + TABLE -> LENLENS -> CODELENS -> LEN_ + LEN_ -> LEN + Read deflate codes in fixed or dynamic block: + LEN -> LENEXT or LIT or TYPE + LENEXT -> DIST -> DISTEXT -> MATCH -> LEN + LIT -> LEN + Process trailer: + CHECK -> LENGTH -> DONE + */ + +/* State maintained between inflate() calls -- approximately 7K bytes, not + including the allocated sliding window, which is up to 32K bytes. */ +struct inflate_state { + z_streamp strm; /* pointer back to this zlib stream */ + inflate_mode mode; /* current inflate mode */ + int last; /* true if processing last block */ + int wrap; /* bit 0 true for zlib, bit 1 true for gzip, + bit 2 true to validate check value */ + int havedict; /* true if dictionary provided */ + int flags; /* gzip header method and flags, 0 if zlib, or + -1 if raw or no header yet */ + unsigned dmax; /* zlib header max distance (INFLATE_STRICT) */ + unsigned long check; /* protected copy of check value */ + unsigned long total; /* protected copy of output count */ + gz_headerp head; /* where to save gzip header information */ + /* sliding window */ + unsigned wbits; /* log base 2 of requested window size */ + unsigned wsize; /* window size or zero if not using window */ + unsigned whave; /* valid bytes in the window */ + unsigned wnext; /* window write index */ + unsigned char FAR *window; /* allocated sliding window, if needed */ + /* bit accumulator */ + unsigned long hold; /* input bit accumulator */ + unsigned bits; /* number of bits in "in" */ + /* for string and stored block copying */ + unsigned length; /* literal or length of data to copy */ + unsigned offset; /* distance back to copy string from */ + /* for table and code decoding */ + unsigned extra; /* extra bits needed */ + /* fixed and dynamic code tables */ + code const FAR *lencode; /* starting table for length/literal codes */ + code const FAR *distcode; /* starting table for distance codes */ + unsigned lenbits; /* index bits for lencode */ + unsigned distbits; /* index bits for distcode */ + /* dynamic table building */ + unsigned ncode; /* number of code length code lengths */ + unsigned nlen; /* number of length code lengths */ + unsigned ndist; /* number of distance code lengths */ + unsigned have; /* number of code lengths in lens[] */ + code FAR *next; /* next available space in codes[] */ + unsigned short lens[320]; /* temporary storage for code lengths */ + unsigned short work[288]; /* work area for code table building */ + code codes[ENOUGH]; /* space for code tables */ + int sane; /* if false, allow invalid distance too far */ + int back; /* bits back of last unprocessed length/lit */ + unsigned was; /* initial length of match */ +}; diff --git a/lib/os/windows/zlib-1.2.13/inftrees.c b/lib/os/windows/zlib-1.2.13/inftrees.c new file mode 100644 index 000000000000..57d2793bec93 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/inftrees.c @@ -0,0 +1,304 @@ +/* inftrees.c -- generate Huffman trees for efficient decoding + * Copyright (C) 1995-2022 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +#include "zutil.h" +#include "inftrees.h" + +#define MAXBITS 15 + +const char inflate_copyright[] = + " inflate 1.2.13 Copyright 1995-2022 Mark Adler "; +/* + If you use the zlib library in a product, an acknowledgment is welcome + in the documentation of your product. If for some reason you cannot + include such an acknowledgment, I would appreciate that you keep this + copyright string in the executable of your product. + */ + +/* + Build a set of tables to decode the provided canonical Huffman code. + The code lengths are lens[0..codes-1]. The result starts at *table, + whose indices are 0..2^bits-1. work is a writable array of at least + lens shorts, which is used as a work area. type is the type of code + to be generated, CODES, LENS, or DISTS. On return, zero is success, + -1 is an invalid code, and +1 means that ENOUGH isn't enough. table + on return points to the next available entry's address. bits is the + requested root table index bits, and on return it is the actual root + table index bits. It will differ if the request is greater than the + longest code or if it is less than the shortest code. + */ +int ZLIB_INTERNAL inflate_table(type, lens, codes, table, bits, work) +codetype type; +unsigned short FAR *lens; +unsigned codes; +code FAR * FAR *table; +unsigned FAR *bits; +unsigned short FAR *work; +{ + unsigned len; /* a code's length in bits */ + unsigned sym; /* index of code symbols */ + unsigned min, max; /* minimum and maximum code lengths */ + unsigned root; /* number of index bits for root table */ + unsigned curr; /* number of index bits for current table */ + unsigned drop; /* code bits to drop for sub-table */ + int left; /* number of prefix codes available */ + unsigned used; /* code entries in table used */ + unsigned huff; /* Huffman code */ + unsigned incr; /* for incrementing code, index */ + unsigned fill; /* index for replicating entries */ + unsigned low; /* low bits for current root entry */ + unsigned mask; /* mask for low root bits */ + code here; /* table entry for duplication */ + code FAR *next; /* next available space in table */ + const unsigned short FAR *base; /* base value table to use */ + const unsigned short FAR *extra; /* extra bits table to use */ + unsigned match; /* use base and extra for symbol >= match */ + unsigned short count[MAXBITS+1]; /* number of codes of each length */ + unsigned short offs[MAXBITS+1]; /* offsets in table for each length */ + static const unsigned short lbase[31] = { /* Length codes 257..285 base */ + 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, + 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0}; + static const unsigned short lext[31] = { /* Length codes 257..285 extra */ + 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 18, 18, 18, 18, + 19, 19, 19, 19, 20, 20, 20, 20, 21, 21, 21, 21, 16, 194, 65}; + static const unsigned short dbase[32] = { /* Distance codes 0..29 base */ + 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, + 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, + 8193, 12289, 16385, 24577, 0, 0}; + static const unsigned short dext[32] = { /* Distance codes 0..29 extra */ + 16, 16, 16, 16, 17, 17, 18, 18, 19, 19, 20, 20, 21, 21, 22, 22, + 23, 23, 24, 24, 25, 25, 26, 26, 27, 27, + 28, 28, 29, 29, 64, 64}; + + /* + Process a set of code lengths to create a canonical Huffman code. The + code lengths are lens[0..codes-1]. Each length corresponds to the + symbols 0..codes-1. The Huffman code is generated by first sorting the + symbols by length from short to long, and retaining the symbol order + for codes with equal lengths. Then the code starts with all zero bits + for the first code of the shortest length, and the codes are integer + increments for the same length, and zeros are appended as the length + increases. For the deflate format, these bits are stored backwards + from their more natural integer increment ordering, and so when the + decoding tables are built in the large loop below, the integer codes + are incremented backwards. + + This routine assumes, but does not check, that all of the entries in + lens[] are in the range 0..MAXBITS. The caller must assure this. + 1..MAXBITS is interpreted as that code length. zero means that that + symbol does not occur in this code. + + The codes are sorted by computing a count of codes for each length, + creating from that a table of starting indices for each length in the + sorted table, and then entering the symbols in order in the sorted + table. The sorted table is work[], with that space being provided by + the caller. + + The length counts are used for other purposes as well, i.e. finding + the minimum and maximum length codes, determining if there are any + codes at all, checking for a valid set of lengths, and looking ahead + at length counts to determine sub-table sizes when building the + decoding tables. + */ + + /* accumulate lengths for codes (assumes lens[] all in 0..MAXBITS) */ + for (len = 0; len <= MAXBITS; len++) + count[len] = 0; + for (sym = 0; sym < codes; sym++) + count[lens[sym]]++; + + /* bound code lengths, force root to be within code lengths */ + root = *bits; + for (max = MAXBITS; max >= 1; max--) + if (count[max] != 0) break; + if (root > max) root = max; + if (max == 0) { /* no symbols to code at all */ + here.op = (unsigned char)64; /* invalid code marker */ + here.bits = (unsigned char)1; + here.val = (unsigned short)0; + *(*table)++ = here; /* make a table to force an error */ + *(*table)++ = here; + *bits = 1; + return 0; /* no symbols, but wait for decoding to report error */ + } + for (min = 1; min < max; min++) + if (count[min] != 0) break; + if (root < min) root = min; + + /* check for an over-subscribed or incomplete set of lengths */ + left = 1; + for (len = 1; len <= MAXBITS; len++) { + left <<= 1; + left -= count[len]; + if (left < 0) return -1; /* over-subscribed */ + } + if (left > 0 && (type == CODES || max != 1)) + return -1; /* incomplete set */ + + /* generate offsets into symbol table for each length for sorting */ + offs[1] = 0; + for (len = 1; len < MAXBITS; len++) + offs[len + 1] = offs[len] + count[len]; + + /* sort symbols by length, by symbol order within each length */ + for (sym = 0; sym < codes; sym++) + if (lens[sym] != 0) work[offs[lens[sym]]++] = (unsigned short)sym; + + /* + Create and fill in decoding tables. In this loop, the table being + filled is at next and has curr index bits. The code being used is huff + with length len. That code is converted to an index by dropping drop + bits off of the bottom. For codes where len is less than drop + curr, + those top drop + curr - len bits are incremented through all values to + fill the table with replicated entries. + + root is the number of index bits for the root table. When len exceeds + root, sub-tables are created pointed to by the root entry with an index + of the low root bits of huff. This is saved in low to check for when a + new sub-table should be started. drop is zero when the root table is + being filled, and drop is root when sub-tables are being filled. + + When a new sub-table is needed, it is necessary to look ahead in the + code lengths to determine what size sub-table is needed. The length + counts are used for this, and so count[] is decremented as codes are + entered in the tables. + + used keeps track of how many table entries have been allocated from the + provided *table space. It is checked for LENS and DIST tables against + the constants ENOUGH_LENS and ENOUGH_DISTS to guard against changes in + the initial root table size constants. See the comments in inftrees.h + for more information. + + sym increments through all symbols, and the loop terminates when + all codes of length max, i.e. all codes, have been processed. This + routine permits incomplete codes, so another loop after this one fills + in the rest of the decoding tables with invalid code markers. + */ + + /* set up for code type */ + switch (type) { + case CODES: + base = extra = work; /* dummy value--not used */ + match = 20; + break; + case LENS: + base = lbase; + extra = lext; + match = 257; + break; + default: /* DISTS */ + base = dbase; + extra = dext; + match = 0; + } + + /* initialize state for loop */ + huff = 0; /* starting code */ + sym = 0; /* starting code symbol */ + len = min; /* starting code length */ + next = *table; /* current table to fill in */ + curr = root; /* current table index bits */ + drop = 0; /* current bits to drop from code for index */ + low = (unsigned)(-1); /* trigger new sub-table when len > root */ + used = 1U << root; /* use root table entries */ + mask = used - 1; /* mask for comparing low */ + + /* check available table space */ + if ((type == LENS && used > ENOUGH_LENS) || + (type == DISTS && used > ENOUGH_DISTS)) + return 1; + + /* process all codes and make table entries */ + for (;;) { + /* create table entry */ + here.bits = (unsigned char)(len - drop); + if (work[sym] + 1U < match) { + here.op = (unsigned char)0; + here.val = work[sym]; + } + else if (work[sym] >= match) { + here.op = (unsigned char)(extra[work[sym] - match]); + here.val = base[work[sym] - match]; + } + else { + here.op = (unsigned char)(32 + 64); /* end of block */ + here.val = 0; + } + + /* replicate for those indices with low len bits equal to huff */ + incr = 1U << (len - drop); + fill = 1U << curr; + min = fill; /* save offset to next table */ + do { + fill -= incr; + next[(huff >> drop) + fill] = here; + } while (fill != 0); + + /* backwards increment the len-bit code huff */ + incr = 1U << (len - 1); + while (huff & incr) + incr >>= 1; + if (incr != 0) { + huff &= incr - 1; + huff += incr; + } + else + huff = 0; + + /* go to next symbol, update count, len */ + sym++; + if (--(count[len]) == 0) { + if (len == max) break; + len = lens[work[sym]]; + } + + /* create new sub-table if needed */ + if (len > root && (huff & mask) != low) { + /* if first time, transition to sub-tables */ + if (drop == 0) + drop = root; + + /* increment past last table */ + next += min; /* here min is 1 << curr */ + + /* determine length of next table */ + curr = len - drop; + left = (int)(1 << curr); + while (curr + drop < max) { + left -= count[curr + drop]; + if (left <= 0) break; + curr++; + left <<= 1; + } + + /* check for enough space */ + used += 1U << curr; + if ((type == LENS && used > ENOUGH_LENS) || + (type == DISTS && used > ENOUGH_DISTS)) + return 1; + + /* point entry in root table to sub-table */ + low = huff & mask; + (*table)[low].op = (unsigned char)curr; + (*table)[low].bits = (unsigned char)root; + (*table)[low].val = (unsigned short)(next - *table); + } + } + + /* fill in remaining table entry if code is incomplete (guaranteed to have + at most one remaining entry, since if the code is incomplete, the + maximum code length that was allowed to get this far is one bit) */ + if (huff != 0) { + here.op = (unsigned char)64; /* invalid code marker */ + here.bits = (unsigned char)(len - drop); + here.val = (unsigned short)0; + next[huff] = here; + } + + /* set return parameters */ + *table += used; + *bits = root; + return 0; +} diff --git a/lib/os/windows/zlib-1.2.13/inftrees.h b/lib/os/windows/zlib-1.2.13/inftrees.h new file mode 100644 index 000000000000..f53665311c16 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/inftrees.h @@ -0,0 +1,62 @@ +/* inftrees.h -- header to use inftrees.c + * Copyright (C) 1995-2005, 2010 Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* WARNING: this file should *not* be used by applications. It is + part of the implementation of the compression library and is + subject to change. Applications should only use zlib.h. + */ + +/* Structure for decoding tables. Each entry provides either the + information needed to do the operation requested by the code that + indexed that table entry, or it provides a pointer to another + table that indexes more bits of the code. op indicates whether + the entry is a pointer to another table, a literal, a length or + distance, an end-of-block, or an invalid code. For a table + pointer, the low four bits of op is the number of index bits of + that table. For a length or distance, the low four bits of op + is the number of extra bits to get after the code. bits is + the number of bits in this code or part of the code to drop off + of the bit buffer. val is the actual byte to output in the case + of a literal, the base length or distance, or the offset from + the current table to the next table. Each entry is four bytes. */ +typedef struct { + unsigned char op; /* operation, extra bits, table bits */ + unsigned char bits; /* bits in this part of the code */ + unsigned short val; /* offset in table or code value */ +} code; + +/* op values as set by inflate_table(): + 00000000 - literal + 0000tttt - table link, tttt != 0 is the number of table index bits + 0001eeee - length or distance, eeee is the number of extra bits + 01100000 - end of block + 01000000 - invalid code + */ + +/* Maximum size of the dynamic table. The maximum number of code structures is + 1444, which is the sum of 852 for literal/length codes and 592 for distance + codes. These values were found by exhaustive searches using the program + examples/enough.c found in the zlib distribution. The arguments to that + program are the number of symbols, the initial root table size, and the + maximum bit length of a code. "enough 286 9 15" for literal/length codes + returns returns 852, and "enough 30 6 15" for distance codes returns 592. + The initial root table size (9 or 6) is found in the fifth argument of the + inflate_table() calls in inflate.c and infback.c. If the root table size is + changed, then these maximum sizes would be need to be recalculated and + updated. */ +#define ENOUGH_LENS 852 +#define ENOUGH_DISTS 592 +#define ENOUGH (ENOUGH_LENS+ENOUGH_DISTS) + +/* Type of code to build for inflate_table() */ +typedef enum { + CODES, + LENS, + DISTS +} codetype; + +int ZLIB_INTERNAL inflate_table OF((codetype type, unsigned short FAR *lens, + unsigned codes, code FAR * FAR *table, + unsigned FAR *bits, unsigned short FAR *work)); diff --git a/lib/os/windows/zlib-1.2.13/test/minigzip.c b/lib/os/windows/zlib-1.2.13/test/minigzip.c new file mode 100644 index 000000000000..a649d2b3d9ba --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/test/minigzip.c @@ -0,0 +1,651 @@ +/* minigzip.c -- simulate gzip using the zlib compression library + * Copyright (C) 1995-2006, 2010, 2011, 2016 Jean-loup Gailly + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* + * minigzip is a minimal implementation of the gzip utility. This is + * only an example of using zlib and isn't meant to replace the + * full-featured gzip. No attempt is made to deal with file systems + * limiting names to 14 or 8+3 characters, etc... Error checking is + * very limited. So use minigzip only for testing; use gzip for the + * real thing. On MSDOS, use only on file names without extension + * or in pipe mode. + */ + +/* @(#) $Id$ */ + +#include "zlib.h" +#include + +#ifdef STDC +# include +# include +#endif + +#ifdef USE_MMAP +# include +# include +# include +#endif + +#if defined(MSDOS) || defined(OS2) || defined(WIN32) || defined(__CYGWIN__) +# include +# include +# ifdef UNDER_CE +# include +# endif +# define SET_BINARY_MODE(file) setmode(fileno(file), O_BINARY) +#else +# define SET_BINARY_MODE(file) +#endif + +#if defined(_MSC_VER) && _MSC_VER < 1900 +# define snprintf _snprintf +#endif + +#ifdef VMS +# define unlink delete +# define GZ_SUFFIX "-gz" +#endif +#ifdef RISCOS +# define unlink remove +# define GZ_SUFFIX "-gz" +# define fileno(file) file->__file +#endif +#if defined(__MWERKS__) && __dest_os != __be_os && __dest_os != __win32_os +# include /* for fileno */ +#endif + +#if !defined(Z_HAVE_UNISTD_H) && !defined(_LARGEFILE64_SOURCE) +#ifndef WIN32 /* unlink already in stdio.h for WIN32 */ + extern int unlink OF((const char *)); +#endif +#endif + +#if defined(UNDER_CE) +# include +# define perror(s) pwinerror(s) + +/* Map the Windows error number in ERROR to a locale-dependent error + message string and return a pointer to it. Typically, the values + for ERROR come from GetLastError. + + The string pointed to shall not be modified by the application, + but may be overwritten by a subsequent call to strwinerror + + The strwinerror function does not change the current setting + of GetLastError. */ + +static char *strwinerror (error) + DWORD error; +{ + static char buf[1024]; + + wchar_t *msgbuf; + DWORD lasterr = GetLastError(); + DWORD chars = FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM + | FORMAT_MESSAGE_ALLOCATE_BUFFER, + NULL, + error, + 0, /* Default language */ + (LPVOID)&msgbuf, + 0, + NULL); + if (chars != 0) { + /* If there is an \r\n appended, zap it. */ + if (chars >= 2 + && msgbuf[chars - 2] == '\r' && msgbuf[chars - 1] == '\n') { + chars -= 2; + msgbuf[chars] = 0; + } + + if (chars > sizeof (buf) - 1) { + chars = sizeof (buf) - 1; + msgbuf[chars] = 0; + } + + wcstombs(buf, msgbuf, chars + 1); + LocalFree(msgbuf); + } + else { + sprintf(buf, "unknown win32 error (%ld)", error); + } + + SetLastError(lasterr); + return buf; +} + +static void pwinerror (s) + const char *s; +{ + if (s && *s) + fprintf(stderr, "%s: %s\n", s, strwinerror(GetLastError ())); + else + fprintf(stderr, "%s\n", strwinerror(GetLastError ())); +} + +#endif /* UNDER_CE */ + +#ifndef GZ_SUFFIX +# define GZ_SUFFIX ".gz" +#endif +#define SUFFIX_LEN (sizeof(GZ_SUFFIX)-1) + +#define BUFLEN 16384 +#define MAX_NAME_LEN 1024 + +#ifdef MAXSEG_64K +# define local static + /* Needed for systems with limitation on stack size. */ +#else +# define local +#endif + +#ifdef Z_SOLO +/* for Z_SOLO, create simplified gz* functions using deflate and inflate */ + +#if defined(Z_HAVE_UNISTD_H) || defined(Z_LARGE) +# include /* for unlink() */ +#endif + +void *myalloc OF((void *, unsigned, unsigned)); +void myfree OF((void *, void *)); + +void *myalloc(q, n, m) + void *q; + unsigned n, m; +{ + (void)q; + return calloc(n, m); +} + +void myfree(q, p) + void *q, *p; +{ + (void)q; + free(p); +} + +typedef struct gzFile_s { + FILE *file; + int write; + int err; + char *msg; + z_stream strm; +} *gzFile; + +gzFile gzopen OF((const char *, const char *)); +gzFile gzdopen OF((int, const char *)); +gzFile gz_open OF((const char *, int, const char *)); + +gzFile gzopen(path, mode) +const char *path; +const char *mode; +{ + return gz_open(path, -1, mode); +} + +gzFile gzdopen(fd, mode) +int fd; +const char *mode; +{ + return gz_open(NULL, fd, mode); +} + +gzFile gz_open(path, fd, mode) + const char *path; + int fd; + const char *mode; +{ + gzFile gz; + int ret; + + gz = malloc(sizeof(struct gzFile_s)); + if (gz == NULL) + return NULL; + gz->write = strchr(mode, 'w') != NULL; + gz->strm.zalloc = myalloc; + gz->strm.zfree = myfree; + gz->strm.opaque = Z_NULL; + if (gz->write) + ret = deflateInit2(&(gz->strm), -1, 8, 15 + 16, 8, 0); + else { + gz->strm.next_in = 0; + gz->strm.avail_in = Z_NULL; + ret = inflateInit2(&(gz->strm), 15 + 16); + } + if (ret != Z_OK) { + free(gz); + return NULL; + } + gz->file = path == NULL ? fdopen(fd, gz->write ? "wb" : "rb") : + fopen(path, gz->write ? "wb" : "rb"); + if (gz->file == NULL) { + gz->write ? deflateEnd(&(gz->strm)) : inflateEnd(&(gz->strm)); + free(gz); + return NULL; + } + gz->err = 0; + gz->msg = ""; + return gz; +} + +int gzwrite OF((gzFile, const void *, unsigned)); + +int gzwrite(gz, buf, len) + gzFile gz; + const void *buf; + unsigned len; +{ + z_stream *strm; + unsigned char out[BUFLEN]; + + if (gz == NULL || !gz->write) + return 0; + strm = &(gz->strm); + strm->next_in = (void *)buf; + strm->avail_in = len; + do { + strm->next_out = out; + strm->avail_out = BUFLEN; + (void)deflate(strm, Z_NO_FLUSH); + fwrite(out, 1, BUFLEN - strm->avail_out, gz->file); + } while (strm->avail_out == 0); + return len; +} + +int gzread OF((gzFile, void *, unsigned)); + +int gzread(gz, buf, len) + gzFile gz; + void *buf; + unsigned len; +{ + int ret; + unsigned got; + unsigned char in[1]; + z_stream *strm; + + if (gz == NULL || gz->write) + return 0; + if (gz->err) + return 0; + strm = &(gz->strm); + strm->next_out = (void *)buf; + strm->avail_out = len; + do { + got = fread(in, 1, 1, gz->file); + if (got == 0) + break; + strm->next_in = in; + strm->avail_in = 1; + ret = inflate(strm, Z_NO_FLUSH); + if (ret == Z_DATA_ERROR) { + gz->err = Z_DATA_ERROR; + gz->msg = strm->msg; + return 0; + } + if (ret == Z_STREAM_END) + inflateReset(strm); + } while (strm->avail_out); + return len - strm->avail_out; +} + +int gzclose OF((gzFile)); + +int gzclose(gz) + gzFile gz; +{ + z_stream *strm; + unsigned char out[BUFLEN]; + + if (gz == NULL) + return Z_STREAM_ERROR; + strm = &(gz->strm); + if (gz->write) { + strm->next_in = Z_NULL; + strm->avail_in = 0; + do { + strm->next_out = out; + strm->avail_out = BUFLEN; + (void)deflate(strm, Z_FINISH); + fwrite(out, 1, BUFLEN - strm->avail_out, gz->file); + } while (strm->avail_out == 0); + deflateEnd(strm); + } + else + inflateEnd(strm); + fclose(gz->file); + free(gz); + return Z_OK; +} + +const char *gzerror OF((gzFile, int *)); + +const char *gzerror(gz, err) + gzFile gz; + int *err; +{ + *err = gz->err; + return gz->msg; +} + +#endif + +static char *prog; + +void error OF((const char *msg)); +void gz_compress OF((FILE *in, gzFile out)); +#ifdef USE_MMAP +int gz_compress_mmap OF((FILE *in, gzFile out)); +#endif +void gz_uncompress OF((gzFile in, FILE *out)); +void file_compress OF((char *file, char *mode)); +void file_uncompress OF((char *file)); +int main OF((int argc, char *argv[])); + +/* =========================================================================== + * Display error message and exit + */ +void error(msg) + const char *msg; +{ + fprintf(stderr, "%s: %s\n", prog, msg); + exit(1); +} + +/* =========================================================================== + * Compress input to output then close both files. + */ + +void gz_compress(in, out) + FILE *in; + gzFile out; +{ + local char buf[BUFLEN]; + int len; + int err; + +#ifdef USE_MMAP + /* Try first compressing with mmap. If mmap fails (minigzip used in a + * pipe), use the normal fread loop. + */ + if (gz_compress_mmap(in, out) == Z_OK) return; +#endif + for (;;) { + len = (int)fread(buf, 1, sizeof(buf), in); + if (ferror(in)) { + perror("fread"); + exit(1); + } + if (len == 0) break; + + if (gzwrite(out, buf, (unsigned)len) != len) error(gzerror(out, &err)); + } + fclose(in); + if (gzclose(out) != Z_OK) error("failed gzclose"); +} + +#ifdef USE_MMAP /* MMAP version, Miguel Albrecht */ + +/* Try compressing the input file at once using mmap. Return Z_OK if + * if success, Z_ERRNO otherwise. + */ +int gz_compress_mmap(in, out) + FILE *in; + gzFile out; +{ + int len; + int err; + int ifd = fileno(in); + caddr_t buf; /* mmap'ed buffer for the entire input file */ + off_t buf_len; /* length of the input file */ + struct stat sb; + + /* Determine the size of the file, needed for mmap: */ + if (fstat(ifd, &sb) < 0) return Z_ERRNO; + buf_len = sb.st_size; + if (buf_len <= 0) return Z_ERRNO; + + /* Now do the actual mmap: */ + buf = mmap((caddr_t) 0, buf_len, PROT_READ, MAP_SHARED, ifd, (off_t)0); + if (buf == (caddr_t)(-1)) return Z_ERRNO; + + /* Compress the whole file at once: */ + len = gzwrite(out, (char *)buf, (unsigned)buf_len); + + if (len != (int)buf_len) error(gzerror(out, &err)); + + munmap(buf, buf_len); + fclose(in); + if (gzclose(out) != Z_OK) error("failed gzclose"); + return Z_OK; +} +#endif /* USE_MMAP */ + +/* =========================================================================== + * Uncompress input to output then close both files. + */ +void gz_uncompress(in, out) + gzFile in; + FILE *out; +{ + local char buf[BUFLEN]; + int len; + int err; + + for (;;) { + len = gzread(in, buf, sizeof(buf)); + if (len < 0) error (gzerror(in, &err)); + if (len == 0) break; + + if ((int)fwrite(buf, 1, (unsigned)len, out) != len) { + error("failed fwrite"); + } + } + if (fclose(out)) error("failed fclose"); + + if (gzclose(in) != Z_OK) error("failed gzclose"); +} + + +/* =========================================================================== + * Compress the given file: create a corresponding .gz file and remove the + * original. + */ +void file_compress(file, mode) + char *file; + char *mode; +{ + local char outfile[MAX_NAME_LEN]; + FILE *in; + gzFile out; + + if (strlen(file) + strlen(GZ_SUFFIX) >= sizeof(outfile)) { + fprintf(stderr, "%s: filename too long\n", prog); + exit(1); + } + +#if !defined(NO_snprintf) && !defined(NO_vsnprintf) + snprintf(outfile, sizeof(outfile), "%s%s", file, GZ_SUFFIX); +#else + strcpy(outfile, file); + strcat(outfile, GZ_SUFFIX); +#endif + + in = fopen(file, "rb"); + if (in == NULL) { + perror(file); + exit(1); + } + out = gzopen(outfile, mode); + if (out == NULL) { + fprintf(stderr, "%s: can't gzopen %s\n", prog, outfile); + exit(1); + } + gz_compress(in, out); + + unlink(file); +} + + +/* =========================================================================== + * Uncompress the given file and remove the original. + */ +void file_uncompress(file) + char *file; +{ + local char buf[MAX_NAME_LEN]; + char *infile, *outfile; + FILE *out; + gzFile in; + z_size_t len = strlen(file); + + if (len + strlen(GZ_SUFFIX) >= sizeof(buf)) { + fprintf(stderr, "%s: filename too long\n", prog); + exit(1); + } + +#if !defined(NO_snprintf) && !defined(NO_vsnprintf) + snprintf(buf, sizeof(buf), "%s", file); +#else + strcpy(buf, file); +#endif + + if (len > SUFFIX_LEN && strcmp(file+len-SUFFIX_LEN, GZ_SUFFIX) == 0) { + infile = file; + outfile = buf; + outfile[len-3] = '\0'; + } else { + outfile = file; + infile = buf; +#if !defined(NO_snprintf) && !defined(NO_vsnprintf) + snprintf(buf + len, sizeof(buf) - len, "%s", GZ_SUFFIX); +#else + strcat(infile, GZ_SUFFIX); +#endif + } + in = gzopen(infile, "rb"); + if (in == NULL) { + fprintf(stderr, "%s: can't gzopen %s\n", prog, infile); + exit(1); + } + out = fopen(outfile, "wb"); + if (out == NULL) { + perror(file); + exit(1); + } + + gz_uncompress(in, out); + + unlink(infile); +} + + +/* =========================================================================== + * Usage: minigzip [-c] [-d] [-f] [-h] [-r] [-1 to -9] [files...] + * -c : write to standard output + * -d : decompress + * -f : compress with Z_FILTERED + * -h : compress with Z_HUFFMAN_ONLY + * -r : compress with Z_RLE + * -1 to -9 : compression level + */ + +int main(argc, argv) + int argc; + char *argv[]; +{ + int copyout = 0; + int uncompr = 0; + gzFile file; + char *bname, outmode[20]; + +#if !defined(NO_snprintf) && !defined(NO_vsnprintf) + snprintf(outmode, sizeof(outmode), "%s", "wb6 "); +#else + strcpy(outmode, "wb6 "); +#endif + + prog = argv[0]; + bname = strrchr(argv[0], '/'); + if (bname) + bname++; + else + bname = argv[0]; + argc--, argv++; + + if (!strcmp(bname, "gunzip")) + uncompr = 1; + else if (!strcmp(bname, "zcat")) + copyout = uncompr = 1; + + while (argc > 0) { + if (strcmp(*argv, "-c") == 0) + copyout = 1; + else if (strcmp(*argv, "-d") == 0) + uncompr = 1; + else if (strcmp(*argv, "-f") == 0) + outmode[3] = 'f'; + else if (strcmp(*argv, "-h") == 0) + outmode[3] = 'h'; + else if (strcmp(*argv, "-r") == 0) + outmode[3] = 'R'; + else if ((*argv)[0] == '-' && (*argv)[1] >= '1' && (*argv)[1] <= '9' && + (*argv)[2] == 0) + outmode[2] = (*argv)[1]; + else + break; + argc--, argv++; + } + if (outmode[3] == ' ') + outmode[3] = 0; + if (argc == 0) { + SET_BINARY_MODE(stdin); + SET_BINARY_MODE(stdout); + if (uncompr) { + file = gzdopen(fileno(stdin), "rb"); + if (file == NULL) error("can't gzdopen stdin"); + gz_uncompress(file, stdout); + } else { + file = gzdopen(fileno(stdout), outmode); + if (file == NULL) error("can't gzdopen stdout"); + gz_compress(stdin, file); + } + } else { + if (copyout) { + SET_BINARY_MODE(stdout); + } + do { + if (uncompr) { + if (copyout) { + file = gzopen(*argv, "rb"); + if (file == NULL) + fprintf(stderr, "%s: can't gzopen %s\n", prog, *argv); + else + gz_uncompress(file, stdout); + } else { + file_uncompress(*argv); + } + } else { + if (copyout) { + FILE * in = fopen(*argv, "rb"); + + if (in == NULL) { + perror(*argv); + } else { + file = gzdopen(fileno(stdout), outmode); + if (file == NULL) error("can't gzdopen stdout"); + + gz_compress(in, file); + } + + } else { + file_compress(*argv, outmode); + } + } + } while (argv++, --argc); + } + return 0; +} diff --git a/lib/os/windows/zlib-1.2.13/trees.c b/lib/os/windows/zlib-1.2.13/trees.c new file mode 100644 index 000000000000..5f305c47221e --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/trees.c @@ -0,0 +1,1181 @@ +/* trees.c -- output deflated data using Huffman coding + * Copyright (C) 1995-2021 Jean-loup Gailly + * detect_data_type() function provided freely by Cosmin Truta, 2006 + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* + * ALGORITHM + * + * The "deflation" process uses several Huffman trees. The more + * common source values are represented by shorter bit sequences. + * + * Each code tree is stored in a compressed form which is itself + * a Huffman encoding of the lengths of all the code strings (in + * ascending order by source values). The actual code strings are + * reconstructed from the lengths in the inflate process, as described + * in the deflate specification. + * + * REFERENCES + * + * Deutsch, L.P.,"'Deflate' Compressed Data Format Specification". + * Available in ftp.uu.net:/pub/archiving/zip/doc/deflate-1.1.doc + * + * Storer, James A. + * Data Compression: Methods and Theory, pp. 49-50. + * Computer Science Press, 1988. ISBN 0-7167-8156-5. + * + * Sedgewick, R. + * Algorithms, p290. + * Addison-Wesley, 1983. ISBN 0-201-06672-6. + */ + +/* @(#) $Id$ */ + +/* #define GEN_TREES_H */ + +#include "deflate.h" + +#ifdef ZLIB_DEBUG +# include +#endif + +/* =========================================================================== + * Constants + */ + +#define MAX_BL_BITS 7 +/* Bit length codes must not exceed MAX_BL_BITS bits */ + +#define END_BLOCK 256 +/* end of block literal code */ + +#define REP_3_6 16 +/* repeat previous bit length 3-6 times (2 bits of repeat count) */ + +#define REPZ_3_10 17 +/* repeat a zero length 3-10 times (3 bits of repeat count) */ + +#define REPZ_11_138 18 +/* repeat a zero length 11-138 times (7 bits of repeat count) */ + +local const int extra_lbits[LENGTH_CODES] /* extra bits for each length code */ + = {0,0,0,0,0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4,5,5,5,5,0}; + +local const int extra_dbits[D_CODES] /* extra bits for each distance code */ + = {0,0,0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,8,8,9,9,10,10,11,11,12,12,13,13}; + +local const int extra_blbits[BL_CODES]/* extra bits for each bit length code */ + = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,3,7}; + +local const uch bl_order[BL_CODES] + = {16,17,18,0,8,7,9,6,10,5,11,4,12,3,13,2,14,1,15}; +/* The lengths of the bit length codes are sent in order of decreasing + * probability, to avoid transmitting the lengths for unused bit length codes. + */ + +/* =========================================================================== + * Local data. These are initialized only once. + */ + +#define DIST_CODE_LEN 512 /* see definition of array dist_code below */ + +#if defined(GEN_TREES_H) || !defined(STDC) +/* non ANSI compilers may not accept trees.h */ + +local ct_data static_ltree[L_CODES+2]; +/* The static literal tree. Since the bit lengths are imposed, there is no + * need for the L_CODES extra codes used during heap construction. However + * The codes 286 and 287 are needed to build a canonical tree (see _tr_init + * below). + */ + +local ct_data static_dtree[D_CODES]; +/* The static distance tree. (Actually a trivial tree since all codes use + * 5 bits.) + */ + +uch _dist_code[DIST_CODE_LEN]; +/* Distance codes. The first 256 values correspond to the distances + * 3 .. 258, the last 256 values correspond to the top 8 bits of + * the 15 bit distances. + */ + +uch _length_code[MAX_MATCH-MIN_MATCH+1]; +/* length code for each normalized match length (0 == MIN_MATCH) */ + +local int base_length[LENGTH_CODES]; +/* First normalized length for each code (0 = MIN_MATCH) */ + +local int base_dist[D_CODES]; +/* First normalized distance for each code (0 = distance of 1) */ + +#else +# include "trees.h" +#endif /* GEN_TREES_H */ + +struct static_tree_desc_s { + const ct_data *static_tree; /* static tree or NULL */ + const intf *extra_bits; /* extra bits for each code or NULL */ + int extra_base; /* base index for extra_bits */ + int elems; /* max number of elements in the tree */ + int max_length; /* max bit length for the codes */ +}; + +local const static_tree_desc static_l_desc = +{static_ltree, extra_lbits, LITERALS+1, L_CODES, MAX_BITS}; + +local const static_tree_desc static_d_desc = +{static_dtree, extra_dbits, 0, D_CODES, MAX_BITS}; + +local const static_tree_desc static_bl_desc = +{(const ct_data *)0, extra_blbits, 0, BL_CODES, MAX_BL_BITS}; + +/* =========================================================================== + * Local (static) routines in this file. + */ + +local void tr_static_init OF((void)); +local void init_block OF((deflate_state *s)); +local void pqdownheap OF((deflate_state *s, ct_data *tree, int k)); +local void gen_bitlen OF((deflate_state *s, tree_desc *desc)); +local void gen_codes OF((ct_data *tree, int max_code, ushf *bl_count)); +local void build_tree OF((deflate_state *s, tree_desc *desc)); +local void scan_tree OF((deflate_state *s, ct_data *tree, int max_code)); +local void send_tree OF((deflate_state *s, ct_data *tree, int max_code)); +local int build_bl_tree OF((deflate_state *s)); +local void send_all_trees OF((deflate_state *s, int lcodes, int dcodes, + int blcodes)); +local void compress_block OF((deflate_state *s, const ct_data *ltree, + const ct_data *dtree)); +local int detect_data_type OF((deflate_state *s)); +local unsigned bi_reverse OF((unsigned code, int len)); +local void bi_windup OF((deflate_state *s)); +local void bi_flush OF((deflate_state *s)); + +#ifdef GEN_TREES_H +local void gen_trees_header OF((void)); +#endif + +#ifndef ZLIB_DEBUG +# define send_code(s, c, tree) send_bits(s, tree[c].Code, tree[c].Len) + /* Send a code of the given tree. c and tree must not have side effects */ + +#else /* !ZLIB_DEBUG */ +# define send_code(s, c, tree) \ + { if (z_verbose>2) fprintf(stderr,"\ncd %3d ",(c)); \ + send_bits(s, tree[c].Code, tree[c].Len); } +#endif + +/* =========================================================================== + * Output a short LSB first on the stream. + * IN assertion: there is enough room in pendingBuf. + */ +#define put_short(s, w) { \ + put_byte(s, (uch)((w) & 0xff)); \ + put_byte(s, (uch)((ush)(w) >> 8)); \ +} + +/* =========================================================================== + * Send a value on a given number of bits. + * IN assertion: length <= 16 and value fits in length bits. + */ +#ifdef ZLIB_DEBUG +local void send_bits OF((deflate_state *s, int value, int length)); + +local void send_bits(s, value, length) + deflate_state *s; + int value; /* value to send */ + int length; /* number of bits */ +{ + Tracevv((stderr," l %2d v %4x ", length, value)); + Assert(length > 0 && length <= 15, "invalid length"); + s->bits_sent += (ulg)length; + + /* If not enough room in bi_buf, use (valid) bits from bi_buf and + * (16 - bi_valid) bits from value, leaving (width - (16 - bi_valid)) + * unused bits in value. + */ + if (s->bi_valid > (int)Buf_size - length) { + s->bi_buf |= (ush)value << s->bi_valid; + put_short(s, s->bi_buf); + s->bi_buf = (ush)value >> (Buf_size - s->bi_valid); + s->bi_valid += length - Buf_size; + } else { + s->bi_buf |= (ush)value << s->bi_valid; + s->bi_valid += length; + } +} +#else /* !ZLIB_DEBUG */ + +#define send_bits(s, value, length) \ +{ int len = length;\ + if (s->bi_valid > (int)Buf_size - len) {\ + int val = (int)value;\ + s->bi_buf |= (ush)val << s->bi_valid;\ + put_short(s, s->bi_buf);\ + s->bi_buf = (ush)val >> (Buf_size - s->bi_valid);\ + s->bi_valid += len - Buf_size;\ + } else {\ + s->bi_buf |= (ush)(value) << s->bi_valid;\ + s->bi_valid += len;\ + }\ +} +#endif /* ZLIB_DEBUG */ + + +/* the arguments must not have side effects */ + +/* =========================================================================== + * Initialize the various 'constant' tables. + */ +local void tr_static_init() +{ +#if defined(GEN_TREES_H) || !defined(STDC) + static int static_init_done = 0; + int n; /* iterates over tree elements */ + int bits; /* bit counter */ + int length; /* length value */ + int code; /* code value */ + int dist; /* distance index */ + ush bl_count[MAX_BITS+1]; + /* number of codes at each bit length for an optimal tree */ + + if (static_init_done) return; + + /* For some embedded targets, global variables are not initialized: */ +#ifdef NO_INIT_GLOBAL_POINTERS + static_l_desc.static_tree = static_ltree; + static_l_desc.extra_bits = extra_lbits; + static_d_desc.static_tree = static_dtree; + static_d_desc.extra_bits = extra_dbits; + static_bl_desc.extra_bits = extra_blbits; +#endif + + /* Initialize the mapping length (0..255) -> length code (0..28) */ + length = 0; + for (code = 0; code < LENGTH_CODES-1; code++) { + base_length[code] = length; + for (n = 0; n < (1 << extra_lbits[code]); n++) { + _length_code[length++] = (uch)code; + } + } + Assert (length == 256, "tr_static_init: length != 256"); + /* Note that the length 255 (match length 258) can be represented + * in two different ways: code 284 + 5 bits or code 285, so we + * overwrite length_code[255] to use the best encoding: + */ + _length_code[length - 1] = (uch)code; + + /* Initialize the mapping dist (0..32K) -> dist code (0..29) */ + dist = 0; + for (code = 0 ; code < 16; code++) { + base_dist[code] = dist; + for (n = 0; n < (1 << extra_dbits[code]); n++) { + _dist_code[dist++] = (uch)code; + } + } + Assert (dist == 256, "tr_static_init: dist != 256"); + dist >>= 7; /* from now on, all distances are divided by 128 */ + for ( ; code < D_CODES; code++) { + base_dist[code] = dist << 7; + for (n = 0; n < (1 << (extra_dbits[code] - 7)); n++) { + _dist_code[256 + dist++] = (uch)code; + } + } + Assert (dist == 256, "tr_static_init: 256 + dist != 512"); + + /* Construct the codes of the static literal tree */ + for (bits = 0; bits <= MAX_BITS; bits++) bl_count[bits] = 0; + n = 0; + while (n <= 143) static_ltree[n++].Len = 8, bl_count[8]++; + while (n <= 255) static_ltree[n++].Len = 9, bl_count[9]++; + while (n <= 279) static_ltree[n++].Len = 7, bl_count[7]++; + while (n <= 287) static_ltree[n++].Len = 8, bl_count[8]++; + /* Codes 286 and 287 do not exist, but we must include them in the + * tree construction to get a canonical Huffman tree (longest code + * all ones) + */ + gen_codes((ct_data *)static_ltree, L_CODES+1, bl_count); + + /* The static distance tree is trivial: */ + for (n = 0; n < D_CODES; n++) { + static_dtree[n].Len = 5; + static_dtree[n].Code = bi_reverse((unsigned)n, 5); + } + static_init_done = 1; + +# ifdef GEN_TREES_H + gen_trees_header(); +# endif +#endif /* defined(GEN_TREES_H) || !defined(STDC) */ +} + +/* =========================================================================== + * Generate the file trees.h describing the static trees. + */ +#ifdef GEN_TREES_H +# ifndef ZLIB_DEBUG +# include +# endif + +# define SEPARATOR(i, last, width) \ + ((i) == (last)? "\n};\n\n" : \ + ((i) % (width) == (width) - 1 ? ",\n" : ", ")) + +void gen_trees_header() +{ + FILE *header = fopen("trees.h", "w"); + int i; + + Assert (header != NULL, "Can't open trees.h"); + fprintf(header, + "/* header created automatically with -DGEN_TREES_H */\n\n"); + + fprintf(header, "local const ct_data static_ltree[L_CODES+2] = {\n"); + for (i = 0; i < L_CODES+2; i++) { + fprintf(header, "{{%3u},{%3u}}%s", static_ltree[i].Code, + static_ltree[i].Len, SEPARATOR(i, L_CODES+1, 5)); + } + + fprintf(header, "local const ct_data static_dtree[D_CODES] = {\n"); + for (i = 0; i < D_CODES; i++) { + fprintf(header, "{{%2u},{%2u}}%s", static_dtree[i].Code, + static_dtree[i].Len, SEPARATOR(i, D_CODES-1, 5)); + } + + fprintf(header, "const uch ZLIB_INTERNAL _dist_code[DIST_CODE_LEN] = {\n"); + for (i = 0; i < DIST_CODE_LEN; i++) { + fprintf(header, "%2u%s", _dist_code[i], + SEPARATOR(i, DIST_CODE_LEN-1, 20)); + } + + fprintf(header, + "const uch ZLIB_INTERNAL _length_code[MAX_MATCH-MIN_MATCH+1]= {\n"); + for (i = 0; i < MAX_MATCH-MIN_MATCH+1; i++) { + fprintf(header, "%2u%s", _length_code[i], + SEPARATOR(i, MAX_MATCH-MIN_MATCH, 20)); + } + + fprintf(header, "local const int base_length[LENGTH_CODES] = {\n"); + for (i = 0; i < LENGTH_CODES; i++) { + fprintf(header, "%1u%s", base_length[i], + SEPARATOR(i, LENGTH_CODES-1, 20)); + } + + fprintf(header, "local const int base_dist[D_CODES] = {\n"); + for (i = 0; i < D_CODES; i++) { + fprintf(header, "%5u%s", base_dist[i], + SEPARATOR(i, D_CODES-1, 10)); + } + + fclose(header); +} +#endif /* GEN_TREES_H */ + +/* =========================================================================== + * Initialize the tree data structures for a new zlib stream. + */ +void ZLIB_INTERNAL _tr_init(s) + deflate_state *s; +{ + tr_static_init(); + + s->l_desc.dyn_tree = s->dyn_ltree; + s->l_desc.stat_desc = &static_l_desc; + + s->d_desc.dyn_tree = s->dyn_dtree; + s->d_desc.stat_desc = &static_d_desc; + + s->bl_desc.dyn_tree = s->bl_tree; + s->bl_desc.stat_desc = &static_bl_desc; + + s->bi_buf = 0; + s->bi_valid = 0; +#ifdef ZLIB_DEBUG + s->compressed_len = 0L; + s->bits_sent = 0L; +#endif + + /* Initialize the first block of the first file: */ + init_block(s); +} + +/* =========================================================================== + * Initialize a new block. + */ +local void init_block(s) + deflate_state *s; +{ + int n; /* iterates over tree elements */ + + /* Initialize the trees. */ + for (n = 0; n < L_CODES; n++) s->dyn_ltree[n].Freq = 0; + for (n = 0; n < D_CODES; n++) s->dyn_dtree[n].Freq = 0; + for (n = 0; n < BL_CODES; n++) s->bl_tree[n].Freq = 0; + + s->dyn_ltree[END_BLOCK].Freq = 1; + s->opt_len = s->static_len = 0L; + s->sym_next = s->matches = 0; +} + +#define SMALLEST 1 +/* Index within the heap array of least frequent node in the Huffman tree */ + + +/* =========================================================================== + * Remove the smallest element from the heap and recreate the heap with + * one less element. Updates heap and heap_len. + */ +#define pqremove(s, tree, top) \ +{\ + top = s->heap[SMALLEST]; \ + s->heap[SMALLEST] = s->heap[s->heap_len--]; \ + pqdownheap(s, tree, SMALLEST); \ +} + +/* =========================================================================== + * Compares to subtrees, using the tree depth as tie breaker when + * the subtrees have equal frequency. This minimizes the worst case length. + */ +#define smaller(tree, n, m, depth) \ + (tree[n].Freq < tree[m].Freq || \ + (tree[n].Freq == tree[m].Freq && depth[n] <= depth[m])) + +/* =========================================================================== + * Restore the heap property by moving down the tree starting at node k, + * exchanging a node with the smallest of its two sons if necessary, stopping + * when the heap property is re-established (each father smaller than its + * two sons). + */ +local void pqdownheap(s, tree, k) + deflate_state *s; + ct_data *tree; /* the tree to restore */ + int k; /* node to move down */ +{ + int v = s->heap[k]; + int j = k << 1; /* left son of k */ + while (j <= s->heap_len) { + /* Set j to the smallest of the two sons: */ + if (j < s->heap_len && + smaller(tree, s->heap[j + 1], s->heap[j], s->depth)) { + j++; + } + /* Exit if v is smaller than both sons */ + if (smaller(tree, v, s->heap[j], s->depth)) break; + + /* Exchange v with the smallest son */ + s->heap[k] = s->heap[j]; k = j; + + /* And continue down the tree, setting j to the left son of k */ + j <<= 1; + } + s->heap[k] = v; +} + +/* =========================================================================== + * Compute the optimal bit lengths for a tree and update the total bit length + * for the current block. + * IN assertion: the fields freq and dad are set, heap[heap_max] and + * above are the tree nodes sorted by increasing frequency. + * OUT assertions: the field len is set to the optimal bit length, the + * array bl_count contains the frequencies for each bit length. + * The length opt_len is updated; static_len is also updated if stree is + * not null. + */ +local void gen_bitlen(s, desc) + deflate_state *s; + tree_desc *desc; /* the tree descriptor */ +{ + ct_data *tree = desc->dyn_tree; + int max_code = desc->max_code; + const ct_data *stree = desc->stat_desc->static_tree; + const intf *extra = desc->stat_desc->extra_bits; + int base = desc->stat_desc->extra_base; + int max_length = desc->stat_desc->max_length; + int h; /* heap index */ + int n, m; /* iterate over the tree elements */ + int bits; /* bit length */ + int xbits; /* extra bits */ + ush f; /* frequency */ + int overflow = 0; /* number of elements with bit length too large */ + + for (bits = 0; bits <= MAX_BITS; bits++) s->bl_count[bits] = 0; + + /* In a first pass, compute the optimal bit lengths (which may + * overflow in the case of the bit length tree). + */ + tree[s->heap[s->heap_max]].Len = 0; /* root of the heap */ + + for (h = s->heap_max + 1; h < HEAP_SIZE; h++) { + n = s->heap[h]; + bits = tree[tree[n].Dad].Len + 1; + if (bits > max_length) bits = max_length, overflow++; + tree[n].Len = (ush)bits; + /* We overwrite tree[n].Dad which is no longer needed */ + + if (n > max_code) continue; /* not a leaf node */ + + s->bl_count[bits]++; + xbits = 0; + if (n >= base) xbits = extra[n - base]; + f = tree[n].Freq; + s->opt_len += (ulg)f * (unsigned)(bits + xbits); + if (stree) s->static_len += (ulg)f * (unsigned)(stree[n].Len + xbits); + } + if (overflow == 0) return; + + Tracev((stderr,"\nbit length overflow\n")); + /* This happens for example on obj2 and pic of the Calgary corpus */ + + /* Find the first bit length which could increase: */ + do { + bits = max_length - 1; + while (s->bl_count[bits] == 0) bits--; + s->bl_count[bits]--; /* move one leaf down the tree */ + s->bl_count[bits + 1] += 2; /* move one overflow item as its brother */ + s->bl_count[max_length]--; + /* The brother of the overflow item also moves one step up, + * but this does not affect bl_count[max_length] + */ + overflow -= 2; + } while (overflow > 0); + + /* Now recompute all bit lengths, scanning in increasing frequency. + * h is still equal to HEAP_SIZE. (It is simpler to reconstruct all + * lengths instead of fixing only the wrong ones. This idea is taken + * from 'ar' written by Haruhiko Okumura.) + */ + for (bits = max_length; bits != 0; bits--) { + n = s->bl_count[bits]; + while (n != 0) { + m = s->heap[--h]; + if (m > max_code) continue; + if ((unsigned) tree[m].Len != (unsigned) bits) { + Tracev((stderr,"code %d bits %d->%d\n", m, tree[m].Len, bits)); + s->opt_len += ((ulg)bits - tree[m].Len) * tree[m].Freq; + tree[m].Len = (ush)bits; + } + n--; + } + } +} + +/* =========================================================================== + * Generate the codes for a given tree and bit counts (which need not be + * optimal). + * IN assertion: the array bl_count contains the bit length statistics for + * the given tree and the field len is set for all tree elements. + * OUT assertion: the field code is set for all tree elements of non + * zero code length. + */ +local void gen_codes(tree, max_code, bl_count) + ct_data *tree; /* the tree to decorate */ + int max_code; /* largest code with non zero frequency */ + ushf *bl_count; /* number of codes at each bit length */ +{ + ush next_code[MAX_BITS+1]; /* next code value for each bit length */ + unsigned code = 0; /* running code value */ + int bits; /* bit index */ + int n; /* code index */ + + /* The distribution counts are first used to generate the code values + * without bit reversal. + */ + for (bits = 1; bits <= MAX_BITS; bits++) { + code = (code + bl_count[bits - 1]) << 1; + next_code[bits] = (ush)code; + } + /* Check that the bit counts in bl_count are consistent. The last code + * must be all ones. + */ + Assert (code + bl_count[MAX_BITS] - 1 == (1 << MAX_BITS) - 1, + "inconsistent bit counts"); + Tracev((stderr,"\ngen_codes: max_code %d ", max_code)); + + for (n = 0; n <= max_code; n++) { + int len = tree[n].Len; + if (len == 0) continue; + /* Now reverse the bits */ + tree[n].Code = (ush)bi_reverse(next_code[len]++, len); + + Tracecv(tree != static_ltree, (stderr,"\nn %3d %c l %2d c %4x (%x) ", + n, (isgraph(n) ? n : ' '), len, tree[n].Code, next_code[len] - 1)); + } +} + +/* =========================================================================== + * Construct one Huffman tree and assigns the code bit strings and lengths. + * Update the total bit length for the current block. + * IN assertion: the field freq is set for all tree elements. + * OUT assertions: the fields len and code are set to the optimal bit length + * and corresponding code. The length opt_len is updated; static_len is + * also updated if stree is not null. The field max_code is set. + */ +local void build_tree(s, desc) + deflate_state *s; + tree_desc *desc; /* the tree descriptor */ +{ + ct_data *tree = desc->dyn_tree; + const ct_data *stree = desc->stat_desc->static_tree; + int elems = desc->stat_desc->elems; + int n, m; /* iterate over heap elements */ + int max_code = -1; /* largest code with non zero frequency */ + int node; /* new node being created */ + + /* Construct the initial heap, with least frequent element in + * heap[SMALLEST]. The sons of heap[n] are heap[2*n] and heap[2*n + 1]. + * heap[0] is not used. + */ + s->heap_len = 0, s->heap_max = HEAP_SIZE; + + for (n = 0; n < elems; n++) { + if (tree[n].Freq != 0) { + s->heap[++(s->heap_len)] = max_code = n; + s->depth[n] = 0; + } else { + tree[n].Len = 0; + } + } + + /* The pkzip format requires that at least one distance code exists, + * and that at least one bit should be sent even if there is only one + * possible code. So to avoid special checks later on we force at least + * two codes of non zero frequency. + */ + while (s->heap_len < 2) { + node = s->heap[++(s->heap_len)] = (max_code < 2 ? ++max_code : 0); + tree[node].Freq = 1; + s->depth[node] = 0; + s->opt_len--; if (stree) s->static_len -= stree[node].Len; + /* node is 0 or 1 so it does not have extra bits */ + } + desc->max_code = max_code; + + /* The elements heap[heap_len/2 + 1 .. heap_len] are leaves of the tree, + * establish sub-heaps of increasing lengths: + */ + for (n = s->heap_len/2; n >= 1; n--) pqdownheap(s, tree, n); + + /* Construct the Huffman tree by repeatedly combining the least two + * frequent nodes. + */ + node = elems; /* next internal node of the tree */ + do { + pqremove(s, tree, n); /* n = node of least frequency */ + m = s->heap[SMALLEST]; /* m = node of next least frequency */ + + s->heap[--(s->heap_max)] = n; /* keep the nodes sorted by frequency */ + s->heap[--(s->heap_max)] = m; + + /* Create a new node father of n and m */ + tree[node].Freq = tree[n].Freq + tree[m].Freq; + s->depth[node] = (uch)((s->depth[n] >= s->depth[m] ? + s->depth[n] : s->depth[m]) + 1); + tree[n].Dad = tree[m].Dad = (ush)node; +#ifdef DUMP_BL_TREE + if (tree == s->bl_tree) { + fprintf(stderr,"\nnode %d(%d), sons %d(%d) %d(%d)", + node, tree[node].Freq, n, tree[n].Freq, m, tree[m].Freq); + } +#endif + /* and insert the new node in the heap */ + s->heap[SMALLEST] = node++; + pqdownheap(s, tree, SMALLEST); + + } while (s->heap_len >= 2); + + s->heap[--(s->heap_max)] = s->heap[SMALLEST]; + + /* At this point, the fields freq and dad are set. We can now + * generate the bit lengths. + */ + gen_bitlen(s, (tree_desc *)desc); + + /* The field len is now set, we can generate the bit codes */ + gen_codes ((ct_data *)tree, max_code, s->bl_count); +} + +/* =========================================================================== + * Scan a literal or distance tree to determine the frequencies of the codes + * in the bit length tree. + */ +local void scan_tree(s, tree, max_code) + deflate_state *s; + ct_data *tree; /* the tree to be scanned */ + int max_code; /* and its largest code of non zero frequency */ +{ + int n; /* iterates over all tree elements */ + int prevlen = -1; /* last emitted length */ + int curlen; /* length of current code */ + int nextlen = tree[0].Len; /* length of next code */ + int count = 0; /* repeat count of the current code */ + int max_count = 7; /* max repeat count */ + int min_count = 4; /* min repeat count */ + + if (nextlen == 0) max_count = 138, min_count = 3; + tree[max_code + 1].Len = (ush)0xffff; /* guard */ + + for (n = 0; n <= max_code; n++) { + curlen = nextlen; nextlen = tree[n + 1].Len; + if (++count < max_count && curlen == nextlen) { + continue; + } else if (count < min_count) { + s->bl_tree[curlen].Freq += count; + } else if (curlen != 0) { + if (curlen != prevlen) s->bl_tree[curlen].Freq++; + s->bl_tree[REP_3_6].Freq++; + } else if (count <= 10) { + s->bl_tree[REPZ_3_10].Freq++; + } else { + s->bl_tree[REPZ_11_138].Freq++; + } + count = 0; prevlen = curlen; + if (nextlen == 0) { + max_count = 138, min_count = 3; + } else if (curlen == nextlen) { + max_count = 6, min_count = 3; + } else { + max_count = 7, min_count = 4; + } + } +} + +/* =========================================================================== + * Send a literal or distance tree in compressed form, using the codes in + * bl_tree. + */ +local void send_tree(s, tree, max_code) + deflate_state *s; + ct_data *tree; /* the tree to be scanned */ + int max_code; /* and its largest code of non zero frequency */ +{ + int n; /* iterates over all tree elements */ + int prevlen = -1; /* last emitted length */ + int curlen; /* length of current code */ + int nextlen = tree[0].Len; /* length of next code */ + int count = 0; /* repeat count of the current code */ + int max_count = 7; /* max repeat count */ + int min_count = 4; /* min repeat count */ + + /* tree[max_code + 1].Len = -1; */ /* guard already set */ + if (nextlen == 0) max_count = 138, min_count = 3; + + for (n = 0; n <= max_code; n++) { + curlen = nextlen; nextlen = tree[n + 1].Len; + if (++count < max_count && curlen == nextlen) { + continue; + } else if (count < min_count) { + do { send_code(s, curlen, s->bl_tree); } while (--count != 0); + + } else if (curlen != 0) { + if (curlen != prevlen) { + send_code(s, curlen, s->bl_tree); count--; + } + Assert(count >= 3 && count <= 6, " 3_6?"); + send_code(s, REP_3_6, s->bl_tree); send_bits(s, count - 3, 2); + + } else if (count <= 10) { + send_code(s, REPZ_3_10, s->bl_tree); send_bits(s, count - 3, 3); + + } else { + send_code(s, REPZ_11_138, s->bl_tree); send_bits(s, count - 11, 7); + } + count = 0; prevlen = curlen; + if (nextlen == 0) { + max_count = 138, min_count = 3; + } else if (curlen == nextlen) { + max_count = 6, min_count = 3; + } else { + max_count = 7, min_count = 4; + } + } +} + +/* =========================================================================== + * Construct the Huffman tree for the bit lengths and return the index in + * bl_order of the last bit length code to send. + */ +local int build_bl_tree(s) + deflate_state *s; +{ + int max_blindex; /* index of last bit length code of non zero freq */ + + /* Determine the bit length frequencies for literal and distance trees */ + scan_tree(s, (ct_data *)s->dyn_ltree, s->l_desc.max_code); + scan_tree(s, (ct_data *)s->dyn_dtree, s->d_desc.max_code); + + /* Build the bit length tree: */ + build_tree(s, (tree_desc *)(&(s->bl_desc))); + /* opt_len now includes the length of the tree representations, except the + * lengths of the bit lengths codes and the 5 + 5 + 4 bits for the counts. + */ + + /* Determine the number of bit length codes to send. The pkzip format + * requires that at least 4 bit length codes be sent. (appnote.txt says + * 3 but the actual value used is 4.) + */ + for (max_blindex = BL_CODES-1; max_blindex >= 3; max_blindex--) { + if (s->bl_tree[bl_order[max_blindex]].Len != 0) break; + } + /* Update opt_len to include the bit length tree and counts */ + s->opt_len += 3*((ulg)max_blindex + 1) + 5 + 5 + 4; + Tracev((stderr, "\ndyn trees: dyn %ld, stat %ld", + s->opt_len, s->static_len)); + + return max_blindex; +} + +/* =========================================================================== + * Send the header for a block using dynamic Huffman trees: the counts, the + * lengths of the bit length codes, the literal tree and the distance tree. + * IN assertion: lcodes >= 257, dcodes >= 1, blcodes >= 4. + */ +local void send_all_trees(s, lcodes, dcodes, blcodes) + deflate_state *s; + int lcodes, dcodes, blcodes; /* number of codes for each tree */ +{ + int rank; /* index in bl_order */ + + Assert (lcodes >= 257 && dcodes >= 1 && blcodes >= 4, "not enough codes"); + Assert (lcodes <= L_CODES && dcodes <= D_CODES && blcodes <= BL_CODES, + "too many codes"); + Tracev((stderr, "\nbl counts: ")); + send_bits(s, lcodes - 257, 5); /* not +255 as stated in appnote.txt */ + send_bits(s, dcodes - 1, 5); + send_bits(s, blcodes - 4, 4); /* not -3 as stated in appnote.txt */ + for (rank = 0; rank < blcodes; rank++) { + Tracev((stderr, "\nbl code %2d ", bl_order[rank])); + send_bits(s, s->bl_tree[bl_order[rank]].Len, 3); + } + Tracev((stderr, "\nbl tree: sent %ld", s->bits_sent)); + + send_tree(s, (ct_data *)s->dyn_ltree, lcodes - 1); /* literal tree */ + Tracev((stderr, "\nlit tree: sent %ld", s->bits_sent)); + + send_tree(s, (ct_data *)s->dyn_dtree, dcodes - 1); /* distance tree */ + Tracev((stderr, "\ndist tree: sent %ld", s->bits_sent)); +} + +/* =========================================================================== + * Send a stored block + */ +void ZLIB_INTERNAL _tr_stored_block(s, buf, stored_len, last) + deflate_state *s; + charf *buf; /* input block */ + ulg stored_len; /* length of input block */ + int last; /* one if this is the last block for a file */ +{ + send_bits(s, (STORED_BLOCK<<1) + last, 3); /* send block type */ + bi_windup(s); /* align on byte boundary */ + put_short(s, (ush)stored_len); + put_short(s, (ush)~stored_len); + if (stored_len) + zmemcpy(s->pending_buf + s->pending, (Bytef *)buf, stored_len); + s->pending += stored_len; +#ifdef ZLIB_DEBUG + s->compressed_len = (s->compressed_len + 3 + 7) & (ulg)~7L; + s->compressed_len += (stored_len + 4) << 3; + s->bits_sent += 2*16; + s->bits_sent += stored_len << 3; +#endif +} + +/* =========================================================================== + * Flush the bits in the bit buffer to pending output (leaves at most 7 bits) + */ +void ZLIB_INTERNAL _tr_flush_bits(s) + deflate_state *s; +{ + bi_flush(s); +} + +/* =========================================================================== + * Send one empty static block to give enough lookahead for inflate. + * This takes 10 bits, of which 7 may remain in the bit buffer. + */ +void ZLIB_INTERNAL _tr_align(s) + deflate_state *s; +{ + send_bits(s, STATIC_TREES<<1, 3); + send_code(s, END_BLOCK, static_ltree); +#ifdef ZLIB_DEBUG + s->compressed_len += 10L; /* 3 for block type, 7 for EOB */ +#endif + bi_flush(s); +} + +/* =========================================================================== + * Determine the best encoding for the current block: dynamic trees, static + * trees or store, and write out the encoded block. + */ +void ZLIB_INTERNAL _tr_flush_block(s, buf, stored_len, last) + deflate_state *s; + charf *buf; /* input block, or NULL if too old */ + ulg stored_len; /* length of input block */ + int last; /* one if this is the last block for a file */ +{ + ulg opt_lenb, static_lenb; /* opt_len and static_len in bytes */ + int max_blindex = 0; /* index of last bit length code of non zero freq */ + + /* Build the Huffman trees unless a stored block is forced */ + if (s->level > 0) { + + /* Check if the file is binary or text */ + if (s->strm->data_type == Z_UNKNOWN) + s->strm->data_type = detect_data_type(s); + + /* Construct the literal and distance trees */ + build_tree(s, (tree_desc *)(&(s->l_desc))); + Tracev((stderr, "\nlit data: dyn %ld, stat %ld", s->opt_len, + s->static_len)); + + build_tree(s, (tree_desc *)(&(s->d_desc))); + Tracev((stderr, "\ndist data: dyn %ld, stat %ld", s->opt_len, + s->static_len)); + /* At this point, opt_len and static_len are the total bit lengths of + * the compressed block data, excluding the tree representations. + */ + + /* Build the bit length tree for the above two trees, and get the index + * in bl_order of the last bit length code to send. + */ + max_blindex = build_bl_tree(s); + + /* Determine the best encoding. Compute the block lengths in bytes. */ + opt_lenb = (s->opt_len + 3 + 7) >> 3; + static_lenb = (s->static_len + 3 + 7) >> 3; + + Tracev((stderr, "\nopt %lu(%lu) stat %lu(%lu) stored %lu lit %u ", + opt_lenb, s->opt_len, static_lenb, s->static_len, stored_len, + s->sym_next / 3)); + +#ifndef FORCE_STATIC + if (static_lenb <= opt_lenb || s->strategy == Z_FIXED) +#endif + opt_lenb = static_lenb; + + } else { + Assert(buf != (char*)0, "lost buf"); + opt_lenb = static_lenb = stored_len + 5; /* force a stored block */ + } + +#ifdef FORCE_STORED + if (buf != (char*)0) { /* force stored block */ +#else + if (stored_len + 4 <= opt_lenb && buf != (char*)0) { + /* 4: two words for the lengths */ +#endif + /* The test buf != NULL is only necessary if LIT_BUFSIZE > WSIZE. + * Otherwise we can't have processed more than WSIZE input bytes since + * the last block flush, because compression would have been + * successful. If LIT_BUFSIZE <= WSIZE, it is never too late to + * transform a block into a stored block. + */ + _tr_stored_block(s, buf, stored_len, last); + + } else if (static_lenb == opt_lenb) { + send_bits(s, (STATIC_TREES<<1) + last, 3); + compress_block(s, (const ct_data *)static_ltree, + (const ct_data *)static_dtree); +#ifdef ZLIB_DEBUG + s->compressed_len += 3 + s->static_len; +#endif + } else { + send_bits(s, (DYN_TREES<<1) + last, 3); + send_all_trees(s, s->l_desc.max_code + 1, s->d_desc.max_code + 1, + max_blindex + 1); + compress_block(s, (const ct_data *)s->dyn_ltree, + (const ct_data *)s->dyn_dtree); +#ifdef ZLIB_DEBUG + s->compressed_len += 3 + s->opt_len; +#endif + } + Assert (s->compressed_len == s->bits_sent, "bad compressed size"); + /* The above check is made mod 2^32, for files larger than 512 MB + * and uLong implemented on 32 bits. + */ + init_block(s); + + if (last) { + bi_windup(s); +#ifdef ZLIB_DEBUG + s->compressed_len += 7; /* align on byte boundary */ +#endif + } + Tracev((stderr,"\ncomprlen %lu(%lu) ", s->compressed_len >> 3, + s->compressed_len - 7*last)); +} + +/* =========================================================================== + * Save the match info and tally the frequency counts. Return true if + * the current block must be flushed. + */ +int ZLIB_INTERNAL _tr_tally(s, dist, lc) + deflate_state *s; + unsigned dist; /* distance of matched string */ + unsigned lc; /* match length - MIN_MATCH or unmatched char (dist==0) */ +{ + s->sym_buf[s->sym_next++] = (uch)dist; + s->sym_buf[s->sym_next++] = (uch)(dist >> 8); + s->sym_buf[s->sym_next++] = (uch)lc; + if (dist == 0) { + /* lc is the unmatched char */ + s->dyn_ltree[lc].Freq++; + } else { + s->matches++; + /* Here, lc is the match length - MIN_MATCH */ + dist--; /* dist = match distance - 1 */ + Assert((ush)dist < (ush)MAX_DIST(s) && + (ush)lc <= (ush)(MAX_MATCH-MIN_MATCH) && + (ush)d_code(dist) < (ush)D_CODES, "_tr_tally: bad match"); + + s->dyn_ltree[_length_code[lc] + LITERALS + 1].Freq++; + s->dyn_dtree[d_code(dist)].Freq++; + } + return (s->sym_next == s->sym_end); +} + +/* =========================================================================== + * Send the block data compressed using the given Huffman trees + */ +local void compress_block(s, ltree, dtree) + deflate_state *s; + const ct_data *ltree; /* literal tree */ + const ct_data *dtree; /* distance tree */ +{ + unsigned dist; /* distance of matched string */ + int lc; /* match length or unmatched char (if dist == 0) */ + unsigned sx = 0; /* running index in sym_buf */ + unsigned code; /* the code to send */ + int extra; /* number of extra bits to send */ + + if (s->sym_next != 0) do { + dist = s->sym_buf[sx++] & 0xff; + dist += (unsigned)(s->sym_buf[sx++] & 0xff) << 8; + lc = s->sym_buf[sx++]; + if (dist == 0) { + send_code(s, lc, ltree); /* send a literal byte */ + Tracecv(isgraph(lc), (stderr," '%c' ", lc)); + } else { + /* Here, lc is the match length - MIN_MATCH */ + code = _length_code[lc]; + send_code(s, code + LITERALS + 1, ltree); /* send length code */ + extra = extra_lbits[code]; + if (extra != 0) { + lc -= base_length[code]; + send_bits(s, lc, extra); /* send the extra length bits */ + } + dist--; /* dist is now the match distance - 1 */ + code = d_code(dist); + Assert (code < D_CODES, "bad d_code"); + + send_code(s, code, dtree); /* send the distance code */ + extra = extra_dbits[code]; + if (extra != 0) { + dist -= (unsigned)base_dist[code]; + send_bits(s, dist, extra); /* send the extra distance bits */ + } + } /* literal or match pair ? */ + + /* Check that the overlay between pending_buf and sym_buf is ok: */ + Assert(s->pending < s->lit_bufsize + sx, "pendingBuf overflow"); + + } while (sx < s->sym_next); + + send_code(s, END_BLOCK, ltree); +} + +/* =========================================================================== + * Check if the data type is TEXT or BINARY, using the following algorithm: + * - TEXT if the two conditions below are satisfied: + * a) There are no non-portable control characters belonging to the + * "block list" (0..6, 14..25, 28..31). + * b) There is at least one printable character belonging to the + * "allow list" (9 {TAB}, 10 {LF}, 13 {CR}, 32..255). + * - BINARY otherwise. + * - The following partially-portable control characters form a + * "gray list" that is ignored in this detection algorithm: + * (7 {BEL}, 8 {BS}, 11 {VT}, 12 {FF}, 26 {SUB}, 27 {ESC}). + * IN assertion: the fields Freq of dyn_ltree are set. + */ +local int detect_data_type(s) + deflate_state *s; +{ + /* block_mask is the bit mask of block-listed bytes + * set bits 0..6, 14..25, and 28..31 + * 0xf3ffc07f = binary 11110011111111111100000001111111 + */ + unsigned long block_mask = 0xf3ffc07fUL; + int n; + + /* Check for non-textual ("block-listed") bytes. */ + for (n = 0; n <= 31; n++, block_mask >>= 1) + if ((block_mask & 1) && (s->dyn_ltree[n].Freq != 0)) + return Z_BINARY; + + /* Check for textual ("allow-listed") bytes. */ + if (s->dyn_ltree[9].Freq != 0 || s->dyn_ltree[10].Freq != 0 + || s->dyn_ltree[13].Freq != 0) + return Z_TEXT; + for (n = 32; n < LITERALS; n++) + if (s->dyn_ltree[n].Freq != 0) + return Z_TEXT; + + /* There are no "block-listed" or "allow-listed" bytes: + * this stream either is empty or has tolerated ("gray-listed") bytes only. + */ + return Z_BINARY; +} + +/* =========================================================================== + * Reverse the first len bits of a code, using straightforward code (a faster + * method would use a table) + * IN assertion: 1 <= len <= 15 + */ +local unsigned bi_reverse(code, len) + unsigned code; /* the value to invert */ + int len; /* its bit length */ +{ + register unsigned res = 0; + do { + res |= code & 1; + code >>= 1, res <<= 1; + } while (--len > 0); + return res >> 1; +} + +/* =========================================================================== + * Flush the bit buffer, keeping at most 7 bits in it. + */ +local void bi_flush(s) + deflate_state *s; +{ + if (s->bi_valid == 16) { + put_short(s, s->bi_buf); + s->bi_buf = 0; + s->bi_valid = 0; + } else if (s->bi_valid >= 8) { + put_byte(s, (Byte)s->bi_buf); + s->bi_buf >>= 8; + s->bi_valid -= 8; + } +} + +/* =========================================================================== + * Flush the bit buffer and align the output on a byte boundary + */ +local void bi_windup(s) + deflate_state *s; +{ + if (s->bi_valid > 8) { + put_short(s, s->bi_buf); + } else if (s->bi_valid > 0) { + put_byte(s, (Byte)s->bi_buf); + } + s->bi_buf = 0; + s->bi_valid = 0; +#ifdef ZLIB_DEBUG + s->bits_sent = (s->bits_sent + 7) & ~7; +#endif +} diff --git a/lib/os/windows/zlib-1.2.13/trees.h b/lib/os/windows/zlib-1.2.13/trees.h new file mode 100644 index 000000000000..d35639d82a27 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/trees.h @@ -0,0 +1,128 @@ +/* header created automatically with -DGEN_TREES_H */ + +local const ct_data static_ltree[L_CODES+2] = { +{{ 12},{ 8}}, {{140},{ 8}}, {{ 76},{ 8}}, {{204},{ 8}}, {{ 44},{ 8}}, +{{172},{ 8}}, {{108},{ 8}}, {{236},{ 8}}, {{ 28},{ 8}}, {{156},{ 8}}, +{{ 92},{ 8}}, {{220},{ 8}}, {{ 60},{ 8}}, {{188},{ 8}}, {{124},{ 8}}, +{{252},{ 8}}, {{ 2},{ 8}}, {{130},{ 8}}, {{ 66},{ 8}}, {{194},{ 8}}, +{{ 34},{ 8}}, {{162},{ 8}}, {{ 98},{ 8}}, {{226},{ 8}}, {{ 18},{ 8}}, +{{146},{ 8}}, {{ 82},{ 8}}, {{210},{ 8}}, {{ 50},{ 8}}, {{178},{ 8}}, +{{114},{ 8}}, {{242},{ 8}}, {{ 10},{ 8}}, {{138},{ 8}}, {{ 74},{ 8}}, +{{202},{ 8}}, {{ 42},{ 8}}, {{170},{ 8}}, {{106},{ 8}}, {{234},{ 8}}, +{{ 26},{ 8}}, {{154},{ 8}}, {{ 90},{ 8}}, {{218},{ 8}}, {{ 58},{ 8}}, +{{186},{ 8}}, {{122},{ 8}}, {{250},{ 8}}, {{ 6},{ 8}}, {{134},{ 8}}, +{{ 70},{ 8}}, {{198},{ 8}}, {{ 38},{ 8}}, {{166},{ 8}}, {{102},{ 8}}, +{{230},{ 8}}, {{ 22},{ 8}}, {{150},{ 8}}, {{ 86},{ 8}}, {{214},{ 8}}, +{{ 54},{ 8}}, {{182},{ 8}}, {{118},{ 8}}, {{246},{ 8}}, {{ 14},{ 8}}, +{{142},{ 8}}, {{ 78},{ 8}}, {{206},{ 8}}, {{ 46},{ 8}}, {{174},{ 8}}, +{{110},{ 8}}, {{238},{ 8}}, {{ 30},{ 8}}, {{158},{ 8}}, {{ 94},{ 8}}, +{{222},{ 8}}, {{ 62},{ 8}}, {{190},{ 8}}, {{126},{ 8}}, {{254},{ 8}}, +{{ 1},{ 8}}, {{129},{ 8}}, {{ 65},{ 8}}, {{193},{ 8}}, {{ 33},{ 8}}, +{{161},{ 8}}, {{ 97},{ 8}}, {{225},{ 8}}, {{ 17},{ 8}}, {{145},{ 8}}, +{{ 81},{ 8}}, {{209},{ 8}}, {{ 49},{ 8}}, {{177},{ 8}}, {{113},{ 8}}, +{{241},{ 8}}, {{ 9},{ 8}}, {{137},{ 8}}, {{ 73},{ 8}}, {{201},{ 8}}, +{{ 41},{ 8}}, {{169},{ 8}}, {{105},{ 8}}, {{233},{ 8}}, {{ 25},{ 8}}, +{{153},{ 8}}, {{ 89},{ 8}}, {{217},{ 8}}, {{ 57},{ 8}}, {{185},{ 8}}, +{{121},{ 8}}, {{249},{ 8}}, {{ 5},{ 8}}, {{133},{ 8}}, {{ 69},{ 8}}, +{{197},{ 8}}, {{ 37},{ 8}}, {{165},{ 8}}, {{101},{ 8}}, {{229},{ 8}}, +{{ 21},{ 8}}, {{149},{ 8}}, {{ 85},{ 8}}, {{213},{ 8}}, {{ 53},{ 8}}, +{{181},{ 8}}, {{117},{ 8}}, {{245},{ 8}}, {{ 13},{ 8}}, {{141},{ 8}}, +{{ 77},{ 8}}, {{205},{ 8}}, {{ 45},{ 8}}, {{173},{ 8}}, {{109},{ 8}}, +{{237},{ 8}}, {{ 29},{ 8}}, {{157},{ 8}}, {{ 93},{ 8}}, {{221},{ 8}}, +{{ 61},{ 8}}, {{189},{ 8}}, {{125},{ 8}}, {{253},{ 8}}, {{ 19},{ 9}}, +{{275},{ 9}}, {{147},{ 9}}, {{403},{ 9}}, {{ 83},{ 9}}, {{339},{ 9}}, +{{211},{ 9}}, {{467},{ 9}}, {{ 51},{ 9}}, {{307},{ 9}}, {{179},{ 9}}, +{{435},{ 9}}, {{115},{ 9}}, {{371},{ 9}}, {{243},{ 9}}, {{499},{ 9}}, +{{ 11},{ 9}}, {{267},{ 9}}, {{139},{ 9}}, {{395},{ 9}}, {{ 75},{ 9}}, +{{331},{ 9}}, {{203},{ 9}}, {{459},{ 9}}, {{ 43},{ 9}}, {{299},{ 9}}, +{{171},{ 9}}, {{427},{ 9}}, {{107},{ 9}}, {{363},{ 9}}, {{235},{ 9}}, +{{491},{ 9}}, {{ 27},{ 9}}, {{283},{ 9}}, {{155},{ 9}}, {{411},{ 9}}, +{{ 91},{ 9}}, {{347},{ 9}}, {{219},{ 9}}, {{475},{ 9}}, {{ 59},{ 9}}, +{{315},{ 9}}, {{187},{ 9}}, {{443},{ 9}}, {{123},{ 9}}, {{379},{ 9}}, +{{251},{ 9}}, {{507},{ 9}}, {{ 7},{ 9}}, {{263},{ 9}}, {{135},{ 9}}, +{{391},{ 9}}, {{ 71},{ 9}}, {{327},{ 9}}, {{199},{ 9}}, {{455},{ 9}}, +{{ 39},{ 9}}, {{295},{ 9}}, {{167},{ 9}}, {{423},{ 9}}, {{103},{ 9}}, +{{359},{ 9}}, {{231},{ 9}}, {{487},{ 9}}, {{ 23},{ 9}}, {{279},{ 9}}, +{{151},{ 9}}, {{407},{ 9}}, {{ 87},{ 9}}, {{343},{ 9}}, {{215},{ 9}}, +{{471},{ 9}}, {{ 55},{ 9}}, {{311},{ 9}}, {{183},{ 9}}, {{439},{ 9}}, +{{119},{ 9}}, {{375},{ 9}}, {{247},{ 9}}, {{503},{ 9}}, {{ 15},{ 9}}, +{{271},{ 9}}, {{143},{ 9}}, {{399},{ 9}}, {{ 79},{ 9}}, {{335},{ 9}}, +{{207},{ 9}}, {{463},{ 9}}, {{ 47},{ 9}}, {{303},{ 9}}, {{175},{ 9}}, +{{431},{ 9}}, {{111},{ 9}}, {{367},{ 9}}, {{239},{ 9}}, {{495},{ 9}}, +{{ 31},{ 9}}, {{287},{ 9}}, {{159},{ 9}}, {{415},{ 9}}, {{ 95},{ 9}}, +{{351},{ 9}}, {{223},{ 9}}, {{479},{ 9}}, {{ 63},{ 9}}, {{319},{ 9}}, +{{191},{ 9}}, {{447},{ 9}}, {{127},{ 9}}, {{383},{ 9}}, {{255},{ 9}}, +{{511},{ 9}}, {{ 0},{ 7}}, {{ 64},{ 7}}, {{ 32},{ 7}}, {{ 96},{ 7}}, +{{ 16},{ 7}}, {{ 80},{ 7}}, {{ 48},{ 7}}, {{112},{ 7}}, {{ 8},{ 7}}, +{{ 72},{ 7}}, {{ 40},{ 7}}, {{104},{ 7}}, {{ 24},{ 7}}, {{ 88},{ 7}}, +{{ 56},{ 7}}, {{120},{ 7}}, {{ 4},{ 7}}, {{ 68},{ 7}}, {{ 36},{ 7}}, +{{100},{ 7}}, {{ 20},{ 7}}, {{ 84},{ 7}}, {{ 52},{ 7}}, {{116},{ 7}}, +{{ 3},{ 8}}, {{131},{ 8}}, {{ 67},{ 8}}, {{195},{ 8}}, {{ 35},{ 8}}, +{{163},{ 8}}, {{ 99},{ 8}}, {{227},{ 8}} +}; + +local const ct_data static_dtree[D_CODES] = { +{{ 0},{ 5}}, {{16},{ 5}}, {{ 8},{ 5}}, {{24},{ 5}}, {{ 4},{ 5}}, +{{20},{ 5}}, {{12},{ 5}}, {{28},{ 5}}, {{ 2},{ 5}}, {{18},{ 5}}, +{{10},{ 5}}, {{26},{ 5}}, {{ 6},{ 5}}, {{22},{ 5}}, {{14},{ 5}}, +{{30},{ 5}}, {{ 1},{ 5}}, {{17},{ 5}}, {{ 9},{ 5}}, {{25},{ 5}}, +{{ 5},{ 5}}, {{21},{ 5}}, {{13},{ 5}}, {{29},{ 5}}, {{ 3},{ 5}}, +{{19},{ 5}}, {{11},{ 5}}, {{27},{ 5}}, {{ 7},{ 5}}, {{23},{ 5}} +}; + +const uch ZLIB_INTERNAL _dist_code[DIST_CODE_LEN] = { + 0, 1, 2, 3, 4, 4, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7, 8, 8, 8, 8, + 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, +10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, +11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, +12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, +13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, +13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, +14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, +14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, +14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, +15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, +15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, +15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 0, 0, 16, 17, +18, 18, 19, 19, 20, 20, 20, 20, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, +23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, +24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, +26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, +26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, +27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, +27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, +28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, +28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, +28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, +29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, +29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, +29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29 +}; + +const uch ZLIB_INTERNAL _length_code[MAX_MATCH-MIN_MATCH+1]= { + 0, 1, 2, 3, 4, 5, 6, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 12, 12, +13, 13, 13, 13, 14, 14, 14, 14, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, +17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, +19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, +21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, +22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, +23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, +24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, +25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, +25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, +26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, +26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, +27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28 +}; + +local const int base_length[LENGTH_CODES] = { +0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 16, 20, 24, 28, 32, 40, 48, 56, +64, 80, 96, 112, 128, 160, 192, 224, 0 +}; + +local const int base_dist[D_CODES] = { + 0, 1, 2, 3, 4, 6, 8, 12, 16, 24, + 32, 48, 64, 96, 128, 192, 256, 384, 512, 768, + 1024, 1536, 2048, 3072, 4096, 6144, 8192, 12288, 16384, 24576 +}; + diff --git a/lib/os/windows/zlib-1.2.13/uncompr.c b/lib/os/windows/zlib-1.2.13/uncompr.c new file mode 100644 index 000000000000..f9532f46c1a6 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/uncompr.c @@ -0,0 +1,93 @@ +/* uncompr.c -- decompress a memory buffer + * Copyright (C) 1995-2003, 2010, 2014, 2016 Jean-loup Gailly, Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* @(#) $Id$ */ + +#define ZLIB_INTERNAL +#include "zlib.h" + +/* =========================================================================== + Decompresses the source buffer into the destination buffer. *sourceLen is + the byte length of the source buffer. Upon entry, *destLen is the total size + of the destination buffer, which must be large enough to hold the entire + uncompressed data. (The size of the uncompressed data must have been saved + previously by the compressor and transmitted to the decompressor by some + mechanism outside the scope of this compression library.) Upon exit, + *destLen is the size of the decompressed data and *sourceLen is the number + of source bytes consumed. Upon return, source + *sourceLen points to the + first unused input byte. + + uncompress returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_BUF_ERROR if there was not enough room in the output buffer, or + Z_DATA_ERROR if the input data was corrupted, including if the input data is + an incomplete zlib stream. +*/ +int ZEXPORT uncompress2(dest, destLen, source, sourceLen) + Bytef *dest; + uLongf *destLen; + const Bytef *source; + uLong *sourceLen; +{ + z_stream stream; + int err; + const uInt max = (uInt)-1; + uLong len, left; + Byte buf[1]; /* for detection of incomplete stream when *destLen == 0 */ + + len = *sourceLen; + if (*destLen) { + left = *destLen; + *destLen = 0; + } + else { + left = 1; + dest = buf; + } + + stream.next_in = (z_const Bytef *)source; + stream.avail_in = 0; + stream.zalloc = (alloc_func)0; + stream.zfree = (free_func)0; + stream.opaque = (voidpf)0; + + err = inflateInit(&stream); + if (err != Z_OK) return err; + + stream.next_out = dest; + stream.avail_out = 0; + + do { + if (stream.avail_out == 0) { + stream.avail_out = left > (uLong)max ? max : (uInt)left; + left -= stream.avail_out; + } + if (stream.avail_in == 0) { + stream.avail_in = len > (uLong)max ? max : (uInt)len; + len -= stream.avail_in; + } + err = inflate(&stream, Z_NO_FLUSH); + } while (err == Z_OK); + + *sourceLen -= len + stream.avail_in; + if (dest != buf) + *destLen = stream.total_out; + else if (stream.total_out && err == Z_BUF_ERROR) + left = 1; + + inflateEnd(&stream); + return err == Z_STREAM_END ? Z_OK : + err == Z_NEED_DICT ? Z_DATA_ERROR : + err == Z_BUF_ERROR && left + stream.avail_out ? Z_DATA_ERROR : + err; +} + +int ZEXPORT uncompress(dest, destLen, source, sourceLen) + Bytef *dest; + uLongf *destLen; + const Bytef *source; + uLong sourceLen; +{ + return uncompress2(dest, destLen, source, &sourceLen); +} diff --git a/lib/os/windows/zlib-1.2.13/zconf.h b/lib/os/windows/zlib-1.2.13/zconf.h new file mode 100644 index 000000000000..590585d1686e --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/zconf.h @@ -0,0 +1,552 @@ +/* zconf.h -- configuration of the zlib compression library + * Copyright (C) 1995-2016 Jean-loup Gailly, Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* @(#) $Id$ */ + +#ifndef ZCONF_H +#define ZCONF_H + +/* + * If you *really* need a unique prefix for all types and library functions, + * compile with -DZ_PREFIX. The "standard" zlib should be compiled without it. + * Even better than compiling with -DZ_PREFIX would be to use configure to set + * this permanently in zconf.h using "./configure --zprefix". + */ +#ifdef Z_PREFIX /* may be set to #if 1 by ./configure */ +# define Z_PREFIX_SET + +/* all linked symbols and init macros */ +# define _dist_code z__dist_code +# define _length_code z__length_code +# define _tr_align z__tr_align +# define _tr_flush_bits z__tr_flush_bits +# define _tr_flush_block z__tr_flush_block +# define _tr_init z__tr_init +# define _tr_stored_block z__tr_stored_block +# define _tr_tally z__tr_tally +# define adler32 z_adler32 +# define adler32_combine z_adler32_combine +# define adler32_combine64 z_adler32_combine64 +# define adler32_z z_adler32_z +# ifndef Z_SOLO +# define compress z_compress +# define compress2 z_compress2 +# define compressBound z_compressBound +# endif +# define crc32 z_crc32 +# define crc32_combine z_crc32_combine +# define crc32_combine64 z_crc32_combine64 +# define crc32_combine_gen z_crc32_combine_gen +# define crc32_combine_gen64 z_crc32_combine_gen64 +# define crc32_combine_op z_crc32_combine_op +# define crc32_z z_crc32_z +# define deflate z_deflate +# define deflateBound z_deflateBound +# define deflateCopy z_deflateCopy +# define deflateEnd z_deflateEnd +# define deflateGetDictionary z_deflateGetDictionary +# define deflateInit z_deflateInit +# define deflateInit2 z_deflateInit2 +# define deflateInit2_ z_deflateInit2_ +# define deflateInit_ z_deflateInit_ +# define deflateParams z_deflateParams +# define deflatePending z_deflatePending +# define deflatePrime z_deflatePrime +# define deflateReset z_deflateReset +# define deflateResetKeep z_deflateResetKeep +# define deflateSetDictionary z_deflateSetDictionary +# define deflateSetHeader z_deflateSetHeader +# define deflateTune z_deflateTune +# define deflate_copyright z_deflate_copyright +# define get_crc_table z_get_crc_table +# ifndef Z_SOLO +# define gz_error z_gz_error +# define gz_intmax z_gz_intmax +# define gz_strwinerror z_gz_strwinerror +# define gzbuffer z_gzbuffer +# define gzclearerr z_gzclearerr +# define gzclose z_gzclose +# define gzclose_r z_gzclose_r +# define gzclose_w z_gzclose_w +# define gzdirect z_gzdirect +# define gzdopen z_gzdopen +# define gzeof z_gzeof +# define gzerror z_gzerror +# define gzflush z_gzflush +# define gzfread z_gzfread +# define gzfwrite z_gzfwrite +# define gzgetc z_gzgetc +# define gzgetc_ z_gzgetc_ +# define gzgets z_gzgets +# define gzoffset z_gzoffset +# define gzoffset64 z_gzoffset64 +# define gzopen z_gzopen +# define gzopen64 z_gzopen64 +# ifdef _WIN32 +# define gzopen_w z_gzopen_w +# endif +# define gzprintf z_gzprintf +# define gzputc z_gzputc +# define gzputs z_gzputs +# define gzread z_gzread +# define gzrewind z_gzrewind +# define gzseek z_gzseek +# define gzseek64 z_gzseek64 +# define gzsetparams z_gzsetparams +# define gztell z_gztell +# define gztell64 z_gztell64 +# define gzungetc z_gzungetc +# define gzvprintf z_gzvprintf +# define gzwrite z_gzwrite +# endif +# define inflate z_inflate +# define inflateBack z_inflateBack +# define inflateBackEnd z_inflateBackEnd +# define inflateBackInit z_inflateBackInit +# define inflateBackInit_ z_inflateBackInit_ +# define inflateCodesUsed z_inflateCodesUsed +# define inflateCopy z_inflateCopy +# define inflateEnd z_inflateEnd +# define inflateGetDictionary z_inflateGetDictionary +# define inflateGetHeader z_inflateGetHeader +# define inflateInit z_inflateInit +# define inflateInit2 z_inflateInit2 +# define inflateInit2_ z_inflateInit2_ +# define inflateInit_ z_inflateInit_ +# define inflateMark z_inflateMark +# define inflatePrime z_inflatePrime +# define inflateReset z_inflateReset +# define inflateReset2 z_inflateReset2 +# define inflateResetKeep z_inflateResetKeep +# define inflateSetDictionary z_inflateSetDictionary +# define inflateSync z_inflateSync +# define inflateSyncPoint z_inflateSyncPoint +# define inflateUndermine z_inflateUndermine +# define inflateValidate z_inflateValidate +# define inflate_copyright z_inflate_copyright +# define inflate_fast z_inflate_fast +# define inflate_table z_inflate_table +# ifndef Z_SOLO +# define uncompress z_uncompress +# define uncompress2 z_uncompress2 +# endif +# define zError z_zError +# ifndef Z_SOLO +# define zcalloc z_zcalloc +# define zcfree z_zcfree +# endif +# define zlibCompileFlags z_zlibCompileFlags +# define zlibVersion z_zlibVersion + +/* all zlib typedefs in zlib.h and zconf.h */ +# define Byte z_Byte +# define Bytef z_Bytef +# define alloc_func z_alloc_func +# define charf z_charf +# define free_func z_free_func +# ifndef Z_SOLO +# define gzFile z_gzFile +# endif +# define gz_header z_gz_header +# define gz_headerp z_gz_headerp +# define in_func z_in_func +# define intf z_intf +# define out_func z_out_func +# define uInt z_uInt +# define uIntf z_uIntf +# define uLong z_uLong +# define uLongf z_uLongf +# define voidp z_voidp +# define voidpc z_voidpc +# define voidpf z_voidpf + +/* all zlib structs in zlib.h and zconf.h */ +# define gz_header_s z_gz_header_s +# define internal_state z_internal_state + +#endif + +#if defined(__MSDOS__) && !defined(MSDOS) +# define MSDOS +#endif +#if (defined(OS_2) || defined(__OS2__)) && !defined(OS2) +# define OS2 +#endif +#if defined(_WINDOWS) && !defined(WINDOWS) +# define WINDOWS +#endif +#if defined(_WIN32) || defined(_WIN32_WCE) || defined(__WIN32__) +# ifndef WIN32 +# define WIN32 +# endif +#endif +#if (defined(MSDOS) || defined(OS2) || defined(WINDOWS)) && !defined(WIN32) +# if !defined(__GNUC__) && !defined(__FLAT__) && !defined(__386__) +# ifndef SYS16BIT +# define SYS16BIT +# endif +# endif +#endif + +/* + * Compile with -DMAXSEG_64K if the alloc function cannot allocate more + * than 64k bytes at a time (needed on systems with 16-bit int). + */ +#ifdef SYS16BIT +# define MAXSEG_64K +#endif +#ifdef MSDOS +# define UNALIGNED_OK +#endif + +#ifdef __STDC_VERSION__ +# ifndef STDC +# define STDC +# endif +# if __STDC_VERSION__ >= 199901L +# ifndef STDC99 +# define STDC99 +# endif +# endif +#endif +#if !defined(STDC) && (defined(__STDC__) || defined(__cplusplus)) +# define STDC +#endif +#if !defined(STDC) && (defined(__GNUC__) || defined(__BORLANDC__)) +# define STDC +#endif +#if !defined(STDC) && (defined(MSDOS) || defined(WINDOWS) || defined(WIN32)) +# define STDC +#endif +#if !defined(STDC) && (defined(OS2) || defined(__HOS_AIX__)) +# define STDC +#endif + +#if defined(__OS400__) && !defined(STDC) /* iSeries (formerly AS/400). */ +# define STDC +#endif + +#ifndef STDC +# ifndef const /* cannot use !defined(STDC) && !defined(const) on Mac */ +# define const /* note: need a more gentle solution here */ +# endif +#endif + +#if defined(ZLIB_CONST) && !defined(z_const) +# define z_const const +#else +# define z_const +#endif + +#ifdef Z_SOLO + typedef unsigned long z_size_t; +#else +# define z_longlong long long +# if defined(NO_SIZE_T) + typedef unsigned NO_SIZE_T z_size_t; +# elif defined(STDC) +# include + typedef size_t z_size_t; +# else + typedef unsigned long z_size_t; +# endif +# undef z_longlong +#endif + +/* Maximum value for memLevel in deflateInit2 */ +#ifndef MAX_MEM_LEVEL +# ifdef MAXSEG_64K +# define MAX_MEM_LEVEL 8 +# else +# define MAX_MEM_LEVEL 9 +# endif +#endif + +/* Maximum value for windowBits in deflateInit2 and inflateInit2. + * WARNING: reducing MAX_WBITS makes minigzip unable to extract .gz files + * created by gzip. (Files created by minigzip can still be extracted by + * gzip.) + */ +#ifndef MAX_WBITS +# define MAX_WBITS 15 /* 32K LZ77 window */ +#endif + +/* The memory requirements for deflate are (in bytes): + (1 << (windowBits+2)) + (1 << (memLevel+9)) + that is: 128K for windowBits=15 + 128K for memLevel = 8 (default values) + plus a few kilobytes for small objects. For example, if you want to reduce + the default memory requirements from 256K to 128K, compile with + make CFLAGS="-O -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7" + Of course this will generally degrade compression (there's no free lunch). + + The memory requirements for inflate are (in bytes) 1 << windowBits + that is, 32K for windowBits=15 (default value) plus about 7 kilobytes + for small objects. +*/ + + /* Type declarations */ + +#ifndef OF /* function prototypes */ +# ifdef STDC +# define OF(args) args +# else +# define OF(args) () +# endif +#endif + +#ifndef Z_ARG /* function prototypes for stdarg */ +# if defined(STDC) || defined(Z_HAVE_STDARG_H) +# define Z_ARG(args) args +# else +# define Z_ARG(args) () +# endif +#endif + +/* The following definitions for FAR are needed only for MSDOS mixed + * model programming (small or medium model with some far allocations). + * This was tested only with MSC; for other MSDOS compilers you may have + * to define NO_MEMCPY in zutil.h. If you don't need the mixed model, + * just define FAR to be empty. + */ +#ifdef SYS16BIT +# if defined(M_I86SM) || defined(M_I86MM) + /* MSC small or medium model */ +# define SMALL_MEDIUM +# ifdef _MSC_VER +# define FAR _far +# else +# define FAR far +# endif +# endif +# if (defined(__SMALL__) || defined(__MEDIUM__)) + /* Turbo C small or medium model */ +# define SMALL_MEDIUM +# ifdef __BORLANDC__ +# define FAR _far +# else +# define FAR far +# endif +# endif +#endif + +#if defined(WINDOWS) || defined(WIN32) +#undef ZLIB_WINAPI + + /* If building or using zlib as a DLL, define ZLIB_DLL. + * This is not mandatory, but it offers a little performance increase. + */ +# ifdef ZLIB_DLL +# if defined(WIN32) && (!defined(__BORLANDC__) || (__BORLANDC__ >= 0x500)) +# ifdef ZLIB_INTERNAL +# define ZEXTERN extern __declspec(dllexport) +# else +# define ZEXTERN extern __declspec(dllimport) +# endif +# endif +# endif /* ZLIB_DLL */ + /* If building or using zlib with the WINAPI/WINAPIV calling convention, + * define ZLIB_WINAPI. + * Caution: the standard ZLIB1.DLL is NOT compiled using ZLIB_WINAPI. + */ +# ifdef ZLIB_WINAPI +# ifdef FAR +# undef FAR +# endif +# ifndef WIN32_LEAN_AND_MEAN +# define WIN32_LEAN_AND_MEAN +# endif +# include + /* No need for _export, use ZLIB.DEF instead. */ + /* For complete Windows compatibility, use WINAPI, not __stdcall. */ +# define ZEXPORT WINAPI +# ifdef WIN32 +# define ZEXPORTVA WINAPIV +# else +# define ZEXPORTVA FAR CDECL +# endif +# endif +#endif + +#if defined (__BEOS__) +# ifdef ZLIB_DLL +# ifdef ZLIB_INTERNAL +# define ZEXPORT __declspec(dllexport) +# define ZEXPORTVA __declspec(dllexport) +# else +# define ZEXPORT __declspec(dllimport) +# define ZEXPORTVA __declspec(dllimport) +# endif +# endif +#endif + +#ifndef ZEXTERN +# define ZEXTERN extern +#endif +#ifndef ZEXPORT +# define ZEXPORT +#endif +#ifndef ZEXPORTVA +# define ZEXPORTVA +#endif + +#ifndef FAR +# define FAR +#endif + +#if !defined(__MACTYPES__) +typedef unsigned char Byte; /* 8 bits */ +#endif +typedef unsigned int uInt; /* 16 bits or more */ +typedef unsigned long uLong; /* 32 bits or more */ + +#ifdef SMALL_MEDIUM + /* Borland C/C++ and some old MSC versions ignore FAR inside typedef */ +# define Bytef Byte FAR +#else + typedef Byte FAR Bytef; +#endif +typedef char FAR charf; +typedef int FAR intf; +typedef uInt FAR uIntf; +typedef uLong FAR uLongf; + +#ifdef STDC + typedef void const *voidpc; + typedef void FAR *voidpf; + typedef void *voidp; +#else + typedef Byte const *voidpc; + typedef Byte FAR *voidpf; + typedef Byte *voidp; +#endif + +#if !defined(Z_U4) && !defined(Z_SOLO) && defined(STDC) +# include +# if (UINT_MAX == 0xffffffffUL) +# define Z_U4 unsigned +# elif (ULONG_MAX == 0xffffffffUL) +# define Z_U4 unsigned long +# elif (USHRT_MAX == 0xffffffffUL) +# define Z_U4 unsigned short +# endif +#endif + +#ifdef Z_U4 + typedef Z_U4 z_crc_t; +#else + typedef unsigned long z_crc_t; +#endif + +#ifdef HAVE_UNISTD_H /* may be set to #if 1 by ./configure */ +# define Z_HAVE_UNISTD_H +#endif + +#ifdef HAVE_STDARG_H /* may be set to #if 1 by ./configure */ +# define Z_HAVE_STDARG_H +#endif + +#ifdef STDC +# ifndef Z_SOLO +/* TODO MERGE: # include /* for off_t */ +# endif +#endif + +#if defined(STDC) || defined(Z_HAVE_STDARG_H) +# ifndef Z_SOLO +/* TODO MERGE: # include /* for va_list */ +# endif +#endif + +#ifdef _WIN32 +# ifndef Z_SOLO +/* TODO MERGE: # include /* for wchar_t */ +# endif +#endif + +/* a little trick to accommodate both "#define _LARGEFILE64_SOURCE" and + * "#define _LARGEFILE64_SOURCE 1" as requesting 64-bit operations, (even + * though the former does not conform to the LFS document), but considering + * both "#undef _LARGEFILE64_SOURCE" and "#define _LARGEFILE64_SOURCE 0" as + * equivalently requesting no 64-bit operations + */ +#if defined(_LARGEFILE64_SOURCE) && -_LARGEFILE64_SOURCE - -1 == 1 +# undef _LARGEFILE64_SOURCE +#endif + +#ifndef Z_HAVE_UNISTD_H +# ifdef __WATCOMC__ +# define Z_HAVE_UNISTD_H +# endif +#endif +#ifndef Z_HAVE_UNISTD_H +# if defined(_LARGEFILE64_SOURCE) && !defined(_WIN32) +# define Z_HAVE_UNISTD_H +# endif +#endif + +#if 0 /* HAVE_UNISTD_H -- this line is updated by ./configure */ +# include /* for off_t */ +/* TODO MERGE: #ifndef Z_SOLO */ +# if defined(Z_HAVE_UNISTD_H) +# include /* for SEEK_*, off_t, and _LFS64_LARGEFILE */ +# ifdef VMS +# include /* for off_t */ +# endif +# ifndef z_off_t +# define z_off_t off_t +# endif +# endif +#endif + +#if defined(_LFS64_LARGEFILE) && _LFS64_LARGEFILE-0 +# define Z_LFS64 +#endif + +#if defined(_LARGEFILE64_SOURCE) && defined(Z_LFS64) +# define Z_LARGE64 +#endif + +#if defined(_FILE_OFFSET_BITS) && _FILE_OFFSET_BITS-0 == 64 && defined(Z_LFS64) +# define Z_WANT64 +#endif + +#if !defined(SEEK_SET) && !defined(Z_SOLO) +# define SEEK_SET 0 /* Seek from beginning of file. */ +# define SEEK_CUR 1 /* Seek from current position. */ +# define SEEK_END 2 /* Set file pointer to EOF plus "offset" */ +#endif + +#ifndef z_off_t +# define z_off_t long +#endif + +#if !defined(_WIN32) && defined(Z_LARGE64) +# define z_off64_t off64_t +#else +# if defined(_WIN32) && !defined(__GNUC__) && !defined(Z_SOLO) +# define z_off64_t __int64 +# else +# define z_off64_t z_off_t +# endif +#endif + +/* MVS linker does not support external names larger than 8 bytes */ +#if defined(__MVS__) + #pragma map(deflateInit_,"DEIN") + #pragma map(deflateInit2_,"DEIN2") + #pragma map(deflateEnd,"DEEND") + #pragma map(deflateBound,"DEBND") + #pragma map(inflateInit_,"ININ") + #pragma map(inflateInit2_,"ININ2") + #pragma map(inflateEnd,"INEND") + #pragma map(inflateSync,"INSY") + #pragma map(inflateSetDictionary,"INSEDI") + #pragma map(compressBound,"CMBND") + #pragma map(inflate_table,"INTABL") + #pragma map(inflate_fast,"INFA") + #pragma map(inflate_copyright,"INCOPY") +#endif + +#endif /* ZCONF_H */ diff --git a/lib/os/windows/zlib-1.2.13/zconf.h.in b/lib/os/windows/zlib-1.2.13/zconf.h.in new file mode 100644 index 000000000000..bf977d3e70ad --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/zconf.h.in @@ -0,0 +1,547 @@ +/* zconf.h -- configuration of the zlib compression library + * Copyright (C) 1995-2016 Jean-loup Gailly, Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* @(#) $Id$ */ + +#ifndef ZCONF_H +#define ZCONF_H + +/* + * If you *really* need a unique prefix for all types and library functions, + * compile with -DZ_PREFIX. The "standard" zlib should be compiled without it. + * Even better than compiling with -DZ_PREFIX would be to use configure to set + * this permanently in zconf.h using "./configure --zprefix". + */ +#ifdef Z_PREFIX /* may be set to #if 1 by ./configure */ +# define Z_PREFIX_SET + +/* all linked symbols and init macros */ +# define _dist_code z__dist_code +# define _length_code z__length_code +# define _tr_align z__tr_align +# define _tr_flush_bits z__tr_flush_bits +# define _tr_flush_block z__tr_flush_block +# define _tr_init z__tr_init +# define _tr_stored_block z__tr_stored_block +# define _tr_tally z__tr_tally +# define adler32 z_adler32 +# define adler32_combine z_adler32_combine +# define adler32_combine64 z_adler32_combine64 +# define adler32_z z_adler32_z +# ifndef Z_SOLO +# define compress z_compress +# define compress2 z_compress2 +# define compressBound z_compressBound +# endif +# define crc32 z_crc32 +# define crc32_combine z_crc32_combine +# define crc32_combine64 z_crc32_combine64 +# define crc32_combine_gen z_crc32_combine_gen +# define crc32_combine_gen64 z_crc32_combine_gen64 +# define crc32_combine_op z_crc32_combine_op +# define crc32_z z_crc32_z +# define deflate z_deflate +# define deflateBound z_deflateBound +# define deflateCopy z_deflateCopy +# define deflateEnd z_deflateEnd +# define deflateGetDictionary z_deflateGetDictionary +# define deflateInit z_deflateInit +# define deflateInit2 z_deflateInit2 +# define deflateInit2_ z_deflateInit2_ +# define deflateInit_ z_deflateInit_ +# define deflateParams z_deflateParams +# define deflatePending z_deflatePending +# define deflatePrime z_deflatePrime +# define deflateReset z_deflateReset +# define deflateResetKeep z_deflateResetKeep +# define deflateSetDictionary z_deflateSetDictionary +# define deflateSetHeader z_deflateSetHeader +# define deflateTune z_deflateTune +# define deflate_copyright z_deflate_copyright +# define get_crc_table z_get_crc_table +# ifndef Z_SOLO +# define gz_error z_gz_error +# define gz_intmax z_gz_intmax +# define gz_strwinerror z_gz_strwinerror +# define gzbuffer z_gzbuffer +# define gzclearerr z_gzclearerr +# define gzclose z_gzclose +# define gzclose_r z_gzclose_r +# define gzclose_w z_gzclose_w +# define gzdirect z_gzdirect +# define gzdopen z_gzdopen +# define gzeof z_gzeof +# define gzerror z_gzerror +# define gzflush z_gzflush +# define gzfread z_gzfread +# define gzfwrite z_gzfwrite +# define gzgetc z_gzgetc +# define gzgetc_ z_gzgetc_ +# define gzgets z_gzgets +# define gzoffset z_gzoffset +# define gzoffset64 z_gzoffset64 +# define gzopen z_gzopen +# define gzopen64 z_gzopen64 +# ifdef _WIN32 +# define gzopen_w z_gzopen_w +# endif +# define gzprintf z_gzprintf +# define gzputc z_gzputc +# define gzputs z_gzputs +# define gzread z_gzread +# define gzrewind z_gzrewind +# define gzseek z_gzseek +# define gzseek64 z_gzseek64 +# define gzsetparams z_gzsetparams +# define gztell z_gztell +# define gztell64 z_gztell64 +# define gzungetc z_gzungetc +# define gzvprintf z_gzvprintf +# define gzwrite z_gzwrite +# endif +# define inflate z_inflate +# define inflateBack z_inflateBack +# define inflateBackEnd z_inflateBackEnd +# define inflateBackInit z_inflateBackInit +# define inflateBackInit_ z_inflateBackInit_ +# define inflateCodesUsed z_inflateCodesUsed +# define inflateCopy z_inflateCopy +# define inflateEnd z_inflateEnd +# define inflateGetDictionary z_inflateGetDictionary +# define inflateGetHeader z_inflateGetHeader +# define inflateInit z_inflateInit +# define inflateInit2 z_inflateInit2 +# define inflateInit2_ z_inflateInit2_ +# define inflateInit_ z_inflateInit_ +# define inflateMark z_inflateMark +# define inflatePrime z_inflatePrime +# define inflateReset z_inflateReset +# define inflateReset2 z_inflateReset2 +# define inflateResetKeep z_inflateResetKeep +# define inflateSetDictionary z_inflateSetDictionary +# define inflateSync z_inflateSync +# define inflateSyncPoint z_inflateSyncPoint +# define inflateUndermine z_inflateUndermine +# define inflateValidate z_inflateValidate +# define inflate_copyright z_inflate_copyright +# define inflate_fast z_inflate_fast +# define inflate_table z_inflate_table +# ifndef Z_SOLO +# define uncompress z_uncompress +# define uncompress2 z_uncompress2 +# endif +# define zError z_zError +# ifndef Z_SOLO +# define zcalloc z_zcalloc +# define zcfree z_zcfree +# endif +# define zlibCompileFlags z_zlibCompileFlags +# define zlibVersion z_zlibVersion + +/* all zlib typedefs in zlib.h and zconf.h */ +# define Byte z_Byte +# define Bytef z_Bytef +# define alloc_func z_alloc_func +# define charf z_charf +# define free_func z_free_func +# ifndef Z_SOLO +# define gzFile z_gzFile +# endif +# define gz_header z_gz_header +# define gz_headerp z_gz_headerp +# define in_func z_in_func +# define intf z_intf +# define out_func z_out_func +# define uInt z_uInt +# define uIntf z_uIntf +# define uLong z_uLong +# define uLongf z_uLongf +# define voidp z_voidp +# define voidpc z_voidpc +# define voidpf z_voidpf + +/* all zlib structs in zlib.h and zconf.h */ +# define gz_header_s z_gz_header_s +# define internal_state z_internal_state + +#endif + +#if defined(__MSDOS__) && !defined(MSDOS) +# define MSDOS +#endif +#if (defined(OS_2) || defined(__OS2__)) && !defined(OS2) +# define OS2 +#endif +#if defined(_WINDOWS) && !defined(WINDOWS) +# define WINDOWS +#endif +#if defined(_WIN32) || defined(_WIN32_WCE) || defined(__WIN32__) +# ifndef WIN32 +# define WIN32 +# endif +#endif +#if (defined(MSDOS) || defined(OS2) || defined(WINDOWS)) && !defined(WIN32) +# if !defined(__GNUC__) && !defined(__FLAT__) && !defined(__386__) +# ifndef SYS16BIT +# define SYS16BIT +# endif +# endif +#endif + +/* + * Compile with -DMAXSEG_64K if the alloc function cannot allocate more + * than 64k bytes at a time (needed on systems with 16-bit int). + */ +#ifdef SYS16BIT +# define MAXSEG_64K +#endif +#ifdef MSDOS +# define UNALIGNED_OK +#endif + +#ifdef __STDC_VERSION__ +# ifndef STDC +# define STDC +# endif +# if __STDC_VERSION__ >= 199901L +# ifndef STDC99 +# define STDC99 +# endif +# endif +#endif +#if !defined(STDC) && (defined(__STDC__) || defined(__cplusplus)) +# define STDC +#endif +#if !defined(STDC) && (defined(__GNUC__) || defined(__BORLANDC__)) +# define STDC +#endif +#if !defined(STDC) && (defined(MSDOS) || defined(WINDOWS) || defined(WIN32)) +# define STDC +#endif +#if !defined(STDC) && (defined(OS2) || defined(__HOS_AIX__)) +# define STDC +#endif + +#if defined(__OS400__) && !defined(STDC) /* iSeries (formerly AS/400). */ +# define STDC +#endif + +#ifndef STDC +# ifndef const /* cannot use !defined(STDC) && !defined(const) on Mac */ +# define const /* note: need a more gentle solution here */ +# endif +#endif + +#if defined(ZLIB_CONST) && !defined(z_const) +# define z_const const +#else +# define z_const +#endif + +#ifdef Z_SOLO + typedef unsigned long z_size_t; +#else +# define z_longlong long long +# if defined(NO_SIZE_T) + typedef unsigned NO_SIZE_T z_size_t; +# elif defined(STDC) +# include + typedef size_t z_size_t; +# else + typedef unsigned long z_size_t; +# endif +# undef z_longlong +#endif + +/* Maximum value for memLevel in deflateInit2 */ +#ifndef MAX_MEM_LEVEL +# ifdef MAXSEG_64K +# define MAX_MEM_LEVEL 8 +# else +# define MAX_MEM_LEVEL 9 +# endif +#endif + +/* Maximum value for windowBits in deflateInit2 and inflateInit2. + * WARNING: reducing MAX_WBITS makes minigzip unable to extract .gz files + * created by gzip. (Files created by minigzip can still be extracted by + * gzip.) + */ +#ifndef MAX_WBITS +# define MAX_WBITS 15 /* 32K LZ77 window */ +#endif + +/* The memory requirements for deflate are (in bytes): + (1 << (windowBits+2)) + (1 << (memLevel+9)) + that is: 128K for windowBits=15 + 128K for memLevel = 8 (default values) + plus a few kilobytes for small objects. For example, if you want to reduce + the default memory requirements from 256K to 128K, compile with + make CFLAGS="-O -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7" + Of course this will generally degrade compression (there's no free lunch). + + The memory requirements for inflate are (in bytes) 1 << windowBits + that is, 32K for windowBits=15 (default value) plus about 7 kilobytes + for small objects. +*/ + + /* Type declarations */ + +#ifndef OF /* function prototypes */ +# ifdef STDC +# define OF(args) args +# else +# define OF(args) () +# endif +#endif + +#ifndef Z_ARG /* function prototypes for stdarg */ +# if defined(STDC) || defined(Z_HAVE_STDARG_H) +# define Z_ARG(args) args +# else +# define Z_ARG(args) () +# endif +#endif + +/* The following definitions for FAR are needed only for MSDOS mixed + * model programming (small or medium model with some far allocations). + * This was tested only with MSC; for other MSDOS compilers you may have + * to define NO_MEMCPY in zutil.h. If you don't need the mixed model, + * just define FAR to be empty. + */ +#ifdef SYS16BIT +# if defined(M_I86SM) || defined(M_I86MM) + /* MSC small or medium model */ +# define SMALL_MEDIUM +# ifdef _MSC_VER +# define FAR _far +# else +# define FAR far +# endif +# endif +# if (defined(__SMALL__) || defined(__MEDIUM__)) + /* Turbo C small or medium model */ +# define SMALL_MEDIUM +# ifdef __BORLANDC__ +# define FAR _far +# else +# define FAR far +# endif +# endif +#endif + +#if defined(WINDOWS) || defined(WIN32) + /* If building or using zlib as a DLL, define ZLIB_DLL. + * This is not mandatory, but it offers a little performance increase. + */ +# ifdef ZLIB_DLL +# if defined(WIN32) && (!defined(__BORLANDC__) || (__BORLANDC__ >= 0x500)) +# ifdef ZLIB_INTERNAL +# define ZEXTERN extern __declspec(dllexport) +# else +# define ZEXTERN extern __declspec(dllimport) +# endif +# endif +# endif /* ZLIB_DLL */ + /* If building or using zlib with the WINAPI/WINAPIV calling convention, + * define ZLIB_WINAPI. + * Caution: the standard ZLIB1.DLL is NOT compiled using ZLIB_WINAPI. + */ +# ifdef ZLIB_WINAPI +# ifdef FAR +# undef FAR +# endif +# ifndef WIN32_LEAN_AND_MEAN +# define WIN32_LEAN_AND_MEAN +# endif +# include + /* No need for _export, use ZLIB.DEF instead. */ + /* For complete Windows compatibility, use WINAPI, not __stdcall. */ +# define ZEXPORT WINAPI +# ifdef WIN32 +# define ZEXPORTVA WINAPIV +# else +# define ZEXPORTVA FAR CDECL +# endif +# endif +#endif + +#if defined (__BEOS__) +# ifdef ZLIB_DLL +# ifdef ZLIB_INTERNAL +# define ZEXPORT __declspec(dllexport) +# define ZEXPORTVA __declspec(dllexport) +# else +# define ZEXPORT __declspec(dllimport) +# define ZEXPORTVA __declspec(dllimport) +# endif +# endif +#endif + +#ifndef ZEXTERN +# define ZEXTERN extern +#endif +#ifndef ZEXPORT +# define ZEXPORT +#endif +#ifndef ZEXPORTVA +# define ZEXPORTVA +#endif + +#ifndef FAR +# define FAR +#endif + +#if !defined(__MACTYPES__) +typedef unsigned char Byte; /* 8 bits */ +#endif +typedef unsigned int uInt; /* 16 bits or more */ +typedef unsigned long uLong; /* 32 bits or more */ + +#ifdef SMALL_MEDIUM + /* Borland C/C++ and some old MSC versions ignore FAR inside typedef */ +# define Bytef Byte FAR +#else + typedef Byte FAR Bytef; +#endif +typedef char FAR charf; +typedef int FAR intf; +typedef uInt FAR uIntf; +typedef uLong FAR uLongf; + +#ifdef STDC + typedef void const *voidpc; + typedef void FAR *voidpf; + typedef void *voidp; +#else + typedef Byte const *voidpc; + typedef Byte FAR *voidpf; + typedef Byte *voidp; +#endif + +#if !defined(Z_U4) && !defined(Z_SOLO) && defined(STDC) +# include +# if (UINT_MAX == 0xffffffffUL) +# define Z_U4 unsigned +# elif (ULONG_MAX == 0xffffffffUL) +# define Z_U4 unsigned long +# elif (USHRT_MAX == 0xffffffffUL) +# define Z_U4 unsigned short +# endif +#endif + +#ifdef Z_U4 + typedef Z_U4 z_crc_t; +#else + typedef unsigned long z_crc_t; +#endif + +#ifdef HAVE_UNISTD_H /* may be set to #if 1 by ./configure */ +# define Z_HAVE_UNISTD_H +#endif + +#ifdef HAVE_STDARG_H /* may be set to #if 1 by ./configure */ +# define Z_HAVE_STDARG_H +#endif + +#ifdef STDC +# ifndef Z_SOLO +# include /* for off_t */ +# endif +#endif + +#if defined(STDC) || defined(Z_HAVE_STDARG_H) +# ifndef Z_SOLO +# include /* for va_list */ +# endif +#endif + +#ifdef _WIN32 +# ifndef Z_SOLO +# include /* for wchar_t */ +# endif +#endif + +/* a little trick to accommodate both "#define _LARGEFILE64_SOURCE" and + * "#define _LARGEFILE64_SOURCE 1" as requesting 64-bit operations, (even + * though the former does not conform to the LFS document), but considering + * both "#undef _LARGEFILE64_SOURCE" and "#define _LARGEFILE64_SOURCE 0" as + * equivalently requesting no 64-bit operations + */ +#if defined(_LARGEFILE64_SOURCE) && -_LARGEFILE64_SOURCE - -1 == 1 +# undef _LARGEFILE64_SOURCE +#endif + +#ifndef Z_HAVE_UNISTD_H +# ifdef __WATCOMC__ +# define Z_HAVE_UNISTD_H +# endif +#endif +#ifndef Z_HAVE_UNISTD_H +# if defined(_LARGEFILE64_SOURCE) && !defined(_WIN32) +# define Z_HAVE_UNISTD_H +# endif +#endif +#ifndef Z_SOLO +# if defined(Z_HAVE_UNISTD_H) +# include /* for SEEK_*, off_t, and _LFS64_LARGEFILE */ +# ifdef VMS +# include /* for off_t */ +# endif +# ifndef z_off_t +# define z_off_t off_t +# endif +# endif +#endif + +#if defined(_LFS64_LARGEFILE) && _LFS64_LARGEFILE-0 +# define Z_LFS64 +#endif + +#if defined(_LARGEFILE64_SOURCE) && defined(Z_LFS64) +# define Z_LARGE64 +#endif + +#if defined(_FILE_OFFSET_BITS) && _FILE_OFFSET_BITS-0 == 64 && defined(Z_LFS64) +# define Z_WANT64 +#endif + +#if !defined(SEEK_SET) && !defined(Z_SOLO) +# define SEEK_SET 0 /* Seek from beginning of file. */ +# define SEEK_CUR 1 /* Seek from current position. */ +# define SEEK_END 2 /* Set file pointer to EOF plus "offset" */ +#endif + +#ifndef z_off_t +# define z_off_t long +#endif + +#if !defined(_WIN32) && defined(Z_LARGE64) +# define z_off64_t off64_t +#else +# if defined(_WIN32) && !defined(__GNUC__) && !defined(Z_SOLO) +# define z_off64_t __int64 +# else +# define z_off64_t z_off_t +# endif +#endif + +/* MVS linker does not support external names larger than 8 bytes */ +#if defined(__MVS__) + #pragma map(deflateInit_,"DEIN") + #pragma map(deflateInit2_,"DEIN2") + #pragma map(deflateEnd,"DEEND") + #pragma map(deflateBound,"DEBND") + #pragma map(inflateInit_,"ININ") + #pragma map(inflateInit2_,"ININ2") + #pragma map(inflateEnd,"INEND") + #pragma map(inflateSync,"INSY") + #pragma map(inflateSetDictionary,"INSEDI") + #pragma map(compressBound,"CMBND") + #pragma map(inflate_table,"INTABL") + #pragma map(inflate_fast,"INFA") + #pragma map(inflate_copyright,"INCOPY") +#endif + +#endif /* ZCONF_H */ diff --git a/lib/os/windows/zlib-1.2.13/zlib-dll-res.rc b/lib/os/windows/zlib-1.2.13/zlib-dll-res.rc new file mode 100644 index 000000000000..538ddbec70f5 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/zlib-dll-res.rc @@ -0,0 +1,134 @@ +#define WIN32_LEAN_AND_MEAN +#include +#include + +#define VER_FILEVERSION 1,2,3,2027 +#define VER_FILEVERSION_STR "1.2.3.2027" +#define VER_FILELTVERSION_STR "Zlib: general purpose data compression / decompression library" +#define VER_LEGALCOPYRIGHT_STR " 2005 Jean-loup Gailly , Mark Adler " +//#define VER_COMMENT_STR "" + +#define VER_FILEDESCRIPTION_STR "Zlib: general purpose data compression / decompression library" +#define VER_INTERNALNAME_STR "zlib1" +#define VER_ORIGINALFILENAME_STR "zlib1.dll" +#define VER_WWW_STR "http://www.zlib.net" +#define VER_COMPANYNAME_STR "Zlib" +#define VER_LICENSE_STR "(C) 1995-2002 Jean-loup Gailly and Mark Adler This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software. Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions: 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 3. This notice may not be removed or altered from any source distribution. Jean-loup Gailly Mark Adler jloup@gzip.org madler@alumni.caltech.edu " +#define VER_LEGALTRADEMARKS_STR "Zlib, Zlib, zlib1" + +#define VER_PRODUCTNAME_STR "Zlib" +#define VER_PRODUCTVERSION 1,2,3,2027 +#define VER_PRODUCTVERSION_STR "1.2.3.2027" + + +#undef OFFICIAL +#define FINAL 1 + +#define GNUWIN32_SPECIALBUILD_STR "GNU for Win32 " + +#define VER_FILETYPE VFT_DLL +#ifndef VER_FILETYPE +#define VER_FILETYPE VFT_APP +#endif + +#define VER_FILESUBTYPE VFT2_UNKNOWN + +#ifndef DEBUG +#define VER_DEBUG 0 +#else /* DEBUG */ +#define VER_DEBUG VS_FF_DEBUG +#endif + +#ifndef PATCHED +#define VER_PATCHED 0 +#else /* PATCHED */ +#define VER_PATCHED VS_FF_PATCHED +#endif + +#ifndef OFFICIAL +#define VER_SPECIALBUILD VS_FF_SPECIALBUILD +#ifndef VER_SPECIALBUILD_STR +#define VER_SPECIALBUILD_STR GNUWIN32_SPECIALBUILD_STR +#endif +#else /* OFFICIAL */ +#define VER_SPECIALBUILD 0 +#endif /* OFFICIAL */ + +#ifndef FINAL +#define VER_PRIVATEBUILD VS_FF_PRIVATEBUILD +#ifndef VER_PRIVATEBUILD_STR +#define VER_PRIVATEBUILD_STR "Pre-release" +#endif /* VER_PRIVATEBUILD_STR */ +#define VER_PRERELEASE VS_FF_PRERELEASE +#else /* FINAL */ +#define VER_PRIVATEBUILD 0 +#define VER_PRERELEASE 0 +#endif /* FINAL */ + +#define VER_FILEFLAGSMASK VS_FFI_FILEFLAGSMASK +#define VER_FILEFLAGS (VER_PRIVATEBUILD|VER_SPECIALBUILD|VER_PRERELEASE|VER_DEBUG|VER_PATCHED) + +#define VER_FILEOS VOS__WINDOWS32 + +#ifdef RC_INVOKED + +VS_VERSION_INFO VERSIONINFO +FILEVERSION VER_FILEVERSION +PRODUCTVERSION VER_PRODUCTVERSION +FILEFLAGSMASK VER_FILEFLAGSMASK +FILEFLAGS VER_FILEFLAGS +FILEOS VER_FILEOS +FILETYPE VER_FILETYPE +FILESUBTYPE VER_FILESUBTYPE + +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904E4" + /* language ID = U.S. English, char set = Windows, Multilingual */ + BEGIN +#ifdef VER_COMMENT_STR + VALUE "Comments", VER_COMMENT_STR +#endif + VALUE "CompanyName", VER_COMPANYNAME_STR + VALUE "License", VER_LICENSE_STR + VALUE "FileDescription", VER_FILEDESCRIPTION_STR + VALUE "FileVersion", VER_FILEVERSION_STR +#if !(VER_FILETYPE-VFT_DLL) + VALUE "LibToolFileVersion",VER_FILELTVERSION_STR +#endif + VALUE "InternalName", VER_INTERNALNAME_STR +#ifdef VER_LEGALCOPYRIGHT_STR + VALUE "LegalCopyright", VER_LEGALCOPYRIGHT_STR +#endif +#ifdef VER_LEGALTRADEMARKS_STR + VALUE "LegalTrademarks", VER_LEGALTRADEMARKS_STR +#endif + VALUE "OriginalFilename", VER_ORIGINALFILENAME_STR + VALUE "ProductName", VER_PRODUCTNAME_STR + VALUE "ProductVersion", VER_PRODUCTVERSION_STR +#ifdef VER_PATCHLEVEL_STR + VALUE "Patch level", VER_PATCHLEVEL_STR +#endif +#ifdef VER_PRIVATEBUILD_STR + VALUE "PrivateBuild", VER_PRIVATEBUILD_STR +#endif +#ifdef VER_SPECIALBUILD_STR + VALUE "SpecialBuild", VER_SPECIALBUILD_STR +#endif +#ifdef VER_AUTHOR_STR + VALUE "Authors", VER_AUTHOR_STR +#endif +#ifdef VER_WWW_STR + VALUE "WWW", VER_WWW_STR +#endif +END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x409, 1252 + END +END + +#endif /* RC_INVOKED */ +500 ICON MOVEABLE PURE LOADONCALL DISCARDABLE "zlib.ico" diff --git a/lib/os/windows/zlib-1.2.13/zlib.3 b/lib/os/windows/zlib-1.2.13/zlib.3 new file mode 100644 index 000000000000..6f6e91404dff --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/zlib.3 @@ -0,0 +1,149 @@ +.TH ZLIB 3 "13 Oct 2022" +.SH NAME +zlib \- compression/decompression library +.SH SYNOPSIS +[see +.I zlib.h +for full description] +.SH DESCRIPTION +The +.I zlib +library is a general purpose data compression library. +The code is thread safe, assuming that the standard library functions +used are thread safe, such as memory allocation routines. +It provides in-memory compression and decompression functions, +including integrity checks of the uncompressed data. +This version of the library supports only one compression method (deflation) +but other algorithms may be added later +with the same stream interface. +.LP +Compression can be done in a single step if the buffers are large enough +or can be done by repeated calls of the compression function. +In the latter case, +the application must provide more input and/or consume the output +(providing more output space) before each call. +.LP +The library also supports reading and writing files in +.IR gzip (1) +(.gz) format +with an interface similar to that of stdio. +.LP +The library does not install any signal handler. +The decoder checks the consistency of the compressed data, +so the library should never crash even in the case of corrupted input. +.LP +All functions of the compression library are documented in the file +.IR zlib.h . +The distribution source includes examples of use of the library +in the files +.I test/example.c +and +.IR test/minigzip.c, +as well as other examples in the +.IR examples/ +directory. +.LP +Changes to this version are documented in the file +.I ChangeLog +that accompanies the source. +.LP +.I zlib +is built in to many languages and operating systems, including but not limited to +Java, Python, .NET, PHP, Perl, Ruby, Swift, and Go. +.LP +An experimental package to read and write files in the .zip format, +written on top of +.I zlib +by Gilles Vollant (info@winimage.com), +is available at: +.IP +http://www.winimage.com/zLibDll/minizip.html +and also in the +.I contrib/minizip +directory of the main +.I zlib +source distribution. +.SH "SEE ALSO" +The +.I zlib +web site can be found at: +.IP +http://zlib.net/ +.LP +The data format used by the +.I zlib +library is described by RFC +(Request for Comments) 1950 to 1952 in the files: +.IP +http://tools.ietf.org/html/rfc1950 (for the zlib header and trailer format) +.br +http://tools.ietf.org/html/rfc1951 (for the deflate compressed data format) +.br +http://tools.ietf.org/html/rfc1952 (for the gzip header and trailer format) +.LP +Mark Nelson wrote an article about +.I zlib +for the Jan. 1997 issue of Dr. Dobb's Journal; +a copy of the article is available at: +.IP +http://marknelson.us/1997/01/01/zlib-engine/ +.SH "REPORTING PROBLEMS" +Before reporting a problem, +please check the +.I zlib +web site to verify that you have the latest version of +.IR zlib ; +otherwise, +obtain the latest version and see if the problem still exists. +Please read the +.I zlib +FAQ at: +.IP +http://zlib.net/zlib_faq.html +.LP +before asking for help. +Send questions and/or comments to zlib@gzip.org, +or (for the Windows DLL version) to Gilles Vollant (info@winimage.com). +.SH AUTHORS AND LICENSE +Version 1.2.13 +.LP +Copyright (C) 1995-2022 Jean-loup Gailly and Mark Adler +.LP +This software is provided 'as-is', without any express or implied +warranty. In no event will the authors be held liable for any damages +arising from the use of this software. +.LP +Permission is granted to anyone to use this software for any purpose, +including commercial applications, and to alter it and redistribute it +freely, subject to the following restrictions: +.LP +.nr step 1 1 +.IP \n[step]. 3 +The origin of this software must not be misrepresented; you must not +claim that you wrote the original software. If you use this software +in a product, an acknowledgment in the product documentation would be +appreciated but is not required. +.IP \n+[step]. +Altered source versions must be plainly marked as such, and must not be +misrepresented as being the original software. +.IP \n+[step]. +This notice may not be removed or altered from any source distribution. +.LP +Jean-loup Gailly Mark Adler +.br +jloup@gzip.org madler@alumni.caltech.edu +.LP +The deflate format used by +.I zlib +was defined by Phil Katz. +The deflate and +.I zlib +specifications were written by L. Peter Deutsch. +Thanks to all the people who reported problems and suggested various +improvements in +.IR zlib ; +who are too numerous to cite here. +.LP +UNIX manual page by R. P. C. Rodgers, +U.S. National Library of Medicine (rodgers@nlm.nih.gov). +.\" end of man page diff --git a/lib/os/windows/zlib-1.2.13/zlib.h b/lib/os/windows/zlib-1.2.13/zlib.h new file mode 100644 index 000000000000..953cb5012dc2 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/zlib.h @@ -0,0 +1,1935 @@ +/* zlib.h -- interface of the 'zlib' general purpose compression library + version 1.2.13, October 13th, 2022 + + Copyright (C) 1995-2022 Jean-loup Gailly and Mark Adler + + This software is provided 'as-is', without any express or implied + warranty. In no event will the authors be held liable for any damages + arising from the use of this software. + + Permission is granted to anyone to use this software for any purpose, + including commercial applications, and to alter it and redistribute it + freely, subject to the following restrictions: + + 1. The origin of this software must not be misrepresented; you must not + claim that you wrote the original software. If you use this software + in a product, an acknowledgment in the product documentation would be + appreciated but is not required. + 2. Altered source versions must be plainly marked as such, and must not be + misrepresented as being the original software. + 3. This notice may not be removed or altered from any source distribution. + + Jean-loup Gailly Mark Adler + jloup@gzip.org madler@alumni.caltech.edu + + + The data format used by the zlib library is described by RFCs (Request for + Comments) 1950 to 1952 in the files http://tools.ietf.org/html/rfc1950 + (zlib format), rfc1951 (deflate format) and rfc1952 (gzip format). +*/ + +#ifndef ZLIB_H +#define ZLIB_H + +#include "zconf.h" + +#ifdef __cplusplus +extern "C" { +#endif + +#define ZLIB_VERSION "1.2.13" +#define ZLIB_VERNUM 0x12d0 +#define ZLIB_VER_MAJOR 1 +#define ZLIB_VER_MINOR 2 +#define ZLIB_VER_REVISION 13 +#define ZLIB_VER_SUBREVISION 0 + +/* + The 'zlib' compression library provides in-memory compression and + decompression functions, including integrity checks of the uncompressed data. + This version of the library supports only one compression method (deflation) + but other algorithms will be added later and will have the same stream + interface. + + Compression can be done in a single step if the buffers are large enough, + or can be done by repeated calls of the compression function. In the latter + case, the application must provide more input and/or consume the output + (providing more output space) before each call. + + The compressed data format used by default by the in-memory functions is + the zlib format, which is a zlib wrapper documented in RFC 1950, wrapped + around a deflate stream, which is itself documented in RFC 1951. + + The library also supports reading and writing files in gzip (.gz) format + with an interface similar to that of stdio using the functions that start + with "gz". The gzip format is different from the zlib format. gzip is a + gzip wrapper, documented in RFC 1952, wrapped around a deflate stream. + + This library can optionally read and write gzip and raw deflate streams in + memory as well. + + The zlib format was designed to be compact and fast for use in memory + and on communications channels. The gzip format was designed for single- + file compression on file systems, has a larger header than zlib to maintain + directory information, and uses a different, slower check method than zlib. + + The library does not install any signal handler. The decoder checks + the consistency of the compressed data, so the library should never crash + even in the case of corrupted input. +*/ + +typedef voidpf (*alloc_func) OF((voidpf opaque, uInt items, uInt size)); +typedef void (*free_func) OF((voidpf opaque, voidpf address)); + +struct internal_state; + +typedef struct z_stream_s { + z_const Bytef *next_in; /* next input byte */ + uInt avail_in; /* number of bytes available at next_in */ + uLong total_in; /* total number of input bytes read so far */ + + Bytef *next_out; /* next output byte will go here */ + uInt avail_out; /* remaining free space at next_out */ + uLong total_out; /* total number of bytes output so far */ + + z_const char *msg; /* last error message, NULL if no error */ + struct internal_state FAR *state; /* not visible by applications */ + + alloc_func zalloc; /* used to allocate the internal state */ + free_func zfree; /* used to free the internal state */ + voidpf opaque; /* private data object passed to zalloc and zfree */ + + int data_type; /* best guess about the data type: binary or text + for deflate, or the decoding state for inflate */ + uLong adler; /* Adler-32 or CRC-32 value of the uncompressed data */ + uLong reserved; /* reserved for future use */ +} z_stream; + +typedef z_stream FAR *z_streamp; + +/* + gzip header information passed to and from zlib routines. See RFC 1952 + for more details on the meanings of these fields. +*/ +typedef struct gz_header_s { + int text; /* true if compressed data believed to be text */ + uLong time; /* modification time */ + int xflags; /* extra flags (not used when writing a gzip file) */ + int os; /* operating system */ + Bytef *extra; /* pointer to extra field or Z_NULL if none */ + uInt extra_len; /* extra field length (valid if extra != Z_NULL) */ + uInt extra_max; /* space at extra (only when reading header) */ + Bytef *name; /* pointer to zero-terminated file name or Z_NULL */ + uInt name_max; /* space at name (only when reading header) */ + Bytef *comment; /* pointer to zero-terminated comment or Z_NULL */ + uInt comm_max; /* space at comment (only when reading header) */ + int hcrc; /* true if there was or will be a header crc */ + int done; /* true when done reading gzip header (not used + when writing a gzip file) */ +} gz_header; + +typedef gz_header FAR *gz_headerp; + +/* + The application must update next_in and avail_in when avail_in has dropped + to zero. It must update next_out and avail_out when avail_out has dropped + to zero. The application must initialize zalloc, zfree and opaque before + calling the init function. All other fields are set by the compression + library and must not be updated by the application. + + The opaque value provided by the application will be passed as the first + parameter for calls of zalloc and zfree. This can be useful for custom + memory management. The compression library attaches no meaning to the + opaque value. + + zalloc must return Z_NULL if there is not enough memory for the object. + If zlib is used in a multi-threaded application, zalloc and zfree must be + thread safe. In that case, zlib is thread-safe. When zalloc and zfree are + Z_NULL on entry to the initialization function, they are set to internal + routines that use the standard library functions malloc() and free(). + + On 16-bit systems, the functions zalloc and zfree must be able to allocate + exactly 65536 bytes, but will not be required to allocate more than this if + the symbol MAXSEG_64K is defined (see zconf.h). WARNING: On MSDOS, pointers + returned by zalloc for objects of exactly 65536 bytes *must* have their + offset normalized to zero. The default allocation function provided by this + library ensures this (see zutil.c). To reduce memory requirements and avoid + any allocation of 64K objects, at the expense of compression ratio, compile + the library with -DMAX_WBITS=14 (see zconf.h). + + The fields total_in and total_out can be used for statistics or progress + reports. After compression, total_in holds the total size of the + uncompressed data and may be saved for use by the decompressor (particularly + if the decompressor wants to decompress everything in a single step). +*/ + + /* constants */ + +#define Z_NO_FLUSH 0 +#define Z_PARTIAL_FLUSH 1 +#define Z_SYNC_FLUSH 2 +#define Z_FULL_FLUSH 3 +#define Z_FINISH 4 +#define Z_BLOCK 5 +#define Z_TREES 6 +/* Allowed flush values; see deflate() and inflate() below for details */ + +#define Z_OK 0 +#define Z_STREAM_END 1 +#define Z_NEED_DICT 2 +#define Z_ERRNO (-1) +#define Z_STREAM_ERROR (-2) +#define Z_DATA_ERROR (-3) +#define Z_MEM_ERROR (-4) +#define Z_BUF_ERROR (-5) +#define Z_VERSION_ERROR (-6) +/* Return codes for the compression/decompression functions. Negative values + * are errors, positive values are used for special but normal events. + */ + +#define Z_NO_COMPRESSION 0 +#define Z_BEST_SPEED 1 +#define Z_BEST_COMPRESSION 9 +#define Z_DEFAULT_COMPRESSION (-1) +/* compression levels */ + +#define Z_FILTERED 1 +#define Z_HUFFMAN_ONLY 2 +#define Z_RLE 3 +#define Z_FIXED 4 +#define Z_DEFAULT_STRATEGY 0 +/* compression strategy; see deflateInit2() below for details */ + +#define Z_BINARY 0 +#define Z_TEXT 1 +#define Z_ASCII Z_TEXT /* for compatibility with 1.2.2 and earlier */ +#define Z_UNKNOWN 2 +/* Possible values of the data_type field for deflate() */ + +#define Z_DEFLATED 8 +/* The deflate compression method (the only one supported in this version) */ + +#define Z_NULL 0 /* for initializing zalloc, zfree, opaque */ + +#define zlib_version zlibVersion() +/* for compatibility with versions < 1.0.2 */ + + + /* basic functions */ + +ZEXTERN const char * ZEXPORT zlibVersion OF((void)); +/* The application can compare zlibVersion and ZLIB_VERSION for consistency. + If the first character differs, the library code actually used is not + compatible with the zlib.h header file used by the application. This check + is automatically made by deflateInit and inflateInit. + */ + +/* +ZEXTERN int ZEXPORT deflateInit OF((z_streamp strm, int level)); + + Initializes the internal stream state for compression. The fields + zalloc, zfree and opaque must be initialized before by the caller. If + zalloc and zfree are set to Z_NULL, deflateInit updates them to use default + allocation functions. + + The compression level must be Z_DEFAULT_COMPRESSION, or between 0 and 9: + 1 gives best speed, 9 gives best compression, 0 gives no compression at all + (the input data is simply copied a block at a time). Z_DEFAULT_COMPRESSION + requests a default compromise between speed and compression (currently + equivalent to level 6). + + deflateInit returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_STREAM_ERROR if level is not a valid compression level, or + Z_VERSION_ERROR if the zlib library version (zlib_version) is incompatible + with the version assumed by the caller (ZLIB_VERSION). msg is set to null + if there is no error message. deflateInit does not perform any compression: + this will be done by deflate(). +*/ + + +ZEXTERN int ZEXPORT deflate OF((z_streamp strm, int flush)); +/* + deflate compresses as much data as possible, and stops when the input + buffer becomes empty or the output buffer becomes full. It may introduce + some output latency (reading input without producing any output) except when + forced to flush. + + The detailed semantics are as follows. deflate performs one or both of the + following actions: + + - Compress more input starting at next_in and update next_in and avail_in + accordingly. If not all input can be processed (because there is not + enough room in the output buffer), next_in and avail_in are updated and + processing will resume at this point for the next call of deflate(). + + - Generate more output starting at next_out and update next_out and avail_out + accordingly. This action is forced if the parameter flush is non zero. + Forcing flush frequently degrades the compression ratio, so this parameter + should be set only when necessary. Some output may be provided even if + flush is zero. + + Before the call of deflate(), the application should ensure that at least + one of the actions is possible, by providing more input and/or consuming more + output, and updating avail_in or avail_out accordingly; avail_out should + never be zero before the call. The application can consume the compressed + output when it wants, for example when the output buffer is full (avail_out + == 0), or after each call of deflate(). If deflate returns Z_OK and with + zero avail_out, it must be called again after making room in the output + buffer because there might be more output pending. See deflatePending(), + which can be used if desired to determine whether or not there is more output + in that case. + + Normally the parameter flush is set to Z_NO_FLUSH, which allows deflate to + decide how much data to accumulate before producing output, in order to + maximize compression. + + If the parameter flush is set to Z_SYNC_FLUSH, all pending output is + flushed to the output buffer and the output is aligned on a byte boundary, so + that the decompressor can get all input data available so far. (In + particular avail_in is zero after the call if enough output space has been + provided before the call.) Flushing may degrade compression for some + compression algorithms and so it should be used only when necessary. This + completes the current deflate block and follows it with an empty stored block + that is three bits plus filler bits to the next byte, followed by four bytes + (00 00 ff ff). + + If flush is set to Z_PARTIAL_FLUSH, all pending output is flushed to the + output buffer, but the output is not aligned to a byte boundary. All of the + input data so far will be available to the decompressor, as for Z_SYNC_FLUSH. + This completes the current deflate block and follows it with an empty fixed + codes block that is 10 bits long. This assures that enough bytes are output + in order for the decompressor to finish the block before the empty fixed + codes block. + + If flush is set to Z_BLOCK, a deflate block is completed and emitted, as + for Z_SYNC_FLUSH, but the output is not aligned on a byte boundary, and up to + seven bits of the current block are held to be written as the next byte after + the next deflate block is completed. In this case, the decompressor may not + be provided enough bits at this point in order to complete decompression of + the data provided so far to the compressor. It may need to wait for the next + block to be emitted. This is for advanced applications that need to control + the emission of deflate blocks. + + If flush is set to Z_FULL_FLUSH, all output is flushed as with + Z_SYNC_FLUSH, and the compression state is reset so that decompression can + restart from this point if previous compressed data has been damaged or if + random access is desired. Using Z_FULL_FLUSH too often can seriously degrade + compression. + + If deflate returns with avail_out == 0, this function must be called again + with the same value of the flush parameter and more output space (updated + avail_out), until the flush is complete (deflate returns with non-zero + avail_out). In the case of a Z_FULL_FLUSH or Z_SYNC_FLUSH, make sure that + avail_out is greater than six to avoid repeated flush markers due to + avail_out == 0 on return. + + If the parameter flush is set to Z_FINISH, pending input is processed, + pending output is flushed and deflate returns with Z_STREAM_END if there was + enough output space. If deflate returns with Z_OK or Z_BUF_ERROR, this + function must be called again with Z_FINISH and more output space (updated + avail_out) but no more input data, until it returns with Z_STREAM_END or an + error. After deflate has returned Z_STREAM_END, the only possible operations + on the stream are deflateReset or deflateEnd. + + Z_FINISH can be used in the first deflate call after deflateInit if all the + compression is to be done in a single step. In order to complete in one + call, avail_out must be at least the value returned by deflateBound (see + below). Then deflate is guaranteed to return Z_STREAM_END. If not enough + output space is provided, deflate will not return Z_STREAM_END, and it must + be called again as described above. + + deflate() sets strm->adler to the Adler-32 checksum of all input read + so far (that is, total_in bytes). If a gzip stream is being generated, then + strm->adler will be the CRC-32 checksum of the input read so far. (See + deflateInit2 below.) + + deflate() may update strm->data_type if it can make a good guess about + the input data type (Z_BINARY or Z_TEXT). If in doubt, the data is + considered binary. This field is only for information purposes and does not + affect the compression algorithm in any manner. + + deflate() returns Z_OK if some progress has been made (more input + processed or more output produced), Z_STREAM_END if all input has been + consumed and all output has been produced (only when flush is set to + Z_FINISH), Z_STREAM_ERROR if the stream state was inconsistent (for example + if next_in or next_out was Z_NULL or the state was inadvertently written over + by the application), or Z_BUF_ERROR if no progress is possible (for example + avail_in or avail_out was zero). Note that Z_BUF_ERROR is not fatal, and + deflate() can be called again with more input and more output space to + continue compressing. +*/ + + +ZEXTERN int ZEXPORT deflateEnd OF((z_streamp strm)); +/* + All dynamically allocated data structures for this stream are freed. + This function discards any unprocessed input and does not flush any pending + output. + + deflateEnd returns Z_OK if success, Z_STREAM_ERROR if the + stream state was inconsistent, Z_DATA_ERROR if the stream was freed + prematurely (some input or output was discarded). In the error case, msg + may be set but then points to a static string (which must not be + deallocated). +*/ + + +/* +ZEXTERN int ZEXPORT inflateInit OF((z_streamp strm)); + + Initializes the internal stream state for decompression. The fields + next_in, avail_in, zalloc, zfree and opaque must be initialized before by + the caller. In the current version of inflate, the provided input is not + read or consumed. The allocation of a sliding window will be deferred to + the first call of inflate (if the decompression does not complete on the + first call). If zalloc and zfree are set to Z_NULL, inflateInit updates + them to use default allocation functions. + + inflateInit returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_VERSION_ERROR if the zlib library version is incompatible with the + version assumed by the caller, or Z_STREAM_ERROR if the parameters are + invalid, such as a null pointer to the structure. msg is set to null if + there is no error message. inflateInit does not perform any decompression. + Actual decompression will be done by inflate(). So next_in, and avail_in, + next_out, and avail_out are unused and unchanged. The current + implementation of inflateInit() does not process any header information -- + that is deferred until inflate() is called. +*/ + + +ZEXTERN int ZEXPORT inflate OF((z_streamp strm, int flush)); +/* + inflate decompresses as much data as possible, and stops when the input + buffer becomes empty or the output buffer becomes full. It may introduce + some output latency (reading input without producing any output) except when + forced to flush. + + The detailed semantics are as follows. inflate performs one or both of the + following actions: + + - Decompress more input starting at next_in and update next_in and avail_in + accordingly. If not all input can be processed (because there is not + enough room in the output buffer), then next_in and avail_in are updated + accordingly, and processing will resume at this point for the next call of + inflate(). + + - Generate more output starting at next_out and update next_out and avail_out + accordingly. inflate() provides as much output as possible, until there is + no more input data or no more space in the output buffer (see below about + the flush parameter). + + Before the call of inflate(), the application should ensure that at least + one of the actions is possible, by providing more input and/or consuming more + output, and updating the next_* and avail_* values accordingly. If the + caller of inflate() does not provide both available input and available + output space, it is possible that there will be no progress made. The + application can consume the uncompressed output when it wants, for example + when the output buffer is full (avail_out == 0), or after each call of + inflate(). If inflate returns Z_OK and with zero avail_out, it must be + called again after making room in the output buffer because there might be + more output pending. + + The flush parameter of inflate() can be Z_NO_FLUSH, Z_SYNC_FLUSH, Z_FINISH, + Z_BLOCK, or Z_TREES. Z_SYNC_FLUSH requests that inflate() flush as much + output as possible to the output buffer. Z_BLOCK requests that inflate() + stop if and when it gets to the next deflate block boundary. When decoding + the zlib or gzip format, this will cause inflate() to return immediately + after the header and before the first block. When doing a raw inflate, + inflate() will go ahead and process the first block, and will return when it + gets to the end of that block, or when it runs out of data. + + The Z_BLOCK option assists in appending to or combining deflate streams. + To assist in this, on return inflate() always sets strm->data_type to the + number of unused bits in the last byte taken from strm->next_in, plus 64 if + inflate() is currently decoding the last block in the deflate stream, plus + 128 if inflate() returned immediately after decoding an end-of-block code or + decoding the complete header up to just before the first byte of the deflate + stream. The end-of-block will not be indicated until all of the uncompressed + data from that block has been written to strm->next_out. The number of + unused bits may in general be greater than seven, except when bit 7 of + data_type is set, in which case the number of unused bits will be less than + eight. data_type is set as noted here every time inflate() returns for all + flush options, and so can be used to determine the amount of currently + consumed input in bits. + + The Z_TREES option behaves as Z_BLOCK does, but it also returns when the + end of each deflate block header is reached, before any actual data in that + block is decoded. This allows the caller to determine the length of the + deflate block header for later use in random access within a deflate block. + 256 is added to the value of strm->data_type when inflate() returns + immediately after reaching the end of the deflate block header. + + inflate() should normally be called until it returns Z_STREAM_END or an + error. However if all decompression is to be performed in a single step (a + single call of inflate), the parameter flush should be set to Z_FINISH. In + this case all pending input is processed and all pending output is flushed; + avail_out must be large enough to hold all of the uncompressed data for the + operation to complete. (The size of the uncompressed data may have been + saved by the compressor for this purpose.) The use of Z_FINISH is not + required to perform an inflation in one step. However it may be used to + inform inflate that a faster approach can be used for the single inflate() + call. Z_FINISH also informs inflate to not maintain a sliding window if the + stream completes, which reduces inflate's memory footprint. If the stream + does not complete, either because not all of the stream is provided or not + enough output space is provided, then a sliding window will be allocated and + inflate() can be called again to continue the operation as if Z_NO_FLUSH had + been used. + + In this implementation, inflate() always flushes as much output as + possible to the output buffer, and always uses the faster approach on the + first call. So the effects of the flush parameter in this implementation are + on the return value of inflate() as noted below, when inflate() returns early + when Z_BLOCK or Z_TREES is used, and when inflate() avoids the allocation of + memory for a sliding window when Z_FINISH is used. + + If a preset dictionary is needed after this call (see inflateSetDictionary + below), inflate sets strm->adler to the Adler-32 checksum of the dictionary + chosen by the compressor and returns Z_NEED_DICT; otherwise it sets + strm->adler to the Adler-32 checksum of all output produced so far (that is, + total_out bytes) and returns Z_OK, Z_STREAM_END or an error code as described + below. At the end of the stream, inflate() checks that its computed Adler-32 + checksum is equal to that saved by the compressor and returns Z_STREAM_END + only if the checksum is correct. + + inflate() can decompress and check either zlib-wrapped or gzip-wrapped + deflate data. The header type is detected automatically, if requested when + initializing with inflateInit2(). Any information contained in the gzip + header is not retained unless inflateGetHeader() is used. When processing + gzip-wrapped deflate data, strm->adler32 is set to the CRC-32 of the output + produced so far. The CRC-32 is checked against the gzip trailer, as is the + uncompressed length, modulo 2^32. + + inflate() returns Z_OK if some progress has been made (more input processed + or more output produced), Z_STREAM_END if the end of the compressed data has + been reached and all uncompressed output has been produced, Z_NEED_DICT if a + preset dictionary is needed at this point, Z_DATA_ERROR if the input data was + corrupted (input stream not conforming to the zlib format or incorrect check + value, in which case strm->msg points to a string with a more specific + error), Z_STREAM_ERROR if the stream structure was inconsistent (for example + next_in or next_out was Z_NULL, or the state was inadvertently written over + by the application), Z_MEM_ERROR if there was not enough memory, Z_BUF_ERROR + if no progress was possible or if there was not enough room in the output + buffer when Z_FINISH is used. Note that Z_BUF_ERROR is not fatal, and + inflate() can be called again with more input and more output space to + continue decompressing. If Z_DATA_ERROR is returned, the application may + then call inflateSync() to look for a good compression block if a partial + recovery of the data is to be attempted. +*/ + + +ZEXTERN int ZEXPORT inflateEnd OF((z_streamp strm)); +/* + All dynamically allocated data structures for this stream are freed. + This function discards any unprocessed input and does not flush any pending + output. + + inflateEnd returns Z_OK if success, or Z_STREAM_ERROR if the stream state + was inconsistent. +*/ + + + /* Advanced functions */ + +/* + The following functions are needed only in some special applications. +*/ + +/* +ZEXTERN int ZEXPORT deflateInit2 OF((z_streamp strm, + int level, + int method, + int windowBits, + int memLevel, + int strategy)); + + This is another version of deflateInit with more compression options. The + fields zalloc, zfree and opaque must be initialized before by the caller. + + The method parameter is the compression method. It must be Z_DEFLATED in + this version of the library. + + The windowBits parameter is the base two logarithm of the window size + (the size of the history buffer). It should be in the range 8..15 for this + version of the library. Larger values of this parameter result in better + compression at the expense of memory usage. The default value is 15 if + deflateInit is used instead. + + For the current implementation of deflate(), a windowBits value of 8 (a + window size of 256 bytes) is not supported. As a result, a request for 8 + will result in 9 (a 512-byte window). In that case, providing 8 to + inflateInit2() will result in an error when the zlib header with 9 is + checked against the initialization of inflate(). The remedy is to not use 8 + with deflateInit2() with this initialization, or at least in that case use 9 + with inflateInit2(). + + windowBits can also be -8..-15 for raw deflate. In this case, -windowBits + determines the window size. deflate() will then generate raw deflate data + with no zlib header or trailer, and will not compute a check value. + + windowBits can also be greater than 15 for optional gzip encoding. Add + 16 to windowBits to write a simple gzip header and trailer around the + compressed data instead of a zlib wrapper. The gzip header will have no + file name, no extra data, no comment, no modification time (set to zero), no + header crc, and the operating system will be set to the appropriate value, + if the operating system was determined at compile time. If a gzip stream is + being written, strm->adler is a CRC-32 instead of an Adler-32. + + For raw deflate or gzip encoding, a request for a 256-byte window is + rejected as invalid, since only the zlib header provides a means of + transmitting the window size to the decompressor. + + The memLevel parameter specifies how much memory should be allocated + for the internal compression state. memLevel=1 uses minimum memory but is + slow and reduces compression ratio; memLevel=9 uses maximum memory for + optimal speed. The default value is 8. See zconf.h for total memory usage + as a function of windowBits and memLevel. + + The strategy parameter is used to tune the compression algorithm. Use the + value Z_DEFAULT_STRATEGY for normal data, Z_FILTERED for data produced by a + filter (or predictor), Z_HUFFMAN_ONLY to force Huffman encoding only (no + string match), or Z_RLE to limit match distances to one (run-length + encoding). Filtered data consists mostly of small values with a somewhat + random distribution. In this case, the compression algorithm is tuned to + compress them better. The effect of Z_FILTERED is to force more Huffman + coding and less string matching; it is somewhat intermediate between + Z_DEFAULT_STRATEGY and Z_HUFFMAN_ONLY. Z_RLE is designed to be almost as + fast as Z_HUFFMAN_ONLY, but give better compression for PNG image data. The + strategy parameter only affects the compression ratio but not the + correctness of the compressed output even if it is not set appropriately. + Z_FIXED prevents the use of dynamic Huffman codes, allowing for a simpler + decoder for special applications. + + deflateInit2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_STREAM_ERROR if any parameter is invalid (such as an invalid + method), or Z_VERSION_ERROR if the zlib library version (zlib_version) is + incompatible with the version assumed by the caller (ZLIB_VERSION). msg is + set to null if there is no error message. deflateInit2 does not perform any + compression: this will be done by deflate(). +*/ + +ZEXTERN int ZEXPORT deflateSetDictionary OF((z_streamp strm, + const Bytef *dictionary, + uInt dictLength)); +/* + Initializes the compression dictionary from the given byte sequence + without producing any compressed output. When using the zlib format, this + function must be called immediately after deflateInit, deflateInit2 or + deflateReset, and before any call of deflate. When doing raw deflate, this + function must be called either before any call of deflate, or immediately + after the completion of a deflate block, i.e. after all input has been + consumed and all output has been delivered when using any of the flush + options Z_BLOCK, Z_PARTIAL_FLUSH, Z_SYNC_FLUSH, or Z_FULL_FLUSH. The + compressor and decompressor must use exactly the same dictionary (see + inflateSetDictionary). + + The dictionary should consist of strings (byte sequences) that are likely + to be encountered later in the data to be compressed, with the most commonly + used strings preferably put towards the end of the dictionary. Using a + dictionary is most useful when the data to be compressed is short and can be + predicted with good accuracy; the data can then be compressed better than + with the default empty dictionary. + + Depending on the size of the compression data structures selected by + deflateInit or deflateInit2, a part of the dictionary may in effect be + discarded, for example if the dictionary is larger than the window size + provided in deflateInit or deflateInit2. Thus the strings most likely to be + useful should be put at the end of the dictionary, not at the front. In + addition, the current implementation of deflate will use at most the window + size minus 262 bytes of the provided dictionary. + + Upon return of this function, strm->adler is set to the Adler-32 value + of the dictionary; the decompressor may later use this value to determine + which dictionary has been used by the compressor. (The Adler-32 value + applies to the whole dictionary even if only a subset of the dictionary is + actually used by the compressor.) If a raw deflate was requested, then the + Adler-32 value is not computed and strm->adler is not set. + + deflateSetDictionary returns Z_OK if success, or Z_STREAM_ERROR if a + parameter is invalid (e.g. dictionary being Z_NULL) or the stream state is + inconsistent (for example if deflate has already been called for this stream + or if not at a block boundary for raw deflate). deflateSetDictionary does + not perform any compression: this will be done by deflate(). +*/ + +ZEXTERN int ZEXPORT deflateGetDictionary OF((z_streamp strm, + Bytef *dictionary, + uInt *dictLength)); +/* + Returns the sliding dictionary being maintained by deflate. dictLength is + set to the number of bytes in the dictionary, and that many bytes are copied + to dictionary. dictionary must have enough space, where 32768 bytes is + always enough. If deflateGetDictionary() is called with dictionary equal to + Z_NULL, then only the dictionary length is returned, and nothing is copied. + Similarly, if dictLength is Z_NULL, then it is not set. + + deflateGetDictionary() may return a length less than the window size, even + when more than the window size in input has been provided. It may return up + to 258 bytes less in that case, due to how zlib's implementation of deflate + manages the sliding window and lookahead for matches, where matches can be + up to 258 bytes long. If the application needs the last window-size bytes of + input, then that would need to be saved by the application outside of zlib. + + deflateGetDictionary returns Z_OK on success, or Z_STREAM_ERROR if the + stream state is inconsistent. +*/ + +ZEXTERN int ZEXPORT deflateCopy OF((z_streamp dest, + z_streamp source)); +/* + Sets the destination stream as a complete copy of the source stream. + + This function can be useful when several compression strategies will be + tried, for example when there are several ways of pre-processing the input + data with a filter. The streams that will be discarded should then be freed + by calling deflateEnd. Note that deflateCopy duplicates the internal + compression state which can be quite large, so this strategy is slow and can + consume lots of memory. + + deflateCopy returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_STREAM_ERROR if the source stream state was inconsistent + (such as zalloc being Z_NULL). msg is left unchanged in both source and + destination. +*/ + +ZEXTERN int ZEXPORT deflateReset OF((z_streamp strm)); +/* + This function is equivalent to deflateEnd followed by deflateInit, but + does not free and reallocate the internal compression state. The stream + will leave the compression level and any other attributes that may have been + set unchanged. + + deflateReset returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent (such as zalloc or state being Z_NULL). +*/ + +ZEXTERN int ZEXPORT deflateParams OF((z_streamp strm, + int level, + int strategy)); +/* + Dynamically update the compression level and compression strategy. The + interpretation of level and strategy is as in deflateInit2(). This can be + used to switch between compression and straight copy of the input data, or + to switch to a different kind of input data requiring a different strategy. + If the compression approach (which is a function of the level) or the + strategy is changed, and if there have been any deflate() calls since the + state was initialized or reset, then the input available so far is + compressed with the old level and strategy using deflate(strm, Z_BLOCK). + There are three approaches for the compression levels 0, 1..3, and 4..9 + respectively. The new level and strategy will take effect at the next call + of deflate(). + + If a deflate(strm, Z_BLOCK) is performed by deflateParams(), and it does + not have enough output space to complete, then the parameter change will not + take effect. In this case, deflateParams() can be called again with the + same parameters and more output space to try again. + + In order to assure a change in the parameters on the first try, the + deflate stream should be flushed using deflate() with Z_BLOCK or other flush + request until strm.avail_out is not zero, before calling deflateParams(). + Then no more input data should be provided before the deflateParams() call. + If this is done, the old level and strategy will be applied to the data + compressed before deflateParams(), and the new level and strategy will be + applied to the the data compressed after deflateParams(). + + deflateParams returns Z_OK on success, Z_STREAM_ERROR if the source stream + state was inconsistent or if a parameter was invalid, or Z_BUF_ERROR if + there was not enough output space to complete the compression of the + available input data before a change in the strategy or approach. Note that + in the case of a Z_BUF_ERROR, the parameters are not changed. A return + value of Z_BUF_ERROR is not fatal, in which case deflateParams() can be + retried with more output space. +*/ + +ZEXTERN int ZEXPORT deflateTune OF((z_streamp strm, + int good_length, + int max_lazy, + int nice_length, + int max_chain)); +/* + Fine tune deflate's internal compression parameters. This should only be + used by someone who understands the algorithm used by zlib's deflate for + searching for the best matching string, and even then only by the most + fanatic optimizer trying to squeeze out the last compressed bit for their + specific input data. Read the deflate.c source code for the meaning of the + max_lazy, good_length, nice_length, and max_chain parameters. + + deflateTune() can be called after deflateInit() or deflateInit2(), and + returns Z_OK on success, or Z_STREAM_ERROR for an invalid deflate stream. + */ + +ZEXTERN uLong ZEXPORT deflateBound OF((z_streamp strm, + uLong sourceLen)); +/* + deflateBound() returns an upper bound on the compressed size after + deflation of sourceLen bytes. It must be called after deflateInit() or + deflateInit2(), and after deflateSetHeader(), if used. This would be used + to allocate an output buffer for deflation in a single pass, and so would be + called before deflate(). If that first deflate() call is provided the + sourceLen input bytes, an output buffer allocated to the size returned by + deflateBound(), and the flush value Z_FINISH, then deflate() is guaranteed + to return Z_STREAM_END. Note that it is possible for the compressed size to + be larger than the value returned by deflateBound() if flush options other + than Z_FINISH or Z_NO_FLUSH are used. +*/ + +ZEXTERN int ZEXPORT deflatePending OF((z_streamp strm, + unsigned *pending, + int *bits)); +/* + deflatePending() returns the number of bytes and bits of output that have + been generated, but not yet provided in the available output. The bytes not + provided would be due to the available output space having being consumed. + The number of bits of output not provided are between 0 and 7, where they + await more bits to join them in order to fill out a full byte. If pending + or bits are Z_NULL, then those values are not set. + + deflatePending returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent. + */ + +ZEXTERN int ZEXPORT deflatePrime OF((z_streamp strm, + int bits, + int value)); +/* + deflatePrime() inserts bits in the deflate output stream. The intent + is that this function is used to start off the deflate output with the bits + leftover from a previous deflate stream when appending to it. As such, this + function can only be used for raw deflate, and must be used before the first + deflate() call after a deflateInit2() or deflateReset(). bits must be less + than or equal to 16, and that many of the least significant bits of value + will be inserted in the output. + + deflatePrime returns Z_OK if success, Z_BUF_ERROR if there was not enough + room in the internal buffer to insert the bits, or Z_STREAM_ERROR if the + source stream state was inconsistent. +*/ + +ZEXTERN int ZEXPORT deflateSetHeader OF((z_streamp strm, + gz_headerp head)); +/* + deflateSetHeader() provides gzip header information for when a gzip + stream is requested by deflateInit2(). deflateSetHeader() may be called + after deflateInit2() or deflateReset() and before the first call of + deflate(). The text, time, os, extra field, name, and comment information + in the provided gz_header structure are written to the gzip header (xflag is + ignored -- the extra flags are set according to the compression level). The + caller must assure that, if not Z_NULL, name and comment are terminated with + a zero byte, and that if extra is not Z_NULL, that extra_len bytes are + available there. If hcrc is true, a gzip header crc is included. Note that + the current versions of the command-line version of gzip (up through version + 1.3.x) do not support header crc's, and will report that it is a "multi-part + gzip file" and give up. + + If deflateSetHeader is not used, the default gzip header has text false, + the time set to zero, and os set to 255, with no extra, name, or comment + fields. The gzip header is returned to the default state by deflateReset(). + + deflateSetHeader returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent. +*/ + +/* +ZEXTERN int ZEXPORT inflateInit2 OF((z_streamp strm, + int windowBits)); + + This is another version of inflateInit with an extra parameter. The + fields next_in, avail_in, zalloc, zfree and opaque must be initialized + before by the caller. + + The windowBits parameter is the base two logarithm of the maximum window + size (the size of the history buffer). It should be in the range 8..15 for + this version of the library. The default value is 15 if inflateInit is used + instead. windowBits must be greater than or equal to the windowBits value + provided to deflateInit2() while compressing, or it must be equal to 15 if + deflateInit2() was not used. If a compressed stream with a larger window + size is given as input, inflate() will return with the error code + Z_DATA_ERROR instead of trying to allocate a larger window. + + windowBits can also be zero to request that inflate use the window size in + the zlib header of the compressed stream. + + windowBits can also be -8..-15 for raw inflate. In this case, -windowBits + determines the window size. inflate() will then process raw deflate data, + not looking for a zlib or gzip header, not generating a check value, and not + looking for any check values for comparison at the end of the stream. This + is for use with other formats that use the deflate compressed data format + such as zip. Those formats provide their own check values. If a custom + format is developed using the raw deflate format for compressed data, it is + recommended that a check value such as an Adler-32 or a CRC-32 be applied to + the uncompressed data as is done in the zlib, gzip, and zip formats. For + most applications, the zlib format should be used as is. Note that comments + above on the use in deflateInit2() applies to the magnitude of windowBits. + + windowBits can also be greater than 15 for optional gzip decoding. Add + 32 to windowBits to enable zlib and gzip decoding with automatic header + detection, or add 16 to decode only the gzip format (the zlib format will + return a Z_DATA_ERROR). If a gzip stream is being decoded, strm->adler is a + CRC-32 instead of an Adler-32. Unlike the gunzip utility and gzread() (see + below), inflate() will *not* automatically decode concatenated gzip members. + inflate() will return Z_STREAM_END at the end of the gzip member. The state + would need to be reset to continue decoding a subsequent gzip member. This + *must* be done if there is more data after a gzip member, in order for the + decompression to be compliant with the gzip standard (RFC 1952). + + inflateInit2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_VERSION_ERROR if the zlib library version is incompatible with the + version assumed by the caller, or Z_STREAM_ERROR if the parameters are + invalid, such as a null pointer to the structure. msg is set to null if + there is no error message. inflateInit2 does not perform any decompression + apart from possibly reading the zlib header if present: actual decompression + will be done by inflate(). (So next_in and avail_in may be modified, but + next_out and avail_out are unused and unchanged.) The current implementation + of inflateInit2() does not process any header information -- that is + deferred until inflate() is called. +*/ + +ZEXTERN int ZEXPORT inflateSetDictionary OF((z_streamp strm, + const Bytef *dictionary, + uInt dictLength)); +/* + Initializes the decompression dictionary from the given uncompressed byte + sequence. This function must be called immediately after a call of inflate, + if that call returned Z_NEED_DICT. The dictionary chosen by the compressor + can be determined from the Adler-32 value returned by that call of inflate. + The compressor and decompressor must use exactly the same dictionary (see + deflateSetDictionary). For raw inflate, this function can be called at any + time to set the dictionary. If the provided dictionary is smaller than the + window and there is already data in the window, then the provided dictionary + will amend what's there. The application must insure that the dictionary + that was used for compression is provided. + + inflateSetDictionary returns Z_OK if success, Z_STREAM_ERROR if a + parameter is invalid (e.g. dictionary being Z_NULL) or the stream state is + inconsistent, Z_DATA_ERROR if the given dictionary doesn't match the + expected one (incorrect Adler-32 value). inflateSetDictionary does not + perform any decompression: this will be done by subsequent calls of + inflate(). +*/ + +ZEXTERN int ZEXPORT inflateGetDictionary OF((z_streamp strm, + Bytef *dictionary, + uInt *dictLength)); +/* + Returns the sliding dictionary being maintained by inflate. dictLength is + set to the number of bytes in the dictionary, and that many bytes are copied + to dictionary. dictionary must have enough space, where 32768 bytes is + always enough. If inflateGetDictionary() is called with dictionary equal to + Z_NULL, then only the dictionary length is returned, and nothing is copied. + Similarly, if dictLength is Z_NULL, then it is not set. + + inflateGetDictionary returns Z_OK on success, or Z_STREAM_ERROR if the + stream state is inconsistent. +*/ + +ZEXTERN int ZEXPORT inflateSync OF((z_streamp strm)); +/* + Skips invalid compressed data until a possible full flush point (see above + for the description of deflate with Z_FULL_FLUSH) can be found, or until all + available input is skipped. No output is provided. + + inflateSync searches for a 00 00 FF FF pattern in the compressed data. + All full flush points have this pattern, but not all occurrences of this + pattern are full flush points. + + inflateSync returns Z_OK if a possible full flush point has been found, + Z_BUF_ERROR if no more input was provided, Z_DATA_ERROR if no flush point + has been found, or Z_STREAM_ERROR if the stream structure was inconsistent. + In the success case, the application may save the current current value of + total_in which indicates where valid compressed data was found. In the + error case, the application may repeatedly call inflateSync, providing more + input each time, until success or end of the input data. +*/ + +ZEXTERN int ZEXPORT inflateCopy OF((z_streamp dest, + z_streamp source)); +/* + Sets the destination stream as a complete copy of the source stream. + + This function can be useful when randomly accessing a large stream. The + first pass through the stream can periodically record the inflate state, + allowing restarting inflate at those points when randomly accessing the + stream. + + inflateCopy returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_STREAM_ERROR if the source stream state was inconsistent + (such as zalloc being Z_NULL). msg is left unchanged in both source and + destination. +*/ + +ZEXTERN int ZEXPORT inflateReset OF((z_streamp strm)); +/* + This function is equivalent to inflateEnd followed by inflateInit, + but does not free and reallocate the internal decompression state. The + stream will keep attributes that may have been set by inflateInit2. + + inflateReset returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent (such as zalloc or state being Z_NULL). +*/ + +ZEXTERN int ZEXPORT inflateReset2 OF((z_streamp strm, + int windowBits)); +/* + This function is the same as inflateReset, but it also permits changing + the wrap and window size requests. The windowBits parameter is interpreted + the same as it is for inflateInit2. If the window size is changed, then the + memory allocated for the window is freed, and the window will be reallocated + by inflate() if needed. + + inflateReset2 returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent (such as zalloc or state being Z_NULL), or if + the windowBits parameter is invalid. +*/ + +ZEXTERN int ZEXPORT inflatePrime OF((z_streamp strm, + int bits, + int value)); +/* + This function inserts bits in the inflate input stream. The intent is + that this function is used to start inflating at a bit position in the + middle of a byte. The provided bits will be used before any bytes are used + from next_in. This function should only be used with raw inflate, and + should be used before the first inflate() call after inflateInit2() or + inflateReset(). bits must be less than or equal to 16, and that many of the + least significant bits of value will be inserted in the input. + + If bits is negative, then the input stream bit buffer is emptied. Then + inflatePrime() can be called again to put bits in the buffer. This is used + to clear out bits leftover after feeding inflate a block description prior + to feeding inflate codes. + + inflatePrime returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent. +*/ + +ZEXTERN long ZEXPORT inflateMark OF((z_streamp strm)); +/* + This function returns two values, one in the lower 16 bits of the return + value, and the other in the remaining upper bits, obtained by shifting the + return value down 16 bits. If the upper value is -1 and the lower value is + zero, then inflate() is currently decoding information outside of a block. + If the upper value is -1 and the lower value is non-zero, then inflate is in + the middle of a stored block, with the lower value equaling the number of + bytes from the input remaining to copy. If the upper value is not -1, then + it is the number of bits back from the current bit position in the input of + the code (literal or length/distance pair) currently being processed. In + that case the lower value is the number of bytes already emitted for that + code. + + A code is being processed if inflate is waiting for more input to complete + decoding of the code, or if it has completed decoding but is waiting for + more output space to write the literal or match data. + + inflateMark() is used to mark locations in the input data for random + access, which may be at bit positions, and to note those cases where the + output of a code may span boundaries of random access blocks. The current + location in the input stream can be determined from avail_in and data_type + as noted in the description for the Z_BLOCK flush parameter for inflate. + + inflateMark returns the value noted above, or -65536 if the provided + source stream state was inconsistent. +*/ + +ZEXTERN int ZEXPORT inflateGetHeader OF((z_streamp strm, + gz_headerp head)); +/* + inflateGetHeader() requests that gzip header information be stored in the + provided gz_header structure. inflateGetHeader() may be called after + inflateInit2() or inflateReset(), and before the first call of inflate(). + As inflate() processes the gzip stream, head->done is zero until the header + is completed, at which time head->done is set to one. If a zlib stream is + being decoded, then head->done is set to -1 to indicate that there will be + no gzip header information forthcoming. Note that Z_BLOCK or Z_TREES can be + used to force inflate() to return immediately after header processing is + complete and before any actual data is decompressed. + + The text, time, xflags, and os fields are filled in with the gzip header + contents. hcrc is set to true if there is a header CRC. (The header CRC + was valid if done is set to one.) If extra is not Z_NULL, then extra_max + contains the maximum number of bytes to write to extra. Once done is true, + extra_len contains the actual extra field length, and extra contains the + extra field, or that field truncated if extra_max is less than extra_len. + If name is not Z_NULL, then up to name_max characters are written there, + terminated with a zero unless the length is greater than name_max. If + comment is not Z_NULL, then up to comm_max characters are written there, + terminated with a zero unless the length is greater than comm_max. When any + of extra, name, or comment are not Z_NULL and the respective field is not + present in the header, then that field is set to Z_NULL to signal its + absence. This allows the use of deflateSetHeader() with the returned + structure to duplicate the header. However if those fields are set to + allocated memory, then the application will need to save those pointers + elsewhere so that they can be eventually freed. + + If inflateGetHeader is not used, then the header information is simply + discarded. The header is always checked for validity, including the header + CRC if present. inflateReset() will reset the process to discard the header + information. The application would need to call inflateGetHeader() again to + retrieve the header from the next gzip stream. + + inflateGetHeader returns Z_OK if success, or Z_STREAM_ERROR if the source + stream state was inconsistent. +*/ + +/* +ZEXTERN int ZEXPORT inflateBackInit OF((z_streamp strm, int windowBits, + unsigned char FAR *window)); + + Initialize the internal stream state for decompression using inflateBack() + calls. The fields zalloc, zfree and opaque in strm must be initialized + before the call. If zalloc and zfree are Z_NULL, then the default library- + derived memory allocation routines are used. windowBits is the base two + logarithm of the window size, in the range 8..15. window is a caller + supplied buffer of that size. Except for special applications where it is + assured that deflate was used with small window sizes, windowBits must be 15 + and a 32K byte window must be supplied to be able to decompress general + deflate streams. + + See inflateBack() for the usage of these routines. + + inflateBackInit will return Z_OK on success, Z_STREAM_ERROR if any of + the parameters are invalid, Z_MEM_ERROR if the internal state could not be + allocated, or Z_VERSION_ERROR if the version of the library does not match + the version of the header file. +*/ + +typedef unsigned (*in_func) OF((void FAR *, + z_const unsigned char FAR * FAR *)); +typedef int (*out_func) OF((void FAR *, unsigned char FAR *, unsigned)); + +ZEXTERN int ZEXPORT inflateBack OF((z_streamp strm, + in_func in, void FAR *in_desc, + out_func out, void FAR *out_desc)); +/* + inflateBack() does a raw inflate with a single call using a call-back + interface for input and output. This is potentially more efficient than + inflate() for file i/o applications, in that it avoids copying between the + output and the sliding window by simply making the window itself the output + buffer. inflate() can be faster on modern CPUs when used with large + buffers. inflateBack() trusts the application to not change the output + buffer passed by the output function, at least until inflateBack() returns. + + inflateBackInit() must be called first to allocate the internal state + and to initialize the state with the user-provided window buffer. + inflateBack() may then be used multiple times to inflate a complete, raw + deflate stream with each call. inflateBackEnd() is then called to free the + allocated state. + + A raw deflate stream is one with no zlib or gzip header or trailer. + This routine would normally be used in a utility that reads zip or gzip + files and writes out uncompressed files. The utility would decode the + header and process the trailer on its own, hence this routine expects only + the raw deflate stream to decompress. This is different from the default + behavior of inflate(), which expects a zlib header and trailer around the + deflate stream. + + inflateBack() uses two subroutines supplied by the caller that are then + called by inflateBack() for input and output. inflateBack() calls those + routines until it reads a complete deflate stream and writes out all of the + uncompressed data, or until it encounters an error. The function's + parameters and return types are defined above in the in_func and out_func + typedefs. inflateBack() will call in(in_desc, &buf) which should return the + number of bytes of provided input, and a pointer to that input in buf. If + there is no input available, in() must return zero -- buf is ignored in that + case -- and inflateBack() will return a buffer error. inflateBack() will + call out(out_desc, buf, len) to write the uncompressed data buf[0..len-1]. + out() should return zero on success, or non-zero on failure. If out() + returns non-zero, inflateBack() will return with an error. Neither in() nor + out() are permitted to change the contents of the window provided to + inflateBackInit(), which is also the buffer that out() uses to write from. + The length written by out() will be at most the window size. Any non-zero + amount of input may be provided by in(). + + For convenience, inflateBack() can be provided input on the first call by + setting strm->next_in and strm->avail_in. If that input is exhausted, then + in() will be called. Therefore strm->next_in must be initialized before + calling inflateBack(). If strm->next_in is Z_NULL, then in() will be called + immediately for input. If strm->next_in is not Z_NULL, then strm->avail_in + must also be initialized, and then if strm->avail_in is not zero, input will + initially be taken from strm->next_in[0 .. strm->avail_in - 1]. + + The in_desc and out_desc parameters of inflateBack() is passed as the + first parameter of in() and out() respectively when they are called. These + descriptors can be optionally used to pass any information that the caller- + supplied in() and out() functions need to do their job. + + On return, inflateBack() will set strm->next_in and strm->avail_in to + pass back any unused input that was provided by the last in() call. The + return values of inflateBack() can be Z_STREAM_END on success, Z_BUF_ERROR + if in() or out() returned an error, Z_DATA_ERROR if there was a format error + in the deflate stream (in which case strm->msg is set to indicate the nature + of the error), or Z_STREAM_ERROR if the stream was not properly initialized. + In the case of Z_BUF_ERROR, an input or output error can be distinguished + using strm->next_in which will be Z_NULL only if in() returned an error. If + strm->next_in is not Z_NULL, then the Z_BUF_ERROR was due to out() returning + non-zero. (in() will always be called before out(), so strm->next_in is + assured to be defined if out() returns non-zero.) Note that inflateBack() + cannot return Z_OK. +*/ + +ZEXTERN int ZEXPORT inflateBackEnd OF((z_streamp strm)); +/* + All memory allocated by inflateBackInit() is freed. + + inflateBackEnd() returns Z_OK on success, or Z_STREAM_ERROR if the stream + state was inconsistent. +*/ + +ZEXTERN uLong ZEXPORT zlibCompileFlags OF((void)); +/* Return flags indicating compile-time options. + + Type sizes, two bits each, 00 = 16 bits, 01 = 32, 10 = 64, 11 = other: + 1.0: size of uInt + 3.2: size of uLong + 5.4: size of voidpf (pointer) + 7.6: size of z_off_t + + Compiler, assembler, and debug options: + 8: ZLIB_DEBUG + 9: ASMV or ASMINF -- use ASM code + 10: ZLIB_WINAPI -- exported functions use the WINAPI calling convention + 11: 0 (reserved) + + One-time table building (smaller code, but not thread-safe if true): + 12: BUILDFIXED -- build static block decoding tables when needed + 13: DYNAMIC_CRC_TABLE -- build CRC calculation tables when needed + 14,15: 0 (reserved) + + Library content (indicates missing functionality): + 16: NO_GZCOMPRESS -- gz* functions cannot compress (to avoid linking + deflate code when not needed) + 17: NO_GZIP -- deflate can't write gzip streams, and inflate can't detect + and decode gzip streams (to avoid linking crc code) + 18-19: 0 (reserved) + + Operation variations (changes in library functionality): + 20: PKZIP_BUG_WORKAROUND -- slightly more permissive inflate + 21: FASTEST -- deflate algorithm with only one, lowest compression level + 22,23: 0 (reserved) + + The sprintf variant used by gzprintf (zero is best): + 24: 0 = vs*, 1 = s* -- 1 means limited to 20 arguments after the format + 25: 0 = *nprintf, 1 = *printf -- 1 means gzprintf() not secure! + 26: 0 = returns value, 1 = void -- 1 means inferred string length returned + + Remainder: + 27-31: 0 (reserved) + */ + +#ifndef Z_SOLO + + /* utility functions */ + +/* + The following utility functions are implemented on top of the basic + stream-oriented functions. To simplify the interface, some default options + are assumed (compression level and memory usage, standard memory allocation + functions). The source code of these utility functions can be modified if + you need special options. +*/ + +ZEXTERN int ZEXPORT compress OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen)); +/* + Compresses the source buffer into the destination buffer. sourceLen is + the byte length of the source buffer. Upon entry, destLen is the total size + of the destination buffer, which must be at least the value returned by + compressBound(sourceLen). Upon exit, destLen is the actual size of the + compressed data. compress() is equivalent to compress2() with a level + parameter of Z_DEFAULT_COMPRESSION. + + compress returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_BUF_ERROR if there was not enough room in the output + buffer. +*/ + +ZEXTERN int ZEXPORT compress2 OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen, + int level)); +/* + Compresses the source buffer into the destination buffer. The level + parameter has the same meaning as in deflateInit. sourceLen is the byte + length of the source buffer. Upon entry, destLen is the total size of the + destination buffer, which must be at least the value returned by + compressBound(sourceLen). Upon exit, destLen is the actual size of the + compressed data. + + compress2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_BUF_ERROR if there was not enough room in the output buffer, + Z_STREAM_ERROR if the level parameter is invalid. +*/ + +ZEXTERN uLong ZEXPORT compressBound OF((uLong sourceLen)); +/* + compressBound() returns an upper bound on the compressed size after + compress() or compress2() on sourceLen bytes. It would be used before a + compress() or compress2() call to allocate the destination buffer. +*/ + +ZEXTERN int ZEXPORT uncompress OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen)); +/* + Decompresses the source buffer into the destination buffer. sourceLen is + the byte length of the source buffer. Upon entry, destLen is the total size + of the destination buffer, which must be large enough to hold the entire + uncompressed data. (The size of the uncompressed data must have been saved + previously by the compressor and transmitted to the decompressor by some + mechanism outside the scope of this compression library.) Upon exit, destLen + is the actual size of the uncompressed data. + + uncompress returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_BUF_ERROR if there was not enough room in the output + buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete. In + the case where there is not enough room, uncompress() will fill the output + buffer with the uncompressed data up to that point. +*/ + +ZEXTERN int ZEXPORT uncompress2 OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong *sourceLen)); +/* + Same as uncompress, except that sourceLen is a pointer, where the + length of the source is *sourceLen. On return, *sourceLen is the number of + source bytes consumed. +*/ + + /* gzip file access functions */ + +/* + This library supports reading and writing files in gzip (.gz) format with + an interface similar to that of stdio, using the functions that start with + "gz". The gzip format is different from the zlib format. gzip is a gzip + wrapper, documented in RFC 1952, wrapped around a deflate stream. +*/ + +typedef struct gzFile_s *gzFile; /* semi-opaque gzip file descriptor */ + +/* +ZEXTERN gzFile ZEXPORT gzopen OF((const char *path, const char *mode)); + + Open the gzip (.gz) file at path for reading and decompressing, or + compressing and writing. The mode parameter is as in fopen ("rb" or "wb") + but can also include a compression level ("wb9") or a strategy: 'f' for + filtered data as in "wb6f", 'h' for Huffman-only compression as in "wb1h", + 'R' for run-length encoding as in "wb1R", or 'F' for fixed code compression + as in "wb9F". (See the description of deflateInit2 for more information + about the strategy parameter.) 'T' will request transparent writing or + appending with no compression and not using the gzip format. + + "a" can be used instead of "w" to request that the gzip stream that will + be written be appended to the file. "+" will result in an error, since + reading and writing to the same gzip file is not supported. The addition of + "x" when writing will create the file exclusively, which fails if the file + already exists. On systems that support it, the addition of "e" when + reading or writing will set the flag to close the file on an execve() call. + + These functions, as well as gzip, will read and decode a sequence of gzip + streams in a file. The append function of gzopen() can be used to create + such a file. (Also see gzflush() for another way to do this.) When + appending, gzopen does not test whether the file begins with a gzip stream, + nor does it look for the end of the gzip streams to begin appending. gzopen + will simply append a gzip stream to the existing file. + + gzopen can be used to read a file which is not in gzip format; in this + case gzread will directly read from the file without decompression. When + reading, this will be detected automatically by looking for the magic two- + byte gzip header. + + gzopen returns NULL if the file could not be opened, if there was + insufficient memory to allocate the gzFile state, or if an invalid mode was + specified (an 'r', 'w', or 'a' was not provided, or '+' was provided). + errno can be checked to determine if the reason gzopen failed was that the + file could not be opened. +*/ + +ZEXTERN gzFile ZEXPORT gzdopen OF((int fd, const char *mode)); +/* + Associate a gzFile with the file descriptor fd. File descriptors are + obtained from calls like open, dup, creat, pipe or fileno (if the file has + been previously opened with fopen). The mode parameter is as in gzopen. + + The next call of gzclose on the returned gzFile will also close the file + descriptor fd, just like fclose(fdopen(fd, mode)) closes the file descriptor + fd. If you want to keep fd open, use fd = dup(fd_keep); gz = gzdopen(fd, + mode);. The duplicated descriptor should be saved to avoid a leak, since + gzdopen does not close fd if it fails. If you are using fileno() to get the + file descriptor from a FILE *, then you will have to use dup() to avoid + double-close()ing the file descriptor. Both gzclose() and fclose() will + close the associated file descriptor, so they need to have different file + descriptors. + + gzdopen returns NULL if there was insufficient memory to allocate the + gzFile state, if an invalid mode was specified (an 'r', 'w', or 'a' was not + provided, or '+' was provided), or if fd is -1. The file descriptor is not + used until the next gz* read, write, seek, or close operation, so gzdopen + will not detect if fd is invalid (unless fd is -1). +*/ + +ZEXTERN int ZEXPORT gzbuffer OF((gzFile file, unsigned size)); +/* + Set the internal buffer size used by this library's functions for file to + size. The default buffer size is 8192 bytes. This function must be called + after gzopen() or gzdopen(), and before any other calls that read or write + the file. The buffer memory allocation is always deferred to the first read + or write. Three times that size in buffer space is allocated. A larger + buffer size of, for example, 64K or 128K bytes will noticeably increase the + speed of decompression (reading). + + The new buffer size also affects the maximum length for gzprintf(). + + gzbuffer() returns 0 on success, or -1 on failure, such as being called + too late. +*/ + +ZEXTERN int ZEXPORT gzsetparams OF((gzFile file, int level, int strategy)); +/* + Dynamically update the compression level and strategy for file. See the + description of deflateInit2 for the meaning of these parameters. Previously + provided data is flushed before applying the parameter changes. + + gzsetparams returns Z_OK if success, Z_STREAM_ERROR if the file was not + opened for writing, Z_ERRNO if there is an error writing the flushed data, + or Z_MEM_ERROR if there is a memory allocation error. +*/ + +ZEXTERN int ZEXPORT gzread OF((gzFile file, voidp buf, unsigned len)); +/* + Read and decompress up to len uncompressed bytes from file into buf. If + the input file is not in gzip format, gzread copies the given number of + bytes into the buffer directly from the file. + + After reaching the end of a gzip stream in the input, gzread will continue + to read, looking for another gzip stream. Any number of gzip streams may be + concatenated in the input file, and will all be decompressed by gzread(). + If something other than a gzip stream is encountered after a gzip stream, + that remaining trailing garbage is ignored (and no error is returned). + + gzread can be used to read a gzip file that is being concurrently written. + Upon reaching the end of the input, gzread will return with the available + data. If the error code returned by gzerror is Z_OK or Z_BUF_ERROR, then + gzclearerr can be used to clear the end of file indicator in order to permit + gzread to be tried again. Z_OK indicates that a gzip stream was completed + on the last gzread. Z_BUF_ERROR indicates that the input file ended in the + middle of a gzip stream. Note that gzread does not return -1 in the event + of an incomplete gzip stream. This error is deferred until gzclose(), which + will return Z_BUF_ERROR if the last gzread ended in the middle of a gzip + stream. Alternatively, gzerror can be used before gzclose to detect this + case. + + gzread returns the number of uncompressed bytes actually read, less than + len for end of file, or -1 for error. If len is too large to fit in an int, + then nothing is read, -1 is returned, and the error state is set to + Z_STREAM_ERROR. +*/ + +ZEXTERN z_size_t ZEXPORT gzfread OF((voidp buf, z_size_t size, z_size_t nitems, + gzFile file)); +/* + Read and decompress up to nitems items of size size from file into buf, + otherwise operating as gzread() does. This duplicates the interface of + stdio's fread(), with size_t request and return types. If the library + defines size_t, then z_size_t is identical to size_t. If not, then z_size_t + is an unsigned integer type that can contain a pointer. + + gzfread() returns the number of full items read of size size, or zero if + the end of the file was reached and a full item could not be read, or if + there was an error. gzerror() must be consulted if zero is returned in + order to determine if there was an error. If the multiplication of size and + nitems overflows, i.e. the product does not fit in a z_size_t, then nothing + is read, zero is returned, and the error state is set to Z_STREAM_ERROR. + + In the event that the end of file is reached and only a partial item is + available at the end, i.e. the remaining uncompressed data length is not a + multiple of size, then the final partial item is nevertheless read into buf + and the end-of-file flag is set. The length of the partial item read is not + provided, but could be inferred from the result of gztell(). This behavior + is the same as the behavior of fread() implementations in common libraries, + but it prevents the direct use of gzfread() to read a concurrently written + file, resetting and retrying on end-of-file, when size is not 1. +*/ + +ZEXTERN int ZEXPORT gzwrite OF((gzFile file, voidpc buf, unsigned len)); +/* + Compress and write the len uncompressed bytes at buf to file. gzwrite + returns the number of uncompressed bytes written or 0 in case of error. +*/ + +ZEXTERN z_size_t ZEXPORT gzfwrite OF((voidpc buf, z_size_t size, + z_size_t nitems, gzFile file)); +/* + Compress and write nitems items of size size from buf to file, duplicating + the interface of stdio's fwrite(), with size_t request and return types. If + the library defines size_t, then z_size_t is identical to size_t. If not, + then z_size_t is an unsigned integer type that can contain a pointer. + + gzfwrite() returns the number of full items written of size size, or zero + if there was an error. If the multiplication of size and nitems overflows, + i.e. the product does not fit in a z_size_t, then nothing is written, zero + is returned, and the error state is set to Z_STREAM_ERROR. +*/ + +ZEXTERN int ZEXPORTVA gzprintf Z_ARG((gzFile file, const char *format, ...)); +/* + Convert, format, compress, and write the arguments (...) to file under + control of the string format, as in fprintf. gzprintf returns the number of + uncompressed bytes actually written, or a negative zlib error code in case + of error. The number of uncompressed bytes written is limited to 8191, or + one less than the buffer size given to gzbuffer(). The caller should assure + that this limit is not exceeded. If it is exceeded, then gzprintf() will + return an error (0) with nothing written. In this case, there may also be a + buffer overflow with unpredictable consequences, which is possible only if + zlib was compiled with the insecure functions sprintf() or vsprintf(), + because the secure snprintf() or vsnprintf() functions were not available. + This can be determined using zlibCompileFlags(). +*/ + +ZEXTERN int ZEXPORT gzputs OF((gzFile file, const char *s)); +/* + Compress and write the given null-terminated string s to file, excluding + the terminating null character. + + gzputs returns the number of characters written, or -1 in case of error. +*/ + +ZEXTERN char * ZEXPORT gzgets OF((gzFile file, char *buf, int len)); +/* + Read and decompress bytes from file into buf, until len-1 characters are + read, or until a newline character is read and transferred to buf, or an + end-of-file condition is encountered. If any characters are read or if len + is one, the string is terminated with a null character. If no characters + are read due to an end-of-file or len is less than one, then the buffer is + left untouched. + + gzgets returns buf which is a null-terminated string, or it returns NULL + for end-of-file or in case of error. If there was an error, the contents at + buf are indeterminate. +*/ + +ZEXTERN int ZEXPORT gzputc OF((gzFile file, int c)); +/* + Compress and write c, converted to an unsigned char, into file. gzputc + returns the value that was written, or -1 in case of error. +*/ + +ZEXTERN int ZEXPORT gzgetc OF((gzFile file)); +/* + Read and decompress one byte from file. gzgetc returns this byte or -1 + in case of end of file or error. This is implemented as a macro for speed. + As such, it does not do all of the checking the other functions do. I.e. + it does not check to see if file is NULL, nor whether the structure file + points to has been clobbered or not. +*/ + +ZEXTERN int ZEXPORT gzungetc OF((int c, gzFile file)); +/* + Push c back onto the stream for file to be read as the first character on + the next read. At least one character of push-back is always allowed. + gzungetc() returns the character pushed, or -1 on failure. gzungetc() will + fail if c is -1, and may fail if a character has been pushed but not read + yet. If gzungetc is used immediately after gzopen or gzdopen, at least the + output buffer size of pushed characters is allowed. (See gzbuffer above.) + The pushed character will be discarded if the stream is repositioned with + gzseek() or gzrewind(). +*/ + +ZEXTERN int ZEXPORT gzflush OF((gzFile file, int flush)); +/* + Flush all pending output to file. The parameter flush is as in the + deflate() function. The return value is the zlib error number (see function + gzerror below). gzflush is only permitted when writing. + + If the flush parameter is Z_FINISH, the remaining data is written and the + gzip stream is completed in the output. If gzwrite() is called again, a new + gzip stream will be started in the output. gzread() is able to read such + concatenated gzip streams. + + gzflush should be called only when strictly necessary because it will + degrade compression if called too often. +*/ + +/* +ZEXTERN z_off_t ZEXPORT gzseek OF((gzFile file, + z_off_t offset, int whence)); + + Set the starting position to offset relative to whence for the next gzread + or gzwrite on file. The offset represents a number of bytes in the + uncompressed data stream. The whence parameter is defined as in lseek(2); + the value SEEK_END is not supported. + + If the file is opened for reading, this function is emulated but can be + extremely slow. If the file is opened for writing, only forward seeks are + supported; gzseek then compresses a sequence of zeroes up to the new + starting position. + + gzseek returns the resulting offset location as measured in bytes from + the beginning of the uncompressed stream, or -1 in case of error, in + particular if the file is opened for writing and the new starting position + would be before the current position. +*/ + +ZEXTERN int ZEXPORT gzrewind OF((gzFile file)); +/* + Rewind file. This function is supported only for reading. + + gzrewind(file) is equivalent to (int)gzseek(file, 0L, SEEK_SET). +*/ + +/* +ZEXTERN z_off_t ZEXPORT gztell OF((gzFile file)); + + Return the starting position for the next gzread or gzwrite on file. + This position represents a number of bytes in the uncompressed data stream, + and is zero when starting, even if appending or reading a gzip stream from + the middle of a file using gzdopen(). + + gztell(file) is equivalent to gzseek(file, 0L, SEEK_CUR) +*/ + +/* +ZEXTERN z_off_t ZEXPORT gzoffset OF((gzFile file)); + + Return the current compressed (actual) read or write offset of file. This + offset includes the count of bytes that precede the gzip stream, for example + when appending or when using gzdopen() for reading. When reading, the + offset does not include as yet unused buffered input. This information can + be used for a progress indicator. On error, gzoffset() returns -1. +*/ + +ZEXTERN int ZEXPORT gzeof OF((gzFile file)); +/* + Return true (1) if the end-of-file indicator for file has been set while + reading, false (0) otherwise. Note that the end-of-file indicator is set + only if the read tried to go past the end of the input, but came up short. + Therefore, just like feof(), gzeof() may return false even if there is no + more data to read, in the event that the last read request was for the exact + number of bytes remaining in the input file. This will happen if the input + file size is an exact multiple of the buffer size. + + If gzeof() returns true, then the read functions will return no more data, + unless the end-of-file indicator is reset by gzclearerr() and the input file + has grown since the previous end of file was detected. +*/ + +ZEXTERN int ZEXPORT gzdirect OF((gzFile file)); +/* + Return true (1) if file is being copied directly while reading, or false + (0) if file is a gzip stream being decompressed. + + If the input file is empty, gzdirect() will return true, since the input + does not contain a gzip stream. + + If gzdirect() is used immediately after gzopen() or gzdopen() it will + cause buffers to be allocated to allow reading the file to determine if it + is a gzip file. Therefore if gzbuffer() is used, it should be called before + gzdirect(). + + When writing, gzdirect() returns true (1) if transparent writing was + requested ("wT" for the gzopen() mode), or false (0) otherwise. (Note: + gzdirect() is not needed when writing. Transparent writing must be + explicitly requested, so the application already knows the answer. When + linking statically, using gzdirect() will include all of the zlib code for + gzip file reading and decompression, which may not be desired.) +*/ + +ZEXTERN int ZEXPORT gzclose OF((gzFile file)); +/* + Flush all pending output for file, if necessary, close file and + deallocate the (de)compression state. Note that once file is closed, you + cannot call gzerror with file, since its structures have been deallocated. + gzclose must not be called more than once on the same file, just as free + must not be called more than once on the same allocation. + + gzclose will return Z_STREAM_ERROR if file is not valid, Z_ERRNO on a + file operation error, Z_MEM_ERROR if out of memory, Z_BUF_ERROR if the + last read ended in the middle of a gzip stream, or Z_OK on success. +*/ + +ZEXTERN int ZEXPORT gzclose_r OF((gzFile file)); +ZEXTERN int ZEXPORT gzclose_w OF((gzFile file)); +/* + Same as gzclose(), but gzclose_r() is only for use when reading, and + gzclose_w() is only for use when writing or appending. The advantage to + using these instead of gzclose() is that they avoid linking in zlib + compression or decompression code that is not used when only reading or only + writing respectively. If gzclose() is used, then both compression and + decompression code will be included the application when linking to a static + zlib library. +*/ + +ZEXTERN const char * ZEXPORT gzerror OF((gzFile file, int *errnum)); +/* + Return the error message for the last error which occurred on file. + errnum is set to zlib error number. If an error occurred in the file system + and not in the compression library, errnum is set to Z_ERRNO and the + application may consult errno to get the exact error code. + + The application must not modify the returned string. Future calls to + this function may invalidate the previously returned string. If file is + closed, then the string previously returned by gzerror will no longer be + available. + + gzerror() should be used to distinguish errors from end-of-file for those + functions above that do not distinguish those cases in their return values. +*/ + +ZEXTERN void ZEXPORT gzclearerr OF((gzFile file)); +/* + Clear the error and end-of-file flags for file. This is analogous to the + clearerr() function in stdio. This is useful for continuing to read a gzip + file that is being written concurrently. +*/ + +#endif /* !Z_SOLO */ + + /* checksum functions */ + +/* + These functions are not related to compression but are exported + anyway because they might be useful in applications using the compression + library. +*/ + +ZEXTERN uLong ZEXPORT adler32 OF((uLong adler, const Bytef *buf, uInt len)); +/* + Update a running Adler-32 checksum with the bytes buf[0..len-1] and + return the updated checksum. An Adler-32 value is in the range of a 32-bit + unsigned integer. If buf is Z_NULL, this function returns the required + initial value for the checksum. + + An Adler-32 checksum is almost as reliable as a CRC-32 but can be computed + much faster. + + Usage example: + + uLong adler = adler32(0L, Z_NULL, 0); + + while (read_buffer(buffer, length) != EOF) { + adler = adler32(adler, buffer, length); + } + if (adler != original_adler) error(); +*/ + +ZEXTERN uLong ZEXPORT adler32_z OF((uLong adler, const Bytef *buf, + z_size_t len)); +/* + Same as adler32(), but with a size_t length. +*/ + +/* +ZEXTERN uLong ZEXPORT adler32_combine OF((uLong adler1, uLong adler2, + z_off_t len2)); + + Combine two Adler-32 checksums into one. For two sequences of bytes, seq1 + and seq2 with lengths len1 and len2, Adler-32 checksums were calculated for + each, adler1 and adler2. adler32_combine() returns the Adler-32 checksum of + seq1 and seq2 concatenated, requiring only adler1, adler2, and len2. Note + that the z_off_t type (like off_t) is a signed integer. If len2 is + negative, the result has no meaning or utility. +*/ + +ZEXTERN uLong ZEXPORT crc32 OF((uLong crc, const Bytef *buf, uInt len)); +/* + Update a running CRC-32 with the bytes buf[0..len-1] and return the + updated CRC-32. A CRC-32 value is in the range of a 32-bit unsigned integer. + If buf is Z_NULL, this function returns the required initial value for the + crc. Pre- and post-conditioning (one's complement) is performed within this + function so it shouldn't be done by the application. + + Usage example: + + uLong crc = crc32(0L, Z_NULL, 0); + + while (read_buffer(buffer, length) != EOF) { + crc = crc32(crc, buffer, length); + } + if (crc != original_crc) error(); +*/ + +ZEXTERN uLong ZEXPORT crc32_z OF((uLong crc, const Bytef *buf, + z_size_t len)); +/* + Same as crc32(), but with a size_t length. +*/ + +/* +ZEXTERN uLong ZEXPORT crc32_combine OF((uLong crc1, uLong crc2, z_off_t len2)); + + Combine two CRC-32 check values into one. For two sequences of bytes, + seq1 and seq2 with lengths len1 and len2, CRC-32 check values were + calculated for each, crc1 and crc2. crc32_combine() returns the CRC-32 + check value of seq1 and seq2 concatenated, requiring only crc1, crc2, and + len2. +*/ + +/* +ZEXTERN uLong ZEXPORT crc32_combine_gen OF((z_off_t len2)); + + Return the operator corresponding to length len2, to be used with + crc32_combine_op(). +*/ + +ZEXTERN uLong ZEXPORT crc32_combine_op OF((uLong crc1, uLong crc2, uLong op)); +/* + Give the same result as crc32_combine(), using op in place of len2. op is + is generated from len2 by crc32_combine_gen(). This will be faster than + crc32_combine() if the generated op is used more than once. +*/ + + + /* various hacks, don't look :) */ + +/* deflateInit and inflateInit are macros to allow checking the zlib version + * and the compiler's view of z_stream: + */ +ZEXTERN int ZEXPORT deflateInit_ OF((z_streamp strm, int level, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT inflateInit_ OF((z_streamp strm, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT deflateInit2_ OF((z_streamp strm, int level, int method, + int windowBits, int memLevel, + int strategy, const char *version, + int stream_size)); +ZEXTERN int ZEXPORT inflateInit2_ OF((z_streamp strm, int windowBits, + const char *version, int stream_size)); +ZEXTERN int ZEXPORT inflateBackInit_ OF((z_streamp strm, int windowBits, + unsigned char FAR *window, + const char *version, + int stream_size)); +#ifdef Z_PREFIX_SET +# define z_deflateInit(strm, level) \ + deflateInit_((strm), (level), ZLIB_VERSION, (int)sizeof(z_stream)) +# define z_inflateInit(strm) \ + inflateInit_((strm), ZLIB_VERSION, (int)sizeof(z_stream)) +# define z_deflateInit2(strm, level, method, windowBits, memLevel, strategy) \ + deflateInit2_((strm),(level),(method),(windowBits),(memLevel),\ + (strategy), ZLIB_VERSION, (int)sizeof(z_stream)) +# define z_inflateInit2(strm, windowBits) \ + inflateInit2_((strm), (windowBits), ZLIB_VERSION, \ + (int)sizeof(z_stream)) +# define z_inflateBackInit(strm, windowBits, window) \ + inflateBackInit_((strm), (windowBits), (window), \ + ZLIB_VERSION, (int)sizeof(z_stream)) +#else +# define deflateInit(strm, level) \ + deflateInit_((strm), (level), ZLIB_VERSION, (int)sizeof(z_stream)) +# define inflateInit(strm) \ + inflateInit_((strm), ZLIB_VERSION, (int)sizeof(z_stream)) +# define deflateInit2(strm, level, method, windowBits, memLevel, strategy) \ + deflateInit2_((strm),(level),(method),(windowBits),(memLevel),\ + (strategy), ZLIB_VERSION, (int)sizeof(z_stream)) +# define inflateInit2(strm, windowBits) \ + inflateInit2_((strm), (windowBits), ZLIB_VERSION, \ + (int)sizeof(z_stream)) +# define inflateBackInit(strm, windowBits, window) \ + inflateBackInit_((strm), (windowBits), (window), \ + ZLIB_VERSION, (int)sizeof(z_stream)) +#endif + +#ifndef Z_SOLO + +/* gzgetc() macro and its supporting function and exposed data structure. Note + * that the real internal state is much larger than the exposed structure. + * This abbreviated structure exposes just enough for the gzgetc() macro. The + * user should not mess with these exposed elements, since their names or + * behavior could change in the future, perhaps even capriciously. They can + * only be used by the gzgetc() macro. You have been warned. + */ +struct gzFile_s { + unsigned have; + unsigned char *next; + z_off64_t pos; +}; +ZEXTERN int ZEXPORT gzgetc_ OF((gzFile file)); /* backward compatibility */ +#ifdef Z_PREFIX_SET +# undef z_gzgetc +# define z_gzgetc(g) \ + ((g)->have ? ((g)->have--, (g)->pos++, *((g)->next)++) : (gzgetc)(g)) +#else +# define gzgetc(g) \ + ((g)->have ? ((g)->have--, (g)->pos++, *((g)->next)++) : (gzgetc)(g)) +#endif + +/* provide 64-bit offset functions if _LARGEFILE64_SOURCE defined, and/or + * change the regular functions to 64 bits if _FILE_OFFSET_BITS is 64 (if + * both are true, the application gets the *64 functions, and the regular + * functions are changed to 64 bits) -- in case these are set on systems + * without large file support, _LFS64_LARGEFILE must also be true + */ +#ifdef Z_LARGE64 + ZEXTERN gzFile ZEXPORT gzopen64 OF((const char *, const char *)); + ZEXTERN z_off64_t ZEXPORT gzseek64 OF((gzFile, z_off64_t, int)); + ZEXTERN z_off64_t ZEXPORT gztell64 OF((gzFile)); + ZEXTERN z_off64_t ZEXPORT gzoffset64 OF((gzFile)); + ZEXTERN uLong ZEXPORT adler32_combine64 OF((uLong, uLong, z_off64_t)); + ZEXTERN uLong ZEXPORT crc32_combine64 OF((uLong, uLong, z_off64_t)); + ZEXTERN uLong ZEXPORT crc32_combine_gen64 OF((z_off64_t)); +#endif + +#if !defined(ZLIB_INTERNAL) && defined(Z_WANT64) +# ifdef Z_PREFIX_SET +# define z_gzopen z_gzopen64 +# define z_gzseek z_gzseek64 +# define z_gztell z_gztell64 +# define z_gzoffset z_gzoffset64 +# define z_adler32_combine z_adler32_combine64 +# define z_crc32_combine z_crc32_combine64 +# define z_crc32_combine_gen z_crc32_combine_gen64 +# else +# define gzopen gzopen64 +# define gzseek gzseek64 +# define gztell gztell64 +# define gzoffset gzoffset64 +# define adler32_combine adler32_combine64 +# define crc32_combine crc32_combine64 +# define crc32_combine_gen crc32_combine_gen64 +# endif +# ifndef Z_LARGE64 + ZEXTERN gzFile ZEXPORT gzopen64 OF((const char *, const char *)); + ZEXTERN z_off_t ZEXPORT gzseek64 OF((gzFile, z_off_t, int)); + ZEXTERN z_off_t ZEXPORT gztell64 OF((gzFile)); + ZEXTERN z_off_t ZEXPORT gzoffset64 OF((gzFile)); + ZEXTERN uLong ZEXPORT adler32_combine64 OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine64 OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine_gen64 OF((z_off_t)); +# endif +#else + ZEXTERN gzFile ZEXPORT gzopen OF((const char *, const char *)); + ZEXTERN z_off_t ZEXPORT gzseek OF((gzFile, z_off_t, int)); + ZEXTERN z_off_t ZEXPORT gztell OF((gzFile)); + ZEXTERN z_off_t ZEXPORT gzoffset OF((gzFile)); + ZEXTERN uLong ZEXPORT adler32_combine OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine_gen OF((z_off_t)); +#endif + +#else /* Z_SOLO */ + + ZEXTERN uLong ZEXPORT adler32_combine OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine_gen OF((z_off_t)); + +#endif /* !Z_SOLO */ + +/* undocumented functions */ +ZEXTERN const char * ZEXPORT zError OF((int)); +ZEXTERN int ZEXPORT inflateSyncPoint OF((z_streamp)); +ZEXTERN const z_crc_t FAR * ZEXPORT get_crc_table OF((void)); +ZEXTERN int ZEXPORT inflateUndermine OF((z_streamp, int)); +ZEXTERN int ZEXPORT inflateValidate OF((z_streamp, int)); +ZEXTERN unsigned long ZEXPORT inflateCodesUsed OF((z_streamp)); +ZEXTERN int ZEXPORT inflateResetKeep OF((z_streamp)); +ZEXTERN int ZEXPORT deflateResetKeep OF((z_streamp)); +#if defined(_WIN32) && !defined(Z_SOLO) +ZEXTERN gzFile ZEXPORT gzopen_w OF((const wchar_t *path, + const char *mode)); +#endif +#if defined(STDC) || defined(Z_HAVE_STDARG_H) +# ifndef Z_SOLO +ZEXTERN int ZEXPORTVA gzvprintf Z_ARG((gzFile file, + const char *format, + va_list va)); +# endif +#endif + +#ifdef __cplusplus +} +#endif + +#endif /* ZLIB_H */ diff --git a/lib/os/windows/zlib-1.2.13/zutil.c b/lib/os/windows/zlib-1.2.13/zutil.c new file mode 100644 index 000000000000..1812e8c6ee70 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/zutil.c @@ -0,0 +1,356 @@ +/* zutil.c -- target dependent utility functions for the compression library + * Copyright (C) 1995-2017 Jean-loup Gailly + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* @(#) $Id$ */ + +#include "zutil.h" +#ifndef Z_SOLO +# include "gzguts.h" +#endif + +z_const char * const z_errmsg[10] = { + (z_const char *)"need dictionary", /* Z_NEED_DICT 2 */ + (z_const char *)"stream end", /* Z_STREAM_END 1 */ + (z_const char *)"", /* Z_OK 0 */ + (z_const char *)"file error", /* Z_ERRNO (-1) */ + (z_const char *)"stream error", /* Z_STREAM_ERROR (-2) */ + (z_const char *)"data error", /* Z_DATA_ERROR (-3) */ + (z_const char *)"insufficient memory", /* Z_MEM_ERROR (-4) */ + (z_const char *)"buffer error", /* Z_BUF_ERROR (-5) */ + (z_const char *)"incompatible version",/* Z_VERSION_ERROR (-6) */ + (z_const char *)"" +}; + + +const char * ZEXPORT zlibVersion() +{ + return ZLIB_VERSION; +} + +uLong ZEXPORT zlibCompileFlags() +{ + uLong flags; + + flags = 0; + switch ((int)(sizeof(uInt))) { + case 2: break; + case 4: flags += 1; break; + case 8: flags += 2; break; + default: flags += 3; + } + switch ((int)(sizeof(uLong))) { + case 2: break; + case 4: flags += 1 << 2; break; + case 8: flags += 2 << 2; break; + default: flags += 3 << 2; + } + switch ((int)(sizeof(voidpf))) { + case 2: break; + case 4: flags += 1 << 4; break; + case 8: flags += 2 << 4; break; + default: flags += 3 << 4; + } + switch ((int)(sizeof(z_off_t))) { + case 2: break; + case 4: flags += 1 << 6; break; + case 8: flags += 2 << 6; break; + default: flags += 3 << 6; + } +#ifdef ZLIB_DEBUG + flags += 1 << 8; +#endif + /* +#if defined(ASMV) || defined(ASMINF) + flags += 1 << 9; +#endif + */ +#ifdef ZLIB_WINAPI + flags += 1 << 10; +#endif +#ifdef BUILDFIXED + flags += 1 << 12; +#endif +#ifdef DYNAMIC_CRC_TABLE + flags += 1 << 13; +#endif +#ifdef NO_GZCOMPRESS + flags += 1L << 16; +#endif +#ifdef NO_GZIP + flags += 1L << 17; +#endif +#ifdef PKZIP_BUG_WORKAROUND + flags += 1L << 20; +#endif +#ifdef FASTEST + flags += 1L << 21; +#endif +#if defined(STDC) || defined(Z_HAVE_STDARG_H) +# ifdef NO_vsnprintf + flags += 1L << 25; +# ifdef HAS_vsprintf_void + flags += 1L << 26; +# endif +# else +# ifdef HAS_vsnprintf_void + flags += 1L << 26; +# endif +# endif +#else + flags += 1L << 24; +# ifdef NO_snprintf + flags += 1L << 25; +# ifdef HAS_sprintf_void + flags += 1L << 26; +# endif +# else +# ifdef HAS_snprintf_void + flags += 1L << 26; +# endif +# endif +#endif + return flags; +} + +#ifdef ZLIB_DEBUG +#include +# ifndef verbose +# define verbose 0 +# endif +int ZLIB_INTERNAL z_verbose = verbose; + +void ZLIB_INTERNAL z_error(m) + char *m; +{ + fprintf(stderr, "%s\n", m); + exit(1); +} +#endif + +/* exported to allow conversion of error code to string for compress() and + * uncompress() + */ +const char * ZEXPORT zError(err) + int err; +{ + return ERR_MSG(err); +} + +#if defined(_WIN32_WCE) && _WIN32_WCE < 0x800 + /* The older Microsoft C Run-Time Library for Windows CE doesn't have + * errno. We define it as a global variable to simplify porting. + * Its value is always 0 and should not be used. + */ + int errno = 0; +#endif + +#ifndef HAVE_MEMCPY + +void ZLIB_INTERNAL zmemcpy(dest, source, len) + Bytef* dest; + const Bytef* source; + uInt len; +{ + if (len == 0) return; + do { + *dest++ = *source++; /* ??? to be unrolled */ + } while (--len != 0); +} + +int ZLIB_INTERNAL zmemcmp(s1, s2, len) + const Bytef* s1; + const Bytef* s2; + uInt len; +{ + uInt j; + + for (j = 0; j < len; j++) { + if (s1[j] != s2[j]) return 2*(s1[j] > s2[j])-1; + } + return 0; +} + +void ZLIB_INTERNAL zmemzero(dest, len) + Bytef* dest; + uInt len; +{ + if (len == 0) return; + do { + *dest++ = 0; /* ??? to be unrolled */ + } while (--len != 0); +} +#endif + +#ifndef Z_SOLO + +#ifdef SYS16BIT + +#ifdef __TURBOC__ +/* Turbo C in 16-bit mode */ + +# define MY_ZCALLOC + +/* Turbo C malloc() does not allow dynamic allocation of 64K bytes + * and farmalloc(64K) returns a pointer with an offset of 8, so we + * must fix the pointer. Warning: the pointer must be put back to its + * original form in order to free it, use zcfree(). + */ + +#define MAX_PTR 10 +/* 10*64K = 640K */ + +local int next_ptr = 0; + +typedef struct ptr_table_s { + voidpf org_ptr; + voidpf new_ptr; +} ptr_table; + +local ptr_table table[MAX_PTR]; +/* This table is used to remember the original form of pointers + * to large buffers (64K). Such pointers are normalized with a zero offset. + * Since MSDOS is not a preemptive multitasking OS, this table is not + * protected from concurrent access. This hack doesn't work anyway on + * a protected system like OS/2. Use Microsoft C instead. + */ + +voidpf ZLIB_INTERNAL zcalloc(voidpf opaque, unsigned items, unsigned size) +{ + voidpf buf; + ulg bsize = (ulg)items*size; + + (void)opaque; + + /* If we allocate less than 65520 bytes, we assume that farmalloc + * will return a usable pointer which doesn't have to be normalized. + */ + if (bsize < 65520L) { + buf = farmalloc(bsize); + if (*(ush*)&buf != 0) return buf; + } else { + buf = farmalloc(bsize + 16L); + } + if (buf == NULL || next_ptr >= MAX_PTR) return NULL; + table[next_ptr].org_ptr = buf; + + /* Normalize the pointer to seg:0 */ + *((ush*)&buf+1) += ((ush)((uch*)buf-0) + 15) >> 4; + *(ush*)&buf = 0; + table[next_ptr++].new_ptr = buf; + return buf; +} + +void ZLIB_INTERNAL zcfree(voidpf opaque, voidpf ptr) +{ + int n; + + (void)opaque; + + if (*(ush*)&ptr != 0) { /* object < 64K */ + farfree(ptr); + return; + } + /* Find the original pointer */ + for (n = 0; n < next_ptr; n++) { + if (ptr != table[n].new_ptr) continue; + + farfree(table[n].org_ptr); + while (++n < next_ptr) { + table[n-1] = table[n]; + } + next_ptr--; + return; + } + Assert(0, "zcfree: ptr not found"); +} + +#endif /* __TURBOC__ */ + + +#ifdef M_I86 +/* Microsoft C in 16-bit mode */ + +# define MY_ZCALLOC + +#if (!defined(_MSC_VER) || (_MSC_VER <= 600)) +# define _halloc halloc +# define _hfree hfree +#endif + +voidpf ZLIB_INTERNAL zcalloc(voidpf opaque, uInt items, uInt size) +{ + (void)opaque; + return _halloc((long)items, size); +} + +void ZLIB_INTERNAL zcfree(voidpf opaque, voidpf ptr) +{ + (void)opaque; + _hfree(ptr); +} + +#endif /* M_I86 */ + +#endif /* SYS16BIT */ + +#ifdef _KERNEL +#include +voidpf +zcalloc( + voidpf opaque, + unsigned items, + unsigned size) +{ + return ((voidpf)zfs_kmem_alloc(items * size, KM_SLEEP)); +} + +void +zcfree(voidpf opaque, voidpf ptr) +{ + zfs_kmem_free(ptr, 0); +} +#endif + +#ifndef MY_ZCALLOC +#ifdef _KERNEL +#error "should not get here in kernel" +#endif +#endif + +#ifdef MY_ZCALLOC +#ifndef _KERNEL +#error "should not get here in userland" +#endif +#endif + +#ifndef MY_ZCALLOC /* Any system without a special alloc function */ + +#ifndef STDC +extern voidp malloc OF((uInt size)); +extern voidp calloc OF((uInt items, uInt size)); +extern void free OF((voidpf ptr)); +#endif + +voidpf ZLIB_INTERNAL zcalloc(opaque, items, size) + voidpf opaque; + unsigned items; + unsigned size; +{ + (void)opaque; + return sizeof(uInt) > 2 ? (voidpf)malloc(items * size) : + (voidpf)calloc(items, size); +} + +void ZLIB_INTERNAL zcfree(opaque, ptr) + voidpf opaque; + voidpf ptr; +{ + (void)opaque; + free(ptr); +} + +#endif /* MY_ZCALLOC */ + +#endif /* !Z_SOLO */ diff --git a/lib/os/windows/zlib-1.2.13/zutil.h b/lib/os/windows/zlib-1.2.13/zutil.h new file mode 100644 index 000000000000..0bc7f4ecd1c0 --- /dev/null +++ b/lib/os/windows/zlib-1.2.13/zutil.h @@ -0,0 +1,275 @@ +/* zutil.h -- internal interface and configuration of the compression library + * Copyright (C) 1995-2022 Jean-loup Gailly, Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* WARNING: this file should *not* be used by applications. It is + part of the implementation of the compression library and is + subject to change. Applications should only use zlib.h. + */ + +/* @(#) $Id$ */ + +#ifndef ZUTIL_H +#define ZUTIL_H + +#ifdef HAVE_HIDDEN +# define ZLIB_INTERNAL __attribute__((visibility ("hidden"))) +#else +# define ZLIB_INTERNAL +#endif + +#include "zlib.h" + +#if defined(STDC) && !defined(Z_SOLO) +# if !(defined(_WIN32_WCE) && defined(_MSC_VER)) +# include +# endif +# include +# include +#endif + +#ifndef local +# define local static +#endif +/* since "static" is used to mean two completely different things in C, we + define "local" for the non-static meaning of "static", for readability + (compile with -Dlocal if your debugger can't find static symbols) */ + +typedef unsigned char uch; +typedef uch FAR uchf; +typedef unsigned short ush; +typedef ush FAR ushf; +typedef unsigned long ulg; + +#if !defined(Z_U8) && !defined(Z_SOLO) && defined(STDC) +# include +# if (ULONG_MAX == 0xffffffffffffffff) +# define Z_U8 unsigned long +# elif (ULLONG_MAX == 0xffffffffffffffff) +# define Z_U8 unsigned long long +# elif (UINT_MAX == 0xffffffffffffffff) +# define Z_U8 unsigned +# endif +#endif + +extern z_const char * const z_errmsg[10]; /* indexed by 2-zlib_error */ +/* (size given to avoid silly warnings with Visual C++) */ + +#define ERR_MSG(err) z_errmsg[Z_NEED_DICT-(err)] + +#define ERR_RETURN(strm,err) \ + return (strm->msg = ERR_MSG(err), (err)) +/* To be used only when the state is known to be valid */ + + /* common constants */ + +#ifndef DEF_WBITS +# define DEF_WBITS MAX_WBITS +#endif +/* default windowBits for decompression. MAX_WBITS is for compression only */ + +#if MAX_MEM_LEVEL >= 8 +# define DEF_MEM_LEVEL 8 +#else +# define DEF_MEM_LEVEL MAX_MEM_LEVEL +#endif +/* default memLevel */ + +#define STORED_BLOCK 0 +#define STATIC_TREES 1 +#define DYN_TREES 2 +/* The three kinds of block type */ + +#define MIN_MATCH 3 +#define MAX_MATCH 258 +/* The minimum and maximum match lengths */ + +#define PRESET_DICT 0x20 /* preset dictionary flag in zlib header */ + + /* target dependencies */ + +#if defined(MSDOS) || (defined(WINDOWS) && !defined(WIN32)) +# define OS_CODE 0x00 +# ifndef Z_SOLO +# if defined(__TURBOC__) || defined(__BORLANDC__) +# if (__STDC__ == 1) && (defined(__LARGE__) || defined(__COMPACT__)) + /* Allow compilation with ANSI keywords only enabled */ + void _Cdecl farfree( void *block ); + void *_Cdecl farmalloc( unsigned long nbytes ); +# else +# include +# endif +# else /* MSC or DJGPP */ +# include +# endif +# endif +#endif + +#ifdef AMIGA +# define OS_CODE 1 +#endif + +#if defined(VAXC) || defined(VMS) +# define OS_CODE 2 +# define F_OPEN(name, mode) \ + fopen((name), (mode), "mbc=60", "ctx=stm", "rfm=fix", "mrs=512") +#endif + +#ifdef __370__ +# if __TARGET_LIB__ < 0x20000000 +# define OS_CODE 4 +# elif __TARGET_LIB__ < 0x40000000 +# define OS_CODE 11 +# else +# define OS_CODE 8 +# endif +#endif + +#if defined(ATARI) || defined(atarist) +# define OS_CODE 5 +#endif + +#ifdef OS2 +# define OS_CODE 6 +# if defined(M_I86) && !defined(Z_SOLO) +# include +# endif +#endif + +#if defined(MACOS) || defined(TARGET_OS_MAC) +# define OS_CODE 7 +# ifndef Z_SOLO +# if defined(__MWERKS__) && __dest_os != __be_os && __dest_os != __win32_os +# include /* for fdopen */ +# else +# ifndef fdopen +# define fdopen(fd,mode) NULL /* No fdopen() */ +# endif +# endif +# endif +#endif + +#ifdef __acorn +# define OS_CODE 13 +#endif + +#if defined(WIN32) && !defined(__CYGWIN__) +# define OS_CODE 10 +#endif + +#ifdef _BEOS_ +# define OS_CODE 16 +#endif + +#ifdef __TOS_OS400__ +# define OS_CODE 18 +#endif + +#ifdef __APPLE__ +# define OS_CODE 19 +#endif + +#if defined(_BEOS_) || defined(RISCOS) +# define fdopen(fd,mode) NULL /* No fdopen() */ +#endif + +#if (defined(_MSC_VER) && (_MSC_VER > 600)) && !defined __INTERIX +# if defined(_WIN32_WCE) +# define fdopen(fd,mode) NULL /* No fdopen() */ +# else +# define fdopen(fd,type) _fdopen(fd,type) +# endif +#endif + +#if defined(__BORLANDC__) && !defined(MSDOS) + #pragma warn -8004 + #pragma warn -8008 + #pragma warn -8066 +#endif + +/* provide prototypes for these when building zlib without LFS */ +#if !defined(_WIN32) && \ + (!defined(_LARGEFILE64_SOURCE) || _LFS64_LARGEFILE-0 == 0) + ZEXTERN uLong ZEXPORT adler32_combine64 OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine64 OF((uLong, uLong, z_off_t)); + ZEXTERN uLong ZEXPORT crc32_combine_gen64 OF((z_off_t)); +#endif + + /* common defaults */ + +#ifndef OS_CODE +# define OS_CODE 3 /* assume Unix */ +#endif + +#ifndef F_OPEN +# define F_OPEN(name, mode) fopen((name), (mode)) +#endif + + /* functions */ + +#if defined(pyr) || defined(Z_SOLO) +# define NO_MEMCPY +#endif +#if defined(SMALL_MEDIUM) && !defined(_MSC_VER) && !defined(__SC__) + /* Use our own functions for small and medium model with MSC <= 5.0. + * You may have to use the same strategy for Borland C (untested). + * The __SC__ check is for Symantec. + */ +# define NO_MEMCPY +#endif +#if defined(STDC) && !defined(HAVE_MEMCPY) && !defined(NO_MEMCPY) +# define HAVE_MEMCPY +#endif +#ifdef HAVE_MEMCPY +# ifdef SMALL_MEDIUM /* MSDOS small or medium model */ +# define zmemcpy _fmemcpy +# define zmemcmp _fmemcmp +# define zmemzero(dest, len) _fmemset(dest, 0, len) +# else +# define zmemcpy memcpy +# define zmemcmp memcmp +# define zmemzero(dest, len) memset(dest, 0, len) +# endif +#else + void ZLIB_INTERNAL zmemcpy OF((Bytef* dest, const Bytef* source, uInt len)); + int ZLIB_INTERNAL zmemcmp OF((const Bytef* s1, const Bytef* s2, uInt len)); + void ZLIB_INTERNAL zmemzero OF((Bytef* dest, uInt len)); +#endif + +/* Diagnostic functions */ +#ifdef ZLIB_DEBUG +# include + extern int ZLIB_INTERNAL z_verbose; + extern void ZLIB_INTERNAL z_error OF((char *m)); +# define Assert(cond,msg) {if(!(cond)) z_error(msg);} +# define Trace(x) {if (z_verbose>=0) fprintf x ;} +# define Tracev(x) {if (z_verbose>0) fprintf x ;} +# define Tracevv(x) {if (z_verbose>1) fprintf x ;} +# define Tracec(c,x) {if (z_verbose>0 && (c)) fprintf x ;} +# define Tracecv(c,x) {if (z_verbose>1 && (c)) fprintf x ;} +#else +# define Assert(cond,msg) +# define Trace(x) +# define Tracev(x) +# define Tracevv(x) +# define Tracec(c,x) +# define Tracecv(c,x) +#endif + +#ifndef Z_SOLO + voidpf ZLIB_INTERNAL zcalloc OF((voidpf opaque, unsigned items, + unsigned size)); + void ZLIB_INTERNAL zcfree OF((voidpf opaque, voidpf ptr)); +#endif + +#define ZALLOC(strm, items, size) \ + (*((strm)->zalloc))((strm)->opaque, (items), (size)) +#define ZFREE(strm, addr) (*((strm)->zfree))((strm)->opaque, (voidpf)(addr)) +#define TRY_FREE(s, p) {if (p) ZFREE(s, p);} + +/* Reverse the bytes in a 32-bit value */ +#define ZSWAP32(q) ((((q) >> 24) & 0xff) + (((q) >> 8) & 0xff00) + \ + (((q) & 0xff00) << 8) + (((q) & 0xff) << 24)) + +#endif /* ZUTIL_H */ diff --git a/module/CMakeLists.txt b/module/CMakeLists.txt new file mode 100644 index 000000000000..0864093ccea4 --- /dev/null +++ b/module/CMakeLists.txt @@ -0,0 +1,135 @@ +find_program(INF2CAT_PROGRAM + inf2cat + PATHS "${WDK_ROOT}/bin/${WDK_VERSION}/x86" # no x64 version of the tool + NO_DEFAULT_PATH +) + +if (NOT INF2CAT_PROGRAM) + message(FATAL_ERROR "Failed to find inf2cat.exe") +endif() + +find_program(SIGNTOOL_PROGRAM + signtool + PATHS "${WDK_ROOT}/bin/${WDK_VERSION}/${WDK_PLATFORM}" + NO_DEFAULT_PATH +) + +if (NOT SIGNTOOL_PROGRAM) + message(FATAL_ERROR "Failed to find signtool.exe") +endif() + + +find_program(STAMPINF_PROGRAM + stampinf + PATHS "${WDK_ROOT}/bin/${WDK_VERSION}/${WDK_PLATFORM}" + NO_DEFAULT_PATH +) + +if (NOT STAMPINF_PROGRAM) + message(FATAL_ERROR "Failed to find stampinf.exe") +endif() + +if (NOT "${ZFSIN_SIGNTOOL_CERTSTORE}") + set(ZFSIN_SIGNTOOL_CERTSTORE PrivateCertStore CACHE STRING "Name of the certificate store (PrivateCertStore) that contains the test certificate.") +endif() +if (NOT "${ZFSIN_SIGNTOOL_CERTNAME}") + set(ZFSIN_SIGNTOOL_CERTNAME "OpenZFS Test Signing Certificate" CACHE STRING "Name of the certificate (OpenZFS Test Signing Certificate) that is installed in the specified certificate store.") +endif() +if (NOT "${ZFSIN_SIGNTOOL_SHA1}") + set(ZFSIN_SIGNTOOL_SHA1 "e96bb80ace0b559239c89a425ba0b58d5590fdb3" CACHE STRING "SHA1 of the certificate.") +endif() +if (NOT "${ZFSIN_SIGNTOOL_TSA}") + set(ZFSIN_SIGNTOOL_TSA "http://timestamp.digicert.com" CACHE STRING "Specifies URL of the TSA (http://timestamp.digicert.com) which will time stamp the digital signature.") +endif() + +add_definitions( + -D_CRT_SECURE_NO_WARNINGS + -D_CRT_NONSTDC_NO_WARNINGS +) +list(APPEND WDK_COMPILE_DEFINITIONS -D_KERNEL -D__KERNEL__) +list(APPEND WDK_COMPILE_FLAGS + $,/MTd,/MT> +) + +set(CMAKE_C_STANDARD_LIBRARIES "") + +# add_definitions(-D_CRT_LOADCFG_DISABLE_CET) +# list(APPEND WDK_COMPILE_DEFINITIONS -D_CRT_LOADCFG_DISABLE_CET) +# What the blazes is __guard_eh_cont_table + + +# Silence MSVC++ things +add_compile_options( + /we4013 # 'function' undefined; assuming extern returning int + + /wd4057 # 'operator' : 'identifier1' indirection to slightly different base types from 'identifier2' + /wd4100 # 'identifier' : unreferenced formal parameter + /wd4152 # nonstandard extension, function/data pointer conversion in expression + /wd4200 # nonstandard extension used : zero-sized array in struct/union + /wd4201 # nonstandard extension used : nameless struct/union + /wd4211 # nonstandard extension used: redefined extern to static + /wd4204 # nonstandard extension used: non-constant aggregate initializer + /wd4210 # nonstandard extension used : function given file scope + /wd4389 # 'operator' : signed/unsigned mismatch + /wd4706 # assignment within conditional expression + /wd4131 # uses old-style declarator + /wd4115 # named type definition in parentheses + /wd4018 # '>=': signed/unsigned mismatch + /wd4206 # nonstandard extension used: translation unit is empty + /wd4324 # structure was padded due to alignment specifier + /wd4053 # one void operand for '?:' +) + +string(REPLACE "/W3" "/W4" CMAKE_C_FLAGS "${CMAKE_C_FLAGS}") +string(REGEX REPLACE "[-/]DWIN32" "" CMAKE_C_FLAGS "${CMAKE_C_FLAGS}") + +set(CRT_LINK_REGEX "/M[TD]d?") +string(REGEX MATCH "${CRT_LINK_REGEX}" CRT_FLAG_DEBUG "${CMAKE_C_FLAGS_DEBUG}") +string(REGEX REPLACE "${CRT_LINK_REGEX}" "" CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG}") + +string(REGEX MATCH "${CRT_LINK_REGEX}" CRT_FLAG_MINSIZEREL "${CMAKE_C_FLAGS_MINSIZEREL}") +string(REGEX REPLACE "${CRT_LINK_REGEX}" "" CMAKE_C_FLAGS_MINSIZEREL "${CMAKE_C_FLAGS_MINSIZEREL}") + +string(REGEX MATCH "${CRT_LINK_REGEX}" CRT_FLAG_RELEASE "${CMAKE_C_FLAGS_RELEASE}") +string(REGEX REPLACE "${CRT_LINK_REGEX}" "" CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE}") + +string(REGEX MATCH "${CRT_LINK_REGEX}" CRT_FLAG_RELWITHDEBINFO "${CMAKE_C_FLAGS_RELWITHDEBINFO}") +string(REGEX REPLACE "${CRT_LINK_REGEX}" "" CMAKE_C_FLAGS_RELWITHDEBINFO "${CMAKE_C_FLAGS_RELWITHDEBINFO}") + +add_compile_options( + $<$,$>>>:${CRT_FLAG_DEBUG}> + $<$,$>>>:${CRT_FLAG_MINSIZEREL}> + $<$,$>>>:${CRT_FLAG_RELEASE}> + $<$,$>>>:${CRT_FLAG_RELWITHDEBINFO}> +) + +function(um_add_executable name) + add_executable(${ARGV}) + + target_link_options(${name} PRIVATE "/MANIFESTUAC:level=\"requireAdministrator\" uiAccess=\"false\"") + add_custom_command( + TARGET ${name} + POST_BUILD + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /fd sha1 /s "${ZFSIN_SIGNTOOL_CERTSTORE}" /n "${ZFSIN_SIGNTOOL_CERTNAME}" /t "${ZFSIN_SIGNTOOL_TSA}" $ + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /as /fd sha256 /s "${ZFSIN_SIGNTOOL_CERTSTORE}" /n "${ZFSIN_SIGNTOOL_CERTNAME}" /tr "${ZFSIN_SIGNTOOL_TSA}" $ + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /fd sha1 /sha1 "${ZFSIN_SIGNTOOL_SHA1}" /t "${ZFSIN_SIGNTOOL_TSA}" $ + # Lundman's line + COMMAND "${SIGNTOOL_PROGRAM}" sign /v /as /fd sha256 /td sha256 /sha1 "${ZFSIN_SIGNTOOL_SHA1}" /tr "${ZFSIN_SIGNTOOL_TSA}" $ + WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}" + VERBATIM COMMENT "Sign userspace tool" + ) +endfunction() + +set(OpenZFS_ROOT_DIR "${CMAKE_CURRENT_LIST_DIR}") + +include_directories("${CMAKE_SOURCE_DIR}/include/os/windows/" "${CMAKE_SOURCE_DIR}/include/os/windows/spl" "${CMAKE_SOURCE_DIR}/include/os/windows/zfs" "${CMAKE_SOURCE_DIR}/include") + +add_subdirectory(icp) +add_subdirectory(lua) +add_subdirectory(zfs) +add_subdirectory(zcommon) +add_subdirectory(nvpair) +add_subdirectory(unicode) +add_subdirectory(zstd) + +add_subdirectory(os/windows) diff --git a/module/icp/CMakeLists.txt b/module/icp/CMakeLists.txt new file mode 100644 index 000000000000..5217d5a3727a --- /dev/null +++ b/module/icp/CMakeLists.txt @@ -0,0 +1,59 @@ + +use_clang() + +wdk_add_library(icpkern + algs/aes/aes_impl_generic.c + algs/aes/aes_impl.c + algs/aes/aes_impl_aesni.c + algs/aes/aes_impl_x86-64.c + algs/aes/aes_modes.c + algs/blake3/blake3.c + algs/blake3/blake3_generic.c + algs/blake3/blake3_impl.c + algs/edonr/edonr.c + algs/modes/cbc.c + algs/modes/ccm.c + algs/modes/ctr.c + algs/modes/ecb.c + algs/modes/gcm.c + algs/modes/gcm_generic.c + algs/modes/gcm_pclmulqdq.c + algs/modes/modes.c + algs/sha2/sha2_generic.c + algs/sha2/sha256_impl.c + algs/sha2/sha512_impl.c + algs/skein/skein.c + algs/skein/skein_block.c + algs/skein/skein_iv.c + algs/skein/skein_impl.h + algs/skein/skein_port.h + api/kcf_cipher.c + api/kcf_ctxops.c + api/kcf_mac.c + asm-x86_64/aes/aeskey.c + asm-x86_64/aes/aes_aesni.S + asm-x86_64/aes/aes_amd64.S + asm-x86_64/modes/gcm_pclmulqdq.S + asm-x86_64/modes/ghash-x86_64.S + asm-x86_64/modes/aesni-gcm-x86_64.S +# asm-x86_64/sha1/sha1-x86_64.S + asm-x86_64/blake3/blake3_avx2.S + asm-x86_64/blake3/blake3_avx512.S + asm-x86_64/blake3/blake3_sse2.S + asm-x86_64/blake3/blake3_sse41.S + asm-x86_64/sha2/sha256-x86_64.S + asm-x86_64/sha2/sha512-x86_64.S + core/kcf_callprov.c + core/kcf_mech_tabs.c + core/kcf_prov_lib.c + core/kcf_prov_tabs.c + core/kcf_sched.c + illumos-crypto.c + io/aes.c + io/sha2_mod.c + io/skein_mod.c + spi/kcf_spi.c +) + +target_link_libraries(icpkern PRIVATE splkern) +target_include_directories(icpkern BEFORE PUBLIC "include") diff --git a/module/icp/algs/aes/aes_impl.c b/module/icp/algs/aes/aes_impl.c index 9daa975226fe..40bae98c7233 100644 --- a/module/icp/algs/aes/aes_impl.c +++ b/module/icp/algs/aes/aes_impl.c @@ -404,7 +404,8 @@ aes_impl_set(const char *val) return (err); } -#if defined(_KERNEL) && defined(__linux__) +#if defined(_KERNEL) +#if defined(__linux__) || defined(_WIN32) static int icp_aes_impl_set(const char *val, zfs_kernel_param_t *kp) @@ -437,8 +438,33 @@ icp_aes_impl_get(char *buffer, zfs_kernel_param_t *kp) return (cnt); } +#endif /* Linux || Windows */ + +#ifdef _WIN32 +int +win32_icp_aes_impl_set(ZFS_MODULE_PARAM_ARGS) +{ + static unsigned char str[1024] = ""; + + *type = ZT_TYPE_STRING; + + if (set == B_FALSE) { + if (aes_impl_initialized) + icp_aes_impl_get((char *)str, NULL); + *ptr = str; + *len = strlen((const char *)str); + return (0); + } + + ASSERT3P(ptr, !=, NULL); + + aes_impl_set(*ptr); + + return (0); +} +#endif /* WIN32 */ module_param_call(icp_aes_impl, icp_aes_impl_set, icp_aes_impl_get, NULL, 0644); MODULE_PARM_DESC(icp_aes_impl, "Select aes implementation."); -#endif +#endif /* KERNEL */ diff --git a/module/icp/algs/blake3/blake3_impl.c b/module/icp/algs/blake3/blake3_impl.c index f3f48c2dfa1a..b7375d937fbd 100644 --- a/module/icp/algs/blake3/blake3_impl.c +++ b/module/icp/algs/blake3/blake3_impl.c @@ -21,6 +21,7 @@ /* * Copyright (c) 2021-2022 Tino Reichardt + * Copyright (c) 2017 Jorgen Lundman */ #include @@ -312,7 +313,7 @@ blake3_per_cpu_ctx_fini(void) #define IMPL_FMT(impl, i) (((impl) == (i)) ? "[%s] " : "%s ") -#if defined(__linux__) +#if defined(__linux__) || defined(_WIN32) static int blake3_param_get(char *buffer, zfs_kernel_param_t *unused) @@ -347,6 +348,30 @@ blake3_param_set(const char *val, zfs_kernel_param_t *unused) return (generic_impl_setname(val)); } +#endif /* Linux || Windows */ + +#if defined(_WIN32) + +int +win32_blake3_param_set(ZFS_MODULE_PARAM_ARGS) +{ + static char str[1024] = ""; + + *type = ZT_TYPE_STRING; + + if (set == B_FALSE) { + char buffer[PAGE_SIZE]; /* Looks like they use page size */ + blake3_param_get(buffer, NULL); + *ptr = buffer; + *len = strlen(buffer); + return (0); + } + + ASSERT3P(ptr, !=, NULL); + + return (-generic_impl_setname(*ptr)); +} + #elif defined(__FreeBSD__) #include diff --git a/module/icp/algs/modes/gcm.c b/module/icp/algs/modes/gcm.c index dd8db6f97460..dd5e9878f6e1 100644 --- a/module/icp/algs/modes/gcm.c +++ b/module/icp/algs/modes/gcm.c @@ -985,7 +985,8 @@ gcm_impl_set(const char *val) return (err); } -#if defined(_KERNEL) && defined(__linux__) +#if defined(_KERNEL) +#if defined(__linux__) || defined(_WIN32) static int icp_gcm_impl_set(const char *val, zfs_kernel_param_t *kp) @@ -1024,6 +1025,32 @@ icp_gcm_impl_get(char *buffer, zfs_kernel_param_t *kp) return (cnt); } +#endif /* linux || windows */ + +#ifdef _WIN32 +int +win32_icp_gcm_impl_set(ZFS_MODULE_PARAM_ARGS) +{ + uint32_t val; + static unsigned char str[1024] = ""; + + *type = ZT_TYPE_STRING; + + if (set == B_FALSE) { + if (gcm_impl_initialized) + icp_gcm_impl_get(str, NULL); + *ptr = str; + *len = strlen(str); + return (0); + } + + ASSERT3P(ptr, !=, NULL); + + gcm_impl_set(*ptr); + + return (0); +} +#endif module_param_call(icp_gcm_impl, icp_gcm_impl_set, icp_gcm_impl_get, NULL, 0644); @@ -1564,6 +1591,36 @@ icp_gcm_avx_set_chunk_size(const char *buf, zfs_kernel_param_t *kp) return (error); } +#ifdef _WIN32 +/* Lives in here to have access to GCM macros */ +int +win32_icp_gcm_avx_set_chunk_size(ZFS_MODULE_PARAM_ARGS) +{ + uint32_t val; + + *type = ZT_TYPE_UINT; + + if (set == B_FALSE) { + *ptr = &gcm_avx_chunk_size; + *len = sizeof (gcm_avx_chunk_size); + return (0); + } + + ASSERT3U(*len, >=, sizeof (gcm_avx_chunk_size)); + + val = *(uint32_t *)(*ptr); + + val = (val / GCM_AVX_MIN_DECRYPT_BYTES) * GCM_AVX_MIN_DECRYPT_BYTES; + + if (val < GCM_AVX_MIN_ENCRYPT_BYTES || val > GCM_AVX_MAX_CHUNK_SIZE) + return (-EINVAL); + + gcm_avx_chunk_size = val; + + return (0); +} +#endif + module_param_call(icp_gcm_avx_chunk_size, icp_gcm_avx_set_chunk_size, param_get_uint, &gcm_avx_chunk_size, 0644); diff --git a/module/icp/algs/sha2/sha256_impl.c b/module/icp/algs/sha2/sha256_impl.c index 01ce5cbd814c..1d758c306f88 100644 --- a/module/icp/algs/sha2/sha256_impl.c +++ b/module/icp/algs/sha2/sha256_impl.c @@ -215,7 +215,7 @@ static const sha256_ops_t *const sha256_impls[] = { #define IMPL_FMT(impl, i) (((impl) == (i)) ? "[%s] " : "%s ") -#if defined(__linux__) +#if defined(__linux__) || defined(_WIN32) static int sha256_param_get(char *buffer, zfs_kernel_param_t *unused) @@ -250,6 +250,30 @@ sha256_param_set(const char *val, zfs_kernel_param_t *unused) return (generic_impl_setname(val)); } +#ifdef _WIN32 +int +win32_sha256_param_set(ZFS_MODULE_PARAM_ARGS) +{ + uint32_t val; + static unsigned char str[1024] = ""; + + *type = ZT_TYPE_STRING; + + if (set == B_FALSE) { + sha256_param_get(str, NULL); + *ptr = str; + *len = strlen(str); + return (0); + } + + ASSERT3P(ptr, !=, NULL); + + generic_impl_setname(*ptr); + + return (0); +} +#endif /* WIN32 */ + #elif defined(__FreeBSD__) #include diff --git a/module/icp/algs/sha2/sha512_impl.c b/module/icp/algs/sha2/sha512_impl.c index 27b35a639a54..6d8af9214669 100644 --- a/module/icp/algs/sha2/sha512_impl.c +++ b/module/icp/algs/sha2/sha512_impl.c @@ -190,7 +190,7 @@ static const sha512_ops_t *const sha512_impls[] = { #define IMPL_FMT(impl, i) (((impl) == (i)) ? "[%s] " : "%s ") -#if defined(__linux__) +#if defined(__linux__) || defined(_WIN32) static int sha512_param_get(char *buffer, zfs_kernel_param_t *unused) @@ -225,6 +225,31 @@ sha512_param_set(const char *val, zfs_kernel_param_t *unused) return (generic_impl_setname(val)); } +#ifdef _WIN32 +int +win32_sha512_param_set(ZFS_MODULE_PARAM_ARGS) +{ + uint32_t val; + static unsigned char str[1024] = ""; + + *type = ZT_TYPE_STRING; + + if (set == B_FALSE) { + sha512_param_get(str, NULL); + *ptr = str; + *len = strlen(str); + return (0); + } + + ASSERT3P(ptr, !=, NULL); + + generic_impl_setname(*ptr); + + return (0); +} +#endif /* WIN32 */ + + #elif defined(__FreeBSD__) #include diff --git a/module/icp/asm-x86_64/sha2/sha256-x86_64.S b/module/icp/asm-x86_64/sha2/sha256-x86_64.S index d3e5e3f0d080..208ca3101cd5 100644 --- a/module/icp/asm-x86_64/sha2/sha256-x86_64.S +++ b/module/icp/asm-x86_64/sha2/sha256-x86_64.S @@ -26,7 +26,7 @@ SECTION_STATIC -.balign 64 +.align 64 SET_OBJ(K256) K256: .long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 @@ -105,7 +105,7 @@ ENTRY_ALIGN(zfs_sha256_transform_x64, 16) movl 24(%rdi),%r10d movl 28(%rdi),%r11d jmp .Lloop -.balign 16 +.align 16 .Lloop: movl %ebx,%edi leaq K256(%rip),%rbp @@ -622,7 +622,7 @@ ENTRY_ALIGN(zfs_sha256_transform_x64, 16) addl %r12d,%eax leaq 20(%rbp),%rbp jmp .Lrounds_16_xx -.balign 16 +.align 16 .Lrounds_16_xx: movl 4(%rsp),%r13d movl 56(%rsp),%r15d @@ -1436,7 +1436,7 @@ ENTRY_ALIGN(zfs_sha256_transform_shani, 64) punpcklqdq %xmm0,%xmm2 jmp .Loop_shani -.balign 16 +.align 16 .Loop_shani: movdqu (%rsi),%xmm3 movdqu 16(%rsi),%xmm4 @@ -1666,7 +1666,7 @@ ENTRY_ALIGN(zfs_sha256_transform_ssse3, 64) movl 28(%rdi),%r11d jmp .Lloop_ssse3 -.balign 16 +.align 16 .Lloop_ssse3: movdqa K256+512(%rip),%xmm7 movdqu 0(%rsi),%xmm0 @@ -1696,7 +1696,7 @@ ENTRY_ALIGN(zfs_sha256_transform_ssse3, 64) movl %r8d,%r13d jmp .Lssse3_00_47 -.balign 16 +.align 16 .Lssse3_00_47: subq $-128,%rbp rorl $14,%r13d @@ -2779,7 +2779,7 @@ ENTRY_ALIGN(zfs_sha256_transform_avx, 64) vmovdqa K256+512+32(%rip),%xmm8 vmovdqa K256+512+64(%rip),%xmm9 jmp .Lloop_avx -.balign 16 +.align 16 .Lloop_avx: vmovdqa K256+512(%rip),%xmm7 vmovdqu 0(%rsi),%xmm0 @@ -2805,7 +2805,7 @@ ENTRY_ALIGN(zfs_sha256_transform_avx, 64) movl %r8d,%r13d jmp .Lavx_00_47 -.balign 16 +.align 16 .Lavx_00_47: subq $-128,%rbp vpalignr $4,%xmm0,%xmm1,%xmm4 @@ -3858,7 +3858,7 @@ ENTRY_ALIGN(zfs_sha256_transform_avx2, 64) vmovdqa K256+512+32(%rip),%ymm8 vmovdqa K256+512+64(%rip),%ymm9 jmp .Loop_avx2 -.balign 16 +.align 16 .Loop_avx2: vmovdqa K256+512(%rip),%ymm7 vmovdqu -64+0(%rsi),%xmm0 @@ -3900,7 +3900,7 @@ ENTRY_ALIGN(zfs_sha256_transform_avx2, 64) subq $-32*4,%rbp jmp .Lavx2_00_47 -.balign 16 +.align 16 .Lavx2_00_47: leaq -64(%rsp),%rsp .cfi_escape 0x0f,0x05,0x77,0x38,0x06,0x23,0x08 @@ -4842,7 +4842,7 @@ ENTRY_ALIGN(zfs_sha256_transform_avx2, 64) xorl %ecx,%edi movl %r9d,%r12d jmp .Lower_avx2 -.balign 16 +.align 16 .Lower_avx2: addl 0+16(%rbp),%r11d andl %r8d,%r12d diff --git a/module/icp/asm-x86_64/sha2/sha512-x86_64.S b/module/icp/asm-x86_64/sha2/sha512-x86_64.S index fbbcca650d10..98bb64a5e9fb 100644 --- a/module/icp/asm-x86_64/sha2/sha512-x86_64.S +++ b/module/icp/asm-x86_64/sha2/sha512-x86_64.S @@ -26,7 +26,7 @@ SECTION_STATIC -.balign 64 +.align 64 SET_OBJ(K512) K512: .quad 0x428a2f98d728ae22,0x7137449123ef65cd @@ -148,7 +148,7 @@ ENTRY_ALIGN(zfs_sha512_transform_x64, 16) movq 48(%rdi),%r10 movq 56(%rdi),%r11 jmp .Lloop -.balign 16 +.align 16 .Lloop: movq %rbx,%rdi leaq K512(%rip),%rbp @@ -665,7 +665,7 @@ ENTRY_ALIGN(zfs_sha512_transform_x64, 16) addq %r12,%rax leaq 24(%rbp),%rbp jmp .Lrounds_16_xx -.balign 16 +.align 16 .Lrounds_16_xx: movq 8(%rsp),%r13 movq 112(%rsp),%r15 @@ -1501,7 +1501,7 @@ ENTRY_ALIGN(zfs_sha512_transform_avx, 64) movq 48(%rdi),%r10 movq 56(%rdi),%r11 jmp .Lloop_avx -.balign 16 +.align 16 .Lloop_avx: vmovdqa K512+1280(%rip),%xmm11 vmovdqu 0(%rsi),%xmm0 @@ -1543,7 +1543,7 @@ ENTRY_ALIGN(zfs_sha512_transform_avx, 64) movq %r8,%r13 jmp .Lavx_00_47 -.balign 16 +.align 16 .Lavx_00_47: addq $256,%rbp vpalignr $8,%xmm0,%xmm1,%xmm8 @@ -2670,7 +2670,7 @@ ENTRY_ALIGN(zfs_sha512_transform_avx2, 64) movq 48(%rdi),%r10 movq 56(%rdi),%r11 jmp .Loop_avx2 -.balign 16 +.align 16 .Loop_avx2: vmovdqu -128(%rsi),%xmm0 vmovdqu -128+16(%rsi),%xmm1 @@ -2732,7 +2732,7 @@ ENTRY_ALIGN(zfs_sha512_transform_avx2, 64) addq $32*8,%rbp jmp .Lavx2_00_47 -.balign 16 +.align 16 .Lavx2_00_47: leaq -128(%rsp),%rsp .cfi_escape 0x0f,0x06,0x77,0xf8,0x00,0x06,0x23,0x08 @@ -3750,7 +3750,7 @@ ENTRY_ALIGN(zfs_sha512_transform_avx2, 64) xorq %rcx,%rdi movq %r9,%r12 jmp .Lower_avx2 -.balign 16 +.align 16 .Lower_avx2: addq 0+16(%rbp),%r11 andq %r8,%r12 diff --git a/module/icp/illumos-crypto.c b/module/icp/illumos-crypto.c index 13f05c06ed5c..e7d5408f007d 100644 --- a/module/icp/illumos-crypto.c +++ b/module/icp/illumos-crypto.c @@ -24,9 +24,11 @@ */ #ifdef _KERNEL +#ifdef __linux__ #include #include #include +#endif #else #define __exit #define __init diff --git a/module/icp/include/aes/aes_impl.h b/module/icp/include/aes/aes_impl.h index 66eb4a6c8fb6..b17abc88e4f7 100644 --- a/module/icp/include/aes/aes_impl.h +++ b/module/icp/include/aes/aes_impl.h @@ -107,13 +107,22 @@ typedef struct aes_impl_ops aes_impl_ops_t; * coded in aesni-gcm-x86_64, so please don't change (or adjust accordingly). */ typedef struct aes_key aes_key_t; + struct aes_key { aes_ks_t encr_ks; /* encryption key schedule */ aes_ks_t decr_ks; /* decryption key schedule */ #ifdef __amd64 long double align128; /* Align fields above for Intel AES-NI */ #endif /* __amd64 */ +#ifdef _WIN32 + /* + * We need to align with aesni-gcm-x64_64.S that pulls this out + * of the struct, and it is aligned to 128 on posix somehow. + */ + const aes_impl_ops_t _Alignas(16) *ops; +#else const aes_impl_ops_t *ops; /* ops associated with this schedule */ +#endif int nr; /* number of rounds (10, 12, or 14) */ int type; /* key schedule size (32 or 64 bits) */ }; diff --git a/module/lua/CMakeLists.txt b/module/lua/CMakeLists.txt new file mode 100644 index 000000000000..2352c9e3e9ef --- /dev/null +++ b/module/lua/CMakeLists.txt @@ -0,0 +1,55 @@ + +use_clang() + +wdk_add_library(luakern + lapi.c + lauxlib.c + lbaselib.c + #lbitlib.c + lcode.c + lcompat.c + lcorolib.c + lctype.c + ldebug.c + ldo.c + #ldump.c + lfunc.c + lgc.c + llex.c + lmem.c + lobject.c + lopcodes.c + lparser.c + lstate.c + lstring.c + lstrlib.c + ltable.c + ltablib.c + ltm.c + #lundump.c + lvm.c + lzio.c + + lapi.h + lcode.h + lctype.h + ldebug.h + ldo.h + lfunc.h + lgc.h + llex.h + llimits.h + lmem.h + lobject.h + lopcodes.h + lparser.h + lstate.h + lstring.h + ltable.h + ltm.h + #lundump.h + lvm.h + lzio.h + setjmp/setjmp_x86_64.S +) +target_link_libraries(luakern PRIVATE splkern) diff --git a/module/lua/ldo.c b/module/lua/ldo.c index 38bd4e08a73d..f8e347b29d43 100644 --- a/module/lua/ldo.c +++ b/module/lua/ldo.c @@ -26,7 +26,7 @@ #include "ltm.h" #include "lvm.h" #include "lzio.h" - +#include /* Return the number of bytes available on the stack. */ #if defined (_KERNEL) && defined(__linux__) @@ -43,6 +43,11 @@ static intptr_t stack_remaining(void) { local = (intptr_t)&local - (intptr_t)curthread->td_kstack; return local; } +#elif defined (_KERNEL) && defined(_WIN32) +#pragma warning "stack_remaining not implemented" +static intptr_t stack_remaining(void) { + return INT_MAX; +} #else static intptr_t stack_remaining(void) { return INTPTR_MAX; @@ -65,11 +70,15 @@ static intptr_t stack_remaining(void) { #ifdef _KERNEL -#ifdef __linux__ +#if defined(__linux__) || defined(__APPLE__) || defined(_WIN32) #if defined(__i386__) #define JMP_BUF_CNT 6 #elif defined(__x86_64__) +#ifdef _WIN32 +#define JMP_BUF_CNT 10 // +rsi +rdi +#else #define JMP_BUF_CNT 8 +#endif #elif defined(__sparc__) && defined(__arch64__) #define JMP_BUF_CNT 6 #elif defined(__powerpc__) diff --git a/module/nvpair/CMakeLists.txt b/module/nvpair/CMakeLists.txt new file mode 100644 index 000000000000..2d731381facf --- /dev/null +++ b/module/nvpair/CMakeLists.txt @@ -0,0 +1,12 @@ + +use_clang() + +wdk_add_library(nvpairkern + fnvpair.c + nvpair.c + nvpair_alloc_fixed.c + nvpair_alloc_spl.c +) + + +target_link_libraries(nvpairkern PRIVATE splkern) diff --git a/module/os/windows/.gitignore b/module/os/windows/.gitignore new file mode 100644 index 000000000000..26b8289b5e17 --- /dev/null +++ b/module/os/windows/.gitignore @@ -0,0 +1,3 @@ +/OpenZFS_counters.h +/OpenZFS_perf.h +/OpenZFS_perf.rc \ No newline at end of file diff --git a/module/os/windows/CMakeLists.txt b/module/os/windows/CMakeLists.txt new file mode 100644 index 000000000000..ac84ce943657 --- /dev/null +++ b/module/os/windows/CMakeLists.txt @@ -0,0 +1,152 @@ + +#include(ExternalProject) + +#ExternalProject_Add ( spl +# PREFIX "spl" +# CONFIGURE_COMMAND ${CMAKE_COMMAND} +# SOURCE_DIR "spl" # Tell CMake to use subdirectory as source. +#) + +#ExternalProject_Add ( zfs +# PREFIX "zfs" +# CONFIGURE_COMMAND ${CMAKE_COMMAND} +# SOURCE_DIR "zfs" # Tell CMake to use subdirectory as source. +#) + + +add_subdirectory(spl) +add_subdirectory(zfs) + +wdk_add_driver(OpenZFS + debug.c + driver.c + Wpp.c + OpenZFS_perf.rc + resource.rc +) + +target_link_libraries(OpenZFS PRIVATE + splkern + zlibkern + icpkern + luakern + zfskern + zfskern_os + zcommonkern + nvpairkern + unicodekern + # WDK libraries + WDK::WDMSEC + WDK::STORPORT + WDK::SCSIWMI +) + +# set(CMAKE_LINKER link.exe) + +set_target_properties(OpenZFS + PROPERTIES + ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/driver" + LIBRARY_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/driver" + RUNTIME_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/driver" +) + +configure_file(OpenZFS.inf "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.inf" COPYONLY) +configure_file(OpenZFS.man "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.man" COPYONLY) + +add_custom_command( + TARGET OpenZFS + POST_BUILD + BYPRODUCTS "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.cat" + COMMAND "${STAMPINF_PROGRAM}" -d "*" -a "amd64" -v "*" -f "${CMAKE_CURRENT_BINARY_DIR}\\driver\\OpenZFS.inf" + COMMAND "${INF2CAT_PROGRAM}" /verbose "/driver:${CMAKE_CURRENT_BINARY_DIR}/driver" /os:10_19H1_X86,10_19H1_X64,ServerRS5_X64 /uselocaltime + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /fd sha1 /s "${ZFSIN_SIGNTOOL_CERTSTORE}" /n "${ZFSIN_SIGNTOOL_CERTNAME}" /t "${ZFSIN_SIGNTOOL_TSA}" "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.cat" + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /fd sha1 /sha1 "${ZFSIN_SIGNTOOL_SHA1}" /t "${ZFSIN_SIGNTOOL_TSA}" "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.cat" + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /as /fd sha256 /s "${ZFSIN_SIGNTOOL_CERTSTORE}" /n "${ZFSIN_SIGNTOOL_CERTNAME}" /tr "${ZFSIN_SIGNTOOL_TSA}" "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.cat" + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /as /fd sha256 /td sha256 /sha1 "${ZFSIN_SIGNTOOL_SHA1}" /tr "${ZFSIN_SIGNTOOL_TSA}" "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.cat" + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /fd sha1 /s "${ZFSIN_SIGNTOOL_CERTSTORE}" /n "${ZFSIN_SIGNTOOL_CERTNAME}" /t "${ZFSIN_SIGNTOOL_TSA}" $ + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /fd sha1 /sha1 "${ZFSIN_SIGNTOOL_SHA1}" /t "${ZFSIN_SIGNTOOL_TSA}" $ + #COMMAND "${SIGNTOOL_PROGRAM}" sign /v /as /fd sha256 /s "${ZFSIN_SIGNTOOL_CERTSTORE}" /n "${ZFSIN_SIGNTOOL_CERTNAME}" /tr "${ZFSIN_SIGNTOOL_TSA}" $ + # lundman's line + COMMAND "${SIGNTOOL_PROGRAM}" sign /v /as /fd sha256 /td sha256 /sha1 "${ZFSIN_SIGNTOOL_SHA1}" /tr "${ZFSIN_SIGNTOOL_TSA}" "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.cat" + COMMAND "${SIGNTOOL_PROGRAM}" sign /v /as /fd sha256 /td sha256 /sha1 "${ZFSIN_SIGNTOOL_SHA1}" /tr "${ZFSIN_SIGNTOOL_TSA}" $ + WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/driver" + VERBATIM COMMENT "Generating and signing .cat file" +) + +install(TARGETS OpenZFS RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}/driver") +install(FILES + "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.inf" + "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.cat" + "${CMAKE_CURRENT_BINARY_DIR}/driver/OpenZFS.man" + DESTINATION "${CMAKE_INSTALL_BINDIR}/driver" +) + +install(FILES + $ + DESTINATION "${CMAKE_INSTALL_BINDIR}/driver" + OPTIONAL +) + +execute_process(COMMAND "${WDK_ROOT}/bin/${WDK_VERSION}/x86/ctrpp.exe" + -o ${CMAKE_CURRENT_SOURCE_DIR}/OpenZFS_perf.h + -rc ${CMAKE_CURRENT_SOURCE_DIR}/OpenZFS_perf.rc + -ch ${CMAKE_CURRENT_SOURCE_DIR}/OpenZFS_counters.h + ${CMAKE_CURRENT_SOURCE_DIR}/OpenZFS.man ) + +file(GLOB CSOURCES CONFIGURE_DEPENDS "*.c") + +function(add_macro_property) + foreach (I ${CSOURCES}) + get_filename_component(OUTPUT_FILE_WE ${I} NAME_WE) + set_source_files_properties(${I} PROPERTIES COMPILE_FLAGS -DWPPFILE=${CMAKE_SOURCE_DIR}/out/build/${OUTPUT_FILE_WE}.tmh) + message(STATUS "FILES_IN ======: ${CMAKE_SOURCE_DIR}/out/build/${OUTPUT_FILE_WE}.tmh") + endforeach() +endfunction() + +function(tracewpp OUTPUT_DIR SOURCE) + # [mlr] list of .tmh files to be generated -> TMH + set(WPP_DIR "${WDK_ROOT}/bin/${WDK_VERSION}") + + get_filename_component(FILEN ${SOURCE} NAME) + set(TMH_FILEN ${FILEN}.tmh) + set(TMH ${OUTPUT_DIR}/${TMH_FILEN}) + set(EXTENSIONS ".c") + + # [mlr] cmake only converts the command name to the native path format. the + # path names to be used in arguments must be converted manually. + + file(TO_NATIVE_PATH ${SOURCE} NATIVE_SOURCE) + file(TO_NATIVE_PATH ${WPP_DIR} NATIVE_WPP_DIR) + file(TO_NATIVE_PATH ${OUTPUT_DIR} NATIVE_OUTPUT_DIR) + + # [mlr] note that if -preserveext: occurs after the source file specification, it has + # no effect. + + set(TRACE "TraceEvent{FLAGS=MYDRIVER_ALL_INFO}(LEVEL, MSG, ...)") + set(DPRINT "dprintf{FLAGS=MYDRIVER_ALL_INFO, LEVEL=TRACE_INFO}(MSG, ...)") + set(CFGDIR "${WPP_DIR}/wppconfig/rev1") + set(SCAN "${WDK_ROOT}/Include/wdf/kmdf/1.9/WdfTraceEnums.h") + set(WPP_MACRO "WPP_INLINE __inline") + + execute_process(COMMAND "${NATIVE_WPP_DIR}/${WDK_PLATFORM}/tracewpp.exe" + -scan:${SCAN} /D${WPP_MACRO} + -cfgdir:${CFGDIR} -I${CMAKE_CURRENT_BINARY_DIR} -odir:${NATIVE_OUTPUT_DIR} -km -func:${TRACE} + -func:${DPRINT} -gen:{km-default.tpl}*.tmh ${NATIVE_SOURCE}) + +endfunction() + + +function(wpp OUTPUT_DIR) + + add_macro_property() + # [mlr] invoke tracewpp() for each source file, adding the resulting file to a list + # named TMH. + message(STATUS "OUTPUT_DIR ======: ${OUTPUT_DIR}") + + foreach ( I ${CSOURCES} ) + tracewpp(${OUTPUT_DIR} ${I}) + endforeach() + +endfunction() + +wpp("${CMAKE_SOURCE_DIR}/out/build") diff --git a/module/os/windows/CMakeSettings.json b/module/os/windows/CMakeSettings.json new file mode 100644 index 000000000000..b82c1d6b219b --- /dev/null +++ b/module/os/windows/CMakeSettings.json @@ -0,0 +1,27 @@ +{ + "configurations": [ + { + "name": "x64-Debug", + "generator": "Ninja", + "configurationType": "Debug", + "inheritEnvironments": [ "clang_cl_x64" ], + "buildRoot": "${projectDir}\\out\\build\\${name}", + "installRoot": "${projectDir}\\out\\install\\${name}", + "cmakeCommandArgs": "", + "buildCommandArgs": "", + "ctestCommandArgs": "", + "variables": [ + { + "name": "INF2CAT_PROGRAM", + "value": "C:/Program Files (x86)/Windows Kits/10/bin/10.0.18362.0/x86/Inf2Cat.exe", + "type": "FILEPATH" + }, + { + "name": "SIGNTOOL_PROGRAM", + "value": "C:/Program Files (x86)/Windows Kits/10/bin/10.0.18362.0/x64/signtool.exe", + "type": "FILEPATH" + } + ] + } + ] +} \ No newline at end of file diff --git a/module/os/windows/OpenZFS.inf b/module/os/windows/OpenZFS.inf new file mode 100644 index 000000000000..8b7628cf1cc8 --- /dev/null +++ b/module/os/windows/OpenZFS.inf @@ -0,0 +1,118 @@ +;;; +;;; OpenZFS +;;; +;;; +;;; Copyright (c) Jorgen Lundman +;;; + +[Version] +Signature = "$Windows NT$" +Class = Volume +ClassGuid = {71a27cdd-812a-11d0-bec7-08002be2092f} +Provider = %Me% +DriverVer = 05/19/2018,1.0.2.0 +CatalogFile = OpenZFS.cat + +[DestinationDirs] +DefaultDestDir = 12 +OpenZFS.DriverFiles = 12 ;%windir%\system32\drivers +OpenZFS.DllFiles = 11 ;%windir%\system32 + +;; +;; Default install sections +;; + +;[DefaultInstall] +;OptionDesc = %ServiceDescription% +;CopyFiles = OpenZFS.DriverFiles +;;;,OpenZFS.DllFiles +;RegisterDlls = shellopenzfs +;CopyINF = OpenZFS.inf +; +;[DefaultInstall.Services] +;AddService = %ServiceName%,0x802,OpenZFS.Service + +[Manufacturer] +%Me%=Standard,NTamd64,NTx86 + +[Standard.NTamd64] +%VolumeName% = OpenZFS_Install, OpenZFSVolume +%ControllerName% = OpenZFS_Install, ROOT\OpenZFS + +[Standard.NTx86] +%VolumeName% = OpenZFS_Install, OpenZFSVolume +%ControllerName% = OpenZFS_Install, ROOT\OpenZFS + +[OpenZFS_Install] +OptionDesc = %ServiceDescription% +CopyFiles = OpenZFS.DriverFiles +;;,OpenZFS.DllFiles +;;RegisterDlls = shellopenzfs + +[OpenZFS_Install.Services] +AddService = %ServiceName%,2,OpenZFS.Service + +;; +;; Default uninstall sections +;; + +[DefaultUninstall] +UnregisterDlls = shellopenzfs +DelFiles = OpenZFS.DriverFiles +;;,OpenZFS.DllFiles + +[DefaultUninstall.Services] +DelService = %ServiceName%,0x200 ;Ensure service is stopped before deleting + +; +; Services Section +; + +[OpenZFS.Service] +DisplayName = %ServiceName% +Description = %ServiceDescription% +ServiceBinary = %12%\OpenZFS.sys ;%windir%\system32\drivers\ +ServiceType = 1 +StartType = 1 ;SERVICE_SYSTEM_START +ErrorControl = 1 +LoadOrderGroup = "File System" + +;;[shellopenzfs] +;;11,,shellopenzfs.dll,1 + +; +; Copy Files +; + +[OpenZFS.DriverFiles] +OpenZFS.sys + +[OpenZFS.DllFiles] +;;zpool.exe +;;zfs.exe + +[SourceDisksFiles] +OpenZFS.sys = 1,, +;;zpool.exe = 1,, +;;zfs.exe = 1,, + +[SourceDisksNames.x86] +1 = %DiskId1%,, +;;,\x86 + +[SourceDisksNames.amd64] +1 = %DiskId1%,, +;;,\x64 + +;; +;; String Section +;; + +[Strings] +Me = "Jorgen Lundman" +ServiceDescription = "OpenZFS driver" +ServiceName = "OpenZFS" +DriverName = "OpenZFS" +DiskId1 = "OpenZFS Device Installation Disk" +VolumeName = "OpenZFS volume" +ControllerName = "OpenZFS controller" diff --git a/module/os/windows/OpenZFS.man b/module/os/windows/OpenZFS.man new file mode 100644 index 000000000000..4bf4f38b5f6c --- /dev/null +++ b/module/os/windows/OpenZFS.man @@ -0,0 +1,1120 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/module/os/windows/PORTING_NOTES.txt b/module/os/windows/PORTING_NOTES.txt new file mode 100644 index 000000000000..e598441f53af --- /dev/null +++ b/module/os/windows/PORTING_NOTES.txt @@ -0,0 +1,89 @@ +=== Unix to Windows porting notes === + + + +All the IO Request Packets (IRP) all come in through the same set of +function handlers, which can get a bit noisy. So ioctls from userland +to, say, listing datasets, come in to the same place, as requests to +list a directory, and volume creation notifications. We split these +out in the tail end of zfs_vnops_windows.c in the function `dispatcher`. + +To trigger a mount, we add two new ZFS ioctls, for mount and unmount. +In mount we will create a new fake disk, then create a filesystem that +we attach to the fake disk. Then we attach the filesystem to the mount +point desired. When IRP requests come in, we will immediately split it +into three; diskDevice to handle the fake disk related requests. +ioctlDevice which handles the ZFS ioctls from userland, and finally +fsDevice which gets the vnop requests from the mounted ZFS filesystem. + + +IRP_MJ_CREATE appears to serve the purpose of vnop_lookup, vnop_open, +vnop_mkdir, and vnop_create. The "create" in this context is more in +the line of "create a handle to file/dir" - existing, or creating, +entries. It has a flag to open "parent" of a file entry as well. + +We will use "fscontext" for the vnode *vp pointer, which gets you +znode_t pointer via VTOZ() macro. This project has created its own +vnode struct, to work more closely to Unix. The refcounting is done +internally to the project, and is separate to any OS related +refcounting (Unlike that of `FileObject`). Some Windows specific +variables are also contained in the vnode struct. + +It is expected that before using a struct vnode, a reference is taken +using `VN_HOLD(vp)` and release it with `VN_RELE(vp)`, for any access +of the vnode struct, and/or, znode. These are short term locks, for +long term (like that of directory handles, mountpoint) use +`vnode_ref()` and `vnode_rele()` respectively. + +Directory listings come in the form of IRP_MJ_DIRECTORY_CONTROL + +IRP_MN_QUERY_DIRECTORY. It comes with a structure "type" requested, +one of nine. Although, it appears mostly three are used, and those are +implemented. Add more as needed... + +Each return struct has an "offset to next node", relative to each +node, and the final is zero to indicate last entry in buffer. Each +struct is followed by filename in typical windows fashion, in 2byte +chars. Due to variable length filename, the next struct start has to +be aligned to 8 bytes. + +As long as there are valid entries in the return buf, it needs to +return STATUS_SUCCESS, even when EOF has been reached. So EOF has to +be remembered until the next call, at which time it should return +STATUS_NO_MORE_FILES. The directory query can also pass along pattern +to match against, which is only passed along in the first call, and +needs to be remembered. Similarly the index (in OpenZFS terms, the +directory offset) needs to be saved. These are stored in the +"fscontext2" void* ptr assigned to the directory in MJ_CREATE. Often +called "Ccb" in Windows. There is no Unix equivalent, there the offset +is stored in the `UIO` offset passed to `vnop_readdir()`. + +Deleting directories are done a little differently. It calls +IRP_MJ_CREATE to open a handle to the directory, then calls +IRP_SET_INFORMATION with type `FileDispositionInformation`, which has a +single boolean `delete`. Then it calls IRP_MJ_CLOSE, and eventually +IRP_MJ_CLEANUP. And if this is the final reference to the directory, +we call `vnop_rmdir`. + +For files, it calls IRP_MJ_CREATE with the flag DELETE_ON_CLOSE, and +closes the file. The IRP_MJ_CLEANUP handler is eventually called. The +"delete flag" is stored in the vnode struct, using the new +vnode_setunlink() and vnode_unlink() API calls. + +Many IRP calls that take structs will check the input size matches +that of sizeof(struct). A few structs will paste variable length +information, like that of Filename, at the end of the struct. A few +observations have come up; + +* Often the struct has WCHAR filename[1], which means you can always +fit the first (wide) character of the name, and the returned Length +needs to be adjusted to be one character less than the filename +length. + +* Those structs that take a variable name will also check if the full +name will fit, and if it does not, returns STATUS_BUFFER_OVERFLOW. +But, it is expected to fill in as much of the name that fits. Other +data, like in the case of FileAllInformation struct, need to be valid, +even though we return "error" and partial filename. + +* FileNameLength should be set to "required" length, and Information +size should be the same (no bigger) than input size. diff --git a/module/os/windows/README.md b/module/os/windows/README.md new file mode 100644 index 000000000000..a8229e1d78e2 --- /dev/null +++ b/module/os/windows/README.md @@ -0,0 +1,454 @@ + +[![Build status](https://ci.appveyor.com/api/projects/status/dcw734sl0prmolwr/branch/master?svg=true)](https://ci.appveyor.com/project/lundman/openzfs/branch/master) + + +# To setup a development environment for compiling OpenZFS on Windows. + + +Download free development Windows 11 image from Microsoft. + +https://developer.microsoft.com/en-us/windows/downloads/virtual-machines + +and create two VMs. + +* Host (running Visual Studio and Kernel Debugger) +* Target (runs the compiled kernel module) + +The VM images comes with Visual Studio 2022, which we use to compile the driver. + +It is recommended that the VMs are placed on static IP, as they can +change IP with all the crashes, and you have to configure the remote +kernel development again. + +Go download the Windows Driver Kit 11 + +https://developer.microsoft.com/en-us/windows/hardware/windows-driver-kit + +and install on both VMs. You will need both the SDK and WDK: +Download the SDK with the Visual Studio 2022 community edition first and install it. +It will update the already installed Visual Studio. +Then install the WDK. At the end of the installer, allow it to install the Visual Studio extension. + + +On Target VM, complete the guide specified here, under +section "Prepare the target computer for provisioning". + +https://msdn.microsoft.com/windows/hardware/drivers/gettingstarted/provision-a-target-computer-wdk-8-1?f=255&MSPPError=-2147217396 + +Which mostly entails running: + +Disable Secure Boot. + +C:\Program Files (x86)\Windows Kits\10\Remote\x64\WDK Test Target Setup x64-x64_en-us.msi + +* reboot Target VM + + +On the Host VM, continue the guide to configure Visual Studio. + +* Load Visual Studio, there is no need to load the project yet. +* Menu > Driver > Test > Configure Devices +* Click "Add New Device" +* In "Display name:" enter "Target" +* In "Device type:" leave as "Computer" +* In "Network host name:" enter IP of Target VM, for me "172.16.248.103" +* Provisioning options: o Provision device and choose debugger settings. +* Click "Next >" + +It now confirms that it talked to the Target, and note here that +"Host IP" it that of the Host VM, for me, "172.16.248.102", and not to +be confused by the Target IP entered on previous screen. + +* Click "Next >" + +Watch and wait as remote items are installed on the Target VM. It +will most likely reboot the Target VM as well. + +I've had dialog boxes pop up and I agree to installation, but I am not +sure they are supposed to. They probably shouldn't, it would seem it +failed to put WDKRemoteUser in Administrators group. If that happens, +use "lusrmgr.msc" to correct it. + +The task "Creating system restore point" will most likely fail and +that is acceptable, however, if other tasks fail, you may need to +retry until they work. + +At the end of the run, the output window offers a link to the full +log, which is worth reading if you encounter issues. + +When things fail, I start a CMD prompt as Administrator, and paste in +the commands that fail, from the log file. It would be nice if this +process just worked though. + +If your version of .NET newer, just move along. + +The Target VM should reboot, and login as "WDKRemoteUser". + +Use your preferred Unix style shell to work with git and +cmake. If you do not yet have a preferred shell, the +"GIT bash for Windows" is small and sufficient: + +https://git-scm.com/downloads + +--- + + + +Host and Target VMs are now configured. + +First time you load the project it might default to + +Debug : ARM + +you probably want to change ARM ==> X64. + +--- + +Currently OpenZFS is compiled using CMake. The easiest +way to do so is with Visual Studio. You will need to +enable "clang" support, as well as Spectre libraries in +the Visual Studio Installer. + +Add individual components: +(Versions refer to what was available at the time) + +* C++ Cmake tools for Windows +* C++ Clang Compiler for Windows (14.0.5) +* C++ Clang-cl for v143 build tools (x64/x86) +* MSVC v143 - VS 2022 C++ x64/x86 Spectre-mitigated libs (v14.33-17.3) + + + + + + +Open Visual Studio 2022 (As of Sep 2022) +File -> Open -> Folder + +and open the top source folder. Hit build when ready. + +It was previously expected of you to set the environment variables +(either globally, or in your CMakeSettings.json) + ${OPENZFS_SIGNTOOL_CERTSTORE} + ${OPENZFS_SIGNTOOL_SHA1} + ${OPENZFS_SIGNTOOL_TSA} + +but these now default to the test signing certificate + +Only the top `driver.c` is compiled using MSVC, and the +linking of OpenZFS.sys. +All other files are compiled using clang, and linked into +libraries. + +--- + +You will need to download OpenSSL for Windows: +Non-Light version recommended for Developers. Install to openssl/bin. +https://slproweb.com/products/Win32OpenSSL.html + +--- + +Deploying with Visual Studio. + +If you wish to use Visual Studio to deploy, and debug, against +remote Windows kernel. You can load the +contrib/windows/OpenZFS.sln Solutions file. + +This file is only used for deploying! + +You are still expected to compile the OpenZFS code using +CMake as described above. This section only compiles +`driver.c` and links against already compiled libraries. + + +* Configuration Properties > Debugging +"Debugging tools for Windows - Kernel Debugger" +Remote Computer Name: Target + +* Configuration Properties > Driver Install > Deployment +Target Device Name: Target +[Tick] Remove previous driver versions +O Hardware ID Driver Update +Root\OpenZFS + + +You can run DbgView on the Target VM to see the kernel prints on that VM. + + +Run the compiled Target + +* Compile solution +* Menu > Debug > Start Debugging (F5) + +wait a while, for Visual Studio to deploy the .sys file on Target and start it. + + + + + +Target VM optionals. + +If you find it frustrating to do development work when Windows Defender or +Windows Updates run, you can disable those in gpedit.msc + +* Computer Configuration > Administrative Templates > + Windows Components > + Windows Defender + Windows Updates + + +--- + +# Milestones + + + ✅ Compile SPL sources + * Godzillion warnings yet to be addressed + + ✅ Port SPL sources, atomics, mutex, kmem, condvars + * C11 _Atomics in kmem not yet handled + + ✅ Compile ZFS sources, stubbing out code as needed + + ✅ Include kernel zlib library + + ✅ Load and Unload SPL and ZFS code + + ✅ Port kernel `zfs_ioctl.c` to accept ioctls from userland + + ✅ Compile userland libspl, libzpool, libzfs, ... + + ✅ Include pthread wrapper library + * Replaced with thin pthread.h file + + ✅ Include userland zlib library + + ✅ Compile cmd/zpool + + ✅ Port functions in libzpool, libzfs. Iterate disks, ioctl + + ✅ Test ioctl from zpool to talk to kernel + + ✅ Port kernel `vdev_disk.c` / `vdev_file.c` to issue IO + + ✅ Port over cmd/zfs + + ✅ Add ioctl calls to MOUNT and create Volume to attach + + ✅ Add ioctl calls to UNMOUNT and detach and delete Volume + + ✅ Port kernel `zfs_vnops.c` / `zfs_vnops_windows.c` + * Many special cases missing, flags to create/read/etc + + ✅ Correct file information (dates, size, etc) + + ✅ Basic DOS usage + + ✅ Simple Notepad text edit, executables also work. + + ✅ Basic drag'n'drop in Explorer + + ✅ zfs send / recv, file and pipe. + + ✅ ZVOL support + + ✅ git clone ZFS repo on ZFS mounted fs + + ✅ Compile ZFS on top of ZFS + + ❎ Scrooge McDuck style swim in cash + +--- + +# Design issues that need addressing. + +* Windows does not handle EFI labels, for now they are parsed with +libefi, and we send offset and size with the filename, that both +libzfs and kernel will parse out and use. This works for a proof +of concept. + +Possibly a more proper solution would be to write a thin virtual +hard disk driver, which reads the EFI label and present just the +partitions. + +* vdev_disk.c spawns a thread to get around that IoCompletionRoutine +is called in a different context, to sleep until signalled. Is there +a better way to do async in Windows? + +* ThreadId should be checked, using PsGetCurrentThreadId() but +it makes zio_taskq_member(taskq_member()) crash. Investigate. + +* Functions in posix.c need sustenance. + +Add dataset property DriveLetter, which is ignored on Unix system. +So for a simple drive letter dataset: + +zfs set driveletter=Z pool + +The default creating of a new pool, AND, importing a UNIX pool, would +set the root dataset to + +driveletter=?: + +So it is assigned first-available drive letter. All lower datasets +will be mounted inside the drive letter. If pool's WinDriveLetter is +not set, it will mount "/pool" as "C:/pool". + +--- + +# Installing a binary release + +Latest binary files are available at [GitHub releases](https://github.com/openzfsonwindows/OpenZFS/releases) +for the lastest builds use [Nightly builds](https://openzfsonosx.org/wiki/Windows_builds) + +Developers may want to enable TestMode for quicker developments. + +* `bcdedit.exe -set testsigning on ` +* Then **reboot**. After restart it should have _Test Mode_ bottom right corner of the screen. + + +After that either + +* Run OpenZFSOnWindows.exe installer to install +* *Would you like to install device software?* should pop up, click install + * If installing an unsigned release, click "Install anyway" in the "unknown developer" popup + +Or if you do not want to run the Installer, run this command by hand from elevated CMD: +* `zfsinstaller.exe install .\OpenZFS.inf` +* *Would you like to install device software?* should pop up, click install + * If installing an unsigned release, click "Install anyway" in the "unknown developer" popup + +Run `zpool.exe status` to confirm it can talk to the kernel + +Failure would be: +``` +Unable to open \\.\ZFS: No error. +``` + +Success would be: +``` +No pools available +``` + +--- + +# Creating your first pool. + +The basic syntax to creating a pool is as below. We use the pool name +"tank" here as with Open ZFS documentation. Feel free to pick your own +pool name. + +``` +# zpool create [options] tank disk + - Create single disk pool + +# zpool create [options] tank mirror disk1 disk2 + - Create mirrored pool ("raid1") + +# zpool create [options] tank raidz disk1 disk2 disk3 .... diskn + - Create raidz ("raid5") pool of multiple disks + +``` + +The default _options_ will "mostly" work in Windows, but for best compatibility +should use a case insensitive filesystem. + +The recommended _options_ string for Windows is currently: + +``` +zpool create -O casesensitivity=insensitive -O compression=lz4 \ + -O atime=off -o ashift=12 tank disk +``` + +* Creating filebased pools would look like: +``` +# fsutil file createnew C:\poolfile.bin 200000000 +# zpool.exe create tank \\?\C:\poolfile.bin + +Note that "\\?\C:\" needs to be escaped in bash shell, ie +"\\\\?\\C:\\". + + TEST ONLINE 0 0 0 + \??\C:\poolfile.bin ONLINE 0 0 0 +``` + +* Creating a HDD pool + +First, locate disk name + +``` +# wmic diskdrive list brief +VMware, VMware Virtual S SCSI Disk Device \\.\PHYSICALDRIVE2 VMware, VMware Virtual S SCSI Disk Device 0 5362882560 +# zpool create tank PHYSICALDRIVE2 +``` + +# Creating a ZVOL virtual hard disk + +Creating a virtual hard disk (ZVOL) is done by passing "-V " to the "zfs create" command. +``` +# zfs create -V 2g tank/hello +``` + +Which would create a disk of 2GB in size, called "tank/hello". +Confirm it was created with: + +``` +# wmic diskdrive list brief +Caption DeviceID Model Partitions Size +ZVOL tank/hello SCSI Disk Device \\.\PHYSICALDRIVE2 ZVOL tank/hello SCSI DiskDevice 0 2105671680 +``` + + +# Exporting the pool + +If you have finished with ZFS, or want to eject the USB or HDD that the +pool resides on, it must first be _exported_. Similar to "ejecting" a +USB device before unplugging it. + +``` +# zpool export tank +``` + +# Importing a pool + +If a zpool has been created on a disk partition from a different system make +sure the partition label contains "zfs". Otherwise `zpool import` won't +recognize the pool and will fail with "no pools available to import". + +``` +# zpool import tank +``` + +# Uninstalling the driver + +If you used the Installer, you can browse to "C:\Program Files (x86)\OpenZFS On Windows" +and run the "uninst000.exe" Uninstaller program. + +You can also use "Add Remove Programs" from the Settings menu, and +click on "OpenZFS On Windows-debug version x.xx" and select Uninstall. + +If you did not use the Installer, you can manually uninstall it: + +``` +zfsinstaller uninstall .\OpenZFS.inf +``` + +To verify that the driver got uninstalled properly you can check "zpool.exe status". + +When uninstalled with success, "zpool.exe status" should return: +``` +Unable to open \\.\ZFS: No error. +``` + +If the driver is still there, it would be: +``` +No pools available +``` + +A reboot might be necessary to uninstall it completely. + +# Tuning + +You can use the [registry](https://openzfsonosx.org/wiki/Windows_Registry) to tune various parameters. +Also, there is [`kstat`](https://openzfsonosx.org/wiki/Windows_kstat) to dynamically change parameters. + diff --git a/module/os/windows/Wpp.c b/module/os/windows/Wpp.c new file mode 100644 index 000000000000..29267c07b97e --- /dev/null +++ b/module/os/windows/Wpp.c @@ -0,0 +1,35 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include +#include + +void +ZFSWppInit(PDRIVER_OBJECT pDriverObject, PUNICODE_STRING pRegistryPath) +{ + WPP_INIT_TRACING(pDriverObject, pRegistryPath); +} + +void +ZFSWppCleanup(PDRIVER_OBJECT pDriverObject) +{ + WPP_CLEANUP(pDriverObject); +} diff --git a/module/os/windows/debug.c b/module/os/windows/debug.c new file mode 100644 index 000000000000..d97637e44861 --- /dev/null +++ b/module/os/windows/debug.c @@ -0,0 +1,186 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2018 Julian Heuking + */ + +// Get "_daylight: has bad storage class" in time.h +#define _INC_TIME + +#define _NO_CRT_STDIO_INLINE + +#include +#include +#include +#include + +#define max_line_length 1024 + +#ifdef windowsStyleLineEndings + char *endLine = "\r\n"; +#else + char *endLine = ""; +#endif + +char *endBuf = "-EB-"; + +KSPIN_LOCK cbuf_spin; +char *cbuf = NULL; + +static unsigned long long cbuf_size = 0x100000; // 1MB +static unsigned long long startOff = 0; + +int +initDbgCircularBuffer(void) +{ + cbuf = ExAllocatePoolWithTag(NonPagedPoolNx, cbuf_size, '!GBD'); + ASSERT(cbuf); + KeInitializeSpinLock(&cbuf_spin); + return (0); +} + +int +finiDbgCircularBuffer(void) +{ + ExFreePoolWithTag(cbuf, '!GBD'); + return (0); +} + +/* + * Howto: Read the circular buffer with windbg + * + * get address of cbuf buffer: + * dt OpenZFS!cbuf --> copy shown address + * + * write memory to file + * .writemem [filepath] [cbuf address] L[Length as hex] + * e.g. .writemem C:\src\cbuf.txt 0xffff870d`d2000000 L100000 + * + * Open in your favourite text editor and + * locate -EB-, there's the start/end of the buffer + * + */ +void +addbuffer(char *buf) +{ + // unsigned long long writtenBytes = 0; + if (buf) { + unsigned long long bufLen = strlen(buf); + unsigned long long endLineLen = strlen(endLine); + unsigned long long endBufLen = strlen(endBuf); + + if (startOff + bufLen + endLineLen + endBufLen >= cbuf_size) { + // too long, set reset start offset + while (startOff < cbuf_size) { + cbuf[startOff] = 0; + startOff++; + } + startOff = 0; + } + + unsigned long long endBufOff = startOff + bufLen + endLineLen; + // print new end buf marker first, + // before overwriting the old one + for (int i = 0; i < endBufLen; i++) { + cbuf[endBufOff + i] = endBuf[i]; + } + + // print buffer + for (int i = 0; i < bufLen; i++) { + cbuf[startOff] = buf[i]; + startOff++; + } + + // print end line marker + for (int i = 0; i < endLineLen; i++) { + cbuf[startOff] = endLine[i]; + startOff++; + } + } +} + +void +printBuffer(const char *fmt, ...) +{ + // DPCs can't block (mutex) - replace this code with spinlocks + KIRQL level; + va_list args; + va_start(args, fmt); + char buf[max_line_length]; + _snprintf(buf, 18, "%p: ", PsGetCurrentThread()); + + int tmp = _vsnprintf_s(&buf[17], sizeof (buf), max_line_length, + fmt, args); + if (tmp >= max_line_length) { + _snprintf(&buf[17], 17, "buffer too small"); + } + + KeAcquireSpinLock(&cbuf_spin, &level); + addbuffer(buf); + KeReleaseSpinLock(&cbuf_spin, level); + + va_end(args); +} + +// Signalled by userland to write out the kernel buffer. +void +saveBuffer(void) +{ + UNICODE_STRING UnicodeFilespec; + OBJECT_ATTRIBUTES ObjectAttributes; + NTSTATUS status; + HANDLE h; + + printBuffer("saving buffer to disk\n"); + + RtlInitUnicodeString(&UnicodeFilespec, + L"\\??\\C:\\Windows\\debug\\ZFSin.txt"); + + // Attempt to create file, make a weak attempt, give up easily. + ObjectAttributes.Length = sizeof (OBJECT_ATTRIBUTES); + ObjectAttributes.RootDirectory = NULL; + ObjectAttributes.Attributes = OBJ_KERNEL_HANDLE; + ObjectAttributes.ObjectName = &UnicodeFilespec; + ObjectAttributes.SecurityDescriptor = NULL; + ObjectAttributes.SecurityQualityOfService = NULL; + IO_STATUS_BLOCK iostatus; + + status = ZwCreateFile(&h, + GENERIC_READ | GENERIC_WRITE | SYNCHRONIZE, + &ObjectAttributes, + &iostatus, + 0, + FILE_ATTRIBUTE_NORMAL, + /* FILE_SHARE_WRITE | */ FILE_SHARE_READ, + FILE_OVERWRITE_IF, + FILE_SYNCHRONOUS_IO_NONALERT | FILE_NO_INTERMEDIATE_BUFFERING, + NULL, + 0); + + if (status != STATUS_SUCCESS) { + printBuffer("failed to save buffer: 0x%lx\n", status); + return; + } + + ZwWriteFile(h, 0, NULL, NULL, &iostatus, cbuf, (ULONG)cbuf_size, + NULL, NULL); + ZwClose(h); +} diff --git a/module/os/windows/driver.c b/module/os/windows/driver.c new file mode 100644 index 000000000000..930049e1a231 --- /dev/null +++ b/module/os/windows/driver.c @@ -0,0 +1,174 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +// Get "_daylight: has bad storage class" in time.h +#define _INC_TIME + + +#include + +#include +#include + +#include +#include + +#include "Trace.h" + +DRIVER_INITIALIZE DriverEntry; + +extern int initDbgCircularBuffer(void); +extern int finiDbgCircularBuffer(void); +extern int spl_start(PUNICODE_STRING RegistryPath); +extern int spl_stop(void); +extern int zfs_start(void); +extern void zfs_stop(void); +extern void windows_delay(int ticks); +extern int zfs_vfsops_init(void); +extern int zfs_vfsops_fini(void); +extern int zfs_kmod_init(void); +extern void zfs_kmod_fini(void); + +extern void sysctl_os_init(PUNICODE_STRING RegistryPath); +extern void sysctl_os_fini(void); + + +#ifdef __clang__ +#error "This file should be compiled with MSVC not Clang" +#endif + +PDRIVER_OBJECT WIN_DriverObject = NULL; +PDRIVER_UNLOAD STOR_DriverUnload = NULL; +PDRIVER_DISPATCH STOR_MajorFunction[IRP_MJ_MAXIMUM_FUNCTION + 1]; + +wzvolDriverInfo STOR_wzvolDriverInfo; + +DRIVER_UNLOAD OpenZFS_Fini; + +extern _Function_class_(WORKER_THREAD_ROUTINE) +void __stdcall +sysctl_os_registry_change(PVOID Parameter); + + +void +OpenZFS_Fini(PDRIVER_OBJECT DriverObject) +{ + KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_INFO_LEVEL, "OpenZFS_Fini\n")); + + zfs_vfsops_fini(); + + if (STOR_DriverUnload != NULL) { + STOR_DriverUnload(DriverObject); + STOR_DriverUnload = NULL; + } + + zfs_kmod_fini(); + + system_taskq_fini(); + + sysctl_os_fini(); + + spl_stop(); + finiDbgCircularBuffer(); + + if (STOR_wzvolDriverInfo.zvContextArray) { + ExFreePoolWithTag(STOR_wzvolDriverInfo.zvContextArray, + MP_TAG_GENERAL); + STOR_wzvolDriverInfo.zvContextArray = NULL; + } + ZFSWppCleanup(DriverObject); +} + +/* + * Setup a Storage Miniport Driver, used only by ZVOL to create virtual disks. + */ +NTSTATUS +DriverEntry(_In_ PDRIVER_OBJECT DriverObject, + _In_ PUNICODE_STRING pRegistryPath) +{ + NTSTATUS status; + KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_INFO_LEVEL, + "OpenZFS: DriverEntry\n")); + + ZFSWppInit(DriverObject, pRegistryPath); + + KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_INFO_LEVEL, + "OpenZFS: DriverEntry\n")); + + // Setup global so zfs_ioctl.c can setup devnode + WIN_DriverObject = DriverObject; + + /* Setup print buffer, since we print from SPL */ + initDbgCircularBuffer(); + +#ifdef DBG + // DEBUG build, let's be noisy by default. + extern int zfs_flags; + zfs_flags |= 1; +#endif + + spl_start(pRegistryPath); + + sysctl_os_init(pRegistryPath); + + system_taskq_init(); + + /* + * Initialise storport for the ZVOL virtual disks. This also + * sets the Driver Callbacks, so we make a copy of them, so + * that Dispatcher can use them. + */ + status = zvol_start(DriverObject, pRegistryPath); + + if (STATUS_SUCCESS != status) { + /* If we failed, we carryon without ZVOL support. */ + KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, + "OpenZFS: StorPortInitialize() failed, no ZVOL. %d/0x%x\n", + status, status)); + memset(STOR_MajorFunction, 0, sizeof (STOR_MajorFunction)); + STOR_DriverUnload = NULL; + } else { + /* Make a copy of the Driver Callbacks for miniport */ + memcpy(STOR_MajorFunction, WIN_DriverObject->MajorFunction, + sizeof (STOR_MajorFunction)); + STOR_DriverUnload = WIN_DriverObject->DriverUnload; + } + + /* Now set the Driver Callbacks to dispatcher and start ZFS */ + WIN_DriverObject->DriverUnload = OpenZFS_Fini; + + /* Start ZFS itself */ + zfs_kmod_init(); + + /* Register fs with Win */ + zfs_vfsops_init(); + + /* Start monitoring Registry for changes */ + sysctl_os_registry_change(pRegistryPath); + + KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, + "OpenZFS: Started\n")); + return (STATUS_SUCCESS); +} diff --git a/module/os/windows/resource.h b/module/os/windows/resource.h new file mode 100644 index 000000000000..8a09cc650089 --- /dev/null +++ b/module/os/windows/resource.h @@ -0,0 +1,15 @@ +// {{NO_DEPENDENCIES}} +// Microsoft Visual C++ generated include file. +// Used by resource.rc +// + +// Next default values for new objects +// +#ifdef APSTUDIO_INVOKED +#ifndef APSTUDIO_READONLY_SYMBOLS +#define _APS_NEXT_RESOURCE_VALUE 101 +#define _APS_NEXT_COMMAND_VALUE 40001 +#define _APS_NEXT_CONTROL_VALUE 1000 +#define _APS_NEXT_SYMED_VALUE 101 +#endif +#endif diff --git a/module/os/windows/resource.rc b/module/os/windows/resource.rc new file mode 100644 index 000000000000..764f4c8edea8 --- /dev/null +++ b/module/os/windows/resource.rc @@ -0,0 +1,89 @@ +// Microsoft Visual C++ generated resource script. +// +#include +#include "resource.h" +///////////////////////////////////////////////////////////////////////////// +// English (United States) resources + +#if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) +LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US +#pragma code_page(1252) + +#ifdef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// TEXTINCLUDE +// + +1 TEXTINCLUDE +BEGIN + "resource.h\0" +END + +2 TEXTINCLUDE +BEGIN + "\0" +END + +3 TEXTINCLUDE +BEGIN + "\r\n" + "\0" +END + +#endif // APSTUDIO_INVOKED + + +///////////////////////////////////////////////////////////////////////////// +// +// Version +// + +VS_VERSION_INFO VERSIONINFO + FILEVERSION 1,0,0,1 + PRODUCTVERSION 1,0,0,1 + FILEFLAGSMASK 0x3fL +#ifdef _DEBUG + FILEFLAGS 0x1L +#else + FILEFLAGS 0x0L +#endif + FILEOS 0x40004L + FILETYPE 0x0L + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "400904b0" + BEGIN + VALUE "CompanyName", "TODO: " + VALUE "FileDescription", "OpenZFS.sys" + VALUE "FileVersion", "1.0.0.1" + VALUE "InternalName", "resource.rc" + VALUE "LegalCopyright", "Copyright (C) 2021" + VALUE "OriginalFilename", "resource.rc" + VALUE "ProductName", "OpenZFS.sys" + VALUE "ProductVersion", "1.0.0.1" + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x4009, 1200 + END +END + +#endif // English (United States) resources +///////////////////////////////////////////////////////////////////////////// + + + +#ifndef APSTUDIO_INVOKED +///////////////////////////////////////////////////////////////////////////// +// +// Generated from the TEXTINCLUDE 3 resource. +// + + +///////////////////////////////////////////////////////////////////////////// +#endif // not APSTUDIO_INVOKED + diff --git a/module/os/windows/spl/CMakeLists.txt b/module/os/windows/spl/CMakeLists.txt new file mode 100644 index 000000000000..cc6ea36641fa --- /dev/null +++ b/module/os/windows/spl/CMakeLists.txt @@ -0,0 +1,97 @@ + +use_clang() + +wdk_add_library(splkern + spl-atomic.c + spl-condvar.c + spl-cred.c + spl-ddi.c + spl-debug.c + spl-err.c + spl-kmem.c + spl-kstat.c + spl-list.c + spl-md5.c + spl-mount.c + spl-mutex.c + spl-policy.c + spl-proc.c + spl-processor.c + spl-proc_list.c + spl-rwlock.c + spl-seg_kmem.c + spl-taskq.c + spl-thread.c + spl-time.c + spl-tsd.c + spl-uio.c + spl-vmem.c + spl-vnode.c + spl-windows.c + spl-xdr.c + spl-lookasidelist.c +) + +# set(CMAKE_TOOLCHAIN_FILE $ENV{CMAKE_TOOLCHAIN_FILE}) + +target_include_directories(splkern BEFORE PUBLIC "${CMAKE_SOURCE_DIR}/include/os/windows/" PUBLIC "${CMAKE_SOURCE_DIR}/include/os/windows/spl") + +file(GLOB CSOURCES CONFIGURE_DEPENDS "*.c") + +list(REMOVE_ITEM CSOURCES "${CMAKE_CURRENT_SOURCE_DIR}/spl-wmsum.c") + +function(add_macro_property) + foreach (I ${CSOURCES}) + get_filename_component(OUTPUT_FILE_WE ${I} NAME_WE) + set_source_files_properties(${I} PROPERTIES COMPILE_FLAGS -DWPPFILE=${CMAKE_SOURCE_DIR}/out/build/${OUTPUT_FILE_WE}.tmh) + message(STATUS "FILES_IN ======: ${CMAKE_SOURCE_DIR}/out/build/${OUTPUT_FILE_WE}.tmh") + endforeach() +endfunction() + +function(tracewpp OUTPUT_DIR SOURCE) + # [mlr] list of .tmh files to be generated -> TMH + set(WPP_DIR "${WDK_ROOT}/bin/${WDK_VERSION}") + + get_filename_component(FILEN ${SOURCE} NAME) + set(TMH_FILEN ${FILEN}.tmh) + set(TMH ${OUTPUT_DIR}/${TMH_FILEN}) + set(EXTENSIONS ".c") + + # [mlr] cmake only converts the command name to the native path format. the + # path names to be used in arguments must be converted manually. + + file(TO_NATIVE_PATH ${SOURCE} NATIVE_SOURCE) + file(TO_NATIVE_PATH ${WPP_DIR} NATIVE_WPP_DIR) + file(TO_NATIVE_PATH ${OUTPUT_DIR} NATIVE_OUTPUT_DIR) + + # [mlr] note that if -preserveext: occurs after the source file specification, it has + # no effect. + + set(TRACE "TraceEvent{FLAGS=MYDRIVER_ALL_INFO}(LEVEL, MSG, ...)") + set(DPRINT "dprintf{FLAGS=MYDRIVER_ALL_INFO, LEVEL=TRACE_INFO}(MSG, ...)") + set(CFGDIR "${WPP_DIR}/wppconfig/rev1") + set(SCAN "${WDK_ROOT}/Include/wdf/kmdf/1.9/WdfTraceEnums.h") + set(WPP_MACRO "WPP_INLINE __inline") + + execute_process(COMMAND "${NATIVE_WPP_DIR}/${WDK_PLATFORM}/tracewpp.exe" + -scan:${SCAN} /D${WPP_MACRO} + -cfgdir:${CFGDIR} -I${CMAKE_CURRENT_BINARY_DIR} -odir:${NATIVE_OUTPUT_DIR} -km -func:${TRACE} + -func:${DPRINT} -gen:{km-default.tpl}*.tmh ${NATIVE_SOURCE}) + +endfunction() + + +function(wpp OUTPUT_DIR) + + add_macro_property() + # [mlr] invoke tracewpp() for each source file, adding the resulting file to a list + # named TMH. + message(STATUS "OUTPUT_DIR ======: ${OUTPUT_DIR}") + + foreach ( I ${CSOURCES} ) + tracewpp(${OUTPUT_DIR} ${I}) + endforeach() + +endfunction() + +wpp("${CMAKE_SOURCE_DIR}/out/build") diff --git a/module/os/windows/spl/spl-atomic.c b/module/os/windows/spl/spl-atomic.c new file mode 100644 index 000000000000..d2a6d1b68bc6 --- /dev/null +++ b/module/os/windows/spl/spl-atomic.c @@ -0,0 +1,46 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + * + * *************************************************************************** + * Solaris Porting Layer (SPL) Atomic Implementation. + * *************************************************************************** + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#include +#include +#include + + +#ifdef _KERNEL + +void * +atomic_cas_ptr(volatile void *_target, void *_cmp, void *_new) +{ + return (InterlockedCompareExchangePointer((volatile PVOID *)_target, + _new, _cmp)); +} + +#endif diff --git a/module/os/windows/spl/spl-condvar.c b/module/os/windows/spl/spl-condvar.c new file mode 100644 index 000000000000..69874adb144c --- /dev/null +++ b/module/os/windows/spl/spl-condvar.c @@ -0,0 +1,292 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + * Following the guide at http://www.cs.wustl.edu/~schmidt/win32-cv-1.html + * and implementing the second-to-last suggestion, albeit in kernel mode, + * and replacing CriticalSection with Atomics. At some point, we should + * perhaps look at the final "SignalObjectAndWait" solution, presumably + * by using the Wait argument to Mutex, and call WaitForObject. + */ + +#include +#include +#include +#include + +#ifdef SPL_DEBUG_MUTEX +void spl_wdlist_settime(void *mpleak, uint64_t value); +#endif + +#define CONDVAR_INIT 0x12345678 + +void +spl_cv_init(kcondvar_t *cvp, char *name, kcv_type_t type, void *arg) +{ + (void) cvp; (void) name; (void) type; (void) arg; + + KeInitializeEvent(&cvp->cv_kevent[CV_SIGNAL], SynchronizationEvent, + FALSE); + KeInitializeEvent(&cvp->cv_kevent[CV_BROADCAST], NotificationEvent, + FALSE); + + cvp->cv_waiters_count = 0; + cvp->cv_initialised = CONDVAR_INIT; +} + +void +spl_cv_destroy(kcondvar_t *cvp) +{ + if (cvp->cv_initialised != CONDVAR_INIT) + panic("%s: not cv_initialised", __func__); + // We have probably already signalled the waiters, but we need to + // kick around long enough for them to wake. + while (cvp->cv_waiters_count > 0) + cv_broadcast(cvp); + ASSERT0(cvp->cv_waiters_count); + cvp->cv_initialised = 0; +} + +void +spl_cv_signal(kcondvar_t *cvp) +{ + if (cvp->cv_initialised != CONDVAR_INIT) + panic("%s: not cv_initialised", __func__); + + uint32_t have_waiters = cvp->cv_waiters_count > 0; + + if (have_waiters) + KeSetEvent(&cvp->cv_kevent[CV_SIGNAL], 0, FALSE); +} + +// WakeConditionVariable or WakeAllConditionVariable function. + +void +spl_cv_broadcast(kcondvar_t *cvp) +{ + if (cvp->cv_initialised != CONDVAR_INIT) + panic("%s: not cv_initialised", __func__); + + int have_waiters = cvp->cv_waiters_count > 0; + + if (have_waiters) + KeSetEvent(&cvp->cv_kevent[CV_BROADCAST], 0, FALSE); +} + +/* + * Block on the indicated condition variable and + * release the associated mutex while blocked. + */ +int +spl_cv_wait(kcondvar_t *cvp, kmutex_t *mp, int flags, const char *msg) +{ + int result; + if (cvp->cv_initialised != CONDVAR_INIT) + panic("%s: not cv_initialised", __func__); + + if (msg != NULL && msg[0] == '&') + ++msg; /* skip over '&' prefixes */ +#ifdef SPL_DEBUG_MUTEX + spl_wdlist_settime(mp->leak, 0); +#endif + + atomic_inc_32(&cvp->cv_waiters_count); + mutex_exit(mp); + + void *locks[CV_MAX_EVENTS] = + { &cvp->cv_kevent[CV_SIGNAL], &cvp->cv_kevent[CV_BROADCAST] }; + + result = KeWaitForMultipleObjects(2, locks, WaitAny, Executive, + KernelMode, FALSE, NULL, NULL); + + // If last listener, clear BROADCAST event. (Even if it was SIGNAL + // overclearing will not hurt?) + mutex_enter(mp); + + if (cvp->cv_waiters_count == 1) + KeClearEvent(&cvp->cv_kevent[CV_BROADCAST]); + + atomic_dec_32(&cvp->cv_waiters_count); + +#ifdef SPL_DEBUG_MUTEX + spl_wdlist_settime(mp->leak, gethrestime_sec()); +#endif + /* + * 1 - condvar got cv_signal()/cv_broadcast() + * 0 - received signal (kill -signal) + */ + return (result == STATUS_ALERTED ? 0 : 1); +} + +/* + * Same as cv_wait except the thread will unblock at 'tim' + * (an absolute time) if it hasn't already unblocked. + * + * Returns the amount of time left from the original 'tim' value + * when it was unblocked. + */ +int +spl_cv_timedwait(kcondvar_t *cvp, kmutex_t *mp, clock_t tim, int flags, + const char *msg) +{ + int result; + clock_t timenow; + LARGE_INTEGER timeout; + (void) cvp; (void) flags; + + if (cvp->cv_initialised != CONDVAR_INIT) + panic("%s: not cv_initialised", __func__); + + if (msg != NULL && msg[0] == '&') + ++msg; /* skip over '&' prefixes */ + + timenow = zfs_lbolt(); + + // Check for events already in the past + if (tim < timenow) + tim = timenow; + + /* + * Pointer to a time-out value that specifies the absolute or + * relative time, in 100-nanosecond units, at which the wait is to + * be completed. A positive value specifies an absolute time, + * relative to January 1, 1601. A negative value specifies an + * interval relative to the current time. + */ + timeout.QuadPart = -100000 * MAX(1, (tim - timenow) / hz); + +#ifdef SPL_DEBUG_MUTEX + spl_wdlist_settime(mp->leak, 0); +#endif + + atomic_inc_32(&cvp->cv_waiters_count); + mutex_exit(mp); + + void *locks[CV_MAX_EVENTS] = + { &cvp->cv_kevent[CV_SIGNAL], &cvp->cv_kevent[CV_BROADCAST] }; + + result = KeWaitForMultipleObjects(2, locks, WaitAny, Executive, + KernelMode, FALSE, &timeout, NULL); + + int last_waiter = + result == STATUS_WAIT_0 + CV_BROADCAST && + cvp->cv_waiters_count == 1; + + if (last_waiter) + KeClearEvent(&cvp->cv_kevent[CV_BROADCAST]); + + atomic_dec_32(&cvp->cv_waiters_count); + + mutex_enter(mp); + +#ifdef SPL_DEBUG_MUTEX + spl_wdlist_settime(mp->leak, gethrestime_sec()); +#endif + + switch (result) { + + case STATUS_ALERTED: /* Signal */ + case ERESTART: + return (0); + + case STATUS_TIMEOUT: /* Timeout */ + return (-1); + } + + return (1); +} + + +/* + * Compatibility wrapper for the cv_timedwait_hires() Illumos interface. + */ +int +cv_timedwait_hires(kcondvar_t *cvp, kmutex_t *mp, hrtime_t tim, + hrtime_t res, int flag) +{ + int result; + LARGE_INTEGER timeout; + + if (cvp->cv_initialised != CONDVAR_INIT) + panic("%s: not cv_initialised", __func__); + ASSERT(cvp->cv_initialised == CONDVAR_INIT); + + if (res > 1) { + /* + * Align expiration to the specified resolution. + */ + if (flag & CALLOUT_FLAG_ROUNDUP) + tim += res - 1; + tim = (tim / res) * res; + } + + if (flag & CALLOUT_FLAG_ABSOLUTE) { + // 'tim' here is absolute UNIX time (from gethrtime()) so + // convert it to absolute Windows time + hrtime_t now = gethrtime(); + + tim -= now; // Remove the ticks, what remains is "sleep" amount. + } + timeout.QuadPart = -tim / 100; + +#ifdef SPL_DEBUG_MUTEX + spl_wdlist_settime(mp->leak, 0); +#endif + + atomic_inc_32(&cvp->cv_waiters_count); + mutex_exit(mp); + + void *locks[CV_MAX_EVENTS] = + { &cvp->cv_kevent[CV_SIGNAL], &cvp->cv_kevent[CV_BROADCAST] }; + + result = KeWaitForMultipleObjects(2, locks, WaitAny, Executive, + KernelMode, FALSE, &timeout, NULL); + + int last_waiter = + result == STATUS_WAIT_0 + CV_BROADCAST && + cvp->cv_waiters_count == 1; + + if (last_waiter) + KeClearEvent(&cvp->cv_kevent[CV_BROADCAST]); + + atomic_dec_32(&cvp->cv_waiters_count); + + mutex_enter(mp); + +#ifdef SPL_DEBUG_MUTEX + spl_wdlist_settime(mp->leak, gethrestime_sec()); +#endif + + switch (result) { + + case STATUS_ALERTED: /* Signal */ + case ERESTART: + return (0); + + case STATUS_TIMEOUT: /* Timeout */ + return (-1); + } + + return (1); +} diff --git a/module/os/windows/spl/spl-cred.c b/module/os/windows/spl/spl-cred.c new file mode 100644 index 000000000000..6e3abcfcb921 --- /dev/null +++ b/module/os/windows/spl/spl-cred.c @@ -0,0 +1,161 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#include +#include + +/* Return the effective user id */ +uid_t +crgetuid(const cred_t *cr) +{ + if (!cr) + return (0); + return ((uint64_t)-1); +} + + +/* Return the real user id */ +uid_t +crgetruid(const cred_t *cr) +{ + if (!cr) + return (0); + return ((uint64_t)-1); +} + +/* Return the saved user id */ +uid_t +crgetsuid(const cred_t *cr) +{ + if (!cr) + return (0); + return ((uint64_t)-1); +} + +/* Return the filesystem user id */ +uid_t +crgetfsuid(const cred_t *cr) +{ + if (!cr) + return (0); + return ((uint64_t)-1); +} + +/* Return the effective group id */ +gid_t +crgetgid(const cred_t *cr) +{ + if (!cr) + return (0); + return ((uint64_t)-1); +} + +/* Return the real group id */ +gid_t +crgetrgid(const cred_t *cr) +{ + if (!cr) + return (0); + return ((uint64_t)-1); +} + +/* Return the saved group id */ +gid_t +crgetsgid(const cred_t *cr) +{ + if (!cr) + return (0); + return ((uint64_t)-1); +} + +/* Return the filesystem group id */ +gid_t +crgetfsgid(const cred_t *cr) +{ + (void) cr; + return ((uint64_t)-1); +} + + +/* + * Unfortunately, to get the count of groups, we have to call XNU which + * memcpy's them over. No real clean way to get around that, but at least + * these calls are done sparingly. + */ +int +crgetngroups(const cred_t *cr) +{ + (void) cr; + return (0); +} + + +/* + * We always allocate NGROUPs here, since we don't know how many there will + * be until after the call. Unlike IllumOS, the ptr returned is allocated + * and must be returned by a call to crgetgroupsfree(). + */ +gid_t * +crgetgroups(const cred_t *cr) +{ + gid_t *gids; + int count = NGROUPS; + (void) cr; + + gids = kmem_zalloc(sizeof (gid_t) * count, KM_SLEEP); + if (!gids) + return (NULL); + + return (gids); +} + +void +crgetgroupsfree(gid_t *gids) +{ + if (!gids) + return; + kmem_free(gids, sizeof (gid_t) * NGROUPS); +} + +/* + * Return true if "cr" belongs in group "gid". + */ +int +spl_cred_ismember_gid(cred_t *cr, gid_t gid) +{ + int ret = 0; // Is not member. + (void) cr; (void) gid; + if (ret == 1) + return (TRUE); + return (FALSE); +} + +int +groupmember(gid_t gid, kauth_cred_t *cred) +{ + return (0); +} diff --git a/module/os/windows/spl/spl-ddi.c b/module/os/windows/spl/spl-ddi.c new file mode 100644 index 000000000000..567232203749 --- /dev/null +++ b/module/os/windows/spl/spl-ddi.c @@ -0,0 +1,622 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include +#include +#include +#include + +/* + * Allocate a set of pointers to 'n_items' objects of size 'size' + * bytes. Each pointer is initialized to nil. + * + * The 'size' and 'n_items' values are stashed in the opaque + * handle returned to the caller. + * + * This implementation interprets 'set of pointers' to mean 'array + * of pointers' but note that nothing in the interface definition + * precludes an implementation that uses, for example, a linked list. + * However there should be a small efficiency gain from using an array + * at lookup time. + * + * NOTE As an optimization, we make our growable array allocations in + * powers of two (bytes), since that's how much kmem_alloc (currently) + * gives us anyway. It should save us some free/realloc's .. + * + * As a further optimization, we make the growable array start out + * with MIN_N_ITEMS in it. + */ + + +int +ddi_soft_state_init(void **state_p, uint32_t size, uint32_t n_items) +{ + struct i_ddi_soft_state *ss; + + if (state_p == NULL || *state_p != NULL || size == 0) + return (EINVAL); + + ss = kmem_zalloc(sizeof (*ss), KM_SLEEP); + mutex_init(&ss->lock, NULL, MUTEX_DRIVER, NULL); + ss->size = size; + + if (n_items < MIN_N_ITEMS) + ss->n_items = MIN_N_ITEMS; + else { + int bitlog; + + if ((bitlog = ddi_fls(n_items)) == ddi_ffs(n_items)) + bitlog--; + ss->n_items = 1 << bitlog; + } + + ASSERT(ss->n_items >= n_items); + + ss->array = kmem_zalloc(ss->n_items * sizeof (void *), KM_SLEEP); + + *state_p = ss; + + return (0); +} + + +/* + * Allocate a state structure of size 'size' to be associated + * with item 'item'. + * + * In this implementation, the array is extended to + * allow the requested offset, if needed. + */ +int +ddi_soft_state_zalloc(void *state, int item) +{ + struct i_ddi_soft_state *ss; + void **array; + void *new_element; + + if ((ss = state) == NULL || item < 0) + return (DDI_FAILURE); + + mutex_enter(&ss->lock); + if (ss->size == 0) { + mutex_exit(&ss->lock); + cmn_err(CE_WARN, "ddi_soft_state_zalloc: bad handle"); + return (DDI_FAILURE); + } + + array = ss->array; /* NULL if ss->n_items == 0 */ + ASSERT(ss->n_items != 0 && array != NULL); + + /* + * refuse to tread on an existing element + */ + if (item < ss->n_items && array[item] != NULL) { + mutex_exit(&ss->lock); + return (DDI_FAILURE); + } + + /* + * Allocate a new element to plug in + */ + new_element = kmem_zalloc(ss->size, KM_SLEEP); + + /* + * Check if the array is big enough, if not, grow it. + */ + if (item >= ss->n_items) { + void **new_array; + uint32_t new_n_items; + struct i_ddi_soft_state *dirty; + + /* + * Allocate a new array of the right length, copy + * all the old pointers to the new array, then + * if it exists at all, put the old array on the + * dirty list. + * + * Note that we can't kmem_free() the old array. + * + * Why -- well the 'get' operation is 'mutex-free', so we + * can't easily catch a suspended thread that is just about + * to dereference the array we just grew out of. So we + * cons up a header and put it on a list of 'dirty' + * pointer arrays. (Dirty in the sense that there may + * be suspended threads somewhere that are in the middle + * of referencing them). Fortunately, we -can- garbage + * collect it all at ddi_soft_state_fini time. + */ + new_n_items = ss->n_items; + while (new_n_items < (1 + item)) + new_n_items <<= 1; /* double array size .. */ + + ASSERT(new_n_items >= (1 + item)); /* sanity check! */ + + new_array = kmem_zalloc(new_n_items * sizeof (void *), + KM_SLEEP); + /* + * Copy the pointers into the new array + */ + memcpy(new_array, array, ss->n_items * sizeof (void *)); + + /* + * Save the old array on the dirty list + */ + dirty = kmem_zalloc(sizeof (*dirty), KM_SLEEP); + dirty->array = ss->array; + dirty->n_items = ss->n_items; + dirty->next = ss->next; + ss->next = dirty; + + ss->array = (array = new_array); + ss->n_items = new_n_items; + } + + ASSERT(array != NULL && item < ss->n_items && array[item] == NULL); + + array[item] = new_element; + + mutex_exit(&ss->lock); + return (DDI_SUCCESS); +} + + +/* + * Fetch a pointer to the allocated soft state structure. + * + * This is designed to be cheap. + * + * There's an argument that there should be more checking for + * nil pointers and out of bounds on the array.. but we do a lot + * of that in the alloc/free routines. + * + * An array has the convenience that we don't need to lock read-access + * to it c.f. a linked list. However our "expanding array" strategy + * means that we should hold a readers lock on the i_ddi_soft_state + * structure. + * + * However, from a performance viewpoint, we need to do it without + * any locks at all -- this also makes it a leaf routine. The algorithm + * is 'lock-free' because we only discard the pointer arrays at + * ddi_soft_state_fini() time. + */ +void * +ddi_get_soft_state(void *state, int item) +{ + struct i_ddi_soft_state *ss = state; + + ASSERT(ss != NULL && item >= 0); + + if (item < ss->n_items && ss->array != NULL) + return (ss->array[item]); + return (NULL); +} + +/* + * Free the state structure corresponding to 'item.' Freeing an + * element that has either gone or was never allocated is not + * considered an error. Note that we free the state structure, but + * we don't shrink our pointer array, or discard 'dirty' arrays, + * since even a few pointers don't really waste too much memory. + * + * Passing an item number that is out of bounds, or a null pointer will + * provoke an error message. + */ +void +ddi_soft_state_free(void *state, int item) +{ + struct i_ddi_soft_state *ss; + void **array; + void *element; + static char msg[] = "ddi_soft_state_free:"; + + if ((ss = state) == NULL) { + cmn_err(CE_WARN, "%s null handle", + msg); + return; + } + + element = NULL; + + mutex_enter(&ss->lock); + + if ((array = ss->array) == NULL || ss->size == 0) { + cmn_err(CE_WARN, "%s bad handle", + msg); + } else if (item < 0 || item >= ss->n_items) { + cmn_err(CE_WARN, "%s item %d not in range [0..%lu]", + msg, item, ss->n_items - 1); + } else if (array[item] != NULL) { + element = array[item]; + array[item] = NULL; + } + + mutex_exit(&ss->lock); + + if (element) + kmem_free(element, ss->size); +} + + +/* + * Free the entire set of pointers, and any + * soft state structures contained therein. + * + * Note that we don't grab the ss->lock mutex, even though + * we're inspecting the various fields of the data structure. + * + * There is an implicit assumption that this routine will + * never run concurrently with any of the above on this + * particular state structure i.e. by the time the driver + * calls this routine, there should be no other threads + * running in the driver. + */ +void +ddi_soft_state_fini(void **state_p) +{ + struct i_ddi_soft_state *ss, *dirty; + int item; + static char msg[] = "ddi_soft_state_fini:"; + + if (state_p == NULL || (ss = *state_p) == NULL) { + return; + } + + if (ss->size == 0) { + cmn_err(CE_WARN, "%s bad handle", + msg); + return; + } + + if (ss->n_items > 0) { + for (item = 0; item < ss->n_items; item++) + ddi_soft_state_free(ss, item); + kmem_free(ss->array, ss->n_items * sizeof (void *)); + } + + /* + * Now delete any dirty arrays from previous 'grow' operations + */ + for (dirty = ss->next; dirty; dirty = ss->next) { + ss->next = dirty->next; + kmem_free(dirty->array, dirty->n_items * sizeof (void *)); + kmem_free(dirty, sizeof (*dirty)); + } + + mutex_destroy(&ss->lock); + kmem_free(ss, sizeof (*ss)); + + *state_p = NULL; +} + +int +ddi_create_minor_node(dev_info_t *dip, char *name, int spec_type, + minor_t minor_num, char *node_type, int flag) +{ + dev_t dev; + int error = 0; + char *r, *dup; + + dev = minor_num; + dip->dev = dev; + + MALLOC(dup, char *, strlen(name)+1, M_TEMP, M_WAITOK); + if (dup == NULL) + return (ENOMEM); + memcpy(dup, name, strlen(name)); + dup[strlen(name)] = '\0'; + + for (r = dup; + (r = strchr(r, '/')); + *r = '_') + /* empty */; + + dip->devc = NULL; + dip->devb = NULL; + + FREE(dup, M_TEMP); + + return (error); +} + + +void +ddi_remove_minor_node(dev_info_t *dip, char *name) +{ + if (dip->devc) { + dip->devc = NULL; + } + if (dip->devb) { + dip->devb = NULL; + } +} + + + +int +ddi_strtol(const char *str, char **nptr, int base, long *result) +{ + long val; + int c; + int xx; + int neg = 0; + long multmin; + long limit; + const char **ptr = (const char **)nptr; + const unsigned char *ustr = (const unsigned char *)str; + + if (ptr != (const char **)0) + *ptr = (char *)ustr; /* in case no number is formed */ + if (base < 0 || base > MBASE || base == 1) { + /* base is invalid -- should be a fatal error */ + return (EINVAL); + } + if (!isalnum(c = *ustr)) { + while (isspace(c)) + c = *++ustr; + switch (c) { + case '-': + neg++; + /* FALLTHROUGH */ + case '+': + c = *++ustr; + } + } + if (base == 0) + if (c != '0') + base = 10; + else if (ustr[1] == 'x' || ustr[1] == 'X') + base = 16; + else + base = 8; + /* + * for any base > 10, the digits incrementally following + * 9 are assumed to be "abc...z" or "ABC...Z" + */ + if (!lisalnum(c) || (xx = DIGIT(c)) >= base) { + /* no number formed */ + return (EINVAL); + } + if (base == 16 && c == '0' && (ustr[1] == 'x' || ustr[1] == 'X') && + isxdigit(ustr[2])) + c = *(ustr += 2); /* skip over leading "0x" or "0X" */ + + /* this code assumes that abs(LONG_MIN) >= abs(LONG_MAX) */ + if (neg) + limit = LONG_MIN; + else + limit = -LONG_MAX; + multmin = limit / (long)base; + val = -DIGIT(c); + for (c = *++ustr; lisalnum(c) && (xx = DIGIT(c)) < base; ) { + /* accumulate neg avoids surprises near LONG_MAX */ + if (val < multmin) + goto overflow; + val *= base; + if (val < limit + xx) + goto overflow; + val -= xx; + c = *++ustr; + } + if (ptr != (const char **)0) + *ptr = (char *)ustr; + *result = neg ? val : -val; + return (0); + +overflow: + for (c = *++ustr; lisalnum(c) && (xx = DIGIT(c)) < base; (c = *++ustr)) + ; + if (ptr != (const char **)0) + *ptr = (char *)ustr; + return (ERANGE); +} + +char *__cdecl +strpbrk(const char *s, const char *b) +{ + const char *p; + + do { + for (p = b; *p != '\0' && *p != *s; ++p) + ; + if (*p != '\0') + return ((char *)s); + } while (*s++); + return (NULL); +} + +int +ddi_strtoul(const char *str, char **nptr, int base, unsigned long *result) +{ + *result = (unsigned long)_strtoui64(str, nptr, base); + if (*result == 0) + return (EINVAL); + else if (*result == (unsigned long)ULONG_MAX) + return (ERANGE); + return (0); +} + +int +ddi_strtoull(const char *str, char **nptr, int base, unsigned long long *result) +{ + *result = (unsigned long long)_strtoui64(str, nptr, base); + if (*result == 0) + return (EINVAL); + else if (*result == ULLONG_MAX) + return (ERANGE); + return (0); +} + +int +ddi_strtoll(const char *str, char **nptr, int base, long long *result) +{ + long long val; + int c; + int xx; + int neg = 0; + long long multmin; + long long limit; + const char **ptr = (const char **)nptr; + const unsigned char *ustr = (const unsigned char *)str; + + if (ptr != (const char **)0) + *ptr = (char *)ustr; /* in case no number is formed */ + if (base < 0 || base > MBASE || base == 1) { + /* base is invalid -- should be a fatal error */ + return (EINVAL); + } + if (!isalnum(c = *ustr)) { + while (isspace(c)) + c = *++ustr; + switch (c) { + case '-': + neg++; + /* FALLTHROUGH */ + case '+': + c = *++ustr; + } + } + if (base == 0) + if (c != '0') + base = 10; + else if (ustr[1] == 'x' || ustr[1] == 'X') + base = 16; + else + base = 8; + /* + * for any base > 10, the digits incrementally following + * 9 are assumed to be "abc...z" or "ABC...Z" + */ + if (!lisalnum(c) || (xx = DIGIT(c)) >= base) { + /* no number formed */ + return (EINVAL); + } + if (base == 16 && c == '0' && (ustr[1] == 'x' || ustr[1] == 'X') && + isxdigit(ustr[2])) + c = *(ustr += 2); /* skip over leading "0x" or "0X" */ + + /* this code assumes that abs(LONG_MIN) >= abs(LONG_MAX) */ + if (neg) + limit = LONGLONG_MIN; + else + limit = -LONGLONG_MAX; + multmin = limit / (long)base; + val = -DIGIT(c); + for (c = *++ustr; lisalnum(c) && (xx = DIGIT(c)) < base; ) { + /* accumulate neg avoids surprises near LONG_MAX */ + if (val < multmin) + goto overflow; + val *= base; + if (val < limit + xx) + goto overflow; + val -= xx; + c = *++ustr; + } + if (ptr != (const char **)0) + *ptr = (char *)ustr; + *result = neg ? val : -val; + return (0); + +overflow: + for (c = *++ustr; lisalnum(c) && (xx = DIGIT(c)) < base; (c = *++ustr)) + ; + if (ptr != (const char **)0) + *ptr = (char *)ustr; + return (ERANGE); +} + +uint32_t +ddi_strcspn(const char *__restrict s, const char *__restrict charset) +{ + /* + * NB: idx and bit are temporaries whose use causes gcc 3.4.2 to + * generate better code. Without them, gcc gets a little confused. + */ + const char *s1; + ulong_t bit; + ulong_t tbl[(255 + 1) / LONG_BIT]; + int idx; + if (*s == '\0') + return (0); + + // 64bit code + tbl[0] = 1; + tbl[3] = tbl[2] = tbl[1] = 0; + for (; *charset != '\0'; charset++) { + idx = IDX(*charset); + bit = BIT(*charset); + tbl[idx] |= bit; + + } + + for (s1 = s; ; s1++) { + idx = IDX(*s1); + bit = BIT(*s1); + if ((tbl[idx] & bit) != 0) + break; + } + return (uint32_t)(s1 - s); +} + +extern size_t +strlcpy(char *s, const char *t, size_t n) +{ + const char *o = t; + + if (n) + do { + if (!--n) { + *s = 0; + break; + } + } while (*s++ = *t++); + if (!n) + while (*t++) + ; + return (uint32_t)(t - o - 1); +} + +extern size_t +strlcat(char *s, const char *t, size_t n) +{ + register size_t m; + const char *o = t; + + if (m = n) { + while (n && *s) { + n--; + s++; + } + m -= n; + if (n) + do { + if (!--n) { + *s = 0; + break; + } + } while (*s++ = *t++); + else + *s = 0; + } + if (!n) + while (*t++) + ; + return ((t - o) + m - 1); +} diff --git a/module/os/windows/spl/spl-debug.c b/module/os/windows/spl/spl-debug.c new file mode 100644 index 000000000000..d0681ec44138 --- /dev/null +++ b/module/os/windows/spl/spl-debug.c @@ -0,0 +1,39 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include +#include + +void +panic(const char *fmt, ...) __attribute__((__noreturn__)) +{ + va_list ap; + va_start(ap, fmt); + do { + KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, fmt, ap)); + DbgBreakPoint(); + windows_delay(hz); + } while (1); + va_end(ap); +} + + +/* Debug log support enabled */ diff --git a/module/os/windows/spl/spl-err.c b/module/os/windows/spl/spl-err.c new file mode 100644 index 000000000000..850bb3ebf273 --- /dev/null +++ b/module/os/windows/spl/spl-err.c @@ -0,0 +1,77 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#include +#include +#include + +#include + +void +vcmn_err(int ce, const char *fmt, va_list ap) +{ + char msg[MAXMSGLEN]; + + _vsnprintf(msg, MAXMSGLEN - 1, fmt, ap); + + switch (ce) { + case CE_IGNORE: + break; + case CE_CONT: + dprintf("%s", msg); + break; + case CE_NOTE: + dprintf("SPL: Notice: %s\n", msg); + break; + case CE_WARN: + TraceEvent(TRACE_WARNING, "SPL: Warning: %s\n", msg); + break; + case CE_PANIC: + PANIC("%s", msg); + break; + } +} /* vcmn_err() */ + +void +cmn_err(int ce, const char *fmt, ...) +{ + va_list ap; + + va_start(ap, fmt); + vcmn_err(ce, fmt, ap); + va_end(ap); +} /* cmn_err() */ + +int +spl_panic(const char *file, const char *func, int line, const char *fmt, ...) +{ + va_list ap; + + va_start(ap, fmt); + PANIC(fmt, ap); + va_end(ap); +} diff --git a/module/os/windows/spl/spl-kmem.c b/module/os/windows/spl/spl-kmem.c new file mode 100644 index 000000000000..eaf417d07a07 --- /dev/null +++ b/module/os/windows/spl/spl-kmem.c @@ -0,0 +1,6630 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2008 MacZFS + * Copyright (C) 2013, 2020 Jorgen Lundman + * Copyright (C) 2014 Brendon Humphrey + * Copyright (C) 2017 Sean Doran + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + * Portions Copyright 2022 Andrew Innes + * + */ + +#include +// #include +#include +#include +// #include +#include +#include +#include +// #include +#include +#include +#include +#include +// #include +#include +#include + +#include + +// TODO, track down what is using floats in this file +int _fltused = 0; + + + +// =============================================================== +// Options +// =============================================================== +// #define PRINT_CACHE_STATS 1 + +// =============================================================== +// OS Interface +// =============================================================== + +// 3500 kern.spl_vm_page_free_min, rarely changes +const unsigned int spl_vm_page_free_min = 3500; + +#define SMALL_PRESSURE_INCURSION_PAGES (spl_vm_page_free_min >> 5) + +static kcondvar_t spl_free_thread_cv; +static kmutex_t spl_free_thread_lock; +static boolean_t spl_free_thread_exit; +static volatile _Atomic int64_t spl_free; +int64_t spl_free_delta_ema; + +static boolean_t spl_event_thread_exit = FALSE; +PKEVENT low_mem_event = NULL; + +static volatile _Atomic int64_t spl_free_manual_pressure = 0; +static volatile _Atomic boolean_t spl_free_fast_pressure = FALSE; +static _Atomic bool spl_free_maybe_reap_flag = false; +static _Atomic uint64_t spl_free_last_pressure = 0; + +_Atomic uint32_t spl_vm_pages_reclaimed = 0; +_Atomic uint32_t spl_vm_pages_wanted = 0; +_Atomic uint32_t spl_vm_pressure_level = 0; + + +/* + * the spl_pressure_level enum only goes to four, + * but we want to watch kstat for whether + * mach's pressure is unavailable + */ +#define MAGIC_PRESSURE_UNAVAILABLE 1001001 + +// Start and end address of kernel memory +extern vm_offset_t virtual_space_start; +extern vm_offset_t virtual_space_end; + +// Can be polled to determine if the VM is experiecing +// a shortage of free pages. +extern int vm_pool_low(void); + +// Invoke the kernel debugger +extern void Debugger(const char *message); + +// Read from /dev/random +void read_random(void *buffer, uint_t numbytes); + +// =============================================================== +// Non Illumos Variables +// =============================================================== + +// Flag to cause tasks and threads to terminate as +// the kmem module is preparing to unload. +static int shutting_down = 0; + +// Amount of RAM in machine +uint64_t physmem = 0; + +// Size in bytes of the memory allocated in seg_kmem +extern uint64_t segkmem_total_mem_allocated; + +// Number of active threads +extern uint64_t zfs_threads; +extern uint64_t zfs_active_mutex; +extern uint64_t zfs_active_rwlock; + +extern uint64_t total_memory; +extern uint64_t real_total_memory; + +#define MULT 1 + +static const char *KMEM_VA_PREFIX = "kmem_va"; +static const char *KMEM_MAGAZINE_PREFIX = "kmem_magazine_"; + +extern void kstat_init(void); + +// =============================================================== +// Illumos Variables +// =============================================================== + +struct kmem_cache_kstat { + kstat_named_t kmc_buf_size; + kstat_named_t kmc_align; + kstat_named_t kmc_chunk_size; + kstat_named_t kmc_slab_size; + kstat_named_t kmc_alloc; + kstat_named_t kmc_alloc_fail; + kstat_named_t kmc_free; + kstat_named_t kmc_depot_alloc; + kstat_named_t kmc_depot_free; + kstat_named_t kmc_depot_contention; + kstat_named_t kmc_slab_alloc; + kstat_named_t kmc_slab_free; + kstat_named_t kmc_buf_constructed; + kstat_named_t kmc_buf_avail; + kstat_named_t kmc_buf_inuse; + kstat_named_t kmc_buf_total; + kstat_named_t kmc_buf_max; + kstat_named_t kmc_slab_create; + kstat_named_t kmc_slab_destroy; + kstat_named_t kmc_vmem_source; + kstat_named_t kmc_hash_size; + kstat_named_t kmc_hash_lookup_depth; + kstat_named_t kmc_hash_rescale; + kstat_named_t kmc_full_magazines; + kstat_named_t kmc_empty_magazines; + kstat_named_t kmc_magazine_size; + kstat_named_t kmc_reap; /* number of kmem_cache_reap() calls */ + kstat_named_t kmc_defrag; /* attempts to defrag all partial slabs */ + kstat_named_t kmc_scan; /* attempts to defrag one partial slab */ + kstat_named_t kmc_move_callbacks; /* sum of yes, no, later, dn, dk */ + kstat_named_t kmc_move_yes; + kstat_named_t kmc_move_no; + kstat_named_t kmc_move_later; + kstat_named_t kmc_move_dont_need; + kstat_named_t kmc_move_dont_know; /* obj unrecognized by client ... */ + kstat_named_t kmc_move_hunt_found; /* ... but found in mag layer */ + kstat_named_t kmc_move_slabs_freed; /* slabs freed by consolidator */ + kstat_named_t kmc_move_reclaimable; /* buffers, if consolidator ran */ + kstat_named_t kmc_no_vba_success; + kstat_named_t kmc_no_vba_fail; + kstat_named_t kmc_arc_no_grow_set; + kstat_named_t kmc_arc_no_grow; +} kmem_cache_kstat = { + { "buf_size", KSTAT_DATA_UINT64 }, + { "align", KSTAT_DATA_UINT64 }, + { "chunk_size", KSTAT_DATA_UINT64 }, + { "slab_size", KSTAT_DATA_UINT64 }, + { "alloc", KSTAT_DATA_UINT64 }, + { "alloc_fail", KSTAT_DATA_UINT64 }, + { "free", KSTAT_DATA_UINT64 }, + { "depot_alloc", KSTAT_DATA_UINT64 }, + { "depot_free", KSTAT_DATA_UINT64 }, + { "depot_contention", KSTAT_DATA_UINT64 }, + { "slab_alloc", KSTAT_DATA_UINT64 }, + { "slab_free", KSTAT_DATA_UINT64 }, + { "buf_constructed", KSTAT_DATA_UINT64 }, + { "buf_avail", KSTAT_DATA_UINT64 }, + { "buf_inuse", KSTAT_DATA_UINT64 }, + { "buf_total", KSTAT_DATA_UINT64 }, + { "buf_max", KSTAT_DATA_UINT64 }, + { "slab_create", KSTAT_DATA_UINT64 }, + { "slab_destroy", KSTAT_DATA_UINT64 }, + { "vmem_source", KSTAT_DATA_UINT64 }, + { "hash_size", KSTAT_DATA_UINT64 }, + { "hash_lookup_depth", KSTAT_DATA_UINT64 }, + { "hash_rescale", KSTAT_DATA_UINT64 }, + { "full_magazines", KSTAT_DATA_UINT64 }, + { "empty_magazines", KSTAT_DATA_UINT64 }, + { "magazine_size", KSTAT_DATA_UINT64 }, + { "reap", KSTAT_DATA_UINT64 }, + { "defrag", KSTAT_DATA_UINT64 }, + { "scan", KSTAT_DATA_UINT64 }, + { "move_callbacks", KSTAT_DATA_UINT64 }, + { "move_yes", KSTAT_DATA_UINT64 }, + { "move_no", KSTAT_DATA_UINT64 }, + { "move_later", KSTAT_DATA_UINT64 }, + { "move_dont_need", KSTAT_DATA_UINT64 }, + { "move_dont_know", KSTAT_DATA_UINT64 }, + { "move_hunt_found", KSTAT_DATA_UINT64 }, + { "move_slabs_freed", KSTAT_DATA_UINT64 }, + { "move_reclaimable", KSTAT_DATA_UINT64 }, + { "no_vba_success", KSTAT_DATA_UINT64 }, + { "no_vba_fail", KSTAT_DATA_UINT64 }, + { "arc_no_grow_set", KSTAT_DATA_UINT64 }, + { "arc_no_grow", KSTAT_DATA_UINT64 }, +}; + +static kmutex_t kmem_cache_kstat_lock; + +/* + * The default set of caches to back kmem_alloc(). + * These sizes should be reevaluated periodically. + * + * We want allocations that are multiples of the coherency granularity + * (64 bytes) to be satisfied from a cache which is a multiple of 64 + * bytes, so that it will be 64-byte aligned. For all multiples of 64, + * the next 1 greater than or equal to it must be a + * multiple of 64. + * + * We split the table into two sections: size <= 4k and size > 4k. This + * saves a lot of space and cache footprint in our cache tables. + */ +static const int kmem_alloc_sizes[] = { + 1 * 8, + 2 * 8, + 3 * 8, + 4 * 8, 5 * 8, 6 * 8, 7 * 8, + 4 * 16, 5 * 16, 6 * 16, 7 * 16, + 4 * 32, 5 * 32, 6 * 32, 7 * 32, + 4 * 64, 5 * 64, 6 * 64, 7 * 64, + 4 * 128, 9*64, 5 * 128, 6 * 128, 13*64, 7 * 128, + P2ALIGN(8192 / 8, 64), + P2ALIGN(8192 / 7, 64), + P2ALIGN(8192 / 6, 64), + P2ALIGN(8192 / 5, 64), + P2ALIGN(8192 / 4, 64), + P2ALIGN(8192 / 3, 64), + P2ALIGN(8192 / 2, 64), +}; + +static const int kmem_big_alloc_sizes[] = { + 2 * 4096, 3 * 4096, + 2 * 8192, 3 * 8192, + 4 * 8192, 5 * 8192, 6 * 8192, 7 * 8192, + 8 * 8192, 9 * 8192, 10 * 8192, 11 * 8192, + 12 * 8192, 13 * 8192, 14 * 8192, 15 * 8192, + 16 * 8192 +}; + +#define KMEM_MAXBUF 4096 +#define KMEM_BIG_MAXBUF_32BIT 32768 +#define KMEM_BIG_MAXBUF 131072 + +#define KMEM_BIG_MULTIPLE 4096 /* big_alloc_sizes must be a multiple */ +#define KMEM_BIG_SHIFT 12 /* lg(KMEM_BIG_MULTIPLE) */ + +static kmem_cache_t *kmem_alloc_table[KMEM_MAXBUF >> KMEM_ALIGN_SHIFT]; +static kmem_cache_t *kmem_big_alloc_table[KMEM_BIG_MAXBUF >> KMEM_BIG_SHIFT]; + +#define KMEM_ALLOC_TABLE_MAX (KMEM_MAXBUF >> KMEM_ALIGN_SHIFT) +static size_t kmem_big_alloc_table_max = 0; /* # of filled elements */ + +static kmem_magtype_t kmem_magtype[] = { + { 1, 8, 3200, 65536 }, + { 3, 16, 256, 32768 }, + { 7, 32, 64, 16384 }, + { 15, 64, 0, 8192 }, + { 31, 64, 0, 4096 }, + { 47, 64, 0, 2048 }, + { 63, 64, 0, 1024 }, + { 95, 64, 0, 512 }, + { 143, 64, 0, 0 }, +}; + +static struct bsd_timeout_wrapper kmem_update_timer; +static struct bsd_timeout_wrapper kmem_reaping; +static struct bsd_timeout_wrapper kmem_reaping_idspace; + +/* + * kmem tunables + */ +static struct timespec kmem_reap_interval = {15, 0}; +int kmem_depot_contention = 3; /* max failed tryenters per real interval */ +pgcnt_t kmem_reapahead = 0; /* start reaping N pages before pageout */ +int kmem_panic = 1; /* whether to panic on error */ +int kmem_logging = 0; /* kmem_log_enter() override */ +uint32_t kmem_mtbf = 0; /* mean time between failures [default: off] */ +size_t kmem_transaction_log_size; /* transaction log size [2% of memory] */ +size_t kmem_content_log_size; /* content log size [2% of memory] */ +size_t kmem_failure_log_size; /* failure log [4 pages per CPU] */ +size_t kmem_slab_log_size; /* slab create log [4 pages per CPU] */ +size_t kmem_content_maxsave = 256; /* KMF_CONTENTS max bytes to log */ +size_t kmem_lite_minsize = 0; /* minimum buffer size for KMF_LITE */ +size_t kmem_lite_maxalign = 8192; /* maximum buffer alignment for KMF_LITE */ +int kmem_lite_pcs = 4; /* number of PCs to store in KMF_LITE mode */ +size_t kmem_maxverify; /* maximum bytes to inspect in debug routines */ +size_t kmem_minfirewall; /* hardware-enforced redzone threshold */ + +size_t kmem_max_cached = KMEM_BIG_MAXBUF; /* maximum kmem_alloc cache */ + +/* + * Be aware that KMF_AUDIT does not release memory, and you will eventually + * grind to a halt. But it is useful to enable if you can trigger a memory + * fault, and wish to see the calling stack. + */ +#ifdef DEBUG +// can be 0 or KMF_LITE +// or KMF_DEADBEEF | KMF_REDZONE | KMF_CONTENTS +// with or without KMF_AUDIT +// int kmem_flags = KMF_DEADBEEF | KMF_REDZONE | KMF_CONTENTS | KMF_AUDIT; +int kmem_flags = KMF_LITE; +#else +int kmem_flags = 0; +#endif +int kmem_ready; + +static kmem_cache_t *kmem_slab_cache; +static kmem_cache_t *kmem_bufctl_cache; +static kmem_cache_t *kmem_bufctl_audit_cache; + +static kmutex_t kmem_cache_lock; /* inter-cache linkage only */ +static list_t kmem_caches; +extern vmem_t *heap_arena; +static taskq_t *kmem_taskq; +static kmutex_t kmem_flags_lock; +static vmem_t *kmem_metadata_arena; +static vmem_t *kmem_msb_arena; /* arena for metadata caches */ +static vmem_t *kmem_cache_arena; +static vmem_t *kmem_hash_arena; +static vmem_t *kmem_log_arena; +static vmem_t *kmem_oversize_arena; +static vmem_t *kmem_va_arena; +static vmem_t *kmem_default_arena; +static vmem_t *kmem_firewall_arena; + +/* + * kmem slab consolidator thresholds (tunables) + */ +size_t kmem_frag_minslabs = 101; /* minimum total slabs */ +size_t kmem_frag_numer = 1; /* free buffers (numerator) */ +size_t kmem_frag_denom = KMEM_VOID_FRACTION; /* buffers (denominator) */ +/* + * Maximum number of slabs from which to move buffers during a single + * maintenance interval while the system is not low on memory. + */ +size_t kmem_reclaim_max_slabs = 4; // smd 1 +/* + * Number of slabs to scan backwards from the end of the partial slab list + * when searching for buffers to relocate. + */ +size_t kmem_reclaim_scan_range = 48; // smd 12 + +/* consolidator knobs */ +static boolean_t kmem_move_noreap; +static boolean_t kmem_move_blocked; +static boolean_t kmem_move_fulltilt; +static boolean_t kmem_move_any_partial; + +#ifdef DEBUG +/* + * kmem consolidator debug tunables: + * Ensure code coverage by occasionally running the consolidator even when the + * caches are not fragmented (they may never be). These intervals are mean time + * in cache maintenance intervals (kmem_cache_update). + */ +uint32_t kmem_mtb_move = 20; /* defrag 1 slab (~5min) */ +uint32_t kmem_mtb_reap = 240; /* defrag all slabs (~1hrs) */ +uint32_t kmem_mtb_reap_count = 0; +#endif /* DEBUG */ + +static kmem_cache_t *kmem_defrag_cache; +static kmem_cache_t *kmem_move_cache; +static taskq_t *kmem_move_taskq; + +static void kmem_cache_scan(kmem_cache_t *); +static void kmem_cache_defrag(kmem_cache_t *); +static void kmem_slab_prefill(kmem_cache_t *, kmem_slab_t *); + + +kmem_log_header_t *kmem_transaction_log; +kmem_log_header_t *kmem_content_log; +kmem_log_header_t *kmem_failure_log; +kmem_log_header_t *kmem_slab_log; + +static int kmem_lite_count; /* # of PCs in kmem_buftag_lite_t */ + +#define KMEM_BUFTAG_LITE_ENTER(bt, count, caller) \ +if ((count) > 0) { \ +pc_t *_s = ((kmem_buftag_lite_t *)(bt))->bt_history; \ +pc_t *_e; \ +/* memmove() the old entries down one notch */ \ +for (_e = &_s[(count) - 1]; _e > _s; _e--) \ +*_e = *(_e - 1); \ +*_s = (uintptr_t)(caller); \ +} + +#define KMERR_MODIFIED 0 /* buffer modified while on freelist */ +#define KMERR_REDZONE 1 /* redzone violation (write past end of buf) */ +#define KMERR_DUPFREE 2 /* freed a buffer twice */ +#define KMERR_BADADDR 3 /* freed a bad (unallocated) address */ +#define KMERR_BADBUFTAG 4 /* buftag corrupted */ +#define KMERR_BADBUFCTL 5 /* bufctl corrupted */ +#define KMERR_BADCACHE 6 /* freed a buffer to the wrong cache */ +#define KMERR_BADSIZE 7 /* alloc size != free size */ +#define KMERR_BADBASE 8 /* buffer base address wrong */ + +struct { + hrtime_t kmp_timestamp; /* timestamp of panic */ + int kmp_error; /* type of kmem error */ + void *kmp_buffer; /* buffer that induced panic */ + void *kmp_realbuf; /* real start address for buffer */ + kmem_cache_t *kmp_cache; /* buffer's cache according to client */ + kmem_cache_t *kmp_realcache; /* actual cache containing buffer */ + kmem_slab_t *kmp_slab; /* slab accoring to kmem_findslab() */ + kmem_bufctl_t *kmp_bufctl; /* bufctl */ +} kmem_panic_info; + +extern uint64_t stat_osif_malloc_success; +extern uint64_t stat_osif_malloc_bytes; +extern uint64_t stat_osif_free; +extern uint64_t stat_osif_free_bytes; + +extern uint64_t spl_bucket_non_pow2_allocs; + +// stats for spl_root_allocator(); +extern uint64_t spl_root_allocator_calls; +extern uint64_t spl_root_allocator_large_bytes_asked; +extern uint64_t spl_root_allocator_small_bytes_asked; +extern uint64_t spl_root_allocator_minalloc_bytes_asked; +extern uint64_t spl_root_allocator_extra_pass; +extern uint64_t spl_root_allocator_recovered; +extern uint64_t spl_root_allocator_recovered_bytes; + +extern uint64_t spl_vmem_unconditional_allocs; +extern uint64_t spl_vmem_unconditional_alloc_bytes; +extern uint64_t spl_vmem_conditional_allocs; +extern uint64_t spl_vmem_conditional_alloc_bytes; +extern uint64_t spl_vmem_conditional_alloc_deny; +extern uint64_t spl_vmem_conditional_alloc_deny_bytes; + +extern uint64_t spl_xat_success; +extern uint64_t spl_xat_late_success; +extern uint64_t spl_xat_late_success_nosleep; +extern uint64_t spl_xat_pressured; +extern uint64_t spl_xat_bailed; +extern uint64_t spl_xat_bailed_contended; +extern uint64_t spl_xat_lastalloc; +extern uint64_t spl_xat_lastfree; +extern uint64_t spl_xat_forced; +extern uint64_t spl_xat_sleep; +extern uint64_t spl_xat_late_deny; +extern uint64_t spl_xat_no_waiters; +extern uint64_t spl_xft_wait; + +extern uint64_t spl_vba_parent_memory_appeared; +extern uint64_t spl_vba_parent_memory_blocked; +extern uint64_t spl_vba_hiprio_blocked; +extern uint64_t spl_vba_cv_timeout; +extern uint64_t spl_vba_loop_timeout; +extern uint64_t spl_vba_cv_timeout_blocked; +extern uint64_t spl_vba_loop_timeout_blocked; +extern uint64_t spl_vba_sleep; +extern uint64_t spl_vba_loop_entries; + +extern uint64_t spl_bucket_tunable_large_span; +extern uint64_t spl_bucket_tunable_small_span; +extern void spl_set_bucket_tunable_large_span(uint64_t); +extern void spl_set_bucket_tunable_small_span(uint64_t); + +extern _Atomic uint64_t spl_arc_no_grow_bits; +extern uint64_t spl_arc_no_grow_count; + +extern uint64_t spl_frag_max_walk; +extern uint64_t spl_frag_walked_out; +extern uint64_t spl_frag_walk_cnt; + +uint64_t spl_buckets_mem_free = 0; +uint64_t spl_arc_reclaim_avoided = 0; + +uint64_t kmem_free_to_slab_when_fragmented = 0; + +// Keep an eye on stack size in known places (macOS) +extern _Atomic uint64_t spl_lowest_vdev_disk_stack_remaining; +extern _Atomic uint64_t spl_lowest_zvol_stack_remaining; +extern _Atomic uint64_t spl_lowest_alloc_stack_remaining; + +typedef struct spl_stats { + kstat_named_t spl_os_alloc; + kstat_named_t spl_active_threads; + kstat_named_t spl_active_mutex; + kstat_named_t spl_active_rwlock; + kstat_named_t spl_active_tsd; + kstat_named_t spl_free_wake_count; + kstat_named_t spl_spl_free; + kstat_named_t spl_spl_free_manual_pressure; + kstat_named_t spl_spl_free_fast_pressure; + kstat_named_t spl_spl_free_delta_ema; + kstat_named_t spl_spl_free_negative_count; + kstat_named_t spl_osif_malloc_success; + kstat_named_t spl_osif_malloc_bytes; + kstat_named_t spl_osif_free; + kstat_named_t spl_osif_free_bytes; + kstat_named_t spl_bucket_non_pow2_allocs; + + kstat_named_t spl_vmem_unconditional_allocs; + kstat_named_t spl_vmem_unconditional_alloc_bytes; + kstat_named_t spl_vmem_conditional_allocs; + kstat_named_t spl_vmem_conditional_alloc_bytes; + kstat_named_t spl_vmem_conditional_alloc_deny; + kstat_named_t spl_vmem_conditional_alloc_deny_bytes; + + kstat_named_t spl_xat_success; + kstat_named_t spl_xat_late_success; + kstat_named_t spl_xat_late_success_nosleep; + kstat_named_t spl_xat_pressured; + kstat_named_t spl_xat_bailed; + kstat_named_t spl_xat_bailed_contended; + kstat_named_t spl_xat_lastalloc; + kstat_named_t spl_xat_lastfree; + kstat_named_t spl_xat_forced; + kstat_named_t spl_xat_sleep; + kstat_named_t spl_xat_late_deny; + kstat_named_t spl_xat_no_waiters; + kstat_named_t spl_xft_wait; + + kstat_named_t spl_vba_parent_memory_appeared; + kstat_named_t spl_vba_parent_memory_blocked; + kstat_named_t spl_vba_hiprio_blocked; + kstat_named_t spl_vba_cv_timeout; + kstat_named_t spl_vba_loop_timeout; + kstat_named_t spl_vba_cv_timeout_blocked; + kstat_named_t spl_vba_loop_timeout_blocked; + kstat_named_t spl_vba_sleep; + kstat_named_t spl_vba_loop_entries; + + kstat_named_t spl_bucket_tunable_large_span; + kstat_named_t spl_bucket_tunable_small_span; + + kstat_named_t spl_buckets_mem_free; + kstat_named_t spl_arc_no_grow_bits; + kstat_named_t spl_arc_no_grow_count; + kstat_named_t spl_frag_max_walk; + kstat_named_t spl_frag_walked_out; + kstat_named_t spl_frag_walk_cnt; + kstat_named_t spl_arc_reclaim_avoided; + + kstat_named_t kmem_free_to_slab_when_fragmented; + + kstat_named_t spl_vm_pages_reclaimed; + kstat_named_t spl_vm_pages_wanted; + kstat_named_t spl_vm_pressure_level; + + kstat_named_t spl_lowest_alloc_stack_remaining; + kstat_named_t spl_lowest_vdev_disk_stack_remaining; + kstat_named_t spl_lowest_zvol_stack_remaining; +} spl_stats_t; + +static spl_stats_t spl_stats = { + {"os_mem_alloc", KSTAT_DATA_UINT64}, + {"active_threads", KSTAT_DATA_UINT64}, + {"active_mutex", KSTAT_DATA_UINT64}, + {"active_rwlock", KSTAT_DATA_UINT64}, + {"active_tsd", KSTAT_DATA_UINT64}, + {"spl_free_wake_count", KSTAT_DATA_UINT64}, + {"spl_spl_free", KSTAT_DATA_INT64}, + {"spl_spl_free_manual_pressure", KSTAT_DATA_UINT64}, + {"spl_spl_free_fast_pressure", KSTAT_DATA_UINT64}, + {"spl_spl_free_delta_ema", KSTAT_DATA_UINT64}, + {"spl_spl_free_negative_count", KSTAT_DATA_UINT64}, + {"spl_osif_malloc_success", KSTAT_DATA_UINT64}, + {"spl_osif_malloc_bytes", KSTAT_DATA_UINT64}, + {"spl_osif_free", KSTAT_DATA_UINT64}, + {"spl_osif_free_bytes", KSTAT_DATA_UINT64}, + {"spl_bucket_non_pow2_allocs", KSTAT_DATA_UINT64}, + + {"vmem_unconditional_allocs", KSTAT_DATA_UINT64}, + {"vmem_unconditional_alloc_bytes", KSTAT_DATA_UINT64}, + {"vmem_conditional_allocs", KSTAT_DATA_UINT64}, + {"vmem_conditional_alloc_bytes", KSTAT_DATA_UINT64}, + {"vmem_conditional_alloc_deny", KSTAT_DATA_UINT64}, + {"vmem_conditional_alloc_deny_bytes", KSTAT_DATA_UINT64}, + + {"spl_xat_success", KSTAT_DATA_UINT64}, + {"spl_xat_late_success", KSTAT_DATA_UINT64}, + {"spl_xat_late_success_nosleep", KSTAT_DATA_UINT64}, + {"spl_xat_pressured", KSTAT_DATA_UINT64}, + {"spl_xat_bailed", KSTAT_DATA_UINT64}, + {"spl_xat_bailed_contended", KSTAT_DATA_UINT64}, + {"spl_xat_lastalloc", KSTAT_DATA_UINT64}, + {"spl_xat_lastfree", KSTAT_DATA_UINT64}, + {"spl_xat_forced", KSTAT_DATA_UINT64}, + {"spl_xat_sleep", KSTAT_DATA_UINT64}, + {"spl_xat_late_deny", KSTAT_DATA_UINT64}, + {"spl_xat_no_waiters", KSTAT_DATA_UINT64}, + {"spl_xft_wait", KSTAT_DATA_UINT64}, + + {"spl_vba_parent_memory_appeared", KSTAT_DATA_UINT64}, + {"spl_vba_parent_memory_blocked", KSTAT_DATA_UINT64}, + {"spl_vba_hiprio_blocked", KSTAT_DATA_UINT64}, + {"spl_vba_cv_timeout", KSTAT_DATA_UINT64}, + {"spl_vba_loop_timeout", KSTAT_DATA_UINT64}, + {"spl_vba_cv_timeout_blocked", KSTAT_DATA_UINT64}, + {"spl_vba_loop_timeout_blocked", KSTAT_DATA_UINT64}, + {"spl_vba_sleep", KSTAT_DATA_UINT64}, + {"spl_vba_loop_entries", KSTAT_DATA_UINT64}, + + {"spl_tunable_large_span", KSTAT_DATA_UINT64}, + {"spl_tunable_small_span", KSTAT_DATA_UINT64}, + + {"spl_buckets_mem_free", KSTAT_DATA_UINT64}, + {"spl_arc_no_grow_bits", KSTAT_DATA_UINT64}, + {"spl_arc_no_grow_count", KSTAT_DATA_UINT64}, + + {"spl_vmem_frag_max_walk", KSTAT_DATA_UINT64}, + {"spl_vmem_frag_walked_out", KSTAT_DATA_UINT64}, + {"spl_vmem_frag_walk_cnt", KSTAT_DATA_UINT64}, + {"spl_arc_reclaim_avoided", KSTAT_DATA_UINT64}, + + {"kmem_free_to_slab_when_fragmented", KSTAT_DATA_UINT64}, + {"spl_vm_pages_reclaimed", KSTAT_DATA_UINT64}, + {"spl_vm_pages_wanted", KSTAT_DATA_UINT64}, + {"spl_vm_pressure_level", KSTAT_DATA_UINT64}, + + {"lowest_alloc_stack_remaining", KSTAT_DATA_UINT64}, + {"lowest_vdev_disk_stack_remaining", KSTAT_DATA_UINT64}, + {"lowest_zvol_stack_remaining", KSTAT_DATA_UINT64}, + +}; + +static kstat_t *spl_ksp = 0; + +// Stub out caller() +caddr_t +caller() +{ + return ((caddr_t)(0)); +} + +void * +calloc(size_t n, size_t s) +{ + return (zfs_kmem_zalloc(n * s, KM_NOSLEEP)); +} + +#define IS_DIGIT(c) ((c) >= '0' && (c) <= '9') + +#define IS_ALPHA(c) \ +(((c) >= 'a' && (c) <= 'z') || ((c) >= 'A' && (c) <= 'Z')) + +/* + * BGH - Missing from Windows? + * + * Convert a string into a valid C identifier by replacing invalid + * characters with '_'. Also makes sure the string is nul-terminated + * and takes up at most n bytes. + */ +void +strident_canon(char *s, size_t n) +{ + char c; + char *end = s + n - 1; + + if ((c = *s) == 0) + return; + + if (!IS_ALPHA(c) && c != '_') + *s = '_'; + + while (s < end && ((c = *(++s)) != 0)) { + if (!IS_ALPHA(c) && !IS_DIGIT(c) && c != '_') + *s = '_'; + } + *s = 0; +} + +int +strident_valid(const char *id) +{ + int c = *id++; + + if (!IS_ALPHA(c) && c != '_') + return (0); + while ((c = *id++) != 0) { + if (!IS_ALPHA(c) && !IS_DIGIT(c) && c != '_') + return (0); + } + return (1); +} + +static void +copy_pattern(uint64_t pattern, void *buf_arg, size_t size) +{ + uint64_t *bufend = (uint64_t *)((char *)buf_arg + size); + uint64_t *buf = buf_arg; + + while (buf < bufend) + *buf++ = pattern; +} + +static void * +verify_pattern(uint64_t pattern, void *buf_arg, size_t size) +{ + uint64_t *bufend = (uint64_t *)((char *)buf_arg + size); + uint64_t *buf; + + for (buf = buf_arg; buf < bufend; buf++) + if (*buf != pattern) + return (buf); + return (NULL); +} + +static void * +verify_and_copy_pattern(uint64_t old, uint64_t new, void *buf_arg, size_t size) +{ + uint64_t *bufend = (uint64_t *)((char *)buf_arg + size); + uint64_t *buf; + + for (buf = buf_arg; buf < bufend; buf++) { + if (*buf != old) { + copy_pattern(old, buf_arg, + (char *)buf - (char *)buf_arg); + return (buf); + } + *buf = new; + } + + return (NULL); +} + +static void +kmem_cache_applyall(void (*func)(kmem_cache_t *), taskq_t *tq, int tqflag) +{ + kmem_cache_t *cp; + + mutex_enter(&kmem_cache_lock); + for (cp = list_head(&kmem_caches); cp != NULL; + cp = list_next(&kmem_caches, cp)) + if (tq != NULL) + (void) taskq_dispatch(tq, (task_func_t *)func, cp, + tqflag); + else + func(cp); + mutex_exit(&kmem_cache_lock); +} + +static void +kmem_cache_applyall_id(void (*func)(kmem_cache_t *), taskq_t *tq, int tqflag) +{ + kmem_cache_t *cp; + + mutex_enter(&kmem_cache_lock); + for (cp = list_head(&kmem_caches); cp != NULL; + cp = list_next(&kmem_caches, cp)) { + if (!(cp->cache_cflags & KMC_IDENTIFIER)) + continue; + if (tq != NULL) + (void) taskq_dispatch(tq, (task_func_t *)func, cp, + tqflag); + else + func(cp); + } + mutex_exit(&kmem_cache_lock); +} + +/* + * Debugging support. Given a buffer address, find its slab. + */ +static kmem_slab_t * +kmem_findslab(kmem_cache_t *cp, void *buf) +{ + kmem_slab_t *sp; + + mutex_enter(&cp->cache_lock); + for (sp = list_head(&cp->cache_complete_slabs); sp != NULL; + sp = list_next(&cp->cache_complete_slabs, sp)) { + if (KMEM_SLAB_MEMBER(sp, buf)) { + mutex_exit(&cp->cache_lock); + return (sp); + } + } + for (sp = avl_first(&cp->cache_partial_slabs); sp != NULL; + sp = AVL_NEXT(&cp->cache_partial_slabs, sp)) { + if (KMEM_SLAB_MEMBER(sp, buf)) { + mutex_exit(&cp->cache_lock); + return (sp); + } + } + mutex_exit(&cp->cache_lock); + + return (NULL); +} + +static void +kmem_error(int error, kmem_cache_t *cparg, void *bufarg) +{ + kmem_buftag_t *btp = NULL; + kmem_bufctl_t *bcp = NULL; + kmem_cache_t *cp = cparg; + kmem_slab_t *sp; + uint64_t *off; + void *buf = bufarg; + + kmem_logging = 0; /* stop logging when a bad thing happens */ + + kmem_panic_info.kmp_timestamp = gethrtime(); + + sp = kmem_findslab(cp, buf); + if (sp == NULL) { + for (cp = list_tail(&kmem_caches); cp != NULL; + cp = list_prev(&kmem_caches, cp)) { + if ((sp = kmem_findslab(cp, buf)) != NULL) + break; + } + } + + if (sp == NULL) { + cp = NULL; + error = KMERR_BADADDR; + } else { + if (cp != cparg) + error = KMERR_BADCACHE; + else + buf = (char *)bufarg - + ((uintptr_t)bufarg - + (uintptr_t)sp->slab_base) % cp->cache_chunksize; + if (buf != bufarg) + error = KMERR_BADBASE; + if (cp->cache_flags & KMF_BUFTAG) + btp = KMEM_BUFTAG(cp, buf); + if (cp->cache_flags & KMF_HASH) { + mutex_enter(&cp->cache_lock); + for (bcp = *KMEM_HASH(cp, buf); bcp; bcp = bcp->bc_next) + if (bcp->bc_addr == buf) + break; + mutex_exit(&cp->cache_lock); + if (bcp == NULL && btp != NULL) + bcp = btp->bt_bufctl; + if (kmem_findslab(cp->cache_bufctl_cache, bcp) == + NULL || P2PHASE((uintptr_t)bcp, KMEM_ALIGN) || + bcp->bc_addr != buf) { + error = KMERR_BADBUFCTL; + bcp = NULL; + } + } + } + + kmem_panic_info.kmp_error = error; + kmem_panic_info.kmp_buffer = bufarg; + kmem_panic_info.kmp_realbuf = buf; + kmem_panic_info.kmp_cache = cparg; + kmem_panic_info.kmp_realcache = cp; + kmem_panic_info.kmp_slab = sp; + kmem_panic_info.kmp_bufctl = bcp; + + dprintf("SPL: kernel memory allocator: "); + + switch (error) { + + case KMERR_MODIFIED: + TraceEvent(TRACE_ERROR, "buffer modified after being" + " freed\n"); + off = verify_pattern(KMEM_FREE_PATTERN, buf, + cp->cache_verify); + if (off == NULL) /* shouldn't happen */ + off = buf; + TraceEvent(TRACE_ERROR, "SPL: modification occurred " + "at offset 0x%lx (0x%llx replaced by 0x%llx)\n", + (uintptr_t)off - (uintptr_t)buf, + (longlong_t)KMEM_FREE_PATTERN, (longlong_t)*off); + break; + + case KMERR_REDZONE: + TraceEvent(TRACE_ERROR, "redzone violation: write past" + " end of buffer\n"); + break; + + case KMERR_BADADDR: + TraceEvent(TRACE_ERROR, "invalid free: buffer not in" + " cache\n"); + break; + + case KMERR_DUPFREE: + TraceEvent(TRACE_ERROR, "duplicate free: buffer freed" + " twice\n"); + break; + + case KMERR_BADBUFTAG: + TraceEvent(TRACE_ERROR, "boundary tag corrupted\n"); + TraceEvent(TRACE_ERROR, "SPL: bcp ^ bxstat = %lx, " + "should be %lx\n", + (intptr_t)btp->bt_bufctl ^ btp->bt_bxstat, + KMEM_BUFTAG_FREE); + break; + + case KMERR_BADBUFCTL: + TraceEvent(TRACE_ERROR, "bufctl corrupted\n"); + break; + + case KMERR_BADCACHE: + TraceEvent(TRACE_ERROR, "buffer freed to wrong " + "cache\n"); + TraceEvent(TRACE_ERROR, "SPL: buffer was allocated" + " from %s,\n", cp->cache_name); + TraceEvent(TRACE_ERROR, "SPL: caller attempting free" + " to %s.\n", cparg->cache_name); + break; + + case KMERR_BADSIZE: + TraceEvent(TRACE_ERROR, "bad free: free size (%u) !=" + " alloc size (%u)\n", + KMEM_SIZE_DECODE(((uint32_t *)btp)[0]), + KMEM_SIZE_DECODE(((uint32_t *)btp)[1])); + break; + + case KMERR_BADBASE: + TraceEvent(TRACE_ERROR, "bad free: free address" + " (%p) != alloc address (%p)\n", bufarg, buf); + break; + } + + dprintf("SPL: buffer=%p bufctl=%p cache: %s\n", + bufarg, (void *)bcp, cparg->cache_name); + + if (bcp != NULL && (cp->cache_flags & KMF_AUDIT) && + error != KMERR_BADBUFCTL) { + int d; + timestruc_t ts = {0, 0}; + kmem_bufctl_audit_t *bcap = (kmem_bufctl_audit_t *)bcp; + + hrt2ts(kmem_panic_info.kmp_timestamp - bcap->bc_timestamp, &ts); + dprintf("SPL: previous transaction on buffer %p:\n", buf); + dprintf("SPL: thread=%p time=T-%ld.%09ld slab=%p cache: " + "%s\n", (void *)bcap->bc_thread, ts.tv_sec, ts.tv_nsec, + (void *)sp, cp->cache_name); + for (d = 0; d < MIN(bcap->bc_depth, KMEM_STACK_DEPTH); d++) { + dprintf(" : %p\n, ", (void *)bcap->bc_stack[d]); + } + } + + if (kmem_panic > 0) { + extern void IODelay(unsigned microseconds); // lh_cpu[max_ncpus]; + int i; + + /* + * Make sure that lhp->lh_cpu[] is nicely aligned + * to prevent false sharing of cache lines. + */ + lhsize = P2ROUNDUP(lhsize, KMEM_ALIGN); + lhp = vmem_xalloc(kmem_log_arena, lhsize, 64, P2NPHASE(lhsize, 64), 0, + NULL, NULL, VM_SLEEP); + memset(lhp, 0, lhsize); + + mutex_init(&lhp->lh_lock, NULL, MUTEX_DEFAULT, NULL); + lhp->lh_nchunks = nchunks; + lhp->lh_chunksize = P2ROUNDUP(logsize / nchunks + 1, PAGESIZE); + lhp->lh_base = vmem_alloc_impl(kmem_log_arena, + lhp->lh_chunksize * nchunks, VM_SLEEP); + lhp->lh_free = vmem_alloc_impl(kmem_log_arena, + nchunks * sizeof (int), VM_SLEEP); + memset(lhp->lh_base, 0, lhp->lh_chunksize * nchunks); + + for (i = 0; i < max_ncpus; i++) { + kmem_cpu_log_header_t *clhp = &lhp->lh_cpu[i]; + mutex_init(&clhp->clh_lock, NULL, MUTEX_DEFAULT, NULL); + clhp->clh_chunk = i; + } + + for (i = max_ncpus; i < nchunks; i++) + lhp->lh_free[i] = i; + + lhp->lh_head = max_ncpus; + lhp->lh_tail = 0; + + return (lhp); +} + + +static void +kmem_log_fini(kmem_log_header_t *lhp) +{ + int nchunks = 4 * max_ncpus; + size_t lhsize = (size_t)&((kmem_log_header_t *)0)->lh_cpu[max_ncpus]; + int i; + + + + for (i = 0; i < max_ncpus; i++) { + kmem_cpu_log_header_t *clhp = &lhp->lh_cpu[i]; + mutex_destroy(&clhp->clh_lock); + } + + vmem_free_impl(kmem_log_arena, lhp->lh_free, nchunks * sizeof (int)); + + vmem_free_impl(kmem_log_arena, lhp->lh_base, + lhp->lh_chunksize * nchunks); + + mutex_destroy(&lhp->lh_lock); + + lhsize = P2ROUNDUP(lhsize, KMEM_ALIGN); + vmem_xfree(kmem_log_arena, lhp, lhsize); +} + + +static void * +kmem_log_enter(kmem_log_header_t *lhp, void *data, size_t size) +{ + void *logspace; + + kmem_cpu_log_header_t *clhp = &lhp->lh_cpu[CPU_SEQID]; + + // if (lhp == NULL || kmem_logging == 0 || panicstr) + if (lhp == NULL || kmem_logging == 0) + return (NULL); + + mutex_enter(&clhp->clh_lock); + clhp->clh_hits++; + if (size > clhp->clh_avail) { + mutex_enter(&lhp->lh_lock); + lhp->lh_hits++; + lhp->lh_free[lhp->lh_tail] = clhp->clh_chunk; + lhp->lh_tail = (lhp->lh_tail + 1) % lhp->lh_nchunks; + clhp->clh_chunk = lhp->lh_free[lhp->lh_head]; + lhp->lh_head = (lhp->lh_head + 1) % lhp->lh_nchunks; + clhp->clh_current = lhp->lh_base + + clhp->clh_chunk * lhp->lh_chunksize; + clhp->clh_avail = lhp->lh_chunksize; + if (size > lhp->lh_chunksize) + size = lhp->lh_chunksize; + mutex_exit(&lhp->lh_lock); + } + logspace = clhp->clh_current; + clhp->clh_current += size; + clhp->clh_avail -= size; + memcpy(logspace, data, size); + mutex_exit(&clhp->clh_lock); + return (logspace); +} + +#define KMEM_AUDIT(lp, cp, bcp) \ +{ \ +kmem_bufctl_audit_t *_bcp = (kmem_bufctl_audit_t *)(bcp); \ +_bcp->bc_timestamp = gethrtime(); \ +_bcp->bc_thread = spl_current_thread(); \ +_bcp->bc_depth = getpcstack(_bcp->bc_stack, KMEM_STACK_DEPTH); \ +_bcp->bc_lastlog = kmem_log_enter((lp), _bcp, sizeof (*_bcp)); \ +} + +/* kmem_cache_alloc 344 -> 54 */ +__attribute__((noinline)) +static void +kmem_log_event(kmem_log_header_t *lp, kmem_cache_t *cp, + kmem_slab_t *sp, void *addr) +{ + kmem_bufctl_audit_t bca; + + memset(&bca, 0, sizeof (kmem_bufctl_audit_t)); + bca.bc_addr = addr; + bca.bc_slab = sp; + KMEM_AUDIT(lp, cp, &bca); +} + +/* + * Create a new slab for cache cp. + */ +__attribute__((noinline)) +static kmem_slab_t * +kmem_slab_create(kmem_cache_t *cp, int kmflag) +{ + size_t slabsize = cp->cache_slabsize; + size_t chunksize = cp->cache_chunksize; + int cache_flags = cp->cache_flags; + size_t color, chunks; + char *buf, *slab; + kmem_slab_t *sp; + kmem_bufctl_t *bcp; + vmem_t *vmp = cp->cache_arena; + + ASSERT(MUTEX_NOT_HELD(&cp->cache_lock)); + + color = cp->cache_color + cp->cache_align; + if (color > cp->cache_maxcolor) + color = cp->cache_mincolor; + cp->cache_color = color; + + slab = vmem_alloc_impl(vmp, slabsize, kmflag & KM_VMFLAGS); + + if (slab == NULL) + goto vmem_alloc_failure; + + ASSERT(P2PHASE((uintptr_t)slab, vmp->vm_quantum) == 0); + + /* + * Reverify what was already checked in kmem_cache_set_move(), since the + * consolidator depends (for correctness) on slabs being initialized + * with the 0xbaddcafe memory pattern (setting a low order bit usable by + * clients to distinguish uninitialized memory from known objects). + */ + ASSERT((cp->cache_move == NULL) || !(cp->cache_cflags & KMC_NOTOUCH)); + if (!(cp->cache_cflags & KMC_NOTOUCH)) + copy_pattern(KMEM_UNINITIALIZED_PATTERN, slab, slabsize); + + if (cache_flags & KMF_HASH) { + if ((sp = kmem_cache_alloc(kmem_slab_cache, kmflag)) == NULL) + goto slab_alloc_failure; + chunks = (slabsize - color) / chunksize; + } else { + sp = KMEM_SLAB(cp, slab); + chunks = (slabsize - sizeof (kmem_slab_t) - color) / chunksize; + } + + sp->slab_cache = cp; + sp->slab_head = NULL; + sp->slab_refcnt = 0; + sp->slab_base = buf = slab + color; + sp->slab_chunks = chunks; + sp->slab_stuck_offset = (uint32_t)-1; + sp->slab_later_count = 0; + sp->slab_flags = 0; + sp->slab_create_time = gethrtime(); + + ASSERT(chunks > 0); + while (chunks-- != 0) { + if (cache_flags & KMF_HASH) { + bcp = kmem_cache_alloc(cp->cache_bufctl_cache, kmflag); + if (bcp == NULL) + goto bufctl_alloc_failure; + if (cache_flags & KMF_AUDIT) { + kmem_bufctl_audit_t *bcap = + (kmem_bufctl_audit_t *)bcp; + memset(bcap, 0, sizeof (kmem_bufctl_audit_t)); + bcap->bc_cache = cp; + } + bcp->bc_addr = buf; + bcp->bc_slab = sp; + } else { + bcp = KMEM_BUFCTL(cp, buf); + } + if (cache_flags & KMF_BUFTAG) { + kmem_buftag_t *btp = KMEM_BUFTAG(cp, buf); + btp->bt_redzone = KMEM_REDZONE_PATTERN; + btp->bt_bufctl = bcp; + btp->bt_bxstat = (intptr_t)bcp ^ KMEM_BUFTAG_FREE; + if (cache_flags & KMF_DEADBEEF) { + copy_pattern(KMEM_FREE_PATTERN, buf, + cp->cache_verify); + } + } + bcp->bc_next = sp->slab_head; + sp->slab_head = bcp; + buf += chunksize; + } + + kmem_log_event(kmem_slab_log, cp, sp, slab); + + return (sp); + +bufctl_alloc_failure: + + while ((bcp = sp->slab_head) != NULL) { + sp->slab_head = bcp->bc_next; + kmem_cache_free(cp->cache_bufctl_cache, bcp); + } + kmem_cache_free(kmem_slab_cache, sp); + +slab_alloc_failure: + + vmem_free_impl(vmp, slab, slabsize); + +vmem_alloc_failure: + + if (0 == (kmflag & KM_NO_VBA)) { + kmem_log_event(kmem_failure_log, cp, NULL, NULL); + atomic_inc_64(&cp->cache_alloc_fail); + } + + return (NULL); +} + +/* + * Destroy a slab. + */ +static void +kmem_slab_destroy(kmem_cache_t *cp, kmem_slab_t *sp) +{ + vmem_t *vmp = cp->cache_arena; + void *slab = (void *)P2ALIGN((uintptr_t)sp->slab_base, vmp->vm_quantum); + + ASSERT(MUTEX_NOT_HELD(&cp->cache_lock)); + ASSERT(sp->slab_refcnt == 0); + + if (cp->cache_flags & KMF_HASH) { + kmem_bufctl_t *bcp; + while ((bcp = sp->slab_head) != NULL) { + sp->slab_head = bcp->bc_next; + kmem_cache_free(cp->cache_bufctl_cache, bcp); + } + kmem_cache_free(kmem_slab_cache, sp); + } + kpreempt(KPREEMPT_SYNC); + vmem_free_impl(vmp, slab, cp->cache_slabsize); +} + +static void * +kmem_slab_alloc_impl(kmem_cache_t *cp, kmem_slab_t *sp, boolean_t prefill) +{ + kmem_bufctl_t *bcp, **hash_bucket; + void *buf; + boolean_t new_slab = (sp->slab_refcnt == 0); + + ASSERT(MUTEX_HELD(&cp->cache_lock)); + /* + * kmem_slab_alloc() drops cache_lock when it creates a new slab, so we + * can't ASSERT(avl_is_empty(&cp->cache_partial_slabs)) here when the + * slab is newly created. + */ + ASSERT(new_slab || (KMEM_SLAB_IS_PARTIAL(sp) && + (sp == avl_first(&cp->cache_partial_slabs)))); + ASSERT(sp->slab_cache == cp); + + cp->cache_slab_alloc++; + cp->cache_bufslab--; + sp->slab_refcnt++; + + bcp = sp->slab_head; + sp->slab_head = bcp->bc_next; + + if (cp->cache_flags & KMF_HASH) { + /* + * Add buffer to allocated-address hash table. + */ + buf = bcp->bc_addr; + hash_bucket = KMEM_HASH(cp, buf); + bcp->bc_next = *hash_bucket; + *hash_bucket = bcp; + if ((cp->cache_flags & (KMF_AUDIT | KMF_BUFTAG)) == KMF_AUDIT) { + KMEM_AUDIT(kmem_transaction_log, cp, bcp); + } + } else { + buf = KMEM_BUF(cp, bcp); + } + + ASSERT(KMEM_SLAB_MEMBER(sp, buf)); + + if (sp->slab_head == NULL) { + ASSERT(KMEM_SLAB_IS_ALL_USED(sp)); + if (new_slab) { + ASSERT(sp->slab_chunks == 1); + } else { + ASSERT(sp->slab_chunks > 1); /* the slab was partial */ + avl_remove(&cp->cache_partial_slabs, sp); + sp->slab_later_count = 0; /* clear history */ + sp->slab_flags &= ~KMEM_SLAB_NOMOVE; + sp->slab_stuck_offset = (uint32_t)-1; + } + list_insert_head(&cp->cache_complete_slabs, sp); + cp->cache_complete_slab_count++; + return (buf); + } + + ASSERT(KMEM_SLAB_IS_PARTIAL(sp)); + /* + * Peek to see if the magazine layer is enabled before + * we prefill. We're not holding the cpu cache lock, + * so the peek could be wrong, but there's no harm in it. + */ + if (new_slab && prefill && (cp->cache_flags & KMF_PREFILL) && + (KMEM_CPU_CACHE(cp)->cc_magsize != 0)) { + kmem_slab_prefill(cp, sp); + return (buf); + } + + if (new_slab) { + avl_add(&cp->cache_partial_slabs, sp); + return (buf); + } + + /* + * The slab is now more allocated than it was, so the + * order remains unchanged. + */ + ASSERT(!avl_update(&cp->cache_partial_slabs, sp)); + return (buf); +} + +/* + * Allocate a raw (unconstructed) buffer from cp's slab layer. + */ +__attribute__((noinline)) +static void * +kmem_slab_alloc(kmem_cache_t *cp, int kmflag) +{ + kmem_slab_t *sp; + void *buf; + boolean_t test_destructor; + + mutex_enter(&cp->cache_lock); + test_destructor = (cp->cache_slab_alloc == 0); + sp = avl_first(&cp->cache_partial_slabs); + if (sp == NULL) { + ASSERT(cp->cache_bufslab == 0); + + /* + * The freelist is empty. Create a new slab. + */ + mutex_exit(&cp->cache_lock); + if ((sp = kmem_slab_create(cp, kmflag)) == NULL) { + return (NULL); + } + mutex_enter(&cp->cache_lock); + cp->cache_slab_create++; + if ((cp->cache_buftotal += sp->slab_chunks) > cp->cache_bufmax) + cp->cache_bufmax = cp->cache_buftotal; + cp->cache_bufslab += sp->slab_chunks; + } + + buf = kmem_slab_alloc_impl(cp, sp, B_TRUE); + ASSERT((cp->cache_slab_create - cp->cache_slab_destroy) == + (cp->cache_complete_slab_count + + avl_numnodes(&cp->cache_partial_slabs) + + (cp->cache_defrag == NULL ? 0 : cp->cache_defrag->kmd_deadcount))); + mutex_exit(&cp->cache_lock); + + if (test_destructor && cp->cache_destructor != NULL) { + copy_pattern(KMEM_UNINITIALIZED_PATTERN, buf, + cp->cache_bufsize); + if (cp->cache_flags & KMF_DEADBEEF) { + copy_pattern(KMEM_FREE_PATTERN, buf, cp->cache_verify); + } + } + + return (buf); +} + +static void kmem_slab_move_yes(kmem_cache_t *, kmem_slab_t *, void *); + +/* + * Free a raw (unconstructed) buffer to cp's slab layer. + */ +static void +kmem_slab_free(kmem_cache_t *cp, void *buf) +{ + kmem_slab_t *sp; + kmem_bufctl_t *bcp, **prev_bcpp; + + ASSERT(buf != NULL); + + mutex_enter(&cp->cache_lock); + cp->cache_slab_free++; + + if (cp->cache_flags & KMF_HASH) { + /* + * Look up buffer in allocated-address hash table. + */ + prev_bcpp = KMEM_HASH(cp, buf); + while ((bcp = *prev_bcpp) != NULL) { + if (bcp->bc_addr == buf) { + *prev_bcpp = bcp->bc_next; + sp = bcp->bc_slab; + break; + } + cp->cache_lookup_depth++; + prev_bcpp = &bcp->bc_next; + } + } else { + bcp = KMEM_BUFCTL(cp, buf); + sp = KMEM_SLAB(cp, buf); + } + + if (bcp == NULL || sp->slab_cache != cp || !KMEM_SLAB_MEMBER(sp, buf)) { + mutex_exit(&cp->cache_lock); + kmem_error(KMERR_BADADDR, cp, buf); + return; + } + + if (KMEM_SLAB_OFFSET(sp, buf) == sp->slab_stuck_offset) { + /* + * If this is the buffer that prevented the consolidator from + * clearing the slab, we can reset the slab flags now that the + * buffer is freed. (It makes sense to do this in + * kmem_cache_free(), where the client gives up ownership of the + * buffer, but on the hot path the test is too expensive.) + */ + kmem_slab_move_yes(cp, sp, buf); + } + + if ((cp->cache_flags & (KMF_AUDIT | KMF_BUFTAG)) == KMF_AUDIT) { + if (cp->cache_flags & KMF_CONTENTS) + ((kmem_bufctl_audit_t *)bcp)->bc_contents = + kmem_log_enter(kmem_content_log, buf, + cp->cache_contents); + KMEM_AUDIT(kmem_transaction_log, cp, bcp); + } + + bcp->bc_next = sp->slab_head; + sp->slab_head = bcp; + + cp->cache_bufslab++; + ASSERT(sp->slab_refcnt >= 1); + + if (--sp->slab_refcnt == 0) { + /* + * There are no outstanding allocations from this slab, + * so we can reclaim the memory. + */ + if (sp->slab_chunks == 1) { + list_remove(&cp->cache_complete_slabs, sp); + cp->cache_complete_slab_count--; + } else { + avl_remove(&cp->cache_partial_slabs, sp); + } + + cp->cache_buftotal -= sp->slab_chunks; + cp->cache_bufslab -= sp->slab_chunks; + /* + * Defer releasing the slab to the virtual memory subsystem + * while there is a pending move callback, since we guarantee + * that buffers passed to the move callback have only been + * touched by kmem or by the client itself. Since the memory + * patterns baddcafe (uninitialized) and deadbeef (freed) both + * set at least one of the two lowest order bits, the client can + * test those bits in the move callback to determine whether or + * not it knows about the buffer (assuming that the client also + * sets one of those low order bits whenever it frees a buffer). + */ + if (cp->cache_defrag == NULL || + (avl_is_empty(&cp->cache_defrag->kmd_moves_pending) && + !(sp->slab_flags & KMEM_SLAB_MOVE_PENDING))) { + cp->cache_slab_destroy++; + mutex_exit(&cp->cache_lock); + kmem_slab_destroy(cp, sp); + } else { + list_t *deadlist = + &cp->cache_defrag->kmd_deadlist; + /* + * Slabs are inserted at both ends of the + * deadlist to distinguish between slabs + * freed while move callbacks are pending + * (list head) and a slab freed while the + * lock is dropped in kmem_move_buffers() + * (list tail) so that in both cases + * slab_destroy() is called from the + * right context. + */ + if (sp->slab_flags & KMEM_SLAB_MOVE_PENDING) { + list_insert_tail(deadlist, sp); + } else { + list_insert_head(deadlist, sp); + } + cp->cache_defrag->kmd_deadcount++; + mutex_exit(&cp->cache_lock); + } + return; + } + + if (bcp->bc_next == NULL) { + /* Transition the slab from completely allocated to partial. */ + ASSERT(sp->slab_refcnt == (sp->slab_chunks - 1)); + ASSERT(sp->slab_chunks > 1); + list_remove(&cp->cache_complete_slabs, sp); + cp->cache_complete_slab_count--; + avl_add(&cp->cache_partial_slabs, sp); + } else { + (void) avl_update_gt(&cp->cache_partial_slabs, sp); + } + + ASSERT((cp->cache_slab_create - cp->cache_slab_destroy) == + (cp->cache_complete_slab_count + + avl_numnodes(&cp->cache_partial_slabs) + + (cp->cache_defrag == NULL ? 0 : cp->cache_defrag->kmd_deadcount))); + mutex_exit(&cp->cache_lock); +} + +/* + * Return -1 if kmem_error, 1 if constructor fails, 0 if successful. + */ +static int +kmem_cache_alloc_debug(kmem_cache_t *cp, void *buf, int kmflag, int construct, + caddr_t caller) +{ + kmem_buftag_t *btp = KMEM_BUFTAG(cp, buf); + kmem_bufctl_audit_t *bcp = (kmem_bufctl_audit_t *)btp->bt_bufctl; + uint32_t mtbf; + + if (btp->bt_bxstat != ((intptr_t)bcp ^ KMEM_BUFTAG_FREE)) { + kmem_error(KMERR_BADBUFTAG, cp, buf); + return (-1); + } + + btp->bt_bxstat = (intptr_t)bcp ^ KMEM_BUFTAG_ALLOC; + + if ((cp->cache_flags & KMF_HASH) && bcp->bc_addr != buf) { + kmem_error(KMERR_BADBUFCTL, cp, buf); + return (-1); + } + + if (cp->cache_flags & KMF_DEADBEEF) { + if (!construct && (cp->cache_flags & KMF_LITE)) { + if (*(uint64_t *)buf != KMEM_FREE_PATTERN) { + kmem_error(KMERR_MODIFIED, cp, buf); + return (-1); + } + if (cp->cache_constructor != NULL) + *(uint64_t *)buf = btp->bt_redzone; + else + *(uint64_t *)buf = KMEM_UNINITIALIZED_PATTERN; + } else { + construct = 1; + if (verify_and_copy_pattern(KMEM_FREE_PATTERN, + KMEM_UNINITIALIZED_PATTERN, buf, + cp->cache_verify)) { + kmem_error(KMERR_MODIFIED, cp, buf); + return (-1); + } + } + } + btp->bt_redzone = KMEM_REDZONE_PATTERN; + + if ((mtbf = kmem_mtbf | cp->cache_mtbf) != 0 && + gethrtime() % mtbf == 0 && + (kmflag & (KM_NOSLEEP | KM_PANIC)) == KM_NOSLEEP) { + kmem_log_event(kmem_failure_log, cp, NULL, NULL); + if (!construct && cp->cache_destructor != NULL) + cp->cache_destructor(buf, cp->cache_private); + } else { + mtbf = 0; + } + + if (mtbf || (construct && cp->cache_constructor != NULL && + cp->cache_constructor(buf, cp->cache_private, kmflag) != 0)) { + atomic_inc_64(&cp->cache_alloc_fail); + btp->bt_bxstat = (intptr_t)bcp ^ KMEM_BUFTAG_FREE; + if (cp->cache_flags & KMF_DEADBEEF) + copy_pattern(KMEM_FREE_PATTERN, buf, cp->cache_verify); + kmem_slab_free(cp, buf); + return (1); + } + + if (cp->cache_flags & KMF_AUDIT) { + KMEM_AUDIT(kmem_transaction_log, cp, bcp); + } + + if ((cp->cache_flags & KMF_LITE) && + !(cp->cache_cflags & KMC_KMEM_ALLOC)) { + KMEM_BUFTAG_LITE_ENTER(btp, kmem_lite_count, caller); + } + + return (0); +} + +static int +kmem_cache_free_debug(kmem_cache_t *cp, void *buf, caddr_t caller) +{ + kmem_buftag_t *btp = KMEM_BUFTAG(cp, buf); + kmem_bufctl_audit_t *bcp = (kmem_bufctl_audit_t *)btp->bt_bufctl; + kmem_slab_t *sp; + + if (btp->bt_bxstat != ((intptr_t)bcp ^ KMEM_BUFTAG_ALLOC)) { + if (btp->bt_bxstat == ((intptr_t)bcp ^ KMEM_BUFTAG_FREE)) { + kmem_error(KMERR_DUPFREE, cp, buf); + return (-1); + } + sp = kmem_findslab(cp, buf); + if (sp == NULL || sp->slab_cache != cp) + kmem_error(KMERR_BADADDR, cp, buf); + else + kmem_error(KMERR_REDZONE, cp, buf); + return (-1); + } + + btp->bt_bxstat = (intptr_t)bcp ^ KMEM_BUFTAG_FREE; + + if ((cp->cache_flags & KMF_HASH) && bcp->bc_addr != buf) { + kmem_error(KMERR_BADBUFCTL, cp, buf); + return (-1); + } + + if (btp->bt_redzone != KMEM_REDZONE_PATTERN) { + kmem_error(KMERR_REDZONE, cp, buf); + return (-1); + } + + if (cp->cache_flags & KMF_AUDIT) { + if (cp->cache_flags & KMF_CONTENTS) + bcp->bc_contents = kmem_log_enter(kmem_content_log, + buf, cp->cache_contents); + KMEM_AUDIT(kmem_transaction_log, cp, bcp); + } + + if ((cp->cache_flags & KMF_LITE) && + !(cp->cache_cflags & KMC_KMEM_ALLOC)) { + KMEM_BUFTAG_LITE_ENTER(btp, kmem_lite_count, caller); + } + + if (cp->cache_flags & KMF_DEADBEEF) { + if (cp->cache_flags & KMF_LITE) + btp->bt_redzone = *(uint64_t *)buf; + else if (cp->cache_destructor != NULL) + cp->cache_destructor(buf, cp->cache_private); + + copy_pattern(KMEM_FREE_PATTERN, buf, cp->cache_verify); + } + + return (0); +} + +/* + * Free each object in magazine mp to cp's slab layer, and free mp itself. + */ +static void +kmem_magazine_destroy(kmem_cache_t *cp, kmem_magazine_t *mp, int nrounds) +{ + int round; + + ASSERT(!list_link_active(&cp->cache_link) || + taskq_member(kmem_taskq, curthread)); + + for (round = 0; round < nrounds; round++) { + void *buf = mp->mag_round[round]; + + if (cp->cache_flags & KMF_DEADBEEF) { + if (verify_pattern(KMEM_FREE_PATTERN, buf, + cp->cache_verify) != NULL) { + kmem_error(KMERR_MODIFIED, cp, buf); + continue; + } + if ((cp->cache_flags & KMF_LITE) && + cp->cache_destructor != NULL) { + kmem_buftag_t *btp = KMEM_BUFTAG(cp, buf); + *(uint64_t *)buf = btp->bt_redzone; + cp->cache_destructor(buf, cp->cache_private); + *(uint64_t *)buf = KMEM_FREE_PATTERN; + } + } else if (cp->cache_destructor != NULL) { + cp->cache_destructor(buf, cp->cache_private); + } + + kmem_slab_free(cp, buf); + kpreempt(KPREEMPT_SYNC); + } + ASSERT(KMEM_MAGAZINE_VALID(cp, mp)); + kmem_cache_free(cp->cache_magtype->mt_cache, mp); +} + +/* + * Allocate a magazine from the depot. + */ +static kmem_magazine_t * +kmem_depot_alloc(kmem_cache_t *cp, kmem_maglist_t *mlp) +{ + kmem_magazine_t *mp; + + /* + * If we can't get the depot lock without contention, + * update our contention count. We use the depot + * contention rate to determine whether we need to + * increase the magazine size for better scalability. + */ + if (!mutex_tryenter(&cp->cache_depot_lock)) { + mutex_enter(&cp->cache_depot_lock); + cp->cache_depot_contention++; + } + + if ((mp = mlp->ml_list) != NULL) { + ASSERT(KMEM_MAGAZINE_VALID(cp, mp)); + mlp->ml_list = mp->mag_next; + if (--mlp->ml_total < mlp->ml_min) + mlp->ml_min = mlp->ml_total; + mlp->ml_alloc++; + } + + mutex_exit(&cp->cache_depot_lock); + + return (mp); +} + +/* + * Free a magazine to the depot. + */ +static void +kmem_depot_free(kmem_cache_t *cp, kmem_maglist_t *mlp, kmem_magazine_t *mp) +{ + mutex_enter(&cp->cache_depot_lock); + ASSERT(KMEM_MAGAZINE_VALID(cp, mp)); + mp->mag_next = mlp->ml_list; + mlp->ml_list = mp; + mlp->ml_total++; + mutex_exit(&cp->cache_depot_lock); +} + +/* + * Update the working set statistics for cp's depot. + */ +static void +kmem_depot_ws_update(kmem_cache_t *cp) +{ + mutex_enter(&cp->cache_depot_lock); + cp->cache_full.ml_reaplimit = cp->cache_full.ml_min; + cp->cache_full.ml_min = cp->cache_full.ml_total; + cp->cache_empty.ml_reaplimit = cp->cache_empty.ml_min; + cp->cache_empty.ml_min = cp->cache_empty.ml_total; + mutex_exit(&cp->cache_depot_lock); +} + +/* + * Set the working set statistics for cp's depot to zero. (Everything is + * eligible for reaping.) + */ +void +kmem_depot_ws_zero(kmem_cache_t *cp) +{ + mutex_enter(&cp->cache_depot_lock); + cp->cache_full.ml_reaplimit = cp->cache_full.ml_total; + cp->cache_full.ml_min = cp->cache_full.ml_total; + cp->cache_empty.ml_reaplimit = cp->cache_empty.ml_total; + cp->cache_empty.ml_min = cp->cache_empty.ml_total; + mutex_exit(&cp->cache_depot_lock); +} + +/* + * The number of bytes to reap before we call kpreempt(). The default (1MB) + * causes us to preempt reaping up to hundres of times per second. Using a + * larger value (1GB) causes this to have virtually no effect. + */ +size_t kmem_reap_preempt_bytes = 64 * 1024 * 1024; + + +/* + * Reap all magazines that have fallen out of the depot's working set. + */ +static void +kmem_depot_ws_reap(kmem_cache_t *cp) +{ + size_t bytes = 0; + long reap; + kmem_magazine_t *mp; + + ASSERT(!list_link_active(&cp->cache_link) || + taskq_member(kmem_taskq, curthread)); + + reap = MIN(cp->cache_full.ml_reaplimit, cp->cache_full.ml_min); + while (reap-- && + (mp = kmem_depot_alloc(cp, &cp->cache_full)) != NULL) { + kmem_magazine_destroy(cp, mp, cp->cache_magtype->mt_magsize); + bytes += cp->cache_magtype->mt_magsize * cp->cache_bufsize; + if (bytes > kmem_reap_preempt_bytes) { + kpreempt(KPREEMPT_SYNC); + bytes = 0; + } + } + + reap = MIN(cp->cache_empty.ml_reaplimit, cp->cache_empty.ml_min); + while (reap-- && + (mp = kmem_depot_alloc(cp, &cp->cache_empty)) != NULL) { + kmem_magazine_destroy(cp, mp, 0); + bytes += cp->cache_magtype->mt_magsize * cp->cache_bufsize; + if (bytes > kmem_reap_preempt_bytes) { + kpreempt(KPREEMPT_SYNC); + bytes = 0; + } + } +} + +static void +kmem_cpu_reload(kmem_cpu_cache_t *ccp, kmem_magazine_t *mp, int rounds) +{ + ASSERT((ccp->cc_loaded == NULL && ccp->cc_rounds == -1) || + (ccp->cc_loaded && ccp->cc_rounds + rounds == ccp->cc_magsize)); + ASSERT(ccp->cc_magsize > 0); + + ccp->cc_ploaded = ccp->cc_loaded; + ccp->cc_prounds = ccp->cc_rounds; + ccp->cc_loaded = mp; + ccp->cc_rounds = rounds; +} + +/* + * Intercept kmem alloc/free calls during crash dump in order to avoid + * changing kmem state while memory is being saved to the dump device. + * Otherwise, ::kmem_verify will report "corrupt buffers". Note that + * there are no locks because only one CPU calls kmem during a crash + * dump. To enable this feature, first create the associated vmem + * arena with VMC_DUMPSAFE. + */ +static void *kmem_dump_start; /* start of pre-reserved heap */ +static void *kmem_dump_end; /* end of heap area */ +static void *kmem_dump_curr; /* current free heap pointer */ +static size_t kmem_dump_size; /* size of heap area */ + +/* append to each buf created in the pre-reserved heap */ +typedef struct kmem_dumpctl { + void *kdc_next; /* cache dump free list linkage */ +} kmem_dumpctl_t; + +#define KMEM_DUMPCTL(cp, buf) \ +((kmem_dumpctl_t *)P2ROUNDUP((uintptr_t)(buf) + (cp)->cache_bufsize, \ +sizeof (void *))) + +/* Keep some simple stats. */ +#define KMEM_DUMP_LOGS (100) + +typedef struct kmem_dump_log { + kmem_cache_t *kdl_cache; + uint_t kdl_allocs; /* # of dump allocations */ + uint_t kdl_frees; /* # of dump frees */ + uint_t kdl_alloc_fails; /* # of allocation failures */ + uint_t kdl_free_nondump; /* # of non-dump frees */ + uint_t kdl_unsafe; /* cache was used, but unsafe */ +} kmem_dump_log_t; + +static kmem_dump_log_t *kmem_dump_log; +static int kmem_dump_log_idx; + +#define KDI_LOG(cp, stat) { \ +kmem_dump_log_t *kdl; \ +if ((kdl = (kmem_dump_log_t *)((cp)->cache_dumplog)) != NULL) { \ +kdl->stat++; \ +} else if (kmem_dump_log_idx < KMEM_DUMP_LOGS) { \ +kdl = &kmem_dump_log[kmem_dump_log_idx++]; \ +kdl->stat++; \ +kdl->kdl_cache = (cp); \ +(cp)->cache_dumplog = kdl; \ +} \ +} + +/* set non zero for full report */ +uint_t kmem_dump_verbose = 0; + +/* stats for overize heap */ +uint_t kmem_dump_oversize_allocs = 0; +uint_t kmem_dump_oversize_max = 0; + +static void +kmem_dumppr(char **pp, char *e, const char *format, ...) +{ + char *p = *pp; + + if (p < e) { + int n; + va_list ap; + + va_start(ap, format); + n = vsnprintf(p, e - p, format, ap); + va_end(ap); + *pp = p + n; + } +} + +/* + * Called when dumpadm(1M) configures dump parameters. + */ +void +kmem_dump_init(size_t size) +{ + if (kmem_dump_start != NULL) + zfs_kmem_free(kmem_dump_start, kmem_dump_size); + + if (kmem_dump_log == NULL) + kmem_dump_log = + (kmem_dump_log_t *)zfs_kmem_zalloc( + KMEM_DUMP_LOGS * sizeof (kmem_dump_log_t), KM_SLEEP); + + kmem_dump_start = zfs_kmem_alloc(size, KM_SLEEP); + + if (kmem_dump_start != NULL) { + kmem_dump_size = size; + kmem_dump_curr = kmem_dump_start; + kmem_dump_end = (void *)((char *)kmem_dump_start + size); + copy_pattern(KMEM_UNINITIALIZED_PATTERN, kmem_dump_start, size); + } else { + kmem_dump_size = 0; + kmem_dump_curr = NULL; + kmem_dump_end = NULL; + } +} + +/* + * Set flag for each kmem_cache_t if is safe to use alternate dump + * memory. Called just before panic crash dump starts. Set the flag + * for the calling CPU. + */ +void +kmem_dump_begin(void) +{ + if (kmem_dump_start != NULL) { + kmem_cache_t *cp; + + for (cp = list_head(&kmem_caches); cp != NULL; + cp = list_next(&kmem_caches, cp)) { + kmem_cpu_cache_t *ccp = KMEM_CPU_CACHE(cp); + + if (cp->cache_arena->vm_cflags & VMC_DUMPSAFE) { + cp->cache_flags |= KMF_DUMPDIVERT; + ccp->cc_flags |= KMF_DUMPDIVERT; + ccp->cc_dump_rounds = ccp->cc_rounds; + ccp->cc_dump_prounds = ccp->cc_prounds; + ccp->cc_rounds = ccp->cc_prounds = -1; + } else { + cp->cache_flags |= KMF_DUMPUNSAFE; + ccp->cc_flags |= KMF_DUMPUNSAFE; + } + } + } +} + +/* + * finished dump intercept + * print any warnings on the console + * return verbose information to dumpsys() in the given buffer + */ +size_t +kmem_dump_finish(char *buf, size_t size) +{ + int kdi_idx; + int kdi_end = kmem_dump_log_idx; + int percent = 0; + int header = 0; + int warn = 0; + size_t used; + kmem_cache_t *cp; + kmem_dump_log_t *kdl; + char *e = buf + size; + char *p = buf; + + if (kmem_dump_size == 0 || kmem_dump_verbose == 0) + return (0); + + used = (char *)kmem_dump_curr - (char *)kmem_dump_start; + percent = (used * 100) / kmem_dump_size; + + kmem_dumppr(&p, e, "%% heap used,%d\n", percent); + kmem_dumppr(&p, e, "used bytes,%ld\n", used); + kmem_dumppr(&p, e, "heap size,%ld\n", kmem_dump_size); + kmem_dumppr(&p, e, "Oversize allocs,%d\n", + kmem_dump_oversize_allocs); + kmem_dumppr(&p, e, "Oversize max size,%ld\n", + kmem_dump_oversize_max); + + for (kdi_idx = 0; kdi_idx < kdi_end; kdi_idx++) { + kdl = &kmem_dump_log[kdi_idx]; + cp = kdl->kdl_cache; + if (cp == NULL) + break; + if (kdl->kdl_alloc_fails) + ++warn; + if (header == 0) { + kmem_dumppr(&p, e, + "Cache Name,Allocs,Frees,Alloc Fails," + "Nondump Frees,Unsafe Allocs/Frees\n"); + header = 1; + } + kmem_dumppr(&p, e, "%s,%d,%d,%d,%d,%d\n", + cp->cache_name, kdl->kdl_allocs, kdl->kdl_frees, + kdl->kdl_alloc_fails, kdl->kdl_free_nondump, + kdl->kdl_unsafe); + } + + /* return buffer size used */ + if (p < e) + memset(p, 0, e - p); + return (p - buf); +} + +/* + * Allocate a constructed object from alternate dump memory. + */ +void * +kmem_cache_alloc_dump(kmem_cache_t *cp, int kmflag) +{ + void *buf; + void *curr; + char *bufend; + + /* return a constructed object */ + if ((buf = cp->cache_dumpfreelist) != NULL) { + cp->cache_dumpfreelist = KMEM_DUMPCTL(cp, buf)->kdc_next; + KDI_LOG(cp, kdl_allocs); + return (buf); + } + + /* create a new constructed object */ + curr = kmem_dump_curr; + buf = (void *)P2ROUNDUP((uintptr_t)curr, cp->cache_align); + bufend = (char *)KMEM_DUMPCTL(cp, buf) + sizeof (kmem_dumpctl_t); + + /* hat layer objects cannot cross a page boundary */ + if (cp->cache_align < PAGESIZE) { + char *page = (char *)P2ROUNDUP((uintptr_t)buf, PAGESIZE); + if (bufend > page) { + bufend += page - (char *)buf; + buf = (void *)page; + } + } + + /* fall back to normal alloc if reserved area is used up */ + if (bufend > (char *)kmem_dump_end) { + kmem_dump_curr = kmem_dump_end; + KDI_LOG(cp, kdl_alloc_fails); + return (NULL); + } + + /* + * Must advance curr pointer before calling a constructor that + * may also allocate memory. + */ + kmem_dump_curr = bufend; + + /* run constructor */ + if (cp->cache_constructor != NULL && + cp->cache_constructor(buf, cp->cache_private, kmflag) + != 0) { +#ifdef DEBUG + dprintf("name='%s' cache=0x%p: kmem cache constructor failed\n", + cp->cache_name, (void *)cp); +#endif + /* reset curr pointer iff no allocs were done */ + if (kmem_dump_curr == bufend) + kmem_dump_curr = curr; + + /* fall back to normal alloc if the constructor fails */ + KDI_LOG(cp, kdl_alloc_fails); + return (NULL); + } + + KDI_LOG(cp, kdl_allocs); + return (buf); +} + +/* + * Free a constructed object in alternate dump memory. + */ +int +kmem_cache_free_dump(kmem_cache_t *cp, void *buf) +{ + /* save constructed buffers for next time */ + if ((char *)buf >= (char *)kmem_dump_start && + (char *)buf < (char *)kmem_dump_end) { + KMEM_DUMPCTL(cp, buf)->kdc_next = cp->cache_dumpfreelist; + cp->cache_dumpfreelist = buf; + KDI_LOG(cp, kdl_frees); + return (0); + } + + /* count all non-dump buf frees */ + KDI_LOG(cp, kdl_free_nondump); + + /* just drop buffers that were allocated before dump started */ + if (kmem_dump_curr < kmem_dump_end) + return (0); + + /* fall back to normal free if reserved area is used up */ + return (1); +} + +/* + * Allocate a constructed object from cache cp. + */ +void * +kmem_cache_alloc(kmem_cache_t *cp, int kmflag) +{ + kmem_cpu_cache_t *ccp = KMEM_CPU_CACHE(cp); + kmem_magazine_t *fmp; + void *buf; + mutex_enter(&ccp->cc_lock); + for (;;) { + /* + * If there's an object available in the current CPU's + * loaded magazine, just take it and return. + */ + if (ccp->cc_rounds > 0) { + buf = ccp->cc_loaded->mag_round[--ccp->cc_rounds]; + ccp->cc_alloc++; + mutex_exit(&ccp->cc_lock); + if (ccp->cc_flags & (KMF_BUFTAG | KMF_DUMPUNSAFE)) { + if (ccp->cc_flags & KMF_DUMPUNSAFE) { + ASSERT(!(ccp->cc_flags & + KMF_DUMPDIVERT)); + KDI_LOG(cp, kdl_unsafe); + } + if ((ccp->cc_flags & KMF_BUFTAG) && + kmem_cache_alloc_debug(cp, buf, kmflag, 0, + caller()) != 0) { + if (kmflag & KM_NOSLEEP) + return (NULL); + mutex_enter(&ccp->cc_lock); + continue; + } + } + return (buf); + } + + /* + * The loaded magazine is empty. If the previously loaded + * magazine was full, exchange them and try again. + */ + if (ccp->cc_prounds > 0) { + kmem_cpu_reload(ccp, ccp->cc_ploaded, ccp->cc_prounds); + continue; + } + + /* + * Return an alternate buffer at dump time to preserve + * the heap. + */ + if (ccp->cc_flags & (KMF_DUMPDIVERT | KMF_DUMPUNSAFE)) { + if (ccp->cc_flags & KMF_DUMPUNSAFE) { + ASSERT(!(ccp->cc_flags & KMF_DUMPDIVERT)); + /* log it so that we can warn about it */ + KDI_LOG(cp, kdl_unsafe); + } else { + if ((buf = kmem_cache_alloc_dump(cp, kmflag)) != + NULL) { + mutex_exit(&ccp->cc_lock); + return (buf); + } + break; /* fall back to slab layer */ + } + } + + /* + * If the magazine layer is disabled, break out now. + */ + if (ccp->cc_magsize == 0) + break; + + /* + * Try to get a full magazine from the depot. + */ + fmp = kmem_depot_alloc(cp, &cp->cache_full); + if (fmp != NULL) { + if (ccp->cc_ploaded != NULL) + kmem_depot_free(cp, &cp->cache_empty, + ccp->cc_ploaded); + kmem_cpu_reload(ccp, fmp, ccp->cc_magsize); + continue; + } + + /* + * There are no full magazines in the depot, + * so fall through to the slab layer. + */ + break; + } + mutex_exit(&ccp->cc_lock); + + /* + * We couldn't allocate a constructed object from the magazine layer, + * so get a raw buffer from the slab layer and apply its constructor. + */ + buf = kmem_slab_alloc(cp, kmflag); + + if (buf == NULL) + return (NULL); + + if (cp->cache_flags & KMF_BUFTAG) { + /* + * Make kmem_cache_alloc_debug() apply the constructor for us. + */ + int rc = kmem_cache_alloc_debug(cp, buf, kmflag, 1, caller()); + if (rc != 0) { + if (kmflag & KM_NOSLEEP) + return (NULL); + /* + * kmem_cache_alloc_debug() detected corruption + * but didn't panic (kmem_panic <= 0). We should not be + * here because the constructor failed (indicated by a + * return code of 1). Try again. + */ + ASSERT(rc == -1); + return (kmem_cache_alloc(cp, kmflag)); + } + return (buf); + } + + if (cp->cache_constructor != NULL && + cp->cache_constructor(buf, cp->cache_private, kmflag) != 0) { + atomic_inc_64(&cp->cache_alloc_fail); + kmem_slab_free(cp, buf); + return (NULL); + } + + return (buf); +} + +/* + * The freed argument tells whether or not kmem_cache_free_debug() has already + * been called so that we can avoid the duplicate free error. For example, a + * buffer on a magazine has already been freed by the client but is still + * constructed. + */ +static void +kmem_slab_free_constructed(kmem_cache_t *cp, void *buf, boolean_t freed) +{ + if (!freed && (cp->cache_flags & KMF_BUFTAG)) + if (kmem_cache_free_debug(cp, buf, caller()) == -1) + return; + + /* + * Note that if KMF_DEADBEEF is in effect and KMF_LITE is not, + * kmem_cache_free_debug() will have already applied the destructor. + */ + if ((cp->cache_flags & (KMF_DEADBEEF | KMF_LITE)) != KMF_DEADBEEF && + cp->cache_destructor != NULL) { + if (cp->cache_flags & KMF_DEADBEEF) { /* KMF_LITE implied */ + kmem_buftag_t *btp = KMEM_BUFTAG(cp, buf); + *(uint64_t *)buf = btp->bt_redzone; + cp->cache_destructor(buf, cp->cache_private); + *(uint64_t *)buf = KMEM_FREE_PATTERN; + } else { + cp->cache_destructor(buf, cp->cache_private); + } + } + + kmem_slab_free(cp, buf); +} + +/* + * Used when there's no room to free a buffer to the per-CPU cache. + * Drops and re-acquires &ccp->cc_lock, and returns non-zero if the + * caller should try freeing to the per-CPU cache again. + * Note that we don't directly install the magazine in the cpu cache, + * since its state may have changed wildly while the lock was dropped. + */ +static int +kmem_cpucache_magazine_alloc(kmem_cpu_cache_t *ccp, kmem_cache_t *cp) +{ + kmem_magazine_t *emp; + kmem_magtype_t *mtp; + + ASSERT(MUTEX_HELD(&ccp->cc_lock)); + ASSERT(((uint_t)ccp->cc_rounds == ccp->cc_magsize || + ((uint_t)ccp->cc_rounds == -1)) && + ((uint_t)ccp->cc_prounds == ccp->cc_magsize || + ((uint_t)ccp->cc_prounds == -1))); + + emp = kmem_depot_alloc(cp, &cp->cache_empty); + if (emp != NULL) { + if (ccp->cc_ploaded != NULL) + kmem_depot_free(cp, &cp->cache_full, + ccp->cc_ploaded); + kmem_cpu_reload(ccp, emp, 0); + return (1); + } + /* + * There are no empty magazines in the depot, + * so try to allocate a new one. We must drop all locks + * across kmem_cache_alloc() because lower layers may + * attempt to allocate from this cache. + */ + mtp = cp->cache_magtype; + mutex_exit(&ccp->cc_lock); + emp = kmem_cache_alloc(mtp->mt_cache, KM_NOSLEEP); + mutex_enter(&ccp->cc_lock); + + if (emp != NULL) { + /* + * We successfully allocated an empty magazine. + * However, we had to drop ccp->cc_lock to do it, + * so the cache's magazine size may have changed. + * If so, free the magazine and try again. + */ + if (ccp->cc_magsize != mtp->mt_magsize) { + mutex_exit(&ccp->cc_lock); + kmem_cache_free(mtp->mt_cache, emp); + mutex_enter(&ccp->cc_lock); + return (1); + } + + /* + * We got a magazine of the right size. Add it to + * the depot and try the whole dance again. + */ + kmem_depot_free(cp, &cp->cache_empty, emp); + return (1); + } + + /* + * We couldn't allocate an empty magazine, + * so fall through to the slab layer. + */ + return (0); +} + +/* + * If the cache's parent arena is a leaf arena (i.e., it imports all its memory) + * then we can consider it fragmented if either there is 1 GiB free in the arena + * or one eighth of the arena is free. + * + * This is useful in kmem_cache_free{_debug} to determine whether to free to the + * slab layer if the loaded magazine is full. + */ +static inline boolean_t +kmem_cache_parent_arena_fragmented(kmem_cache_t *cp) +{ + const vmem_kstat_t *kp = &cp->cache_arena->vm_kstat; + const int64_t vk_import = kp->vk_mem_import.value.ui64; + const int64_t vk_inuse = kp->vk_mem_inuse.value.ui64; + const int64_t vk_total = kp->vk_mem_total.value.ui64; + + if (vk_import == vk_total && vk_inuse < vk_total) { + const int64_t vk_free = vk_total - vk_inuse; + const int64_t highthresh = 1024LL*1024LL*1024LL; + // we are fragmented if we have 1GiB free + if (vk_free >= highthresh) + return (B_TRUE); + // we are fragmented if at least 1/8 of the + // total arena space is free + if (vk_free > 0 && vk_total > 0) { + const int64_t eighth_total = vk_total / 8; + if (vk_free >= eighth_total) + return (B_TRUE); + } + } + return (B_FALSE); +} + +/* + * Free a constructed object to cache cp. + */ +void +kmem_cache_free(kmem_cache_t *cp, void *buf) +{ + kmem_cpu_cache_t *ccp = KMEM_CPU_CACHE(cp); + + /* + * The client must not free either of the buffers passed to the move + * callback function. + */ + ASSERT(cp->cache_defrag == NULL || + cp->cache_defrag->kmd_thread != spl_current_thread() || + (buf != cp->cache_defrag->kmd_from_buf && + buf != cp->cache_defrag->kmd_to_buf)); + + if (ccp->cc_flags & (KMF_BUFTAG | KMF_DUMPDIVERT | KMF_DUMPUNSAFE)) { + if (ccp->cc_flags & KMF_DUMPUNSAFE) { + ASSERT(!(ccp->cc_flags & KMF_DUMPDIVERT)); + /* log it so that we can warn about it */ + KDI_LOG(cp, kdl_unsafe); + } else if (KMEM_DUMPCC(ccp) && !kmem_cache_free_dump(cp, buf)) { + return; + } + if (ccp->cc_flags & KMF_BUFTAG) { + if (kmem_cache_free_debug(cp, buf, caller()) == -1) + return; + } + } + + mutex_enter(&ccp->cc_lock); + /* + * Any changes to this logic should be reflected in kmem_slab_prefill() + */ + for (;;) { + /* + * If there's a slot available in the current CPU's + * loaded magazine, just put the object there and return. + */ + if ((uint_t)ccp->cc_rounds < ccp->cc_magsize) { + ccp->cc_loaded->mag_round[ccp->cc_rounds++] = buf; + ccp->cc_free++; + mutex_exit(&ccp->cc_lock); + return; + } + + /* + * If the magazine layer is disabled, break out now. + */ + if (ccp->cc_magsize == 0) { + break; + } + + /* + * The magazine layer is on, but the loaded magazine is now + * full (of allocatable constructed elements). + * + * If the cache's arena is badly fragmented, break out now; + * this frees to the slab layer. + * + * Note: this is not reflected in kmem_slab_prefill() which + * deals with a freshly allocated slab. + */ + + if (kmem_free_to_slab_when_fragmented == 1 && + kmem_cache_parent_arena_fragmented(cp)) + break; + + /* + * The loaded magazine is full. If the previously loaded + * magazine was empty, exchange them and try again. + */ + if (ccp->cc_prounds == 0) { + kmem_cpu_reload(ccp, ccp->cc_ploaded, ccp->cc_prounds); + continue; + } + + if (!kmem_cpucache_magazine_alloc(ccp, cp)) { + /* + * We couldn't free our constructed object to the + * magazine layer, so apply its destructor and free it + * to the slab layer. + */ + break; + } + } + mutex_exit(&ccp->cc_lock); + kpreempt(KPREEMPT_SYNC); + kmem_slab_free_constructed(cp, buf, B_TRUE); +} + +/* + * Free a constructed object to cache cp. + * Do not free to the magazine layer. + * This is essentially just kmem_cache_free() without + * the for(;;) loop or the ccp critical section. + */ +void +kmem_cache_free_to_slab(kmem_cache_t *cp, void *buf) +{ + kmem_cpu_cache_t *ccp = KMEM_CPU_CACHE(cp); + + /* + * The client must not free either of the buffers passed to the move + * callback function. + */ + ASSERT(cp->cache_defrag == NULL || + cp->cache_defrag->kmd_thread != spl_current_thread() || + (buf != cp->cache_defrag->kmd_from_buf && + buf != cp->cache_defrag->kmd_to_buf)); + + if (ccp->cc_flags & (KMF_BUFTAG | KMF_DUMPDIVERT | KMF_DUMPUNSAFE)) { + if (ccp->cc_flags & KMF_DUMPUNSAFE) { + ASSERT(!(ccp->cc_flags & KMF_DUMPDIVERT)); + /* log it so that we can warn about it */ + KDI_LOG(cp, kdl_unsafe); + } else if (KMEM_DUMPCC(ccp) && !kmem_cache_free_dump(cp, buf)) { + return; + } + if (ccp->cc_flags & KMF_BUFTAG) { + if (kmem_cache_free_debug(cp, buf, caller()) == -1) + return; + } + } + + /* omitted the for(;;) loop from kmem_cache_free */ + /* also do not take ccp mutex */ + + kmem_slab_free_constructed(cp, buf, B_TRUE); +} + +static void +kmem_slab_prefill(kmem_cache_t *cp, kmem_slab_t *sp) +{ + kmem_cpu_cache_t *ccp = KMEM_CPU_CACHE(cp); + + kmem_bufctl_t *next, *head; + size_t nbufs; + + /* + * Completely allocate the newly created slab and put the pre-allocated + * buffers in magazines. Any of the buffers that cannot be put in + * magazines must be returned to the slab. + */ + ASSERT(MUTEX_HELD(&cp->cache_lock)); + ASSERT(cp->cache_constructor == NULL); + ASSERT(sp->slab_cache == cp); + ASSERT(sp->slab_refcnt == 1); + ASSERT(sp->slab_head != NULL && sp->slab_chunks > sp->slab_refcnt); + ASSERT(avl_find(&cp->cache_partial_slabs, sp, NULL) == NULL); + + head = sp->slab_head; + nbufs = (sp->slab_chunks - sp->slab_refcnt); + sp->slab_head = NULL; + sp->slab_refcnt += nbufs; + cp->cache_bufslab -= nbufs; + cp->cache_slab_alloc += nbufs; + list_insert_head(&cp->cache_complete_slabs, sp); + cp->cache_complete_slab_count++; + mutex_exit(&cp->cache_lock); + mutex_enter(&ccp->cc_lock); + + while (head != NULL) { + void *buf = KMEM_BUF(cp, head); + /* + * If there's a slot available in the current CPU's + * loaded magazine, just put the object there and + * continue. + */ + if ((uint_t)ccp->cc_rounds < ccp->cc_magsize) { + ccp->cc_loaded->mag_round[ccp->cc_rounds++] = + buf; + ccp->cc_free++; + nbufs--; + head = head->bc_next; + continue; + } + + /* + * The loaded magazine is full. If the previously + * loaded magazine was empty, exchange them and try + * again. + */ + if (ccp->cc_prounds == 0) { + kmem_cpu_reload(ccp, ccp->cc_ploaded, + ccp->cc_prounds); + continue; + } + + /* + * If the magazine layer is disabled, break out now. + */ + + if (ccp->cc_magsize == 0) { + break; + } + + if (!kmem_cpucache_magazine_alloc(ccp, cp)) + break; + } + mutex_exit(&ccp->cc_lock); + if (nbufs != 0) { + ASSERT(head != NULL); + + /* + * If there was a failure, return remaining objects to + * the slab + */ + while (head != NULL) { + ASSERT(nbufs != 0); + next = head->bc_next; + head->bc_next = NULL; + kmem_slab_free(cp, KMEM_BUF(cp, head)); + head = next; + nbufs--; + } + } + ASSERT(head == NULL); + ASSERT(nbufs == 0); + mutex_enter(&cp->cache_lock); +} + +void * +zfs_kmem_zalloc(size_t size, int kmflag) +{ + size_t index; + void *buf; + + if ((index = ((size - 1) >> KMEM_ALIGN_SHIFT)) < KMEM_ALLOC_TABLE_MAX) { + kmem_cache_t *cp = kmem_alloc_table[index]; + buf = kmem_cache_alloc(cp, kmflag); + if (buf != NULL) { + if ((cp->cache_flags & KMF_BUFTAG) && !KMEM_DUMP(cp)) { + kmem_buftag_t *btp = KMEM_BUFTAG(cp, buf); + ((uint8_t *)buf)[size] = KMEM_REDZONE_BYTE; + ((uint32_t *)btp)[1] = KMEM_SIZE_ENCODE(size); + + if (cp->cache_flags & KMF_LITE) { + KMEM_BUFTAG_LITE_ENTER(btp, + kmem_lite_count, caller()); + } + } + memset(buf, 0, size); + } + } else { + buf = zfs_kmem_alloc(size, kmflag); + if (buf != NULL) + memset(buf, 0, size); + } + return (buf); +} + +void * +zfs_kmem_alloc(size_t size, int kmflag) +{ + size_t index; + kmem_cache_t *cp; + void *buf; + + if (size == 0) + return (KMEM_ZERO_SIZE_PTR); + + if ((index = ((size - 1) >> KMEM_ALIGN_SHIFT)) < KMEM_ALLOC_TABLE_MAX) { + cp = kmem_alloc_table[index]; + /* fall through to kmem_cache_alloc() */ + + } else if ((index = ((size - 1) >> KMEM_BIG_SHIFT)) < + kmem_big_alloc_table_max) { + cp = kmem_big_alloc_table[index]; + /* fall through to kmem_cache_alloc() */ + + } else { + + buf = vmem_alloc_impl(kmem_oversize_arena, size, + kmflag & KM_VMFLAGS); + if (buf == NULL) + kmem_log_event(kmem_failure_log, NULL, NULL, + (void *)size); + else if (KMEM_DUMP(kmem_slab_cache)) { + /* stats for dump intercept */ + kmem_dump_oversize_allocs++; + if (size > kmem_dump_oversize_max) + kmem_dump_oversize_max = size; + } + return (buf); + } + + buf = kmem_cache_alloc(cp, kmflag); + if ((cp->cache_flags & KMF_BUFTAG) && !KMEM_DUMP(cp) && buf != NULL) { + kmem_buftag_t *btp = KMEM_BUFTAG(cp, buf); + ((uint8_t *)buf)[size] = KMEM_REDZONE_BYTE; + ((uint32_t *)btp)[1] = KMEM_SIZE_ENCODE(size); + + if (cp->cache_flags & KMF_LITE) { + KMEM_BUFTAG_LITE_ENTER(btp, kmem_lite_count, caller()); + } + } + return (buf); +} + +void +zfs_kmem_free(void *buf, size_t size) +{ + size_t index; + kmem_cache_t *cp; + + if (size == 0 || buf == KMEM_ZERO_SIZE_PTR || buf == NULL) + return; + + if ((index = (size - 1) >> KMEM_ALIGN_SHIFT) < KMEM_ALLOC_TABLE_MAX) { + cp = kmem_alloc_table[index]; + /* fall through to kmem_cache_free() */ + + } else if ((index = ((size - 1) >> KMEM_BIG_SHIFT)) < + kmem_big_alloc_table_max) { + cp = kmem_big_alloc_table[index]; + /* fall through to kmem_cache_free() */ + + } else { + vmem_free_impl(kmem_oversize_arena, buf, size); + return; + } + + if ((cp->cache_flags & KMF_BUFTAG) && !KMEM_DUMP(cp)) { + kmem_buftag_t *btp = KMEM_BUFTAG(cp, buf); + uint32_t *ip = (uint32_t *)btp; + if (ip[1] != KMEM_SIZE_ENCODE(size)) { + if (*(uint64_t *)buf == KMEM_FREE_PATTERN) { + kmem_error(KMERR_DUPFREE, cp, buf); + return; + } + if (KMEM_SIZE_VALID(ip[1])) { + ip[0] = KMEM_SIZE_ENCODE(size); + kmem_error(KMERR_BADSIZE, cp, buf); + } else { + kmem_error(KMERR_REDZONE, cp, buf); + } + return; + } + if (((uint8_t *)buf)[size] != KMEM_REDZONE_BYTE) { + kmem_error(KMERR_REDZONE, cp, buf); + return; + } + btp->bt_redzone = KMEM_REDZONE_PATTERN; + if (cp->cache_flags & KMF_LITE) { + KMEM_BUFTAG_LITE_ENTER(btp, kmem_lite_count, + caller()); + } + } + kmem_cache_free(cp, buf); +} + +/* + * Try to allocate at least `size' bytes of memory without sleeping or + * panicking. Return actual allocated size in `asize'. If allocation failed, + * try final allocation with sleep or panic allowed. + */ +void * +kmem_alloc_tryhard(size_t size, size_t *asize, int kmflag) +{ + void *p; + + *asize = P2ROUNDUP(size, KMEM_ALIGN); + do { + p = kmem_alloc(*asize, (kmflag | KM_NOSLEEP) & ~KM_PANIC); + if (p != NULL) + return (p); + *asize += KMEM_ALIGN; + } while (*asize <= PAGESIZE); + + *asize = P2ROUNDUP(size, KMEM_ALIGN); + return (zfs_kmem_alloc(*asize, kmflag)); +} + +/* + * Reclaim all unused memory from a cache. + */ +static void +kmem_cache_reap(kmem_cache_t *cp) +{ + ASSERT(taskq_member(kmem_taskq, curthread)); + + cp->cache_reap++; + + /* + * Ask the cache's owner to free some memory if possible. + * The idea is to handle things like the inode cache, which + * typically sits on a bunch of memory that it doesn't truly + * *need*. Reclaim policy is entirely up to the owner; this + * callback is just an advisory plea for help. + */ + if (cp->cache_reclaim != NULL) { + long delta; + + /* + * Reclaimed memory should be reapable (not included in the + * depot's working set). + */ + delta = cp->cache_full.ml_total; + cp->cache_reclaim(cp->cache_private); + delta = cp->cache_full.ml_total - delta; + if (delta > 0) { + mutex_enter(&cp->cache_depot_lock); + cp->cache_full.ml_reaplimit += delta; + cp->cache_full.ml_min += delta; + mutex_exit(&cp->cache_depot_lock); + } + } + + kmem_depot_ws_reap(cp); + + if (cp->cache_defrag != NULL && !kmem_move_noreap) { + kmem_cache_defrag(cp); + } +} + +static void +kmem_reap_timeout(void *flag_arg) +{ + uint32_t *flag = (uint32_t *)flag_arg; + + ASSERT(flag == (uint32_t *)&kmem_reaping || + flag == (uint32_t *)&kmem_reaping_idspace); + *flag = 0; +} + +static void +kmem_reap_done(void *flag) +{ + (void) bsd_timeout(kmem_reap_timeout, flag, &kmem_reap_interval); +} + +static void +kmem_reap_start(void *flag) +{ + ASSERT(flag == (uint32_t *)&kmem_reaping || + (uint32_t *)flag == &kmem_reaping_idspace); + + if (flag == &kmem_reaping) { + kmem_cache_applyall(kmem_cache_reap, kmem_taskq, TQ_NOSLEEP); + /* + * if we have segkp under heap, reap segkp cache. + */ + } + else + kmem_cache_applyall_id(kmem_cache_reap, kmem_taskq, TQ_NOSLEEP); + + /* + * We use taskq_dispatch() to schedule a timeout to clear + * the flag so that kmem_reap() becomes self-throttling: + * we won't reap again until the current reap completes *and* + * at least kmem_reap_interval ticks have elapsed. + */ + if (!taskq_dispatch(kmem_taskq, kmem_reap_done, flag, TQ_NOSLEEP)) + kmem_reap_done(flag); +} + +static void +kmem_reap_common(void *flag_arg) +{ + uint32_t *flag = (uint32_t *)flag_arg; + + + if (MUTEX_HELD(&kmem_cache_lock) || kmem_taskq == NULL || + atomic_cas_32(flag, 0, 1) != 0) + return; + + /* + * It may not be kosher to do memory allocation when a reap is called + * is called (for example, if vmem_populate() is in the call chain). + * So we start the reap going with a TQ_NOALLOC dispatch. If the + * dispatch fails, we reset the flag, and the next reap will try again. + */ + if (!taskq_dispatch(kmem_taskq, kmem_reap_start, flag, TQ_NOALLOC)) + *flag = 0; +} + +/* + * Reclaim all unused memory from all caches. Called from the VM system + * when memory gets tight. + */ +void +kmem_reap(void) +{ + kmem_reap_common(&kmem_reaping); +} + +/* + * Reclaim all unused memory from identifier arenas, called when a vmem + * arena not back by memory is exhausted. Since reaping memory-backed caches + * cannot help with identifier exhaustion, we avoid both a large amount of + * work and unwanted side-effects from reclaim callbacks. + */ +void +kmem_reap_idspace(void) +{ + kmem_reap_common(&kmem_reaping_idspace); +} + +/* + * Purge all magazines from a cache and set its magazine limit to zero. + * All calls are serialized by the kmem_taskq lock, except for the final + * call from kmem_cache_destroy(). + */ +static void +kmem_cache_magazine_purge(kmem_cache_t *cp) +{ + kmem_cpu_cache_t *ccp; + kmem_magazine_t *mp, *pmp; + int rounds, prounds, cpu_seqid; + + ASSERT(!list_link_active(&cp->cache_link) || + taskq_member(kmem_taskq, curthread)); + ASSERT(MUTEX_NOT_HELD(&cp->cache_lock)); + + for (cpu_seqid = 0; cpu_seqid < max_ncpus; cpu_seqid++) { + ccp = &cp->cache_cpu[cpu_seqid]; + + mutex_enter(&ccp->cc_lock); + mp = ccp->cc_loaded; + pmp = ccp->cc_ploaded; + rounds = ccp->cc_rounds; + prounds = ccp->cc_prounds; + ccp->cc_loaded = NULL; + ccp->cc_ploaded = NULL; + ccp->cc_rounds = -1; + ccp->cc_prounds = -1; + ccp->cc_magsize = 0; + mutex_exit(&ccp->cc_lock); + + if (mp) + kmem_magazine_destroy(cp, mp, rounds); + + if (pmp) + kmem_magazine_destroy(cp, pmp, prounds); + } + + kmem_depot_ws_zero(cp); + kmem_depot_ws_reap(cp); +} + +/* + * Enable per-cpu magazines on a cache. + */ +static void +kmem_cache_magazine_enable(kmem_cache_t *cp) +{ + int cpu_seqid; + + if (cp->cache_flags & KMF_NOMAGAZINE) + return; + + for (cpu_seqid = 0; cpu_seqid < max_ncpus; cpu_seqid++) { + kmem_cpu_cache_t *ccp = &cp->cache_cpu[cpu_seqid]; + mutex_enter(&ccp->cc_lock); + ccp->cc_magsize = cp->cache_magtype->mt_magsize; + mutex_exit(&ccp->cc_lock); + } + +} + +static void +kmem_cache_magazine_disable(kmem_cache_t *cp) +{ + int cpu_seqid; + + if (cp->cache_flags & KMF_NOMAGAZINE) + return; + + for (cpu_seqid = 0; cpu_seqid < max_ncpus; cpu_seqid++) { + kmem_cpu_cache_t *ccp = &cp->cache_cpu[cpu_seqid]; + mutex_enter(&ccp->cc_lock); + ccp->cc_magsize = 0; + mutex_exit(&ccp->cc_lock); + } + +} + +/* + * Allow our caller to determine if there are running reaps. + * + * This call is very conservative and may return B_TRUE even when + * reaping activity isn't active. If it returns B_FALSE, then reaping + * activity is definitely inactive. + */ +boolean_t +kmem_cache_reap_active(void) +{ + return (B_FALSE); +} + +/* + * Reap (almost) everything right now. + */ +void +kmem_cache_reap_now(kmem_cache_t *cp) +{ + ASSERT(list_link_active(&cp->cache_link)); + + kmem_depot_ws_zero(cp); + + (void) taskq_dispatch(kmem_taskq, + (task_func_t *)kmem_depot_ws_reap, cp, TQ_SLEEP); +} + +/* + * Recompute a cache's magazine size. The trade-off is that larger magazines + * provide a higher transfer rate with the depot, while smaller magazines + * reduce memory consumption. Magazine resizing is an expensive operation; + * it should not be done frequently. + * + * Changes to the magazine size are serialized by the kmem_taskq lock. + * + * Note: at present this only grows the magazine size. It might be useful + * to allow shrinkage too. + */ +static void +kmem_cache_magazine_resize(kmem_cache_t *cp) +{ + kmem_magtype_t *mtp = cp->cache_magtype; + + ASSERT(taskq_member(kmem_taskq, curthread)); + + if (cp->cache_chunksize < mtp->mt_maxbuf) { + kmem_cache_magazine_purge(cp); + mutex_enter(&cp->cache_depot_lock); + cp->cache_magtype = ++mtp; + cp->cache_depot_contention_prev = + cp->cache_depot_contention + INT_MAX; + mutex_exit(&cp->cache_depot_lock); + kmem_cache_magazine_enable(cp); + } +} + +/* + * Rescale a cache's hash table, so that the table size is roughly the + * cache size. We want the average lookup time to be extremely small. + */ +static void +kmem_hash_rescale(kmem_cache_t *cp) +{ + kmem_bufctl_t **old_table, **new_table, *bcp; + size_t old_size, new_size, h; + + ASSERT(taskq_member(kmem_taskq, curthread)); + + new_size = MAX(KMEM_HASH_INITIAL, + 1 << (highbit(3 * cp->cache_buftotal + 4) - 2)); + old_size = cp->cache_hash_mask + 1; + + if ((old_size >> 1) <= new_size && new_size <= (old_size << 1)) + return; + + new_table = vmem_alloc_impl(kmem_hash_arena, new_size * sizeof (void *), + VM_NOSLEEP); + if (new_table == NULL) + return; + memset(new_table, 0, new_size * sizeof (void *)); + + mutex_enter(&cp->cache_lock); + + old_size = cp->cache_hash_mask + 1; + old_table = cp->cache_hash_table; + + cp->cache_hash_mask = new_size - 1; + cp->cache_hash_table = new_table; + cp->cache_rescale++; + + for (h = 0; h < old_size; h++) { + bcp = old_table[h]; + while (bcp != NULL) { + void *addr = bcp->bc_addr; + kmem_bufctl_t *next_bcp = bcp->bc_next; + kmem_bufctl_t **hash_bucket = KMEM_HASH(cp, addr); + bcp->bc_next = *hash_bucket; + *hash_bucket = bcp; + bcp = next_bcp; + } + } + + mutex_exit(&cp->cache_lock); + + vmem_free_impl(kmem_hash_arena, old_table, old_size * sizeof (void *)); +} + +/* + * Perform periodic maintenance on a cache: hash rescaling, depot working-set + * update, magazine resizing, and slab consolidation. + */ +static void +kmem_cache_update(kmem_cache_t *cp) +{ + int need_hash_rescale = 0; + int need_magazine_resize = 0; + + /* + * If the cache has become much larger or smaller than its hash table, + * fire off a request to rescale the hash table. + */ + mutex_enter(&cp->cache_lock); + + if ((cp->cache_flags & KMF_HASH) && + (cp->cache_buftotal > (cp->cache_hash_mask << 1) || + (cp->cache_buftotal < (cp->cache_hash_mask >> 1) && + cp->cache_hash_mask > KMEM_HASH_INITIAL))) + need_hash_rescale = 1; + + mutex_exit(&cp->cache_lock); + + /* + * Update the depot working set statistics. + */ + kmem_depot_ws_update(cp); + + /* + * If there's a lot of contention in the depot, + * increase the magazine size. + */ + mutex_enter(&cp->cache_depot_lock); + + if (cp->cache_chunksize < cp->cache_magtype->mt_maxbuf && + (int)(cp->cache_depot_contention - + cp->cache_depot_contention_prev) > kmem_depot_contention) + need_magazine_resize = 1; + + cp->cache_depot_contention_prev = cp->cache_depot_contention; + + mutex_exit(&cp->cache_depot_lock); + + if (need_hash_rescale) + (void) taskq_dispatch(kmem_taskq, + (task_func_t *)kmem_hash_rescale, cp, TQ_NOSLEEP); + + if (need_magazine_resize) + (void) taskq_dispatch(kmem_taskq, + (task_func_t *)kmem_cache_magazine_resize, + cp, TQ_NOSLEEP); + + // smd : the following if is only true for the dnode cache + if (cp->cache_defrag != NULL) + (void) taskq_dispatch(kmem_taskq, + (task_func_t *)kmem_cache_scan, cp, TQ_NOSLEEP); + +#ifdef DEBUG + else { + // for every other cache, duplicate some of the logic from + // kmem_cache_scan() below + // run reap occasionally even if there is plenty of memory + uint16_t debug_rand; + + (void) random_get_bytes((uint8_t *)&debug_rand, 2); + if (!kmem_move_noreap && + ((debug_rand % kmem_mtb_reap) == 0)) { + /* + * no mutex above, so no need to give it up as + * in kmem_cache_scan() + */ + } + } +#endif + +} + +static void kmem_update(void *); + +static void +kmem_update_timeout(void *dummy) +{ + (void) bsd_timeout(kmem_update, dummy, &kmem_reap_interval); +} + +static void +kmem_update(void *dummy) +{ + kmem_cache_applyall(kmem_cache_update, NULL, TQ_NOSLEEP); + + /* + * We use taskq_dispatch() to reschedule the timeout so that + * kmem_update() becomes self-throttling: it won't schedule + * new tasks until all previous tasks have completed. + */ + if (!taskq_dispatch(kmem_taskq, kmem_update_timeout, dummy, TQ_NOSLEEP)) + kmem_update_timeout(NULL); + +} + +static int +kmem_cache_kstat_update(kstat_t *ksp, int rw) +{ + struct kmem_cache_kstat *kmcp = &kmem_cache_kstat; + kmem_cache_t *cp = ksp->ks_private; + uint64_t cpu_buf_avail; + uint64_t buf_avail = 0; + int cpu_seqid; + long reap; + + if (rw == KSTAT_WRITE) + return (EACCES); + + mutex_enter(&cp->cache_lock); + + kmcp->kmc_alloc_fail.value.ui64 = cp->cache_alloc_fail; + kmcp->kmc_alloc.value.ui64 = cp->cache_slab_alloc; + kmcp->kmc_free.value.ui64 = cp->cache_slab_free; + kmcp->kmc_slab_alloc.value.ui64 = cp->cache_slab_alloc; + kmcp->kmc_slab_free.value.ui64 = cp->cache_slab_free; + kmcp->kmc_no_vba_success.value.ui64 = cp->no_vba_success; + kmcp->kmc_no_vba_fail.value.ui64 = cp->no_vba_fail; + kmcp->kmc_arc_no_grow_set.value.ui64 = cp->arc_no_grow_set; + kmcp->kmc_arc_no_grow.value.ui64 = cp->arc_no_grow; + + for (cpu_seqid = 0; cpu_seqid < max_ncpus; cpu_seqid++) { + kmem_cpu_cache_t *ccp = &cp->cache_cpu[cpu_seqid]; + + mutex_enter(&ccp->cc_lock); + + cpu_buf_avail = 0; + if (ccp->cc_rounds > 0) + cpu_buf_avail += ccp->cc_rounds; + if (ccp->cc_prounds > 0) + cpu_buf_avail += ccp->cc_prounds; + + kmcp->kmc_alloc.value.ui64 += ccp->cc_alloc; + kmcp->kmc_free.value.ui64 += ccp->cc_free; + buf_avail += cpu_buf_avail; + + mutex_exit(&ccp->cc_lock); + } + + mutex_enter(&cp->cache_depot_lock); + + kmcp->kmc_depot_alloc.value.ui64 = cp->cache_full.ml_alloc; + kmcp->kmc_depot_free.value.ui64 = cp->cache_empty.ml_alloc; + kmcp->kmc_depot_contention.value.ui64 = cp->cache_depot_contention; + kmcp->kmc_full_magazines.value.ui64 = cp->cache_full.ml_total; + kmcp->kmc_empty_magazines.value.ui64 = cp->cache_empty.ml_total; + kmcp->kmc_magazine_size.value.ui64 = + (cp->cache_flags & KMF_NOMAGAZINE) ? + 0 : cp->cache_magtype->mt_magsize; + + kmcp->kmc_alloc.value.ui64 += cp->cache_full.ml_alloc; + kmcp->kmc_free.value.ui64 += cp->cache_empty.ml_alloc; + buf_avail += cp->cache_full.ml_total * cp->cache_magtype->mt_magsize; + + reap = MIN(cp->cache_full.ml_reaplimit, cp->cache_full.ml_min); + reap = MIN(reap, cp->cache_full.ml_total); + + mutex_exit(&cp->cache_depot_lock); + + kmcp->kmc_buf_size.value.ui64 = cp->cache_bufsize; + kmcp->kmc_align.value.ui64 = cp->cache_align; + kmcp->kmc_chunk_size.value.ui64 = cp->cache_chunksize; + kmcp->kmc_slab_size.value.ui64 = cp->cache_slabsize; + kmcp->kmc_buf_constructed.value.ui64 = buf_avail; + buf_avail += cp->cache_bufslab; + kmcp->kmc_buf_avail.value.ui64 = buf_avail; + kmcp->kmc_buf_inuse.value.ui64 = cp->cache_buftotal - buf_avail; + kmcp->kmc_buf_total.value.ui64 = cp->cache_buftotal; + kmcp->kmc_buf_max.value.ui64 = cp->cache_bufmax; + kmcp->kmc_slab_create.value.ui64 = cp->cache_slab_create; + kmcp->kmc_slab_destroy.value.ui64 = cp->cache_slab_destroy; + kmcp->kmc_hash_size.value.ui64 = (cp->cache_flags & KMF_HASH) ? + cp->cache_hash_mask + 1 : 0; + kmcp->kmc_hash_lookup_depth.value.ui64 = cp->cache_lookup_depth; + kmcp->kmc_hash_rescale.value.ui64 = cp->cache_rescale; + kmcp->kmc_vmem_source.value.ui64 = cp->cache_arena->vm_id; + kmcp->kmc_reap.value.ui64 = cp->cache_reap; + + if (cp->cache_defrag == NULL) { + kmcp->kmc_move_callbacks.value.ui64 = 0; + kmcp->kmc_move_yes.value.ui64 = 0; + kmcp->kmc_move_no.value.ui64 = 0; + kmcp->kmc_move_later.value.ui64 = 0; + kmcp->kmc_move_dont_need.value.ui64 = 0; + kmcp->kmc_move_dont_know.value.ui64 = 0; + kmcp->kmc_move_hunt_found.value.ui64 = 0; + kmcp->kmc_move_slabs_freed.value.ui64 = 0; + kmcp->kmc_defrag.value.ui64 = 0; + kmcp->kmc_scan.value.ui64 = 0; + kmcp->kmc_move_reclaimable.value.ui64 = 0; + } else { + int64_t reclaimable; + + kmem_defrag_t *kd = cp->cache_defrag; + kmcp->kmc_move_callbacks.value.ui64 = kd->kmd_callbacks; + kmcp->kmc_move_yes.value.ui64 = kd->kmd_yes; + kmcp->kmc_move_no.value.ui64 = kd->kmd_no; + kmcp->kmc_move_later.value.ui64 = kd->kmd_later; + kmcp->kmc_move_dont_need.value.ui64 = kd->kmd_dont_need; + kmcp->kmc_move_dont_know.value.ui64 = kd->kmd_dont_know; + kmcp->kmc_move_hunt_found.value.ui64 = 0; + kmcp->kmc_move_slabs_freed.value.ui64 = kd->kmd_slabs_freed; + kmcp->kmc_defrag.value.ui64 = kd->kmd_defrags; + kmcp->kmc_scan.value.ui64 = kd->kmd_scans; + + reclaimable = cp->cache_bufslab - (cp->cache_maxchunks - 1); + reclaimable = MAX(reclaimable, 0); + reclaimable += ((uint64_t)reap * cp->cache_magtype->mt_magsize); + kmcp->kmc_move_reclaimable.value.ui64 = reclaimable; + } + + mutex_exit(&cp->cache_lock); + return (0); +} + +/* + * Return a named statistic about a particular cache. + * This shouldn't be called very often, so it's currently designed for + * simplicity (leverages existing kstat support) rather than efficiency. + */ +uint64_t +kmem_cache_stat(kmem_cache_t *cp, char *name) +{ + int i; + kstat_t *ksp = cp->cache_kstat; + kstat_named_t *knp = (kstat_named_t *)&kmem_cache_kstat; + uint64_t value = 0; + + if (ksp != NULL) { + mutex_enter(&kmem_cache_kstat_lock); + (void) kmem_cache_kstat_update(ksp, KSTAT_READ); + for (i = 0; i < ksp->ks_ndata; i++) { + if (strcmp(knp[i].name, name) == 0) { + value = knp[i].value.ui64; + break; + } + } + mutex_exit(&kmem_cache_kstat_lock); + } + return (value); +} + +// TRUE if we have more than a critical minimum of memory +// used in arc_memory_throttle; if FALSE, we throttle +bool +spl_minimal_physmem_p_logic() +{ + // do we have enough memory to avoid throttling? + if (spl_vm_pages_wanted > 0 || + (spl_vm_pressure_level > 0 && + spl_vm_pressure_level != MAGIC_PRESSURE_UNAVAILABLE)) + return (false); + // XXX : check reclaiming load? + return (true); +} + +int64_t +spl_minimal_physmem_p(void) +{ + + // arc will throttle throttle if we are paging, otherwise + // we want a small bit of pressure here so that we can compete + // a little with the xnu buffer cache + + return (spl_minimal_physmem_p_logic() && spl_free > -4096LL); +} + +/* + * Return the maximum amount of memory that is (in theory) allocatable + * from the heap. This may be used as an estimate only since there + * is no guarentee this space will still be available when an allocation + * request is made, nor that the space may be allocated in one big request + * due to kernel heap fragmentation. + */ +size_t +kmem_maxavail(void) +{ +#ifndef APPLE + // spgcnt_t pmem = availrmem - tune.t_minarmem; + // spgcnt_t vmem = btop(vmem_size(heap_arena, VMEM_FREE)); + // + // return ((size_t)ptob(MAX(MIN(pmem, vmem), 0))); +#endif + return (physmem * PAGE_SIZE); +} + +/* + * Indicate whether memory-intensive kmem debugging is enabled. + */ +int +kmem_debugging(void) +{ + return (kmem_flags & (KMF_AUDIT | KMF_REDZONE)); +} + +/* binning function, sorts finely at the two extremes */ +#define KMEM_PARTIAL_SLAB_WEIGHT(sp, binshift) \ +((((sp)->slab_refcnt <= (binshift)) || \ +(((sp)->slab_chunks - (sp)->slab_refcnt) <= (binshift))) \ +? -(sp)->slab_refcnt \ +: -((binshift) + ((sp)->slab_refcnt >> (binshift)))) + +/* + * Minimizing the number of partial slabs on the freelist minimizes + * fragmentation (the ratio of unused buffers held by the slab layer). There are + * two ways to get a slab off of the freelist: 1) free all the buffers on the + * slab, and 2) allocate all the buffers on the slab. It follows that we want + * the most-used slabs at the front of the list where they have the best chance + * of being completely allocated, and the least-used slabs at a safe distance + * from the front to improve the odds that the few remaining buffers will all be + * freed before another allocation can tie up the slab. For that reason a slab + * with a higher slab_refcnt sorts less than than a slab with a lower + * slab_refcnt. + * + * However, if a slab has at least one buffer that is deemed unfreeable, we + * would rather have that slab at the front of the list regardless of + * slab_refcnt, since even one unfreeable buffer makes the entire slab + * unfreeable. If the client returns KMEM_CBRC_NO in response to a cache_move() + * callback, the slab is marked unfreeable for as long as it remains on the + * freelist. + */ +static int +kmem_partial_slab_cmp(const void *pp0, const void *pp1) +{ + const kmem_cache_t *cp; + const kmem_slab_t *s0 = pp0; + const kmem_slab_t *s1 = pp1; + int w0, w1; + size_t binshift; + + ASSERT(KMEM_SLAB_IS_PARTIAL(s0)); + ASSERT(KMEM_SLAB_IS_PARTIAL(s1)); + ASSERT(s0->slab_cache == s1->slab_cache); + cp = s1->slab_cache; + ASSERT(MUTEX_HELD((struct kmutex *)&cp->cache_lock)); + binshift = cp->cache_partial_binshift; + + /* weight of first slab */ + w0 = KMEM_PARTIAL_SLAB_WEIGHT(s0, binshift); + if (s0->slab_flags & KMEM_SLAB_NOMOVE) { + w0 -= cp->cache_maxchunks; + } + + /* weight of second slab */ + w1 = KMEM_PARTIAL_SLAB_WEIGHT(s1, binshift); + if (s1->slab_flags & KMEM_SLAB_NOMOVE) { + w1 -= cp->cache_maxchunks; + } + + if (w0 < w1) + return (-1); + if (w0 > w1) + return (1); + + // compare slab age if available + hrtime_t c0 = s0->slab_create_time, c1 = s1->slab_create_time; + if (c0 != 0 && c1 != 0 && c0 != c1) { + // higher time is newer; newer sorts before older + if (c0 < c1) // c0 is older than c1 + return (1); // so c0 sorts after c1 + if (c0 > c1) + return (-1); + } + + /* compare pointer values */ + if ((uintptr_t)s0 < (uintptr_t)s1) + return (-1); + if ((uintptr_t)s0 > (uintptr_t)s1) + return (1); + + return (0); +} + +/* + * It must be valid to call the destructor (if any) on a newly created object. + * That is, the constructor (if any) must leave the object in a valid state for + * the destructor. + */ +kmem_cache_t * +kmem_cache_create( + char *name, /* descriptive name for this cache */ + size_t bufsize, /* size of the objects it manages */ + size_t align, /* required object alignment */ + int (*constructor)(void *, void *, int), /* object constructor */ + void (*destructor)(void *, void *), /* object destructor */ + void (*reclaim)(void *), /* memory reclaim callback */ + void *private, /* pass-thru arg for constr/destr/reclaim */ + vmem_t *vmp, /* vmem source for slab allocation */ + int cflags) /* cache creation flags */ +{ + int cpu_seqid; + size_t chunksize; + kmem_cache_t *cp; + kmem_magtype_t *mtp; + size_t csize = KMEM_CACHE_SIZE(max_ncpus); + +#ifdef DEBUG + /* + * Cache names should conform to the rules for valid C identifiers + */ + if (!strident_valid(name)) { + cmn_err(CE_CONT, + "kmem_cache_create: '%s' is an invalid cache name\n" + "cache names must conform to the rules for " + "C identifiers\n", name); + } +#endif /* DEBUG */ + + if (vmp == NULL) + vmp = kmem_default_arena; + + /* + * If this kmem cache has an identifier vmem arena as its source, mark + * it such to allow kmem_reap_idspace(). + */ + ASSERT(!(cflags & KMC_IDENTIFIER)); /* consumer should not set this */ + if (vmp->vm_cflags & VMC_IDENTIFIER) + cflags |= KMC_IDENTIFIER; + + /* + * Get a kmem_cache structure. We arrange that cp->cache_cpu[] + * is aligned on a KMEM_CPU_CACHE_SIZE boundary to prevent + * false sharing of per-CPU data. + */ + cp = vmem_xalloc(kmem_cache_arena, csize, + KMEM_CPU_CACHE_SIZE, + P2NPHASE(csize, KMEM_CPU_CACHE_SIZE), + 0, NULL, NULL, VM_SLEEP); + memset(cp, 0, csize); + list_link_init(&cp->cache_link); + + if (align == 0) + align = KMEM_ALIGN; + + /* + * If we're not at least KMEM_ALIGN aligned, we can't use free + * memory to hold bufctl information (because we can't safely + * perform word loads and stores on it). + */ + if (align < KMEM_ALIGN) + cflags |= KMC_NOTOUCH; + + if ((align & (align - 1)) != 0 || align > vmp->vm_quantum) + panic("kmem_cache_create: bad alignment %lu", align); + + mutex_enter(&kmem_flags_lock); + if (kmem_flags & KMF_RANDOMIZE) + kmem_flags = (((kmem_flags | ~KMF_RANDOM) + 1) & KMF_RANDOM) | + KMF_RANDOMIZE; + cp->cache_flags = (kmem_flags | cflags) & KMF_DEBUG; + mutex_exit(&kmem_flags_lock); + + /* + * Make sure all the various flags are reasonable. + */ + ASSERT(!(cflags & KMC_NOHASH) || !(cflags & KMC_NOTOUCH)); + + if (cp->cache_flags & KMF_LITE) { + if (bufsize >= kmem_lite_minsize && + align <= kmem_lite_maxalign && + P2PHASE(bufsize, kmem_lite_maxalign) != 0) { + cp->cache_flags |= KMF_BUFTAG; + cp->cache_flags &= ~(KMF_AUDIT | KMF_FIREWALL); + } else { + cp->cache_flags &= ~KMF_DEBUG; + } + } + + if (cp->cache_flags & KMF_DEADBEEF) + cp->cache_flags |= KMF_REDZONE; + + if ((cflags & KMC_QCACHE) && (cp->cache_flags & KMF_AUDIT)) + cp->cache_flags |= KMF_NOMAGAZINE; + + if (cflags & KMC_NODEBUG) + cp->cache_flags &= ~KMF_DEBUG; + + if (cflags & KMC_NOTOUCH) + cp->cache_flags &= ~KMF_TOUCH; + + if (cflags & KMC_PREFILL) + cp->cache_flags |= KMF_PREFILL; + + if (cflags & KMC_NOHASH) + cp->cache_flags &= ~(KMF_AUDIT | KMF_FIREWALL); + + if (cflags & KMC_NOMAGAZINE) + cp->cache_flags |= KMF_NOMAGAZINE; + + if ((cp->cache_flags & KMF_AUDIT) && !(cflags & KMC_NOTOUCH)) + cp->cache_flags |= KMF_REDZONE; + + if (!(cp->cache_flags & KMF_AUDIT)) + cp->cache_flags &= ~KMF_CONTENTS; + + if ((cp->cache_flags & KMF_BUFTAG) && bufsize >= kmem_minfirewall && + !(cp->cache_flags & KMF_LITE) && !(cflags & KMC_NOHASH)) + cp->cache_flags |= KMF_FIREWALL; + + if (vmp != kmem_default_arena || kmem_firewall_arena == NULL) + cp->cache_flags &= ~KMF_FIREWALL; + + if (cp->cache_flags & KMF_FIREWALL) { + cp->cache_flags &= ~KMF_BUFTAG; + cp->cache_flags |= KMF_NOMAGAZINE; + ASSERT(vmp == kmem_default_arena); + vmp = kmem_firewall_arena; + } + + /* + * Set cache properties. + */ + (void) strlcpy(cp->cache_name, name, sizeof (cp->cache_name)); + strident_canon(cp->cache_name, sizeof (cp->cache_name)); + cp->cache_bufsize = bufsize; + cp->cache_align = align; + cp->cache_constructor = constructor; + cp->cache_destructor = destructor; + cp->cache_reclaim = reclaim; + cp->cache_private = private; + cp->cache_arena = vmp; + cp->cache_cflags = cflags; + + /* + * Determine the chunk size. + */ + chunksize = bufsize; + + if (align >= KMEM_ALIGN) { + chunksize = P2ROUNDUP(chunksize, KMEM_ALIGN); + cp->cache_bufctl = chunksize - KMEM_ALIGN; + } + + if (cp->cache_flags & KMF_BUFTAG) { + cp->cache_bufctl = chunksize; + cp->cache_buftag = chunksize; + if (cp->cache_flags & KMF_LITE) + chunksize += KMEM_BUFTAG_LITE_SIZE(kmem_lite_count); + else + chunksize += sizeof (kmem_buftag_t); + } + + if (cp->cache_flags & KMF_DEADBEEF) { + cp->cache_verify = MIN(cp->cache_buftag, kmem_maxverify); + if (cp->cache_flags & KMF_LITE) + cp->cache_verify = sizeof (uint64_t); + } + + cp->cache_contents = MIN(cp->cache_bufctl, kmem_content_maxsave); + + cp->cache_chunksize = chunksize = P2ROUNDUP(chunksize, align); + + /* + * Now that we know the chunk size, determine the optimal slab size. + */ + + size_t vquantum = vmp->vm_quantum; + + if ((cflags & KMC_ARENA_SLAB) == KMC_ARENA_SLAB) { + VERIFY3U((vmp->vm_cflags & VMC_NO_QCACHE), ==, VMC_NO_QCACHE); + VERIFY3U(vmp->vm_min_import, >, 0); + VERIFY3U(vmp->vm_min_import, >=, (2 * vmp->vm_quantum)); + VERIFY(ISP2(vmp->vm_min_import)); + vquantum = vmp->vm_min_import >> 1; + } + + if (vmp == kmem_firewall_arena) { + cp->cache_slabsize = P2ROUNDUP(chunksize, vquantum); + cp->cache_mincolor = cp->cache_slabsize - chunksize; + cp->cache_maxcolor = cp->cache_mincolor; + cp->cache_flags |= KMF_HASH; + ASSERT(!(cp->cache_flags & KMF_BUFTAG)); + } else if ((cflags & KMC_NOHASH) || (!(cflags & KMC_NOTOUCH) && + !(cp->cache_flags & KMF_AUDIT) && + chunksize < vquantum / + KMEM_VOID_FRACTION)) { + cp->cache_slabsize = vquantum; + cp->cache_mincolor = 0; + cp->cache_maxcolor = + (cp->cache_slabsize - sizeof (kmem_slab_t)) % chunksize; + ASSERT(chunksize + sizeof (kmem_slab_t) <= cp->cache_slabsize); + ASSERT(!(cp->cache_flags & KMF_AUDIT)); + } else { + size_t chunks, bestfit, waste, slabsize; + size_t minwaste = LONG_MAX; + + for (chunks = 1; chunks <= KMEM_VOID_FRACTION; chunks++) { + slabsize = P2ROUNDUP(chunksize * chunks, + vquantum); + chunks = slabsize / chunksize; + waste = (slabsize % chunksize) / chunks; + if (waste < minwaste) { + minwaste = waste; + bestfit = slabsize; + } + } + if (cflags & KMC_QCACHE) + bestfit = VMEM_QCACHE_SLABSIZE(vmp->vm_qcache_max); + cp->cache_slabsize = bestfit; + cp->cache_mincolor = 0; + cp->cache_maxcolor = bestfit % chunksize; + cp->cache_flags |= KMF_HASH; + } + + cp->cache_maxchunks = (cp->cache_slabsize / cp->cache_chunksize); + cp->cache_partial_binshift = highbit(cp->cache_maxchunks / 16) + 1; + + /* + * Disallowing prefill when either the DEBUG or HASH flag is set or when + * there is a constructor avoids some tricky issues with debug setup + * that may be revisited later. We cannot allow prefill in a + * metadata cache because of potential recursion. + */ + if (vmp == kmem_msb_arena || + cp->cache_flags & (KMF_HASH | KMF_BUFTAG) || + cp->cache_constructor != NULL) + cp->cache_flags &= ~KMF_PREFILL; + + if (cp->cache_flags & KMF_HASH) { + ASSERT(!(cflags & KMC_NOHASH)); + cp->cache_bufctl_cache = (cp->cache_flags & KMF_AUDIT) ? + kmem_bufctl_audit_cache : kmem_bufctl_cache; + } + + if (cp->cache_maxcolor >= vquantum) + cp->cache_maxcolor = vquantum - 1; + + cp->cache_color = cp->cache_mincolor; + + /* + * Initialize the rest of the slab layer. + */ + mutex_init(&cp->cache_lock, NULL, MUTEX_DEFAULT, NULL); + + avl_create(&cp->cache_partial_slabs, kmem_partial_slab_cmp, + sizeof (kmem_slab_t), offsetof(kmem_slab_t, slab_link)); + /* LINTED: E_TRUE_LOGICAL_EXPR */ + ASSERT(sizeof (list_node_t) <= sizeof (avl_node_t)); + /* reuse partial slab AVL linkage for complete slab list linkage */ + list_create(&cp->cache_complete_slabs, + sizeof (kmem_slab_t), offsetof(kmem_slab_t, slab_link)); + + if (cp->cache_flags & KMF_HASH) { + cp->cache_hash_table = vmem_alloc_impl(kmem_hash_arena, + KMEM_HASH_INITIAL * sizeof (void *), + VM_SLEEP); + memset(cp->cache_hash_table, 0, + KMEM_HASH_INITIAL * sizeof (void *)); + cp->cache_hash_mask = KMEM_HASH_INITIAL - 1; + cp->cache_hash_shift = highbit((ulong_t)chunksize) - 1; + } + + /* + * Initialize the depot. + */ + mutex_init(&cp->cache_depot_lock, NULL, MUTEX_DEFAULT, NULL); + + for (mtp = kmem_magtype; chunksize <= mtp->mt_minbuf; mtp++) + continue; + + cp->cache_magtype = mtp; + + /* + * Initialize the CPU layer. + */ + for (cpu_seqid = 0; cpu_seqid < max_ncpus; cpu_seqid++) { + kmem_cpu_cache_t *ccp = &cp->cache_cpu[cpu_seqid]; + mutex_init(&ccp->cc_lock, NULL, MUTEX_DEFAULT, NULL); // XNU + ccp->cc_flags = cp->cache_flags; + ccp->cc_rounds = -1; + ccp->cc_prounds = -1; + } + + /* + * Create the cache's kstats. + */ + if ((cp->cache_kstat = kstat_create("unix", 0, cp->cache_name, + "kmem_cache", KSTAT_TYPE_NAMED, + sizeof (kmem_cache_kstat) / sizeof (kstat_named_t), + KSTAT_FLAG_VIRTUAL)) != NULL) { + cp->cache_kstat->ks_data = &kmem_cache_kstat; + cp->cache_kstat->ks_update = kmem_cache_kstat_update; + cp->cache_kstat->ks_private = cp; + cp->cache_kstat->ks_lock = &kmem_cache_kstat_lock; + kstat_install(cp->cache_kstat); + } + + /* + * Add the cache to the global list. This makes it visible + * to kmem_update(), so the cache must be ready for business. + */ + mutex_enter(&kmem_cache_lock); + list_insert_tail(&kmem_caches, cp); + mutex_exit(&kmem_cache_lock); + + if (kmem_ready) + kmem_cache_magazine_enable(cp); + + return (cp); +} + +static int +kmem_move_cmp(const void *buf, const void *p) +{ + const kmem_move_t *kmm = p; + uintptr_t v1 = (uintptr_t)buf; + uintptr_t v2 = (uintptr_t)kmm->kmm_from_buf; + return (v1 < v2 ? -1 : (v1 > v2 ? 1 : 0)); +} + +static void +kmem_reset_reclaim_threshold(kmem_defrag_t *kmd) +{ + kmd->kmd_reclaim_numer = 1; +} + +/* + * Initially, when choosing candidate slabs for buffers to move, we want to be + * very selective and take only slabs that are less than + * (1 / KMEM_VOID_FRACTION) allocated. If we have difficulty finding candidate + * slabs, then we raise the allocation ceiling incrementally. The reclaim + * threshold is reset to (1 / KMEM_VOID_FRACTION) as soon as the cache is no + * longer fragmented. + */ +static void +kmem_adjust_reclaim_threshold(kmem_defrag_t *kmd, int direction) +{ + if (direction > 0) { + /* make it easier to find a candidate slab */ + if (kmd->kmd_reclaim_numer < (KMEM_VOID_FRACTION - 1)) { + kmd->kmd_reclaim_numer++; + } + } else { + /* be more selective */ + if (kmd->kmd_reclaim_numer > 1) { + kmd->kmd_reclaim_numer--; + } + } +} + +uint64_t +spl_kmem_cache_inuse(kmem_cache_t *cache) +{ + return (cache->cache_buftotal); +} + +uint64_t +spl_kmem_cache_entry_size(kmem_cache_t *cache) +{ + return (cache->cache_bufsize); +} + +void +kmem_cache_set_move(kmem_cache_t *cp, + kmem_cbrc_t (*move)(void *, void *, size_t, void *)) +{ + kmem_defrag_t *defrag; + + ASSERT(move != NULL); + /* + * The consolidator does not support NOTOUCH caches because kmem cannot + * initialize their slabs with the 0xbaddcafe memory pattern, which sets + * a low order bit usable by clients to distinguish uninitialized memory + * from known objects (see kmem_slab_create). + */ + ASSERT(!(cp->cache_cflags & KMC_NOTOUCH)); + ASSERT(!(cp->cache_cflags & KMC_IDENTIFIER)); + + /* + * We should not be holding anyone's cache lock when calling + * kmem_cache_alloc(), so allocate in all cases before acquiring the + * lock. + */ + defrag = kmem_cache_alloc(kmem_defrag_cache, KM_SLEEP); + + mutex_enter(&cp->cache_lock); + + if (KMEM_IS_MOVABLE(cp)) { + if (cp->cache_move == NULL) { + ASSERT(cp->cache_slab_alloc == 0); + + cp->cache_defrag = defrag; + defrag = NULL; /* nothing to free */ + memset(cp->cache_defrag, 0, sizeof (kmem_defrag_t)); + avl_create(&cp->cache_defrag->kmd_moves_pending, + kmem_move_cmp, sizeof (kmem_move_t), + offsetof(kmem_move_t, kmm_entry)); + /* LINTED: E_TRUE_LOGICAL_EXPR */ + ASSERT(sizeof (list_node_t) <= sizeof (avl_node_t)); + /* reuse the slab's AVL linkage for deadlist linkage */ + list_create(&cp->cache_defrag->kmd_deadlist, + sizeof (kmem_slab_t), + offsetof(kmem_slab_t, slab_link)); + kmem_reset_reclaim_threshold(cp->cache_defrag); + } + cp->cache_move = move; + } + + mutex_exit(&cp->cache_lock); + + if (defrag != NULL) { + kmem_cache_free(kmem_defrag_cache, defrag); /* unused */ + } +} + +void +kmem_qcache_destroy() +{ + kmem_cache_t *cp; + kmem_cache_t *cache_to_destroy = NULL; + + do { + cache_to_destroy = NULL; + mutex_enter(&kmem_cache_lock); + for (cp = list_head(&kmem_caches); cp != NULL; + cp = list_next(&kmem_caches, cp)) { + if (cp->cache_cflags & KMC_QCACHE) { + cache_to_destroy = cp; + break; + } + } + mutex_exit(&kmem_cache_lock); + + if (cache_to_destroy) { + kmem_cache_destroy(cache_to_destroy); + } + } while (cache_to_destroy); +} + +void +kmem_cache_destroy(kmem_cache_t *cp) +{ + int cpu_seqid; + + /* + * Remove the cache from the global cache list so that no one else + * can schedule tasks on its behalf, wait for any pending tasks to + * complete, purge the cache, and then destroy it. + */ + mutex_enter(&kmem_cache_lock); + list_remove(&kmem_caches, cp); + mutex_exit(&kmem_cache_lock); + + if (kmem_taskq != NULL) + taskq_wait(kmem_taskq); + + if (kmem_move_taskq != NULL && cp->cache_defrag != NULL) + taskq_wait(kmem_move_taskq); + + kmem_cache_magazine_purge(cp); + + mutex_enter(&cp->cache_lock); + + if (cp->cache_buftotal != 0) + cmn_err(CE_WARN, "kmem_cache_destroy: '%s' (%p) not empty", + cp->cache_name, (void *)cp); + if (cp->cache_defrag != NULL) { + avl_destroy(&cp->cache_defrag->kmd_moves_pending); + list_destroy(&cp->cache_defrag->kmd_deadlist); + kmem_cache_free(kmem_defrag_cache, cp->cache_defrag); + cp->cache_defrag = NULL; + } + /* + * The cache is now dead. There should be no further activity. We + * enforce this by setting land mines in the constructor, destructor, + * reclaim, and move routines that induce a kernel text fault if + * invoked. + */ +#if defined(__aarch64__) + /* Setting landmines causes Break 0xC470: Ptrauth failure */ + cp->cache_constructor = (int (*)(void *, void *, int))NULL; + cp->cache_destructor = (void (*)(void *, void *))NULL; + cp->cache_reclaim = (void (*)(void *))NULL; + cp->cache_move = (kmem_cbrc_t (*)(void *, void *, size_t, void *))NULL; +#else + cp->cache_constructor = (int (*)(void *, void *, int))1; + cp->cache_destructor = (void (*)(void *, void *))2; + cp->cache_reclaim = (void (*)(void *))3; + cp->cache_move = (kmem_cbrc_t (*)(void *, void *, size_t, void *))4; +#endif + mutex_exit(&cp->cache_lock); + + kstat_delete(cp->cache_kstat); + + if (cp->cache_hash_table != NULL) + vmem_free_impl(kmem_hash_arena, cp->cache_hash_table, + (cp->cache_hash_mask + 1) * sizeof (void *)); + + for (cpu_seqid = 0; cpu_seqid < max_ncpus; cpu_seqid++) + mutex_destroy(&cp->cache_cpu[cpu_seqid].cc_lock); + + mutex_destroy(&cp->cache_depot_lock); + mutex_destroy(&cp->cache_lock); + + vmem_free_impl(kmem_cache_arena, cp, KMEM_CACHE_SIZE(max_ncpus)); +} + +static void +kmem_alloc_caches_create(const int *array, size_t count, + kmem_cache_t **alloc_table, size_t maxbuf, + uint_t shift) +{ + char name[KMEM_CACHE_NAMELEN + 1]; + size_t table_unit = (1 << shift); /* range of one alloc_table entry */ + size_t size = table_unit; + int i; + + for (i = 0; i < count; i++) { + size_t cache_size = array[i]; + size_t align = KMEM_ALIGN; + kmem_cache_t *cp; + + /* if the table has an entry for maxbuf, we're done */ + if (size > maxbuf) + break; + + /* cache size must be a multiple of the table unit */ + ASSERT(P2PHASE(cache_size, table_unit) == 0); + + /* + * If they allocate a multiple of the coherency granularity, + * they get a coherency-granularity-aligned address. + */ + if (IS_P2ALIGNED(cache_size, 64)) + align = 64; + if (IS_P2ALIGNED(cache_size, PAGESIZE)) + align = PAGESIZE; + (void) snprintf(name, sizeof (name), + "kmem_alloc_%lu", cache_size); + cp = kmem_cache_create(name, cache_size, align, + NULL, NULL, NULL, NULL, NULL, KMC_KMEM_ALLOC | KMF_HASH); + + while (size <= cache_size) { + alloc_table[(size - 1) >> shift] = cp; + size += table_unit; + } + } + + ASSERT(size > maxbuf); /* i.e. maxbuf <= max(cache_size) */ +} + +static void +kmem_alloc_caches_destroy() +{ + kmem_cache_t *cache_to_destroy = NULL; + kmem_cache_t *cp = NULL; + + do { + cache_to_destroy = NULL; + + // Locate the first cache that has the KMC_KMEM_ALLOC flag. + mutex_enter(&kmem_cache_lock); + + for (cp = list_head(&kmem_caches); cp != NULL; + cp = list_next(&kmem_caches, cp)) { + if (cp->cache_cflags & KMC_KMEM_ALLOC) { + cache_to_destroy = cp; + break; + } + } + + mutex_exit(&kmem_cache_lock); + + // Destroy the cache + if (cache_to_destroy) { + kmem_cache_destroy(cache_to_destroy); + } + + } while (cache_to_destroy); +} + +static void +kmem_destroy_cache_by_name(const char *substr) +{ + kmem_cache_t *cache_to_destroy = NULL; + kmem_cache_t *cp = NULL; + + do { + cache_to_destroy = NULL; + + // Locate the first cache that has the KMC_KMEM_ALLOC flag. + mutex_enter(&kmem_cache_lock); + + for (cp = list_head(&kmem_caches); cp != NULL; + cp = list_next(&kmem_caches, cp)) { + if (kmem_strstr(cp->cache_name, substr)) { + cache_to_destroy = cp; + break; + } + } + + mutex_exit(&kmem_cache_lock); + + // Destroy the cache + if (cache_to_destroy) { + kmem_cache_destroy(cache_to_destroy); + } + + } while (cache_to_destroy); +} + +static void +kmem_cache_init(int pass, int use_large_pages) +{ + int i; + size_t maxbuf; + kmem_magtype_t *mtp; + + for (i = 0; i < sizeof (kmem_magtype) / sizeof (*mtp); i++) { + char name[KMEM_CACHE_NAMELEN + 1]; + + mtp = &kmem_magtype[i]; + (void) snprintf(name, KMEM_CACHE_NAMELEN, "%s%d", + KMEM_MAGAZINE_PREFIX, + mtp->mt_magsize); + mtp->mt_cache = kmem_cache_create( + name, + (mtp->mt_magsize + 1) * sizeof (void *), + mtp->mt_align, NULL, NULL, NULL, NULL, + kmem_msb_arena, KMC_NOHASH); + } + + kmem_slab_cache = kmem_cache_create("kmem_slab_cache", + sizeof (kmem_slab_t), 0, NULL, NULL, + NULL, NULL, + kmem_msb_arena, KMC_NOHASH); + + kmem_bufctl_cache = kmem_cache_create("kmem_bufctl_cache", + sizeof (kmem_bufctl_t), 0, + NULL, NULL, NULL, NULL, + kmem_msb_arena, KMC_NOHASH); + + kmem_bufctl_audit_cache = kmem_cache_create("kmem_bufctl_audit_cache", + sizeof (kmem_bufctl_audit_t), + 0, NULL, NULL, NULL, NULL, + kmem_msb_arena, KMC_NOHASH); + + if (pass == 2) { + kmem_va_arena = vmem_create(KMEM_VA_PREFIX, + NULL, 0, PAGESIZE, + vmem_alloc_impl, vmem_free_impl, heap_arena, + 2 * PAGESIZE, VM_SLEEP); + + kmem_default_arena = vmem_create("kmem_default", + NULL, 0, PAGESIZE, + vmem_alloc_impl, vmem_free_impl, kmem_va_arena, + 0, VMC_DUMPSAFE | VM_SLEEP); + + /* Figure out what our maximum cache size is */ + maxbuf = kmem_max_cached; + if (maxbuf <= KMEM_MAXBUF) { + maxbuf = 0; + kmem_max_cached = KMEM_MAXBUF; + } else { + size_t size = 0; + size_t max = + sizeof (kmem_big_alloc_sizes) / sizeof (int); + /* + * Round maxbuf up to an existing cache size. If maxbuf + * is larger than the largest cache, we truncate it to + * the largest cache's size. + */ + for (i = 0; i < max; i++) { + size = kmem_big_alloc_sizes[i]; + if (maxbuf <= size) + break; + } + kmem_max_cached = maxbuf = size; + } + + /* + * The big alloc table may not be completely overwritten, so + * we clear out any stale cache pointers from the first pass. + */ + memset(kmem_big_alloc_table, 0, sizeof (kmem_big_alloc_table)); + } else { + /* + * During the first pass, the kmem_alloc_* caches + * are treated as metadata. + */ + kmem_default_arena = kmem_msb_arena; + maxbuf = KMEM_BIG_MAXBUF_32BIT; + } + + /* + * Set up the default caches to back kmem_alloc() + */ + kmem_alloc_caches_create( + kmem_alloc_sizes, sizeof (kmem_alloc_sizes) / sizeof (int), + kmem_alloc_table, KMEM_MAXBUF, KMEM_ALIGN_SHIFT); + + kmem_alloc_caches_create( + kmem_big_alloc_sizes, sizeof (kmem_big_alloc_sizes) / sizeof (int), + kmem_big_alloc_table, maxbuf, KMEM_BIG_SHIFT); + + kmem_big_alloc_table_max = maxbuf >> KMEM_BIG_SHIFT; +} + +struct free_slab { + vmem_t *vmp; + size_t slabsize; + void *slab; + list_node_t next; +}; + +static list_t freelist; + + +void +kmem_cache_build_slablist(kmem_cache_t *cp) +{ + int cpu_seqid; + + vmem_t *vmp = cp->cache_arena; + kmem_slab_t *sp; + struct free_slab *fs; + + for (sp = list_head(&cp->cache_complete_slabs); sp != NULL; + sp = list_next(&cp->cache_complete_slabs, sp)) { + + MALLOC(fs, struct free_slab *, sizeof (struct free_slab), + M_TEMP, M_WAITOK); + fs->vmp = vmp; + fs->slabsize = cp->cache_slabsize; + fs->slab = (void *)P2ALIGN((uintptr_t)sp->slab_base, + vmp->vm_quantum); + list_link_init(&fs->next); + list_insert_tail(&freelist, fs); + } + + for (sp = avl_first(&cp->cache_partial_slabs); sp != NULL; + sp = AVL_NEXT(&cp->cache_partial_slabs, sp)) { + + MALLOC(fs, struct free_slab *, sizeof (struct free_slab), + M_TEMP, M_WAITOK); + fs->vmp = vmp; + fs->slabsize = cp->cache_slabsize; + fs->slab = (void *)P2ALIGN((uintptr_t)sp->slab_base, + vmp->vm_quantum); + list_link_init(&fs->next); + list_insert_tail(&freelist, fs); + } + + + kstat_delete(cp->cache_kstat); + + if (cp->cache_hash_table != NULL) + vmem_free_impl(kmem_hash_arena, cp->cache_hash_table, + (cp->cache_hash_mask + 1) * sizeof (void *)); + + for (cpu_seqid = 0; cpu_seqid < max_ncpus; cpu_seqid++) + mutex_destroy(&cp->cache_cpu[cpu_seqid].cc_lock); + + mutex_destroy(&cp->cache_depot_lock); + mutex_destroy(&cp->cache_lock); + + vmem_free_impl(kmem_cache_arena, cp, KMEM_CACHE_SIZE(max_ncpus)); +} + + +static void +kmem_cache_fini() +{ + kmem_cache_t *cp; + int i; + struct free_slab *fs; + + list_create(&freelist, sizeof (struct free_slab), + offsetof(struct free_slab, next)); + + mutex_enter(&kmem_cache_lock); + + while ((cp = list_head(&kmem_caches))) { + list_remove(&kmem_caches, cp); + mutex_exit(&kmem_cache_lock); + kmem_cache_build_slablist(cp); + mutex_enter(&kmem_cache_lock); + } + + mutex_exit(&kmem_cache_lock); + + i = 0; + while ((fs = list_head(&freelist))) { + i++; + list_remove(&freelist, fs); + vmem_free_impl(fs->vmp, fs->slab, fs->slabsize); + FREE(fs, M_TEMP); + + } + xprintf("SPL: Released %u slabs\n", i); + list_destroy(&freelist); +} + +// this is intended to substitute for kmem_avail() in arc.c +int64_t +spl_free_wrapper(void) +{ + return (spl_free); +} + +// this is intended to substitute for kmem_avail() in arc.c +// when arc_reclaim_thread() calls spl_free_set_pressure(0); +int64_t +spl_free_manual_pressure_wrapper(void) +{ + return (spl_free_manual_pressure); +} + +uint64_t +spl_free_last_pressure_wrapper(void) +{ + return (spl_free_last_pressure); +} + +int64_t +spl_free_set_and_wait_pressure(int64_t new_p, boolean_t fast, + clock_t check_interval) +{ + + int64_t snapshot_pressure = 0; + + if (new_p <= 0) + return (0); + + spl_free_fast_pressure = fast; + + if (spl_free_manual_pressure >= 0) + spl_free_manual_pressure += new_p; + else + spl_free_manual_pressure = new_p; + + // wait for another thread to reset pressure + const uint64_t start = zfs_lbolt(); + const uint64_t end_by = start + (hz*60); + const uint64_t double_at = start + (hz/2); + const uint64_t double_again_at = start + hz; + bool doubled = false, doubled_again = false; + uint64_t now; + + spl_free_last_pressure = start; + + for (; spl_free_manual_pressure != 0; ) { + // has another thread set spl_free_manual_pressure? + if (spl_free_manual_pressure < new_p) + spl_free_manual_pressure = new_p; + snapshot_pressure = spl_free_manual_pressure; + mutex_enter(&spl_free_thread_lock); + cv_timedwait_hires(&spl_free_thread_cv, + &spl_free_thread_lock, check_interval, 0, 0); + mutex_exit(&spl_free_thread_lock); + now = zfs_lbolt(); + if (now > end_by) { + TraceEvent(TRACE_ERROR, "%s: ERROR: timed out " + "after one minute!\n", __func__); + break; + } else if (now > double_again_at && !doubled_again) { + doubled_again = true; + new_p *= 2; + } else if (now > double_at) { + doubled = true; + new_p *= 2; + } + } + return (snapshot_pressure); +} + +// routinely called by arc_reclaim_thread() with new_p == 0 +void +spl_free_set_pressure(int64_t new_p) +{ + if (new_p > spl_free_manual_pressure || new_p <= 0) + spl_free_manual_pressure = new_p; + if (new_p == 0) { + spl_free_fast_pressure = FALSE; + // wake up both spl_free_thread() to recalculate spl_free + // and any spl_free_set_and_wait_pressure() threads + cv_broadcast(&spl_free_thread_cv); + } + spl_free_last_pressure = zfs_lbolt(); +} + +void +spl_free_set_pressure_both(int64_t new_p, boolean_t fast) +{ + spl_free_fast_pressure = fast; + if (new_p > spl_free_manual_pressure || new_p <= 0) + spl_free_manual_pressure = new_p; + spl_free_last_pressure = zfs_lbolt(); +} + +void spl_free_maybe_reap(void); + +void +spl_free_set_emergency_pressure(int64_t new_p) +{ + spl_free_fast_pressure = TRUE; + if (new_p > spl_free_manual_pressure || new_p <= 0) + spl_free_manual_pressure = new_p; + spl_free_maybe_reap(); + spl_free_last_pressure = zfs_lbolt(); +} + +void +spl_free_set_emergency_pressure_additive(int64_t new_p) +{ + spl_free_fast_pressure = TRUE; + spl_free_manual_pressure += new_p; + spl_free_last_pressure = zfs_lbolt(); +} + +void +spl_free_set_pressure_additive(int64_t new_p) +{ + spl_free_manual_pressure += new_p; + spl_free_last_pressure = zfs_lbolt(); +} + +boolean_t +spl_free_fast_pressure_wrapper() +{ + return (spl_free_fast_pressure); +} + +void +spl_free_set_fast_pressure(boolean_t state) +{ + spl_free_fast_pressure = state; + spl_free_last_pressure = zfs_lbolt(); +} + +void +spl_free_reap_caches(void) +{ + // note: this may take some time + static hrtime_t last_reap = 0; + const hrtime_t reap_after = SEC2NSEC(60); + const hrtime_t curtime = gethrtime(); + + if (curtime - last_reap < reap_after) + return; + + kmem_reap(); + vmem_qcache_reap(kmem_va_arena); +} + +void +spl_free_maybe_reap(void) +{ + static _Atomic uint64_t last_reap = 0; + const uint64_t lockout_time = 60 * hz; + + uint64_t now = zfs_lbolt(); + if (now > last_reap + lockout_time) { + last_reap = now; + spl_free_maybe_reap_flag = true; + } +} + +boolean_t +spl_maybe_send_large_pressure(uint64_t now, uint64_t minutes, boolean_t full) +{ + static volatile _Atomic uint64_t spl_last_large_pressure = 0; + const uint64_t interval_ticks = minutes * 60ULL * (uint64_t)hz; + + if (spl_last_large_pressure + interval_ticks > now) + return (false); + + spl_last_large_pressure = now; + + const int64_t sixteenth_physmem = (int64_t)real_total_memory / 16LL; + const int64_t sixtyfourth_physmem = sixteenth_physmem / 4LL; + int64_t howmuch = sixteenth_physmem; + + if (full == false) + howmuch = sixtyfourth_physmem; + + + dprintf("SPL: %s: %lld bytes at time %llu\n", + __func__, howmuch, now); + + spl_free_set_emergency_pressure(howmuch); + + return (true); +} + +static void +spl_free_thread() +{ + callb_cpr_t cpr; + uint64_t last_update = zfs_lbolt(); + int64_t last_spl_free; + double ema_new = 0; + double ema_old = 0; + double alpha; + + CALLB_CPR_INIT(&cpr, &spl_free_thread_lock, callb_generic_cpr, FTAG); + + /* initialize with a reasonably large amount of memory */ + spl_free = MAX(4*1024*1024*1024, + total_memory * 75ULL / 100ULL); + + mutex_enter(&spl_free_thread_lock); + + dprintf("SPL: beginning spl_free_thread() loop, spl_free == %lld\n", + spl_free); + + uint64_t recent_lowmem = 0; + uint64_t last_disequilibrium = 0; + + while (!spl_free_thread_exit) { + mutex_exit(&spl_free_thread_lock); + boolean_t lowmem = false; + boolean_t emergency_lowmem = false; + int64_t base; + int64_t new_spl_free = 0LL; + + spl_stats.spl_free_wake_count.value.ui64++; + + if (spl_free_maybe_reap_flag == true) { + spl_free_maybe_reap_flag = false; + spl_free_reap_caches(); + } + + uint64_t time_now = zfs_lbolt(); + uint64_t time_now_seconds = 0; + if (time_now > hz) + time_now_seconds = time_now / hz; + + last_spl_free = spl_free; + + new_spl_free = total_memory - + segkmem_total_mem_allocated; + + /* Ask Mach about pressure */ + + /* + * Don't wait for pressure, just report back + * how much has changed while we were asleep + * (cf. the duration of cv_timedwait_hires + * at justwait: below -- 10 msec is an eternity + * on most hardware, and the osfmk/vm/vm_pageout.c + * code is linear in the nsecs_wanted parameter + * + */ + + // uint32_t pages_reclaimed = 0; + // uint32_t pages_wanted = 0; + +/* get pressure here */ + + if (spl_vm_pressure_level > 0 && + spl_vm_pressure_level != MAGIC_PRESSURE_UNAVAILABLE) { + /* there is pressure */ + lowmem = true; + new_spl_free = -(2LL * PAGE_SIZE * spl_vm_pages_wanted); + if (spl_vm_pressure_level > 1) { + emergency_lowmem = true; + if (new_spl_free > 0) + new_spl_free = -(4LL * + PAGE_SIZE * + spl_vm_pages_wanted); + spl_free_fast_pressure = TRUE; + } + spl_free_manual_pressure += PAGE_SIZE * + spl_vm_pages_wanted; + } else if (spl_vm_pages_wanted > 0) { + /* + * kVMPressureNormal but pages wanted : react more + * strongly if there was some transient pressure that + * was weakly absorbed by the virtual memory system + */ + /* XXX : additional hysteresis maintained below */ + int64_t m = 2LL; + if (spl_vm_pages_wanted * 8 > + spl_vm_pages_reclaimed) + m = 8LL; + + new_spl_free -= m * + PAGE_SIZE * spl_vm_pages_wanted; + } else { + /* + * No pressure. Xnu has freed up some memory + * which we can use. + */ + if (spl_vm_pages_reclaimed > 0) + new_spl_free += (PAGE_SIZE * + spl_vm_pages_reclaimed) >> 1LL; + else { + /* grow a little every pressure-free pass */ + new_spl_free += 1024LL*1024LL; + } + /* + * Cap, bearing in mind that we deflate + * total_memory by 50% at initialization + */ + if (new_spl_free > total_memory) + new_spl_free = total_memory; + } + + /* + * if there is pressure that has not yet reached + * arc_reclaim_thread() then start with a negative + * new_spl_free + */ + if (spl_free_manual_pressure > 0) { + int64_t old_pressure = spl_free_manual_pressure; + new_spl_free -= old_pressure * 2LL; + lowmem = true; + if (spl_free_fast_pressure) { + emergency_lowmem = true; + new_spl_free -= old_pressure * 4LL; + } + } + + /* + * can we allocate at least a 64 MiB segment + * from spl_heap_arena? this probes the reserve + * and also the largest imported spans, which + * vmem_alloc can fragment if needed. + */ + boolean_t reserve_low = false; + extern vmem_t *spl_heap_arena; + const uint64_t sixtyfour = 64ULL*1024ULL*1024ULL; + const uint64_t rvallones = (sixtyfour << 1ULL) - 1ULL; + const uint64_t rvmask = ~rvallones; + uint64_t rvfreebits = spl_heap_arena->vm_freemap; + + if ((rvfreebits & rvmask) == 0) { + reserve_low = true; + } else { + new_spl_free += (int64_t)sixtyfour; + } + + // do we have lots of memory in the spl_heap_arena ? + + boolean_t early_lots_free = false; + const uint64_t onetwentyeight = 128ULL*1024ULL*1024ULL; + const uint64_t sixteen = 16ULL*1024ULL*1024ULL; + if (!reserve_low) { + early_lots_free = true; + } else if (vmem_size_semi_atomic(spl_heap_arena, + VMEM_FREE) > onetwentyeight) { + early_lots_free = true; + new_spl_free += (int64_t)sixteen; + } + + // do we have lots of memory in the bucket_arenas ? + + extern int64_t vmem_buckets_size(int); // non-locking + int64_t buckets_free = vmem_buckets_size(VMEM_FREE); + if ((uint64_t)buckets_free != spl_buckets_mem_free) + spl_buckets_mem_free = (uint64_t)buckets_free; + + if (buckets_free >= 512LL*1024LL*1024LL) { + early_lots_free = true; + new_spl_free += (int64_t)sixteen; + } + if (buckets_free >= 1024LL*1024LL*1024LL) { + reserve_low = false; + new_spl_free += (int64_t)sixteen; + } + + /* + * if we have neither alloced or freed in + * several minutes, then we do not need to + * shrink back if there is a momentary transient + * memory spike (i.e., one that lasts less than a second) + */ + boolean_t memory_equilibrium = false; + const uint64_t five_minutes = 300ULL; + const uint64_t one_minute = 60ULL; + uint64_t last_xat_alloc_seconds = spl_xat_lastalloc; + uint64_t last_xat_free_seconds = spl_xat_lastfree; + + if (last_xat_alloc_seconds + five_minutes > time_now_seconds && + last_xat_free_seconds + five_minutes > time_now_seconds) { + if (last_disequilibrium + one_minute > + time_now_seconds) { + memory_equilibrium = true; + last_disequilibrium = 0; + } + } else { + last_disequilibrium = time_now_seconds; + } + + boolean_t just_alloced = false; + if (last_xat_alloc_seconds + 1 > time_now_seconds) + just_alloced = true; + + /* + * this is a sign of a period of time of low system + * memory, however XNU's generation of this variable + * is not very predictable, but generally it should be + * taken seriously when it's positive (it is often falsely 0) + */ + if ((spl_vm_pages_wanted > 0 && reserve_low && + !early_lots_free && !memory_equilibrium && + !just_alloced) || spl_vm_pages_wanted >= 1024) { + int64_t bminus = (int64_t)spl_vm_pages_wanted * + (int64_t)PAGESIZE * -16LL; + if (bminus > -16LL*1024LL*1024LL) + bminus = -16LL*1024LL*1024LL; + new_spl_free += bminus; + lowmem = true; + emergency_lowmem = true; + // atomic swaps to set these variables used in arc.c + int64_t previous_highest_pressure = 0; + int64_t new_p = -bminus; + previous_highest_pressure = spl_free_manual_pressure; + if (new_p > previous_highest_pressure || new_p <= 0) { + boolean_t fast = FALSE; + if (spl_vm_pages_wanted > + spl_vm_page_free_min / 8) + fast = TRUE; + spl_free_set_pressure_both(-16LL * new_spl_free, + fast); + } + last_disequilibrium = time_now_seconds; + } else if (spl_vm_pages_wanted > 0) { + int64_t bytes_wanted = + (int64_t)spl_vm_pages_wanted * + (int64_t)PAGESIZE; + new_spl_free -= bytes_wanted; + if (reserve_low && !early_lots_free) { + lowmem = true; + if (recent_lowmem == 0) { + recent_lowmem = time_now; + } + if (!memory_equilibrium) { + last_disequilibrium = time_now_seconds; + } + } + } + + /* + * If we have already detected a memory shortage + * and we have not reaped in a while (a short while + * for emergency_lowmem), then do a kmem_reap() now. + * See http://comments.gmane.org/gmane.os.illumos.devel/22552 + * (notably Richard Elling's "A kernel module can call + * kmem_reap() whenever it wishes and some modules, + * like zfs, do so." If we reap, stop processing spl_free + * on this pass, to let the reaps (and arc, if pressure + * has been set above) do their job for a few milliseconds. + */ + if (emergency_lowmem || lowmem) { + static uint64_t last_reap = 0; + uint64_t now = time_now; + uint64_t elapsed = 60*hz; + if (emergency_lowmem) + elapsed = 15*hz; // min.freq. kmem_reap_interval + if (now - last_reap > elapsed) { + last_reap = now; + /* + * spl_free_reap_caches() calls functions + * that will acquire locks and can take a while + * so set spl_free to a small positive value + * to stop arc shrinking too much during this + * period when we expect to be freeing up + * arc-usable memory, but low enough that + * arc_no_grow likely will be set. + */ + const int64_t two_spamax = 32LL * 1024LL * + 1024LL; + if (spl_free < two_spamax) + spl_free = two_spamax; // atomic! + spl_free_reap_caches(); + // we do not have any lock now, so we can jump + // to just before the thread-suspending code + goto justwait; + } + } + + /* + * a number or exceptions to reverse the lowmem + * / emergency_lowmem states if we have recently reaped. + * we also take the strong reaction sting out of + * the set pressure by turning off spl_free_fast_pressure, + * since that automatically provokes an arc shrink + * and arc reap. + */ + + if (!reserve_low || early_lots_free || memory_equilibrium || + just_alloced) { + lowmem = false; + emergency_lowmem = false; + spl_free_fast_pressure = FALSE; + } + + /* + * Stay in a low memory condition for several seconds + * after we first detect that we are in it, giving the + * system (arc, xnu and userland) time to adapt + */ + if (!lowmem && recent_lowmem > 0) { + if (recent_lowmem + 4*hz < time_now) + lowmem = true; + else + recent_lowmem = 0; + } + + base = new_spl_free; + + // adjust for available memory in spl_heap_arena + // cf arc_available_memory() + if (!emergency_lowmem) { + extern vmem_t *spl_default_arena; + int64_t heap_free = (int64_t)vmem_size_semi_atomic( + spl_heap_arena, VMEM_FREE); + // grabbed buckets_free up above; we are OK with + // change to it in the meanwhile, + // it'll get an update on the next run. + int64_t combined_free = heap_free + buckets_free; + + if (combined_free != 0) { + const int64_t mb = 1024*1024; + if (!lowmem) { + new_spl_free += combined_free / + 4; + } else { + new_spl_free -= 16LL * mb; + } + } + + // memory footprint has gotten really big, + // decrease spl_free substantially + int64_t total_mem_used = (int64_t) + segkmem_total_mem_allocated; + if ((segkmem_total_mem_allocated * 100LL / + real_total_memory) > 70) { + new_spl_free -= total_mem_used / 64; + } else if ((segkmem_total_mem_allocated * 100LL / + real_total_memory) > 75) { + new_spl_free -= total_mem_used / 32; + lowmem = true; + } + } + + // try to get 1/64 of spl_heap_arena freed up + if (emergency_lowmem && new_spl_free >= 0LL) { + extern vmem_t *spl_root_arena; + uint64_t root_size = vmem_size_semi_atomic( + spl_heap_arena, VMEM_ALLOC | VMEM_FREE); + uint64_t root_free = vmem_size_semi_atomic( + spl_heap_arena, VMEM_FREE); + int64_t difference = root_size - root_free; + int64_t target = root_size / 64; + if (difference < target) { + new_spl_free -= target; + } + // and we should definitely not be returning + // positive now + if (new_spl_free >= 0LL) + new_spl_free = -1024LL; + } + + double delta = (double)new_spl_free - (double)last_spl_free; + + boolean_t spl_free_is_negative = false; + + if (new_spl_free < 0LL) { + spl_stats.spl_spl_free_negative_count.value.ui64++; + spl_free_is_negative = true; + } + + /* + * leave a little headroom if we have hit our + * allocation maximum + */ + const int64_t spamaxblksz = 16LL * 1024LL; + if ((4LL * spamaxblksz) > + (total_memory - segkmem_total_mem_allocated)) { + if (new_spl_free > 2LL * spamaxblksz) + new_spl_free = 2LL * spamaxblksz; + } + + // NOW set spl_free from calculated new_spl_free + spl_free = new_spl_free; + // the direct equivalent of : + // __c11_atomic_store(&spl_free, new_spl_free, + // __ATOMIC_SEQ_CST); + + /* + * Because we're already negative, arc is likely to have + * been signalled already. We can rely on the _maybe_ in + * spl-vmem.c:xnu_alloc_throttled() [XAT] to try to give + * arc a kick with greater probability. However, if we've + * gone negative several times, and have not tried a full + * kick in a long time, do so now; if the full kick is + * refused because there has been a kick too few minutes + * ago, try a gentler kick. We do this outside the lock, + * as spl_maybe_send_large_pressure may need to take a + * mutex, and we forbid further mutex entry when + * spl_free_lock is held. + */ + + if (spl_free_is_negative) { + static volatile _Atomic uint32_t + negatives_since_last_kick = 0; + + if (negatives_since_last_kick++ > 8) { + if (spl_maybe_send_large_pressure(time_now, 360, + true) || + spl_maybe_send_large_pressure(time_now, 60, + false)) { + negatives_since_last_kick = 0; + } + } + } + + if (lowmem) + recent_lowmem = time_now; + + // maintain an exponential moving average for the ema kstat + if (last_update > hz) + alpha = 1.0; + else { + double td_tick = (double)(time_now - last_update); + alpha = td_tick / (double)(hz*50.0); // roughly 0.02 + } + + ema_new = (alpha * delta) + (1.0 - alpha)*ema_old; + spl_free_delta_ema = ema_new; + ema_old = ema_new; + + justwait: + mutex_enter(&spl_free_thread_lock); + CALLB_CPR_SAFE_BEGIN(&cpr); + (void) cv_timedwait_hires(&spl_free_thread_cv, + &spl_free_thread_lock, MSEC2NSEC(10), 0, 0); + CALLB_CPR_SAFE_END(&cpr, &spl_free_thread_lock); + } + spl_free_thread_exit = FALSE; + dprintf("SPL: spl_free_thread_exit set to FALSE " \ + "and exiting: cv_broadcasting\n"); + spl_free_manual_pressure = 0; + cv_broadcast(&spl_free_thread_cv); + CALLB_CPR_EXIT(&cpr); + dprintf("SPL: %s thread_exit\n", __func__); + thread_exit(); +} + +static void +spl_event_thread(void *notused) +{ + // callb_cpr_t cpr; + NTSTATUS Status; + + DECLARE_CONST_UNICODE_STRING(low_mem_name, + L"\\KernelObjects\\LowMemoryCondition"); + HANDLE low_mem_handle; + low_mem_event = + IoCreateNotificationEvent((PUNICODE_STRING)&low_mem_name, + &low_mem_handle); + if (low_mem_event == NULL) { + TraceEvent(TRACE_ERROR, + "%s: failed IoCreateNotificationEvent(\\KernelObjects" + "\\LowMemoryCondition)", __func__); + thread_exit(); + } + KeClearEvent(low_mem_event); + + dprintf("SPL: beginning spl_event_thread() loop\n"); + + while (!spl_event_thread_exit) { + + /* Don't busy loop */ + delay(hz); + + /* Sleep forever waiting for event */ + Status = KeWaitForSingleObject(low_mem_event, Executive, + KernelMode, FALSE, NULL); + KeClearEvent(low_mem_event); + + dprintf("%s: LOWMEMORY EVENT *** 0x%x (memusage: %llu)\n", + __func__, Status, segkmem_total_mem_allocated); + /* We were signalled */ + // vm_page_free_wanted = vm_page_free_min; + spl_free_set_pressure(spl_vm_page_free_min); + cv_broadcast(&spl_free_thread_cv); + } + + ZwClose(low_mem_handle); + + spl_event_thread_exit = FALSE; + dprintf("SPL: %s thread_exit\n", __func__); + thread_exit(); +} + + +static int +spl_kstat_update(kstat_t *ksp, int rw) +{ + spl_stats_t *ks = ksp->ks_data; + + if (rw == KSTAT_WRITE) { + + if (ks->spl_spl_free_manual_pressure.value.i64 != + spl_free_manual_pressure) { + spl_free_set_pressure( + ks->spl_spl_free_manual_pressure.value.i64 * 1024 * + 1024); + if (ks->spl_spl_free_manual_pressure.value.i64 > 0) { + spl_free_reap_caches(); + } + } + + if (ks->spl_spl_free_fast_pressure.value.i64 != + spl_free_fast_pressure) { + if (spl_free_wrapper() != 0) { + spl_free_set_fast_pressure(TRUE); + } + } + + if (ks->spl_bucket_tunable_large_span.value.ui64 != + spl_bucket_tunable_large_span) { + spl_set_bucket_tunable_large_span( + ks->spl_bucket_tunable_large_span.value.ui64); + } + + if (ks->spl_bucket_tunable_small_span.value.ui64 != + spl_bucket_tunable_small_span) { + spl_set_bucket_tunable_small_span( + ks->spl_bucket_tunable_small_span.value.ui64); + } + + if (ks->spl_frag_max_walk.value.ui64 != spl_frag_max_walk) { + spl_frag_max_walk = ks->spl_frag_max_walk.value.ui64; + } + + if (ks->kmem_free_to_slab_when_fragmented.value.ui64 != + kmem_free_to_slab_when_fragmented) { + kmem_free_to_slab_when_fragmented = + ks->kmem_free_to_slab_when_fragmented.value.ui64; + } + + } else { + ks->spl_os_alloc.value.ui64 = segkmem_total_mem_allocated; + ks->spl_active_threads.value.ui64 = zfs_threads; + ks->spl_active_mutex.value.ui64 = zfs_active_mutex; + ks->spl_active_rwlock.value.ui64 = zfs_active_rwlock; + ks->spl_active_tsd.value.ui64 = spl_tsd_size(); + ks->spl_spl_free.value.i64 = spl_free; + ks->spl_spl_free_manual_pressure.value.i64 = + spl_free_manual_pressure; + ks->spl_spl_free_fast_pressure.value.i64 = + spl_free_fast_pressure; + ks->spl_spl_free_delta_ema.value.i64 = spl_free_delta_ema; + ks->spl_osif_malloc_success.value.ui64 = + stat_osif_malloc_success; + ks->spl_osif_malloc_bytes.value.ui64 = stat_osif_malloc_bytes; + ks->spl_osif_free.value.ui64 = stat_osif_free; + ks->spl_osif_free_bytes.value.ui64 = stat_osif_free_bytes; + ks->spl_bucket_non_pow2_allocs.value.ui64 = + spl_bucket_non_pow2_allocs; + + ks->spl_vmem_unconditional_allocs.value.ui64 = + spl_vmem_unconditional_allocs; + ks->spl_vmem_unconditional_alloc_bytes.value.ui64 = + spl_vmem_unconditional_alloc_bytes; + ks->spl_vmem_conditional_allocs.value.ui64 = + spl_vmem_conditional_allocs; + ks->spl_vmem_conditional_alloc_bytes.value.ui64 = + spl_vmem_conditional_alloc_bytes; + ks->spl_vmem_conditional_alloc_deny.value.ui64 = + spl_vmem_conditional_alloc_deny; + ks->spl_vmem_conditional_alloc_deny_bytes.value.ui64 = + spl_vmem_conditional_alloc_deny_bytes; + + ks->spl_xat_success.value.ui64 = spl_xat_success; + ks->spl_xat_late_success.value.ui64 = spl_xat_late_success; + ks->spl_xat_late_success_nosleep.value.ui64 = + spl_xat_late_success_nosleep; + ks->spl_xat_pressured.value.ui64 = spl_xat_pressured; + ks->spl_xat_bailed.value.ui64 = spl_xat_bailed; + ks->spl_xat_bailed_contended.value.ui64 = + spl_xat_bailed_contended; + ks->spl_xat_lastalloc.value.ui64 = spl_xat_lastalloc; + ks->spl_xat_lastfree.value.ui64 = spl_xat_lastfree; + ks->spl_xat_forced.value.ui64 = spl_xat_forced; + ks->spl_xat_sleep.value.ui64 = spl_xat_sleep; + ks->spl_xat_late_deny.value.ui64 = spl_xat_late_deny; + ks->spl_xat_no_waiters.value.ui64 = spl_xat_no_waiters; + ks->spl_xft_wait.value.ui64 = spl_xft_wait; + + ks->spl_vba_parent_memory_appeared.value.ui64 = + spl_vba_parent_memory_appeared; + ks->spl_vba_parent_memory_blocked.value.ui64 = + spl_vba_parent_memory_blocked; + ks->spl_vba_hiprio_blocked.value.ui64 = spl_vba_hiprio_blocked; + ks->spl_vba_cv_timeout.value.ui64 = spl_vba_cv_timeout; + ks->spl_vba_loop_timeout.value.ui64 = spl_vba_loop_timeout; + ks->spl_vba_cv_timeout_blocked.value.ui64 = + spl_vba_cv_timeout_blocked; + ks->spl_vba_loop_timeout_blocked.value.ui64 = + spl_vba_loop_timeout_blocked; + ks->spl_vba_sleep.value.ui64 = spl_vba_sleep; + ks->spl_vba_loop_entries.value.ui64 = spl_vba_loop_entries; + + ks->spl_bucket_tunable_large_span.value.ui64 = + spl_bucket_tunable_large_span; + ks->spl_bucket_tunable_small_span.value.ui64 = + spl_bucket_tunable_small_span; + + ks->spl_buckets_mem_free.value.ui64 = spl_buckets_mem_free; + ks->spl_arc_no_grow_bits.value.ui64 = spl_arc_no_grow_bits; + ks->spl_arc_no_grow_count.value.ui64 = spl_arc_no_grow_count; + + ks->spl_frag_max_walk.value.ui64 = spl_frag_max_walk; + ks->spl_frag_walked_out.value.ui64 = spl_frag_walked_out; + ks->spl_frag_walk_cnt.value.ui64 = spl_frag_walk_cnt; + + ks->spl_arc_reclaim_avoided.value.ui64 = + spl_arc_reclaim_avoided; + + ks->kmem_free_to_slab_when_fragmented.value.ui64 = + kmem_free_to_slab_when_fragmented; + + ks->spl_vm_pages_reclaimed.value.ui64 = + spl_vm_pages_reclaimed; + ks->spl_vm_pages_wanted.value.ui64 = + spl_vm_pages_wanted; + ks->spl_vm_pressure_level.value.ui64 = + spl_vm_pressure_level; + + ks->spl_lowest_alloc_stack_remaining.value.ui64 = + spl_lowest_alloc_stack_remaining; + ks->spl_lowest_vdev_disk_stack_remaining.value.ui64 = + spl_lowest_vdev_disk_stack_remaining; + ks->spl_lowest_zvol_stack_remaining.value.ui64 = + spl_lowest_zvol_stack_remaining; + } + + return (0); +} + +void +spl_kmem_init(uint64_t xtotal_memory) +{ + int old_kmem_flags = kmem_flags; + int use_large_pages = 0; + size_t maxverify, minfirewall; + + dprintf("SPL: KMEM starting. Total memory %llu\n", xtotal_memory); + + // Initialise the kstat lock + mutex_init(&kmem_cache_lock, "kmem_cache_lock", MUTEX_DEFAULT, NULL); + mutex_init(&kmem_flags_lock, "kmem_flags_lock", MUTEX_DEFAULT, NULL); + mutex_init(&kmem_cache_kstat_lock, "kmem_kstat_lock", MUTEX_DEFAULT, + NULL); + + spl_kstat_init(); + + + /* + * Small-memory systems (< 24 MB) can't handle kmem_flags overhead. + */ + if (physmem < btop(24 << 20) && !(old_kmem_flags & KMF_STICKY)) + kmem_flags = 0; + + /* + * Don't do firewalled allocations if the heap is less than 1TB + * (i.e. on a 32-bit kernel) + * The resulting VM_NEXTFIT allocations would create too much + * fragmentation in a small heap. + */ + maxverify = minfirewall = PAGESIZE / 2; + + + /* LINTED */ + ASSERT(sizeof (kmem_cpu_cache_t) == KMEM_CPU_CACHE_SIZE); + + list_create(&kmem_caches, sizeof (kmem_cache_t), + offsetof(kmem_cache_t, cache_link)); + + kernelheap_init(); + + kmem_metadata_arena = vmem_create("kmem_metadata", NULL, 0, PAGESIZE, + vmem_alloc_impl, vmem_free_impl, heap_arena, 8 * PAGESIZE, + VM_SLEEP | VMC_NO_QCACHE); + + kmem_msb_arena = vmem_create("kmem_msb", NULL, 0, + PAGESIZE, vmem_alloc_impl, vmem_free_impl, kmem_metadata_arena, 0, + VMC_DUMPSAFE | VM_SLEEP); + + kmem_cache_arena = vmem_create("kmem_cache", NULL, 0, KMEM_ALIGN, + vmem_alloc_impl, vmem_free_impl, kmem_metadata_arena, 0, VM_SLEEP); + + kmem_hash_arena = vmem_create("kmem_hash", NULL, 0, KMEM_ALIGN, + vmem_alloc_impl, vmem_free_impl, kmem_metadata_arena, 0, VM_SLEEP); + + kmem_log_arena = vmem_create("kmem_log", NULL, 0, KMEM_ALIGN, + vmem_alloc_impl, vmem_free_impl, kmem_metadata_arena, 0, VM_SLEEP); + + /* temporary oversize arena for mod_read_system_file */ + kmem_oversize_arena = vmem_create("kmem_oversize", NULL, 0, PAGESIZE, + vmem_alloc_impl, vmem_free_impl, heap_arena, 0, VM_SLEEP); + + // statically declared above kmem_reap_interval = 15 * hz; + + /* + * Read /etc/system. This is a chicken-and-egg problem because + * kmem_flags may be set in /etc/system, but mod_read_system_file() + * needs to use the allocator. The simplest solution is to create + * all the standard kmem caches, read /etc/system, destroy all the + * caches we just created, and then create them all again in light + * of the (possibly) new kmem_flags and other kmem tunables. + */ + + if (old_kmem_flags & KMF_STICKY) + kmem_flags = old_kmem_flags; + + if (!(kmem_flags & KMF_AUDIT)) + vmem_seg_size = offsetof(vmem_seg_t, vs_thread); + + if (kmem_maxverify == 0) + kmem_maxverify = maxverify; + + if (kmem_minfirewall == 0) + kmem_minfirewall = minfirewall; + + /* + * give segkmem a chance to figure out if we are using large pages + * for the kernel heap + */ + // use_large_pages = segkmem_lpsetup(); + use_large_pages = 0; + + /* + * To protect against corruption, we keep the actual number of callers + * KMF_LITE records seperate from the tunable. We arbitrarily clamp + * to 16, since the overhead for small buffers quickly gets out of + * hand. + * + * The real limit would depend on the needs of the largest KMC_NOHASH + * cache. + */ + kmem_lite_count = MIN(MAX(0, kmem_lite_pcs), 16); + kmem_lite_pcs = kmem_lite_count; + + kmem_cache_init(2, use_large_pages); + + if (kmem_flags & (KMF_AUDIT | KMF_RANDOMIZE)) { + if (kmem_transaction_log_size == 0) + kmem_transaction_log_size = MIN(kmem_maxavail() / 50ULL, + PAGESIZE<<4); + kmem_transaction_log = kmem_log_init(kmem_transaction_log_size); + } + + if (kmem_flags & (KMF_CONTENTS | KMF_RANDOMIZE)) { + if (kmem_content_log_size == 0) + kmem_content_log_size = MIN(kmem_maxavail() / 50ULL, + PAGESIZE<<4); + kmem_content_log = kmem_log_init(kmem_content_log_size); + } + + kmem_failure_log = kmem_log_init(kmem_failure_log_size); + + kmem_slab_log = kmem_log_init(kmem_slab_log_size); + + spl_tsd_init(); + spl_rwlock_init(); + spl_taskq_init(); + + /* + * Warn about invalid or dangerous values of kmem_flags. + * Always warn about unsupported values. + */ + if (((kmem_flags & ~(KMF_AUDIT | KMF_DEADBEEF | KMF_REDZONE | + KMF_CONTENTS | KMF_LITE)) != 0) || + ((kmem_flags & KMF_LITE) && kmem_flags != KMF_LITE)) + cmn_err(CE_WARN, "kmem_flags set to unsupported value 0x%x. " + "See the Solaris Tunable Parameters Reference Manual.", + kmem_flags); + +#ifdef DEBUG + if ((kmem_flags & KMF_DEBUG) == 0) + cmn_err(CE_NOTE, "kmem debugging disabled."); +#else + /* + * For non-debug kernels, the only "normal" flags are 0, KMF_LITE, + * KMF_REDZONE, and KMF_CONTENTS (the last because it is only enabled + * if KMF_AUDIT is set). We should warn the user about the performance + * penalty of KMF_AUDIT or KMF_DEADBEEF if they are set and KMF_LITE + * isn't set (since that disables AUDIT). + */ + if (!(kmem_flags & KMF_LITE) && + (kmem_flags & (KMF_AUDIT | KMF_DEADBEEF)) != 0) + cmn_err(CE_WARN, "High-overhead kmem debugging features " + "enabled (kmem_flags = 0x%x). Performance degradation " + "and large memory overhead possible. See the Solaris " + "Tunable Parameters Reference Manual.", kmem_flags); +#endif /* not DEBUG */ + + segkmem_abd_init(); + + kmem_cache_applyall(kmem_cache_magazine_enable, NULL, TQ_SLEEP); + + kmem_ready = 1; + + // Install spl kstats + spl_ksp = kstat_create("spl", 0, "spl_misc", "misc", KSTAT_TYPE_NAMED, + sizeof (spl_stats) / sizeof (kstat_named_t), + KSTAT_FLAG_VIRTUAL|KSTAT_FLAG_WRITABLE); + + if (spl_ksp != NULL) { + spl_ksp->ks_data = &spl_stats; + spl_ksp->ks_update = spl_kstat_update; + kstat_install(spl_ksp); + } +} + +void +spl_kmem_fini(void) +{ + + kmem_cache_applyall(kmem_cache_magazine_disable, NULL, TQ_SLEEP); + + kstat_delete(spl_ksp); + + kmem_log_fini(kmem_slab_log); + kmem_log_fini(kmem_failure_log); + + if (kmem_flags & (KMF_CONTENTS | KMF_RANDOMIZE)) { + if (kmem_content_log_size == 0) + kmem_content_log_size = kmem_maxavail() / 50; + kmem_log_fini(kmem_content_log); + } + + if (kmem_flags & (KMF_AUDIT | KMF_RANDOMIZE)) { + if (kmem_transaction_log_size == 0) + kmem_transaction_log_size = kmem_maxavail() / 50; + kmem_log_fini(kmem_transaction_log); + } + + // Destroy all the "general allocation" caches + kmem_alloc_caches_destroy(); + + // Destroy the VA associated caches + kmem_destroy_cache_by_name(KMEM_VA_PREFIX); + + kmem_qcache_destroy(); + // Destroy metadata caches + kmem_cache_destroy(kmem_bufctl_cache); + kmem_cache_destroy(kmem_bufctl_audit_cache); + kmem_cache_destroy(kmem_slab_cache); // Dont think this one + + // Some caches cannot be destroyed as + // they mutually reference each other. + // So we explicitly pull them apart piece-by-piece. + kmem_cache_fini(); + + segkmem_abd_fini(); + + // Now destroy the vmem arenas used by kmem. + vmem_destroy(kmem_default_arena); + vmem_destroy(kmem_va_arena); + vmem_destroy(kmem_oversize_arena); + vmem_destroy(kmem_log_arena); + vmem_destroy(kmem_hash_arena); + vmem_destroy(kmem_cache_arena); + vmem_destroy(kmem_msb_arena); + vmem_destroy(kmem_metadata_arena); + + kernelheap_fini(); + + list_destroy(&kmem_caches); + + mutex_destroy(&kmem_cache_kstat_lock); + mutex_destroy(&kmem_flags_lock); + mutex_destroy(&kmem_cache_lock); +} + +static void +kmem_move_init(void) +{ + kmem_defrag_cache = kmem_cache_create("kmem_defrag_cache", + sizeof (kmem_defrag_t), 0, NULL, NULL, NULL, NULL, + kmem_msb_arena, KMC_NOHASH); + kmem_move_cache = kmem_cache_create("kmem_move_cache", + sizeof (kmem_move_t), 0, NULL, NULL, NULL, NULL, + kmem_msb_arena, KMC_NOHASH); + + /* + * kmem guarantees that move callbacks are sequential and that even + * across multiple caches no two moves ever execute simultaneously. + * Move callbacks are processed on a separate taskq so that client code + * does not interfere with internal maintenance tasks. + */ + kmem_move_taskq = taskq_create("kmem_move_taskq", 1, + minclsyspri, 100, INT_MAX, TASKQ_PREPOPULATE); +} + +void +kmem_move_fini(void) +{ + + taskq_wait(kmem_move_taskq); + taskq_destroy(kmem_move_taskq); + kmem_move_taskq = 0; + + kmem_cache_destroy(kmem_move_cache); + kmem_cache_destroy(kmem_defrag_cache); + +} + +void +spl_kmem_thread_init(void) +{ + kmem_move_init(); + + // Initialize the spl_free locks + mutex_init(&spl_free_thread_lock, "spl_free_thead_lock", MUTEX_DEFAULT, + NULL); + + kmem_taskq = taskq_create("kmem_taskq", 1, minclsyspri, + 600, INT_MAX, TASKQ_PREPOPULATE); + + spl_free_thread_exit = FALSE; + (void) cv_init(&spl_free_thread_cv, NULL, CV_DEFAULT, NULL); + (void) thread_create(NULL, 0, spl_free_thread, 0, 0, 0, 0, 92); + + spl_event_thread_exit = FALSE; + (void) thread_create(NULL, 0, spl_event_thread, 0, 0, 0, 0, 92); +} + +void +spl_kmem_thread_fini(void) +{ + shutting_down = 1; + + if (low_mem_event != NULL) { + dprintf("SPL: stopping spl_event_thread\n"); + spl_event_thread_exit = TRUE; + KeSetEvent(low_mem_event, 0, FALSE); + while (spl_event_thread_exit) { + delay(hz >> 4); + } + dprintf("SPL: stopped spl_event_thread\n"); + } + + mutex_enter(&spl_free_thread_lock); + spl_free_thread_exit = TRUE; + while (spl_free_thread_exit) { + cv_signal(&spl_free_thread_cv); + cv_wait(&spl_free_thread_cv, &spl_free_thread_lock); + } + mutex_exit(&spl_free_thread_lock); + cv_destroy(&spl_free_thread_cv); + mutex_destroy(&spl_free_thread_lock); + + bsd_untimeout(kmem_update, &kmem_update_timer); + bsd_untimeout(kmem_reap_timeout, &kmem_reaping); + bsd_untimeout(kmem_reap_timeout, &kmem_reaping_idspace); + + taskq_wait(kmem_taskq); + + taskq_destroy(kmem_taskq); + kmem_taskq = 0; + + kmem_move_fini(); + +} + +void +spl_kmem_timer_fini(void) +{ + bsd_timeout_cancel(&kmem_update_timer); + bsd_timeout_cancel(&kmem_reaping); + bsd_timeout_cancel(&kmem_reaping_idspace); +} + +void +spl_kmem_mp_init(void) +{ + kmem_update_timeout(&kmem_update_timer); +} + +/* + * Return the slab of the allocated buffer, or NULL if the buffer is not + * allocated. This function may be called with a known slab address to determine + * whether or not the buffer is allocated, or with a NULL slab address to obtain + * an allocated buffer's slab. + */ +static kmem_slab_t * +kmem_slab_allocated(kmem_cache_t *cp, kmem_slab_t *sp, void *buf) +{ + kmem_bufctl_t *bcp, *bufbcp; + + ASSERT(MUTEX_HELD(&cp->cache_lock)); + ASSERT(sp == NULL || KMEM_SLAB_MEMBER(sp, buf)); + + if (cp->cache_flags & KMF_HASH) { + for (bcp = *KMEM_HASH(cp, buf); + (bcp != NULL) && (bcp->bc_addr != buf); + bcp = bcp->bc_next) { + continue; + } + ASSERT(sp != NULL && bcp != NULL ? sp == bcp->bc_slab : 1); + return (bcp == NULL ? NULL : bcp->bc_slab); + } + + if (sp == NULL) { + sp = KMEM_SLAB(cp, buf); + } + bufbcp = KMEM_BUFCTL(cp, buf); + for (bcp = sp->slab_head; + (bcp != NULL) && (bcp != bufbcp); + bcp = bcp->bc_next) { + continue; + } + return (bcp == NULL ? sp : NULL); +} + +static boolean_t +kmem_slab_is_reclaimable(kmem_cache_t *cp, kmem_slab_t *sp, int flags) +{ + long refcnt = sp->slab_refcnt; + + ASSERT(cp->cache_defrag != NULL); + + /* + * For code coverage we want to be able to move an object within the + * same slab (the only partial slab) even if allocating the destination + * buffer resulted in a completely allocated slab. + */ + if (flags & KMM_DEBUG) { + return ((flags & KMM_DESPERATE) || + ((sp->slab_flags & KMEM_SLAB_NOMOVE) == 0)); + } + + /* If we're desperate, we don't care if the client said NO. */ + if (flags & KMM_DESPERATE) { + return (refcnt < sp->slab_chunks); /* any partial */ + } + + if (sp->slab_flags & KMEM_SLAB_NOMOVE) { + return (B_FALSE); + } + + if ((refcnt == 1) || kmem_move_any_partial) { + return (refcnt < sp->slab_chunks); + } + + /* + * The reclaim threshold is adjusted at each kmem_cache_scan() so that + * slabs with a progressively higher percentage of used buffers can be + * reclaimed until the cache as a whole is no longer fragmented. + * + * sp->slab_refcnt kmd_reclaim_numer + * --------------- < ------------------ + * sp->slab_chunks KMEM_VOID_FRACTION + */ + return ((refcnt * KMEM_VOID_FRACTION) < + (sp->slab_chunks * cp->cache_defrag->kmd_reclaim_numer)); +} + +/* + * May be called from the kmem_move_taskq, from kmem_cache_move_notify_task(), + * or when the buffer is freed. + */ +static void +kmem_slab_move_yes(kmem_cache_t *cp, kmem_slab_t *sp, void *from_buf) +{ + ASSERT(MUTEX_HELD(&cp->cache_lock)); + ASSERT(KMEM_SLAB_MEMBER(sp, from_buf)); + + if (!KMEM_SLAB_IS_PARTIAL(sp)) { + return; + } + + if (sp->slab_flags & KMEM_SLAB_NOMOVE) { + if (KMEM_SLAB_OFFSET(sp, from_buf) == sp->slab_stuck_offset) { + avl_remove(&cp->cache_partial_slabs, sp); + sp->slab_flags &= ~KMEM_SLAB_NOMOVE; + sp->slab_stuck_offset = (uint32_t)-1; + avl_add(&cp->cache_partial_slabs, sp); + } + } else { + sp->slab_later_count = 0; + sp->slab_stuck_offset = (uint32_t)-1; + } +} + +static void +kmem_slab_move_no(kmem_cache_t *cp, kmem_slab_t *sp, void *from_buf) +{ + ASSERT(taskq_member(kmem_move_taskq, curthread)); + ASSERT(MUTEX_HELD(&cp->cache_lock)); + ASSERT(KMEM_SLAB_MEMBER(sp, from_buf)); + + if (!KMEM_SLAB_IS_PARTIAL(sp)) { + return; + } + + avl_remove(&cp->cache_partial_slabs, sp); + sp->slab_later_count = 0; + sp->slab_flags |= KMEM_SLAB_NOMOVE; + sp->slab_stuck_offset = KMEM_SLAB_OFFSET(sp, from_buf); + avl_add(&cp->cache_partial_slabs, sp); +} + +static void kmem_move_end(kmem_cache_t *, kmem_move_t *); + +/* + * The move callback takes two buffer addresses, the buffer to be moved, and a + * newly allocated and constructed buffer selected by kmem as the destination. + * It also takes the size of the buffer and an optional user argument specified + * at cache creation time. kmem guarantees that the buffer to be moved has not + * been unmapped by the virtual memory subsystem. Beyond that, it cannot + * guarantee the present whereabouts of the buffer to be moved, so it is up to + * the client to safely determine whether or not it is still using the buffer. + * The client must not free either of the buffers passed to the move callback, + * since kmem wants to free them directly to the slab layer. The client response + * tells kmem which of the two buffers to free: + * + * YES kmem frees the old buffer (the move was successful) + * NO kmem frees the new buffer, marks the slab of the old buffer + * non-reclaimable to avoid bothering the client again + * LATER kmem frees the new buffer, increments slab_later_count + * DONT_KNOW kmem frees the new buffer + * DONT_NEED kmem frees both the old buffer and the new buffer + * + * The pending callback argument now being processed contains both of the + * buffers (old and new) passed to the move callback function, the slab of the + * old buffer, and flags related to the move request, such as whether or not the + * system was desperate for memory. + * + * Slabs are not freed while there is a pending callback, but instead are kept + * on a deadlist, which is drained after the last callback completes. This means + * that slabs are safe to access until kmem_move_end(), no matter how many of + * their buffers have been freed. Once slab_refcnt reaches zero, it stays at + * zero for as long as the slab remains on the deadlist and until the slab is + * freed. + */ +static void +kmem_move_buffer(kmem_move_t *callback) +{ + kmem_cbrc_t response; + kmem_slab_t *sp = callback->kmm_from_slab; + kmem_cache_t *cp = sp->slab_cache; + boolean_t free_on_slab; + + ASSERT(taskq_member(kmem_move_taskq, curthread)); + ASSERT(MUTEX_NOT_HELD(&cp->cache_lock)); + ASSERT(KMEM_SLAB_MEMBER(sp, callback->kmm_from_buf)); + + /* + * The number of allocated buffers on the slab may have changed since we + * last checked the slab's reclaimability (when the pending move was + * enqueued), or the client may have responded NO when asked to move + * another buffer on the same slab. + */ + if (!kmem_slab_is_reclaimable(cp, sp, callback->kmm_flags)) { + kmem_slab_free(cp, callback->kmm_to_buf); + kmem_move_end(cp, callback); + return; + } + + /* + * Checking the slab layer is easy, so we might as well do that here + * in case we can avoid bothering the client. + */ + mutex_enter(&cp->cache_lock); + free_on_slab = (kmem_slab_allocated(cp, sp, + callback->kmm_from_buf) == NULL); + mutex_exit(&cp->cache_lock); + + if (free_on_slab) { + kmem_slab_free(cp, callback->kmm_to_buf); + kmem_move_end(cp, callback); + return; + } + + if (cp->cache_flags & KMF_BUFTAG) { + /* + * Make kmem_cache_alloc_debug() apply the constructor for us. + */ + if (kmem_cache_alloc_debug(cp, callback->kmm_to_buf, + KM_NOSLEEP, 1, caller()) != 0) { + kmem_move_end(cp, callback); + return; + } + } else if (cp->cache_constructor != NULL && + cp->cache_constructor(callback->kmm_to_buf, cp->cache_private, + KM_NOSLEEP) != 0) { + atomic_inc_64(&cp->cache_alloc_fail); + kmem_slab_free(cp, callback->kmm_to_buf); + kmem_move_end(cp, callback); + return; + } + + cp->cache_defrag->kmd_callbacks++; + cp->cache_defrag->kmd_thread = spl_current_thread(); + cp->cache_defrag->kmd_from_buf = callback->kmm_from_buf; + cp->cache_defrag->kmd_to_buf = callback->kmm_to_buf; + DTRACE_PROBE2(kmem__move__start, kmem_cache_t *, cp, kmem_move_t *, + callback); + + response = cp->cache_move(callback->kmm_from_buf, + callback->kmm_to_buf, cp->cache_bufsize, cp->cache_private); + + DTRACE_PROBE3(kmem__move__end, kmem_cache_t *, cp, kmem_move_t *, + callback, kmem_cbrc_t, response); + cp->cache_defrag->kmd_thread = NULL; + cp->cache_defrag->kmd_from_buf = NULL; + cp->cache_defrag->kmd_to_buf = NULL; + + if (response == KMEM_CBRC_YES) { + cp->cache_defrag->kmd_yes++; + kmem_slab_free_constructed(cp, callback->kmm_from_buf, B_FALSE); + /* slab safe to access until kmem_move_end() */ + if (sp->slab_refcnt == 0) + cp->cache_defrag->kmd_slabs_freed++; + mutex_enter(&cp->cache_lock); + kmem_slab_move_yes(cp, sp, callback->kmm_from_buf); + mutex_exit(&cp->cache_lock); + kmem_move_end(cp, callback); + return; + } + + switch (response) { + case KMEM_CBRC_NO: + cp->cache_defrag->kmd_no++; + mutex_enter(&cp->cache_lock); + kmem_slab_move_no(cp, sp, callback->kmm_from_buf); + mutex_exit(&cp->cache_lock); + break; + case KMEM_CBRC_LATER: + cp->cache_defrag->kmd_later++; + mutex_enter(&cp->cache_lock); + if (!KMEM_SLAB_IS_PARTIAL(sp)) { + mutex_exit(&cp->cache_lock); + break; + } + + if (++sp->slab_later_count >= KMEM_DISBELIEF) { + kmem_slab_move_no(cp, sp, + callback->kmm_from_buf); + } else if (!(sp->slab_flags & KMEM_SLAB_NOMOVE)) { + sp->slab_stuck_offset = KMEM_SLAB_OFFSET(sp, + callback->kmm_from_buf); + } + mutex_exit(&cp->cache_lock); + break; + case KMEM_CBRC_DONT_NEED: + cp->cache_defrag->kmd_dont_need++; + kmem_slab_free_constructed(cp, callback->kmm_from_buf, + B_FALSE); + if (sp->slab_refcnt == 0) + cp->cache_defrag->kmd_slabs_freed++; + mutex_enter(&cp->cache_lock); + kmem_slab_move_yes(cp, sp, callback->kmm_from_buf); + mutex_exit(&cp->cache_lock); + break; + case KMEM_CBRC_DONT_KNOW: + /* + * If we don't know if we can move this buffer or not, + * we'll just assume that we can't: if the buffer is + * in fact free, then it is sitting in one of the + * per-CPU magazines or in a full magazine in the depot + * layer. Either way, because defrag is induced in the + * same logic that reaps a cache, it's likely that full + * magazines will be returned to the system soon + * (thereby accomplishing what we're trying to + * accomplish here: return those magazines to their + * slabs). Given this, any work that we might do now to + * locate a buffer in a magazine is wasted (and + * expensive!) work; we bump a counter in this case and + * otherwise assume that we can't move it. + */ + cp->cache_defrag->kmd_dont_know++; + break; + default: + panic("'%s' (%p) unexpected move callback " + "response %d\n", cp->cache_name, (void *)cp, + response); + } + + kmem_slab_free_constructed(cp, callback->kmm_to_buf, B_FALSE); + kmem_move_end(cp, callback); +} + +/* Return B_FALSE if there is insufficient memory for the move request. */ +static boolean_t +kmem_move_begin(kmem_cache_t *cp, kmem_slab_t *sp, void *buf, int flags) +{ + void *to_buf; + avl_index_t index; + kmem_move_t *callback, *pending; + ulong_t n; + + ASSERT(taskq_member(kmem_taskq, curthread)); + ASSERT(MUTEX_NOT_HELD(&cp->cache_lock)); + ASSERT(sp->slab_flags & KMEM_SLAB_MOVE_PENDING); + + callback = kmem_cache_alloc(kmem_move_cache, KM_NOSLEEP); + + if (callback == NULL) + return (B_FALSE); + + callback->kmm_from_slab = sp; + callback->kmm_from_buf = buf; + callback->kmm_flags = flags; + + mutex_enter(&cp->cache_lock); + + n = avl_numnodes(&cp->cache_partial_slabs); + if ((n == 0) || ((n == 1) && !(flags & KMM_DEBUG))) { + mutex_exit(&cp->cache_lock); + kmem_cache_free(kmem_move_cache, callback); + return (B_TRUE); /* there is no need for the move request */ + } + + pending = avl_find(&cp->cache_defrag->kmd_moves_pending, buf, &index); + if (pending != NULL) { + /* + * If the move is already pending and we're desperate now, + * update the move flags. + */ + if (flags & KMM_DESPERATE) { + pending->kmm_flags |= KMM_DESPERATE; + } + mutex_exit(&cp->cache_lock); + kmem_cache_free(kmem_move_cache, callback); + return (B_TRUE); + } + + to_buf = kmem_slab_alloc_impl(cp, avl_first(&cp->cache_partial_slabs), + B_FALSE); + callback->kmm_to_buf = to_buf; + avl_insert(&cp->cache_defrag->kmd_moves_pending, callback, index); + + mutex_exit(&cp->cache_lock); + + if (!taskq_dispatch(kmem_move_taskq, (task_func_t *)kmem_move_buffer, + callback, TQ_NOSLEEP)) { + mutex_enter(&cp->cache_lock); + avl_remove(&cp->cache_defrag->kmd_moves_pending, callback); + mutex_exit(&cp->cache_lock); + kmem_slab_free(cp, to_buf); + kmem_cache_free(kmem_move_cache, callback); + return (B_FALSE); + } + + return (B_TRUE); +} + +static void +kmem_move_end(kmem_cache_t *cp, kmem_move_t *callback) +{ + avl_index_t index; + + ASSERT(cp->cache_defrag != NULL); + ASSERT(taskq_member(kmem_move_taskq, curthread)); + ASSERT(MUTEX_NOT_HELD(&cp->cache_lock)); + + mutex_enter(&cp->cache_lock); + VERIFY(avl_find(&cp->cache_defrag->kmd_moves_pending, + callback->kmm_from_buf, &index) != NULL); + avl_remove(&cp->cache_defrag->kmd_moves_pending, callback); + if (avl_is_empty(&cp->cache_defrag->kmd_moves_pending)) { + list_t *deadlist = &cp->cache_defrag->kmd_deadlist; + kmem_slab_t *sp; + + /* + * The last pending move completed. Release all slabs + * from the front of the dead list except for any slab + * at the tail that needs to be released from the context + * of kmem_move_buffers(). kmem deferred unmapping the + * buffers on these slabs in order to guarantee that + * buffers passed to the move callback have been touched + * only by kmem or by the client itself. + */ + while ((sp = list_remove_head(deadlist)) != NULL) { + if (sp->slab_flags & KMEM_SLAB_MOVE_PENDING) { + list_insert_tail(deadlist, sp); + break; + } + cp->cache_defrag->kmd_deadcount--; + cp->cache_slab_destroy++; + mutex_exit(&cp->cache_lock); + kmem_slab_destroy(cp, sp); + mutex_enter(&cp->cache_lock); + } + } + mutex_exit(&cp->cache_lock); + kmem_cache_free(kmem_move_cache, callback); +} + +/* + * Move buffers from least used slabs first by scanning backwards from the end + * of the partial slab list. Scan at most max_scan candidate slabs and move + * buffers from at most max_slabs slabs (0 for all partial slabs in both cases). + * If desperate to reclaim memory, move buffers from any partial slab, otherwise + * skip slabs with a ratio of allocated buffers at or above the current + * threshold. Return the number of unskipped slabs (at most max_slabs, -1 if the + * scan is aborted) so that the caller can adjust the reclaimability threshold + * depending on how many reclaimable slabs it finds. + * + * kmem_move_buffers() drops and reacquires cache_lock every time it issues a + * move request, since it is not valid for kmem_move_begin() to call + * kmem_cache_alloc() or taskq_dispatch() with cache_lock held. + */ +static int +kmem_move_buffers(kmem_cache_t *cp, size_t max_scan, size_t max_slabs, + int flags) +{ + kmem_slab_t *sp; + void *buf; + int i, j; /* slab index, buffer index */ + int s; /* reclaimable slabs */ + int b; /* allocated (movable) buffers on reclaimable slab */ + boolean_t success; + int refcnt; + int nomove; + + ASSERT(taskq_member(kmem_taskq, curthread)); + ASSERT(MUTEX_HELD(&cp->cache_lock)); + ASSERT(kmem_move_cache != NULL); + ASSERT(cp->cache_move != NULL && cp->cache_defrag != NULL); + ASSERT((flags & KMM_DEBUG) ? !avl_is_empty(&cp->cache_partial_slabs) : + avl_numnodes(&cp->cache_partial_slabs) > 1); + + if (kmem_move_blocked) { + return (0); + } + + if (kmem_move_fulltilt) { + flags |= KMM_DESPERATE; + } + + if (max_scan == 0 || (flags & KMM_DESPERATE)) { + /* + * Scan as many slabs as needed to find the desired number of + * candidate slabs. + */ + max_scan = (size_t)-1; + } + + if (max_slabs == 0 || (flags & KMM_DESPERATE)) { + /* Find as many candidate slabs as possible. */ + max_slabs = (size_t)-1; + } + + sp = avl_last(&cp->cache_partial_slabs); + ASSERT(KMEM_SLAB_IS_PARTIAL(sp)); + for (i = 0, s = 0; (i < max_scan) && (s < max_slabs) && (sp != NULL) && + ((sp != avl_first(&cp->cache_partial_slabs)) || + (flags & KMM_DEBUG)); + sp = AVL_PREV(&cp->cache_partial_slabs, sp), i++) { + + if (!kmem_slab_is_reclaimable(cp, sp, flags)) { + continue; + } + s++; + + /* Look for allocated buffers to move. */ + for (j = 0, b = 0, buf = sp->slab_base; + (j < sp->slab_chunks) && (b < sp->slab_refcnt); + buf = (((char *)buf) + cp->cache_chunksize), j++) { + + if (kmem_slab_allocated(cp, sp, buf) == NULL) { + continue; + } + + b++; + + /* + * Prevent the slab from being destroyed while we drop + * cache_lock and while the pending move is not yet + * registered. Flag the pending move while + * kmd_moves_pending may still be empty, since we can't + * yet rely on a non-zero pending move count to prevent + * the slab from being destroyed. + */ + ASSERT(!(sp->slab_flags & KMEM_SLAB_MOVE_PENDING)); + sp->slab_flags |= KMEM_SLAB_MOVE_PENDING; + /* + * Recheck refcnt and nomove after reacquiring the lock, + * since these control the order of partial slabs, and + * we want to know if we can pick up the scan where we + * left off. + */ + refcnt = sp->slab_refcnt; + nomove = (sp->slab_flags & KMEM_SLAB_NOMOVE); + mutex_exit(&cp->cache_lock); + + success = kmem_move_begin(cp, sp, buf, flags); + + /* + * Now, before the lock is reacquired, kmem could + * process all pending move requests and purge the + * deadlist, so that upon reacquiring the lock, sp has + * been remapped. Or, the client may free all the + * objects on the slab while the pending moves are still + * on the taskq. Therefore, the KMEM_SLAB_MOVE_PENDING + * flag causes the slab to be put at the end of the + * deadlist and prevents it from being destroyed, since + * we plan to destroy it here after reacquiring the + * lock. + */ + mutex_enter(&cp->cache_lock); + ASSERT(sp->slab_flags & KMEM_SLAB_MOVE_PENDING); + sp->slab_flags &= ~KMEM_SLAB_MOVE_PENDING; + + if (sp->slab_refcnt == 0) { + list_t *deadlist = + &cp->cache_defrag->kmd_deadlist; + list_remove(deadlist, sp); + + if (!avl_is_empty( + &cp->cache_defrag->kmd_moves_pending)) { + /* + * A pending move makes it unsafe to + * destroy the slab, because even though + * the move is no longer needed, the + * context where that is determined + * requires the slab to exist. + * Fortunately, a pending move also + * means we don't need to destroy the + * slab here, since it will get + * destroyed along with any other slabs + * on the deadlist after the last + * pending move completes. + */ + list_insert_head(deadlist, sp); + return (-1); + } + + /* + * Destroy the slab now if it was completely + * freed while we dropped cache_lock and there + * are no pending moves. Since slab_refcnt + * cannot change once it reaches zero, no new + * pending moves from that slab are possible. + */ + cp->cache_defrag->kmd_deadcount--; + cp->cache_slab_destroy++; + mutex_exit(&cp->cache_lock); + kmem_slab_destroy(cp, sp); + mutex_enter(&cp->cache_lock); + /* + * Since we can't pick up the scan where we left + * off, abort the scan and say nothing about the + * number of reclaimable slabs. + */ + return (-1); + } + + if (!success) { + /* + * Abort the scan if there is not enough memory + * for the request and say nothing about the + * number of reclaimable slabs. + */ + return (-1); + } + + /* + * The slab's position changed while the lock was + * dropped, so we don't know where we are in the + * sequence any more. + */ + if (sp->slab_refcnt != refcnt) { + /* + * If this is a KMM_DEBUG move, the slab_refcnt + * may have changed because we allocated a + * destination buffer on the same slab. In that + * case, we're not interested in counting it. + */ + return (-1); + } + if ((sp->slab_flags & KMEM_SLAB_NOMOVE) != nomove) + return (-1); + + /* + * Generating a move request allocates a destination + * buffer from the slab layer, bumping the first partial + * slab if it is completely allocated. If the current + * slab becomes the first partial slab as a result, we + * can't continue to scan backwards. + * + * If this is a KMM_DEBUG move and we allocated the + * destination buffer from the last partial slab, then + * the buffer we're moving is on the same slab and our + * slab_refcnt has changed, causing us to return before + * reaching here if there are no partial slabs left. + */ + ASSERT(!avl_is_empty(&cp->cache_partial_slabs)); + if (sp == avl_first(&cp->cache_partial_slabs)) { + /* + * We're not interested in a second KMM_DEBUG + * move. + */ + goto end_scan; + } + } + } +end_scan: + + return (s); +} + +typedef struct kmem_move_notify_args { + kmem_cache_t *kmna_cache; + void *kmna_buf; +} kmem_move_notify_args_t; + +static void +kmem_cache_move_notify_task(void *arg) +{ + kmem_move_notify_args_t *args = arg; + kmem_cache_t *cp = args->kmna_cache; + void *buf = args->kmna_buf; + kmem_slab_t *sp; + + ASSERT(taskq_member(kmem_taskq, curthread)); + ASSERT(list_link_active(&cp->cache_link)); + + zfs_kmem_free(args, sizeof (kmem_move_notify_args_t)); + mutex_enter(&cp->cache_lock); + sp = kmem_slab_allocated(cp, NULL, buf); + + /* Ignore the notification if the buffer is no longer allocated. */ + if (sp == NULL) { + mutex_exit(&cp->cache_lock); + return; + } + + /* Ignore the notification if there's no reason to move the buffer. */ + if (avl_numnodes(&cp->cache_partial_slabs) > 1) { + /* + * So far the notification is not ignored. Ignore the + * notification if the slab is not marked by an earlier refusal + * to move a buffer. + */ + if (!(sp->slab_flags & KMEM_SLAB_NOMOVE) && + (sp->slab_later_count == 0)) { + mutex_exit(&cp->cache_lock); + return; + } + + kmem_slab_move_yes(cp, sp, buf); + ASSERT(!(sp->slab_flags & KMEM_SLAB_MOVE_PENDING)); + sp->slab_flags |= KMEM_SLAB_MOVE_PENDING; + mutex_exit(&cp->cache_lock); + /* see kmem_move_buffers() about dropping the lock */ + (void) kmem_move_begin(cp, sp, buf, KMM_NOTIFY); + mutex_enter(&cp->cache_lock); + ASSERT(sp->slab_flags & KMEM_SLAB_MOVE_PENDING); + sp->slab_flags &= ~KMEM_SLAB_MOVE_PENDING; + if (sp->slab_refcnt == 0) { + list_t *deadlist = &cp->cache_defrag->kmd_deadlist; + list_remove(deadlist, sp); + + if (!avl_is_empty( + &cp->cache_defrag->kmd_moves_pending)) { + list_insert_head(deadlist, sp); + mutex_exit(&cp->cache_lock); + return; + } + + cp->cache_defrag->kmd_deadcount--; + cp->cache_slab_destroy++; + mutex_exit(&cp->cache_lock); + kmem_slab_destroy(cp, sp); + return; + } + } else { + kmem_slab_move_yes(cp, sp, buf); + } + mutex_exit(&cp->cache_lock); +} + +void +kmem_cache_move_notify(kmem_cache_t *cp, void *buf) +{ + kmem_move_notify_args_t *args; + + args = zfs_kmem_alloc(sizeof (kmem_move_notify_args_t), KM_NOSLEEP); + if (args != NULL) { + args->kmna_cache = cp; + args->kmna_buf = buf; + if (!taskq_dispatch(kmem_taskq, + (task_func_t *)kmem_cache_move_notify_task, args, + TQ_NOSLEEP)) + zfs_kmem_free(args, sizeof (kmem_move_notify_args_t)); + } +} + +static void +kmem_cache_defrag(kmem_cache_t *cp) +{ + size_t n; + + ASSERT(cp->cache_defrag != NULL); + + mutex_enter(&cp->cache_lock); + n = avl_numnodes(&cp->cache_partial_slabs); + if (n > 1) { + /* kmem_move_buffers() drops and reacquires cache_lock */ + cp->cache_defrag->kmd_defrags++; + (void) kmem_move_buffers(cp, n, 0, KMM_DESPERATE); + } + mutex_exit(&cp->cache_lock); +} + +/* Is this cache above the fragmentation threshold? */ +static boolean_t +kmem_cache_frag_threshold(kmem_cache_t *cp, uint64_t nfree) +{ + /* + * nfree kmem_frag_numer + * ------------------ > --------------- + * cp->cache_buftotal kmem_frag_denom + */ + return ((nfree * kmem_frag_denom) > + (cp->cache_buftotal * kmem_frag_numer)); +} + +static boolean_t +kmem_cache_is_fragmented(kmem_cache_t *cp, boolean_t *doreap) +{ + boolean_t fragmented; + uint64_t nfree; + + ASSERT(MUTEX_HELD(&cp->cache_lock)); + *doreap = B_FALSE; + + if (kmem_move_fulltilt) { + if (avl_numnodes(&cp->cache_partial_slabs) > 1) { + return (B_TRUE); + } + } else { + if ((cp->cache_complete_slab_count + avl_numnodes( + &cp->cache_partial_slabs)) < kmem_frag_minslabs) { + return (B_FALSE); + } + } + + nfree = cp->cache_bufslab; + fragmented = ((avl_numnodes(&cp->cache_partial_slabs) > 1) && + kmem_cache_frag_threshold(cp, nfree)); + + /* + * Free buffers in the magazine layer appear allocated from the point of + * view of the slab layer. We want to know if the slab layer would + * appear fragmented if we included free buffers from magazines that + * have fallen out of the working set. + */ + if (!fragmented) { + long reap; + + mutex_enter(&cp->cache_depot_lock); + reap = MIN(cp->cache_full.ml_reaplimit, cp->cache_full.ml_min); + reap = MIN(reap, cp->cache_full.ml_total); + mutex_exit(&cp->cache_depot_lock); + + nfree += ((uint64_t)reap * cp->cache_magtype->mt_magsize); + if (kmem_cache_frag_threshold(cp, nfree)) { + *doreap = B_TRUE; + } + } + + return (fragmented); +} + +/* Called periodically from kmem_taskq */ +static void +kmem_cache_scan(kmem_cache_t *cp) +{ + boolean_t reap = B_FALSE; + kmem_defrag_t *kmd; + + ASSERT(taskq_member(kmem_taskq, curthread)); + + mutex_enter(&cp->cache_lock); + + kmd = cp->cache_defrag; + if (kmd->kmd_consolidate > 0) { + kmd->kmd_consolidate--; + mutex_exit(&cp->cache_lock); + kmem_cache_reap(cp); + return; + } + + if (kmem_cache_is_fragmented(cp, &reap)) { + size_t slabs_found; + + /* + * Consolidate reclaimable slabs from the end of the partial + * slab list (scan at most kmem_reclaim_scan_range slabs to find + * reclaimable slabs). Keep track of how many candidate slabs we + * looked for and how many we actually found so we can adjust + * the definition of a candidate slab if we're having trouble + * finding them. + * + * kmem_move_buffers() drops and reacquires cache_lock. + */ + kmd->kmd_scans++; + slabs_found = kmem_move_buffers(cp, kmem_reclaim_scan_range, + kmem_reclaim_max_slabs, 0); + kmd->kmd_slabs_sought += kmem_reclaim_max_slabs; + kmd->kmd_slabs_found += slabs_found; + + if (++kmd->kmd_tries >= kmem_reclaim_scan_range) { + kmd->kmd_tries = 0; + + /* + * If we had difficulty finding candidate slabs in + * previous scans, adjust the threshold so that + * candidates are easier to find. + */ + if (kmd->kmd_slabs_found == kmd->kmd_slabs_sought) { + kmem_adjust_reclaim_threshold(kmd, -1); + } else if ((kmd->kmd_slabs_found * 2) < + kmd->kmd_slabs_sought) { + kmem_adjust_reclaim_threshold(kmd, 1); + } + kmd->kmd_slabs_sought = 0; + kmd->kmd_slabs_found = 0; + } + } else { + kmem_reset_reclaim_threshold(cp->cache_defrag); +#ifdef DEBUG + if (!avl_is_empty(&cp->cache_partial_slabs)) { + /* + * In a debug kernel we want the consolidator to + * run occasionally even when there is plenty of + * memory. + */ + uint16_t debug_rand; + + /* + * smd: note that this only gets called for the + * dnode cache because only the dnode cache has + * kmem_cache_set_move() applied to it + * brendon says move is voluntary and "tricky" + * the reason this is not called is because the source + * is kmem_cache_update(), that only calls this + * function (kmem_cache_scan()) + * if there is a move/defrag (same thing) associated + * with it so hoist some of this code up to to + * kmem_cache_update + */ + + (void) random_get_bytes((uint8_t *)&debug_rand, 2); + if (!kmem_move_noreap && + ((debug_rand % kmem_mtb_reap) == 0)) { + mutex_exit(&cp->cache_lock); + kmem_mtb_reap_count++; + return; + } else if ((debug_rand % kmem_mtb_move) == 0) { + kmd->kmd_scans++; + (void) kmem_move_buffers(cp, + kmem_reclaim_scan_range, 1, KMM_DEBUG); + } + } +#endif /* DEBUG */ + } + + mutex_exit(&cp->cache_lock); + +} + +// =============================================================== +// Status +// =============================================================== + + +size_t +kmem_size(void) +{ + return (total_memory); // smd +} + +// this is used in arc_reclaim_needed. if 1, reclaim is needed. +// returning 1 has the effect of throttling ARC, so be careful. +int +spl_vm_pool_low(void) +{ + bool m = spl_minimal_physmem_p_logic(); + + if (m) + return (0); + else + return (1); +} + +// =============================================================== +// String handling +// =============================================================== + +char * +kmem_strdup(const char *str) +{ + char *buf; + int len; + len = strlen(str) + 1; + buf = kmem_alloc(len, KM_SLEEP); + strlcpy(buf, str, len); + return (buf); +} + +void +kmem_strfree(char *str) +{ + zfs_kmem_free(str, strlen(str) + 1); +} + +/* + * kmem_scnprintf() will return the number of characters that it would have + * printed whenever it is limited by value of the size variable, rather than + * the number of characters that it did print. This can cause misbehavior on + * subsequent uses of the return value, so we define a safe version that will + * return the number of characters actually printed, minus the NULL format + * character. Subsequent use of this by the safe string functions is safe + * whether it is snprintf(), strlcat() or strlcpy(). + */ + +int +kmem_scnprintf(char *restrict str, size_t size, const char *restrict fmt, ...) +{ + int n; + va_list ap; + + /* Make the 0 case a no-op so that we do not return -1 */ + if (size == 0) + return (0); + + va_start(ap, fmt); + n = vsnprintf(str, size, fmt, ap); + va_end(ap); + + if (n >= size) + n = size - 1; + + return (n); +} + +char * +kvasdprintf(const char *fmt, va_list ap) +{ + char *p = NULL; + + if (!p) + return (NULL); + + return (p); +} + +char * +kmem_vasdprintf(const char *fmt, va_list ap) +{ + char *ptr = NULL; + + return (ptr); +} + +char * +kmem_asdprintf(const char *fmt, ...) +{ + va_list ap; + char *ptr; + + do { + va_start(ap, fmt); + ptr = kvasdprintf(fmt, ap); + va_end(ap); + } while (ptr == NULL); + + return (ptr); +} + +char * +kmem_asprintf(const char *fmt, ...) +{ + int size; + va_list adx; + char *buf; + + va_start(adx, fmt); + size = _vsnprintf(NULL, 0, fmt, adx) + 1; + va_end(adx); + + buf = kmem_alloc(size, KM_SLEEP); + + va_start(adx, fmt); + (void) _vsnprintf(buf, size, fmt, adx); + va_end(adx); + + return (buf); +} + + +/* + * Copyright (C) 2014 insane coder + * (http://insanecoding.blogspot.com/, http://asprintf.insanecoding.org/) + */ +char * +kmem_vasprintf(const char *fmt, va_list ap) +{ + char *ptr; + int size; + int r = -1; + + size = vsnprintf(NULL, 0, fmt, ap); + if ((size >= 0) && (size < INT_MAX)) { + ptr = (char *)kmem_alloc(size + 1, KM_SLEEP); // +1 for null + if (ptr) { + r = vsnprintf(ptr, size + 1, fmt, ap); // +1 for null + if ((r < 0) || (r > size)) { + kmem_free(ptr, size); + r = -1; + } + } + } else { + ptr = 0; + } + + return (ptr); +} + +char * +kmem_strstr(const char *in, const char *str) +{ + char c; + size_t len; + + c = *str++; + if (!c) + return ((char *)in); // Trivial empty string case + + len = strlen(str); + do { + char sc; + + do { + sc = *in++; + if (!sc) + return ((char *)0); + } while (sc != c); + } while (strncmp(in, str, len) != 0); + + return ((char *)(in - 1)); +} + + +// suppress timer and related logic for this kmem cache can live here +// three new per-kmem-cache stats: counters: non-vba-success non-vba-fail; +// flag: arc_no_grow +// from zfs/include/sys/spa.h + +#define SPA_MINBLOCKSHIFT 9 +#define SPA_MAXBLOCKSHIFT 24 +#define SPA_MINBLOCKSIZE (1ULL << SPA_MINBLOCKSHIFT) +#define SPA_MAXBLOCKSIZE (1ULL << SPA_MAXBLOCKSHIFT) + +typedef struct { + _Atomic(kmem_cache_t *)cp_metadata; + _Atomic(kmem_cache_t *)cp_filedata; + uint16_t pointed_to; + _Atomic int64_t suppress_count; + _Atomic uint64_t last_bumped; +} ksupp_t; + +typedef struct { + ksupp_t *ks_entry; +} iksupp_t; + +ksupp_t ksvec[SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT] = + { { NULL, NULL, false, 0, 0 } }; +iksupp_t iksvec[SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT] = + { { NULL } }; + +/* + * return true if the reclaim thread should be awakened + * because we do not have enough memory on hand + */ +boolean_t +spl_arc_reclaim_needed(const size_t bytes, kmem_cache_t **zp) +{ + + /* + * fast path: + * if our argument is 0, then do the equivalent of + * if (arc_available_memory() < 0) return (B_TRUE); + * which is traditional arc.c appraoch + * so we can arc_reclaim_needed() -> spl_arc_reclaim_needed(0) + * if we desire. + */ + if (bytes == 0 && spl_free < 0) { + return (B_TRUE); + } + + // copy some code from zio_buf_alloc() + size_t c = (bytes - 1) >> SPA_MINBLOCKSHIFT; + VERIFY3U(c, <, SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT); + + // if there is free memory in the kmem cache slab layer + // then we do not have to reclaim + + if (zp[c]->cache_bufslab > 1) { + if (spl_free < 0) + atomic_inc_64(&spl_arc_reclaim_avoided); + return (B_FALSE); + } + + extern uint64_t vmem_xnu_useful_bytes_free(void); + const uint64_t min_threshold = 64ULL*1024ULL*1024ULL; + const uint64_t pm_pct = real_total_memory >> 8; + const uint64_t high_threshold = MAX(min_threshold, (uint64_t)pm_pct); + const uint64_t low_threshold = bytes; + + const uint64_t f = vmem_xnu_useful_bytes_free(); + + if (f <= low_threshold) { + return (B_TRUE); + } else if (f > high_threshold) { + if (spl_free < 0) + atomic_inc_64(&spl_arc_reclaim_avoided); + return (B_FALSE); + } + + if (spl_free < 0) { + return (B_TRUE); + } else { + return (B_FALSE); + } +} + +/* small auxiliary function since we do not export struct kmem_cache to zfs */ +size_t +kmem_cache_bufsize(kmem_cache_t *cp) +{ + return (cp->cache_bufsize); +} + +/* + * check that we would not have KMERR_BADCACHE error in the event + * we did kmem_cache_free(cp, buf) in a DEBUG setting + * + * returns: NULL if the buf is not found in any cache + * cparg if the buf is found in cparg + * a pointer to the cache the buf is found in, if not cparg + */ + +kmem_cache_t * +kmem_cache_buf_in_cache(kmem_cache_t *cparg, void *bufarg) +{ + kmem_cache_t *cp = cparg; + kmem_slab_t *sp; + void *buf = bufarg; + + sp = kmem_findslab(cp, buf); + if (sp == NULL) { + for (cp = list_tail(&kmem_caches); cp != NULL; + cp = list_prev(&kmem_caches, cp)) { + if ((sp = kmem_findslab(cp, buf)) != NULL) + break; + } + } + + if (sp == NULL) { + TraceEvent(TRACE_ERROR, "SPL: %s: KMERR_BADADDR orig cache =" + " %s\n", __func__, cparg->cache_name); + return (NULL); + } + + if (cp == NULL) { + TraceEvent(TRACE_ERROR, "SPL: %s: ERROR cp == NULL; cparg ==" + " %s", __func__, cparg->cache_name); + return (NULL); + } + + if (cp != cparg) { + TraceEvent(TRACE_ERROR, "SPL: %s: KMERR_BADCACHE arg cache =" + " %s but found in %s instead\n", + __func__, cparg->cache_name, cp->cache_name); + return (cp); + } + + ASSERT(cp == cparg); + + return (cp); +} diff --git a/module/os/windows/spl/spl-kstat.c b/module/os/windows/spl/spl-kstat.c new file mode 100644 index 000000000000..aa2f1ea2f423 --- /dev/null +++ b/module/os/windows/spl/spl-kstat.c @@ -0,0 +1,2409 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + * + */ + +/* + * Provides an implementation of kstat that is backed by whatever windows has ? + */ + +#include +#include +#include +#include +#include + +/* kstat_fr.c */ + +/* + * Copyright (c) 1992, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright 2014, Joyent, Inc. All rights reserved. + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + */ + +/* + * Kernel statistics framework + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +/* + * Global lock to protect the AVL trees and kstat_chain_id. + */ +static kmutex_t kstat_chain_lock; + +/* + * Every install/delete kstat bumps kstat_chain_id. This is used by: + * + * (1) /dev/kstat, to detect changes in the kstat chain across ioctls; + * + * (2) kstat_create(), to assign a KID (kstat ID) to each new kstat. + * /dev/kstat uses the KID as a cookie for kstat lookups. + * + * We reserve the first two IDs because some kstats are created before + * the well-known ones (kstat_headers = 0, kstat_types = 1). + * + * We also bump the kstat_chain_id if a zone is gaining or losing visibility + * into a particular kstat, which is logically equivalent to a kstat being + * installed/deleted. + */ + +kid_t kstat_chain_id = 2; + +/* + * As far as zones are concerned, there are 3 types of kstat: + * + * 1) Those which have a well-known name, and which should return per-zone data + * depending on which zone is doing the kstat_read(). sockfs:0:sock_unix_list + * is an example of this type of kstat. + * + * 2) Those which should only be exported to a particular list of zones. + * For example, in the case of nfs:*:mntinfo, we don't want zone A to be + * able to see NFS mounts associated with zone B, while we want the + * global zone to be able to see all mounts on the system. + * + * 3) Those that can be exported to all zones. Most system-related + * kstats fall within this category. + * + * An ekstat_t thus contains a list of kstats that the zone is to be + * exported to. The lookup of a name:instance:module thus translates to a + * lookup of name:instance:module:myzone; if the kstat is not exported + * to all zones, and does not have the caller's zoneid explicitly + * enumerated in the list of zones to be exported to, it is the same as + * if the kstat didn't exist. + * + * Writing to kstats is currently disallowed from within a non-global + * zone, although this restriction could be removed in the future. + */ +typedef struct kstat_zone { + zoneid_t zoneid; + struct kstat_zone *next; +} kstat_zone_t; + +/* + * Extended kstat structure -- for internal use only. + */ +typedef struct ekstat { + kstat_t e_ks; /* the kstat itself */ + size_t e_size; /* total allocation size */ + kthread_t *e_owner; /* thread holding this kstat */ + kcondvar_t e_cv; /* wait for owner == NULL */ + avl_node_t e_avl_bykid; /* AVL tree to sort by KID */ + avl_node_t e_avl_byname; /* AVL tree to sort by name */ + kstat_zone_t e_zone; /* zone to export stats to */ +} ekstat_t; + +struct sbuf { + char *s_buf; /* storage buffer */ + void *s_unused; /* binary compatibility. */ + int s_size; /* size of storage buffer */ + int s_len; /* current length of string */ +#define SBUF_FIXEDLEN 0x00000000 /* fixed length buffer (default) */ +#define SBUF_AUTOEXTEND 0x00000001 /* automatically extend buffer */ +#define SBUF_USRFLAGMSK 0x0000ffff /* mask of flags the user may specify */ +#define SBUF_DYNAMIC 0x00010000 /* s_buf must be freed */ +#define SBUF_FINISHED 0x00020000 /* set by sbuf_finish() */ +#define SBUF_OVERFLOWED 0x00040000 /* sbuf overflowed */ +#define SBUF_DYNSTRUCT 0x00080000 /* sbuf must be freed */ + int s_flags; /* flags */ +}; + +/* sbuf_new() and family does exist in XNU, but Apple wont let us call them */ +#define M_SBUF 105 /* string buffers */ +#define SBMALLOC(size) \ + (struct sbuf *)ExAllocatePoolWithTag(NonPagedPoolNx, (size), '!SFZ') +#define SBFREE(buf) ExFreePoolWithTag((buf), '!SFZ') + +#define SBUF_SETFLAG(s, f) do { (s)->s_flags |= (f); } while (0) +#define SBUF_CLEARFLAG(s, f) do { (s)->s_flags &= ~(f); } while (0) +#define SBUF_ISDYNAMIC(s) ((s)->s_flags & SBUF_DYNAMIC) +#define SBUF_ISDYNSTRUCT(s) ((s)->s_flags & SBUF_DYNSTRUCT) +#define SBUF_HASOVERFLOWED(s) ((s)->s_flags & SBUF_OVERFLOWED) +#define SBUF_HASROOM(s) ((s)->s_len < (s)->s_size - 1) +#define SBUF_FREESPACE(s) ((s)->s_size - (s)->s_len - 1) +#define SBUF_CANEXTEND(s) ((s)->s_flags & SBUF_AUTOEXTEND) +#define SBUF_ISFINISHED(s) ((s)->s_flags & SBUF_FINISHED) + +#define SBUF_MINEXTENDSIZE 16 /* Should be power of 2. */ +#define SBUF_MAXEXTENDSIZE PAGE_SIZE +#define SBUF_MAXEXTENDINCR PAGE_SIZE + +#define SBUF_INCLUDENUL 0x00000002 /* FBSD: nulterm byte is counted in len */ +#define SBUF_NULINCLUDED(s) ((s)->s_flags & SBUF_INCLUDENUL) + +void +sbuf_finish(struct sbuf *s) +{ + s->s_buf[s->s_len] = '\0'; + if (SBUF_NULINCLUDED(s)) + s->s_len++; + SBUF_CLEARFLAG(s, SBUF_OVERFLOWED); + SBUF_SETFLAG(s, SBUF_FINISHED); +} + +char * +sbuf_data(struct sbuf *s) +{ + return (s->s_buf); +} + +int +sbuf_len(struct sbuf *s) +{ + if (SBUF_HASOVERFLOWED(s)) { + return (-1); + } + /* If finished, nulterm is already in len, else add one. */ + if (SBUF_NULINCLUDED(s) && !SBUF_ISFINISHED(s)) + return (s->s_len + 1); + return (s->s_len); +} + +void +sbuf_delete(struct sbuf *s) +{ + int isdyn; + if (SBUF_ISDYNAMIC(s)) { + SBFREE(s->s_buf); + } + isdyn = SBUF_ISDYNSTRUCT(s); + memset(s, 0, sizeof (*s)); + if (isdyn) { + SBFREE(s); + } +} + +static int +sbuf_extendsize(int size) +{ + int newsize; + + newsize = SBUF_MINEXTENDSIZE; + while (newsize < size) { + if (newsize < (int)SBUF_MAXEXTENDSIZE) { + newsize *= 2; + } else { + newsize += SBUF_MAXEXTENDINCR; + } + } + + return (newsize); +} + +static int +sbuf_extend(struct sbuf *s, int addlen) +{ + char *newbuf; + int newsize; + + if (!SBUF_CANEXTEND(s)) { + return (-1); + } + + newsize = sbuf_extendsize(s->s_size + addlen); + newbuf = (char *)SBMALLOC(newsize); + if (newbuf == NULL) { + return (-1); + } + memcpy(newbuf, s->s_buf, s->s_size); + if (SBUF_ISDYNAMIC(s)) { + SBFREE(s->s_buf); + } else { + SBUF_SETFLAG(s, SBUF_DYNAMIC); + } + s->s_buf = newbuf; + s->s_size = newsize; + return (0); +} + +struct sbuf * +sbuf_new(struct sbuf *s, char *buf, int length, int flags) +{ + flags &= SBUF_USRFLAGMSK; + if (s == NULL) { + s = (struct sbuf *)SBMALLOC(sizeof (*s)); + if (s == NULL) { + return (NULL); + } + memset(s, 0, sizeof (*s)); + s->s_flags = flags; + SBUF_SETFLAG(s, SBUF_DYNSTRUCT); + } else { + memset(s, 0, sizeof (*s)); + s->s_flags = flags; + } + s->s_size = length; + if (buf) { + s->s_buf = buf; + return (s); + } + if (flags & SBUF_AUTOEXTEND) { + s->s_size = sbuf_extendsize(s->s_size); + } + s->s_buf = (char *)SBMALLOC(s->s_size); + if (s->s_buf == NULL) { + if (SBUF_ISDYNSTRUCT(s)) { + SBFREE(s); + } + return (NULL); + } + SBUF_SETFLAG(s, SBUF_DYNAMIC); + return (s); +} + +#define va_copy(A, B) A = B + +int +sbuf_vprintf(struct sbuf *s, const char *fmt, va_list ap) +{ + __builtin_va_list ap_copy; + int len; + + if (SBUF_HASOVERFLOWED(s)) { + return (-1); + } + + do { + va_copy(ap_copy, ap); + len = vsnprintf(&s->s_buf[s->s_len], SBUF_FREESPACE(s) + 1, + fmt, ap_copy); + // left-side must be assignable. Win tries to set to 0. + // va_end(ap_copy); + } while (len > SBUF_FREESPACE(s) && + sbuf_extend(s, len - SBUF_FREESPACE(s)) == 0); + s->s_len += min(len, SBUF_FREESPACE(s)); + if (!SBUF_HASROOM(s) && !SBUF_CANEXTEND(s)) { + SBUF_SETFLAG(s, SBUF_OVERFLOWED); + } + + if (SBUF_HASOVERFLOWED(s)) { + return (-1); + } + return (0); +} + +int +sbuf_printf(struct sbuf *s, const char *fmt, ...) +{ + va_list ap; + int result; + + va_start(ap, fmt); + result = sbuf_vprintf(s, fmt, ap); + va_end(ap); + return (result); +} + +static int +kstat_default_update(kstat_t *ksp, int rw) +{ + ASSERT(ksp != NULL); + + if (rw == KSTAT_WRITE) + return (EACCES); + + return (0); +} + +static int +kstat_resize_raw(kstat_t *ksp) +{ + if (ksp->ks_raw_bufsize == KSTAT_RAW_MAX) + return (ENOMEM); + + SBMFREE(ksp->ks_raw_buf, ksp->ks_raw_bufsize); + ksp->ks_raw_bufsize = MIN(ksp->ks_raw_bufsize * 2, KSTAT_RAW_MAX); + ksp->ks_raw_buf = SBMALLOC(ksp->ks_raw_bufsize); + + return (0); +} + +static void * +kstat_raw_default_addr(kstat_t *ksp, loff_t n) +{ + if (n == 0) + return (ksp->ks_data); + return (NULL); +} + +#define HD_COLUMN_MASK 0xff +#define HD_DELIM_MASK 0xff00 +#define HD_OMIT_COUNT (1 << 16) +#define HD_OMIT_HEX (1 << 17) +#define HD_OMIT_CHARS (1 << 18) + +void +sbuf_hexdump(struct sbuf *sb, const void *ptr, int length, const char *hdr, + int flags) +{ + int i, j, k; + int cols; + const unsigned char *cp; + char delim; + + if ((flags & HD_DELIM_MASK) != 0) + delim = (flags & HD_DELIM_MASK) >> 8; + else + delim = ' '; + + if ((flags & HD_COLUMN_MASK) != 0) + cols = flags & HD_COLUMN_MASK; + else + cols = 16; + + cp = ptr; + for (i = 0; i < length; i += cols) { + if (hdr != NULL) + sbuf_printf(sb, "%s", hdr); + + if ((flags & HD_OMIT_COUNT) == 0) + sbuf_printf(sb, "%04x ", i); + + if ((flags & HD_OMIT_HEX) == 0) { + for (j = 0; j < cols; j++) { + k = i + j; + if (k < length) + sbuf_printf(sb, "%c%02x", delim, cp[k]); + else + sbuf_printf(sb, " "); + } + } + + if ((flags & HD_OMIT_CHARS) == 0) { + sbuf_printf(sb, " |"); + for (j = 0; j < cols; j++) { + k = i + j; + if (k >= length) + sbuf_printf(sb, " "); + else if (cp[k] >= ' ' && cp[k] <= '~') + sbuf_printf(sb, "%c", cp[k]); + else + sbuf_printf(sb, "."); + } + sbuf_printf(sb, "|"); + } + sbuf_printf(sb, "\n"); + } +} + +/* + * kmem sets up kstats before it is ready to be called, + * so have some initial space + */ +static uint64_t kstat_initial[16384]; +static void *kstat_initial_ptr = kstat_initial; +static size_t kstat_initial_avail = sizeof (kstat_initial); +static vmem_t *kstat_arena; + +#define KSTAT_ALIGN (sizeof (uint64_t)) + +static avl_tree_t kstat_avl_bykid; +static avl_tree_t kstat_avl_byname; + +/* + * Various pointers we need to create kstats at boot time in kstat_init() + */ +extern kstat_named_t *segmapcnt_ptr; +extern uint_t segmapcnt_ndata; +extern int segmap_kstat_update(kstat_t *, int); +extern kstat_named_t *biostats_ptr; +extern uint_t biostats_ndata; +extern kstat_named_t *pollstats_ptr; +extern uint_t pollstats_ndata; + +extern int vac; +extern uint_t nproc; +extern time_t boot_time; + +static struct { + char name[KSTAT_STRLEN]; + size_t size; + uint_t min_ndata; + uint_t max_ndata; +} kstat_data_type[KSTAT_NUM_TYPES] = { + { "raw", 1, 0, INT_MAX }, + { "name=value", sizeof (kstat_named_t), 0, INT_MAX }, + { "interrupt", sizeof (kstat_intr_t), 1, 1 }, + { "i/o", sizeof (kstat_io_t), 1, 1 }, + { "event_timer", sizeof (kstat_timer_t), 0, INT_MAX }, +}; + +static int header_kstat_update(kstat_t *, int); +static int header_kstat_snapshot(kstat_t *, void *, int); + +int +kstat_zone_find(kstat_t *k, zoneid_t zoneid) +{ + ekstat_t *e = (ekstat_t *)k; + kstat_zone_t *kz; + + ASSERT(MUTEX_HELD(&kstat_chain_lock)); + for (kz = &e->e_zone; kz != NULL; kz = kz->next) { + if (zoneid == ALL_ZONES || kz->zoneid == ALL_ZONES) + return (1); + if (zoneid == kz->zoneid) + return (1); + } + return (0); +} + +void +kstat_zone_remove(kstat_t *k, zoneid_t zoneid) +{ + ekstat_t *e = (ekstat_t *)k; + kstat_zone_t *kz, *t = NULL; + + mutex_enter(&kstat_chain_lock); + if (zoneid == e->e_zone.zoneid) { + kz = e->e_zone.next; + ASSERT(kz != NULL); + e->e_zone.zoneid = kz->zoneid; + e->e_zone.next = kz->next; + goto out; + } + for (kz = &e->e_zone; kz->next != NULL; kz = kz->next) { + if (kz->next->zoneid == zoneid) { + t = kz->next; + kz->next = t->next; + break; + } + } + ASSERT(t != NULL); /* we removed something */ + kz = t; +out: + kstat_chain_id++; + mutex_exit(&kstat_chain_lock); + kmem_free(kz, sizeof (*kz)); +} + +void +kstat_zone_add(kstat_t *k, zoneid_t zoneid) +{ + ekstat_t *e = (ekstat_t *)k; + kstat_zone_t *kz; + + kz = kmem_alloc(sizeof (*kz), KM_NOSLEEP); + if (kz == NULL) + return; + mutex_enter(&kstat_chain_lock); + kz->zoneid = zoneid; + kz->next = e->e_zone.next; + e->e_zone.next = kz; + kstat_chain_id++; + mutex_exit(&kstat_chain_lock); +} + +/* + * Compare the list of zones for the given kstats, returning 0 if they match + * (ie, one list contains ALL_ZONES or both lists contain the same zoneid). + * In practice, this is called indirectly by kstat_hold_byname(), so one of the + * two lists always has one element, and this is an O(n) operation rather than + * O(n^2). + */ +static int +kstat_zone_compare(ekstat_t *e1, ekstat_t *e2) +{ + kstat_zone_t *kz1, *kz2; + + ASSERT(MUTEX_HELD(&kstat_chain_lock)); + for (kz1 = &e1->e_zone; kz1 != NULL; kz1 = kz1->next) { + for (kz2 = &e2->e_zone; kz2 != NULL; kz2 = kz2->next) { + if (kz1->zoneid == ALL_ZONES || + kz2->zoneid == ALL_ZONES) + return (0); + if (kz1->zoneid == kz2->zoneid) + return (0); + } + } + return (e1->e_zone.zoneid < e2->e_zone.zoneid ? -1 : 1); +} + +/* + * Support for keeping kstats sorted in AVL trees for fast lookups. + */ +static int +kstat_compare_bykid(const void *a1, const void *a2) +{ + const kstat_t *k1 = a1; + const kstat_t *k2 = a2; + + if (k1->ks_kid < k2->ks_kid) + return (-1); + if (k1->ks_kid > k2->ks_kid) + return (1); + return (kstat_zone_compare((ekstat_t *)k1, (ekstat_t *)k2)); +} + +static int +kstat_compare_byname(const void *a1, const void *a2) +{ + const kstat_t *k1 = a1; + const kstat_t *k2 = a2; + int s; + + s = strcmp(k1->ks_module, k2->ks_module); + if (s > 0) + return (1); + if (s < 0) + return (-1); + + if (k1->ks_instance < k2->ks_instance) + return (-1); + if (k1->ks_instance > k2->ks_instance) + return (1); + + s = strcmp(k1->ks_name, k2->ks_name); + if (s > 0) + return (1); + if (s < 0) + return (-1); + + return (kstat_zone_compare((ekstat_t *)k1, (ekstat_t *)k2)); +} + +static kstat_t * +kstat_hold(avl_tree_t *t, ekstat_t *template) +{ + kstat_t *ksp; + ekstat_t *e; + + mutex_enter(&kstat_chain_lock); + for (;;) { + ksp = avl_find(t, template, NULL); + if (ksp == NULL) + break; + e = (ekstat_t *)ksp; + if (e->e_owner == NULL) { + e->e_owner = (void *)curthread; + break; + } + cv_wait(&e->e_cv, &kstat_chain_lock); + } + mutex_exit(&kstat_chain_lock); + return (ksp); +} + +void +kstat_rele(kstat_t *ksp) +{ + ekstat_t *e = (ekstat_t *)ksp; + + mutex_enter(&kstat_chain_lock); + ASSERT(e->e_owner == (void *)curthread); + e->e_owner = NULL; + cv_broadcast(&e->e_cv); + mutex_exit(&kstat_chain_lock); +} + +kstat_t * +kstat_hold_bykid(kid_t kid, zoneid_t zoneid) +{ + ekstat_t e; + + e.e_ks.ks_kid = kid; + e.e_zone.zoneid = zoneid; + e.e_zone.next = NULL; + + return (kstat_hold(&kstat_avl_bykid, &e)); +} + +kstat_t * +kstat_hold_byname(const char *ks_module, int ks_instance, const char *ks_name, + zoneid_t ks_zoneid) +{ + ekstat_t e; + + kstat_set_string(e.e_ks.ks_module, ks_module); + e.e_ks.ks_instance = ks_instance; + kstat_set_string(e.e_ks.ks_name, ks_name); + e.e_zone.zoneid = ks_zoneid; + e.e_zone.next = NULL; + return (kstat_hold(&kstat_avl_byname, &e)); +} + +static ekstat_t * +kstat_alloc(size_t size) +{ + ekstat_t *e = NULL; + + size = P2ROUNDUP(sizeof (ekstat_t) + size, KSTAT_ALIGN); + + if (kstat_arena == NULL) { + if (size <= kstat_initial_avail) { + e = kstat_initial_ptr; + kstat_initial_ptr = (char *)kstat_initial_ptr + size; + kstat_initial_avail -= size; + } + } else { + e = vmem_alloc_impl(kstat_arena, size, VM_NOSLEEP); + } + + if (e != NULL) { + memset(e, 0, size); + e->e_size = size; + cv_init(&e->e_cv, NULL, CV_DEFAULT, NULL); + } + + return (e); +} + +static void +kstat_free(ekstat_t *e) +{ + cv_destroy(&e->e_cv); + vmem_free_impl(kstat_arena, e, e->e_size); +} + +extern vmem_t *heap_arena; +void *segkmem_alloc(vmem_t *vmp, uint32_t size, int vmflag); +void segkmem_free(vmem_t *vmp, void *inaddr, uint32_t size); + +/* + * Create various system kstats. + */ +void +kstat_init(void) +{ + kstat_t *ksp; + ekstat_t *e; + avl_tree_t *t = &kstat_avl_bykid; + + /* + * Set up the kstat vmem arena. + */ + kstat_arena = vmem_create("kstat", + (void *)kstat_initial, sizeof (kstat_initial), KSTAT_ALIGN, + segkmem_alloc, segkmem_free, heap_arena, 0, VM_SLEEP); + + /* + * Make initial kstats appear as though they were allocated. + */ + for (e = avl_first(t); e != NULL; e = avl_walk(t, e, AVL_AFTER)) + (void) vmem_xalloc(kstat_arena, e->e_size, KSTAT_ALIGN, + 0, 0, e, (char *)e + e->e_size, + VM_NOSLEEP | VM_BESTFIT | VM_PANIC); + + /* + * The mother of all kstats. The first kstat in the system, which + * always has KID 0, has the headers for all kstats (including itself) + * as its data. Thus, the kstat driver does not need any special + * interface to extract the kstat chain. + */ + kstat_chain_id = 0; + ksp = kstat_create("unix", 0, "kstat_headers", "kstat", KSTAT_TYPE_RAW, + 0, KSTAT_FLAG_VIRTUAL | KSTAT_FLAG_VAR_SIZE); + if (ksp) { + ksp->ks_lock = &kstat_chain_lock; + ksp->ks_update = header_kstat_update; + ksp->ks_snapshot = header_kstat_snapshot; + kstat_install(ksp); + } else { + panic("cannot create kstat 'kstat_headers'"); + } + + ksp = kstat_create("unix", 0, "kstat_types", "kstat", + KSTAT_TYPE_NAMED, KSTAT_NUM_TYPES, 0); + if (ksp) { + int i; + kstat_named_t *kn = KSTAT_NAMED_PTR(ksp); + + for (i = 0; i < KSTAT_NUM_TYPES; i++) { + kstat_named_init(&kn[i], kstat_data_type[i].name, + KSTAT_DATA_ULONG); + kn[i].value.ul = i; + } + kstat_install(ksp); + } + +} + +/* + * Caller of this should ensure that the string pointed by src + * doesn't change while kstat's lock is held. Not doing so defeats + * kstat's snapshot strategy as explained in + */ +void +kstat_named_setstr(kstat_named_t *knp, const char *src) +{ + if (knp->data_type != KSTAT_DATA_STRING) + panic("kstat_named_setstr('%p', '%p'): " + "named kstat is not of type KSTAT_DATA_STRING", + (void *)knp, (void *)src); + + KSTAT_NAMED_STR_PTR(knp) = (char *)src; + if (src != NULL) + KSTAT_NAMED_STR_BUFLEN(knp) = strlen(src) + 1; + else + KSTAT_NAMED_STR_BUFLEN(knp) = 0; +} + +void +kstat_set_string(char *dst, const char *src) +{ + (void) strlcpy(dst, src, KSTAT_STRLEN); +} + +void +kstat_named_init(kstat_named_t *knp, const char *name, uchar_t data_type) +{ + kstat_set_string(knp->name, name); + knp->data_type = data_type; + + if (data_type == KSTAT_DATA_STRING) + kstat_named_setstr(knp, NULL); +} + +void +kstat_timer_init(kstat_timer_t *ktp, const char *name) +{ + kstat_set_string(ktp->name, name); +} + +static int +default_kstat_update(kstat_t *ksp, int rw) +{ + uint_t i; + size_t len = 0; + kstat_named_t *knp; + + /* + * Named kstats with variable-length long strings have a standard + * way of determining how much space is needed to hold the snapshot: + */ + if (ksp->ks_data != NULL && ksp->ks_type == KSTAT_TYPE_NAMED && + (ksp->ks_flags & (KSTAT_FLAG_VAR_SIZE | KSTAT_FLAG_LONGSTRINGS))) { + + /* + * Add in the space required for the strings + */ + knp = KSTAT_NAMED_PTR(ksp); + for (i = 0; i < ksp->ks_ndata; i++, knp++) { + if (knp->data_type == KSTAT_DATA_STRING) + len += KSTAT_NAMED_STR_BUFLEN(knp); + } + ksp->ks_data_size = + ksp->ks_ndata * sizeof (kstat_named_t) + len; + } + return (0); +} + +static int +default_kstat_snapshot(kstat_t *ksp, void *buf, int rw) +{ + // kstat_io_t *kiop; + hrtime_t cur_time; + size_t namedsz; + + ksp->ks_snaptime = cur_time = gethrtime(); + + if (rw == KSTAT_WRITE) { + if (!(ksp->ks_flags & KSTAT_FLAG_WRITABLE)) + return (EACCES); + memcpy(ksp->ks_data, buf, ksp->ks_data_size); + return (0); + } + + /* + * KSTAT_TYPE_NAMED kstats are defined to have ks_ndata + * number of kstat_named_t structures, followed by an optional + * string segment. The ks_data generally holds only the + * kstat_named_t structures. So we copy it first. The strings, + * if any, are copied below. For other kstat types, ks_data holds the + * entire buffer. + */ + + namedsz = sizeof (kstat_named_t) * ksp->ks_ndata; + if (ksp->ks_type == KSTAT_TYPE_NAMED && ksp->ks_data_size > namedsz) + memcpy(buf, ksp->ks_data, namedsz); + else + memcpy(buf, ksp->ks_data, ksp->ks_data_size); + + /* + * Apply kstat type-specific data massaging + */ + switch (ksp->ks_type) { + + case KSTAT_TYPE_IO: + /* + * Normalize time units and deal with incomplete transactions + */ +#if 0 + kiop = (kstat_io_t *)buf; + + scalehrtime(&kiop->wtime); + scalehrtime(&kiop->wlentime); + scalehrtime(&kiop->wlastupdate); + scalehrtime(&kiop->rtime); + scalehrtime(&kiop->rlentime); + scalehrtime(&kiop->rlastupdate); + + if (kiop->wcnt != 0) { + /* like kstat_waitq_exit */ + hrtime_t wfix = cur_time - kiop->wlastupdate; + kiop->wlastupdate = cur_time; + kiop->wlentime += kiop->wcnt * wfix; + kiop->wtime += wfix; + } + + if (kiop->rcnt != 0) { + /* like kstat_runq_exit */ + hrtime_t rfix = cur_time - kiop->rlastupdate; + kiop->rlastupdate = cur_time; + kiop->rlentime += kiop->rcnt * rfix; + kiop->rtime += rfix; + } +#endif + break; + + case KSTAT_TYPE_NAMED: + /* + * Massage any long strings in at the end of the buffer + */ + if (ksp->ks_data_size > namedsz) { + uint_t i; + kstat_named_t *knp = buf; + char *dst = (char *)(knp + ksp->ks_ndata); + /* + * Copy strings and update pointers + */ + for (i = 0; i < ksp->ks_ndata; i++, knp++) { + if (knp->data_type == KSTAT_DATA_STRING && + KSTAT_NAMED_STR_PTR(knp) != NULL) { + memcpy(dst, KSTAT_NAMED_STR_PTR(knp), + KSTAT_NAMED_STR_BUFLEN(knp)); + KSTAT_NAMED_STR_PTR(knp) = dst; + dst += KSTAT_NAMED_STR_BUFLEN(knp); + } + } + ASSERT(dst <= ((char *)buf + ksp->ks_data_size)); + } + break; + } + return (0); +} + +static int +header_kstat_update(kstat_t *header_ksp, int rw) +{ + int nkstats = 0; + ekstat_t *e; + avl_tree_t *t = &kstat_avl_bykid; + zoneid_t zoneid; + + if (rw == KSTAT_WRITE) + return (EACCES); + + ASSERT(MUTEX_HELD(&kstat_chain_lock)); + + zoneid = crgetzoneid(); + for (e = avl_first(t); e != NULL; e = avl_walk(t, e, AVL_AFTER)) { + if (kstat_zone_find((kstat_t *)e, zoneid) && + (e->e_ks.ks_flags & KSTAT_FLAG_INVALID) == 0) { + nkstats++; + } + } + header_ksp->ks_ndata = nkstats; + header_ksp->ks_data_size = nkstats * sizeof (kstat_t); + return (0); +} + +/* + * Copy out the data section of kstat 0, which consists of the list + * of all kstat headers. By specification, these headers must be + * copied out in order of increasing KID. + */ +static int +header_kstat_snapshot(kstat_t *header_ksp, void *buf, int rw) +{ + ekstat_t *e; + avl_tree_t *t = &kstat_avl_bykid; + zoneid_t zoneid; + + header_ksp->ks_snaptime = gethrtime(); + + if (rw == KSTAT_WRITE) + return (EACCES); + + ASSERT(MUTEX_HELD(&kstat_chain_lock)); + + zoneid = crgetzoneid(); + for (e = avl_first(t); e != NULL; e = avl_walk(t, e, AVL_AFTER)) { + if (kstat_zone_find((kstat_t *)e, zoneid) && + (e->e_ks.ks_flags & KSTAT_FLAG_INVALID) == 0) { + memcpy(buf, &e->e_ks, sizeof (kstat_t)); + buf = (char *)buf + sizeof (kstat_t); + } + } + + return (0); +} + +kstat_t *perf_arc_ksp, *perf_zil_ksp; + +kstat_t * +kstat_create(const char *ks_module, int ks_instance, const char *ks_name, + const char *ks_class, uchar_t ks_type, uint_t ks_ndata, uchar_t ks_flags) +{ + kstat_t *kstat_temp = (kstat_create_zone(ks_module, ks_instance, + ks_name, ks_class, ks_type, ks_ndata, ks_flags, ALL_ZONES)); + + if (kstat_temp && strcmp(ks_module, "zfs") == 0 && strcmp(ks_name, + "arcstats") == 0) + perf_arc_ksp = kstat_temp; + if (kstat_temp && strcmp(ks_module, "zfs") == 0 && strcmp(ks_name, + "zil") == 0) + perf_zil_ksp = kstat_temp; + + return (kstat_temp); +} + +/* + * Allocate and initialize a kstat structure. Or, if a dormant kstat with + * the specified name exists, reactivate it. Returns a pointer to the kstat + * on success, NULL on failure. The kstat will not be visible to the + * kstat driver until kstat_install(). + */ +kstat_t * +kstat_create_zone(const char *ks_module, int ks_instance, const char *ks_name, + const char *ks_class, uchar_t ks_type, uint_t ks_ndata, uchar_t ks_flags, + zoneid_t ks_zoneid) +{ + size_t ks_data_size; + kstat_t *ksp; + ekstat_t *e; + avl_index_t where; + char namebuf[KSTAT_STRLEN + 16]; + + if (ks_class == NULL) + ks_class = "misc"; + + if (avl_numnodes(&kstat_avl_bykid) == 0) { + avl_create(&kstat_avl_bykid, kstat_compare_bykid, + sizeof (ekstat_t), offsetof(struct ekstat, e_avl_bykid)); + + avl_create(&kstat_avl_byname, kstat_compare_byname, + sizeof (ekstat_t), offsetof(struct ekstat, e_avl_byname)); + } + + /* + * If ks_name == NULL, set the ks_name to . + */ + if (ks_name == NULL) { + char buf[KSTAT_STRLEN]; + kstat_set_string(buf, ks_module); + (void) sprintf(namebuf, "%s%d", buf, ks_instance); + ks_name = namebuf; + } + + /* + * Make sure it's a valid kstat data type + */ + if (ks_type >= KSTAT_NUM_TYPES) { + cmn_err(CE_WARN, "kstat_create('%s', %d, '%s'): " + "invalid kstat type %d", + ks_module, ks_instance, ks_name, ks_type); + return (NULL); + } + + /* + * Don't allow persistent virtual kstats -- it makes no sense. + * ks_data points to garbage when the client goes away. + */ + if ((ks_flags & KSTAT_FLAG_PERSISTENT) && + (ks_flags & KSTAT_FLAG_VIRTUAL)) { + cmn_err(CE_WARN, "kstat_create('%s', %d, '%s'): " + "cannot create persistent virtual kstat", + ks_module, ks_instance, ks_name); + return (NULL); + } + + /* + * Don't allow variable-size physical kstats, since the framework's + * memory allocation for physical kstat data is fixed at creation time. + */ + if ((ks_flags & KSTAT_FLAG_VAR_SIZE) && + !(ks_flags & KSTAT_FLAG_VIRTUAL)) { + cmn_err(CE_WARN, "kstat_create('%s', %d, '%s'): " + "cannot create variable-size physical kstat", + ks_module, ks_instance, ks_name); + return (NULL); + } + + /* + * Make sure the number of data fields is within legal range + */ + if (ks_ndata < kstat_data_type[ks_type].min_ndata || + ks_ndata > kstat_data_type[ks_type].max_ndata) { + cmn_err(CE_WARN, "kstat_create('%s', %d, '%s'): " + "ks_ndata=%d out of range [%d, %d]", + ks_module, ks_instance, ks_name, (int)ks_ndata, + kstat_data_type[ks_type].min_ndata, + kstat_data_type[ks_type].max_ndata); + return (NULL); + } + + ks_data_size = kstat_data_type[ks_type].size * ks_ndata; + + /* + * If the named kstat already exists and is dormant, reactivate it. + */ + ksp = kstat_hold_byname(ks_module, ks_instance, ks_name, ks_zoneid); + if (ksp != NULL) { + if (!(ksp->ks_flags & KSTAT_FLAG_DORMANT)) { + /* + * The named kstat exists but is not dormant -- + * this is a kstat namespace collision. + */ + kstat_rele(ksp); + cmn_err(CE_WARN, + "kstat_create('%s', %d, '%s'): namespace collision", + ks_module, ks_instance, ks_name); + return (NULL); + } + if ((strcmp(ksp->ks_class, ks_class) != 0) || + (ksp->ks_type != ks_type) || + (ksp->ks_ndata != ks_ndata) || + (ks_flags & KSTAT_FLAG_VIRTUAL)) { + /* + * The name is the same, but the other key parameters + * differ from those of the dormant kstat -- bogus. + */ + kstat_rele(ksp); + cmn_err(CE_WARN, "kstat_create('%s', %d, '%s'): " + "invalid reactivation of dormant kstat", + ks_module, ks_instance, ks_name); + return (NULL); + } + /* + * Return dormant kstat pointer to caller. As usual, + * the kstat is marked invalid until kstat_install(). + */ + ksp->ks_flags |= KSTAT_FLAG_INVALID; + kstat_rele(ksp); + return (ksp); + } + + /* + * Allocate memory for the new kstat header and, if this is a physical + * kstat, the data section. + */ + e = kstat_alloc(ks_flags & KSTAT_FLAG_VIRTUAL ? 0 : ks_data_size); + if (e == NULL) { + cmn_err(CE_NOTE, "kstat_create('%s', %d, '%s'): " + "insufficient kernel memory", + ks_module, ks_instance, ks_name); + return (NULL); + } + + /* + * Initialize as many fields as we can. The caller may reset + * ks_lock, ks_update, ks_private, and ks_snapshot as necessary. + * Creators of virtual kstats may also reset ks_data. It is + * also up to the caller to initialize the kstat data section, + * if necessary. All initialization must be complete before + * calling kstat_install(). + */ + e->e_zone.zoneid = ks_zoneid; + e->e_zone.next = NULL; + + ksp = &e->e_ks; + ksp->ks_crtime = gethrtime(); + kstat_set_string(ksp->ks_module, ks_module); + ksp->ks_instance = ks_instance; + kstat_set_string(ksp->ks_name, ks_name); + ksp->ks_type = ks_type; + kstat_set_string(ksp->ks_class, ks_class); + ksp->ks_flags = ks_flags | KSTAT_FLAG_INVALID; + if (ks_flags & KSTAT_FLAG_VIRTUAL) + ksp->ks_data = NULL; + else + ksp->ks_data = (void *)(e + 1); + ksp->ks_ndata = ks_ndata; + ksp->ks_data_size = ks_data_size; + ksp->ks_snaptime = ksp->ks_crtime; + ksp->ks_update = default_kstat_update; + ksp->ks_private = NULL; + ksp->ks_private1 = NULL; + ksp->ks_snapshot = default_kstat_snapshot; + mutex_init(&ksp->ks_private_lock, NULL, MUTEX_DEFAULT, NULL); + ksp->ks_lock = &ksp->ks_private_lock; + + mutex_enter(&kstat_chain_lock); + + /* + * Add our kstat to the AVL trees. + */ + if (avl_find(&kstat_avl_byname, e, &where) != NULL) { + mutex_exit(&kstat_chain_lock); + cmn_err(CE_WARN, + "kstat_create('%s', %d, '%s'): namespace collision", + ks_module, ks_instance, ks_name); + kstat_free(e); + return (NULL); + } + avl_insert(&kstat_avl_byname, e, where); + + /* + * Loop around until we find an unused KID. + */ + do { + ksp->ks_kid = kstat_chain_id++; + } while (avl_find(&kstat_avl_bykid, e, &where) != NULL); + avl_insert(&kstat_avl_bykid, e, where); + + mutex_exit(&kstat_chain_lock); + + return (ksp); +} + +/* + * Activate a fully initialized kstat and make it visible to /dev/kstat. + */ +void +kstat_install(kstat_t *ksp) +{ + zoneid_t zoneid = ((ekstat_t *)ksp)->e_zone.zoneid; + + /* + * If this is a variable-size kstat, it MUST provide kstat data locking + * to prevent data-size races with kstat readers. + */ + if ((ksp->ks_flags & KSTAT_FLAG_VAR_SIZE) && ksp->ks_lock == NULL) { + panic("kstat_install('%s', %d, '%s'): " + "cannot create variable-size kstat without data lock", + ksp->ks_module, ksp->ks_instance, ksp->ks_name); + } + + if (kstat_hold_bykid(ksp->ks_kid, zoneid) != ksp) { + cmn_err(CE_WARN, "kstat_install(%p): does not exist", + (void *)ksp); + return; + } + + if (ksp->ks_type == KSTAT_TYPE_NAMED && ksp->ks_data != NULL) { + uint_t i; + kstat_named_t *knp = KSTAT_NAMED_PTR(ksp); + + for (i = 0; i < ksp->ks_ndata; i++, knp++) { + if (knp->data_type == KSTAT_DATA_STRING) { + ksp->ks_flags |= KSTAT_FLAG_LONGSTRINGS; + break; + } + } + /* + * The default snapshot routine does not handle KSTAT_WRITE + * for long strings. + */ + if ((ksp->ks_flags & KSTAT_FLAG_LONGSTRINGS) && + (ksp->ks_flags & KSTAT_FLAG_WRITABLE) && + (ksp->ks_snapshot == default_kstat_snapshot)) { + panic("kstat_install('%s', %d, '%s'): " + "named kstat containing KSTAT_DATA_STRING " + "is writable but uses default snapshot routine", + ksp->ks_module, ksp->ks_instance, ksp->ks_name); + } + } + + if (ksp->ks_flags & KSTAT_FLAG_DORMANT) { + + /* + * We are reactivating a dormant kstat. Initialize the + * caller's underlying data to the value it had when the + * kstat went dormant, and mark the kstat as active. + * Grab the provider's kstat lock if it's not already held. + */ + kmutex_t *lp = ksp->ks_lock; + if (lp != NULL && MUTEX_NOT_HELD(lp)) { + mutex_enter(lp); + (void) KSTAT_UPDATE(ksp, KSTAT_WRITE); + mutex_exit(lp); + } else { + (void) KSTAT_UPDATE(ksp, KSTAT_WRITE); + } + ksp->ks_flags &= ~KSTAT_FLAG_DORMANT; + } + + /* + * Now that the kstat is active, make it visible to the kstat driver. + * When copying out kstats the count is determined in + * header_kstat_update() and actually copied into kbuf in + * header_kstat_snapshot(). kstat_chain_lock is held across the two + * calls to ensure that this list doesn't change. Thus, we need to + * also take the lock to ensure that the we don't copy the new kstat + * in the 2nd pass and overrun the buf. + */ + mutex_enter(&kstat_chain_lock); + ksp->ks_flags &= ~KSTAT_FLAG_INVALID; + mutex_exit(&kstat_chain_lock); + kstat_rele(ksp); +} + +/* + * Remove a kstat from the system. Or, if it's a persistent kstat, + * just update the data and mark it as dormant. + */ +void +kstat_delete(kstat_t *ksp) +{ + kmutex_t *lp; + ekstat_t *e = (ekstat_t *)ksp; + zoneid_t zoneid; + kstat_zone_t *kz; + + ASSERT(ksp != NULL); + + if (ksp == NULL) + return; + + zoneid = e->e_zone.zoneid; + + lp = ksp->ks_lock; + ksp->ks_lock = NULL; + + if (lp != NULL && MUTEX_HELD(lp)) { + panic("kstat_delete(%p): caller holds data lock %p", + (void *)ksp, (void *)lp); + } + + if (kstat_hold_bykid(ksp->ks_kid, zoneid) != ksp) { + cmn_err(CE_WARN, "kstat_delete(%p): does not exist", + (void *)ksp); + return; + } + + if (ksp->ks_flags & KSTAT_FLAG_PERSISTENT) { + /* + * Update the data one last time, so that all activity + * prior to going dormant has been accounted for. + */ + KSTAT_ENTER(ksp); + (void) KSTAT_UPDATE(ksp, KSTAT_READ); + KSTAT_EXIT(ksp); + + /* + * Mark the kstat as dormant and restore caller-modifiable + * fields to default values, so the kstat is readable during + * the dormant phase. + */ + ksp->ks_flags |= KSTAT_FLAG_DORMANT; + ksp->ks_lock = NULL; + ksp->ks_update = default_kstat_update; + ksp->ks_private = NULL; + ksp->ks_snapshot = default_kstat_snapshot; + kstat_rele(ksp); + return; + } + + /* + * Remove the kstat from the framework's AVL trees, + * free the allocated memory, and increment kstat_chain_id so + * /dev/kstat clients can detect the event. + */ + mutex_enter(&kstat_chain_lock); + avl_remove(&kstat_avl_bykid, e); + avl_remove(&kstat_avl_byname, e); + kstat_chain_id++; + mutex_exit(&kstat_chain_lock); + + mutex_destroy(&ksp->ks_private_lock); + + kz = e->e_zone.next; + while (kz != NULL) { + kstat_zone_t *t = kz; + + kz = kz->next; + kmem_free(t, sizeof (*t)); + } + kstat_rele(ksp); + kstat_free(e); +} + +void +kstat_delete_byname_zone(const char *ks_module, int ks_instance, + const char *ks_name, zoneid_t ks_zoneid) +{ + kstat_t *ksp; + + ksp = kstat_hold_byname(ks_module, ks_instance, ks_name, ks_zoneid); + if (ksp != NULL) { + kstat_rele(ksp); + kstat_delete(ksp); + } +} + +void +kstat_delete_byname(const char *ks_module, int ks_instance, const char *ks_name) +{ + kstat_delete_byname_zone(ks_module, ks_instance, ks_name, ALL_ZONES); +} + +/* + * The sparc V9 versions of these routines can be much cheaper than + * the poor 32-bit compiler can comprehend, so they're in sparcv9_subr.s. + * For simplicity, however, we always feed the C versions to lint. + */ +#if !defined(__sparc) || defined(lint) || defined(__lint) + +void +kstat_waitq_enter(kstat_io_t *kiop) +{ + hrtime_t new, delta; + ulong_t wcnt; + + new = gethrtime(); + delta = new - kiop->wlastupdate; + kiop->wlastupdate = new; + wcnt = kiop->wcnt++; + if (wcnt != 0) { + kiop->wlentime += delta * wcnt; + kiop->wtime += delta; + } +} + +void +kstat_waitq_exit(kstat_io_t *kiop) +{ + hrtime_t new, delta; + ulong_t wcnt; + + new = gethrtime(); + delta = new - kiop->wlastupdate; + kiop->wlastupdate = new; + wcnt = kiop->wcnt--; + ASSERT((int)wcnt > 0); + kiop->wlentime += delta * wcnt; + kiop->wtime += delta; +} + +void +kstat_runq_enter(kstat_io_t *kiop) +{ + hrtime_t new, delta; + ulong_t rcnt; + + new = gethrtime(); + delta = new - kiop->rlastupdate; + kiop->rlastupdate = new; + rcnt = kiop->rcnt++; + if (rcnt != 0) { + kiop->rlentime += delta * rcnt; + kiop->rtime += delta; + } +} + +void +kstat_runq_exit(kstat_io_t *kiop) +{ + hrtime_t new, delta; + ulong_t rcnt; + + new = gethrtime(); + delta = new - kiop->rlastupdate; + kiop->rlastupdate = new; + rcnt = kiop->rcnt--; + ASSERT((int)rcnt > 0); + kiop->rlentime += delta * rcnt; + kiop->rtime += delta; +} + +void +kstat_waitq_to_runq(kstat_io_t *kiop) +{ + hrtime_t new, delta; + ulong_t wcnt, rcnt; + + new = gethrtime(); + + delta = new - kiop->wlastupdate; + kiop->wlastupdate = new; + wcnt = kiop->wcnt--; + ASSERT((int)wcnt > 0); + kiop->wlentime += delta * wcnt; + kiop->wtime += delta; + + delta = new - kiop->rlastupdate; + kiop->rlastupdate = new; + rcnt = kiop->rcnt++; + if (rcnt != 0) { + kiop->rlentime += delta * rcnt; + kiop->rtime += delta; + } +} + +void +kstat_runq_back_to_waitq(kstat_io_t *kiop) +{ + hrtime_t new, delta; + ulong_t wcnt, rcnt; + + new = gethrtime(); + + delta = new - kiop->rlastupdate; + kiop->rlastupdate = new; + rcnt = kiop->rcnt--; + ASSERT((int)rcnt > 0); + kiop->rlentime += delta * rcnt; + kiop->rtime += delta; + + delta = new - kiop->wlastupdate; + kiop->wlastupdate = new; + wcnt = kiop->wcnt++; + if (wcnt != 0) { + kiop->wlentime += delta * wcnt; + kiop->wtime += delta; + } +} + +#endif + +void +kstat_timer_start(kstat_timer_t *ktp) +{ + ktp->start_time = gethrtime(); +} + +void +kstat_timer_stop(kstat_timer_t *ktp) +{ + hrtime_t etime; + u_longlong_t num_events; + + ktp->stop_time = etime = gethrtime(); + etime -= ktp->start_time; + num_events = ktp->num_events; + if (etime < ktp->min_time || num_events == 0) + ktp->min_time = etime; + if (etime > ktp->max_time) + ktp->max_time = etime; + ktp->elapsed_time += etime; + ktp->num_events = num_events + 1; +} + +/* io/kstat.c */ + +/* + * kernel statistics driver + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// static dev_info_t *kstat_devi; + +static int +read_kstat_data(int *rvalp, void *user_ksp, int flag) +{ + kstat_t user_kstat, *ksp; +#ifdef _MULTI_DATAMODEL + kstat32_t user_kstat32; +#endif + void *kbuf = NULL; + size_t kbufsize, ubufsize, copysize, allocsize; + int error = 0; + uint_t model; + +#define DDI_MODEL_NONE 0 + switch (model = DDI_MODEL_NONE) { +#ifdef _MULTI_DATAMODEL + case DDI_MODEL_ILP32: + if (copyin(user_ksp, &user_kstat32, sizeof (kstat32_t)) != 0) + return (EFAULT); + user_kstat.ks_kid = user_kstat32.ks_kid; + user_kstat.ks_data = (void *)(uintptr_t)user_kstat32.ks_data; + user_kstat.ks_data_size = (size_t)user_kstat32.ks_data_size; + break; +#endif + default: + case DDI_MODEL_NONE: + if (ddi_copyin(user_ksp, &user_kstat, sizeof (kstat_t), 0) != 0) + return (EFAULT); + } + + ksp = kstat_hold_bykid(user_kstat.ks_kid, crgetzoneid()); + if (ksp == NULL) { + /* + * There is no kstat with the specified KID + */ + return (ENXIO); + } + if (ksp->ks_flags & KSTAT_FLAG_INVALID) { + /* + * The kstat exists, but is momentarily in some + * indeterminate state (e.g. the data section is not + * yet initialized). Try again in a few milliseconds. + */ + kstat_rele(ksp); + return (EAGAIN); + } + + /* + * If it's a fixed-size kstat, allocate the buffer now, so we + * don't have to do it under the kstat's data lock. (If it's a + * var-size kstat or one with long strings, we don't know the size + * until after the update routine is called, so we can't do this + * optimization.) + * The allocator relies on this behavior to prevent recursive + * mutex_enter in its (fixed-size) kstat update routine. + * It's a zalloc to prevent unintentional exposure of random + * juicy morsels of (old) kernel data. + */ + if (!(ksp->ks_flags & (KSTAT_FLAG_VAR_SIZE | KSTAT_FLAG_LONGSTRINGS))) { + kbufsize = ksp->ks_data_size; + allocsize = kbufsize + 1; + kbuf = kmem_zalloc(allocsize, KM_NOSLEEP); + if (kbuf == NULL) { + kstat_rele(ksp); + return (EAGAIN); + } + } + KSTAT_ENTER(ksp); + if ((error = KSTAT_UPDATE(ksp, KSTAT_READ)) != 0) { + KSTAT_EXIT(ksp); + kstat_rele(ksp); + if (kbuf != NULL) + kmem_free(kbuf, allocsize); + return (error); + } + + kbufsize = ksp->ks_data_size; + ubufsize = user_kstat.ks_data_size; + + if (ubufsize < kbufsize) { + error = ENOMEM; + } else { + if (kbuf == NULL) { + allocsize = kbufsize + 1; + kbuf = kmem_zalloc(allocsize, KM_NOSLEEP); + } + if (kbuf == NULL) { + error = EAGAIN; + } else { + error = KSTAT_SNAPSHOT(ksp, kbuf, KSTAT_READ); + } + } + + /* + * The following info must be returned to user level, + * even if the the update or snapshot failed. This allows + * kstat readers to get a handle on variable-size kstats, + * detect dormant kstats, etc. + */ + user_kstat.ks_ndata = ksp->ks_ndata; + user_kstat.ks_data_size = kbufsize; + user_kstat.ks_flags = ksp->ks_flags; + user_kstat.ks_snaptime = ksp->ks_snaptime; +#ifndef _WIN32 + *rvalp = kstat_chain_id; +#else + // The above doesn't work, as rvalp refers to the userland struct, + // before copyin() and we need to write value to kernel version. + user_kstat.ks_returnvalue = kstat_chain_id; +#endif + KSTAT_EXIT(ksp); + kstat_rele(ksp); + + if (kbuf == NULL) + goto out; + + /* + * Copy the buffer containing the kstat back to userland. + */ + copysize = kbufsize; + + switch (model) { + int i; +#ifdef _MULTI_DATAMODEL + kstat32_t *k32; + kstat_t *k; + + case DDI_MODEL_ILP32: + + if (ksp->ks_type == KSTAT_TYPE_NAMED) { + kstat_named_t *kn = kbuf; + char *strbuf = (char *)((kstat_named_t *)kn + + ksp->ks_ndata); + + for (i = 0; i < user_kstat.ks_ndata; kn++, i++) + switch (kn->data_type) { + /* + * Named statistics have fields of type 'long'. + * For a 32-bit application looking at a 64-bit + * kernel, forcibly truncate these 64-bit + * quantities to 32-bit values. + */ + case KSTAT_DATA_LONG: + kn->value.i32 = (int32_t)kn->value.l; + kn->data_type = KSTAT_DATA_INT32; + break; + case KSTAT_DATA_ULONG: + kn->value.ui32 = (uint32_t)kn->value.ul; + kn->data_type = KSTAT_DATA_UINT32; + break; + /* + * Long strings must be massaged before being + * copied out to userland. Do that here. + */ + case KSTAT_DATA_STRING: + if (KSTAT_NAMED_STR_PTR(kn) == NULL) + break; + /* + * If the string lies outside of kbuf + * copy it there and update the pointer. + */ + if (KSTAT_NAMED_STR_PTR(kn) < + (char *)kbuf || + KSTAT_NAMED_STR_PTR(kn) + + KSTAT_NAMED_STR_BUFLEN(kn) > + (char *)kbuf + kbufsize + 1) { + memcpy(strbuf, + KSTAT_NAMED_STR_PTR(kn), + KSTAT_NAMED_STR_BUFLEN(kn)); + + KSTAT_NAMED_STR_PTR(kn) = + strbuf; + strbuf += + KSTAT_NAMED_STR_BUFLEN(kn); + ASSERT(strbuf <= + (char *)kbuf + + kbufsize + 1); + } + /* + * The offsets within the buffers are + * the same, so add the offset to the + * beginning of the new buffer to fix + * the pointer. + */ + KSTAT_NAMED_STR_PTR(kn) = + (char *)user_kstat.ks_data + + (KSTAT_NAMED_STR_PTR(kn) - + (char *)kbuf); + /* + * Make sure the string pointer lies + * within the allocated buffer. + */ + ASSERT(KSTAT_NAMED_STR_PTR(kn) + + KSTAT_NAMED_STR_BUFLEN(kn) <= + ((char *)user_kstat.ks_data + + ubufsize)); + ASSERT(KSTAT_NAMED_STR_PTR(kn) >= + (char *)((kstat_named_t *) + user_kstat.ks_data + + user_kstat.ks_ndata)); + /* + * Cast 64-bit ptr to 32-bit. + */ + kn->value.str.addr.ptr32 = + (caddr32_t)(uintptr_t) + KSTAT_NAMED_STR_PTR(kn); + break; + default: + break; + } + } + + if (user_kstat.ks_kid != 0) + break; + + /* + * This is the special case of the kstat header + * list for the entire system. Reshape the + * array in place, then copy it out. + */ + k32 = kbuf; + k = kbuf; + for (i = 0; i < user_kstat.ks_ndata; k32++, k++, i++) { + k32->ks_crtime = k->ks_crtime; + k32->ks_next = 0; + k32->ks_kid = k->ks_kid; + (void) strcpy(k32->ks_module, k->ks_module); + k32->ks_resv = k->ks_resv; + k32->ks_instance = k->ks_instance; + (void) strcpy(k32->ks_name, k->ks_name); + k32->ks_type = k->ks_type; + (void) strcpy(k32->ks_class, k->ks_class); + k32->ks_flags = k->ks_flags; + k32->ks_data = 0; + k32->ks_ndata = k->ks_ndata; + if (k->ks_data_size > UINT32_MAX) { + error = EOVERFLOW; + break; + } + k32->ks_data_size = (size32_t)k->ks_data_size; + k32->ks_snaptime = k->ks_snaptime; + } + + /* + * XXX In this case we copy less data than is + * claimed in the header. + */ + copysize = user_kstat.ks_ndata * sizeof (kstat32_t); + break; +#endif /* _MULTI_DATAMODEL */ + default: + case DDI_MODEL_NONE: + if (ksp->ks_type == KSTAT_TYPE_NAMED) { + kstat_named_t *kn = kbuf; + char *strbuf = (char *)((kstat_named_t *)kn + + ksp->ks_ndata); + + for (i = 0; i < user_kstat.ks_ndata; kn++, i++) + switch (kn->data_type) { +#ifdef _LP64 + case KSTAT_DATA_LONG: + kn->data_type = + KSTAT_DATA_INT64; + break; + case KSTAT_DATA_ULONG: + kn->data_type = + KSTAT_DATA_UINT64; + break; +#endif /* _LP64 */ + case KSTAT_DATA_STRING: + if (KSTAT_NAMED_STR_PTR(kn) == NULL) + break; + /* + * If the string lies outside of kbuf + * copy it there and update the pointer. + */ + if (KSTAT_NAMED_STR_PTR(kn) < + (char *)kbuf || + KSTAT_NAMED_STR_PTR(kn) + + KSTAT_NAMED_STR_BUFLEN(kn) > + (char *)kbuf + kbufsize + 1) { + memcpy(strbuf, + KSTAT_NAMED_STR_PTR(kn), + KSTAT_NAMED_STR_BUFLEN(kn)); + + KSTAT_NAMED_STR_PTR(kn) = + strbuf; + strbuf += + KSTAT_NAMED_STR_BUFLEN(kn); + ASSERT(strbuf <= + (char *)kbuf + + kbufsize + 1); + } + + KSTAT_NAMED_STR_PTR(kn) = + (char *)user_kstat.ks_data + + (KSTAT_NAMED_STR_PTR(kn) - + (char *)kbuf); + ASSERT(KSTAT_NAMED_STR_PTR(kn) + + KSTAT_NAMED_STR_BUFLEN(kn) <= + ((char *)user_kstat.ks_data + + ubufsize)); + ASSERT(KSTAT_NAMED_STR_PTR(kn) >= + (char *)((kstat_named_t *) + user_kstat.ks_data + + user_kstat.ks_ndata)); + break; + default: + break; + } + } + break; + } + + if (error == 0 && + ddi_copyout(kbuf, user_kstat.ks_data, copysize, 0)) + error = EFAULT; + kmem_free(kbuf, allocsize); + +out: + /* + * We have modified the ks_ndata, ks_data_size, ks_flags, and + * ks_snaptime fields of the user kstat; now copy it back to userland. + */ + switch (model) { +#ifdef _MULTI_DATAMODEL + case DDI_MODEL_ILP32: + if (kbufsize > UINT32_MAX) { + error = EOVERFLOW; + break; + } + user_kstat32.ks_ndata = user_kstat.ks_ndata; + user_kstat32.ks_data_size = (size32_t)kbufsize; + user_kstat32.ks_flags = user_kstat.ks_flags; + user_kstat32.ks_snaptime = user_kstat.ks_snaptime; + if (copyout(&user_kstat32, user_ksp, sizeof (kstat32_t)) && + error == 0) + error = EFAULT; + break; +#endif + default: + case DDI_MODEL_NONE: + // If we have an errorcode, set it in ks_errnovalue + // Above sets returnvalue with *rval = + // Must be done before this copyout() + user_kstat.ks_errnovalue = 0; + if (error) { + user_kstat.ks_errnovalue = error; + user_kstat.ks_returnvalue = -1; + } + if (ddi_copyout(&user_kstat, user_ksp, sizeof (kstat_t), 0) && + error == 0) + error = EFAULT; + break; + } + + return (error); +} + +static int +write_kstat_data(int *rvalp, void *user_ksp, int flag, cred_t *cred) +{ + kstat_t user_kstat, *ksp; + void *buf = NULL; + size_t bufsize; + int error = 0; + + if (secpolicy_sys_config(cred, B_FALSE) != 0) + return (EPERM); + + switch (DDI_MODEL_NONE) { +#ifdef _MULTI_DATAMODEL + kstat32_t user_kstat32; + + case DDI_MODEL_ILP32: + if (copyin(user_ksp, &user_kstat32, sizeof (kstat32_t))) + return (EFAULT); + /* + * These are the only fields we actually look at. + */ + user_kstat.ks_kid = user_kstat32.ks_kid; + user_kstat.ks_data = (void *)(uintptr_t)user_kstat32.ks_data; + user_kstat.ks_data_size = (size_t)user_kstat32.ks_data_size; + user_kstat.ks_ndata = user_kstat32.ks_ndata; + break; +#endif + default: + case DDI_MODEL_NONE: + if (ddi_copyin(user_ksp, &user_kstat, sizeof (kstat_t), 0)) + return (EFAULT); + } + + bufsize = user_kstat.ks_data_size; + buf = kmem_alloc(bufsize + 1, KM_NOSLEEP); + if (buf == NULL) + return (EAGAIN); + + if (ddi_copyin(user_kstat.ks_data, buf, bufsize, 0)) { + kmem_free(buf, bufsize + 1); + return (EFAULT); + } + + ksp = kstat_hold_bykid(user_kstat.ks_kid, crgetzoneid()); + if (ksp == NULL) { + kmem_free(buf, bufsize + 1); + return (ENXIO); + } + if (ksp->ks_flags & KSTAT_FLAG_INVALID) { + kstat_rele(ksp); + kmem_free(buf, bufsize + 1); + return (EAGAIN); + } + if (!(ksp->ks_flags & KSTAT_FLAG_WRITABLE)) { + kstat_rele(ksp); + kmem_free(buf, bufsize + 1); + return (EACCES); + } + + /* + * With KSTAT_FLAG_VAR_SIZE, one must call the kstat's update callback + * routine to ensure ks_data_size is up to date. + * In this case it makes sense to do it anyhow, as it will be shortly + * followed by a KSTAT_SNAPSHOT(). + */ + KSTAT_ENTER(ksp); + error = KSTAT_UPDATE(ksp, KSTAT_READ); + if (error || user_kstat.ks_data_size != ksp->ks_data_size || + user_kstat.ks_ndata != ksp->ks_ndata) { + KSTAT_EXIT(ksp); + kstat_rele(ksp); + kmem_free(buf, bufsize + 1); + return (error ? error : EINVAL); + } + + /* + * We have to ensure that we don't accidentally change the type of + * existing kstat_named statistics when writing over them. + * Since read_kstat_data() modifies some of the types on their way + * out, we need to be sure to handle these types seperately. + */ + if (ksp->ks_type == KSTAT_TYPE_NAMED) { + void *kbuf; + kstat_named_t *kold; + kstat_named_t *knew = buf; + int i; + +#ifdef _MULTI_DATAMODEL + int model = ddi_model_convert_from(flag & FMODELS); +#endif + + /* + * Since ksp->ks_data may be NULL, we need to take a snapshot + * of the published data to look at the types. + */ + kbuf = kmem_alloc(bufsize + 1, KM_NOSLEEP); + if (kbuf == NULL) { + KSTAT_EXIT(ksp); + kstat_rele(ksp); + kmem_free(buf, bufsize + 1); + return (EAGAIN); + } + error = KSTAT_SNAPSHOT(ksp, kbuf, KSTAT_READ); + if (error) { + KSTAT_EXIT(ksp); + kstat_rele(ksp); + kmem_free(kbuf, bufsize + 1); + kmem_free(buf, bufsize + 1); + return (error); + } + kold = kbuf; + + /* + * read_kstat_data() changes the types of + * KSTAT_DATA_LONG / KSTAT_DATA_ULONG, so we need to + * make sure that these (modified) types are considered + * valid. + */ + for (i = 0; i < ksp->ks_ndata; i++, kold++, knew++) { + switch (kold->data_type) { +#ifdef _MULTI_DATAMODEL + case KSTAT_DATA_LONG: + switch (model) { + case DDI_MODEL_ILP32: + if (knew->data_type == + KSTAT_DATA_INT32) { + knew->value.l = + (long)knew->value.i32; + knew->data_type = + KSTAT_DATA_LONG; + } + break; + default: + case DDI_MODEL_NONE: +#ifdef _LP64 + if (knew->data_type == + KSTAT_DATA_INT64) { + knew->value.l = + (long)knew->value.i64; + knew->data_type = + KSTAT_DATA_LONG; + } +#endif /* _LP64 */ + break; + } + break; + case KSTAT_DATA_ULONG: + switch (model) { + case DDI_MODEL_ILP32: + if (knew->data_type == + KSTAT_DATA_UINT32) { + knew->value.ul = + (ulong_t)knew->value.ui32; + knew->data_type = + KSTAT_DATA_ULONG; + } + break; + default: + case DDI_MODEL_NONE: +#ifdef _LP64 + if (knew->data_type == + KSTAT_DATA_UINT64) { + knew->value.ul = + (ulong_t)knew->value.ui64; + knew->data_type = + KSTAT_DATA_ULONG; + } +#endif /* _LP64 */ + break; + } + break; +#endif /* _MULTI_DATAMODEL */ + case KSTAT_DATA_STRING: + if (knew->data_type != KSTAT_DATA_STRING) { + KSTAT_EXIT(ksp); + kstat_rele(ksp); + kmem_free(kbuf, bufsize + 1); + kmem_free(buf, bufsize + 1); + return (EINVAL); + } + +#ifdef _MULTI_DATAMODEL + if (model == DDI_MODEL_ILP32) + KSTAT_NAMED_STR_PTR(knew) = + (char *)(uintptr_t) + knew->value.str.addr.ptr32; +#endif + /* + * Nothing special for NULL + */ + if (KSTAT_NAMED_STR_PTR(knew) == NULL) + break; + + /* + * Check to see that the pointers all point + * to within the buffer and after the array + * of kstat_named_t's. + */ + if (KSTAT_NAMED_STR_PTR(knew) < + (char *) + ((kstat_named_t *)user_kstat.ks_data + + ksp->ks_ndata)) { + KSTAT_EXIT(ksp); + kstat_rele(ksp); + kmem_free(kbuf, bufsize + 1); + kmem_free(buf, bufsize + 1); + return (EINVAL); + } + if (KSTAT_NAMED_STR_PTR(knew) + + KSTAT_NAMED_STR_BUFLEN(knew) > + ((char *)user_kstat.ks_data + + ksp->ks_data_size)) { + KSTAT_EXIT(ksp); + kstat_rele(ksp); + kmem_free(kbuf, bufsize + 1); + kmem_free(buf, bufsize + 1); + return (EINVAL); + } + + /* + * Update the pointers within the buffer + */ + KSTAT_NAMED_STR_PTR(knew) = + (char *)buf + + (KSTAT_NAMED_STR_PTR(knew) - + (char *)user_kstat.ks_data); + break; + default: + break; + } + } + + kold = kbuf; + knew = buf; + + /* + * Now make sure the types are what we expected them to be. + */ + for (i = 0; i < ksp->ks_ndata; i++, kold++, knew++) + if (kold->data_type != knew->data_type) { + KSTAT_EXIT(ksp); + kstat_rele(ksp); + kmem_free(kbuf, bufsize + 1); + kmem_free(buf, bufsize + 1); + return (EINVAL); + } + + kmem_free(kbuf, bufsize + 1); + } + + error = KSTAT_SNAPSHOT(ksp, buf, KSTAT_WRITE); + if (!error) + error = KSTAT_UPDATE(ksp, KSTAT_WRITE); +#ifndef _WIN32 + *rvalp = kstat_chain_id; +#else + // The above doesn't work, as rvalp refers to the userland struct, + // before copyin() and we need to write value to kernel version. + user_kstat.ks_returnvalue = kstat_chain_id; + // We need to copyout() so userland will get the return values. +#endif + + KSTAT_EXIT(ksp); + kstat_rele(ksp); + kmem_free(buf, bufsize + 1); + return (error); +} + +/* spl-kstat.c */ + +void +spl_kstat_init() +{ + /* + * Create the kstat root OID + */ + mutex_init(&kstat_chain_lock, NULL, MUTEX_DEFAULT, NULL); +} + +void +spl_kstat_fini() +{ + /* + * Destroy the kstat module/class/name tree + * + * Done in two passes, first unregisters all + * of the oids, second releases all the memory. + */ + + vmem_fini(kstat_arena); + mutex_destroy(&kstat_chain_lock); +} + +void +__kstat_set_raw_ops(kstat_t *ksp, + int (*headers)(char *buf, size_t size), + int (*data)(char *buf, size_t size, void *data), + void *(*addr)(kstat_t *ksp, loff_t index)) +{ + ksp->ks_raw_ops.headers = headers; + ksp->ks_raw_ops.data = data; + ksp->ks_raw_ops.addr = addr; +} + +void +__kstat_set_seq_raw_ops(kstat_t *ksp, + int (*headers)(struct seq_file *f), + int (*data)(char *buf, size_t size, void *data), + void *(*addr)(kstat_t *ksp, loff_t index)) +{ + ksp->ks_raw_ops.seq_headers = headers; + ksp->ks_raw_ops.data = data; + ksp->ks_raw_ops.addr = addr; +} + +int spl_kstat_chain_id(PDEVICE_OBJECT DiskDevice, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + kstat_t ksp = { 0 }; + ksp.ks_returnvalue = kstat_chain_id; + ASSERT3U(IrpSp->Parameters.DeviceIoControl.OutputBufferLength, >=, + sizeof (ksp)); + ddi_copyout(&ksp, IrpSp->Parameters.DeviceIoControl.Type3InputBuffer, + sizeof (ksp), 0); + dprintf("%s:%d: returning kstat_chain_id %d\n", + __func__, __LINE__, kstat_chain_id); + return (0); +} + +int spl_kstat_read(PDEVICE_OBJECT DiskDevice, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + int rc; + kstat_t *ksp; + ksp = (kstat_t *)IrpSp->Parameters.DeviceIoControl.Type3InputBuffer; + rc = read_kstat_data(&ksp->ks_returnvalue, (void *)ksp, 0); + return (0); +} + +int spl_kstat_write(PDEVICE_OBJECT DiskDevice, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + int rc; + kstat_t *ksp; + ksp = (kstat_t *)IrpSp->Parameters.DeviceIoControl.Type3InputBuffer; + rc = write_kstat_data(&ksp->ks_returnvalue, (void *)ksp, 0, NULL); + return (0); +} + +// Added comments inline referring to perl arcstat.pl +void +arc_cache_counters_perfmon(cache_counters *perf, arc_stats_t *arc_ptr) +{ + // $v{ "hits" } = $d{ "hits" } / $int; + perf->arcstat_hits = arc_ptr->arcstat_hits.value.ui64; + + // $v{"miss"} = $d{"misses"}/$int; + perf->arcstat_misses = arc_ptr->arcstat_misses.value.ui64; + + // $v{"dhit"} = ($d{"demand_data_hits"} + + // $d{"demand_metadata_hits"})/$int; + perf->arcstat_total_demand_hits = arc_ptr-> + arcstat_demand_data_hits.value.ui64 + + arc_ptr->arcstat_demand_metadata_hits.value.ui64; + + // $v{"dmis"} = ($d{"demand_data_misses"}+ + // $d{"demand_metadata_misses"})/$int; + perf->arcstat_total_demand_miss = arc_ptr-> + arcstat_demand_data_misses.value.ui64 + + arc_ptr->arcstat_demand_metadata_misses.value.ui64; + + // $v{"phit"}=($d{"prefetch_data_hits"} + + // $d{"prefetch_metadata_hits"})/$int; + + perf->arcstat_perfetch_hits = + arc_ptr->arcstat_prefetch_data_hits.value.ui64 + + arc_ptr->arcstat_prefetch_metadata_hits.value.ui64; + + // $v{ "pmis" } = ($d{ "prefetch_data_misses" } + + // $d{ "prefetch_metadata_misses" }) / $int; + perf->arcstat_perfetch_miss = + arc_ptr->arcstat_prefetch_data_misses.value.ui64 + + arc_ptr->arcstat_prefetch_metadata_misses.value.ui64; + + // $v{"pread"} = $v{"phit"} + $v{"pmis"}; + perf->arcstat_perfetch_ps = perf->arcstat_perfetch_hits + + perf->arcstat_perfetch_miss; + + // $v{"dread"} = $v{"dhit"} + $v{"dmis"}; + perf->arcstat_demand_ps = perf->arcstat_total_demand_hits + + perf->arcstat_total_demand_miss; + + // $v{"size"} = $cur{"size"}; + perf->arcstat_size = arc_ptr->arcstat_size.value.ui64; + + // $v{"tsize"} = $cur{"c"}; + perf->arcstat_c = arc_ptr->arcstat_c.value.ui64; + + // $v{"mfu"} = $d{"hits"}/$int; + perf->arcstat_mfu_hits = arc_ptr->arcstat_mfu_hits.value.ui64; + + // $v{"mru"} = $d{"mru_hits"}/$int; + perf->arcstat_mru_hits = arc_ptr->arcstat_mru_hits.value.ui64; + + // $v{"mrug"} = $d{"mru_ghost_hits"}/$int; + perf->arcstat_mru_ghost_hits = + arc_ptr->arcstat_mru_ghost_hits.value.ui64; + + // $v{"mfug"} = $d{"mfu_ghost_hits"}/$int; + perf->arcstat_mfu_ghost_hits = + arc_ptr->arcstat_mfu_ghost_hits.value.ui64; + + // $v{"eskip"} = $d{"evict_skip"}/$int; + perf->arcstat_evict_skip = arc_ptr->arcstat_evict_skip.value.ui64; + + // $v{"mtxmis"} = $d{"mutex_miss"}/$int; + perf->arcstat_mutex_miss = arc_ptr->arcstat_mutex_miss.value.ui64; + + // $v{"comprs"} = $cur{"compressed_size"}; + perf->arcstat_compressed_size = + arc_ptr->arcstat_compressed_size.value.ui64; + + // $v{"uncomp"} = $cur{"uncompressed_size"}; + perf->arcstat_uncompressed_size = arc_ptr->arcstat_uncompressed_size + .value.ui64; + + // $v{"l2hits"} = $d{"l2_hits"}/$int; + perf->arcstat_l2_hits = arc_ptr->arcstat_l2_hits.value.ui64; + + // $v{ "l2miss" } = $d{ "l2_misses" } / $int; + perf->arcstat_l2_misses = arc_ptr->arcstat_l2_misses.value.ui64; + + // $v{"l2read"} = $d{"l2_read_bytes"}/$int; + perf->arcstat_l2_read_bytes = + arc_ptr->arcstat_l2_read_bytes.value.ui64; + + // $v{"l2write"} = $d{"l2_write_bytes"}/$int; + perf->arcstat_l2_write_bytes = arc_ptr-> + arcstat_l2_write_bytes.value.ui64; + + // $v{l2 access per second} = $v{"l2hits"} + $v{ "l2miss" } + perf->arcstat_l2_access_ps = perf->arcstat_l2_hits + + perf->arcstat_l2_misses; + + // $v{ "read" } = $v{ "hits" } +$v{ "miss" }; + perf->arcstat_read_ps = perf->arcstat_hits + perf->arcstat_misses; + + // $v{"mhit"}=($d{"prefetch_metadata_hits"}+ + // $d{"demand_metadata_hits"})/$int; + perf->arcstat_metadata_hit_ps = arc_ptr->arcstat_prefetch_metadata_hits + .value.ui64 + arc_ptr->arcstat_demand_metadata_hits.value.ui64; + + // $v{"mmis"}=($d{"prefetch_metadata_misses"} + + // $d{ "demand_metadata_misses" }) / $int; + perf->arcstat_metadata_miss_ps = + arc_ptr->arcstat_prefetch_metadata_misses.value.ui64 + + arc_ptr->arcstat_demand_metadata_misses.value.ui64; + + // $v{"mread"} = $v{"mhit"} + $v{"mmis"}; + perf->arcstat_metadata_accesses_ps = perf->arcstat_metadata_hit_ps + + perf->arcstat_metadata_miss_ps; + + // $v{"ovrhd"} = $cur{"overhead_size"}; + perf->arcstat_overhead_size = arc_ptr->arcstat_overhead_size.value.ui64; +} +void +zil_cache_counters_perfmon(cache_counters *perf, zil_kstat_values_t *zil_ptr) +{ + perf->zil_commit_count = zil_ptr->zil_commit_count.value.ui64; + perf->zil_commit_writer_count = zil_ptr-> + zil_commit_writer_count.value.ui64; + perf->zil_itx_count = zil_ptr->zil_itx_count.value.ui64; + perf->zil_itx_indirect_count = zil_ptr-> + zil_itx_indirect_count.value.ui64; + perf->zil_itx_indirect_bytes = zil_ptr-> + zil_itx_indirect_bytes.value.ui64; + perf->zil_itx_copied_count = zil_ptr->zil_itx_copied_count.value.ui64; + perf->zil_itx_copied_bytes = zil_ptr->zil_itx_copied_bytes.value.ui64; + perf->zil_itx_needcopy_count = zil_ptr-> + zil_itx_needcopy_count.value.ui64; + perf->zil_itx_needcopy_bytes = zil_ptr-> + zil_itx_needcopy_bytes.value.ui64; + perf->zil_itx_metaslab_normal_count = zil_ptr-> + zil_itx_metaslab_normal_count + .value.ui64; + perf->zil_itx_metaslab_normal_bytes = zil_ptr-> + zil_itx_metaslab_normal_bytes + .value.ui64; + perf->zil_itx_metaslab_slog_count = zil_ptr-> + zil_itx_metaslab_slog_count.value.ui64; + perf->zil_itx_metaslab_slog_bytes = zil_ptr-> + zil_itx_metaslab_slog_bytes.value.ui64; +} diff --git a/module/os/windows/spl/spl-list.c b/module/os/windows/spl/spl-list.c new file mode 100644 index 000000000000..64740b1d67e5 --- /dev/null +++ b/module/os/windows/spl/spl-list.c @@ -0,0 +1,197 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Generic doubly-linked list implementation + */ + +#include +#include +#include +#include + + +#define list_insert_after_node(list, node, object) { \ + list_node_t *lnew = list_d2l(list, object); \ + lnew->list_prev = node; \ + lnew->list_next = node->list_next; \ + node->list_next->list_prev = lnew; \ + node->list_next = lnew; \ +} + +#define list_insert_before_node(list, node, object) { \ + list_node_t *lnew = list_d2l(list, object); \ + lnew->list_next = node; \ + lnew->list_prev = node->list_prev; \ + node->list_prev->list_next = lnew; \ + node->list_prev = lnew; \ +} + +void +list_create(list_t *list, size_t size, size_t offset) +{ + ASSERT(list); + ASSERT(size > 0); + ASSERT(size >= offset + sizeof (list_node_t)); + + list->list_size = size; + list->list_offset = offset; + list->list_head.list_next = list->list_head.list_prev = + &list->list_head; +} + +void +list_destroy(list_t *list) +{ + list_node_t *node = &list->list_head; + + ASSERT(list); + ASSERT(list->list_head.list_next == node); + ASSERT(list->list_head.list_prev == node); + + node->list_next = node->list_prev = NULL; +} + +void +list_insert_after(list_t *list, void *object, void *nobject) +{ + if (object == NULL) { + list_insert_head(list, nobject); + } else { + list_node_t *lold = list_d2l(list, object); + list_insert_after_node(list, lold, nobject); + } +} + +void +list_insert_before(list_t *list, void *object, void *nobject) +{ + if (object == NULL) { + list_insert_tail(list, nobject); + } else { + list_node_t *lold = list_d2l(list, object); + list_insert_before_node(list, lold, nobject); + } +} + +void +list_insert_head(list_t *list, void *object) +{ + list_node_t *lold = &list->list_head; + list_insert_after_node(list, lold, object); +} + +void +list_insert_tail(list_t *list, void *object) +{ + list_node_t *lold = &list->list_head; + list_insert_before_node(list, lold, object); +} + +void +list_remove(list_t *list, void *object) +{ + list_node_t *lold = list_d2l(list, object); + ASSERT(!list_empty(list)); + ASSERT(lold->list_next != NULL); + lold->list_prev->list_next = lold->list_next; + lold->list_next->list_prev = lold->list_prev; + lold->list_next = lold->list_prev = NULL; +} + + +void * +list_head(list_t *list) +{ + if (list_empty(list)) + return (NULL); + return (list_object(list, list->list_head.list_next)); +} + +void * +list_tail(list_t *list) +{ + if (list_empty(list)) + return (NULL); + return (list_object(list, list->list_head.list_prev)); +} + +void * +list_next(list_t *list, void *object) +{ + list_node_t *node = list_d2l(list, object); + + if (node->list_next != &list->list_head) + return (list_object(list, node->list_next)); + + return (NULL); +} + +void * +list_prev(list_t *list, void *object) +{ + list_node_t *node = list_d2l(list, object); + + if (node->list_prev != &list->list_head) + return (list_object(list, node->list_prev)); + + return (NULL); +} + +/* + * Insert src list after dst list. Empty src list thereafter. + */ +void +list_move_tail(list_t *dst, list_t *src) +{ + list_node_t *dstnode = &dst->list_head; + list_node_t *srcnode = &src->list_head; + + ASSERT(dst->list_size == src->list_size); + ASSERT(dst->list_offset == src->list_offset); + + if (list_empty(src)) + return; + + dstnode->list_prev->list_next = srcnode->list_next; + srcnode->list_next->list_prev = dstnode->list_prev; + dstnode->list_prev = srcnode->list_prev; + srcnode->list_prev->list_next = dstnode; + + /* empty src list */ + srcnode->list_next = srcnode->list_prev = srcnode; +} + +int +list_link_active(list_node_t *link) +{ + return (link->list_next != NULL); +} + +int +list_is_empty(list_t *list) +{ + return (list_empty(list)); +} diff --git a/module/os/windows/spl/spl-lookasidelist.c b/module/os/windows/spl/spl-lookasidelist.c new file mode 100644 index 000000000000..37cd81d2775f --- /dev/null +++ b/module/os/windows/spl/spl-lookasidelist.c @@ -0,0 +1,192 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2020, DataCore Software Corp. + * Portions Copyright 2022 Andrew Innes + */ + +#include +#include + + +typedef struct lookasidelist_stats { + kstat_named_t lookasidestat_active_alloc; + kstat_named_t lookasidestat_total_alloc; + kstat_named_t lookasidestat_total_free; + kstat_named_t lookasidestat_chunk_size; +} lookasidelist_stats_t; + +static lookasidelist_stats_t lookasidelist_stats = { + /* Number of active allocations */ + { "active_alloc", KSTAT_DATA_UINT64 }, + /* The total number of allocations */ + { "total_alloc", KSTAT_DATA_UINT64 }, + /* The total number of frees */ + { "total_free", KSTAT_DATA_UINT64 }, + /* The size of each object/chunk in lookaside list */ + { "chunk_size", KSTAT_DATA_UINT64 }, +}; + +static int +lookaside_kstat_update(kstat_t *ksp, int rw) +{ + lookasidelist_stats_t *ks = ksp->ks_data; + lookasidelist_cache_t *cp = ksp->ks_private; + + if (rw == KSTAT_WRITE) { + return (EACCES); + } + + ks->lookasidestat_active_alloc.value.ui64 = + cp->cache_active_allocations; + ks->lookasidestat_total_alloc.value.ui64 = + cp->total_alloc; + ks->lookasidestat_total_free.value.ui64 = + cp->total_free; + ks->lookasidestat_chunk_size.value.ui64 = + cp->cache_chunksize; + + return (0); +} + +void * +allocate_func( + __in POOL_TYPE PoolType, + __in SIZE_T NumberOfBytes, + __in ULONG Tag, + __inout PLOOKASIDE_LIST_EX Lookaside) +{ + lookasidelist_cache_t *pLookasidelist_cache; + pLookasidelist_cache = CONTAINING_RECORD(Lookaside, + lookasidelist_cache_t, lookasideField); + void *buf = osif_malloc(NumberOfBytes); + ASSERT(buf != NULL); + + if (buf != NULL) { + atomic_inc_64(&pLookasidelist_cache->cache_active_allocations); + atomic_inc_64(&pLookasidelist_cache->total_alloc); + } + + return (buf); +} + +void free_func( + __in PVOID Buffer, + __inout PLOOKASIDE_LIST_EX Lookaside +) { + lookasidelist_cache_t *pLookasidelist_cache; + pLookasidelist_cache = CONTAINING_RECORD(Lookaside, + lookasidelist_cache_t, lookasideField); + osif_free(Buffer, pLookasidelist_cache->cache_chunksize); + atomic_dec_64(&pLookasidelist_cache->cache_active_allocations); + atomic_inc_64(&pLookasidelist_cache->total_free); +} + +lookasidelist_cache_t * +lookasidelist_cache_create(char *name, /* descriptive name for this cache */ + size_t size) /* size of the objects it manages */ +{ + lookasidelist_cache_t *pLookasidelist_cache; + pLookasidelist_cache = ExAllocatePoolWithTag(NonPagedPoolNx, + sizeof (lookasidelist_cache_t), ZFS_LookAsideList_DRV_TAG); + + if (pLookasidelist_cache != NULL) { + memset(pLookasidelist_cache, 0, sizeof (lookasidelist_cache_t)); + pLookasidelist_cache->cache_chunksize = size; + + if (name != NULL) { + strlcpy(pLookasidelist_cache->cache_name, name, + sizeof (pLookasidelist_cache->cache_name)); + } + + NTSTATUS retval = ExInitializeLookasideListEx( + &pLookasidelist_cache->lookasideField, + allocate_func, + free_func, + NonPagedPoolNx, + 0, + size, + ZFS_LookAsideList_DRV_TAG, + 0); + + ASSERT(retval == STATUS_SUCCESS); + + if (retval != STATUS_SUCCESS) { + ExFreePoolWithTag(pLookasidelist_cache, + ZFS_LookAsideList_DRV_TAG); + pLookasidelist_cache = NULL; + } + } + + if (pLookasidelist_cache != NULL) { + kstat_t *ksp = kstat_create("spl", 0, + pLookasidelist_cache->cache_name, + "lookasidelist_cache", KSTAT_TYPE_NAMED, + sizeof (lookasidelist_stats) / sizeof (kstat_named_t), + KSTAT_FLAG_VIRTUAL); + + if (ksp != NULL) { + ksp->ks_data = &lookasidelist_stats; + ksp->ks_update = lookaside_kstat_update; + ksp->ks_private = pLookasidelist_cache; + pLookasidelist_cache->cache_kstat = ksp; + kstat_install(ksp); + } + } + + return (pLookasidelist_cache); +} + +void +lookasidelist_cache_destroy(lookasidelist_cache_t *pLookasidelist_cache) +{ + if (pLookasidelist_cache != NULL) { + ExFlushLookasideListEx(&pLookasidelist_cache->lookasideField); + ExDeleteLookasideListEx(&pLookasidelist_cache->lookasideField); + + if (pLookasidelist_cache->cache_kstat != NULL) { + kstat_delete(pLookasidelist_cache->cache_kstat); + pLookasidelist_cache->cache_kstat = NULL; + } + + ExFreePoolWithTag(pLookasidelist_cache, + ZFS_LookAsideList_DRV_TAG); + } +} + +void * +lookasidelist_cache_alloc(lookasidelist_cache_t *pLookasidelist_cache) +{ + void *buf = ExAllocateFromLookasideListEx( + &pLookasidelist_cache->lookasideField); + ASSERT(buf != NULL); + return (buf); +} + +void +lookasidelist_cache_free(lookasidelist_cache_t *pLookasidelist_cache, void *buf) +{ + ASSERT(buf != NULL); + if (buf != NULL) { + ExFreeToLookasideListEx(&pLookasidelist_cache->lookasideField, + buf); + } +} diff --git a/module/os/windows/spl/spl-md5.c b/module/os/windows/spl/spl-md5.c new file mode 100644 index 000000000000..2c903f148be7 --- /dev/null +++ b/module/os/windows/spl/spl-md5.c @@ -0,0 +1,663 @@ +/* + * Copyright 2009 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Cleaned-up and optimized version of MD5, based on the reference + * implementation provided in RFC 1321. See RSA Copyright information + * below. + */ + +/* + * MD5C.C - RSA Data Security, Inc., MD5 message-digest algorithm + */ + +/* + * Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All + * rights reserved. + * + * License to copy and use this software is granted provided that it + * is identified as the "RSA Data Security, Inc. MD5 Message-Digest + * Algorithm" in all material mentioning or referencing this software + * or this function. + * + * License is also granted to make and use derivative works provided + * that such works are identified as "derived from the RSA Data + * Security, Inc. MD5 Message-Digest Algorithm" in all material + * mentioning or referencing the derived work. + * + * RSA Data Security, Inc. makes no representations concerning either + * the merchantability of this software or the suitability of this + * software for any particular purpose. It is provided "as is" + * without express or implied warranty of any kind. + * + * These notices must be retained in any copies of any part of this + * documentation and/or software. + */ + +#ifndef _KERNEL +#include +#endif /* _KERNEL */ + +#include +#include +#include /* MD5_CONST() optimization */ + +#ifdef _KERNEL +#include +#endif /* _KERNEL */ + +static void Encode(uint8_t *, const uint32_t *, size_t); + +static void MD5Transform(uint32_t, uint32_t, uint32_t, uint32_t, MD5_CTX *, + const uint8_t [64]); + +static uint8_t PADDING[64] = { 0x80, /* all zeros */ }; + +/* + * F, G, H and I are the basic MD5 functions. + */ +#define F(b, c, d) (((b) & (c)) | ((~b) & (d))) +#define G(b, c, d) (((b) & (d)) | ((c) & (~d))) +#define H(b, c, d) ((b) ^ (c) ^ (d)) +#define I(b, c, d) ((c) ^ ((b) | (~d))) + +/* + * ROTATE_LEFT rotates x left n bits. + */ +#define ROTATE_LEFT(x, n) \ + (((x) << (n)) | ((x) >> ((sizeof (x) << 3) - (n)))) + +/* + * FF, GG, HH, and II transformations for rounds 1, 2, 3, and 4. + * Rotation is separate from addition to prevent recomputation. + */ + +#define FF(a, b, c, d, x, s, ac) { \ + (a) += F((b), (c), (d)) + (x) + ((unsigned long long)(ac)); \ + (a) = ROTATE_LEFT((a), (s)); \ + (a) += (b); \ + } + +#define GG(a, b, c, d, x, s, ac) { \ + (a) += G((b), (c), (d)) + (x) + ((unsigned long long)(ac)); \ + (a) = ROTATE_LEFT((a), (s)); \ + (a) += (b); \ + } + +#define HH(a, b, c, d, x, s, ac) { \ + (a) += H((b), (c), (d)) + (x) + ((unsigned long long)(ac)); \ + (a) = ROTATE_LEFT((a), (s)); \ + (a) += (b); \ + } + +#define II(a, b, c, d, x, s, ac) { \ + (a) += I((b), (c), (d)) + (x) + ((unsigned long long)(ac)); \ + (a) = ROTATE_LEFT((a), (s)); \ + (a) += (b); \ + } + +/* + * Loading 32-bit constants on a RISC is expensive since it involves both a + * `sethi' and an `or'. thus, we instead have the compiler generate `ld's to + * load the constants from an array called `md5_consts'. however, on intel + * (and other CISC processors), it is cheaper to load the constant + * directly. thus, the c code in MD5Transform() uses the macro MD5_CONST() + * which either expands to a constant or an array reference, depending on the + * architecture the code is being compiled for. + * + * Right now, i386 and amd64 are the CISC exceptions. + * If we get another CISC ISA, we'll have to change the ifdef. + */ + +#if defined(__i386) || defined(__amd64) + +#define MD5_CONST(x) (MD5_CONST_ ## x) +#define MD5_CONST_e(x) MD5_CONST(x) +#define MD5_CONST_o(x) MD5_CONST(x) + +#else +/* + * sparc/RISC optimization: + * + * while it is somewhat counter-intuitive, on sparc (and presumably other RISC + * machines), it is more efficient to place all the constants used in this + * function in an array and load the values out of the array than to manually + * load the constants. this is because setting a register to a 32-bit value + * takes two ops in most cases: a `sethi' and an `or', but loading a 32-bit + * value from memory only takes one `ld' (or `lduw' on v9). while this + * increases memory usage, the compiler can find enough other things to do + * while waiting to keep the pipeline does not stall. additionally, it is + * likely that many of these constants are cached so that later accesses do + * not even go out to the bus. + * + * this array is declared `static' to keep the compiler from having to + * memcpy() this array onto the stack frame of MD5Transform() each time it is + * called -- which is unacceptably expensive. + * + * the `const' is to ensure that callers are good citizens and do not try to + * munge the array. since these routines are going to be called from inside + * multithreaded kernelland, this is a good safety check. -- `constants' will + * end up in .rodata. + * + * unfortunately, loading from an array in this manner hurts performance under + * intel (and presumably other CISC machines). so, there is a macro, + * MD5_CONST(), used in MD5Transform(), that either expands to a reference to + * this array, or to the actual constant, depending on what platform this code + * is compiled for. + */ + +#ifdef sun4v + +/* + * Going to load these consts in 8B chunks, so need to enforce 8B alignment + */ + +/* CSTYLED */ +#pragma align 64 (md5_consts) +#define _MD5_CHECK_ALIGNMENT + +#endif /* sun4v */ + +static const uint32_t md5_consts[] = { + MD5_CONST_0, MD5_CONST_1, MD5_CONST_2, MD5_CONST_3, + MD5_CONST_4, MD5_CONST_5, MD5_CONST_6, MD5_CONST_7, + MD5_CONST_8, MD5_CONST_9, MD5_CONST_10, MD5_CONST_11, + MD5_CONST_12, MD5_CONST_13, MD5_CONST_14, MD5_CONST_15, + MD5_CONST_16, MD5_CONST_17, MD5_CONST_18, MD5_CONST_19, + MD5_CONST_20, MD5_CONST_21, MD5_CONST_22, MD5_CONST_23, + MD5_CONST_24, MD5_CONST_25, MD5_CONST_26, MD5_CONST_27, + MD5_CONST_28, MD5_CONST_29, MD5_CONST_30, MD5_CONST_31, + MD5_CONST_32, MD5_CONST_33, MD5_CONST_34, MD5_CONST_35, + MD5_CONST_36, MD5_CONST_37, MD5_CONST_38, MD5_CONST_39, + MD5_CONST_40, MD5_CONST_41, MD5_CONST_42, MD5_CONST_43, + MD5_CONST_44, MD5_CONST_45, MD5_CONST_46, MD5_CONST_47, + MD5_CONST_48, MD5_CONST_49, MD5_CONST_50, MD5_CONST_51, + MD5_CONST_52, MD5_CONST_53, MD5_CONST_54, MD5_CONST_55, + MD5_CONST_56, MD5_CONST_57, MD5_CONST_58, MD5_CONST_59, + MD5_CONST_60, MD5_CONST_61, MD5_CONST_62, MD5_CONST_63 +}; + + +#ifdef sun4v +/* + * To reduce the number of loads, load consts in 64-bit + * chunks and then split. + * + * No need to mask upper 32-bits, as just interested in + * low 32-bits (saves an & operation and means that this + * optimization doesn't increases the icount. + */ +#define MD5_CONST_e(x) (md5_consts64[x/2] >> 32) +#define MD5_CONST_o(x) (md5_consts64[x/2]) + +#else + +#define MD5_CONST_e(x) (md5_consts[x]) +#define MD5_CONST_o(x) (md5_consts[x]) + +#endif /* sun4v */ + +#endif + +/* + * MD5Init() + * + * purpose: initializes the md5 context and begins and md5 digest operation + * input: MD5_CTX * : the context to initialize. + * output: void + */ + +void +MD5Init(MD5_CTX *ctx) +{ + ctx->count[0] = ctx->count[1] = 0; + + /* load magic initialization constants */ + ctx->state[0] = MD5_INIT_CONST_1; + ctx->state[1] = MD5_INIT_CONST_2; + ctx->state[2] = MD5_INIT_CONST_3; + ctx->state[3] = MD5_INIT_CONST_4; +} + +/* + * MD5Update() + * + * purpose: continues an md5 digest operation, using the message block + * to update the context. + * input: MD5_CTX * : the context to update + * uint8_t * : the message block + * uint32_t : the length of the message block in bytes + * output: void + * + * MD5 crunches in 64-byte blocks. All numeric constants here are related to + * that property of MD5. + */ + +void +MD5Update(MD5_CTX *ctx, const void *inpp, unsigned int input_len) +{ + uint32_t i, buf_index, buf_len; +#ifdef sun4v + uint32_t old_asi; +#endif /* sun4v */ +#if defined(__amd64) + // uint32_t block_count; +#endif /* !defined(__amd64) */ + const unsigned char *input = (const unsigned char *)inpp; + + /* compute (number of bytes computed so far) mod 64 */ + buf_index = (ctx->count[0] >> 3) & 0x3F; + + /* update number of bits hashed into this MD5 computation so far */ + if ((ctx->count[0] += (input_len << 3)) < (input_len << 3)) + ctx->count[1]++; + ctx->count[1] += (input_len >> 29); + + buf_len = 64 - buf_index; + + /* transform as many times as possible */ + i = 0; + if (input_len >= buf_len) { + + /* + * general optimization: + * + * only do initial memcpy() and MD5Transform() if + * buf_index != 0. if buf_index == 0, we're just + * wasting our time doing the memcpy() since there + * wasn't any data left over from a previous call to + * MD5Update(). + */ + +#ifdef sun4v + /* + * For N1 use %asi register. However, costly to repeatedly set + * in MD5Transform. Therefore, set once here. + * Should probably restore the old value afterwards... + */ + old_asi = get_little(); + set_little(0x88); +#endif /* sun4v */ + + if (buf_index) { + memcpy(&ctx->buf_un.buf8[buf_index], input, buf_len); + + MD5Transform(ctx->state[0], ctx->state[1], + ctx->state[2], ctx->state[3], ctx, + ctx->buf_un.buf8); + + i = buf_len; + } + + for (; i + 63 < input_len; i += 64) + MD5Transform(ctx->state[0], ctx->state[1], + ctx->state[2], ctx->state[3], ctx, &input[i]); + + +#ifdef sun4v + /* + * Restore old %ASI value + */ + set_little(old_asi); +#endif /* sun4v */ + + /* + * general optimization: + * + * if i and input_len are the same, return now instead + * of calling memcpy(), since the memcpy() in this + * case will be an expensive nop. + */ + + if (input_len == i) + return; + + buf_index = 0; + } + + /* buffer remaining input */ + memcpy(&ctx->buf_un.buf8[buf_index], &input[i], input_len - i); +} + +/* + * MD5Final() + * + * purpose: ends an md5 digest operation, finalizing the message digest and + * zeroing the context. + * input: uchar_t * : a buffer to store the digest in + * : The function actually uses void* because many + * : callers pass things other than uchar_t here. + * MD5_CTX * : the context to finalize, save, and zero + * output: void + */ + +void +MD5Final(void *digest, MD5_CTX *ctx) +{ + uint8_t bitcount_le[sizeof (ctx->count)]; + uint32_t index = (ctx->count[0] >> 3) & 0x3f; + + /* store bit count, little endian */ + Encode(bitcount_le, ctx->count, sizeof (bitcount_le)); + + /* pad out to 56 mod 64 */ + MD5Update(ctx, PADDING, ((index < 56) ? 56 : 120) - index); + + /* append length (before padding) */ + MD5Update(ctx, bitcount_le, sizeof (bitcount_le)); + + /* store state in digest */ + Encode(digest, ctx->state, sizeof (ctx->state)); + + /* zeroize sensitive information */ + memset(ctx, 0, sizeof (*ctx)); +} + +#ifndef _KERNEL + +void +md5_calc(unsigned char *output, unsigned char *input, unsigned int inlen) +{ + MD5_CTX context; + + MD5Init(&context); + MD5Update(&context, input, inlen); + MD5Final(output, &context); +} + +#endif /* !_KERNEL */ + +/* + * sparc register window optimization: + * + * `a', `b', `c', and `d' are passed into MD5Transform explicitly + * since it increases the number of registers available to the + * compiler. under this scheme, these variables can be held in + * %i0 - %i3, which leaves more local and out registers available. + */ + +/* + * MD5Transform() + * + * purpose: md5 transformation -- updates the digest based on `block' + * input: uint32_t : bytes 1 - 4 of the digest + * uint32_t : bytes 5 - 8 of the digest + * uint32_t : bytes 9 - 12 of the digest + * uint32_t : bytes 12 - 16 of the digest + * MD5_CTX * : the context to update + * uint8_t [64]: the block to use to update the digest + * output: void + */ + +static void +MD5Transform(uint32_t a, uint32_t b, uint32_t c, uint32_t d, + MD5_CTX *ctx, const uint8_t block[64]) +{ + /* + * general optimization: + * + * use individual integers instead of using an array. this is a + * win, although the amount it wins by seems to vary quite a bit. + */ + + register uint32_t x_0, x_1, x_2, x_3, x_4, x_5, x_6, x_7; + register uint32_t x_8, x_9, x_10, x_11, x_12, x_13, x_14, x_15; +#ifdef sun4v + unsigned long long *md5_consts64; + + /* LINTED E_BAD_PTR_CAST_ALIGN */ + md5_consts64 = (unsigned long long *) md5_consts; +#endif /* sun4v */ + + /* + * general optimization: + * + * the compiler (at least SC4.2/5.x) generates better code if + * variable use is localized. in this case, swapping the integers in + * this order allows `x_0 'to be swapped nearest to its first use in + * FF(), and likewise for `x_1' and up. note that the compiler + * prefers this to doing each swap right before the FF() that + * uses it. + */ + + /* + * sparc v9/v8plus optimization: + * + * if `block' is already aligned on a 4-byte boundary, use the + * optimized load_little_32() directly. otherwise, memcpy() + * into a buffer that *is* aligned on a 4-byte boundary and + * then do the load_little_32() on that buffer. benchmarks + * have shown that using the memcpy() is better than loading + * the bytes individually and doing the endian-swap by hand. + * + * even though it's quite tempting to assign to do: + * + * blk = memcpy(ctx->buf_un.buf32, blk, sizeof (ctx->buf_un.buf32)); + * + * and only have one set of LOAD_LITTLE_32()'s, the compiler (at least + * SC4.2/5.x) *does not* like that, so please resist the urge. + */ + +#ifdef _MD5_CHECK_ALIGNMENT + if ((uintptr_t)block & 0x3) { /* not 4-byte aligned? */ + memcpy(ctx->buf_un.buf32, block, sizeof (ctx->buf_un.buf32)); + +#ifdef sun4v + x_15 = LOAD_LITTLE_32_f(ctx->buf_un.buf32); + x_14 = LOAD_LITTLE_32_e(ctx->buf_un.buf32); + x_13 = LOAD_LITTLE_32_d(ctx->buf_un.buf32); + x_12 = LOAD_LITTLE_32_c(ctx->buf_un.buf32); + x_11 = LOAD_LITTLE_32_b(ctx->buf_un.buf32); + x_10 = LOAD_LITTLE_32_a(ctx->buf_un.buf32); + x_9 = LOAD_LITTLE_32_9(ctx->buf_un.buf32); + x_8 = LOAD_LITTLE_32_8(ctx->buf_un.buf32); + x_7 = LOAD_LITTLE_32_7(ctx->buf_un.buf32); + x_6 = LOAD_LITTLE_32_6(ctx->buf_un.buf32); + x_5 = LOAD_LITTLE_32_5(ctx->buf_un.buf32); + x_4 = LOAD_LITTLE_32_4(ctx->buf_un.buf32); + x_3 = LOAD_LITTLE_32_3(ctx->buf_un.buf32); + x_2 = LOAD_LITTLE_32_2(ctx->buf_un.buf32); + x_1 = LOAD_LITTLE_32_1(ctx->buf_un.buf32); + x_0 = LOAD_LITTLE_32_0(ctx->buf_un.buf32); +#else + x_15 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 15); + x_14 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 14); + x_13 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 13); + x_12 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 12); + x_11 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 11); + x_10 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 10); + x_9 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 9); + x_8 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 8); + x_7 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 7); + x_6 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 6); + x_5 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 5); + x_4 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 4); + x_3 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 3); + x_2 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 2); + x_1 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 1); + x_0 = LOAD_LITTLE_32(ctx->buf_un.buf32 + 0); +#endif /* sun4v */ + } else +#endif + { + +#ifdef sun4v + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_15 = LOAD_LITTLE_32_f(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_14 = LOAD_LITTLE_32_e(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_13 = LOAD_LITTLE_32_d(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_12 = LOAD_LITTLE_32_c(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_11 = LOAD_LITTLE_32_b(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_10 = LOAD_LITTLE_32_a(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_9 = LOAD_LITTLE_32_9(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_8 = LOAD_LITTLE_32_8(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_7 = LOAD_LITTLE_32_7(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_6 = LOAD_LITTLE_32_6(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_5 = LOAD_LITTLE_32_5(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_4 = LOAD_LITTLE_32_4(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_3 = LOAD_LITTLE_32_3(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_2 = LOAD_LITTLE_32_2(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_1 = LOAD_LITTLE_32_1(block); + /* LINTED E_BAD_PTR_CAST_ALIGN */ + x_0 = LOAD_LITTLE_32_0(block); +#else +#define LOAD_LITTLE_32(addr) (*(uint32_t *)(void *)(addr)) + x_15 = LOAD_LITTLE_32(block + 60); + x_14 = LOAD_LITTLE_32(block + 56); + x_13 = LOAD_LITTLE_32(block + 52); + x_12 = LOAD_LITTLE_32(block + 48); + x_11 = LOAD_LITTLE_32(block + 44); + x_10 = LOAD_LITTLE_32(block + 40); + x_9 = LOAD_LITTLE_32(block + 36); + x_8 = LOAD_LITTLE_32(block + 32); + x_7 = LOAD_LITTLE_32(block + 28); + x_6 = LOAD_LITTLE_32(block + 24); + x_5 = LOAD_LITTLE_32(block + 20); + x_4 = LOAD_LITTLE_32(block + 16); + x_3 = LOAD_LITTLE_32(block + 12); + x_2 = LOAD_LITTLE_32(block + 8); + x_1 = LOAD_LITTLE_32(block + 4); + x_0 = LOAD_LITTLE_32(block + 0); +#endif /* sun4v */ + } + + /* round 1 */ + FF(a, b, c, d, x_0, MD5_SHIFT_11, MD5_CONST_e(0)); /* 1 */ + FF(d, a, b, c, x_1, MD5_SHIFT_12, MD5_CONST_o(1)); /* 2 */ + FF(c, d, a, b, x_2, MD5_SHIFT_13, MD5_CONST_e(2)); /* 3 */ + FF(b, c, d, a, x_3, MD5_SHIFT_14, MD5_CONST_o(3)); /* 4 */ + FF(a, b, c, d, x_4, MD5_SHIFT_11, MD5_CONST_e(4)); /* 5 */ + FF(d, a, b, c, x_5, MD5_SHIFT_12, MD5_CONST_o(5)); /* 6 */ + FF(c, d, a, b, x_6, MD5_SHIFT_13, MD5_CONST_e(6)); /* 7 */ + FF(b, c, d, a, x_7, MD5_SHIFT_14, MD5_CONST_o(7)); /* 8 */ + FF(a, b, c, d, x_8, MD5_SHIFT_11, MD5_CONST_e(8)); /* 9 */ + FF(d, a, b, c, x_9, MD5_SHIFT_12, MD5_CONST_o(9)); /* 10 */ + FF(c, d, a, b, x_10, MD5_SHIFT_13, MD5_CONST_e(10)); /* 11 */ + FF(b, c, d, a, x_11, MD5_SHIFT_14, MD5_CONST_o(11)); /* 12 */ + FF(a, b, c, d, x_12, MD5_SHIFT_11, MD5_CONST_e(12)); /* 13 */ + FF(d, a, b, c, x_13, MD5_SHIFT_12, MD5_CONST_o(13)); /* 14 */ + FF(c, d, a, b, x_14, MD5_SHIFT_13, MD5_CONST_e(14)); /* 15 */ + FF(b, c, d, a, x_15, MD5_SHIFT_14, MD5_CONST_o(15)); /* 16 */ + + /* round 2 */ + GG(a, b, c, d, x_1, MD5_SHIFT_21, MD5_CONST_e(16)); /* 17 */ + GG(d, a, b, c, x_6, MD5_SHIFT_22, MD5_CONST_o(17)); /* 18 */ + GG(c, d, a, b, x_11, MD5_SHIFT_23, MD5_CONST_e(18)); /* 19 */ + GG(b, c, d, a, x_0, MD5_SHIFT_24, MD5_CONST_o(19)); /* 20 */ + GG(a, b, c, d, x_5, MD5_SHIFT_21, MD5_CONST_e(20)); /* 21 */ + GG(d, a, b, c, x_10, MD5_SHIFT_22, MD5_CONST_o(21)); /* 22 */ + GG(c, d, a, b, x_15, MD5_SHIFT_23, MD5_CONST_e(22)); /* 23 */ + GG(b, c, d, a, x_4, MD5_SHIFT_24, MD5_CONST_o(23)); /* 24 */ + GG(a, b, c, d, x_9, MD5_SHIFT_21, MD5_CONST_e(24)); /* 25 */ + GG(d, a, b, c, x_14, MD5_SHIFT_22, MD5_CONST_o(25)); /* 26 */ + GG(c, d, a, b, x_3, MD5_SHIFT_23, MD5_CONST_e(26)); /* 27 */ + GG(b, c, d, a, x_8, MD5_SHIFT_24, MD5_CONST_o(27)); /* 28 */ + GG(a, b, c, d, x_13, MD5_SHIFT_21, MD5_CONST_e(28)); /* 29 */ + GG(d, a, b, c, x_2, MD5_SHIFT_22, MD5_CONST_o(29)); /* 30 */ + GG(c, d, a, b, x_7, MD5_SHIFT_23, MD5_CONST_e(30)); /* 31 */ + GG(b, c, d, a, x_12, MD5_SHIFT_24, MD5_CONST_o(31)); /* 32 */ + + /* round 3 */ + HH(a, b, c, d, x_5, MD5_SHIFT_31, MD5_CONST_e(32)); /* 33 */ + HH(d, a, b, c, x_8, MD5_SHIFT_32, MD5_CONST_o(33)); /* 34 */ + HH(c, d, a, b, x_11, MD5_SHIFT_33, MD5_CONST_e(34)); /* 35 */ + HH(b, c, d, a, x_14, MD5_SHIFT_34, MD5_CONST_o(35)); /* 36 */ + HH(a, b, c, d, x_1, MD5_SHIFT_31, MD5_CONST_e(36)); /* 37 */ + HH(d, a, b, c, x_4, MD5_SHIFT_32, MD5_CONST_o(37)); /* 38 */ + HH(c, d, a, b, x_7, MD5_SHIFT_33, MD5_CONST_e(38)); /* 39 */ + HH(b, c, d, a, x_10, MD5_SHIFT_34, MD5_CONST_o(39)); /* 40 */ + HH(a, b, c, d, x_13, MD5_SHIFT_31, MD5_CONST_e(40)); /* 41 */ + HH(d, a, b, c, x_0, MD5_SHIFT_32, MD5_CONST_o(41)); /* 42 */ + HH(c, d, a, b, x_3, MD5_SHIFT_33, MD5_CONST_e(42)); /* 43 */ + HH(b, c, d, a, x_6, MD5_SHIFT_34, MD5_CONST_o(43)); /* 44 */ + HH(a, b, c, d, x_9, MD5_SHIFT_31, MD5_CONST_e(44)); /* 45 */ + HH(d, a, b, c, x_12, MD5_SHIFT_32, MD5_CONST_o(45)); /* 46 */ + HH(c, d, a, b, x_15, MD5_SHIFT_33, MD5_CONST_e(46)); /* 47 */ + HH(b, c, d, a, x_2, MD5_SHIFT_34, MD5_CONST_o(47)); /* 48 */ + + /* round 4 */ + II(a, b, c, d, x_0, MD5_SHIFT_41, MD5_CONST_e(48)); /* 49 */ + II(d, a, b, c, x_7, MD5_SHIFT_42, MD5_CONST_o(49)); /* 50 */ + II(c, d, a, b, x_14, MD5_SHIFT_43, MD5_CONST_e(50)); /* 51 */ + II(b, c, d, a, x_5, MD5_SHIFT_44, MD5_CONST_o(51)); /* 52 */ + II(a, b, c, d, x_12, MD5_SHIFT_41, MD5_CONST_e(52)); /* 53 */ + II(d, a, b, c, x_3, MD5_SHIFT_42, MD5_CONST_o(53)); /* 54 */ + II(c, d, a, b, x_10, MD5_SHIFT_43, MD5_CONST_e(54)); /* 55 */ + II(b, c, d, a, x_1, MD5_SHIFT_44, MD5_CONST_o(55)); /* 56 */ + II(a, b, c, d, x_8, MD5_SHIFT_41, MD5_CONST_e(56)); /* 57 */ + II(d, a, b, c, x_15, MD5_SHIFT_42, MD5_CONST_o(57)); /* 58 */ + II(c, d, a, b, x_6, MD5_SHIFT_43, MD5_CONST_e(58)); /* 59 */ + II(b, c, d, a, x_13, MD5_SHIFT_44, MD5_CONST_o(59)); /* 60 */ + II(a, b, c, d, x_4, MD5_SHIFT_41, MD5_CONST_e(60)); /* 61 */ + II(d, a, b, c, x_11, MD5_SHIFT_42, MD5_CONST_o(61)); /* 62 */ + II(c, d, a, b, x_2, MD5_SHIFT_43, MD5_CONST_e(62)); /* 63 */ + II(b, c, d, a, x_9, MD5_SHIFT_44, MD5_CONST_o(63)); /* 64 */ + + ctx->state[0] += a; + ctx->state[1] += b; + ctx->state[2] += c; + ctx->state[3] += d; + + /* + * zeroize sensitive information -- compiler will optimize + * this out if everything is kept in registers + */ + + x_0 = x_1 = x_2 = x_3 = x_4 = x_5 = x_6 = x_7 = x_8 = 0; + x_9 = x_10 = x_11 = x_12 = x_13 = x_14 = x_15 = 0; +} + +/* + * Encode() + * + * purpose: to convert a list of numbers from big endian to little endian + * input: uint8_t * : place to store the converted little endian numbers + * uint32_t * : place to get numbers to convert from + * size_t : the length of the input in bytes + * output: void + */ + +static void +Encode(uint8_t *output, const uint32_t *input, + size_t input_len) +{ + size_t i, j; + + for (i = 0, j = 0; j < input_len; i++, j += sizeof (uint32_t)) { + +#ifdef _LITTLE_ENDIAN + +#ifdef _MD5_CHECK_ALIGNMENT + if ((uintptr_t)output & 0x3) /* Not 4-byte aligned */ + memcpy(output + j, input + i, 4); + else *(uint32_t *)(output + j) = input[i]; +#else + /*LINTED E_BAD_PTR_CAST_ALIGN*/ + *(uint32_t *)(output + j) = input[i]; +#endif /* _MD5_CHECK_ALIGNMENT */ + +#else /* big endian -- will work on little endian, but slowly */ + + output[j] = input[i] & 0xff; + output[j + 1] = (input[i] >> 8) & 0xff; + output[j + 2] = (input[i] >> 16) & 0xff; + output[j + 3] = (input[i] >> 24) & 0xff; +#endif + } +} diff --git a/module/os/windows/spl/spl-mount.c b/module/os/windows/spl/spl-mount.c new file mode 100644 index 000000000000..df43a0448929 --- /dev/null +++ b/module/os/windows/spl/spl-mount.c @@ -0,0 +1,128 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#include +#include +#include + +int +vfs_busy(mount_t *mp, int flags) +{ + return (0); +} + +void +vfs_unbusy(mount_t *mp) +{ +} + +int +vfs_isrdonly(mount_t *mp) +{ + return (mp->mountflags & MNT_RDONLY); +} + +void +vfs_setrdonly(mount_t *mp) +{ + mp->mountflags |= MNT_RDONLY; +} + +void +vfs_clearrdonly(mount_t *mp) +{ + mp->mountflags &= ~MNT_RDONLY; +} + +void * +vfs_fsprivate(mount_t *mp) +{ + return (mp->fsprivate); +} + +void +vfs_setfsprivate(mount_t *mp, void *mntdata) +{ + mp->fsprivate = mntdata; +} + +void +vfs_clearflags(mount_t *mp, uint64_t flags) +{ + mp->mountflags &= ~flags; +} + +void +vfs_setflags(mount_t *mp, uint64_t flags) +{ + mp->mountflags |= flags; +} + +uint64_t +vfs_flags(mount_t *mp) +{ + return (mp->mountflags); +} + +struct vfsstatfs * +vfs_statfs(mount_t *mp) +{ + return (NULL); +} + +void +vfs_setlocklocal(mount_t *mp) +{ +} + +int +vfs_typenum(mount_t *mp) +{ + return (0); +} + +void +vfs_getnewfsid(struct mount *mp) +{ +} + +int +vfs_isunmount(mount_t *mp) +{ + return (0); +} + +int +vfs_iswriteupgrade(mount_t *mp) /* ronly && MNTK_WANTRDWR */ +{ + return (FALSE); +} + +void +vfs_setextendedsecurity(mount_t *mp) +{ +} diff --git a/module/os/windows/spl/spl-mutex.c b/module/os/windows/spl/spl-mutex.c new file mode 100644 index 000000000000..62e5efcf3977 --- /dev/null +++ b/module/os/windows/spl/spl-mutex.c @@ -0,0 +1,205 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017,2019 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + * + */ + +/* + * Implementation details. + * Using SynchronizationEvent that autoresets. When in 'Signaled' + * state the mutex is considered FREE/Available to be locked. + * Call KeWaitForSingleObject() to wait for it to be made + * 'available' (either blocking, or polling for *Try method) + * Calling KeSetEvent() sets event to Signaled, and wakes 'one' + * waiter, before Clearing it again. + * We attempt to avoid calling KeWaitForSingleObject() by + * using atomic CAS on m_owner, in the simple cases. + */ + +#include +#include +#include +#include +#include +#include +#include + +uint64_t zfs_active_mutex = 0; + +#define MUTEX_INITIALISED 0x23456789 +#define MUTEX_DESTROYED 0x98765432 + +int +spl_mutex_subsystem_init(void) +{ + return (0); +} + +void +spl_mutex_subsystem_fini(void) +{ + +} + +void +spl_mutex_init(kmutex_t *mp, char *name, kmutex_type_t type, void *ibc) +{ + (void) name; + ASSERT(type != MUTEX_SPIN); + ASSERT(ibc == NULL); + + if (mp->m_initialised == MUTEX_INITIALISED) + panic("%s: mutex already m_initialised\n", __func__); + mp->m_initialised = MUTEX_INITIALISED; + KeInitializeSpinLock(&mp->m_destroy_lock); + + mp->m_owner = NULL; + + // Initialise it to 'Signaled' as mutex is 'free'. + KeInitializeEvent((PRKEVENT)&mp->m_lock, SynchronizationEvent, TRUE); + atomic_inc_64(&zfs_active_mutex); +} + +void +spl_mutex_destroy(kmutex_t *mp) +{ + KIRQL oldq; + + if (!mp) + return; + + if (mp->m_initialised != MUTEX_INITIALISED) + panic("%s: mutex not m_initialised\n", __func__); + + if (mp->m_owner != 0) + panic("SPL: releasing held mutex"); + + // Make sure any call to KeSetEvent() has completed. + KeAcquireSpinLock(&mp->m_destroy_lock, &oldq); + mp->m_initialised = MUTEX_DESTROYED; + KeReleaseSpinLock(&mp->m_destroy_lock, oldq); + + // There is no FREE member for events + // KeDeleteEvent(); + + atomic_dec_64(&zfs_active_mutex); +} + +void +spl_mutex_enter(kmutex_t *mp) +{ + NTSTATUS Status; + kthread_t *thisthread = current_thread(); + + if (mp->m_initialised != MUTEX_INITIALISED) + panic("%s: mutex not m_initialised\n", __func__); + + if (mp->m_owner == thisthread) + panic("mutex_enter: locking against myself!"); + + VERIFY3P(mp->m_owner, !=, 0xdeadbeefdeadbeef); + + // Test if "m_owner" is NULL, if so, set it to "thisthread". + // Returns original value, so if NULL, it succeeded. +again: + if (InterlockedCompareExchangePointer(&mp->m_owner, + thisthread, NULL) != NULL) { + + // Failed to CAS-in 'thisthread', as owner was not NULL + // Wait forever for event to be signaled. + Status = KeWaitForSingleObject( + (PRKEVENT)&mp->m_lock, + Executive, + KernelMode, + FALSE, + NULL); + + // We waited, but someone else may have beaten us to it + // so we need to attempt CAS again + goto again; + } + + ASSERT(mp->m_owner == thisthread); +} + +void +spl_mutex_exit(kmutex_t *mp) +{ + if (mp->m_owner != current_thread()) + panic("%s: releasing not held/not our lock?\n", __func__); + + VERIFY3P(mp->m_owner, !=, 0xdeadbeefdeadbeef); + + KIRQL oldq; + + /* + * Hold spinlock so the SetEvent() does't accidentally alow + * another thread to turbo into mutex_destroy() + */ + KeAcquireSpinLock(&mp->m_destroy_lock, &oldq); + mp->m_owner = NULL; + // Wake up one waiter now that it is available. + KeSetEvent((PRKEVENT)&mp->m_lock, SEMAPHORE_INCREMENT, FALSE); + KeReleaseSpinLock(&mp->m_destroy_lock, oldq); + + VERIFY3U(KeGetCurrentIrql(), <=, DISPATCH_LEVEL); +} + +int +spl_mutex_tryenter(kmutex_t *mp) +{ + // LARGE_INTEGER timeout; + // NTSTATUS Status; + kthread_t *thisthread = current_thread(); + + if (mp->m_initialised != MUTEX_INITIALISED) + panic("%s: mutex not m_initialised\n", __func__); + + + + // Test if "m_owner" is NULL, if so, set it to "thisthread". + // Returns original value, so if NULL, it succeeded. + if (InterlockedCompareExchangePointer(&mp->m_owner, + thisthread, NULL) != NULL) { + return (0); // Not held. + } + + ASSERT(mp->m_owner == thisthread); + + // held + return (1); +} + +int +spl_mutex_owned(kmutex_t *mp) +{ + return (mp->m_owner == current_thread()); +} + +struct kthread * +spl_mutex_owner(kmutex_t *mp) +{ + return (mp->m_owner); +} diff --git a/module/os/windows/spl/spl-policy.c b/module/os/windows/spl/spl-policy.c new file mode 100644 index 000000000000..9ff411f31177 --- /dev/null +++ b/module/os/windows/spl/spl-policy.c @@ -0,0 +1,188 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +#include +#include +#include + +int +spl_priv_check_cred(const cred_t *cred, int priv, int flags) +{ + int error = 0; + + /* + * The default is deny, so if no policies have granted it, reject + * with a privilege error here. + */ + + // WIN32 - only root can run commands at the moment, but we should + // fix this test + // SC_HANDLE SCM; + // SCL = OpenSCManager(NULL, NULL, SC_MANAGER_CREATE_SERVICE) + // if (SCM != NULL) { + // isAdmin = TRUE; + // CloseServiceHandle(SCM); + // } + // error = EPERM; +// out: + return (error); +} + +int +secpolicy_fs_unmount(cred_t *cr, struct mount *vfsp) +{ + return (spl_priv_check_cred(cr, PRIV_VFS_UNMOUNT, 0)); +} + +int +secpolicy_nfs(const cred_t *cr) +{ + return (spl_priv_check_cred(cr, PRIV_NFS_DAEMON, 0)); +} + +int +secpolicy_sys_config(const cred_t *cr, boolean_t checkonly) +{ + return (spl_priv_check_cred(cr, PRIV_ZFS_POOL_CONFIG, 0)); +} + +int +secpolicy_zfs(const cred_t *cr) +{ + return (spl_priv_check_cred(cr, PRIV_VFS_MOUNT, 0)); +} + +int +secpolicy_zinject(const cred_t *cr) +{ + return (spl_priv_check_cred(cr, PRIV_ZFS_INJECT, 0)); +} + +int +secpolicy_vnode_any_access(const cred_t *cr, vnode_t *vp, uid_t owner) +{ + // FIXME + return (0); +} + +int +secpolicy_vnode_access2(const cred_t *cr, vnode_t *vp, uid_t owner, + mode_t curmode, mode_t wantmode) +{ + // FIXME + return (0); +} + +int +secpolicy_vnode_setattr(cred_t *cr, struct vnode *vp, vattr_t *vap, + const vattr_t *ovap, int flags, + int unlocked_access(void *, int, cred_t *), + void *node) +{ + // FIXME + return (0); +} + +int +secpolicy_vnode_stky_modify(const cred_t *cred) +{ + return (EPERM); +} + +int +secpolicy_setid_setsticky_clear(vnode_t *vp, vattr_t *vap, const vattr_t *ovap, + cred_t *cr) +{ + // FIXME + return (0); +} + +int +secpolicy_vnode_remove(struct vnode *vp, const cred_t *cr) +{ + return (0); +} + +int +secpolicy_vnode_create_gid(const cred_t *cred) +{ + return (0); +} + +int +secpolicy_vnode_setids_setgids(struct vnode *vp, const cred_t *cr, + gid_t gid) +{ + return (0); +} + +int +secpolicy_vnode_setdac(struct vnode *vp, const cred_t *cr, uid_t u) +{ + return (0); +} + +int +secpolicy_vnode_chown(struct vnode *vp, const cred_t *cr, uid_t u) +{ + return (0); +} + +int +secpolicy_vnode_setid_retain(struct znode *zp, const cred_t *cr, int fal) +{ + return (0); +} + +int +secpolicy_xvattr(vattr_t *vap, uid_t uid, const cred_t *cr, mode_t mod) +{ + return (0); +} + +int +secpolicy_setid_clear(vattr_t *vap, const cred_t *cr) +{ + return (0); +} + +int +secpolicy_basic_link(const cred_t *cr) +{ + return (0); +} + +int +secpolicy_fs_mount_clearopts(const cred_t *cr, struct mount *mp) +{ + return (0); +} + +int +secpolicy_fs_mount(const cred_t *cr, struct vnode *vp, struct mount *mp) +{ + return (spl_priv_check_cred(cr, PRIV_VFS_MOUNT, 0)); +} + +int +secpolicy_zfs_proc(const cred_t *cr, proc_t *proc) +{ + return (spl_priv_check_cred(cr, PRIV_VFS_MOUNT, 0)); +} diff --git a/module/os/windows/spl/spl-proc.c b/module/os/windows/spl/spl-proc.c new file mode 100644 index 000000000000..28ba36cd3463 --- /dev/null +++ b/module/os/windows/spl/spl-proc.c @@ -0,0 +1,36 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include +#include +#include + +/* + * "p0" is the first process/kernel in illumos/solaris - it is only used as + * an address to know if we are first process or not. It needs no allocated + * space, just "an address". It should be a "proc_t *". + */ + +struct _KPROCESS { + void *something; +}; + +proc_t p0 = {0}; diff --git a/module/os/windows/spl/spl-proc_list.c b/module/os/windows/spl/spl-proc_list.c new file mode 100644 index 000000000000..7fc3407c274a --- /dev/null +++ b/module/os/windows/spl/spl-proc_list.c @@ -0,0 +1,168 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include +#include +#include +#include + +typedef struct procfs_list_iter { + procfs_list_t *pli_pl; + void *pli_elt; +} pli_t; + +void +seq_printf(struct seq_file *f, const char *fmt, ...) +{ + va_list adx; + + va_start(adx, fmt); + (void) vsnprintf(f->sf_buf, f->sf_size, fmt, adx); + va_end(adx); +} + +static int +procfs_list_update(kstat_t *ksp, int rw) +{ + procfs_list_t *pl = ksp->ks_private; + + if (rw == KSTAT_WRITE) + pl->pl_clear(pl); + + return (0); +} + +static int +procfs_list_data(char *buf, size_t size, void *data) +{ + pli_t *p; + void *elt; + procfs_list_t *pl; + struct seq_file f; + + p = data; + pl = p->pli_pl; + elt = p->pli_elt; + ExFreePoolWithTag(p, '!SFZ'); + f.sf_buf = buf; + f.sf_size = size; + return (pl->pl_show(&f, elt)); +} + +static void * +procfs_list_addr(kstat_t *ksp, loff_t n) +{ + procfs_list_t *pl = ksp->ks_private; + void *elt = ksp->ks_private1; + pli_t *p = NULL; + + + if (n == 0) + ksp->ks_private1 = list_head(&pl->pl_list); + else if (elt) + ksp->ks_private1 = list_next(&pl->pl_list, elt); + + if (ksp->ks_private1) { + p = ExAllocatePoolWithTag(NonPagedPoolNx, sizeof (*p), '!SFZ'); + p->pli_pl = pl; + p->pli_elt = ksp->ks_private1; + } + + return (p); +} + + +void +procfs_list_install(const char *module, + const char *submodule, + const char *name, + mode_t mode, + procfs_list_t *procfs_list, + int (*show)(struct seq_file *f, void *p), + int (*show_header)(struct seq_file *f), + int (*clear)(procfs_list_t *procfs_list), + size_t procfs_list_node_off) +{ + kstat_t *procfs_kstat; + char *fullmod = module; + char combined[KSTAT_STRLEN]; + + mutex_init(&procfs_list->pl_lock, NULL, MUTEX_DEFAULT, NULL); + list_create(&procfs_list->pl_list, + procfs_list_node_off + sizeof (procfs_list_node_t), + procfs_list_node_off + offsetof(procfs_list_node_t, pln_link)); + procfs_list->pl_show = show; + procfs_list->pl_show_header = show_header; + procfs_list->pl_clear = clear; + procfs_list->pl_next_id = 1; + procfs_list->pl_node_offset = procfs_list_node_off; + + // Unfortunately, "submodule" (ks_class) is not used when + // considering unique names. Best way around that is to + // make "module" be "module/submodule". Ie "zfs/poolname". + if (submodule != NULL) { + snprintf(combined, sizeof (combined), "%s/%s", module, + submodule); + fullmod = combined; + } + + procfs_kstat = kstat_create(fullmod, 0, name, submodule, + KSTAT_TYPE_RAW, 0, KSTAT_FLAG_VIRTUAL); + + if (procfs_kstat) { + procfs_kstat->ks_lock = &procfs_list->pl_lock; + procfs_kstat->ks_ndata = UINT32_MAX; + procfs_kstat->ks_private = procfs_list; + procfs_kstat->ks_update = procfs_list_update; + kstat_set_seq_raw_ops(procfs_kstat, show_header, + procfs_list_data, procfs_list_addr); + kstat_install(procfs_kstat); + } + + procfs_list->pl_private = procfs_kstat; +} + +void +procfs_list_uninstall(procfs_list_t *procfs_list) +{ +} + +void +procfs_list_destroy(procfs_list_t *procfs_list) +{ + ASSERT(list_is_empty(&procfs_list->pl_list)); + if (procfs_list->pl_private != NULL) + kstat_delete(procfs_list->pl_private); + list_destroy(&procfs_list->pl_list); + mutex_destroy(&procfs_list->pl_lock); +} + +#define NODE_ID(procfs_list, obj) \ + (((procfs_list_node_t *)(((char *)obj) + \ + (procfs_list)->pl_node_offset))->pln_id) + +void +procfs_list_add(procfs_list_t *procfs_list, void *p) +{ + ASSERT(MUTEX_HELD(&procfs_list->pl_lock)); + NODE_ID(procfs_list, p) = procfs_list->pl_next_id++; + list_insert_tail(&procfs_list->pl_list, p); +} diff --git a/module/os/windows/spl/spl-processor.c b/module/os/windows/spl/spl-processor.c new file mode 100644 index 000000000000..0eb4347fabdb --- /dev/null +++ b/module/os/windows/spl/spl-processor.c @@ -0,0 +1,127 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#include + +/* Holds the flags for KeSaveExtendedProcessorState() in simd.h */ +uint32_t kfpu_state = 0; + +#ifdef __x86_64__ + +/* Should probably use MS cpuid() call */ + +#define _spl_cpuid(func, a, b, c, d) \ + __asm__ __volatile__( \ + " pushq %%rbx \n" \ + " xorq %%rcx,%%rcx \n" \ + " cpuid \n" \ + " movq %%rbx, %%rsi \n" \ + " popq %%rbx \n" : \ + "=a" (a), "=S" (b), "=c" (c), "=d" (d) : "a" (func)) + +#else /* Add ARM */ + +#define _spl_cpuid(func, a, b, c, d) \ + a = b = c = d = 0 + +#endif + +static uint64_t _spl_cpuid_features = 0ULL; +static uint64_t _spl_cpuid_features_leaf7 = 0ULL; +static boolean_t _spl_cpuid_has_xgetbv = B_FALSE; + +uint32_t +cpu_number(void) +{ + uint32_t cpuid; + cpuid = (uint32_t)KeGetCurrentProcessorIndex(); + return (cpuid % max_ncpus); +} + +uint32_t +getcpuid() +{ + uint32_t cpuid; + cpuid = (uint32_t)KeGetCurrentProcessorIndex(); + return (cpuid % max_ncpus); +} + +uint64_t +spl_cpuid_features(void) +{ + +#if defined(__aarch64__) + + // Find arm64 solution. + _spl_cpuid_has_xgetbv = B_FALSE; /* Silence unused */ + +#else /* X64 */ + + static int first_time = 1; + uint64_t a, b, c, d; + + if (first_time == 1) { + first_time = 0; + // Wikipedia: stored in EAX, EBX, EDX, ECX (in that order). + _spl_cpuid(0, a, b, d, c); + if (a >= 1) { + _spl_cpuid(1, a, b, d, c); + _spl_cpuid_features = d | (c << 32); + + // GETBV is bit 26 in ECX. Apple defines it as: + // CPUID_FEATURE_XSAVE _HBit(26) + // ( ECX & (1 << 26) + // or, (feature & 400000000000000) + _spl_cpuid_has_xgetbv = + _spl_cpuid_features & CPUID_FEATURE_XSAVE; + } + if (a >= 7) { + c = 0; + _spl_cpuid(7, a, b, d, c); + _spl_cpuid_features_leaf7 = b | (c << 32); + } + xprintf("SPL: CPUID 0x%08llx and leaf7 0x%08llx\n", + _spl_cpuid_features, _spl_cpuid_features_leaf7); + + if (_spl_cpuid_features & CPUID_FEATURE_AVX1_0) + kfpu_state |= XSTATE_MASK_AVX; + if (_spl_cpuid_features_leaf7 & CPUID_LEAF7_FEATURE_AVX2) + kfpu_state |= XSTATE_MASK_AVX; + if (_spl_cpuid_features_leaf7 & CPUID_LEAF7_FEATURE_AVX512F) + kfpu_state |= XSTATE_MASK_AVX512; + + } +#endif + + return (_spl_cpuid_features); +} + +uint64_t +spl_cpuid_leaf7_features(void) +{ + return (_spl_cpuid_features_leaf7); +} diff --git a/module/os/windows/spl/spl-rwlock.c b/module/os/windows/spl/spl-rwlock.c new file mode 100644 index 000000000000..6f518af25cba --- /dev/null +++ b/module/os/windows/spl/spl-rwlock.c @@ -0,0 +1,246 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2018 Jorgen Lundman + * + */ + +#include +#include +#include + +uint64_t zfs_active_rwlock = 0; + +/* + * We run rwlock with DEBUG on for now, as it protects against + * uninitialised access etc, and almost no cost. + */ +#ifndef DEBUG +#define DEBUG +#endif + +#ifdef DEBUG +int +rw_isinit(krwlock_t *rwlp) +{ + if (rwlp->rw_pad != 0x012345678) + return (0); + return (1); +} +#endif + + +void +rw_init(krwlock_t *rwlp, char *name, krw_type_t type, __unused void *arg) +{ + ASSERT(type != RW_DRIVER); + +#ifdef DEBUG + VERIFY3U(rwlp->rw_pad, !=, 0x012345678); +#endif + ExInitializeResourceLite(&rwlp->rw_lock); + rwlp->rw_owner = NULL; + rwlp->rw_readers = 0; +#ifdef DEBUG + rwlp->rw_pad = 0x012345678; +#endif + atomic_inc_64(&zfs_active_rwlock); +} + +void +rw_destroy(krwlock_t *rwlp) +{ + // Confirm it was initialised, and is unlocked, + // and not already destroyed. +#ifdef DEBUG + VERIFY3U(rwlp->rw_pad, ==, 0x012345678); +#endif + VERIFY3U(rwlp->rw_owner, ==, 0); + VERIFY3U(rwlp->rw_readers, ==, 0); + + // This has caused panic due to IRQL panic, from + // taskq->zap_evict->rw_destroy + ExDeleteResourceLite(&rwlp->rw_lock); +#ifdef DEBUG + rwlp->rw_pad = 0x99; +#endif + atomic_dec_64(&zfs_active_rwlock); +} + +void +rw_enter(krwlock_t *rwlp, krw_t rw) +{ +#ifdef DEBUG + if (rwlp->rw_pad != 0x012345678) + panic("rwlock %p not initialised\n", rwlp); +#endif + + if (rw == RW_READER) { + ExAcquireResourceSharedLite(&rwlp->rw_lock, TRUE); + atomic_inc_32((volatile uint32_t *)&rwlp->rw_readers); + ASSERT(rwlp->rw_owner == 0); + } else { + if (rwlp->rw_owner == current_thread()) + panic("rw_enter: locking against myself!"); + ExAcquireResourceExclusiveLite(&rwlp->rw_lock, TRUE); + ASSERT(rwlp->rw_owner == 0); + ASSERT(rwlp->rw_readers == 0); + rwlp->rw_owner = current_thread(); + } +} + +/* + * kernel private from osfmk/kern/locks.h + */ + +int +rw_tryenter(krwlock_t *rwlp, krw_t rw) +{ + int held = 0; + +#ifdef DEBUG + if (rwlp->rw_pad != 0x012345678) + panic("rwlock %p not initialised\n", rwlp); +#endif + + if (rw == RW_READER) { + held = ExAcquireResourceSharedLite(&rwlp->rw_lock, FALSE); + if (held) + atomic_inc_32((volatile uint32_t *)&rwlp->rw_readers); + } else { + if (rwlp->rw_owner == current_thread()) + panic("rw_tryenter: locking against myself!"); + + held = ExAcquireResourceExclusiveLite(&rwlp->rw_lock, FALSE); + if (held) + rwlp->rw_owner = current_thread(); + } + + return (held); +} + +/* + * It appears a difference between Darwin's + * lck_rw_lock_shared_to_exclusive() and Solaris's rw_tryupgrade() and + * FreeBSD's sx_try_upgrade() is that on failure to upgrade, the prior + * held shared/reader lock is lost on Darwin, but retained on + * Solaris/FreeBSD. We could re-acquire the lock in this situation, + * but it enters a possibility of blocking, when tryupgrade is meant + * to be non-blocking. + * Also note that XNU's lck_rw_lock_shared_to_exclusive() is always + * blocking (when waiting on readers), which means we can not use it. + */ +int +rw_tryupgrade(krwlock_t *rwlp) +{ + int held = 0; + + if (rwlp->rw_owner == current_thread()) + panic("rw_enter: locking against myself!"); + + /* More readers than us? give up */ + if (rwlp->rw_readers != 1) + return (0); + + /* + * It is ON. We need to drop our READER lock, and try to + * grab the WRITER as quickly as possible. + */ + atomic_dec_32((volatile uint32_t *)&rwlp->rw_readers); + ExReleaseResourceLite(&rwlp->rw_lock); + + /* Grab the WRITER lock */ + held = ExAcquireResourceExclusiveLite(&rwlp->rw_lock, FALSE); + + if (held) { + /* Looks like we won */ + rwlp->rw_owner = current_thread(); + ASSERT(rwlp->rw_readers == 0); + return (1); + } + + /* + * The worst has happened, we failed to grab WRITE lock, either + * due to another WRITER lock, or, some READER came along. + * IllumOS implementation returns with the READER lock again + * so we need to grab it. + */ + rw_enter(rwlp, RW_READER); + return (0); +} + +void +rw_exit(krwlock_t *rwlp) +{ + if (rwlp->rw_owner == current_thread()) { + rwlp->rw_owner = NULL; + ASSERT(rwlp->rw_readers == 0); + ExReleaseResourceLite(&rwlp->rw_lock); + } else { + atomic_dec_32((volatile uint32_t *)&rwlp->rw_readers); + ASSERT(rwlp->rw_owner == 0); + ExReleaseResourceLite(&rwlp->rw_lock); + } +} + +int +rw_read_held(krwlock_t *rwlp) +{ + return (rw_lock_held(rwlp) && rwlp->rw_owner == NULL); +} + +int +rw_lock_held(krwlock_t *rwlp) +{ + /* + * ### not sure about this one ### + */ + return (rwlp->rw_owner == current_thread() || rwlp->rw_readers > 0); +} + +int +rw_write_held(krwlock_t *rwlp) +{ + return (rwlp->rw_owner == current_thread()); +} + +void +rw_downgrade(krwlock_t *rwlp) +{ + if (rwlp->rw_owner != current_thread()) + panic("SPL: rw_downgrade not WRITE lock held\n"); + rw_exit(rwlp); + rw_enter(rwlp, RW_READER); +} + +int +spl_rwlock_init(void) +{ + return (0); +} + +void +spl_rwlock_fini(void) +{ + ASSERT(zfs_active_rwlock == 0); +} diff --git a/module/os/windows/spl/spl-seg_kmem.c b/module/os/windows/spl/spl-seg_kmem.c new file mode 100644 index 000000000000..cd86479995a3 --- /dev/null +++ b/module/os/windows/spl/spl-seg_kmem.c @@ -0,0 +1,248 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 1998, 2010, Oracle and/or its affiliates. All rights reserved. + */ + +#include + +#include +#include + +#include +#include +#include + +#include + +/* + * seg_kmem is the primary kernel memory segment driver. It + * maps the kernel heap [kernelheap, ekernelheap), module text, + * and all memory which was allocated before the VM was initialized + * into kas. + * + * Pages which belong to seg_kmem are hashed into &kvp vnode at + * an offset equal to (u_offset_t)virt_addr, and have p_lckcnt >= 1. + * They must never be paged out since segkmem_fault() is a no-op to + * prevent recursive faults. + * + * Currently, seg_kmem pages are sharelocked (p_sharelock == 1) on + * __x86 and are unlocked (p_sharelock == 0) on __sparc. Once __x86 + * supports relocation the #ifdef kludges can be removed. + * + * seg_kmem pages may be subject to relocation by page_relocate(), + * provided that the HAT supports it; if this is so, segkmem_reloc + * will be set to a nonzero value. All boot time allocated memory as + * well as static memory is considered off limits to relocation. + * Pages are "relocatable" if p_state does not have P_NORELOC set, so + * we request P_NORELOC pages for memory that isn't safe to relocate. + * + * The kernel heap is logically divided up into four pieces: + * + * heap32_arena is for allocations that require 32-bit absolute + * virtual addresses (e.g. code that uses 32-bit pointers/offsets). + * + * heap_core is for allocations that require 2GB *relative* + * offsets; in other words all memory from heap_core is within + * 2GB of all other memory from the same arena. This is a requirement + * of the addressing modes of some processors in supervisor code. + * + * heap_arena is the general heap arena. + * + * static_arena is the static memory arena. Allocations from it + * are not subject to relocation so it is safe to use the memory + * physical address as well as the virtual address (e.g. the VA to + * PA translations are static). Caches may import from static_arena; + * all other static memory allocations should use static_alloc_arena. + * + * On some platforms which have limited virtual address space, seg_kmem + * may share [kernelheap, ekernelheap) with seg_kp; if this is so, + * segkp_bitmap is non-NULL, and each bit represents a page of virtual + * address space which is actually seg_kp mapped. + */ + +/* + * Rough stubbed Port for XNU. + * + * Copyright (c) 2014 Brendon Humphrey (brendon.humphrey@mac.com) + */ + + +#ifdef _KERNEL +#define XNU_KERNEL_PRIVATE + +#include + +#endif /* _KERNEL */ + +typedef int page_t; + +void *segkmem_alloc(vmem_t *vmp, size_t size, int vmflag); +void segkmem_free(vmem_t *vmp, void *inaddr, size_t size); + +/* Total memory held allocated */ +uint64_t segkmem_total_mem_allocated = 0; + +/* primary kernel heap arena */ +vmem_t *heap_arena; + +/* qcaches abd */ +vmem_t *abd_arena; + +#ifdef _KERNEL +extern uint64_t total_memory; +uint64_t stat_osif_malloc_success = 0; +uint64_t stat_osif_free = 0; +uint64_t stat_osif_malloc_bytes = 0; +uint64_t stat_osif_free_bytes = 0; +#endif + +void * +osif_malloc(uint64_t size) +{ +#ifdef _KERNEL + void *tr = NULL; + + tr = ExAllocatePoolWithTag(NonPagedPoolNx, size, '!SFZ'); + ASSERT(P2PHASE(tr, PAGE_SIZE) == 0); + if (tr != NULL) { + atomic_inc_64(&stat_osif_malloc_success); + atomic_add_64(&segkmem_total_mem_allocated, size); + atomic_add_64(&stat_osif_malloc_bytes, size); + return (tr); + } else { + dprintf("%s:%d: ExAllocatePoolWithTag failed (memusage: %llu)" + "\n", __func__, __LINE__, segkmem_total_mem_allocated); + ASSERT(0); + extern volatile unsigned int vm_page_free_wanted; + extern volatile unsigned int vm_page_free_min; + spl_free_set_pressure(vm_page_free_min); + vm_page_free_wanted = vm_page_free_min; + return (NULL); + } +#else + return (malloc(size)); +#endif +} + +void +osif_free(void *buf, uint64_t size) +{ +#ifdef _KERNEL + ExFreePoolWithTag(buf, '!SFZ'); + atomic_inc_64(&stat_osif_free); + atomic_sub_64(&segkmem_total_mem_allocated, size); + atomic_add_64(&stat_osif_free_bytes, size); +#else + free(buf); +#endif /* _KERNEL */ +} + +/* + * Configure vmem, such that the heap arena is fed, + * and drains to the kernel low level allocator. + */ +void +kernelheap_init() +{ + heap_arena = vmem_init("heap", NULL, 0, PAGESIZE, segkmem_alloc, + segkmem_free); +} + + +void +kernelheap_fini(void) +{ + vmem_fini(heap_arena); +} + +void * +segkmem_alloc(vmem_t *vmp, size_t size, int maybe_unmasked_vmflag) +{ + return (osif_malloc(size)); +} + +void +segkmem_free(vmem_t *vmp, void *inaddr, size_t size) +{ + osif_free(inaddr, size); + // since this is mainly called by spl_root_arena and free_arena, + // do we really want to wake up a waiter, just because we have + // transferred from one to the other? + // we already have vmem_add_a_gibibyte waking up waiters + // so specializing here seems wasteful + // (originally included in vmem_experiments) + // cv_signal(&vmp->vm_cv); +} + +/* + * OSX does not use separate heaps for the ZIO buffers, + * the ZFS code is structured such that the zio caches will + * fallback to using the kmem_default arena same + * as all the other caches. + */ +// smd: we nevertheless plumb in an arena with heap as parent, so that +// we can track stats and maintain the VM_ / qc settings differently +void +segkmem_abd_init() +{ + /* + * OpenZFS does not segregate the abd kmem cache out of the general + * heap, leading to large numbers of short-lived slabs exchanged + * between the kmem cache and it's parent. XNU absorbs this with a + * qcache, following its history of absorbing the pre-ABD zio file and + * metadata caches being qcached (which raises the exchanges with the + * general heap from PAGESIZE to 256k). + */ + + extern vmem_t *spl_heap_arena; + + abd_arena = vmem_create("abd_cache", NULL, 0, + PAGESIZE, vmem_alloc_impl, vmem_free_impl, spl_heap_arena, + 131072, VM_SLEEP | VMC_NO_QCACHE | VM_FIRSTFIT); + + VERIFY3P(abd_arena, !=, NULL); + + /* + * We also have a sub-arena for sub-page allocations, so as to avoid + * memory waste, while segregating ABDs for visibility and + * fragmentation control. + * + * This approach presently assumes SPA_MINBLOCKSIZE is 512 and that + * PAGESIZE is an even multiple of at least several SPA_MINBLOCKSIZE. + * This will be _Static_assert-ed in abd_os.c. + */ +#if 0 // macos + abd_subpage_arena = vmem_create("abd_subpage_cache", NULL, 0, + 512, vmem_alloc_impl, vmem_free_impl, abd_arena, + 131072, VM_SLEEP | VMC_NO_QCACHE | VM_FIRSTFIT); + + VERIFY3P(abd_subpage_arena, !=, NULL); +#endif +} + +void +segkmem_abd_fini(void) +{ + if (abd_arena) { + vmem_destroy(abd_arena); + } +} diff --git a/module/os/windows/spl/spl-taskq.c b/module/os/windows/spl/spl-taskq.c new file mode 100644 index 000000000000..979c91844b5c --- /dev/null +++ b/module/os/windows/spl/spl-taskq.c @@ -0,0 +1,2868 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + */ + +/* + * Copyright (C) 2015, 2020 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ + +/* + * Kernel task queues: general-purpose asynchronous task scheduling. + * + * A common problem in kernel programming is the need to schedule tasks + * to be performed later, by another thread. There are several reasons + * you may want or need to do this: + * + * (1) The task isn't time-critical, but your current code path is. + * + * (2) The task may require grabbing locks that you already hold. + * + * (3) The task may need to block (e.g. to wait for memory), but you + * cannot block in your current context. + * + * (4) Your code path can't complete because of some condition, but you can't + * sleep or fail, so you queue the task for later execution when condition + * disappears. + * + * (5) You just want a simple way to launch multiple tasks in parallel. + * + * Task queues provide such a facility. In its simplest form (used when + * performance is not a critical consideration) a task queue consists of a + * single list of tasks, together with one or more threads to service the + * list. There are some cases when this simple queue is not sufficient: + * + * (1) The task queues are very hot and there is a need to avoid data and lock + * contention over global resources. + * + * (2) Some tasks may depend on other tasks to complete, so they can't be put in + * the same list managed by the same thread. + * + * (3) Some tasks may block for a long time, and this should not block other + * tasks in the queue. + * + * To provide useful service in such cases we define a "dynamic task queue" + * which has an individual thread for each of the tasks. These threads are + * dynamically created as they are needed and destroyed when they are not in + * use. The API for managing task pools is the same as for managing task queues + * with the exception of a taskq creation flag TASKQ_DYNAMIC which tells that + * dynamic task pool behavior is desired. + * + * Dynamic task queues may also place tasks in the normal queue (called "backing + * queue") when task pool runs out of resources. Users of task queues may + * disallow such queued scheduling by specifying TQ_NOQUEUE in the dispatch + * flags. + * + * The backing task queue is also used for scheduling internal tasks needed for + * dynamic task queue maintenance. + * + * INTERFACES ================================================================== + * + * taskq_t *taskq_create(name, nthreads, pri, minalloc, maxall, flags); + * + * Create a taskq with specified properties. + * Possible 'flags': + * + * TASKQ_DYNAMIC: Create task pool for task management. If this flag is + * specified, 'nthreads' specifies the maximum number of threads in + * the task queue. Task execution order for dynamic task queues is + * not predictable. + * + * If this flag is not specified (default case) a + * single-list task queue is created with 'nthreads' threads + * servicing it. Entries in this queue are managed by + * taskq_ent_alloc() and taskq_ent_free() which try to keep the + * task population between 'minalloc' and 'maxalloc', but the + * latter limit is only advisory for TQ_SLEEP dispatches and the + * former limit is only advisory for TQ_NOALLOC dispatches. If + * TASKQ_PREPOPULATE is set in 'flags', the taskq will be + * prepopulated with 'minalloc' task structures. + * + * Since non-DYNAMIC taskqs are queues, tasks are guaranteed to be + * executed in the order they are scheduled if nthreads == 1. + * If nthreads > 1, task execution order is not predictable. + * + * TASKQ_PREPOPULATE: Prepopulate task queue with threads. + * Also prepopulate the task queue with 'minalloc' task structures. + * + * TASKQ_THREADS_CPU_PCT: This flag specifies that 'nthreads' should be + * interpreted as a percentage of the # of online CPUs on the + * system. The taskq subsystem will automatically adjust the + * number of threads in the taskq in response to CPU online + * and offline events, to keep the ratio. nthreads must be in + * the range [0,100]. + * + * The calculation used is: + * + * MAX((ncpus_online * percentage)/100, 1) + * + * This flag is not supported for DYNAMIC task queues. + * This flag is not compatible with TASKQ_CPR_SAFE. + * + * TASKQ_CPR_SAFE: This flag specifies that users of the task queue will + * use their own protocol for handling CPR issues. This flag is not + * supported for DYNAMIC task queues. This flag is not compatible + * with TASKQ_THREADS_CPU_PCT. + * + * The 'pri' field specifies the default priority for the threads that + * service all scheduled tasks. + * + * taskq_t *taskq_create_instance(name, instance, nthreads, pri, minalloc, + * maxall, flags); + * + * Like taskq_create(), but takes an instance number (or -1 to indicate + * no instance). + * + * taskq_t *taskq_create_proc(name, nthreads, pri, minalloc, maxall, proc, + * flags); + * + * Like taskq_create(), but creates the taskq threads in the specified + * system process. If proc != &p0, this must be called from a thread + * in that process. + * + * taskq_t *taskq_create_sysdc(name, nthreads, minalloc, maxall, proc, + * dc, flags); + * + * Like taskq_create_proc(), but the taskq threads will use the + * System Duty Cycle (SDC) scheduling class with a duty cycle of dc. + * + * void taskq_destroy(tap): + * + * Waits for any scheduled tasks to complete, then destroys the taskq. + * Caller should guarantee that no new tasks are scheduled in the closing + * taskq. + * + * taskqid_t taskq_dispatch(tq, func, arg, flags): + * + * Dispatches the task "func(arg)" to taskq. The 'flags' indicates whether + * the caller is willing to block for memory. The function returns an + * opaque value which is zero iff dispatch fails. If flags is TQ_NOSLEEP + * or TQ_NOALLOC and the task can't be dispatched, taskq_dispatch() fails + * and returns (taskqid_t)0. + * + * ASSUMES: func != NULL. + * + * Possible flags: + * TQ_NOSLEEP: Do not wait for resources; may fail. + * + * TQ_NOALLOC: Do not allocate memory; may fail. May only be used with + * non-dynamic task queues. + * + * TQ_NOQUEUE: Do not enqueue a task if it can't dispatch it due to + * lack of available resources and fail. If this flag is not + * set, and the task pool is exhausted, the task may be scheduled + * in the backing queue. This flag may ONLY be used with dynamic + * task queues. + * + * NOTE: This flag should always be used when a task queue is used + * for tasks that may depend on each other for completion. + * Enqueueing dependent tasks may create deadlocks. + * + * TQ_SLEEP: May block waiting for resources. May still fail for + * dynamic task queues if TQ_NOQUEUE is also specified, otherwise + * always succeed. + * + * TQ_FRONT: Puts the new task at the front of the queue. Be careful. + * + * NOTE: Dynamic task queues are much more likely to fail in + * taskq_dispatch() (especially if TQ_NOQUEUE was specified), so it + * is important to have backup strategies handling such failures. + * + * void taskq_dispatch_ent(tq, func, arg, flags, tqent) + * + * This is a light-weight form of taskq_dispatch(), that uses a + * preallocated taskq_ent_t structure for scheduling. As a + * result, it does not perform allocations and cannot ever fail. + * Note especially that it cannot be used with TASKQ_DYNAMIC + * taskqs. The memory for the tqent must not be modified or used + * until the function (func) is called. (However, func itself + * may safely modify or free this memory, once it is called.) + * Note that the taskq framework will NOT free this memory. + * + * void taskq_wait(tq): + * + * Waits for all previously scheduled tasks to complete. + * + * NOTE: It does not stop any new task dispatches. + * Do NOT call taskq_wait() from a task: it will cause deadlock. + * + * void taskq_suspend(tq) + * + * Suspend all task execution. Tasks already scheduled for a dynamic task + * queue will still be executed, but all new scheduled tasks will be + * suspended until taskq_resume() is called. + * + * int taskq_suspended(tq) + * + * Returns 1 if taskq is suspended and 0 otherwise. It is intended to + * ASSERT that the task queue is suspended. + * + * void taskq_resume(tq) + * + * Resume task queue execution. + * + * int taskq_member(tq, thread) + * + * Returns 1 if 'thread' belongs to taskq 'tq' and 0 otherwise. The + * intended use is to ASSERT that a given function is called in taskq + * context only. + * + * system_taskq + * + * Global system-wide dynamic task queue for common uses. It may be used by + * any subsystem that needs to schedule tasks and does not need to manage + * its own task queues. It is initialized quite early during system boot. + * + * IMPLEMENTATION ============================================================== + * + * This is schematic representation of the task queue structures. + * + * taskq: + * +-------------+ + * | tq_lock | +---< taskq_ent_free() + * +-------------+ | + * |... | | tqent: tqent: + * +-------------+ | +------------+ +------------+ + * | tq_freelist |-->| tqent_next |--> ... ->| tqent_next | + * +-------------+ +------------+ +------------+ + * |... | | ... | | ... | + * +-------------+ +------------+ +------------+ + * | tq_task | | + * | | +-------------->taskq_ent_alloc() + * +--------------------------------------------------------------------------+ + * | | | tqent tqent | + * | +---------------------+ +--> +------------+ +--> +------------+ | + * | | ... | | | func, arg | | | func, arg | | + * +>+---------------------+ <---|-+ +------------+ <---|-+ +------------+ | + * | tq_taskq.tqent_next | ----+ | | tqent_next | --->+ | | tqent_next |--+ + * +---------------------+ | +------------+ ^ | +------------+ + * +-| tq_task.tqent_prev | +--| tqent_prev | | +--| tqent_prev | ^ + * | +---------------------+ +------------+ | +------------+ | + * | |... | | ... | | | ... | | + * | +---------------------+ +------------+ | +------------+ | + * | ^ | | + * | | | | + * +--------------------------------------+--------------+ TQ_APPEND() -+ + * | | | + * |... | taskq_thread()-----+ + * +-------------+ + * | tq_buckets |--+-------> [ NULL ] (for regular task queues) + * +-------------+ | + * | DYNAMIC TASK QUEUES: + * | + * +-> taskq_bucket[nCPU] taskq_bucket_dispatch() + * +-------------------+ ^ + * +--->| tqbucket_lock | | + * | +-------------------+ +--------+ +--------+ + * | | tqbucket_freelist |-->| tqent |-->...| tqent | ^ + * | +-------------------+<--+--------+<--...+--------+ | + * | | ... | | thread | | thread | | + * | +-------------------+ +--------+ +--------+ | + * | +-------------------+ | + * taskq_dispatch()--+--->| tqbucket_lock | TQ_APPEND()------+ + * TQ_HASH() | +-------------------+ +--------+ +--------+ + * | | tqbucket_freelist |-->| tqent |-->...| tqent | + * | +-------------------+<--+--------+<--...+--------+ + * | | ... | | thread | | thread | + * | +-------------------+ +--------+ +--------+ + * +---> ... + * + * + * Task queues use tq_task field to link new entry in the queue. The queue is a + * circular doubly-linked list. Entries are put in the end of the list with + * TQ_APPEND() and processed from the front of the list by taskq_thread() in + * FIFO order. Task queue entries are cached in the free list managed by + * taskq_ent_alloc() and taskq_ent_free() functions. + * + * All threads used by task queues mark t_taskq field of the thread to + * point to the task queue. + * + * Taskq Thread Management ----------------------------------------------------- + * + * Taskq's non-dynamic threads are managed with several variables and flags: + * + * * tq_nthreads - The number of threads in taskq_thread() for the + * taskq. + * + * * tq_active - The number of threads not waiting on a CV in + * taskq_thread(); includes newly created threads + * not yet counted in tq_nthreads. + * + * * tq_nthreads_target + * - The number of threads desired for the taskq. + * + * * tq_flags & TASKQ_CHANGING + * - Indicates that tq_nthreads != tq_nthreads_target. + * + * * tq_flags & TASKQ_THREAD_CREATED + * - Indicates that a thread is being created in the taskq. + * + * During creation, tq_nthreads and tq_active are set to 0, and + * tq_nthreads_target is set to the number of threads desired. The + * TASKQ_CHANGING flag is set, and taskq_thread_create() is called to + * create the first thread. taskq_thread_create() increments tq_active, + * sets TASKQ_THREAD_CREATED, and creates the new thread. + * + * Each thread starts in taskq_thread(), clears the TASKQ_THREAD_CREATED + * flag, and increments tq_nthreads. It stores the new value of + * tq_nthreads as its "thread_id", and stores its thread pointer in the + * tq_threadlist at the (thread_id - 1). We keep the thread_id space + * densely packed by requiring that only the largest thread_id can exit during + * normal adjustment. The exception is during the destruction of the + * taskq; once tq_nthreads_target is set to zero, no new threads will be created + * for the taskq queue, so every thread can exit without any ordering being + * necessary. + * + * Threads will only process work if their thread id is <= tq_nthreads_target. + * + * When TASKQ_CHANGING is set, threads will check the current thread target + * whenever they wake up, and do whatever they can to apply its effects. + * + * TASKQ_THREAD_CPU_PCT -------------------------------------------------------- + * + * When a taskq is created with TASKQ_THREAD_CPU_PCT, we store their requested + * percentage in tq_threads_ncpus_pct, start them off with the correct thread + * target, and add them to the taskq_cpupct_list for later adjustment. + * + * We register taskq_cpu_setup() to be called whenever a CPU changes state. It + * walks the list of TASKQ_THREAD_CPU_PCT taskqs, adjusts their nthread_target + * if need be, and wakes up all of the threads to process the change. + * + * Dynamic Task Queues Implementation ------------------------------------------ + * + * For a dynamic task queues there is a 1-to-1 mapping between a thread and + * taskq_ent_structure. Each entry is serviced by its own thread and each thread + * is controlled by a single entry. + * + * Entries are distributed over a set of buckets. To avoid using modulo + * arithmetics the number of buckets is 2^n and is determined as the nearest + * power of two roundown of the number of CPUs in the system. Tunable + * variable 'taskq_maxbuckets' limits the maximum number of buckets. Each entry + * is attached to a bucket for its lifetime and can't migrate to other buckets. + * + * Entries that have scheduled tasks are not placed in any list. The dispatch + * function sets their "func" and "arg" fields and signals the corresponding + * thread to execute the task. Once the thread executes the task it clears the + * "func" field and places an entry on the bucket cache of free entries pointed + * by "tqbucket_freelist" field. ALL entries on the free list should have "func" + * field equal to NULL. The free list is a circular doubly-linked list identical + * in structure to the tq_task list above, but entries are taken from it in LIFO + * order - the last freed entry is the first to be allocated. The + * taskq_bucket_dispatch() function gets the most recently used entry from the + * free list, sets its "func" and "arg" fields and signals a worker thread. + * + * After executing each task a per-entry thread taskq_d_thread() places its + * entry on the bucket free list and goes to a timed sleep. If it wakes up + * without getting new task it removes the entry from the free list and destroys + * itself. The thread sleep time is controlled by a tunable variable + * `taskq_thread_timeout'. + * + * There are various statistics kept in the bucket which allows for later + * analysis of taskq usage patterns. Also, a global copy of taskq creation and + * death statistics is kept in the global taskq data structure. Since thread + * creation and death happen rarely, updating such global data does not present + * a performance problem. + * + * NOTE: Threads are not bound to any CPU and there is absolutely no association + * between the bucket and actual thread CPU, so buckets are used only to + * split resources and reduce resource contention. Having threads attached + * to the CPU denoted by a bucket may reduce number of times the job + * switches between CPUs. + * + * Current algorithm creates a thread whenever a bucket has no free + * entries. It would be nice to know how many threads are in the running + * state and don't create threads if all CPUs are busy with existing + * tasks, but it is unclear how such strategy can be implemented. + * + * Currently buckets are created statically as an array attached to task + * queue. On some system with nCPUs < max_ncpus it may waste system + * memory. One solution may be allocation of buckets when they are first + * touched, but it is not clear how useful it is. + * + * SUSPEND/RESUME implementation ----------------------------------------------- + * + * Before executing a task taskq_thread() (executing non-dynamic task + * queues) obtains taskq's thread lock as a reader. The taskq_suspend() + * function gets the same lock as a writer blocking all non-dynamic task + * execution. The taskq_resume() function releases the lock allowing + * taskq_thread to continue execution. + * + * For dynamic task queues, each bucket is marked as TQBUCKET_SUSPEND by + * taskq_suspend() function. After that taskq_bucket_dispatch() always + * fails, so that taskq_dispatch() will either enqueue tasks for a + * suspended backing queue or fail if TQ_NOQUEUE is specified in dispatch + * flags. + * + * NOTE: taskq_suspend() does not immediately block any tasks already + * scheduled for dynamic task queues. It only suspends new tasks + * scheduled after taskq_suspend() was called. + * + * taskq_member() function works by comparing a thread t_taskq pointer with + * the passed thread pointer. + * + * LOCKS and LOCK Hierarchy ---------------------------------------------------- + * + * There are three locks used in task queues: + * + * 1) The taskq_t's tq_lock, protecting global task queue state. + * + * 2) Each per-CPU bucket has a lock for bucket management. + * + * 3) The global taskq_cpupct_lock, which protects the list of + * TASKQ_THREADS_CPU_PCT taskqs. + * + * If both (1) and (2) are needed, tq_lock should be taken *after* the bucket + * lock. + * + * If both (1) and (3) are needed, tq_lock should be taken *after* + * taskq_cpupct_lock. + * + * DEBUG FACILITIES ------------------------------------------------------------ + * + * For DEBUG kernels it is possible to induce random failures to + * taskq_dispatch() function when it is given TQ_NOSLEEP argument. The value of + * taskq_dmtbf and taskq_smtbf tunables control the mean time between induced + * failures for dynamic and static task queues respectively. + * + * Setting TASKQ_STATISTIC to 0 will disable per-bucket statistics. + * + * TUNABLES -------------------------------------------------------------------- + * + * system_taskq_size - Size of the global system_taskq. + * This value is multiplied by nCPUs to determine + * actual size. + * Default value: 64 + * + * taskq_minimum_nthreads_max + * - Minimum size of the thread list for a taskq. + * Useful for testing different thread pool + * sizes by overwriting tq_nthreads_target. + * + * taskq_thread_timeout - Maximum idle time for taskq_d_thread() + * Default value: 5 minutes + * + * taskq_maxbuckets - Maximum number of buckets in any task queue + * Default value: 128 + * + * taskq_search_depth - Maximum # of buckets searched for a free entry + * Default value: 4 + * + * taskq_dmtbf - Mean time between induced dispatch failures + * for dynamic task queues. + * Default value: UINT_MAX (no induced failures) + * + * taskq_smtbf - Mean time between induced dispatch failures + * for static task queues. + * Default value: UINT_MAX (no induced failures) + * + * CONDITIONAL compilation ----------------------------------------------------- + * + * TASKQ_STATISTIC - If set will enable bucket statistic (default). + * + * DELAY DISPATCH -------------------------------------------------------------- + * + * taskq_delay_dispatch(): + * Create a tqd_delay node containing the dispatch information, and + * expire time. It is inserted sorted by time. The "tqd_delay" pointer is + * returned as "taskqid_t" (ID). The dispatcher thread is signalled. + * List is controlled by tqd_delay_lock and tqd_delay_cv. + * + * taskq_delay_dispatcher_thread(): + * Thread sitting in cv_wait()/cv_timedwait(), waiting either to be + * signalled or time expires. The sleeping time is the head of the list + * (expires the soonest due to sorting). If the head-node is expired + * we call taskq_dispatch() on the information. The taskqid_t is assigned + * to the node as the taskq is active. It also tagged the tqent as + * DELAYED. + * + * Once the taskqent completes, as DELAYED is set, it will run through + * the list to locate the tqdelay node, and free it. + * + * taskq_cancel_id(): + * If called, it uses the given taskqid_t/tqdelay to locate a match + * in the list (can't trust the node until it is confirmed to exist on list) + * If it present, and the tqe is not set (not yet active), the node is + * removed from the list. Cancel is complete. + * + * If tqe is set (active), taskq_cancel_id() will need to wait for + * the taskq to complete, sitting in cv_wait(). The active tqe will + * signal back when completed. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include /* For throttlefree */ +#include +#include +#include +#include + +#include + +static kmem_cache_t *taskq_ent_cache, *taskq_cache; + +static uint_t taskq_tsd; + +/* + * Pseudo instance numbers for taskqs without explicitly provided instance. + */ +static vmem_t *taskq_id_arena; + +/* Global system task queue for common use */ +taskq_t *system_taskq = NULL; +taskq_t *system_delay_taskq = NULL; + +/* + * Maximum number of entries in global system taskq is + * system_taskq_size * max_ncpus + */ +#ifdef _WIN32 +#define SYSTEM_TASKQ_SIZE 128 +#else +#define SYSTEM_TASKQ_SIZE 64 +#endif +int system_taskq_size = SYSTEM_TASKQ_SIZE; + +/* + * Minimum size for tq_nthreads_max; useful for those who want to play around + * with increasing a taskq's tq_nthreads_target. + */ +int taskq_minimum_nthreads_max = 1; + +/* + * We want to ensure that when taskq_create() returns, there is at least + * one thread ready to handle requests. To guarantee this, we have to wait + * for the second thread, since the first one cannot process requests until + * the second thread has been created. + */ +#define TASKQ_CREATE_ACTIVE_THREADS 2 + +/* Maximum percentage allowed for TASKQ_THREADS_CPU_PCT */ +#define TASKQ_CPUPCT_MAX_PERCENT 1000 +int taskq_cpupct_max_percent = TASKQ_CPUPCT_MAX_PERCENT; + +/* + * Dynamic task queue threads that don't get any work within + * taskq_thread_timeout destroy themselves + */ +#define TASKQ_THREAD_TIMEOUT (60 * 5) +int taskq_thread_timeout = TASKQ_THREAD_TIMEOUT; + +#define TASKQ_MAXBUCKETS 128 +int taskq_maxbuckets = TASKQ_MAXBUCKETS; + +/* + * When a bucket has no available entries another buckets are tried. + * taskq_search_depth parameter limits the amount of buckets that we search + * before failing. This is mostly useful in systems with many CPUs where we may + * spend too much time scanning busy buckets. + */ +#define TASKQ_SEARCH_DEPTH 4 +int taskq_search_depth = TASKQ_SEARCH_DEPTH; + +/* + * Hashing function: mix various bits of x. May be pretty much anything. + */ +#define TQ_HASH(x) ((x) ^ ((x) >> 11) ^ ((x) >> 17) ^ ((x) ^ 27)) + +/* + * We do not create any new threads when the system is low on memory and start + * throttling memory allocations. The following macro tries to estimate such + * condition. + */ +#ifdef _WIN32 +#define ENOUGH_MEMORY() (!spl_vm_pool_low()) +#else +#define ENOUGH_MEMORY() (freemem > throttlefree) +#endif + +/* + * Static functions. + */ +static taskq_t *taskq_create_common(const char *, int, int, pri_t, int, + int, proc_t *, uint_t, uint_t); +static void taskq_thread(void *); +static void taskq_d_thread(taskq_ent_t *); +static void taskq_bucket_extend(void *); +static int taskq_constructor(void *, void *, int); +static void taskq_destructor(void *, void *); +static int taskq_ent_constructor(void *, void *, int); +static void taskq_ent_destructor(void *, void *); +static taskq_ent_t *taskq_ent_alloc(taskq_t *, int); +static void taskq_ent_free(taskq_t *, taskq_ent_t *); +static int taskq_ent_exists(taskq_t *, task_func_t, void *); +static taskq_ent_t *taskq_bucket_dispatch(taskq_bucket_t *, task_func_t, + void *); + +/* + * Task queues kstats. + */ +struct taskq_kstat { + kstat_named_t tq_pid; + kstat_named_t tq_tasks; + kstat_named_t tq_executed; + kstat_named_t tq_maxtasks; + kstat_named_t tq_totaltime; + kstat_named_t tq_nalloc; + kstat_named_t tq_nactive; + kstat_named_t tq_pri; + kstat_named_t tq_nthreads; +} taskq_kstat = { + { "pid", KSTAT_DATA_UINT64 }, + { "tasks", KSTAT_DATA_UINT64 }, + { "executed", KSTAT_DATA_UINT64 }, + { "maxtasks", KSTAT_DATA_UINT64 }, + { "totaltime", KSTAT_DATA_UINT64 }, + { "nactive", KSTAT_DATA_UINT64 }, + { "nalloc", KSTAT_DATA_UINT64 }, + { "priority", KSTAT_DATA_UINT64 }, + { "threads", KSTAT_DATA_UINT64 }, +}; + +struct taskq_d_kstat { + kstat_named_t tqd_pri; + kstat_named_t tqd_btasks; + kstat_named_t tqd_bexecuted; + kstat_named_t tqd_bmaxtasks; + kstat_named_t tqd_bnalloc; + kstat_named_t tqd_bnactive; + kstat_named_t tqd_btotaltime; + kstat_named_t tqd_hits; + kstat_named_t tqd_misses; + kstat_named_t tqd_overflows; + kstat_named_t tqd_tcreates; + kstat_named_t tqd_tdeaths; + kstat_named_t tqd_maxthreads; + kstat_named_t tqd_nomem; + kstat_named_t tqd_disptcreates; + kstat_named_t tqd_totaltime; + kstat_named_t tqd_nalloc; + kstat_named_t tqd_nfree; +} taskq_d_kstat = { + { "priority", KSTAT_DATA_UINT64 }, + { "btasks", KSTAT_DATA_UINT64 }, + { "bexecuted", KSTAT_DATA_UINT64 }, + { "bmaxtasks", KSTAT_DATA_UINT64 }, + { "bnalloc", KSTAT_DATA_UINT64 }, + { "bnactive", KSTAT_DATA_UINT64 }, + { "btotaltime", KSTAT_DATA_UINT64 }, + { "hits", KSTAT_DATA_UINT64 }, + { "misses", KSTAT_DATA_UINT64 }, + { "overflows", KSTAT_DATA_UINT64 }, + { "tcreates", KSTAT_DATA_UINT64 }, + { "tdeaths", KSTAT_DATA_UINT64 }, + { "maxthreads", KSTAT_DATA_UINT64 }, + { "nomem", KSTAT_DATA_UINT64 }, + { "disptcreates", KSTAT_DATA_UINT64 }, + { "totaltime", KSTAT_DATA_UINT64 }, + { "nalloc", KSTAT_DATA_UINT64 }, + { "nfree", KSTAT_DATA_UINT64 }, +}; + +static kmutex_t taskq_kstat_lock; +static kmutex_t taskq_d_kstat_lock; +static int taskq_kstat_update(kstat_t *, int); +static int taskq_d_kstat_update(kstat_t *, int); + +/* + * List of all TASKQ_THREADS_CPU_PCT taskqs. + */ +static list_t taskq_cpupct_list; /* protected by cpu_lock */ + +/* + * Collect per-bucket statistic when TASKQ_STATISTIC is defined. + */ +#define TASKQ_STATISTIC 1 + +#if TASKQ_STATISTIC +#define TQ_STAT(b, x) b->tqbucket_stat.x++ +#else +#define TQ_STAT(b, x) +#endif + +/* + * Random fault injection. + */ +uint_t taskq_random; +uint_t taskq_dmtbf = UINT_MAX; /* mean time between injected failures */ +uint_t taskq_smtbf = UINT_MAX; /* mean time between injected failures */ + +/* + * TQ_NOSLEEP dispatches on dynamic task queues are always allowed to fail. + * + * TQ_NOSLEEP dispatches on static task queues can't arbitrarily fail because + * they could prepopulate the cache and make sure that they do not use more + * then minalloc entries. So, fault injection in this case insures that + * either TASKQ_PREPOPULATE is not set or there are more entries allocated + * than is specified by minalloc. TQ_NOALLOC dispatches are always allowed + * to fail, but for simplicity we treat them identically to TQ_NOSLEEP + * dispatches. + */ +#ifdef DEBUG +#define TASKQ_D_RANDOM_DISPATCH_FAILURE(tq, flag) \ + taskq_random = (taskq_random * 2416 + 374441) % 1771875;\ + if ((flag & TQ_NOSLEEP) && \ + taskq_random < 1771875 / taskq_dmtbf) { \ + return (0); \ + } + +#define TASKQ_S_RANDOM_DISPATCH_FAILURE(tq, flag) \ + taskq_random = (taskq_random * 2416 + 374441) % 1771875;\ + if ((flag & (TQ_NOSLEEP | TQ_NOALLOC)) && \ + (!(tq->tq_flags & TASKQ_PREPOPULATE) || \ + (tq->tq_nalloc > tq->tq_minalloc)) && \ + (taskq_random < (1771875 / taskq_smtbf))) { \ + mutex_exit(&tq->tq_lock); \ + return (0); \ + } +#else +#define TASKQ_S_RANDOM_DISPATCH_FAILURE(tq, flag) +#define TASKQ_D_RANDOM_DISPATCH_FAILURE(tq, flag) +#endif + +#define IS_EMPTY(l) (((l).tqent_prev == (l).tqent_next) && \ + ((l).tqent_prev == &(l))) + +/* + * Append `tqe' in the end of the doubly-linked list denoted by l. + */ +#define TQ_APPEND(l, tqe) { \ + tqe->tqent_next = &l; \ + tqe->tqent_prev = l.tqent_prev; \ + tqe->tqent_next->tqent_prev = tqe; \ + tqe->tqent_prev->tqent_next = tqe; \ +} +/* + * Prepend 'tqe' to the beginning of l + */ +#define TQ_PREPEND(l, tqe) { \ + tqe->tqent_next = l.tqent_next; \ + tqe->tqent_prev = &l; \ + tqe->tqent_next->tqent_prev = tqe; \ + tqe->tqent_prev->tqent_next = tqe; \ +} + +/* + * Schedule a task specified by func and arg into the task queue entry tqe. + */ +#define TQ_DO_ENQUEUE(tq, tqe, func, arg, front) { \ + ASSERT(MUTEX_HELD(&tq->tq_lock)); \ + _NOTE(CONSTCOND) \ + if (front) { \ + TQ_PREPEND(tq->tq_task, tqe); \ + } else { \ + TQ_APPEND(tq->tq_task, tqe); \ + } \ + tqe->tqent_func = (func); \ + tqe->tqent_arg = (arg); \ + tq->tq_tasks++; \ + if (tq->tq_tasks - tq->tq_executed > tq->tq_maxtasks) \ + tq->tq_maxtasks = tq->tq_tasks - tq->tq_executed; \ + cv_signal(&tq->tq_dispatch_cv); \ + DTRACE_PROBE2(taskq__enqueue, taskq_t *, tq, taskq_ent_t *, tqe); \ +} + +#define TQ_ENQUEUE(tq, tqe, func, arg) \ + TQ_DO_ENQUEUE(tq, tqe, func, arg, 0) + +#define TQ_ENQUEUE_FRONT(tq, tqe, func, arg) \ + TQ_DO_ENQUEUE(tq, tqe, func, arg, 1) + +/* + * Do-nothing task which may be used to prepopulate thread caches. + */ + +void +nulltask(void *unused) +{ +} + + +static int +taskq_constructor(void *buf, void *cdrarg, int kmflags) +{ + taskq_t *tq = buf; + + memset(tq, 0, sizeof (taskq_t)); + + mutex_init(&tq->tq_lock, NULL, MUTEX_DEFAULT, NULL); + rw_init(&tq->tq_threadlock, NULL, RW_DEFAULT, NULL); + cv_init(&tq->tq_dispatch_cv, NULL, CV_DEFAULT, NULL); + cv_init(&tq->tq_exit_cv, NULL, CV_DEFAULT, NULL); + cv_init(&tq->tq_wait_cv, NULL, CV_DEFAULT, NULL); + cv_init(&tq->tq_maxalloc_cv, NULL, CV_DEFAULT, NULL); + + tq->tq_task.tqent_next = &tq->tq_task; + tq->tq_task.tqent_prev = &tq->tq_task; + + return (0); +} + + +static void +taskq_destructor(void *buf, void *cdrarg) +{ + taskq_t *tq = buf; + + ASSERT(tq->tq_nthreads == 0); + ASSERT(tq->tq_buckets == NULL); + ASSERT(tq->tq_tcreates == 0); + ASSERT(tq->tq_tdeaths == 0); + + mutex_destroy(&tq->tq_lock); + rw_destroy(&tq->tq_threadlock); + cv_destroy(&tq->tq_dispatch_cv); + cv_destroy(&tq->tq_exit_cv); + cv_destroy(&tq->tq_wait_cv); + cv_destroy(&tq->tq_maxalloc_cv); +} + + +static int +taskq_ent_constructor(void *buf, void *cdrarg, int kmflags) +{ + taskq_ent_t *tqe = buf; + + tqe->tqent_thread = NULL; + cv_init(&tqe->tqent_cv, NULL, CV_DEFAULT, NULL); +#ifdef _WIN32 + /* Simulate TS_STOPPED */ + mutex_init(&tqe->tqent_thread_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&tqe->tqent_thread_cv, NULL, CV_DEFAULT, NULL); +#endif /* __APPLE__ */ + return (0); +} + + +static void +taskq_ent_destructor(void *buf, void *cdrarg) +{ + taskq_ent_t *tqe = buf; + + ASSERT(tqe->tqent_thread == NULL); + cv_destroy(&tqe->tqent_cv); +#ifdef _WIN32 + /* See comment in taskq_d_thread(). */ + mutex_destroy(&tqe->tqent_thread_lock); + cv_destroy(&tqe->tqent_thread_cv); +#endif /* __APPLE__ */ +} + + +struct tqdelay { + /* list of all dispatch_delay */ + list_node_t tqd_listnode; + /* time (list sorted on this, soonest first) */ + clock_t tqd_time; + /* func+arg for taskq_dispatch */ + taskq_t *tqd_taskq; + task_func_t *tqd_func; + void *tqd_arg; + uint_t tqd_tqflags; + /* tqe once dispatch called (currently executing) */ + taskqid_t tqd_ent; +}; + +typedef struct tqdelay tqdelay_t; + +static list_t tqd_list; +static kmutex_t tqd_delay_lock; +static kcondvar_t tqd_delay_cv; +static int tqd_do_exit = 0; + +static void +taskq_delay_dispatcher_thread(void *notused) +{ + callb_cpr_t cpr; + + dprintf("%s: starting\n", __func__); + CALLB_CPR_INIT(&cpr, &tqd_delay_lock, callb_generic_cpr, FTAG); + + mutex_enter(&tqd_delay_lock); + while (tqd_do_exit == 0) { + int didsleep = 0; + tqdelay_t *tqdnode; + CALLB_CPR_SAFE_BEGIN(&cpr); + + /* + * If list is empty, just sleep until signal, + * otherwise, sleep on list_head (lowest in the list) + */ + tqdnode = list_head(&tqd_list); + + if (tqdnode == NULL) + (void) cv_wait(&tqd_delay_cv, &tqd_delay_lock); + else + didsleep = cv_timedwait(&tqd_delay_cv, + &tqd_delay_lock, tqdnode->tqd_time); + CALLB_CPR_SAFE_END(&cpr, &tqd_delay_lock); + + if (tqd_do_exit != 0) + break; + + /* If we got a node, and we slept until expired, run it. */ + tqdnode = list_head(&tqd_list); + if (tqdnode != NULL) { + clock_t now = ddi_get_lbolt(); + + /* First node is 'running', nothing to do. */ + if (tqdnode->tqd_ent != TASKQID_INVALID) { + /* Set time in the future, just stick on end */ + tqdnode->tqd_time = now + SEC_TO_TICK(1); + list_remove(&tqd_list, tqdnode); + list_insert_tail(&tqd_list, tqdnode); + continue; + } + + /* Time has arrived */ + if (tqdnode->tqd_time <= now) { + + mutex_exit(&tqd_delay_lock); + /* Dispatch the taskq */ + tqdnode->tqd_ent = taskq_dispatch( + tqdnode->tqd_taskq, + tqdnode->tqd_func, tqdnode->tqd_arg, + tqdnode->tqd_tqflags | TQ_DELAYED); + + mutex_enter(&tqd_delay_lock); + if (tqdnode->tqd_ent == TASKQID_INVALID) { + /* Failed to dispatch, if !TQ_SLEEP */ + list_remove(&tqd_list, tqdnode); + kmem_free(tqdnode, sizeof (tqdelay_t)); + } + } + // + } + } + + tqd_do_exit = 0; + cv_broadcast(&tqd_delay_cv); + CALLB_CPR_EXIT(&cpr); /* drops lock */ + dprintf("%s: exit\n", __func__); + thread_exit(); +} + +taskqid_t +taskq_dispatch_delay(taskq_t *tq, task_func_t func, void *arg, uint_t tqflags, + clock_t expire_time) +{ + tqdelay_t *tqdnode; + + tqdnode = kmem_alloc(sizeof (tqdelay_t), KM_SLEEP); + + tqdnode->tqd_time = expire_time; + tqdnode->tqd_taskq = tq; + tqdnode->tqd_func = func; + tqdnode->tqd_arg = arg; + tqdnode->tqd_tqflags = tqflags; + tqdnode->tqd_ent = TASKQID_INVALID; + + mutex_enter(&tqd_delay_lock); + + /* Insert sorted on time */ + tqdelay_t *runner; + for (runner = list_head(&tqd_list); + runner != NULL; + runner = list_next(&tqd_list, runner)) + if (tqdnode->tqd_time < runner->tqd_time) { + list_insert_before(&tqd_list, runner, tqdnode); + break; + } + if (runner == NULL) { + list_insert_tail(&tqd_list, tqdnode); + } + + /* We have added to the list, wake the thread up */ + cv_broadcast(&tqd_delay_cv); + mutex_exit(&tqd_delay_lock); + + return ((taskqid_t)tqdnode); +} + +int +taskq_cancel_id(taskq_t *tq, taskqid_t id) +{ + tqdelay_t *task = (tqdelay_t *)id; + tqdelay_t *tqdnode; + + /* delay_taskq active? Linux will call with id==NULL */ + if (task != NULL) { + + /* Don't trust 'task' until it is found in the list */ +again: + mutex_enter(&tqd_delay_lock); + + for (tqdnode = list_head(&tqd_list); + tqdnode != NULL; + tqdnode = list_next(&tqd_list, tqdnode)) { + + if (tqdnode == task) { + /* + * First check if it has already started + * executing, if so, we want for signal + * that it has finished and restart the loop. + */ + if (tqdnode->tqd_ent != TASKQID_INVALID) { + (void) cv_timedwait(&tqd_delay_cv, + &tqd_delay_lock, + ddi_get_lbolt() + SEC_TO_TICK(1)); + mutex_exit(&tqd_delay_lock); + goto again; + } + + /* + * task exists and needs to be cancelled. + * remove it from list, and wake the thread up + * as it might be sleeping on this node. We can + * free the memory as "time" is passed in as a + * variable. + */ + list_remove(&tqd_list, tqdnode); + cv_signal(&tqd_delay_cv); + mutex_exit(&tqd_delay_lock); + + kmem_free(tqdnode, sizeof (tqdelay_t)); + + return (1); + + } // task == tqdnode + } // for + mutex_exit(&tqd_delay_lock); + } // task != NULL + return (0); +} + +void +taskq_start_delay_thread(void) +{ + list_create(&tqd_list, sizeof (tqdelay_t), + offsetof(tqdelay_t, tqd_listnode)); + mutex_init(&tqd_delay_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&tqd_delay_cv, NULL, CV_DEFAULT, NULL); + tqd_do_exit = 0; + (void) thread_create(NULL, 0, taskq_delay_dispatcher_thread, + NULL, 0, &p0, TS_RUN, minclsyspri); +} + +void +taskq_stop_delay_thread(void) +{ + tqdelay_t *tqdnode; + + mutex_enter(&tqd_delay_lock); + tqd_do_exit = 1; + /* + * The reclaim thread will set arc_reclaim_thread_exit back to + * FALSE when it is finished exiting; we're waiting for that. + */ + while (tqd_do_exit) { + cv_signal(&tqd_delay_cv); + cv_wait(&tqd_delay_cv, &tqd_delay_lock); + } + mutex_exit(&tqd_delay_lock); + mutex_destroy(&tqd_delay_lock); + cv_destroy(&tqd_delay_cv); + + while ((tqdnode = list_head(&tqd_list)) != NULL) { + list_remove(&tqd_list, tqdnode); + kmem_free(tqdnode, sizeof (tqdelay_t)); + } + + list_destroy(&tqd_list); +} + +int +spl_taskq_init(void) +{ + tsd_create(&taskq_tsd, NULL); + + taskq_ent_cache = kmem_cache_create("taskq_ent_cache", + sizeof (taskq_ent_t), 0, taskq_ent_constructor, + taskq_ent_destructor, NULL, NULL, NULL, 0); + taskq_cache = kmem_cache_create("taskq_cache", sizeof (taskq_t), + 0, taskq_constructor, taskq_destructor, NULL, NULL, NULL, 0); + taskq_id_arena = vmem_create("taskq_id_arena", + (void *)1, INT32_MAX, 1, NULL, NULL, NULL, 0, + VM_SLEEP | VMC_IDENTIFIER); + + list_create(&taskq_cpupct_list, sizeof (taskq_t), + offsetof(taskq_t, tq_cpupct_link)); + + mutex_init(&taskq_kstat_lock, NULL, MUTEX_DEFAULT, NULL); + mutex_init(&taskq_d_kstat_lock, NULL, MUTEX_DEFAULT, NULL); + + return (0); +} + +void +spl_taskq_fini(void) +{ + mutex_destroy(&taskq_d_kstat_lock); + mutex_destroy(&taskq_kstat_lock); + + if (taskq_cache) { + kmem_cache_destroy(taskq_cache); + taskq_cache = NULL; + } + if (taskq_ent_cache) { + kmem_cache_destroy(taskq_ent_cache); + taskq_ent_cache = NULL; + } + + list_destroy(&taskq_cpupct_list); + + vmem_destroy(taskq_id_arena); + + tsd_destroy(&taskq_tsd); +} + + + + +static void +taskq_update_nthreads(taskq_t *tq, uint_t ncpus) +{ + uint_t newtarget = TASKQ_THREADS_PCT(ncpus, tq->tq_threads_ncpus_pct); + +#ifndef _WIN32 + ASSERT(MUTEX_HELD(&cpu_lock)); +#endif + ASSERT(MUTEX_HELD(&tq->tq_lock)); + + /* We must be going from non-zero to non-zero; no exiting. */ + ASSERT3U(tq->tq_nthreads_target, !=, 0); + ASSERT3U(newtarget, !=, 0); + + ASSERT3U(newtarget, <=, tq->tq_nthreads_max); + if (newtarget != tq->tq_nthreads_target) { + tq->tq_flags |= TASKQ_CHANGING; + tq->tq_nthreads_target = newtarget; + cv_broadcast(&tq->tq_dispatch_cv); + cv_broadcast(&tq->tq_exit_cv); + } +} + +#ifndef _WIN32 +/* No dynamic CPU add/remove in XNU, so we can just use static ncpu math */ + +/* called during task queue creation */ +static void +taskq_cpupct_install(taskq_t *tq, cpupart_t *cpup) +{ + ASSERT(tq->tq_flags & TASKQ_THREADS_CPU_PCT); + + mutex_enter(&cpu_lock); + mutex_enter(&tq->tq_lock); + tq->tq_cpupart = cpup->cp_id; + taskq_update_nthreads(tq, cpup->cp_ncpus); + mutex_exit(&tq->tq_lock); + + list_insert_tail(&taskq_cpupct_list, tq); + mutex_exit(&cpu_lock); +} + +static void +taskq_cpupct_remove(taskq_t *tq) +{ + ASSERT(tq->tq_flags & TASKQ_THREADS_CPU_PCT); + + mutex_enter(&cpu_lock); + list_remove(&taskq_cpupct_list, tq); + mutex_exit(&cpu_lock); +} + + +static int +taskq_cpu_setup(cpu_setup_t what, int id, void *arg) +{ + taskq_t *tq; + cpupart_t *cp = cpu[id]->cpu_part; + uint_t ncpus = cp->cp_ncpus; + + ASSERT(MUTEX_HELD(&cpu_lock)); + ASSERT(ncpus > 0); + + switch (what) { + case CPU_OFF: + case CPU_CPUPART_OUT: + /* offlines are called *before* the cpu is offlined. */ + if (ncpus > 1) + ncpus--; + break; + + case CPU_ON: + case CPU_CPUPART_IN: + break; + + default: + return (0); /* doesn't affect cpu count */ + } + + for (tq = list_head(&taskq_cpupct_list); tq != NULL; + tq = list_next(&taskq_cpupct_list, tq)) { + + mutex_enter(&tq->tq_lock); + /* + * If the taskq is part of the cpuset which is changing, + * update its nthreads_target. + */ + if (tq->tq_cpupart == cp->cp_id) { + taskq_update_nthreads(tq, ncpus); + } + mutex_exit(&tq->tq_lock); + } + return (0); +} + +void +taskq_mp_init(void) +{ + mutex_enter(&cpu_lock); + register_cpu_setup_func(taskq_cpu_setup, NULL); + /* + * Make sure we're up to date. At this point in boot, there is only + * one processor set, so we only have to update the current CPU. + */ + (void) taskq_cpu_setup(CPU_ON, CPU->cpu_id, NULL); + mutex_exit(&cpu_lock); +} +#endif /* __APPLE__ */ + +/* + * Create global system dynamic task queue. + */ +void +system_taskq_init(void) +{ +#ifdef _WIN32 + system_taskq = taskq_create_common("system_taskq", 0, + system_taskq_size * max_ncpus, minclsyspri, 4, 512, &p0, 0, + TASKQ_DYNAMIC | TASKQ_PREPOPULATE | TASKQ_REALLY_DYNAMIC); +#else + system_taskq = taskq_create_common("system_taskq", 0, + system_taskq_size * max_ncpus, minclsyspri, 4, 512, &p0, 0, + TASKQ_DYNAMIC | TASKQ_PREPOPULATE); +#endif + + system_delay_taskq = taskq_create("system_delay_taskq", max_ncpus, + minclsyspri, max_ncpus, INT_MAX, TASKQ_PREPOPULATE); + + taskq_start_delay_thread(); +} + +void +system_taskq_fini(void) +{ + taskq_stop_delay_thread(); + + if (system_delay_taskq) + taskq_destroy(system_delay_taskq); + if (system_taskq) + taskq_destroy(system_taskq); + system_taskq = NULL; +} + +/* + * taskq_ent_alloc() + * + * Allocates a new taskq_ent_t structure either from the free list or from the + * cache. Returns NULL if it can't be allocated. + * + * Assumes: tq->tq_lock is held. + */ +static taskq_ent_t * +taskq_ent_alloc(taskq_t *tq, int flags) +{ + int kmflags = (flags & TQ_NOSLEEP) ? KM_NOSLEEP : KM_SLEEP; + taskq_ent_t *tqe; + clock_t wait_time; + clock_t wait_rv; + + ASSERT(MUTEX_HELD(&tq->tq_lock)); + + /* + * TQ_NOALLOC allocations are allowed to use the freelist, even if + * we are below tq_minalloc. + */ +again: if ((tqe = tq->tq_freelist) != NULL && + ((flags & TQ_NOALLOC) || tq->tq_nalloc >= tq->tq_minalloc)) { + tq->tq_freelist = tqe->tqent_next; + } else { + if (flags & TQ_NOALLOC) + return (NULL); + + if (tq->tq_nalloc >= tq->tq_maxalloc) { + if (kmflags & KM_NOSLEEP) + return (NULL); + + /* + * We don't want to exceed tq_maxalloc, but we can't + * wait for other tasks to complete (and thus free up + * task structures) without risking deadlock with + * the caller. So, we just delay for one second + * to throttle the allocation rate. If we have tasks + * complete before one second timeout expires then + * taskq_ent_free will signal us and we will + * immediately retry the allocation (reap free). + */ + wait_time = ddi_get_lbolt() + hz; + while (tq->tq_freelist == NULL) { + tq->tq_maxalloc_wait++; + wait_rv = cv_timedwait(&tq->tq_maxalloc_cv, + &tq->tq_lock, wait_time); + tq->tq_maxalloc_wait--; + if (wait_rv == -1) + break; + } + if (tq->tq_freelist) + goto again; /* reap freelist */ + + } + mutex_exit(&tq->tq_lock); + + tqe = kmem_cache_alloc(taskq_ent_cache, kmflags); + + mutex_enter(&tq->tq_lock); + if (tqe != NULL) + tq->tq_nalloc++; + } + + return (tqe); +} + +/* + * taskq_ent_free() + * + * Free taskq_ent_t structure by either putting it on the free list or freeing + * it to the cache. + * + * Assumes: tq->tq_lock is held. + */ +static void +taskq_ent_free(taskq_t *tq, taskq_ent_t *tqe) +{ + ASSERT(MUTEX_HELD(&tq->tq_lock)); + + if (tq->tq_nalloc <= tq->tq_minalloc) { + tqe->tqent_next = tq->tq_freelist; + tq->tq_freelist = tqe; + } else { + tq->tq_nalloc--; + mutex_exit(&tq->tq_lock); + kmem_cache_free(taskq_ent_cache, tqe); + mutex_enter(&tq->tq_lock); + } + + if (tq->tq_maxalloc_wait) + cv_signal(&tq->tq_maxalloc_cv); +} + +/* + * taskq_ent_exists() + * + * Return 1 if taskq already has entry for calling 'func(arg)'. + * + * Assumes: tq->tq_lock is held. + */ +static int +taskq_ent_exists(taskq_t *tq, task_func_t func, void *arg) +{ + taskq_ent_t *tqe; + + ASSERT(MUTEX_HELD(&tq->tq_lock)); + + for (tqe = tq->tq_task.tqent_next; tqe != &tq->tq_task; + tqe = tqe->tqent_next) + if ((tqe->tqent_func == func) && (tqe->tqent_arg == arg)) + return (1); + return (0); +} + +/* + * Dispatch a task "func(arg)" to a free entry of bucket b. + * + * Assumes: no bucket locks is held. + * + * Returns: a pointer to an entry if dispatch was successful. + * NULL if there are no free entries or if the bucket is suspended. + */ +static taskq_ent_t * +taskq_bucket_dispatch(taskq_bucket_t *b, task_func_t func, void *arg) +{ + taskq_ent_t *tqe; + + ASSERT(MUTEX_NOT_HELD(&b->tqbucket_lock)); + ASSERT(func != NULL); + + mutex_enter(&b->tqbucket_lock); + + ASSERT(b->tqbucket_nfree != 0 || IS_EMPTY(b->tqbucket_freelist)); + ASSERT(b->tqbucket_nfree == 0 || !IS_EMPTY(b->tqbucket_freelist)); + + /* + * Get en entry from the freelist if there is one. + * Schedule task into the entry. + */ + if ((b->tqbucket_nfree != 0) && + !(b->tqbucket_flags & TQBUCKET_SUSPEND)) { + tqe = b->tqbucket_freelist.tqent_prev; + + ASSERT(tqe != &b->tqbucket_freelist); + ASSERT(tqe->tqent_thread != NULL); + + tqe->tqent_prev->tqent_next = tqe->tqent_next; + tqe->tqent_next->tqent_prev = tqe->tqent_prev; + b->tqbucket_nalloc++; + b->tqbucket_nfree--; + tqe->tqent_func = func; + tqe->tqent_arg = arg; + TQ_STAT(b, tqs_hits); + cv_signal(&tqe->tqent_cv); + DTRACE_PROBE2(taskq__d__enqueue, taskq_bucket_t *, b, + taskq_ent_t *, tqe); + } else { + tqe = NULL; + TQ_STAT(b, tqs_misses); + } + mutex_exit(&b->tqbucket_lock); + return (tqe); +} + +/* + * Dispatch a task. + * + * Assumes: func != NULL + * + * Returns: NULL if dispatch failed. + * non-NULL if task dispatched successfully. + * Actual return value is the pointer to taskq entry that was used to + * dispatch a task. This is useful for debugging. + */ +taskqid_t +taskq_dispatch(taskq_t *tq, task_func_t func, void *arg, uint_t flags) +{ + taskq_bucket_t *bucket = NULL; /* Which bucket needs extension */ + taskq_ent_t *tqe = NULL; + taskq_ent_t *tqe1; + uint_t bsize; + + ASSERT(tq != NULL); + ASSERT(func != NULL); + + if (!(tq->tq_flags & TASKQ_DYNAMIC)) { + /* + * TQ_NOQUEUE flag can't be used with non-dynamic task queues. + */ + ASSERT(!(flags & TQ_NOQUEUE)); + /* + * Enqueue the task to the underlying queue. + */ + mutex_enter(&tq->tq_lock); + + TASKQ_S_RANDOM_DISPATCH_FAILURE(tq, flags); + + if ((tqe = taskq_ent_alloc(tq, flags)) == NULL) { + mutex_exit(&tq->tq_lock); + return (0); + } + /* Make sure we start without any flags */ + tqe->tqent_un.tqent_flags = 0; + + if (flags & TQ_DELAYED) + tqe->tqent_un.tqent_flags = TQENT_FLAG_DELAYED; + + if (flags & TQ_FRONT) { + TQ_ENQUEUE_FRONT(tq, tqe, func, arg); + } else { + TQ_ENQUEUE(tq, tqe, func, arg); + } + mutex_exit(&tq->tq_lock); + return ((taskqid_t)tqe); + } + + /* + * Dynamic taskq dispatching. + */ + ASSERT(!(flags & (TQ_NOALLOC | TQ_FRONT))); + TASKQ_D_RANDOM_DISPATCH_FAILURE(tq, flags); + + bsize = tq->tq_nbuckets; + + if (bsize == 1) { + /* + * In a single-CPU case there is only one bucket, so get + * entry directly from there. + */ + if ((tqe = taskq_bucket_dispatch(tq->tq_buckets, func, arg)) + != NULL) + return ((taskqid_t)tqe); /* Fastpath */ + bucket = tq->tq_buckets; + } else { + int loopcount; + taskq_bucket_t *b; + // uintptr_t h = ((uintptr_t)CPU + (uintptr_t)arg) >> 3; + uintptr_t h = ((uintptr_t)(CPU_SEQID<<3) + + (uintptr_t)arg) >> 3; + + h = TQ_HASH(h); + + /* + * The 'bucket' points to the original bucket that we hit. If we + * can't allocate from it, we search other buckets, but only + * extend this one. + */ + b = &tq->tq_buckets[h & (bsize - 1)]; + ASSERT(b->tqbucket_taskq == tq); /* Sanity check */ + + /* + * Do a quick check before grabbing the lock. If the bucket does + * not have free entries now, chances are very small that it + * will after we take the lock, so we just skip it. + */ + if (b->tqbucket_nfree != 0) { + if ((tqe = taskq_bucket_dispatch(b, func, arg)) != NULL) + return ((taskqid_t)tqe); /* Fastpath */ + } else { + TQ_STAT(b, tqs_misses); + } + + bucket = b; + loopcount = MIN(taskq_search_depth, bsize); + /* + * If bucket dispatch failed, search loopcount number of buckets + * before we give up and fail. + */ + do { + b = &tq->tq_buckets[++h & (bsize - 1)]; + ASSERT(b->tqbucket_taskq == tq); /* Sanity check */ + loopcount--; + + if (b->tqbucket_nfree != 0) { + tqe = taskq_bucket_dispatch(b, func, arg); + } else { + TQ_STAT(b, tqs_misses); + } + } while ((tqe == NULL) && (loopcount > 0)); + } + + /* + * At this point we either scheduled a task and (tqe != NULL) or failed + * (tqe == NULL). Try to recover from fails. + */ + + /* + * For KM_SLEEP dispatches, try to extend the bucket and retry dispatch. + */ + if ((tqe == NULL) && !(flags & TQ_NOSLEEP)) { + /* + * taskq_bucket_extend() may fail to do anything, but this is + * fine - we deal with it later. If the bucket was successfully + * extended, there is a good chance that taskq_bucket_dispatch() + * will get this new entry, unless someone is racing with us and + * stealing the new entry from under our nose. + * taskq_bucket_extend() may sleep. + */ + taskq_bucket_extend(bucket); + TQ_STAT(bucket, tqs_disptcreates); + if ((tqe = taskq_bucket_dispatch(bucket, func, arg)) != NULL) + return ((taskqid_t)tqe); + } + + ASSERT(bucket != NULL); + + /* + * Since there are not enough free entries in the bucket, add a + * taskq entry to extend it in the background using backing queue + * (unless we already have a taskq entry to perform that extension). + */ + mutex_enter(&tq->tq_lock); + if (!taskq_ent_exists(tq, taskq_bucket_extend, bucket)) { + if ((tqe1 = taskq_ent_alloc(tq, TQ_NOSLEEP)) != NULL) { + TQ_ENQUEUE_FRONT(tq, tqe1, taskq_bucket_extend, bucket); + } else { + TQ_STAT(bucket, tqs_nomem); + } + } + + /* + * Dispatch failed and we can't find an entry to schedule a task. + * Revert to the backing queue unless TQ_NOQUEUE was asked. + */ + if ((tqe == NULL) && !(flags & TQ_NOQUEUE)) { + if ((tqe = taskq_ent_alloc(tq, flags)) != NULL) { + TQ_ENQUEUE(tq, tqe, func, arg); + } else { + TQ_STAT(bucket, tqs_nomem); + } + } + mutex_exit(&tq->tq_lock); + + return ((taskqid_t)tqe); +} + +void +taskq_init_ent(taskq_ent_t *t) +{ + memset(t, 0, sizeof (*t)); +} + +void +taskq_dispatch_ent(taskq_t *tq, task_func_t func, void *arg, uint_t flags, + taskq_ent_t *tqe) +{ + ASSERT(func != NULL); + ASSERT(!(tq->tq_flags & TASKQ_DYNAMIC)); + + /* + * Mark it as a prealloc'd task. This is important + * to ensure that we don't free it later. + */ + tqe->tqent_un.tqent_flags |= TQENT_FLAG_PREALLOC; + tqe->tqent_un.tqent_flags &= ~TQENT_FLAG_DELAYED; + /* + * Enqueue the task to the underlying queue. + */ + mutex_enter(&tq->tq_lock); + + if (flags & TQ_FRONT) { + TQ_ENQUEUE_FRONT(tq, tqe, func, arg); + } else { + TQ_ENQUEUE(tq, tqe, func, arg); + } + mutex_exit(&tq->tq_lock); +} + +int +taskq_empty_ent(taskq_ent_t *t) +{ + if (t->tqent_prev == NULL && + t->tqent_next == NULL) + return (TRUE); + return (IS_EMPTY(*t)); +} + +/* + * Wait for all pending tasks to complete. + * Calling taskq_wait from a task will cause deadlock. + */ +void +taskq_wait(taskq_t *tq) +{ +#ifndef _WIN32 + ASSERT(tq != curthread->t_taskq); +#endif + + if (tq == NULL) + return; + + mutex_enter(&tq->tq_lock); + while (tq->tq_task.tqent_next != &tq->tq_task || tq->tq_active != 0) + cv_wait(&tq->tq_wait_cv, &tq->tq_lock); + mutex_exit(&tq->tq_lock); + + if (tq->tq_flags & TASKQ_DYNAMIC) { + taskq_bucket_t *b = tq->tq_buckets; + int bid = 0; + for (; (b != NULL) && (bid < tq->tq_nbuckets); b++, bid++) { + mutex_enter(&b->tqbucket_lock); + while (b->tqbucket_nalloc > 0) + cv_wait(&b->tqbucket_cv, &b->tqbucket_lock); + mutex_exit(&b->tqbucket_lock); + } + } +} + +/* + * ZOL implements taskq_wait_id() that can wait for a specific + * taskq to finish, rather than all active taskqs. Until it is + * implemented, we wait for all to complete. + */ +void +taskq_wait_id(taskq_t *tq, taskqid_t id) +{ + return (taskq_wait(tq)); +} + +void +taskq_wait_outstanding(taskq_t *tq, taskqid_t id) +{ + return (taskq_wait(tq)); +} + +/* + * Suspend execution of tasks. + * + * Tasks in the queue part will be suspended immediately upon return from this + * function. Pending tasks in the dynamic part will continue to execute, but all + * new tasks will be suspended. + */ +void +taskq_suspend(taskq_t *tq) +{ + rw_enter(&tq->tq_threadlock, RW_WRITER); + + if (tq->tq_flags & TASKQ_DYNAMIC) { + taskq_bucket_t *b = tq->tq_buckets; + int bid = 0; + for (; (b != NULL) && (bid < tq->tq_nbuckets); b++, bid++) { + mutex_enter(&b->tqbucket_lock); + b->tqbucket_flags |= TQBUCKET_SUSPEND; + mutex_exit(&b->tqbucket_lock); + } + } + /* + * Mark task queue as being suspended. Needed for taskq_suspended(). + */ + mutex_enter(&tq->tq_lock); + ASSERT(!(tq->tq_flags & TASKQ_SUSPENDED)); + tq->tq_flags |= TASKQ_SUSPENDED; + mutex_exit(&tq->tq_lock); +} + +/* + * returns: 1 if tq is suspended, 0 otherwise. + */ +int +taskq_suspended(taskq_t *tq) +{ + return ((tq->tq_flags & TASKQ_SUSPENDED) != 0); +} + +/* + * Resume taskq execution. + */ +void +taskq_resume(taskq_t *tq) +{ + ASSERT(RW_WRITE_HELD(&tq->tq_threadlock)); + + if (tq->tq_flags & TASKQ_DYNAMIC) { + taskq_bucket_t *b = tq->tq_buckets; + int bid = 0; + for (; (b != NULL) && (bid < tq->tq_nbuckets); b++, bid++) { + mutex_enter(&b->tqbucket_lock); + b->tqbucket_flags &= ~TQBUCKET_SUSPEND; + mutex_exit(&b->tqbucket_lock); + } + } + mutex_enter(&tq->tq_lock); + ASSERT(tq->tq_flags & TASKQ_SUSPENDED); + tq->tq_flags &= ~TASKQ_SUSPENDED; + mutex_exit(&tq->tq_lock); + + rw_exit(&tq->tq_threadlock); +} + +int +taskq_member(taskq_t *tq, struct kthread *thread) +{ + return (tq == (taskq_t *)tsd_get_by_thread(taskq_tsd, thread)); +} + +taskq_t * +taskq_of_curthread(void) +{ + return (tsd_get(taskq_tsd)); +} + +/* + * Creates a thread in the taskq. We only allow one outstanding create at + * a time. We drop and reacquire the tq_lock in order to avoid blocking other + * taskq activity while thread_create() or lwp_kernel_create() run. + * + * The first time we're called, we do some additional setup, and do not + * return until there are enough threads to start servicing requests. + */ +static void +taskq_thread_create(taskq_t *tq) +{ + kthread_t *t; + const boolean_t first = (tq->tq_nthreads == 0); + + ASSERT(MUTEX_HELD(&tq->tq_lock)); + ASSERT(tq->tq_flags & TASKQ_CHANGING); + ASSERT(tq->tq_nthreads < tq->tq_nthreads_target); + ASSERT(!(tq->tq_flags & TASKQ_THREAD_CREATED)); + + tq->tq_flags |= TASKQ_THREAD_CREATED; + tq->tq_active++; + mutex_exit(&tq->tq_lock); + + /* + * With TASKQ_DUTY_CYCLE the new thread must have an LWP + * as explained in ../disp/sysdc.c (for the msacct data). + * Otherwise simple kthreads are preferred. + */ + if ((tq->tq_flags & TASKQ_DUTY_CYCLE) != 0) { + /* Enforced in taskq_create_common */ + dprintf("SPL: taskq_thread_create(TASKQ_DUTY_CYCLE) seen\n"); +#ifndef _WIN32 + ASSERT3P(tq->tq_proc, !=, &p0); + t = lwp_kernel_create(tq->tq_proc, taskq_thread, tq, TS_RUN, + tq->tq_pri); +#else + t = thread_create_named(tq->tq_name, + NULL, 0, taskq_thread, tq, 0, tq->tq_proc, + TS_RUN, tq->tq_pri); +#endif + } else { + t = thread_create_named(tq->tq_name, + NULL, 0, taskq_thread, tq, 0, tq->tq_proc, + TS_RUN, tq->tq_pri); + } + + if (!first) { + mutex_enter(&tq->tq_lock); + return; + } + + /* + * We know the thread cannot go away, since tq cannot be + * destroyed until creation has completed. We can therefore + * safely dereference t. + */ + if (tq->tq_flags & TASKQ_THREADS_CPU_PCT) { +#ifdef _WIN32 + mutex_enter(&tq->tq_lock); + taskq_update_nthreads(tq, max_ncpus); + mutex_exit(&tq->tq_lock); +#else + taskq_cpupct_install(tq, t->t_cpupart); +#endif + } + mutex_enter(&tq->tq_lock); + + /* Wait until we can service requests. */ + while (tq->tq_nthreads != tq->tq_nthreads_target && + tq->tq_nthreads < TASKQ_CREATE_ACTIVE_THREADS) { + cv_wait(&tq->tq_wait_cv, &tq->tq_lock); + } +} + +/* + * Common "sleep taskq thread" function, which handles CPR stuff, as well + * as giving a nice common point for debuggers to find inactive threads. + */ +static clock_t +taskq_thread_wait(taskq_t *tq, kmutex_t *mx, kcondvar_t *cv, + callb_cpr_t *cprinfo, clock_t timeout) +{ + clock_t ret = 0; + + if (!(tq->tq_flags & TASKQ_CPR_SAFE)) { + CALLB_CPR_SAFE_BEGIN(cprinfo); + } + if ((signed long)timeout < 0) + cv_wait(cv, mx); + else + ret = cv_reltimedwait(cv, mx, timeout, TR_CLOCK_TICK); + + if (!(tq->tq_flags & TASKQ_CPR_SAFE)) { + CALLB_CPR_SAFE_END(cprinfo, mx); + } + + return (ret); +} + +#ifdef _WIN32 +/* + * Adjust thread policies for SYSDC and BATCH task threads + */ + +/* + * from osfmk/kern/thread.[hc] and osfmk/kern/ledger.c + * + * limit [is] a percentage of CPU over an interval in nanoseconds + * + * in particular limittime = (interval_ns * percentage) / 100 + * + * when a thread has enough cpu time accumulated to hit limittime, + * ast_taken->thread_block is seen in a stackshot (e.g. spindump) + * + * thread.h 204:#define MINIMUM_CPULIMIT_INTERVAL_MS 1 + * + * Illumos's sysdc updates its stats every 20 ms + * (sysdc_update_interval_msec) + * which is the tunable we can deal with here; xnu will + * take care of the bookkeeping and the amount of "break", + * which are the other Illumos tunables. + */ +#define CPULIMIT_INTERVAL (MSEC2NSEC(100ULL)) +#define THREAD_CPULIMIT_BLOCK 0x1 + + +static void +taskq_thread_set_cpulimit(taskq_t *tq) +{ +} + +/* + * Set up xnu thread importance, + * throughput and latency QOS. + * + * Approximate Illumos's SYSDC + * (/usr/src/uts/common/disp/sysdc.c) + * + * SYSDC tracks cpu runtime itself, and yields to + * other threads if + * onproc time / (onproc time + runnable time) + * exceeds the Duty Cycle threshold. + * + * Approximate this by + * [a] setting a thread_cpu_limit percentage, + * [b] setting the thread precedence + * slightly higher than normal, + * [c] setting the thread throughput and latency policies + * just less than USER_INTERACTIVE, and + * [d] turning on the + * TIMESHARE policy, which adjusts the thread + * priority based on cpu usage. + */ + +static void +set_taskq_thread_attributes(struct kthread *thread, taskq_t *tq) +{ +} + +#endif // __APPLE__ + +/* + * Worker thread for processing task queue. + */ +static void +taskq_thread(void *arg) +{ + int thread_id; + + taskq_t *tq = arg; + taskq_ent_t *tqe; + callb_cpr_t cprinfo; + hrtime_t start, end; + boolean_t freeit; + +#ifdef __APPLE__ + set_taskq_thread_attributes(current_thread(), tq); +#endif + + CALLB_CPR_INIT(&cprinfo, &tq->tq_lock, callb_generic_cpr, + tq->tq_name); + + tsd_set(taskq_tsd, tq); + mutex_enter(&tq->tq_lock); + thread_id = ++tq->tq_nthreads; + ASSERT(tq->tq_flags & TASKQ_THREAD_CREATED); + ASSERT(tq->tq_flags & TASKQ_CHANGING); + tq->tq_flags &= ~TASKQ_THREAD_CREATED; + + VERIFY3S(thread_id, <=, tq->tq_nthreads_max); + + if (tq->tq_nthreads_max == 1) + tq->tq_thread = (kthread_t *)curthread; + else + tq->tq_threadlist[thread_id - 1] = (kthread_t *)curthread; + + /* Allow taskq_create_common()'s taskq_thread_create() to return. */ + if (tq->tq_nthreads == TASKQ_CREATE_ACTIVE_THREADS) + cv_broadcast(&tq->tq_wait_cv); + + for (;;) { + if (tq->tq_flags & TASKQ_CHANGING) { + /* See if we're no longer needed */ + if (thread_id > tq->tq_nthreads_target) { + /* + * To preserve the one-to-one mapping between + * thread_id and thread, we must exit from + * highest thread ID to least. + * + * However, if everyone is exiting, the order + * doesn't matter, so just exit immediately. + * (this is safe, since you must wait for + * nthreads to reach 0 after setting + * tq_nthreads_target to 0) + */ + if (thread_id == tq->tq_nthreads || + tq->tq_nthreads_target == 0) { + break; + } + + /* Wait for higher thread_ids to exit */ + (void) taskq_thread_wait(tq, &tq->tq_lock, + &tq->tq_exit_cv, &cprinfo, -1); + continue; + } + + /* + * If no thread is starting taskq_thread(), we can + * do some bookkeeping. + */ + if (!(tq->tq_flags & TASKQ_THREAD_CREATED)) { + /* Check if we've reached our target */ + if (tq->tq_nthreads == tq->tq_nthreads_target) { + tq->tq_flags &= ~TASKQ_CHANGING; + cv_broadcast(&tq->tq_wait_cv); + } + /* Check if we need to create a thread */ + if (tq->tq_nthreads < tq->tq_nthreads_target) { + taskq_thread_create(tq); + continue; /* tq_lock was dropped */ + } + } + } + if ((tqe = tq->tq_task.tqent_next) == &tq->tq_task) { + if (--tq->tq_active == 0) + cv_broadcast(&tq->tq_wait_cv); + (void) taskq_thread_wait(tq, &tq->tq_lock, + &tq->tq_dispatch_cv, &cprinfo, -1); + tq->tq_active++; + continue; + } + + tqe->tqent_prev->tqent_next = tqe->tqent_next; + tqe->tqent_next->tqent_prev = tqe->tqent_prev; + mutex_exit(&tq->tq_lock); + + /* + * For prealloc'd tasks, we don't free anything. We + * have to check this now, because once we call the + * function for a prealloc'd taskq, we can't touch the + * tqent any longer (calling the function returns the + * ownershp of the tqent back to caller of + * taskq_dispatch.) + */ + if ((!(tq->tq_flags & TASKQ_DYNAMIC)) && + (tqe->tqent_un.tqent_flags & TQENT_FLAG_PREALLOC)) { + /* clear pointers to assist assertion checks */ + tqe->tqent_next = tqe->tqent_prev = NULL; + freeit = B_FALSE; + } else { + freeit = B_TRUE; + } + + int is_delayed = tqe->tqent_un.tqent_flags & TQENT_FLAG_DELAYED; + + rw_enter(&tq->tq_threadlock, RW_READER); + start = gethrtime(); + DTRACE_PROBE2(taskq__exec__start, taskq_t *, tq, + taskq_ent_t *, tqe); + tqe->tqent_func(tqe->tqent_arg); + DTRACE_PROBE2(taskq__exec__end, taskq_t *, tq, + taskq_ent_t *, tqe); + end = gethrtime(); + rw_exit(&tq->tq_threadlock); + + /* If we were launched as canceled, do some book keeping */ + if (!(tq->tq_flags & TASKQ_DYNAMIC) && is_delayed) { + tqdelay_t *tqdnode; + mutex_enter(&tqd_delay_lock); + /* Try to find this node on list */ + for (tqdnode = list_head(&tqd_list); + tqdnode != NULL; + tqdnode = list_next(&tqd_list, tqdnode)) { + if (tqdnode->tqd_ent == (taskqid_t)tqe) { + list_remove(&tqd_list, tqdnode); + kmem_free(tqdnode, sizeof (tqdelay_t)); + cv_broadcast(&tqd_delay_cv); + break; + } + } + mutex_exit(&tqd_delay_lock); + } + + mutex_enter(&tq->tq_lock); + tq->tq_totaltime += end - start; + tq->tq_executed++; + + if (freeit) + taskq_ent_free(tq, tqe); + } + + if (tq->tq_nthreads_max == 1) + tq->tq_thread = NULL; + else + tq->tq_threadlist[thread_id - 1] = NULL; + + /* We're exiting, and therefore no longer active */ + ASSERT(tq->tq_active > 0); + tq->tq_active--; + + ASSERT(tq->tq_nthreads > 0); + tq->tq_nthreads--; + + /* Wake up anyone waiting for us to exit */ + cv_broadcast(&tq->tq_exit_cv); + if (tq->tq_nthreads == tq->tq_nthreads_target) { + if (!(tq->tq_flags & TASKQ_THREAD_CREATED)) + tq->tq_flags &= ~TASKQ_CHANGING; + + cv_broadcast(&tq->tq_wait_cv); + } + + tsd_set(taskq_tsd, NULL); + + CALLB_CPR_EXIT(&cprinfo); + thread_exit(); + +} + +/* + * Worker per-entry thread for dynamic dispatches. + */ +static void +taskq_d_thread(taskq_ent_t *tqe) +{ + taskq_bucket_t *bucket = tqe->tqent_un.tqent_bucket; + taskq_t *tq = bucket->tqbucket_taskq; + kmutex_t *lock = &bucket->tqbucket_lock; + kcondvar_t *cv = &tqe->tqent_cv; + callb_cpr_t cprinfo; + clock_t w; + + CALLB_CPR_INIT(&cprinfo, lock, callb_generic_cpr, tq->tq_name); + +#ifdef _WIN32 + /* + * There's no way in Mac OS X KPI to create a thread + * in a suspended state (TS_STOPPED). So instead we + * use tqent_thread as a flag and wait for it to get + * initialized. + */ + mutex_enter(&tqe->tqent_thread_lock); + while (tqe->tqent_thread == (kthread_t *)0xCEDEC0DE) + cv_wait(&tqe->tqent_thread_cv, &tqe->tqent_thread_lock); + mutex_exit(&tqe->tqent_thread_lock); +#endif + + mutex_enter(lock); + + for (;;) { + /* + * If a task is scheduled (func != NULL), execute it, otherwise + * sleep, waiting for a job. + */ + if (tqe->tqent_func != NULL) { + hrtime_t start; + hrtime_t end; + + ASSERT(bucket->tqbucket_nalloc > 0); + + /* + * It is possible to free the entry right away before + * actually executing the task so that subsequent + * dispatches may immediately reuse it. But this, + * effectively, creates a two-length queue in the entry + * and may lead to a deadlock if the execution of the + * current task depends on the execution of the next + * scheduled task. So, we keep the entry busy until the + * task is processed. + */ + + mutex_exit(lock); + start = gethrtime(); + DTRACE_PROBE3(taskq__d__exec__start, taskq_t *, tq, + taskq_bucket_t *, bucket, taskq_ent_t *, tqe); + tqe->tqent_func(tqe->tqent_arg); + DTRACE_PROBE3(taskq__d__exec__end, taskq_t *, tq, + taskq_bucket_t *, bucket, taskq_ent_t *, tqe); + end = gethrtime(); + mutex_enter(lock); + bucket->tqbucket_totaltime += end - start; + + /* + * Return the entry to the bucket free list. + */ + tqe->tqent_func = NULL; + TQ_APPEND(bucket->tqbucket_freelist, tqe); + bucket->tqbucket_nalloc--; + bucket->tqbucket_nfree++; + ASSERT(!IS_EMPTY(bucket->tqbucket_freelist)); + /* + * taskq_wait() waits for nalloc to drop to zero on + * tqbucket_cv. + */ + cv_signal(&bucket->tqbucket_cv); + } + + /* + * At this point the entry must be in the bucket free list - + * either because it was there initially or because it just + * finished executing a task and put itself on the free list. + */ + ASSERT(bucket->tqbucket_nfree > 0); + /* + * Go to sleep unless we are closing. + * If a thread is sleeping too long, it dies. + */ + if (! (bucket->tqbucket_flags & TQBUCKET_CLOSE)) { + w = taskq_thread_wait(tq, lock, cv, + &cprinfo, taskq_thread_timeout * hz); + } + + /* + * At this point we may be in two different states: + * + * (1) tqent_func is set which means that a new task is + * dispatched and we need to execute it. + * + * (2) Thread is sleeping for too long or we are closing. In + * both cases destroy the thread and the entry. + */ + + /* If func is NULL we should be on the freelist. */ + ASSERT((tqe->tqent_func != NULL) || + (bucket->tqbucket_nfree > 0)); + /* If func is non-NULL we should be allocated */ + ASSERT((tqe->tqent_func == NULL) || + (bucket->tqbucket_nalloc > 0)); + + /* Check freelist consistency */ + ASSERT((bucket->tqbucket_nfree > 0) || + IS_EMPTY(bucket->tqbucket_freelist)); + ASSERT((bucket->tqbucket_nfree == 0) || + !IS_EMPTY(bucket->tqbucket_freelist)); + + if ((tqe->tqent_func == NULL) && + ((w == -1) || (bucket->tqbucket_flags & TQBUCKET_CLOSE))) { + /* + * This thread is sleeping for too long or we are + * closing - time to die. + * Thread creation/destruction happens rarely, + * so grabbing the lock is not a big performance issue. + * The bucket lock is dropped by CALLB_CPR_EXIT(). + */ + + /* Remove the entry from the free list. */ + tqe->tqent_prev->tqent_next = tqe->tqent_next; + tqe->tqent_next->tqent_prev = tqe->tqent_prev; + ASSERT(bucket->tqbucket_nfree > 0); + bucket->tqbucket_nfree--; + + TQ_STAT(bucket, tqs_tdeaths); + cv_signal(&bucket->tqbucket_cv); + tqe->tqent_thread = NULL; + mutex_enter(&tq->tq_lock); + tq->tq_tdeaths++; + mutex_exit(&tq->tq_lock); + CALLB_CPR_EXIT(&cprinfo); + kmem_cache_free(taskq_ent_cache, tqe); + thread_exit(); + } + } +} + + +/* + * Taskq creation. May sleep for memory. + * Always use automatically generated instances to avoid kstat name space + * collisions. + */ + +taskq_t * +taskq_create(const char *name, int nthreads, pri_t pri, int minalloc, + int maxalloc, uint_t flags) +{ + ASSERT((flags & ~TASKQ_INTERFACE_FLAGS) == 0); + + return (taskq_create_common(name, 0, nthreads, pri, minalloc, + maxalloc, &p0, 0, flags | TASKQ_NOINSTANCE)); +} + +/* + * Create an instance of task queue. It is legal to create task queues with the + * same name and different instances. + * + * taskq_create_instance is used by ddi_taskq_create() where it gets the + * instance from ddi_get_instance(). In some cases the instance is not + * initialized and is set to -1. This case is handled as if no instance was + * passed at all. + */ +taskq_t * +taskq_create_instance(const char *name, int instance, int nthreads, pri_t pri, + int minalloc, int maxalloc, uint_t flags) +{ + ASSERT((flags & ~TASKQ_INTERFACE_FLAGS) == 0); + ASSERT((instance >= 0) || (instance == -1)); + + if (instance < 0) { + flags |= TASKQ_NOINSTANCE; + } + + return (taskq_create_common(name, instance, nthreads, + pri, minalloc, maxalloc, &p0, 0, flags)); +} + +taskq_t * +taskq_create_proc(const char *name, int nthreads, pri_t pri, int minalloc, + int maxalloc, proc_t *proc, uint_t flags) +{ + ASSERT((flags & ~TASKQ_INTERFACE_FLAGS) == 0); +#ifndef _WIN32 + ASSERT(proc->p_flag & SSYS); +#endif + return (taskq_create_common(name, 0, nthreads, pri, minalloc, + maxalloc, proc, 0, flags | TASKQ_NOINSTANCE)); +} + +taskq_t * +taskq_create_sysdc(const char *name, int nthreads, int minalloc, + int maxalloc, proc_t *proc, uint_t dc, uint_t flags) +{ + ASSERT((flags & ~TASKQ_INTERFACE_FLAGS) == 0); +#ifndef _WIN32 + ASSERT(proc->p_flag & SSYS); +#else + dprintf("SPL: %s:%d: taskq_create_sysdc(%s, nthreads: %d," + " minalloc: %d, maxalloc: %d, proc, dc: %u, flags: %x)\n", + __func__, __LINE__, name, nthreads, + minalloc, maxalloc, dc, flags); +#endif + return (taskq_create_common(name, 0, nthreads, minclsyspri, minalloc, + maxalloc, proc, dc, flags | TASKQ_NOINSTANCE | TASKQ_DUTY_CYCLE)); +} + +static taskq_t * +taskq_create_common(const char *name, int instance, int nthreads, pri_t pri, + int minalloc, int maxalloc, proc_t *proc, uint_t dc, uint_t flags) +{ + taskq_t *tq = kmem_cache_alloc(taskq_cache, KM_SLEEP); +#ifdef _WIN32 + uint_t ncpus = max_ncpus; +#else + uint_t ncpus = ((boot_max_ncpus == -1) ? max_ncpus : boot_max_ncpus); +#endif + uint_t bsize; /* # of buckets - always power of 2 */ + int max_nthreads; + + /* + * We are not allowed to use TASKQ_DYNAMIC with taskq_dispatch_ent() + * but that is done by spa.c - so we will simply mask DYNAMIC out. + */ + if (!(flags & TASKQ_REALLY_DYNAMIC)) + flags &= ~TASKQ_DYNAMIC; + + /* + * TASKQ_DYNAMIC, TASKQ_CPR_SAFE and TASKQ_THREADS_CPU_PCT are all + * mutually incompatible. + */ + IMPLY((flags & TASKQ_DYNAMIC), !(flags & TASKQ_CPR_SAFE)); + IMPLY((flags & TASKQ_DYNAMIC), !(flags & TASKQ_THREADS_CPU_PCT)); + IMPLY((flags & TASKQ_CPR_SAFE), !(flags & TASKQ_THREADS_CPU_PCT)); + + /* Cannot have DYNAMIC with DUTY_CYCLE */ + IMPLY((flags & TASKQ_DYNAMIC), !(flags & TASKQ_DUTY_CYCLE)); + + /* Cannot have DUTY_CYCLE with a p0 kernel process */ + IMPLY((flags & TASKQ_DUTY_CYCLE), proc != &p0); + + /* Cannot have DC_BATCH without DUTY_CYCLE */ + ASSERT((flags & (TASKQ_DUTY_CYCLE|TASKQ_DC_BATCH)) + != TASKQ_DC_BATCH); + +#ifdef _WIN32 + /* Cannot have DC_BATCH or DUTY_CYCLE with TIMESHARE */ + IMPLY((flags & (TASKQ_DUTY_CYCLE|TASKQ_DC_BATCH)), + !(flags & TASKQ_TIMESHARE)); +#endif + + ASSERT(proc != NULL); + + bsize = 1 << (highbit(ncpus) - 1); + ASSERT(bsize >= 1); + bsize = MIN(bsize, taskq_maxbuckets); + + if (flags & TASKQ_DYNAMIC) { + ASSERT3S(nthreads, >=, 1); + tq->tq_maxsize = nthreads; + + /* For dynamic task queues use just one backup thread */ + nthreads = max_nthreads = 1; + + } else if (flags & TASKQ_THREADS_CPU_PCT) { + uint_t pct; + ASSERT3S(nthreads, >=, 0); + pct = nthreads; + + if (pct > taskq_cpupct_max_percent) + pct = taskq_cpupct_max_percent; + + /* + * If you're using THREADS_CPU_PCT, the process for the + * taskq threads must be curproc. This allows any pset + * binding to be inherited correctly. If proc is &p0, + * we won't be creating LWPs, so new threads will be assigned + * to the default processor set. + */ + /* ASSERT(curproc == proc || proc == &p0); */ + tq->tq_threads_ncpus_pct = pct; + nthreads = 1; /* corrected in taskq_thread_create() */ + max_nthreads = TASKQ_THREADS_PCT(max_ncpus, pct); + + } else { + ASSERT3S(nthreads, >=, 1); + max_nthreads = nthreads; + } + + if (max_nthreads < taskq_minimum_nthreads_max) + max_nthreads = taskq_minimum_nthreads_max; + + /* + * Make sure the name is 0-terminated, and conforms to the rules for + * C indentifiers + */ + (void) strlcpy(tq->tq_name, name, sizeof (tq->tq_name)); + strident_canon(tq->tq_name, sizeof (tq->tq_name)); + + tq->tq_flags = flags | TASKQ_CHANGING; + tq->tq_active = 0; + tq->tq_instance = instance; + tq->tq_nthreads_target = nthreads; + tq->tq_nthreads_max = max_nthreads; + tq->tq_minalloc = minalloc; + tq->tq_maxalloc = maxalloc; + tq->tq_nbuckets = bsize; + tq->tq_proc = proc; + tq->tq_pri = pri; + tq->tq_DC = dc; + list_link_init(&tq->tq_cpupct_link); + + if (max_nthreads > 1) + tq->tq_threadlist = kmem_alloc( + sizeof (kthread_t *) * max_nthreads, KM_SLEEP); + + mutex_enter(&tq->tq_lock); + if (flags & TASKQ_PREPOPULATE) { + while (minalloc-- > 0) + taskq_ent_free(tq, taskq_ent_alloc(tq, TQ_SLEEP)); + } + + /* + * Before we start creating threads for this taskq, take a + * zone hold so the zone can't go away before taskq_destroy + * makes sure all the taskq threads are gone. This hold is + * similar in purpose to those taken by zthread_create(). + */ +#ifndef _WIN32 + zone_hold(tq->tq_proc->p_zone); +#endif + /* + * Create the first thread, which will create any other threads + * necessary. taskq_thread_create will not return until we have + * enough threads to be able to process requests. + */ + taskq_thread_create(tq); + mutex_exit(&tq->tq_lock); + + if (flags & TASKQ_DYNAMIC) { + taskq_bucket_t *bucket = kmem_zalloc(sizeof (taskq_bucket_t) * + bsize, KM_SLEEP); + int b_id; + + tq->tq_buckets = bucket; + + /* Initialize each bucket */ + for (b_id = 0; b_id < bsize; b_id++, bucket++) { + mutex_init(&bucket->tqbucket_lock, NULL, MUTEX_DEFAULT, + NULL); + cv_init(&bucket->tqbucket_cv, NULL, CV_DEFAULT, NULL); + bucket->tqbucket_taskq = tq; + bucket->tqbucket_freelist.tqent_next = + bucket->tqbucket_freelist.tqent_prev = + &bucket->tqbucket_freelist; + if (flags & TASKQ_PREPOPULATE) + taskq_bucket_extend(bucket); + } + } + + /* + * Install kstats. + * We have two cases: + * 1) Instance is provided to taskq_create_instance(). In this case it + * should be >= 0 and we use it. + * + * 2) Instance is not provided and is automatically generated + */ + if (flags & TASKQ_NOINSTANCE) { + instance = tq->tq_instance = + (int)(uintptr_t)vmem_alloc_impl(taskq_id_arena, 1, + VM_SLEEP); + } + + if (flags & TASKQ_DYNAMIC) { + if ((tq->tq_kstat = kstat_create("unix", instance, + tq->tq_name, "taskq_d", KSTAT_TYPE_NAMED, + sizeof (taskq_d_kstat) / sizeof (kstat_named_t), + KSTAT_FLAG_VIRTUAL)) != NULL) { + tq->tq_kstat->ks_lock = &taskq_d_kstat_lock; + tq->tq_kstat->ks_data = &taskq_d_kstat; + tq->tq_kstat->ks_update = taskq_d_kstat_update; + tq->tq_kstat->ks_private = tq; + kstat_install(tq->tq_kstat); + } + } else { + if ((tq->tq_kstat = kstat_create("unix", instance, tq->tq_name, + "taskq", KSTAT_TYPE_NAMED, + sizeof (taskq_kstat) / sizeof (kstat_named_t), + KSTAT_FLAG_VIRTUAL)) != NULL) { + tq->tq_kstat->ks_lock = &taskq_kstat_lock; + tq->tq_kstat->ks_data = &taskq_kstat; + tq->tq_kstat->ks_update = taskq_kstat_update; + tq->tq_kstat->ks_private = tq; + kstat_install(tq->tq_kstat); + } + } + + return (tq); +} + +/* + * taskq_destroy(). + * + * Assumes: by the time taskq_destroy is called no one will use this task queue + * in any way and no one will try to dispatch entries in it. + */ +void +taskq_destroy(taskq_t *tq) +{ + taskq_bucket_t *b = tq->tq_buckets; + int bid = 0; + + ASSERT(! (tq->tq_flags & TASKQ_CPR_SAFE)); + + /* + * Destroy kstats. + */ + if (tq->tq_kstat != NULL) { + kstat_delete(tq->tq_kstat); + tq->tq_kstat = NULL; + } + + /* + * Destroy instance if needed. + */ + if (tq->tq_flags & TASKQ_NOINSTANCE) { + vmem_free_impl(taskq_id_arena, + (void *)(uintptr_t)(tq->tq_instance), 1); + tq->tq_instance = 0; + } + + /* + * Unregister from the cpupct list. + */ +#ifndef _WIN32 + if (tq->tq_flags & TASKQ_THREADS_CPU_PCT) { + taskq_cpupct_remove(tq); + } +#endif + + /* + * Wait for any pending entries to complete. + */ + taskq_wait(tq); + + mutex_enter(&tq->tq_lock); + ASSERT((tq->tq_task.tqent_next == &tq->tq_task) && + (tq->tq_active == 0)); + + /* notify all the threads that they need to exit */ + tq->tq_nthreads_target = 0; + + tq->tq_flags |= TASKQ_CHANGING; + cv_broadcast(&tq->tq_dispatch_cv); + cv_broadcast(&tq->tq_exit_cv); + + while (tq->tq_nthreads != 0) + cv_wait(&tq->tq_wait_cv, &tq->tq_lock); + + if (tq->tq_nthreads_max != 1) + kmem_free(tq->tq_threadlist, sizeof (kthread_t *) * + tq->tq_nthreads_max); + + tq->tq_minalloc = 0; + while (tq->tq_nalloc != 0) + taskq_ent_free(tq, taskq_ent_alloc(tq, TQ_SLEEP)); + + mutex_exit(&tq->tq_lock); + + /* + * Mark each bucket as closing and wakeup all sleeping threads. + */ + for (; (b != NULL) && (bid < tq->tq_nbuckets); b++, bid++) { + taskq_ent_t *tqe; + + mutex_enter(&b->tqbucket_lock); + + b->tqbucket_flags |= TQBUCKET_CLOSE; + /* Wakeup all sleeping threads */ + + for (tqe = b->tqbucket_freelist.tqent_next; + tqe != &b->tqbucket_freelist; tqe = tqe->tqent_next) + cv_signal(&tqe->tqent_cv); + + ASSERT(b->tqbucket_nalloc == 0); + + /* + * At this point we waited for all pending jobs to complete (in + * both the task queue and the bucket and no new jobs should + * arrive. Wait for all threads to die. + */ + while (b->tqbucket_nfree > 0) + cv_wait(&b->tqbucket_cv, &b->tqbucket_lock); + mutex_exit(&b->tqbucket_lock); + mutex_destroy(&b->tqbucket_lock); + cv_destroy(&b->tqbucket_cv); + } + + if (tq->tq_buckets != NULL) { + ASSERT(tq->tq_flags & TASKQ_DYNAMIC); + kmem_free(tq->tq_buckets, + sizeof (taskq_bucket_t) * tq->tq_nbuckets); + + /* Cleanup fields before returning tq to the cache */ + tq->tq_buckets = NULL; + tq->tq_tcreates = 0; + tq->tq_tdeaths = 0; + } else { + ASSERT(!(tq->tq_flags & TASKQ_DYNAMIC)); + } + + /* + * Now that all the taskq threads are gone, we can + * drop the zone hold taken in taskq_create_common + */ +#ifndef _WIN32 + zone_rele(tq->tq_proc->p_zone); +#endif + + tq->tq_threads_ncpus_pct = 0; + tq->tq_totaltime = 0; + tq->tq_tasks = 0; + tq->tq_maxtasks = 0; + tq->tq_executed = 0; + kmem_cache_free(taskq_cache, tq); +} + +/* + * Extend a bucket with a new entry on the free list and attach a worker thread + * to it. + * + * Argument: pointer to the bucket. + * + * This function may quietly fail. It is only used by taskq_dispatch() which + * handles such failures properly. + */ +static void +taskq_bucket_extend(void *arg) +{ + taskq_ent_t *tqe; + taskq_bucket_t *b = (taskq_bucket_t *)arg; + taskq_t *tq = b->tqbucket_taskq; + int nthreads; +#ifdef _WIN32 + kthread_t *thread; +#endif + + if (! ENOUGH_MEMORY()) { + TQ_STAT(b, tqs_nomem); + return; + } + + mutex_enter(&tq->tq_lock); + + /* + * Observe global taskq limits on the number of threads. + */ + if (tq->tq_tcreates++ - tq->tq_tdeaths > tq->tq_maxsize) { + tq->tq_tcreates--; + mutex_exit(&tq->tq_lock); + return; + } + mutex_exit(&tq->tq_lock); + + tqe = kmem_cache_alloc(taskq_ent_cache, KM_NOSLEEP); + + if (tqe == NULL) { + mutex_enter(&tq->tq_lock); + TQ_STAT(b, tqs_nomem); + tq->tq_tcreates--; + mutex_exit(&tq->tq_lock); + return; + } + + ASSERT(tqe->tqent_thread == NULL); + + tqe->tqent_un.tqent_bucket = b; + +#ifdef _WIN32 + /* + * There's no way in Mac OS X KPI to create a thread + * in a suspended state (TS_STOPPED). So instead we + * use tqent_thread as a flag and the thread must wait + * for it to be initialized (below). + */ + tqe->tqent_thread = (kthread_t *)0xCEDEC0DE; + thread = thread_create_named(tq->tq_name, + NULL, 0, (void (*)(void *))taskq_d_thread, + tqe, 0, pp0, TS_RUN, tq->tq_pri); + + set_taskq_thread_attributes(thread, tq); +#else + + /* + * Create a thread in a TS_STOPPED state first. If it is successfully + * created, place the entry on the free list and start the thread. + */ + tqe->tqent_thread = thread_create(NULL, 0, taskq_d_thread, tqe, + 0, tq->tq_proc, TS_STOPPED, tq->tq_pri); +#endif /* __APPLE__ */ + + /* + * Once the entry is ready, link it to the the bucket free list. + */ + mutex_enter(&b->tqbucket_lock); + tqe->tqent_func = NULL; + TQ_APPEND(b->tqbucket_freelist, tqe); + b->tqbucket_nfree++; + TQ_STAT(b, tqs_tcreates); + +#if TASKQ_STATISTIC + nthreads = b->tqbucket_stat.tqs_tcreates - + b->tqbucket_stat.tqs_tdeaths; + b->tqbucket_stat.tqs_maxthreads = MAX(nthreads, + b->tqbucket_stat.tqs_maxthreads); +#endif + + mutex_exit(&b->tqbucket_lock); + /* + * Start the stopped thread. + */ +#ifdef _WIN32 + mutex_enter(&tqe->tqent_thread_lock); + tqe->tqent_thread = thread; + cv_signal(&tqe->tqent_thread_cv); + mutex_exit(&tqe->tqent_thread_lock); +#else + thread_lock(tqe->tqent_thread); + tqe->tqent_thread->t_taskq = tq; + tqe->tqent_thread->t_schedflag |= TS_ALLSTART; + setrun_locked(tqe->tqent_thread); + thread_unlock(tqe->tqent_thread); +#endif /* __APPLE__ */ +} + +static int +taskq_kstat_update(kstat_t *ksp, int rw) +{ + struct taskq_kstat *tqsp = &taskq_kstat; + taskq_t *tq = ksp->ks_private; + + if (rw == KSTAT_WRITE) + return (EACCES); + +#ifdef _WIN32 + tqsp->tq_pid.value.ui64 = 0; /* kernel_task'd pid is 0 */ +#else + tqsp->tq_pid.value.ui64 = proc_pid(tq->tq_proc->p_pid); +#endif + tqsp->tq_tasks.value.ui64 = tq->tq_tasks; + tqsp->tq_executed.value.ui64 = tq->tq_executed; + tqsp->tq_maxtasks.value.ui64 = tq->tq_maxtasks; + tqsp->tq_totaltime.value.ui64 = tq->tq_totaltime; + tqsp->tq_nactive.value.ui64 = tq->tq_active; + tqsp->tq_nalloc.value.ui64 = tq->tq_nalloc; + tqsp->tq_pri.value.ui64 = tq->tq_pri; + tqsp->tq_nthreads.value.ui64 = tq->tq_nthreads; + return (0); +} + +static int +taskq_d_kstat_update(kstat_t *ksp, int rw) +{ + struct taskq_d_kstat *tqsp = &taskq_d_kstat; + taskq_t *tq = ksp->ks_private; + taskq_bucket_t *b = tq->tq_buckets; + int bid = 0; + + if (rw == KSTAT_WRITE) + return (EACCES); + + ASSERT(tq->tq_flags & TASKQ_DYNAMIC); + + tqsp->tqd_btasks.value.ui64 = tq->tq_tasks; + tqsp->tqd_bexecuted.value.ui64 = tq->tq_executed; + tqsp->tqd_bmaxtasks.value.ui64 = tq->tq_maxtasks; + tqsp->tqd_bnalloc.value.ui64 = tq->tq_nalloc; + tqsp->tqd_bnactive.value.ui64 = tq->tq_active; + tqsp->tqd_btotaltime.value.ui64 = tq->tq_totaltime; + tqsp->tqd_pri.value.ui64 = tq->tq_pri; + + tqsp->tqd_hits.value.ui64 = 0; + tqsp->tqd_misses.value.ui64 = 0; + tqsp->tqd_overflows.value.ui64 = 0; + tqsp->tqd_tcreates.value.ui64 = 0; + tqsp->tqd_tdeaths.value.ui64 = 0; + tqsp->tqd_maxthreads.value.ui64 = 0; + tqsp->tqd_nomem.value.ui64 = 0; + tqsp->tqd_disptcreates.value.ui64 = 0; + tqsp->tqd_totaltime.value.ui64 = 0; + tqsp->tqd_nalloc.value.ui64 = 0; + tqsp->tqd_nfree.value.ui64 = 0; + + for (; (b != NULL) && (bid < tq->tq_nbuckets); b++, bid++) { + tqsp->tqd_hits.value.ui64 += b->tqbucket_stat.tqs_hits; + tqsp->tqd_misses.value.ui64 += b->tqbucket_stat.tqs_misses; + tqsp->tqd_overflows.value.ui64 += b->tqbucket_stat.tqs_overflow; + tqsp->tqd_tcreates.value.ui64 += b->tqbucket_stat.tqs_tcreates; + tqsp->tqd_tdeaths.value.ui64 += b->tqbucket_stat.tqs_tdeaths; + tqsp->tqd_maxthreads.value.ui64 += + b->tqbucket_stat.tqs_maxthreads; + tqsp->tqd_nomem.value.ui64 += b->tqbucket_stat.tqs_nomem; + tqsp->tqd_disptcreates.value.ui64 += + b->tqbucket_stat.tqs_disptcreates; + tqsp->tqd_totaltime.value.ui64 += b->tqbucket_totaltime; + tqsp->tqd_nalloc.value.ui64 += b->tqbucket_nalloc; + tqsp->tqd_nfree.value.ui64 += b->tqbucket_nfree; + } + return (0); +} + +int +EMPTY_TASKQ(taskq_t *tq) +{ +#ifdef _KERNEL + return ((tq)->tq_task.tqent_next == &(tq)->tq_task); +#else + return (tq->tq_task.tqent_next == &tq->tq_task || tq->tq_active == 0); +#endif +} diff --git a/module/os/windows/spl/spl-thread.c b/module/os/windows/spl/spl-thread.c new file mode 100644 index 000000000000..6d2118136616 --- /dev/null +++ b/module/os/windows/spl/spl-thread.c @@ -0,0 +1,142 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2019 Jorgen Lundman + * + */ + +#include +#include +#include +#include +#include +#include +#include + +#include + +#include + +uint64_t zfs_threads = 0; + +kthread_t * +spl_thread_create( + caddr_t stk, + size_t stksize, + void (*proc)(void *), + void *arg, + size_t len, + int state, +#ifdef SPL_DEBUG_THREAD + char *filename, + int line, +#endif + pri_t pri) +{ + NTSTATUS result; + struct _KTHREAD *thread; + +#ifdef SPL_DEBUG_THREAD + dprintf("Start thread pri %d by '%s':%d\n", pri, + filename, line); +#endif + result = PsCreateSystemThread( + (void **)&thread, + 0, // DesiredAccess, + NULL, // ObjectAttributes, + NULL, // ProcessHandle, + 0, // ClientId, + proc, // StartRoutine, + arg); // StartContext + + if (result != STATUS_SUCCESS) + return (NULL); + + /* + * Improve the priority when asked to do so + * Thread priorities range from 0 to 31, where 0 is the lowest + * priority and 31 is the highest + */ + + if (pri > minclsyspri) { + // thread_precedence_policy_data_t policy; + // policy.importance = pri - minclsyspri; + + // thread_policy_set(thread, + // THREAD_PRECEDENCE_POLICY, + // (thread_policy_t)&policy, + // THREAD_PRECEDENCE_POLICY_COUNT); + + // TODO: Windows thread priority? + + // why is this call missing? + // KeSetBasePriorityThread(thread, 1); + } + + atomic_inc_64(&zfs_threads); + + // Convert thread handle to pethread, so it matches current_thread() + PETHREAD eThread; + ObReferenceObjectByHandle(thread, THREAD_ALL_ACCESS, 0, + KernelMode, (void **)&eThread, 0); + ObDereferenceObject(eThread); + ZwClose(thread); + return ((kthread_t *)eThread); +} + +kthread_t * +spl_current_thread(void) +{ + thread_t *cur_thread = current_thread(); + return ((kthread_t *)cur_thread); +} + +extern __declspec(noreturn) NTSTATUS PsTerminateSystemThread(NTSTATUS); + +__declspec(noreturn) void +spl_thread_exit(void) +{ + atomic_dec_64(&zfs_threads); + + tsd_thread_exit(); + (void) PsTerminateSystemThread(0); +} + + +/* + * IllumOS has callout.c - place it here until we find a better place + */ +callout_id_t +timeout_generic(int type, void (*func)(void *), void *arg, + hrtime_t expiration, hrtime_t resolution, int flags) +{ + // struct timespec ts; + // hrt2ts(expiration, &ts); + // bsd_timeout(func, arg, &ts); + /* + * bsd_untimeout() requires func and arg to cancel the timeout, so + * pass it back as the callout_id. If we one day were to implement + * untimeout_generic() they would pass it back to us + */ + return ((callout_id_t)arg); +} diff --git a/module/os/windows/spl/spl-time.c b/module/os/windows/spl/spl-time.c new file mode 100644 index 000000000000..94c5ed51da90 --- /dev/null +++ b/module/os/windows/spl/spl-time.c @@ -0,0 +1,150 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ +#define _KERNEL_MODE +#include +#include + +#include +#include +#include + +/* + * gethrtime() provides high-resolution timestamps with machine-dependent + * origin. Hence its primary use is to specify intervals. + */ + +static hrtime_t +zfs_abs_to_nano(uint64_t elapsed) +{ + return (elapsed * KeQueryTimeIncrement() * 100); +} + +/* Open Solaris lbolt is in hz */ +uint64_t +zfs_lbolt(void) +{ + uint64_t lbolt_hz; + LARGE_INTEGER ticks; + KeQueryTickCount(&ticks); + lbolt_hz = ticks.QuadPart * KeQueryTimeIncrement(); + lbolt_hz /= (10000000 / 119); // Solaris hz ? + return (lbolt_hz); +} + +hrtime_t +gethrtime(void) +{ + static LARGE_INTEGER start = { 0 }; + LARGE_INTEGER now; + if (start.QuadPart == 0) { + KeQueryTickCount(&start); + start.QuadPart--; + } + KeQueryTickCount(&now); + ASSERT((now.QuadPart != start.QuadPart)); + return (zfs_abs_to_nano(now.QuadPart - start.QuadPart)); +} + +/* + * Get bytes from the /dev/random generator. Returns 0 + * on success. Returns EAGAIN if there is insufficient entropy. + */ +int +random_get_bytes(uint8_t *ptr, uint32_t len) +{ + LARGE_INTEGER TickCount; + ULONG r; + PULONG b; + int i; + + KeQueryTickCount(&TickCount); + + b = (PULONG) ptr; + + for (i = 0; i < len / sizeof (ULONG); i++) + b[i] = RtlRandomEx(&TickCount.LowPart); + + len &= (sizeof (ULONG) - 1); + if (len > 0) { + r = RtlRandomEx(&TickCount.LowPart); + RtlCopyMemory(&b[i], &r, len); + } + return (0); +} + + +void +gethrestime(struct timespec *ts) +{ + LARGE_INTEGER now; + uint64_t tv[2]; +#if _WIN32_WINNT >= 0x0602 + KeQuerySystemTimePrecise(&now); +#else + KeQuerySystemTime(&now); +#endif + TIME_WINDOWS_TO_UNIX(now.QuadPart, tv); + // change macro to take 2 dst args, "sec and nsec" to avoid this step? + ts->tv_sec = tv[0]; + ts->tv_nsec = tv[1]; +} + +time_t +gethrestime_sec(void) +{ + struct timespec tv; + gethrestime(&tv); + return (tv.tv_sec); +} + +#if 0 +void +hrt2ts(hrtime_t hrt, struct timespec *tsp) +{ + uint32_t sec, nsec, tmp; + + tmp = (uint32_t)(hrt >> 30); + sec = tmp - (tmp >> 2); + sec = tmp - (sec >> 5); + sec = tmp + (sec >> 1); + sec = tmp - (sec >> 6) + 7; + sec = tmp - (sec >> 3); + sec = tmp + (sec >> 1); + sec = tmp + (sec >> 3); + sec = tmp + (sec >> 4); + tmp = (sec << 7) - sec - sec - sec; + tmp = (tmp << 7) - tmp - tmp - tmp; + tmp = (tmp << 7) - tmp - tmp - tmp; + nsec = (uint32_t)hrt - (tmp << 9); + while (nsec >= NANOSEC) { + nsec -= NANOSEC; + sec++; + } + tsp->tv_sec = (time_t)sec; + tsp->tv_nsec = nsec; +} +#endif diff --git a/module/os/windows/spl/spl-tsd.c b/module/os/windows/spl/spl-tsd.c new file mode 100644 index 000000000000..6e30c337ef6e --- /dev/null +++ b/module/os/windows/spl/spl-tsd.c @@ -0,0 +1,396 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2014,2017 Jorgen Lundman + * + * A thread will call tsd_create(&key, dtor) to allocate a new + * "variable" placement, called a "key". In IllumOS, this is the index + * into an array of dtors. (If dtor is passed as NULL, TSD internally + * set it to an empty function). So if the dtor array[i] is NULL, it + * is "free" and can be allocated. (returned as *key = i). + * IllumOS will grow this dtor array with realloc when required. + * Then Any Thread can set a value on this "key index", and this value + * is specific to each thread by calling tsd_set(key, value). + * And can be retrieved with tsd_get(key). + * When tsd_destroy(key) is called, we need to loop through all + * threads different "values", and call the dtor on each one. + * Likewise, we need to know when a thread exists, so we can clean up + * the values (by calling dtor for each one) so we patch into the + * thread_exit() call, to also call tsd_thread_exit(). + * + * In OsX, we build an array of the dtors, and return the key index, + * this is to store the dtor, and know which "key" values are valid. + * Then we build an AVL tree, indexed by , to store + * each thread's value. This allows us to do key access quick. + * On thread_exit, we iterate the dtor array, and for each key + * remove . + * On tsd_destroy(key), we use AVL find nearest with , then + * avl_next as long as key remains the same, to remove each thread value. + * + * Note a key of "0" is considered "invalid" in IllumOS, so we return + * a "1" based index, even though internally it is 0 based. + * + */ + +#include +#include +#include +#include +#include + +#include + +/* Initial size of array, and realloc growth size */ +#define TSD_ALLOC_SIZE 10 + +/* array of dtors, allocated in init */ +static dtor_func_t *tsd_dtor_array = NULL; +static uint32_t tsd_dtor_size = 0; + +static avl_tree_t tsd_tree; + +struct spl_tsd_node_s +{ + /* The index/key */ + uint_t tsd_key; + thread_t *tsd_thread; + + /* The payload */ + void *tsd_value; + + /* Internal mumbo */ + avl_node_t tsd_link_node; +}; +typedef struct spl_tsd_node_s spl_tsd_node_t; + +static kmutex_t spl_tsd_mutex; + +/* + * tsd_set - set thread specific data + * @key: lookup key + * @value: value to set + * + * Caller must prevent racing tsd_create() or tsd_destroy(), protected + * from racing tsd_get() or tsd_set() because it is thread specific. + * This function has been optimized to be fast for the update case. + * When setting the tsd initially it will be slower due to additional + * required locking and potential memory allocations. + * If the value is set to NULL, we also release it. + */ +int +tsd_set(uint_t key, void *value) +{ + spl_tsd_node_t *entry = NULL; + spl_tsd_node_t search; + avl_index_t loc; + uint_t i; + + /* Invalid key values? */ + if ((key < 1) || + (key >= tsd_dtor_size)) { + return (EINVAL); + } + + i = key - 1; + + /* + * First handle the easy case, already has a node/value + * so we just need to find it, update it. + */ + + search.tsd_key = i; + search.tsd_thread = current_thread(); + + mutex_enter(&spl_tsd_mutex); + entry = avl_find(&tsd_tree, &search, &loc); + mutex_exit(&spl_tsd_mutex); + + if (entry) { + + /* If value is set to NULL, release it as well */ + if (value == NULL) { + mutex_enter(&spl_tsd_mutex); + avl_remove(&tsd_tree, entry); + mutex_exit(&spl_tsd_mutex); + + kmem_free(entry, sizeof (*entry)); + return (0); + } + + entry->tsd_value = value; + return (0); + } + + /* No node, we need to create a new one and insert it. */ + /* But if the value is NULL, then why create one eh? */ + if (value == NULL) + return (0); + + entry = kmem_alloc(sizeof (spl_tsd_node_t), KM_SLEEP); + + entry->tsd_key = i; + entry->tsd_thread = current_thread(); + entry->tsd_value = value; + + mutex_enter(&spl_tsd_mutex); + avl_add(&tsd_tree, entry); + mutex_exit(&spl_tsd_mutex); + + return (0); +} + +/* + * tsd_get - get thread specific data for specified thread + * @key: lookup key + * + * Caller must prevent racing tsd_create() or tsd_destroy(). This + * implementation is designed to be fast and scalable, it does not + * lock the entire table only a single hash bin. + */ +void * +tsd_get_by_thread(uint_t key, thread_t *thread) +{ + spl_tsd_node_t *entry = NULL; + spl_tsd_node_t search; + avl_index_t loc; + uint_t i; + + /* Invalid key values? */ + if ((key < 1) || + (key >= tsd_dtor_size)) { + return (NULL); + } + + i = key - 1; + + search.tsd_key = i; + search.tsd_thread = thread; + + mutex_enter(&spl_tsd_mutex); + entry = avl_find(&tsd_tree, &search, &loc); + mutex_exit(&spl_tsd_mutex); + + return (entry ? entry->tsd_value : NULL); +} + +void * +tsd_get(uint_t key) +{ + return (tsd_get_by_thread(key, current_thread())); +} + +static void +tsd_internal_dtor(void *value) +{ +} + +/* + * Create TSD for a pid and fill in key with unique value, remember the dtor + * + * We cheat and create an entry with pid=0, to keep the dtor. + */ +void +tsd_create(uint_t *keyp, dtor_func_t dtor) +{ + uint_t i; + + if (*keyp) + return; // Should be 0 + + // Iterate the dtor_array, looking for first NULL + for (i = 0; i < TSD_ALLOC_SIZE; i++) { + if (tsd_dtor_array[i] == NULL) break; + } + + /* Do we need to grow the list? */ + if (i >= tsd_dtor_size) { + dprintf("SPL: tsd list growing not implemented\n"); + return; + } + + if (dtor == NULL) + dtor = tsd_internal_dtor; + + tsd_dtor_array[i] = dtor; + + *keyp = i + 1; +} + +void +tsd_destroy(uint_t *keyp) +{ + spl_tsd_node_t *entry = NULL, *next = NULL; + spl_tsd_node_t search; + avl_index_t loc; + dtor_func_t dtor = NULL; + uint_t i; + + /* Invalid key values? */ + if ((*keyp < 1) || + (*keyp >= tsd_dtor_size)) { + return; + } + + i = *keyp - 1; + *keyp = 0; + + ASSERT(tsd_dtor_array[i] != NULL); + + dtor = tsd_dtor_array[i]; + tsd_dtor_array[i] = NULL; + + /* + * For each thread; + * if it has a value + * call the dtor + */ + search.tsd_key = i; + search.tsd_thread = NULL; + + mutex_enter(&spl_tsd_mutex); + entry = avl_find(&tsd_tree, &search, &loc); + + /* + * "entry" should really be NULL here, as we searched for the + * NULL thread + */ + if (entry == NULL) + entry = avl_nearest(&tsd_tree, loc, AVL_AFTER); + + /* Now, free node, and go to next, as long as the key matches */ + while (entry && (entry->tsd_key == i)) { + next = AVL_NEXT(&tsd_tree, entry); + + /* If we have a value, call the dtor for this thread */ + if (entry->tsd_value) + dtor(entry->tsd_value); + + avl_remove(&tsd_tree, entry); + + kmem_free(entry, sizeof (*entry)); + + entry = next; + } + + mutex_exit(&spl_tsd_mutex); +} + + + +/* + * A thread is exiting, clear out any tsd values it might have. + */ +void +tsd_thread_exit(void) +{ + spl_tsd_node_t *entry = NULL; + spl_tsd_node_t search; + avl_index_t loc; + int i; + + search.tsd_thread = current_thread(); + + /* For all defined dtor/values */ + for (i = 0; i < tsd_dtor_size; i++) { + + /* If not allocated, skip */ + if (tsd_dtor_array[i] == NULL) + continue; + + /* Find out of this thread has a value */ + search.tsd_key = i; + + mutex_enter(&spl_tsd_mutex); + entry = avl_find(&tsd_tree, &search, &loc); + if (entry) avl_remove(&tsd_tree, entry); + mutex_exit(&spl_tsd_mutex); + + if (entry == NULL) + continue; + + /* If we have a value, call dtor */ + if (entry->tsd_value) + tsd_dtor_array[i](entry->tsd_value); + + kmem_free(entry, sizeof (*entry)); + } // for all i +} + +static int +tsd_tree_cmp(const void *arg1, const void *arg2) +{ + const spl_tsd_node_t *node1 = arg1; + const spl_tsd_node_t *node2 = arg2; + if (node1->tsd_key > node2->tsd_key) + return (1); + if (node1->tsd_key < node2->tsd_key) + return (-1); + if (node1->tsd_thread > node2->tsd_thread) + return (1); + if (node1->tsd_thread < node2->tsd_thread) + return (-1); + return (0); +} + +int +spl_tsd_init(void) +{ + tsd_dtor_array = kmem_zalloc(sizeof (dtor_func_t) * TSD_ALLOC_SIZE, + KM_SLEEP); + tsd_dtor_size = TSD_ALLOC_SIZE; + + mutex_init(&spl_tsd_mutex, NULL, MUTEX_DEFAULT, NULL); + avl_create(&tsd_tree, tsd_tree_cmp, + sizeof (spl_tsd_node_t), + offsetof(spl_tsd_node_t, tsd_link_node)); + return (0); +} + + +uint64_t +spl_tsd_size(void) +{ + return (avl_numnodes(&tsd_tree)); +} + +void +spl_tsd_fini(void) +{ + spl_tsd_node_t *entry = NULL; + void *cookie = NULL; + + dprintf("SPL: tsd unloading %llu\n", spl_tsd_size()); + + mutex_enter(&spl_tsd_mutex); + cookie = NULL; + while ((entry = avl_destroy_nodes(&tsd_tree, &cookie))) { + kmem_free(entry, sizeof (*entry)); + } + mutex_exit(&spl_tsd_mutex); + + avl_destroy(&tsd_tree); + mutex_destroy(&spl_tsd_mutex); + + kmem_free(tsd_dtor_array, sizeof (dtor_func_t) * tsd_dtor_size); + tsd_dtor_size = 0; +} diff --git a/module/os/windows/spl/spl-uio.c b/module/os/windows/spl/spl-uio.c new file mode 100644 index 000000000000..8da487b2fbda --- /dev/null +++ b/module/os/windows/spl/spl-uio.c @@ -0,0 +1,117 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#include +#include + +static int +zfs_uiomove_iov(void *p, size_t n, zfs_uio_rw_t rw, zfs_uio_t *uio) +{ + const struct iovec *iov = uio->uio_iov; + size_t skip = uio->uio_skip; + int cnt; + + while (n && uio->uio_resid) { + cnt = MIN(iov->iov_len - skip, n); + switch (uio->uio_segflg) { + case UIO_SYSSPACE: + if (rw == UIO_READ) + memcpy(iov->iov_base + skip, p, cnt); + else + memcpy((void *)p, iov->iov_base + skip, + cnt); + break; + default: + /* Probably have no uio from userland in Windows */ + VERIFY(0); + return (-1); + } + skip += cnt; + if (skip == iov->iov_len) { + skip = 0; + uio->uio_iov = (++iov); + uio->uio_iovcnt--; + } + uio->uio_skip = skip; + uio->uio_resid -= cnt; + uio->uio_loffset += cnt; + p = (caddr_t)p + cnt; + n -= cnt; + } + return (0); +} + +int +zfs_uiomove(const char *p, size_t n, enum uio_rw rw, zfs_uio_t *uio) +{ + int result; + + result = zfs_uiomove_iov((void *)p, n, rw, uio); + return (SET_ERROR(result)); +} + +/* + * same as uiomove() but doesn't modify uio structure. + * return in cbytes how many bytes were copied. + */ +int +zfs_uiocopy(const char *p, size_t n, enum uio_rw rw, zfs_uio_t *uio, + size_t *cbytes) +{ + int result; + + zfs_uio_t uio_copy; + + memcpy(&uio_copy, uio, sizeof (zfs_uio_t)); + result = zfs_uiomove_iov((void *)p, n, rw, &uio_copy); + + *cbytes = uio->uio_resid - uio_copy.uio_resid; + + return (result); +} + +void +zfs_uioskip(zfs_uio_t *uio, size_t n) +{ + if (n > uio->uio_resid) + return; + uio->uio_skip += n; + while (uio->uio_iovcnt && + uio->uio_skip >= uio->uio_iov->iov_len) { + uio->uio_skip -= uio->uio_iov->iov_len; + uio->uio_iov++; + uio->uio_iovcnt--; + } + uio->uio_loffset += n; + uio->uio_resid -= n; +} + +int +zfs_uio_prefaultpages(ssize_t n, zfs_uio_t *uio) +{ + return (0); +} diff --git a/module/os/windows/spl/spl-vmem.c b/module/os/windows/spl/spl-vmem.c new file mode 100644 index 000000000000..03e3e702251a --- /dev/null +++ b/module/os/windows/spl/spl-vmem.c @@ -0,0 +1,3960 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + +/* + * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2012, Joyent, Inc. All rights reserved. + * Copyright (c) 2017 Sean Doran + */ + +/* + * Big Theory Statement for the virtual memory allocator. + * + * For a more complete description of the main ideas, see: + * + * Jeff Bonwick and Jonathan Adams, + * + * Magazines and vmem: Extending the Slab Allocator to Many CPUs and + * Arbitrary Resources. + * + * Proceedings of the 2001 Usenix Conference. + * Available as http://www.usenix.org/event/usenix01/bonwick.html + * + * + * 1. General Concepts + * ------------------- + * + * 1.1 Overview + * ------------ + * We divide the kernel address space into a number of logically distinct + * pieces, or *arenas*: text, data, heap, stack, and so on. Within these + * arenas we often subdivide further; for example, we use heap addresses + * not only for the kernel heap (kmem_alloc() space), but also for DVMA, + * bp_mapin(), /dev/kmem, and even some device mappings like the TOD chip. + * The kernel address space, therefore, is most accurately described as + * a tree of arenas in which each node of the tree *imports* some subset + * of its parent. The virtual memory allocator manages these arenas and + * supports their natural hierarchical structure. + * + * 1.2 Arenas + * ---------- + * An arena is nothing more than a set of integers. These integers most + * commonly represent virtual addresses, but in fact they can represent + * anything at all. For example, we could use an arena containing the + * integers minpid through maxpid to allocate process IDs. vmem_create() + * and vmem_destroy() create and destroy vmem arenas. In order to + * differentiate between arenas used for adresses and arenas used for + * identifiers, the VMC_IDENTIFIER flag is passed to vmem_create(). This + * prevents identifier exhaustion from being diagnosed as general memory + * failure. + * + * 1.3 Spans + * --------- + * We represent the integers in an arena as a collection of *spans*, or + * contiguous ranges of integers. For example, the kernel heap consists + * of just one span: [kernelheap, ekernelheap). Spans can be added to an + * arena in two ways: explicitly, by vmem_add(), or implicitly, by + * importing, as described in Section 1.5 below. + * + * 1.4 Segments + * ------------ + * Spans are subdivided into *segments*, each of which is either allocated + * or free. A segment, like a span, is a contiguous range of integers. + * Each allocated segment [addr, addr + size) represents exactly one + * vmem_alloc_impl(size) that returned addr. Free segments represent the space + * between allocated segments. If two free segments are adjacent, we + * coalesce them into one larger segment; that is, if segments [a, b) and + * [b, c) are both free, we merge them into a single segment [a, c). + * The segments within a span are linked together in increasing-address order + * so we can easily determine whether coalescing is possible. + * + * Segments never cross span boundaries. When all segments within + * an imported span become free, we return the span to its source. + * + * 1.5 Imported Memory + * ------------------- + * As mentioned in the overview, some arenas are logical subsets of + * other arenas. For example, kmem_va_arena (a virtual address cache + * that satisfies most kmem_slab_create() requests) is just a subset + * of heap_arena (the kernel heap) that provides caching for the most + * common slab sizes. When kmem_va_arena runs out of virtual memory, + * it *imports* more from the heap; we say that heap_arena is the + * *vmem source* for kmem_va_arena. vmem_create() allows you to + * specify any existing vmem arena as the source for your new arena. + * Topologically, since every arena is a child of at most one source, + * the set of all arenas forms a collection of trees. + * + * 1.6 Constrained Allocations + * --------------------------- + * Some vmem clients are quite picky about the kind of address they want. + * For example, the DVMA code may need an address that is at a particular + * phase with respect to some alignment (to get good cache coloring), or + * that lies within certain limits (the addressable range of a device), + * or that doesn't cross some boundary (a DMA counter restriction) -- + * or all of the above. vmem_xalloc() allows the client to specify any + * or all of these constraints. + * + * 1.7 The Vmem Quantum + * -------------------- + * Every arena has a notion of 'quantum', specified at vmem_create() time, + * that defines the arena's minimum unit of currency. Most commonly the + * quantum is either 1 or PAGESIZE, but any power of 2 is legal. + * All vmem allocations are guaranteed to be quantum-aligned. + * + * 1.8 Quantum Caching + * ------------------- + * A vmem arena may be so hot (frequently used) that the scalability of vmem + * allocation is a significant concern. We address this by allowing the most + * common allocation sizes to be serviced by the kernel memory allocator, + * which provides low-latency per-cpu caching. The qcache_max argument to + * vmem_create() specifies the largest allocation size to cache. + * + * 1.9 Relationship to Kernel Memory Allocator + * ------------------------------------------- + * Every kmem cache has a vmem arena as its slab supplier. The kernel memory + * allocator uses vmem_alloc_impl() and vmem_free_impl() to create and + * destroy slabs. + * + * + * 2. Implementation + * ----------------- + * + * 2.1 Segment lists and markers + * ----------------------------- + * The segment structure (vmem_seg_t) contains two doubly-linked lists. + * + * The arena list (vs_anext/vs_aprev) links all segments in the arena. + * In addition to the allocated and free segments, the arena contains + * special marker segments at span boundaries. Span markers simplify + * coalescing and importing logic by making it easy to tell both when + * we're at a span boundary (so we don't coalesce across it), and when + * a span is completely free (its neighbors will both be span markers). + * + * Imported spans will have vs_import set. + * + * The next-of-kin list (vs_knext/vs_kprev) links segments of the same type: + * (1) for allocated segments, vs_knext is the hash chain linkage; + * (2) for free segments, vs_knext is the freelist linkage; + * (3) for span marker segments, vs_knext is the next span marker. + * + * 2.2 Allocation hashing + * ---------------------- + * We maintain a hash table of all allocated segments, hashed by address. + * This allows vmem_free_impl() to discover the target segment in constant + * time. + * vmem_update() periodically resizes hash tables to keep hash chains short. + * + * 2.3 Freelist management + * ----------------------- + * We maintain power-of-2 freelists for free segments, i.e. free segments + * of size >= 2^n reside in vmp->vm_freelist[n]. To ensure constant-time + * allocation, vmem_xalloc() looks not in the first freelist that *might* + * satisfy the allocation, but in the first freelist that *definitely* + * satisfies the allocation (unless VM_BESTFIT is specified, or all larger + * freelists are empty). For example, a 1000-byte allocation will be + * satisfied not from the 512..1023-byte freelist, whose members *might* + * contains a 1000-byte segment, but from a 1024-byte or larger freelist, + * the first member of which will *definitely* satisfy the allocation. + * This ensures that vmem_xalloc() works in constant time. + * + * We maintain a bit map to determine quickly which freelists are non-empty. + * vmp->vm_freemap & (1 << n) is non-zero iff vmp->vm_freelist[n] is non-empty. + * + * The different freelists are linked together into one large freelist, + * with the freelist heads serving as markers. Freelist markers simplify + * the maintenance of vm_freemap by making it easy to tell when we're taking + * the last member of a freelist (both of its neighbors will be markers). + * + * 2.4 Vmem Locking + * ---------------- + * For simplicity, all arena state is protected by a per-arena lock. + * For very hot arenas, use quantum caching for scalability. + * + * 2.5 Vmem Population + * ------------------- + * Any internal vmem routine that might need to allocate new segment + * structures must prepare in advance by calling vmem_populate(), which + * will preallocate enough vmem_seg_t's to get is through the entire + * operation without dropping the arena lock. + * + * 2.6 Auditing + * ------------ + * If KMF_AUDIT is set in kmem_flags, we audit vmem allocations as well. + * Since virtual addresses cannot be scribbled on, there is no equivalent + * in vmem to redzone checking, deadbeef, or other kmem debugging features. + * Moreover, we do not audit frees because segment coalescing destroys the + * association between an address and its segment structure. Auditing is + * thus intended primarily to keep track of who's consuming the arena. + * Debugging support could certainly be extended in the future if it proves + * necessary, but we do so much live checking via the allocation hash table + * that even non-DEBUG systems get quite a bit of sanity checking already. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#define VMEM_INITIAL 21 /* early vmem arenas */ +#define VMEM_SEG_INITIAL 800 + +/* + * Adding a new span to an arena requires two segment structures: one to + * represent the span, and one to represent the free segment it contains. + */ +#define VMEM_SEGS_PER_SPAN_CREATE 2 + +/* + * Allocating a piece of an existing segment requires 0-2 segment structures + * depending on how much of the segment we're allocating. + * + * To allocate the entire segment, no new segment structures are needed; we + * simply move the existing segment structure from the freelist to the + * allocation hash table. + * + * To allocate a piece from the left or right end of the segment, we must + * split the segment into two pieces (allocated part and remainder), so we + * need one new segment structure to represent the remainder. + * + * To allocate from the middle of a segment, we need two new segment strucures + * to represent the remainders on either side of the allocated part. + */ +#define VMEM_SEGS_PER_EXACT_ALLOC 0 +#define VMEM_SEGS_PER_LEFT_ALLOC 1 +#define VMEM_SEGS_PER_RIGHT_ALLOC 1 +#define VMEM_SEGS_PER_MIDDLE_ALLOC 2 + +/* + * vmem_populate() preallocates segment structures for vmem to do its work. + * It must preallocate enough for the worst case, which is when we must import + * a new span and then allocate from the middle of it. + */ +#define VMEM_SEGS_PER_ALLOC_MAX \ +(VMEM_SEGS_PER_SPAN_CREATE + VMEM_SEGS_PER_MIDDLE_ALLOC) + +/* + * The segment structures themselves are allocated from vmem_seg_arena, so + * we have a recursion problem when vmem_seg_arena needs to populate itself. + * We address this by working out the maximum number of segment structures + * this act will require, and multiplying by the maximum number of threads + * that we'll allow to do it simultaneously. + * + * The worst-case segment consumption to populate vmem_seg_arena is as + * follows (depicted as a stack trace to indicate why events are occurring): + * + * (In order to lower the fragmentation in the heap_arena, we specify a + * minimum import size for the vmem_metadata_arena which is the same size + * as the kmem_va quantum cache allocations. This causes the worst-case + * allocation from the vmem_metadata_arena to be 3 segments.) + * + * vmem_alloc_impl(vmem_seg_arena) -> 2 segs (span create + exact alloc) + * segkmem_alloc(vmem_metadata_arena) + * vmem_alloc_impl(vmem_metadata_arena) -> 3 segs (span create + left alloc) + * vmem_alloc_impl(heap_arena) -> 1 seg (left alloc) + * page_create() + * hat_memload() + * kmem_cache_alloc() + * kmem_slab_create() + * vmem_alloc_impl(hat_memload_arena) -> 2 segs (span create + exact alloc) + * segkmem_alloc(heap_arena) + * vmem_alloc_impl(heap_arena) -> 1 seg (left alloc) + * page_create() + * hat_memload() -> (hat layer won't recurse further) + * + * The worst-case consumption for each arena is 3 segment structures. + * Of course, a 3-seg reserve could easily be blown by multiple threads. + * Therefore, we serialize all allocations from vmem_seg_arena (which is OK + * because they're rare). We cannot allow a non-blocking allocation to get + * tied up behind a blocking allocation, however, so we use separate locks + * for VM_SLEEP and VM_NOSLEEP allocations. Similarly, VM_PUSHPAGE allocations + * must not block behind ordinary VM_SLEEPs. In addition, if the system is + * panicking then we must keep enough resources for panic_thread to do its + * work. Thus we have at most four threads trying to allocate from + * vmem_seg_arena, and each thread consumes at most three segment structures, + * so we must maintain a 12-seg reserve. + */ +#define VMEM_POPULATE_RESERVE 12 + +/* + * vmem_populate() ensures that each arena has VMEM_MINFREE seg structures + * so that it can satisfy the worst-case allocation *and* participate in + * worst-case allocation from vmem_seg_arena. + */ +#define VMEM_MINFREE (VMEM_POPULATE_RESERVE + VMEM_SEGS_PER_ALLOC_MAX) + +static vmem_t vmem0[VMEM_INITIAL]; +static vmem_t *vmem_populator[VMEM_INITIAL]; +static uint32_t vmem_id; +static uint32_t vmem_populators; +static vmem_seg_t vmem_seg0[VMEM_SEG_INITIAL]; +static vmem_seg_t *vmem_segfree; +static kmutex_t vmem_list_lock; +static kmutex_t vmem_segfree_lock; +static kmutex_t vmem_sleep_lock; +static kmutex_t vmem_nosleep_lock; +static kmutex_t vmem_pushpage_lock; +static kmutex_t vmem_panic_lock; +static kmutex_t vmem_xnu_alloc_lock; +static vmem_t *vmem_list; +static vmem_t *vmem_metadata_arena; +static vmem_t *vmem_seg_arena; +static vmem_t *vmem_hash_arena; +static vmem_t *vmem_vmem_arena; +vmem_t *spl_default_arena; // The bottom-most arena for SPL +static vmem_t *spl_default_arena_parent; // dummy arena as a placeholder +#define VMEM_BUCKETS 13 +#define VMEM_BUCKET_LOWBIT 12 +#define VMEM_BUCKET_HIBIT 24 +static vmem_t *vmem_bucket_arena[VMEM_BUCKETS]; +vmem_t *spl_heap_arena; +static void *spl_heap_arena_initial_alloc; +static size_t spl_heap_arena_initial_alloc_size = 0; +#define NUMBER_OF_ARENAS_IN_VMEM_INIT 21 +/* vmem_update() every 15 seconds */ +static struct timespec vmem_update_interval = {15, 0}; +uint32_t vmem_mtbf; /* mean time between failures [default: off] */ +size_t vmem_seg_size = sizeof (vmem_seg_t); + +static struct bsd_timeout_wrapper vmem_update_timer; + + +// must match with include/sys/vmem_impl.h +static vmem_kstat_t vmem_kstat_template = { + { "mem_inuse", KSTAT_DATA_UINT64 }, + { "mem_import", KSTAT_DATA_UINT64 }, + { "mem_total", KSTAT_DATA_UINT64 }, + { "vmem_source", KSTAT_DATA_UINT32 }, + { "alloc", KSTAT_DATA_UINT64 }, + { "free", KSTAT_DATA_UINT64 }, + { "wait", KSTAT_DATA_UINT64 }, + { "fail", KSTAT_DATA_UINT64 }, + { "lookup", KSTAT_DATA_UINT64 }, + { "search", KSTAT_DATA_UINT64 }, + { "populate_fail", KSTAT_DATA_UINT64 }, + { "contains", KSTAT_DATA_UINT64 }, + { "contains_search", KSTAT_DATA_UINT64 }, + { "parent_alloc", KSTAT_DATA_UINT64 }, + { "parent_free", KSTAT_DATA_UINT64 }, + { "threads_waiting", KSTAT_DATA_UINT64 }, + { "excess", KSTAT_DATA_UINT64 }, +}; + + +/* + * Insert/delete from arena list (type 'a') or next-of-kin list (type 'k'). + */ +#define VMEM_INSERT(vprev, vsp, type) \ +{ \ +vmem_seg_t *_vnext = (vprev)->vs_##type##next; \ +(vsp)->vs_##type##next = (_vnext); \ +(vsp)->vs_##type##prev = (vprev); \ +(vprev)->vs_##type##next = (vsp); \ +(_vnext)->vs_##type##prev = (vsp); \ +} + +#define VMEM_DELETE(vsp, type) \ +{ \ +vmem_seg_t *_vprev = (vsp)->vs_##type##prev; \ +vmem_seg_t *_vnext = (vsp)->vs_##type##next; \ +(_vprev)->vs_##type##next = (_vnext); \ +(_vnext)->vs_##type##prev = (_vprev); \ +} + +// vmem thread block count +uint64_t spl_vmem_threads_waiting = 0; + +// number of allocations > minalloc +uint64_t spl_bucket_non_pow2_allocs = 0; + +// allocator kstats +uint64_t spl_vmem_unconditional_allocs = 0; +uint64_t spl_vmem_unconditional_alloc_bytes = 0; +uint64_t spl_vmem_conditional_allocs = 0; +uint64_t spl_vmem_conditional_alloc_bytes = 0; +uint64_t spl_vmem_conditional_alloc_deny = 0; +uint64_t spl_vmem_conditional_alloc_deny_bytes = 0; + +// bucket allocator kstat +uint64_t spl_xat_success = 0; +uint64_t spl_xat_late_success = 0; +uint64_t spl_xat_late_success_nosleep = 0; +uint64_t spl_xat_pressured = 0; +uint64_t spl_xat_bailed = 0; +uint64_t spl_xat_bailed_contended = 0; +uint64_t spl_xat_lastalloc = 0; +uint64_t spl_xat_lastfree = 0; +uint64_t spl_xat_forced = 0; +uint64_t spl_xat_sleep = 0; +uint64_t spl_xat_late_deny = 0; +uint64_t spl_xat_no_waiters = 0; +uint64_t spl_xft_wait = 0; + +uint64_t spl_vba_parent_memory_appeared = 0; +uint64_t spl_vba_parent_memory_blocked = 0; +uint64_t spl_vba_hiprio_blocked = 0; +uint64_t spl_vba_cv_timeout = 0; +uint64_t spl_vba_loop_timeout = 0; +uint64_t spl_vba_cv_timeout_blocked = 0; +uint64_t spl_vba_loop_timeout_blocked = 0; +uint64_t spl_vba_sleep = 0; +uint64_t spl_vba_loop_entries = 0; + +// bucket minimum span size tunables +uint64_t spl_bucket_tunable_large_span = 0; +uint64_t spl_bucket_tunable_small_span = 0; + +// for XAT & XATB visibility into VBA queue +static _Atomic uint32_t spl_vba_threads[VMEM_BUCKETS] = { 0 }; +static uint32_t + vmem_bucket_id_to_bucket_number[NUMBER_OF_ARENAS_IN_VMEM_INIT] = { 0 }; +boolean_t spl_arc_no_grow(size_t, boolean_t, kmem_cache_t **); +_Atomic uint64_t spl_arc_no_grow_bits = 0; +uint64_t spl_arc_no_grow_count = 0; + +// compare span ages this many steps from the head of the freelist +uint64_t spl_frag_max_walk = 1000; +uint64_t spl_frag_walked_out = 0; +uint64_t spl_frag_walk_cnt = 0; + +extern void spl_free_set_emergency_pressure(int64_t p); +extern uint64_t segkmem_total_mem_allocated; +extern uint64_t total_memory; + +_Atomic uint64_t spl_lowest_alloc_stack_remaining = 0; + +/* + * Get a vmem_seg_t from the global segfree list. + */ +static vmem_seg_t * +vmem_getseg_global(void) +{ + vmem_seg_t *vsp; + + mutex_enter(&vmem_segfree_lock); + if ((vsp = vmem_segfree) != NULL) + vmem_segfree = vsp->vs_knext; + mutex_exit(&vmem_segfree_lock); + + if (vsp != NULL) + vsp->vs_span_createtime = 0; + + return (vsp); +} + +/* + * Put a vmem_seg_t on the global segfree list. + */ +static void +vmem_putseg_global(vmem_seg_t *vsp) +{ + mutex_enter(&vmem_segfree_lock); + vsp->vs_knext = vmem_segfree; + vmem_segfree = vsp; + mutex_exit(&vmem_segfree_lock); +} + +/* + * Get a vmem_seg_t from vmp's segfree list. + */ +static vmem_seg_t * +vmem_getseg(vmem_t *vmp) +{ + vmem_seg_t *vsp; + + ASSERT(vmp->vm_nsegfree > 0); + + vsp = vmp->vm_segfree; + vmp->vm_segfree = vsp->vs_knext; + vmp->vm_nsegfree--; + + return (vsp); +} + +/* + * Put a vmem_seg_t on vmp's segfree list. + */ +static void +vmem_putseg(vmem_t *vmp, vmem_seg_t *vsp) +{ + vsp->vs_knext = vmp->vm_segfree; + vmp->vm_segfree = vsp; + vmp->vm_nsegfree++; +} + + +/* + * Add vsp to the appropriate freelist, at the appropriate location, + * keeping the freelist sorted by age. + */ + +/* + * return true when we continue the for loop in + * vmem_freelist_insert_sort_by_time + */ +static inline bool +flist_sort_compare(bool newfirst, + const vmem_seg_t *vhead, + const vmem_seg_t *nextlist, + vmem_seg_t *p, vmem_seg_t *to_insert) +{ + /* + * vsp is the segment we are inserting into the freelist + * p is a freelist poniter or an element inside a non-empty freelist + * if we return false, then vsp is inserted immedaitely after p, + */ + + // always enter the for loop if we're at the front of a flist + if (p == vhead) + return (true); + + const vmem_seg_t *n = p->vs_knext; + + if (n == nextlist || n == NULL) { + // if we are at the tail of the flist, then + // insert vsp between p and n + return (false); + } + + if (n->vs_import == true && to_insert->vs_import == false) { + /* + * put non-imported segments before imported segments + * no matter what their respective create times are, + * thereby making imported segments more likely "age out" + */ + return (false); // inserts to_insert between p and n + } + + if (newfirst == true) { + if (n->vs_span_createtime < to_insert->vs_span_createtime) { + // n is older than me, so insert me between p and n + return (false); + } + } else { + if (n->vs_span_createtime > to_insert->vs_span_createtime) { + // n is newer than me, so insert me between p and n + return (false); + } + } + // continue iterating + return (true); +} + +static void +vmem_freelist_insert_sort_by_time(vmem_t *vmp, vmem_seg_t *vsp) +{ + ASSERT(vmp->vm_cflags & VMC_TIMEFREE); + ASSERT(vsp->vs_span_createtime > 0); + + const bool newfirst = 0 == (vmp->vm_cflags & VMC_OLDFIRST); + + const uint64_t abs_max_walk_steps = 1ULL << 30ULL; + uint32_t max_walk_steps = (uint32_t)MIN(spl_frag_max_walk, + abs_max_walk_steps); + + vmem_seg_t *vprev; + + ASSERT(*VMEM_HASH(vmp, vsp->vs_start) != vsp); + + /* + * in vmem_create_common() the freelists are arranged: + * freelist[0].vs_kprev = NULL, freelist[VMEM_FREELISTS].vs_knext = NULL + * freelist[1].vs_kprev = freelist[0], freelist[1].vs_knext = + * freelist[2] ... + * from vmem_freelist_insert(): + * VS_SIZE is the segment size (->vs_end - ->vs_start), so say 8k-512 + * highbit is the higest bit set PLUS 1, so in this case would be the + * 16k list. so below, vprev is therefore pointing to the 8k list + * in vmem_alloc_impl, the unconstrained allocation takes, for a 8k-512 + * block: vsp = flist[8k].vs_knext + * and calls vmem_seg_create() which sends any leftovers from vsp + * to vmem_freelist_insert + * + * vmem_freelist_insert would take the seg (as above, 8k-512 size), + * vprev points to the 16k list, and VMEM_INSERT(vprev, vsp, k) + * inserts the segment immediately after + * + * so vmem_seg_create(...8k-512...) pushes to the head of the 8k list, + * and vmem_alloc_impl(...8-512k...) will pull from the head of + * the 8k list + * + * below we may want to push to the TAIL of the 8k list, which is + * just before flist[16k]. + */ + + vprev = (vmem_seg_t *)&vmp->vm_freelist[highbit(VS_SIZE(vsp)) - 1]; + + int my_listnum = highbit(VS_SIZE(vsp)) - 1; + + ASSERT(my_listnum >= 1); + ASSERT(my_listnum < VMEM_FREELISTS); + + int next_listnum = my_listnum + 1; + + const vmem_seg_t *nextlist = + (vmem_seg_t *)&vmp->vm_freelist[next_listnum]; + + ASSERT(vsp->vs_span_createtime != 0); + if (vsp->vs_span_createtime == 0) { + TraceEvent(TRACE_WARNING, "SPL: %s: WARNING: " + "vsp->vs_span_createtime == 0 (%s)!\n", + __func__, vmp->vm_name); + } + + // continuing our example, starts with p at flist[8k] + // and n at the following freelist entry + + const vmem_seg_t *vhead = vprev; + vmem_seg_t *p = vprev; + vmem_seg_t *n = p->vs_knext; + + // walk from the freelist head looking for + // a segment whose creation time is earlier than + // the segment to be inserted's creation time, + // then insert before that segment. + + for (uint32_t step = 0; + flist_sort_compare(newfirst, vhead, nextlist, p, vsp) == true; + step++) { + // iterating while predecessor pointer p was created + // at a later tick than funcarg vsp. + // + // below we set p to n and update n. + ASSERT(n != NULL); + if (n == nextlist) { + dprintf("SPL: %s: at marker (%s)(steps: %u) " + "p->vs_start, end == %lu, %lu\n", + __func__, vmp->vm_name, step, + (uintptr_t)p->vs_start, (uintptr_t)p->vs_end); + // IOSleep(1); + // the next entry is the next marker (e.g. 16k marker) + break; + } + if (n->vs_start == 0) { + // from vmem_freelist_delete, this is a head + dprintf("SPL: %s: n->vs_start == 0 (%s)(steps: %u) " + "p->vs_start, end == %lu, %lu\n", + __func__, vmp->vm_name, step, + (uintptr_t)p->vs_start, (uintptr_t)p->vs_end); + // IOSleep(1); + break; + } + if (step >= max_walk_steps) { + ASSERT(nextlist->vs_kprev != NULL); + // we have walked far enough. + // put this segment at the tail of the freelist. + if (nextlist->vs_kprev != NULL) { + n = (vmem_seg_t *)nextlist; + p = nextlist->vs_kprev; + } + dprintf("SPL: %s: walked out (%s)\n", __func__, + vmp->vm_name); + // IOSleep(1); + atomic_inc_64(&spl_frag_walked_out); + break; + } + if (n->vs_knext == NULL) { + dprintf("SPL: %s: n->vs_knext == NULL (my_listnum " + "== %d)\n", __func__, my_listnum); + // IOSleep(1); + break; + } + p = n; + n = n->vs_knext; + atomic_inc_64(&spl_frag_walk_cnt); + } + + ASSERT(p != NULL); + + // insert segment between p and n + + vsp->vs_type = VMEM_FREE; + vmp->vm_freemap |= VS_SIZE(vprev); + VMEM_INSERT(p, vsp, k); + + cv_broadcast(&vmp->vm_cv); +} + +/* + * Add vsp to the appropriate freelist. + */ +static void +vmem_freelist_insert(vmem_t *vmp, vmem_seg_t *vsp) +{ + + if (vmp->vm_cflags & VMC_TIMEFREE) { + vmem_freelist_insert_sort_by_time(vmp, vsp); + return; + } + + vmem_seg_t *vprev; + + ASSERT(*VMEM_HASH(vmp, vsp->vs_start) != vsp); + + vprev = (vmem_seg_t *)&vmp->vm_freelist[highbit(VS_SIZE(vsp)) - 1]; + vsp->vs_type = VMEM_FREE; + vmp->vm_freemap |= VS_SIZE(vprev); + VMEM_INSERT(vprev, vsp, k); + + cv_broadcast(&vmp->vm_cv); +} + +/* + * Take vsp from the freelist. + */ +static void +vmem_freelist_delete(vmem_t *vmp, vmem_seg_t *vsp) +{ + ASSERT(*VMEM_HASH(vmp, vsp->vs_start) != vsp); + ASSERT(vsp->vs_type == VMEM_FREE); + + if (vsp->vs_knext->vs_start == 0 && vsp->vs_kprev->vs_start == 0) { + /* + * The segments on both sides of 'vsp' are freelist heads, + * so taking vsp leaves the freelist at vsp->vs_kprev empty. + */ + ASSERT(vmp->vm_freemap & VS_SIZE(vsp->vs_kprev)); + vmp->vm_freemap ^= VS_SIZE(vsp->vs_kprev); + } + VMEM_DELETE(vsp, k); +} + +/* + * Add vsp to the allocated-segment hash table and update kstats. + */ +static void +vmem_hash_insert(vmem_t *vmp, vmem_seg_t *vsp) +{ + vmem_seg_t **bucket; + + vsp->vs_type = VMEM_ALLOC; + bucket = VMEM_HASH(vmp, vsp->vs_start); + vsp->vs_knext = *bucket; + *bucket = vsp; + + if (vmem_seg_size == sizeof (vmem_seg_t)) { + // vsp->vs_depth = (uint8_t)getpcstack(vsp->vs_stack, + // VMEM_STACK_DEPTH); + // vsp->vs_thread = curthread; + vsp->vs_depth = 0; + vsp->vs_thread = 0; + vsp->vs_timestamp = gethrtime(); + } else { + vsp->vs_depth = 0; + } + + vmp->vm_kstat.vk_alloc.value.ui64++; + vmp->vm_kstat.vk_mem_inuse.value.ui64 += VS_SIZE(vsp); +} + +/* + * Remove vsp from the allocated-segment hash table and update kstats. + */ +static vmem_seg_t * +vmem_hash_delete(vmem_t *vmp, uintptr_t addr, size_t size) +{ + vmem_seg_t *vsp, **prev_vspp; + + prev_vspp = VMEM_HASH(vmp, addr); + while ((vsp = *prev_vspp) != NULL) { + if (vsp->vs_start == addr) { + *prev_vspp = vsp->vs_knext; + break; + } + vmp->vm_kstat.vk_lookup.value.ui64++; + prev_vspp = &vsp->vs_knext; + } + + if (vsp == NULL) + panic("vmem_hash_delete(%p, %lx, %lu): bad free " + "(name: %s, addr, size)", + (void *)vmp, addr, size, vmp->vm_name); + if (VS_SIZE(vsp) != size) + panic("vmem_hash_delete(%p, %lx, %lu): (%s) wrong size" + "(expect %lu)", + (void *)vmp, addr, size, vmp->vm_name, VS_SIZE(vsp)); + + vmp->vm_kstat.vk_free.value.ui64++; + vmp->vm_kstat.vk_mem_inuse.value.ui64 -= size; + + return (vsp); +} + +/* + * Create a segment spanning the range [start, end) and add it to the arena. + */ +static vmem_seg_t * +vmem_seg_create(vmem_t *vmp, vmem_seg_t *vprev, uintptr_t start, uintptr_t end) +{ + vmem_seg_t *newseg = vmem_getseg(vmp); + + newseg->vs_start = start; + newseg->vs_end = end; + newseg->vs_type = 0; + newseg->vs_import = 0; + newseg->vs_span_createtime = 0; + + VMEM_INSERT(vprev, newseg, a); + + return (newseg); +} + +/* + * Remove segment vsp from the arena. + */ +static void +vmem_seg_destroy(vmem_t *vmp, vmem_seg_t *vsp) +{ + ASSERT(vsp->vs_type != VMEM_ROTOR); + VMEM_DELETE(vsp, a); + + vmem_putseg(vmp, vsp); +} + +/* + * Add the span [vaddr, vaddr + size) to vmp and update kstats. + */ +static vmem_seg_t * +vmem_span_create(vmem_t *vmp, void *vaddr, size_t size, uint8_t import) +{ + vmem_seg_t *newseg, *span; + uintptr_t start = (uintptr_t)vaddr; + uintptr_t end = start + size; + + ASSERT(MUTEX_HELD(&vmp->vm_lock)); + + /* + * AllocatePoolWithTag does not handle alignment: + * no windows equivalent? + */ + + if ((start | end) & (vmp->vm_quantum - 1)) + panic("vmem_span_create(%p, %p, %lu): misaligned (%s)", + (void *)vmp, vaddr, size, vmp->vm_name); + + span = vmem_seg_create(vmp, vmp->vm_seg0.vs_aprev, start, end); + span->vs_type = VMEM_SPAN; + span->vs_import = import; + + hrtime_t t = 0; + if (vmp->vm_cflags & VMC_TIMEFREE) { + t = gethrtime(); + } + span->vs_span_createtime = t; + + VMEM_INSERT(vmp->vm_seg0.vs_kprev, span, k); + + newseg = vmem_seg_create(vmp, span, start, end); + newseg->vs_span_createtime = t; + + vmem_freelist_insert(vmp, newseg); + + if (import) + vmp->vm_kstat.vk_mem_import.value.ui64 += size; + vmp->vm_kstat.vk_mem_total.value.ui64 += size; + + return (newseg); +} + +/* + * Remove span vsp from vmp and update kstats. + */ +static void +vmem_span_destroy(vmem_t *vmp, vmem_seg_t *vsp) +{ + vmem_seg_t *span = vsp->vs_aprev; + size_t size = VS_SIZE(vsp); + + ASSERT(MUTEX_HELD(&vmp->vm_lock)); + ASSERT(span->vs_type == VMEM_SPAN); + + if (span->vs_import) + vmp->vm_kstat.vk_mem_import.value.ui64 -= size; + vmp->vm_kstat.vk_mem_total.value.ui64 -= size; + + VMEM_DELETE(span, k); + + vmem_seg_destroy(vmp, vsp); + vmem_seg_destroy(vmp, span); +} + +/* + * Allocate the subrange [addr, addr + size) from segment vsp. + * If there are leftovers on either side, place them on the freelist. + * Returns a pointer to the segment representing [addr, addr + size). + */ +static vmem_seg_t * +vmem_seg_alloc(vmem_t *vmp, vmem_seg_t *vsp, uintptr_t addr, size_t size) +{ + uintptr_t vs_start = vsp->vs_start; + uintptr_t vs_end = vsp->vs_end; + size_t vs_size = vs_end - vs_start; + size_t realsize = P2ROUNDUP(size, vmp->vm_quantum); + uintptr_t addr_end = addr + realsize; + + ASSERT(P2PHASE(vs_start, vmp->vm_quantum) == 0); + ASSERT(P2PHASE(addr, vmp->vm_quantum) == 0); + ASSERT(vsp->vs_type == VMEM_FREE); + ASSERT(addr >= vs_start && addr_end - 1 <= vs_end - 1); + ASSERT(addr - 1 <= addr_end - 1); + + hrtime_t parent_seg_span_createtime = vsp->vs_span_createtime; + + /* + * If we're allocating from the start of the segment, and the + * remainder will be on the same freelist, we can save quite + * a bit of work. + */ + if (P2SAMEHIGHBIT(vs_size, vs_size - realsize) && addr == vs_start) { + ASSERT(highbit(vs_size) == highbit(vs_size - realsize)); + vsp->vs_start = addr_end; + vsp = vmem_seg_create(vmp, vsp->vs_aprev, addr, addr + size); + vsp->vs_span_createtime = parent_seg_span_createtime; + vmem_hash_insert(vmp, vsp); + return (vsp); + } + + vmem_freelist_delete(vmp, vsp); + + if (vs_end != addr_end) { + vmem_seg_t *v = vmem_seg_create(vmp, vsp, addr_end, vs_end); + v->vs_span_createtime = parent_seg_span_createtime; + vmem_freelist_insert(vmp, v); + } + + if (vs_start != addr) { + vmem_seg_t *v = + vmem_seg_create(vmp, vsp->vs_aprev, vs_start, addr); + v->vs_span_createtime = parent_seg_span_createtime; + vmem_freelist_insert(vmp, v); + } + + vsp->vs_start = addr; + vsp->vs_end = addr + size; + + vsp->vs_span_createtime = parent_seg_span_createtime; + + vmem_hash_insert(vmp, vsp); + return (vsp); +} + +/* + * Returns 1 if we are populating, 0 otherwise. + * Call it if we want to prevent recursion from HAT. + */ +int +vmem_is_populator() +{ + return (mutex_owner(&vmem_sleep_lock) == curthread || + mutex_owner(&vmem_nosleep_lock) == curthread || + mutex_owner(&vmem_pushpage_lock) == curthread || + mutex_owner(&vmem_panic_lock) == curthread); +} + +/* + * Populate vmp's segfree list with VMEM_MINFREE vmem_seg_t structures. + */ +static int +vmem_populate(vmem_t *vmp, int vmflag) +{ + char *p; + vmem_seg_t *vsp; + ssize_t nseg; + size_t size; + kmutex_t *lp; + int i; + + while (vmp->vm_nsegfree < VMEM_MINFREE && + (vsp = vmem_getseg_global()) != NULL) + vmem_putseg(vmp, vsp); + + if (vmp->vm_nsegfree >= VMEM_MINFREE) + return (1); + + /* + * If we're already populating, tap the reserve. + */ + if (vmem_is_populator()) { + ASSERT(vmp->vm_cflags & VMC_POPULATOR); + return (1); + } + + mutex_exit(&vmp->vm_lock); + + // if (panic_thread == curthread) + // lp = &vmem_panic_lock; + // else + + if (vmflag & VM_NOSLEEP) + lp = &vmem_nosleep_lock; + else if (vmflag & VM_PUSHPAGE) + lp = &vmem_pushpage_lock; + else + lp = &vmem_sleep_lock; + + mutex_enter(lp); + + nseg = VMEM_MINFREE + vmem_populators * VMEM_POPULATE_RESERVE; + size = P2ROUNDUP(nseg * vmem_seg_size, vmem_seg_arena->vm_quantum); + nseg = size / vmem_seg_size; + + /* + * The following vmem_alloc_impl() may need to populate vmem_seg_arena + * and all the things it imports from. When doing so, it will tap + * each arena's reserve to prevent recursion (see the block comment + * above the definition of VMEM_POPULATE_RESERVE). + */ + p = vmem_alloc_impl(vmem_seg_arena, size, vmflag & VM_KMFLAGS); + if (p == NULL) { + mutex_exit(lp); + mutex_enter(&vmp->vm_lock); + vmp->vm_kstat.vk_populate_fail.value.ui64++; + return (0); + } + + /* + * Restock the arenas that may have been depleted during population. + */ + for (i = 0; i < vmem_populators; i++) { + mutex_enter(&vmem_populator[i]->vm_lock); + while (vmem_populator[i]->vm_nsegfree < VMEM_POPULATE_RESERVE) + vmem_putseg(vmem_populator[i], + (vmem_seg_t *)(p + --nseg * vmem_seg_size)); + mutex_exit(&vmem_populator[i]->vm_lock); + } + + mutex_exit(lp); + mutex_enter(&vmp->vm_lock); + + /* + * Now take our own segments. + */ + ASSERT(nseg >= VMEM_MINFREE); + while (vmp->vm_nsegfree < VMEM_MINFREE) + vmem_putseg(vmp, (vmem_seg_t *)(p + --nseg * vmem_seg_size)); + + /* + * Give the remainder to charity. + */ + while (nseg > 0) + vmem_putseg_global((vmem_seg_t *)(p + --nseg * vmem_seg_size)); + + return (1); +} + +/* + * Advance a walker from its previous position to 'afterme'. + * Note: may drop and reacquire vmp->vm_lock. + */ +static void +vmem_advance(vmem_t *vmp, vmem_seg_t *walker, vmem_seg_t *afterme) +{ + vmem_seg_t *vprev = walker->vs_aprev; + vmem_seg_t *vnext = walker->vs_anext; + vmem_seg_t *vsp = NULL; + + VMEM_DELETE(walker, a); + + if (afterme != NULL) + VMEM_INSERT(afterme, walker, a); + + /* + * The walker segment's presence may have prevented its neighbors + * from coalescing. If so, coalesce them now. + */ + if (vprev->vs_type == VMEM_FREE) { + if (vnext->vs_type == VMEM_FREE) { + ASSERT(vprev->vs_end == vnext->vs_start); + ASSERT(vprev->vs_span_createtime == + vnext->vs_span_createtime); + vmem_freelist_delete(vmp, vnext); + vmem_freelist_delete(vmp, vprev); + vprev->vs_end = vnext->vs_end; + vmem_freelist_insert(vmp, vprev); + vmem_seg_destroy(vmp, vnext); + } + vsp = vprev; + } else if (vnext->vs_type == VMEM_FREE) { + vsp = vnext; + } + + /* + * vsp could represent a complete imported span, + * in which case we must return it to the source. + */ + if (vsp != NULL && vsp->vs_aprev->vs_import && + vmp->vm_source_free != NULL && + vsp->vs_aprev->vs_type == VMEM_SPAN && + vsp->vs_anext->vs_type == VMEM_SPAN) { + void *vaddr = (void *)vsp->vs_start; + size_t size = VS_SIZE(vsp); + ASSERT(size == VS_SIZE(vsp->vs_aprev)); + vmem_freelist_delete(vmp, vsp); + vmem_span_destroy(vmp, vsp); + vmp->vm_kstat.vk_parent_free.value.ui64++; + mutex_exit(&vmp->vm_lock); + vmp->vm_source_free(vmp->vm_source, vaddr, size); + mutex_enter(&vmp->vm_lock); + } +} + +/* + * VM_NEXTFIT allocations deliberately cycle through all virtual addresses + * in an arena, so that we avoid reusing addresses for as long as possible. + * This helps to catch used-after-freed bugs. It's also the perfect policy + * for allocating things like process IDs, where we want to cycle through + * all values in order. + */ +static void * +vmem_nextfit_alloc(vmem_t *vmp, size_t size, int vmflag) +{ + vmem_seg_t *vsp, *rotor; + uintptr_t addr; + size_t realsize = P2ROUNDUP(size, vmp->vm_quantum); + size_t vs_size; + + mutex_enter(&vmp->vm_lock); + + if (vmp->vm_nsegfree < VMEM_MINFREE && !vmem_populate(vmp, vmflag)) { + mutex_exit(&vmp->vm_lock); + return (NULL); + } + + /* + * The common case is that the segment right after the rotor is free, + * and large enough that extracting 'size' bytes won't change which + * freelist it's on. In this case we can avoid a *lot* of work. + * Instead of the normal vmem_seg_alloc(), we just advance the start + * address of the victim segment. Instead of moving the rotor, we + * create the new segment structure *behind the rotor*, which has + * the same effect. And finally, we know we don't have to coalesce + * the rotor's neighbors because the new segment lies between them. + */ + rotor = &vmp->vm_rotor; + vsp = rotor->vs_anext; + if (vsp->vs_type == VMEM_FREE && (vs_size = VS_SIZE(vsp)) > realsize && + P2SAMEHIGHBIT(vs_size, vs_size - realsize)) { + ASSERT(highbit(vs_size) == highbit(vs_size - realsize)); + addr = vsp->vs_start; + vsp->vs_start = addr + realsize; + hrtime_t t = vsp->vs_span_createtime; + vmem_hash_insert(vmp, + vmem_seg_create(vmp, rotor->vs_aprev, addr, addr + size)); + vsp->vs_span_createtime = t; + mutex_exit(&vmp->vm_lock); + return ((void *)addr); + } + + /* + * Starting at the rotor, look for a segment large enough to + * satisfy the allocation. + */ + for (;;) { + atomic_inc_64(&vmp->vm_kstat.vk_search.value.ui64); + if (vsp->vs_type == VMEM_FREE && VS_SIZE(vsp) >= size) + break; + vsp = vsp->vs_anext; + if (vsp == rotor) { + /* + * We've come full circle. One possibility is that the + * there's actually enough space, but the rotor itself + * is preventing the allocation from succeeding because + * it's sitting between two free segments. Therefore, + * we advance the rotor and see if that liberates a + * suitable segment. + */ + vmem_advance(vmp, rotor, rotor->vs_anext); + vsp = rotor->vs_aprev; + if (vsp->vs_type == VMEM_FREE && VS_SIZE(vsp) >= size) + break; + /* + * If there's a lower arena we can import from, or it's + * a VM_NOSLEEP allocation, let vmem_xalloc() handle it. + * Otherwise, wait until another thread frees something. + */ + if (vmp->vm_source_alloc != NULL || + (vmflag & VM_NOSLEEP)) { + mutex_exit(&vmp->vm_lock); + return (vmem_xalloc(vmp, size, vmp->vm_quantum, + 0, 0, NULL, NULL, + vmflag & (VM_KMFLAGS | VM_NEXTFIT))); + } + atomic_inc_64(&vmp->vm_kstat.vk_wait.value.ui64); + atomic_inc_64( + &vmp->vm_kstat.vk_threads_waiting.value.ui64); + atomic_inc_64(&spl_vmem_threads_waiting); + if (spl_vmem_threads_waiting > 1) + dprintf("SPL: %s: waiting for %lu sized alloc " + "after full circle of %s, waiting " + "threads %llu, total threads waiting " + "= %llu.\n", + __func__, size, vmp->vm_name, + vmp->vm_kstat.vk_threads_waiting.value.ui64, + spl_vmem_threads_waiting); + cv_wait(&vmp->vm_cv, &vmp->vm_lock); + atomic_dec_64(&spl_vmem_threads_waiting); + atomic_dec_64( + &vmp->vm_kstat.vk_threads_waiting.value.ui64); + vsp = rotor->vs_anext; + } + } + + /* + * We found a segment. Extract enough space to satisfy the allocation. + */ + addr = vsp->vs_start; + vsp = vmem_seg_alloc(vmp, vsp, addr, size); + ASSERT(vsp->vs_type == VMEM_ALLOC && + vsp->vs_start == addr && vsp->vs_end == addr + size); + + /* + * Advance the rotor to right after the newly-allocated segment. + * That's where the next VM_NEXTFIT allocation will begin searching. + */ + vmem_advance(vmp, rotor, vsp); + mutex_exit(&vmp->vm_lock); + return ((void *)addr); +} + +/* + * Checks if vmp is guaranteed to have a size-byte buffer somewhere on its + * freelist. If size is not a power-of-2, it can return a false-negative. + * + * Used to decide if a newly imported span is superfluous after re-acquiring + * the arena lock. + */ +static int +vmem_canalloc(vmem_t *vmp, size_t size) +{ + int hb; + int flist = 0; + ASSERT(MUTEX_HELD(&vmp->vm_lock)); + + if ((size & (size - 1)) == 0) + flist = lowbit(P2ALIGN(vmp->vm_freemap, size)); + else if ((hb = highbit(size)) < VMEM_FREELISTS) + flist = lowbit(P2ALIGN(vmp->vm_freemap, 1ULL << hb)); + + return (flist); +} + +// Convenience functions for use when gauging +// allocation ability when not holding the lock. +// These are unreliable because vmp->vm_freemap is +// liable to change immediately after being examined. +int +vmem_canalloc_lock(vmem_t *vmp, size_t size) +{ + mutex_enter(&vmp->vm_lock); + int i = vmem_canalloc(vmp, size); + mutex_exit(&vmp->vm_lock); + return (i); +} + +int +vmem_canalloc_atomic(vmem_t *vmp, size_t size) +{ + int hb; + int flist = 0; + + ulong_t freemap = + __c11_atomic_load((_Atomic ulong_t *)&vmp->vm_freemap, + __ATOMIC_SEQ_CST); + + if (ISP2(size)) + flist = lowbit(P2ALIGN(freemap, size)); + else if ((hb = highbit(size)) < VMEM_FREELISTS) + flist = lowbit(P2ALIGN(freemap, 1ULL << hb)); + + return (flist); +} + +static inline uint64_t +spl_vmem_xnu_useful_bytes_free(void) +{ + extern _Atomic uint32_t spl_vm_pages_reclaimed; + extern _Atomic uint32_t spl_vm_pages_wanted; + extern _Atomic uint32_t spl_vm_pressure_level; + + if (spl_vm_pages_wanted > 0) + return (PAGE_SIZE * spl_vm_pages_reclaimed); + + /* + * beware of large magic guard values, + * the pressure enum only goes to 4 + */ + if (spl_vm_pressure_level > 0 && + spl_vm_pressure_level < 100) + return (0); + + return (total_memory - segkmem_total_mem_allocated); +} + +uint64_t +vmem_xnu_useful_bytes_free(void) +{ + return (spl_vmem_xnu_useful_bytes_free()); +} + + +static void * +spl_vmem_malloc_unconditionally_unlocked(size_t size) +{ + extern void *osif_malloc(uint64_t); + atomic_inc_64(&spl_vmem_unconditional_allocs); + atomic_add_64(&spl_vmem_unconditional_alloc_bytes, size); + return (osif_malloc(size)); +} + +static void * +spl_vmem_malloc_unconditionally(size_t size) +{ + mutex_enter(&vmem_xnu_alloc_lock); + void *m = spl_vmem_malloc_unconditionally_unlocked(size); + mutex_exit(&vmem_xnu_alloc_lock); + return (m); +} + +static void * +spl_vmem_malloc_if_no_pressure(size_t size) +{ + // The mutex serializes concurrent callers, providing time for + // the variables in spl_vmem_xnu_useful_bytes_free() to be updated. + mutex_enter(&vmem_xnu_alloc_lock); + if (spl_vmem_xnu_useful_bytes_free() > (MAX(size, 1024ULL*1024ULL))) { + extern void *osif_malloc(uint64_t); + void *p = osif_malloc(size); + if (p != NULL) { + spl_vmem_conditional_allocs++; + spl_vmem_conditional_alloc_bytes += size; + } + mutex_exit(&vmem_xnu_alloc_lock); + return (p); + } else { + spl_vmem_conditional_alloc_deny++; + spl_vmem_conditional_alloc_deny_bytes += size; + mutex_exit(&vmem_xnu_alloc_lock); + return (NULL); + } +} + +/* + * Allocate size bytes at offset phase from an align boundary such that the + * resulting segment [addr, addr + size) is a subset of [minaddr, maxaddr) + * that does not straddle a nocross-aligned boundary. + */ +void * +vmem_xalloc(vmem_t *vmp, size_t size, size_t align_arg, size_t phase, + size_t nocross, void *minaddr, void *maxaddr, int vmflag) +{ + vmem_seg_t *vsp; + vmem_seg_t *vbest = NULL; + uintptr_t addr, taddr, start, end; + uintptr_t align = (align_arg != 0) ? align_arg : vmp->vm_quantum; + void *vaddr, *xvaddr = NULL; + size_t xsize; + int hb, flist, resv; + uint32_t mtbf; + + if ((align | phase | nocross) & (vmp->vm_quantum - 1)) + panic("vmem_xalloc(%p, %lu, %lu, %lu, %lu, %p, %p, %x): " + "parameters not vm_quantum aligned", + (void *)vmp, size, align_arg, phase, nocross, + minaddr, maxaddr, vmflag); + + if (nocross != 0 && + (align > nocross || P2ROUNDUP(phase + size, align) > nocross)) + panic("vmem_xalloc(%p, %lu, %lu, %lu, %lu, %p, %p, %x): " + "overconstrained allocation", + (void *)vmp, size, align_arg, phase, nocross, + minaddr, maxaddr, vmflag); + + if (phase >= align || (align & (align - 1)) != 0 || + (nocross & (nocross - 1)) != 0) + panic("vmem_xalloc(%p, %lu, %lu, %lu, %lu, %p, %p, %x): " + "parameters inconsistent or invalid", + (void *)vmp, size, align_arg, phase, nocross, + minaddr, maxaddr, vmflag); + + if ((mtbf = vmem_mtbf | vmp->vm_mtbf) != 0 && gethrtime() % mtbf == 0 && + (vmflag & (VM_NOSLEEP | VM_PANIC)) == VM_NOSLEEP) + return (NULL); + + mutex_enter(&vmp->vm_lock); + for (;;) { + if (vmp->vm_nsegfree < VMEM_MINFREE && + !vmem_populate(vmp, vmflag)) + break; +do_alloc: + /* + * highbit() returns the highest bit + 1, which is exactly + * what we want: we want to search the first freelist whose + * members are *definitely* large enough to satisfy our + * allocation. However, there are certain cases in which we + * want to look at the next-smallest freelist (which *might* + * be able to satisfy the allocation): + * + * (1) The size is exactly a power of 2, in which case + * the smaller freelist is always big enough; + * + * (2) All other freelists are empty; + * + * (3) We're in the highest possible freelist, which is + * always empty (e.g. the 4GB freelist on 32-bit systems); + * + * (4) We're doing a best-fit or first-fit allocation. + */ + if ((size & (size - 1)) == 0) { + flist = lowbit(P2ALIGN(vmp->vm_freemap, size)); + } else { + hb = highbit(size); + if ((vmp->vm_freemap >> hb) == 0 || + hb == VMEM_FREELISTS || + (vmflag & (VM_BESTFIT | VM_FIRSTFIT))) + hb--; + flist = lowbit(P2ALIGN(vmp->vm_freemap, 1UL << hb)); + } + + for (vbest = NULL, vsp = (flist == 0) ? NULL : + vmp->vm_freelist[flist - 1].vs_knext; + vsp != NULL; vsp = vsp->vs_knext) { + atomic_inc_64(&vmp->vm_kstat.vk_search.value.ui64); + if (vsp->vs_start == 0) { + /* + * We're moving up to a larger freelist, + * so if we've already found a candidate, + * the fit can't possibly get any better. + */ + if (vbest != NULL) + break; + /* + * Find the next non-empty freelist. + */ + flist = lowbit(P2ALIGN(vmp->vm_freemap, + VS_SIZE(vsp))); + if (flist-- == 0) + break; + vsp = (vmem_seg_t *)&vmp->vm_freelist[flist]; + ASSERT(vsp->vs_knext->vs_type == VMEM_FREE); + continue; + } + if (vsp->vs_end - 1 < (uintptr_t)minaddr) + continue; + if (vsp->vs_start > (uintptr_t)maxaddr - 1) + continue; + start = MAX(vsp->vs_start, (uintptr_t)minaddr); + end = MIN(vsp->vs_end - 1, (uintptr_t)maxaddr - 1) + 1; + taddr = P2PHASEUP(start, align, phase); + if (P2BOUNDARY(taddr, size, nocross)) + taddr += + P2ROUNDUP(P2NPHASE(taddr, nocross), align); + if ((taddr - start) + size > end - start || + (vbest != NULL && VS_SIZE(vsp) >= VS_SIZE(vbest))) + continue; + vbest = vsp; + addr = taddr; + if (!(vmflag & VM_BESTFIT) || VS_SIZE(vbest) == size) + break; + } + if (vbest != NULL) + break; + ASSERT(xvaddr == NULL); + if (size == 0) + panic("vmem_xalloc(): size == 0"); + if (vmp->vm_source_alloc != NULL && nocross == 0 && + minaddr == NULL && maxaddr == NULL) { + size_t aneeded, asize; + size_t aquantum = MAX(vmp->vm_quantum, + vmp->vm_source->vm_quantum); + size_t aphase = phase; + if ((align > aquantum) && + !(vmp->vm_cflags & VMC_XALIGN)) { + aphase = (P2PHASE(phase, aquantum) != 0) ? + align - vmp->vm_quantum : align - aquantum; + ASSERT(aphase >= phase); + } + aneeded = MAX(size + aphase, vmp->vm_min_import); + asize = P2ROUNDUP(aneeded, aquantum); + + if (asize < size) { + /* + * The rounding induced overflow; return NULL + * if we are permitted to fail the allocation + * (and explicitly panic if we aren't). + */ + if ((vmflag & VM_NOSLEEP) && + !(vmflag & VM_PANIC)) { + mutex_exit(&vmp->vm_lock); + return (NULL); + } + + panic("vmem_xalloc(): size overflow"); + } + + /* + * Determine how many segment structures we'll consume. + * The calculation must be precise because if we're + * here on behalf of vmem_populate(), we are taking + * segments from a very limited reserve. + */ + if (size == asize && !(vmp->vm_cflags & VMC_XALLOC)) + resv = VMEM_SEGS_PER_SPAN_CREATE + + VMEM_SEGS_PER_EXACT_ALLOC; + else if (phase == 0 && + align <= vmp->vm_source->vm_quantum) + resv = VMEM_SEGS_PER_SPAN_CREATE + + VMEM_SEGS_PER_LEFT_ALLOC; + else + resv = VMEM_SEGS_PER_ALLOC_MAX; + + ASSERT(vmp->vm_nsegfree >= resv); + vmp->vm_nsegfree -= resv; /* reserve our segs */ + mutex_exit(&vmp->vm_lock); + if (vmp->vm_cflags & VMC_XALLOC) { + size_t oasize = asize; + vaddr = ((vmem_ximport_t *) + vmp->vm_source_alloc)(vmp->vm_source, + &asize, align, vmflag & VM_KMFLAGS); + ASSERT(asize >= oasize); + ASSERT(P2PHASE(asize, + vmp->vm_source->vm_quantum) == 0); + ASSERT(!(vmp->vm_cflags & VMC_XALIGN) || + IS_P2ALIGNED(vaddr, align)); + } else { + atomic_inc_64( + &vmp->vm_kstat.vk_parent_alloc.value.ui64); + vaddr = vmp->vm_source_alloc(vmp->vm_source, + asize, vmflag & (VM_KMFLAGS | VM_NEXTFIT)); + } + mutex_enter(&vmp->vm_lock); + vmp->vm_nsegfree += resv; /* claim reservation */ + aneeded = size + align - vmp->vm_quantum; + aneeded = P2ROUNDUP(aneeded, vmp->vm_quantum); + if (vaddr != NULL) { + /* + * Since we dropped the vmem lock while + * calling the import function, other + * threads could have imported space + * and made our import unnecessary. In + * order to save space, we return + * excess imports immediately. + */ + // but if there are threads waiting below, + // do not return the excess import, rather + // wake those threads up so they can use it. + if (asize > aneeded && + vmp->vm_source_free != NULL && + vmp->vm_kstat.vk_threads_waiting.value.ui64 + == 0 && vmem_canalloc(vmp, aneeded)) { + ASSERT(resv >= + VMEM_SEGS_PER_MIDDLE_ALLOC); + xvaddr = vaddr; + xsize = asize; + goto do_alloc; + } else if ( + vmp->vm_kstat.vk_threads_waiting.value.ui64 + > 0) { + vmp->vm_kstat.vk_excess.value.ui64++; + cv_broadcast(&vmp->vm_cv); + } + vbest = vmem_span_create(vmp, vaddr, asize, 1); + addr = P2PHASEUP(vbest->vs_start, align, phase); + break; + } else if (vmem_canalloc(vmp, aneeded)) { + /* + * Our import failed, but another thread + * added sufficient free memory to the arena + * to satisfy our request. Go back and + * grab it. + */ + ASSERT(resv >= VMEM_SEGS_PER_MIDDLE_ALLOC); + goto do_alloc; + } + } + + /* + * If the requestor chooses to fail the allocation attempt + * rather than reap wait and retry - get out of the loop. + */ + if (vmflag & VM_ABORT) + break; + mutex_exit(&vmp->vm_lock); + +#if 0 + if (vmp->vm_cflags & VMC_IDENTIFIER) + kmem_reap_idspace(); + else + kmem_reap(); +#endif + + mutex_enter(&vmp->vm_lock); + if (vmflag & VM_NOSLEEP) + break; + atomic_inc_64(&vmp->vm_kstat.vk_wait.value.ui64); + atomic_inc_64(&vmp->vm_kstat.vk_threads_waiting.value.ui64); + atomic_inc_64(&spl_vmem_threads_waiting); + if (spl_vmem_threads_waiting > 0) { + dprintf("SPL: %s: vmem waiting for %lu sized alloc " + "for %s, waiting threads %llu, total threads " + "waiting = %llu\n", + __func__, size, vmp->vm_name, + vmp->vm_kstat.vk_threads_waiting.value.ui64, + spl_vmem_threads_waiting); + extern int64_t spl_free_set_and_wait_pressure(int64_t, + boolean_t, clock_t); + extern int64_t spl_free_manual_pressure_wrapper(void); + mutex_exit(&vmp->vm_lock); + // release other waiting threads + spl_free_set_pressure(0); + int64_t target_pressure = size * + spl_vmem_threads_waiting; + int64_t delivered_pressure = + spl_free_set_and_wait_pressure(target_pressure, + TRUE, USEC2NSEC(500)); + dprintf("SPL: %s: pressure %lld targeted, %lld " + "delivered\n", __func__, target_pressure, + delivered_pressure); + mutex_enter(&vmp->vm_lock); + } + cv_wait(&vmp->vm_cv, &vmp->vm_lock); + atomic_dec_64(&spl_vmem_threads_waiting); + atomic_dec_64(&vmp->vm_kstat.vk_threads_waiting.value.ui64); + } + if (vbest != NULL) { + ASSERT(vbest->vs_type == VMEM_FREE); + ASSERT(vbest->vs_knext != vbest); + /* re-position to end of buffer */ + if (vmflag & VM_ENDALLOC) { + addr += ((vbest->vs_end - (addr + size)) / align) * + align; + } + (void) vmem_seg_alloc(vmp, vbest, addr, size); + mutex_exit(&vmp->vm_lock); + if (xvaddr) { + atomic_inc_64(&vmp->vm_kstat.vk_parent_free.value.ui64); + vmp->vm_source_free(vmp->vm_source, xvaddr, xsize); + } + ASSERT(P2PHASE(addr, align) == phase); + ASSERT(!P2BOUNDARY(addr, size, nocross)); + ASSERT(addr >= (uintptr_t)minaddr); + ASSERT(addr + size - 1 <= (uintptr_t)maxaddr - 1); + return ((void *)addr); + } + if (0 == (vmflag & VM_NO_VBA)) { + vmp->vm_kstat.vk_fail.value.ui64++; + } + mutex_exit(&vmp->vm_lock); + if (vmflag & VM_PANIC) + panic("vmem_xalloc(%p, %lu, %lu, %lu, %lu, %p, %p, %x): " + "cannot satisfy mandatory allocation", + (void *)vmp, size, align_arg, phase, nocross, + minaddr, maxaddr, vmflag); + ASSERT(xvaddr == NULL); + return (NULL); +} + +/* + * Free the segment [vaddr, vaddr + size), where vaddr was a constrained + * allocation. vmem_xalloc() and vmem_xfree() must always be paired because + * both routines bypass the quantum caches. + */ +void +vmem_xfree(vmem_t *vmp, void *vaddr, size_t size) +{ + vmem_seg_t *vsp, *vnext, *vprev; + + mutex_enter(&vmp->vm_lock); + + vsp = vmem_hash_delete(vmp, (uintptr_t)vaddr, size); + vsp->vs_end = P2ROUNDUP(vsp->vs_end, vmp->vm_quantum); + + /* + * Attempt to coalesce with the next segment. + */ + vnext = vsp->vs_anext; + if (vnext->vs_type == VMEM_FREE) { + ASSERT(vsp->vs_end == vnext->vs_start); + vmem_freelist_delete(vmp, vnext); + vsp->vs_end = vnext->vs_end; + vmem_seg_destroy(vmp, vnext); + } + + /* + * Attempt to coalesce with the previous segment. + */ + vprev = vsp->vs_aprev; + if (vprev->vs_type == VMEM_FREE) { + ASSERT(vprev->vs_end == vsp->vs_start); + vmem_freelist_delete(vmp, vprev); + vprev->vs_end = vsp->vs_end; + vmem_seg_destroy(vmp, vsp); + vsp = vprev; + } + + /* + * If the entire span is free, return it to the source. + */ + if (vsp->vs_aprev->vs_import && vmp->vm_source_free != NULL && + vsp->vs_aprev->vs_type == VMEM_SPAN && + vsp->vs_anext->vs_type == VMEM_SPAN) { + vaddr = (void *)vsp->vs_start; + size = VS_SIZE(vsp); + ASSERT(size == VS_SIZE(vsp->vs_aprev)); + vmem_span_destroy(vmp, vsp); + vmp->vm_kstat.vk_parent_free.value.ui64++; + mutex_exit(&vmp->vm_lock); + vmp->vm_source_free(vmp->vm_source, vaddr, size); + } else { + vmem_freelist_insert(vmp, vsp); + mutex_exit(&vmp->vm_lock); + } +} + +/* + * vmem_alloc_impl() and auxiliary functions : + * + * Allocate size bytes from arena vmp. Returns the allocated address + * on success, NULL on failure. vmflag specifies VM_SLEEP or VM_NOSLEEP, + * and may also specify best-fit, first-fit, or next-fit allocation policy + * instead of the default instant-fit policy. VM_SLEEP allocations are + * guaranteed to succeed. + */ +void * +vmem_alloc_impl(vmem_t *vmp, size_t size, int vmflag) +{ + vmem_seg_t *vsp; + uintptr_t addr; + int hb; + int flist = 0; + uint32_t mtbf; + + const ULONG_PTR r = IoGetRemainingStackSize(); + + if (spl_lowest_alloc_stack_remaining == 0) { + spl_lowest_alloc_stack_remaining = r; + } else if (spl_lowest_alloc_stack_remaining > r) { + spl_lowest_alloc_stack_remaining = r; + } + + if (size - 1 < vmp->vm_qcache_max) + return (kmem_cache_alloc(vmp->vm_qcache[(size - 1) >> + vmp->vm_qshift], vmflag & VM_KMFLAGS)); + + if ((mtbf = vmem_mtbf | vmp->vm_mtbf) != 0 && gethrtime() % mtbf == 0 && + (vmflag & (VM_NOSLEEP | VM_PANIC)) == VM_NOSLEEP) + return (NULL); + + if (vmflag & VM_NEXTFIT) + return (vmem_nextfit_alloc(vmp, size, vmflag)); + + if (vmflag & (VM_BESTFIT | VM_FIRSTFIT)) + return (vmem_xalloc(vmp, size, vmp->vm_quantum, 0, 0, + NULL, NULL, vmflag)); + if (vmp->vm_cflags & VM_NEXTFIT) + return (vmem_nextfit_alloc(vmp, size, vmflag)); + + /* + * Unconstrained instant-fit allocation from the segment list. + */ + mutex_enter(&vmp->vm_lock); + + if (vmp->vm_nsegfree >= VMEM_MINFREE || vmem_populate(vmp, vmflag)) { + if ((size & (size - 1)) == 0) + flist = lowbit(P2ALIGN(vmp->vm_freemap, size)); + else if ((hb = highbit(size)) < VMEM_FREELISTS) + flist = lowbit(P2ALIGN(vmp->vm_freemap, 1UL << hb)); + } + + if (flist-- == 0) { + mutex_exit(&vmp->vm_lock); + return (vmem_xalloc(vmp, size, vmp->vm_quantum, + 0, 0, NULL, NULL, vmflag)); + } + + ASSERT(size <= (1UL << flist)); + vsp = vmp->vm_freelist[flist].vs_knext; + addr = vsp->vs_start; + if (vmflag & VM_ENDALLOC) { + addr += vsp->vs_end - (addr + size); + } + (void) vmem_seg_alloc(vmp, vsp, addr, size); + mutex_exit(&vmp->vm_lock); + return ((void *)addr); +} + +/* + * Free the segment [vaddr, vaddr + size). + */ +void +vmem_free_impl(vmem_t *vmp, void *vaddr, size_t size) +{ + if (size - 1 < vmp->vm_qcache_max) + kmem_cache_free(vmp->vm_qcache[(size - 1) >> vmp->vm_qshift], + vaddr); + else + vmem_xfree(vmp, vaddr, size); +} + +/* + * Determine whether arena vmp contains the segment [vaddr, vaddr + size). + */ +int +vmem_contains(vmem_t *vmp, void *vaddr, size_t size) +{ + uintptr_t start = (uintptr_t)vaddr; + uintptr_t end = start + size; + vmem_seg_t *vsp; + vmem_seg_t *seg0 = &vmp->vm_seg0; + + mutex_enter(&vmp->vm_lock); + vmp->vm_kstat.vk_contains.value.ui64++; + for (vsp = seg0->vs_knext; vsp != seg0; vsp = vsp->vs_knext) { + vmp->vm_kstat.vk_contains_search.value.ui64++; + ASSERT(vsp->vs_type == VMEM_SPAN); + if (start >= vsp->vs_start && end - 1 <= vsp->vs_end - 1) + break; + } + mutex_exit(&vmp->vm_lock); + return (vsp != seg0); +} + +/* + * Add the span [vaddr, vaddr + size) to arena vmp. + */ +void * +vmem_add(vmem_t *vmp, void *vaddr, size_t size, int vmflag) +{ + if (vaddr == NULL || size == 0) + panic("vmem_add(%p, %p, %lu): bad arguments", + (void *)vmp, vaddr, size); + + ASSERT(!vmem_contains(vmp, vaddr, size)); + + mutex_enter(&vmp->vm_lock); + if (vmem_populate(vmp, vmflag)) + (void) vmem_span_create(vmp, vaddr, size, 0); + else + vaddr = NULL; + mutex_exit(&vmp->vm_lock); + return (vaddr); +} + +/* + * Walk the vmp arena, applying func to each segment matching typemask. + * If VMEM_REENTRANT is specified, the arena lock is dropped across each + * call to func(); otherwise, it is held for the duration of vmem_walk() + * to ensure a consistent snapshot. Note that VMEM_REENTRANT callbacks + * are *not* necessarily consistent, so they may only be used when a hint + * is adequate. + */ +void +vmem_walk(vmem_t *vmp, int typemask, + void (*func)(void *, void *, size_t), void *arg) +{ + vmem_seg_t *vsp; + vmem_seg_t *seg0 = &vmp->vm_seg0; + vmem_seg_t walker; + + if (typemask & VMEM_WALKER) + return; + + memset(&walker, 0, sizeof (walker)); + walker.vs_type = VMEM_WALKER; + + mutex_enter(&vmp->vm_lock); + VMEM_INSERT(seg0, &walker, a); + for (vsp = seg0->vs_anext; vsp != seg0; vsp = vsp->vs_anext) { + if (vsp->vs_type & typemask) { + void *start = (void *)vsp->vs_start; + size_t size = VS_SIZE(vsp); + if (typemask & VMEM_REENTRANT) { + vmem_advance(vmp, &walker, vsp); + mutex_exit(&vmp->vm_lock); + func(arg, start, size); + mutex_enter(&vmp->vm_lock); + vsp = &walker; + } else { + func(arg, start, size); + } + } + } + vmem_advance(vmp, &walker, NULL); + mutex_exit(&vmp->vm_lock); +} + +/* + * Return the total amount of memory whose type matches typemask. Thus: + * + * typemask VMEM_ALLOC yields total memory allocated (in use). + * typemask VMEM_FREE yields total memory free (available). + * typemask (VMEM_ALLOC | VMEM_FREE) yields total arena size. + */ +size_t +vmem_size(vmem_t *vmp, int typemask) +{ + int64_t size = 0; + + if (typemask & VMEM_ALLOC) + size += (int64_t)vmp->vm_kstat.vk_mem_inuse.value.ui64; + if (typemask & VMEM_FREE) + size += (int64_t)vmp->vm_kstat.vk_mem_total.value.ui64 - + (int64_t)vmp->vm_kstat.vk_mem_inuse.value.ui64; + if (size < 0) + size = 0; + + return ((size_t)size); +} + +size_t +vmem_size_locked(vmem_t *vmp, int typemask) +{ + boolean_t m = (mutex_owner(&vmp->vm_lock) == curthread); + + if (!m) + mutex_enter(&vmp->vm_lock); + size_t s = vmem_size(vmp, typemask); + if (!m) + mutex_exit(&vmp->vm_lock); + return (s); +} + +size_t +vmem_size_semi_atomic(vmem_t *vmp, int typemask) +{ + int64_t size = 0; + uint64_t inuse = 0; + uint64_t total = 0; + + __sync_swap(&total, vmp->vm_kstat.vk_mem_total.value.ui64); + __sync_swap(&inuse, vmp->vm_kstat.vk_mem_inuse.value.ui64); + + int64_t inuse_signed = (int64_t)inuse; + int64_t total_signed = (int64_t)total; + + if (typemask & VMEM_ALLOC) + size += inuse_signed; + if (typemask & VMEM_FREE) + size += total_signed - inuse_signed; + + if (size < 0) + size = 0; + + return ((size_t)size); +} + +size_t +spl_vmem_size(vmem_t *vmp, int typemask) +{ + return (vmem_size_locked(vmp, typemask)); +} + +/* + * Create an arena called name whose initial span is [base, base + size). + * The arena's natural unit of currency is quantum, so vmem_alloc_impl() + * guarantees quantum-aligned results. The arena may import new spans + * by invoking afunc() on source, and may return those spans by invoking + * ffunc() on source. To make small allocations fast and scalable, + * the arena offers high-performance caching for each integer multiple + * of quantum up to qcache_max. + */ +static vmem_t * +vmem_create_common(const char *name, void *base, size_t size, size_t quantum, + void *(*afunc)(vmem_t *, size_t, int), + void (*ffunc)(vmem_t *, void *, size_t), + vmem_t *source, size_t qcache_max, int vmflag) +{ + int i; + size_t nqcache; + vmem_t *vmp, *cur, **vmpp; + vmem_seg_t *vsp; + vmem_freelist_t *vfp; + uint32_t id = atomic_inc_32_nv(&vmem_id); + + if (vmem_vmem_arena != NULL) { + vmp = vmem_alloc_impl(vmem_vmem_arena, sizeof (vmem_t), + vmflag & VM_KMFLAGS); + } else { + ASSERT(id <= VMEM_INITIAL); + vmp = &vmem0[id - 1]; + } + + /* An identifier arena must inherit from another identifier arena */ + ASSERT(source == NULL || ((source->vm_cflags & VMC_IDENTIFIER) == + (vmflag & VMC_IDENTIFIER))); + + if (vmp == NULL) + return (NULL); + memset(vmp, 0, sizeof (vmem_t)); + + (void) snprintf(vmp->vm_name, VMEM_NAMELEN, "%s", name); + mutex_init(&vmp->vm_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&vmp->vm_cv, NULL, CV_DEFAULT, NULL); + vmp->vm_cflags = vmflag; + vmflag &= VM_KMFLAGS; + + hrtime_t hrnow = gethrtime(); + + vmp->vm_createtime = hrnow; + + vmp->vm_quantum = quantum; + vmp->vm_qshift = highbit(quantum) - 1; + nqcache = MIN(qcache_max >> vmp->vm_qshift, VMEM_NQCACHE_MAX); + + for (i = 0; i <= VMEM_FREELISTS; i++) { + vfp = &vmp->vm_freelist[i]; + vfp->vs_end = 1UL << i; + vfp->vs_knext = (vmem_seg_t *)(vfp + 1); + vfp->vs_kprev = (vmem_seg_t *)(vfp - 1); + } + + vmp->vm_freelist[0].vs_kprev = NULL; + vmp->vm_freelist[VMEM_FREELISTS].vs_knext = NULL; + vmp->vm_freelist[VMEM_FREELISTS].vs_end = 0; + vmp->vm_hash_table = vmp->vm_hash0; + vmp->vm_hash_mask = VMEM_HASH_INITIAL - 1; + vmp->vm_hash_shift = highbit(vmp->vm_hash_mask); + + vsp = &vmp->vm_seg0; + vsp->vs_anext = vsp; + vsp->vs_aprev = vsp; + vsp->vs_knext = vsp; + vsp->vs_kprev = vsp; + vsp->vs_type = VMEM_SPAN; + vsp->vs_span_createtime = hrnow; + + vsp = &vmp->vm_rotor; + vsp->vs_type = VMEM_ROTOR; + VMEM_INSERT(&vmp->vm_seg0, vsp, a); + + memcpy(&vmp->vm_kstat, &vmem_kstat_template, sizeof (vmem_kstat_t)); + + vmp->vm_id = id; + if (source != NULL) + vmp->vm_kstat.vk_source_id.value.ui32 = source->vm_id; + vmp->vm_source = source; + vmp->vm_source_alloc = afunc; + vmp->vm_source_free = ffunc; + + /* + * Some arenas (like vmem_metadata and kmem_metadata) cannot + * use quantum caching to lower fragmentation. Instead, we + * increase their imports, giving a similar effect. + */ + if (vmp->vm_cflags & VMC_NO_QCACHE) { + if (qcache_max > VMEM_NQCACHE_MAX && ISP2(qcache_max)) { + vmp->vm_min_import = qcache_max; + } else { + vmp->vm_min_import = + VMEM_QCACHE_SLABSIZE(nqcache << vmp->vm_qshift); + } + nqcache = 0; + } + + if (nqcache != 0) { + ASSERT(!(vmflag & VM_NOSLEEP)); + vmp->vm_qcache_max = nqcache << vmp->vm_qshift; + for (i = 0; i < nqcache; i++) { + char buf[VMEM_NAMELEN + 21]; + (void) snprintf(buf, VMEM_NAMELEN + 20, "%s_%lu", + vmp->vm_name, (i + 1) * quantum); + vmp->vm_qcache[i] = kmem_cache_create(buf, + (i + 1) * quantum, quantum, NULL, NULL, NULL, + NULL, vmp, KMC_QCACHE | KMC_NOTOUCH); + } + } + + if ((vmp->vm_ksp = kstat_create("vmem", vmp->vm_id, vmp->vm_name, + "vmem", KSTAT_TYPE_NAMED, sizeof (vmem_kstat_t) / + sizeof (kstat_named_t), KSTAT_FLAG_VIRTUAL)) != NULL) { + vmp->vm_ksp->ks_data = &vmp->vm_kstat; + kstat_install(vmp->vm_ksp); + } + + mutex_enter(&vmem_list_lock); + vmpp = &vmem_list; + while ((cur = *vmpp) != NULL) + vmpp = &cur->vm_next; + *vmpp = vmp; + mutex_exit(&vmem_list_lock); + + if (vmp->vm_cflags & VMC_POPULATOR) { + ASSERT(vmem_populators < VMEM_INITIAL); + vmem_populator[atomic_inc_32_nv(&vmem_populators) - 1] = vmp; + mutex_enter(&vmp->vm_lock); + (void) vmem_populate(vmp, vmflag | VM_PANIC); + mutex_exit(&vmp->vm_lock); + } + + if ((base || size) && vmem_add(vmp, base, size, vmflag) == NULL) { + vmem_destroy(vmp); + return (NULL); + } + + return (vmp); +} + +vmem_t * +vmem_xcreate(const char *name, void *base, size_t size, size_t quantum, + vmem_ximport_t *afunc, vmem_free_t *ffunc, vmem_t *source, + size_t qcache_max, int vmflag) +{ + ASSERT(!(vmflag & (VMC_POPULATOR | VMC_XALLOC))); + vmflag &= ~(VMC_POPULATOR | VMC_XALLOC); + + return (vmem_create_common(name, base, size, quantum, + (vmem_alloc_t *)afunc, ffunc, source, qcache_max, + vmflag | VMC_XALLOC)); +} + +vmem_t * +vmem_create(const char *name, void *base, size_t size, size_t quantum, + vmem_alloc_t *afunc, vmem_free_t *ffunc, vmem_t *source, + size_t qcache_max, int vmflag) +{ + ASSERT(!(vmflag & (VMC_XALLOC | VMC_XALIGN))); + vmflag &= ~(VMC_XALLOC | VMC_XALIGN); + + return (vmem_create_common(name, base, size, quantum, + afunc, ffunc, source, qcache_max, vmflag)); +} + +/* + * Destroy arena vmp. + */ +void +vmem_destroy(vmem_t *vmp) +{ + vmem_t *cur, **vmpp; + vmem_seg_t *seg0 = &vmp->vm_seg0; + vmem_seg_t *vsp, *anext; + size_t leaked; + + /* + * set vm_nsegfree to zero because vmem_free_span_list + * would have already freed vm_segfree. + */ + vmp->vm_nsegfree = 0; + mutex_enter(&vmem_list_lock); + vmpp = &vmem_list; + while ((cur = *vmpp) != vmp) + vmpp = &cur->vm_next; + *vmpp = vmp->vm_next; + mutex_exit(&vmem_list_lock); + + leaked = vmem_size(vmp, VMEM_ALLOC); + if (leaked != 0) + dprintf("SPL: vmem_destroy('%s'): leaked %lu %s\n", + vmp->vm_name, leaked, (vmp->vm_cflags & VMC_IDENTIFIER) ? + "identifiers" : "bytes"); + + if (vmp->vm_hash_table != vmp->vm_hash0) + if (vmem_hash_arena != NULL) + vmem_free_impl(vmem_hash_arena, vmp->vm_hash_table, + (vmp->vm_hash_mask + 1) * sizeof (void *)); + + /* + * Give back the segment structures for anything that's left in the + * arena, e.g. the primary spans and their free segments. + */ + VMEM_DELETE(&vmp->vm_rotor, a); + for (vsp = seg0->vs_anext; vsp != seg0; vsp = anext) { + anext = vsp->vs_anext; + vmem_putseg_global(vsp); + } + + while (vmp->vm_nsegfree > 0) + vmem_putseg_global(vmem_getseg(vmp)); + + kstat_delete(vmp->vm_ksp); + + mutex_destroy(&vmp->vm_lock); + cv_destroy(&vmp->vm_cv); + vmem_free_impl(vmem_vmem_arena, vmp, sizeof (vmem_t)); +} + + +/* + * Destroy arena vmp. + */ +void +vmem_destroy_internal(vmem_t *vmp) +{ + vmem_t *cur, **vmpp; + vmem_seg_t *seg0 = &vmp->vm_seg0; + vmem_seg_t *vsp, *anext; + size_t leaked; + + mutex_enter(&vmem_list_lock); + vmpp = &vmem_list; + while ((cur = *vmpp) != vmp) + vmpp = &cur->vm_next; + *vmpp = vmp->vm_next; + mutex_exit(&vmem_list_lock); + + leaked = vmem_size(vmp, VMEM_ALLOC); + if (leaked != 0) + dprintf("SPL: vmem_destroy('%s'): leaked %lu %s\n", + vmp->vm_name, leaked, (vmp->vm_cflags & VMC_IDENTIFIER) ? + "identifiers" : "bytes"); + + if (vmp->vm_hash_table != vmp->vm_hash0) + if (vmem_hash_arena != NULL) + vmem_free_impl(vmem_hash_arena, vmp->vm_hash_table, + (vmp->vm_hash_mask + 1) * sizeof (void *)); + + /* + * Give back the segment structures for anything that's left in the + * arena, e.g. the primary spans and their free segments. + */ + VMEM_DELETE(&vmp->vm_rotor, a); + for (vsp = seg0->vs_anext; vsp != seg0; vsp = anext) { + anext = vsp->vs_anext; + vmem_putseg_global(vsp); + } + + while (vmp->vm_nsegfree > 0) + vmem_putseg_global(vmem_getseg(vmp)); + + if (!(vmp->vm_cflags & VMC_IDENTIFIER) && + vmem_size(vmp, VMEM_ALLOC) != 0) + dprintf("SPL: vmem_destroy('%s'): STILL %lu bytes at " + "kstat_delete() time\n", + vmp->vm_name, vmem_size(vmp, VMEM_ALLOC)); + + kstat_delete(vmp->vm_ksp); + + mutex_destroy(&vmp->vm_lock); + cv_destroy(&vmp->vm_cv); + + // Alas, to free, requires access to "vmem_vmem_arena" the very thing + // we release first. + // vmem_free_impl(vmem_vmem_arena, vmp, sizeof (vmem_t)); +} + +/* + * Only shrink vmem hashtable if it is 1<vm_kstat.vk_alloc.value.ui64 - + vmp->vm_kstat.vk_free.value.ui64); + + new_size = MAX(VMEM_HASH_INITIAL, 1 << (highbit(3 * nseg + 4) - 2)); + old_size = vmp->vm_hash_mask + 1; + + if ((old_size >> vmem_rescale_minshift) <= new_size && + new_size <= (old_size << 1)) + return; + + new_table = vmem_alloc_impl(vmem_hash_arena, new_size * sizeof (void *), + VM_NOSLEEP); + if (new_table == NULL) + return; + memset(new_table, 0, new_size * sizeof (void *)); + + mutex_enter(&vmp->vm_lock); + + old_size = vmp->vm_hash_mask + 1; + old_table = vmp->vm_hash_table; + + vmp->vm_hash_mask = new_size - 1; + vmp->vm_hash_table = new_table; + vmp->vm_hash_shift = highbit(vmp->vm_hash_mask); + + for (h = 0; h < old_size; h++) { + vsp = old_table[h]; + while (vsp != NULL) { + uintptr_t addr = vsp->vs_start; + vmem_seg_t *next_vsp = vsp->vs_knext; + vmem_seg_t **hash_bucket = VMEM_HASH(vmp, addr); + vsp->vs_knext = *hash_bucket; + *hash_bucket = vsp; + vsp = next_vsp; + } + } + + mutex_exit(&vmp->vm_lock); + + if (old_table != vmp->vm_hash0) + vmem_free_impl(vmem_hash_arena, old_table, + old_size * sizeof (void *)); +} + +/* + * Perform periodic maintenance on all vmem arenas. + */ + +void +vmem_update(void *dummy) +{ + vmem_t *vmp; + + mutex_enter(&vmem_list_lock); + for (vmp = vmem_list; vmp != NULL; vmp = vmp->vm_next) { + /* + * If threads are waiting for resources, wake them up + * periodically so they can issue another kmem_reap() + * to reclaim resources cached by the slab allocator. + */ + cv_broadcast(&vmp->vm_cv); + + /* + * Rescale the hash table to keep the hash chains short. + */ + vmem_hash_rescale(vmp); + } + mutex_exit(&vmem_list_lock); + + (void) bsd_timeout(vmem_update, dummy, &vmem_update_interval); +} + +void +vmem_qcache_reap(vmem_t *vmp) +{ + int i; + + /* + * Reap any quantum caches that may be part of this vmem. + */ + for (i = 0; i < VMEM_NQCACHE_MAX; i++) + if (vmp->vm_qcache[i]) + kmem_cache_reap_now(vmp->vm_qcache[i]); +} + +/* given a size, return the appropriate vmem_bucket_arena[] entry */ + +static inline uint16_t +vmem_bucket_number(size_t size) +{ + // For VMEM_BUCKET_HIBIT == 12, + // vmem_bucket_arena[n] holds allocations from 2^[n+11]+1 to 2^[n+12], + // so for [n] = 0, 2049-4096, for [n]=5 65537-131072, + // for [n]=7 (256k+1)-512k + // set hb: 512k == 19, 256k+1 == 19, 256k == 18, ... + const int hb = highbit(size-1); + + int bucket = hb - VMEM_BUCKET_LOWBIT; + + // very large allocations go into the 16 MiB bucket + if (hb > VMEM_BUCKET_HIBIT) + bucket = VMEM_BUCKET_HIBIT - VMEM_BUCKET_LOWBIT; + + // very small allocations go into the 4 kiB bucket + if (bucket < 0) + bucket = 0; + + return (bucket); +} + +static inline vmem_t * +vmem_bucket_arena_by_size(size_t size) +{ + uint16_t bucket = vmem_bucket_number(size); + + return (vmem_bucket_arena[bucket]); +} + +vmem_t * +spl_vmem_bucket_arena_by_size(size_t size) +{ + return (vmem_bucket_arena_by_size(size)); +} + +static inline void +vmem_bucket_wake_all_waiters(void) +{ + for (int i = VMEM_BUCKET_LOWBIT; i < VMEM_BUCKET_HIBIT; i++) { + const int bucket = i - VMEM_BUCKET_LOWBIT; + vmem_t *bvmp = vmem_bucket_arena[bucket]; + cv_broadcast(&bvmp->vm_cv); + } + cv_broadcast(&spl_heap_arena->vm_cv); +} + +/* + * xnu_alloc_throttled_bail() : spin looking for memory + * + */ + +static inline void * +xnu_alloc_throttled_bail(uint64_t now_ticks, vmem_t *calling_vmp, + size_t size, int vmflags) +{ + // spin looking for memory + const uint64_t bigtarget = MAX(size, 16ULL*1024ULL*1024ULL); + static volatile _Atomic bool alloc_lock = false; + static volatile _Atomic uint64_t force_time = 0; + + uint64_t timeout_ticks = hz / 2; + if (vmflags & VM_PUSHPAGE) + timeout_ticks = hz / 4; + + uint64_t timeout_time = now_ticks + timeout_ticks; + + for (uint32_t suspends = 0, blocked_suspends = 0, + try_no_pressure = 0; /* empty */; /* empty */) { + if (force_time + timeout_ticks > timeout_time) { + // another thread has forced an allocation + // by timing out. push our deadline into the future. + timeout_time = force_time + timeout_ticks; + } + if (alloc_lock) { + blocked_suspends++; + IOSleep(1); + } else if (spl_vmem_xnu_useful_bytes_free() >= bigtarget) { + bool f = false; + // if alloc_lock == f then alloc_lock = true and result + // is true otherwise result is false and f = true + if (!__c11_atomic_compare_exchange_strong(&alloc_lock, + &f, true, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED)) { + /* + * avoid (highly unlikely) data race on + * alloc_lock. if alloc_lock has become true + * while we were in the else if expression + * then we effectively optimize away the + * (relaxed) load of alloc_lock (== true) + * into f and continue. + */ + continue; + } + // alloc_lock is now visible as true to all threads + try_no_pressure++; + void *m = spl_vmem_malloc_if_no_pressure(size); + if (m != NULL) { + uint64_t ticks = zfs_lbolt() - now_ticks; + dprintf("SPL: %s returning %llu bytes after " + "%llu ticks (hz=%u, seconds = %llu), " + "%u suspends, %u blocked, %u tries (%s)\n", + __func__, (uint64_t)size, + ticks, hz, ticks/hz, suspends, + blocked_suspends, try_no_pressure, + calling_vmp->vm_name); + // atomic seq cst, so is published to all + // threads + alloc_lock = false; + return (m); + } else { + alloc_lock = false; + spl_free_set_emergency_pressure(bigtarget); + suspends++; + IOSleep(1); + } + } else if (zfs_lbolt() > timeout_time) { + bool f = false; + if (!__c11_atomic_compare_exchange_strong(&alloc_lock, + &f, true, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED)) { + // avoid (highly unlikely) data race on + // alloc_lock as above + continue; + } + void *mp = spl_vmem_malloc_unconditionally(size); + uint64_t now = zfs_lbolt(); + uint64_t ticks = now - now_ticks; + force_time = now; + dprintf("SPL: %s TIMEOUT %llu bytes after " + "%llu ticks (hz=%u, seconds=%llu), " + "%u suspends, %u blocked, %u tries (%s)\n", + __func__, (uint64_t)size, + ticks, hz, ticks/hz, suspends, + blocked_suspends, try_no_pressure, + calling_vmp->vm_name); + alloc_lock = false; + atomic_inc_64(&spl_xat_forced); + return (mp); + } else { + spl_free_set_emergency_pressure(bigtarget); + suspends++; + IOSleep(1); + } + } +} + +static void * +xnu_alloc_throttled(vmem_t *bvmp, size_t size, int vmflag) +{ + // the caller is one of the bucket arenas. + // null_vmp will be spl_default_arena_parent, which is + // just a placeholder. + + uint64_t now = zfs_lbolt(); + const uint64_t entry_now = now; + + void *m = spl_vmem_malloc_if_no_pressure(size); + + if (m != NULL) { + atomic_inc_64(&spl_xat_success); + spl_xat_lastalloc = gethrtime(); + // wake up waiters on all the arena condvars + // since there is apparently no memory shortage. + vmem_bucket_wake_all_waiters(); + return (m); + } else { + spl_free_set_emergency_pressure((int64_t)size); + } + + if (vmflag & VM_PANIC) { + // force an allocation now to avoid a panic + spl_xat_lastalloc = gethrtime(); + spl_free_set_emergency_pressure(4LL * (int64_t)size); + void *p = spl_vmem_malloc_unconditionally(size); + // p cannot be NULL (unconditional kernel malloc always works + // or panics) + // therefore: success, wake all waiters on alloc|free condvar + // wake up arena waiters to let them know there is memory + // available in the arena; let waiters on other bucket arenas + // continue sleeping. + cv_broadcast(&bvmp->vm_cv); + return (p); + } + + if (vmflag & VM_NOSLEEP) { + spl_free_set_emergency_pressure(MAX(2LL * (int64_t)size, + 16LL*1024LL*1024LL)); + /* cheating a bit, but not really waiting */ + kpreempt(KPREEMPT_SYNC); + void *p = spl_vmem_malloc_if_no_pressure(size); + if (p != NULL) { + atomic_inc_64(&spl_xat_late_success_nosleep); + cv_broadcast(&bvmp->vm_cv); + spl_xat_lastalloc = gethrtime(); + } + // if p == NULL, then there will be an increment in + // the fail kstat + return (p); + } + + /* + * Loop for a while trying to satisfy VM_SLEEP allocations. + * + * If we are able to allocate memory, then return the pointer. + * + * We return NULL if some other thread's activity has caused + * sufficient memory to appear in this arena that we can satisfy + * the allocation. + * + * We call xnu_alloc_throttle_bail() after a few milliseconds of + * waiting; it will either return a pointer to newly allocated + * memory or NULL. We return the result. + * + */ + + const uint32_t bucket_number = + vmem_bucket_id_to_bucket_number[bvmp->vm_id]; + static volatile _Atomic uint32_t waiters = 0; + + waiters++; + + if (waiters == 1UL) + atomic_inc_64(&spl_xat_no_waiters); + + static _Atomic uint32_t max_waiters_seen = 0; + + if (waiters > max_waiters_seen) { + max_waiters_seen = waiters; + dprintf("SPL: %s: max_waiters_seen increased to %u\n", __func__, + max_waiters_seen); + } + + boolean_t local_xat_pressured = false; + + for (; /* empty */; /* empty */) { + clock_t wait_time = USEC2NSEC(500UL * MAX(waiters, 1UL)); + mutex_enter(&bvmp->vm_lock); + spl_xat_sleep++; + if (local_xat_pressured) { + spl_xat_pressured++; + local_xat_pressured = false; + } + (void) cv_timedwait_hires(&bvmp->vm_cv, &bvmp->vm_lock, + wait_time, 0, 0); + mutex_exit(&bvmp->vm_lock); + now = zfs_lbolt(); + // We may be here because of a broadcast to &vmp->vm_cv, + // causing xnu to schedule all the sleepers in priority-weighted + // FIFO order. Because of the mutex_exit(), the sections below + // here may be entered concurrently. + // spl_vmem_malloc_if_no_pressure does a mutex, so avoid calling + // it unless there is a chance it will succeed. + if (spl_vmem_xnu_useful_bytes_free() > (MAX(size, + 16ULL*1024ULL*1024ULL))) { + void *a = spl_vmem_malloc_if_no_pressure(size); + if (a != NULL) { + atomic_inc_64(&spl_xat_late_success); + spl_xat_lastalloc = gethrtime(); + waiters--; + // Wake up all waiters on the bucket arena + // locks, since the system apparently has + // memory again. + vmem_bucket_wake_all_waiters(); + return (a); + } else { + // Probably spl_vm_page_free_count changed while + // we were in the mutex queue in + // spl_vmem_malloc_if_no_pressure(). There is + // therefore no point in doing the bail-out + // check below, so go back to the top of the + // for loop. + atomic_inc_64(&spl_xat_late_deny); + continue; + } + } + if (now > entry_now + hz / 4 || + spl_vba_threads[bucket_number] > 1UL) { + // If there are other threads waiting for us + // in vba() then when we satisfy this allocation, + // we satisfy more than one thread, so invoke XATB(). + // Otherwise, if we have had no luck for 250 ms, then + // switch to XATB() which is much more aggressive. + if (spl_vba_threads[bucket_number] > 1UL) + atomic_inc_64(&spl_xat_bailed_contended); + atomic_inc_64(&spl_xat_bailed); + static _Atomic uint32_t bailing_threads = 0; + static _Atomic uint32_t max_bailers_seen = 0; + bailing_threads++; + if (bailing_threads > max_bailers_seen) { + max_bailers_seen = bailing_threads; + dprintf("SPL: %s: max_bailers_seen increased " + "to %u\n", __func__, max_bailers_seen); + } + void *b = + xnu_alloc_throttled_bail(now, bvmp, size, vmflag); + bailing_threads--; + spl_xat_lastalloc = gethrtime(); + // wake up waiters on the arena lock, + // since they now have memory they can use. + cv_broadcast(&bvmp->vm_cv); + // open turnstile after having bailed, rather + // than before + waiters--; + return (b); + } else if (now - entry_now > 0 && + ((now - entry_now) % (hz/10))) { + spl_free_set_emergency_pressure(MAX(size, + 16LL*1024LL*1024LL)); + local_xat_pressured = true; + } + } +} + +static void +xnu_free_throttled(vmem_t *vmp, void *vaddr, size_t size) +{ + extern void osif_free(void *, uint64_t); + + // Serialize behind a (short) spin-sleep delay, giving + // xnu time to do freelist management and + // PT teardowns + + // In the usual case there is only one thread in this function, + // so we proceed waitlessly to osif_free(). + + // When there are multiple threads here, we delay the 2nd and later. + + // Explict race: + // The osif_free() is not protected by the vmem_xnu_alloc_lock + // mutex; that is just used for implementing the delay. Consequently, + // the waiters on the same lock in spl_vmem_malloc_if_no_pressure may + // falsely see too small a value for spl_vm_page_free_count. We don't + // care in part because xnu performs poorly when doing + // free-then-allocate anwyay. + + // a_waiters gauges the loop exit checking and sleep duration; + // it is a count of the number of threads trying to do work + // in this function. + static volatile _Atomic uint32_t a_waiters = 0; + + // is_freeing protects the osif_free() call; see comment below + static volatile _Atomic bool is_freeing = false; + + a_waiters++; // generates "lock incl ..." + + static _Atomic uint32_t max_waiters_seen = 0; + + if (a_waiters > max_waiters_seen) { + max_waiters_seen = a_waiters; + dprintf("SPL: %s: max_waiters_seen increased to %u\n", + __func__, max_waiters_seen); + } + + for (uint32_t iter = 0; a_waiters > 1UL; iter++) { + // there is more than one thread here, so suspend and + // sleep for 1 ms + atomic_inc_64(&spl_xft_wait); + IOSleep(1); + // If are growing old in this loop, then see if + // anyone else is still in osif_free. If not, + // we can exit. + if (iter >= a_waiters) { + // if is_freeing == f, then set is_freeing to true with + // release semantics (i.e. "push" it to other cores) + // then break; otherwise, set f to true relaxedly (i.e., + // optimize it out) + bool f = false; + if (__c11_atomic_compare_exchange_weak(&is_freeing, + &f, true, __ATOMIC_RELEASE, __ATOMIC_RELAXED)) { + break; + } + } + } + // If there is more than one thread in this function, osif_free() is + // protected by is_freeing. Release it after the osif_free() + // call has been made and the lastfree bookkeeping has been done. + osif_free(vaddr, size); + spl_xat_lastfree = gethrtime(); + is_freeing = false; + a_waiters--; + kpreempt(KPREEMPT_SYNC); + // since we just gave back xnu enough to satisfy an allocation + // in at least the smaller buckets, let's wake up anyone in + // the cv_wait() in vmem_xalloc([bucket_#], ...) + vmem_bucket_wake_all_waiters(); +} + +// return 0 if the bit was unset before the atomic OR. +static inline bool +vba_atomic_lock_bucket(volatile _Atomic uint16_t *bbap, uint16_t bucket_bit) +{ + + // We use a test-and-set of the appropriate bit + // in buckets_busy_allocating; if it was not set, + // then break out of the loop. + // + // This compiles into an orl, cmpxchgw instruction pair. + // the return from __c11_atomic_fetch_or() is the + // previous value of buckets_busy_allocating. + + uint16_t prev = + __c11_atomic_fetch_or(bbap, bucket_bit, __ATOMIC_SEQ_CST); + if (prev & bucket_bit) + return (false); // we did not acquire the bit lock here + else + return (true); // we turned the bit from 0 to 1 +} + +static void * +vmem_bucket_alloc(vmem_t *null_vmp, size_t size, const int vmflags) +{ + + if (vmflags & VM_NO_VBA) + return (NULL); + + // caller is spl_heap_arena looking for memory. + // null_vmp will be spl_default_arena_parent, and so + // is just a placeholder. + + vmem_t *calling_arena = spl_heap_arena; + + static volatile _Atomic uint32_t hipriority_allocators = 0; + boolean_t local_hipriority_allocator = false; + + if (0 != (vmflags & (VM_PUSHPAGE | VM_NOSLEEP | VM_PANIC | VM_ABORT))) { + local_hipriority_allocator = true; + hipriority_allocators++; + } + + if (!ISP2(size)) + atomic_inc_64(&spl_bucket_non_pow2_allocs); + + vmem_t *bvmp = vmem_bucket_arena_by_size(size); + + // there are 13 buckets, so use a 16-bit scalar to hold + // a set of bits, where each bit corresponds to an in-progress + // vmem_alloc_impl(bucket, ...) below. + + static volatile _Atomic uint16_t buckets_busy_allocating = 0; + const uint16_t bucket_number = vmem_bucket_number(size); + const uint16_t bucket_bit = (uint16_t)1 << bucket_number; + + spl_vba_threads[bucket_number]++; + + static volatile _Atomic uint32_t waiters = 0; + + // First, if we are VM_SLEEP, check for memory, try some pressure, + // and if that doesn't work, force entry into the loop below. + + bool loop_once = false; + + if ((vmflags & (VM_NOSLEEP | VM_PANIC | VM_ABORT)) == 0 && + ! vmem_canalloc_atomic(bvmp, size)) { + if (spl_vmem_xnu_useful_bytes_free() < (MAX(size, + 16ULL*1024ULL*1024ULL))) { + spl_free_set_emergency_pressure(size); + IOSleep(1); + if (!vmem_canalloc_atomic(bvmp, size) && + (spl_vmem_xnu_useful_bytes_free() < (MAX(size, + 16ULL*1024ULL*1024ULL)))) { + loop_once = true; + } + } + } + + // spin-sleep: if we would need to go to the xnu allocator. + // + // We want to avoid a burst of allocs from bucket_heap's children + // successively hitting a low-memory condition, or alternatively + // each successfully importing memory from xnu when they can share + // a single import. + // + // We also want to take advantage of any memory that becomes available + // in bucket_heap. + // + // If there is more than one thread in this function (~ few percent) + // then the subsequent threads are put into the loop below. They + // can escape the loop if they are [1]non-waiting allocations, or + // [2]if they become the only waiting thread, or + // [3]if the cv_timedwait_hires returns -1 (which represents EWOULDBLOCK + // from msleep() which gets it from _sleep()'s THREAD_TIMED_OUT) + // allocating in the bucket, or [4]if this thread has (rare condition) + // spent a quarter of a second in the loop. + + if (waiters++ > 1 || loop_once) { + atomic_inc_64(&spl_vba_loop_entries); + } + + static _Atomic uint32_t max_waiters_seen = 0; + + if (waiters > max_waiters_seen) { + max_waiters_seen = waiters; + dprintf("SPL: %s: max_waiters_seen increased to %u\n", __func__, + max_waiters_seen); + } + + // local counters, to be added atomically to global kstat variables + uint64_t local_memory_blocked = 0, local_cv_timeout = 0; + uint64_t local_loop_timeout = 0; + uint64_t local_cv_timeout_blocked = 0, local_loop_timeout_blocked = 0; + uint64_t local_sleep = 0, local_hipriority_blocked = 0; + + const uint64_t loop_ticks = 25; // a tick is 10 msec, so 250 msec + const uint64_t hiprio_loop_ticks = 4; // 40 msec + + for (uint64_t entry_time = zfs_lbolt(), + loop_timeout = entry_time + loop_ticks, + hiprio_timeout = entry_time + hiprio_loop_ticks, timedout = 0; + waiters > 1UL || loop_once; /* empty */) { + loop_once = false; + // non-waiting allocations should proceeed to vmem_alloc_impl() + // immediately + if (vmflags & (VM_NOSLEEP | VM_PANIC | VM_ABORT)) { + break; + } + if (vmem_canalloc_atomic(bvmp, size)) { + // We can probably vmem_alloc(bvmp, size, vmflags). + // At worst case it will give us a NULL and we will + // end up on the vmp's cv_wait. + // + // We can have threads with different bvmp + // taking this exit, and will proceed concurrently. + // + // However, we should protect against a burst of + // callers hitting the same bvmp before the allocation + // results are reflected in + // vmem_canalloc_atomic(bvmp, ...) + if (local_hipriority_allocator == false && + hipriority_allocators > 0) { + // more high priority allocations are wanted, + // so this thread stays here + local_hipriority_blocked++; + } else if (vba_atomic_lock_bucket( + &buckets_busy_allocating, bucket_bit)) { + // we are not being blocked by another allocator + // to the same bucket, or any higher priority + // allocator + atomic_inc_64(&spl_vba_parent_memory_appeared); + break; + // The vmem_alloc_impl() should return extremely + // quickly from an INSTANTFIT allocation that + // canalloc predicts will succeed. + } else { + // another thread is trying to use the free + // memory in the bucket_## arena; there might + // still be free memory there after its + // allocation is completed, and there might be + // excess in the bucket_heap arena, so stick + // around in this loop. + local_memory_blocked++; + cv_broadcast(&bvmp->vm_cv); + } + } + if (timedout > 0) { + if (local_hipriority_allocator == false && + hipriority_allocators > 0) { + local_hipriority_blocked++; + } else if (vba_atomic_lock_bucket( + &buckets_busy_allocating, bucket_bit)) { + if (timedout & 1) + local_cv_timeout++; + if (timedout & 6 || zfs_lbolt() >= loop_timeout) + local_loop_timeout++; + break; + } else { + if (timedout & 1) { + local_cv_timeout_blocked++; + } + if (timedout & 6) { + local_loop_timeout_blocked++; + } else if (zfs_lbolt() > loop_timeout) { + timedout |= 2; + } + // flush the current thread in xat() out of + // xat()'s for() loop and into xat_bail() + cv_broadcast(&bvmp->vm_cv); + } + } + // The bucket is already allocating, or the bucket needs + // more memory to satisfy vmem_allocat(bvmp, size, VM_NOSLEEP), + // or we want to give the bucket some time to acquire more + // memory. + // substitute for the vmp arena's cv_wait in vmem_xalloc() + // (vmp is the bucket_heap AKA spl_heap_arena) + mutex_enter(&calling_arena->vm_lock); + local_sleep++; + if (local_sleep >= 1000ULL) { + atomic_add_64(&spl_vba_sleep, local_sleep - 1ULL); + local_sleep = 1ULL; + atomic_add_64(&spl_vba_cv_timeout_blocked, + local_cv_timeout_blocked); + local_cv_timeout_blocked = 0; + atomic_add_64(&spl_vba_loop_timeout_blocked, + local_loop_timeout_blocked); + local_loop_timeout_blocked = 0; + atomic_add_64(&spl_vba_hiprio_blocked, + local_hipriority_blocked); + local_hipriority_blocked = 0; + if (local_memory_blocked > 1ULL) { + atomic_add_64(&spl_vba_parent_memory_blocked, + local_memory_blocked - 1ULL); + local_memory_blocked = 1ULL; + } + } + clock_t wait_time = MSEC2NSEC(30); + if (timedout > 0 || local_memory_blocked > 0) { + wait_time = MSEC2NSEC(1); + } + int ret = cv_timedwait_hires(&calling_arena->vm_cv, + &calling_arena->vm_lock, + wait_time, 0, 0); + // We almost certainly have exited because of a + // signal/broadcast, but maybe just timed out. + // Either way, recheck memory. + mutex_exit(&calling_arena->vm_lock); + if (ret == -1) { + // cv_timedwait_hires timer expired + timedout |= 1; + cv_broadcast(&bvmp->vm_cv); + } else if ((timedout & 2) == 0) { + // we were awakened; check to see if we have been + // in the for loop for a long time + uint64_t n = zfs_lbolt(); + if (n > loop_timeout) { + timedout |= 2; + extern uint64_t real_total_memory; + spl_free_set_emergency_pressure( + real_total_memory / 64LL); + // flush the current thread in xat() out of + // xat()'s for() loop and into xat_bail() + cv_broadcast(&bvmp->vm_cv); + } else if (local_hipriority_allocator && + n > hiprio_timeout && waiters > 1UL) { + timedout |= 4; + } + } + } + + /* + * Turn on the exclusion bit in buckets_busy_allocating, to + * prevent multiple threads from calling vmem_alloc_impl() on the + * same bucket arena concurrently rather than serially. + * + * This principally reduces the liklihood of asking xnu for + * more memory when other memory is or becomes available. + * + * This exclusion only applies to VM_SLEEP allocations; + * others (VM_PANIC, VM_NOSLEEP, VM_ABORT) will go to + * vmem_alloc_impl() concurrently with any other threads. + * + * Since we aren't doing a test-and-set operation like above, + * we can just use |= and &= below and get correct atomic + * results, instead of using: + * + * __c11_atomic_fetch_or(&buckets_busy_allocating, + * bucket_bit, __ATOMIC_SEQ_CST); + * with the &= down below being written as + * __c11_atomic_fetch_and(&buckets_busy_allocating, + * ~bucket_bit, __ATOMIC_SEQ_CST); + * + * and this makes a difference with no optimization either + * compiling the whole file or with __attribute((optnone)) + * in front of the function decl. In particular, the non- + * optimized version that uses the builtin __c11_atomic_fetch_{and,or} + * preserves the C program order in the machine language output, + * inersting cmpxchgws, while all optimized versions, and the + * non-optimized version using the plainly-written version, reorder + * the "orw regr, memory" and "andw register, memory" (these are atomic + * RMW operations in x86-64 when the memory is naturally aligned) so + * that the strong memory model x86-64 promise that later loads see the + * results of earlier stores. + * + * clang+llvm simply are good at optimizing _Atomics and + * the optimized code differs only in line numbers and + * among all three approaches (as plainly written, using + * the __c11_atomic_fetch_{or,and} with sequential consistency, + * or when compiling with at least -O optimization so an + * atomic_or_16(&buckets_busy_allocating) built with GCC intrinsics + * is actually inlined rather than a function call). + * + */ + + // in case we left the loop by being the only waiter, stop the + // next thread arriving from leaving the for loop because + // vmem_canalloc(bvmp, that_thread's_size) is true. + + buckets_busy_allocating |= bucket_bit; + + // update counters + if (local_sleep > 0) + atomic_add_64(&spl_vba_sleep, local_sleep); + if (local_memory_blocked > 0) + atomic_add_64(&spl_vba_parent_memory_blocked, + local_memory_blocked); + if (local_cv_timeout > 0) + atomic_add_64(&spl_vba_cv_timeout, local_cv_timeout); + if (local_cv_timeout_blocked > 0) + atomic_add_64(&spl_vba_cv_timeout_blocked, + local_cv_timeout_blocked); + if (local_loop_timeout > 0) + atomic_add_64(&spl_vba_loop_timeout, local_loop_timeout); + if (local_loop_timeout_blocked > 0) + atomic_add_64(&spl_vba_loop_timeout_blocked, + local_loop_timeout_blocked); + if (local_hipriority_blocked > 0) + atomic_add_64(&spl_vba_hiprio_blocked, + local_hipriority_blocked); + + // There is memory in this bucket, or there are no other waiters, + // or we aren't a VM_SLEEP allocation, or we iterated out of the + // for loop. + // vmem_alloc_impl() and vmem_xalloc() do their own mutex serializing + // on bvmp->vm_lock, so we don't have to here. + // + // vmem_alloc may take some time to return (especially for VM_SLEEP + // allocations where we did not take the vm_canalloc(bvmp...) break out + // of the for loop). Therefore, if we didn't enter the for loop at all + // because waiters was 0 when we entered this function, + // subsequent callers will enter the for loop. + + void *m = vmem_alloc_impl(bvmp, size, vmflags); + + // allow another vmem_canalloc() through for this bucket + // by atomically turning off the appropriate bit + + /* + * Except clang+llvm DTRT because of _Atomic, could be written as: + * __c11_atomic_fetch_and(&buckets_busy_allocating, + * ~bucket_bit, __ATOMIC_SEQ_CST); + * + * On processors with more relaxed memory models, it might be + * more efficient to do so with release semantics here, and + * in the atomic |= above, with acquire semantics in the bit tests, + * but on the other hand it may be hard to do better than clang+llvm. + */ + + buckets_busy_allocating &= ~bucket_bit; + + if (local_hipriority_allocator) + hipriority_allocators--; + + // if we got an allocation, wake up the arena cv waiters + // to let them try to exit the for(;;) loop above and + // exit the cv_wait() in vmem_xalloc(vmp, ...) + + if (m != NULL) { + cv_broadcast(&calling_arena->vm_cv); + } + + waiters--; + spl_vba_threads[bucket_number]--; + return (m); +} + +static void +vmem_bucket_free(vmem_t *null_vmp, void *vaddr, size_t size) +{ + vmem_t *calling_arena = spl_heap_arena; + + vmem_free_impl(vmem_bucket_arena_by_size(size), vaddr, size); + + // wake up arena waiters to let them try an alloc + cv_broadcast(&calling_arena->vm_cv); +} + +static inline int64_t +vmem_bucket_arena_free(uint16_t bucket) +{ + VERIFY(bucket < VMEM_BUCKETS); + return ((int64_t)vmem_size_semi_atomic(vmem_bucket_arena[bucket], + VMEM_FREE)); +} + +static inline int64_t +vmem_bucket_arena_used(int bucket) +{ + VERIFY(bucket < VMEM_BUCKETS); + return ((int64_t)vmem_size_semi_atomic(vmem_bucket_arena[bucket], + VMEM_ALLOC)); +} + + +int64_t +vmem_buckets_size(int typemask) +{ + int64_t total_size = 0; + + for (int i = 0; i < VMEM_BUCKETS; i++) { + int64_t u = vmem_bucket_arena_used(i); + int64_t f = vmem_bucket_arena_free(i); + if (typemask & VMEM_ALLOC) + total_size += u; + if (typemask & VMEM_FREE) + total_size += f; + } + if (total_size < 0) + total_size = 0; + + return ((size_t)total_size); +} + +static uint64_t +spl_validate_bucket_span_size(uint64_t val) +{ + if (!ISP2(val)) { + TraceEvent(TRACE_WARNING, "SPL: %s: WARNING %llu is not a " + "power of two, not changing.\n", __func__, val); + return (0); + } + if (val < 128ULL*1024ULL || val > 16ULL*1024ULL*1024ULL) { + TraceEvent(TRACE_WARNING, "SPL: %s: WARNING %llu is out " + "of range [128k - 16M], not changing.\n", __func__, val); + return (0); + } + return (val); +} + +static inline void +spl_modify_bucket_span_size(int bucket, uint64_t size) +{ + vmem_t *bvmp = vmem_bucket_arena[bucket]; + + mutex_enter(&bvmp->vm_lock); + bvmp->vm_min_import = size; + mutex_exit(&bvmp->vm_lock); +} + +static inline void +spl_modify_bucket_array() +{ + for (int i = VMEM_BUCKET_LOWBIT; i < VMEM_BUCKET_HIBIT; i++) { + // i = 12, bucket = 0, contains allocs from 8192 to 16383 bytes, + // and should never ask xnu for < 16384 bytes, so as to avoid + // asking xnu for a non-power-of-two size. + const int bucket = i - VMEM_BUCKET_LOWBIT; + const uint32_t bucket_alloc_minimum_size = 1UL << (uint32_t)i; + const uint32_t bucket_parent_alloc_minimum_size = + bucket_alloc_minimum_size * 2UL; + + switch (i) { + // see vmem_init() below for details + case 16: + case 17: + spl_modify_bucket_span_size(bucket, + MAX(spl_bucket_tunable_small_span, + bucket_parent_alloc_minimum_size)); + break; + default: + spl_modify_bucket_span_size(bucket, + MAX(spl_bucket_tunable_large_span, + bucket_parent_alloc_minimum_size)); + break; + } + } +} + +static inline void +spl_printf_bucket_span_sizes(void) +{ + // this doesn't have to be super-exact + dprintf("SPL: %s: ", __func__); + for (int i = VMEM_BUCKET_LOWBIT; i < VMEM_BUCKET_HIBIT; i++) { + int bnum = i - VMEM_BUCKET_LOWBIT; + vmem_t *bvmp = vmem_bucket_arena[bnum]; + dprintf("%llu ", (uint64_t)bvmp->vm_min_import); + } + dprintf("\n"); +} + +static inline void +spl_set_bucket_spans(uint64_t l, uint64_t s) +{ + if (spl_validate_bucket_span_size(l) && + spl_validate_bucket_span_size(s)) { + atomic_swap_64(&spl_bucket_tunable_large_span, l); + atomic_swap_64(&spl_bucket_tunable_small_span, s); + spl_modify_bucket_array(); + } +} + +void +spl_set_bucket_tunable_large_span(uint64_t size) +{ + uint64_t s = 0; + + mutex_enter(&vmem_xnu_alloc_lock); + atomic_swap_64(&s, spl_bucket_tunable_small_span); + spl_set_bucket_spans(size, s); + mutex_exit(&vmem_xnu_alloc_lock); + + spl_printf_bucket_span_sizes(); +} + +void +spl_set_bucket_tunable_small_span(uint64_t size) +{ + uint64_t l = 0; + + mutex_enter(&vmem_xnu_alloc_lock); + atomic_swap_64(&l, spl_bucket_tunable_large_span); + spl_set_bucket_spans(l, size); + mutex_exit(&vmem_xnu_alloc_lock); + + spl_printf_bucket_span_sizes(); +} + +static void * +spl_vmem_default_alloc(vmem_t *vmp, size_t size, int vmflags) +{ + extern void *osif_malloc(uint64_t); + return (osif_malloc(size)); +} + +static void +spl_vmem_default_free(vmem_t *vmp, void *vaddr, size_t size) +{ + extern void osif_free(void *, uint64_t); + osif_free(vaddr, size); +} + +vmem_t * +vmem_init(const char *heap_name, + void *heap_start, size_t heap_size, size_t heap_quantum, + void *(*heap_alloc)(vmem_t *, size_t, int), + void (*heap_free)(vmem_t *, void *, size_t)) +{ + uint32_t id; + int nseg = VMEM_SEG_INITIAL; + vmem_t *heap; + + // XNU mutexes need initialisation + mutex_init(&vmem_list_lock, "vmem_list_lock", MUTEX_DEFAULT, + NULL); + mutex_init(&vmem_segfree_lock, "vmem_segfree_lock", MUTEX_DEFAULT, + NULL); + mutex_init(&vmem_sleep_lock, "vmem_sleep_lock", MUTEX_DEFAULT, + NULL); + mutex_init(&vmem_nosleep_lock, "vmem_nosleep_lock", MUTEX_DEFAULT, + NULL); + mutex_init(&vmem_pushpage_lock, "vmem_pushpage_lock", MUTEX_DEFAULT, + NULL); + mutex_init(&vmem_panic_lock, "vmem_panic_lock", MUTEX_DEFAULT, + NULL); + mutex_init(&vmem_xnu_alloc_lock, "vmem_xnu_alloc_lock", MUTEX_DEFAULT, + NULL); + + while (--nseg >= 0) + vmem_putseg_global(&vmem_seg0[nseg]); + + /* + * On OSX we ultimately have to use the OS allocator + * as the ource and sink of memory as it is allocated + * and freed. + * + * The spl_root_arena_parent is needed in order to provide a + * base arena with an always-NULL afunc and ffunc in order to + * end the searches done by vmem_[x]alloc and vm_xfree; it + * serves no other purpose; its stats will always be zero. + * + */ + + // id 0 + spl_default_arena_parent = vmem_create("spl_default_arena_parent", + NULL, 0, heap_quantum, NULL, NULL, NULL, 0, VM_SLEEP); + + // illumos/openzfs has a gigantic pile of memory that it can use + // for its first arena; + // o3x is not so lucky, so we start with this + // Intel can go with 4096 alignment, but arm64 needs 16384. So + // we just use the larger. + // turns out that Windows refuses alignment over 8192 + __declspec(align(PAGE_SIZE)) static char + initial_default_block[16ULL * 1024ULL * 1024ULL] = { 0 }; + + // The default arena is very low-bandwidth; it supplies the initial + // large allocation for the heap arena below, and it serves as the + // parent of the vmem_metadata arena. It will typically do only 2 + // or 3 parent_alloc calls (to spl_vmem_default_alloc) in total. + + spl_default_arena = vmem_create("spl_default_arena", // id 1 + initial_default_block, 16ULL*1024ULL*1024ULL, + heap_quantum, spl_vmem_default_alloc, spl_vmem_default_free, + spl_default_arena_parent, 16ULL*1024ULL*1024ULL, + VM_SLEEP | VMC_POPULATOR | VMC_NO_QCACHE); + + VERIFY(spl_default_arena != NULL); + + // The bucket arenas satisfy allocations & frees from the bucket heap + // that are dispatched to the bucket whose power-of-two label is the + // smallest allocation that vmem_bucket_allocate will ask for. + // + // The bucket arenas in turn exchange memory with XNU's allocator/freer + // in large spans (~ 1 MiB is stable on all systems but creates bucket + // fragmentation) + // + // Segregating by size constrains internal fragmentation within the + // bucket and provides kstat.vmem visiblity and span-size policy to + // be applied to particular buckets (notably the sources of most + // allocations, see the comments below) + // + // For VMEM_BUCKET_HIBIT == 12, + // vmem_bucket_arena[n] holds allocations from 2^[n+11]+1 to 2^[n+12], + // so for [n] = 0, 2049-4096, for [n]=5 65537-131072, + // for [n]=7 (256k+1)-512k + // + // so "kstat.vmvm.vmem.bucket_1048576" should be read as the bucket + // arena containing allocations 1 MiB and smaller, but larger + // than 512 kiB. + + // create arenas for the VMEM_BUCKETS, id 2 - id 14 + + extern uint64_t real_total_memory; + VERIFY3U(real_total_memory, >=, 1024ULL*1024ULL*1024ULL); + + // adjust minimum bucket span size for memory size + // see comments in the switch below + // large span: 1 MiB and bigger on large-memory (> 32 GiB) systems + // small span: 256 kiB and bigger on large-memory systems + const uint64_t k = 1024ULL; + const uint64_t qm = 256ULL * k; + const uint64_t m = 1024ULL* k; + const uint64_t big = MAX(real_total_memory / (k * 32ULL), m); + const uint64_t small = MAX(real_total_memory / (k * 128ULL), qm); + spl_bucket_tunable_large_span = MIN(big, 16ULL * m); + spl_bucket_tunable_small_span = small; + dprintf("SPL: %s: real_total_memory %llu, large spans %llu, small " + "spans %llu\n", __func__, real_total_memory, + spl_bucket_tunable_large_span, spl_bucket_tunable_small_span); + + char *buf; + buf = vmem_alloc_impl(spl_default_arena, VMEM_NAMELEN + 21, VM_SLEEP); + + for (int32_t i = VMEM_BUCKET_LOWBIT; i <= VMEM_BUCKET_HIBIT; i++) { + const uint64_t bucket_largest_size = (1ULL << (uint64_t)i); + (void) snprintf(buf, VMEM_NAMELEN + 20, "%s_%llu", + "bucket", bucket_largest_size); + dprintf("SPL: %s creating arena %s (i == %d)\n", __func__, buf, + i); + const int bucket_number = i - VMEM_BUCKET_LOWBIT; + /* + * To reduce the number of IOMalloc/IOFree transactions with + * the kernel, we create vmem bucket arenas with a PAGESIZE or + * bigger quantum, and a minimum import that is several pages + * for small bucket sizes, and twice the bucket size. + * These will serve power-of-two sized blocks to the + * bucket_heap arena. + */ + vmem_t *b = vmem_create(buf, NULL, 0, + // MAX(heap_quantum, bucket_largest_size), + heap_quantum, + xnu_alloc_throttled, xnu_free_throttled, + spl_default_arena_parent, + MAX(heap_quantum * 8, bucket_largest_size * 2), + VM_SLEEP | VMC_POPULATOR | VMC_NO_QCACHE | VMC_TIMEFREE); + VERIFY(b != NULL); + b->vm_source = b; + vmem_bucket_arena[bucket_number] = b; + vmem_bucket_id_to_bucket_number[b->vm_id] = bucket_number; + } + + vmem_free_impl(spl_default_arena, buf, VMEM_NAMELEN + 21); + // spl_heap_arena, the bucket heap, is the primary interface + // to the vmem system + + // all arenas not rooted to vmem_metadata will be rooted to + // spl_heap arena. + + spl_heap_arena = vmem_create("bucket_heap", // id 15 + NULL, 0, heap_quantum, + vmem_bucket_alloc, vmem_bucket_free, spl_default_arena_parent, 0, + VM_SLEEP | VMC_TIMEFREE | VMC_OLDFIRST); + + VERIFY(spl_heap_arena != NULL); + + // add a fixed-sized allocation to spl_heap_arena; this reduces the + // need to talk to the bucket arenas by a substantial margin + // (kstat.vmem.vmem.bucket_heap.{alloc+free} is much greater than + // kstat.vmem.vmem.bucket_heap.parent_{alloc+free}, and improves with + // increasing initial fixed allocation size. + + const size_t mib = 1024ULL * 1024ULL; + const size_t gib = 1024ULL * mib; + size_t resv_size = 128ULL * mib; + extern uint64_t real_total_memory; + + if (real_total_memory >= 4ULL * gib) + resv_size = 256ULL * mib; + if (real_total_memory >= 8ULL * gib) + resv_size = 512ULL * mib; + if (real_total_memory >= 16ULL * gib) + resv_size = gib; + + dprintf("SPL: %s adding fixed allocation of %llu to the bucket_heap\n", + __func__, (uint64_t)resv_size); + + spl_heap_arena_initial_alloc = vmem_add(spl_heap_arena, + vmem_xalloc(spl_default_arena, resv_size, resv_size, + 0, 0, NULL, NULL, VM_SLEEP), + resv_size, VM_SLEEP); + + VERIFY(spl_heap_arena_initial_alloc != NULL); + + spl_heap_arena_initial_alloc_size = resv_size; + + // kstat.vmem.vmem.heap : kmem_cache_alloc() and similar calls + // to handle in-memory datastructures other than abd + + heap = vmem_create(heap_name, // id 16 + NULL, 0, heap_quantum, + vmem_alloc_impl, vmem_free_impl, spl_heap_arena, 0, + VM_SLEEP); + + VERIFY(heap != NULL); + + // Root all the low bandwidth metadata arenas to the default arena. + // The vmem_metadata allocations will all be 32 kiB or larger, + // and the total allocation will generally cap off around 24 MiB. + + vmem_metadata_arena = vmem_create("vmem_metadata", // id 17 + NULL, 0, heap_quantum, vmem_alloc_impl, vmem_free_impl, + spl_default_arena, + 8 * PAGESIZE, VM_SLEEP | VMC_POPULATOR | VMC_NO_QCACHE); + + VERIFY(vmem_metadata_arena != NULL); + + vmem_seg_arena = vmem_create("vmem_seg", // id 18 + NULL, 0, heap_quantum, + vmem_alloc_impl, vmem_free_impl, vmem_metadata_arena, 0, + VM_SLEEP | VMC_POPULATOR); + + VERIFY(vmem_seg_arena != NULL); + + vmem_hash_arena = vmem_create("vmem_hash", // id 19 + NULL, 0, 8, + vmem_alloc_impl, vmem_free_impl, vmem_metadata_arena, 0, + VM_SLEEP); + + VERIFY(vmem_hash_arena != NULL); + + vmem_vmem_arena = vmem_create("vmem_vmem", // id 20 + vmem0, sizeof (vmem0), 1, + vmem_alloc_impl, vmem_free_impl, vmem_metadata_arena, 0, + VM_SLEEP); + + VERIFY(vmem_vmem_arena != NULL); + + // 21 (0-based) vmem_create before this line. - macroized + // NUMBER_OF_ARENAS_IN_VMEM_INIT + for (id = 0; id < vmem_id; id++) { + (void) vmem_xalloc(vmem_vmem_arena, sizeof (vmem_t), + 1, 0, 0, &vmem0[id], &vmem0[id + 1], + VM_NOSLEEP | VM_BESTFIT | VM_PANIC); + } + + dprintf("SPL: starting vmem_update() thread\n"); + vmem_update(&vmem_update_timer); + + return (heap); +} + +struct free_slab { + vmem_t *vmp; + size_t slabsize; + void *slab; + list_node_t next; +}; +static list_t freelist; + +static void vmem_fini_freelist(void *vmp, void *start, size_t size) +{ + struct free_slab *fs; + + MALLOC(fs, struct free_slab *, sizeof (struct free_slab), M_TEMP, + M_WAITOK); + fs->vmp = vmp; + fs->slabsize = size; + fs->slab = start; + list_link_init(&fs->next); + list_insert_tail(&freelist, fs); +} + +void +vmem_free_span_list(void) +{ + int total = 0; + int total_count = 0; + struct free_slab *fs; +// int release = 1; + + while ((fs = list_head(&freelist))) { + total_count++; + total += fs->slabsize; + list_remove(&freelist, fs); + /* + * Commenting out due to BSOD during uninstallation, + * will revisit later. + * + * for (int id = 0; id < VMEM_INITIAL; id++) { + * if (&vmem0[id] == fs->slab) { + * release = 0; + * break; + * } + * } + * + * if (release) + * fs->vmp->vm_source_free(fs->vmp, fs->slab, + * fs->slabsize); + * release = 1; + * + */ + FREE(fs, M_TEMP); + } +} + +static void +vmem_fini_void(void *vmp, void *start, size_t size) +{ +} + +void +vmem_timer_fini() +{ + bsd_timeout_cancel(&vmem_update_timer); +} + +void +vmem_fini(vmem_t *heap) +{ + struct free_slab *fs; + uint64_t total; + + bsd_untimeout(vmem_update, &vmem_update_timer); + + dprintf("SPL: %s: stopped vmem_update. Creating list and walking " + "arenas.\n", __func__); + + /* Create a list of slabs to free by walking the list of allocs */ + list_create(&freelist, sizeof (struct free_slab), + offsetof(struct free_slab, next)); + + /* Walk to list of allocations */ + + /* + * walking with VMEM_REENTRANT causes segment consolidation and + * freeing of spans the freelist contains a list of segments that + * are still allocated at the time of the walk; unfortunately the + * lists cannot be exact without complex multiple passes, locking, + * and a more complex vmem_fini_freelist(). + * + * Walking without VMEM_REENTRANT can produce a nearly-exact list + * of unfreed spans, which Illumos would then free directly after + * the list is complete. + * + * Unfortunately in O3X, that lack of exactness can lead to a panic + * caused by attempting to free to xnu memory that we already freed + * to xnu. Fortunately, we can get a sense of what would have been + * destroyed after the (non-reentrant) walking, and we printf that + * at the end of this function. + */ + + // Walk all still-alive arenas from leaves to the root + + vmem_walk(heap, VMEM_ALLOC | VMEM_REENTRANT, vmem_fini_void, heap); + + vmem_walk(heap, VMEM_ALLOC, vmem_fini_freelist, heap); + + vmem_free_span_list(); + dprintf("SPL: %s destroying heap\n", __func__); + vmem_destroy(heap); // PARENT: spl_heap_arena + + dprintf("SPL: %s: walking spl_heap_arena, aka bucket_heap (pass 1)\n", + __func__); + + vmem_walk(spl_heap_arena, VMEM_ALLOC | VMEM_REENTRANT, vmem_fini_void, + spl_heap_arena); + + dprintf("SPL: %s: calling vmem_xfree(spl_default_arena, ptr, %llu);\n", + __func__, (uint64_t)spl_heap_arena_initial_alloc_size); + + // forcibly remove the initial alloc from spl_heap_arena arena, whether + // or not it is empty. below this point, any activity on + // spl_default_arena other than a non-reentrant(!) walk and a destroy + // is unsafe (UAF or MAF). + // However, all the children of spl_heap_arena should now be destroyed. + + vmem_xfree(spl_default_arena, spl_heap_arena_initial_alloc, + spl_heap_arena_initial_alloc_size); + + dprintf("SPL: %s: walking spl_heap_arena, aka bucket_heap (pass 2)\n", + __func__); + + vmem_walk(spl_heap_arena, VMEM_ALLOC, vmem_fini_freelist, + spl_heap_arena); + vmem_free_span_list(); + + dprintf("SPL: %s: walking bucket arenas...\n", __func__); + + for (int i = VMEM_BUCKET_LOWBIT; i <= VMEM_BUCKET_HIBIT; i++) { + const int bucket = i - VMEM_BUCKET_LOWBIT; + vmem_walk(vmem_bucket_arena[bucket], + VMEM_ALLOC | VMEM_REENTRANT, vmem_fini_void, + vmem_bucket_arena[bucket]); + + vmem_walk(vmem_bucket_arena[bucket], VMEM_ALLOC, + vmem_fini_freelist, vmem_bucket_arena[bucket]); + } + vmem_free_span_list(); + + dprintf("SPL: %s destroying spl_bucket_arenas...", __func__); + for (int32_t i = VMEM_BUCKET_LOWBIT; i <= VMEM_BUCKET_HIBIT; i++) { + vmem_t *vmpt = vmem_bucket_arena[i - VMEM_BUCKET_LOWBIT]; + dprintf(" %llu", (1ULL << i)); + vmem_destroy(vmpt); // parent: spl_default_arena_parent + } + dprintf("\n"); + + dprintf("SPL: %s: walking vmem metadata-related arenas...\n", __func__); + + vmem_walk(vmem_vmem_arena, VMEM_ALLOC | VMEM_REENTRANT, + vmem_fini_void, vmem_vmem_arena); + + vmem_walk(vmem_vmem_arena, VMEM_ALLOC, + vmem_fini_freelist, vmem_vmem_arena); + + vmem_free_span_list(); + + // We should not do VMEM_REENTRANT on vmem_seg_arena or + // vmem_hash_arena or below to avoid causing work in + // vmem_seg_arena and vmem_hash_arena. + + vmem_walk(vmem_seg_arena, VMEM_ALLOC, + vmem_fini_freelist, vmem_seg_arena); + + vmem_free_span_list(); + + vmem_walk(vmem_hash_arena, VMEM_ALLOC, + vmem_fini_freelist, vmem_hash_arena); + vmem_free_span_list(); + + vmem_walk(vmem_metadata_arena, VMEM_ALLOC, + vmem_fini_freelist, vmem_metadata_arena); + + vmem_free_span_list(); + dprintf("SPL: %s walking the root arena (spl_default_arena)...\n", + __func__); + + vmem_walk(spl_default_arena, VMEM_ALLOC, + vmem_fini_freelist, spl_default_arena); + + vmem_free_span_list(); + + dprintf("SPL: %s destroying bucket heap\n", __func__); + // PARENT: spl_default_arena_parent (but depends on buckets) + vmem_destroy(spl_heap_arena); + + // destroying the vmem_vmem arena and any arena afterwards + // requires the use of vmem_destroy_internal(), which does + // not talk to vmem_vmem_arena like vmem_destroy() does. + // dprintf("SPL: %s destroying vmem_vmem_arena\n", __func__); + // vmem_destroy_internal(vmem_vmem_arena); + // parent: vmem_metadata_arena + + // destroying the seg arena means we must no longer + // talk to vmem_populate() + dprintf("SPL: %s destroying vmem_seg_arena\n", __func__); + vmem_destroy(vmem_seg_arena); + + // vmem_hash_arena may be freed-to in vmem_destroy_internal() + // so it should be just before the vmem_metadata_arena. + dprintf("SPL: %s destroying vmem_hash_arena\n", __func__); + vmem_destroy(vmem_hash_arena); // parent: vmem_metadata_arena + vmem_hash_arena = NULL; + + // XXX: if we panic on unload below here due to destroyed mutex, + // vmem_init() will need some reworking (e.g. have + // vmem_metadata_arena talk directly to xnu), or alternatively a + // vmem_destroy_internal_internal() function that does not touch + // vmem_hash_arena will need writing. + + dprintf("SPL: %s destroying vmem_metadata_arena\n", __func__); + vmem_destroy(vmem_metadata_arena); // parent: spl_default_arena + + dprintf("SPL: %s destroying spl_default_arena\n", __func__); + vmem_destroy(spl_default_arena); // parent: spl_default_arena_parent + dprintf("SPL: %s destroying spl_default_arena_parant\n", __func__); + vmem_destroy(spl_default_arena_parent); + + dprintf("SPL: %s destroying vmem_vmem_arena\n", __func__); + vmem_destroy_internal(vmem_vmem_arena); + + dprintf("SPL: arenas removed, now try destroying mutexes... "); + + dprintf("vmem_xnu_alloc_lock "); + mutex_destroy(&vmem_xnu_alloc_lock); + dprintf("vmem_panic_lock "); + mutex_destroy(&vmem_panic_lock); + dprintf("vmem_pushpage_lock "); + mutex_destroy(&vmem_pushpage_lock); + dprintf("vmem_nosleep_lock "); + mutex_destroy(&vmem_nosleep_lock); + dprintf("vmem_sleep_lock "); + mutex_destroy(&vmem_sleep_lock); + dprintf("vmem_segfree_lock "); + mutex_destroy(&vmem_segfree_lock); + dprintf("vmem_list_lock "); + mutex_destroy(&vmem_list_lock); + + dprintf("SPL: %s: walking list of live slabs at time of call to %s\n", + __func__, __func__); + + // annoyingly, some of these should be returned to xnu, but + // we have no idea which have already been freed to xnu, and + // freeing a second time results in a panic. + + /* Now release the list of allocs to built above */ + total = 0; + uint64_t total_count = 0; + while ((fs = list_head(&freelist))) { + total_count++; + total += fs->slabsize; + list_remove(&freelist, fs); + // extern void segkmem_free(vmem_t *, void *, size_t); + // segkmem_free(fs->vmp, fs->slab, fs->slabsize); + FREE(fs, M_TEMP); + } + dprintf("SPL: WOULD HAVE released %llu bytes (%llu spans) from" + " arenas\n", total, total_count); + list_destroy(&freelist); + dprintf("SPL: %s: Brief delay for readability...\n", __func__); + delay(hz); + dprintf("SPL: %s: done!\n", __func__); +} + +/* + * return true if inuse is much smaller than imported + */ +static inline bool +bucket_fragmented(const uint16_t bn, const uint64_t now) +{ + + // early during uptime, just let buckets grow. + + if (now < 600 * hz) + return (false); + + // if there has been no pressure in the past five minutes, + // then we will just let the bucket grow. + + const uint64_t timeout = 5ULL * 60ULL * hz; + + if (spl_free_last_pressure_wrapper() + timeout < now) + return (false); + + const vmem_t *vmp = vmem_bucket_arena[bn]; + + const int64_t imported = + (int64_t)vmp->vm_kstat.vk_mem_import.value.ui64; + const int64_t inuse = + (int64_t)vmp->vm_kstat.vk_mem_inuse.value.ui64; + const int64_t tiny = 64LL*1024LL*1024LL; + const int64_t small = tiny * 2LL; // 128 M + const int64_t medium = small * 2LL; // 256 + const int64_t large = medium * 2LL; // 512 + const int64_t huge = large * 2LL; // 1 G + const int64_t super_huge = huge * 2LL; // 2 + + const int64_t amount_free = imported - inuse; + + if (amount_free <= tiny || imported <= small) + return (false); + + const int64_t percent_free = (amount_free * 100LL) / imported; + + if (percent_free > 75LL) { + return (true); + } else if (imported <= medium) { + return (percent_free >= 50); + } else if (imported <= large) { + return (percent_free >= 33); + } else if (imported <= huge) { + return (percent_free >= 25); + } else if (imported <= super_huge) { + return (percent_free >= 15); + } else { + return (percent_free >= 10); + } +} + +/* + * Return an adjusted number of bytes free in the + * abd_cache_arena (if it exists), for arc_no_grow + * policy: if there's lots of space, don't allow + * arc growth for a while to see if the gap + * between imported and inuse drops. + */ +int64_t +abd_arena_empty_space(void) +{ + extern vmem_t *abd_arena; + + if (abd_arena == NULL) + return (0); + + const int64_t imported = + (int64_t)abd_arena->vm_kstat.vk_mem_import.value.ui64; + const int64_t inuse = + (int64_t)abd_arena->vm_kstat.vk_mem_inuse.value.ui64; + + /* Hide 10% or 1GiB fragmentation from arc_no_grow */ + int64_t headroom = + (imported * 90LL / 100LL) - inuse; + + if (headroom < 1024LL*1024LL*1024LL) + headroom = 0; + + return (headroom); +} + +int64_t +abd_arena_total_size(void) +{ + extern vmem_t *abd_arena; + + if (abd_arena != NULL) + return (abd_arena->vm_kstat.vk_mem_total.value.ui64); + return (0LL); +} + + +/* + * return true if the bucket for size is fragmented + */ +static inline bool +spl_arc_no_grow_impl(const uint16_t b, const size_t size, + const boolean_t buf_is_metadata, kmem_cache_t **kc) +{ + static _Atomic uint8_t frag_suppression_counter[VMEM_BUCKETS] = { 0 }; + + const uint64_t now = zfs_lbolt(); + + const bool fragmented = bucket_fragmented(b, now); + + if (fragmented) { + if (size < 32768) { + // Don't suppress small qcached blocks when the + // qcache size (bucket_262144) is fragmented, + // since they will push everything else towards + // the tails of ARC lists without eating up a large + // amount of space themselves. + return (false); + } + const uint32_t b_bit = (uint32_t)1 << (uint32_t)b; + spl_arc_no_grow_bits |= b_bit; + const uint32_t sup_at_least_every = MIN(b_bit, 255); + const uint32_t sup_at_most_every = MAX(b_bit, 16); + const uint32_t sup_every = MIN(sup_at_least_every, + sup_at_most_every); + if (frag_suppression_counter[b] >= sup_every) { + frag_suppression_counter[b] = 0; + return (true); + } else { + frag_suppression_counter[b]++; + return (false); + } + } else { + const uint32_t b_bit = (uint32_t)1 << (uint32_t)b; + spl_arc_no_grow_bits &= ~b_bit; + } + + return (false); +} + +static inline uint16_t +vmem_bucket_number_arc_no_grow(const size_t size) +{ + // qcaching on arc + if (size < 128*1024) + return (vmem_bucket_number(262144)); + else + return (vmem_bucket_number(size)); +} + +boolean_t +spl_arc_no_grow(size_t size, boolean_t buf_is_metadata, kmem_cache_t **zp) +{ + const uint16_t b = vmem_bucket_number_arc_no_grow(size); + + const bool rv = spl_arc_no_grow_impl(b, size, buf_is_metadata, zp); + + if (rv) { + atomic_inc_64(&spl_arc_no_grow_count); + } + + return ((boolean_t)rv); +} diff --git a/module/os/windows/spl/spl-vnode.c b/module/os/windows/spl/spl-vnode.c new file mode 100644 index 000000000000..035678a3b593 --- /dev/null +++ b/module/os/windows/spl/spl-vnode.c @@ -0,0 +1,2101 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#include +#include +#include +#include + +#include + +#include +#include + +#ifdef DEBUG_IOCOUNT +#include +#endif + +#include + +// #define FIND_MAF + + +/* Counter for unique vnode ID */ +static uint64_t vnode_vid_counter = 6; /* ZFSCTL_INO_SHARES + 1; */ + +/* Total number of active vnodes */ +static uint64_t vnode_active = 0; + +/* The kmem cache for vnodes */ +static kmem_cache_t *vnode_cache = NULL; + +/* List of all vnodes */ +static kmutex_t vnode_all_list_lock; +static list_t vnode_all_list; + +/* list of all getf/releasef active */ +static kmutex_t spl_getf_lock; +static list_t spl_getf_list; + +enum vtype iftovt_tab[16] = { + VNON, VFIFO, VCHR, VNON, VDIR, VNON, VBLK, VNON, + VREG, VNON, VLNK, VNON, VSOCK, VNON, VNON, VBAD, +}; +int vttoif_tab[9] = { + 0, S_IFREG, S_IFDIR, S_IFBLK, S_IFCHR, S_IFLNK, + S_IFSOCK, S_IFIFO, S_IFMT, +}; + +/* + * In a real VFS the filesystem would register the callbacks for + * VNOP_ACTIVE and VNOP_RECLAIM - but here we just call them direct + */ +extern int zfs_vnop_reclaim(struct vnode *); + +int vnode_recycle_int(vnode_t *vp, int flags); + +int +vn_open(char *pnamep, enum zfs_uio_seg seg, int filemode, int createmode, + struct vnode **vpp, enum create crwhy, mode_t umask) +{ + // vfs_context_t *vctx; + int fmode; + int error = 0; + + fmode = filemode; + if (crwhy) + fmode |= O_CREAT; + + return (error); +} + +int +vn_openat(char *pnamep, enum zfs_uio_seg seg, int filemode, int createmode, + struct vnode **vpp, enum create crwhy, + mode_t umask, struct vnode *startvp) +{ + char *path; + // int pathlen = MAXPATHLEN; + int error = 0; + + path = (char *)kmem_zalloc(MAXPATHLEN, KM_SLEEP); + + if (error == 0) { + } + + kmem_free(path, MAXPATHLEN); + return (error); +} + +extern errno_t vnode_rename(const char *, const char *, int, vfs_context_t *); + +errno_t +vnode_rename(const char *from, const char *to, int flags, vfs_context_t *vctx) +{ + /* + * We need proper KPI changes to be able to safely update + * the zpool.cache file. For now, we return EPERM. + */ + return (EPERM); +} + +int +vn_rename(char *from, char *to, enum zfs_uio_seg seg) +{ + // vfs_context_t *vctx; + int error = 0; + + // vctx = vfs_context_create((vfs_context_t)0); + + // error = vnode_rename(from, to, 0, vctx); + + // (void) vfs_context_rele(vctx); + + return (error); +} + +extern errno_t vnode_remove(const char *, int, enum vtype, vfs_context_t *); + +errno_t +vnode_remove(const char *name, int flag, enum vtype type, vfs_context_t *vctx) +{ + /* + * Now that zed ZFS Event Daemon can handle the rename of zpool.cache + * we will silence this limitation, and look in zed.d/config.sync.sh + */ + return (EPERM); +} + + +int +vn_remove(char *fnamep, enum zfs_uio_seg seg, enum rm dirflag) +{ + // vfs_context_t *vctx; + // enum vtype type; + int error = 0; + return (error); +} + +int zfs_vn_rdwr(enum zfs_uio_rw rw, struct vnode *vp, caddr_t base, ssize_t len, + offset_t offset, enum zfs_uio_seg seg, int ioflag, rlim64_t ulimit, + cred_t *cr, ssize_t *residp) +{ + int error = 0; + + if (rw == UIO_READ) { + // error = VNOP_READ(vp, auio, ioflag, vctx); + } else { + // error = VNOP_WRITE(vp, auio, ioflag, vctx); + } + + if (residp) { + // *residp = uio_resid(auio); + } else { + // if (uio_resid(auio) && error == 0) + error = EIO; + } + + return (error); +} + +int +kernel_ioctl(PDEVICE_OBJECT DeviceObject, FILE_OBJECT *FileObject, + long cmd, void *inbuf, uint32_t inlen, + void *outbuf, uint32_t outlen) +{ + // NTSTATUS status; + // PFILE_OBJECT FileObject; + + dprintf("%s: trying to send kernel ioctl %x\n", __func__, cmd); + + IO_STATUS_BLOCK IoStatusBlock; + KEVENT Event; + PIRP Irp; + NTSTATUS Status; + // ULONG Remainder; + PAGED_CODE(); + + /* Build the information IRP */ + KeInitializeEvent(&Event, SynchronizationEvent, FALSE); + Irp = IoBuildDeviceIoControlRequest(cmd, + DeviceObject, + inbuf, + inlen, + outbuf, + outlen, + FALSE, + &Event, + &IoStatusBlock); + if (!Irp) + return (STATUS_NO_MEMORY); + + /* Override verification */ + IoGetNextIrpStackLocation(Irp)->Flags |= SL_OVERRIDE_VERIFY_VOLUME; + + if (FileObject != NULL) + IoGetNextIrpStackLocation(Irp)->FileObject = FileObject; + + /* Do the request */ + Status = IoCallDriver(DeviceObject, Irp); + if (Status == STATUS_PENDING) { + /* Wait for completion */ + KeWaitForSingleObject(&Event, + Executive, + KernelMode, + FALSE, + NULL); + Status = IoStatusBlock.Status; + } + + return (Status); +} + +/* Linux TRIM API */ +int +blk_queue_discard(PDEVICE_OBJECT dev) +{ + STORAGE_PROPERTY_QUERY spqTrim; + spqTrim.PropertyId = (STORAGE_PROPERTY_ID)StorageDeviceTrimProperty; + spqTrim.QueryType = PropertyStandardQuery; + + // DWORD bytesReturned = 0; + DEVICE_TRIM_DESCRIPTOR dtd = { 0 }; + + if (kernel_ioctl(dev, NULL, IOCTL_STORAGE_QUERY_PROPERTY, + &spqTrim, sizeof (spqTrim), &dtd, sizeof (dtd)) == 0) { + return (dtd.TrimEnabled); + } + return (0); // No trim +} + +int +blk_queue_discard_secure(PDEVICE_OBJECT dev) +{ + return (0); // No secure trim +} + +int +blk_queue_nonrot(PDEVICE_OBJECT dev) +{ + STORAGE_PROPERTY_QUERY spqSeekP; + spqSeekP.PropertyId = + (STORAGE_PROPERTY_ID)StorageDeviceSeekPenaltyProperty; + spqSeekP.QueryType = PropertyStandardQuery; + // DWORD bytesReturned = 0; + DEVICE_SEEK_PENALTY_DESCRIPTOR dspd = { 0 }; + if (kernel_ioctl(dev, NULL, IOCTL_STORAGE_QUERY_PROPERTY, + &spqSeekP, sizeof (spqSeekP), &dspd, sizeof (dspd)) == 0) { + return (!dspd.IncursSeekPenalty); + } + return (0); // Not SSD; +} + +int +blkdev_issue_discard_bytes(PDEVICE_OBJECT dev, uint64_t offset, + uint64_t size, uint32_t flags) +{ + int Status = 0; + struct setAttrAndRange { + DEVICE_MANAGE_DATA_SET_ATTRIBUTES dmdsa; + DEVICE_DATA_SET_RANGE range; + } set; + + set.dmdsa.Size = sizeof (DEVICE_MANAGE_DATA_SET_ATTRIBUTES); + set.dmdsa.Action = DeviceDsmAction_Trim; + set.dmdsa.Flags = DEVICE_DSM_FLAG_TRIM_NOT_FS_ALLOCATED; + set.dmdsa.ParameterBlockOffset = 0; + set.dmdsa.ParameterBlockLength = 0; + set.dmdsa.DataSetRangesOffset = + FIELD_OFFSET(struct setAttrAndRange, range); + set.dmdsa.DataSetRangesLength = 1 * sizeof (DEVICE_DATA_SET_RANGE); + + set.range.LengthInBytes = size; + set.range.StartingOffset = offset; + + Status = kernel_ioctl(dev, NULL, + IOCTL_STORAGE_MANAGE_DATA_SET_ATTRIBUTES, + &set, sizeof (set), NULL, 0); + + if (Status == 0) + return (0); // TRIM OK + + // Linux returncodes are negative + return (-Status); +} + + +int +VOP_SPACE(HANDLE h, int cmd, struct flock *fl, int flags, offset_t off, + cred_t *cr, void *ctx) +{ + if (cmd == F_FREESP) { + NTSTATUS Status; + // DWORD ret; + FILE_ZERO_DATA_INFORMATION fzdi; + fzdi.FileOffset.QuadPart = fl->l_start; + fzdi.BeyondFinalZero.QuadPart = fl->l_start + fl->l_len; + + Status = ZwFsControlFile( + h, + NULL, + NULL, + NULL, + NULL, + FSCTL_SET_ZERO_DATA, + &fzdi, sizeof (fzdi), + NULL, + 0); + + return (Status); + } + + return (STATUS_NOT_SUPPORTED); +} + +int +VOP_CLOSE(struct vnode *vp, int flag, int count, offset_t off, + void *cr, void *k) +{ + int error = 0; + + return (error); +} + +int +VOP_FSYNC(struct vnode *vp, int flags, void* unused, void *uused2) +{ + int error = 0; + + return (error); +} + +int +VOP_GETATTR(struct vnode *vp, vattr_t *vap, int flags, void *x3, void *x4) +{ + int error = 0; + return (error); +} + +#if 1 +errno_t VNOP_LOOKUP(struct vnode *, struct vnode **, struct componentname *, + vfs_context_t *); + +errno_t +VOP_LOOKUP(struct vnode *vp, struct vnode **vpp, struct componentname *cn, + vfs_context_t *ct) +{ + return (ENOTSUP); +} +#endif + +#if 0 +extern errno_t VNOP_REMOVE(struct vnode *, struct vnode *, + struct componentname *, int, vfs_context_t); +errno_t +VOP_REMOVE(struct vnode *vp, struct vnode *dp, + struct componentname *cn, int flags, + vfs_context_t ct) +{ + return (VNOP_REMOVE(vp, dp, cn, flags, ct)); +} + + +extern errno_t VNOP_SYMLINK(struct vnode *, struct vnode **, + struct componentname *, struct vnode_attr *, + char *, vfs_context_t); +errno_t +VOP_SYMLINK(struct vnode *vp, struct vnode **vpp, + struct componentname *cn, struct vnode_attr *attr, + char *name, vfs_context_t ct) +{ + return (VNOP_SYMLINK(vp, vpp, cn, attr, name, ct)); +} +#endif + +#undef VFS_ROOT + +extern int VFS_ROOT(mount_t *, struct vnode **, vfs_context_t); +int +spl_vfs_root(mount_t *mount, struct vnode **vp) +{ + *vp = NULL; + return (-1); +} + +void +vfs_mountedfrom(struct mount *vfsp, char *osname) +{ +} + +/* + * DNLC Name Cache Support + */ +struct vnode * +dnlc_lookup(struct vnode *dvp, char *name) +{ + struct componentname cn; + struct vnode *vp = NULL; + + memset(&cn, 0, sizeof (cn)); + + switch (0 /* cache_lookup(dvp, &vp, &cn) */) { + case -1: + break; + case ENOENT: + vp = DNLC_NO_VNODE; + break; + default: + vp = NULL; + } + return (vp); +} + +int +dnlc_purge_vfsp(struct mount *mp, int flags) +{ + return (0); +} + +void +dnlc_remove(struct vnode *vp, char *name) +{ +} + +/* + * + * + */ +void +dnlc_update(struct vnode *vp, char *name, struct vnode *tp) +{ +} + +static int +vnode_fileobject_compare(const void *arg1, const void *arg2) +{ + const vnode_fileobjects_t *node1 = arg1; + const vnode_fileobjects_t *node2 = arg2; + if (node1->fileobject > node2->fileobject) + return (1); + if (node1->fileobject < node2->fileobject) + return (-1); + return (0); +} + +static int +zfs_vnode_cache_constructor(void *buf, void *arg, int kmflags) +{ + vnode_t *vp = buf; + + // So the Windows structs have to be zerod, even though we call + // their setup functions. + memset(vp, 0, sizeof (*vp)); + + mutex_init(&vp->v_mutex, NULL, MUTEX_DEFAULT, NULL); + avl_create(&vp->v_fileobjects, vnode_fileobject_compare, + sizeof (vnode_fileobjects_t), offsetof(vnode_fileobjects_t, + avlnode)); + + ExInitializeResourceLite(&vp->resource); + ExInitializeResourceLite(&vp->pageio_resource); + ExInitializeFastMutex(&vp->AdvancedFcbHeaderMutex); + + return (0); +} + +static void +zfs_vnode_cache_destructor(void *buf, void *arg) +{ + vnode_t *vp = buf; + + ExDeleteResourceLite(&vp->pageio_resource); + ExDeleteResourceLite(&vp->resource); + + avl_destroy(&vp->v_fileobjects); + mutex_destroy(&vp->v_mutex); +} + +int +spl_vnode_init(void) +{ + mutex_init(&spl_getf_lock, NULL, MUTEX_DEFAULT, NULL); + list_create(&spl_getf_list, sizeof (struct spl_fileproc), + offsetof(struct spl_fileproc, f_next)); + mutex_init(&vnode_all_list_lock, NULL, MUTEX_DEFAULT, NULL); + list_create(&vnode_all_list, sizeof (struct vnode), + offsetof(struct vnode, v_list)); + + vnode_cache = kmem_cache_create("zfs_vnode_cache", + sizeof (vnode_t), 0, + zfs_vnode_cache_constructor, + zfs_vnode_cache_destructor, NULL, NULL, + NULL, 0); + + return (0); +} + +void +spl_vnode_fini(void) +{ + // We need to free all delayed vnodes - this can easily go + // wrong, still haven't figured out how to tell Windows + // to let go for a FILEOBJECT. + if (vnode_active > 0) { + vnode_drain_delayclose(1); + if (vnode_active > 0) { + // vnode ages up to 5s. then, we loop all + // still active nodes, mark them dead, and old + // so they are immediately freed, as well as + // go through the tree of fileobjects to free. + + delay(hz*5); + // hardcoded age, see vnode_drain_delayclose + + dprintf("%s: forcing free (this can go wrong)n", + __func__); + struct vnode *rvp; + clock_t then = gethrtime() - SEC2NSEC(6); // hardcoded + + mutex_enter(&vnode_all_list_lock); + for (rvp = list_head(&vnode_all_list); + rvp; + rvp = list_next(&vnode_all_list, rvp)) { + vnode_fileobjects_t *node; + + rvp->v_flags |= VNODE_DEAD|VNODE_FLUSHING; + rvp->v_age = then; + + mutex_enter(&rvp->v_mutex); + while (node = avl_first(&rvp->v_fileobjects)) { + avl_remove(&rvp->v_fileobjects, node); + kmem_free(node, sizeof (*node)); + } + mutex_exit(&rvp->v_mutex); + } + mutex_exit(&vnode_all_list_lock); + } + } + + // age all marked "old", so here's hopin' + vnode_drain_delayclose(1); + + ASSERT(list_empty(&vnode_all_list)); + + mutex_destroy(&vnode_all_list_lock); + list_destroy(&vnode_all_list); + mutex_destroy(&spl_getf_lock); + list_destroy(&spl_getf_list); + + if (vnode_cache) + kmem_cache_destroy(vnode_cache); + vnode_cache = NULL; +} + +#include + +void +cache_purgevfs(mount_t *mp) +{ +} + +dev_t +vnode_specrdev(vnode_t *vp) +{ + return (0); +} + +void +cache_purge(vnode_t *vp) +{ +} + +void +cache_purge_negatives(vnode_t *vp) +{ +} + +int +vnode_removefsref(vnode_t *vp) +{ + return (0); +} + +/* + * getf(int fd) - hold a lock on a file descriptor, to be released by calling + * releasef(). On OSX we will also look up the vnode of the fd for calls + * to spl_vn_rdwr(). + */ +void * +getf(uint64_t fd) +{ + struct spl_fileproc *sfp = NULL; + // HANDLE h; + +#if 1 + struct fileproc *fp = NULL; + // struct vnode *vp; + // uint64_t vid; + + /* + * We keep the "fp" pointer as well, both for unlocking in releasef() + * and used in vn_rdwr(). + */ + + sfp = kmem_alloc(sizeof (*sfp), KM_SLEEP); + if (!sfp) + return (NULL); + + /* + * The f_vnode ptr is used to point back to the "sfp" node itself, + * as it is the only information passed to vn_rdwr. + */ + if (ObReferenceObjectByHandle((HANDLE)fd, 0, 0, KernelMode, + (void **)&fp, 0) != STATUS_SUCCESS) { + dprintf("%s: failed to get fd %d fp 0x\n", __func__, fd); + } + + sfp->f_vnode = sfp; + + sfp->f_handle = fd; + sfp->f_offset = 0; + sfp->f_proc = current_proc(); + sfp->f_fp = (void *)fp; + sfp->f_file = (uint64_t)fp; + + mutex_enter(&spl_getf_lock); + list_insert_tail(&spl_getf_list, sfp); + mutex_exit(&spl_getf_lock); + + // printf("SPL: new getf(%d) ret %p fp is %p so vnode set to %p\n", + // fd, sfp, fp, sfp->f_vnode); +#endif + return (sfp); +} + +struct vnode * +getf_vnode(void *fp) +{ + struct vnode *vp = NULL; +#if 0 + struct spl_fileproc *sfp = (struct spl_fileproc *)fp; + uint32_t vid; + + if (!file_vnode_withvid(sfp->f_fd, &vp, &vid)) { + file_drop(sfp->f_fd); + } +#endif + return (vp); +} + +void +releasefp(struct spl_fileproc *fp) +{ + if (fp->f_fp) + ObDereferenceObject(fp->f_fp); + + /* Remove node from the list */ + mutex_enter(&spl_getf_lock); + list_remove(&spl_getf_list, fp); + mutex_exit(&spl_getf_lock); + + /* Free the node */ + kmem_free(fp, sizeof (*fp)); +} +void +releasef(uint64_t fd) +{ + +#if 1 + struct spl_fileproc *fp = NULL; + struct proc *p = NULL; + + // printf("SPL: releasef(%d)\n", fd); + + p = (void *)current_proc(); + mutex_enter(&spl_getf_lock); + for (fp = list_head(&spl_getf_list); fp != NULL; + fp = list_next(&spl_getf_list, fp)) { + if ((fp->f_proc == p) && fp->f_fd == fd) + break; + } + mutex_exit(&spl_getf_lock); + if (!fp) + return; // Not found + + // printf("SPL: releasing %p\n", fp); + releasefp(fp); + +#endif +} + +/* + * Our version of vn_rdwr, here "vp" is not actually a vnode, but a ptr + * to the node allocated in getf(). We use the "fp" part of the node to + * be able to issue IO. + * You must call getf() before calling spl_vn_rdwr(). + */ +int +spl_vn_rdwr(enum zfs_uio_rw rw, + struct vnode *vp, + caddr_t base, + ssize_t len, + offset_t offset, + enum zfs_uio_seg seg, + int ioflag, + rlim64_t ulimit, /* meaningful only if rw is UIO_WRITE */ + cred_t *cr, + ssize_t *residp) +{ + struct spl_fileproc *sfp = (struct spl_fileproc *)vp; + int error = 0; + IO_STATUS_BLOCK iob; + LARGE_INTEGER off; + + off.QuadPart = offset; + + if (rw == UIO_READ) { + error = ZwReadFile((HANDLE)sfp->f_fd, NULL, NULL, NULL, &iob, + base, (ULONG)len, &off, NULL); + } else { + error = ZwWriteFile((HANDLE)sfp->f_fd, NULL, NULL, NULL, &iob, + base, (ULONG)len, &off, NULL); + sfp->f_writes = 1; + } + + if (residp) { + *residp = len - iob.Information; + } else { + if ((iob.Information < len) && error == 0) + error = EIO; + } + + return (error); +} + +void +spl_rele_async(void *arg) +{ + struct vnode *vp = (struct vnode *)arg; +#ifdef DEBUG_IOCOUNT + if (vp) { + znode_t *zp = VTOZ(vp); + if (zp) + dprintf("%s: Dec iocount from %u for '%s' \n", __func__, + &vp->v_iocount, + zp->z_name_cache); + } +#endif + if (vp) VN_RELE(vp); +} + +void +vn_rele_async(struct vnode *vp, void *taskq) +{ +#ifdef DEBUG_IOCOUNT + if (vp) { + znode_t *zp = VTOZ(vp); + if (zp) + dprintf("%s: Dec iocount in future, now %u for '%s' \n", + __func__, vp->v_iocount, zp->z_name_cache); + } +#endif + VERIFY(taskq_dispatch((taskq_t *)taskq, + (task_func_t *)spl_rele_async, vp, TQ_SLEEP) != 0); +} + +vfs_context_t * +spl_vfs_context_kernel(void) +{ + return (NULL); +} + +extern int build_path(struct vnode *vp, char *buff, int buflen, int *outlen, + int flags, vfs_context_t *ctx); + +int +spl_build_path(struct vnode *vp, char *buff, int buflen, int *outlen, + int flags, vfs_context_t *ctx) +{ + return (0); +} + +/* + * vnode_notify was moved from KERNEL_PRIVATE to KERNEL in 10.11, but to be + * backward compatible, we keep the wrapper for now. + */ +extern int vnode_notify(struct vnode *, uint32_t, struct vnode_attr *); +int +spl_vnode_notify(struct vnode *vp, uint32_t type, struct vnode_attr *vap) +{ + return (0); +} + +extern intvfs_get_notify_attributes(struct vnode_attr *vap); +int +spl_vfs_get_notify_attributes(struct vnode_attr *vap) +{ + return (0); +} + +/* Root directory vnode for the system a.k.a. '/' */ +/* + * Must use vfs_rootvnode() to acquire a reference, and + * vnode_put() to release it + */ + +/* + * From early boot (mountroot) we can not call vfs_rootvnode() + * or it will panic. So the default here is to return NULL until + * root has been mounted. XNU will call vfs_root() once that is + * done, so we use that to inform us that root is mounted. In nonboot, + * vfs_start is called early from kextload (zfs_osx.cpp). + */ +static int spl_skip_getrootdir = 1; + +struct vnode * +getrootdir(void) +{ + struct vnode *rvnode = NULL; + if (spl_skip_getrootdir) + return (NULL); + + return (rvnode); +} + +void +spl_vfs_start() +{ + spl_skip_getrootdir = 0; +} + +int +vnode_vfsisrdonly(vnode_t *vp) +{ + return (0); +} + +uint64_t +vnode_vid(vnode_t *vp) +{ + return (vp->v_id); +} + +int +vnode_isreg(vnode_t *vp) +{ + return (vp->v_type == VREG); +} + +int +vnode_isdir(vnode_t *vp) +{ + return (vp->v_type == VDIR); +} + +void * +vnode_fsnode(struct vnode *dvp) +{ + return (dvp->v_data); +} + +enum vtype +vnode_vtype(vnode_t *vp) +{ + return (vp->v_type); +} + +int +vnode_isblk(vnode_t *vp) +{ + return (vp->v_type == VBLK); +} + +int +vnode_ischr(vnode_t *vp) +{ + return (vp->v_type == VCHR); +} + +int +vnode_isswap(vnode_t *vp) +{ + return (0); +} + +int +vnode_isfifo(vnode_t *vp) +{ + return (0); +} + +int +vnode_islnk(vnode_t *vp) +{ + return (0); +} + +mount_t * +vnode_mountedhere(vnode_t *vp) +{ + return (NULL); +} + +void +ubc_setsize(struct vnode *vp, uint64_t size) +{ +} + +int +vnode_isinuse(vnode_t *vp, uint64_t refcnt) +{ + // xnu uses usecount +kusecount, not iocount + if (((vp->v_usecount /* + vp->v_iocount */) > refcnt)) + return (1); + return (0); +} + +int +vnode_isidle(vnode_t *vp) +{ + if ((vp->v_usecount == 0) && (vp->v_iocount <= 1)) + return (1); + return (0); +} + +int +vnode_iocount(vnode_t *vp) +{ + return (vp->v_iocount); +} + +#ifdef DEBUG_IOCOUNT +int +vnode_getwithref(vnode_t *vp, char *file, int line) +#else +int +vnode_getwithref(vnode_t *vp) +#endif +{ + // KIRQL OldIrql; + int error = 0; +#ifdef FIND_MAF + ASSERT(!(vp->v_flags & 0x8000)); +#endif + + mutex_enter(&vp->v_mutex); + if ((vp->v_flags & VNODE_DEAD)) { + error = ENOENT; +// } else if (vnode_deleted(vp)) { +// error = ENOENT; + } else { +#ifdef DEBUG_IOCOUNT + if (vp) { + znode_t *zp = VTOZ(vp); + if (zp) + dprintf("%s: Inc iocount now %u for '%s' " + "(%s:%d) thread %p \n", __func__, + atomic_inc_32_nv(&vp->v_iocount), + zp->z_name_cache, + file, line, current_thread()); + } +#else + atomic_inc_32(&vp->v_iocount); +#endif + } + + mutex_exit(&vp->v_mutex); + return (error); +} + +#ifdef DEBUG_IOCOUNT +int +vnode_debug_getwithvid(vnode_t *vp, uint64_t id, char *file, int line) +#else +int +vnode_getwithvid(vnode_t *vp, uint64_t id) +#endif +{ + // KIRQL OldIrql; + int error = 0; + +#ifdef FIND_MAF + ASSERT(!(vp->v_flags & 0x8000)); +#endif + + mutex_enter(&vp->v_mutex); + if ((vp->v_flags & VNODE_DEAD)) { + error = ENOENT; + } else if (id != vp->v_id) { + error = ENOENT; +// } else if (vnode_deleted(vp)) { +// error = ENOENT; + } else { +#ifdef DEBUG_IOCOUNT + if (vp) { + znode_t *zp = VTOZ(vp); + if (zp) + dprintf("%s: Inc iocount now %u for '%s' " + "(%s:%d) thread %p\n", __func__, + atomic_inc_32_nv(&vp->v_iocount), + zp->z_name_cache, file, line, + current_thread()); + } +#else + atomic_inc_32(&vp->v_iocount); +#endif + } + + mutex_exit(&vp->v_mutex); + return (error); +} + +extern void zfs_inactive(vnode_t *vp, cred_t *cr, caller_context_t *ct); + +#ifdef DEBUG_IOCOUNT +int +vnode_put(vnode_t *vp, char *file, int line) +#else +int +vnode_put(vnode_t *vp) +#endif +{ + // KIRQL OldIrql; + int calldrain = 0; + ASSERT(!(vp->v_flags & VNODE_DEAD)); + ASSERT(vp->v_iocount > 0); + ASSERT((vp->v_flags & ~VNODE_VALIDBITS) == 0); +#ifdef DEBUG_IOCOUNT + if (vp) { + znode_t *zp = VTOZ(vp); + if (zp) + dprintf("%s: Dec iocount now %u for '%s' (%s:%d) " + "thread %p \n", __func__, + atomic_dec_32_nv(&vp->v_iocount), + zp->z_name_cache, file, line, current_thread()); + } +#else + atomic_dec_32(&vp->v_iocount); +#endif + // Now idle? + mutex_enter(&vp->v_mutex); + + if (vp->v_iocount == 0) { + + if (vp->v_usecount == 0) + calldrain = 1; + + if (vp->v_flags & VNODE_NEEDINACTIVE) { + vp->v_flags &= ~VNODE_NEEDINACTIVE; + mutex_exit(&vp->v_mutex); + zfs_inactive(vp, NULL, NULL); + mutex_enter(&vp->v_mutex); + } + } + + vp->v_flags &= ~VNODE_NEEDINACTIVE; + +#if 0 + // Re-test for idle, as we may have dropped lock for inactive + if ((vp->v_usecount == 0) && (vp->v_iocount == 0)) { + // Was it marked TERM, but we were waiting for last ref + if ((vp->v_flags & VNODE_MARKTERM)) { + KeReleaseSpinLock(&vp->v_spinlock, OldIrql); + vnode_recycle_int(vp, 0); // OldIrql is lost! + return (0); + } + } +#endif + mutex_exit(&vp->v_mutex); + + // Temporarily - should perhaps be own thread? + // if (calldrain) + // vnode_drain_delayclose(0); + + return (0); +} + +int +vnode_recycle_int(vnode_t *vp, int flags) +{ + // KIRQL OldIrql; + ASSERT((vp->v_flags & VNODE_DEAD) == 0); + + // Mark it for recycle, if we are not ROOT. + if (!(vp->v_flags&VNODE_MARKROOT)) { + if (vp->v_flags & VNODE_MARKTERM) + dprintf("already marked\n"); + vp->v_flags |= VNODE_MARKTERM; // Mark it terminating + dprintf("%s: marking %p VNODE_MARKTERM\n", __func__, vp); + } + + // Already locked calling in... + if (!(flags & VNODELOCKED)) { + mutex_enter(&vp->v_mutex); + } + + // Doublecheck CcMgr is gone (should be if avl is empty) + // If it hasn't quite let go yet, let the node linger on deadlist. +#if 1 + if (vp->SectionObjectPointers.DataSectionObject != NULL || + vp->SectionObjectPointers.ImageSectionObject != NULL || + vp->SectionObjectPointers.SharedCacheMap != NULL) { + dprintf("%s: %p still has CcMgr, lingering on dead list.\n", + __func__, vp); + mutex_exit(&vp->v_mutex); + return (-1); + } +#endif + + // We will only reclaim idle nodes, and not mountpoints(ROOT) + if ((flags & FORCECLOSE) || + ((vp->v_usecount == 0) && + (vp->v_iocount <= 1) && + avl_is_empty(&vp->v_fileobjects) && + ((vp->v_flags&VNODE_MARKROOT) == 0))) { + + ASSERT3P(vp->SectionObjectPointers.DataSectionObject, ==, NULL); + ASSERT3P(vp->SectionObjectPointers.ImageSectionObject, ==, + NULL); + ASSERT3P(vp->SectionObjectPointers.SharedCacheMap, ==, NULL); + + vp->v_flags |= VNODE_DEAD; // Mark it dead +// Since we might get swapped out (noticably FsRtlTeardownPerStreamContexts) +// we hold a look until the very end. + vp->v_iocount = 1; + + mutex_exit(&vp->v_mutex); + + FsRtlTeardownPerStreamContexts(&vp->FileHeader); + FsRtlUninitializeFileLock(&vp->lock); + + // Call sync? If vnode_write + // zfs_fsync(vp, 0, NULL, NULL); + + // Call inactive? + if (vp->v_flags & VNODE_NEEDINACTIVE) { + vp->v_flags &= ~VNODE_NEEDINACTIVE; + zfs_inactive(vp, NULL, NULL); + } + + + // Tell FS to release node. + if (zfs_vnop_reclaim(vp)) + panic("vnode_recycle: cannot reclaim\n"); + + // KIRQL OldIrql; + mutex_enter(&vp->v_mutex); + ASSERT(avl_is_empty(&vp->v_fileobjects)); + // We are all done with it. + vp->v_iocount = 0; + mutex_exit(&vp->v_mutex); + +#ifdef FIND_MAF + vp->v_flags |= 0x8000; +#endif + +/* + * Windows has a habit of copying FsContext (vp) without our knowledge and + * attempt to call fsDispatcher. We notice in vnode_getwithref(), which + * calls mutex_enter so we can not free the vp right here like we want to, + * or that would be a MAF. + * So we let it linger and age, there is no great way to know for sure that it + * has finished trying. + */ + dprintf("vp %p left on DEAD list\n", vp); + vp->v_age = gethrtime(); + + return (0); + } + + mutex_exit(&vp->v_mutex); + return (-1); +} + + +int +vnode_recycle(vnode_t *vp) +{ + if (vp->v_flags & VNODE_FLUSHING) + return (-1); + return (vnode_recycle_int(vp, 0)); +} + +typedef struct { + FSRTL_COMMON_FCB_HEADER Header; + PFAST_MUTEX FastMutex; + LIST_ENTRY FilterContexts; + EX_PUSH_LOCK PushLock; + PVOID *FileContextSupportPointer; + union { + OPLOCK Oplock; + PVOID ReservedForRemote; + }; + PVOID ReservedContext; +} FSRTL_ADVANCED_FCB_HEADER_NEW; + +POPLOCK +vp_oplock(struct vnode *vp) +{ + // The oplock in header starts with Win8 + if (vp->FileHeader.Version >= FSRTL_FCB_HEADER_V2) + return (&((FSRTL_ADVANCED_FCB_HEADER_NEW *)&vp-> + FileHeader)->Oplock); + else + return (&vp->oplock); +} + +void +vnode_create(mount_t *mp, void *v_data, int type, int flags, + struct vnode **vpp) +{ + struct vnode *vp; + // cache_alloc does not zero the struct, we need to + // make sure that those things that need clearing is + // done here. + vp = kmem_cache_alloc(vnode_cache, KM_SLEEP); + *vpp = vp; + vp->v_flags = 0; + vp->v_mount = mp; + vp->v_data = v_data; + vp->v_type = type; + vp->v_id = atomic_inc_64_nv(&(vnode_vid_counter)); + vp->v_iocount = 1; + vp->v_usecount = 0; + vp->v_unlink = 0; + vp->v_reparse = NULL; + vp->v_reparse_size = 0; + + atomic_inc_64(&vnode_active); + + list_link_init(&vp->v_list); + ASSERT(vnode_fileobject_empty(vp, 1)); // lying about locked is ok. + + if (flags & VNODE_MARKROOT) + vp->v_flags |= VNODE_MARKROOT; + + + // Initialise the Windows specific data. + memset(&vp->SectionObjectPointers, 0, + sizeof (vp->SectionObjectPointers)); + + FsRtlSetupAdvancedHeader(&vp->FileHeader, &vp->AdvancedFcbHeaderMutex); + + FsRtlInitializeFileLock(&vp->lock, NULL, NULL); + FsRtlInitializeOplock(vp_oplock(vp)); + + vp->FileHeader.Resource = &vp->resource; + vp->FileHeader.PagingIoResource = &vp->pageio_resource; + + // Add only to list once we have finished initialising. + mutex_enter(&vnode_all_list_lock); + list_insert_tail(&vnode_all_list, vp); + mutex_exit(&vnode_all_list_lock); +} + +int +vnode_isvroot(vnode_t *vp) +{ + return (vp->v_flags & VNODE_MARKROOT); +} + +mount_t * +vnode_mount(vnode_t *vp) +{ + return (vp->v_mount); +} + +void +vnode_clearfsnode(vnode_t *vp) +{ + vp->v_data = NULL; +} + +void * +vnode_sectionpointer(vnode_t *vp) +{ + return (&vp->SectionObjectPointers); +} + +int +vnode_ref(vnode_t *vp) +{ + ASSERT(vp->v_iocount > 0); + ASSERT(!(vp->v_flags & VNODE_DEAD)); + atomic_inc_32(&vp->v_usecount); + return (0); +} + +void +vnode_rele(vnode_t *vp) +{ + // KIRQL OldIrql; + + ASSERT(!(vp->v_flags & VNODE_DEAD)); + ASSERT(vp->v_iocount > 0); + ASSERT(vp->v_usecount > 0); + atomic_dec_32(&vp->v_usecount); + + // Grab lock and inspect + mutex_enter(&vp->v_mutex); + + // If we were the last usecount, but vp is still + // busy, we set NEEDINACTIVE + if (vp->v_usecount > 0 || vp->v_iocount > 0) { + vp->v_flags |= VNODE_NEEDINACTIVE; + } else { + // We are idle, call inactive, grab a hold + // so we can call inactive unlocked + vp->v_flags &= ~VNODE_NEEDINACTIVE; + mutex_exit(&vp->v_mutex); + atomic_inc_32(&vp->v_iocount); + + zfs_inactive(vp, NULL, NULL); +#ifdef DEBUG_VERBOSE + if (vp) { + znode_t *zp = VTOZ(vp); + if (zp) + dprintf("%s: Inc iocount to %u for %s \n", + __func__, vp->v_iocount, zp->z_name_cache); + } +#endif + atomic_dec_32(&vp->v_iocount); + // Re-check we are still free, and recycle (markterm) was called + // we can reclaim now + mutex_enter(&vp->v_mutex); + if ((vp->v_iocount == 0) && (vp->v_usecount == 0) && + ((vp->v_flags & (VNODE_MARKTERM)))) { + mutex_exit(&vp->v_mutex); + vnode_recycle_int(vp, 0); + return; + } + } + + mutex_exit(&vp->v_mutex); +} + +/* + * Periodically walk through list and release vnodes that are now idle. + * Set force=1 to perform check now. + * Will return number of vnodes with delete set, but not yet reclaimed. + */ +int +vnode_drain_delayclose(int force) +{ + struct vnode *vp, *next = NULL; + int ret = 0; + int candidate = 0; + static hrtime_t last = 0; + const hrtime_t interval = SEC2NSEC(2); + const hrtime_t curtime = gethrtime(); + + mutex_enter(&vnode_all_list_lock); + // This should probably be its own thread, but for now, run every 2s + if (!force && curtime - last < interval) { + mutex_exit(&vnode_all_list_lock); + return (0); + } + last = curtime; + + TraceEvent(8, "%s: scanning\n", __func__); + + for (vp = list_head(&vnode_all_list); + vp; + vp = next) { + + next = list_next(&vnode_all_list, vp); + + // Make sure everything about the vp has been released. + vnode_lock(vp); + + // If we see a deleted node awaiting recycle, signal return code + if ((vp->v_flags & VNODE_MARKTERM)) + candidate = 1; + else + candidate = 0; + + if ((vp->v_flags & VNODE_MARKTERM) && + !(vp->v_flags & VNODE_DEAD) && + (vp->v_iocount == 0) && + (vp->v_usecount == 0) && + vnode_fileobject_empty(vp, /* locked */ 1) && + !vnode_isvroot(vp) && + (vp->SectionObjectPointers.ImageSectionObject == NULL) && + (vp->SectionObjectPointers.DataSectionObject == NULL)) { + // We are ready to let go + dprintf("%s: drain %p\n", __func__, vp); + +// Pass VNODELOCKED as we hold vp, recycle will unlock. +// We have to give up all_list due to +// recycle -> reclaim -> rmnode -> purgedir -> zget -> vnode_create + mutex_exit(&vnode_all_list_lock); + if (vnode_recycle_int(vp, VNODELOCKED) == 0) + candidate = 0; +// If recycle was ok, this isnt a node we wait for + + mutex_enter(&vnode_all_list_lock); + + // If successful, vp is freed. Do not use vp from here: + + } else if ((vp->v_flags & VNODE_DEAD) && + (vp->v_age != 0) && + (curtime - vp->v_age > SEC2NSEC(5))) { +// Arbitrary time! fixme? It would be nice to know when Windows really +// wont try this vp again. fastfat seems to clear up the cache of the +// parent directory, perhaps this is the missing bit. It is non-trivial +// to get parent from here though. + + // dprintf("age is %llu %d\n", (curtime - vp->v_age), + // NSEC2SEC(curtime - vp->v_age)); + + // Finally free vp. + list_remove(&vnode_all_list, vp); + vnode_unlock(vp); + dprintf("%s: freeing DEAD vp %p\n", __func__, vp); + + kmem_cache_free(vnode_cache, vp); + atomic_dec_64(&vnode_active); + + } else { + vnode_unlock(vp); + } + + if (candidate) ret++; + } + mutex_exit(&vnode_all_list_lock); + + return (ret); +} + +int +mount_count_nodes(struct mount *mp, int flags) +{ + int count = 0; + struct vnode *rvp; + + mutex_enter(&vnode_all_list_lock); + for (rvp = list_head(&vnode_all_list); + rvp; + rvp = list_next(&vnode_all_list, rvp)) { + if (rvp->v_mount != mp) + continue; + if ((flags&SKIPROOT) && vnode_isvroot(rvp)) + continue; + // if (rvp->v_flags&VNODE_DEAD) + // continue; + // if (rvp->has_uninit) + // continue; + count++; + } + mutex_exit(&vnode_all_list_lock); + return (count); +} + + +/* + * Let's try something new. If we are to vflush, lets do everything we can + * then release the znode struct, and leave vnode with a NULL ptr, marked + * dead. Future access to vnode will be refused. Move the vnode from + * the mount's list, onto a deadlist. Only stop module unload + * until deadlist is empty. + */ +int +vflush(struct mount *mp, struct vnode *skipvp, int flags) +{ + // Iterate the vnode list and call reclaim + // flags: + // SKIPROOT : dont release root nodes (mountpoints) + // SKIPSYSTEM : dont release vnodes marked as system + // FORCECLOSE : release everything, force unmount + + // if mp is NULL, we are reclaiming nodes, until threshold + int isbusy = 0; + int reclaims = 0; + vnode_fileobjects_t *node; + struct vnode *rvp; + int Status; + + dprintf("vflush start\n"); + + mutex_enter(&vnode_all_list_lock); + while (1) { + for (rvp = list_head(&vnode_all_list); + rvp; + rvp = list_next(&vnode_all_list, rvp)) { + + // skip vnodes not belonging to this mount + if (mp && rvp->v_mount != mp) + continue; + + // If we aren't FORCE and asked to SKIPROOT, and node + // is MARKROOT, then go to next. + if (!(flags & FORCECLOSE)) + if ((flags & SKIPROOT)) + if (rvp->v_flags & VNODE_MARKROOT) + continue; + + // We are to remove this node, even if ROOT - unmark it. + mutex_exit(&vnode_all_list_lock); + + // Release the AVL tree + // KIRQL OldIrql; + + // Attempt to flush out any caches; + mutex_enter(&rvp->v_mutex); + // Make sure we don't call vnode_cacheflush() again + // from IRP_MJ_CLOSE. + rvp->v_flags |= VNODE_FLUSHING; + + for (node = avl_first(&rvp->v_fileobjects); + node != NULL; + node = AVL_NEXT(&rvp->v_fileobjects, node)) { + FILE_OBJECT *fileobject = node->fileobject; + + // Because the CC* calls can re-enter ZFS, we need to + // release the lock, and because we release the lock the + // while has to start from the top each time. We release + // the node at end of this while. + + try { + Status = ObReferenceObjectByPointer( + fileobject, + 0, + *IoFileObjectType, + KernelMode); + } except(EXCEPTION_EXECUTE_HANDLER) { + Status = GetExceptionCode(); + } + + // Try to lock fileobject before we use it. + if (NT_SUCCESS(Status)) { + int ok; + + // Let go of mutex, as flushcache will re-enter + // (IRP_MJ_CLEANUP) + mutex_exit(&rvp->v_mutex); + node->remove = vnode_flushcache(rvp, + fileobject, TRUE); + + ObDereferenceObject(fileobject); + + mutex_enter(&rvp->v_mutex); + + } // if ObReferenceObjectByPointer + } // for + + // Remove any nodes we successfully closed. +restart: + for (node = avl_first(&rvp->v_fileobjects); + node != NULL; + node = AVL_NEXT(&rvp->v_fileobjects, node)) { + if (node->remove) { + avl_remove(&rvp->v_fileobjects, node); + kmem_free(node, sizeof (*node)); + goto restart; + } + } + + dprintf("vp %p has %d fileobject(s) remaining\n", rvp, + avl_numnodes(&rvp->v_fileobjects)); + + // vnode_recycle_int() will call mutex_exit(&rvp->v_mutex); + // re-check flags, due to releasing locks + isbusy = 1; + if (!(rvp->v_flags & VNODE_DEAD)) + isbusy = vnode_recycle_int(rvp, + (flags & FORCECLOSE) | VNODELOCKED); + else + mutex_exit(&rvp->v_mutex); + + mutex_enter(&vnode_all_list_lock); + + if (!isbusy) { + reclaims++; + break; // must restart loop if unlinked node + } + } + + // If the end of the list was reached, stop entirely + if (!rvp) + break; + } + + mutex_exit(&vnode_all_list_lock); + + if (mp == NULL && reclaims > 0) { + dprintf("%s: %llu reclaims processed.\n", __func__, reclaims); + } + + + kpreempt(KPREEMPT_SYNC); + + /* + * Process all remaining nodes, release znode, and set vnode to NULL + * move to dead list. + */ + int deadlist = 0; + mutex_enter(&vnode_all_list_lock); + for (rvp = list_head(&vnode_all_list); + rvp; + rvp = list_next(&vnode_all_list, rvp)) { + if (rvp->v_mount == mp) { + if (rvp->v_data) { + deadlist++; + // mutex_exit(&vnode_all_list_lock); + zfs_vnop_reclaim(rvp); + // mutex_enter(&vnode_all_list_lock); + // Also empty fileobjects + while (node = avl_first(&rvp->v_fileobjects)) { + avl_remove(&rvp->v_fileobjects, node); + kmem_free(node, sizeof (*node)); + } + } else { + rvp->v_age = gethrtime() - SEC2NSEC(6); + } + rvp->v_flags |= VNODE_DEAD; + rvp->v_data = NULL; + } + } + mutex_exit(&vnode_all_list_lock); + + if (FORCECLOSE) + vnode_drain_delayclose(1); + + xprintf("vflush end: deadlisted %d nodes\n", deadlist); + + return (0); +} + +/* + * Set the Windows SecurityPolicy + */ +void +vnode_setsecurity(vnode_t *vp, void *sd) +{ + vp->security_descriptor = sd; +} + +void * +vnode_security(vnode_t *vp) +{ + return (vp->security_descriptor); +} + +extern CACHE_MANAGER_CALLBACKS CacheManagerCallbacks; + +void +vnode_couplefileobject(vnode_t *vp, FILE_OBJECT *fileobject, uint64_t size) +{ + if (fileobject) { + + fileobject->FsContext = vp; + + // Make sure it is pointing to the right vp. + if (fileobject->SectionObjectPointer != NULL) + VERIFY3P(vnode_sectionpointer(vp), ==, fileobject-> + SectionObjectPointer); + + if (fileobject->SectionObjectPointer != + vnode_sectionpointer(vp)) { + fileobject->SectionObjectPointer = + vnode_sectionpointer(vp); + } + + // If this fo's CcMgr hasn't been initialised, do so now + // this ties each fileobject to CcMgr, it is not about + // the vp itself. CcInit will be called many times on a vp, + // once for each fileobject. + dprintf("%s: vp %p fo %p\n", __func__, vp, fileobject); + + // Add this fileobject to the list of known ones. + vnode_fileobject_add(vp, fileobject); + + if (vnode_isvroot(vp)) + return; + + vnode_pager_setsize(fileobject, vp, size, FALSE); + + } +} + +// Attempt to boot CcMgr out of the fileobject, return +// true if we could +int +vnode_flushcache(vnode_t *vp, FILE_OBJECT *fileobject, boolean_t hard) +{ + CACHE_UNINITIALIZE_EVENT UninitializeCompleteEvent; + // NTSTATUS WaitStatus; + LARGE_INTEGER Zero = { .QuadPart = 0 }; + int ret = 1; + + if (vp == NULL) + return (1); + + if (fileobject == NULL) + return (1); + + // Have CcMgr already released it? + if (fileobject->SectionObjectPointer == NULL) + return (1); + + if (FlagOn(fileobject->Flags, FO_CLEANUP_COMPLETE)) { + // return 1; + } + + if (avl_numnodes(&vp->v_fileobjects) > 1) { + dprintf("warning, has other fileobjects: %d\n", + avl_numnodes(&vp->v_fileobjects)); + } + + int lastclose = 0; + + if (vp->v_iocount <= 1 && vp->v_usecount == 0) + lastclose = 1; + +// Because CcUninitializeCacheMap() can call MJ_CLOSE immediately, and we +// don't want to free anything in *that* call, take a usecount++ here, that +// way we skip the vnode_isinuse() test + atomic_inc_32(&vp->v_usecount); + + if (fileobject->SectionObjectPointer->ImageSectionObject) { + if (hard) + (void) MmForceSectionClosed( + fileobject->SectionObjectPointer, TRUE); + else + (void) MmFlushImageSection( + fileobject->SectionObjectPointer, MmFlushForWrite); + } + + if (lastclose && FlagOn(fileobject->Flags, FO_CACHE_SUPPORTED)) { + // DataSection next + if (fileobject->SectionObjectPointer->DataSectionObject) { + CcFlushCache(fileobject->SectionObjectPointer, NULL, 0, + NULL); + ExAcquireResourceExclusiveLite( + vp->FileHeader.PagingIoResource, TRUE); + ExReleaseResourceLite(vp->FileHeader.PagingIoResource); + } + + CcPurgeCacheSection(fileobject->SectionObjectPointer, NULL, + 0, hard); + +#if 0 + if (vp->has_uninit > 3) { + NTSTATUS ntstatus; + IO_STATUS_BLOCK IoStatus; + ntstatus = ZwDeviceIoControlFile(fileobject, 0, 0, 0, + &IoStatus, FSCTL_DISMOUNT_VOLUME, 0, 0, 0, 0); + dprintf("Said 0x%x\n", ntstatus); + ntstatus = ZwDeviceIoControlFile(fileobject, 0, 0, 0, + &IoStatus, IRP_MN_REMOVE_DEVICE, 0, 0, 0, 0); + dprintf("Said 0x%x\n", ntstatus); + } +#endif + + } + + if (!hard && avl_numnodes(&vp->v_fileobjects) > 1) { + // dprintf("leaving early due to v_fileobjects > 1 - flush only\n"); + // ret = 0; + // goto out; + } + + if (fileobject->PrivateCacheMap == NULL) { + KeInitializeEvent(&UninitializeCompleteEvent.Event, + SynchronizationEvent, + FALSE); + + // Try to release cache + TraceEvent(8, "calling CcUninit: fo %p\n", fileobject); + CcUninitializeCacheMap(fileobject, + hard ? &Zero : NULL, + NULL); + TraceEvent(8, "complete CcUninit\n"); + } + + ret = 1; + if (fileobject && fileobject->SectionObjectPointer) + if ((fileobject->SectionObjectPointer->ImageSectionObject != + NULL) || + (fileobject->SectionObjectPointer->DataSectionObject != + NULL) || + (fileobject->SectionObjectPointer->SharedCacheMap != + NULL)) { + ret = 0; + dprintf("vp %p: Non^NULL entires so saying failed\n", vp); + } + + // if (ret) + // fileobject->SectionObjectPointer = NULL; +// out: + // Remove usecount lock held above. + atomic_dec_32(&vp->v_usecount); + + // Unable to fully release CcMgr + TraceEvent(8, "%s: ret %d : vp %p fo %p\n", __func__, ret, + vp, fileobject); + + return (ret); +} + + +void +vnode_decouplefileobject(vnode_t *vp, FILE_OBJECT *fileobject) +{ + if (fileobject && fileobject->FsContext) { + dprintf("%s: fo %p -X-> %p\n", __func__, fileobject, vp); + + // If we are flushing, we do nothing here. + if (vp->v_flags & VNODE_FLUSHING) { + dprintf("Already flushing; FS re-entry\n"); + return; + } + + // if (vnode_flushcache(vp, fileobject, FALSE)) + vnode_fileobject_remove(vp, fileobject); + + // fileobject->FsContext = NULL; + } +} + +void +vnode_setsizechange(vnode_t *vp, int set) +{ + if (set) + vp->v_flags |= VNODE_SIZECHANGE; + else + vp->v_flags &= ~VNODE_SIZECHANGE; +} + +int +vnode_sizechange(vnode_t *vp) +{ + return (vp->v_flags & VNODE_SIZECHANGE); +} + +int +vnode_isrecycled(vnode_t *vp) +{ + return (vp->v_flags&(VNODE_MARKTERM | VNODE_DEAD)); +} + +void +vnode_lock(vnode_t *vp) +{ + mutex_enter(&vp->v_mutex); +} + +void +vnode_unlock(vnode_t *vp) +{ + mutex_exit(&vp->v_mutex); +} + +int +vnode_fileobject_member(vnode_t *vp, void *fo) +{ + avl_index_t idx; + mutex_enter(&vp->v_mutex); + // Early out to avoid memory alloc + vnode_fileobjects_t search; + search.fileobject = fo; + if (avl_find(&vp->v_fileobjects, &search, &idx) != NULL) { + mutex_exit(&vp->v_mutex); + return (1); + } + mutex_exit(&vp->v_mutex); + return (0); +} + +/* + * Add a FileObject to the list of FO in the vnode. + * Return 1 if we actually added it + * Return 0 if it was already in the list. + */ +int +vnode_fileobject_add(vnode_t *vp, void *fo) +{ + vnode_fileobjects_t *node; + avl_index_t idx; + // KIRQL OldIrql; + mutex_enter(&vp->v_mutex); + // Early out to avoid memory alloc + vnode_fileobjects_t search; + search.fileobject = fo; + if (avl_find(&vp->v_fileobjects, &search, &idx) != NULL) { + mutex_exit(&vp->v_mutex); + return (0); + } + mutex_exit(&vp->v_mutex); + + node = kmem_alloc(sizeof (*node), KM_SLEEP); + node->fileobject = fo; + node->remove = 0; + + mutex_enter(&vp->v_mutex); + if (avl_find(&vp->v_fileobjects, node, &idx) == NULL) { + avl_insert(&vp->v_fileobjects, node, idx); + mutex_exit(&vp->v_mutex); + dprintf("%s: added FO %p to vp %p\n", __func__, fo, vp); + return (1); + } else { + mutex_exit(&vp->v_mutex); + kmem_free(node, sizeof (*node)); + return (0); + } + // not reached. + mutex_exit(&vp->v_mutex); + return (0); +} + +/* + * Remove a FileObject from the list of FO in the vnode. + * Return 1 if we actually removed it + * Return 0 if it was not in the list. + */ +int +vnode_fileobject_remove(vnode_t *vp, void *fo) +{ + vnode_fileobjects_t search, *node; + // KIRQL OldIrql; + mutex_enter(&vp->v_mutex); + search.fileobject = fo; + node = avl_find(&vp->v_fileobjects, &search, NULL); + if (node == NULL) { + mutex_exit(&vp->v_mutex); + return (0); + } + avl_remove(&vp->v_fileobjects, node); + mutex_exit(&vp->v_mutex); + kmem_free(node, sizeof (*node)); + + dprintf("%s: remed FO %p fm vp %p\n", __func__, fo, vp); + + if (avl_numnodes(&vp->v_fileobjects) == 0) + dprintf("vp %p no more fileobjects, it should be released\n", + vp); + + return (1); +} + +/* + * Check and make sure the list of FileObjects is empty + */ +int +vnode_fileobject_empty(vnode_t *vp, int locked) +{ + // KIRQL OldIrql; + + if (!locked) + mutex_enter(&vp->v_mutex); + boolean_t ret = avl_is_empty(&vp->v_fileobjects); + if (!locked) + mutex_exit(&vp->v_mutex); + + return (ret); +} + +// Get cached EA size, returns 1 is it is cached, 0 if not. +int +vnode_easize(struct vnode *vp, uint64_t *size) +{ + if (vp->v_flags & VNODE_EASIZE) { + *size = vp->v_easize; + return (1); + } + return (0); +} + +void +vnode_set_easize(struct vnode *vp, uint64_t size) +{ + vp->v_easize = size; + vp->v_flags |= VNODE_EASIZE; +} + +void +vnode_clear_easize(struct vnode *vp) +{ + vp->v_flags &= ~VNODE_EASIZE; +} + +void +vnode_set_reparse(struct vnode *vp, REPARSE_DATA_BUFFER *rpp, size_t size) +{ + if (vp->v_reparse != NULL && size > 0) + kmem_free(vp->v_reparse, vp->v_reparse_size); + vp->v_reparse = NULL; + vp->v_reparse_size = 0; + + if (rpp != NULL && size > 0) { + vp->v_reparse = kmem_alloc(size, KM_SLEEP); + vp->v_reparse_size = size; + memcpy(vp->v_reparse, rpp, size); + } +} + +ULONG +vnode_get_reparse_tag(struct vnode *vp) +{ + return (vp->v_reparse ? vp->v_reparse->ReparseTag : 0); +} + +int +vnode_get_reparse_point(struct vnode *vp, REPARSE_DATA_BUFFER **rpp, + size_t *size) +{ + if (vp->v_reparse == NULL || vp->v_reparse_size == 0) + return (ENOENT); + ASSERT3P(rpp, !=, NULL); + ASSERT3P(size, !=, NULL); + *rpp = vp->v_reparse; + *size = vp->v_reparse_size; + return (0); +} + +#ifdef DEBUG_IOCOUNT +void +vnode_check_iocount(void) +{ + /* Iterate all vnodes, checking that iocount is zero. */ + struct vnode *rvp; + mutex_enter(&vnode_all_list_lock); + for (rvp = list_head(&vnode_all_list); + rvp; + rvp = list_next(&vnode_all_list, rvp)) { + ASSERT0(rvp->v_iocount); + } + mutex_exit(&vnode_all_list_lock); +} +#endif + +// Call CcSetFileSizes() either directly, or delayed +// if delay=false, uses FileObject +// if we fail to set, remember it with setsizechange +void +vnode_pager_setsize(void *fo, vnode_t *vp, uint64_t size, boolean_t delay) +{ + FILE_OBJECT *fileObject = fo; + vp->FileHeader.AllocationSize.QuadPart = + P2ROUNDUP(size, PAGE_SIZE); + vp->FileHeader.FileSize.QuadPart = size; + vp->FileHeader.ValidDataLength.QuadPart = size; + vnode_setsizechange(vp, 1); + if (!delay && fileObject && + fileObject->SectionObjectPointer && + fileObject->SectionObjectPointer->SharedCacheMap) { + DWORD __status = STATUS_SUCCESS; + + try { + CcSetFileSizes(fileObject, (PCC_FILE_SIZES) &vp-> + FileHeader.AllocationSize); + } + except(FsRtlIsNtstatusExpected(GetExceptionCode()) ? + EXCEPTION_EXECUTE_HANDLER : + EXCEPTION_CONTINUE_SEARCH) { + __status = STATUS_UNEXPECTED_IO_ERROR; + } + + if (NT_SUCCESS(__status)) + vnode_setsizechange(vp, 0); + } + +} + +void +vfs_changeowner(mount_t *from, mount_t *to) +{ + struct vnode *rvp; + mutex_enter(&vnode_all_list_lock); + for (rvp = list_head(&vnode_all_list); + rvp; + rvp = list_next(&vnode_all_list, rvp)) { + if (rvp->v_mount == from) + rvp->v_mount = to; + + } + mutex_exit(&vnode_all_list_lock); +} diff --git a/module/os/windows/spl/spl-windows.c b/module/os/windows/spl/spl-windows.c new file mode 100644 index 000000000000..29e2d957213b --- /dev/null +++ b/module/os/windows/spl/spl-windows.c @@ -0,0 +1,773 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2018 Jorgen Lundman + * + */ + +#include +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#define DEBUG 1 // for backtrace debugging info + +static utsname_t utsname_static = { { 0 } }; + +unsigned int max_ncpus = 0; +uint64_t total_memory = 0; +uint64_t real_total_memory = 0; + +volatile unsigned int vm_page_free_wanted = 0; +volatile unsigned int vm_page_free_min = 512; +volatile unsigned int vm_page_free_count = 5000; +volatile unsigned int vm_page_speculative_count = 5500; + +uint64_t spl_GetPhysMem(void); +uint64_t spl_GetZfsTotalMemory(PUNICODE_STRING RegistryPath); + +#include +#include + +// Size in bytes of the memory allocated in seg_kmem +extern uint64_t segkmem_total_mem_allocated; +#define MAXHOSTNAMELEN 64 +extern char hostname[MAXHOSTNAMELEN]; + +#define ZFS_MIN_MEMORY_LIMIT 2ULL * 1024ULL * 1024ULL * 1024ULL + +/* + * Windows internal tunables, we use the RAW method when + * we want more control over "name" and "variable" used. + * First argument is the "subfolder" wanted in the Registry, + * and most will most likely be in "root". + */ +uint32_t spl_hostid = 0; +ZFS_MODULE_RAW(, hostid, spl_hostid, + UINT, ZMOD_RW, 0, "The system hostid."); + +extern uchar_t zfs_vdev_protection_filter[ZFS_MODULE_STRMAX]; +ZFS_MODULE_RAW(, zfs_vdev_protection_filter, zfs_vdev_protection_filter, + STRING, ZMOD_RW, ZT_FLAG_STATIC, "vdev_protection_filter"); + +static uchar_t zfs_version[] = ZFS_META_GITREV; +ZFS_MODULE_RAW(, zfs_version, zfs_version, STRING, ZMOD_RD, + ZT_FLAG_STATIC | ZT_FLAG_WRITEONLY, "OpenZFS Windows Driver Version"); + +#if defined(__clang__) +/* + * Try to figure out why we fail linking with these two missing + * Appears to come from including intrin.h - except we don't. + */ +uint64_t +__readcr8(void) +{ + return (0ULL); +} + +unsigned long +_byteswap_ulong(unsigned long b) +{ + return (__builtin_bswap32(b)); +} +#endif + +utsname_t * +utsname(void) +{ + return (&utsname_static); +} + +/* + * Solaris delay is in ticks (hz) and Windows in 100 nanosecs + * 1 HZ is 10 milliseconds, 10000000 nanoseconds. + */ +void +windows_delay(int ticks) +{ + LARGE_INTEGER interval; + // * 10000000 / 100 + interval.QuadPart = -((uint64_t)ticks) * 100000ULL; + KeDelayExecutionThread(KernelMode, FALSE, &interval); +} + +uint32_t +zone_get_hostid(void *zone) +{ + return (spl_hostid); +} + +const char * +spl_panicstr(void) +{ + return (""); +} + +int +spl_system_inshutdown(void) +{ + return (0); +} + +void +hrt2ts(hrtime_t hrt, timespec_t *tsp) +{ + tsp->tv_sec = (time_t)(hrt / NANOSEC); + tsp->tv_nsec = (hrt % NANOSEC); +} + +// If we want to implement this on Windows, we could probably use +// https:/ /stackoverflow.com/questions/590160/how-to-log- +// stack-frames-with-windows-x64 +// which calls RtlCaptureStackBackTrace(); +int +getpcstack(uintptr_t *pcstack, int pcstack_limit) +{ + return (RtlCaptureStackBackTrace(1, pcstack_limit, (PVOID *)pcstack, + NULL)); +} + +/* + * fnv_32a_str - perform a 32 bit Fowler/Noll/Vo FNV-1a hash on a string + * + * input: + * str - string to hash + * hval - previous hash value or 0 if first call + * + * returns: + * 32 bit hash as a static hash type + * + * NOTE: To use the recommended 32 bit FNV-1a hash, use FNV1_32A_INIT as the + * hval arg on the first call to either fnv_32a_buf() or fnv_32a_str(). + */ +uint32_t +fnv_32a_str(const char *str, uint32_t hval) +{ + unsigned char *s = (unsigned char *)str; /* unsigned string */ + + /* + * FNV-1a hash each octet in the buffer + */ + while (*s) { + + /* xor the bottom with the current octet */ + hval ^= (uint32_t)*s++; + + /* multiply by the 32 bit FNV magic prime mod 2^32 */ +#if defined(NO_FNV_GCC_OPTIMIZATION) + hval *= FNV_32_PRIME; +#else + hval += (hval << 1) + (hval << 4) + (hval << 7) + + (hval << 8) + (hval << 24); +#endif + } + + /* return our new hash value */ + return (hval); +} + +/* + * fnv_32a_buf - perform a 32 bit Fowler/Noll/Vo FNV-1a hash on a buffer + * + * input: + * buf- start of buffer to hash + * len- length of buffer in octets + * hval- previous hash value or 0 if first call + * + * returns: + * 32 bit hash as a static hash type + * + * NOTE: To use the recommended 32 bit FNV-1a hash, use FNV1_32A_INIT as the + * hval arg on the first call to either fnv_32a_buf() or fnv_32a_str(). + */ +uint32_t +fnv_32a_buf(void *buf, size_t len, uint32_t hval) +{ + unsigned char *bp = (unsigned char *)buf; /* start of buffer */ + unsigned char *be = bp + len; /* beyond end of buffer */ + + /* + * FNV-1a hash each octet in the buffer + */ + while (bp < be) { + + /* xor the bottom with the current octet */ + hval ^= (uint32_t)*bp++; + + /* multiply by the 32 bit FNV magic prime mod 2^32 */ +#if defined(NO_FNV_GCC_OPTIMIZATION) + hval *= FNV_32_PRIME; +#else + hval += (hval << 1) + (hval << 4) + (hval << 7) + + (hval << 8) + (hval << 24); +#endif + } + + /* return our new hash value */ + return (hval); +} + +/* + * Function to free a MDL chain + */ +void +UnlockAndFreeMdl(PMDL Mdl) +{ + PMDL currentMdl, nextMdl; + + for (currentMdl = Mdl; currentMdl != NULL; currentMdl = nextMdl) { + nextMdl = currentMdl->Next; + if (currentMdl->MdlFlags & MDL_PAGES_LOCKED) { + MmUnlockPages(currentMdl); + } + IoFreeMdl(currentMdl); + } +} + +int +ddi_copyin(const void *from, void *to, size_t len, int flags) +{ + int error = 0; + PMDL mdl = NULL; + PCHAR buffer = NULL; + + if (from == NULL || + to == NULL || + len == 0) + return (0); + + /* Fake ioctl() issued by kernel, so we just need to memcpy */ + if (flags & FKIOCTL) { + if (flags & FCOPYSTR) + strlcpy(to, from, len); + else + memcpy(to, from, len); + return (0); + } + + // Lets try reading from the input nvlist + TraceEvent(TRACE_NOISY, "%s:%d: SPL: trying windows copyin: %p:%d\n", + __func__, __LINE__, from, len); + + try { + ProbeForRead((void *)from, len, sizeof (UCHAR)); + } + except(EXCEPTION_EXECUTE_HANDLER) { + error = GetExceptionCode(); + } + if (error) { + TraceEvent(TRACE_ERROR, + "SPL: Exception while accessing inBuf 0X%08X\n", error); + goto out; + } + + mdl = IoAllocateMdl((void *)from, len, FALSE, FALSE, NULL); + if (!mdl) { + error = STATUS_INSUFFICIENT_RESOURCES; + goto out; + } + + try { + MmProbeAndLockPages(mdl, UserMode, IoReadAccess); + } + except(EXCEPTION_EXECUTE_HANDLER) { + error = GetExceptionCode(); + } + if (error) { + TraceEvent(TRACE_ERROR, + "SPL: Exception while locking inBuf 0X%08X\n", error); + goto out; + } + + buffer = MmGetSystemAddressForMdlSafe(mdl, + NormalPagePriority | MdlMappingNoExecute); + + if (!buffer) { + error = STATUS_INSUFFICIENT_RESOURCES; + } else { + // Success, copy over the data. + if (flags & FCOPYSTR) + strlcpy(to, buffer, len); + else + memcpy(to, buffer, len); + } + + TraceEvent(TRACE_NOISY, "SPL: copyin return %d (%d bytes)\n", + error, len); + +out: + if (mdl) { + UnlockAndFreeMdl(mdl); + } + + return (error); +} + +int +ddi_copyout(const void *from, void *to, size_t len, int flags) +{ + int error = 0; + PMDL mdl = NULL; + PCHAR buffer = NULL; + + if (from == NULL || + to == NULL || + len == 0) + return (0); + + /* Fake ioctl() issued by kernel, 'from' is a kernel address */ + if (flags & FKIOCTL) { + memcpy(to, from, len); + return (0); + } + + // dprintf("SPL: trying windows copyout: %p:%d\n", to, len); + + mdl = IoAllocateMdl(to, len, FALSE, FALSE, NULL); + if (!mdl) { + error = STATUS_INSUFFICIENT_RESOURCES; + TraceEvent(TRACE_ERROR, + "SPL: copyout failed to allocate mdl\n"); + goto out; + } + + try { + MmProbeAndLockPages(mdl, UserMode, IoWriteAccess); + } + except(EXCEPTION_EXECUTE_HANDLER) { + error = GetExceptionCode(); + } + if (error != 0) { + TraceEvent(TRACE_ERROR, + "SPL: Exception while locking outBuf 0X%08X\n", + error); + goto out; + } + + buffer = MmGetSystemAddressForMdlSafe(mdl, + NormalPagePriority | MdlMappingNoExecute); + + if (!buffer) { + error = STATUS_INSUFFICIENT_RESOURCES; + goto out; + } else { + // Success, copy over the data. + memcpy(buffer, from, len); + } + // dprintf("SPL: copyout return %d (%d bytes)\n", error, len); +out: + if (mdl) { + UnlockAndFreeMdl(mdl); + } + + return (error); +} + +int +ddi_copysetup(void *to, size_t len, void **out_buffer, PMDL *out_mdl) +{ + int error = 0; + PMDL mdl = NULL; + PCHAR buffer = NULL; + + if (to == NULL || + out_buffer == NULL || + out_mdl == NULL || + len == 0) + return (0); + + // dprintf("SPL: trying windows copyout_ex: %p:%d\n", to, len); + + // Do we have to call both? Or is calling ProbeForWrite enough? + try { + ProbeForRead(to, len, sizeof (UCHAR)); + } + except(EXCEPTION_EXECUTE_HANDLER) { + error = GetExceptionCode(); + } + if (error) { + TraceEvent(TRACE_ERROR, + "SPL: Exception while accessing inBuf 0X%08X\n", error); + goto out; + } + + try { + ProbeForWrite(to, len, sizeof (UCHAR)); + } + except(EXCEPTION_EXECUTE_HANDLER) { + error = GetExceptionCode(); + } + if (error) { + TraceEvent(TRACE_ERROR, + "SPL: Exception while accessing inBuf 0X%08X\n", error); + goto out; + } + + mdl = IoAllocateMdl(to, len, FALSE, FALSE, NULL); + if (!mdl) { + error = STATUS_INSUFFICIENT_RESOURCES; + TraceEvent(TRACE_ERROR, + "SPL: copyout failed to allocate mdl\n"); + goto out; + } + + try { + MmProbeAndLockPages(mdl, UserMode, IoWriteAccess); + } + except(EXCEPTION_EXECUTE_HANDLER) { + error = GetExceptionCode(); + } + if (error != 0) { + TraceEvent(TRACE_ERROR, + "SPL: Exception while locking outBuf 0X%08X\n", + error); + goto out; + } + + buffer = MmGetSystemAddressForMdlSafe(mdl, + NormalPagePriority | MdlMappingNoExecute); + + if (!buffer) { + error = STATUS_INSUFFICIENT_RESOURCES; + goto out; + } + + *out_buffer = buffer; + *out_mdl = mdl; + return (0); + +out: + if (mdl) { + UnlockAndFreeMdl(mdl); + } + + return (error); +} + + +/* + * Technically, this call does not exist in IllumOS, but we use it for + * consistency. + */ +int +ddi_copyinstr(const void *uaddr, void *kaddr, size_t len, size_t *done) +{ + int ret = 0; + + ret = ddi_copyin((const void *)uaddr, kaddr, len, FCOPYSTR); + if ((ret == STATUS_SUCCESS) && done) { + *done = strlen(kaddr) + 1; // copyinstr includes the NULL byte + } + return (ret); +} + +int +spl_start(PUNICODE_STRING RegistryPath) +{ + + uint64_t zfs_total_memory_limit = 0; + dprintf("SPL: start\n"); + max_ncpus = KeQueryActiveProcessorCountEx(ALL_PROCESSOR_GROUPS); + if (!max_ncpus) max_ncpus = 1; + dprintf("SPL: total ncpu %d\n", max_ncpus); + + // Not sure how to get physical RAM size in a Windows Driver + // So until then, pull some numbers out of the aether. Next + // we could let users pass in a value, somehow... + total_memory = spl_GetPhysMem(); + real_total_memory = spl_GetPhysMem(); + + // Set 2GB as code above doesnt work + if (real_total_memory) { + zfs_total_memory_limit = spl_GetZfsTotalMemory(RegistryPath); + if (zfs_total_memory_limit > ZFS_MIN_MEMORY_LIMIT && + zfs_total_memory_limit < real_total_memory) + total_memory = zfs_total_memory_limit; + else + total_memory = real_total_memory * 50ULL / 100ULL; + } else { + real_total_memory = ZFS_MIN_MEMORY_LIMIT; + total_memory = real_total_memory * 50ULL / 100ULL; + } + + dprintf("%s real_total_memory: %llu zfs_total_memory_limit: %llu " + "total_memory: %llu\n", __func__, real_total_memory, + zfs_total_memory_limit, total_memory); + physmem = total_memory / PAGE_SIZE; + + // We need to set these to some non-zero values + // so we don't think there is permanent memory + // pressure. + vm_page_free_count = (unsigned int)(physmem / 2ULL); + vm_page_speculative_count = vm_page_free_count; + + // Set hostid here, it will be overwritten if it is in registry + if (spl_hostid == 0) + random_get_bytes(&spl_hostid, sizeof (spl_hostid)); + + /* + * For some reason, (CTLFLAG_KERN is not set) looking up hostname + * returns 1. So we set it to uuid just to give it *something*. + * As it happens, ZFS sets the nodename on init. + */ + strlcpy(utsname_static.nodename, "Windows", + sizeof (utsname_static.nodename)); + spl_mutex_subsystem_init(); + spl_kmem_init(total_memory); + + spl_vnode_init(); + spl_kmem_thread_init(); + spl_kmem_mp_init(); + + kstat_init(); + + IOLog("SPL: Loaded module v%s-%s%s, " + "(ncpu %d, memsize %llu, pages %llu)\n", + SPL_META_VERSION, SPL_META_RELEASE, SPL_DEBUG_STR, + max_ncpus, total_memory, physmem); + return (STATUS_SUCCESS); +} + +extern uint64_t zfs_threads; + +int +spl_stop(void) +{ + spl_kmem_thread_fini(); + spl_vnode_fini(); + spl_taskq_fini(); + spl_rwlock_fini(); + spl_tsd_fini(); + spl_kmem_fini(); + spl_mutex_subsystem_fini(); + IOLog("SPL: Unloaded module v%s-%s " + "(os_mem_alloc: %llu)\n", + SPL_META_VERSION, SPL_META_RELEASE, + segkmem_total_mem_allocated); + while (zfs_threads >= 1) { + IOLog("SPL: active threads %d\n", zfs_threads); + delay(hz << 2); + } + + /* + * At this point, all threads waiting on bsd_timers in + * bsd_timeout_handler() are exited and timer can be cancelled. If the + * timer is still loaded,it could fire after driver unload and bugcheck + */ + spl_kmem_timer_fini(); + vmem_timer_fini(); + + return (STATUS_SUCCESS); +} + +#define UNICODE + +#pragma pack(push, 4) +typedef struct { + UCHAR Type; + UCHAR ShareDisposition; + USHORT Flags; + ULONGLONG Start; + ULONG Length; +} MEMORY, *PMEMORY; +#pragma pack(pop) + +/* TimoVJL */ +LONGLONG +GetMemResources(char *pData) +{ + LONGLONG llMem = 0; + char *pPtr; + uint32_t *pDW; + pDW = (uint32_t *)pData; + if (*pDW != 1) + return (0); + DWORD nCnt = *(uint32_t *)(pData + 0x10); // Count + pPtr = pData + 0x14; + DWORD nRLen = 0; + if (*(pData + 0x14) == *(pData + 0x24)) nRLen = 16; + if (*(pData + 0x14) == *(pData + 0x28)) nRLen = 20; + PMEMORY pMem; + for (DWORD nIdx = 0; nRLen && nIdx < nCnt; nIdx++) { + pMem = (PMEMORY)(pPtr + nRLen * nIdx); + if (pMem->Type == 3) llMem += pMem->Length; + if (pMem->Type == 7 && pMem->Flags == 0x200) + llMem += ((LONGLONG)pMem->Length) << 8; + pMem += nRLen; + } + return (llMem); +} + +NTSTATUS +spl_query_memsize( + IN PWSTR ValueName, + IN ULONG ValueType, + IN PVOID ValueData, + IN ULONG ValueLength, + IN PVOID Context, + IN PVOID EntryContext) +{ + + dprintf("%s: '%S' type 0x%x len 0x%x\n", __func__, + ValueName, ValueType, ValueLength); + + if ((ValueType == REG_RESOURCE_LIST) && + (_wcsicmp(L".Translated", ValueName) == 0)) { + uint64_t *value; + value = EntryContext; + if (value) + *value = GetMemResources(ValueData); + dprintf("%s: memsize is %llu\n", __func__, value ? *value : 0); + } + + return (STATUS_SUCCESS); +} + +uint64_t +spl_GetPhysMem(void) +{ + uint64_t memory; + NTSTATUS status; + static RTL_QUERY_REGISTRY_TABLE query[2] = + { + { + .Flags = RTL_QUERY_REGISTRY_REQUIRED + /* | RTL_QUERY_REGISTRY_DIRECT */ + | RTL_QUERY_REGISTRY_NOEXPAND + | RTL_QUERY_REGISTRY_TYPECHECK, + .QueryRoutine = spl_query_memsize, + } + }; + + query[0].EntryContext = &memory; + status = RtlQueryRegistryValues( + RTL_REGISTRY_ABSOLUTE, + L"\\REGISTRY\\MACHINE\\HARDWARE\\RESOURCEMAP\\" + "System Resources\\Physical Memory", + query, NULL, NULL); + + if (status != STATUS_SUCCESS) { + TraceEvent(TRACE_ERROR, + "%s: size query failed: 0x%x\n", __func__, status); + return (0ULL); + } + + return (memory); +} + +uint64_t +spl_GetZfsTotalMemory(PUNICODE_STRING RegistryPath) +{ + OBJECT_ATTRIBUTES ObjectAttributes; + HANDLE h; + NTSTATUS status; + uint64_t newvalue = 0; + + InitializeObjectAttributes(&ObjectAttributes, + RegistryPath, + OBJ_KERNEL_HANDLE | OBJ_CASE_INSENSITIVE, + NULL, + NULL); + + status = ZwOpenKey(&h, // KeyHandle + KEY_ALL_ACCESS, // DesiredAccess + &ObjectAttributes); // ObjectAttributes + + if (!NT_SUCCESS(status)) { + dprintf("%s: Unable to open Registry %wZ: 0x%x. " + "Going with defaults.\n", __func__, RegistryPath, status); + return (0); + } + + ULONG index = 0; + ULONG length = 0; + PKEY_VALUE_FULL_INFORMATION regBuffer = NULL; + + for (index = 0; status != STATUS_NO_MORE_ENTRIES; index++) { + // Get the buffer size necessary + status = ZwEnumerateValueKey(h, index, KeyValueFullInformation, + NULL, 0, &length); + + if ((status != STATUS_BUFFER_TOO_SMALL) && + (status != STATUS_BUFFER_OVERFLOW)) + break; // Something is wrong - or we finished + + // Allocate space to hold + regBuffer = (PKEY_VALUE_FULL_INFORMATION)ExAllocatePoolWithTag( + NonPagedPoolNx, length, 'zfsr'); + + if (regBuffer == NULL) + break; + + status = ZwEnumerateValueKey(h, index, KeyValueFullInformation, + regBuffer, length, &length); + if (!NT_SUCCESS(status)) { + break; + } + // Convert name to straight ascii so we compare with kstat + ULONG outlen = 0; + char keyname[KSTAT_STRLEN + 1] = { 0 }; + status = RtlUnicodeToUTF8N(keyname, KSTAT_STRLEN, &outlen, + regBuffer->Name, regBuffer->NameLength); + + // Conversion failed? move along.. + if (status != STATUS_SUCCESS && status + != STATUS_SOME_NOT_MAPPED) + break; + + // Output string is only null terminated if input is, + // so do so now. + keyname[outlen] = 0; + if (strcasecmp("zfs_total_memory_limit", keyname) == 0) { + if (regBuffer->Type != REG_QWORD || + regBuffer->DataLength != sizeof (uint64_t)) { + dprintf("%s: registry '%s' did not match. " + "Type needs to be REG_QWORD. (8 bytes)\n", + __func__, keyname); + } else { + newvalue = *(uint64_t *)((uint8_t *)regBuffer + + regBuffer->DataOffset); + dprintf("%s: zfs_total_memory_limit is set to:" + " %llu\n", __func__, newvalue); + } + break; + } + ExFreePool(regBuffer); + regBuffer = NULL; + } + + if (regBuffer) + ExFreePool(regBuffer); + + ZwClose(h); + return (newvalue); +} diff --git a/module/os/windows/spl/spl-wmsum.c b/module/os/windows/spl/spl-wmsum.c new file mode 100644 index 000000000000..7812add812b3 --- /dev/null +++ b/module/os/windows/spl/spl-wmsum.c @@ -0,0 +1 @@ +// This space intentionally left blank diff --git a/module/os/windows/spl/spl-xdr.c b/module/os/windows/spl/spl-xdr.c new file mode 100644 index 000000000000..1a02711485f1 --- /dev/null +++ b/module/os/windows/spl/spl-xdr.c @@ -0,0 +1,527 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * + * Copyright (C) 2017 Jorgen Lundman + * + */ + +#include +#include +#include +#include +#include +#include + +#include + +/* + * SPL's XDR mem implementation. + * + * This is used by libnvpair to serialize/deserialize the name-value pair data + * structures into byte arrays in a well-defined and portable manner. + * + * These data structures are used by the DMU/ZFS to flexibly manipulate various + * information in memory and later serialize it/deserialize it to disk. + * Examples of usages include the pool configuration, lists of pool and dataset + * properties, etc. + * + * Reference documentation for the XDR representation and XDR operations can be + * found in RFC 1832 and xdr(3), respectively. + * + * === Implementation shortcomings === + * + * It is assumed that the following C types have the following sizes: + * + * char/unsigned char: 1 byte + * short/unsigned short: 2 bytes + * int/unsigned int: 4 bytes + * longlong_t/u_longlong_t: 8 bytes + * + * The C standard allows these types to be larger (and in the case of ints, + * shorter), so if that is the case on some compiler/architecture, the build + * will fail (on purpose). + * + * If someone wants to fix the code to work properly on such environments, then: + * + * 1) Preconditions should be added to xdrmem_enc functions to make sure the + * caller doesn't pass arguments which exceed the expected range. + * 2) Functions which take signed integers should be changed to properly do + * sign extension. + * 3) For ints with less than 32 bits, well.. I suspect you'll have bigger + * problems than this implementation. + * + * It is also assumed that: + * + * 1) Chars have 8 bits. + * 2) We can always do 32-bit-aligned int memory accesses and byte-aligned + * memcpy, memset and memcmp. + * 3) Arrays passed to xdr_array() are packed and the compiler/architecture + * supports element-sized-aligned memory accesses. + * 4) Negative integers are natively stored in two's complement binary + * representation. + * + * No checks are done for the 4 assumptions above, though. + * + * === Caller expectations === + * + * Existing documentation does not describe the semantics of XDR operations very + * well. Therefore, some assumptions about failure semantics will be made and + * will be described below: + * + * 1) If any encoding operation fails (e.g., due to lack of buffer space), the + * the stream should be considered valid only up to the encoding operation + * previous to the one that first failed. However, the stream size as returned + * by xdr_control() cannot be considered to be strictly correct (it may be + * bigger). + * + * Putting it another way, if there is an encoding failure it's undefined + * whether anything is added to the stream in that operation and therefore + * neither xdr_control() nor future encoding operations on the same stream can + * be relied upon to produce correct results. + * + * 2) If a decoding operation fails, it's undefined whether anything will be + * decoded into passed buffers/pointers during that operation, or what the + * values on those buffers will look like. + * + * Future decoding operations on the same stream will also have similar + * undefined behavior. + * + * 3) When the first decoding operation fails it is OK to trust the results of + * previous decoding operations on the same stream, as long as the caller + * expects a failure to be possible (e.g. due to end-of-stream). + * + * However, this is highly discouraged because the caller should know the + * stream size and should be coded to expect any decoding failure to be data + * corruption due to hardware, accidental or even malicious causes, which should + * be handled gracefully in all cases. + * + * In very rare situations where there are strong reasons to believe the data + * can be trusted to be valid and non-tampered with, then the caller may assume + * a decoding failure to be a bug (e.g. due to mismatched data types) and may + * fail non-gracefully. + * + * 4) Non-zero padding bytes will cause the decoding operation to fail. + * + * 5) Zero bytes on string types will also cause the decoding operation to fail. + * + * 6) It is assumed that either the pointer to the stream buffer given by the + * caller is 32-bit aligned or the architecture supports non-32-bit-aligned int + * memory accesses. + * + * 7) The stream buffer and encoding/decoding buffers/ptrs should not overlap. + * + * 8) If a caller passes pointers to non-kernel memory (e.g., pointers to user + * space or MMIO space), the computer may explode. + */ + +static struct xdr_ops xdrmem_encode_ops; +static struct xdr_ops xdrmem_decode_ops; + +void +xdrmem_create(XDR *xdrs, const caddr_t addr, const uint_t size, + const enum xdr_op op) +{ + switch (op) { + case XDR_ENCODE: + xdrs->x_ops = &xdrmem_encode_ops; + break; + case XDR_DECODE: + xdrs->x_ops = &xdrmem_decode_ops; + break; + default: + TraceEvent(TRACE_ERROR, + "SPL: Invalid op value: %d\n", op); + xdrs->x_ops = NULL; /* Let the caller know we failed */ + return; + } + + xdrs->x_op = op; + xdrs->x_addr = addr; + xdrs->x_addr_end = addr + size; + + if (xdrs->x_addr_end < xdrs->x_addr) { + TraceEvent(TRACE_ERROR, + "SPL: Overflow while creating xdrmem: %p, %u\n", + addr, size); + xdrs->x_ops = NULL; + } +} +EXPORT_SYMBOL(xdrmem_create); + +static bool_t +xdrmem_control(XDR *xdrs, int req, void *info) +{ + struct xdr_bytesrec *rec = (struct xdr_bytesrec *)info; + + if (req != XDR_GET_BYTES_AVAIL) { + TraceEvent(TRACE_ERROR, + "SPL: Called with unknown request: %d\n", req); + return (FALSE); + } + + rec->xc_is_last_record = TRUE; /* always TRUE in xdrmem streams */ + rec->xc_num_avail = (size_t)(xdrs->x_addr_end - xdrs->x_addr); + + return (TRUE); +} + +static bool_t +xdrmem_enc_bytes(XDR *xdrs, caddr_t cp, const uint_t cnt) +{ + uint_t size = P2ROUNDUP(cnt, 4); + uint_t pad; + + if (size < cnt) + return (FALSE); /* Integer overflow */ + + if (xdrs->x_addr > xdrs->x_addr_end) + return (FALSE); + + if (xdrs->x_addr_end - xdrs->x_addr < size) + return (FALSE); + + memcpy(xdrs->x_addr, cp, cnt); + + xdrs->x_addr += cnt; + + pad = size - cnt; + if (pad > 0) { + memset(xdrs->x_addr, 0, pad); + xdrs->x_addr += pad; + } + + return (TRUE); +} + +static bool_t +xdrmem_dec_bytes(XDR *xdrs, caddr_t cp, const uint_t cnt) +{ + static uint32_t zero = 0; + uint_t size = P2ROUNDUP(cnt, 4); + uint_t pad; + + if (size < cnt) + return (FALSE); /* Integer overflow */ + + if (xdrs->x_addr > xdrs->x_addr_end) + return (FALSE); + + if (xdrs->x_addr_end - xdrs->x_addr < size) + return (FALSE); + + memcpy(cp, xdrs->x_addr, cnt); + xdrs->x_addr += cnt; + + pad = size - cnt; + if (pad > 0) { + /* An inverted memchr() would be useful here... */ + if (memcmp(&zero, xdrs->x_addr, pad) != 0) + return (FALSE); + + xdrs->x_addr += pad; + } + + return (TRUE); +} + +static bool_t +xdrmem_enc_uint32(XDR *xdrs, uint32_t val) +{ + if (xdrs->x_addr + sizeof (uint32_t) > xdrs->x_addr_end) + return (FALSE); + + *((uint32_t *)xdrs->x_addr) = BE_32(val); + + xdrs->x_addr += sizeof (uint32_t); + + return (TRUE); +} + +static bool_t +xdrmem_dec_uint32(XDR *xdrs, uint32_t *val) +{ + if (xdrs->x_addr + sizeof (uint32_t) > xdrs->x_addr_end) + return (FALSE); + + *val = BE_32(*((uint32_t *)xdrs->x_addr)); + + xdrs->x_addr += sizeof (uint32_t); + + return (TRUE); +} + +static bool_t +xdrmem_enc_char(XDR *xdrs, char *cp) +{ + uint32_t val; + + // BUILD_BUG_ON(sizeof(char) != 1); + val = *((unsigned char *) cp); + + return (xdrmem_enc_uint32(xdrs, val)); +} + +static bool_t +xdrmem_dec_char(XDR *xdrs, char *cp) +{ + uint32_t val; + + // BUILD_BUG_ON(sizeof(char) != 1); + + if (!xdrmem_dec_uint32(xdrs, &val)) + return (FALSE); + + /* + * If any of the 3 other bytes are non-zero then val will be greater + * than 0xff and we fail because according to the RFC, this block does + * not have a char encoded in it. + */ + if (val > 0xff) + return (FALSE); + + *((unsigned char *) cp) = (uint8_t)val; + + return (TRUE); +} + +static bool_t +xdrmem_enc_ushort(XDR *xdrs, unsigned short *usp) +{ + // BUILD_BUG_ON(sizeof(unsigned short) != 2); + + return (xdrmem_enc_uint32(xdrs, *usp)); +} + +static bool_t +xdrmem_dec_ushort(XDR *xdrs, unsigned short *usp) +{ + uint32_t val; + + // BUILD_BUG_ON(sizeof(unsigned short) != 2); + + if (!xdrmem_dec_uint32(xdrs, &val)) + return (FALSE); + + /* + * Short ints are not in the RFC, but we assume similar logic as in + * xdrmem_dec_char(). + */ + if (val > 0xffff) + return (FALSE); + + *usp = (uint16_t)val; + + return (TRUE); +} + +static bool_t +xdrmem_enc_uint(XDR *xdrs, unsigned *up) +{ + // BUILD_BUG_ON(sizeof(unsigned) != 4); + + return (xdrmem_enc_uint32(xdrs, *up)); +} + +static bool_t +xdrmem_dec_uint(XDR *xdrs, unsigned *up) +{ + // BUILD_BUG_ON(sizeof(unsigned) != 4); + + return (xdrmem_dec_uint32(xdrs, (uint32_t *)up)); +} + +static bool_t +xdrmem_enc_ulonglong(XDR *xdrs, u_longlong_t *ullp) +{ + // BUILD_BUG_ON(sizeof(u_longlong_t) != 8); + + if (!xdrmem_enc_uint32(xdrs, *ullp >> 32)) + return (FALSE); + + return (xdrmem_enc_uint32(xdrs, *ullp & 0xffffffff)); +} + +static bool_t +xdrmem_dec_ulonglong(XDR *xdrs, u_longlong_t *ullp) +{ + uint32_t low, high; + + // BUILD_BUG_ON(sizeof(u_longlong_t) != 8); + + if (!xdrmem_dec_uint32(xdrs, &high)) + return (FALSE); + if (!xdrmem_dec_uint32(xdrs, &low)) + return (FALSE); + + *ullp = ((u_longlong_t)high << 32) | low; + + return (TRUE); +} + +static bool_t +xdr_enc_array(XDR *xdrs, caddr_t *arrp, uint_t *sizep, const uint_t maxsize, + const uint_t elsize, const xdrproc_t elproc) +{ + uint_t i; + caddr_t addr = *arrp; + + if (*sizep > maxsize || *sizep > UINT_MAX / elsize) + return (FALSE); + + if (!xdrmem_enc_uint(xdrs, sizep)) + return (FALSE); + + for (i = 0; i < *sizep; i++) { + if (!elproc(xdrs, addr)) + return (FALSE); + addr += elsize; + } + + return (TRUE); +} + +static bool_t +xdr_dec_array(XDR *xdrs, caddr_t *arrp, uint_t *sizep, const uint_t maxsize, + const uint_t elsize, const xdrproc_t elproc) +{ + uint_t i, size; + bool_t alloc = FALSE; + caddr_t addr; + + if (!xdrmem_dec_uint(xdrs, sizep)) + return (FALSE); + + size = *sizep; + + if (size > maxsize || size > UINT_MAX / elsize) + return (FALSE); + + /* + * The Solaris man page says: "If *arrp is NULL when decoding, + * xdr_array() allocates memory and *arrp points to it". + */ + if (*arrp == NULL) { + // BUILD_BUG_ON(sizeof(uint_t) > sizeof(size_t)); + + *arrp = kmem_alloc(size * elsize, KM_NOSLEEP); + if (*arrp == NULL) + return (FALSE); + + alloc = TRUE; + } + + addr = *arrp; + + for (i = 0; i < size; i++) { + if (!elproc(xdrs, addr)) { + if (alloc) + kmem_free(*arrp, size * elsize); + return (FALSE); + } + addr += elsize; + } + + return (TRUE); +} + +static bool_t +xdr_enc_string(XDR *xdrs, char **sp, const uint_t maxsize) +{ + size_t slen = strlen(*sp); + uint_t len; + + if (slen > maxsize) + return (FALSE); + + len = slen; + + if (!xdrmem_enc_uint(xdrs, &len)) + return (FALSE); + + return (xdrmem_enc_bytes(xdrs, *sp, len)); +} + +static bool_t +xdr_dec_string(XDR *xdrs, char **sp, const uint_t maxsize) +{ + uint_t size; + bool_t alloc = FALSE; + + if (!xdrmem_dec_uint(xdrs, &size)) + return (FALSE); + + if (size > maxsize || size > UINT_MAX - 1) + return (FALSE); + + /* + * Solaris man page: "If *sp is NULL when decoding, xdr_string() + * allocates memory and *sp points to it". + */ + if (*sp == NULL) { + // BUILD_BUG_ON(sizeof(uint_t) > sizeof(size_t)); + + *sp = kmem_alloc(size + 1, KM_NOSLEEP); + if (*sp == NULL) + return (FALSE); + + alloc = TRUE; + } + + if (!xdrmem_dec_bytes(xdrs, *sp, size)) + goto fail; + + if (kmemchr(*sp, 0, size) != NULL) + goto fail; + + (*sp)[size] = '\0'; + + return (TRUE); + +fail: + if (alloc) + kmem_free(*sp, size + 1); + + return (FALSE); +} + +static struct xdr_ops xdrmem_encode_ops = { + .xdr_control = xdrmem_control, + .xdr_char = xdrmem_enc_char, + .xdr_u_short = xdrmem_enc_ushort, + .xdr_u_int = xdrmem_enc_uint, + .xdr_u_longlong_t = xdrmem_enc_ulonglong, + .xdr_opaque = xdrmem_enc_bytes, + .xdr_string = xdr_enc_string, + .xdr_array = xdr_enc_array +}; + +static struct xdr_ops xdrmem_decode_ops = { + .xdr_control = xdrmem_control, + .xdr_char = xdrmem_dec_char, + .xdr_u_short = xdrmem_dec_ushort, + .xdr_u_int = xdrmem_dec_uint, + .xdr_u_longlong_t = xdrmem_dec_ulonglong, + .xdr_opaque = xdrmem_dec_bytes, + .xdr_string = xdr_dec_string, + .xdr_array = xdr_dec_array +}; diff --git a/module/os/windows/spl/spl-zlib.c b/module/os/windows/spl/spl-zlib.c new file mode 100644 index 000000000000..d066bbd2822f --- /dev/null +++ b/module/os/windows/spl/spl-zlib.c @@ -0,0 +1,198 @@ +/* + * + * zlib.h -- interface of the 'zlib' general purpose compression library + * version 1.2.5, April 19th, 2010 + * + * Copyright (C) 1995-2010 Jean-loup Gailly and Mark Adler + * + * This software is provided 'as-is', without any express or implied + * warranty. In no event will the authors be held liable for any damages + * arising from the use of this software. + * + * Permission is granted to anyone to use this software for any purpose, + * including commercial applications, and to alter it and redistribute it + * freely, subject to the following restrictions: + * + * 1. The origin of this software must not be misrepresented; you must not + * claim that you wrote the original software. If you use this software + * in a product, an acknowledgment in the product documentation would be + * appreciated but is not required. + * 2. Altered source versions must be plainly marked as such, and must not be + * misrepresented as being the original software. + * 3. This notice may not be removed or altered from any source distribution. + * + * Jean-loup Gailly + * Mark Adler + */ + +#include +#include +#include +#include + +#ifdef DEBUG_SUBSYSTEM +#undef DEBUG_SUBSYSTEM +#endif + +#define DEBUG_SUBSYSTEM SS_ZLIB + +static spl_kmem_cache_t *zlib_workspace_cache; + +/* + * A kmem_cache is used for the zlib workspaces to avoid having to vmalloc + * and vfree for every call. Using a kmem_cache also has the advantage + * that improves the odds that the memory used will be local to this cpu. + * To further improve things it might be wise to create a dedicated per-cpu + * workspace for use. This would take some additional care because we then + * must disable preemption around the critical section, and verify that + * zlib_deflate* and zlib_inflate* never internally call schedule(). + */ +static void * +zlib_workspace_alloc(int flags) +{ + return (kmem_cache_alloc(zlib_workspace_cache, flags & ~(__GFP_FS))); +} + +static void +zlib_workspace_free(void *workspace) +{ + kmem_cache_free(zlib_workspace_cache, workspace); +} + +/* + * Compresses the source buffer into the destination buffer. The level + * parameter has the same meaning as in deflateInit. sourceLen is the byte + * length of the source buffer. Upon entry, destLen is the total size of the + * destination buffer, which must be at least 0.1% larger than sourceLen plus + * 12 bytes. Upon exit, destLen is the actual size of the compressed buffer. + * + * compress2 returns Z_OK if success, Z_MEM_ERROR if there was not enough + * memory, Z_BUF_ERROR if there was not enough room in the output buffer, + * Z_STREAM_ERROR if the level parameter is invalid. + */ +int +z_compress_level(void *dest, size_t *destLen, const void *source, + size_t sourceLen, int level) +{ + z_stream stream; + int err; + + stream.next_in = (Byte *)source; + stream.avail_in = (uInt)sourceLen; + stream.next_out = dest; + stream.avail_out = (uInt)*destLen; + + if ((size_t)stream.avail_out != *destLen) + return (Z_BUF_ERROR); + + stream.workspace = zlib_workspace_alloc(KM_SLEEP); + if (!stream.workspace) + return (Z_MEM_ERROR); + + err = zlib_deflateInit(&stream, level); + if (err != Z_OK) { + zlib_workspace_free(stream.workspace); + return (err); + } + + err = zlib_deflate(&stream, Z_FINISH); + if (err != Z_STREAM_END) { + zlib_deflateEnd(&stream); + zlib_workspace_free(stream.workspace); + return (err == Z_OK ? Z_BUF_ERROR : err); + } + *destLen = stream.total_out; + + err = zlib_deflateEnd(&stream); + zlib_workspace_free(stream.workspace); + + return (err); +} +EXPORT_SYMBOL(z_compress_level); + +/* + * Decompresses the source buffer into the destination buffer. sourceLen is + * the byte length of the source buffer. Upon entry, destLen is the total + * size of the destination buffer, which must be large enough to hold the + * entire uncompressed data. (The size of the uncompressed data must have + * been saved previously by the compressor and transmitted to the decompressor + * by some mechanism outside the scope of this compression library.) + * Upon exit, destLen is the actual size of the compressed buffer. + * This function can be used to decompress a whole file at once if the + * input file is mmap'ed. + * + * uncompress returns Z_OK if success, Z_MEM_ERROR if there was not + * enough memory, Z_BUF_ERROR if there was not enough room in the output + * buffer, or Z_DATA_ERROR if the input data was corrupted. + */ +int +z_uncompress(void *dest, size_t *destLen, const void *source, size_t sourceLen) +{ + z_stream stream; + int err; + + stream.next_in = (Byte *)source; + stream.avail_in = (uInt)sourceLen; + stream.next_out = dest; + stream.avail_out = (uInt)*destLen; + + if ((size_t)stream.avail_out != *destLen) + return (Z_BUF_ERROR); + + stream.workspace = zlib_workspace_alloc(KM_SLEEP); + if (!stream.workspace) + return (Z_MEM_ERROR); + + err = zlib_inflateInit(&stream); + if (err != Z_OK) { + zlib_workspace_free(stream.workspace); + return (err); + } + + err = zlib_inflate(&stream, Z_FINISH); + if (err != Z_STREAM_END) { + zlib_inflateEnd(&stream); + zlib_workspace_free(stream.workspace); + + if (err == Z_NEED_DICT || + (err == Z_BUF_ERROR && stream.avail_in == 0)) + return (Z_DATA_ERROR); + + return (err); + } + *destLen = stream.total_out; + + err = zlib_inflateEnd(&stream); + zlib_workspace_free(stream.workspace); + + return (err); +} +EXPORT_SYMBOL(z_uncompress); + +int +spl_zlib_init(void) +{ + int size; + SENTRY; + + size = MAX(spl_zlib_deflate_workspacesize(MAX_WBITS, MAX_MEM_LEVEL), + zlib_inflate_workspacesize()); + + zlib_workspace_cache = kmem_cache_create( + "spl_zlib_workspace_cache", + size, 0, NULL, NULL, NULL, NULL, NULL, + KMC_VMEM | KMC_NOEMERGENCY); + if (!zlib_workspace_cache) + SRETURN(1); + + SRETURN(0); +} + +void +spl_zlib_fini(void) +{ + SENTRY; + kmem_cache_destroy(zlib_workspace_cache); + zlib_workspace_cache = NULL; + SEXIT; +} diff --git a/module/os/windows/zfs/CMakeLists.txt b/module/os/windows/zfs/CMakeLists.txt new file mode 100644 index 000000000000..96d8ec50a7de --- /dev/null +++ b/module/os/windows/zfs/CMakeLists.txt @@ -0,0 +1,102 @@ + +use_clang() + +wdk_add_library(zfskern_os + KMDF 1.9 +abd_os.c +arc_os.c +qat.c +qat_compress.c +qat_crypt.c +spa_misc_os.c +sysctl_os.c +vdev_disk.c +vdev_file.c +zfs_acl.c +zfs_racct.c +zfs_ctldir.c +zfs_debug.c +zfs_dir.c +zfs_file_os.c +zfs_fuid_os.c +zfs_ioctl_os.c +zfs_vfsops.c +zfs_vnops_os.c +zfs_vnops_windows.c +zfs_vnops_windows_lib.c +zfs_vnops_windows_mount.c +zfs_vnops_windows_xattr.c +zfs_windows_zvol.c +zfs_windows_zvol_scsi.c +zfs_windows_zvol_wmi.c +zfs_znode.c +zio_crypt.c +zvol_os.c +) + +target_link_libraries(zfskern_os PRIVATE splkern icpkern) + +file(GLOB CSOURCES CONFIGURE_DEPENDS "*.c") + +list(REMOVE_ITEM CSOURCES "${CMAKE_CURRENT_SOURCE_DIR}/zfs_vnops_os.c" + "${CMAKE_CURRENT_SOURCE_DIR}/zfs_vnops_windows.c" + "${CMAKE_CURRENT_SOURCE_DIR}/zfs_vnops_windows_lib.c" + "${CMAKE_CURRENT_SOURCE_DIR}/zfs_vnops_windows_mount.c" + "${CMAKE_CURRENT_SOURCE_DIR}/zfs_windows_zvol.c" + "${CMAKE_CURRENT_SOURCE_DIR}/zvol_os.c") + +function(add_macro_property) + foreach (I ${CSOURCES}) + get_filename_component(OUTPUT_FILE_WE ${I} NAME_WE) + set_source_files_properties(${I} PROPERTIES COMPILE_FLAGS -DWPPFILE=${CMAKE_SOURCE_DIR}/out/build/${OUTPUT_FILE_WE}.tmh) + message(STATUS "FILES_IN ======: ${CMAKE_SOURCE_DIR}/out/build/${OUTPUT_FILE_WE}.tmh") + endforeach() +endfunction() + +function(tracewpp OUTPUT_DIR SOURCE) + # [mlr] list of .tmh files to be generated -> TMH + set(WPP_DIR "${WDK_ROOT}/bin/${WDK_VERSION}") + + get_filename_component(FILEN ${SOURCE} NAME) + set(TMH_FILEN ${FILEN}.tmh) + set(TMH ${OUTPUT_DIR}/${TMH_FILEN}) + set(EXTENSIONS ".c") + + # [mlr] cmake only converts the command name to the native path format. the + # path names to be used in arguments must be converted manually. + + file(TO_NATIVE_PATH ${SOURCE} NATIVE_SOURCE) + file(TO_NATIVE_PATH ${WPP_DIR} NATIVE_WPP_DIR) + file(TO_NATIVE_PATH ${OUTPUT_DIR} NATIVE_OUTPUT_DIR) + + # [mlr] note that if -preserveext: occurs after the source file specification, it has + # no effect. + + set(TRACE "TraceEvent{FLAGS=MYDRIVER_ALL_INFO}(LEVEL, MSG, ...)") + set(DPRINT "dprintf{FLAGS=MYDRIVER_ALL_INFO, LEVEL=TRACE_INFO}(MSG, ...)") + set(CFGDIR "${WPP_DIR}/wppconfig/rev1") + set(SCAN "${WDK_ROOT}/Include/wdf/kmdf/1.9/WdfTraceEnums.h") + set(WPP_MACRO "WPP_INLINE __inline") + + execute_process(COMMAND "${NATIVE_WPP_DIR}/${WDK_PLATFORM}/tracewpp.exe" + -scan:${SCAN} /D${WPP_MACRO} + -cfgdir:${CFGDIR} -I${CMAKE_CURRENT_BINARY_DIR} -odir:${NATIVE_OUTPUT_DIR} -km -func:${TRACE} + -func:${DPRINT} -gen:{km-default.tpl}*.tmh ${NATIVE_SOURCE}) + +endfunction() + + +function(wpp OUTPUT_DIR) + + add_macro_property() + # [mlr] invoke tracewpp() for each source file, adding the resulting file to a list + # named TMH. + message(STATUS "OUTPUT_DIR ======: ${OUTPUT_DIR}") + + foreach ( I ${CSOURCES} ) + tracewpp(${OUTPUT_DIR} ${I}) + endforeach() + +endfunction() + +wpp("${CMAKE_SOURCE_DIR}/out/build") diff --git a/module/os/windows/zfs/abd_os.c b/module/os/windows/zfs/abd_os.c new file mode 100644 index 000000000000..8a4e06b0d1a4 --- /dev/null +++ b/module/os/windows/zfs/abd_os.c @@ -0,0 +1,490 @@ +/* + * This file and its contents are supplied under the terms of the + * Common Development and Distribution License ("CDDL"), version 1.0. + * You may only use this file in accordance with the terms of version + * 1.0 of the CDDL. + * + * A full copy of the text of the CDDL should have accompanied this + * source. A copy of the CDDL is also available via the Internet at + * http://www.illumos.org/license/CDDL. + */ + +/* + * Copyright (c) 2014 by Chunwei Chen. All rights reserved. + * Copyright (c) 2016 by Delphix. All rights reserved. + * Copyright (c) 2020 by Jorgen Lundman. All rights reserved. + */ + +/* + * See abd.c for a general overview of the arc buffered data (ABD). + * + * Using a large proportion of scattered ABDs decreases ARC fragmentation since + * when we are at the limit of allocatable space, using equal-size chunks will + * allow us to quickly reclaim enough space for a new large allocation (assuming + * it is also scattered). + * + * ABDs are allocated scattered by default unless the caller uses + * abd_alloc_linear() or zfs_abd_scatter_enabled is disabled. + */ + +#include +#include +#include +#include +#include +#include + +typedef struct abd_stats { + kstat_named_t abdstat_struct_size; + kstat_named_t abdstat_scatter_cnt; + kstat_named_t abdstat_scatter_data_size; + kstat_named_t abdstat_scatter_chunk_waste; + kstat_named_t abdstat_linear_cnt; + kstat_named_t abdstat_linear_data_size; +} abd_stats_t; + +static abd_stats_t abd_stats = { + /* Amount of memory occupied by all of the abd_t struct allocations */ + { "struct_size", KSTAT_DATA_UINT64 }, + /* + * The number of scatter ABDs which are currently allocated, excluding + * ABDs which don't own their data (for instance the ones which were + * allocated through abd_get_offset()). + */ + { "scatter_cnt", KSTAT_DATA_UINT64 }, + /* Amount of data stored in all scatter ABDs tracked by scatter_cnt */ + { "scatter_data_size", KSTAT_DATA_UINT64 }, + /* + * The amount of space wasted at the end of the last chunk across all + * scatter ABDs tracked by scatter_cnt. + */ + { "scatter_chunk_waste", KSTAT_DATA_UINT64 }, + /* + * The number of linear ABDs which are currently allocated, excluding + * ABDs which don't own their data (for instance the ones which were + * allocated through abd_get_offset() and abd_get_from_buf()). If an + * ABD takes ownership of its buf then it will become tracked. + */ + { "linear_cnt", KSTAT_DATA_UINT64 }, + /* Amount of data stored in all linear ABDs tracked by linear_cnt */ + { "linear_data_size", KSTAT_DATA_UINT64 }, +}; + +static struct { + wmsum_t abdstat_struct_size; + wmsum_t abdstat_scatter_cnt; + wmsum_t abdstat_scatter_data_size; + wmsum_t abdstat_scatter_chunk_waste; + wmsum_t abdstat_linear_cnt; + wmsum_t abdstat_linear_data_size; +} abd_sums; + +/* + * The size of the chunks ABD allocates. Because the sizes allocated from the + * kmem_cache can't change, this tunable can only be modified at boot. Changing + * it at runtime would cause ABD iteration to work incorrectly for ABDs which + * were allocated with the old size, so a safeguard has been put in place which + * will cause the machine to panic if you change it and try to access the data + * within a scattered ABD. + */ +size_t zfs_abd_chunk_size = 4096; + +lookasidelist_cache_t *abd_chunk_cache; +static kstat_t *abd_ksp; + + +/* + * We use a scattered SPA_MAXBLOCKSIZE sized ABD whose chunks are + * just a single zero'd sized zfs_abd_chunk_size buffer. This + * allows us to conserve memory by only using a single zero buffer + * for the scatter chunks. + */ +abd_t *abd_zero_scatter = NULL; +static char *abd_zero_buf = NULL; + +static void +abd_free_chunk(void *c) +{ + lookasidelist_cache_free(abd_chunk_cache, c); +} + +static size_t +abd_chunkcnt_for_bytes(size_t size) +{ + return (P2ROUNDUP(size, zfs_abd_chunk_size) / zfs_abd_chunk_size); +} + +static inline size_t +abd_scatter_chunkcnt(abd_t *abd) +{ + ASSERT(!abd_is_linear(abd)); + return (abd_chunkcnt_for_bytes( + ABD_SCATTER(abd).abd_offset + abd->abd_size)); +} + +boolean_t +abd_size_alloc_linear(size_t size) +{ + return (size <= zfs_abd_chunk_size ? B_TRUE : B_FALSE); +} + +void +abd_update_scatter_stats(abd_t *abd, abd_stats_op_t op) +{ + size_t n = abd_scatter_chunkcnt(abd); + ASSERT(op == ABDSTAT_INCR || op == ABDSTAT_DECR); + if (op == ABDSTAT_INCR) { + ABDSTAT_BUMP(abdstat_scatter_cnt); + ABDSTAT_INCR(abdstat_scatter_data_size, abd->abd_size); + ABDSTAT_INCR(abdstat_scatter_chunk_waste, + n * zfs_abd_chunk_size - abd->abd_size); + } else { + ABDSTAT_BUMPDOWN(abdstat_scatter_cnt); + ABDSTAT_INCR(abdstat_scatter_data_size, -(int)abd->abd_size); + ABDSTAT_INCR(abdstat_scatter_chunk_waste, + abd->abd_size - n * zfs_abd_chunk_size); + } +} + +void +abd_update_linear_stats(abd_t *abd, abd_stats_op_t op) +{ + ASSERT(op == ABDSTAT_INCR || op == ABDSTAT_DECR); + if (op == ABDSTAT_INCR) { + ABDSTAT_BUMP(abdstat_linear_cnt); + ABDSTAT_INCR(abdstat_linear_data_size, abd->abd_size); + } else { + ABDSTAT_BUMPDOWN(abdstat_linear_cnt); + ABDSTAT_INCR(abdstat_linear_data_size, -(int)abd->abd_size); + } +} + +void +abd_verify_scatter(abd_t *abd) +{ + /* + * There is no scatter linear pages in FreeBSD so there is an + * if an error if the ABD has been marked as a linear page. + */ + VERIFY(!abd_is_linear_page(abd)); + VERIFY3U(ABD_SCATTER(abd).abd_offset, <, + zfs_abd_chunk_size); + + size_t n = abd_scatter_chunkcnt(abd); + for (int i = 0; i < n; i++) { + ASSERT3P( + ABD_SCATTER(abd).abd_chunks[i], !=, NULL); + } +} + +void +abd_alloc_chunks(abd_t *abd, size_t size) +{ + size_t n = abd_chunkcnt_for_bytes(size); + for (int i = 0; i < n; i++) { + void *c = lookasidelist_cache_alloc(abd_chunk_cache); + ABD_SCATTER(abd).abd_chunks[i] = c; + } + ABD_SCATTER(abd).abd_chunk_size = zfs_abd_chunk_size; +} + +void +abd_free_chunks(abd_t *abd) +{ + size_t n = abd_scatter_chunkcnt(abd); + for (int i = 0; i < n; i++) { + abd_free_chunk(ABD_SCATTER(abd).abd_chunks[i]); + } +} + +abd_t * +abd_alloc_struct_impl(size_t size) +{ + size_t chunkcnt = abd_chunkcnt_for_bytes(size); + /* + * In the event we are allocating a gang ABD, the size passed in + * will be 0. We must make sure to set abd_size to the size of an + * ABD struct as opposed to an ABD scatter with 0 chunks. The gang + * ABD struct allocation accounts for an additional 24 bytes over + * a scatter ABD with 0 chunks. + */ + size_t abd_size = MAX(sizeof (abd_t), + offsetof(abd_t, abd_u.abd_scatter.abd_chunks[chunkcnt])); + abd_t *abd = kmem_zalloc(abd_size, KM_PUSHPAGE); + ABDSTAT_INCR(abdstat_struct_size, abd_size); + + return (abd); +} + +void +abd_free_struct_impl(abd_t *abd) +{ + uint_t chunkcnt = abd_is_linear(abd) || abd_is_gang(abd) ? 0 : + abd_scatter_chunkcnt(abd); + ssize_t size = MAX(sizeof (abd_t), + offsetof(abd_t, abd_u.abd_scatter.abd_chunks[chunkcnt])); + + kmem_free(abd, size); + ABDSTAT_INCR(abdstat_struct_size, -size); +} + +/* + * Allocate scatter ABD of size SPA_MAXBLOCKSIZE, where + * each chunk in the scatterlist will be set to abd_zero_buf. + */ +static void +abd_alloc_zero_scatter(void) +{ + size_t n = abd_chunkcnt_for_bytes(SPA_MAXBLOCKSIZE); + abd_zero_buf = lookasidelist_cache_alloc(abd_chunk_cache); + memset(abd_zero_buf, 0, zfs_abd_chunk_size); + abd_zero_scatter = abd_alloc_struct(SPA_MAXBLOCKSIZE); + + abd_zero_scatter->abd_flags |= ABD_FLAG_OWNER | ABD_FLAG_ZEROS; + abd_zero_scatter->abd_size = SPA_MAXBLOCKSIZE; + + ABD_SCATTER(abd_zero_scatter).abd_offset = 0; + ABD_SCATTER(abd_zero_scatter).abd_chunk_size = + zfs_abd_chunk_size; + + for (int i = 0; i < n; i++) { + ABD_SCATTER(abd_zero_scatter).abd_chunks[i] = + abd_zero_buf; + } + + ABDSTAT_BUMP(abdstat_scatter_cnt); + ABDSTAT_INCR(abdstat_scatter_data_size, zfs_abd_chunk_size); +} + +static void +abd_free_zero_scatter(void) +{ + ABDSTAT_BUMPDOWN(abdstat_scatter_cnt); + ABDSTAT_INCR(abdstat_scatter_data_size, -(int)zfs_abd_chunk_size); + + abd_free_struct(abd_zero_scatter); + abd_zero_scatter = NULL; + lookasidelist_cache_free(abd_chunk_cache, abd_zero_buf); +} + +void +abd_init(void) +{ + abd_chunk_cache = lookasidelist_cache_create("abd_chunk", + zfs_abd_chunk_size); + + abd_ksp = kstat_create("zfs", 0, "abdstats", "misc", KSTAT_TYPE_NAMED, + sizeof (abd_stats) / sizeof (kstat_named_t), KSTAT_FLAG_VIRTUAL); + if (abd_ksp != NULL) { + abd_ksp->ks_data = &abd_stats; + kstat_install(abd_ksp); + } + + abd_alloc_zero_scatter(); +} + +void +abd_fini(void) +{ + abd_free_zero_scatter(); + + if (abd_ksp != NULL) { + kstat_delete(abd_ksp); + abd_ksp = NULL; + } + + lookasidelist_cache_destroy(abd_chunk_cache); + abd_chunk_cache = NULL; +} + +void +abd_free_linear_page(abd_t *abd) +{ + /* + * FreeBSD does not have have scatter linear pages + * so there is an error. + */ + VERIFY(0); +} + +/* + * If we're going to use this ABD for doing I/O using the block layer, the + * consumer of the ABD data doesn't care if it's scattered or not, and we don't + * plan to store this ABD in memory for a long period of time, we should + * allocate the ABD type that requires the least data copying to do the I/O. + * + * Currently this is linear ABDs, however if ldi_strategy() can ever issue I/Os + * using a scatter/gather list we should switch to that and replace this call + * with vanilla abd_alloc(). + */ +abd_t * +abd_alloc_for_io(size_t size, boolean_t is_metadata) +{ + return (abd_alloc_linear(size, is_metadata)); +} + +abd_t * +abd_get_offset_scatter(abd_t *abd, abd_t *sabd, size_t off, size_t size) +{ + abd_verify(sabd); + VERIFY3U(off, <=, sabd->abd_size); + + size_t new_offset = ABD_SCATTER(sabd).abd_offset + off; + + /* + * chunkcnt is abd_chunkcnt_for_bytes(size), which rounds + * up to the nearest chunk, but we also must take care + * of the offset *in the leading chunk* + */ + size_t chunkcnt = abd_chunkcnt_for_bytes( + (new_offset % zfs_abd_chunk_size) + size); + + VERIFY3U(chunkcnt, <=, abd_scatter_chunkcnt(sabd)); + + /* + * If an abd struct is provided, it is only the minimum size. If we + * need additional chunks, we need to allocate a new struct. + */ + if (abd != NULL && + offsetof(abd_t, abd_u.abd_scatter.abd_chunks[chunkcnt]) > + sizeof (abd_t)) { + abd = NULL; + } + + if (abd == NULL) + abd = abd_alloc_struct(chunkcnt * zfs_abd_chunk_size); + + /* + * Even if this buf is filesystem metadata, we only track that + * if we own the underlying data buffer, which is not true in + * this case. Therefore, we don't ever use ABD_FLAG_META here. + */ + + ABD_SCATTER(abd).abd_offset = new_offset % zfs_abd_chunk_size; + ABD_SCATTER(abd).abd_chunk_size = zfs_abd_chunk_size; + + /* Copy the scatterlist starting at the correct offset */ + (void) memcpy(&ABD_SCATTER(abd).abd_chunks, + &ABD_SCATTER(sabd).abd_chunks[new_offset / + zfs_abd_chunk_size], + chunkcnt * sizeof (void *)); + + return (abd); +} + +static inline size_t +abd_iter_scatter_chunk_offset(struct abd_iter *aiter) +{ + ASSERT(!abd_is_linear(aiter->iter_abd)); + return ((ABD_SCATTER(aiter->iter_abd).abd_offset + + aiter->iter_pos) % zfs_abd_chunk_size); +} + +static inline size_t +abd_iter_scatter_chunk_index(struct abd_iter *aiter) +{ + ASSERT(!abd_is_linear(aiter->iter_abd)); + return ((ABD_SCATTER(aiter->iter_abd).abd_offset + + aiter->iter_pos) / zfs_abd_chunk_size); +} + +/* + * Initialize the abd_iter. + */ +void +abd_iter_init(struct abd_iter *aiter, abd_t *abd) +{ + ASSERT(!abd_is_gang(abd)); + abd_verify(abd); + aiter->iter_abd = abd; + aiter->iter_pos = 0; + aiter->iter_mapaddr = NULL; + aiter->iter_mapsize = 0; +} + +/* + * This is just a helper function to see if we have exhausted the + * abd_iter and reached the end. + */ +boolean_t +abd_iter_at_end(struct abd_iter *aiter) +{ + return (aiter->iter_pos == aiter->iter_abd->abd_size); +} + +/* + * Advance the iterator by a certain amount. Cannot be called when a chunk is + * in use. This can be safely called when the aiter has already exhausted, in + * which case this does nothing. + */ +void +abd_iter_advance(struct abd_iter *aiter, size_t amount) +{ + ASSERT3P(aiter->iter_mapaddr, ==, NULL); + ASSERT0(aiter->iter_mapsize); + + /* There's nothing left to advance to, so do nothing */ + if (abd_iter_at_end(aiter)) + return; + + aiter->iter_pos += amount; +} + +/* + * Map the current chunk into aiter. This can be safely called when the aiter + * has already exhausted, in which case this does nothing. + */ +void +abd_iter_map(struct abd_iter *aiter) +{ + void *paddr; + size_t offset = 0; + + ASSERT3P(aiter->iter_mapaddr, ==, NULL); + ASSERT0(aiter->iter_mapsize); + + /* Panic if someone has changed zfs_abd_chunk_size */ + IMPLY(!abd_is_linear(aiter->iter_abd), zfs_abd_chunk_size == + ABD_SCATTER(aiter->iter_abd).abd_chunk_size); + + /* There's nothing left to iterate over, so do nothing */ + if (abd_iter_at_end(aiter)) + return; + + if (abd_is_linear(aiter->iter_abd)) { + offset = aiter->iter_pos; + aiter->iter_mapsize = aiter->iter_abd->abd_size - offset; + paddr = ABD_LINEAR_BUF(aiter->iter_abd); + } else { + size_t index = abd_iter_scatter_chunk_index(aiter); + offset = abd_iter_scatter_chunk_offset(aiter); + aiter->iter_mapsize = MIN(zfs_abd_chunk_size - offset, + aiter->iter_abd->abd_size - aiter->iter_pos); + paddr = ABD_SCATTER(aiter->iter_abd).abd_chunks[index]; + } + aiter->iter_mapaddr = (char *)paddr + offset; +} + +/* + * Unmap the current chunk from aiter. This can be safely called when the aiter + * has already exhausted, in which case this does nothing. + */ +void +abd_iter_unmap(struct abd_iter *aiter) +{ + /* There's nothing left to unmap, so do nothing */ + if (abd_iter_at_end(aiter)) + return; + + ASSERT3P(aiter->iter_mapaddr, !=, NULL); + ASSERT3U(aiter->iter_mapsize, >, 0); + + aiter->iter_mapaddr = NULL; + aiter->iter_mapsize = 0; +} + +void +abd_cache_reap_now(void) +{ + // do nothing +} diff --git a/module/os/windows/zfs/arc_os.c b/module/os/windows/zfs/arc_os.c new file mode 100644 index 000000000000..65a2ab075f42 --- /dev/null +++ b/module/os/windows/zfs/arc_os.c @@ -0,0 +1,828 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2018, Joyent, Inc. + * Copyright (c) 2011, 2019 by Delphix. All rights reserved. + * Copyright (c) 2014 by Saso Kiselkov. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. All rights reserved. + * Portions Copyright 2022 Andrew Innes + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#ifdef _KERNEL +#include +#include +#endif +#include +#include +#include +#include +#include +#include +#include + +extern arc_stats_t arc_stats; + +static kmutex_t arc_reclaim_lock; +static kcondvar_t arc_reclaim_thread_cv; +static boolean_t arc_reclaim_thread_exit; +static kcondvar_t arc_reclaim_waiters_cv; + +/* + * log2(fraction of ARC which must be free to allow growing). + * I.e. If there is less than arc_c >> arc_no_grow_shift free memory, + * when reading a new block into the ARC, we will evict an equal-sized block + * from the ARC. + * + * This must be less than arc_shrink_shift, so that when we shrink the ARC, + * we will still not allow it to grow. + */ +extern uint_t arc_no_grow_shift; + +extern uint64_t total_memory; +extern uint64_t real_total_memory; + +extern boolean_t spl_minimal_physmem_p_logic(); + +/* + * Return a default max arc size based on the amount of physical memory. + */ +uint64_t +arc_default_max(uint64_t min, uint64_t allmem) +{ + /* Default to 1/3 of all memory. */ + return (MAX(allmem, min)); +} + +#ifdef _KERNEL + +static _Atomic boolean_t arc_reclaim_in_loop = B_FALSE; + +/* + * Return maximum amount of memory that we could possibly use. Reduced + * to half of all memory in user space which is primarily used for testing. + */ +uint64_t +arc_all_memory(void) +{ + return (kmem_size()); +} + +/* + * Return the amount of memory that is considered free. In user space + * which is primarily used for testing we pretend that free memory ranges + * from 0-20% of all memory. + */ +uint64_t +arc_free_memory(void) +{ + int64_t avail; + + avail = spl_free_wrapper(); + return (avail >= 0LL ? avail : 0LL); +} + +/* + * Return the amount of memory that can be consumed before reclaim will be + * needed. Positive if there is sufficient free memory, negative indicates + * the amount of memory that needs to be freed up. + */ +int64_t +arc_available_memory(void) +{ + return (arc_free_memory() - arc_sys_free); +} + +int +arc_memory_throttle(spa_t *spa, uint64_t reserve, uint64_t txg) +{ + + /* possibly wake up arc reclaim thread */ + + if (arc_reclaim_in_loop == B_FALSE) { + if (spl_free_manual_pressure_wrapper() != 0 || + !spl_minimal_physmem_p() || + arc_reclaim_needed()) { + cv_signal(&arc_reclaim_thread_cv); + kpreempt(KPREEMPT_SYNC); + ARCSTAT_INCR(arcstat_memory_throttle_count, 1); + } + } + + return (0); +} + +/* + * arc.c has a arc_reap_zthr we should probably use, instead of + * having our own legacy arc_reclaim_thread(). + */ +static void arc_kmem_reap_now(void) +{ + arc_wait_for_eviction(0, B_FALSE); + + /* arc.c will do the heavy lifting */ + arc_kmem_reap_soon(); +} + +/* + * Threads can block in arc_get_data_impl() waiting for this thread to evict + * enough data and signal them to proceed. When this happens, the threads in + * arc_get_data_impl() are sleeping while holding the hash lock for their + * particular arc header. Thus, we must be careful to never sleep on a + * hash lock in this thread. This is to prevent the following deadlock: + * + * - Thread A sleeps on CV in arc_get_data_impl() holding hash lock "L", + * waiting for the reclaim thread to signal it. + * + * - arc_reclaim_thread() tries to acquire hash lock "L" using mutex_enter, + * fails, and goes to sleep forever. + * + * This possible deadlock is avoided by always acquiring a hash lock + * using mutex_tryenter() from arc_reclaim_thread(). + */ +static void +arc_reclaim_thread(void *unused) +{ + hrtime_t growtime = 0; + + callb_cpr_t cpr; + + CALLB_CPR_INIT(&cpr, &arc_reclaim_lock, callb_generic_cpr, FTAG); + + mutex_enter(&arc_reclaim_lock); + while (!arc_reclaim_thread_exit) { + arc_reclaim_in_loop = B_TRUE; + + mutex_exit(&arc_reclaim_lock); + + const int64_t pre_adjust_free_memory = MIN(spl_free_wrapper(), + arc_available_memory()); + + int64_t manual_pressure = spl_free_manual_pressure_wrapper(); + spl_free_set_pressure(0); // clears both spl pressure variables + + /* + * We call arc_adjust() before (possibly) calling + * arc_kmem_reap_now(), so that we can wake up + * arc_get_data_impl() sooner. + */ + + if (manual_pressure > 0) { + arc_reduce_target_size(MIN(manual_pressure, + (arc_c >> arc_shrink_shift))); + } + + arc_wait_for_eviction(0, B_FALSE); + + int64_t free_memory = arc_available_memory(); + + const int64_t post_adjust_manual_pressure = + spl_free_manual_pressure_wrapper(); + + /* maybe we are getting lots of pressure from spl */ + manual_pressure = MAX(manual_pressure, + post_adjust_manual_pressure); + + spl_free_set_pressure(0); + + const int64_t post_adjust_free_memory = + MIN(spl_free_wrapper(), arc_available_memory()); + + // if arc_adjust() evicted, we expect post_adjust_free_memory + // to be larger than pre_adjust_free_memory (as there should + // be more free memory). + + /* + * d_adj tracks the change of memory across the call + * to arc_wait_for_eviction(), and will count the number + * of bytes the spl_free_thread calculates has been + * made free (signed) + */ + + const int64_t d_adj = post_adjust_free_memory - + pre_adjust_free_memory; + + if (manual_pressure > 0 && post_adjust_manual_pressure == 0) { + // pressure did not get re-signalled during arc_adjust() + if (d_adj > 0) + manual_pressure -= d_adj; + } else if (manual_pressure > 0 && + post_adjust_manual_pressure > 0) { + // otherwise use the most recent pressure value + manual_pressure = post_adjust_manual_pressure; + } + + /* + * If we have successfully freed a bunch of memory, + * it is worth reaping the abd_chunk_cache + */ + if (d_adj >= 64LL*1024LL*1024LL) { + abd_cache_reap_now(); + } + + free_memory = post_adjust_free_memory; + + const hrtime_t curtime = gethrtime(); + + if (free_memory < 0 || manual_pressure > 0) { + + if (manual_pressure > 0 || free_memory <= + (arc_c >> arc_no_grow_shift) + SPA_MAXBLOCKSIZE) { + + arc_no_grow = B_TRUE; + + /* + * Absorb occasional low memory conditions, as + * they may be caused by a single sequentially + * writing thread pushing a lot of dirty data + * into the ARC. + * + * In particular, we want to quickly begin + * re-growing the ARC if we are not in chronic + * high pressure. However, if we're in + * chronic high pressure, we want to reduce + * reclaim thread work by keeping arc_no_grow + * set. + * + * If growtime is in the past, then set it to + * last half a second (which is the length of + * the cv_timedwait_hires() call below). + * + * If growtime is in the future, then make + * sure that it is no further than 60 seconds + * into the future. + * + * If growtime is less than 60 seconds in the + * future, then grow growtime by an + * exponentially increasing value starting + * with 500msec. + * + */ + const hrtime_t agr = SEC2NSEC(arc_grow_retry); + static int grow_pass = 0; + + if (growtime == 0) { + growtime = curtime + MSEC2NSEC(500); + grow_pass = 0; + } else { + // check for 500ms not being enough + // ASSERT3U(growtime, >, curtime); + if (growtime <= curtime) + growtime = curtime + + MSEC2NSEC(500); + + // growtime is in the future! + const hrtime_t difference = + growtime - curtime; + + if (difference >= agr) { + // cap arc_grow_retry secs now + growtime = curtime + agr - 1LL; + grow_pass = 0; + } else { + /* + * with each pass, push + * turning off arc_no_grow + * by longer + */ + hrtime_t grow_by = + MSEC2NSEC(500) * + (1LL << grow_pass); + + if (grow_by > (agr >> 1)) + grow_by = agr >> 1; + + growtime += grow_by; + + // add 512 seconds maximum + if (grow_pass < 10) + grow_pass++; + } + } + } + + arc_warm = B_TRUE; + + arc_kmem_reap_now(); + + /* + * If we are still low on memory, shrink the ARC + * so that we have arc_shrink_min free space. + */ + free_memory = arc_available_memory(); + + int64_t to_free = + (arc_c >> arc_shrink_shift) - free_memory; + + if (to_free > 0 || manual_pressure != 0) { + + to_free = MAX(to_free, manual_pressure); + + arc_reduce_target_size(to_free); + + goto lock_and_sleep; + } + } else if (free_memory < (arc_c >> arc_no_grow_shift) && + aggsum_value(&arc_sums.arcstat_size) > + arc_c_min + SPA_MAXBLOCKSIZE) { + // relatively low memory and arc is above arc_c_min + arc_no_grow = B_TRUE; + growtime = curtime + SEC2NSEC(1); + goto lock_and_sleep; + } + + /* + * The abd vmem layer can see a large number of + * frees from the abd kmem cache layer, and unfortunately + * the abd vmem layer might end up fragmented as a result. + * + * Watch for this fragmentation and if it arises + * suppress ARC growth for ten minutes in hopes that + * abd activity driven by ARC replacement or further ARC + * shrinking lets the abd vmem layer defragment. + */ + + if (arc_no_grow != B_TRUE) { + /* + * The gap is between imported and inuse + * in the abd vmem layer + */ + + static hrtime_t when_gap_grew = 0; + static int64_t previous_gap = 0; + static int64_t previous_abd_size = 0; + + int64_t gap = abd_arena_empty_space(); + int64_t abd_size = abd_arena_total_size(); + + if (gap == 0) { + /* + * no abd vmem layer fragmentation + * so don't adjust arc_no_grow + */ + previous_gap = 0; + previous_abd_size = abd_size; + } else if (gap > 0 && gap == previous_gap && + abd_size == previous_abd_size) { + if (curtime < when_gap_grew + SEC2NSEC(600)) { + /* + * our abd arena is unchanged + * try up to ten minutes for kmem layer + * to free slabs to abd vmem layer + */ + arc_no_grow = B_TRUE; + growtime = curtime + + SEC2NSEC(arc_grow_retry); + previous_abd_size = abd_size; + } else { + /* + * ten minutes have expired with no + * good result, shrink the arc a little, + * no more than once every + * arc_grow_retry (5) seconds + */ + arc_no_grow = B_TRUE; + growtime = curtime + + SEC2NSEC(arc_grow_retry); + previous_abd_size = abd_size; + + const int64_t sb = + arc_c >> arc_shrink_shift; + if (arc_c_min + sb > arc_c) { + arc_reduce_target_size(sb); + goto lock_and_sleep; + } + } + } else if (gap > 0 && gap > previous_gap) { + /* + * kmem layer must have freed slabs + * but vmem layer is holding on because + * of fragmentation. Don't grow ARC + * for a minute. + */ + arc_no_grow = B_TRUE; + growtime = curtime + SEC2NSEC(arc_grow_retry); + previous_gap = gap; + when_gap_grew = curtime; + /* + * but if we're growing the abd + * as well as its gap, shrink + */ + if (abd_size > previous_abd_size) { + const int64_t sb = + arc_c >> arc_shrink_shift; + if (arc_c_min + sb > arc_c) + arc_reduce_target_size(sb); + } + previous_abd_size = abd_size; + } else if (gap > 0 && gap < previous_gap) { + /* + * vmem layer successfully freeing. + */ + if (curtime < when_gap_grew + SEC2NSEC(600)) { + arc_no_grow = B_TRUE; + growtime = curtime + + SEC2NSEC(arc_grow_retry); + } + previous_gap = gap; + previous_abd_size = abd_size; + } else { + previous_abd_size = abd_size; + } + } + + if (growtime > 0 && curtime >= growtime) { + if (arc_no_grow == B_TRUE) + dprintf("ZFS: arc growtime expired\n"); + growtime = 0; + arc_no_grow = B_FALSE; + } + +lock_and_sleep: + + arc_reclaim_in_loop = B_FALSE; + + mutex_enter(&arc_reclaim_lock); + + /* + * If d_adj is non-positive, we didn't evict anything, + * perhaps because nothing was evictable. Immediately + * running another pass is unlikely to be helpful. + */ + + if (aggsum_compare(&arc_sums.arcstat_size, arc_c) <= 0 || + d_adj <= 0) { + /* + * We're either no longer overflowing, or we + * can't evict anything more, so we should wake + * up any threads before we go to sleep. + */ + cv_broadcast(&arc_reclaim_waiters_cv); + + /* + * Block until signaled, or after one second (we + * might need to perform arc_kmem_reap_now() + * even if we aren't being signalled) + */ + CALLB_CPR_SAFE_BEGIN(&cpr); + (void) cv_timedwait_hires(&arc_reclaim_thread_cv, + &arc_reclaim_lock, MSEC2NSEC(500), MSEC2NSEC(1), 0); + CALLB_CPR_SAFE_END(&cpr, &arc_reclaim_lock); + + } else if (d_adj >= SPA_MAXBLOCKSIZE * 3) { + // we evicted plenty of buffers, so let's wake up + // all the waiters rather than having them stall + cv_broadcast(&arc_reclaim_waiters_cv); + } else { + // we evicted some buffers but are still overflowing, + // so wake up only one waiter + cv_signal(&arc_reclaim_waiters_cv); + } + } + + arc_reclaim_thread_exit = B_FALSE; + cv_broadcast(&arc_reclaim_thread_cv); + CALLB_CPR_EXIT(&cpr); /* drops arc_reclaim_lock */ + thread_exit(); +} + +uint64_t +isqrt(uint64_t n) +{ + int i; + uint64_t r, tmp; + r = 0; + for (i = 64/2-1; i >= 0; i--) { + tmp = r | (1 << i); + if (tmp*tmp <= n) + r = tmp; + } + return (r); +} + +/* This is called before arc is initialized, and threads are not running */ +void +arc_lowmem_init(void) +{ + /* + * The ARC tries to keep at least this much memory available for the + * system. This gives the ARC time to shrink in response to memory + * pressure, before running completely out of memory and invoking the + * direct-reclaim ARC shrinker. + * + * arc_wait_for_eviction() waits for half of arc_sys_free. Bump this up + * to 3x to ensure we're above it. + */ + VERIFY3U(arc_all_memory(), >, 0); + arc_sys_free = arc_all_memory() / 128LL; + +} + +/* This is called after arc is initialized, and thread are running */ +void +arc_os_init(void) +{ + mutex_init(&arc_reclaim_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&arc_reclaim_thread_cv, NULL, CV_DEFAULT, NULL); + cv_init(&arc_reclaim_waiters_cv, NULL, CV_DEFAULT, NULL); + + arc_reclaim_thread_exit = B_FALSE; + + (void) thread_create(NULL, 0, arc_reclaim_thread, NULL, 0, &p0, + TS_RUN, minclsyspri); + + arc_warm = B_FALSE; + +} + +void +arc_lowmem_fini(void) +{ +} + +void +arc_os_fini(void) +{ + mutex_enter(&arc_reclaim_lock); + arc_reclaim_thread_exit = B_TRUE; + /* + * The reclaim thread will set arc_reclaim_thread_exit back to + * B_FALSE when it is finished exiting; we're waiting for that. + */ + while (arc_reclaim_thread_exit) { + cv_signal(&arc_reclaim_thread_cv); + cv_wait(&arc_reclaim_thread_cv, &arc_reclaim_lock); + } + mutex_exit(&arc_reclaim_lock); + + mutex_destroy(&arc_reclaim_lock); + cv_destroy(&arc_reclaim_thread_cv); + cv_destroy(&arc_reclaim_waiters_cv); +} + +/* + * Uses ARC static variables in logic. + */ +#define arc_meta_limit ARCSTAT(arcstat_meta_limit) /* max size for metadata */ +/* max size for dnodes */ +#define arc_dnode_size_limit ARCSTAT(arcstat_dnode_limit) +#define arc_meta_min ARCSTAT(arcstat_meta_min) /* min size for metadata */ +#define arc_meta_max ARCSTAT(arcstat_meta_max) /* max size of metadata */ + +/* So close, they made arc_min_prefetch_ms be static, but no others */ +#if 0 +int +arc_kstat_update_windows(kstat_t *ksp, int rw) +{ + windows_kstat_t *ks = ksp->ks_data; + + if (rw == KSTAT_WRITE) { + + /* Did we change the value ? */ + if (ks->arc_zfs_arc_max.value.ui64 != zfs_arc_max) { + + /* Assign new value */ + zfs_arc_max = ks->arc_zfs_arc_max.value.ui64; + + /* Update ARC with new value */ + if (zfs_arc_max > 64<<20 && zfs_arc_max < + physmem * PAGESIZE) + arc_c_max = zfs_arc_max; + + arc_c = arc_c_max; + arc_p = (arc_c >> 1); + + /* If meta_limit is not set, adjust it automatically */ + if (!zfs_arc_meta_limit) + arc_meta_limit = arc_c_max / 4; + } + + if (ks->arc_zfs_arc_min.value.ui64 != zfs_arc_min) { + zfs_arc_min = ks->arc_zfs_arc_min.value.ui64; + if (zfs_arc_min > 64<<20 && zfs_arc_min <= arc_c_max) { + arc_c_min = zfs_arc_min; + dprintf("ZFS: set arc_c_min %llu, arc_meta_min " + "%llu, zfs_arc_meta_min %llu\n", + arc_c_min, arc_meta_min, zfs_arc_meta_min); + if (arc_c < arc_c_min) { + dprintf("ZFS: raise arc_c %llu to " + "arc_c_min %llu\n", arc_c, + arc_c_min); + arc_c = arc_c_min; + if (arc_p < (arc_c >> 1)) { + dprintf("ZFS: raise arc_p %llu " + "to %llu\n", + arc_p, (arc_c >> 1)); + arc_p = (arc_c >> 1); + } + } + } + } + + if (ks->arc_zfs_arc_meta_limit.value.ui64 != + zfs_arc_meta_limit) { + zfs_arc_meta_limit = + ks->arc_zfs_arc_meta_limit.value.ui64; + + /* Allow the tunable to override if it is reasonable */ + if (zfs_arc_meta_limit > 0 && + zfs_arc_meta_limit <= arc_c_max) + arc_meta_limit = zfs_arc_meta_limit; + + if (arc_c_min < arc_meta_limit / 2 && + zfs_arc_min == 0) + arc_c_min = arc_meta_limit / 2; + + dprintf("ZFS: set arc_meta_limit %llu, arc_c_min %llu," + "zfs_arc_meta_limit %llu\n", + arc_meta_limit, arc_c_min, zfs_arc_meta_limit); + } + + if (ks->arc_zfs_arc_meta_min.value.ui64 != zfs_arc_meta_min) { + zfs_arc_meta_min = ks->arc_zfs_arc_meta_min.value.ui64; + if (zfs_arc_meta_min >= arc_c_min) { + dprintf("ZFS: probable error, zfs_arc_meta_min " + "%llu >= arc_c_min %llu\n", + zfs_arc_meta_min, arc_c_min); + } + if (zfs_arc_meta_min > 0 && + zfs_arc_meta_min <= arc_meta_limit) + arc_meta_min = zfs_arc_meta_min; + dprintf("ZFS: set arc_meta_min %llu\n", arc_meta_min); + } + + zfs_arc_grow_retry = ks->arc_zfs_arc_grow_retry.value.ui64; + arc_grow_retry = zfs_arc_grow_retry; + zfs_arc_shrink_shift = ks->arc_zfs_arc_shrink_shift.value.ui64; + zfs_arc_p_min_shift = ks->arc_zfs_arc_p_min_shift.value.ui64; + zfs_arc_average_blocksize = + ks->arc_zfs_arc_average_blocksize.value.ui64; + +#ifdef _KERNEL + if (ks->zfs_total_memory_limit.value.ui64 > total_memory && + ks->zfs_total_memory_limit.value.ui64 + < real_total_memory) { + dprintf("%s Changing total memory limit to: %llu " + "from: %llu. total physical memory: %llu\n", + __func__, ks->zfs_total_memory_limit.value.ui64, + total_memory, real_total_memory); + total_memory = ks->zfs_total_memory_limit.value.ui64; + physmem = total_memory / PAGE_SIZE; + spl_minimal_physmem_p_logic(); + } else { + dprintf("%s Skip changing total memory limit to: %llu" + " from: %llu. total physical memory: %llu\n", + __func__, ks->zfs_total_memory_limit.value.ui64, + total_memory, real_total_memory); + } +#endif + + } else { + + ks->arc_zfs_arc_max.value.ui64 = zfs_arc_max; + ks->arc_zfs_arc_min.value.ui64 = zfs_arc_min; + + ks->arc_zfs_arc_meta_limit.value.ui64 = zfs_arc_meta_limit; + ks->arc_zfs_arc_meta_min.value.ui64 = zfs_arc_meta_min; + + ks->arc_zfs_arc_grow_retry.value.ui64 = + zfs_arc_grow_retry ? zfs_arc_grow_retry : arc_grow_retry; + ks->arc_zfs_arc_shrink_shift.value.ui64 = zfs_arc_shrink_shift; + ks->arc_zfs_arc_p_min_shift.value.ui64 = zfs_arc_p_min_shift; + ks->arc_zfs_arc_average_blocksize.value.ui64 = + zfs_arc_average_blocksize; + +#ifdef _KERNEL + ks->zfs_total_memory_limit.value.ui64 = total_memory; +#endif + } + return (0); +} +#endif + +/* + * Helper function for arc_prune_async() it is responsible for safely + * handling the execution of a registered arc_prune_func_t. + */ +static void +arc_prune_task(void *ptr) +{ + arc_prune_t *ap = (arc_prune_t *)ptr; + arc_prune_func_t *func = ap->p_pfunc; + + if (func != NULL) + func(ap->p_adjust, ap->p_private); + + zfs_refcount_remove(&ap->p_refcnt, func); +} + +/* + * Notify registered consumers they must drop holds on a portion of the ARC + * buffered they reference. This provides a mechanism to ensure the ARC can + * honor the arc_meta_limit and reclaim otherwise pinned ARC buffers. This + * is analogous to dnlc_reduce_cache() but more generic. + * + * This operation is performed asynchronously so it may be safely called + * in the context of the arc_reclaim_thread(). A reference is taken here + * for each registered arc_prune_t and the arc_prune_task() is responsible + * for releasing it once the registered arc_prune_func_t has completed. + */ +void +arc_prune_async(uint64_t adjust) +{ + arc_prune_t *ap; + + mutex_enter(&arc_prune_mtx); + for (ap = list_head(&arc_prune_list); ap != NULL; + ap = list_next(&arc_prune_list, ap)) { + + if (zfs_refcount_count(&ap->p_refcnt) >= 2) + continue; + + zfs_refcount_add(&ap->p_refcnt, ap->p_pfunc); + ap->p_adjust = adjust; + if (taskq_dispatch(arc_prune_taskq, arc_prune_task, + ap, TQ_SLEEP) == TASKQID_INVALID) { + zfs_refcount_remove(&ap->p_refcnt, ap->p_pfunc); + continue; + } + ARCSTAT_BUMP(arcstat_prune); + } + mutex_exit(&arc_prune_mtx); +} + +#else /* from above ifdef _KERNEL */ + +int64_t +arc_available_memory(void) +{ + return (arc_free_memory() - arc_sys_free); +} + +int +arc_memory_throttle(spa_t *spa, uint64_t reserve, uint64_t txg) +{ + return (0); +} + +uint64_t +arc_all_memory(void) +{ + return (ptob(physmem) / 2); +} + +uint64_t +arc_free_memory(void) +{ + int64_t avail; + + avail = spl_free_wrapper(); + return (avail >= 0LL ? avail : 0LL); +} +#endif /* KERNEL */ + +void +arc_register_hotplug(void) +{ +} + +void +arc_unregister_hotplug(void) +{ +} diff --git a/module/os/windows/zfs/qat.c b/module/os/windows/zfs/qat.c new file mode 100644 index 000000000000..08613b3a2042 --- /dev/null +++ b/module/os/windows/zfs/qat.c @@ -0,0 +1,105 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#if defined(_KERNEL) && defined(HAVE_QAT) +#include +#include + +qat_stats_t qat_stats = { + { "comp_requests", KSTAT_DATA_UINT64 }, + { "comp_total_in_bytes", KSTAT_DATA_UINT64 }, + { "comp_total_out_bytes", KSTAT_DATA_UINT64 }, + { "decomp_requests", KSTAT_DATA_UINT64 }, + { "decomp_total_in_bytes", KSTAT_DATA_UINT64 }, + { "decomp_total_out_bytes", KSTAT_DATA_UINT64 }, + { "dc_fails", KSTAT_DATA_UINT64 }, + { "encrypt_requests", KSTAT_DATA_UINT64 }, + { "encrypt_total_in_bytes", KSTAT_DATA_UINT64 }, + { "encrypt_total_out_bytes", KSTAT_DATA_UINT64 }, + { "decrypt_requests", KSTAT_DATA_UINT64 }, + { "decrypt_total_in_bytes", KSTAT_DATA_UINT64 }, + { "decrypt_total_out_bytes", KSTAT_DATA_UINT64 }, + { "crypt_fails", KSTAT_DATA_UINT64 }, + { "cksum_requests", KSTAT_DATA_UINT64 }, + { "cksum_total_in_bytes", KSTAT_DATA_UINT64 }, + { "cksum_fails", KSTAT_DATA_UINT64 }, +}; + +static kstat_t *qat_ksp = NULL; + +CpaStatus +qat_mem_alloc_contig(void **pp_mem_addr, Cpa32U size_bytes) +{ + *pp_mem_addr = kmalloc(size_bytes, GFP_KERNEL); + if (*pp_mem_addr == NULL) + return (CPA_STATUS_RESOURCE); + return (CPA_STATUS_SUCCESS); +} + +void +qat_mem_free_contig(void **pp_mem_addr) +{ + if (*pp_mem_addr != NULL) { + kfree(*pp_mem_addr); + *pp_mem_addr = NULL; + } +} + +int +qat_init(void) +{ + qat_ksp = kstat_create("zfs", 0, "qat", "misc", + KSTAT_TYPE_NAMED, sizeof (qat_stats) / sizeof (kstat_named_t), + KSTAT_FLAG_VIRTUAL); + if (qat_ksp != NULL) { + qat_ksp->ks_data = &qat_stats; + kstat_install(qat_ksp); + } + + /* + * Just set the disable flag when qat init failed, qat can be + * turned on again in post-process after zfs module is loaded, e.g.: + * echo 0 > /sys/module/zfs/parameters/zfs_qat_compress_disable + */ + if (qat_dc_init() != 0) + zfs_qat_compress_disable = 1; + + if (qat_cy_init() != 0) { + zfs_qat_checksum_disable = 1; + zfs_qat_encrypt_disable = 1; + } + + return (0); +} + +void +qat_fini(void) +{ + if (qat_ksp != NULL) { + kstat_delete(qat_ksp); + qat_ksp = NULL; + } + + qat_cy_fini(); + qat_dc_fini(); +} + +#endif diff --git a/module/os/windows/zfs/qat_compress.c b/module/os/windows/zfs/qat_compress.c new file mode 100644 index 000000000000..ad3ead3b16e3 --- /dev/null +++ b/module/os/windows/zfs/qat_compress.c @@ -0,0 +1,569 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#if defined(_KERNEL) && defined(HAVE_QAT) +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * Max instances in a QAT device, each instance is a channel to submit + * jobs to QAT hardware, this is only for pre-allocating instance and + * session arrays; the actual number of instances are defined in the + * QAT driver's configuration file. + */ +#define QAT_DC_MAX_INSTANCES 48 + +/* + * ZLIB head and foot size + */ +#define ZLIB_HEAD_SZ 2 +#define ZLIB_FOOT_SZ 4 + +static CpaInstanceHandle dc_inst_handles[QAT_DC_MAX_INSTANCES]; +static CpaDcSessionHandle session_handles[QAT_DC_MAX_INSTANCES]; +static CpaBufferList **buffer_array[QAT_DC_MAX_INSTANCES]; +static Cpa16U num_inst = 0; +static Cpa32U inst_num = 0; +static boolean_t qat_dc_init_done = B_FALSE; +int zfs_qat_compress_disable = 0; + +boolean_t +qat_dc_use_accel(size_t s_len) +{ + return (!zfs_qat_compress_disable && + qat_dc_init_done && + s_len >= QAT_MIN_BUF_SIZE && + s_len <= QAT_MAX_BUF_SIZE); +} + +static void +qat_dc_callback(void *p_callback, CpaStatus status) +{ + if (p_callback != NULL) + complete((struct completion *)p_callback); +} + +static void +qat_dc_clean(void) +{ + Cpa16U buff_num = 0; + Cpa16U num_inter_buff_lists = 0; + + for (Cpa16U i = 0; i < num_inst; i++) { + cpaDcStopInstance(dc_inst_handles[i]); + QAT_PHYS_CONTIG_FREE(session_handles[i]); + /* free intermediate buffers */ + if (buffer_array[i] != NULL) { + cpaDcGetNumIntermediateBuffers( + dc_inst_handles[i], &num_inter_buff_lists); + for (buff_num = 0; buff_num < num_inter_buff_lists; + buff_num++) { + CpaBufferList *buffer_inter = + buffer_array[i][buff_num]; + if (buffer_inter->pBuffers) { + QAT_PHYS_CONTIG_FREE( + buffer_inter->pBuffers->pData); + QAT_PHYS_CONTIG_FREE( + buffer_inter->pBuffers); + } + QAT_PHYS_CONTIG_FREE( + buffer_inter->pPrivateMetaData); + QAT_PHYS_CONTIG_FREE(buffer_inter); + } + } + } + + num_inst = 0; + qat_dc_init_done = B_FALSE; +} + +int +qat_dc_init(void) +{ + CpaStatus status = CPA_STATUS_SUCCESS; + Cpa32U sess_size = 0; + Cpa32U ctx_size = 0; + Cpa16U num_inter_buff_lists = 0; + Cpa16U buff_num = 0; + Cpa32U buff_meta_size = 0; + CpaDcSessionSetupData sd = {0}; + + if (qat_dc_init_done) + return (0); + + status = cpaDcGetNumInstances(&num_inst); + if (status != CPA_STATUS_SUCCESS) + return (-1); + + /* if the user has configured no QAT compression units just return */ + if (num_inst == 0) + return (0); + + if (num_inst > QAT_DC_MAX_INSTANCES) + num_inst = QAT_DC_MAX_INSTANCES; + + status = cpaDcGetInstances(num_inst, &dc_inst_handles[0]); + if (status != CPA_STATUS_SUCCESS) + return (-1); + + for (Cpa16U i = 0; i < num_inst; i++) { + cpaDcSetAddressTranslation(dc_inst_handles[i], + (void*)virt_to_phys); + + status = cpaDcBufferListGetMetaSize(dc_inst_handles[i], + 1, &buff_meta_size); + + if (status == CPA_STATUS_SUCCESS) + status = cpaDcGetNumIntermediateBuffers( + dc_inst_handles[i], &num_inter_buff_lists); + + if (status == CPA_STATUS_SUCCESS && num_inter_buff_lists != 0) + status = QAT_PHYS_CONTIG_ALLOC(&buffer_array[i], + num_inter_buff_lists * + sizeof (CpaBufferList *)); + + for (buff_num = 0; buff_num < num_inter_buff_lists; + buff_num++) { + if (status == CPA_STATUS_SUCCESS) + status = QAT_PHYS_CONTIG_ALLOC( + &buffer_array[i][buff_num], + sizeof (CpaBufferList)); + + if (status == CPA_STATUS_SUCCESS) + status = QAT_PHYS_CONTIG_ALLOC( + &buffer_array[i][buff_num]-> + pPrivateMetaData, + buff_meta_size); + + if (status == CPA_STATUS_SUCCESS) + status = QAT_PHYS_CONTIG_ALLOC( + &buffer_array[i][buff_num]->pBuffers, + sizeof (CpaFlatBuffer)); + + if (status == CPA_STATUS_SUCCESS) { + /* + * implementation requires an intermediate + * buffer approximately twice the size of + * output buffer, which is 2x max buffer + * size here. + */ + status = QAT_PHYS_CONTIG_ALLOC( + &buffer_array[i][buff_num]->pBuffers-> + pData, 2 * QAT_MAX_BUF_SIZE); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + buffer_array[i][buff_num]->numBuffers = 1; + buffer_array[i][buff_num]->pBuffers-> + dataLenInBytes = 2 * QAT_MAX_BUF_SIZE; + } + } + + status = cpaDcStartInstance(dc_inst_handles[i], + num_inter_buff_lists, buffer_array[i]); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + sd.compLevel = CPA_DC_L1; + sd.compType = CPA_DC_DEFLATE; + sd.huffType = CPA_DC_HT_FULL_DYNAMIC; + sd.sessDirection = CPA_DC_DIR_COMBINED; + sd.sessState = CPA_DC_STATELESS; + sd.deflateWindowSize = 7; + sd.checksum = CPA_DC_ADLER32; + status = cpaDcGetSessionSize(dc_inst_handles[i], + &sd, &sess_size, &ctx_size); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + QAT_PHYS_CONTIG_ALLOC(&session_handles[i], sess_size); + if (session_handles[i] == NULL) + goto fail; + + status = cpaDcInitSession(dc_inst_handles[i], + session_handles[i], + &sd, NULL, qat_dc_callback); + if (status != CPA_STATUS_SUCCESS) + goto fail; + } + + qat_dc_init_done = B_TRUE; + return (0); +fail: + qat_dc_clean(); + return (-1); +} + +void +qat_dc_fini(void) +{ + if (!qat_dc_init_done) + return; + + qat_dc_clean(); +} + +/* + * The "add" parameter is an additional buffer which is passed + * to QAT as a scratch buffer alongside the destination buffer + * in case the "compressed" data ends up being larger than the + * original source data. This is necessary to prevent QAT from + * generating buffer overflow warnings for incompressible data. + */ +static int +qat_compress_impl(qat_compress_dir_t dir, char *src, int src_len, + char *dst, int dst_len, char *add, int add_len, size_t *c_len) +{ + CpaInstanceHandle dc_inst_handle; + CpaDcSessionHandle session_handle; + CpaBufferList *buf_list_src = NULL; + CpaBufferList *buf_list_dst = NULL; + CpaFlatBuffer *flat_buf_src = NULL; + CpaFlatBuffer *flat_buf_dst = NULL; + Cpa8U *buffer_meta_src = NULL; + Cpa8U *buffer_meta_dst = NULL; + Cpa32U buffer_meta_size = 0; + CpaDcRqResults dc_results; + CpaStatus status = CPA_STATUS_FAIL; + Cpa32U hdr_sz = 0; + Cpa32U compressed_sz; + Cpa32U num_src_buf = (src_len >> PAGE_SHIFT) + 2; + Cpa32U num_dst_buf = (dst_len >> PAGE_SHIFT) + 2; + Cpa32U num_add_buf = (add_len >> PAGE_SHIFT) + 2; + Cpa32U bytes_left; + Cpa32U dst_pages = 0; + Cpa32U adler32 = 0; + char *data; + struct page *page; + struct page **in_pages = NULL; + struct page **out_pages = NULL; + struct page **add_pages = NULL; + Cpa32U page_off = 0; + struct completion complete; + Cpa32U page_num = 0; + Cpa16U i; + + /* + * We increment num_src_buf and num_dst_buf by 2 to allow + * us to handle non page-aligned buffer addresses and buffers + * whose sizes are not divisible by PAGE_SIZE. + */ + Cpa32U src_buffer_list_mem_size = sizeof (CpaBufferList) + + (num_src_buf * sizeof (CpaFlatBuffer)); + Cpa32U dst_buffer_list_mem_size = sizeof (CpaBufferList) + + ((num_dst_buf + num_add_buf) * sizeof (CpaFlatBuffer)); + + status = QAT_PHYS_CONTIG_ALLOC(&in_pages, + num_src_buf * sizeof (struct page *)); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + status = QAT_PHYS_CONTIG_ALLOC(&out_pages, + num_dst_buf * sizeof (struct page *)); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + status = QAT_PHYS_CONTIG_ALLOC(&add_pages, + num_add_buf * sizeof (struct page *)); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + i = (Cpa32U)atomic_inc_32_nv(&inst_num) % num_inst; + dc_inst_handle = dc_inst_handles[i]; + session_handle = session_handles[i]; + + cpaDcBufferListGetMetaSize(dc_inst_handle, num_src_buf, + &buffer_meta_size); + status = QAT_PHYS_CONTIG_ALLOC(&buffer_meta_src, buffer_meta_size); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + cpaDcBufferListGetMetaSize(dc_inst_handle, num_dst_buf + num_add_buf, + &buffer_meta_size); + status = QAT_PHYS_CONTIG_ALLOC(&buffer_meta_dst, buffer_meta_size); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + /* build source buffer list */ + status = QAT_PHYS_CONTIG_ALLOC(&buf_list_src, src_buffer_list_mem_size); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + flat_buf_src = (CpaFlatBuffer *)(buf_list_src + 1); + + buf_list_src->pBuffers = flat_buf_src; /* always point to first one */ + + /* build destination buffer list */ + status = QAT_PHYS_CONTIG_ALLOC(&buf_list_dst, dst_buffer_list_mem_size); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + flat_buf_dst = (CpaFlatBuffer *)(buf_list_dst + 1); + + buf_list_dst->pBuffers = flat_buf_dst; /* always point to first one */ + + buf_list_src->numBuffers = 0; + buf_list_src->pPrivateMetaData = buffer_meta_src; + bytes_left = src_len; + data = src; + page_num = 0; + while (bytes_left > 0) { + page_off = ((long)data & ~PAGE_MASK); + page = qat_mem_to_page(data); + in_pages[page_num] = page; + flat_buf_src->pData = kmap(page) + page_off; + flat_buf_src->dataLenInBytes = + min((long)PAGE_SIZE - page_off, (long)bytes_left); + + bytes_left -= flat_buf_src->dataLenInBytes; + data += flat_buf_src->dataLenInBytes; + flat_buf_src++; + buf_list_src->numBuffers++; + page_num++; + } + + buf_list_dst->numBuffers = 0; + buf_list_dst->pPrivateMetaData = buffer_meta_dst; + bytes_left = dst_len; + data = dst; + page_num = 0; + while (bytes_left > 0) { + page_off = ((long)data & ~PAGE_MASK); + page = qat_mem_to_page(data); + flat_buf_dst->pData = kmap(page) + page_off; + out_pages[page_num] = page; + flat_buf_dst->dataLenInBytes = + min((long)PAGE_SIZE - page_off, (long)bytes_left); + + bytes_left -= flat_buf_dst->dataLenInBytes; + data += flat_buf_dst->dataLenInBytes; + flat_buf_dst++; + buf_list_dst->numBuffers++; + page_num++; + dst_pages++; + } + + /* map additional scratch pages into the destination buffer list */ + bytes_left = add_len; + data = add; + page_num = 0; + while (bytes_left > 0) { + page_off = ((long)data & ~PAGE_MASK); + page = qat_mem_to_page(data); + flat_buf_dst->pData = kmap(page) + page_off; + add_pages[page_num] = page; + flat_buf_dst->dataLenInBytes = + min((long)PAGE_SIZE - page_off, (long)bytes_left); + + bytes_left -= flat_buf_dst->dataLenInBytes; + data += flat_buf_dst->dataLenInBytes; + flat_buf_dst++; + buf_list_dst->numBuffers++; + page_num++; + } + + init_completion(&complete); + + if (dir == QAT_COMPRESS) { + QAT_STAT_BUMP(comp_requests); + QAT_STAT_INCR(comp_total_in_bytes, src_len); + + cpaDcGenerateHeader(session_handle, + buf_list_dst->pBuffers, &hdr_sz); + buf_list_dst->pBuffers->pData += hdr_sz; + buf_list_dst->pBuffers->dataLenInBytes -= hdr_sz; + status = cpaDcCompressData( + dc_inst_handle, session_handle, + buf_list_src, buf_list_dst, + &dc_results, CPA_DC_FLUSH_FINAL, + &complete); + if (status != CPA_STATUS_SUCCESS) { + goto fail; + } + + /* we now wait until the completion of the operation. */ + wait_for_completion(&complete); + + if (dc_results.status != CPA_STATUS_SUCCESS) { + status = CPA_STATUS_FAIL; + goto fail; + } + + compressed_sz = dc_results.produced; + if (compressed_sz + hdr_sz + ZLIB_FOOT_SZ > dst_len) { + status = CPA_STATUS_INCOMPRESSIBLE; + goto fail; + } + + flat_buf_dst = (CpaFlatBuffer *)(buf_list_dst + 1); + /* move to the last page */ + flat_buf_dst += (compressed_sz + hdr_sz) >> PAGE_SHIFT; + + /* no space for gzip footer in the last page */ + if (((compressed_sz + hdr_sz) % PAGE_SIZE) + + ZLIB_FOOT_SZ > PAGE_SIZE) { + status = CPA_STATUS_INCOMPRESSIBLE; + goto fail; + } + + /* jump to the end of the buffer and append footer */ + flat_buf_dst->pData = + (char *)((unsigned long)flat_buf_dst->pData & PAGE_MASK) + + ((compressed_sz + hdr_sz) % PAGE_SIZE); + flat_buf_dst->dataLenInBytes = ZLIB_FOOT_SZ; + + dc_results.produced = 0; + status = cpaDcGenerateFooter(session_handle, + flat_buf_dst, &dc_results); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + *c_len = compressed_sz + dc_results.produced + hdr_sz; + QAT_STAT_INCR(comp_total_out_bytes, *c_len); + } else { + ASSERT3U(dir, ==, QAT_DECOMPRESS); + QAT_STAT_BUMP(decomp_requests); + QAT_STAT_INCR(decomp_total_in_bytes, src_len); + + buf_list_src->pBuffers->pData += ZLIB_HEAD_SZ; + buf_list_src->pBuffers->dataLenInBytes -= ZLIB_HEAD_SZ; + status = cpaDcDecompressData(dc_inst_handle, session_handle, + buf_list_src, buf_list_dst, &dc_results, CPA_DC_FLUSH_FINAL, + &complete); + + if (CPA_STATUS_SUCCESS != status) { + status = CPA_STATUS_FAIL; + goto fail; + } + + /* we now wait until the completion of the operation. */ + wait_for_completion(&complete); + + if (dc_results.status != CPA_STATUS_SUCCESS) { + status = CPA_STATUS_FAIL; + goto fail; + } + + /* verify adler checksum */ + adler32 = *(Cpa32U *)(src + dc_results.consumed + ZLIB_HEAD_SZ); + if (adler32 != BSWAP_32(dc_results.checksum)) { + status = CPA_STATUS_FAIL; + goto fail; + } + *c_len = dc_results.produced; + QAT_STAT_INCR(decomp_total_out_bytes, *c_len); + } + +fail: + if (status != CPA_STATUS_SUCCESS && status != CPA_STATUS_INCOMPRESSIBLE) + QAT_STAT_BUMP(dc_fails); + + if (in_pages) { + for (page_num = 0; + page_num < buf_list_src->numBuffers; + page_num++) { + kunmap(in_pages[page_num]); + } + QAT_PHYS_CONTIG_FREE(in_pages); + } + + if (out_pages) { + for (page_num = 0; page_num < dst_pages; page_num++) { + kunmap(out_pages[page_num]); + } + QAT_PHYS_CONTIG_FREE(out_pages); + } + + if (add_pages) { + for (page_num = 0; + page_num < buf_list_dst->numBuffers - dst_pages; + page_num++) { + kunmap(add_pages[page_num]); + } + QAT_PHYS_CONTIG_FREE(add_pages); + } + + QAT_PHYS_CONTIG_FREE(buffer_meta_src); + QAT_PHYS_CONTIG_FREE(buffer_meta_dst); + QAT_PHYS_CONTIG_FREE(buf_list_src); + QAT_PHYS_CONTIG_FREE(buf_list_dst); + + return (status); +} + +/* + * Entry point for QAT accelerated compression / decompression. + */ +int +qat_compress(qat_compress_dir_t dir, char *src, int src_len, + char *dst, int dst_len, size_t *c_len) +{ + int ret; + size_t add_len = 0; + void *add = NULL; + + if (dir == QAT_COMPRESS) { + add_len = dst_len; + add = zio_data_buf_alloc(add_len); + } + + ret = qat_compress_impl(dir, src, src_len, dst, + dst_len, add, add_len, c_len); + + if (dir == QAT_COMPRESS) + zio_data_buf_free(add, add_len); + + return (ret); +} + +static int +param_set_qat_compress(const char *val, zfs_kernel_param_t *kp) +{ + int ret; + int *pvalue = kp->arg; + ret = param_set_int(val, kp); + if (ret) + return (ret); + /* + * zfs_qat_compress_disable = 0: enable qat compress + * try to initialize qat instance if it has not been done + */ + if (*pvalue == 0 && !qat_dc_init_done) { + ret = qat_dc_init(); + if (ret != 0) { + zfs_qat_compress_disable = 1; + return (ret); + } + } + return (ret); +} + +module_param_call(zfs_qat_compress_disable, param_set_qat_compress, + param_get_int, &zfs_qat_compress_disable, 0644); +MODULE_PARM_DESC(zfs_qat_compress_disable, "Enable/Disable QAT compression"); + +#endif diff --git a/module/os/windows/zfs/qat_crypt.c b/module/os/windows/zfs/qat_crypt.c new file mode 100644 index 000000000000..18b6e38d1a6e --- /dev/null +++ b/module/os/windows/zfs/qat_crypt.c @@ -0,0 +1,630 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * This file represents the QAT implementation of checksums and encryption. + * Internally, QAT shares the same cryptographic instances for both of these + * operations, so the code has been combined here. QAT data compression uses + * compression instances, so that code is separated into qat_compress.c + */ + +#if defined(_KERNEL) && defined(HAVE_QAT) +#include +#include +#include +#include +#include +#include +#include "lac/cpa_cy_im.h" +#include "lac/cpa_cy_common.h" +#include + +/* + * Max instances in a QAT device, each instance is a channel to submit + * jobs to QAT hardware, this is only for pre-allocating instances + * and session arrays; the actual number of instances are defined in + * the QAT driver's configure file. + */ +#define QAT_CRYPT_MAX_INSTANCES 48 + +#define MAX_PAGE_NUM 1024 + +static Cpa32U inst_num = 0; +static Cpa16U num_inst = 0; +static CpaInstanceHandle cy_inst_handles[QAT_CRYPT_MAX_INSTANCES]; +static boolean_t qat_cy_init_done = B_FALSE; +int zfs_qat_encrypt_disable = 0; +int zfs_qat_checksum_disable = 0; + +typedef struct cy_callback { + CpaBoolean verify_result; + struct completion complete; +} cy_callback_t; + +static void +symcallback(void *p_callback, CpaStatus status, const CpaCySymOp operation, + void *op_data, CpaBufferList *buf_list_dst, CpaBoolean verify) +{ + cy_callback_t *cb = p_callback; + + if (cb != NULL) { + /* indicate that the function has been called */ + cb->verify_result = verify; + complete(&cb->complete); + } +} + +boolean_t +qat_crypt_use_accel(size_t s_len) +{ + return (!zfs_qat_encrypt_disable && + qat_cy_init_done && + s_len >= QAT_MIN_BUF_SIZE && + s_len <= QAT_MAX_BUF_SIZE); +} + +boolean_t +qat_checksum_use_accel(size_t s_len) +{ + return (!zfs_qat_checksum_disable && + qat_cy_init_done && + s_len >= QAT_MIN_BUF_SIZE && + s_len <= QAT_MAX_BUF_SIZE); +} + +void +qat_cy_clean(void) +{ + for (Cpa16U i = 0; i < num_inst; i++) + cpaCyStopInstance(cy_inst_handles[i]); + + num_inst = 0; + qat_cy_init_done = B_FALSE; +} + +int +qat_cy_init(void) +{ + CpaStatus status = CPA_STATUS_FAIL; + + if (qat_cy_init_done) + return (0); + + status = cpaCyGetNumInstances(&num_inst); + if (status != CPA_STATUS_SUCCESS) + return (-1); + + /* if the user has configured no QAT encryption units just return */ + if (num_inst == 0) + return (0); + + if (num_inst > QAT_CRYPT_MAX_INSTANCES) + num_inst = QAT_CRYPT_MAX_INSTANCES; + + status = cpaCyGetInstances(num_inst, &cy_inst_handles[0]); + if (status != CPA_STATUS_SUCCESS) + return (-1); + + for (Cpa16U i = 0; i < num_inst; i++) { + status = cpaCySetAddressTranslation(cy_inst_handles[i], + (void *)virt_to_phys); + if (status != CPA_STATUS_SUCCESS) + goto error; + + status = cpaCyStartInstance(cy_inst_handles[i]); + if (status != CPA_STATUS_SUCCESS) + goto error; + } + + qat_cy_init_done = B_TRUE; + return (0); + +error: + qat_cy_clean(); + return (-1); +} + +void +qat_cy_fini(void) +{ + if (!qat_cy_init_done) + return; + + qat_cy_clean(); +} + +static CpaStatus +qat_init_crypt_session_ctx(qat_encrypt_dir_t dir, CpaInstanceHandle inst_handle, + CpaCySymSessionCtx **cy_session_ctx, crypto_key_t *key, + Cpa64U crypt, Cpa32U aad_len) +{ + CpaStatus status = CPA_STATUS_SUCCESS; + Cpa32U ctx_size; + Cpa32U ciper_algorithm; + Cpa32U hash_algorithm; + CpaCySymSessionSetupData sd = { 0 }; + + if (zio_crypt_table[crypt].ci_crypt_type == ZC_TYPE_CCM) { + return (CPA_STATUS_FAIL); + } else { + ciper_algorithm = CPA_CY_SYM_CIPHER_AES_GCM; + hash_algorithm = CPA_CY_SYM_HASH_AES_GCM; + } + + sd.cipherSetupData.cipherAlgorithm = ciper_algorithm; + sd.cipherSetupData.pCipherKey = key->ck_data; + sd.cipherSetupData.cipherKeyLenInBytes = key->ck_length / 8; + sd.hashSetupData.hashAlgorithm = hash_algorithm; + sd.hashSetupData.hashMode = CPA_CY_SYM_HASH_MODE_AUTH; + sd.hashSetupData.digestResultLenInBytes = ZIO_DATA_MAC_LEN; + sd.hashSetupData.authModeSetupData.aadLenInBytes = aad_len; + sd.sessionPriority = CPA_CY_PRIORITY_NORMAL; + sd.symOperation = CPA_CY_SYM_OP_ALGORITHM_CHAINING; + sd.digestIsAppended = CPA_FALSE; + sd.verifyDigest = CPA_FALSE; + + if (dir == QAT_ENCRYPT) { + sd.cipherSetupData.cipherDirection = + CPA_CY_SYM_CIPHER_DIRECTION_ENCRYPT; + sd.algChainOrder = + CPA_CY_SYM_ALG_CHAIN_ORDER_HASH_THEN_CIPHER; + } else { + ASSERT3U(dir, ==, QAT_DECRYPT); + sd.cipherSetupData.cipherDirection = + CPA_CY_SYM_CIPHER_DIRECTION_DECRYPT; + sd.algChainOrder = + CPA_CY_SYM_ALG_CHAIN_ORDER_CIPHER_THEN_HASH; + } + + status = cpaCySymSessionCtxGetSize(inst_handle, &sd, &ctx_size); + if (status != CPA_STATUS_SUCCESS) + return (status); + + status = QAT_PHYS_CONTIG_ALLOC(cy_session_ctx, ctx_size); + if (status != CPA_STATUS_SUCCESS) + return (status); + + status = cpaCySymInitSession(inst_handle, symcallback, &sd, + *cy_session_ctx); + if (status != CPA_STATUS_SUCCESS) { + QAT_PHYS_CONTIG_FREE(*cy_session_ctx); + return (status); + } + + return (CPA_STATUS_SUCCESS); +} + +static CpaStatus +qat_init_checksum_session_ctx(CpaInstanceHandle inst_handle, + CpaCySymSessionCtx **cy_session_ctx, Cpa64U cksum) +{ + CpaStatus status = CPA_STATUS_SUCCESS; + Cpa32U ctx_size; + Cpa32U hash_algorithm; + CpaCySymSessionSetupData sd = { 0 }; + + /* + * ZFS's SHA512 checksum is actually SHA512/256, which uses + * a different IV from standard SHA512. QAT does not support + * SHA512/256, so we can only support SHA256. + */ + if (cksum == ZIO_CHECKSUM_SHA256) + hash_algorithm = CPA_CY_SYM_HASH_SHA256; + else + return (CPA_STATUS_FAIL); + + sd.sessionPriority = CPA_CY_PRIORITY_NORMAL; + sd.symOperation = CPA_CY_SYM_OP_HASH; + sd.hashSetupData.hashAlgorithm = hash_algorithm; + sd.hashSetupData.hashMode = CPA_CY_SYM_HASH_MODE_PLAIN; + sd.hashSetupData.digestResultLenInBytes = sizeof (zio_cksum_t); + sd.digestIsAppended = CPA_FALSE; + sd.verifyDigest = CPA_FALSE; + + status = cpaCySymSessionCtxGetSize(inst_handle, &sd, &ctx_size); + if (status != CPA_STATUS_SUCCESS) + return (status); + + status = QAT_PHYS_CONTIG_ALLOC(cy_session_ctx, ctx_size); + if (status != CPA_STATUS_SUCCESS) + return (status); + + status = cpaCySymInitSession(inst_handle, symcallback, &sd, + *cy_session_ctx); + if (status != CPA_STATUS_SUCCESS) { + QAT_PHYS_CONTIG_FREE(*cy_session_ctx); + return (status); + } + + return (CPA_STATUS_SUCCESS); +} + +static CpaStatus +qat_init_cy_buffer_lists(CpaInstanceHandle inst_handle, uint32_t nr_bufs, + CpaBufferList *src, CpaBufferList *dst) +{ + CpaStatus status = CPA_STATUS_SUCCESS; + Cpa32U meta_size = 0; + + status = cpaCyBufferListGetMetaSize(inst_handle, nr_bufs, &meta_size); + if (status != CPA_STATUS_SUCCESS) + return (status); + + status = QAT_PHYS_CONTIG_ALLOC(&src->pPrivateMetaData, meta_size); + if (status != CPA_STATUS_SUCCESS) + goto error; + + if (src != dst) { + status = QAT_PHYS_CONTIG_ALLOC(&dst->pPrivateMetaData, + meta_size); + if (status != CPA_STATUS_SUCCESS) + goto error; + } + + return (CPA_STATUS_SUCCESS); + +error: + QAT_PHYS_CONTIG_FREE(src->pPrivateMetaData); + if (src != dst) + QAT_PHYS_CONTIG_FREE(dst->pPrivateMetaData); + + return (status); +} + +int +qat_crypt(qat_encrypt_dir_t dir, uint8_t *src_buf, uint8_t *dst_buf, + uint8_t *aad_buf, uint32_t aad_len, uint8_t *iv_buf, uint8_t *digest_buf, + crypto_key_t *key, uint64_t crypt, uint32_t enc_len) +{ + CpaStatus status = CPA_STATUS_SUCCESS; + Cpa16U i; + CpaInstanceHandle cy_inst_handle; + Cpa16U nr_bufs = (enc_len >> PAGE_SHIFT) + 2; + Cpa32U bytes_left = 0; + Cpa8S *data = NULL; + CpaCySymSessionCtx *cy_session_ctx = NULL; + cy_callback_t cb; + CpaCySymOpData op_data = { 0 }; + CpaBufferList src_buffer_list = { 0 }; + CpaBufferList dst_buffer_list = { 0 }; + CpaFlatBuffer *flat_src_buf_array = NULL; + CpaFlatBuffer *flat_src_buf = NULL; + CpaFlatBuffer *flat_dst_buf_array = NULL; + CpaFlatBuffer *flat_dst_buf = NULL; + struct page *in_pages[MAX_PAGE_NUM]; + struct page *out_pages[MAX_PAGE_NUM]; + Cpa32U in_page_num = 0; + Cpa32U out_page_num = 0; + Cpa32U in_page_off = 0; + Cpa32U out_page_off = 0; + + if (dir == QAT_ENCRYPT) { + QAT_STAT_BUMP(encrypt_requests); + QAT_STAT_INCR(encrypt_total_in_bytes, enc_len); + } else { + QAT_STAT_BUMP(decrypt_requests); + QAT_STAT_INCR(decrypt_total_in_bytes, enc_len); + } + + i = (Cpa32U)atomic_inc_32_nv(&inst_num) % num_inst; + cy_inst_handle = cy_inst_handles[i]; + + status = qat_init_crypt_session_ctx(dir, cy_inst_handle, + &cy_session_ctx, key, crypt, aad_len); + if (status != CPA_STATUS_SUCCESS) { + /* don't count CCM as a failure since it's not supported */ + if (zio_crypt_table[crypt].ci_crypt_type == ZC_TYPE_GCM) + QAT_STAT_BUMP(crypt_fails); + return (status); + } + + /* + * We increment nr_bufs by 2 to allow us to handle non + * page-aligned buffer addresses and buffers whose sizes + * are not divisible by PAGE_SIZE. + */ + status = qat_init_cy_buffer_lists(cy_inst_handle, nr_bufs, + &src_buffer_list, &dst_buffer_list); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + status = QAT_PHYS_CONTIG_ALLOC(&flat_src_buf_array, + nr_bufs * sizeof (CpaFlatBuffer)); + if (status != CPA_STATUS_SUCCESS) + goto fail; + status = QAT_PHYS_CONTIG_ALLOC(&flat_dst_buf_array, + nr_bufs * sizeof (CpaFlatBuffer)); + if (status != CPA_STATUS_SUCCESS) + goto fail; + status = QAT_PHYS_CONTIG_ALLOC(&op_data.pDigestResult, + ZIO_DATA_MAC_LEN); + if (status != CPA_STATUS_SUCCESS) + goto fail; + status = QAT_PHYS_CONTIG_ALLOC(&op_data.pIv, + ZIO_DATA_IV_LEN); + if (status != CPA_STATUS_SUCCESS) + goto fail; + if (aad_len > 0) { + status = QAT_PHYS_CONTIG_ALLOC(&op_data.pAdditionalAuthData, + aad_len); + if (status != CPA_STATUS_SUCCESS) + goto fail; + memcpy(op_data.pAdditionalAuthData, aad_buf, aad_len); + } + + bytes_left = enc_len; + data = src_buf; + flat_src_buf = flat_src_buf_array; + while (bytes_left > 0) { + in_page_off = ((long)data & ~PAGE_MASK); + in_pages[in_page_num] = qat_mem_to_page(data); + flat_src_buf->pData = kmap(in_pages[in_page_num]) + in_page_off; + flat_src_buf->dataLenInBytes = + min((long)PAGE_SIZE - in_page_off, (long)bytes_left); + data += flat_src_buf->dataLenInBytes; + bytes_left -= flat_src_buf->dataLenInBytes; + flat_src_buf++; + in_page_num++; + } + src_buffer_list.pBuffers = flat_src_buf_array; + src_buffer_list.numBuffers = in_page_num; + + bytes_left = enc_len; + data = dst_buf; + flat_dst_buf = flat_dst_buf_array; + while (bytes_left > 0) { + out_page_off = ((long)data & ~PAGE_MASK); + out_pages[out_page_num] = qat_mem_to_page(data); + flat_dst_buf->pData = kmap(out_pages[out_page_num]) + + out_page_off; + flat_dst_buf->dataLenInBytes = + min((long)PAGE_SIZE - out_page_off, (long)bytes_left); + data += flat_dst_buf->dataLenInBytes; + bytes_left -= flat_dst_buf->dataLenInBytes; + flat_dst_buf++; + out_page_num++; + } + dst_buffer_list.pBuffers = flat_dst_buf_array; + dst_buffer_list.numBuffers = out_page_num; + + op_data.sessionCtx = cy_session_ctx; + op_data.packetType = CPA_CY_SYM_PACKET_TYPE_FULL; + op_data.cryptoStartSrcOffsetInBytes = 0; + op_data.messageLenToCipherInBytes = 0; + op_data.hashStartSrcOffsetInBytes = 0; + op_data.messageLenToHashInBytes = 0; + op_data.messageLenToCipherInBytes = enc_len; + op_data.ivLenInBytes = ZIO_DATA_IV_LEN; + memcpy(op_data.pIv, iv_buf, ZIO_DATA_IV_LEN); + /* if dir is QAT_DECRYPT, copy digest_buf to pDigestResult */ + if (dir == QAT_DECRYPT) + memcpy(op_data.pDigestResult, digest_buf, ZIO_DATA_MAC_LEN); + + cb.verify_result = CPA_FALSE; + init_completion(&cb.complete); + status = cpaCySymPerformOp(cy_inst_handle, &cb, &op_data, + &src_buffer_list, &dst_buffer_list, NULL); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + /* we now wait until the completion of the operation. */ + wait_for_completion(&cb.complete); + + if (cb.verify_result == CPA_FALSE) { + status = CPA_STATUS_FAIL; + goto fail; + } + + if (dir == QAT_ENCRYPT) { + /* if dir is QAT_ENCRYPT, save pDigestResult to digest_buf */ + memcpy(digest_buf, op_data.pDigestResult, ZIO_DATA_MAC_LEN); + QAT_STAT_INCR(encrypt_total_out_bytes, enc_len); + } else { + QAT_STAT_INCR(decrypt_total_out_bytes, enc_len); + } + +fail: + if (status != CPA_STATUS_SUCCESS) + QAT_STAT_BUMP(crypt_fails); + + for (i = 0; i < in_page_num; i++) + kunmap(in_pages[i]); + for (i = 0; i < out_page_num; i++) + kunmap(out_pages[i]); + + cpaCySymRemoveSession(cy_inst_handle, cy_session_ctx); + if (aad_len > 0) + QAT_PHYS_CONTIG_FREE(op_data.pAdditionalAuthData); + QAT_PHYS_CONTIG_FREE(op_data.pIv); + QAT_PHYS_CONTIG_FREE(op_data.pDigestResult); + QAT_PHYS_CONTIG_FREE(src_buffer_list.pPrivateMetaData); + QAT_PHYS_CONTIG_FREE(dst_buffer_list.pPrivateMetaData); + QAT_PHYS_CONTIG_FREE(cy_session_ctx); + QAT_PHYS_CONTIG_FREE(flat_src_buf_array); + QAT_PHYS_CONTIG_FREE(flat_dst_buf_array); + + return (status); +} + +int +qat_checksum(uint64_t cksum, uint8_t *buf, uint64_t size, zio_cksum_t *zcp) +{ + CpaStatus status; + Cpa16U i; + CpaInstanceHandle cy_inst_handle; + Cpa16U nr_bufs = (size >> PAGE_SHIFT) + 2; + Cpa32U bytes_left = 0; + Cpa8S *data = NULL; + CpaCySymSessionCtx *cy_session_ctx = NULL; + cy_callback_t cb; + Cpa8U *digest_buffer = NULL; + CpaCySymOpData op_data = { 0 }; + CpaBufferList src_buffer_list = { 0 }; + CpaFlatBuffer *flat_src_buf_array = NULL; + CpaFlatBuffer *flat_src_buf = NULL; + struct page *in_pages[MAX_PAGE_NUM]; + Cpa32U page_num = 0; + Cpa32U page_off = 0; + + QAT_STAT_BUMP(cksum_requests); + QAT_STAT_INCR(cksum_total_in_bytes, size); + + i = (Cpa32U)atomic_inc_32_nv(&inst_num) % num_inst; + cy_inst_handle = cy_inst_handles[i]; + + status = qat_init_checksum_session_ctx(cy_inst_handle, + &cy_session_ctx, cksum); + if (status != CPA_STATUS_SUCCESS) { + /* don't count unsupported checksums as a failure */ + if (cksum == ZIO_CHECKSUM_SHA256 || + cksum == ZIO_CHECKSUM_SHA512) + QAT_STAT_BUMP(cksum_fails); + return (status); + } + + /* + * We increment nr_bufs by 2 to allow us to handle non + * page-aligned buffer addresses and buffers whose sizes + * are not divisible by PAGE_SIZE. + */ + status = qat_init_cy_buffer_lists(cy_inst_handle, nr_bufs, + &src_buffer_list, &src_buffer_list); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + status = QAT_PHYS_CONTIG_ALLOC(&flat_src_buf_array, + nr_bufs * sizeof (CpaFlatBuffer)); + if (status != CPA_STATUS_SUCCESS) + goto fail; + status = QAT_PHYS_CONTIG_ALLOC(&digest_buffer, + sizeof (zio_cksum_t)); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + bytes_left = size; + data = buf; + flat_src_buf = flat_src_buf_array; + while (bytes_left > 0) { + page_off = ((long)data & ~PAGE_MASK); + in_pages[page_num] = qat_mem_to_page(data); + flat_src_buf->pData = kmap(in_pages[page_num]) + page_off; + flat_src_buf->dataLenInBytes = + min((long)PAGE_SIZE - page_off, (long)bytes_left); + data += flat_src_buf->dataLenInBytes; + bytes_left -= flat_src_buf->dataLenInBytes; + flat_src_buf++; + page_num++; + } + src_buffer_list.pBuffers = flat_src_buf_array; + src_buffer_list.numBuffers = page_num; + + op_data.sessionCtx = cy_session_ctx; + op_data.packetType = CPA_CY_SYM_PACKET_TYPE_FULL; + op_data.hashStartSrcOffsetInBytes = 0; + op_data.messageLenToHashInBytes = size; + op_data.pDigestResult = digest_buffer; + + cb.verify_result = CPA_FALSE; + init_completion(&cb.complete); + status = cpaCySymPerformOp(cy_inst_handle, &cb, &op_data, + &src_buffer_list, &src_buffer_list, NULL); + if (status != CPA_STATUS_SUCCESS) + goto fail; + + /* we now wait until the completion of the operation. */ + wait_for_completion(&cb.complete); + + if (cb.verify_result == CPA_FALSE) { + status = CPA_STATUS_FAIL; + goto fail; + } + + memcpy(zcp, digest_buffer, sizeof (zio_cksum_t)); + +fail: + if (status != CPA_STATUS_SUCCESS) + QAT_STAT_BUMP(cksum_fails); + + for (i = 0; i < page_num; i++) + kunmap(in_pages[i]); + + cpaCySymRemoveSession(cy_inst_handle, cy_session_ctx); + QAT_PHYS_CONTIG_FREE(digest_buffer); + QAT_PHYS_CONTIG_FREE(src_buffer_list.pPrivateMetaData); + QAT_PHYS_CONTIG_FREE(cy_session_ctx); + QAT_PHYS_CONTIG_FREE(flat_src_buf_array); + + return (status); +} + +static int +param_set_qat_encrypt(const char *val, zfs_kernel_param_t *kp) +{ + int ret; + int *pvalue = kp->arg; + ret = param_set_int(val, kp); + if (ret) + return (ret); + /* + * zfs_qat_encrypt_disable = 0: enable qat encrypt + * try to initialize qat instance if it has not been done + */ + if (*pvalue == 0 && !qat_cy_init_done) { + ret = qat_cy_init(); + if (ret != 0) { + zfs_qat_encrypt_disable = 1; + return (ret); + } + } + return (ret); +} + +static int +param_set_qat_checksum(const char *val, zfs_kernel_param_t *kp) +{ + int ret; + int *pvalue = kp->arg; + ret = param_set_int(val, kp); + if (ret) + return (ret); + /* + * set_checksum_param_ops = 0: enable qat checksum + * try to initialize qat instance if it has not been done + */ + if (*pvalue == 0 && !qat_cy_init_done) { + ret = qat_cy_init(); + if (ret != 0) { + zfs_qat_checksum_disable = 1; + return (ret); + } + } + return (ret); +} + +module_param_call(zfs_qat_encrypt_disable, param_set_qat_encrypt, + param_get_int, &zfs_qat_encrypt_disable, 0644); +MODULE_PARM_DESC(zfs_qat_encrypt_disable, "Enable/Disable QAT encryption"); + +module_param_call(zfs_qat_checksum_disable, param_set_qat_checksum, + param_get_int, &zfs_qat_checksum_disable, 0644); +MODULE_PARM_DESC(zfs_qat_checksum_disable, "Enable/Disable QAT checksumming"); + +#endif diff --git a/module/os/windows/zfs/spa_misc_os.c b/module/os/windows/zfs/spa_misc_os.c new file mode 100644 index 000000000000..b6f5b8792264 --- /dev/null +++ b/module/os/windows/zfs/spa_misc_os.c @@ -0,0 +1,77 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011, 2019 by Delphix. All rights reserved. + * Copyright 2015 Nexenta Systems, Inc. All rights reserved. + * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. + * Copyright 2013 Saso Kiselkov. All rights reserved. + * Copyright (c) 2017 Datto Inc. + * Copyright (c) 2017, Intel Corporation. + * Portions Copyright 2022 Andrew Innes + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "zfs_prop.h" + +// Windows might have something built-in to busy a driver? +uint64_t zfs_module_busy = 0; + +const char * +spa_history_zone(void) +{ + return ("windows"); +} + +void +spa_import_os(spa_t *arg) +{ +} + +void +spa_export_os(spa_t *arg) +{ +} + +void +spa_activate_os(spa_t *arg) +{ + atomic_inc_64(&zfs_module_busy); +} + +void +spa_deactivate_os(spa_t *arg) +{ + atomic_dec_64(&zfs_module_busy); +} diff --git a/module/os/windows/zfs/sysctl_os.c b/module/os/windows/zfs/sysctl_os.c new file mode 100644 index 000000000000..9951919b40bd --- /dev/null +++ b/module/os/windows/zfs/sysctl_os.c @@ -0,0 +1,755 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright(c) 2022 Jorgen Lundman + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +/* Wait for Registry changes */ +static WORK_QUEUE_ITEM wqi; +static HANDLE registry_notify_fd = 0; +static UNICODE_STRING sysctl_os_RegistryPath; + +void sysctl_os_init(PUNICODE_STRING RegistryPath); + +extern uint32_t spl_hostid; + +HANDLE +sysctl_os_open_registry(PUNICODE_STRING pRegistryPath) +{ + OBJECT_ATTRIBUTES ObjectAttributes; + HANDLE h; + NTSTATUS Status; + + InitializeObjectAttributes(&ObjectAttributes, + pRegistryPath, + OBJ_KERNEL_HANDLE | OBJ_CASE_INSENSITIVE, + NULL, + NULL); + + Status = ZwCreateKey(&h, // KeyHandle + KEY_ALL_ACCESS | KEY_CREATE_SUB_KEY | KEY_NOTIFY, // DesiredAccess + &ObjectAttributes, + 0, + NULL, + REG_OPTION_NON_VOLATILE, + NULL); + + if (!NT_SUCCESS(Status)) { + KdPrintEx((DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, + "%s: Unable to open Registry %wZ: 0x%x -- skipping tunables\n", + __func__, pRegistryPath, Status)); + return (0); + } + return (h); +} + +void +sysctl_os_close_registry(HANDLE regfd) +{ + ZwClose(regfd); +} + +int +sysctl_os_write_registry(HANDLE regfd, ztunable_t *zt, UNICODE_STRING *entry) +{ + void *val = NULL; + ULONG len = 0; + ULONG type = 0; // Registry type + UNICODE_STRING str = { 0 }; + NTSTATUS Status; + ULONG length; + + ZT_GET_VALUE(zt, &val, &len, &type); + + ASSERT3P(val, !=, NULL); + + if (type == ZT_TYPE_STRING) { + + /* + * STRINGS: from zfs/ZT struct to write out to Registry + * Check how much space convert will need, allocate + * buffer + * Convert ascii -> utf8 the string + * Assign to Registry update. + */ + Status = RtlUTF8ToUnicodeN(NULL, 0, + &length, val, len); + if (!NT_SUCCESS(Status)) + goto skip; + if (length == 0) length++; + str.Length = str.MaximumLength = length; + str.Buffer = ExAllocatePoolWithTag(PagedPool, length, + 'ZTST'); + if (str.Buffer == NULL) { + Status = STATUS_NO_MEMORY; + goto skip; + } + + Status = RtlUTF8ToUnicodeN(str.Buffer, + str.MaximumLength, &length, val, len); + str.Length = length; + + len = length; + val = str.Buffer; + + if (!NT_SUCCESS(Status)) + goto skip; + } + + Status = ZwSetValueKey( + regfd, + entry, + 0, + ZT_TYPE_REGISTRY(type), + val, + len); + +skip: + if ((type == ZT_TYPE_STRING) && + str.Buffer != NULL) + ExFreePool(str.Buffer); + + return (Status); +} + +void +sysctl_os_process(PUNICODE_STRING pRegistryPath, ztunable_t *zt) +{ + HANDLE regfd; + + dprintf( + "tunable: '%s/%s' type %d at %p\n", + zt->zt_prefix, zt->zt_name, + zt->zt_type, + zt->zt_ptr); + + /* + * tunable: 'zfs_prefetch_disable' type 0 at FFFFF80731A1B770 + * tunable: 'zfs_prefetch_max_streams' type 1 at FFFFF80730769404 + * tunable: 'zfs_prefetch_array_rd_sz' type 3 at FFFFF80730768EC8 + */ + + /* + * For each tunable; + * - check if registry entry exists + * - no; create entry and set value of tunable. + * - yes; read registry value and set tunable (if different) + */ + NTSTATUS Status; + ULONG length; + + // Linux MODULEPARAM limit is 1024 + static DECLARE_UNICODE_STRING_SIZE(entry, LINUX_MAX_MODULE_PARAM_LEN); + + // Use RegistryPath + Status = RtlUnicodeStringCopy(&entry, pRegistryPath); + if (!NT_SUCCESS(Status)) + return; + + // Add backslash? + Status = RtlUnicodeStringCatString(&entry, L"\\"); + if (!NT_SUCCESS(Status)) + return; + + // keys are "prefix", add to entry + Status = RtlUTF8ToUnicodeN( + (PWSTR)&((uchar_t *)entry.Buffer)[entry.Length], + LINUX_MAX_MODULE_PARAM_LEN - entry.Length, + &length, zt->zt_prefix, strlen(zt->zt_prefix)); + entry.Length += length; + + // If we failed to convert it, just skip it. + if (Status != STATUS_SUCCESS && + Status != STATUS_SOME_NOT_MAPPED) + return; + + // Open registry + regfd = sysctl_os_open_registry(&entry); + if (regfd == 0) + return; + + // create key entry name + Status = RtlUTF8ToUnicodeN(entry.Buffer, LINUX_MAX_MODULE_PARAM_LEN, + &length, zt->zt_name, strlen(zt->zt_name)); + entry.Length = length; + + // If we failed to convert it, just skip it. + if (Status != STATUS_SUCCESS && + Status != STATUS_SOME_NOT_MAPPED) + return; + + // Do we have key? + Status = ZwQueryValueKey( + regfd, + &entry, + KeyValueFullInformation, + NULL, + 0, + &length); + + /* Some tunables need to always be written, think zfs_version */ + if (zt->zt_flag & ZT_FLAG_WRITEONLY) + Status = STATUS_OBJECT_NAME_NOT_FOUND; + + if (Status == STATUS_OBJECT_NAME_NOT_FOUND) { + + Status = sysctl_os_write_registry(regfd, zt, &entry); + + } else { + // Has entry in Registry, read it, and update tunable + // Biggest value we store at the moment is uint64_t + uchar_t *buffer; + buffer = ExAllocatePoolWithTag(PagedPool, length, '!SFZ'); + if (buffer == NULL) + goto failed; + + Status = ZwQueryValueKey( + regfd, + &entry, + KeyValueFullInformation, + buffer, + length, + &length); + + if (NT_SUCCESS(Status)) { + char *strval = NULL; + KEY_VALUE_FULL_INFORMATION *kv = + (KEY_VALUE_FULL_INFORMATION *)buffer; + void *val = NULL; + ULONG len = 0; + ULONG type = 0; + + // _CALL style has to 'type', so look it up first. + if (zt->zt_perm == ZT_ZMOD_RW) { + char **maybestr = NULL; + ZT_GET_VALUE(zt, &maybestr, &len, &type); + + // Set up buffers to SET value. + val = &buffer[kv->DataOffset]; + len = kv->DataLength; + + // If string, make ascii + if (type == ZT_TYPE_STRING) { + /* + * STRINGS: + * + * Static? Convert into buffer assuming + * static MAX. + * Dynamic? + * First if it has a value and + * ALLOCATED, then free(). + * Check string kength needed, allocate + * Then convert ascii -> utf8 the string + */ + /* Already set? free it */ + if (!(zt->zt_flag & ZT_FLAG_STATIC)) { + + if (maybestr != NULL && + *maybestr != NULL) + ExFreePool(*maybestr); + + *maybestr = NULL; + } + /* How much space needed? */ + Status = RtlUnicodeToUTF8N(NULL, 0, + &length, val, len); + if (!NT_SUCCESS(Status)) + goto failed; + + /* Get space */ + strval = ExAllocatePoolWithTag( + PagedPool, length + 1, 'ZTST'); + if (strval == NULL) + goto failed; + + /* Convert to ascii */ + Status = RtlUnicodeToUTF8N(strval, + length, &length, val, len); + if (!NT_SUCCESS(Status)) + goto failed; + + strval[length] = 0; + val = strval; + + } + + ZT_SET_VALUE(zt, &val, &len, &type); + + if ((zt->zt_flag & ZT_FLAG_STATIC) && + strval != NULL) { + ExFreePoolWithTag(strval, '!SFZ'); + } + + + /* + * If the registry exists, it is written to by + * user, the actual value may be changed by the + * _set functions, so we need to call GET again, + * and if it differs, update Registry with real + * (new) value. + * So if its a call-out type, it could have been + * adjusted by the call. + */ + if (zt->zt_func != NULL) { + Status = sysctl_os_write_registry(regfd, + zt, &entry); + } + + + } // RD vs RW + } + +failed: + if (buffer != NULL) + ExFreePoolWithTag(buffer, '!SFZ'); + } + + // Close registry + sysctl_os_close_registry(regfd); + +} + +static void +sysctl_os_fix(void) +{ + uint64_t allmem = arc_all_memory(); + +#ifdef __LP64__ + if (zfs_dirty_data_max_max == 0) + zfs_dirty_data_max_max = MIN(4ULL * 1024 * 1024 * 1024, + allmem * zfs_dirty_data_max_max_percent / 100); +#else + if (zfs_dirty_data_max_max == 0) + zfs_dirty_data_max_max = MIN(1ULL * 1024 * 1024 * 1024, + allmem * zfs_dirty_data_max_max_percent / 100); +#endif + + if (zfs_dirty_data_max == 0) { + zfs_dirty_data_max = allmem * + zfs_dirty_data_max_percent / 100; + zfs_dirty_data_max = MIN(zfs_dirty_data_max, + zfs_dirty_data_max_max); + } + + if (zfs_wrlog_data_max == 0) { + zfs_wrlog_data_max = zfs_dirty_data_max * 2; + } + +} + +_Function_class_(WORKER_THREAD_ROUTINE) void __stdcall +sysctl_os_registry_change(PVOID Parameter) +{ + PUNICODE_STRING RegistryPath = Parameter; + + IO_STATUS_BLOCK iosb; + + // open if first time here + if (registry_notify_fd == 0) { + + registry_notify_fd = sysctl_os_open_registry(RegistryPath); + + if (registry_notify_fd != 0) { + RtlDuplicateUnicodeString( + RTL_DUPLICATE_UNICODE_STRING_ALLOCATE_NULL_STRING | + RTL_DUPLICATE_UNICODE_STRING_NULL_TERMINATE, + RegistryPath, + &sysctl_os_RegistryPath); + ExInitializeWorkItem(&wqi, sysctl_os_registry_change, + &sysctl_os_RegistryPath); + } + } else { + // Notified, re-scan registry + sysctl_os_init(RegistryPath); + // Some tunables must not be unset. + sysctl_os_fix(); + } + + if (registry_notify_fd == 0) + return; + + // Arm / Re-Arm + ZwNotifyChangeKey(registry_notify_fd, NULL, + (PVOID)&wqi, (PVOID)DelayedWorkQueue, + &iosb, + REG_NOTIFY_CHANGE_LAST_SET, + TRUE, NULL, 0, TRUE); +} + +/* + * ZFS_MODULE_PARAM() will create a ztunable_t struct for + * each tunable, so at startup, iterate the "zt" linker-set + * to access all tunables. + */ +void +sysctl_os_init(PUNICODE_STRING RegistryPath) +{ + // iterate the linker-set + ztunable_t **iter = NULL; + int count = 0; + + SET_DECLARE(zt, ztunable_t); + + SET_FOREACH(iter, zt) { + if (iter != NULL && *iter != NULL) { // Sanity + ztunable_t *zt = *iter; + + sysctl_os_process(RegistryPath, zt); + count++; + } + } +} + +void +sysctl_os_fini(void) +{ + HANDLE fd = registry_notify_fd; + registry_notify_fd = 0; + RtlFreeUnicodeString(&sysctl_os_RegistryPath); + + if (fd != 0) + ZwClose(fd); +} + +int +param_set_arc_max(ZFS_MODULE_PARAM_ARGS) +{ + uint64_t val; + + *type = ZT_TYPE_U64; + + if (set == B_FALSE) { + *ptr = &zfs_arc_max; + *len = sizeof (zfs_arc_max); + return (0); + } + + ASSERT3U(*len, >=, sizeof (zfs_arc_max)); + + val = *(uint64_t *)(*ptr); + + if (val != 0 && (val < MIN_ARC_MAX || val <= arc_c_min || + val >= arc_all_memory())) + return (SET_ERROR(EINVAL)); + + zfs_arc_max = val; + arc_tuning_update(B_TRUE); + + /* Update the sysctl to the tuned value */ + if (val != 0) + zfs_arc_max = arc_c_max; + + return (0); +} + +int +param_set_arc_min(ZFS_MODULE_PARAM_ARGS) +{ + uint64_t val; + + *type = ZT_TYPE_U64; + + if (set == B_FALSE) { + *ptr = &zfs_arc_min; + *len = sizeof (zfs_arc_min); + return (0); + } + + ASSERT3U(*len, >=, sizeof (zfs_arc_min)); + + val = *(uint64_t *)(*ptr); + + if (val != 0 && (val < 2ULL << SPA_MAXBLOCKSHIFT || val > arc_c_max)) + return (SET_ERROR(EINVAL)); + + zfs_arc_min = val; + arc_tuning_update(B_TRUE); + + if (val != 0) + zfs_arc_min = arc_c_min; + + return (0); +} + +static int +sysctl_vfs_zfs_arc_no_grow_shift(ZFS_MODULE_PARAM_ARGS) +{ + int val; + + *type = ZT_TYPE_INT; + + if (set == B_FALSE) { + *ptr = &arc_no_grow_shift; + *len = sizeof (arc_no_grow_shift); + return (0); + } + + ASSERT3U(*len, >=, sizeof (arc_no_grow_shift)); + + val = *(int *)(*ptr); + + if (val < 0 || val >= arc_shrink_shift) + return (EINVAL); + + arc_no_grow_shift = val; + return (0); +} + +int +param_set_arc_u64(ZFS_MODULE_PARAM_ARGS) +{ + *ptr = zt->zt_ptr; + *len = sizeof (uint64_t); + *type = ZT_TYPE_U64; + + arc_tuning_update(B_TRUE); + + return (0); +} + +int +param_set_arc_int(ZFS_MODULE_PARAM_ARGS) +{ + *ptr = zt->zt_ptr; + *len = sizeof (uint64_t); + *type = ZT_TYPE_U64; + + arc_tuning_update(B_TRUE); + + return (0); +} + +/* dbuf.c */ + +/* dmu.c */ + +/* dmu_zfetch.c */ + +/* dsl_pool.c */ + +/* dnode.c */ + +/* dsl_scan.c */ + +/* metaslab.c */ + +/* spa_misc.c */ +extern int zfs_flags; +static int +sysctl_vfs_zfs_debug_flags(ZFS_MODULE_PARAM_ARGS) +{ + int val; + + *type = ZT_TYPE_INT; + + if (set == B_FALSE) { + *ptr = &zfs_flags; + *len = sizeof (zfs_flags); + return (0); + } + + ASSERT3U(*len, >=, sizeof (zfs_flags)); + + val = *(int *)(*ptr); + + /* + * ZFS_DEBUG_MODIFY must be enabled prior to boot so all + * arc buffers in the system have the necessary additional + * checksum data. However, it is safe to disable at any + * time. + */ + if (!(zfs_flags & ZFS_DEBUG_MODIFY)) + val &= ~ZFS_DEBUG_MODIFY; + zfs_flags = val; + + return (0); +} + +int +param_set_deadman_synctime(ZFS_MODULE_PARAM_ARGS) +{ + uint64_t val; + + *type = ZT_TYPE_U64; + + if (set == B_FALSE) { + *ptr = &zfs_deadman_synctime_ms; + *len = sizeof (zfs_deadman_synctime_ms); + return (0); + } + + ASSERT3U(*len, >=, sizeof (zfs_deadman_synctime_ms)); + + val = *(uint64_t *)(*ptr); + + zfs_deadman_synctime_ms = val; + + spa_set_deadman_synctime(MSEC2NSEC(zfs_deadman_synctime_ms)); + + return (0); +} + +int +param_set_deadman_ziotime(ZFS_MODULE_PARAM_ARGS) +{ + uint64_t val; + + *type = ZT_TYPE_U64; + + if (set == B_FALSE) { + *ptr = &zfs_deadman_ziotime_ms; + *len = sizeof (zfs_deadman_ziotime_ms); + return (0); + } + + ASSERT3U(*len, >=, sizeof (zfs_deadman_ziotime_ms)); + + val = *(uint64_t *)(*ptr); + + zfs_deadman_ziotime_ms = val; + + spa_set_deadman_ziotime(MSEC2NSEC(zfs_deadman_synctime_ms)); + + return (0); +} + +int +param_set_deadman_failmode(ZFS_MODULE_PARAM_ARGS) +{ + char buf[16]; + + *type = ZT_TYPE_STRING; + + if (set == B_FALSE) { + *ptr = (void *)zfs_deadman_failmode; + *len = strlen(zfs_deadman_failmode) + 1; + return (0); + } + + strlcpy(buf, *ptr, sizeof (buf)); + + if (strcmp(buf, zfs_deadman_failmode) == 0) + return (0); + if (strcmp(buf, "wait") == 0) + zfs_deadman_failmode = "wait"; + if (strcmp(buf, "continue") == 0) + zfs_deadman_failmode = "continue"; + if (strcmp(buf, "panic") == 0) + zfs_deadman_failmode = "panic"; + + return (-param_set_deadman_failmode_common(buf)); +} + + +/* spacemap.c */ + +/* vdev.c */ +int +param_set_min_auto_ashift(ZFS_MODULE_PARAM_ARGS) +{ + uint64_t val; + + *type = ZT_TYPE_U64; + + *ptr = &zfs_vdev_min_auto_ashift; + *len = sizeof (zfs_vdev_min_auto_ashift); + + val = zfs_vdev_min_auto_ashift; + + if (val < ASHIFT_MIN || val > zfs_vdev_max_auto_ashift) + return (SET_ERROR(EINVAL)); + + zfs_vdev_min_auto_ashift = val; + + return (0); +} + +int +param_set_max_auto_ashift(ZFS_MODULE_PARAM_ARGS) +{ + uint64_t val; + + *type = ZT_TYPE_U64; + + *ptr = &zfs_vdev_max_auto_ashift; + *len = sizeof (zfs_vdev_max_auto_ashift); + + val = zfs_vdev_max_auto_ashift; + + if (val > ASHIFT_MAX || val < zfs_vdev_min_auto_ashift) + return (SET_ERROR(EINVAL)); + + zfs_vdev_max_auto_ashift = val; + + return (0); +} + +int +param_set_slop_shift(ZFS_MODULE_PARAM_ARGS) +{ + int val; + + *type = ZT_TYPE_INT; + + if (set == B_FALSE) { + *ptr = &spa_slop_shift; + *len = sizeof (spa_slop_shift); + return (0); + } + + ASSERT3U(*len, >=, sizeof (spa_slop_shift)); + + val = *(int *)(*ptr); + + if (val < 1 || val > 31) + return (EINVAL); + + spa_slop_shift = val; + + return (0); +} + +int +param_set_multihost_interval(ZFS_MODULE_PARAM_ARGS) +{ + *ptr = zt->zt_ptr; + *len = sizeof (uint64_t); + *type = ZT_TYPE_U64; + + if (spa_mode_global != SPA_MODE_UNINIT) + mmp_signal_all_threads(); + + return (0); +} diff --git a/module/os/windows/zfs/vdev_disk.c b/module/os/windows/zfs/vdev_disk.c new file mode 100644 index 000000000000..743fad1ecc1b --- /dev/null +++ b/module/os/windows/zfs/vdev_disk.c @@ -0,0 +1,904 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2017 Jorgen Lundman + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + + +/* + * Virtual device vector for disks. + */ + + +wchar_t zfs_vdev_protection_filter[ZFS_MODULE_STRMAX] = { L"\0" }; + +static void vdev_disk_close(vdev_t *); + +extern void UnlockAndFreeMdl(PMDL); + +_Atomic uint64_t spl_lowest_vdev_disk_stack_remaining = 0; + +static void +vdev_disk_alloc(vdev_t *vd) +{ + vdev_disk_t *dvd; + + dvd = vd->vdev_tsd = kmem_zalloc(sizeof (vdev_disk_t), KM_SLEEP); + +} + +static void +vdev_disk_free(vdev_t *vd) +{ + vdev_disk_t *dvd = vd->vdev_tsd; + + if (dvd == NULL) + return; + + kmem_free(dvd, sizeof (vdev_disk_t)); + vd->vdev_tsd = NULL; +} + +static void disk_exclusive(DEVICE_OBJECT *device, boolean_t excl) +{ + SET_DISK_ATTRIBUTES diskAttrs = { 0 }; + DWORD requiredSize; + DWORD returnedSize; + + // Set disk attributes. + diskAttrs.Version = sizeof (diskAttrs); + diskAttrs.AttributesMask = + DISK_ATTRIBUTE_OFFLINE | DISK_ATTRIBUTE_READ_ONLY; + diskAttrs.Attributes = + excl ? DISK_ATTRIBUTE_OFFLINE | DISK_ATTRIBUTE_READ_ONLY : 0; + diskAttrs.Persist = FALSE; + + if (kernel_ioctl(device, NULL, IOCTL_DISK_SET_DISK_ATTRIBUTES, + &diskAttrs, sizeof (diskAttrs), NULL, 0) != 0) { + dprintf("IOCTL_DISK_SET_DISK_ATTRIBUTES"); + return; + } + + // Tell the system that the disk was changed. + if (kernel_ioctl(device, NULL, IOCTL_DISK_UPDATE_PROPERTIES, NULL, + 0, NULL, 0) != 0) + dprintf("IOCTL_DISK_UPDATE_PROPERTIES"); + +} + + +/* + * We want to be loud in DEBUG kernels when DKIOCGMEDIAINFOEXT fails, or when + * even a fallback to DKIOCGMEDIAINFO fails. + */ +#ifdef DEBUG +#define VDEV_DEBUG(...) cmn_err(CE_NOTE, __VA_ARGS__) +#else +#define VDEV_DEBUG(...) /* Nothing... */ +#endif + +static int +vdev_disk_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, + uint64_t *ashift, uint64_t *physical_ashif) +{ + spa_t *spa = vd->vdev_spa; + vdev_disk_t *dvd = vd->vdev_tsd; + int error = EINVAL; + uint64_t capacity = 0, blksz = 0, pbsize = 0; + int isssd; + char *vdev_path = NULL; + + PAGED_CODE(); + + dprintf("%s: open of '%s' (physpath '%s')\n", __func__, + vd->vdev_path, vd->vdev_physpath ? vd->vdev_physpath : ""); + /* + * We must have a pathname, and it must be absolute. + * It can also start with # for partition encoded paths + */ + if (vd->vdev_path == NULL || + (vd->vdev_path[0] != '/' && vd->vdev_path[0] != '#' && + vd->vdev_path[0] != '\\')) { + vd->vdev_stat.vs_aux = VDEV_AUX_BAD_LABEL; + return (SET_ERROR(EINVAL)); + } + + /* + * Reopen the device if it's not currently open. Otherwise, + * just update the physical size of the device. + */ + if (dvd != NULL) { + if (dvd->vd_ldi_offline && dvd->vd_lh == NULL) { + /* + * If we are opening a device in its offline notify + * context, the LDI handle was just closed. Clean + * up the LDI event callbacks and free vd->vdev_tsd. + */ + vdev_disk_free(vd); + } else { + ASSERT(vd->vdev_reopening); + goto skip_open; + } + } + + /* + * Create vd->vdev_tsd. + */ + vdev_disk_alloc(vd); + dvd = vd->vdev_tsd; + + /* + * If we have not yet opened the device, try to open it by the + * specified path. + */ + NTSTATUS ntstatus; + uint8_t *FileName = NULL; + uint32_t FileLength; + + // Use vd->vdev_physpath first, if set, otherwise + // usual vd->vdev_path + if (vd->vdev_physpath) + vdev_path = spa_strdup(vd->vdev_physpath); + else + vdev_path = spa_strdup(vd->vdev_path); + + /* Check for partition encoded paths */ + if (vdev_path[0] == '#') { + uint8_t *end; + end = &vdev_path[0]; + while (end && end[0] == '#') end++; + ddi_strtoull(end, &end, 10, &dvd->vdev_win_offset); + while (end && end[0] == '#') end++; + ddi_strtoull(end, &end, 10, &dvd->vdev_win_length); + while (end && end[0] == '#') end++; + + FileName = end; + } else { + FileName = vdev_path; + + // Sometimes only vdev_path is set, with "/dev/physicaldrive" + // make it be " \??\physicaldrive" space skipped over. + if (strncmp("/dev/", FileName, 5) == 0) { + FileName[0] = ' '; + FileName[1] = '\\'; + FileName[2] = '?'; + FileName[3] = '?'; + FileName[4] = '\\'; + FileName++; + } + } + + // Apparently in Userland it is "\\?\" but in + // kernel has to be "\??\" - is there not a name that works in both? + if (strncmp("\\\\?\\", FileName, 4) == 0) { + FileName[1] = '?'; + } + + dprintf("%s: opening '%s'\n", __func__, FileName); + + ANSI_STRING AnsiFilespec; + UNICODE_STRING UnicodeFilespec; + OBJECT_ATTRIBUTES ObjectAttributes; + + SHORT UnicodeName[PATH_MAX]; + CHAR AnsiName[PATH_MAX]; + USHORT NameLength = 0; + + memset(UnicodeName, 0, sizeof (SHORT) * PATH_MAX); + memset(AnsiName, 0, sizeof (UCHAR) * PATH_MAX); + + NameLength = strlen(FileName); + ASSERT(NameLength < PATH_MAX); + + memmove(AnsiName, FileName, NameLength); + + AnsiFilespec.MaximumLength = AnsiFilespec.Length = NameLength; + AnsiFilespec.Buffer = AnsiName; + + UnicodeFilespec.MaximumLength = PATH_MAX * 2; + UnicodeFilespec.Length = 0; + UnicodeFilespec.Buffer = (PWSTR)UnicodeName; + + RtlAnsiStringToUnicodeString(&UnicodeFilespec, &AnsiFilespec, FALSE); + + ObjectAttributes.Length = sizeof (OBJECT_ATTRIBUTES); + ObjectAttributes.RootDirectory = NULL; + ObjectAttributes.Attributes = OBJ_KERNEL_HANDLE; + ObjectAttributes.ObjectName = &UnicodeFilespec; + ObjectAttributes.SecurityDescriptor = NULL; + ObjectAttributes.SecurityQualityOfService = NULL; + IO_STATUS_BLOCK iostatus; + + ntstatus = ZwCreateFile(&dvd->vd_lh, + spa_mode(spa) == SPA_MODE_READ ? GENERIC_READ | SYNCHRONIZE : + GENERIC_READ | GENERIC_WRITE | SYNCHRONIZE, + &ObjectAttributes, + &iostatus, + 0, + FILE_ATTRIBUTE_NORMAL, + /* FILE_SHARE_WRITE | */ FILE_SHARE_READ, + FILE_OPEN, + FILE_SYNCHRONOUS_IO_NONALERT | + (spa_mode(spa) == SPA_MODE_READ ? 0 : + FILE_NO_INTERMEDIATE_BUFFERING), + NULL, + 0); + + if (ntstatus == STATUS_SUCCESS) { + error = 0; + } else { + error = EINVAL; // GetLastError(); + dvd->vd_lh = NULL; + } + + /* + * If we succeeded in opening the device, but 'vdev_wholedisk' + * is not yet set, then this must be a slice. + */ + if (error == 0 && vd->vdev_wholedisk == -1ULL) + vd->vdev_wholedisk = 0; + + if (error) { + vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; + spa_strfree(vdev_path); + return (error); + } + + // Since we will use DeviceObject and FileObject to do ioctl and IO + // we grab them now and lock them in place. + // Convert HANDLE to FileObject + PFILE_OBJECT FileObject; + PDEVICE_OBJECT DeviceObject; + NTSTATUS status; + + // This adds a reference to FileObject + status = ObReferenceObjectByHandle( + dvd->vd_lh, // fixme, keep this in dvd + 0, + *IoFileObjectType, + KernelMode, + &FileObject, + NULL); + if (status != STATUS_SUCCESS) { + ZwClose(dvd->vd_lh); + dvd->vd_lh = NULL; + vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; + spa_strfree(vdev_path); + return (EIO); + } + + + // Convert FileObject to DeviceObject + PDEVICE_OBJECT pTopDevice = IoGetRelatedDeviceObject(FileObject); + PDEVICE_OBJECT pSendToDevice = pTopDevice; // default + +/* + * Move to the top of the device stack or until we find the protection + * filter driver. We need to stay under that driver so we can still + * access the disk after protecting it. + * The custom protection filter is optional: if none set we stay under + * the default "partmgr" driver; otherwise we will stay under the first + * one found. By default the disk gets minimal protection being set offline + * and read only through "partmgr". A custom filter driver can provide + * enhanced protection for the vdev disk. + */ + UNICODE_STRING customFilterName; + UNICODE_STRING defaultFilterName; + RtlInitUnicodeString(&customFilterName, zfs_vdev_protection_filter); + RtlInitUnicodeString(&defaultFilterName, L"\\Driver\\partmgr"); + + DeviceObject = FileObject->DeviceObject; // bottom of stack + + /* + * Determine the actual size of the device. + */ + + // If we are given the size in "#offset#size#path" + capacity = dvd->vdev_win_length; + + // STORAGE_DEVICE_NUMBER sdn = { 0 }; + // kernel_ioctl(DeviceObject, FileObject, + // IOCTL_STORAGE_GET_DEVICE_NUMBER, + // NULL, 0, &sdn, sizeof(sdn)); + + if (capacity == 0ULL) { + PARTITION_INFORMATION_EX pix = {0}; + if (kernel_ioctl(DeviceObject, FileObject, + IOCTL_DISK_GET_PARTITION_INFO_EX, + NULL, 0, &pix, sizeof (pix)) == 0) + capacity = pix.PartitionLength.QuadPart; + } + + if (capacity == 0ULL) { + PARTITION_INFORMATION pi = {0}; + if (kernel_ioctl(DeviceObject, FileObject, + IOCTL_DISK_GET_PARTITION_INFO, + NULL, 0, &pi, sizeof (pi)) == 0) + capacity = pi.PartitionLength.QuadPart; + } + + if (capacity == 0ULL) { + GET_LENGTH_INFORMATION len; + + if (kernel_ioctl(DeviceObject, FileObject, + IOCTL_DISK_GET_LENGTH_INFO, + NULL, 0, &len, sizeof (len)) == 0) + capacity = len.Length.QuadPart; + } + + if (capacity == 0ULL && vd->vdev_wholedisk) { + DISK_GEOMETRY_EX geometry_ex = {0}; + DWORD len; + + if (kernel_ioctl(DeviceObject, FileObject, + IOCTL_DISK_GET_DRIVE_GEOMETRY_EX, + NULL, 0, &geometry_ex, sizeof (geometry_ex)) == 0) + capacity = geometry_ex.DiskSize.QuadPart; + } + + if (capacity == 0ULL && vd->vdev_wholedisk) { + STORAGE_READ_CAPACITY cap = {0}; + cap.Version = sizeof (STORAGE_READ_CAPACITY); + if (kernel_ioctl(DeviceObject, FileObject, + IOCTL_STORAGE_READ_CAPACITY, + NULL, 0, &cap, sizeof (cap)) == 0) + capacity = cap.DiskLength.QuadPart; + } + + // Remember the size for when we skip_open + dvd->vdev_win_length = capacity; + + // Now walk up the stack locating the device to lock + while (DeviceObject) { + if ((zfs_vdev_protection_filter[0] != L'\0' ? + !RtlCompareUnicodeString( + &DeviceObject->DriverObject->DriverName, + &customFilterName, TRUE) : FALSE) || + !RtlCompareUnicodeString( + &DeviceObject->DriverObject->DriverName, + &defaultFilterName, TRUE)) { + dprintf("%s: disk %s : vdev protection " + "filter set to %S\n", + __func__, FileName, + DeviceObject->DriverObject->DriverName.Buffer); + break; + } + pSendToDevice = DeviceObject; + DeviceObject = DeviceObject->AttachedDevice; + } + DeviceObject = pSendToDevice; + + // Grab a reference to DeviceObject + ObReferenceObject(DeviceObject); + + dvd->vd_FileObject = FileObject; + dvd->vd_ExclusiveObject = DeviceObject; + + dvd->vd_DeviceObject = DeviceObject; + ObReferenceObject(dvd->vd_DeviceObject); + + // Make disk readonly and offline, so that users can't + // partition/format it. + if (vd->vdev_wholedisk) + disk_exclusive(dvd->vd_ExclusiveObject, TRUE); + spa_strfree(vdev_path); + +skip_open: + + capacity = dvd->vdev_win_length; + + /* + * Determine the device's minimum transfer size. + * If the ioctl isn't supported, assume DEV_BSIZE. + */ + // fill in capacity, blksz, pbsize + STORAGE_PROPERTY_QUERY storageQuery; + memset(&storageQuery, 0, sizeof (STORAGE_PROPERTY_QUERY)); + storageQuery.PropertyId = StorageAccessAlignmentProperty; + storageQuery.QueryType = PropertyStandardQuery; + + STORAGE_ACCESS_ALIGNMENT_DESCRIPTOR diskAlignment = { 0 }; + memset(&diskAlignment, 0, sizeof (STORAGE_ACCESS_ALIGNMENT_DESCRIPTOR)); + DWORD outsize; + + error = kernel_ioctl(dvd->vd_DeviceObject, dvd->vd_FileObject, + IOCTL_STORAGE_QUERY_PROPERTY, + &storageQuery, sizeof (STORAGE_PROPERTY_QUERY), + &diskAlignment, sizeof (STORAGE_ACCESS_ALIGNMENT_DESCRIPTOR)); + + if (error == 0) { + blksz = diskAlignment.BytesPerLogicalSector; + pbsize = diskAlignment.BytesPerPhysicalSector; + } else { + blksz = pbsize = DEV_BSIZE; + } + + // Set psize to the size of the partition. For now, assume virtual + // since ioctls do not seem to work. + *psize = capacity; + + // Set max_psize to the biggest it can be, expanding.. + *max_psize = *psize; + + if (!blksz) blksz = DEV_BSIZE; + if (!pbsize) pbsize = DEV_BSIZE; + + *ashift = highbit64(MAX(pbsize, SPA_MINBLOCKSIZE)) - 1; + dprintf("%s: picked ashift %llu for device\n", __func__, *ashift); + + /* + * Clear the nowritecache bit, so that on a vdev_reopen() we will + * try again. + */ + vd->vdev_nowritecache = B_FALSE; + + /* Set when device reports it supports TRIM. */ + vd->vdev_has_trim = !!blk_queue_discard(dvd->vd_DeviceObject); + + /* Set when device reports it supports secure TRIM. */ + vd->vdev_has_securetrim = + !!blk_queue_discard_secure(dvd->vd_DeviceObject); + + /* Inform the ZIO pipeline that we are non-rotational */ + /* Best choice seems to be either TRIM, or SeekPenalty */ + vd->vdev_nonrot = + vd->vdev_has_trim || blk_queue_nonrot(dvd->vd_DeviceObject); + + dprintf("%s: nonrot %d, trim %d, securetrim %d\n", __func__, + vd->vdev_nonrot, vd->vdev_has_trim, vd->vdev_has_securetrim); + + return (0); +} + + +static void +vdev_disk_close(vdev_t *vd) +{ + vdev_disk_t *dvd = vd->vdev_tsd; + + if (vd->vdev_reopening || dvd == NULL) + return; + + vd->vdev_delayed_close = B_FALSE; + /* + * If we closed the LDI handle due to an offline notify from LDI, + * don't free vd->vdev_tsd or unregister the callbacks here; + * the offline finalize callback or a reopen will take care of it. + */ + if (dvd->vd_ldi_offline) + return; + + if (dvd->vd_lh != NULL) { + dprintf("%s: \n", __func__); + + // Undo disk readonly and offline. + if (vd->vdev_wholedisk) + disk_exclusive(dvd->vd_ExclusiveObject, FALSE); + + // Release our holds + ObDereferenceObject(dvd->vd_FileObject); + ObDereferenceObject(dvd->vd_DeviceObject); + ObDereferenceObject(dvd->vd_ExclusiveObject); + // Close file + ZwClose(dvd->vd_lh); + } + + dvd->vd_lh = NULL; + dvd->vd_FileObject = NULL; + dvd->vd_DeviceObject = NULL; + dvd->vd_ExclusiveObject = NULL; + + vdev_disk_free(vd); +} + +int +vdev_disk_physio(vdev_t *vd, caddr_t data, + size_t size, uint64_t offset, int flags, boolean_t isdump) +{ + vdev_disk_t *dvd = vd->vdev_tsd; + + // dprintf("%s: \n", __func__); + + /* + * If the vdev is closed, it's likely in the REMOVED or FAULTED state. + * Nothing to be done here but return failure. + */ + if (dvd == NULL || (dvd->vd_ldi_offline)) + return (EIO); + + ASSERT(vd->vdev_ops == &vdev_disk_ops); + + return (EIO); +} + +static void +vdev_disk_ioctl_free(zio_t *zio) +{ + kmem_free(zio->io_vsd, sizeof (struct dk_callback)); +} + +static const zio_vsd_ops_t vdev_disk_vsd_ops = { + vdev_disk_ioctl_free, + zio_vsd_default_cksum_report +}; + +static void +vdev_disk_ioctl_done(void *zio_arg, int error) +{ + zio_t *zio = zio_arg; + + zio->io_error = error; + + zio_interrupt(zio); +} + +static void +vdev_disk_io_start_done(__in PVOID pDummy, __in PVOID pWkParms) +{ + zio_t *zio = (zio_t *)pWkParms; + UNREFERENCED_PARAMETER(pDummy); + + ASSERT(zio != NULL); + + IoFreeWorkItem(zio->windows.work_item); + zio->windows.work_item = NULL; + + NTSTATUS status = zio->windows.irp->IoStatus.Status; + zio->io_error = (!NT_SUCCESS(status) ? EIO : 0); + + UnlockAndFreeMdl(zio->windows.irp->MdlAddress); + IoFreeIrp(zio->windows.irp); + + // Return abd buf + if (zio->io_type == ZIO_TYPE_READ) { + VERIFY3S(zio->io_abd->abd_size, >=, zio->io_size); + abd_return_buf_copy(zio->io_abd, zio->windows.b_addr, + zio->io_size); + } else { + VERIFY3S(zio->io_abd->abd_size, >=, zio->io_size); + abd_return_buf(zio->io_abd, zio->windows.b_addr, + zio->io_size); + } + + zio_delay_interrupt(zio); +} + +/* + * IO has finished callback, in Windows this is called as a different + * IRQ level, so we can practically do nothing here. (Can't call mutex + * locking, like from kmem_free()) + */ +IO_COMPLETION_ROUTINE vdev_disk_io_intr; + +static NTSTATUS +vdev_disk_io_intr(PDEVICE_OBJECT DeviceObject, PIRP irp, PVOID Context) +{ + zio_t *zio = (zio_t *)Context; + + ASSERT(zio != NULL); + +/* + * Unfortunately: + * Whatever thread happened to be running is "borrowed" to handle + * the completion. + * As about DPCs - in Windows, they are not bound to a thread, + * KeGetCurrentThread is just nonsense in them. + * + * So, the whole Windows kernel framework just does not support the notion of + * "current thread" in DPCs and thus completion routines. + * + * This means our call to "mutex_enter()" will "panic: lock against myself" + * if that thread it "borrowed" is the actual owner thread. + * + * So schedule IoQueueWorkItem() to call the done function in a real context. + */ + + VERIFY3P(zio->windows.work_item, !=, NULL); + IoQueueWorkItem(zio->windows.work_item, vdev_disk_io_start_done, + DelayedWorkQueue, zio); + return (STATUS_MORE_PROCESSING_REQUIRED); +} + +static void +vdev_disk_io_start(zio_t *zio) +{ + vdev_t *vd = zio->io_vd; + vdev_disk_t *dvd = vd->vdev_tsd; + struct dk_callback *dkc; + buf_t *bp; + unsigned long trim_flags = 0; + int flags, error = 0; + + // dprintf("%s: type 0x%x offset 0x%llx len 0x%llx \n", + // __func__, zio->io_type, zio->io_offset, zio->io_size); + + /* + * If the vdev is closed, it's likely in the REMOVED or FAULTED state. + * Nothing to be done here but return failure. + */ + if (dvd == NULL || (dvd->vd_ldi_offline)) { + zio->io_error = ENXIO; + zio_interrupt(zio); + return; + } + + switch (zio->io_type) { + case ZIO_TYPE_IOCTL: + + if (!vdev_readable(vd)) { + zio->io_error = SET_ERROR(ENXIO); + zio_interrupt(zio); + return; + } + + switch (zio->io_cmd) { + case DKIOCFLUSHWRITECACHE: + + if (zfs_nocacheflush) + break; + + if (vd->vdev_nowritecache) { + zio->io_error = SET_ERROR(ENOTSUP); + break; + } + + zio->io_vsd = dkc = kmem_alloc(sizeof (*dkc), KM_SLEEP); + zio->io_vsd_ops = &vdev_disk_vsd_ops; + + dkc->dkc_callback = vdev_disk_ioctl_done; +// dkc->dkc_flag = FLUSH_VOLATILE; + dkc->dkc_cookie = zio; + + // Windows: find me +// error = ldi_ioctl(dvd->vd_lh, zio->io_cmd, +// (uintptr_t)dkc, FKIOCTL, kcred, NULL); + + if (error == 0) { + /* + * The ioctl will be done asychronously, + * and will call vdev_disk_ioctl_done() + * upon completion. + */ + zio_execute(zio); // until we have ioctl + return; + } + + zio->io_error = error; + + break; + + default: + zio->io_error = SET_ERROR(ENOTSUP); + } /* io_cmd */ + + zio_execute(zio); + return; + + case ZIO_TYPE_WRITE: + // if (zio->io_priority == ZIO_PRIORITY_SYNC_WRITE) + // Windows: do we need to pass this down IRP somehow? + break; + + case ZIO_TYPE_READ: + // if (zio->io_priority == ZIO_PRIORITY_SYNC_READ) + // Windows: do we need to pass this down IRP somehow? + break; + + case ZIO_TYPE_TRIM: +#if defined(BLKDEV_DISCARD_SECURE) + if (zio->io_trim_flags & ZIO_TRIM_SECURE) + trim_flags |= BLKDEV_DISCARD_SECURE; +#endif + zio->io_error = -blkdev_issue_discard_bytes( + dvd->vd_DeviceObject, + zio->io_offset, zio->io_size, trim_flags); + zio_interrupt(zio); + return; + + default: + zio->io_error = SET_ERROR(ENOTSUP); + zio_execute(zio); + return; + } /* io_type */ + + ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE); + + zio->io_target_timestamp = zio_handle_io_delay(zio); + + const ULONG_PTR r = IoGetRemainingStackSize(); + + if (spl_lowest_vdev_disk_stack_remaining == 0) { + spl_lowest_vdev_disk_stack_remaining = r; + } else if (spl_lowest_vdev_disk_stack_remaining > r) { + spl_lowest_vdev_disk_stack_remaining = r; + } + + ASSERT(zio->io_size != 0); + + PIRP irp = NULL; + PIO_STACK_LOCATION irpStack = NULL; + LARGE_INTEGER offset; + + offset.QuadPart = zio->io_offset + dvd->vdev_win_offset; + + /* + * Start IO -> IOCompletion callback 'vdev_disk_io_intr()' + * -> IoQueueWorkItem(DelayedWorkQueue) -> callback vdev_disk_io_done() + */ + + zio->windows.work_item = IoAllocateWorkItem(dvd->vd_DeviceObject); + if (zio->windows.work_item == NULL) { + zio->io_error = EIO; + zio_interrupt(zio); + return; + } + + if (zio->io_type == ZIO_TYPE_READ) { + + zio->windows.b_addr = + abd_borrow_buf(zio->io_abd, zio->io_size); + + irp = IoBuildAsynchronousFsdRequest(IRP_MJ_READ, + dvd->vd_DeviceObject, + zio->windows.b_addr, + (ULONG)zio->io_size, + &offset, + &zio->windows.IoStatus); + + } else { + zio->windows.b_addr = + abd_borrow_buf_copy(zio->io_abd, zio->io_size); + + irp = IoBuildAsynchronousFsdRequest(IRP_MJ_WRITE, + dvd->vd_DeviceObject, + zio->windows.b_addr, + (ULONG)zio->io_size, + &offset, + &zio->windows.IoStatus); + } + + if (!irp) { + + if (zio->io_type == ZIO_TYPE_READ) { + abd_return_buf_copy(zio->io_abd, zio->windows.b_addr, + zio->io_size); + } else { + abd_return_buf(zio->io_abd, zio->windows.b_addr, + zio->io_size); + } + + IoFreeWorkItem(zio->windows.work_item); + zio->io_error = EIO; + zio_interrupt(zio); + return; + } + + zio->windows.irp = irp; + + irpStack = IoGetNextIrpStackLocation(irp); + + irpStack->Flags |= SL_OVERRIDE_VERIFY_VOLUME; + irpStack->FileObject = dvd->vd_FileObject; + + IoSetCompletionRoutine(irp, + vdev_disk_io_intr, + zio, // "Context" in vdev_disk_io_intr() + TRUE, // On Success + TRUE, // On Error + TRUE); // On Cancel + + IoCallDriver(dvd->vd_DeviceObject, irp); + +} + +static void +vdev_disk_io_done(zio_t *zio) +{ + vdev_t *vd = zio->io_vd; + + /* + * If the device returned EIO, then attempt a DKIOCSTATE ioctl to see if + * the device has been removed. If this is the case, then we trigger an + * asynchronous removal of the device. Otherwise, probe the device and + * make sure it's still accessible. + */ + if (zio->io_error == EIO && !vd->vdev_remove_wanted) { + vdev_disk_t *dvd = vd->vdev_tsd; +// int state = DKIO_NONE; + } else if (!vd->vdev_delayed_close) { + vd->vdev_delayed_close = B_TRUE; + } +} + +static void +vdev_disk_hold(vdev_t *vd) +{ + ASSERT(spa_config_held(vd->vdev_spa, SCL_STATE, RW_WRITER)); + + /* We must have a pathname, and it must be absolute. */ + if (vd->vdev_path == NULL || vd->vdev_path[0] != '/') + return; + + /* + * Only prefetch path and devid info if the device has + * never been opened. + */ + if (vd->vdev_tsd != NULL) + return; + +} + +static void +vdev_disk_rele(vdev_t *vd) +{ + ASSERT(spa_config_held(vd->vdev_spa, SCL_STATE, RW_WRITER)); + + /* XXX: Implement me as a vnode rele for the device */ +} + +vdev_ops_t vdev_disk_ops = { + .vdev_op_init = NULL, + .vdev_op_fini = NULL, + .vdev_op_open = vdev_disk_open, + .vdev_op_close = vdev_disk_close, + .vdev_op_asize = vdev_default_asize, + .vdev_op_min_asize = vdev_default_min_asize, + .vdev_op_min_alloc = NULL, + .vdev_op_io_start = vdev_disk_io_start, + .vdev_op_io_done = vdev_disk_io_done, + .vdev_op_state_change = NULL, + .vdev_op_need_resilver = NULL, + .vdev_op_hold = vdev_disk_hold, + .vdev_op_rele = vdev_disk_rele, + .vdev_op_remap = NULL, + .vdev_op_xlate = vdev_default_xlate, + .vdev_op_rebuild_asize = NULL, + .vdev_op_metaslab_init = NULL, + .vdev_op_config_generate = NULL, + .vdev_op_nparity = NULL, + .vdev_op_ndisks = NULL, + .vdev_op_type = VDEV_TYPE_DISK, /* name of this vdev type */ + .vdev_op_leaf = B_TRUE /* leaf vdev */ +}; + +/* + * Given the root disk device devid or pathname, read the label from + * the device, and construct a configuration nvlist. + */ +int +vdev_disk_read_rootlabel(char *devpath, char *devid, nvlist_t **config) +{ + return (-1); +} diff --git a/module/os/windows/zfs/vdev_file.c b/module/os/windows/zfs/vdev_file.c new file mode 100644 index 000000000000..7108967593ae --- /dev/null +++ b/module/os/windows/zfs/vdev_file.c @@ -0,0 +1,502 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011, 2015 by Delphix. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * Virtual device vector for files. + */ + +static taskq_t *vdev_file_taskq; + +#ifdef _KERNEL +extern void UnlockAndFreeMdl(PMDL); +#endif + +static void +vdev_file_hold(vdev_t *vd) +{ + ASSERT(vd->vdev_path != NULL); +} + +static void +vdev_file_rele(vdev_t *vd) +{ + ASSERT(vd->vdev_path != NULL); +} + +#ifdef _KERNEL +extern int VOP_GETATTR(struct vnode *vp, vattr_t *vap, int flags, + void *x3, void *x4); +#endif + +static mode_t +vdev_file_open_mode(spa_mode_t spa_mode) +{ + mode_t mode = 0; + // TODO :- Add flags + if ((spa_mode & SPA_MODE_READ) && (spa_mode & SPA_MODE_WRITE)) { + mode = O_RDWR; + } else if (spa_mode & SPA_MODE_READ) { + mode = O_RDONLY; + } else if (spa_mode & SPA_MODE_WRITE) { + mode = O_WRONLY; + } + return (mode); +} + +static void +vdev_file_close(vdev_t *vd); + +static int +vdev_file_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, + uint64_t *ashift, uint64_t *physical_ashift) +{ + vdev_file_t *vf; + zfs_file_t *fp; + zfs_file_attr_t zfa; + int error = 0; + char *vdev_path = NULL; + uint8_t *FileName = NULL; + + dprintf("vdev_file_open %p\n", vd->vdev_tsd); + /* Rotational optimizations only make sense on block devices */ + vd->vdev_nonrot = B_TRUE; + + /* + * Allow TRIM on file based vdevs. This may not always be supported, + * since it depends on your kernel version and underlying filesystem + * type but it is always safe to attempt. + */ + vd->vdev_has_trim = B_TRUE; + + /* + * Disable secure TRIM on file based vdevs. There is no way to + * request this behavior from the underlying filesystem. + */ + vd->vdev_has_securetrim = B_FALSE; + + /* + * We must have a pathname, and it must be absolute. + */ + if (vd->vdev_path == NULL || (vd->vdev_path[0] != '/' && + vd->vdev_path[0] != '\\')) { + vd->vdev_stat.vs_aux = VDEV_AUX_BAD_LABEL; + return (SET_ERROR(EINVAL)); + } + + /* + * Reopen the device if it's not currently open. Otherwise, + * just update the physical size of the device. + */ +#ifdef _KERNEL + if (vd->vdev_tsd != NULL) { + ASSERT(vd->vdev_reopening); + vf = vd->vdev_tsd; + goto skip_open; + } +#endif + + vf = vd->vdev_tsd = kmem_zalloc(sizeof (vdev_file_t), KM_SLEEP); + + /* + * We have to do the same checks as vdev_disk.c, since this is run + * in userland as zdb. Use physpath if given, with offset and size + */ + if (vd->vdev_physpath) + vdev_path = spa_strdup(vd->vdev_physpath); + else + vdev_path = spa_strdup(vd->vdev_path); + + if (vdev_path[0] == '#') { + uint8_t *end; + end = &vdev_path[0]; + while (end && end[0] == '#') end++; + ddi_strtoull(end, &end, 10, &vf->vdev_win_offset); + while (end && end[0] == '#') end++; + ddi_strtoull(end, &end, 10, &vf->vdev_win_length); + while (end && end[0] == '#') end++; + + FileName = end; + } else { + FileName = vdev_path; + + // Sometimes only vdev_path is set, with "/dev/physicaldrive" + // make it be " \??\physicaldrive" space skipped over. + if (strncmp("/dev/", FileName, 5) == 0) { + FileName[0] = ' '; + FileName[1] = '\\'; + FileName[2] = '?'; + FileName[3] = '?'; + FileName[4] = '\\'; + FileName++; + } + } + +#ifdef _KERNEL + if (strncmp("\\\\?\\", FileName, 4) == 0) { + FileName[1] = '?'; + } + if (strncmp("//./", FileName, 4) == 0) { + FileName[1] = '?'; + FileName[2] = '?'; + for (int i = 0; FileName[i] != 0; i++) + if (FileName[i] == '/') + FileName[i] = '\\'; + } +#endif + + error = zfs_file_open(FileName, + vdev_file_open_mode(spa_mode(vd->vdev_spa)), 0, &fp); + + if (error) { + vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; + if (vdev_path != NULL) + spa_strfree(vdev_path); + return (error); + } + + vf->vf_file = fp; + + /* + * Make sure it's a regular file. + */ + if (zfs_file_getattr(fp, &zfa)) { + /* + * It is possible users copied file to virtual disk, + * try that. + */ + // zfs_file_close(fp); + vdev_file_close(vd); + if (vdev_path != NULL) + spa_strfree(vdev_path); + return (SET_ERROR(ENODEV)); + } + +#ifdef _KERNEL + // Change it to SPARSE, so TRIM might work + error = ZwFsControlFile( + fp->f_handle, + NULL, + NULL, + NULL, + NULL, + FSCTL_SET_SPARSE, + NULL, + 0, + NULL, + 0); + dprintf("%s: set Sparse 0x%x.\n", __func__, error); +#else + // Userland? +#endif + +skip_open: + /* + * Determine the physical size of the file. + */ + if (vf->vdev_win_length != 0) + zfa.zfa_size = vf->vdev_win_length; + else + error = zfs_file_getattr(vf->vf_file, &zfa); + + if (error) { + vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; + if (vdev_path != NULL) + spa_strfree(vdev_path); + return (error); + } + + *max_psize = *psize = zfa.zfa_size; + *ashift = SPA_MINBLOCKSHIFT; + *physical_ashift = SPA_MINBLOCKSHIFT; + if (vdev_path != NULL) + spa_strfree(vdev_path); + return (0); +} + +static void +vdev_file_close(vdev_t *vd) +{ + vdev_file_t *vf = vd->vdev_tsd; + + if (vd->vdev_reopening || vf == NULL) + return; + + + // Release our holds + if (vf->vf_file != NULL) { + zfs_file_close(vf->vf_file); + } + + vd->vdev_delayed_close = B_FALSE; + kmem_free(vf, sizeof (vdev_file_t)); + vd->vdev_tsd = NULL; +} + +#ifdef _KERNEL_THISISNTUSEDANYMORE +struct vdev_file_callback_struct { + zio_t *zio; + PIRP irp; + void *b_data; + char work_item[0]; +}; +typedef struct vdev_file_callback_struct vf_callback_t; + +static void +vdev_file_io_start_done(void *param) +{ + vf_callback_t *vb = (vf_callback_t *)param; + + ASSERT(vb != NULL); + + NTSTATUS status = vb->irp->IoStatus.Status; + zio_t *zio = vb->zio; + zio->io_error = (!NT_SUCCESS(status) ? EIO : 0); + + // Return abd buf + if (zio->io_type == ZIO_TYPE_READ) { + abd_return_buf_copy(zio->io_abd, vb->b_data, + zio->io_size); + } else { + abd_return_buf(zio->io_abd, vb->b_data, + zio->io_size); + } + + UnlockAndFreeMdl(vb->irp->MdlAddress); + IoFreeIrp(vb->irp); + kmem_free(vb, sizeof (vf_callback_t) + IoSizeofWorkItem()); + vb = NULL; + zio_delay_interrupt(zio); +} + +static void +FileIoWkRtn(__in PVOID pDummy, + __in PVOID pWkParms) +{ + vf_callback_t *vb = (vf_callback_t *)pWkParms; + + UNREFERENCED_PARAMETER(pDummy); + IoUninitializeWorkItem((PIO_WORKITEM)vb->work_item); + vdev_file_io_start_done(vb); +} + +static NTSTATUS +vdev_file_io_intr(PDEVICE_OBJECT DeviceObject, PIRP irp, PVOID Context) +{ + vf_callback_t *vb = (vf_callback_t *)Context; + + ASSERT(vb != NULL); + + /* + * If IRQL is below DIPATCH_LEVEL then there is no issue in calling + * vdev_file_io_start_done() directly; otherwise queue a new Work Item + */ + if (KeGetCurrentIrql() < DISPATCH_LEVEL) { + vdev_file_io_start_done(vb); + } else { + vdev_file_t *vf = vb->zio->io_vd->vdev_tsd; + zfs_file_t *fp = vf->vf_file; + IoInitializeWorkItem(fp->f_deviceobject, + (PIO_WORKITEM)vb->work_item); + IoQueueWorkItem((PIO_WORKITEM)vb->work_item, FileIoWkRtn, + DelayedWorkQueue, vb); + } + + return (STATUS_MORE_PROCESSING_REQUIRED); +} +#endif + +static void +vdev_file_io_strategy(void *arg) +{ + zio_t *zio = (zio_t *)arg; + vdev_t *vd = zio->io_vd; + vdev_file_t *vf = vd->vdev_tsd; + zfs_file_t *fp = vf->vf_file; + ssize_t resid; + loff_t off; + void *data; + ssize_t size; + int err; + + off = zio->io_offset + vf->vdev_win_offset; + size = zio->io_size; + resid = 0; + + if (zio->io_type == ZIO_TYPE_READ) { + data = + abd_borrow_buf(zio->io_abd, size); + err = zfs_file_pread(vf->vf_file, data, size, off, &resid); + abd_return_buf_copy(zio->io_abd, data, size); + } else { + data = + abd_borrow_buf_copy(zio->io_abd, size); + err = zfs_file_pwrite(vf->vf_file, data, size, off, &resid); + abd_return_buf(zio->io_abd, data, size); + } + + zio->io_error = (err != 0 ? EIO : 0); + + if (zio->io_error == 0 && resid != 0) + zio->io_error = SET_ERROR(ENOSPC); + + zio_delay_interrupt(zio); +} + +static void +vdev_file_io_start(zio_t *zio) +{ + vdev_t *vd = zio->io_vd; + ssize_t resid = 0; + vdev_file_t *vf = vd->vdev_tsd; + zfs_file_t *fp = vf->vf_file; + + if (zio->io_type == ZIO_TYPE_IOCTL) { + + if (!vdev_readable(vd)) { + zio->io_error = SET_ERROR(ENXIO); + zio_interrupt(zio); + return; + } + + switch (zio->io_cmd) { + case DKIOCFLUSHWRITECACHE: + zio->io_error = zfs_file_fsync(fp, 0); + break; + default: + zio->io_error = SET_ERROR(ENOTSUP); + } + + zio_execute(zio); + return; + + } else if (zio->io_type == ZIO_TYPE_TRIM) { + int mode = 0; + zio->io_error = zfs_file_fallocate(vf->vf_file, + mode, zio->io_offset, zio->io_size); + zio_execute(zio); + return; + } + + ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE); + zio->io_target_timestamp = zio_handle_io_delay(zio); + + + ASSERT(zio->io_size != 0); + LARGE_INTEGER offset; + offset.QuadPart = zio->io_offset + vf->vdev_win_offset; + + VERIFY3U(taskq_dispatch(system_taskq, vdev_file_io_strategy, zio, + TQ_SLEEP), !=, 0); +} + + +static void +vdev_file_io_done(zio_t *zio) +{ +} + +vdev_ops_t vdev_file_ops = { + .vdev_op_init = NULL, + .vdev_op_fini = NULL, + .vdev_op_open = vdev_file_open, + .vdev_op_close = vdev_file_close, + .vdev_op_asize = vdev_default_asize, + .vdev_op_min_asize = vdev_default_min_asize, + .vdev_op_min_alloc = NULL, + .vdev_op_io_start = vdev_file_io_start, + .vdev_op_io_done = vdev_file_io_done, + .vdev_op_state_change = NULL, + .vdev_op_need_resilver = NULL, + .vdev_op_hold = vdev_file_hold, + .vdev_op_rele = vdev_file_rele, + .vdev_op_remap = NULL, + .vdev_op_xlate = vdev_default_xlate, + .vdev_op_rebuild_asize = NULL, + .vdev_op_metaslab_init = NULL, + .vdev_op_config_generate = NULL, + .vdev_op_nparity = NULL, + .vdev_op_ndisks = NULL, + .vdev_op_type = VDEV_TYPE_FILE, /* name of this vdev type */ + .vdev_op_leaf = B_TRUE /* leaf vdev */ +}; + +void +vdev_file_init(void) +{ + vdev_file_taskq = taskq_create("vdev_file_taskq", 100, minclsyspri, + max_ncpus, INT_MAX, TASKQ_PREPOPULATE | TASKQ_THREADS_CPU_PCT); + + VERIFY(vdev_file_taskq); +} + +void +vdev_file_fini(void) +{ + taskq_destroy(vdev_file_taskq); +} + +/* + * From userland we access disks just like files. + */ +#ifndef _KERNEL + +vdev_ops_t vdev_disk_ops = { + .vdev_op_init = NULL, + .vdev_op_fini = NULL, + .vdev_op_open = vdev_file_open, + .vdev_op_close = vdev_file_close, + .vdev_op_asize = vdev_default_asize, + .vdev_op_min_asize = vdev_default_min_asize, + .vdev_op_min_alloc = NULL, + .vdev_op_io_start = vdev_file_io_start, + .vdev_op_io_done = vdev_file_io_done, + .vdev_op_state_change = NULL, + .vdev_op_need_resilver = NULL, + .vdev_op_hold = vdev_file_hold, + .vdev_op_rele = vdev_file_rele, + .vdev_op_remap = NULL, + .vdev_op_xlate = vdev_default_xlate, + .vdev_op_rebuild_asize = NULL, + .vdev_op_metaslab_init = NULL, + .vdev_op_config_generate = NULL, + .vdev_op_nparity = NULL, + .vdev_op_ndisks = NULL, + .vdev_op_type = VDEV_TYPE_DISK, /* name of this vdev type */ + .vdev_op_leaf = B_TRUE /* leaf vdev */ +}; +#endif diff --git a/module/os/windows/zfs/zfs_acl.c b/module/os/windows/zfs/zfs_acl.c new file mode 100644 index 000000000000..7382c26d07e5 --- /dev/null +++ b/module/os/windows/zfs/zfs_acl.c @@ -0,0 +1,3007 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. All rights reserved. + * Portions Copyright 2022 Andrew Innes + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define ALLOW ACE_ACCESS_ALLOWED_ACE_TYPE +#define DENY ACE_ACCESS_DENIED_ACE_TYPE +#define MAX_ACE_TYPE ACE_SYSTEM_ALARM_CALLBACK_OBJECT_ACE_TYPE +#define MIN_ACE_TYPE ALLOW + +#define OWNING_GROUP (ACE_GROUP|ACE_IDENTIFIER_GROUP) +#define EVERYONE_ALLOW_MASK (ACE_READ_ACL|ACE_READ_ATTRIBUTES | \ + ACE_READ_NAMED_ATTRS|ACE_SYNCHRONIZE) +#define EVERYONE_DENY_MASK (ACE_WRITE_ACL|ACE_WRITE_OWNER | \ + ACE_WRITE_ATTRIBUTES|ACE_WRITE_NAMED_ATTRS) +#define OWNER_ALLOW_MASK (ACE_WRITE_ACL | ACE_WRITE_OWNER | \ + ACE_WRITE_ATTRIBUTES|ACE_WRITE_NAMED_ATTRS) + +#define ZFS_CHECKED_MASKS (ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_DATA| \ + ACE_READ_NAMED_ATTRS|ACE_WRITE_DATA|ACE_WRITE_ATTRIBUTES| \ + ACE_WRITE_NAMED_ATTRS|ACE_APPEND_DATA|ACE_EXECUTE|ACE_WRITE_OWNER| \ + ACE_WRITE_ACL|ACE_DELETE|ACE_DELETE_CHILD|ACE_SYNCHRONIZE) + +#define WRITE_MASK_DATA (ACE_WRITE_DATA|ACE_APPEND_DATA|ACE_WRITE_NAMED_ATTRS) +#define WRITE_MASK_ATTRS (ACE_WRITE_ACL|ACE_WRITE_OWNER|ACE_WRITE_ATTRIBUTES| \ + ACE_DELETE|ACE_DELETE_CHILD) +#define WRITE_MASK (WRITE_MASK_DATA|WRITE_MASK_ATTRS) + +#define OGE_CLEAR (ACE_READ_DATA|ACE_LIST_DIRECTORY|ACE_WRITE_DATA| \ + ACE_ADD_FILE|ACE_APPEND_DATA|ACE_ADD_SUBDIRECTORY|ACE_EXECUTE) + +#define OKAY_MASK_BITS (ACE_READ_DATA|ACE_LIST_DIRECTORY|ACE_WRITE_DATA| \ + ACE_ADD_FILE|ACE_APPEND_DATA|ACE_ADD_SUBDIRECTORY|ACE_EXECUTE) + +#define ALL_INHERIT (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE | \ + ACE_NO_PROPAGATE_INHERIT_ACE|ACE_INHERIT_ONLY_ACE|ACE_INHERITED_ACE) + +#define RESTRICTED_CLEAR (ACE_WRITE_ACL|ACE_WRITE_OWNER) + +#define V4_ACL_WIDE_FLAGS (ZFS_ACL_AUTO_INHERIT|ZFS_ACL_DEFAULTED|\ + ZFS_ACL_PROTECTED) + +#define ZFS_ACL_WIDE_FLAGS (V4_ACL_WIDE_FLAGS|ZFS_ACL_TRIVIAL|ZFS_INHERIT_ACE|\ + ZFS_ACL_OBJ_ACE) + +#define ALL_MODE_EXECS (S_IXUSR | S_IXGRP | S_IXOTH) + +static uint16_t +zfs_ace_v0_get_type(void *acep) +{ + return (((zfs_oldace_t *)acep)->z_type); +} + +static uint16_t +zfs_ace_v0_get_flags(void *acep) +{ + return (((zfs_oldace_t *)acep)->z_flags); +} + +static uint32_t +zfs_ace_v0_get_mask(void *acep) +{ + return (((zfs_oldace_t *)acep)->z_access_mask); +} + +static uint64_t +zfs_ace_v0_get_who(void *acep) +{ + return (((zfs_oldace_t *)acep)->z_fuid); +} + +static void +zfs_ace_v0_set_type(void *acep, uint16_t type) +{ + ((zfs_oldace_t *)acep)->z_type = type; +} + +static void +zfs_ace_v0_set_flags(void *acep, uint16_t flags) +{ + ((zfs_oldace_t *)acep)->z_flags = flags; +} + +static void +zfs_ace_v0_set_mask(void *acep, uint32_t mask) +{ + ((zfs_oldace_t *)acep)->z_access_mask = mask; +} + +static void +zfs_ace_v0_set_who(void *acep, uint64_t who) +{ + ((zfs_oldace_t *)acep)->z_fuid = who; +} + +static size_t +zfs_ace_v0_size(void *acep) +{ + (void) acep; + return (sizeof (zfs_oldace_t)); +} + +static size_t +zfs_ace_v0_abstract_size(void) +{ + return (sizeof (zfs_oldace_t)); +} + +static int +zfs_ace_v0_mask_off(void) +{ + return (offsetof(zfs_oldace_t, z_access_mask)); +} + +static int +zfs_ace_v0_data(void *acep, void **datap) +{ + (void) acep; + *datap = NULL; + return (0); +} + +static acl_ops_t zfs_acl_v0_ops = { + zfs_ace_v0_get_mask, + zfs_ace_v0_set_mask, + zfs_ace_v0_get_flags, + zfs_ace_v0_set_flags, + zfs_ace_v0_get_type, + zfs_ace_v0_set_type, + zfs_ace_v0_get_who, + zfs_ace_v0_set_who, + zfs_ace_v0_size, + zfs_ace_v0_abstract_size, + zfs_ace_v0_mask_off, + zfs_ace_v0_data +}; + +static uint16_t +zfs_ace_fuid_get_type(void *acep) +{ + return (((zfs_ace_hdr_t *)acep)->z_type); +} + +static uint16_t +zfs_ace_fuid_get_flags(void *acep) +{ + return (((zfs_ace_hdr_t *)acep)->z_flags); +} + +static uint32_t +zfs_ace_fuid_get_mask(void *acep) +{ + return (((zfs_ace_hdr_t *)acep)->z_access_mask); +} + +static uint64_t +zfs_ace_fuid_get_who(void *args) +{ + uint16_t entry_type; + zfs_ace_t *acep = args; + + entry_type = acep->z_hdr.z_flags & ACE_TYPE_FLAGS; + + if (entry_type == ACE_OWNER || entry_type == OWNING_GROUP || + entry_type == ACE_EVERYONE) + return (-1); + return (((zfs_ace_t *)acep)->z_fuid); +} + +static void +zfs_ace_fuid_set_type(void *acep, uint16_t type) +{ + ((zfs_ace_hdr_t *)acep)->z_type = type; +} + +static void +zfs_ace_fuid_set_flags(void *acep, uint16_t flags) +{ + ((zfs_ace_hdr_t *)acep)->z_flags = flags; +} + +static void +zfs_ace_fuid_set_mask(void *acep, uint32_t mask) +{ + ((zfs_ace_hdr_t *)acep)->z_access_mask = mask; +} + +static void +zfs_ace_fuid_set_who(void *arg, uint64_t who) +{ + zfs_ace_t *acep = arg; + + uint16_t entry_type = acep->z_hdr.z_flags & ACE_TYPE_FLAGS; + + if (entry_type == ACE_OWNER || entry_type == OWNING_GROUP || + entry_type == ACE_EVERYONE) + return; + acep->z_fuid = who; +} + +static size_t +zfs_ace_fuid_size(void *acep) +{ + zfs_ace_hdr_t *zacep = acep; + uint16_t entry_type; + + switch (zacep->z_type) { + case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: + case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: + case ACE_SYSTEM_AUDIT_OBJECT_ACE_TYPE: + case ACE_SYSTEM_ALARM_OBJECT_ACE_TYPE: + return (sizeof (zfs_object_ace_t)); + case ALLOW: + case DENY: + entry_type = + (((zfs_ace_hdr_t *)acep)->z_flags & ACE_TYPE_FLAGS); + if (entry_type == ACE_OWNER || + entry_type == OWNING_GROUP || + entry_type == ACE_EVERYONE) + return (sizeof (zfs_ace_hdr_t)); + /*FALLTHROUGH*/ + default: + return (sizeof (zfs_ace_t)); + } +} + +static size_t +zfs_ace_fuid_abstract_size(void) +{ + return (sizeof (zfs_ace_hdr_t)); +} + +static int +zfs_ace_fuid_mask_off(void) +{ + return (offsetof(zfs_ace_hdr_t, z_access_mask)); +} + +static int +zfs_ace_fuid_data(void *acep, void **datap) +{ + zfs_ace_t *zacep = acep; + zfs_object_ace_t *zobjp; + + switch (zacep->z_hdr.z_type) { + case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: + case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: + case ACE_SYSTEM_AUDIT_OBJECT_ACE_TYPE: + case ACE_SYSTEM_ALARM_OBJECT_ACE_TYPE: + zobjp = acep; + *datap = (caddr_t)zobjp + sizeof (zfs_ace_t); + return (sizeof (zfs_object_ace_t) - sizeof (zfs_ace_t)); + default: + *datap = NULL; + return (0); + } +} + +static acl_ops_t zfs_acl_fuid_ops = { + zfs_ace_fuid_get_mask, + zfs_ace_fuid_set_mask, + zfs_ace_fuid_get_flags, + zfs_ace_fuid_set_flags, + zfs_ace_fuid_get_type, + zfs_ace_fuid_set_type, + zfs_ace_fuid_get_who, + zfs_ace_fuid_set_who, + zfs_ace_fuid_size, + zfs_ace_fuid_abstract_size, + zfs_ace_fuid_mask_off, + zfs_ace_fuid_data +}; + +/* + * The following three functions are provided for compatibility with + * older ZPL version in order to determine if the file use to have + * an external ACL and what version of ACL previously existed on the + * file. Would really be nice to not need this, sigh. + */ +uint64_t +zfs_external_acl(znode_t *zp) +{ + zfs_acl_phys_t acl_phys; + int error; + + if (zp->z_is_sa) + return (0); + + /* + * Need to deal with a potential + * race where zfs_sa_upgrade could cause + * z_isa_sa to change. + * + * If the lookup fails then the state of z_is_sa should have + * changed. + */ + + if ((error = sa_lookup(zp->z_sa_hdl, SA_ZPL_ZNODE_ACL(zp->z_zfsvfs), + &acl_phys, sizeof (acl_phys))) == 0) + return (acl_phys.z_acl_extern_obj); + else { + /* + * after upgrade the SA_ZPL_ZNODE_ACL should have been + * removed + */ + VERIFY(zp->z_is_sa && error == ENOENT); + return (0); + } +} + +/* + * Determine size of ACL in bytes + * + * This is more complicated than it should be since we have to deal + * with old external ACLs. + */ +static int +zfs_acl_znode_info(znode_t *zp, int *aclsize, int *aclcount, + zfs_acl_phys_t *aclphys) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + uint64_t acl_count; + int size; + int error; + + ASSERT(MUTEX_HELD(&zp->z_acl_lock)); + if (zp->z_is_sa) { + if ((error = sa_size(zp->z_sa_hdl, SA_ZPL_DACL_ACES(zfsvfs), + &size)) != 0) + return (error); + *aclsize = size; + if ((error = sa_lookup(zp->z_sa_hdl, SA_ZPL_DACL_COUNT(zfsvfs), + &acl_count, sizeof (acl_count))) != 0) + return (error); + *aclcount = acl_count; + } else { + if ((error = sa_lookup(zp->z_sa_hdl, SA_ZPL_ZNODE_ACL(zfsvfs), + aclphys, sizeof (*aclphys))) != 0) + return (error); + + if (aclphys->z_acl_version == ZFS_ACL_VERSION_INITIAL) { + *aclsize = ZFS_ACL_SIZE(aclphys->z_acl_size); + *aclcount = aclphys->z_acl_size; + } else { + *aclsize = aclphys->z_acl_size; + *aclcount = aclphys->z_acl_count; + } + } + return (0); +} + +int +zfs_znode_acl_version(znode_t *zp) +{ + zfs_acl_phys_t acl_phys; + + if (zp->z_is_sa) + return (ZFS_ACL_VERSION_FUID); + else { + int error; + + /* + * Need to deal with a potential + * race where zfs_sa_upgrade could cause + * z_isa_sa to change. + * + * If the lookup fails then the state of z_is_sa should have + * changed. + */ + if ((error = sa_lookup(zp->z_sa_hdl, + SA_ZPL_ZNODE_ACL(zp->z_zfsvfs), + &acl_phys, sizeof (acl_phys))) == 0) + return (acl_phys.z_acl_version); + else { + /* + * After upgrade SA_ZPL_ZNODE_ACL should have + * been removed. + */ + VERIFY(zp->z_is_sa && error == ENOENT); + return (ZFS_ACL_VERSION_FUID); + } + } +} + +static int +zfs_acl_version(int version) +{ + if (version < ZPL_VERSION_FUID) + return (ZFS_ACL_VERSION_INITIAL); + else + return (ZFS_ACL_VERSION_FUID); +} + +static int +zfs_acl_version_zp(znode_t *zp) +{ + return (zfs_acl_version(zp->z_zfsvfs->z_version)); +} + +zfs_acl_t * +zfs_acl_alloc(int vers) +{ + zfs_acl_t *aclp; + + aclp = kmem_zalloc(sizeof (zfs_acl_t), KM_SLEEP); + list_create(&aclp->z_acl, sizeof (zfs_acl_node_t), + offsetof(zfs_acl_node_t, z_next)); + aclp->z_version = vers; + if (vers == ZFS_ACL_VERSION_FUID) + aclp->z_ops = &zfs_acl_fuid_ops; + else + aclp->z_ops = &zfs_acl_v0_ops; + return (aclp); +} + +zfs_acl_node_t * +zfs_acl_node_alloc(size_t bytes) +{ + zfs_acl_node_t *aclnode; + + aclnode = kmem_zalloc(sizeof (zfs_acl_node_t), KM_SLEEP); + if (bytes) { + aclnode->z_acldata = kmem_alloc(bytes, KM_SLEEP); + aclnode->z_allocdata = aclnode->z_acldata; + aclnode->z_allocsize = bytes; + aclnode->z_size = bytes; + } + + return (aclnode); +} + +static void +zfs_acl_node_free(zfs_acl_node_t *aclnode) +{ + if (aclnode->z_allocsize) + kmem_free(aclnode->z_allocdata, aclnode->z_allocsize); + kmem_free(aclnode, sizeof (zfs_acl_node_t)); +} + +static void +zfs_acl_release_nodes(zfs_acl_t *aclp) +{ + zfs_acl_node_t *aclnode; + + while ((aclnode = list_remove_head(&aclp->z_acl))) + zfs_acl_node_free(aclnode); + aclp->z_acl_count = 0; + aclp->z_acl_bytes = 0; +} + +void +zfs_acl_free(zfs_acl_t *aclp) +{ + zfs_acl_release_nodes(aclp); + list_destroy(&aclp->z_acl); + kmem_free(aclp, sizeof (zfs_acl_t)); +} + +static boolean_t +zfs_acl_valid_ace_type(uint_t type, uint_t flags) +{ + uint16_t entry_type; + + switch (type) { + case ALLOW: + case DENY: + case ACE_SYSTEM_AUDIT_ACE_TYPE: + case ACE_SYSTEM_ALARM_ACE_TYPE: + entry_type = flags & ACE_TYPE_FLAGS; + return (entry_type == ACE_OWNER || + entry_type == OWNING_GROUP || + entry_type == ACE_EVERYONE || entry_type == 0 || + entry_type == ACE_IDENTIFIER_GROUP); + default: + if (type >= MIN_ACE_TYPE && type <= MAX_ACE_TYPE) + return (B_TRUE); + } + return (B_FALSE); +} + +static boolean_t +zfs_ace_valid(umode_t obj_mode, zfs_acl_t *aclp, uint16_t type, uint16_t iflags) +{ + /* + * first check type of entry + */ + + if (!zfs_acl_valid_ace_type(type, iflags)) + return (B_FALSE); + + switch (type) { + case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: + case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: + case ACE_SYSTEM_AUDIT_OBJECT_ACE_TYPE: + case ACE_SYSTEM_ALARM_OBJECT_ACE_TYPE: + if (aclp->z_version < ZFS_ACL_VERSION_FUID) + return (B_FALSE); + aclp->z_hints |= ZFS_ACL_OBJ_ACE; + } + + /* + * next check inheritance level flags + */ + + if (S_ISDIR(obj_mode) && + (iflags & (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) + aclp->z_hints |= ZFS_INHERIT_ACE; + + if (iflags & (ACE_INHERIT_ONLY_ACE|ACE_NO_PROPAGATE_INHERIT_ACE)) { + if ((iflags & (ACE_FILE_INHERIT_ACE| + ACE_DIRECTORY_INHERIT_ACE)) == 0) + return (B_FALSE); + } + + return (B_TRUE); +} + +static void * +zfs_acl_next_ace(zfs_acl_t *aclp, void *start, uint64_t *who, + uint32_t *access_mask, uint16_t *iflags, uint16_t *type) +{ + zfs_acl_node_t *aclnode; + + ASSERT(aclp); + + if (start == NULL) { + aclnode = list_head(&aclp->z_acl); + if (aclnode == NULL) + return (NULL); + + aclp->z_next_ace = aclnode->z_acldata; + aclp->z_curr_node = aclnode; + aclnode->z_ace_idx = 0; + } + + aclnode = aclp->z_curr_node; + + if (aclnode == NULL) + return (NULL); + + if (aclnode->z_ace_idx >= aclnode->z_ace_count) { + aclnode = list_next(&aclp->z_acl, aclnode); + if (aclnode == NULL) + return (NULL); + else { + aclp->z_curr_node = aclnode; + aclnode->z_ace_idx = 0; + aclp->z_next_ace = aclnode->z_acldata; + } + } + + if (aclnode->z_ace_idx < aclnode->z_ace_count) { + void *acep = aclp->z_next_ace; + size_t ace_size; + + /* + * Make sure we don't overstep our bounds + */ + ace_size = aclp->z_ops->ace_size(acep); + + if (((caddr_t)acep + ace_size) > + ((caddr_t)aclnode->z_acldata + aclnode->z_size)) { + return (NULL); + } + + *iflags = aclp->z_ops->ace_flags_get(acep); + *type = aclp->z_ops->ace_type_get(acep); + *access_mask = aclp->z_ops->ace_mask_get(acep); + *who = aclp->z_ops->ace_who_get(acep); + aclp->z_next_ace = (caddr_t)aclp->z_next_ace + ace_size; + aclnode->z_ace_idx++; + + return ((void *)acep); + } + return (NULL); +} + +static uintptr_t +zfs_ace_walk(void *datap, uintptr_t cookie, int aclcnt, + uint16_t *flags, uint16_t *type, uint32_t *mask) +{ + (void) aclcnt; + zfs_acl_t *aclp = datap; + zfs_ace_hdr_t *acep = (zfs_ace_hdr_t *)cookie; + uint64_t who; + + acep = zfs_acl_next_ace(aclp, acep, &who, mask, + flags, type); + return ((uintptr_t)acep); +} + +#if 0 // unused function +static zfs_acl_node_t * +zfs_acl_curr_node(zfs_acl_t *aclp) +{ + ASSERT(aclp->z_curr_node); + return (aclp->z_curr_node); +} +#endif + +/* + * Copy ACE to internal ZFS format. + * While processing the ACL each ACE will be validated for correctness. + * ACE FUIDs will be created later. + */ +int +zfs_copy_ace_2_fuid(zfsvfs_t *zfsvfs, umode_t obj_mode, zfs_acl_t *aclp, + void *datap, zfs_ace_t *z_acl, uint64_t aclcnt, size_t *size, + zfs_fuid_info_t **fuidp, cred_t *cr) +{ + int i; + uint16_t entry_type; + zfs_ace_t *aceptr = z_acl; + ace_t *acep = datap; + zfs_object_ace_t *zobjacep; + ace_object_t *aceobjp; + + for (i = 0; i != aclcnt; i++) { + aceptr->z_hdr.z_access_mask = acep->a_access_mask; + aceptr->z_hdr.z_flags = acep->a_flags; + aceptr->z_hdr.z_type = acep->a_type; + entry_type = aceptr->z_hdr.z_flags & ACE_TYPE_FLAGS; + if (entry_type != ACE_OWNER && entry_type != OWNING_GROUP && + entry_type != ACE_EVERYONE) { + aceptr->z_fuid = zfs_fuid_create(zfsvfs, acep->a_who, + cr, (entry_type == 0) ? + ZFS_ACE_USER : ZFS_ACE_GROUP, fuidp); + } + + /* + * Make sure ACE is valid + */ + if (zfs_ace_valid(obj_mode, aclp, aceptr->z_hdr.z_type, + aceptr->z_hdr.z_flags) != B_TRUE) + return (SET_ERROR(EINVAL)); + + switch (acep->a_type) { + case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: + case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: + case ACE_SYSTEM_AUDIT_OBJECT_ACE_TYPE: + case ACE_SYSTEM_ALARM_OBJECT_ACE_TYPE: + zobjacep = (zfs_object_ace_t *)aceptr; + aceobjp = (ace_object_t *)acep; + + memcpy(zobjacep->z_object_type, aceobjp->a_obj_type, + sizeof (aceobjp->a_obj_type)); + memcpy(zobjacep->z_inherit_type, + aceobjp->a_inherit_obj_type, + sizeof (aceobjp->a_inherit_obj_type)); + acep = (ace_t *)((caddr_t)acep + sizeof (ace_object_t)); + break; + default: + acep = (ace_t *)((caddr_t)acep + sizeof (ace_t)); + } + + aceptr = (zfs_ace_t *)((caddr_t)aceptr + + aclp->z_ops->ace_size(aceptr)); + } + + *size = (caddr_t)aceptr - (caddr_t)z_acl; + + return (0); +} + +/* + * Copy ZFS ACEs to fixed size ace_t layout + */ +#if 0 // unused function +static void +zfs_copy_fuid_2_ace(zfsvfs_t *zfsvfs, zfs_acl_t *aclp, cred_t *cr, + void *datap, int filter) +{ + uint64_t who; + uint32_t access_mask; + uint16_t iflags, type; + zfs_ace_hdr_t *zacep = NULL; + ace_t *acep = datap; + ace_object_t *objacep; + zfs_object_ace_t *zobjacep; + size_t ace_size; + uint16_t entry_type; + + while ((zacep = zfs_acl_next_ace(aclp, zacep, + &who, &access_mask, &iflags, &type))) { + + switch (type) { + case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: + case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: + case ACE_SYSTEM_AUDIT_OBJECT_ACE_TYPE: + case ACE_SYSTEM_ALARM_OBJECT_ACE_TYPE: + if (filter) { + continue; + } + zobjacep = (zfs_object_ace_t *)zacep; + objacep = (ace_object_t *)acep; + memcpy(objacep->a_obj_type, + zobjacep->z_object_type, + sizeof (zobjacep->z_object_type)); + memcpy(objacep->a_inherit_obj_type, + zobjacep->z_inherit_type, + sizeof (zobjacep->z_inherit_type)); + ace_size = sizeof (ace_object_t); + break; + default: + ace_size = sizeof (ace_t); + break; + } + + entry_type = (iflags & ACE_TYPE_FLAGS); + if ((entry_type != ACE_OWNER && + entry_type != OWNING_GROUP && + entry_type != ACE_EVERYONE)) { + acep->a_who = zfs_fuid_map_id(zfsvfs, who, + cr, (entry_type & ACE_IDENTIFIER_GROUP) ? + ZFS_ACE_GROUP : ZFS_ACE_USER); + } else { + acep->a_who = (uid_t)(int64_t)who; + } + acep->a_access_mask = access_mask; + acep->a_flags = iflags; + acep->a_type = type; + acep = (ace_t *)((caddr_t)acep + ace_size); + } +} +#endif + +static int +zfs_copy_ace_2_oldace(umode_t obj_mode, zfs_acl_t *aclp, ace_t *acep, + zfs_oldace_t *z_acl, int aclcnt, size_t *size) +{ + int i; + zfs_oldace_t *aceptr = z_acl; + + for (i = 0; i != aclcnt; i++, aceptr++) { + aceptr->z_access_mask = acep[i].a_access_mask; + aceptr->z_type = acep[i].a_type; + aceptr->z_flags = acep[i].a_flags; + aceptr->z_fuid = acep[i].a_who; + /* + * Make sure ACE is valid + */ + if (zfs_ace_valid(obj_mode, aclp, aceptr->z_type, + aceptr->z_flags) != B_TRUE) + return (SET_ERROR(EINVAL)); + } + *size = (caddr_t)aceptr - (caddr_t)z_acl; + return (0); +} + +/* + * convert old ACL format to new + */ +void +zfs_acl_xform(znode_t *zp, zfs_acl_t *aclp, cred_t *cr) +{ + zfs_oldace_t *oldaclp; + int i; + uint16_t type, iflags; + uint32_t access_mask; + uint64_t who; + void *cookie = NULL; + zfs_acl_node_t *newaclnode; + + ASSERT(aclp->z_version == ZFS_ACL_VERSION_INITIAL); + /* + * First create the ACE in a contiguous piece of memory + * for zfs_copy_ace_2_fuid(). + * + * We only convert an ACL once, so this won't happen + * everytime. + */ + oldaclp = kmem_alloc(sizeof (zfs_oldace_t) * aclp->z_acl_count, + KM_SLEEP); + i = 0; + while ((cookie = zfs_acl_next_ace(aclp, cookie, &who, + &access_mask, &iflags, &type))) { + oldaclp[i].z_flags = iflags; + oldaclp[i].z_type = type; + oldaclp[i].z_fuid = who; + oldaclp[i++].z_access_mask = access_mask; + } + + newaclnode = zfs_acl_node_alloc(aclp->z_acl_count * + sizeof (zfs_object_ace_t)); + aclp->z_ops = &zfs_acl_fuid_ops; + VERIFY(zfs_copy_ace_2_fuid(zp->z_zfsvfs, zp->z_mode, aclp, + oldaclp, newaclnode->z_acldata, aclp->z_acl_count, + &newaclnode->z_size, NULL, cr) == 0); + newaclnode->z_ace_count = aclp->z_acl_count; + aclp->z_version = ZFS_ACL_VERSION; + kmem_free(oldaclp, aclp->z_acl_count * sizeof (zfs_oldace_t)); + + /* + * Release all previous ACL nodes + */ + + zfs_acl_release_nodes(aclp); + + list_insert_head(&aclp->z_acl, newaclnode); + + aclp->z_acl_bytes = newaclnode->z_size; + aclp->z_acl_count = newaclnode->z_ace_count; + +} + +/* + * Convert unix access mask to v4 access mask + */ +static uint32_t +zfs_unix_to_v4(uint32_t access_mask) +{ + uint32_t new_mask = 0; + + if (access_mask & S_IXOTH) + new_mask |= ACE_EXECUTE; + if (access_mask & S_IWOTH) + new_mask |= ACE_WRITE_DATA; + if (access_mask & S_IROTH) + new_mask |= ACE_READ_DATA; + return (new_mask); +} + +static void +zfs_set_ace(zfs_acl_t *aclp, void *acep, uint32_t access_mask, + uint16_t access_type, uint64_t fuid, uint16_t entry_type) +{ + uint16_t type = entry_type & ACE_TYPE_FLAGS; + + aclp->z_ops->ace_mask_set(acep, access_mask); + aclp->z_ops->ace_type_set(acep, access_type); + aclp->z_ops->ace_flags_set(acep, entry_type); + if ((type != ACE_OWNER && type != OWNING_GROUP && + type != ACE_EVERYONE)) + aclp->z_ops->ace_who_set(acep, fuid); +} + +/* + * Determine mode of file based on ACL. + */ +uint64_t +zfs_mode_compute(uint64_t fmode, zfs_acl_t *aclp, + uint64_t *pflags, uint64_t fuid, uint64_t fgid) +{ + int entry_type; + mode_t mode; + mode_t seen = 0; + zfs_ace_hdr_t *acep = NULL; + uint64_t who; + uint16_t iflags, type; + uint32_t access_mask; + boolean_t an_exec_denied = B_FALSE; + + + mode = (fmode & (S_IFMT | S_ISUID | S_ISGID | S_ISVTX)); + + while ((acep = zfs_acl_next_ace(aclp, acep, &who, + &access_mask, &iflags, &type))) { + + if (!zfs_acl_valid_ace_type(type, iflags)) + continue; + + entry_type = (iflags & ACE_TYPE_FLAGS); + + /* + * Skip over any inherit_only ACEs + */ + if (iflags & ACE_INHERIT_ONLY_ACE) + continue; + + + /* + * Apple has unusual expectations to emulate hfs in that the mode is not + * updated: + * -rw-r--r-- 1 root wheel 0 Nov 12 12:39 file.txt + * chmod +a "root allow execute" file.txt + * ZFS: -rwxr--r--+ 1 root wheel 0 Nov 12 12:39 file.txt + * HFS: -rw-r--r--+ 1 root wheel 0 Nov 12 12:39 file.txt + * 0: user:root allow execute + */ +#ifndef _WIN32 + if (entry_type == ACE_OWNER || + (entry_type == 0 && who == fuid)) { +#else + if (entry_type == ACE_OWNER) { +#endif + if ((access_mask & ACE_READ_DATA) && + (!(seen & S_IRUSR))) { + seen |= S_IRUSR; + if (type == ALLOW) { + mode |= S_IRUSR; + } + } + if ((access_mask & ACE_WRITE_DATA) && + (!(seen & S_IWUSR))) { + seen |= S_IWUSR; + if (type == ALLOW) { + mode |= S_IWUSR; + } + } + if ((access_mask & ACE_EXECUTE) && + (!(seen & S_IXUSR))) { + seen |= S_IXUSR; + if (type == ALLOW) { + mode |= S_IXUSR; + } + } +#ifndef _WIN32 + } else if (entry_type == OWNING_GROUP || + (entry_type == ACE_IDENTIFIER_GROUP && who == fgid)) { +#else + } else if (entry_type == OWNING_GROUP) { + +#endif + if ((access_mask & ACE_READ_DATA) && + (!(seen & S_IRGRP))) { + seen |= S_IRGRP; + if (type == ALLOW) { + mode |= S_IRGRP; + } + } + if ((access_mask & ACE_WRITE_DATA) && + (!(seen & S_IWGRP))) { + seen |= S_IWGRP; + if (type == ALLOW) { + mode |= S_IWGRP; + } + } + if ((access_mask & ACE_EXECUTE) && + (!(seen & S_IXGRP))) { + seen |= S_IXGRP; + if (type == ALLOW) { + mode |= S_IXGRP; + } + } + } else if (entry_type == ACE_EVERYONE) { + if ((access_mask & ACE_READ_DATA)) { + if (!(seen & S_IRUSR)) { + seen |= S_IRUSR; + if (type == ALLOW) { + mode |= S_IRUSR; + } + } + if (!(seen & S_IRGRP)) { + seen |= S_IRGRP; + if (type == ALLOW) { + mode |= S_IRGRP; + } + } + if (!(seen & S_IROTH)) { + seen |= S_IROTH; + if (type == ALLOW) { + mode |= S_IROTH; + } + } + } + if ((access_mask & ACE_WRITE_DATA)) { + if (!(seen & S_IWUSR)) { + seen |= S_IWUSR; + if (type == ALLOW) { + mode |= S_IWUSR; + } + } + if (!(seen & S_IWGRP)) { + seen |= S_IWGRP; + if (type == ALLOW) { + mode |= S_IWGRP; + } + } + if (!(seen & S_IWOTH)) { + seen |= S_IWOTH; + if (type == ALLOW) { + mode |= S_IWOTH; + } + } + } + if ((access_mask & ACE_EXECUTE)) { + if (!(seen & S_IXUSR)) { + seen |= S_IXUSR; + if (type == ALLOW) { + mode |= S_IXUSR; + } + } + if (!(seen & S_IXGRP)) { + seen |= S_IXGRP; + if (type == ALLOW) { + mode |= S_IXGRP; + } + } + if (!(seen & S_IXOTH)) { + seen |= S_IXOTH; + if (type == ALLOW) { + mode |= S_IXOTH; + } + } + } + } else { + /* + * Only care if this IDENTIFIER_GROUP or + * USER ACE denies execute access to someone, + * mode is not affected + */ + if ((access_mask & ACE_EXECUTE) && type == DENY) + an_exec_denied = B_TRUE; + } + } + + /* + * Failure to allow is effectively a deny, so execute permission + * is denied if it was never mentioned or if we explicitly + * weren't allowed it. + */ + if (!an_exec_denied && + ((seen & ALL_MODE_EXECS) != ALL_MODE_EXECS || + (mode & ALL_MODE_EXECS) != ALL_MODE_EXECS)) + an_exec_denied = B_TRUE; + + if (an_exec_denied) + *pflags &= ~ZFS_NO_EXECS_DENIED; + else + *pflags |= ZFS_NO_EXECS_DENIED; + + return (mode); +} + +/* + * Read an external acl object. If the intent is to modify, always + * create a new acl and leave any cached acl in place. + */ +static int +zfs_acl_node_read(znode_t *zp, boolean_t have_lock, zfs_acl_t **aclpp, + boolean_t will_modify) +{ + zfs_acl_t *aclp; + int aclsize = 0; + int acl_count = 0; + zfs_acl_node_t *aclnode; + zfs_acl_phys_t znode_acl; + int version; + int error; + boolean_t drop_lock = B_FALSE; + + ASSERT(MUTEX_HELD(&zp->z_acl_lock)); + + if (zp->z_acl_cached && !will_modify) { + *aclpp = zp->z_acl_cached; + return (0); + } + + /* + * close race where znode could be upgrade while trying to + * read the znode attributes. + * + * But this could only happen if the file isn't already an SA + * znode + */ + if (!zp->z_is_sa && !have_lock) { + mutex_enter(&zp->z_lock); + drop_lock = B_TRUE; + } + version = zfs_znode_acl_version(zp); + + if ((error = zfs_acl_znode_info(zp, &aclsize, + &acl_count, &znode_acl)) != 0) { + goto done; + } + + aclp = zfs_acl_alloc(version); + + aclp->z_acl_count = acl_count; + aclp->z_acl_bytes = aclsize; + + aclnode = zfs_acl_node_alloc(aclsize); + aclnode->z_ace_count = aclp->z_acl_count; + aclnode->z_size = aclsize; + + if (!zp->z_is_sa) { + if (znode_acl.z_acl_extern_obj) { + error = dmu_read(zp->z_zfsvfs->z_os, + znode_acl.z_acl_extern_obj, 0, aclnode->z_size, + aclnode->z_acldata, DMU_READ_PREFETCH); + } else { + memcpy(aclnode->z_acldata, znode_acl.z_ace_data, + aclnode->z_size); + } + } else { + error = sa_lookup(zp->z_sa_hdl, SA_ZPL_DACL_ACES(zp->z_zfsvfs), + aclnode->z_acldata, aclnode->z_size); + } + + if (error != 0) { + zfs_acl_free(aclp); + zfs_acl_node_free(aclnode); + /* convert checksum errors into IO errors */ + if (error == ECKSUM) + error = SET_ERROR(EIO); + goto done; + } + + list_insert_head(&aclp->z_acl, aclnode); + + *aclpp = aclp; + if (!will_modify) + zp->z_acl_cached = aclp; +done: + if (drop_lock) + mutex_exit(&zp->z_lock); + return (error); +} + +void +zfs_acl_data_locator(void **dataptr, uint32_t *length, uint32_t buflen, + boolean_t start, void *userdata) +{ + (void) buflen; + zfs_acl_locator_cb_t *cb = (zfs_acl_locator_cb_t *)userdata; + + if (start) { + cb->cb_acl_node = list_head(&cb->cb_aclp->z_acl); + } else { + cb->cb_acl_node = list_next(&cb->cb_aclp->z_acl, + cb->cb_acl_node); + } + ASSERT3P(cb->cb_acl_node, !=, NULL); + *dataptr = cb->cb_acl_node->z_acldata; + *length = cb->cb_acl_node->z_size; +} + +int +zfs_acl_chown_setattr(znode_t *zp) +{ + int error; + zfs_acl_t *aclp; + + if (zp->z_zfsvfs->z_acl_mode == ZFS_ACLTYPE_POSIX) + return (0); + + ASSERT(MUTEX_HELD(&zp->z_lock)); + ASSERT(MUTEX_HELD(&zp->z_acl_lock)); + + error = zfs_acl_node_read(zp, B_TRUE, &aclp, B_FALSE); + if (error == 0 && aclp->z_acl_count > 0) + zp->z_mode = zfs_mode_compute(zp->z_mode, aclp, + &zp->z_pflags, zp->z_uid, zp->z_gid); + + /* + * Some ZFS implementations (ZEVO) create neither a ZNODE_ACL + * nor a DACL_ACES SA in which case ENOENT is returned from + * zfs_acl_node_read() when the SA can't be located. + * Allow chown/chgrp to succeed in these cases rather than + * returning an error that makes no sense in the context of + * the caller. + */ + if (error == ENOENT) + return (0); + + return (error); +} + +/* + * common code for setting ACLs. + * + * This function is called from zfs_mode_update, zfs_perm_init, and zfs_setacl. + * zfs_setacl passes a non-NULL inherit pointer (ihp) to indicate that it's + * already checked the acl and knows whether to inherit. + */ +int +zfs_aclset_common(znode_t *zp, zfs_acl_t *aclp, cred_t *cr, dmu_tx_t *tx) +{ + int error; + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + dmu_object_type_t otype; + zfs_acl_locator_cb_t locate = { 0 }; + uint64_t mode; + sa_bulk_attr_t bulk[5]; + uint64_t ctime[2]; + int count = 0; + zfs_acl_phys_t acl_phys; + + mode = zp->z_mode; + + mode = zfs_mode_compute(mode, aclp, &zp->z_pflags, + zp->z_uid, zp->z_gid); + + zp->z_mode = mode; + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MODE(zfsvfs), NULL, + &mode, sizeof (mode)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), NULL, + &zp->z_pflags, sizeof (zp->z_pflags)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), NULL, + &ctime, sizeof (ctime)); + + if (zp->z_acl_cached) { + zfs_acl_free(zp->z_acl_cached); + zp->z_acl_cached = NULL; + } + + /* + * Upgrade needed? + */ + if (!zfsvfs->z_use_fuids) { + otype = DMU_OT_OLDACL; + } else { + if ((aclp->z_version == ZFS_ACL_VERSION_INITIAL) && + (zfsvfs->z_version >= ZPL_VERSION_FUID)) + zfs_acl_xform(zp, aclp, cr); + ASSERT(aclp->z_version >= ZFS_ACL_VERSION_FUID); + otype = DMU_OT_ACL; + } + + /* + * Arrgh, we have to handle old on disk format + * as well as newer (preferred) SA format. + */ + + if (zp->z_is_sa) { /* the easy case, just update the ACL attribute */ + locate.cb_aclp = aclp; + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_DACL_ACES(zfsvfs), + zfs_acl_data_locator, &locate, aclp->z_acl_bytes); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_DACL_COUNT(zfsvfs), + NULL, &aclp->z_acl_count, sizeof (uint64_t)); + } else { /* Painful legacy way */ + zfs_acl_node_t *aclnode; + uint64_t off = 0; + uint64_t aoid; + + if ((error = sa_lookup(zp->z_sa_hdl, SA_ZPL_ZNODE_ACL(zfsvfs), + &acl_phys, sizeof (acl_phys))) != 0) + return (error); + + aoid = acl_phys.z_acl_extern_obj; + + if (aclp->z_acl_bytes > ZFS_ACE_SPACE) { + /* + * If ACL was previously external and we are now + * converting to new ACL format then release old + * ACL object and create a new one. + */ + if (aoid && + aclp->z_version != acl_phys.z_acl_version) { + error = dmu_object_free(zfsvfs->z_os, aoid, tx); + if (error) + return (error); + aoid = 0; + } + if (aoid == 0) { + aoid = dmu_object_alloc(zfsvfs->z_os, + otype, aclp->z_acl_bytes, + otype == DMU_OT_ACL ? + DMU_OT_SYSACL : DMU_OT_NONE, + otype == DMU_OT_ACL ? + DN_OLD_MAX_BONUSLEN : 0, tx); + } else { + (void) dmu_object_set_blocksize(zfsvfs->z_os, + aoid, aclp->z_acl_bytes, 0, tx); + } + acl_phys.z_acl_extern_obj = aoid; + for (aclnode = list_head(&aclp->z_acl); aclnode; + aclnode = list_next(&aclp->z_acl, aclnode)) { + if (aclnode->z_ace_count == 0) + continue; + dmu_write(zfsvfs->z_os, aoid, off, + aclnode->z_size, aclnode->z_acldata, tx); + off += aclnode->z_size; + } + } else { + void *start = acl_phys.z_ace_data; + /* + * Migrating back embedded? + */ + if (acl_phys.z_acl_extern_obj) { + error = dmu_object_free(zfsvfs->z_os, + acl_phys.z_acl_extern_obj, tx); + if (error) + return (error); + acl_phys.z_acl_extern_obj = 0; + } + + for (aclnode = list_head(&aclp->z_acl); aclnode; + aclnode = list_next(&aclp->z_acl, aclnode)) { + if (aclnode->z_ace_count == 0) + continue; + memcpy(start, aclnode->z_acldata, + aclnode->z_size); + start = (caddr_t)start + aclnode->z_size; + } + } + /* + * If Old version then swap count/bytes to match old + * layout of znode_acl_phys_t. + */ + if (aclp->z_version == ZFS_ACL_VERSION_INITIAL) { + acl_phys.z_acl_size = aclp->z_acl_count; + acl_phys.z_acl_count = aclp->z_acl_bytes; + } else { + acl_phys.z_acl_size = aclp->z_acl_bytes; + acl_phys.z_acl_count = aclp->z_acl_count; + } + acl_phys.z_acl_version = aclp->z_version; + + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_ZNODE_ACL(zfsvfs), NULL, + &acl_phys, sizeof (acl_phys)); + } + + /* + * Replace ACL wide bits, but first clear them. + */ + zp->z_pflags &= ~ZFS_ACL_WIDE_FLAGS; + + zp->z_pflags |= aclp->z_hints; + + if (ace_trivial_common(aclp, 0, zfs_ace_walk) == 0) + zp->z_pflags |= ZFS_ACL_TRIVIAL; + +// zfs_tstamp_update_setup(zp, STATE_CHANGED, NULL, ctime, B_TRUE); + return (sa_bulk_update(zp->z_sa_hdl, bulk, count, tx)); +} + +static void +zfs_acl_chmod(umode_t umode, uint64_t mode, boolean_t split, boolean_t trim, + zfs_acl_t *aclp) +{ + void *acep = NULL; + uint64_t who; + int new_count, new_bytes; + int ace_size; + int entry_type; + uint16_t iflags, type; + uint32_t access_mask; + zfs_acl_node_t *newnode; + size_t abstract_size = aclp->z_ops->ace_abstract_size(); + void *zacep; + boolean_t isdir; + trivial_acl_t masks; + + new_count = new_bytes = 0; + + isdir = S_ISDIR(umode); + + acl_trivial_access_masks((mode_t)mode, isdir, &masks); + + newnode = zfs_acl_node_alloc((abstract_size * 6) + aclp->z_acl_bytes); + + zacep = newnode->z_acldata; + if (masks.allow0) { + zfs_set_ace(aclp, zacep, masks.allow0, ALLOW, -1, ACE_OWNER); + zacep = (void *)((uintptr_t)zacep + abstract_size); + new_count++; + new_bytes += abstract_size; + } if (masks.deny1) { + zfs_set_ace(aclp, zacep, masks.deny1, DENY, -1, ACE_OWNER); + zacep = (void *)((uintptr_t)zacep + abstract_size); + new_count++; + new_bytes += abstract_size; + } + if (masks.deny2) { + zfs_set_ace(aclp, zacep, masks.deny2, DENY, -1, OWNING_GROUP); + zacep = (void *)((uintptr_t)zacep + abstract_size); + new_count++; + new_bytes += abstract_size; + } + + while ((acep = zfs_acl_next_ace(aclp, acep, &who, &access_mask, + &iflags, &type))) { + entry_type = (iflags & ACE_TYPE_FLAGS); + /* + * ACEs used to represent the file mode may be divided + * into an equivalent pair of inherit-only and regular + * ACEs, if they are inheritable. + * Skip regular ACEs, which are replaced by the new mode. + */ + if (split && (entry_type == ACE_OWNER || + entry_type == OWNING_GROUP || + entry_type == ACE_EVERYONE)) { + if (!isdir || !(iflags & + (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) + continue; + /* + * We preserve owner@, group@, or @everyone + * permissions, if they are inheritable, by + * copying them to inherit_only ACEs. This + * prevents inheritable permissions from being + * altered along with the file mode. + */ + iflags |= ACE_INHERIT_ONLY_ACE; + } + + /* + * If this ACL has any inheritable ACEs, mark that in + * the hints (which are later masked into the pflags) + * so create knows to do inheritance. + */ + if (isdir && (iflags & + (ACE_FILE_INHERIT_ACE|ACE_DIRECTORY_INHERIT_ACE))) + aclp->z_hints |= ZFS_INHERIT_ACE; + + if ((type != ALLOW && type != DENY) || + (iflags & ACE_INHERIT_ONLY_ACE)) { + switch (type) { + case ACE_ACCESS_ALLOWED_OBJECT_ACE_TYPE: + case ACE_ACCESS_DENIED_OBJECT_ACE_TYPE: + case ACE_SYSTEM_AUDIT_OBJECT_ACE_TYPE: + case ACE_SYSTEM_ALARM_OBJECT_ACE_TYPE: + aclp->z_hints |= ZFS_ACL_OBJ_ACE; + break; + } + } else { + /* + * Limit permissions granted by ACEs to be no greater + * than permissions of the requested group mode. + * Applies when the "aclmode" property is set to + * "groupmask". + */ + if ((type == ALLOW) && trim) + access_mask &= masks.group; + } + zfs_set_ace(aclp, zacep, access_mask, type, who, iflags); + ace_size = aclp->z_ops->ace_size(acep); + zacep = (void *)((uintptr_t)zacep + ace_size); + new_count++; + new_bytes += ace_size; + } + zfs_set_ace(aclp, zacep, masks.owner, ALLOW, -1, ACE_OWNER); + zacep = (void *)((uintptr_t)zacep + abstract_size); + zfs_set_ace(aclp, zacep, masks.group, ALLOW, -1, OWNING_GROUP); + zacep = (void *)((uintptr_t)zacep + abstract_size); + zfs_set_ace(aclp, zacep, masks.everyone, ALLOW, -1, ACE_EVERYONE); + + new_count += 3; + new_bytes += abstract_size * 3; + zfs_acl_release_nodes(aclp); + aclp->z_acl_count = new_count; + aclp->z_acl_bytes = new_bytes; + newnode->z_ace_count = new_count; + newnode->z_size = new_bytes; + list_insert_tail(&aclp->z_acl, newnode); +} + +int +zfs_acl_chmod_setattr(znode_t *zp, zfs_acl_t **aclp, uint64_t mode) +{ + int error = 0; + + mutex_enter(&zp->z_acl_lock); + mutex_enter(&zp->z_lock); + + if (zp->z_zfsvfs->z_acl_mode == ZFS_ACL_DISCARD) + *aclp = zfs_acl_alloc(zfs_acl_version_zp(zp)); + else + error = zfs_acl_node_read(zp, B_TRUE, aclp, B_TRUE); + + if (error == 0) { + (*aclp)->z_hints = zp->z_pflags & V4_ACL_WIDE_FLAGS; + zfs_acl_chmod(zp->z_mode, mode, B_TRUE, + (zp->z_zfsvfs->z_acl_mode == ZFS_ACL_GROUPMASK), *aclp); + } + mutex_exit(&zp->z_lock); + mutex_exit(&zp->z_acl_lock); + + return (error); +} + +/* + * Should ACE be inherited? + */ +static int +zfs_ace_can_use(umode_t umode, uint16_t acep_flags) +{ + int iflags = (acep_flags & 0xf); + + if (S_ISDIR(umode) && (iflags & ACE_DIRECTORY_INHERIT_ACE)) + return (1); + else if (iflags & ACE_FILE_INHERIT_ACE) + return (!S_ISDIR((umode) && + (iflags & ACE_NO_PROPAGATE_INHERIT_ACE))); + return (0); +} + +/* + * inherit inheritable ACEs from parent + */ +static zfs_acl_t * +zfs_acl_inherit(zfsvfs_t *zfsvfs, zfs_acl_t *paclp, + uint64_t umode, boolean_t *need_chmod) +{ + void *pacep = NULL; + void *acep; + zfs_acl_node_t *aclnode; + zfs_acl_t *aclp = NULL; + uint64_t who; + uint32_t access_mask; + uint16_t iflags, newflags, type; + size_t ace_size; + void *data1, *data2 = NULL; + size_t data1sz, data2sz = 0; + uint_t aclinherit; + boolean_t isdir = S_ISDIR(umode); + boolean_t islnk = S_ISLNK(umode); + boolean_t isreg = S_ISREG(umode); + + *need_chmod = B_TRUE; + + aclp = zfs_acl_alloc(paclp->z_version); + aclinherit = zfsvfs->z_acl_inherit; + if (aclinherit == ZFS_ACL_DISCARD || islnk) + return (aclp); + + while ((pacep = zfs_acl_next_ace(paclp, pacep, &who, + &access_mask, &iflags, &type))) { + + /* + * don't inherit bogus ACEs + */ + if (!zfs_acl_valid_ace_type(type, iflags)) + continue; + + /* + * Check if ACE is inheritable by this vnode + */ + if ((aclinherit == ZFS_ACL_NOALLOW && type == ALLOW) || + !zfs_ace_can_use(umode, iflags)) + continue; + + /* + * If owner@, group@, or everyone@ inheritable + * then zfs_acl_chmod() isn't needed. + */ + if ((aclinherit == ZFS_ACL_PASSTHROUGH || + aclinherit == ZFS_ACL_PASSTHROUGH_X) && + ((iflags & (ACE_OWNER|ACE_EVERYONE)) || + ((iflags & OWNING_GROUP) == OWNING_GROUP)) && + (isreg || (isdir && (iflags & ACE_DIRECTORY_INHERIT_ACE)))) + *need_chmod = B_FALSE; + + /* + * Strip inherited execute permission from file if + * not in mode + */ + if (aclinherit == ZFS_ACL_PASSTHROUGH_X && type == ALLOW && + !isdir && ((umode & (S_IXUSR|S_IXGRP|S_IXOTH)) == 0)) { + access_mask &= ~ACE_EXECUTE; + } + + /* + * Strip write_acl and write_owner from permissions + * when inheriting an ACE + */ + if (aclinherit == ZFS_ACL_RESTRICTED && type == ALLOW) { + access_mask &= ~RESTRICTED_CLEAR; + } + + ace_size = aclp->z_ops->ace_size(pacep); + aclnode = zfs_acl_node_alloc(ace_size); + list_insert_tail(&aclp->z_acl, aclnode); + acep = aclnode->z_acldata; + + zfs_set_ace(aclp, acep, access_mask, type, + who, iflags|ACE_INHERITED_ACE); + + /* + * Copy special opaque data if any + */ + if ((data1sz = paclp->z_ops->ace_data(pacep, &data1)) != 0) { + VERIFY((data2sz = aclp->z_ops->ace_data(acep, + &data2)) == data1sz); + memcpy(data2, data1, data2sz); + } + + aclp->z_acl_count++; + aclnode->z_ace_count++; + aclp->z_acl_bytes += aclnode->z_size; + newflags = aclp->z_ops->ace_flags_get(acep); + + /* + * If ACE is not to be inherited further, or if the vnode is + * not a directory, remove all inheritance flags + */ + if (!isdir || (iflags & ACE_NO_PROPAGATE_INHERIT_ACE)) { + newflags &= ~ALL_INHERIT; + aclp->z_ops->ace_flags_set(acep, + newflags|ACE_INHERITED_ACE); + continue; + } + + /* + * This directory has an inheritable ACE + */ + aclp->z_hints |= ZFS_INHERIT_ACE; + + /* + * If only FILE_INHERIT is set then turn on + * inherit_only + */ + if ((iflags & (ACE_FILE_INHERIT_ACE | + ACE_DIRECTORY_INHERIT_ACE)) == ACE_FILE_INHERIT_ACE) { + newflags |= ACE_INHERIT_ONLY_ACE; + aclp->z_ops->ace_flags_set(acep, + newflags|ACE_INHERITED_ACE); + } else { + newflags &= ~ACE_INHERIT_ONLY_ACE; + aclp->z_ops->ace_flags_set(acep, + newflags|ACE_INHERITED_ACE); + } + } + + return (aclp); +} + +/* + * Create file system object initial permissions + * including inheritable ACEs. + * Also, create FUIDs for owner and group. + */ +int +zfs_acl_ids_create(znode_t *dzp, int flag, vattr_t *vap, cred_t *cr, + vsecattr_t *vsecp, zfs_acl_ids_t *acl_ids, zidmap_t *mnt_ns) +{ + int error; + zfsvfs_t *zfsvfs = dzp->z_zfsvfs; + zfs_acl_t *paclp = NULL; + gid_t gid; + boolean_t need_chmod = B_TRUE; + boolean_t trim = B_FALSE; + boolean_t inherited = B_FALSE; + + memset(acl_ids, 0, sizeof (zfs_acl_ids_t)); + acl_ids->z_mode = MAKEIMODE(vap->va_type, vap->va_mode); + + if (vsecp) + if ((error = zfs_vsec_2_aclp(zfsvfs, vap->va_type, vsecp, cr, + &acl_ids->z_fuidp, &acl_ids->z_aclp)) != 0) + return (error); + /* + * Determine uid and gid. + */ + if ((flag & IS_ROOT_NODE) || zfsvfs->z_replay || + ((flag & IS_XATTR) && (vap->va_type == VDIR))) { + acl_ids->z_fuid = zfs_fuid_create(zfsvfs, + (uint64_t)vap->va_uid, cr, + ZFS_OWNER, &acl_ids->z_fuidp); + acl_ids->z_fgid = zfs_fuid_create(zfsvfs, + (uint64_t)vap->va_gid, cr, + ZFS_GROUP, &acl_ids->z_fuidp); + gid = vap->va_gid; + } else { + acl_ids->z_fuid = zfs_fuid_create_cred(zfsvfs, ZFS_OWNER, + cr, &acl_ids->z_fuidp); + acl_ids->z_fgid = 0; + if (vap->va_mask & ATTR_GID) { + acl_ids->z_fgid = zfs_fuid_create(zfsvfs, + (uint64_t)vap->va_gid, + cr, ZFS_GROUP, &acl_ids->z_fuidp); + gid = vap->va_gid; + if (acl_ids->z_fgid != dzp->z_gid && + !groupmember(vap->va_gid, cr) && + secpolicy_vnode_create_gid(cr) != 0) + acl_ids->z_fgid = 0; + } + if (acl_ids->z_fgid == 0) { + if (dzp->z_mode & S_ISGID) { + char *domain; + uint32_t rid; + + acl_ids->z_fgid = dzp->z_gid; + gid = zfs_fuid_map_id(zfsvfs, acl_ids->z_fgid, + cr, ZFS_GROUP); + + if (zfsvfs->z_use_fuids && + IS_EPHEMERAL(acl_ids->z_fgid)) { + domain = zfs_fuid_idx_domain( + &zfsvfs->z_fuid_idx, + FUID_INDEX(acl_ids->z_fgid)); + rid = FUID_RID(acl_ids->z_fgid); + zfs_fuid_node_add(&acl_ids->z_fuidp, + domain, rid, + FUID_INDEX(acl_ids->z_fgid), + acl_ids->z_fgid, ZFS_GROUP); + } + } else { + acl_ids->z_fgid = zfs_fuid_create_cred(zfsvfs, + ZFS_GROUP, cr, &acl_ids->z_fuidp); +#ifdef __FreeBSD__ + gid = acl_ids->z_fgid = dzp->z_gid; +#else + gid = crgetgid(cr); +#endif + } + } + } + + /* + * If we're creating a directory, and the parent directory has the + * set-GID bit set, set in on the new directory. + * Otherwise, if the user is neither privileged nor a member of the + * file's new group, clear the file's set-GID bit. + */ + + if (!(flag & IS_ROOT_NODE) && (dzp->z_mode & S_ISGID) && + (vap->va_type == VDIR)) { + acl_ids->z_mode |= S_ISGID; + } else { + if ((acl_ids->z_mode & S_ISGID) && + secpolicy_vnode_setids_setgids(ZTOV(dzp), cr, gid) != 0) + acl_ids->z_mode &= ~S_ISGID; + } + + if (acl_ids->z_aclp == NULL) { + mutex_enter(&dzp->z_acl_lock); + mutex_enter(&dzp->z_lock); + if (!(flag & IS_ROOT_NODE) && + (dzp->z_pflags & ZFS_INHERIT_ACE) && + !(dzp->z_pflags & ZFS_XATTR)) { + VERIFY(0 == zfs_acl_node_read(dzp, B_TRUE, + &paclp, B_FALSE)); + acl_ids->z_aclp = zfs_acl_inherit(zfsvfs, + paclp, acl_ids->z_mode, &need_chmod); + inherited = B_TRUE; + } else { + acl_ids->z_aclp = + zfs_acl_alloc(zfs_acl_version_zp(dzp)); + acl_ids->z_aclp->z_hints |= ZFS_ACL_TRIVIAL; + } + mutex_exit(&dzp->z_lock); + mutex_exit(&dzp->z_acl_lock); + + if (need_chmod) { + if (vap->va_type == VDIR) + acl_ids->z_aclp->z_hints |= + ZFS_ACL_AUTO_INHERIT; + + if (zfsvfs->z_acl_mode == ZFS_ACL_GROUPMASK && + zfsvfs->z_acl_inherit != ZFS_ACL_PASSTHROUGH && + zfsvfs->z_acl_inherit != ZFS_ACL_PASSTHROUGH_X) + trim = B_TRUE; + zfs_acl_chmod(vap->va_type, acl_ids->z_mode, B_FALSE, + trim, acl_ids->z_aclp); + } + } + + if (inherited || vsecp) { + acl_ids->z_mode = zfs_mode_compute(acl_ids->z_mode, + acl_ids->z_aclp, &acl_ids->z_aclp->z_hints, + acl_ids->z_fuid, acl_ids->z_fgid); + if (ace_trivial_common(acl_ids->z_aclp, 0, zfs_ace_walk) == 0) + acl_ids->z_aclp->z_hints |= ZFS_ACL_TRIVIAL; + } + + return (0); +} + +/* + * Free ACL and fuid_infop, but not the acl_ids structure + */ +void +zfs_acl_ids_free(zfs_acl_ids_t *acl_ids) +{ + if (acl_ids->z_aclp) + zfs_acl_free(acl_ids->z_aclp); + if (acl_ids->z_fuidp) + zfs_fuid_info_free(acl_ids->z_fuidp); + acl_ids->z_aclp = NULL; + acl_ids->z_fuidp = NULL; +} + +boolean_t +zfs_acl_ids_overquota(zfsvfs_t *zv, zfs_acl_ids_t *acl_ids, uint64_t projid) +{ + return (zfs_id_overquota(zv, DMU_USERUSED_OBJECT, acl_ids->z_fuid) || + zfs_id_overquota(zv, DMU_GROUPUSED_OBJECT, acl_ids->z_fgid) || + (projid != ZFS_DEFAULT_PROJID && projid != ZFS_INVALID_PROJID && + zfs_id_overquota(zv, DMU_PROJECTUSED_OBJECT, projid))); +} + +/* + * Retrieve a file's ACL + */ +int +zfs_getacl(znode_t *zp, vsecattr_t *aclpp, boolean_t skipaclcheck, + cred_t *cr) +{ + zfs_acl_t *aclp; + kauth_acl_t *k_acl; + uint32_t ace_flags = 0; + kauth_ace_rights_t *rights = 0; + guid_t *guidp; + uint64_t who; + uint32_t access_mask; + uint16_t flags; + uint16_t type; + int i; + int error; + void *zacep = NULL; + + mutex_enter(&zp->z_acl_lock); + + error = zfs_acl_node_read(zp, B_FALSE, &aclp, B_TRUE); + if (error != 0) { + mutex_exit(&zp->z_acl_lock); + return (error); + } +// if ((k_acl = kauth_acl_alloc(aclp->z_acl_count)) == NULL) { +// mutex_exit(&zp->z_acl_lock); +// *aclpp = (kauth_acl_t *) KAUTH_FILESEC_NONE; +// return (ENOMEM); +// } + + dprintf("acl_count %d\n", aclp->z_acl_count); + +// k_acl->acl_entrycount = aclp->z_acl_count; +// k_acl->acl_flags = 0; +#if 0 + *aclpp = k_acl; + + /* + * Translate Open Solaris ACEs to Mac OS X ACLs + */ + i = 0; + while ((zacep = zfs_acl_next_ace(aclp, zacep, + &who, &access_mask, &flags, &type))) { + rights = 0; + ace_flags = 0; + +// guidp = &k_acl->acl_ace[i].ace_applicable; + + if (flags & ACE_OWNER) { +#if HIDE_TRIVIAL_ACL + continue; +#endif + who = -1; +// nfsacl_set_wellknown(KAUTH_WKG_OWNER, guidp); + } else if ((flags & OWNING_GROUP) == OWNING_GROUP) { +#if HIDE_TRIVIAL_ACL + continue; +#endif + who = -1; +// nfsacl_set_wellknown(KAUTH_WKG_GROUP, guidp); + } else if (flags & ACE_EVERYONE) { +#if HIDE_TRIVIAL_ACL + continue; +#endif + who = -1; +// nfsacl_set_wellknown(KAUTH_WKG_EVERYBODY, guidp); + /* Try to get a guid from our uid */ + } else { + + dprintf("ZFS: trying to map uid %d flags %x type %x\n", + who, flags, type); + + if (flags & OWNING_GROUP) { +// if (kauth_cred_gid2guid(who, guidp) == 0) { +// dprintf("ZFS: appears to be a group\n"); +// } +// } else if (kauth_cred_uid2guid(who, guidp) == 0) { +// dprintf("ZFS: appears to be a user\n"); + } else { + dprintf("ZFS: Unable to map\n"); + memset(guidp, 0, sizeof (guid_t)); + } + } + + // access_mask = aclp->z_acl[i].a_access_mask; + if (access_mask & ACE_READ_DATA) + rights |= KAUTH_VNODE_READ_DATA; + if (access_mask & ACE_WRITE_DATA) + rights |= KAUTH_VNODE_WRITE_DATA; + if (access_mask & ACE_APPEND_DATA) + rights |= KAUTH_VNODE_APPEND_DATA; + if (access_mask & ACE_READ_NAMED_ATTRS) + rights |= KAUTH_VNODE_READ_EXTATTRIBUTES; + if (access_mask & ACE_WRITE_NAMED_ATTRS) + rights |= KAUTH_VNODE_WRITE_EXTATTRIBUTES; + if (access_mask & ACE_EXECUTE) + rights |= KAUTH_VNODE_EXECUTE; + if (access_mask & ACE_DELETE_CHILD) + rights |= KAUTH_VNODE_DELETE_CHILD; + if (access_mask & ACE_READ_ATTRIBUTES) + rights |= KAUTH_VNODE_READ_ATTRIBUTES; + if (access_mask & ACE_WRITE_ATTRIBUTES) + rights |= KAUTH_VNODE_WRITE_ATTRIBUTES; + if (access_mask & ACE_DELETE) + rights |= KAUTH_VNODE_DELETE; + if (access_mask & ACE_READ_ACL) + rights |= KAUTH_VNODE_READ_SECURITY; + if (access_mask & ACE_WRITE_ACL) + rights |= KAUTH_VNODE_WRITE_SECURITY; + if (access_mask & ACE_WRITE_OWNER) + rights |= KAUTH_VNODE_TAKE_OWNERSHIP; + if (access_mask & ACE_SYNCHRONIZE) + rights |= KAUTH_VNODE_SYNCHRONIZE; + k_acl->acl_ace[i].ace_rights = rights; + + // flags = aclp->z_acl[i].a_flags; + if (flags & ACE_FILE_INHERIT_ACE) + ace_flags |= KAUTH_ACE_FILE_INHERIT; + if (flags & ACE_DIRECTORY_INHERIT_ACE) + ace_flags |= KAUTH_ACE_DIRECTORY_INHERIT; + if (flags & ACE_NO_PROPAGATE_INHERIT_ACE) + ace_flags |= KAUTH_ACE_LIMIT_INHERIT; + if (flags & ACE_INHERIT_ONLY_ACE) + ace_flags |= KAUTH_ACE_ONLY_INHERIT; + + // type = aclp->z_acl[i].a_type; + switch (type) { + case ACE_ACCESS_ALLOWED_ACE_TYPE: + ace_flags |= KAUTH_ACE_PERMIT; + break; + case ACE_ACCESS_DENIED_ACE_TYPE: + ace_flags |= KAUTH_ACE_DENY; + break; + case ACE_SYSTEM_AUDIT_ACE_TYPE: + ace_flags |= KAUTH_ACE_AUDIT; + break; + case ACE_SYSTEM_ALARM_ACE_TYPE: + ace_flags |= KAUTH_ACE_ALARM; + break; + } + k_acl->acl_ace[i].ace_flags = ace_flags; + i++; + } + k_acl->acl_entrycount = i; +#endif + + mutex_exit(&zp->z_acl_lock); + + zfs_acl_free(aclp); + + return (0); +} + +int +zfs_addacl_trivial(znode_t *zp, ace_t *aces, int *nentries, int seen_type) +{ + zfs_acl_t *aclp; + uint64_t who; + uint32_t access_mask; + uint16_t flags; + uint16_t type; + int i; + int error; + void *zacep = NULL; + + mutex_enter(&zp->z_acl_lock); + + error = zfs_acl_node_read(zp, B_FALSE, &aclp, B_TRUE); + if (error != 0) { + mutex_exit(&zp->z_acl_lock); + return (error); + } + + dprintf("ondisk acl_count %d\n", aclp->z_acl_count); + + // Start at the end + i = *nentries; + + /* + * Translate Open Solaris ACEs to Mac OS X ACLs + */ + while ((zacep = zfs_acl_next_ace(aclp, zacep, + &who, &access_mask, &flags, &type))) { + + if (flags & ACE_OWNER) { + if (seen_type & ACE_OWNER) continue; + seen_type |= ACE_OWNER; + who = -1; + } else if ((flags & OWNING_GROUP) == OWNING_GROUP) { + if (seen_type & ACE_GROUP) continue; + seen_type |= ACE_GROUP; + who = -1; + } else if (flags & ACE_EVERYONE) { + if (seen_type & ACE_EVERYONE) continue; + seen_type |= ACE_EVERYONE; + who = -1; + /* Try to get a guid from our uid */ + } else { + + // Only deal with the trivials + continue; + } + + aces[i].a_who = who; + aces[i].a_access_mask = access_mask; + aces[i].a_flags = flags; + aces[i].a_type = type; + + dprintf("zfs: adding entry %d for type %x sizeof %d\n", i, type, + sizeof (aces[i])); + i++; + } + + *nentries = i; + mutex_exit(&zp->z_acl_lock); + + zfs_acl_free(aclp); + + return (0); +} + + +int +zfs_vsec_2_aclp(zfsvfs_t *zfsvfs, umode_t obj_mode, + vsecattr_t *vsecp, cred_t *cr, zfs_fuid_info_t **fuidp, zfs_acl_t **zaclp) +{ + zfs_acl_t *aclp; + zfs_acl_node_t *aclnode; + int aclcnt = vsecp->vsa_aclcnt; + int error; + + if (vsecp->vsa_aclcnt > MAX_ACL_ENTRIES || vsecp->vsa_aclcnt <= 0) + return (SET_ERROR(EINVAL)); + + aclp = zfs_acl_alloc(zfs_acl_version(zfsvfs->z_version)); + + aclp->z_hints = 0; + aclnode = zfs_acl_node_alloc(aclcnt * sizeof (zfs_object_ace_t)); + if (aclp->z_version == ZFS_ACL_VERSION_INITIAL) { + if ((error = zfs_copy_ace_2_oldace(obj_mode, aclp, + (ace_t *)vsecp->vsa_aclentp, aclnode->z_acldata, + aclcnt, &aclnode->z_size)) != 0) { + zfs_acl_free(aclp); + zfs_acl_node_free(aclnode); + return (error); + } + } else { + if ((error = zfs_copy_ace_2_fuid(zfsvfs, obj_mode, aclp, + vsecp->vsa_aclentp, aclnode->z_acldata, aclcnt, + &aclnode->z_size, fuidp, cr)) != 0) { + zfs_acl_free(aclp); + zfs_acl_node_free(aclnode); + return (error); + } + } + aclp->z_acl_bytes = aclnode->z_size; + aclnode->z_ace_count = aclcnt; + aclp->z_acl_count = aclcnt; + list_insert_head(&aclp->z_acl, aclnode); + + /* + * If flags are being set then add them to z_hints + */ + if (vsecp->vsa_mask & VSA_ACE_ACLFLAGS) { + if (vsecp->vsa_aclflags & ACL_PROTECTED) + aclp->z_hints |= ZFS_ACL_PROTECTED; + if (vsecp->vsa_aclflags & ACL_DEFAULTED) + aclp->z_hints |= ZFS_ACL_DEFAULTED; + if (vsecp->vsa_aclflags & ACL_AUTO_INHERIT) + aclp->z_hints |= ZFS_ACL_AUTO_INHERIT; + } + + *zaclp = aclp; + + return (0); +} + + + +/* + * Set a file's ACL + */ +int +zfs_setacl(znode_t *zp, vsecattr_t *vsecp, boolean_t skipaclchk, cred_t *cr) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + zilog_t *zilog = zfsvfs->z_log; + // ulong_t mask = vsecp->vsa_mask & (VSA_ACE | VSA_ACECNT); + dmu_tx_t *tx; + int error; + zfs_acl_t *aclp; + zfs_fuid_info_t *fuidp = NULL; + boolean_t fuid_dirtied; + uint64_t acl_obj; + + // Anyone remember why we commented this out? + // if (mask == 0) + // return (ENOSYS); + + if (zp->z_pflags & ZFS_IMMUTABLE) + return (SET_ERROR(EPERM)); + + if ((error = zfs_zaccess(zp, ACE_WRITE_ACL, 0, skipaclchk, cr, NULL))) + return (error); + + error = zfs_vsec_2_aclp(zfsvfs, vnode_vtype(ZTOV(zp)), vsecp, cr, + &fuidp, &aclp); + if (error) + return (error); + + /* + * If ACL wide flags aren't being set then preserve any + * existing flags. + */ + // if (!(vsecp->vsa_mask & VSA_ACE_ACLFLAGS)) { + // aclp->z_hints |= + // (zp->z_pflags & V4_ACL_WIDE_FLAGS); + // } +top: + mutex_enter(&zp->z_acl_lock); + mutex_enter(&zp->z_lock); + + tx = dmu_tx_create(zfsvfs->z_os); + + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_TRUE); + + fuid_dirtied = zfsvfs->z_fuid_dirty; + if (fuid_dirtied) + zfs_fuid_txhold(zfsvfs, tx); + + /* + * If old version and ACL won't fit in bonus and we aren't + * upgrading then take out necessary DMU holds + */ + + if ((acl_obj = zfs_external_acl(zp)) != 0) { + if (zfsvfs->z_version >= ZPL_VERSION_FUID && + zfs_znode_acl_version(zp) <= ZFS_ACL_VERSION_INITIAL) { + dmu_tx_hold_free(tx, acl_obj, 0, + DMU_OBJECT_END); + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, + aclp->z_acl_bytes); + } else { + dmu_tx_hold_write(tx, acl_obj, 0, aclp->z_acl_bytes); + } + } else if (!zp->z_is_sa && aclp->z_acl_bytes > ZFS_ACE_SPACE) { + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, aclp->z_acl_bytes); + } + + zfs_sa_upgrade_txholds(tx, zp); + error = dmu_tx_assign(tx, TXG_NOWAIT); + if (error) { + mutex_exit(&zp->z_acl_lock); + mutex_exit(&zp->z_lock); + + if (error == ERESTART) { + dmu_tx_wait(tx); + dmu_tx_abort(tx); + goto top; + } + dmu_tx_abort(tx); + zfs_acl_free(aclp); + return (error); + } + + error = zfs_aclset_common(zp, aclp, cr, tx); + ASSERT(error == 0); + ASSERT(zp->z_acl_cached == NULL); + zp->z_acl_cached = aclp; + + if (fuid_dirtied) + zfs_fuid_sync(zfsvfs, tx); + + zfs_log_acl(zilog, tx, zp, vsecp, fuidp); + + if (fuidp) + zfs_fuid_info_free(fuidp); + dmu_tx_commit(tx); + + mutex_exit(&zp->z_lock); + mutex_exit(&zp->z_acl_lock); + + return (error); +} + + +/* + * Check accesses of interest (AoI) against attributes of the dataset + * such as read-only. Returns zero if no AoI conflict with dataset + * attributes, otherwise an appropriate errno is returned. + */ +static int +zfs_zaccess_dataset_check(znode_t *zp, uint32_t v4_mode) +{ + if ((v4_mode & WRITE_MASK) && + (vfs_isrdonly(zp->z_zfsvfs->z_vfs)) && + (!IS_DEVVP(ZTOV(zp)) || + (IS_DEVVP(ZTOV(zp)) && (v4_mode & WRITE_MASK_ATTRS)))) { + return (SET_ERROR(EROFS)); + } + + /* + * Intentionally allow ZFS_READONLY through here. + * See zfs_zaccess_common(). + */ + if ((v4_mode & WRITE_MASK_DATA) && + (zp->z_pflags & ZFS_IMMUTABLE)) { + return (EPERM); + } +#ifdef sun + if ((v4_mode & (ACE_DELETE | ACE_DELETE_CHILD)) && + (zp->z_pflags & ZFS_NOUNLINK)) { + return (EPERM); +#else + /* + * In FreeBSD we allow to modify directory's content is ZFS_NOUNLINK + * (sunlnk) is set. We just don't allow directory removal, which is + * handled in zfs_zaccess_delete(). + */ + if ((v4_mode & ACE_DELETE) && + (zp->z_pflags & ZFS_NOUNLINK)) { + return (SET_ERROR(EPERM)); + } +#endif + + if (((v4_mode & (ACE_READ_DATA|ACE_EXECUTE)) && + (zp->z_pflags & ZFS_AV_QUARANTINED))) { + return (SET_ERROR(EACCES)); + } + + return (0); +} + +/* + * The primary usage of this function is to loop through all of the + * ACEs in the znode, determining what accesses of interest (AoI) to + * the caller are allowed or denied. The AoI are expressed as bits in + * the working_mode parameter. As each ACE is processed, bits covered + * by that ACE are removed from the working_mode. This removal + * facilitates two things. The first is that when the working mode is + * empty (= 0), we know we've looked at all the AoI. The second is + * that the ACE interpretation rules don't allow a later ACE to undo + * something granted or denied by an earlier ACE. Removing the + * discovered access or denial enforces this rule. At the end of + * processing the ACEs, all AoI that were found to be denied are + * placed into the working_mode, giving the caller a mask of denied + * accesses. Returns: + * 0 if all AoI granted + * EACCES if the denied mask is non-zero + * other error if abnormal failure (e.g., IO error) + * + * A secondary usage of the function is to determine if any of the + * AoI are granted. If an ACE grants any access in + * the working_mode, we immediately short circuit out of the function. + * This mode is chosen by setting anyaccess to B_TRUE. The + * working_mode is not a denied access mask upon exit if the function + * is used in this manner. + */ +static int +zfs_zaccess_aces_check(znode_t *zp, uint32_t *working_mode, + boolean_t anyaccess, cred_t *cr) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + zfs_acl_t *aclp; + int error; + uid_t uid = crgetuid(cr); + uint64_t who; + uint16_t type, iflags; + uint16_t entry_type; + uint32_t access_mask; + uint32_t deny_mask = 0; + zfs_ace_hdr_t *acep = NULL; + boolean_t checkit; + uid_t gowner; + uid_t fowner; + + zfs_fuid_map_ids(zp, cr, &fowner, &gowner); + + mutex_enter(&zp->z_acl_lock); + + error = zfs_acl_node_read(zp, B_FALSE, &aclp, B_FALSE); + if (error != 0) { + mutex_exit(&zp->z_acl_lock); + return (error); + } + + ASSERT(zp->z_acl_cached); + + while ((acep = zfs_acl_next_ace(aclp, acep, &who, &access_mask, + &iflags, &type))) { + uint32_t mask_matched; + + if (!zfs_acl_valid_ace_type(type, iflags)) + continue; + + if (vnode_isdir(ZTOV(zp)) && (iflags & ACE_INHERIT_ONLY_ACE)) + continue; + + /* Skip ACE if it does not affect any AoI */ + mask_matched = (access_mask & *working_mode); + if (!mask_matched) + continue; + + entry_type = (iflags & ACE_TYPE_FLAGS); + + checkit = B_FALSE; + + switch (entry_type) { + case ACE_OWNER: + if (uid == fowner) + checkit = B_TRUE; + break; + case OWNING_GROUP: + who = gowner; + /*FALLTHROUGH*/ + case ACE_IDENTIFIER_GROUP: + checkit = zfs_groupmember(zfsvfs, who, cr); + break; + case ACE_EVERYONE: + checkit = B_TRUE; + break; + + /* USER Entry */ + default: + if (entry_type == 0) { + uid_t newid; + + newid = zfs_fuid_map_id(zfsvfs, who, cr, + ZFS_ACE_USER); + if (newid != IDMAP_WK_CREATOR_OWNER_UID && + uid == newid) + checkit = B_TRUE; + break; + } else { + mutex_exit(&zp->z_acl_lock); + return (SET_ERROR(EIO)); + } + } + + if (checkit) { + if (type == DENY) { + DTRACE_PROBE3(zfs__ace__denies, + znode_t *, zp, + zfs_ace_hdr_t *, acep, + uint32_t, mask_matched); + deny_mask |= mask_matched; + } else { + DTRACE_PROBE3(zfs__ace__allows, + znode_t *, zp, + zfs_ace_hdr_t *, acep, + uint32_t, mask_matched); + if (anyaccess) { + mutex_exit(&zp->z_acl_lock); + return (0); + } + } + *working_mode &= ~mask_matched; + } + + /* Are we done? */ + if (*working_mode == 0) + break; + } + + mutex_exit(&zp->z_acl_lock); + + /* Put the found 'denies' back on the working mode */ + if (deny_mask) { + *working_mode |= deny_mask; + return (SET_ERROR(EACCES)); + } else if (*working_mode) { + return (-1); + } + + return (0); +} + +/* + * Return true if any access whatsoever granted, we don't actually + * care what access is granted. + */ +boolean_t +zfs_has_access(znode_t *zp, cred_t *cr) +{ + uint32_t have = ACE_ALL_PERMS; + + if (zfs_zaccess_aces_check(zp, &have, B_TRUE, cr) != 0) { + uid_t owner; + + owner = zfs_fuid_map_id(zp->z_zfsvfs, zp->z_uid, cr, ZFS_OWNER); + return (secpolicy_vnode_any_access(cr, ZTOV(zp), owner) == 0); + } + return (B_TRUE); +} + +static int +zfs_zaccess_common(znode_t *zp, uint32_t v4_mode, uint32_t *working_mode, + boolean_t *check_privs, boolean_t skipaclchk, cred_t *cr) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + int err; + + *working_mode = v4_mode; + *check_privs = B_TRUE; + + /* + * Short circuit empty requests + */ + if (v4_mode == 0 || zfsvfs->z_replay) { + *working_mode = 0; + return (0); + } + + if ((err = zfs_zaccess_dataset_check(zp, v4_mode)) != 0) { + *check_privs = B_FALSE; + return (err); + } + + /* + * The caller requested that the ACL check be skipped. This + * would only happen if the caller checked VOP_ACCESS() with a + * 32 bit ACE mask and already had the appropriate permissions. + */ + if (skipaclchk) { + *working_mode = 0; + return (0); + } + + /* + * Note: ZFS_READONLY represents the "DOS R/O" attribute. + * When that flag is set, we should behave as if write access + * were not granted by anything in the ACL. In particular: + * We _must_ allow writes after opening the file r/w, then + * setting the DOS R/O attribute, and writing some more. + * (Similar to how you can write after fchmod(fd, 0444).) + * + * Therefore ZFS_READONLY is ignored in the dataset check + * above, and checked here as if part of the ACL check. + * Also note: DOS R/O is ignored for directories. + */ + if ((v4_mode & WRITE_MASK_DATA) && + !vnode_isdir(ZTOV(zp)) && + (zp->z_pflags & ZFS_READONLY)) { + return (SET_ERROR(EPERM)); + } + + return (zfs_zaccess_aces_check(zp, working_mode, B_FALSE, cr)); +} + +static int +zfs_zaccess_append(znode_t *zp, uint32_t *working_mode, boolean_t *check_privs, + cred_t *cr) +{ + if (*working_mode != ACE_WRITE_DATA) + return (SET_ERROR(EACCES)); + + return (zfs_zaccess_common(zp, ACE_APPEND_DATA, working_mode, + check_privs, B_FALSE, cr)); +} + +int +zfs_fastaccesschk_execute(znode_t *zdp, cred_t *cr) +{ + boolean_t owner = B_FALSE; + boolean_t groupmbr = B_FALSE; + boolean_t is_attr; + uid_t uid = crgetuid(cr); + int error; + + if (zdp->z_pflags & ZFS_AV_QUARANTINED) + return (SET_ERROR(EACCES)); + + is_attr = ((zdp->z_pflags & ZFS_XATTR) && + (vnode_isdir(ZTOV(zdp)))); + if (is_attr) + goto slow; + + + mutex_enter(&zdp->z_acl_lock); + + if (zdp->z_pflags & ZFS_NO_EXECS_DENIED) { + mutex_exit(&zdp->z_acl_lock); + return (0); + } + + if (FUID_INDEX(zdp->z_uid) != 0 || FUID_INDEX(zdp->z_gid) != 0) { + mutex_exit(&zdp->z_acl_lock); + goto slow; + } + + if (uid == zdp->z_uid) { + owner = B_TRUE; + if (zdp->z_mode & S_IXUSR) { + mutex_exit(&zdp->z_acl_lock); + return (0); + } else { + mutex_exit(&zdp->z_acl_lock); + goto slow; + } + } + if (groupmember(zdp->z_gid, cr)) { + groupmbr = B_TRUE; + if (zdp->z_mode & S_IXGRP) { + mutex_exit(&zdp->z_acl_lock); + return (0); + } else { + mutex_exit(&zdp->z_acl_lock); + goto slow; + } + } + if (!owner && !groupmbr) { + if (zdp->z_mode & S_IXOTH) { + mutex_exit(&zdp->z_acl_lock); + return (0); + } + } + + mutex_exit(&zdp->z_acl_lock); + +slow: + DTRACE_PROBE(zfs__fastpath__execute__access__miss); + if ((error = zfs_enter(zdp->z_zfsvfs, FTAG)) != 0) + return (error); + error = zfs_zaccess(zdp, ACE_EXECUTE, 0, B_FALSE, cr, NULL); + zfs_exit(zdp->z_zfsvfs, FTAG); + return (error); +} + +/* + * Determine whether Access should be granted/denied. + * + * The least priv subsytem is always consulted as a basic privilege + * can define any form of access. + */ +int +zfs_zaccess(znode_t *zp, int mode, int flags, boolean_t skipaclchk, cred_t *cr, + zidmap_t *mnt_ns) +{ + uint32_t working_mode; + int error; + int is_attr; + boolean_t check_privs; + znode_t *xzp = NULL; + znode_t *check_zp = zp; + mode_t needed_bits; + uid_t owner; + + is_attr = ((zp->z_pflags & ZFS_XATTR) && (vnode_isdir(ZTOV(zp)))); + +#ifdef _WIN32 + /* + * In FreeBSD, we don't care about permissions of individual ADS. + * Note that not checking them is not just an optimization - without + * this shortcut, EA operations may bogusly fail with EACCES. + */ + if (zp->z_pflags & ZFS_XATTR) + return (0); +#else + /* + * If attribute then validate against base file + */ + if (is_attr) { + uint64_t parent; + + if ((error = sa_lookup(zp->z_sa_hdl, + SA_ZPL_PARENT(zp->z_zfsvfs), &parent, + sizeof (parent))) != 0) + return (error); + + /* + * Cache the lookup on the parent file znode as + * zp->z_xattr_parent and hold a reference. This + * effectively pins the parent in memory until all + * child xattr znodes have been destroyed and + * release their references in zfs_inode_destroy(). + */ + error = zfs_zget(zp->z_zfsvfs, parent, &check_zp); + if (error) + return (error); + + rw_enter(&zp->z_xattr_lock, RW_WRITER); + if (zp->z_xattr_parent == NULL) + zp->z_xattr_parent = check_zp; + rw_exit(&zp->z_xattr_lock); + } + + check_zp = xzp; + + /* + * fixup mode to map to xattr perms + */ + + if (mode & (ACE_WRITE_DATA|ACE_APPEND_DATA)) { + mode &= ~(ACE_WRITE_DATA|ACE_APPEND_DATA); + mode |= ACE_WRITE_NAMED_ATTRS; + } + + if (mode & (ACE_READ_DATA|ACE_EXECUTE)) { + mode &= ~(ACE_READ_DATA|ACE_EXECUTE); + mode |= ACE_READ_NAMED_ATTRS; + } + } +#endif + + owner = zfs_fuid_map_id(zp->z_zfsvfs, zp->z_uid, cr, ZFS_OWNER); + /* + * Map the bits required to the standard vnode flags VREAD|VWRITE|VEXEC + * in needed_bits. Map the bits mapped by working_mode (currently + * missing) in missing_bits. + * Call secpolicy_vnode_access2() with (needed_bits & ~checkmode), + * needed_bits. + */ + needed_bits = 0; + + working_mode = mode; + if ((working_mode & (ACE_READ_ACL|ACE_READ_ATTRIBUTES)) && + owner == crgetuid(cr)) + working_mode &= ~(ACE_READ_ACL|ACE_READ_ATTRIBUTES); + + if (working_mode & (ACE_READ_DATA|ACE_READ_NAMED_ATTRS| + ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_SYNCHRONIZE)) + needed_bits |= VREAD; + if (working_mode & (ACE_WRITE_DATA|ACE_WRITE_NAMED_ATTRS| + ACE_APPEND_DATA|ACE_WRITE_ATTRIBUTES|ACE_SYNCHRONIZE)) + needed_bits |= VWRITE; + if (working_mode & ACE_EXECUTE) + needed_bits |= VEXEC; + + if ((error = zfs_zaccess_common(check_zp, mode, &working_mode, + &check_privs, skipaclchk, cr)) == 0) { + if (is_attr) + VN_RELE(ZTOV(xzp)); + return (secpolicy_vnode_access2(cr, ZTOV(zp), owner, + needed_bits, needed_bits)); + } + + if (error && !check_privs) { + if (is_attr) + VN_RELE(ZTOV(xzp)); + return (error); + } + + if (error && (flags & V_APPEND)) { + error = zfs_zaccess_append(zp, &working_mode, &check_privs, cr); + } + + if (error && check_privs) { + mode_t checkmode = 0; + + /* + * First check for implicit owner permission on + * read_acl/read_attributes + */ + + error = 0; + ASSERT(working_mode != 0); + + if ((working_mode & (ACE_READ_ACL|ACE_READ_ATTRIBUTES) && + owner == crgetuid(cr))) + working_mode &= ~(ACE_READ_ACL|ACE_READ_ATTRIBUTES); + + if (working_mode & (ACE_READ_DATA|ACE_READ_NAMED_ATTRS| + ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_SYNCHRONIZE)) + checkmode |= VREAD; + if (working_mode & (ACE_WRITE_DATA|ACE_WRITE_NAMED_ATTRS| + ACE_APPEND_DATA|ACE_WRITE_ATTRIBUTES|ACE_SYNCHRONIZE)) + checkmode |= VWRITE; + if (working_mode & ACE_EXECUTE) + checkmode |= VEXEC; + + error = secpolicy_vnode_access2(cr, ZTOV(check_zp), owner, + needed_bits & ~checkmode, needed_bits); + + if (error == 0 && (working_mode & ACE_WRITE_OWNER)) + error = secpolicy_vnode_chown(ZTOV(check_zp), cr, + owner); + if (error == 0 && (working_mode & ACE_WRITE_ACL)) + error = secpolicy_vnode_setdac(ZTOV(check_zp), cr, + owner); + + if (error == 0 && (working_mode & + (ACE_DELETE|ACE_DELETE_CHILD))) + error = secpolicy_vnode_remove(ZTOV(check_zp), cr); + + if (error == 0 && (working_mode & ACE_SYNCHRONIZE)) { + error = secpolicy_vnode_chown(ZTOV(check_zp), cr, + owner); + } + if (error == 0) { + /* + * See if any bits other than those already checked + * for are still present. If so then return EACCES + */ + if (working_mode & ~(ZFS_CHECKED_MASKS)) { + error = SET_ERROR(EACCES); + } + } + } else if (error == 0) { + error = secpolicy_vnode_access2(cr, ZTOV(zp), owner, + needed_bits, needed_bits); + } + + + if (is_attr) + VN_RELE(ZTOV(xzp)); + + return (error); +} + +/* + * Translate traditional unix VREAD/VWRITE/VEXEC mode into + * native ACL format and call zfs_zaccess() + */ +int +zfs_zaccess_rwx(znode_t *zp, mode_t mode, int flags, cred_t *cr, + zidmap_t *mnt_ns) +{ + return (zfs_zaccess(zp, zfs_unix_to_v4(mode >> 6), flags, B_FALSE, cr, + NULL)); +} + +/* + * Access function for secpolicy_vnode_setattr + */ +int +zfs_zaccess_unix(void *zp, int mode, cred_t *cr) +{ + int v4_mode = zfs_unix_to_v4(mode >> 6); + + return (zfs_zaccess(zp, v4_mode, 0, B_FALSE, cr, NULL)); +} + +/* See zfs_zaccess_delete() */ +uint64_t zfs_write_implies_delete_child = 1; + +/* + * Determine whether delete access should be granted. + * + * The following chart outlines how we handle delete permissions which is + * how recent versions of windows (Windows 2008) handles it. The efficiency + * comes from not having to check the parent ACL where the object itself grants + * delete: + * + * ------------------------------------------------------- + * | Parent Dir | Target Object Permissions | + * | permissions | | + * ------------------------------------------------------- + * | | ACL Allows | ACL Denies| Delete | + * | | Delete | Delete | unspecified| + * ------------------------------------------------------- + * | ACL Allows | Permit | Deny * | Permit | + * | DELETE_CHILD | | | | + * ------------------------------------------------------- + * | ACL Denies | Permit | Deny | Deny | + * | DELETE_CHILD | | | | + * ------------------------------------------------------- + * | ACL specifies | | | | + * | only allow | Permit | Deny * | Permit | + * | write and | | | | + * | execute | | | | + * ------------------------------------------------------- + * | ACL denies | | | | + * | write and | Permit | Deny | Deny | + * | execute | | | | + * ------------------------------------------------------- + * ^ + * | + * Re. execute permission on the directory: if that's missing, + * the vnode lookup of the target will fail before we get here. + * + * Re [*] in the table above: NFSv4 would normally Permit delete for + * these two cells of the matrix. + * See acl.h for notes on which ACE_... flags should be checked for which + * operations. Specifically, the NFSv4 committee recommendation is in + * conflict with the Windows interpretation of DENY ACEs, where DENY ACEs + * should take precedence ahead of ALLOW ACEs. + * + * This implementation always consults the target object's ACL first. + * If a DENY ACE is present on the target object that specifies ACE_DELETE, + * delete access is denied. If an ALLOW ACE with ACE_DELETE is present on + * the target object, access is allowed. If and only if no entries with + * ACE_DELETE are present in the object's ACL, check the container's ACL + * for entries with ACE_DELETE_CHILD. + * + * A summary of the logic implemented from the table above is as follows: + * + * First check for DENY ACEs that apply. + * If either target or container has a deny, EACCES. + * + * Delete access can then be summarized as follows: + * 1: The object to be deleted grants ACE_DELETE, or + * 2: The containing directory grants ACE_DELETE_CHILD. + * In a Windows system, that would be the end of the story. + * In this system, (2) has some complications... + * 2a: "sticky" bit on a directory adds restrictions, and + * 2b: existing ACEs from previous versions of ZFS may + * not carry ACE_DELETE_CHILD where they should, so we + * also allow delete when ACE_WRITE_DATA is granted. + * + * Note: 2b is technically a work-around for a prior bug, + * which hopefully can go away some day. For those who + * no longer need the work around, and for testing, this + * work-around is made conditional via the tunable: + * zfs_write_implies_delete_child + */ +int +zfs_zaccess_delete(znode_t *dzp, znode_t *zp, cred_t *cr, zidmap_t *mnt_ns) +{ + uint32_t wanted_dirperms; + uint32_t dzp_working_mode = 0; + uint32_t zp_working_mode = 0; + int dzp_error, zp_error; + boolean_t dzpcheck_privs; + boolean_t zpcheck_privs; + + if (zp->z_pflags & (ZFS_IMMUTABLE | ZFS_NOUNLINK)) + return (SET_ERROR(EPERM)); + + /* + * Case 1: + * If target object grants ACE_DELETE then we are done. This is + * indicated by a return value of 0. For this case we don't worry + * about the sticky bit because sticky only applies to the parent + * directory and this is the child access result. + * + * If we encounter a DENY ACE here, we're also done (EACCES). + * Note that if we hit a DENY ACE here (on the target) it should + * take precedence over a DENY ACE on the container, so that when + * we have more complete auditing support we will be able to + * report an access failure against the specific target. + * (This is part of why we're checking the target first.) + */ + zp_error = zfs_zaccess_common(zp, ACE_DELETE, &zp_working_mode, + &zpcheck_privs, B_FALSE, cr); + if (zp_error == EACCES) { + /* We hit a DENY ACE. */ + if (!zpcheck_privs) + return (SET_ERROR(zp_error)); + + return (secpolicy_vnode_remove(ZTOV(zp), cr)); + } + if (zp_error == 0) + return (0); + + /* + * Case 2: + * If the containing directory grants ACE_DELETE_CHILD, + * or we're in backward compatibility mode and the + * containing directory has ACE_WRITE_DATA, allow. + * Case 2b is handled with wanted_dirperms. + */ + wanted_dirperms = ACE_DELETE_CHILD; + if (zfs_write_implies_delete_child) + wanted_dirperms |= ACE_WRITE_DATA; + dzp_error = zfs_zaccess_common(dzp, wanted_dirperms, + &dzp_working_mode, &dzpcheck_privs, B_FALSE, cr); + if (dzp_error == EACCES) { + /* We hit a DENY ACE. */ + if (!dzpcheck_privs) + return (SET_ERROR(dzp_error)); + return (secpolicy_vnode_remove(ZTOV(zp), cr)); + } + + /* + * Cases 2a, 2b (continued) + * + * Note: dzp_working_mode now contains any permissions + * that were NOT granted. Therefore, if any of the + * wanted_dirperms WERE granted, we will have: + * dzp_working_mode != wanted_dirperms + * We're really asking if ANY of those permissions + * were granted, and if so, grant delete access. + */ + if (dzp_working_mode != wanted_dirperms) + dzp_error = 0; + + /* + * dzp_error is 0 if the container granted us permissions to "modify". + * If we do not have permission via one or more ACEs, our current + * privileges may still permit us to modify the container. + * + * dzpcheck_privs is false when i.e. the FS is read-only. + * Otherwise, do privilege checks for the container. + */ + if (dzp_error != 0 && dzpcheck_privs) { + uid_t owner; + /* + * The secpolicy call needs the requested access and + * the current access mode of the container, but it + * only knows about Unix-style modes (VEXEC, VWRITE), + * so this must condense the fine-grained ACE bits into + * Unix modes. + * + * The VEXEC flag is easy, because we know that has + * always been checked before we get here (during the + * lookup of the target vnode). The container has not + * granted us permissions to "modify", so we do not set + * the VWRITE flag in the current access mode. + */ + owner = zfs_fuid_map_id(dzp->z_zfsvfs, dzp->z_uid, cr, + ZFS_OWNER); + dzp_error = secpolicy_vnode_access2(cr, ZTOV(dzp), + owner, VEXEC, VWRITE|VEXEC); + } + if (dzp_error != 0) { + /* + * Note: We may have dzp_error = -1 here (from + * zfs_zacess_common). Don't return that. + */ + return (SET_ERROR(EACCES)); + } + + /* + * At this point, we know that the directory permissions allow + * us to modify, but we still need to check for the additional + * restrictions that apply when the "sticky bit" is set. + * + * Yes, zfs_sticky_remove_access() also checks this bit, but + * checking it here and skipping the call below is nice when + * you're watching all of this with dtrace. + */ + if ((dzp->z_mode & S_ISVTX) == 0) + return (0); + /* + * zfs_sticky_remove_access will succeed if: + * 1. The sticky bit is absent. + * 2. We pass the sticky bit restrictions. + * 3. We have privileges that always allow file removal. + */ + return (zfs_sticky_remove_access(dzp, zp, cr)); +} + +int +zfs_zaccess_rename(znode_t *sdzp, znode_t *szp, znode_t *tdzp, + znode_t *tzp, cred_t *cr, zidmap_t *mnt_ns) +{ + int add_perm; + int error; + + if (szp->z_pflags & ZFS_AV_QUARANTINED) + return (SET_ERROR(EACCES)); + + add_perm = (vnode_isdir(ZTOV(szp))) ? + ACE_ADD_SUBDIRECTORY : ACE_ADD_FILE; + + /* + * Rename permissions are combination of delete permission + + * add file/subdir permission. + * + * BSD operating systems also require write permission + * on the directory being moved from one parent directory + * to another. + */ + if (vnode_isdir(ZTOV(szp)) && ZTOV(sdzp) != ZTOV(tdzp)) { + if ((error = zfs_zaccess(szp, ACE_WRITE_DATA, 0, B_FALSE, cr, + NULL))) + return (error); + } + + /* + * first make sure we do the delete portion. + * + * If that succeeds then check for add_file/add_subdir permissions + */ + + if ((error = zfs_zaccess_delete(sdzp, szp, cr, mnt_ns))) + return (error); + + /* + * If we have a tzp, see if we can delete it? + */ + if (tzp) { + if ((error = zfs_zaccess_delete(tdzp, tzp, cr, mnt_ns))) + return (error); + } + + /* + * Now check for add permissions + */ + error = zfs_zaccess(tdzp, add_perm, 0, B_FALSE, cr, NULL); + + return (error); +} diff --git a/module/os/windows/zfs/zfs_ctldir.c b/module/os/windows/zfs/zfs_ctldir.c new file mode 100644 index 000000000000..07e1dac44c3c --- /dev/null +++ b/module/os/windows/zfs/zfs_ctldir.c @@ -0,0 +1,1425 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (C) 2011 Lawrence Livermore National Security, LLC. + * Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). + * LLNL-CODE-403049. + * Rewritten for Linux by: + * Rohan Puri + * Brian Behlendorf + * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright 2015, OmniTI Computer Consulting, Inc. All rights reserved. + * Copyright (c) 2018 George Melikov. All Rights Reserved. + * Copyright (c) 2019 Datto, Inc. All rights reserved. + * Copyright (c) 2020 Jorgen Lundman. All rights reserved. + * Portions Copyright 2022 Andrew Innes + */ + +/* + * ZFS control directory (a.k.a. ".zfs") + * + * This directory provides a common location for all ZFS meta-objects. + * Currently, this is only the 'snapshot' and 'shares' directory, but this may + * expand in the future. The elements are built dynamically, as the hierarchy + * does not actually exist on disk. + * + * For 'snapshot', we don't want to have all snapshots always mounted, because + * this would take up a huge amount of space in /etc/mnttab. We have three + * types of objects: + * + * ctldir ------> snapshotdir -------> snapshot + * | + * | + * V + * mounted fs + * + * The 'snapshot' node contains just enough information to lookup '..' and act + * as a mountpoint for the snapshot. Whenever we lookup a specific snapshot, we + * perform an automount of the underlying filesystem and return the + * corresponding vnode. + * + * All mounts are handled automatically by an user mode helper which invokes + * the mount procedure. Unmounts are handled by allowing the mount + * point to expire so the kernel may automatically unmount it. + * + * The '.zfs', '.zfs/snapshot', and all directories created under + * '.zfs/snapshot' (ie: '.zfs/snapshot/') all share the same + * zfsvfs_t as the head filesystem (what '.zfs' lives under). + * + * File systems mounted on top of the '.zfs/snapshot/' paths + * (ie: snapshots) are complete ZFS filesystems and have their own unique + * zfsvfs_t. However, the fsid reported by these mounts will be the same + * as that used by the parent zfsvfs_t to make NFS happy. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "zfs_namecheck.h" + +extern kmem_cache_t *znode_cache; +extern uint64_t vnop_num_vnodes; + +/* + * Windows differences; + * + * We don't have 'shares' directory, so only 'snapshot' is relevant. + * + * - see zfs_ctldir_snapdir.c + * + * All vnodes point to znode_t, no special case nodes. + */ + + + +/* List of zfsctl mounts waiting to be mounted */ +static kmutex_t zfsctl_mounts_lock; +static list_t zfsctl_mounts_list; +struct zfsctl_mounts_waiting { + kmutex_t zcm_lock; + kcondvar_t zcm_cv; + list_node_t zcm_node; + char zcm_name[ZFS_MAX_DATASET_NAME_LEN]; +}; +typedef struct zfsctl_mounts_waiting zfsctl_mounts_waiting_t; + + +/* + * Control Directory Tunables (.zfs) + */ +int zfs_expire_snapshot = ZFSCTL_EXPIRE_SNAPSHOT; +int zfs_admin_snapshot = 1; +int zfs_auto_snapshot = 1; + +static kmutex_t zfsctl_unmount_lock; +static kcondvar_t zfsctl_unmount_cv; +static boolean_t zfsctl_unmount_thread_exit; + +static kmutex_t zfsctl_unmount_list_lock; +static list_t zfsctl_unmount_list; + +struct zfsctl_unmount_delay { + char *se_name; /* full snapshot name */ + spa_t *se_spa; /* pool spa */ + struct vnode *se_vnode; + uint64_t se_objsetid; /* snapshot objset id */ + time_t se_time; + list_node_t se_nodelink; +}; +typedef struct zfsctl_unmount_delay zfsctl_unmount_delay_t; + + +/* + * Check if the given vnode is a part of the virtual .zfs directory. + */ +boolean_t +zfsctl_is_node(znode_t *zp) +{ + return (zp->z_is_ctldir); +} + +/* + * /.zfs/snapshot/nameofsnapshot/ + * -------------------^ + */ +boolean_t +zfsctl_is_leafnode(znode_t *zp) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + return (zp->z_is_ctldir && zp->z_id != ZFSCTL_INO_ROOT && + zp->z_id != ZFSCTL_INO_SNAPDIR && + (zp->z_id >= zfsvfs->z_ctldir_startid) && + (zp->z_id <= ZFSCTL_INO_SNAPDIRS)); +} + +typedef int (**vnode_operations)(void *); + + +/* + * Allocate a new vnode with the passed id and ops. + */ +static struct vnode * +zfsctl_vnode_alloc(zfsvfs_t *zfsvfs, uint64_t id, + char *name) +{ + timestruc_t now; + struct vnode *vp = NULL; + znode_t *zp = NULL; + int flags = 0; + + dprintf("%s\n", __func__); + + zp = kmem_cache_alloc(znode_cache, KM_SLEEP); + + gethrestime(&now); + ASSERT3P(zp->z_dirlocks, ==, NULL); + ASSERT3P(zp->z_acl_cached, ==, NULL); + ASSERT3P(zp->z_xattr_cached, ==, NULL); + zp->z_zfsvfs = zfsvfs; + zp->z_id = id; + zp->z_unlinked = B_FALSE; + zp->z_atime_dirty = B_FALSE; + zp->z_zn_prefetch = B_FALSE; + zp->z_is_sa = B_FALSE; + zp->z_is_mapped = B_FALSE; + zp->z_is_ctldir = B_TRUE; + zp->z_sa_hdl = NULL; + zp->z_blksz = 0; + zp->z_seq = 0; + zp->z_mapcnt = 0; + zp->z_size = 0; + zp->z_pflags = 0; + zp->z_mode = 0; + zp->z_sync_cnt = 0; + zp->z_gen = 0; + zp->z_links = 2; + zp->z_mode = (S_IFDIR | (S_IRWXU|S_IRWXG|S_IRWXO)); + zp->z_uid = 0; + zp->z_gid = 0; + ZFS_TIME_ENCODE(&now, zp->z_atime); + + zp->z_snap_mount_time = 0; /* Allow automount attempt */ + zp->z_name_cache = NULL; + zp->z_name_len = 0; + + dprintf("%s zp %p with vp %p zfsvfs %p vfs %p\n", __func__, + zp, vp, zfsvfs, zfsvfs->z_vfs); + + /* Tag root directory */ + if (id == ZFSCTL_INO_ROOT) + flags |= VNODE_MARKROOT; + + /* + * This creates a vnode with VSYSTEM set, this is so that unmount's + * vflush() (called before our vfs_unmount) will pass (and not block + * waiting for the usercount ref to be released). We then release the + * VROOT vnode in zfsctl_destroy, and release the usercount ref. + * Because of this, we need to call vnode_recycle() ourselves in destroy + */ + + + vnode_create(zfsvfs->z_vfs, zp, VDIR, flags, &vp); + + dprintf("Assigned zp %p with vp %p zfsvfs %p\n", zp, vp, zp->z_zfsvfs); + + zp->z_vid = vnode_vid(vp); + zp->z_vnode = vp; + + // Update startid before buildpath. + if (id < zfsvfs->z_ctldir_startid) + zfsvfs->z_ctldir_startid = id; + + // Build fullpath string here, for Notifications & set_name_information + if (zfs_build_path(zp, NULL, &zp->z_name_cache, + &zp->z_name_len, &zp->z_name_offset) == -1) { + dprintf("%s: failed to build fullpath\n", __func__); + } + + zfs_set_security(vp, NULL); + + mutex_enter(&zfsvfs->z_znodes_lock); + list_insert_tail(&zfsvfs->z_all_znodes, zp); + membar_producer(); + mutex_exit(&zfsvfs->z_znodes_lock); + + return (vp); +} + +/* + * Lookup the vnode with given id, it will be allocated if needed. + */ +static struct vnode * +zfsctl_vnode_lookup(zfsvfs_t *zfsvfs, uint64_t id, + char *name) +{ + struct vnode *ip = NULL; + int error = 0; + + dprintf("%s\n", __func__); + + while (ip == NULL) { + + error = zfs_vfs_vget(zfsvfs->z_vfs, id, &ip, NULL); + if (error == 0 && ip != NULL) + break; + + /* May fail due to concurrent zfsctl_vnode_alloc() */ + ip = zfsctl_vnode_alloc(zfsvfs, id, name); + } + + return (ip); +} + +/* + * Create the '.zfs' directory. This directory is cached as part of the VFS + * structure. This results in a hold on the zfsvfs_t. The code in zfs_umount() + * therefore checks against a vfs_count of 2 instead of 1. This reference + * is removed when the ctldir is destroyed in the unmount. All other entities + * under the '.zfs' directory are created dynamically as needed. + * + * Because the dynamically created '.zfs' directory entries assume the use + * of 64-bit vnode numbers this support must be disabled on 32-bit systems. + */ +int +zfsctl_create(zfsvfs_t *zfsvfs) +{ + ASSERT(zfsvfs->z_ctldir == NULL); + + dprintf("%s\n", __func__); + + /* Create root node, tagged with VSYSTEM - see above */ + zfsvfs->z_ctldir = zfsctl_vnode_alloc(zfsvfs, ZFSCTL_INO_ROOT, + ZFS_CTLDIR_NAME); + if (zfsvfs->z_ctldir == NULL) + return (SET_ERROR(ENOENT)); + + vnode_ref(zfsvfs->z_ctldir); + VN_RELE(zfsvfs->z_ctldir); + + dprintf("%s: done %p\n", __func__, zfsvfs->z_ctldir); + + return (0); +} + +/* + * Destroy the '.zfs' directory or remove a snapshot from + * zfs_snapshots_by_name. Only called when the filesystem is unmounted. + */ +void +zfsctl_destroy(zfsvfs_t *zfsvfs) +{ + if (zfsvfs->z_ctldir) { + if (VN_HOLD(zfsvfs->z_ctldir) == 0) { + vnode_rele(zfsvfs->z_ctldir); + /* Because tagged VSYSTEM, we manually call recycle */ + zfsvfs->z_ctldir->v_flags &= ~VNODE_MARKROOT; // Hack + VN_RELE(zfsvfs->z_ctldir); + } + zfsvfs->z_ctldir = NULL; + } +} + +/* + * Given a root znode, retrieve the associated .zfs directory. + * Add a hold to the vnode and return it. + */ +struct vnode * +zfsctl_root(znode_t *zp) +{ + VN_HOLD(ZTOZSB(zp)->z_ctldir); + return (ZTOZSB(zp)->z_ctldir); +} + +/* Given a entry in .zfs, find its parent */ +struct vnode * +zfs_root_dotdot(struct vnode *vp) +{ + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + znode_t *retzp = NULL; + struct vnode *retvp = NULL; + int error = 0; + + dprintf("%s: for id %llu\n", __func__, zp->z_id); + + if (zp->z_id == ZFSCTL_INO_ROOT) + error = zfs_zget(zfsvfs, zfsvfs->z_root, &retzp); + else if (zp->z_id == ZFSCTL_INO_SNAPDIR) + retvp = zfsctl_root(zp); + else + retvp = zfsctl_vnode_lookup(zfsvfs, ZFSCTL_INO_SNAPDIR, + ZFS_SNAPDIR_NAME); + + if (retzp != NULL) + return (ZTOV(retzp)); + if (retvp != NULL) + return (retvp); + + return (NULL); +} + +/* + * Special case the handling of "..". + */ +int +zfsctl_root_lookup(struct vnode *dvp, char *name, znode_t **zpp, + int flags, cred_t *cr, int *direntflags, struct componentname *realpnp) +{ + znode_t *dzp = VTOZ(dvp); + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + int error = 0; + uint64_t id; + struct vnode *vp; + + dprintf("%s: '%s'\n", __func__, name); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + if (strcmp(name, "..") == 0) { + vp = zfs_root_dotdot(dvp); + } else if (strcmp(name, ZFS_SNAPDIR_NAME) == 0) { + vp = zfsctl_vnode_lookup(zfsvfs, ZFSCTL_INO_SNAPDIR, + name); + } else { + error = dmu_snapshot_lookup(zfsvfs->z_os, name, &id); + if (error != 0) + goto out; + vp = zfsctl_vnode_lookup(zfsvfs, ZFSCTL_INO_SNAPDIRS - id, + name); + } + + if (vp == NULL) + error = SET_ERROR(ENOENT); + else + *zpp = VTOZ(vp); + +out: + zfs_exit(zfsvfs, FTAG); + + return (error); +} + +int +zfsctl_vnop_readdir_root(vnode_t *vp, emitdir_ptr_t *ctx, cred_t *cr, + zfs_dirlist_t *zccb, int flags) +{ + int error = 0; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + dprintf("%s\n", __func__); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + ctx->numdirent = 0; + + while (ctx->offset < 3 && error == 0) { + + switch (ctx->offset) { + case 0: /* "." */ + error = zfs_readdir_emitdir(zfsvfs, ".", + ctx, zccb, ZFSCTL_INO_ROOT); + break; + + case 1: /* ".." */ + error = zfs_readdir_emitdir(zfsvfs, "..", + ctx, zccb, zfsvfs->z_root); + break; + + case 2: + error = zfs_readdir_emitdir(zfsvfs, ZFS_SNAPDIR_NAME, + ctx, zccb, ZFSCTL_INO_SNAPDIR); + break; + } + + if (error == 0) { + ctx->numdirent++; + } else if (error == ENOSPC) { /* stop iterating */ + break; + } else { + /* other error, skip over entry */ + error = 0; + } + + ctx->offset++; + } + + if (error == ENOENT) { + dprintf("end of snapshots reached\n"); + } + + if (error != 0) { + dprintf("emit error\n"); + } + + zfs_readdir_complete(ctx); + + /* Finished without error? Set EOF */ + if (ctx->offset >= 3 && error == 0) { + error = ENOENT; + dprintf("Setting eof\n"); + } + + zfs_exit(zfsvfs, FTAG); + + return (error); +} + +int +zfsctl_vnop_readdir_snapdir(vnode_t *vp, emitdir_ptr_t *ctx, cred_t *cr, + zfs_dirlist_t *zccb, int flags) +{ + int error = 0; + boolean_t case_conflict; + uint64_t id; + char snapname[MAXNAMELEN]; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + dprintf("%s\n", __func__); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + while (error == 0) { + + switch (ctx->offset) { + case 0: /* "." */ + error = zfs_readdir_emitdir(zfsvfs, ".", + ctx, zccb, ZFSCTL_INO_SNAPDIR); + break; + + case 1: /* ".." */ + error = zfs_readdir_emitdir(zfsvfs, "..", + ctx, zccb, ZFSCTL_INO_ROOT); + break; + + default: + dsl_pool_config_enter(dmu_objset_pool(zfsvfs->z_os), + FTAG); + error = dmu_snapshot_list_next(zfsvfs->z_os, + MAXNAMELEN, snapname, &id, &ctx->offset, + &case_conflict); + dsl_pool_config_exit(dmu_objset_pool(zfsvfs->z_os), + FTAG); + if (error) + break; + + error = zfs_readdir_emitdir(zfsvfs, snapname, + ctx, zccb, ZFSCTL_INO_SNAPDIRS - id); + break; + } + + if (error != 0) { + dprintf("emit error\n"); + break; + } + + ctx->offset++; + } + + zfs_readdir_complete(ctx); + + zfs_exit(zfsvfs, FTAG); + + return (error); +} + + +/* We need to spit out a valid "." ".." entries for mount to work */ +int +zfsctl_vnop_readdir_snapdirs(vnode_t *vp, emitdir_ptr_t *ctx, cred_t *cr, + zfs_dirlist_t *zccb, int flags) +{ + int error = 0; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + while (error == 0) { + + switch (ctx->offset) { + case 0: /* "." */ + error = zfs_readdir_emitdir(zfsvfs, ".", + ctx, zccb, zp->z_id); + break; + + case 1: /* ".." */ + error = zfs_readdir_emitdir(zfsvfs, "..", + ctx, zccb, ZFSCTL_INO_SNAPDIR); + break; + + default: + error = ENOENT; + break; + } + + if (error != 0) { + dprintf("emit error\n"); + break; + } + + ctx->offset++; + } + + zfs_exit(zfsvfs, FTAG); + + zfs_readdir_complete(ctx); + + return (error); +} + +int +zfsctl_readdir(vnode_t *vp, emitdir_ptr_t *ctx, cred_t *cr, + zfs_dirlist_t *zccb, int flags) +{ + znode_t *zp = VTOZ(vp); + + dprintf("%s\n", __func__); + + /* Which directory are we to output? */ + switch (zp->z_id) { + case ZFSCTL_INO_ROOT: + return (zfsctl_vnop_readdir_root(vp, ctx, cr, + zccb, flags)); + case ZFSCTL_INO_SNAPDIR: + return (zfsctl_vnop_readdir_snapdir(vp, ctx, cr, + zccb, flags)); + default: + return (zfsctl_vnop_readdir_snapdirs(vp, ctx, cr, + zccb, flags)); + break; + } + panic("%s: weird snapshot state\n", __func__); + return (EINVAL); +} + + +int +zfsctl_vnop_getattr(struct vnop_getattr_args *ap) +#if 0 + struct vnop_getattr_args { + struct vnode *a_vp; + struct vnode_vattr *a_vap; + vfs_context_t a_context; + }; +#endif +{ + vattr_t *vap = ap->a_vap; + struct vnode *vp = ap->a_vp; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + timestruc_t now; + int error = 0; + + dprintf("%s: active x%llx\n", __func__, vap->va_active); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + gethrestime(&now); +#if 0 // WIN32 me + if (VATTR_IS_ACTIVE(vap, va_rdev)) + VATTR_RETURN(vap, va_rdev, zfsvfs->z_rdev); + if (VATTR_IS_ACTIVE(vap, va_nlink)) + VATTR_RETURN(vap, va_nlink, + vnode_isdir(vp) ? zp->z_size : zp->z_links); + if (VATTR_IS_ACTIVE(vap, va_total_size)) + VATTR_RETURN(vap, va_total_size, 512); + if (VATTR_IS_ACTIVE(vap, va_total_alloc)) + VATTR_RETURN(vap, va_total_alloc, 512); + if (VATTR_IS_ACTIVE(vap, va_data_size)) + VATTR_RETURN(vap, va_data_size, 0); + if (VATTR_IS_ACTIVE(vap, va_data_alloc)) + VATTR_RETURN(vap, va_data_alloc, 0); + if (VATTR_IS_ACTIVE(vap, va_iosize)) + VATTR_RETURN(vap, va_iosize, 512); + if (VATTR_IS_ACTIVE(vap, va_uid)) + VATTR_RETURN(vap, va_uid, 0); + if (VATTR_IS_ACTIVE(vap, va_gid)) + VATTR_RETURN(vap, va_gid, 0); + if (VATTR_IS_ACTIVE(vap, va_mode)) + VATTR_RETURN(vap, va_mode, S_IFDIR | + S_IRUSR | S_IXUSR | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH); + if (VATTR_IS_ACTIVE(vap, va_flags)) + VATTR_RETURN(vap, va_flags, zfs_getbsdflags(zp)); + + if (VATTR_IS_ACTIVE(vap, va_acl)) { + VATTR_RETURN(vap, va_uuuid, kauth_null_guid); + VATTR_RETURN(vap, va_guuid, kauth_null_guid); + VATTR_RETURN(vap, va_acl, NULL); + } + + // crtime, atime, mtime, ctime, btime + uint64_t timez[2]; + timez[0] = zfsvfs->z_mount_time; + timez[1] = 0; + + if (VATTR_IS_ACTIVE(vap, va_create_time)) { + ZFS_TIME_DECODE(&vap->va_create_time, timez); + VATTR_SET_SUPPORTED(vap, va_create_time); + } + if (VATTR_IS_ACTIVE(vap, va_access_time)) { + ZFS_TIME_DECODE(&vap->va_access_time, timez); + VATTR_SET_SUPPORTED(vap, va_access_time); + } + if (VATTR_IS_ACTIVE(vap, va_modify_time)) { + ZFS_TIME_DECODE(&vap->va_modify_time, timez); + VATTR_SET_SUPPORTED(vap, va_modify_time); + } + if (VATTR_IS_ACTIVE(vap, va_change_time)) { + ZFS_TIME_DECODE(&vap->va_change_time, timez); + VATTR_SET_SUPPORTED(vap, va_change_time); + } + if (VATTR_IS_ACTIVE(vap, va_backup_time)) { + ZFS_TIME_DECODE(&vap->va_backup_time, timez); + VATTR_SET_SUPPORTED(vap, va_backup_time); + } + if (VATTR_IS_ACTIVE(vap, va_addedtime)) { + ZFS_TIME_DECODE(&vap->va_addedtime, timez); + VATTR_SET_SUPPORTED(vap, va_addedtime); + } + + if (VATTR_IS_ACTIVE(vap, va_fileid)) + VATTR_RETURN(vap, va_fileid, zp->z_id); + if (VATTR_IS_ACTIVE(vap, va_linkid)) + VATTR_RETURN(vap, va_linkid, zp->z_id); + if (VATTR_IS_ACTIVE(vap, va_parentid)) { + switch (zp->z_id) { + case ZFSCTL_INO_ROOT: + // ".zfs" parent is mount, 2 on osx + VATTR_RETURN(vap, va_linkid, 2); + break; + case ZFSCTL_INO_SNAPDIR: + // ".zfs/snapshot" parent is ".zfs" + VATTR_RETURN(vap, va_linkid, ZFSCTL_INO_ROOT); + break; + default: + // ".zfs/snapshot/$name" parent ".zfs/snapshot" + VATTR_RETURN(vap, va_linkid, + ZFSCTL_INO_SNAPDIR); + break; + } + } + if (VATTR_IS_ACTIVE(vap, va_fsid)) + VATTR_RETURN(vap, va_fsid, zfsvfs->z_rdev); + + if (VATTR_IS_ACTIVE(vap, va_filerev)) + VATTR_RETURN(vap, va_filerev, 0); + if (VATTR_IS_ACTIVE(vap, va_gen)) + VATTR_RETURN(vap, va_gen, zp->z_gen); + if (VATTR_IS_ACTIVE(vap, va_type)) + VATTR_RETURN(vap, va_type, vnode_vtype(ZTOV(zp))); + if (VATTR_IS_ACTIVE(vap, va_name)) { + strlcpy(vap->va_name, zp->z_name_cache, MAXPATHLEN); + VATTR_SET_SUPPORTED(vap, va_name); + } + + /* Don't include '.' and '..' in the number of entries */ + if (VATTR_IS_ACTIVE(vap, va_nchildren) && vnode_isdir(vp)) { + VATTR_RETURN(vap, va_nchildren, + zp->z_links > 3 ? zp->z_links-2 : 1); + } + if (VATTR_IS_ACTIVE(vap, va_dirlinkcount) && vnode_isdir(vp)) + VATTR_RETURN(vap, va_dirlinkcount, 1); + +#ifdef VNODE_ATTR_va_fsid64 + if (VATTR_IS_ACTIVE(vap, va_fsid64)) { + vap->va_fsid64.val[0] = + vfs_statfs(zfsvfs->z_vfs)->f_fsid.val[0]; + vap->va_fsid64.val[1] = vfs_typenum(zfsvfs->z_vfs); + VATTR_SET_SUPPORTED(vap, va_fsid64); + } +#endif +#endif + + zfs_exit(zfsvfs, FTAG); + + dprintf("%s: returned x%llx missed: x%llx\n", __func__, + vap->va_supported, vap->va_active &= ~vap->va_supported); + return (0); +} + +int +zfsctl_vnop_access(struct vnop_access_args *ap) +{ + int accmode = ap->a_action; + dprintf("zfsctl_access\n"); + + if (accmode & VWRITE) + return (EACCES); + return (0); +} + +int +zfsctl_vnop_open(struct vnop_open_args *ap) +{ + int flags = ap->a_mode; + + if (flags & FWRITE) + return (EACCES); + return (zfsctl_snapshot_mount(ap->a_vp, 0)); +} + +int +zfsctl_vnop_close(struct vnop_close_args *ap) +{ + dprintf("%s\n", __func__); + return (0); +} + +int +zfsctl_vnop_inactive(struct vnop_inactive_args *ap) +{ + dprintf("%s\n", __func__); + return (0); +} + +int +zfsctl_vnop_reclaim(struct vnop_reclaim_args *ap) +{ + struct vnode *vp = ap->a_vp; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + dprintf("%s vp %p\n", __func__, vp); + vnode_removefsref(vp); /* ADDREF from vnode_create */ + vnode_clearfsnode(vp); /* vp->v_data = NULL */ + + mutex_enter(&zfsvfs->z_znodes_lock); + if (list_link_active(&zp->z_link_node)) { + list_remove(&zfsvfs->z_all_znodes, zp); + } + mutex_exit(&zfsvfs->z_znodes_lock); + + zp->z_vnode = NULL; + kmem_cache_free(znode_cache, zp); + + return (0); +} + +/* + * Construct a full dataset name in full_name: "pool/dataset@snap_name" + */ +static int +zfsctl_snapshot_name(zfsvfs_t *zfsvfs, const char *snap_name, int len, + char *full_name) +{ + objset_t *os = zfsvfs->z_os; + + if (zfs_component_namecheck(snap_name, NULL, NULL) != 0) + return (SET_ERROR(EILSEQ)); + + dmu_objset_name(os, full_name); + if ((strlen(full_name) + 1 + strlen(snap_name)) >= len) + return (SET_ERROR(ENAMETOOLONG)); + + (void) strcat(full_name, "@"); + (void) strcat(full_name, snap_name); + + return (0); +} + +int +zfsctl_snapshot_mount(struct vnode *vp, int flags) +{ + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + int ret = 0; + int error = 0; + /* + * If we are here for a snapdirs directory, attempt to get zed + * to mount the snapshot for the user. If successful, forward the + * vnop_open() to them (ourselves). + * Use a timeout in case zed is not running. + */ + + if (zfs_auto_snapshot == 0) + return (0); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + if (((zp->z_id >= zfsvfs->z_ctldir_startid) && + (zp->z_id <= ZFSCTL_INO_SNAPDIRS))) { + hrtime_t now; + now = gethrtime(); + + /* + * If z_snap_mount_time is set, check if it is old enough to + * retry, if so, set z_snap_mount_time to zero. + */ + if (now - zp->z_snap_mount_time > SEC2NSEC(60)) + atomic_cas_64((uint64_t *)&zp->z_snap_mount_time, + (uint64_t)zp->z_snap_mount_time, + 0ULL); + + /* + * Attempt mount, make sure only to issue one request, by + * attempting to CAS in current time in place of zero. + */ + if (atomic_cas_64((uint64_t *)&zp->z_snap_mount_time, 0ULL, + (uint64_t)now) == 0ULL) { + char full_name[ZFS_MAX_DATASET_NAME_LEN]; + + /* First! */ + ret = zfsctl_snapshot_name(zfsvfs, zp->z_name_cache, + ZFS_MAX_DATASET_NAME_LEN, full_name); + + if (ret == 0) { + zfsctl_mounts_waiting_t *zcm; + + /* Create condvar to wait for mount to happen */ + + zcm = kmem_alloc( + sizeof (zfsctl_mounts_waiting_t), KM_SLEEP); + mutex_init(&zcm->zcm_lock, NULL, MUTEX_DEFAULT, + NULL); + cv_init(&zcm->zcm_cv, NULL, CV_DEFAULT, NULL); + strlcpy(zcm->zcm_name, full_name, + sizeof (zcm->zcm_name)); + + dprintf("%s: requesting mount for '%s'\n", + __func__, full_name); + + mutex_enter(&zfsctl_mounts_lock); + list_insert_tail(&zfsctl_mounts_list, zcm); + mutex_exit(&zfsctl_mounts_lock); + + mutex_enter(&zcm->zcm_lock); + zfs_ereport_snapshot_post( + FM_EREPORT_ZFS_SNAPSHOT_MOUNT, + dmu_objset_spa(zfsvfs->z_os), full_name); + + /* Now we wait hoping zed comes back to us */ + ret = cv_timedwait(&zcm->zcm_cv, &zcm->zcm_lock, + ddi_get_lbolt() + (hz * 3)); + + dprintf("%s: finished waiting %d\n", + __func__, ret); + + mutex_exit(&zcm->zcm_lock); + + mutex_enter(&zfsctl_mounts_lock); + list_remove(&zfsctl_mounts_list, zcm); + mutex_exit(&zfsctl_mounts_lock); + + mutex_destroy(&zcm->zcm_lock); + cv_destroy(&zcm->zcm_cv); + + kmem_free(zcm, + sizeof (zfsctl_mounts_waiting_t)); + + /* + * If we mounted, make it re-open it so + * the process that issued the access will + * see the mounted content + */ + if (ret >= 0) { + /* Remove the cache entry */ + cache_purge(vp); + cache_purge_negatives(vp); + ret = ERESTART; + } + } + } + } + + zfs_exit(zfsvfs, FTAG); + + return (ret); +} + +/* Called whenever zfs_vfs_mount() is called with a snapshot */ +void +zfsctl_mount_signal(zfsvfs_t *zfsvfs, char *osname, boolean_t mounting) +{ + zfsctl_mounts_waiting_t *zcm; + + dprintf("%s: looking for snapshot '%s'\n", __func__, osname); + + mutex_enter(&zfsctl_mounts_lock); + for (zcm = list_head(&zfsctl_mounts_list); + zcm; + zcm = list_next(&zfsctl_mounts_list, zcm)) { + if (strncmp(zcm->zcm_name, osname, sizeof (zcm->zcm_name)) == 0) + break; + } + mutex_exit(&zfsctl_mounts_lock); + + /* Is there someone to wake up? */ + if (zcm != NULL) { + mutex_enter(&zcm->zcm_lock); + cv_signal(&zcm->zcm_cv); + mutex_exit(&zcm->zcm_lock); + dprintf("%s: mount waiter found and signalled\n", __func__); + } + + zfsctl_unmount_delay_t *zcu; + + /* Add or remove mount to/from list of active mounts */ + + if (mounting) { + /* Add active mounts to the list */ + zcu = kmem_alloc(sizeof (zfsctl_unmount_delay_t), KM_SLEEP); + zcu->se_name = kmem_strdup(osname); + zcu->se_time = gethrestime_sec(); + list_link_init(&zcu->se_nodelink); + + mutex_enter(&zfsctl_unmount_list_lock); + list_insert_tail(&zfsctl_unmount_list, zcu); + mutex_exit(&zfsctl_unmount_list_lock); + + } else { + /* Unmounting */ + mutex_enter(&zfsctl_unmount_list_lock); + for (zcu = list_head(&zfsctl_unmount_list); + zcu != NULL; + zcu = list_next(&zfsctl_unmount_list, zcu)) { + if (strcmp(osname, zcu->se_name) == 0) + break; + } + mutex_exit(&zfsctl_unmount_list_lock); + + + if (zcu != NULL) { + list_remove(&zfsctl_unmount_list, zcu); + kmem_strfree(zcu->se_name); + kmem_free(zcu, sizeof (zfsctl_unmount_delay_t)); + } + } +} + +int +zfsctl_snapshot_unmount_node(struct vnode *vp, const char *full_name, + int flags) +{ + znode_t *zp = VTOZ(vp); + int error = 0; + + dprintf("%s\n", __func__); + + if (zp == NULL) + return (ENOENT); + return (0); + + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + int ret = ENOENT; + /* + * If we are here for a snapdirs directory, attempt to get zed + * to mount the snapshot for the user. If successful, forward the + * vnop_open() to them (ourselves). + * Use a timeout in case zed is not running. + */ + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + if (zp->z_id == zfsvfs->z_root) { + hrtime_t now; + now = gethrtime(); + + /* + * If z_snap_mount_time is set, check if it is old enough to + * retry, if so, set z_snap_mount_time to zero. + */ + if (now - zp->z_snap_mount_time > SEC2NSEC(60)) + atomic_cas_64((uint64_t *)&zp->z_snap_mount_time, + (uint64_t)zp->z_snap_mount_time, + 0ULL); + + /* + * Attempt unmount, make sure only to issue one request, by + * attempting to CAS in current time in place of zero. + */ + if (atomic_cas_64((uint64_t *)&zp->z_snap_mount_time, 0ULL, + (uint64_t)now) == 0ULL) { + + ret = 0; + + /* First! */ + + if (ret == 0) { + zfsctl_mounts_waiting_t *zcm; + + /* Create condvar to wait for mount to happen */ + + zcm = kmem_alloc( + sizeof (zfsctl_mounts_waiting_t), KM_SLEEP); + mutex_init(&zcm->zcm_lock, NULL, MUTEX_DEFAULT, + NULL); + cv_init(&zcm->zcm_cv, NULL, CV_DEFAULT, NULL); + strlcpy(zcm->zcm_name, full_name, + sizeof (zcm->zcm_name)); + + dprintf("%s: requesting unmount for '%s'\n", + __func__, full_name); + + mutex_enter(&zfsctl_mounts_lock); + list_insert_tail(&zfsctl_mounts_list, zcm); + mutex_exit(&zfsctl_mounts_lock); + + mutex_enter(&zcm->zcm_lock); + zfs_ereport_snapshot_post( + FM_EREPORT_ZFS_SNAPSHOT_UNMOUNT, + dmu_objset_spa(zfsvfs->z_os), full_name); + + /* Now we wait hoping zed comes back to us */ + ret = cv_timedwait(&zcm->zcm_cv, &zcm->zcm_lock, + ddi_get_lbolt() + (hz * 3)); + + dprintf("%s: finished waiting %d\n", + __func__, ret); + + mutex_exit(&zcm->zcm_lock); + + mutex_enter(&zfsctl_mounts_lock); + list_remove(&zfsctl_mounts_list, zcm); + mutex_exit(&zfsctl_mounts_lock); + + kmem_free(zcm, + sizeof (zfsctl_mounts_waiting_t)); + + /* Allow mounts to happen immediately */ + zp->z_snap_mount_time = 0; + + /* + * If we unmounted, alert caller + */ + if (ret >= 0) + ret = 0; + + } + } + } + + zfs_exit(zfsvfs, FTAG); + + return (ret); +} + +int +zfsctl_snapshot_unmount(const char *snapname, int flags) +{ + znode_t *rootzp; + zfsvfs_t *zfsvfs; + + dprintf("%s\n", __func__); + + if (strchr(snapname, '@') == NULL) + return (0); + + int err = getzfsvfs(snapname, &zfsvfs); + if (err != 0) { + ASSERT3P(zfsvfs, ==, NULL); + return (0); + } + ASSERT(!dsl_pool_config_held(dmu_objset_pool(zfsvfs->z_os))); + + err = zfs_zget(zfsvfs, zfsvfs->z_root, &rootzp); + if (err == 0) { + zfsctl_snapshot_unmount_node(ZTOV(rootzp), snapname, flags); + VN_RELE(ZTOV(rootzp)); + } + + vfs_unbusy(zfsvfs->z_vfs); + return (0); +} + +int +zfsctl_mkdir(znode_t *dzp, znode_t **zpp, char *dirname) +{ + int error; + error = zfsctl_root_lookup(ZTOV(dzp), dirname, zpp, 0, NULL, + NULL, NULL); + if (error == 0) { + ZTOV(*zpp)->v_unlink = 0; + } + return (error); +} + +int +zfsctl_set_reparse_point(znode_t *zp, REPARSE_DATA_BUFFER *rdb, size_t size) +{ + if (!zfsctl_is_leafnode(zp)) + return (STATUS_INVALID_PARAMETER); + + zp->z_pflags |= ZFS_REPARSE; + vnode_set_reparse(ZTOV(zp), rdb, size); + return (STATUS_SUCCESS); +} + +int +zfsctl_delete_reparse_point(znode_t *zp) +{ + if (!zfsctl_is_leafnode(zp)) + return (STATUS_INVALID_PARAMETER); + + zp->z_pflags &= ~ZFS_REPARSE; + vnode_set_reparse(ZTOV(zp), NULL, 0); + + return (STATUS_SUCCESS); +} + +// make Tag vs Point call, to return just ULONG tag or copy over point +ULONG +zfsctl_get_reparse_tag(znode_t *zp) +{ + + if (!zfsctl_is_leafnode(zp)) + return (0); + + if (!(zp->z_pflags & ZFS_REPARSE)) + return (0); + + return (vnode_get_reparse_tag(ZTOV(zp))); +} + +int +zfsctl_get_reparse_point(znode_t *zp, REPARSE_DATA_BUFFER **buffer, + size_t *size) +{ + + if (!zfsctl_is_leafnode(zp)) + return (STATUS_NOT_A_REPARSE_POINT); + + if (!(zp->z_pflags & ZFS_REPARSE)) + return (STATUS_NOT_A_REPARSE_POINT); + + return (vnode_get_reparse_point(ZTOV(zp), buffer, size)); +} + +int +zfsctl_vnop_mkdir(struct vnop_mkdir_args *ap) +#if 0 + struct vnop_mkdir_args { + struct vnode *a_dvp; + struct vnode **a_vpp; + struct componentname *a_cnp; + struct vnode_vattr *a_vap; + vfs_context_t a_context; + }; +#endif +{ + cred_t *cr = NULL; // (cred_t *)vfs_context_ucred((ap)->a_context); + znode_t *dzp = VTOZ(ap->a_dvp); + zfsvfs_t *zfsvfs = dzp->z_zfsvfs; + char *dsname; + int error; + + if (zfs_admin_snapshot == 0) + return (SET_ERROR(EACCES)); + + dsname = kmem_alloc(ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + + if (zfs_component_namecheck(ap->a_cnp->cn_nameptr, NULL, NULL) != 0) { + error = SET_ERROR(EILSEQ); + goto out; + } + + dmu_objset_name(zfsvfs->z_os, dsname); + + error = zfs_secpolicy_snapshot_perms(dsname, cr); + if (error != 0) + goto out; + + if (error == 0) { + error = dmu_objset_snapshot_one(dsname, ap->a_cnp->cn_nameptr); + if (error != 0) + goto out; + + // error = zfsctl_root_lookup(ap->a_dvp, ap->a_cnp->cn_nameptr, + // ap->a_vpp, 0, cr, NULL, NULL); + } + +out: + kmem_free(dsname, ZFS_MAX_DATASET_NAME_LEN); + + return (error); +} + +int +zfsctl_vnop_rmdir(struct vnop_rmdir_args *ap) +#if 0 + struct vnop_rmdir_args { + struct vnode *a_dvp; + struct vnode *a_vp; + struct componentname *a_cnp; + vfs_context_t a_context; + }; +#endif +{ + cred_t *cr = NULL; // (cred_t *)vfs_context_ucred((ap)->a_context); + znode_t *dzp = VTOZ(ap->a_dvp); + zfsvfs_t *zfsvfs = dzp->z_zfsvfs; + char *snapname, *real; + char *name = ap->a_cnp->cn_nameptr; + int error; + + dprintf("%s: '%s'\n", __func__, name); + + if (zfs_admin_snapshot == 0) + return (SET_ERROR(EACCES)); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + snapname = kmem_alloc(ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + real = kmem_alloc(ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + + if (zfsvfs->z_case == ZFS_CASE_INSENSITIVE) { + error = dmu_snapshot_realname(zfsvfs->z_os, name, + real, ZFS_MAX_DATASET_NAME_LEN, NULL); + if (error == 0) { + name = real; + } else if (error != ENOTSUP) { + goto out; + } + } + + error = zfsctl_snapshot_name(zfsvfs, name, + ZFS_MAX_DATASET_NAME_LEN, snapname); + if (error == 0) + error = zfs_secpolicy_destroy_perms(snapname, cr); + if (error != 0) + goto out; + + error = zfsctl_snapshot_unmount_node(ap->a_vp, snapname, MNT_FORCE); + if ((error == 0) || (error == ENOENT)) { + error = dsl_destroy_snapshot(snapname, B_FALSE); + + /* Destroy the vnode */ + if (ap->a_vp != NULL) { + dprintf("%s: releasing vp\n", __func__); + vnode_recycle(ap->a_vp); + } + } + +out: + kmem_free(snapname, ZFS_MAX_DATASET_NAME_LEN); + kmem_free(real, ZFS_MAX_DATASET_NAME_LEN); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +static void +zfsctl_unmount_thread(void *notused) +{ + callb_cpr_t cpr; + zfsctl_unmount_delay_t *zcu; + time_t now; + CALLB_CPR_INIT(&cpr, &zfsctl_unmount_lock, callb_generic_cpr, FTAG); + + dprintf("%s is alive\n", __func__); + + mutex_enter(&zfsctl_unmount_lock); + while (!zfsctl_unmount_thread_exit) { + + CALLB_CPR_SAFE_BEGIN(&cpr); + (void) cv_timedwait(&zfsctl_unmount_cv, + &zfsctl_unmount_lock, ddi_get_lbolt() + (hz<<6)); + CALLB_CPR_SAFE_END(&cpr, &zfsctl_unmount_lock); + + if (!zfsctl_unmount_thread_exit) { + /* + * Loop all active mounts, if any are older + * than ZFSCTL_EXPIRE_SNAPSHOT, then we update + * their timestamp and attempt unmount. + */ + now = gethrestime_sec(); + mutex_enter(&zfsctl_unmount_list_lock); + for (zcu = list_head(&zfsctl_unmount_list); + zcu != NULL; + zcu = list_next(&zfsctl_unmount_list, zcu)) { + if ((now > zcu->se_time) && + ((now - zcu->se_time) > + zfs_expire_snapshot)) { + zcu->se_time = now; + zfsctl_snapshot_unmount(zcu->se_name, + 0); + } + } + mutex_exit(&zfsctl_unmount_list_lock); + } + } + + zfsctl_unmount_thread_exit = FALSE; + cv_broadcast(&zfsctl_unmount_cv); + CALLB_CPR_EXIT(&cpr); + dprintf("ZFS: zfsctl_unmount thread exit\n"); + thread_exit(); +} + +/* + * Initialize the various pieces we'll need to create and manipulate .zfs + * directories. Currently this is unused but available. + */ +void +zfsctl_init(void) +{ + mutex_init(&zfsctl_mounts_lock, NULL, MUTEX_DEFAULT, NULL); + list_create(&zfsctl_mounts_list, sizeof (zfsctl_mounts_waiting_t), + offsetof(zfsctl_mounts_waiting_t, zcm_node)); + + mutex_init(&zfsctl_unmount_list_lock, NULL, MUTEX_DEFAULT, NULL); + list_create(&zfsctl_unmount_list, sizeof (zfsctl_unmount_delay_t), + offsetof(zfsctl_unmount_delay_t, se_nodelink)); + + mutex_init(&zfsctl_unmount_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&zfsctl_unmount_cv, NULL, CV_DEFAULT, NULL); + zfsctl_unmount_thread_exit = FALSE; + + (void) thread_create(NULL, 0, zfsctl_unmount_thread, NULL, 0, &p0, + TS_RUN, minclsyspri); +} + +/* + * Cleanup the various pieces we needed for .zfs directories. In particular + * ensure the expiry timer is canceled safely. + */ +void +zfsctl_fini(void) +{ + mutex_destroy(&zfsctl_mounts_lock); + list_destroy(&zfsctl_mounts_list); + + mutex_destroy(&zfsctl_unmount_list_lock); + list_destroy(&zfsctl_unmount_list); + + mutex_enter(&zfsctl_unmount_lock); + zfsctl_unmount_thread_exit = TRUE; + while (zfsctl_unmount_thread_exit) { + cv_signal(&zfsctl_unmount_cv); + cv_wait(&zfsctl_unmount_cv, &zfsctl_unmount_lock); + } + mutex_exit(&zfsctl_unmount_lock); + + mutex_destroy(&zfsctl_unmount_lock); + cv_destroy(&zfsctl_unmount_cv); +} + +module_param(zfs_admin_snapshot, int, 0644); +MODULE_PARM_DESC(zfs_admin_snapshot, "Enable mkdir/rmdir/mv in .zfs/snapshot"); + +module_param(zfs_expire_snapshot, int, 0644); +MODULE_PARM_DESC(zfs_expire_snapshot, "Seconds to expire .zfs/snapshot"); diff --git a/module/os/windows/zfs/zfs_debug.c b/module/os/windows/zfs/zfs_debug.c new file mode 100644 index 000000000000..76ba385fee0d --- /dev/null +++ b/module/os/windows/zfs/zfs_debug.c @@ -0,0 +1,288 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2014 by Delphix. All rights reserved. + * Portions Copyright 2022 Andrew Innes + */ + +#include + +typedef struct zfs_dbgmsg { + list_node_t zdm_node; + time_t zdm_timestamp; + uint_t zdm_size; + char zdm_msg[1]; /* variable length allocation */ +} zfs_dbgmsg_t; + +list_t zfs_dbgmsgs; +uint_t zfs_dbgmsg_size; +kmutex_t zfs_dbgmsgs_lock; +static uint_t zfs_dbgmsg_maxsize = 4<<20; /* 4MB */ +kstat_t *zfs_dbgmsg_kstat; + +int zfs_dbgmsg_enable = 1; + +static int +zfs_dbgmsg_headers(char *buf, size_t size) +{ + (void) snprintf(buf, size, "%-12s %-8s\n", "timestamp", "message"); + + return (0); +} + +static int +zfs_dbgmsg_data(char *buf, size_t size, void *data) +{ + zfs_dbgmsg_t *zdm = (zfs_dbgmsg_t *)data; + + (void) snprintf(buf, size, "%-12llu %-s\n", + (u_longlong_t)zdm->zdm_timestamp, zdm->zdm_msg); + + return (0); +} + +static void * +zfs_dbgmsg_addr(kstat_t *ksp, loff_t n) +{ + zfs_dbgmsg_t *zdm = (zfs_dbgmsg_t *)ksp->ks_private; + + ASSERT(MUTEX_HELD(&zfs_dbgmsgs_lock)); + + if (n == 0) + ksp->ks_private = list_head(&zfs_dbgmsgs); + else if (zdm) + ksp->ks_private = list_next(&zfs_dbgmsgs, zdm); + + return (ksp->ks_private); +} + +static void +zfs_dbgmsg_purge(uint_t max_size) +{ + zfs_dbgmsg_t *zdm; + uint_t size; + + ASSERT(MUTEX_HELD(&zfs_dbgmsgs_lock)); + + while (zfs_dbgmsg_size > max_size) { + zdm = list_remove_head(&zfs_dbgmsgs); + if (zdm == NULL) + return; + + size = zdm->zdm_size; + kmem_free(zdm, size); + zfs_dbgmsg_size -= size; + } +} + +static int +zfs_dbgmsg_update(kstat_t *ksp, int rw) +{ + if (rw == KSTAT_WRITE) + zfs_dbgmsg_purge(0); + + return (0); +} + +/* + * Debug logging is enabled by default for production kernel builds. + * The overhead for this is negligible and the logs can be valuable when + * debugging. For non-production user space builds all debugging except + * logging is enabled since performance is no longer a concern. + */ +void +zfs_dbgmsg_init(void) +{ + list_create(&zfs_dbgmsgs, sizeof (zfs_dbgmsg_t), + offsetof(zfs_dbgmsg_t, zdm_node)); + mutex_init(&zfs_dbgmsgs_lock, NULL, MUTEX_DEFAULT, NULL); + + zfs_dbgmsg_kstat = kstat_create("zfs", 0, "dbgmsg", "misc", + KSTAT_TYPE_RAW, 0, KSTAT_FLAG_VIRTUAL); + if (zfs_dbgmsg_kstat) { + zfs_dbgmsg_kstat->ks_lock = &zfs_dbgmsgs_lock; + zfs_dbgmsg_kstat->ks_ndata = UINT32_MAX; + zfs_dbgmsg_kstat->ks_private = NULL; + zfs_dbgmsg_kstat->ks_update = zfs_dbgmsg_update; + kstat_set_raw_ops(zfs_dbgmsg_kstat, zfs_dbgmsg_headers, + zfs_dbgmsg_data, zfs_dbgmsg_addr); + kstat_install(zfs_dbgmsg_kstat); + } +} + +void +zfs_dbgmsg_fini(void) +{ + zfs_dbgmsg_t *zdm; + + if (zfs_dbgmsg_kstat) + kstat_delete(zfs_dbgmsg_kstat); + + while ((zdm = list_remove_head(&zfs_dbgmsgs)) != NULL) { + int size = sizeof (zfs_dbgmsg_t) + strlen(zdm->zdm_msg); + kmem_free(zdm, size); + zfs_dbgmsg_size -= size; + } + mutex_destroy(&zfs_dbgmsgs_lock); + ASSERT0(zfs_dbgmsg_size); +} + +void +__set_error(const char *file, const char *func, int line, int err) +{ + /* + * To enable this: + * + * $ echo 512 >/sys/module/zfs/parameters/zfs_flags + */ + if (zfs_flags & ZFS_DEBUG_SET_ERROR) + __dprintf(B_FALSE, file, func, line, "error %lu", err); + + TraceEvent(5, "%s:%s Line:%d Error:%d", file, func, line, err); +} + +/* + * Print these messages by running: + * echo ::zfs_dbgmsg | mdb -k + * + * Monitor these messages by running: + * dtrace -qn 'zfs-dbgmsg{printf("%s\n", stringof(arg0))}' + * + * When used with libzpool, monitor with: + * dtrace -qn 'zfs$pid::zfs_dbgmsg:probe1{printf("%s\n", copyinstr(arg1))}' + */ + +/* + * Look into Windows dtrace? + * MacOS X's dtrace doesn't handle the PROBEs, so + * we have a utility function that we can watch with + * sudo dtrace -qn '__zfs_dbgmsg:entry{printf("%s\n", stringof(arg0));}' + */ +noinline void +__zfs_dbgmsg(char *buf) +{ + uint_t size = sizeof (zfs_dbgmsg_t) + strlen(buf); + zfs_dbgmsg_t *zdm = kmem_zalloc(size, KM_SLEEP); + zdm->zdm_size = size; + zdm->zdm_timestamp = gethrestime_sec(); + strlcpy(zdm->zdm_msg, buf, size); + + mutex_enter(&zfs_dbgmsgs_lock); + list_insert_tail(&zfs_dbgmsgs, zdm); + zfs_dbgmsg_size += size; + zfs_dbgmsg_purge(zfs_dbgmsg_maxsize); + mutex_exit(&zfs_dbgmsgs_lock); +} + +#ifdef _KERNEL +void +__dprintf(boolean_t dprint, const char *file, const char *func, + int line, const char *fmt, ...) +{ + int size, i; + va_list adx; + char *buf, *nl; + char *prefix = (dprint) ? "dprintf: " : ""; + const char *newfile; + + /* + * Skip everything if we can't write to the debug log due to + * being in a DPC. + */ + if (KeGetCurrentIrql() >= DISPATCH_LEVEL) + return; + + /* + * Get rid of annoying prefix to filename. + */ + newfile = strrchr(file, '/'); + if (newfile != NULL) { + newfile = newfile + 1; /* Get rid of leading / */ + } else { + newfile = file; + } + newfile = strrchr(file, '\\'); + if (newfile != NULL) { + newfile = newfile + 1; /* Get rid of leading / */ + } else { + newfile = file; + } + + va_start(adx, fmt); + size = vsnprintf(NULL, 0, fmt, adx); + va_end(adx); + + size += snprintf(NULL, 0, "%s%s:%d:%s(): ", prefix, newfile, line, + func); + + size++; /* null byte in the "buf" string */ + + /* + * There is one byte of string in sizeof (zfs_dbgmsg_t), used + * for the terminating null. + */ + buf = kmem_alloc(size, KM_SLEEP); + int roger = 0; + + va_start(adx, fmt); + i = snprintf(buf, size + 1, "%s%s:%d:%s(): ", + prefix, newfile, line, func); + roger = vsnprintf(buf + i, size -i + 1, fmt, adx); + va_end(adx); + + /* + * Get rid of trailing newline for dprintf logs. + */ + if (dprint && buf[0] != '\0') { + nl = &buf[strlen(buf) - 1]; + if (*nl == '\n') + *nl = '\0'; + } + + DTRACE_PROBE1(zfs__dbgmsg, char *, zdm->zdm_msg); + + __zfs_dbgmsg(buf); + + /* Also emit string to log/console */ + printBuffer("%s\n", buf); + + kmem_free(buf, size); +} + +#else + +#define printBuffer printf + +#endif + +void +zfs_dbgmsg_print(const char *tag) +{ + zfs_dbgmsg_t *zdm; + + (void) printBuffer("ZFS_DBGMSG(%s):\n", tag); + mutex_enter(&zfs_dbgmsgs_lock); + for (zdm = list_head(&zfs_dbgmsgs); zdm; + zdm = list_next(&zfs_dbgmsgs, zdm)) + (void) printBuffer("%s\n", zdm->zdm_msg); + mutex_exit(&zfs_dbgmsgs_lock); +} diff --git a/module/os/windows/zfs/zfs_dir.c b/module/os/windows/zfs/zfs_dir.c new file mode 100644 index 000000000000..b95b8df3cc84 --- /dev/null +++ b/module/os/windows/zfs/zfs_dir.c @@ -0,0 +1,1208 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2013, 2016 by Delphix. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * zfs_match_find() is used by zfs_dirent_lock() to perform zap lookups + * of names after deciding which is the appropriate lookup interface. + */ +static int +zfs_match_find(zfsvfs_t *zfsvfs, znode_t *dzp, char *name, matchtype_t mt, + boolean_t update, int *deflags, struct componentname *rpnp, uint64_t *zoid) +{ + boolean_t conflict = B_FALSE; + int error; + + if (zfsvfs->z_norm) { + size_t bufsz = 0; + char *buf = NULL; + + if (rpnp) { + buf = rpnp->cn_nameptr; + bufsz = rpnp->cn_pnlen; + } + + /* + * In the non-mixed case we only expect there would ever + * be one match, but we need to use the normalizing lookup. + */ + error = zap_lookup_norm(zfsvfs->z_os, dzp->z_id, name, 8, 1, + zoid, mt, buf, bufsz, &conflict); + } else { + error = zap_lookup(zfsvfs->z_os, dzp->z_id, name, 8, 1, zoid); + } + + /* + * Allow multiple entries provided the first entry is + * the object id. Non-zpl consumers may safely make + * use of the additional space. + * + * XXX: This should be a feature flag for compatibility + */ + if (error == EOVERFLOW) + error = 0; + + if (zfsvfs->z_norm && !error && deflags) + *deflags = conflict ? ED_CASE_CONFLICT : 0; + + *zoid = ZFS_DIRENT_OBJ(*zoid); + + return (error); +} + +/* + * Lock a directory entry. A dirlock on protects that name + * in dzp's directory zap object. As long as you hold a dirlock, you can + * assume two things: (1) dzp cannot be reaped, and (2) no other thread + * can change the zap entry for (i.e. link or unlink) this name. + * + * Input arguments: + * dzp - znode for directory + * name - name of entry to lock + * flag - ZNEW: if the entry already exists, fail with EEXIST. + * ZEXISTS: if the entry does not exist, fail with ENOENT. + * ZSHARED: allow concurrent access with other ZSHARED callers. + * ZXATTR: we want dzp's xattr directory + * ZCILOOK: On a mixed sensitivity file system, + * this lookup should be case-insensitive. + * ZCIEXACT: On a purely case-insensitive file system, + * this lookup should be case-sensitive. + * ZRENAMING: we are locking for renaming, force narrow locks + * ZHAVELOCK: Don't grab the z_name_lock for this call. The + * current thread already holds it. + * + * Output arguments: + * zpp - pointer to the znode for the entry (NULL if there isn't one) + * dlpp - pointer to the dirlock for this entry (NULL on error) + * direntflags - (case-insensitive lookup only) + * flags if multiple case-sensitive matches exist in directory + * realpnp - (case-insensitive lookup only) + * actual name matched within the directory + * + * Return value: 0 on success or errno on failure. + * + * NOTE: Always checks for, and rejects, '.' and '..'. + * NOTE: For case-insensitive file systems we take wide locks (see below), + * but return znode pointers to a single match. + */ +int +zfs_dirent_lock(zfs_dirlock_t **dlpp, znode_t *dzp, char *name, znode_t **zpp, + int flag, int *direntflags, struct componentname *realpnp) +{ + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + zfs_dirlock_t *dl; + boolean_t update; + matchtype_t mt = 0; + uint64_t zoid; + int error = 0; + int cmpflags; + + *zpp = NULL; + *dlpp = NULL; + + /* + * Verify that we are not trying to lock '.', '..', or '.zfs' + */ + if ((name[0] == '.' && + (name[1] == '\0' || (name[1] == '.' && name[2] == '\0'))) || + (zfs_has_ctldir(dzp) && strcmp(name, ZFS_CTLDIR_NAME) == 0)) + return (SET_ERROR(EEXIST)); + + /* + * Case sensitivity and normalization preferences are set when + * the file system is created. These are stored in the + * zfsvfs->z_case and zfsvfs->z_norm fields. These choices + * affect what vnodes can be cached in the DNLC, how we + * perform zap lookups, and the "width" of our dirlocks. + * + * A normal dirlock locks a single name. Note that with + * normalization a name can be composed multiple ways, but + * when normalized, these names all compare equal. A wide + * dirlock locks multiple names. We need these when the file + * system is supporting mixed-mode access. It is sometimes + * necessary to lock all case permutations of file name at + * once so that simultaneous case-insensitive/case-sensitive + * behaves as rationally as possible. + */ + + /* + * When matching we may need to normalize & change case according to + * FS settings. + * + * Note that a normalized match is necessary for a case insensitive + * filesystem when the lookup request is not exact because normalization + * can fold case independent of normalizing code point sequences. + * + * See the table above zfs_dropname(). + */ + if (zfsvfs->z_norm != 0) { + mt = MT_NORMALIZE; + + /* + * Determine if the match needs to honor the case specified in + * lookup, and if so keep track of that so that during + * normalization we don't fold case. + */ + if ((zfsvfs->z_case == ZFS_CASE_INSENSITIVE && + (flag & ZCIEXACT)) || + (zfsvfs->z_case == ZFS_CASE_MIXED && !(flag & ZCILOOK))) { + mt |= MT_MATCH_CASE; + } + } + + /* + * Only look in or update the DNLC if we are looking for the + * name on a file system that does not require normalization + * or case folding. We can also look there if we happen to be + * on a non-normalizing, mixed sensitivity file system IF we + * are looking for the exact name. + * + * Maybe can add TO-UPPERed version of name to dnlc in ci-only + * case for performance improvement? + */ + update = !zfsvfs->z_norm || + (zfsvfs->z_case == ZFS_CASE_MIXED && + !(zfsvfs->z_norm & ~U8_TEXTPREP_TOUPPER) && !(flag & ZCILOOK)); + + /* + * ZRENAMING indicates we are in a situation where we should + * take narrow locks regardless of the file system's + * preferences for normalizing and case folding. This will + * prevent us deadlocking trying to grab the same wide lock + * twice if the two names happen to be case-insensitive + * matches. + */ + if (flag & ZRENAMING) + cmpflags = 0; + else + cmpflags = zfsvfs->z_norm; + + /* + * Wait until there are no locks on this name. + * + * Don't grab the lock if it is already held. However, cannot + * have both ZSHARED and ZHAVELOCK together. + */ + ASSERT(!(flag & ZSHARED) || !(flag & ZHAVELOCK)); + if (!(flag & ZHAVELOCK)) + rw_enter(&dzp->z_name_lock, RW_READER); + + mutex_enter(&dzp->z_lock); + for (;;) { + if (dzp->z_unlinked && !(flag & ZXATTR)) { + mutex_exit(&dzp->z_lock); + if (!(flag & ZHAVELOCK)) + rw_exit(&dzp->z_name_lock); + return (SET_ERROR(ENOENT)); + } + for (dl = dzp->z_dirlocks; dl != NULL; dl = dl->dl_next) { + if ((u8_strcmp(name, dl->dl_name, 0, cmpflags, + U8_UNICODE_LATEST, &error) == 0) || error != 0) + break; + } + if (error != 0) { + mutex_exit(&dzp->z_lock); + if (!(flag & ZHAVELOCK)) + rw_exit(&dzp->z_name_lock); + return (SET_ERROR(ENOENT)); + } + if (dl == NULL) { + /* + * Allocate a new dirlock and add it to the list. + */ + dl = kmem_alloc(sizeof (zfs_dirlock_t), KM_SLEEP); + cv_init(&dl->dl_cv, NULL, CV_DEFAULT, NULL); + dl->dl_name = name; + dl->dl_sharecnt = 0; + dl->dl_namelock = 0; + dl->dl_namesize = 0; + dl->dl_dzp = dzp; + dl->dl_next = dzp->z_dirlocks; + dzp->z_dirlocks = dl; + break; + } + if ((flag & ZSHARED) && dl->dl_sharecnt != 0) + break; + cv_wait(&dl->dl_cv, &dzp->z_lock); + } + + /* + * If the z_name_lock was NOT held for this dirlock record it. + */ + if (flag & ZHAVELOCK) + dl->dl_namelock = 1; + + if ((flag & ZSHARED) && ++dl->dl_sharecnt > 1 && dl->dl_namesize == 0) { + /* + * We're the second shared reference to dl. Make a copy of + * dl_name in case the first thread goes away before we do. + * Note that we initialize the new name before storing its + * pointer into dl_name, because the first thread may load + * dl->dl_name at any time. It'll either see the old value, + * which belongs to it, or the new shared copy; either is OK. + */ + dl->dl_namesize = strlen(dl->dl_name) + 1; + name = kmem_alloc(dl->dl_namesize, KM_SLEEP); + memcpy(name, dl->dl_name, dl->dl_namesize); + dl->dl_name = name; + } + + mutex_exit(&dzp->z_lock); + + /* + * We have a dirlock on the name. (Note that it is the dirlock, + * not the dzp's z_lock, that protects the name in the zap object.) + * See if there's an object by this name; if so, put a hold on it. + */ + if (flag & ZXATTR) { + error = sa_lookup(dzp->z_sa_hdl, SA_ZPL_XATTR(zfsvfs), &zoid, + sizeof (zoid)); + if (error == 0) + error = (zoid == 0 ? SET_ERROR(ENOENT) : 0); + } else { + error = zfs_match_find(zfsvfs, dzp, name, mt, + update, direntflags, realpnp, &zoid); + } + if (error) { + if (error != ENOENT || (flag & ZEXISTS)) { + zfs_dirent_unlock(dl); + return (error); + } + } else { + if (flag & ZNEW) { + zfs_dirent_unlock(dl); + return (SET_ERROR(EEXIST)); + } + error = zfs_zget(zfsvfs, zoid, zpp); + if (error) { + zfs_dirent_unlock(dl); + return (error); + } + } + + *dlpp = dl; + + return (0); +} + +/* + * Unlock this directory entry and wake anyone who was waiting for it. + */ +void +zfs_dirent_unlock(zfs_dirlock_t *dl) +{ + znode_t *dzp = dl->dl_dzp; + zfs_dirlock_t **prev_dl, *cur_dl; + + mutex_enter(&dzp->z_lock); + + if (!dl->dl_namelock) + rw_exit(&dzp->z_name_lock); + + if (dl->dl_sharecnt > 1) { + dl->dl_sharecnt--; + mutex_exit(&dzp->z_lock); + return; + } + prev_dl = &dzp->z_dirlocks; + while ((cur_dl = *prev_dl) != dl) + prev_dl = &cur_dl->dl_next; + *prev_dl = dl->dl_next; + cv_broadcast(&dl->dl_cv); + mutex_exit(&dzp->z_lock); + + if (dl->dl_namesize != 0) + kmem_free(dl->dl_name, dl->dl_namesize); + cv_destroy(&dl->dl_cv); + kmem_free(dl, sizeof (*dl)); +} + +/* + * Look up an entry in a directory. + * + * NOTE: '.' and '..' are handled as special cases because + * no directory entries are actually stored for them. If this is + * the root of a filesystem, then '.zfs' is also treated as a + * special pseudo-directory. + */ +int +zfs_dirlook(znode_t *dzp, char *name, znode_t **zpp, int flags, + int *deflg, struct componentname *rpnp) +{ + zfs_dirlock_t *dl; + znode_t *zp; + struct vnode *vp; + int error = 0; + uint64_t parent; + + if (name[0] == 0 || (name[0] == '.' && name[1] == 0)) { + *zpp = dzp; + zhold(*zpp); + } else if (name[0] == '.' && name[1] == '.' && name[2] == 0) { + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + + /* + * If we are a snapshot mounted under .zfs, return + * the inode pointer for the snapshot directory. + */ + if ((error = sa_lookup(dzp->z_sa_hdl, + SA_ZPL_PARENT(zfsvfs), &parent, sizeof (parent))) != 0) + return (error); + + if (parent == dzp->z_id && zfsvfs->z_parent != zfsvfs) { + error = zfsctl_root_lookup(zfsvfs->z_parent->z_ctldir, + ZFS_SNAPDIR_NAME, &vp, 0, kcred, NULL, NULL); + if (error == 0) + *zpp = VTOZ(vp); + return (error); + } + rw_enter(&dzp->z_parent_lock, RW_READER); + error = zfs_zget(zfsvfs, parent, &zp); + if (error == 0) + *zpp = zp; + rw_exit(&dzp->z_parent_lock); + } else if (zfs_has_ctldir(dzp) && strcmp(name, ZFS_CTLDIR_NAME) == 0) { + vp = zfsctl_root(dzp); + if (vp != NULL) + *zpp = VTOZ(vp); + else + error = ENOENT; + } else { + int zf; + + zf = ZEXISTS | ZSHARED; + if (flags & FIGNORECASE) + zf |= ZCILOOK; + + error = zfs_dirent_lock(&dl, dzp, name, &zp, zf, deflg, rpnp); + if (error == 0) { + *zpp = zp; + zfs_dirent_unlock(dl); + dzp->z_zn_prefetch = B_TRUE; /* enable prefetching */ + } + rpnp = NULL; + } + + if ((flags & FIGNORECASE) && rpnp && !error) + (void) strlcpy(rpnp->cn_nameptr, name, rpnp->cn_namelen); + + return (error); +} + +/* + * unlinked Set (formerly known as the "delete queue") Error Handling + * + * When dealing with the unlinked set, we dmu_tx_hold_zap(), but we + * don't specify the name of the entry that we will be manipulating. We + * also fib and say that we won't be adding any new entries to the + * unlinked set, even though we might (this is to lower the minimum file + * size that can be deleted in a full filesystem). So on the small + * chance that the nlink list is using a fat zap (ie. has more than + * 2000 entries), we *may* not pre-read a block that's needed. + * Therefore it is remotely possible for some of the assertions + * regarding the unlinked set below to fail due to i/o error. On a + * nondebug system, this will result in the space being leaked. + */ +void +zfs_unlinked_add(znode_t *zp, dmu_tx_t *tx) +{ + zfsvfs_t *zfsvfs = ZTOZSB(zp); + + ASSERT(zp->z_unlinked); + + VERIFY3U(0, ==, + zap_add_int(zfsvfs->z_os, zfsvfs->z_unlinkedobj, zp->z_id, tx)); + +} + +/* + * Clean up any znodes that had no links when we either crashed or + * (force) umounted the file system. + */ +static void +zfs_unlinked_drain_task(void *arg) +{ + zfsvfs_t *zfsvfs = arg; + zap_cursor_t zc; + zap_attribute_t zap; + dmu_object_info_t doi; + znode_t *zp; + int error; + + /* + * Iterate over the contents of the unlinked set. + */ + for (zap_cursor_init(&zc, zfsvfs->z_os, zfsvfs->z_unlinkedobj); + zap_cursor_retrieve(&zc, &zap) == 0 && + zfsvfs->z_drain_state == ZFS_DRAIN_RUNNING; + zap_cursor_advance(&zc)) { + + /* + * See what kind of object we have in list + */ + + error = dmu_object_info(zfsvfs->z_os, + zap.za_first_integer, &doi); + if (error != 0) + continue; + + ASSERT((doi.doi_type == DMU_OT_PLAIN_FILE_CONTENTS) || + (doi.doi_type == DMU_OT_DIRECTORY_CONTENTS)); + /* + * We need to re-mark these list entries for deletion, + * so we pull them back into core and set zp->z_unlinked. + */ + error = zfs_zget(zfsvfs, zap.za_first_integer, &zp); + + /* + * We may pick up znodes that are already marked for deletion. + * This could happen during the purge of an extended attribute + * directory. All we need to do is skip over them, since they + * are already in the system marked z_unlinked. + */ + if (error != 0) + continue; + + zp->z_unlinked = B_TRUE; + + /* + * zrele() decrements the znode's ref count and may cause + * it to be synchronously freed. We interrupt freeing + * of this znode by checking the return value of + * dmu_objset_zfs_unmounting() in dmu_free_long_range() + * when an unmount is requested. + */ + zrele(zp); + ASSERT3B(zfsvfs->z_unmounted, ==, B_FALSE); + } + zap_cursor_fini(&zc); + + mutex_enter(&zfsvfs->z_drain_lock); + zfsvfs->z_drain_state = ZFS_DRAIN_SHUTDOWN; + cv_broadcast(&zfsvfs->z_drain_cv); + mutex_exit(&zfsvfs->z_drain_lock); +} + +/* + * Sets z_draining then tries to dispatch async unlinked drain. + * If that fails executes synchronous unlinked drain. + */ +void +zfs_unlinked_drain(zfsvfs_t *zfsvfs) +{ + ASSERT3B(zfsvfs->z_unmounted, ==, B_FALSE); + + mutex_enter(&zfsvfs->z_drain_lock); + ASSERT(zfsvfs->z_drain_state == ZFS_DRAIN_SHUTDOWN); + zfsvfs->z_drain_state = ZFS_DRAIN_RUNNING; + mutex_exit(&zfsvfs->z_drain_lock); + + if (taskq_dispatch( + dsl_pool_unlinked_drain_taskq(dmu_objset_pool(zfsvfs->z_os)), + zfs_unlinked_drain_task, zfsvfs, TQ_SLEEP) == 0) { + zfs_dbgmsg("async zfs_unlinked_drain dispatch failed"); + zfs_unlinked_drain_task(zfsvfs); + } +} + +/* + * Wait for the unlinked drain taskq task to stop. This will interrupt the + * unlinked set processing if it is in progress. + */ +void +zfs_unlinked_drain_stop_wait(zfsvfs_t *zfsvfs) +{ + ASSERT3B(zfsvfs->z_unmounted, ==, B_FALSE); + + mutex_enter(&zfsvfs->z_drain_lock); + while (zfsvfs->z_drain_state != ZFS_DRAIN_SHUTDOWN) { + zfsvfs->z_drain_state = ZFS_DRAIN_SHUTDOWN_REQ; + cv_wait(&zfsvfs->z_drain_cv, &zfsvfs->z_drain_lock); + } + mutex_exit(&zfsvfs->z_drain_lock); +} + +/* + * Delete the entire contents of a directory. Return a count + * of the number of entries that could not be deleted. If we encounter + * an error, return a count of at least one so that the directory stays + * in the unlinked set. + * + * NOTE: this function assumes that the directory is inactive, + * so there is no need to lock its entries before deletion. + * Also, it assumes the directory contents is *only* regular + * files. + */ +static int +zfs_purgedir(znode_t *dzp) +{ + zap_cursor_t zc; + zap_attribute_t zap; + znode_t *xzp; + dmu_tx_t *tx; + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + zfs_dirlock_t dl; + int skipped = 0; + int error; + + for (zap_cursor_init(&zc, zfsvfs->z_os, dzp->z_id); + (error = zap_cursor_retrieve(&zc, &zap)) == 0; + zap_cursor_advance(&zc)) { + error = zfs_zget(zfsvfs, + ZFS_DIRENT_OBJ(zap.za_first_integer), &xzp); + if (error) { + skipped += 1; + continue; + } + + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_sa(tx, dzp->z_sa_hdl, B_FALSE); + dmu_tx_hold_zap(tx, dzp->z_id, FALSE, zap.za_name); + dmu_tx_hold_sa(tx, xzp->z_sa_hdl, B_FALSE); + dmu_tx_hold_zap(tx, zfsvfs->z_unlinkedobj, FALSE, NULL); + /* Is this really needed ? */ + zfs_sa_upgrade_txholds(tx, xzp); + dmu_tx_mark_netfree(tx); + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + zfs_zrele_async(xzp); + skipped += 1; + continue; + } + memset(&dl, 0, sizeof (dl)); + dl.dl_dzp = dzp; + dl.dl_name = zap.za_name; + + error = zfs_link_destroy(&dl, xzp, tx, 0, NULL); + if (error) + skipped += 1; + dmu_tx_commit(tx); + + zfs_zrele_async(xzp); + } + zap_cursor_fini(&zc); + if (error != ENOENT) + skipped += 1; + return (skipped); +} + +void +zfs_rmnode(znode_t *zp) +{ + zfsvfs_t *zfsvfs = ZTOZSB(zp); + objset_t *os = zfsvfs->z_os; + znode_t *xzp = NULL; + dmu_tx_t *tx; + uint64_t acl_obj; + uint64_t xattr_obj; + int error; + + /* + * If this is an attribute directory, purge its contents. + */ + if (S_ISDIR(zp->z_mode) && (zp->z_pflags & ZFS_XATTR)) { + if (zfs_purgedir(zp) != 0) { + /* + * Not enough space to delete some xattrs. + * Leave it in the unlinked set. + */ + zfs_znode_dmu_fini(zp); + + return; + } + } + + /* + * Free up all the data in the file. We don't do this for directories + * because we need truncate and remove to be in the same tx, like in + * zfs_znode_delete(). Otherwise, if we crash here we'll end up with + * an inconsistent truncated zap object in the delete queue. Note a + * truncated file is harmless since it only contains user data. + */ + if (S_ISREG(zp->z_mode)) { + error = dmu_free_long_range(os, zp->z_id, 0, DMU_OBJECT_END); + if (error) { + /* + * Not enough space or we were interrupted by unmount. + * Leave the file in the unlinked set. + */ + zfs_znode_dmu_fini(zp); + return; + } + } + + /* + * If the file has extended attributes, we're going to unlink + * the xattr dir. + */ + error = sa_lookup(zp->z_sa_hdl, SA_ZPL_XATTR(zfsvfs), + &xattr_obj, sizeof (xattr_obj)); + if (error == 0 && xattr_obj) { + error = zfs_zget(zfsvfs, xattr_obj, &xzp); + ASSERT(error == 0); + } + + acl_obj = zfs_external_acl(zp); + + /* + * Set up the final transaction. + */ + tx = dmu_tx_create(os); + dmu_tx_hold_free(tx, zp->z_id, 0, DMU_OBJECT_END); + dmu_tx_hold_zap(tx, zfsvfs->z_unlinkedobj, FALSE, NULL); + if (xzp) { + dmu_tx_hold_zap(tx, zfsvfs->z_unlinkedobj, TRUE, NULL); + dmu_tx_hold_sa(tx, xzp->z_sa_hdl, B_FALSE); + } + if (acl_obj) + dmu_tx_hold_free(tx, acl_obj, 0, DMU_OBJECT_END); + + zfs_sa_upgrade_txholds(tx, zp); + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + /* + * Not enough space to delete the file. Leave it in the + * unlinked set, leaking it until the fs is remounted (at + * which point we'll call zfs_unlinked_drain() to process it). + */ + dmu_tx_abort(tx); + zfs_znode_dmu_fini(zp); + goto out; + } + + if (xzp) { + ASSERT(error == 0); + mutex_enter(&xzp->z_lock); + xzp->z_unlinked = B_TRUE; /* mark xzp for deletion */ + xzp->z_links = 0; /* no more links to it */ + VERIFY(0 == sa_update(xzp->z_sa_hdl, SA_ZPL_LINKS(zfsvfs), + &xzp->z_links, sizeof (xzp->z_links), tx)); + mutex_exit(&xzp->z_lock); + zfs_unlinked_add(xzp, tx); + } + + mutex_enter(&os->os_dsl_dataset->ds_dir->dd_activity_lock); + + /* + * Remove this znode from the unlinked set. If a has rollback has + * occurred while a file is open and unlinked. Then when the file + * is closed post rollback it will not exist in the rolled back + * version of the unlinked object. + */ + error = zap_remove_int(zfsvfs->z_os, zfsvfs->z_unlinkedobj, + zp->z_id, tx); + VERIFY(error == 0 || error == ENOENT); + + uint64_t count; + if (zap_count(os, zfsvfs->z_unlinkedobj, &count) == 0 && count == 0) { + cv_broadcast(&os->os_dsl_dataset->ds_dir->dd_activity_cv); + } + + mutex_exit(&os->os_dsl_dataset->ds_dir->dd_activity_lock); + + zfs_znode_delete(zp, tx); + + dmu_tx_commit(tx); +out: + if (xzp) + zfs_zrele_async(xzp); +} + +static uint64_t +zfs_dirent(znode_t *zp, uint64_t mode) +{ + uint64_t de = zp->z_id; + + if (ZTOZSB(zp)->z_version >= ZPL_VERSION_DIRENT_TYPE) + de |= IFTODT(mode) << 60; + return (de); +} + +/* + * Link zp into dl. Can fail in the following cases : + * - if zp has been unlinked. + * - if the number of entries with the same hash (aka. colliding entries) + * exceed the capacity of a leaf-block of fatzap and splitting of the + * leaf-block does not help. + */ +int +zfs_link_create(zfs_dirlock_t *dl, znode_t *zp, dmu_tx_t *tx, int flag) +{ + znode_t *dzp = dl->dl_dzp; + zfsvfs_t *zfsvfs = ZTOZSB(zp); + uint64_t value; + int zp_is_dir = S_ISDIR(zp->z_mode); + sa_bulk_attr_t bulk[5]; + uint64_t mtime[2], ctime[2]; + int count = 0; + int error; + + mutex_enter(&zp->z_lock); + + if (!(flag & ZRENAMING)) { + if (zp->z_unlinked) { /* no new links to unlinked zp */ + ASSERT(!(flag & (ZNEW | ZEXISTS))); + mutex_exit(&zp->z_lock); + return (SET_ERROR(ENOENT)); + } + zp->z_links++; + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_LINKS(zfsvfs), + NULL, &zp->z_links, sizeof (zp->z_links)); + } + + value = zfs_dirent(zp, zp->z_mode); + error = zap_add(ZTOZSB(zp)->z_os, dzp->z_id, dl->dl_name, 8, 1, + &value, tx); + + /* + * zap_add could fail to add the entry if it exceeds the capacity of the + * leaf-block and zap_leaf_split() failed to help. + * The caller of this routine is responsible for failing the transaction + * which will rollback the SA updates done above. + */ + if (error != 0) { + mutex_exit(&zp->z_lock); + return (error); + } + + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_PARENT(zfsvfs), NULL, + &dzp->z_id, sizeof (dzp->z_id)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), NULL, + &zp->z_pflags, sizeof (zp->z_pflags)); + + if (!(flag & ZNEW)) { + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), NULL, + ctime, sizeof (ctime)); + zfs_tstamp_update_setup(zp, STATE_CHANGED, mtime, + ctime); + } + error = sa_bulk_update(zp->z_sa_hdl, bulk, count, tx); + ASSERT(error == 0); + + mutex_exit(&zp->z_lock); + + mutex_enter(&dzp->z_lock); + dzp->z_size++; + if (zp_is_dir) + dzp->z_links++; + count = 0; + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_SIZE(zfsvfs), NULL, + &dzp->z_size, sizeof (dzp->z_size)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_LINKS(zfsvfs), NULL, + &dzp->z_links, sizeof (dzp->z_links)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MTIME(zfsvfs), NULL, + mtime, sizeof (mtime)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), NULL, + ctime, sizeof (ctime)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), NULL, + &dzp->z_pflags, sizeof (dzp->z_pflags)); + zfs_tstamp_update_setup(dzp, CONTENT_MODIFIED, mtime, ctime); + error = sa_bulk_update(dzp->z_sa_hdl, bulk, count, tx); + ASSERT(error == 0); + mutex_exit(&dzp->z_lock); + + return (0); +} + +/* + * The match type in the code for this function should conform to: + * + * ------------------------------------------------------------------------ + * fs type | z_norm | lookup type | match type + * ---------|-------------|-------------|---------------------------------- + * CS !norm | 0 | 0 | 0 (exact) + * CS norm | formX | 0 | MT_NORMALIZE + * CI !norm | upper | !ZCIEXACT | MT_NORMALIZE + * CI !norm | upper | ZCIEXACT | MT_NORMALIZE | MT_MATCH_CASE + * CI norm | upper|formX | !ZCIEXACT | MT_NORMALIZE + * CI norm | upper|formX | ZCIEXACT | MT_NORMALIZE | MT_MATCH_CASE + * CM !norm | upper | !ZCILOOK | MT_NORMALIZE | MT_MATCH_CASE + * CM !norm | upper | ZCILOOK | MT_NORMALIZE + * CM norm | upper|formX | !ZCILOOK | MT_NORMALIZE | MT_MATCH_CASE + * CM norm | upper|formX | ZCILOOK | MT_NORMALIZE + * + * Abbreviations: + * CS = Case Sensitive, CI = Case Insensitive, CM = Case Mixed + * upper = case folding set by fs type on creation (U8_TEXTPREP_TOUPPER) + * formX = unicode normalization form set on fs creation + */ +static int +zfs_dropname(zfs_dirlock_t *dl, znode_t *zp, znode_t *dzp, dmu_tx_t *tx, + int flag) +{ + int error; + + if (ZTOZSB(zp)->z_norm) { + matchtype_t mt = MT_NORMALIZE; + + if ((ZTOZSB(zp)->z_case == ZFS_CASE_INSENSITIVE && + (flag & ZCIEXACT)) || + (ZTOZSB(zp)->z_case == ZFS_CASE_MIXED && + !(flag & ZCILOOK))) { + mt |= MT_MATCH_CASE; + } + + error = zap_remove_norm(ZTOZSB(zp)->z_os, dzp->z_id, + dl->dl_name, mt, tx); + } else { + error = zap_remove(ZTOZSB(zp)->z_os, dzp->z_id, dl->dl_name, + tx); + } + + return (error); +} + +/* + * Unlink zp from dl, and mark zp for deletion if this was the last link. Can + * fail if zp is a mount point (EBUSY) or a non-empty directory (ENOTEMPTY). + * If 'unlinkedp' is NULL, we put unlinked znodes on the unlinked list. + * If it's non-NULL, we use it to indicate whether the znode needs deletion, + * and it's the caller's job to do it. + */ +int +zfs_link_destroy(zfs_dirlock_t *dl, znode_t *zp, dmu_tx_t *tx, int flag, + boolean_t *unlinkedp) +{ + znode_t *dzp = dl->dl_dzp; + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + int zp_is_dir = S_ISDIR(zp->z_mode); + boolean_t unlinked = B_FALSE; + sa_bulk_attr_t bulk[5]; + uint64_t mtime[2], ctime[2]; + int count = 0; + int error; + + if (!(flag & ZRENAMING)) { + mutex_enter(&zp->z_lock); + + if (zp_is_dir && !zfs_dirempty(zp)) { + mutex_exit(&zp->z_lock); + return (SET_ERROR(ENOTEMPTY)); + } + + /* + * If we get here, we are going to try to remove the object. + * First try removing the name from the directory; if that + * fails, return the error. + */ + error = zfs_dropname(dl, zp, dzp, tx, flag); + if (error != 0) { + mutex_exit(&zp->z_lock); + return (error); + } + + if (zp->z_links <= zp_is_dir) { + zfs_panic_recover("zfs: link count on %lu is %u, " + "should be at least %u", zp->z_id, + (int)zp->z_links, zp_is_dir + 1); + zp->z_links = zp_is_dir + 1; + } + if (--zp->z_links == zp_is_dir) { + zp->z_unlinked = B_TRUE; + zp->z_links = 0; + unlinked = B_TRUE; + } else { + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), + NULL, &ctime, sizeof (ctime)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), + NULL, &zp->z_pflags, sizeof (zp->z_pflags)); + zfs_tstamp_update_setup(zp, STATE_CHANGED, mtime, + ctime); + } + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_LINKS(zfsvfs), + NULL, &zp->z_links, sizeof (zp->z_links)); + error = sa_bulk_update(zp->z_sa_hdl, bulk, count, tx); + count = 0; + ASSERT(error == 0); + mutex_exit(&zp->z_lock); + } else { + error = zfs_dropname(dl, zp, dzp, tx, flag); + if (error != 0) + return (error); + } + + mutex_enter(&dzp->z_lock); + dzp->z_size--; /* one dirent removed */ + if (zp_is_dir) + dzp->z_links--; /* ".." link from zp */ + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_LINKS(zfsvfs), + NULL, &dzp->z_links, sizeof (dzp->z_links)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_SIZE(zfsvfs), + NULL, &dzp->z_size, sizeof (dzp->z_size)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), + NULL, ctime, sizeof (ctime)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MTIME(zfsvfs), + NULL, mtime, sizeof (mtime)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), + NULL, &dzp->z_pflags, sizeof (dzp->z_pflags)); + zfs_tstamp_update_setup(dzp, CONTENT_MODIFIED, mtime, ctime); + error = sa_bulk_update(dzp->z_sa_hdl, bulk, count, tx); + ASSERT(error == 0); + mutex_exit(&dzp->z_lock); + + if (unlinkedp != NULL) + *unlinkedp = unlinked; + else if (unlinked) + zfs_unlinked_add(zp, tx); + + return (0); +} + +/* + * Indicate whether the directory is empty. Works with or without z_lock + * held, but can only be consider a hint in the latter case. Returns true + * if only "." and ".." remain and there's no work in progress. + * + * The internal ZAP size, rather than zp->z_size, needs to be checked since + * some consumers (Lustre) do not strictly maintain an accurate SA_ZPL_SIZE. + */ +boolean_t +zfs_dirempty(znode_t *dzp) +{ + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + uint64_t count; + int error; + + if (dzp->z_dirlocks != NULL) + return (B_FALSE); + + error = zap_count(zfsvfs->z_os, dzp->z_id, &count); + if (error != 0 || count != 0) + return (B_FALSE); + + return (B_TRUE); +} + +int +zfs_make_xattrdir(znode_t *zp, vattr_t *vap, znode_t **xzpp, cred_t *cr) +{ + zfsvfs_t *zfsvfs = ZTOZSB(zp); + znode_t *xzp; + dmu_tx_t *tx; + int error; + zfs_acl_ids_t acl_ids; + boolean_t fuid_dirtied; +#ifdef DEBUG + uint64_t parent; +#endif + + *xzpp = NULL; + + if ((error = zfs_zaccess(zp, ACE_WRITE_NAMED_ATTRS, 0, B_FALSE, cr, + NULL))) + return (error); + + if ((error = zfs_acl_ids_create(zp, IS_XATTR, vap, cr, NULL, + &acl_ids, NULL)) != 0) + return (error); + if (zfs_acl_ids_overquota(zfsvfs, &acl_ids, zp->z_projid)) { + zfs_acl_ids_free(&acl_ids); + return (SET_ERROR(EDQUOT)); + } + + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_sa_create(tx, acl_ids.z_aclp->z_acl_bytes + + ZFS_SA_BASE_ATTR_SIZE); + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_TRUE); + dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); + fuid_dirtied = zfsvfs->z_fuid_dirty; + if (fuid_dirtied) + zfs_fuid_txhold(zfsvfs, tx); + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + zfs_acl_ids_free(&acl_ids); + dmu_tx_abort(tx); + return (error); + } + zfs_mknode(zp, vap, tx, cr, IS_XATTR, &xzp, &acl_ids); + + if (fuid_dirtied) + zfs_fuid_sync(zfsvfs, tx); + +#ifdef DEBUG + error = sa_lookup(xzp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)); + ASSERT(error == 0 && parent == zp->z_id); +#endif + + VERIFY(0 == sa_update(zp->z_sa_hdl, SA_ZPL_XATTR(zfsvfs), &xzp->z_id, + sizeof (xzp->z_id), tx)); + + if (!zp->z_unlinked) + (void) zfs_log_create(zfsvfs->z_log, tx, TX_MKXATTR, zp, + xzp, "", NULL, acl_ids.z_fuidp, vap); + + zfs_acl_ids_free(&acl_ids); + dmu_tx_commit(tx); + + /* + * Windows - attach the vnode _after_ committing the transaction + */ + zfs_znode_getvnode(xzp, zp, zfsvfs); + + *xzpp = xzp; + + return (0); +} + +/* + * Return a znode for the extended attribute directory for zp. + * ** If the directory does not already exist, it is created ** + * + * IN: zp - znode to obtain attribute directory from + * cr - credentials of caller + * flags - flags from the VOP_LOOKUP call + * + * OUT: xipp - pointer to extended attribute znode + * + * RETURN: 0 on success + * error number on failure + */ +int +zfs_get_xattrdir(znode_t *zp, znode_t **xzpp, cred_t *cr, int flags) +{ + zfsvfs_t *zfsvfs = ZTOZSB(zp); + znode_t *xzp; + zfs_dirlock_t *dl; + vattr_t va; + int error; +top: + error = zfs_dirent_lock(&dl, zp, "", &xzp, ZXATTR, NULL, NULL); + if (error) + return (error); + + if (xzp != NULL) { + *xzpp = xzp; + zfs_dirent_unlock(dl); + return (0); + } + + if (!(flags & CREATE_XATTR_DIR)) { + zfs_dirent_unlock(dl); + return (SET_ERROR(ENOENT)); + } + + if (zfs_is_readonly(zfsvfs)) { + zfs_dirent_unlock(dl); + return (SET_ERROR(EROFS)); + } + + /* + * The ability to 'create' files in an attribute + * directory comes from the write_xattr permission on the base file. + * + * The ability to 'search' an attribute directory requires + * read_xattr permission on the base file. + * + * Once in a directory the ability to read/write attributes + * is controlled by the permissions on the attribute file. + */ + va.va_mask = ATTR_TYPE | ATTR_MODE | ATTR_UID | ATTR_GID; + va.va_type = VDIR; + va.va_mode = S_IFDIR | S_ISVTX | 0777; + zfs_fuid_map_ids(zp, cr, &va.va_uid, &va.va_gid); + + error = zfs_make_xattrdir(zp, &va, xzpp, cr); + zfs_dirent_unlock(dl); + + if (error == ERESTART) { + /* NB: we already did dmu_tx_wait() if necessary */ + goto top; + } + + return (error); +} + +/* + * Decide whether it is okay to remove within a sticky directory. + * + * In sticky directories, write access is not sufficient; + * you can remove entries from a directory only if: + * + * you own the directory, + * you own the entry, + * you have write access to the entry, + * or you are privileged (checked in secpolicy...). + * + * The function returns 0 if remove access is granted. + */ +int +zfs_sticky_remove_access(znode_t *zdp, znode_t *zp, cred_t *cr) +{ + uid_t uid; + uid_t downer; + uid_t fowner; + zfsvfs_t *zfsvfs = ZTOZSB(zdp); + + if (zfsvfs->z_replay) + return (0); + + if ((zdp->z_mode & S_ISVTX) == 0) + return (0); + + downer = zfs_fuid_map_id(zfsvfs, zdp->z_uid, cr, ZFS_OWNER); + fowner = zfs_fuid_map_id(zfsvfs, zp->z_uid, cr, ZFS_OWNER); + + if ((uid = crgetuid(cr)) == downer || uid == fowner || + (vnode_isreg(ZTOV(zp)) && + zfs_zaccess(zp, ACE_WRITE_DATA, 0, B_FALSE, cr, NULL) == 0)) + return (0); + else + return (secpolicy_vnode_remove(ZTOV(zp), cr)); +} diff --git a/module/os/windows/zfs/zfs_file_os.c b/module/os/windows/zfs/zfs_file_os.c new file mode 100644 index 000000000000..d2ee44776f32 --- /dev/null +++ b/module/os/windows/zfs/zfs_file_os.c @@ -0,0 +1,431 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright(c) 2021 Jorgen Lundman + */ + +#include +#include +#include +#include +#include +#include +#include + +/* + * Open file + * + * path - fully qualified path to file + * flags - file attributes O_READ / O_WRITE / O_EXCL + * fpp - pointer to return file pointer + * + * Returns 0 on success underlying error on failure. + */ +int +zfs_file_open(const char *path, int flags, int mode, zfs_file_t **fpp) +{ + vnode_t *vp; + wchar_t buf[PATH_MAX]; + UNICODE_STRING uniName; + OBJECT_ATTRIBUTES objAttr; + HANDLE handle; + NTSTATUS ntstatus; + IO_STATUS_BLOCK ioStatusBlock; + DWORD desiredAccess = 0; + DWORD dwCreationDisposition; + + mbstowcs(buf, path, sizeof (buf)); + + desiredAccess = GENERIC_READ; + if (flags&O_WRONLY) + desiredAccess = GENERIC_WRITE; + if (flags&O_RDWR) + desiredAccess = GENERIC_READ | GENERIC_WRITE; + + switch (flags&(O_CREAT | O_TRUNC | O_EXCL)) { + case O_CREAT: + dwCreationDisposition = FILE_OPEN_IF; + break; + case O_TRUNC: + dwCreationDisposition = FILE_SUPERSEDE; + break; + case (O_CREAT | O_EXCL): + // Only creating new implies starting from 0 + case (O_CREAT | O_EXCL | O_TRUNC): + dwCreationDisposition = FILE_CREATE; + break; + case (O_CREAT | O_TRUNC): + dwCreationDisposition = FILE_OVERWRITE_IF; + break; + default: + case O_EXCL: // Invalid, ignore bit - treat as normal open + dwCreationDisposition = FILE_OPEN; + break; + } + + if (flags&O_APPEND) mode |= FILE_APPEND_DATA; + +#ifdef O_EXLOCK + // if (flags&O_EXLOCK) share &= ~FILE_SHARE_WRITE; +#endif + + RtlInitUnicodeString(&uniName, buf); + InitializeObjectAttributes(&objAttr, &uniName, + OBJ_CASE_INSENSITIVE | OBJ_KERNEL_HANDLE, + NULL, NULL); + + if (KeGetCurrentIrql() != PASSIVE_LEVEL) + return (-1); + + ntstatus = ZwCreateFile(&handle, + desiredAccess, + &objAttr, &ioStatusBlock, NULL, + FILE_ATTRIBUTE_NORMAL, + 0, + dwCreationDisposition, + FILE_SYNCHRONOUS_IO_NONALERT, + NULL, 0); + + if (ntstatus != STATUS_SUCCESS) + return (-1); + + // Since we will use DeviceObject and FileObject to do ioctl and IO + // we grab them now and lock them in place. + // Convert HANDLE to FileObject + PFILE_OBJECT FileObject; + PDEVICE_OBJECT DeviceObject; + NTSTATUS status; + + // This adds a reference to FileObject + status = ObReferenceObjectByHandle( + handle, + 0, + *IoFileObjectType, + KernelMode, + &FileObject, + NULL); + if (status != STATUS_SUCCESS) { + ZwClose(handle); + return (EIO); + } + + // Convert FileObject to DeviceObject + DeviceObject = IoGetRelatedDeviceObject(FileObject); + + // Grab a reference to DeviceObject + ObReferenceObject(DeviceObject); + + zfs_file_t *fp; + fp = (zfs_file_t *)kmem_zalloc(sizeof (zfs_file_t), KM_SLEEP); + fp->f_vnode = NULL; + fp->f_handle = handle; + fp->f_fileobject = FileObject; + fp->f_deviceobject = DeviceObject; + + *fpp = fp; + + return (0); +} + +void +zfs_file_close(zfs_file_t *fp) +{ + if (fp->f_fileobject != NULL) + ObDereferenceObject(fp->f_fileobject); + if (fp->f_deviceobject != NULL) + ObDereferenceObject(fp->f_deviceobject); + + ZwClose(fp->f_handle); + + kmem_free(fp, sizeof (zfs_file_t)); +} + +/* + * Stateful write - use os internal file pointer to determine where to + * write and update on successful completion. + * + * fp - pointer to file (pipe, socket, etc) to write to + * buf - buffer to write + * count - # of bytes to write + * resid - pointer to count of unwritten bytes (if short write) + * + * Returns 0 on success errno on failure. + */ +int +zfs_file_write(zfs_file_t *fp, const void *buf, size_t count, ssize_t *resid) +{ + NTSTATUS ntstatus; + IO_STATUS_BLOCK ioStatusBlock; + + ntstatus = ZwWriteFile(fp->f_handle, NULL, NULL, NULL, + &ioStatusBlock, buf, count, NULL, NULL); + + if (resid) + *resid = 0; + + if (STATUS_SUCCESS != ntstatus) + return (EIO); + return (0); +} + +/* + * Stateful read - use os internal file pointer to determine where to + * read and update on successful completion. + * + * fp - pointer to file (pipe, socket, etc) to read from + * buf - buffer to write + * count - # of bytes to read + * resid - pointer to count of unread bytes (if short read) + * + * Returns 0 on success errno on failure. + */ +int +zfs_file_read(zfs_file_t *fp, void *buf, size_t count, ssize_t *resid) +{ + NTSTATUS ntstatus; + IO_STATUS_BLOCK ioStatusBlock; + ntstatus = ZwReadFile(fp->f_handle, NULL, NULL, NULL, + &ioStatusBlock, buf, count, NULL, NULL); + if (STATUS_SUCCESS != ntstatus) + return (EIO); + if (resid) + *resid = 0; + return (0); +} + +/* + * Stateless write - os internal file pointer is not updated. + * + * fp - pointer to file (pipe, socket, etc) to write to + * buf - buffer to write + * count - # of bytes to write + * off - file offset to write to (only valid for seekable types) + * resid - pointer to count of unwritten bytes + * + * Returns 0 on success errno on failure. + */ +int +zfs_file_pwrite(zfs_file_t *fp, const void *buf, size_t count, loff_t off, + ssize_t *resid) +{ + NTSTATUS ntstatus; + IO_STATUS_BLOCK ioStatusBlock; + LARGE_INTEGER offset = { 0 }; + offset.QuadPart = off; + ntstatus = ZwWriteFile(fp->f_handle, NULL, NULL, NULL, + &ioStatusBlock, buf, count, &offset, NULL); + // reset fp to its original position + if (STATUS_SUCCESS != ntstatus) + return (EIO); + if (resid) + *resid = 0; + return (0); +} + +/* + * Stateless read - os internal file pointer is not updated. + * + * fp - pointer to file (pipe, socket, etc) to read from + * buf - buffer to write + * count - # of bytes to write + * off - file offset to read from (only valid for seekable types) + * resid - pointer to count of unwritten bytes (if short write) + * + * Returns 0 on success errno on failure. + */ +int +zfs_file_pread(zfs_file_t *fp, void *buf, size_t count, loff_t off, + ssize_t *resid) +{ + NTSTATUS ntstatus; + IO_STATUS_BLOCK ioStatusBlock; + LARGE_INTEGER offset = { 0 }; + offset.QuadPart = off; + ntstatus = ZwReadFile(fp->f_handle, NULL, NULL, NULL, + &ioStatusBlock, buf, count, &offset, NULL); + if (STATUS_SUCCESS != ntstatus) + return (EIO); + if (resid) + *resid = 0; + return (0); +} + +/* + * Sync file to disk + * + * hFile - handle to file + * + * Returns 0 on success or error code of underlying sync call on failure. + */ +int +zfs_file_fsync(zfs_file_t *fp, int flags) +{ + if (KeGetCurrentIrql() != PASSIVE_LEVEL) + return (-1); + IO_STATUS_BLOCK ioStatusBlock; + NTSTATUS ntStatus; + ntStatus = ZwFlushBuffersFile( + fp->f_handle, + &ioStatusBlock); + if (ntStatus != STATUS_SUCCESS) { + return (-1); + } + return (0); +} +/* + * fallocate - allocate or free space on disk + * + * fp - file pointer + * mode (non-standard options for hole punching etc) + * offset - offset to start allocating or freeing from + * len - length to free / allocate + * + * OPTIONAL + */ +int +zfs_file_fallocate(zfs_file_t *fp, int mode, loff_t offset, loff_t len) +{ + int error; + struct flock flck; + + memset(&flck, 0, sizeof (flck)); + flck.l_type = F_FREESP; + flck.l_start = offset; + flck.l_len = len; + flck.l_whence = 0; + + error = VOP_SPACE(fp->f_handle, F_FREESP, &flck, + 0, 0, kcred, NULL); + + return (error); +} + +/* + * Get file attributes + * + * filp - file pointer + * zfattr - pointer to file attr structure + * + * Currently only used for fetching size and file mode. + * + * Returns 0 on success or error code of underlying getattr call on failure. + */ +int +zfs_file_getattr(zfs_file_t *fp, zfs_file_attr_t *zfattr) +{ + FILE_STANDARD_INFORMATION fileInfo = { 0 }; + IO_STATUS_BLOCK ioStatusBlock; + NTSTATUS ntStatus; + ntStatus = ZwQueryInformationFile( + fp->f_handle, + &ioStatusBlock, + &fileInfo, + sizeof (fileInfo), + FileStandardInformation); + if (ntStatus != STATUS_SUCCESS) { + return (-1); + } + zfattr->zfa_size = fileInfo.EndOfFile.QuadPart; + return (0); +} + +/* + * Request current file pointer offset + * + * fp - pointer to file + * + * Returns current file offset. + */ +loff_t +zfs_file_off(zfs_file_t *fp) +{ + return (fp->f_offset); +} + +/* + * Request file pointer private data + * + * fp - pointer to file + * + * Returns pointer to file private data. + */ +extern kmutex_t zfsdev_state_lock; +dev_t zfsdev_get_dev(void); + +void * +zfs_file_private(zfs_file_t *fp) +{ + dev_t dev; + void *zs; + + dev = zfsdev_get_dev(); + dprintf("%s: fetching dev x%x\n", __func__, dev); + if (dev == 0) + return (NULL); + + mutex_enter(&zfsdev_state_lock); + zs = zfsdev_get_state(minor(dev), ZST_ALL); + mutex_exit(&zfsdev_state_lock); + dprintf("%s: searching minor %d %p\n", __func__, minor(dev), zs); + + return (zs); +} + +/* + * unlink file + * + * path - fully qualified file path + * + * Returns 0 on success. + * + * OPTIONAL + */ +int +zfs_file_unlink(const char *path) +{ + return (EOPNOTSUPP); +} + +/* + * Get reference to file pointer + * + * fd - input file descriptor + * fpp - pointer to file pointer + * + * Returns 0 on success EBADF on failure. + */ +zfs_file_t * +zfs_file_get(int fd) +{ + return (getf(fd)); +} + +/* + * Drop reference to file pointer + * + * fd - input file descriptor + */ +void +zfs_file_put(zfs_file_t *fp) +{ + releasefp(fp); +} diff --git a/module/os/windows/zfs/zfs_fuid_os.c b/module/os/windows/zfs/zfs_fuid_os.c new file mode 100644 index 000000000000..8d6e9b9c54cd --- /dev/null +++ b/module/os/windows/zfs/zfs_fuid_os.c @@ -0,0 +1,52 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2007, 2010, Oracle and/or its affiliates. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#ifdef _KERNEL +#include +#include +#include +#endif +#include + +uint64_t +zfs_fuid_create_cred(zfsvfs_t *zfsvfs, zfs_fuid_type_t type, + cred_t *cr, zfs_fuid_info_t **fuidp) +{ + uid_t id; + + VERIFY(type == ZFS_OWNER || type == ZFS_GROUP); + + id = (type == ZFS_OWNER) ? crgetuid(cr) : crgetgid(cr); + + if (IS_EPHEMERAL(id)) + return ((type == ZFS_OWNER) ? UID_NOBODY : GID_NOBODY); + + return ((uint64_t)id); +} diff --git a/module/os/windows/zfs/zfs_ioctl_os.c b/module/os/windows/zfs/zfs_ioctl_os.c new file mode 100644 index 000000000000..41895fd3efad --- /dev/null +++ b/module/os/windows/zfs/zfs_ioctl_os.c @@ -0,0 +1,1232 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2013, 2020 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include + +#include + + + +typedef struct { + unsigned __int64 read_iops; + unsigned __int64 write_iops; + unsigned __int64 total_iops; + unsigned __int64 read_bytes; + unsigned __int64 write_bytes; + unsigned __int64 total_bytes; + unsigned __int64 ddt_entry_count; // number of elments in ddt ,zpool only + unsigned __int64 ddt_dspace; // size of ddt on disk ,zpool only + unsigned __int64 ddt_mspace; // size of ddt in-core ,zpool only + unsigned __int64 vsx_active_queue_sync_read; + unsigned __int64 vsx_active_queue_sync_write; + unsigned __int64 vsx_active_queue_async_read; + unsigned __int64 vsx_active_queue_async_write; + unsigned __int64 vsx_pend_queue_sync_read; + unsigned __int64 vsx_pend_queue_sync_write; + unsigned __int64 vsx_pend_queue_async_read; + unsigned __int64 vsx_pend_queue_async_write; + unsigned __int64 vsx_queue_histo_sync_read_time; + unsigned __int64 vsx_queue_histo_sync_read_count; + unsigned __int64 vsx_queue_histo_async_read_time; + unsigned __int64 vsx_queue_histo_async_read_count; + unsigned __int64 vsx_queue_histo_sync_write_time; + unsigned __int64 vsx_queue_histo_sync_write_count; + unsigned __int64 vsx_queue_histo_async_write_time; + unsigned __int64 vsx_queue_histo_async_write_count; + unsigned __int64 vsx_total_histo_read_time; + unsigned __int64 vsx_total_histo_read_count; + unsigned __int64 vsx_total_histo_write_time; + unsigned __int64 vsx_total_histo_write_count; + unsigned __int64 vsx_disk_histo_read_time; + unsigned __int64 vsx_disk_histo_read_count; + unsigned __int64 vsx_disk_histo_write_time; + unsigned __int64 vsx_disk_histo_write_count; + unsigned __int64 dp_dirty_total_io; // zpool only +} zpool_perf_counters; + +NTSTATUS NTAPI +ZFSinPerfCallBack(PCW_CALLBACK_TYPE Type, PPCW_CALLBACK_INFORMATION Info, + PVOID Context); + +void ZFSinPerfCollect(PCW_MASK_INFORMATION CollectData); +void ZFSinPerfVdevCollect(PCW_MASK_INFORMATION CollectData); +void ZFSinCachePerfCollect(PCW_MASK_INFORMATION CollectData); + +PUNICODE_STRING MapInvalidChars(PUNICODE_STRING InstanceName); + +void ZFSinPerfEnumerate(PCW_MASK_INFORMATION EnumerateInstances); +void ZFSinPerfVdevEnumerate(PCW_MASK_INFORMATION EnumerateInstances); +void ZFSinCachePerfEnumerate(PCW_MASK_INFORMATION EnumerateInstances); + +#include +#include +#include "../OpenZFS_perf.h" +#include "../OpenZFS_counters.h" +#include + +// extern void zfs_windows_vnops_callback(PDEVICE_OBJECT deviceObject); + +int zfs_major = 0; +int zfs_bmajor = 0; +static void *zfs_devnode = NULL; +#define ZFS_MAJOR -24 + +boolean_t +zfs_vfs_held(zfsvfs_t *zfsvfs) +{ + return (zfsvfs->z_vfs != NULL); +} + +int +zfs_vfs_ref(zfsvfs_t **zfvp) +{ + int error = 0; + + if (*zfvp == NULL || (*zfvp)->z_vfs == NULL) + return (SET_ERROR(ESRCH)); + + error = vfs_busy((*zfvp)->z_vfs, LK_NOWAIT); + if (error != 0) { + *zfvp = NULL; + error = SET_ERROR(ESRCH); + } + return (error); +} + +NTSTATUS NTAPI +ZFSinPerfCallBack(PCW_CALLBACK_TYPE Type, PPCW_CALLBACK_INFORMATION Info, + PVOID Context) +{ + UNREFERENCED_PARAMETER(Context); + + switch (Type) { + case PcwCallbackEnumerateInstances: + { + ZFSinPerfEnumerate(Info->EnumerateInstances); + break; + } + case PcwCallbackCollectData: + { + ZFSinPerfCollect(Info->CollectData); + + break; + } + default: break; + } + + return (STATUS_SUCCESS); +} + +NTSTATUS NTAPI +ZFSinPerfVdevCallBack(PCW_CALLBACK_TYPE Type, PPCW_CALLBACK_INFORMATION Info, + PVOID Context) +{ + UNREFERENCED_PARAMETER(Context); + + switch (Type) { + case PcwCallbackEnumerateInstances: + { + ZFSinPerfVdevEnumerate(Info->EnumerateInstances); + + break; + } + case PcwCallbackCollectData: + { + ZFSinPerfVdevCollect(Info->CollectData); + + break; + } + default: break; + } + + return (STATUS_SUCCESS); +} + +NTSTATUS NTAPI +ZFSinCachePerfCallBack(PCW_CALLBACK_TYPE Type, PPCW_CALLBACK_INFORMATION + Info, PVOID Context) +{ + UNREFERENCED_PARAMETER(Context); + + switch (Type) { + case PcwCallbackEnumerateInstances: + { + ZFSinCachePerfEnumerate(Info->EnumerateInstances); + + break; + } + case PcwCallbackCollectData: + { + ZFSinCachePerfCollect(Info->CollectData); + + break; + } + default: break; + } + + return (STATUS_SUCCESS); + +} + + + + +PUNICODE_STRING +MapInvalidChars(PUNICODE_STRING InstanceName) +{ + const WCHAR wInvalidChars[] = L"()#\\/"; + const WCHAR wMappedChars[] = L"[]___"; + const LONG lArraySize = ARRAY_SIZE(wInvalidChars) - 1; + + for (LONG i = 0; i < InstanceName->Length / sizeof (WCHAR); ++i) { + for (LONG j = 0; j < lArraySize; ++j) { + if (InstanceName->Buffer[i] == wInvalidChars[j]) { + InstanceName->Buffer[i] = wMappedChars[j]; + break; + } + } + } + return (InstanceName); +} + +void +ZFSinPerfVdevEnumerate(PCW_MASK_INFORMATION EnumerateInstances) +{ + spa_t *spa_perf = NULL; + NTSTATUS status; + UNICODE_STRING unicodeName; + unicodeName.Buffer = kmem_alloc(sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + unicodeName.MaximumLength = ZFS_MAX_DATASET_NAME_LEN; + mutex_enter(&spa_namespace_lock); + while ((spa_perf = spa_next(spa_perf)) != NULL) { + vdev_t *vd = spa_perf->spa_root_vdev; + char vdev_zpool[ZFS_MAX_DATASET_NAME_LEN] = { 0 }; + + for (int c = 0; c < vd->vdev_children; c++) { + char *vdev_name = vd->vdev_child[c]->vdev_path; + if (!vdev_name || !vdev_name[0]) + continue; + + snprintf(vdev_zpool, ZFS_MAX_DATASET_NAME_LEN, "%s_%s", + vdev_name + 5, spa_perf->spa_name); + /* Neglecting first five characters of vdev_name */ + + ANSI_STRING ansi_vdev; + RtlInitAnsiString(&ansi_vdev, vdev_zpool); + status = RtlAnsiStringToUnicodeString(&unicodeName, + &ansi_vdev, FALSE); + + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: Ansi to Unicode string conversion failed for %Z\n", + __func__, __LINE__, &ansi_vdev); + continue; + } + + status = AddZFSinPerfVdev(EnumerateInstances.Buffer, + MapInvalidChars(&unicodeName), 0, NULL); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: AddZFSinPerfVdev failed - status 0x%x\n", + __func__, __LINE__, status); + } + } + } + mutex_exit(&spa_namespace_lock); + UNICODE_STRING total; + RtlInitUnicodeString(&total, L"_Total"); + status = AddZFSinPerfVdev(EnumerateInstances.Buffer, + MapInvalidChars(&total), 0, NULL); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: AddZFSinPerfVdev failed - status 0x%x\n", + __func__, __LINE__, status); + } + kmem_free(unicodeName.Buffer, sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN); +} + + +void ZFSinPerfEnumerate(PCW_MASK_INFORMATION EnumerateInstances) { + NTSTATUS status = STATUS_SUCCESS; + UNICODE_STRING unicodeName; + unicodeName.Buffer = kmem_alloc(sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + unicodeName.MaximumLength = ZFS_MAX_DATASET_NAME_LEN; + + spa_t *spa_perf = NULL; + ANSI_STRING ansi_spa; + + mutex_enter(&spa_namespace_lock); + while ((spa_perf = spa_next(spa_perf)) != NULL) { + spa_open_ref(spa_perf, FTAG); + RtlInitAnsiString(&ansi_spa, spa_perf->spa_name); + spa_close(spa_perf, FTAG); + + status = RtlAnsiStringToUnicodeString(&unicodeName, &ansi_spa, + FALSE); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: Ansi to Unicode string conversion failed for %Z\n", + __func__, __LINE__, &ansi_spa); + continue; + } + + status = AddZFSinPerf(EnumerateInstances.Buffer, + MapInvalidChars(&unicodeName), 0, NULL); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: AddZFSinPerf failed - status 0x%x\n", + __func__, __LINE__, status); + } + } + mutex_exit(&spa_namespace_lock); + + UNICODE_STRING total; + RtlInitUnicodeString(&total, L"_Total"); + status = AddZFSinPerf(EnumerateInstances.Buffer, + MapInvalidChars(&total), 0, NULL); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: AddZFSinPerf failed - status 0x%x\n", + __func__, __LINE__, status); + } + + kmem_free(unicodeName.Buffer, sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN); +} + +void ZFSinCachePerfEnumerate(PCW_MASK_INFORMATION EnumerateInstances) { + UNICODE_STRING unicodeName; + unicodeName.Buffer = kmem_alloc(sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + unicodeName.MaximumLength = ZFS_MAX_DATASET_NAME_LEN; + + ANSI_STRING ansi_spa; + RtlInitAnsiString(&ansi_spa, "Total"); + + NTSTATUS status = RtlAnsiStringToUnicodeString( + &unicodeName, &ansi_spa, FALSE); + if (!NT_SUCCESS(status)) + { + TraceEvent(TRACE_ERROR, + "%s:%d: Ansi to Unicode string conversion failed for %Z\n", + __func__, __LINE__, &ansi_spa); + } + else + { + status = AddZFSinCachePerf(EnumerateInstances.Buffer, + MapInvalidChars(&unicodeName), 0, NULL); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: AddZFSinCachePerf failed - status 0x%x\n", + __func__, __LINE__, status); + } + } + kmem_free(unicodeName.Buffer, sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN); +} + +void +latency_stats(uint64_t *histo, unsigned int buckets, stat_pair* lat) +{ + int i; + lat->count = 0; + lat->total = 0; + + for (i = 0; i < buckets; i++) { + /* + * Our buckets are power-of-two latency ranges. Use the + * midpoint latency of each bucket to calculate the average. + * For example: + * + * Bucket Midpoint + * 8ns-15ns: 12ns + * 16ns-31ns: 24ns + * ... + */ + if (histo[i] != 0) { + lat->total += histo[i] * + (((1UL << i) + ((1UL << i) / 2))); + lat->count += histo[i]; + } + } +} + +void +update_perf(vdev_stat_ex_t *vsx, vdev_stat_t *vs, ddt_object_t *ddo, + dsl_pool_t *spad, zpool_perf_counters *perf) +{ + if (!perf) + return; + + if (ddo) { + perf->ddt_entry_count = ddo->ddo_count; + perf->ddt_dspace = ddo->ddo_dspace * ddo->ddo_count; + perf->ddt_mspace = ddo->ddo_mspace * ddo->ddo_count; + } + + if (vs) { + perf->read_iops = vs->vs_ops[ZIO_TYPE_READ]; + perf->write_iops = vs->vs_ops[ZIO_TYPE_WRITE]; + perf->read_bytes = vs->vs_bytes[ZIO_TYPE_READ]; + perf->write_bytes = vs->vs_bytes[ZIO_TYPE_WRITE]; + perf->total_bytes = vs->vs_bytes[ZIO_TYPE_WRITE] + + vs->vs_bytes[ZIO_TYPE_READ]; + perf->total_iops = vs->vs_ops[ZIO_TYPE_WRITE] + + vs->vs_ops[ZIO_TYPE_READ]; + } + + if (vsx) { + perf->vsx_active_queue_sync_read = + vsx->vsx_active_queue[ZIO_PRIORITY_SYNC_READ]; + perf->vsx_active_queue_sync_write = + vsx->vsx_active_queue[ZIO_PRIORITY_SYNC_WRITE]; + perf->vsx_active_queue_async_read = + vsx->vsx_active_queue[ZIO_PRIORITY_ASYNC_READ]; + perf->vsx_active_queue_async_write = + vsx->vsx_active_queue[ZIO_PRIORITY_ASYNC_WRITE]; + perf->vsx_pend_queue_sync_read = + vsx->vsx_pend_queue[ZIO_PRIORITY_SYNC_READ]; + perf->vsx_pend_queue_sync_write = + vsx->vsx_pend_queue[ZIO_PRIORITY_SYNC_WRITE]; + perf->vsx_pend_queue_async_read = + vsx->vsx_pend_queue[ZIO_PRIORITY_ASYNC_READ]; + perf->vsx_pend_queue_async_write = + vsx->vsx_pend_queue[ZIO_PRIORITY_ASYNC_WRITE]; + + + stat_pair lat; + latency_stats(&vsx->vsx_queue_histo[ZIO_PRIORITY_SYNC_READ][0], + VDEV_L_HISTO_BUCKETS, &lat); + perf->vsx_queue_histo_sync_read_time = lat.total; + perf->vsx_queue_histo_sync_read_count = lat.count; + + latency_stats(&vsx->vsx_queue_histo[ZIO_PRIORITY_SYNC_WRITE][0], + VDEV_L_HISTO_BUCKETS, &lat); + perf->vsx_queue_histo_sync_write_time = lat.total; + perf->vsx_queue_histo_sync_write_count = lat.count; + + latency_stats(&vsx->vsx_queue_histo[ZIO_PRIORITY_ASYNC_READ][0], + VDEV_L_HISTO_BUCKETS, &lat); + perf->vsx_queue_histo_async_read_time = lat.total; + perf->vsx_queue_histo_async_read_count = lat.count; + + latency_stats(&vsx->vsx_queue_histo[ZIO_PRIORITY_ASYNC_WRITE] + [0], VDEV_L_HISTO_BUCKETS, &lat); + perf->vsx_queue_histo_async_write_time = lat.total; + perf->vsx_queue_histo_async_write_count = lat.count; + + latency_stats(&vsx->vsx_total_histo[ZIO_TYPE_READ][0], + VDEV_L_HISTO_BUCKETS, &lat); + perf->vsx_total_histo_read_time = lat.total; + perf->vsx_total_histo_read_count = lat.count; + + latency_stats(&vsx->vsx_total_histo[ZIO_TYPE_WRITE][0], + VDEV_L_HISTO_BUCKETS, &lat); + perf->vsx_total_histo_write_time = lat.total; + perf->vsx_total_histo_write_count = lat.count; + + latency_stats(&vsx->vsx_disk_histo[ZIO_TYPE_READ][0], + VDEV_L_HISTO_BUCKETS, &lat); + perf->vsx_disk_histo_read_time = lat.total; + perf->vsx_disk_histo_read_count = lat.count; + + latency_stats(&vsx->vsx_disk_histo[ZIO_TYPE_WRITE][0], + VDEV_L_HISTO_BUCKETS, &lat); + perf->vsx_disk_histo_write_time = lat.total; + perf->vsx_disk_histo_write_count = lat.count; + } + + if (spad) + perf->dp_dirty_total_io = spad->dp_dirty_total; +} + + +void +update_total_perf(zpool_perf_counters* perf, zpool_perf_counters* total_perf) +{ + total_perf->ddt_entry_count += perf->ddt_entry_count; + total_perf->ddt_dspace += perf->ddt_dspace; + total_perf->ddt_mspace += perf->ddt_mspace; + total_perf->read_iops += perf->read_iops; + total_perf->write_iops += perf->write_iops; + total_perf->read_bytes += perf->read_bytes; + total_perf->write_bytes += perf->write_bytes; + total_perf->total_iops += (perf->read_iops + perf->write_iops); + total_perf->total_bytes += (perf->read_bytes + perf->write_bytes); + total_perf->vsx_active_queue_sync_read += + perf->vsx_active_queue_sync_read; + total_perf->vsx_active_queue_sync_write += + perf->vsx_active_queue_sync_write; + total_perf->vsx_active_queue_async_read += + perf->vsx_active_queue_async_read; + total_perf->vsx_active_queue_async_write += + perf->vsx_active_queue_async_write; + total_perf->vsx_pend_queue_sync_read += + perf->vsx_pend_queue_sync_read; + total_perf->vsx_pend_queue_sync_write += + perf->vsx_pend_queue_sync_write; + total_perf->vsx_pend_queue_async_read += + perf->vsx_pend_queue_async_read; + total_perf->vsx_pend_queue_async_write += + perf->vsx_pend_queue_async_write; + total_perf->vsx_disk_histo_read_time += + perf->vsx_disk_histo_read_time; + total_perf->vsx_disk_histo_read_count += + perf->vsx_disk_histo_read_count; + total_perf->vsx_disk_histo_write_time += + perf->vsx_disk_histo_write_time; + total_perf->vsx_disk_histo_write_count += + perf->vsx_disk_histo_write_count; + total_perf->vsx_total_histo_read_time += + perf->vsx_total_histo_read_time; + total_perf->vsx_total_histo_read_count += + perf->vsx_total_histo_read_count; + total_perf->vsx_total_histo_write_time += + perf->vsx_total_histo_write_time; + total_perf->vsx_total_histo_write_count += + perf->vsx_total_histo_write_count; + total_perf->vsx_queue_histo_sync_read_time += + perf->vsx_queue_histo_sync_read_time; + total_perf->vsx_queue_histo_sync_read_count += + perf->vsx_queue_histo_sync_read_count; + total_perf->vsx_queue_histo_sync_write_time += + perf->vsx_queue_histo_sync_write_time; + total_perf->vsx_queue_histo_sync_write_count += + perf->vsx_queue_histo_sync_write_count; + total_perf->vsx_queue_histo_async_read_time += + perf->vsx_queue_histo_async_read_time; + total_perf->vsx_queue_histo_async_read_count += + perf->vsx_queue_histo_async_read_count; + total_perf->vsx_queue_histo_async_write_time += + perf->vsx_queue_histo_async_write_time; + total_perf->vsx_queue_histo_async_write_count += + perf->vsx_queue_histo_async_write_count; + total_perf->dp_dirty_total_io += perf->dp_dirty_total_io; +} + + +void ZFSinPerfCollect(PCW_MASK_INFORMATION CollectData) { + NTSTATUS status = STATUS_SUCCESS; + UNICODE_STRING unicodeName; + unicodeName.Buffer = kmem_alloc(sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + unicodeName.MaximumLength = ZFS_MAX_DATASET_NAME_LEN; + + ANSI_STRING ansi_spa; + spa_t *spa_perf = NULL; + zpool_perf_counters total_perf = { 0 }; + + mutex_enter(&spa_namespace_lock); + while ((spa_perf = spa_next(spa_perf)) != NULL) { + zpool_perf_counters perf = { 0 }; + spa_open_ref(spa_perf, FTAG); + RtlInitAnsiString(&ansi_spa, spa_perf->spa_name); + ddt_object_t ddo = { 0 }; + vdev_stat_t vs = { 0 }; + vdev_stat_ex_t vsx = { 0 }; + + spa_config_enter(spa_perf, SCL_ALL, FTAG, RW_READER); + vdev_get_stats_ex(spa_perf->spa_root_vdev, &vs, &vsx); + ddt_get_dedup_object_stats(spa_perf, &ddo); + dsl_pool_t *spad = spa_get_dsl(spa_perf); + + update_perf(&vsx, &vs, &ddo, spad, &perf); + spa_config_exit(spa_perf, SCL_ALL, FTAG); + spa_close(spa_perf, FTAG); + + update_total_perf(&perf, &total_perf); + + status = RtlAnsiStringToUnicodeString(&unicodeName, &ansi_spa, + FALSE); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: Ansi to Unicode string conversion failed for %Z\n", + __func__, __LINE__, &ansi_spa); + continue; + } + + status = AddZFSinPerf(CollectData.Buffer, + MapInvalidChars(&unicodeName), + 0, &perf); + + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: AddZFSinPerf failed - status 0x%x\n", + __func__, __LINE__, status); + } + } + mutex_exit(&spa_namespace_lock); + + UNICODE_STRING total; + RtlInitUnicodeString(&total, L"_Total"); + status = AddZFSinPerf(CollectData.Buffer, MapInvalidChars(&total), + 0, &total_perf); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: AddZFSinPerf failed-status 0x%x\n", + __func__, __LINE__, status); + } + + kmem_free(unicodeName.Buffer, sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN); +} + + +void ZFSinPerfVdevCollect(PCW_MASK_INFORMATION CollectData) { + NTSTATUS status = STATUS_SUCCESS; + UNICODE_STRING unicodeName; + unicodeName.Buffer = kmem_alloc(sizeof (WCHAR) + * ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + unicodeName.MaximumLength = ZFS_MAX_DATASET_NAME_LEN; + + spa_t *spa_perf = NULL; + zpool_perf_counters total_perf_vdev = { 0 }; + mutex_enter(&spa_namespace_lock); + while ((spa_perf = spa_next(spa_perf)) != NULL) { + spa_config_enter(spa_perf, SCL_ALL, FTAG, RW_READER); + vdev_t *vd = spa_perf->spa_root_vdev; + char vdev_zpool[ZFS_MAX_DATASET_NAME_LEN] = { 0 }; + zpool_perf_counters perf_vdev = { 0 }; + + for (int c = 0; c < vd->vdev_children; c++) { + char *vdev_name = vd->vdev_child[c]->vdev_path; + if (!vdev_name || !vdev_name[0]) + continue; + + snprintf(vdev_zpool, ZFS_MAX_DATASET_NAME_LEN, "%s_%s", + vdev_name + 5, spa_perf-> + spa_name); + // Neglecting first five characters of vdev_name + + ANSI_STRING ansi_vdev; + RtlInitAnsiString(&ansi_vdev, vdev_zpool); + status = RtlAnsiStringToUnicodeString(&unicodeName, + &ansi_vdev, FALSE); + + vdev_t *cvd = vd->vdev_child[c]; + + update_perf(&cvd->vdev_stat_ex, &cvd->vdev_stat, NULL, + NULL, &perf_vdev); + update_total_perf(&perf_vdev, &total_perf_vdev); + + status = AddZFSinPerfVdev(CollectData.Buffer, + MapInvalidChars(&unicodeName), 0, &perf_vdev); + + if (!NT_SUCCESS(status)) { + TraceEvent( + TRACE_ERROR, + "%s:%d: AddZFSinPerfVdev failed-status 0x%x\n", + __func__, __LINE__, status); + } + } + spa_config_exit(spa_perf, SCL_ALL, FTAG); + } + mutex_exit(&spa_namespace_lock); + + UNICODE_STRING total; + RtlInitUnicodeString(&total, L"_Total"); + status = AddZFSinPerfVdev(CollectData.Buffer, MapInvalidChars(&total), + 0, &total_perf_vdev); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: AddZFSinPerfVdev failed-status 0x%x\n", + __func__, __LINE__, status); + } + kmem_free(unicodeName.Buffer, sizeof (WCHAR) + * ZFS_MAX_DATASET_NAME_LEN); +} + +extern kstat_t *perf_arc_ksp, *perf_zil_ksp; +void ZFSinCachePerfCollect(PCW_MASK_INFORMATION CollectData) { + UNICODE_STRING unicodeName; + unicodeName.Buffer = kmem_alloc(sizeof (WCHAR) * + ZFS_MAX_DATASET_NAME_LEN, KM_SLEEP); + unicodeName.MaximumLength = ZFS_MAX_DATASET_NAME_LEN; + + ANSI_STRING ansi_spa; + RtlInitAnsiString(&ansi_spa, "Total"); + + NTSTATUS status = RtlAnsiStringToUnicodeString(&unicodeName, + &ansi_spa, FALSE); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d: Ansi to Unicode string conversion failed for %Z\n", + __func__, __LINE__, &ansi_spa); + } else { + cache_counters perf_cache = { 0 }; + + KSTAT_ENTER(perf_arc_ksp); + int error = KSTAT_UPDATE(perf_arc_ksp, KSTAT_READ); + if (!error) + arc_cache_counters_perfmon(&perf_cache, + perf_arc_ksp->ks_data); + KSTAT_EXIT(perf_arc_ksp); + + KSTAT_ENTER(perf_zil_ksp); + error = KSTAT_UPDATE(perf_zil_ksp, KSTAT_READ); + if (!error) + zil_cache_counters_perfmon(&perf_cache, + perf_zil_ksp->ks_data); + KSTAT_EXIT(perf_zil_ksp); + + status = AddZFSinCachePerf(CollectData.Buffer, + MapInvalidChars(&unicodeName), 0, &perf_cache); + if (!NT_SUCCESS(status)) { + TraceEvent(TRACE_ERROR, + "%s:%d:AddZFSinCachePerf failed-status 0x%x\n", + __func__, __LINE__, status); + } + } + kmem_free(unicodeName.Buffer, + sizeof (WCHAR) * ZFS_MAX_DATASET_NAME_LEN); +} + + +void +zfs_vfs_rele(zfsvfs_t *zfsvfs) +{ + vfs_unbusy(zfsvfs->z_vfs); +} + +static uint_t zfsdev_private_tsd; + +dev_t +zfsdev_get_dev(void) +{ + return ((dev_t)tsd_get(zfsdev_private_tsd)); +} + +/* We can't set ->private method, so this function does nothing */ +void +zfsdev_private_set_state(void *priv, zfsdev_state_t *zs) +{ + UNREFERENCED_PARAMETER(priv); + UNREFERENCED_PARAMETER(zs); +} + +/* Loop all zs looking for matching dev_t */ +zfsdev_state_t * +zfsdev_private_get_state(void *priv) +{ + dev_t dev = (dev_t)priv; + zfsdev_state_t *zs; + mutex_enter(&zfsdev_state_lock); + zs = zfsdev_get_state(dev, ZST_ALL); + mutex_exit(&zfsdev_state_lock); + return (zs); +} + +static NTSTATUS +zfsdev_open(dev_t dev, PIRP Irp) +{ + int error; + int flags = 0; + int devtype = 0; + struct proc *p = current_proc(); + PAGED_CODE(); + + mutex_enter(&zfsdev_state_lock); + if (zfsdev_get_state(minor(dev), ZST_ALL)) { + mutex_exit(&zfsdev_state_lock); + return (0); + } + error = zfsdev_state_init((void *)dev); + mutex_exit(&zfsdev_state_lock); + + return (-error); +} + +static NTSTATUS +zfsdev_release(dev_t dev, PIRP Irp) +{ + /* zfsdev_state_destroy() doesn't check for NULL, so pre-lookup here */ + void *priv; + + priv = (void *)(uintptr_t)minor(dev); + zfsdev_state_t *zs = zfsdev_private_get_state(priv); + if (zs != NULL) + zfsdev_state_destroy(priv); + return (0); +} + +static NTSTATUS +zfsdev_ioctl(PDEVICE_OBJECT DeviceObject, PIRP Irp, int flag) +{ + uint_t len, vecnum; + zfs_iocparm_t zit; + zfs_cmd_t *zc; + int error, rc; + user_addr_t uaddr; + ulong_t cmd = 0; + caddr_t arg = NULL; + + PIO_STACK_LOCATION irpSp; + irpSp = IoGetCurrentIrpStackLocation(Irp); + + len = irpSp->Parameters.DeviceIoControl.InputBufferLength; + cmd = irpSp->Parameters.DeviceIoControl.IoControlCode; + arg = irpSp->Parameters.DeviceIoControl.Type3InputBuffer; + + // vecnum = cmd - CTL_CODE(ZFSIOCTL_TYPE, ZFSIOCTL_BASE, + // METHOD_NEITHER, FILE_ANY_ACCESS); + + vecnum = DEVICE_FUNCTION_FROM_CTL_CODE(cmd); + ASSERT3U(vecnum, >=, ZFSIOCTL_BASE + ZFS_IOC_FIRST); + ASSERT3U(vecnum, <, ZFSIOCTL_BASE + ZFS_IOC_LAST); + vecnum -= ZFSIOCTL_BASE; + + if (len != sizeof (zfs_iocparm_t)) { + /* + * printf("len %d vecnum: %d sizeof (zfs_cmd_t) %lu\n", + * len, vecnum, sizeof (zfs_cmd_t)); + */ + return (EINVAL); + } + + // Copy in the wrapper, which contains real zfs_cmd_t addr, len, + // and compat version + error = ddi_copyin((void *)arg, &zit, len, 0); + if (error != 0) + return (EINVAL); + + uaddr = (user_addr_t)zit.zfs_cmd; + + // get ready for zfs_cmd_t + zc = kmem_zalloc(sizeof (zfs_cmd_t), KM_SLEEP); + + if (copyin((void *)uaddr, zc, sizeof (zfs_cmd_t))) { + error = SET_ERROR(EFAULT); + goto out; + } + + error = zfsdev_ioctl_common(vecnum, zc, 0); + + rc = copyout(zc, (void *)uaddr, sizeof (zfs_cmd_t)); + + if (error == 0 && rc != 0) + error = -SET_ERROR(EFAULT); + + // Set the real return code in struct. + // XNU only calls copyout if error=0, but + // presumably we can skip that in Windows and just return? + zit.zfs_ioc_error = error; + error = ddi_copyout(&zit, (void *)arg, len, 0); + error = 0; + +out: + kmem_free(zc, sizeof (zfs_cmd_t)); + return (error); + +} + +/* + * inputs: + * zc_name dataset name to mount + * zc_value path location to mount + * + * outputs: + * return code + */ +int zfs_windows_mount(zfs_cmd_t *zc); // move me to headers + +static int +zfs_ioc_mount(zfs_cmd_t *zc) +{ + return (zfs_windows_mount(zc)); +} + +/* + * inputs: + * zc_name dataset name to unmount + * zc_value path location to unmount + * + * outputs: + * return code + */ +int zfs_windows_unmount(zfs_cmd_t *zc); // move me to headers + +static int +zfs_ioc_unmount(zfs_cmd_t *zc) +{ + dprintf("%s: enter\n", __func__); + return (zfs_windows_unmount(zc)); +} + +void +zfs_ioctl_init_os(void) +{ + /* + * Windows functions + */ + zfs_ioctl_register_legacy(ZFS_IOC_MOUNT, zfs_ioc_mount, + zfs_secpolicy_config, NO_NAME, B_FALSE, POOL_CHECK_NONE); + zfs_ioctl_register_legacy(ZFS_IOC_UNMOUNT, zfs_ioc_unmount, + zfs_secpolicy_config, NO_NAME, B_FALSE, POOL_CHECK_NONE); + +} + + +/* ioctl handler for block device. Relay to zvol */ +static int +zfsdev_bioctl(dev_t dev, ulong_t cmd, caddr_t data, + __unused int flag, struct proc *p) +{ + return (zvol_os_ioctl(dev, cmd, data, 1, NULL, NULL)); +} + +// Callback to print registered filesystems. Not needed +void +DriverNotificationRoutine(_In_ struct _DEVICE_OBJECT *DeviceObject, + _In_ BOOLEAN FsActive) +{ + CHAR nibuf[512]; // buffer that receives name information and name + POBJECT_NAME_INFORMATION name_info = (POBJECT_NAME_INFORMATION)nibuf; + ULONG ret_len; + NTSTATUS status; + + status = ObQueryNameString(DeviceObject, name_info, + sizeof (nibuf), &ret_len); + if (NT_SUCCESS(status)) { + dprintf("Filesystem %p: '%wZ'\n", DeviceObject, + &name_info->Name); + } else { + dprintf("Filesystem %p: '%wZ'\n", DeviceObject, + &DeviceObject->DriverObject->DriverName); + } +} + +// extern PDRIVER_UNLOAD STOR_DriverUnload; +uint64_t +zfs_ioc_unregister_fs(void) +{ + dprintf("%s\n", __func__); + if (zfs_module_busy != 0) { + dprintf("%s: datasets still busy: %llu pool(s)\n", __func__, + zfs_module_busy); + return (zfs_module_busy); + } + if (fsDiskDeviceObject != NULL) { + IoUnregisterFsRegistrationChange(WIN_DriverObject, + DriverNotificationRoutine); + IoUnregisterFileSystem(fsDiskDeviceObject); + ObDereferenceObject(fsDiskDeviceObject); + UNICODE_STRING ntWin32NameString; + RtlInitUnicodeString(&ntWin32NameString, ZFS_DEV_DOS); + IoDeleteSymbolicLink(&ntWin32NameString); + IoDeleteDevice(fsDiskDeviceObject); + fsDiskDeviceObject = NULL; + } +#if 0 + // Do not unload these, so that the zfsinstaller uninstall can + // find the devnode to trigger uninstall. + if (STOR_DriverUnload != NULL) { + STOR_DriverUnload(WIN_DriverObject); + STOR_DriverUnload = NULL; + } +#endif + return (0); +} + +#ifdef ZFS_DEBUG +#define ZFS_DEBUG_STR " (DEBUG mode)" +#else +#define ZFS_DEBUG_STR "" +#endif + +static int +openzfs_init_os(void) +{ + return (0); +} + +static void +openzfs_fini_os(void) +{ +} + +int +zfsdev_attach(void) +{ + NTSTATUS ntStatus; + UNICODE_STRING ntUnicodeString; // NT Device Name + UNICODE_STRING ntWin32NameString; // Win32 Name + int err; + + static UNICODE_STRING sddl = RTL_CONSTANT_STRING( + L"D:P(A;;GA;;;SY)(A;;GRGWGX;;;BA)" + "(A;;GRGWGX;;;WD)(A;;GRGX;;;RC)"); + // Or use &SDDL_DEVOBJ_SYS_ALL_ADM_RWX_WORLD_RW_RES_R + + RtlInitUnicodeString(&ntUnicodeString, ZFS_DEV_KERNEL); + ntStatus = IoCreateDeviceSecure( + WIN_DriverObject, + sizeof (mount_t), + &ntUnicodeString, // Device name "\Device\SIOCTL" + FILE_DEVICE_UNKNOWN, // Device type + /* FILE_DEVICE_SECURE_OPEN */ 0, // Device characteristics + FALSE, // Not an exclusive device + &sddl, + NULL, + &ioctlDeviceObject); // Returned ptr to Device Object + + if (!NT_SUCCESS(ntStatus)) { + dprintf("ZFS: Couldn't create the device object " + "/dev/zfs (%S)\n", ZFS_DEV_KERNEL); + return (ntStatus); + } + dprintf("ZFS: created kernel device node: %p: name %S\n", + ioctlDeviceObject, ZFS_DEV_KERNEL); + + UNICODE_STRING fsDiskDeviceName; + RtlInitUnicodeString(&fsDiskDeviceName, ZFS_GLOBAL_FS_DISK_DEVICE_NAME); + + ntStatus = IoCreateDeviceSecure(WIN_DriverObject, // DriverObject + sizeof (mount_t), // DeviceExtensionSize + &fsDiskDeviceName, // DeviceName + FILE_DEVICE_DISK_FILE_SYSTEM, // DeviceType + 0, + FALSE, + &sddl, + NULL, + &fsDiskDeviceObject); // DeviceObject + + ObReferenceObject(ioctlDeviceObject); + + mount_t *dgl; + dgl = ioctlDeviceObject->DeviceExtension; + dgl->type = MOUNT_TYPE_DGL; + dgl->size = sizeof (mount_t); + + mount_t *vcb; + vcb = fsDiskDeviceObject->DeviceExtension; + vcb->type = MOUNT_TYPE_VCB; + vcb->size = sizeof (mount_t); + + if (ntStatus == STATUS_SUCCESS) { + dprintf("DiskFileSystemDevice: 0x%0x %wZ created\n", + ntStatus, &fsDiskDeviceName); + } + + // Initialize a Unicode String containing the Win32 name + // for our device. + RtlInitUnicodeString(&ntWin32NameString, ZFS_DEV_DOS); + + // Create a symbolic link between our device name and the Win32 name + ntStatus = IoCreateSymbolicLink( + &ntWin32NameString, &ntUnicodeString); + + if (!NT_SUCCESS(ntStatus)) { + dprintf("ZFS: Couldn't create userland symbolic link to " + "/dev/zfs (%wZ)\n", ZFS_DEV); + ObDereferenceObject(ioctlDeviceObject); + IoDeleteDevice(ioctlDeviceObject); + return (-1); + } + dprintf("ZFS: created userland device symlink\n"); + + fsDiskDeviceObject->Flags |= DO_DIRECT_IO; + fsDiskDeviceObject->Flags &= ~DO_DEVICE_INITIALIZING; + IoRegisterFileSystem(fsDiskDeviceObject); + ObReferenceObject(fsDiskDeviceObject); + + NTSTATUS pcwStatus = RegisterZFSinPerf(ZFSinPerfCallBack, NULL); + if (!NT_SUCCESS(pcwStatus)) { + TraceEvent(TRACE_ERROR, "ZFSin perf registration failed\n"); + } + pcwStatus = RegisterZFSinPerfVdev(ZFSinPerfVdevCallBack, NULL); + if (!NT_SUCCESS(pcwStatus)) { + TraceEvent(TRACE_ERROR, + "ZFSin vdev perf registration failed\n"); + } + pcwStatus = RegisterZFSinCachePerf(ZFSinCachePerfCallBack, NULL); + if (!NT_SUCCESS(pcwStatus)) { + TraceEvent(TRACE_ERROR, + "ZFSin cache perf registration failed\n"); + } + + + // Set all the callbacks to "dispatch()" + WIN_DriverObject->MajorFunction[IRP_MJ_CREATE] = + (PDRIVER_DISPATCH)dispatcher; // zfs_ioctl.c + WIN_DriverObject->MajorFunction[IRP_MJ_CLOSE] = + (PDRIVER_DISPATCH)dispatcher; // zfs_ioctl.c + WIN_DriverObject->MajorFunction[IRP_MJ_READ] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_WRITE] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_QUERY_INFORMATION] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_SET_INFORMATION] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_QUERY_EA] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_SET_EA] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_FLUSH_BUFFERS] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_QUERY_VOLUME_INFORMATION] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_SET_VOLUME_INFORMATION] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_DIRECTORY_CONTROL] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_FILE_SYSTEM_CONTROL] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_DEVICE_CONTROL] = + (PDRIVER_DISPATCH)dispatcher; // zfs_ioctl.c + WIN_DriverObject->MajorFunction[IRP_MJ_INTERNAL_DEVICE_CONTROL] = + (PDRIVER_DISPATCH)dispatcher; // zfs_ioctl.c + WIN_DriverObject->MajorFunction[IRP_MJ_SHUTDOWN] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_LOCK_CONTROL] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_CLEANUP] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_SYSTEM_CONTROL] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_DEVICE_CHANGE] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_PNP] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_QUERY_SECURITY] = + (PDRIVER_DISPATCH)dispatcher; + WIN_DriverObject->MajorFunction[IRP_MJ_SET_SECURITY] = + (PDRIVER_DISPATCH)dispatcher; + + // Dump all registered filesystems + ntStatus = IoRegisterFsRegistrationChange(WIN_DriverObject, + DriverNotificationRoutine); + + if ((err = zcommon_init()) != 0) + goto zcommon_failed; + if ((err = icp_init()) != 0) + goto icp_failed; + if ((err = zstd_init()) != 0) + goto zstd_failed; + if ((err = openzfs_init_os()) != 0) + goto openzfs_os_failed; + + tsd_create(&zfsdev_private_tsd, NULL); + + DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, + "ZFS: Loaded module %s, " + "ZFS pool version %s, ZFS filesystem version %s\n", + ZFS_META_GITREV, + SPA_VERSION_STRING, ZPL_VERSION_STRING); + + return (0); + +openzfs_os_failed: + zstd_fini(); +zstd_failed: + icp_fini(); +icp_failed: + zcommon_fini(); +zcommon_failed: + return (err); +} + +void +zfsdev_detach(void) +{ + zfsdev_state_t *zs, *zsprev = NULL; + + UnregisterZFSinPerf(); + UnregisterZFSinPerfVdev(); + UnregisterZFSinCachePerf(); + + PDEVICE_OBJECT deviceObject = WIN_DriverObject->DeviceObject; + UNICODE_STRING uniWin32NameString; + + RtlInitUnicodeString(&uniWin32NameString, ZFS_DEV_DOS); + IoDeleteSymbolicLink(&uniWin32NameString); + if (deviceObject != NULL) { + ObDereferenceObject(deviceObject); + IoDeleteDevice(deviceObject); + } + + tsd_destroy(&zfsdev_private_tsd); + + openzfs_fini_os(); + zstd_fini(); + icp_fini(); + zcommon_fini(); + +} + +/* Update the VFS's cache of mountpoint properties */ +void +zfs_ioctl_update_mount_cache(const char *dsname) +{ + zfsvfs_t *zfsvfs; + + if (getzfsvfs(dsname, &zfsvfs) == 0) { + /* insert code here */ + zfs_vfs_rele(zfsvfs); + } + /* + * Ignore errors; we can't do anything useful if either getzfsvfs or + * VFS_STATFS fails. + */ +} + +uint64_t +zfs_max_nvlist_src_size_os(void) +{ + if (zfs_max_nvlist_src_size != 0) + return (zfs_max_nvlist_src_size); + + return (KMALLOC_MAX_SIZE); +} diff --git a/module/os/windows/zfs/zfs_racct.c b/module/os/windows/zfs/zfs_racct.c new file mode 100644 index 000000000000..7b5d510c303f --- /dev/null +++ b/module/os/windows/zfs/zfs_racct.c @@ -0,0 +1,32 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +#include + +void +zfs_racct_read(uint64_t size, uint64_t iops) +{ +} + +void +zfs_racct_write(uint64_t size, uint64_t iops) +{ +} diff --git a/module/os/windows/zfs/zfs_vfsops.c b/module/os/windows/zfs/zfs_vfsops.c new file mode 100644 index 000000000000..582abecb4f0b --- /dev/null +++ b/module/os/windows/zfs/zfs_vfsops.c @@ -0,0 +1,2308 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2011 Pawel Jakub Dawidek . + * Copyright (c) 2012, 2015 by Delphix. All rights reserved. + * Copyright 2016 Nexenta Systems, Inc. All rights reserved. + */ + +/* Portions Copyright 2010 Robert Milkowski */ +/* Portions Copyright 2013,2020 Jorgen Lundman */ +/* Portions Copyright 2022 Andrew Innes */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "zfs_comutil.h" + +#include +#include +#include +#include +#include +#include + +unsigned int zfs_vnop_skip_unlinked_drain = 0; + +extern int getzfsvfs(const char *dsname, zfsvfs_t **zfvp); + +void arc_os_init(void); +void arc_os_fini(void); + +/* + * AVL tree of hardlink entries, which we need to map for Finder. The va_linkid + * needs to be unique for each hardlink target, as well as, return the znode + * in vget(va_linkid). Unfortunately, the va_linkid is 32bit (lost in the + * syscall translation to userland struct). We sort the AVL tree by + * -> directory id + * -> z_id + * -> name + * + */ +static int hardlinks_compare(const void *arg1, const void *arg2) +{ + const hardlinks_t *node1 = arg1; + const hardlinks_t *node2 = arg2; + int value; + if (node1->hl_parent > node2->hl_parent) + return (1); + if (node1->hl_parent < node2->hl_parent) + return (-1); + if (node1->hl_fileid > node2->hl_fileid) + return (1); + if (node1->hl_fileid < node2->hl_fileid) + return (-1); + + value = strncmp(node1->hl_name, node2->hl_name, PATH_MAX); + if (value < 0) + return (-1); + if (value > 0) + return (1); + return (0); +} + +/* + * Lookup same information from linkid, to get at parentid, objid and name + */ +static int hardlinks_compare_linkid(const void *arg1, const void *arg2) +{ + const hardlinks_t *node1 = arg1; + const hardlinks_t *node2 = arg2; + if (node1->hl_linkid > node2->hl_linkid) + return (1); + if (node1->hl_linkid < node2->hl_linkid) + return (-1); + return (0); +} + +/* + * We need to keep a count of active fs's. + * This is necessary to prevent our kext + * from being unloaded after a umount -f + */ +uint32_t zfs_active_fs_count = 0; + +extern void zfs_ioctl_init(void); +extern void zfs_ioctl_fini(void); + +boolean_t +zfs_is_readonly(zfsvfs_t *zfsvfs) +{ + return (!!(vfs_isrdonly(zfsvfs->z_vfs))); +} + +/* + * The OS sync ignored by default, as ZFS handles internal periodic + * syncs. (As per illumos) Unfortunately, we can not tell the difference + * of when users run "sync" by hand. Sync is called on umount though. + */ +uint64_t zfs_vfs_sync_paranoia = 0; + +int +zfs_vfs_sync(struct mount *vfsp, __unused int waitfor, + __unused vfs_context_t context) +{ + int error = 0; + /* + * Data integrity is job one. We don't want a compromised kernel + * writing to the storage pool, so we never sync during panic. + */ + if (spl_panicstr()) + return (0); + + /* Check if sysctl setting wants sync - and we are not unmounting */ + if (zfs_vfs_sync_paranoia == 0 && + !vfs_isunmount(vfsp)) + return (0); + + if (vfsp != NULL) { + /* + * Sync a specific filesystem. + */ + zfsvfs_t *zfsvfs = vfs_fsprivate(vfsp); + dsl_pool_t *dp; + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + dp = dmu_objset_pool(zfsvfs->z_os); + + /* + * If the system is shutting down, then skip any + * filesystems which may exist on a suspended pool. + */ + if (spl_system_inshutdown() && spa_suspended(dp->dp_spa)) { + zfs_exit(zfsvfs, FTAG); + return (0); + } + + if (zfsvfs->z_log != NULL) + zil_commit(zfsvfs->z_log, 0); + + zfs_exit(zfsvfs, FTAG); + + } else { + /* + * Sync all ZFS filesystems. This is what happens when you + * run sync(1M). Unlike other filesystems, ZFS honors the + * request by waiting for all pools to commit all dirty data. + */ + spa_sync_allpools(); + } + + return (0); +} + +static void +atime_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + + if (newval == B_TRUE) { + zfsvfs->z_atime = B_TRUE; + vfs_clearflags(zfsvfs->z_vfs, (uint64_t)MNT_NOATIME); + } else { + zfsvfs->z_atime = B_FALSE; + vfs_setflags(zfsvfs->z_vfs, (uint64_t)MNT_NOATIME); + } +} + +static void +xattr_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + + /* + * Apple does have MNT_NOUSERXATTR mount option, but unfortunately + * the VFS layer returns EACCESS if xattr access is attempted. + * Finder etc, will do so, even if filesystem capabilities is set + * without xattr, rendering the mount option useless. We no longer + * set it, and handle xattrs being disabled internally. + */ + + if (newval == ZFS_XATTR_OFF) { + zfsvfs->z_xattr = B_FALSE; + // vfs_setflags(zfsvfs->z_vfs, (uint64_t)MNT_NOUSERXATTR); + } else { + zfsvfs->z_xattr = B_TRUE; + // vfs_clearflags(zfsvfs->z_vfs, (uint64_t)MNT_NOUSERXATTR); + + if (newval == ZFS_XATTR_SA) + zfsvfs->z_xattr_sa = B_TRUE; + else + zfsvfs->z_xattr_sa = B_FALSE; + } +} + +static void +blksz_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + ASSERT3U(newval, <=, spa_maxblocksize(dmu_objset_spa(zfsvfs->z_os))); + ASSERT3U(newval, >=, SPA_MINBLOCKSIZE); + ASSERT(ISP2(newval)); + + zfsvfs->z_max_blksz = newval; + // zfsvfs->z_vfs->mnt_stat.f_iosize = newval; +} + +static void +readonly_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + if (newval == B_TRUE) { + vfs_setflags(zfsvfs->z_vfs, (uint64_t)MNT_RDONLY); + } else { + vfs_clearflags(zfsvfs->z_vfs, (uint64_t)MNT_RDONLY); + } +} + +static void +devices_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + if (newval == B_FALSE) { + vfs_setflags(zfsvfs->z_vfs, (uint64_t)MNT_NODEV); + } else { + vfs_clearflags(zfsvfs->z_vfs, (uint64_t)MNT_NODEV); + } +} + +static void +setuid_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + if (newval == B_FALSE) { + vfs_setflags(zfsvfs->z_vfs, (uint64_t)MNT_NOSUID); + } else { + vfs_clearflags(zfsvfs->z_vfs, (uint64_t)MNT_NOSUID); + } +} + +static void +exec_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + if (newval == B_FALSE) { + vfs_setflags(zfsvfs->z_vfs, (uint64_t)MNT_NOEXEC); + } else { + vfs_clearflags(zfsvfs->z_vfs, (uint64_t)MNT_NOEXEC); + } +} + +static void +snapdir_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + zfsvfs->z_show_ctldir = newval; + cache_purgevfs(zfsvfs->z_vfs); +} + +static void +vscan_changed_cb(void *arg, uint64_t newval) +{ + // zfsvfs_t *zfsvfs = arg; + // zfsvfs->z_vscan = newval; +} + +static void +acl_mode_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + zfsvfs->z_acl_mode = newval; +} + +static void +acl_inherit_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + + zfsvfs->z_acl_inherit = newval; +} + +static void +finderbrowse_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + if (newval == B_FALSE) { + vfs_setflags(zfsvfs->z_vfs, (uint64_t)MNT_DONTBROWSE); + } else { + vfs_clearflags(zfsvfs->z_vfs, (uint64_t)MNT_DONTBROWSE); + } +} +static void +ignoreowner_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + if (newval == B_FALSE) { + vfs_clearflags(zfsvfs->z_vfs, (uint64_t)MNT_IGNORE_OWNERSHIP); + } else { + vfs_setflags(zfsvfs->z_vfs, (uint64_t)MNT_IGNORE_OWNERSHIP); + } +} + +static void +mimic_changed_cb(void *arg, uint64_t newval) +{ + zfsvfs_t *zfsvfs = arg; + struct vfsstatfs *vfsstatfs; + vfsstatfs = vfs_statfs(zfsvfs->z_vfs); + + if (newval == 0) { + // strlcpy(vfsstatfs->f_fstypename, "zfs", MFSTYPENAMELEN); + } else { + // strlcpy(vfsstatfs->f_fstypename, "hfs", MFSTYPENAMELEN); + } +} + +static int +zfs_register_callbacks(struct mount *vfsp) +{ + struct dsl_dataset *ds = NULL; + + objset_t *os = NULL; + zfsvfs_t *zfsvfs = NULL; + boolean_t readonly = B_FALSE; + boolean_t do_readonly = B_FALSE; + boolean_t setuid = B_FALSE; + boolean_t do_setuid = B_FALSE; + boolean_t exec = B_FALSE; + boolean_t do_exec = B_FALSE; + boolean_t devices = B_FALSE; + boolean_t do_devices = B_FALSE; + boolean_t xattr = B_FALSE; + boolean_t do_xattr = B_FALSE; + boolean_t atime = B_FALSE; + boolean_t do_atime = B_FALSE; + boolean_t finderbrowse = B_FALSE; + boolean_t do_finderbrowse = B_FALSE; + boolean_t ignoreowner = B_FALSE; + boolean_t do_ignoreowner = B_FALSE; + int error = 0; + + ASSERT(vfsp); + zfsvfs = vfs_fsprivate(vfsp); + ASSERT(zfsvfs); + os = zfsvfs->z_os; + + /* + * This function can be called for a snapshot when we update snapshot's + * mount point, which isn't really supported. + */ + if (dmu_objset_is_snapshot(os)) + return (EOPNOTSUPP); + + /* + * The act of registering our callbacks will destroy any mount + * options we may have. In order to enable temporary overrides + * of mount options, we stash away the current values and + * restore them after we register the callbacks. + */ +#define vfs_optionisset(X, Y, Z) (vfs_flags(X)&(Y)) + + if (vfs_optionisset(vfsp, MNT_RDONLY, NULL) || + !spa_writeable(dmu_objset_spa(os))) { + readonly = B_TRUE; + do_readonly = B_TRUE; + } + if (vfs_optionisset(vfsp, MNT_NODEV, NULL)) { + devices = B_FALSE; + do_devices = B_TRUE; + } + /* xnu SETUID, not IllumOS SUID */ + if (vfs_optionisset(vfsp, MNT_NOSUID, NULL)) { + setuid = B_FALSE; + do_setuid = B_TRUE; + } + if (vfs_optionisset(vfsp, MNT_NOEXEC, NULL)) { + exec = B_FALSE; + do_exec = B_TRUE; + } + if (vfs_optionisset(vfsp, MNT_NOUSERXATTR, NULL)) { + xattr = B_FALSE; + do_xattr = B_TRUE; + } + if (vfs_optionisset(vfsp, MNT_NOATIME, NULL)) { + atime = B_FALSE; + do_atime = B_TRUE; + } + if (vfs_optionisset(vfsp, MNT_DONTBROWSE, NULL)) { + finderbrowse = B_FALSE; + do_finderbrowse = B_TRUE; + } + if (vfs_optionisset(vfsp, MNT_IGNORE_OWNERSHIP, NULL)) { + ignoreowner = B_TRUE; + do_ignoreowner = B_TRUE; + } + + /* + * nbmand is a special property. It can only be changed at + * mount time. + * + * This is weird, but it is documented to only be changeable + * at mount time. + */ + + /* + * Register property callbacks. + * + * It would probably be fine to just check for i/o error from + * the first prop_register(), but I guess I like to go + * overboard... + */ + ds = dmu_objset_ds(os); + dsl_pool_config_enter(dmu_objset_pool(os), FTAG); + error = dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_ATIME), atime_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_XATTR), xattr_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_RECORDSIZE), blksz_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_READONLY), readonly_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_DEVICES), devices_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_SETUID), setuid_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_EXEC), exec_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_SNAPDIR), snapdir_changed_cb, zfsvfs); + // This appears to be PROP_PRIVATE, investigate if we want this + // ZOL calls this ACLTYPE + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_ACLMODE), acl_mode_changed_cb, zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_ACLINHERIT), acl_inherit_changed_cb, + zfsvfs); + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_VSCAN), vscan_changed_cb, zfsvfs); + + error = error ? error : dsl_prop_register(ds, + zfs_prop_to_name(ZFS_PROP_MIMIC), mimic_changed_cb, zfsvfs); + + dsl_pool_config_exit(dmu_objset_pool(os), FTAG); + if (error) + goto unregister; + + /* + * Invoke our callbacks to restore temporary mount options. + */ + if (do_readonly) + readonly_changed_cb(zfsvfs, readonly); + if (do_setuid) + setuid_changed_cb(zfsvfs, setuid); + if (do_exec) + exec_changed_cb(zfsvfs, exec); + if (do_devices) + devices_changed_cb(zfsvfs, devices); + if (do_xattr) + xattr_changed_cb(zfsvfs, xattr); + if (do_atime) + atime_changed_cb(zfsvfs, atime); + + if (do_finderbrowse) + finderbrowse_changed_cb(zfsvfs, finderbrowse); + if (do_ignoreowner) + ignoreowner_changed_cb(zfsvfs, ignoreowner); + + return (0); + +unregister: + dsl_prop_unregister_all(ds, zfsvfs); + return (error); +} + +/* + * Takes a dataset, a property, a value and that value's setpoint as + * found in the ZAP. Checks if the property has been changed in the vfs. + * If so, val and setpoint will be overwritten with updated content. + * Otherwise, they are left unchanged. + */ +int +zfs_get_temporary_prop(dsl_dataset_t *ds, zfs_prop_t zfs_prop, uint64_t *val, + char *setpoint) +{ + int error; + zfsvfs_t *zfvp; + mount_t *vfsp; + objset_t *os; + uint64_t tmp = *val; + + error = dmu_objset_from_ds(ds, &os); + if (error != 0) + return (error); + + if (dmu_objset_type(os) != DMU_OST_ZFS) + return (EINVAL); + + mutex_enter(&os->os_user_ptr_lock); + zfvp = dmu_objset_get_user(os); + mutex_exit(&os->os_user_ptr_lock); + if (zfvp == NULL) + return (ESRCH); + + vfsp = zfvp->z_vfs; + + switch (zfs_prop) { + case ZFS_PROP_ATIME: +// if (vfsp->vfs_do_atime) +// tmp = vfsp->vfs_atime; + break; + case ZFS_PROP_RELATIME: +// if (vfsp->vfs_do_relatime) +// tmp = vfsp->vfs_relatime; + break; + case ZFS_PROP_DEVICES: +// if (vfsp->vfs_do_devices) +// tmp = vfsp->vfs_devices; + break; + case ZFS_PROP_EXEC: +// if (vfsp->vfs_do_exec) +// tmp = vfsp->vfs_exec; + break; + case ZFS_PROP_SETUID: +// if (vfsp->vfs_do_setuid) +// tmp = vfsp->vfs_setuid; + break; + case ZFS_PROP_READONLY: +// if (vfsp->vfs_do_readonly) +// tmp = vfsp->vfs_readonly; + break; + case ZFS_PROP_XATTR: +// if (vfsp->vfs_do_xattr) +// tmp = vfsp->vfs_xattr; + break; + case ZFS_PROP_NBMAND: +// if (vfsp->vfs_do_nbmand) +// tmp = vfsp->vfs_nbmand; + break; + default: + return (ENOENT); + } + + if (tmp != *val) { + (void) strlcpy(setpoint, "temporary", ZFS_MAX_DATASET_NAME_LEN); + *val = tmp; + } + return (0); +} + + +/* + * Associate this zfsvfs with the given objset, which must be owned. + * This will cache a bunch of on-disk state from the objset in the + * zfsvfs. + */ +static int +zfsvfs_init(zfsvfs_t *zfsvfs, objset_t *os) +{ + int error; + uint64_t val; + + zfsvfs->z_max_blksz = SPA_OLD_MAXBLOCKSIZE; + zfsvfs->z_show_ctldir = ZFS_SNAPDIR_VISIBLE; + zfsvfs->z_os = os; + + /* Volume status "all ok" */ + zfsvfs->z_notification_conditions = 0; + zfsvfs->z_freespace_notify_warninglimit = 0; + zfsvfs->z_freespace_notify_dangerlimit = 0; + zfsvfs->z_freespace_notify_desiredlevel = 0; + + error = zfs_get_zplprop(os, ZFS_PROP_VERSION, &zfsvfs->z_version); + if (error != 0) + return (error); + if (zfsvfs->z_version > + zfs_zpl_version_map(spa_version(dmu_objset_spa(os)))) { + dprintf("Can't mount a version %lld file system " + "on a version %lld pool\n. Pool must be upgraded to mount " + "this file system.\n", (u_longlong_t)zfsvfs->z_version, + (u_longlong_t)spa_version(dmu_objset_spa(os))); + return (SET_ERROR(ENOTSUP)); + } + error = zfs_get_zplprop(os, ZFS_PROP_NORMALIZE, &val); + if (error != 0) + return (error); + zfsvfs->z_norm = (int)val; + + error = zfs_get_zplprop(os, ZFS_PROP_UTF8ONLY, &val); + if (error != 0) + return (error); + zfsvfs->z_utf8 = (val != 0); + + error = zfs_get_zplprop(os, ZFS_PROP_CASE, &val); + if (error != 0) + return (error); + zfsvfs->z_case = (uint_t)val; + + error = zfs_get_zplprop(os, ZFS_PROP_ACLMODE, &val); + if (error != 0) + return (error); + zfsvfs->z_acl_mode = (uint_t)val; + + // zfs_get_zplprop(os, ZFS_PROP_LASTUNMOUNT, &val); + // zfsvfs->z_last_unmount_time = val; + + /* + * Fold case on file systems that are always or sometimes case + * insensitive. + */ + if (zfsvfs->z_case == ZFS_CASE_INSENSITIVE || + zfsvfs->z_case == ZFS_CASE_MIXED) + zfsvfs->z_norm |= U8_TEXTPREP_TOUPPER; + + zfsvfs->z_use_fuids = USE_FUIDS(zfsvfs->z_version, zfsvfs->z_os); + zfsvfs->z_use_sa = USE_SA(zfsvfs->z_version, zfsvfs->z_os); + + uint64_t sa_obj = 0; + if (zfsvfs->z_use_sa) { + /* should either have both of these objects or none */ + error = zap_lookup(os, MASTER_NODE_OBJ, ZFS_SA_ATTRS, 8, 1, + &sa_obj); + + if (error != 0) + return (error); + + error = zfs_get_zplprop(os, ZFS_PROP_XATTR, &val); + if ((error == 0) && (val == ZFS_XATTR_SA)) + zfsvfs->z_xattr_sa = B_TRUE; + } + + error = sa_setup(os, sa_obj, zfs_attr_table, ZPL_END, + &zfsvfs->z_attr_table); + if (error != 0) + return (error); + + if (zfsvfs->z_version >= ZPL_VERSION_SA) + sa_register_update_callback(os, zfs_sa_upgrade); + + error = zap_lookup(os, MASTER_NODE_OBJ, ZFS_ROOT_OBJ, 8, 1, + &zfsvfs->z_root); + if (error != 0) + return (error); + ASSERT(zfsvfs->z_root != 0); + + error = zap_lookup(os, MASTER_NODE_OBJ, ZFS_UNLINKED_SET, 8, 1, + &zfsvfs->z_unlinkedobj); + if (error != 0) + return (error); + + error = zap_lookup(os, MASTER_NODE_OBJ, + zfs_userquota_prop_prefixes[ZFS_PROP_USERQUOTA], + 8, 1, &zfsvfs->z_userquota_obj); + if (error == ENOENT) + zfsvfs->z_userquota_obj = 0; + else if (error != 0) + return (error); + + error = zap_lookup(os, MASTER_NODE_OBJ, + zfs_userquota_prop_prefixes[ZFS_PROP_GROUPQUOTA], + 8, 1, &zfsvfs->z_groupquota_obj); + if (error == ENOENT) + zfsvfs->z_groupquota_obj = 0; + else if (error != 0) + return (error); + + error = zap_lookup(os, MASTER_NODE_OBJ, ZFS_FUID_TABLES, 8, 1, + &zfsvfs->z_fuid_obj); + if (error == ENOENT) + zfsvfs->z_fuid_obj = 0; + else if (error != 0) + return (error); + + error = zap_lookup(os, MASTER_NODE_OBJ, ZFS_SHARES_DIR, 8, 1, + &zfsvfs->z_shares_dir); + if (error == ENOENT) + zfsvfs->z_shares_dir = 0; + else if (error != 0) + return (error); + + return (0); +} + +int +zfsvfs_create(const char *osname, boolean_t readonly, zfsvfs_t **zfvp) +{ + objset_t *os; + zfsvfs_t *zfsvfs; + int error; + + zfsvfs = kmem_zalloc(sizeof (zfsvfs_t), KM_SLEEP); + + /* + * We claim to always be readonly so we can open snapshots; + * other ZPL code will prevent us from writing to snapshots. + */ + error = dmu_objset_own(osname, DMU_OST_ZFS, B_TRUE, B_TRUE, + zfsvfs, &os); + if (error != 0) { + kmem_free(zfsvfs, sizeof (zfsvfs_t)); + return (error); + } + + error = zfsvfs_create_impl(zfvp, zfsvfs, os); + + return (error); +} + +int +zfsvfs_create_impl(zfsvfs_t **zfvp, zfsvfs_t *zfsvfs, objset_t *os) +{ + int error; + + zfsvfs->z_vfs = NULL; + zfsvfs->z_parent = zfsvfs; + + mutex_init(&zfsvfs->z_znodes_lock, NULL, MUTEX_DEFAULT, NULL); + mutex_init(&zfsvfs->z_lock, NULL, MUTEX_DEFAULT, NULL); + list_create(&zfsvfs->z_all_znodes, sizeof (znode_t), + offsetof(znode_t, z_link_node)); + + zfsvfs->z_ctldir_startid = ZFSCTL_INO_SNAPDIRS; + + ZFS_TEARDOWN_INIT(zfsvfs); + rw_init(&zfsvfs->z_teardown_inactive_lock, NULL, RW_DEFAULT, NULL); + rw_init(&zfsvfs->z_fuid_lock, NULL, RW_DEFAULT, NULL); + + int size = MIN(1 << (highbit64(zfs_object_mutex_size) - 1), + ZFS_OBJ_MTX_MAX); + zfsvfs->z_hold_size = size; + zfsvfs->z_hold_trees = vmem_zalloc(sizeof (avl_tree_t) * size, + KM_SLEEP); + zfsvfs->z_hold_locks = vmem_zalloc(sizeof (kmutex_t) * size, KM_SLEEP); + for (int i = 0; i != size; i++) { + avl_create(&zfsvfs->z_hold_trees[i], zfs_znode_hold_compare, + sizeof (znode_hold_t), offsetof(znode_hold_t, zh_node)); + mutex_init(&zfsvfs->z_hold_locks[i], NULL, MUTEX_DEFAULT, NULL); + } + + rw_init(&zfsvfs->z_hardlinks_lock, NULL, RW_DEFAULT, NULL); + avl_create(&zfsvfs->z_hardlinks, hardlinks_compare, + sizeof (hardlinks_t), offsetof(hardlinks_t, hl_node)); + avl_create(&zfsvfs->z_hardlinks_linkid, hardlinks_compare_linkid, + sizeof (hardlinks_t), offsetof(hardlinks_t, hl_node_linkid)); + zfsvfs->z_rdonly = 0; + + mutex_init(&zfsvfs->z_drain_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&zfsvfs->z_drain_cv, NULL, CV_DEFAULT, NULL); + + error = zfsvfs_init(zfsvfs, os); + if (error != 0) { + dmu_objset_disown(os, B_TRUE, zfsvfs); + *zfvp = NULL; + kmem_free(zfsvfs, sizeof (zfsvfs_t)); + return (error); + } + + *zfvp = zfsvfs; + return (0); +} + +static int +zfsvfs_setup(zfsvfs_t *zfsvfs, boolean_t mounting) +{ + int error; + boolean_t readonly = vfs_isrdonly(zfsvfs->z_vfs); + + error = zfs_register_callbacks(zfsvfs->z_vfs); + if (error) + return (error); + + zfsvfs->z_log = zil_open(zfsvfs->z_os, zfs_get_data, NULL); + + /* + * If we are not mounting (ie: online recv), then we don't + * have to worry about replaying the log as we blocked all + * operations out since we closed the ZIL. + */ + if (mounting) { + + /* + * During replay we remove the read only flag to + * allow replays to succeed. + */ + + if (readonly != 0) + readonly_changed_cb(zfsvfs, B_FALSE); + else + if (!zfs_vnop_skip_unlinked_drain) + zfs_unlinked_drain(zfsvfs); + + /* + * Parse and replay the intent log. + * + * Because of ziltest, this must be done after + * zfs_unlinked_drain(). (Further note: ziltest + * doesn't use readonly mounts, where + * zfs_unlinked_drain() isn't called.) This is because + * ziltest causes spa_sync() to think it's committed, + * but actually it is not, so the intent log contains + * many txg's worth of changes. + * + * In particular, if object N is in the unlinked set in + * the last txg to actually sync, then it could be + * actually freed in a later txg and then reallocated + * in a yet later txg. This would write a "create + * object N" record to the intent log. Normally, this + * would be fine because the spa_sync() would have + * written out the fact that object N is free, before + * we could write the "create object N" intent log + * record. + * + * But when we are in ziltest mode, we advance the "open + * txg" without actually spa_sync()-ing the changes to + * disk. So we would see that object N is still + * allocated and in the unlinked set, and there is an + * intent log record saying to allocate it. + */ + if (spa_writeable(dmu_objset_spa(zfsvfs->z_os))) { + if (zil_replay_disable) { + zil_destroy(zfsvfs->z_log, B_FALSE); + } else { + zfsvfs->z_replay = B_TRUE; + zil_replay(zfsvfs->z_os, zfsvfs, + zfs_replay_vector); + zfsvfs->z_replay = B_FALSE; + } + } + + /* restore readonly bit */ + if (readonly != 0) + readonly_changed_cb(zfsvfs, B_TRUE); + } + + /* + * Set the objset user_ptr to track its zfsvfs. + */ + mutex_enter(&zfsvfs->z_os->os_user_ptr_lock); + dmu_objset_set_user(zfsvfs->z_os, zfsvfs); + mutex_exit(&zfsvfs->z_os->os_user_ptr_lock); + + return (0); +} + +extern krwlock_t zfsvfs_lock; /* in zfs_znode.c */ + +void +zfsvfs_free(zfsvfs_t *zfsvfs) +{ + int i, size = zfsvfs->z_hold_size; + + dprintf("+zfsvfs_free\n"); + + zfs_fuid_destroy(zfsvfs); + + cv_destroy(&zfsvfs->z_drain_cv); + mutex_destroy(&zfsvfs->z_drain_lock); + mutex_destroy(&zfsvfs->z_znodes_lock); + mutex_destroy(&zfsvfs->z_lock); + list_destroy(&zfsvfs->z_all_znodes); + ZFS_TEARDOWN_DESTROY(zfsvfs); + rw_destroy(&zfsvfs->z_teardown_inactive_lock); + rw_destroy(&zfsvfs->z_fuid_lock); + + for (i = 0; i != size; i++) { + avl_destroy(&zfsvfs->z_hold_trees[i]); + mutex_destroy(&zfsvfs->z_hold_locks[i]); + } + kmem_free(zfsvfs->z_hold_trees, sizeof (avl_tree_t) * size); + kmem_free(zfsvfs->z_hold_locks, sizeof (kmutex_t) * size); + + dprintf("ZFS: Unloading hardlink AVLtree: %lu\n", + avl_numnodes(&zfsvfs->z_hardlinks)); + void *cookie = NULL; + hardlinks_t *hardlink; + rw_destroy(&zfsvfs->z_hardlinks_lock); + while ((hardlink = avl_destroy_nodes(&zfsvfs->z_hardlinks_linkid, + &cookie))) { + } + cookie = NULL; + while ((hardlink = avl_destroy_nodes(&zfsvfs->z_hardlinks, &cookie))) { + kmem_free(hardlink, sizeof (*hardlink)); + } + avl_destroy(&zfsvfs->z_hardlinks); + avl_destroy(&zfsvfs->z_hardlinks_linkid); + + kmem_free(zfsvfs, sizeof (zfsvfs_t)); + dprintf("-zfsvfs_free\n"); +} + +static void +zfs_set_fuid_feature(zfsvfs_t *zfsvfs) +{ + zfsvfs->z_use_fuids = USE_FUIDS(zfsvfs->z_version, zfsvfs->z_os); + if (zfsvfs->z_vfs) { +#if 0 + if (zfsvfs->z_use_fuids) { + vfs_set_feature(zfsvfs->z_vfs, VFSFT_XVATTR); + vfs_set_feature(zfsvfs->z_vfs, VFSFT_SYSATTR_VIEWS); + vfs_set_feature(zfsvfs->z_vfs, VFSFT_ACEMASKONACCESS); + vfs_set_feature(zfsvfs->z_vfs, VFSFT_ACLONCREATE); + vfs_set_feature(zfsvfs->z_vfs, VFSFT_ACCESS_FILTER); + vfs_set_feature(zfsvfs->z_vfs, VFSFT_REPARSE); + } else { + vfs_clear_feature(zfsvfs->z_vfs, VFSFT_XVATTR); + vfs_clear_feature(zfsvfs->z_vfs, VFSFT_SYSATTR_VIEWS); + vfs_clear_feature(zfsvfs->z_vfs, VFSFT_ACEMASKONACCESS); + vfs_clear_feature(zfsvfs->z_vfs, VFSFT_ACLONCREATE); + vfs_clear_feature(zfsvfs->z_vfs, VFSFT_ACCESS_FILTER); + vfs_clear_feature(zfsvfs->z_vfs, VFSFT_REPARSE); + } +#endif + } + zfsvfs->z_use_sa = USE_SA(zfsvfs->z_version, zfsvfs->z_os); +} + + +static int +zfs_domount(struct mount *vfsp, dev_t mount_dev, char *osname, + vfs_context_t ctx) +{ + int error = 0; + zfsvfs_t *zfsvfs; + uint64_t mimic = 0; + // struct timeval tv; + + ASSERT(vfsp); + ASSERT(osname); + + error = zfsvfs_create(osname, B_FALSE, &zfsvfs); + if (error) + return (error); + zfsvfs->z_vfs = vfsp; + + zfsvfs->z_rdev = mount_dev; + + /* HFS sets this prior to mounting */ + vfs_setflags(vfsp, (uint64_t)((unsigned int)MNT_DOVOLFS)); + /* Advisory locking should be handled at the VFS layer */ + vfs_setlocklocal(vfsp); + + /* + * Record the mount time (for Spotlight) + */ + + vfs_setfsprivate(vfsp, zfsvfs); + + /* + * The fsid is 64 bits, composed of an 8-bit fs type, which + * separates our fsid from any other filesystem types, and a + * 56-bit objset unique ID. The objset unique ID is unique to + * all objsets open on this system, provided by unique_create(). + * The 8-bit fs type must be put in the low bits of fsid[1] + * because that's where other Solaris filesystems put it. + */ + + error = dsl_prop_get_integer(osname, "com.apple.mimic", &mimic, NULL); + + /* + * If we are readonly (ie, waiting for rootmount) we need to reply + * honestly, so launchd runs fsck_zfs and mount_zfs + */ + if (mimic /* == ZFS_MIMIC_NTFS */) { + struct vfsstatfs *vfsstatfs; + vfsstatfs = vfs_statfs(vfsp); + strlcpy(vfsstatfs->f_fstypename, "ntfs", MFSTYPENAMELEN); + } + + /* + * Set features for file system. + */ + zfs_set_fuid_feature(zfsvfs); + + if (dmu_objset_is_snapshot(zfsvfs->z_os)) { + uint64_t pval; + char fsname[ZFS_MAX_DATASET_NAME_LEN]; + zfsvfs_t *fs_zfsvfs; + + dmu_fsname(osname, fsname); + error = getzfsvfs(fsname, &fs_zfsvfs); + if (error == 0) { + if (fs_zfsvfs->z_unmounted) + error = SET_ERROR(EINVAL); + vfs_unbusy(fs_zfsvfs->z_vfs); + } + if (error) { + dprintf("file system '%s' is unmounted : error %d\n", + fsname, + error); + goto out; + } + + atime_changed_cb(zfsvfs, B_FALSE); + readonly_changed_cb(zfsvfs, B_TRUE); + if ((error = dsl_prop_get_integer(osname, "xattr", &pval, + NULL))) + goto out; + xattr_changed_cb(zfsvfs, pval); + zfsvfs->z_rdonly = B_TRUE; + zfsvfs->z_issnap = B_TRUE; + zfsvfs->z_os->os_sync = ZFS_SYNC_DISABLED; + + mutex_enter(&zfsvfs->z_os->os_user_ptr_lock); + dmu_objset_set_user(zfsvfs->z_os, zfsvfs); + mutex_exit(&zfsvfs->z_os->os_user_ptr_lock); + + zfsctl_mount_signal(zfsvfs, osname, B_TRUE); + + } else { + if ((error = zfsvfs_setup(zfsvfs, B_TRUE))) + goto out; + } + + vfs_setflags(vfsp, (uint64_t)((unsigned int)MNT_JOURNALED)); + + if ((vfs_flags(vfsp) & MNT_ROOTFS) != 0) { + /* Root FS */ + vfs_clearflags(vfsp, + (uint64_t)((unsigned int)MNT_UNKNOWNPERMISSIONS)); + vfs_clearflags(vfsp, + (uint64_t)((unsigned int)MNT_IGNORE_OWNERSHIP)); + } + +#if 1 // Want .zfs or not + if (!zfsvfs->z_issnap) { + zfsctl_create(zfsvfs); + } +#endif + +out: + if (error) { + vfs_setfsprivate(vfsp, NULL); + dmu_objset_disown(zfsvfs->z_os, B_TRUE, zfsvfs); + zfsvfs_free(zfsvfs); + } else { + atomic_inc_32(&zfs_active_fs_count); + } + + return (error); +} + +void +zfs_unregister_callbacks(zfsvfs_t *zfsvfs) +{ + objset_t *os = zfsvfs->z_os; + + /* + * Unregister properties. + */ + if (!dmu_objset_is_snapshot(os)) { + dsl_prop_unregister_all(dmu_objset_ds(os), zfsvfs); + } +} + +/* + * zfs_vfs_mountroot + * Given a device vnode created by vfs_mountroot bdevvp, + * and with the root pool already imported, root mount the + * dataset specified in the pool's bootfs property. + * + * Inputs: + * mp: VFS mount struct + * devvp: device vnode, currently only used to retrieve the + * dev_t for the fsid. Could vnode_get, vnode_ref, vnode_put, + * with matching get/rele/put in zfs_vfs_umount, but this is + * already done by XNU as well. + * ctx: VFS context, unused. + * + * Return: + * 0 on success, positive int on failure. + */ +int +zfs_vfs_mountroot(struct mount *mp, struct vnode *devvp, vfs_context_t ctx) +{ + /* + * static int zfsrootdone = 0; + */ + zfsvfs_t *zfsvfs = NULL; + spa_t *spa = 0; + char *zfs_bootfs = 0; + dev_t dev = 0; + int error = EINVAL; + + dprintf("ZFS: %s\n", __func__); + ASSERT(mp); + ASSERT(devvp); + ASSERT(ctx); + if (!mp || !devvp | !ctx) { + cmn_err(CE_NOTE, "%s: missing one of mp %p devvp %p" + " or ctx %p", __func__, mp, devvp, ctx); + return (EINVAL); + } + + /* Look up bootfs variable from pool here */ + zfs_bootfs = kmem_alloc(MAXPATHLEN, KM_SLEEP); + if (!zfs_bootfs) { + cmn_err(CE_NOTE, "%s: bootfs alloc failed", + __func__); + return (ENOMEM); + } + + mutex_enter(&spa_namespace_lock); + spa = spa_next(NULL); + if (!spa) { + mutex_exit(&spa_namespace_lock); + cmn_err(CE_NOTE, "%s: no pool available", + __func__); + goto out; + } + + error = dsl_dsobj_to_dsname(spa_name(spa), + spa_bootfs(spa), zfs_bootfs); + if (error != 0) { + mutex_exit(&spa_namespace_lock); + cmn_err(CE_NOTE, "%s: bootfs to name error %d", + __func__, error); + goto out; + } + mutex_exit(&spa_namespace_lock); + + /* + * By setting the dev_t value in the mount vfsp, + * mount_zfs will be called with the /dev/diskN + * proxy, but we can leave the dataset name in + * the mountedfrom field + */ + dev = vnode_specrdev(devvp); + + dprintf("Setting readonly\n"); + + if ((error = zfs_domount(mp, dev, zfs_bootfs, ctx)) != 0) { + dprintf("zfs_domount: error %d", error); + goto out; + } + + zfsvfs = (zfsvfs_t *)vfs_fsprivate(mp); + ASSERT(zfsvfs); + if (!zfsvfs) { + cmn_err(CE_NOTE, "missing zfsvfs"); + goto out; + } + + /* Set this mount to read-only */ + zfsvfs->z_rdonly = 1; + + /* + * Due to XNU mount flags, readonly gets set off for a short + * while, which means mimic will kick in if enabled. But we need + * to reply with true "zfs" until root has been remounted RW, so + * that launchd tries to run mount_zfs instead of mount_hfs + */ + mimic_changed_cb(zfsvfs, B_FALSE); + + /* + * Leave rootvp held. The root file system is never unmounted. + * + * XXX APPLE + * xnu will in fact call vfs_unmount on the root filesystem + * during shutdown/reboot. + */ + +out: + + if (zfs_bootfs) { + kmem_free(zfs_bootfs, MAXPATHLEN); + } + return (error); + +} + +int +zfs_vfs_mount(struct mount *vfsp, vnode_t *mvp /* devvp */, + user_addr_t data, vfs_context_t context) +{ + int error = 0; + cred_t *cr = NULL; // (cred_t *)vfs_context_ucred(context); + char *osname = NULL; + char *options = NULL; + uint64_t flags = vfs_flags(vfsp); + int canwrite; + int rdonly = 0; + int mflag = 0; + + struct zfs_mount_args *mnt_args = (struct zfs_mount_args *)data; + size_t osnamelen = 0; + uint32_t cmdflags = 0; + + dprintf("%s\n", __func__); + cmdflags = (uint32_t)vfs_flags(vfsp) & MNT_CMDFLAGS; + rdonly = vfs_isrdonly(vfsp); + dprintf("%s cmdflags %u rdonly %d\n", __func__, cmdflags, rdonly); + + /* + * Get the objset name (the "special" mount argument). + */ + if (data) { + + // Allocate string area + osname = kmem_alloc(MAXPATHLEN, KM_SLEEP); + + strlcpy(osname, mnt_args->fspec, MAXPATHLEN); + + } + + if (mnt_args->struct_size == sizeof (*mnt_args)) { + + mflag = mnt_args->mflag; + + if (mnt_args->optlen) { + options = kmem_alloc(mnt_args->optlen, KM_SLEEP); + strlcpy(options, mnt_args->optptr, mnt_args->optlen); + } + + dprintf("%s: fspec '%s' : mflag %04x : optptr %p : optlen %d :" + " options %s\n", __func__, + osname, + mnt_args->mflag, + mnt_args->optptr, + mnt_args->optlen, + options); + } + + if (mflag & MS_RDONLY) { + dprintf("%s: adding MNT_RDONLY\n", __func__); + cmdflags |= MNT_RDONLY; + } + + if (mflag & MS_OVERLAY) { + dprintf("%s: adding MNT_UNION\n", __func__); + cmdflags |= MNT_UNION; + } + + if (mflag & MS_FORCE) { + dprintf("%s: adding MNT_FORCE\n", __func__); + cmdflags |= MNT_FORCE; + } + + if (mflag & MS_REMOUNT) { + dprintf("%s: adding MNT_UPDATE on MS_REMOUNT\n", __func__); + cmdflags |= MNT_UPDATE; + } + + vfs_setflags(vfsp, (uint64_t)cmdflags); + + /* + * When doing a remount, we simply refresh our temporary properties + * according to those options set in the current VFS options. + */ + if (cmdflags & MNT_UPDATE) { + + error = 0; + // Used after fsck + if (cmdflags & MNT_RELOAD) { + goto out; + } + + /* refresh mount options */ + zfsvfs_t *zfsvfs = vfs_fsprivate(vfsp); + + if (zfsvfs != NULL) { + if (zfsvfs->z_rdonly == 0 && + (cmdflags & MNT_RDONLY || + vfs_isrdonly(vfsp))) { + /* downgrade */ + dprintf("%s: downgrade requested\n", __func__); + zfsvfs->z_rdonly = 1; + readonly_changed_cb(zfsvfs, B_TRUE); + zfs_unregister_callbacks(zfsvfs); + error = zfs_register_callbacks(vfsp); + if (error) { + dprintf("%s: remount returned %d", + __func__, error); + } + } + + if (vfs_iswriteupgrade(vfsp)) { + /* upgrade */ + dprintf("%s: upgrade requested\n", __func__); + zfsvfs->z_rdonly = 0; + readonly_changed_cb(zfsvfs, B_FALSE); + zfs_unregister_callbacks(zfsvfs); + error = zfs_register_callbacks(vfsp); + if (error) { + dprintf("%s: remount returned %d", + __func__, error); + } + } + } + goto out; + } + + if (vfs_fsprivate(vfsp) != NULL) { + dprintf("already mounted\n"); + error = 0; + goto out; + } + + error = zfs_domount(vfsp, 0, osname, context); + if (error) { + dprintf("%s: zfs_domount returned %d\n", + __func__, error); + goto out; + } + +out: + + if (error == 0) { + + /* Indicate to VFS that we support ACLs. */ + vfs_setextendedsecurity(vfsp); + + } + + if (error) + dprintf("zfs_vfs_mount: error %d\n", error); + + if (osname) + kmem_free(osname, MAXPATHLEN); + + if (options) + kmem_free(options, mnt_args->optlen); + + return (error); +} + +int +zfs_vnode_lock(vnode_t *vp, int flags) +{ + int error; + + ASSERT(vp != NULL); + + error = vn_lock(vp, flags); + return (error); +} + +/* + * The ARC has requested that the filesystem drop entries from the dentry + * and inode caches. This can occur when the ARC needs to free meta data + * blocks but can't because they are all pinned by entries in these caches. + */ + +/* Get vnode for the root object of this mount */ +int +zfs_vfs_root(struct mount *mp, vnode_t **vpp, __unused vfs_context_t context) +{ + zfsvfs_t *zfsvfs = vfs_fsprivate(mp); + znode_t *rootzp = NULL; + int error; + + if (!zfsvfs) { + struct vfsstatfs *stat = 0; + if (mp) stat = vfs_statfs(mp); + if (stat) + dprintf("%s mp on %s from %s\n", __func__, + stat->f_mntonname, stat->f_mntfromname); + dprintf("%s no zfsvfs yet for mp\n", __func__); + return (EINVAL); + } + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + error = zfs_zget(zfsvfs, zfsvfs->z_root, &rootzp); + if (error == 0) + *vpp = ZTOV(rootzp); + else + *vpp = NULL; + + zfs_exit(zfsvfs, FTAG); + + if (error == 0 && *vpp != NULL) + if (vnode_vtype(*vpp) != VDIR) { + panic("%s: not a directory\n", __func__); + } + + return (error); +} + +/* + * Teardown the zfsvfs::z_os. + * + * Note, if 'unmounting' is FALSE, we return with the 'z_teardown_lock' + * and 'z_teardown_inactive_lock' held. + */ +static int +zfsvfs_teardown(zfsvfs_t *zfsvfs, boolean_t unmounting) +{ + znode_t *zp; + /* + * We have experienced deadlocks with dmu_recv_end happening between + * suspend_fs() and resume_fs(). Clearly something is not quite ready + * so we will wait for pools to be synced first. + * This is considered a temporary solution until we can work out + * the full issue. + */ + + zfs_unlinked_drain_stop_wait(zfsvfs); + + /* + * If someone has not already unmounted this file system, + * drain the iput_taskq to ensure all active references to the + * zfs_sb_t have been handled only then can it be safely destroyed. + */ + if (zfsvfs->z_os) { + /* + * If we're unmounting we have to wait for the list to + * drain completely. + * + * If we're not unmounting there's no guarantee the list + * will drain completely, but iputs run from the taskq + * may add the parents of dir-based xattrs to the taskq + * so we want to wait for these. + * + * We can safely read z_nr_znodes without locking because the + * VFS has already blocked operations which add to the + * z_all_znodes list and thus increment z_nr_znodes. + */ + int round = 0; + while (!list_empty(&zfsvfs->z_all_znodes)) { + taskq_wait_outstanding(dsl_pool_zrele_taskq( + dmu_objset_pool(zfsvfs->z_os)), 0); + if (++round > 1 && !unmounting) + break; + break; /* Only loop once - osx can get stuck */ + } + } + + ZFS_TEARDOWN_ENTER_WRITE(zfsvfs, FTAG); + + if (!unmounting) { + /* + * We purge the parent filesystem's vfsp as the parent + * filesystem and all of its snapshots have their vnode's + * v_vfsp set to the parent's filesystem's vfsp. Note, + * 'z_parent' is self referential for non-snapshots. + */ + cache_purgevfs(zfsvfs->z_parent->z_vfs); + } + + /* + * Close the zil. NB: Can't close the zil while zfs_inactive + * threads are blocked as zil_close can call zfs_inactive. + */ + if (zfsvfs->z_log) { + zil_close(zfsvfs->z_log); + zfsvfs->z_log = NULL; + } + + rw_enter(&zfsvfs->z_teardown_inactive_lock, RW_WRITER); + + /* + * If we are not unmounting (ie: online recv) and someone already + * unmounted this file system while we were doing the switcheroo, + * or a reopen of z_os failed then just bail out now. + */ + if (!unmounting && (zfsvfs->z_unmounted || zfsvfs->z_os == NULL)) { + rw_exit(&zfsvfs->z_teardown_inactive_lock); + ZFS_TEARDOWN_EXIT(zfsvfs, FTAG); + return (SET_ERROR(EIO)); + } + /* + * At this point there are no VFS ops active, and any new VFS ops + * will fail with EIO since we have z_teardown_lock for writer (only + * relevant for forced unmount). + * + * Release all holds on dbufs. We also grab an extra reference to all + * the remaining inodes so that the kernel does not attempt to free + * any inodes of a suspended fs. This can cause deadlocks since the + * zfs_resume_fs() process may involve starting threads, which might + * attempt to free unreferenced inodes to free up memory for the new + * thread. + */ + if (!unmounting) { + mutex_enter(&zfsvfs->z_znodes_lock); + for (zp = list_head(&zfsvfs->z_all_znodes); zp != NULL; + zp = list_next(&zfsvfs->z_all_znodes, zp)) { + if (zp->z_sa_hdl) + zfs_znode_dmu_fini(zp); + if (VN_HOLD(ZTOV(zp)) == 0) { + vnode_ref(ZTOV(zp)); + zp->z_suspended = B_TRUE; + VN_RELE(ZTOV(zp)); + } + } + mutex_exit(&zfsvfs->z_znodes_lock); + } + + /* + * If we are unmounting, set the unmounted flag and let new VFS ops + * unblock. zfs_inactive will have the unmounted behavior, and all + * other VFS ops will fail with EIO. + */ + if (unmounting) { + zfsvfs->z_unmounted = B_TRUE; + rw_exit(&zfsvfs->z_teardown_inactive_lock); + ZFS_TEARDOWN_EXIT(zfsvfs, FTAG); + } + + /* + * z_os will be NULL if there was an error in attempting to reopen + * zfsvfs, so just return as the properties had already been + * unregistered and cached data had been evicted before. + */ + if (zfsvfs->z_os == NULL) + return (0); + + /* + * Unregister properties. + */ + zfs_unregister_callbacks(zfsvfs); + + /* + * Evict cached data + */ + /* + * Evict cached data. We must write out any dirty data before + * disowning the dataset. + */ + objset_t *os = zfsvfs->z_os; + boolean_t os_dirty = B_FALSE; + for (int t = 0; t < TXG_SIZE; t++) { + if (dmu_objset_is_dirty(os, t)) { + os_dirty = B_TRUE; + break; + } + } + if (!zfs_is_readonly(zfsvfs) && os_dirty) { + txg_wait_synced(dmu_objset_pool(zfsvfs->z_os), 0); + } + dmu_objset_evict_dbufs(zfsvfs->z_os); + dsl_dir_t *dd = os->os_dsl_dataset->ds_dir; + dsl_dir_cancel_waiters(dd); + + return (0); +} + +int +zfs_vfs_unmount(struct mount *mp, int mntflags, vfs_context_t context) +{ + zfsvfs_t *zfsvfs = vfs_fsprivate(mp); + objset_t *os; + char osname[MAXNAMELEN]; + int ret; + /* cred_t *cr = (cred_t *)vfs_context_ucred(context); */ + int destroyed_zfsctl = 0; + + dprintf("%s\n", __func__); + + zfs_unlinked_drain_stop_wait(zfsvfs); + + /* Save osname for later */ + dmu_objset_name(zfsvfs->z_os, osname); + + /* + * We might skip the sync called in the unmount path, since + * zfs_vfs_sync() is generally ignoring xnu's calls, and alas, + * mount_isforce() is set AFTER that sync call, so we can not + * detect unmount is inflight. But why not just sync now, it + * is safe. Optionally, sync if (mount_isforce()); + */ + // spa_sync_allpools(); + + /* + * We purge the parent filesystem's vfsp as the parent filesystem + * and all of its snapshots have their vnode's v_vfsp set to the + * parent's filesystem's vfsp. Note, 'z_parent' is self + * referential for non-snapshots. + */ + cache_purgevfs(zfsvfs->z_parent->z_vfs); + + /* + * Unmount any snapshots mounted under .zfs before unmounting the + * dataset itself. + * + * Unfortunately, XNU will check for mounts in preflight, and + * simply not call us at all if snapshots are mounted. + * We expect userland to unmount snapshots now. + */ + + ret = vflush(mp, NULLVP, SKIPSYSTEM); + + if (mntflags & MNT_FORCE) { + /* + * Mark file system as unmounted before calling + * vflush(FORCECLOSE). This way we ensure no future vnops + * will be called and risk operating on DOOMED vnodes. + */ + ZFS_TEARDOWN_ENTER_WRITE(zfsvfs, FTAG); + zfsvfs->z_unmounted = B_TRUE; + ZFS_TEARDOWN_EXIT(zfsvfs, FTAG); + } + + /* + * We must release ctldir before vflush on osx. + */ + if (zfsvfs->z_ctldir != NULL) { + destroyed_zfsctl = 1; + zfsctl_destroy(zfsvfs); + } + + /* + * Flush all the files. + */ + ret = vflush(mp, NULLVP, + (mntflags & MNT_FORCE) ? FORCECLOSE|SKIPSYSTEM : SKIPSYSTEM); + + if ((ret != 0) && !(mntflags & MNT_FORCE)) { + if (destroyed_zfsctl) + zfsctl_create(zfsvfs); + return (ret); + } + + /* If we are ourselves a snapshot */ + if (dmu_objset_is_snapshot(zfsvfs->z_os)) { + /* Wake up anyone waiting for unmount */ + zfsctl_mount_signal(zfsvfs, osname, B_FALSE); + } + + /* + * Last chance to dump unreferenced system files. + */ + (void) vflush(mp, NULLVP, FORCECLOSE); + + VERIFY(zfsvfs_teardown(zfsvfs, B_TRUE) == 0); + os = zfsvfs->z_os; + + /* + * z_os will be NULL if there was an error in + * attempting to reopen zfsvfs. + */ + if (os != NULL) { + /* + * Unset the objset user_ptr. + */ + mutex_enter(&os->os_user_ptr_lock); + dmu_objset_set_user(os, NULL); + mutex_exit(&os->os_user_ptr_lock); + + /* + * Finally release the objset + */ + dmu_objset_disown(os, B_TRUE, zfsvfs); + } + + zfs_freevfs(zfsvfs->z_vfs); + + return (0); +} + +static int +zfs_vget_internal(zfsvfs_t *zfsvfs, ino64_t ino, vnode_t **vpp) +{ + znode_t *zp; + int err; + + dprintf("vget get %llu\n", ino); + + /* + * Check to see if we expect to find this in the hardlink avl tree of + * hashes. Use the MSB set high as indicator. + */ + hardlinks_t *findnode = NULL; + if ((1ULL<<31) & ino) { + hardlinks_t *searchnode; + avl_index_t loc; + + searchnode = kmem_alloc(sizeof (hardlinks_t), KM_SLEEP); + + dprintf("ZFS: vget looking for (%llx,%llu)\n", ino, ino); + + searchnode->hl_linkid = ino; + + rw_enter(&zfsvfs->z_hardlinks_lock, RW_READER); + findnode = avl_find(&zfsvfs->z_hardlinks_linkid, searchnode, + &loc); + rw_exit(&zfsvfs->z_hardlinks_lock); + + kmem_free(searchnode, sizeof (hardlinks_t)); + + if (findnode) { + dprintf("ZFS: vget found (%llu, %llu, %u): '%s'\n", + findnode->hl_parent, + findnode->hl_fileid, findnode->hl_linkid, + findnode->hl_name); + // Lookup the actual zp instead + ino = findnode->hl_fileid; + } // findnode + } // MSB set + + + /* We can not be locked during zget. */ + if (!ino) { + dprintf("%s: setting ino from %lld to 2\n", __func__, ino); + ino = 2; + } + + err = zfs_zget(zfsvfs, ino, &zp); + + if (err) { + dprintf("zget failed %d\n", err); + return (err); + } + + /* Don't expose EA objects! */ + if (zp->z_pflags & ZFS_XATTR) { + err = ENOENT; + goto out; + } + if (zp->z_unlinked) { + err = EINVAL; + goto out; + } + + *vpp = ZTOV(zp); + + err = zfs_vnode_lock(*vpp, 0 /* flags */); + + /* + * Spotlight requires that vap->va_name() is set when returning + * from vfs_vget, so that vfs_getrealpath() can succeed in returning + * a path to mds. + */ + char *name = kmem_alloc(MAXPATHLEN + 2, KM_SLEEP); + + /* Root can't lookup in ZAP */ + if (zp->z_id == zfsvfs->z_root) { + + dmu_objset_name(zfsvfs->z_os, name); + dprintf("vget: set root '%s'\n", name); + // vnode_update_identity(*vpp, NULL, name, + // strlen(name), 0, VNODE_UPDATE_NAME); + + } else { + uint64_t parent; + + // if its a hardlink cache + if (findnode) { + + dprintf("vget: updating vnode to '%s' parent %llu\n", + findnode->hl_name, findnode->hl_parent); + + // vnode_update_identity(*vpp, + // NULL, findnode->hl_name, + // strlen(findnode->hl_name), 0, + // VNODE_UPDATE_NAME|VNODE_UPDATE_PARENT); + mutex_enter(&zp->z_lock); + strlcpy(zp->z_name_cache, findnode->hl_name, PATH_MAX); + // zp->z_finder_parentid = findnode->hl_parent; + mutex_exit(&zp->z_lock); + + + // If we already have the name, cached in zfs_vnop_lookup + } else if (zp->z_name_cache[0]) { + dprintf("vget: cached name '%s'\n", zp->z_name_cache); + // vnode_update_identity(*vpp, NULL, zp->z_name_cache, + // strlen(zp->z_name_cache), 0, + // VNODE_UPDATE_NAME); + + /* If needed, if findnode is set, update the parentid */ + + } else { + + /* Lookup name from ID, grab parent */ + VERIFY(sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)) == 0); + + if (zap_value_search(zfsvfs->z_os, parent, zp->z_id, + ZFS_DIRENT_OBJ(-1ULL), name) == 0) { + + dprintf("vget: set name '%s'\n", name); + // vnode_update_identity(*vpp, NULL, name, + // strlen(name), 0, + // VNODE_UPDATE_NAME); + } else { + dprintf("vget: unable to get name for %llu\n", + zp->z_id); + } // !zap_search + } + } // rootid + + kmem_free(name, MAXPATHLEN + 2); + +out: + + if (err != 0) { + VN_RELE(ZTOV(zp)); + *vpp = NULL; + } + + dprintf("vget return %d\n", err); + return (err); +} + +/* + * Get a vnode from a file id (ignoring the generation) + * + * Use by NFS Server (readdirplus) and VFS (build_path) + */ +int +zfs_vfs_vget(struct mount *mp, ino64_t ino, vnode_t **vpp, + __unused vfs_context_t context) +{ + zfsvfs_t *zfsvfs = vfs_fsprivate(mp); + int error; + + dprintf("%s: %llu\n", __func__, ino); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + /* We also need to handle (.zfs) and (.zfs/snapshot). */ + if ((ino == ZFSCTL_INO_ROOT) && (zfsvfs->z_ctldir != NULL)) { + if (VN_HOLD(zfsvfs->z_ctldir) == 0) { + *vpp = zfsvfs->z_ctldir; + error = 0; + } else { + error = ENOENT; + } + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * This one is trickier, we have no reference to it, but it is + * in the all list. A little expensive to search list, but at + * least "snapshot" is infrequently accessed + * We also need to check if it is a ".zfs/snapshot/$name" entry - + * luckily we keep the "lowest" ID seen, so we only need to check + * when it is in the range. + */ + if (zfsvfs->z_ctldir != NULL) { + + /* + * Either it is the snapdir itself, or one of the snapshot + * directories inside it + */ + if ((ino == ZFSCTL_INO_SNAPDIR) || + ((ino >= zfsvfs->z_ctldir_startid) && + (ino <= ZFSCTL_INO_SNAPDIRS))) { + znode_t *zp; + + mutex_enter(&zfsvfs->z_znodes_lock); + for (zp = list_head(&zfsvfs->z_all_znodes); zp; + zp = list_next(&zfsvfs->z_all_znodes, zp)) { + if (zp->z_id == ino) + break; + if (zp->z_id == ZFSCTL_INO_SNAPDIRS - ino) + break; + } + mutex_exit(&zfsvfs->z_znodes_lock); + + error = ENOENT; + if (zp != NULL) { + if (VN_HOLD(ZTOV(zp)) == 0) { + *vpp = ZTOV(zp); + error = 0; + } + } + + zfs_exit(zfsvfs, FTAG); + return (error); + } + } + + error = zfs_vget_internal(zfsvfs, ino, vpp); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * NFS Server File Handle File ID + */ +typedef struct zfs_zfid { + uint8_t zf_object[8]; /* obj[i] = obj >> (8 * i) */ + uint8_t zf_gen[8]; /* gen[i] = gen >> (8 * i) */ +} zfs_zfid_t; + +/* + * File handle to vnode pointer + */ +int +zfs_vfs_fhtovp(struct mount *mp, int fhlen, unsigned char *fhp, + vnode_t **vpp, __unused vfs_context_t context) +{ + dprintf("%s\n", __func__); + zfsvfs_t *zfsvfs = vfs_fsprivate(mp); + zfs_zfid_t *zfid = (zfs_zfid_t *)fhp; + znode_t *zp; + uint64_t obj_num = 0; + uint64_t fid_gen = 0; + uint64_t zp_gen; + int i; + int error; + + *vpp = NULL; + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + if (fhlen < sizeof (zfs_zfid_t)) { + error = EINVAL; + goto out; + } + + /* + * Grab the object and gen numbers in an endian neutral manner + */ + for (i = 0; i < sizeof (zfid->zf_object); i++) + obj_num |= ((uint64_t)zfid->zf_object[i]) << (8 * i); + + for (i = 0; i < sizeof (zfid->zf_gen); i++) + fid_gen |= ((uint64_t)zfid->zf_gen[i]) << (8 * i); + + if ((error = zfs_zget(zfsvfs, obj_num, &zp))) { + goto out; + } + + zp_gen = zp->z_gen; + if (zp_gen == 0) + zp_gen = 1; + + if (zp->z_unlinked || zp_gen != fid_gen) { + vnode_put(ZTOV(zp)); + error = EINVAL; + goto out; + } + *vpp = ZTOV(zp); +out: + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Vnode pointer to File handle + * + * XXX Do we want to check the DSL sharenfs property? + */ +int +zfs_vfs_vptofh(vnode_t *vp, int *fhlenp, unsigned char *fhp, + __unused vfs_context_t context) +{ + dprintf("%s\n", __func__); + zfsvfs_t *zfsvfs = vfs_fsprivate(vnode_mount(vp)); + zfs_zfid_t *zfid = (zfs_zfid_t *)fhp; + znode_t *zp = VTOZ(vp); + uint64_t obj_num; + uint64_t zp_gen; + int i; + int error = 0; + + if (*fhlenp < sizeof (zfs_zfid_t)) { + return (EOVERFLOW); + } + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + obj_num = zp->z_id; + zp_gen = zp->z_gen; + if (zp_gen == 0) + zp_gen = 1; + + /* + * Store the object and gen numbers in an endian neutral manner + */ + for (i = 0; i < sizeof (zfid->zf_object); i++) + zfid->zf_object[i] = (uint8_t)(obj_num >> (8 * i)); + + for (i = 0; i < sizeof (zfid->zf_gen); i++) + zfid->zf_gen[i] = (uint8_t)(zp_gen >> (8 * i)); + + *fhlenp = sizeof (zfs_zfid_t); + + zfs_exit(zfsvfs, FTAG); + return (0); +} + +/* + * Block out VOPs and close zfsvfs_t::z_os + * + * Note, if successful, then we return with the 'z_teardown_lock' and + * 'z_teardown_inactive_lock' write held. We leave ownership of the underlying + * dataset and objset intact so that they can be atomically handed off during + * a subsequent rollback or recv operation and the resume thereafter. + */ +int +zfs_suspend_fs(zfsvfs_t *zfsvfs) +{ + int error; + + if ((error = zfsvfs_teardown(zfsvfs, B_FALSE)) != 0) + return (error); + + return (0); +} + +/* + * Reopen zfsvfs_t::z_os and release VOPs. + */ +int +zfs_resume_fs(zfsvfs_t *zfsvfs, dsl_dataset_t *ds) +{ + int err; + znode_t *zp; + + ASSERT(ZFS_TEARDOWN_WRITE_HELD(zfsvfs)); + ASSERT(RW_WRITE_HELD(&zfsvfs->z_teardown_inactive_lock)); + + /* + * We already own this, so just update the objset_t, as the one we + * had before may have been evicted. + */ + objset_t *os; + VERIFY3P(ds->ds_owner, ==, zfsvfs); + VERIFY(dsl_dataset_long_held(ds)); + dsl_pool_t *dp = spa_get_dsl(dsl_dataset_get_spa(ds)); + dsl_pool_config_enter(dp, FTAG); + VERIFY0(dmu_objset_from_ds(ds, &os)); + dsl_pool_config_exit(dp, FTAG); + + err = zfsvfs_init(zfsvfs, os); + if (err != 0) + goto bail; + + ds->ds_dir->dd_activity_cancelled = B_FALSE; + VERIFY(zfsvfs_setup(zfsvfs, B_FALSE) == 0); + + zfs_set_fuid_feature(zfsvfs); + + /* + * Attempt to re-establish all the active inodes with their + * dbufs. If a zfs_rezget() fails, then we unhash the inode + * and mark it stale. This prevents a collision if a new + * inode/object is created which must use the same inode + * number. The stale inode will be be released when the + * VFS prunes the dentry holding the remaining references + * on the stale inode. + */ + mutex_enter(&zfsvfs->z_znodes_lock); + for (zp = list_head(&zfsvfs->z_all_znodes); zp; + zp = list_next(&zfsvfs->z_all_znodes, zp)) { + (void) zfs_rezget(zp); + + /* see comment in zfs_suspend_fs() */ + if (zp->z_suspended) { + if (vnode_getwithref(ZTOV(zp)) == 0) { + vnode_rele(ZTOV(zp)); + zfs_zrele_async(zp); + zp->z_suspended = B_FALSE; + } + } + } + mutex_exit(&zfsvfs->z_znodes_lock); + + if (!vfs_isrdonly(zfsvfs->z_vfs) && !zfsvfs->z_unmounted) { + /* + * zfs_suspend_fs() could have interrupted freeing + * of dnodes. We need to restart this freeing so + * that we don't "leak" the space. + */ + zfs_unlinked_drain(zfsvfs); + } + + cache_purgevfs(zfsvfs->z_parent->z_vfs); + +bail: + /* release the VFS ops */ + rw_exit(&zfsvfs->z_teardown_inactive_lock); + ZFS_TEARDOWN_EXIT(zfsvfs, FTAG); + + if (err) { + /* + * Since we couldn't setup the sa framework, try to force + * unmount this file system. + */ + if (zfsvfs->z_os) + zfs_vfs_unmount(zfsvfs->z_vfs, 0, NULL); + } + return (err); +} + + +void +zfs_freevfs(struct mount *vfsp) +{ + zfsvfs_t *zfsvfs = vfs_fsprivate(vfsp); + + dprintf("+freevfs\n"); + + vfs_setfsprivate(vfsp, NULL); + + zfsvfs_free(zfsvfs); + + atomic_dec_32(&zfs_active_fs_count); + dprintf("-freevfs\n"); +} + +struct fromname_struct { + char *oldname; + char *newname; +}; +typedef struct fromname_struct fromname_t; + +void +zfsvfs_update_fromname(const char *oldname, const char *newname) +{ + fromname_t frna; + + // find oldname's vfsp + // vfs_mountedfrom(vfsp, newname); + frna.oldname = oldname; + frna.newname = newname; +} + +void +zfs_init(void) +{ + + dprintf("ZFS filesystem version: " ZPL_VERSION_STRING "\n"); + + /* + * Initialize .zfs directory structures + */ + zfsctl_init(); + + /* + * Initialize znode cache, vnode ops, etc... + */ + zfs_znode_init(); + + dmu_objset_register_type(DMU_OST_ZFS, zpl_get_file_info); + + /* Start arc_os - reclaim thread */ + arc_os_init(); + +} + +void +zfs_fini(void) +{ + arc_os_fini(); + zfsctl_fini(); + zfs_znode_fini(); +} + +int +zfs_busy(void) +{ + return (zfs_active_fs_count != 0); +} + +/* + * Release VOPs and unmount a suspended filesystem. + */ +int +zfs_end_fs(zfsvfs_t *zfsvfs, dsl_dataset_t *ds) +{ + ASSERT(ZFS_TEARDOWN_WRITE_HELD(zfsvfs)); + ASSERT(RW_WRITE_HELD(&zfsvfs->z_teardown_inactive_lock)); + + /* + * We already own this, so just hold and rele it to update the + * objset_t, as the one we had before may have been evicted. + */ + objset_t *os; + VERIFY3P(ds->ds_owner, ==, zfsvfs); + VERIFY(dsl_dataset_long_held(ds)); + dsl_pool_t *dp = spa_get_dsl(dsl_dataset_get_spa(ds)); + dsl_pool_config_enter(dp, FTAG); + VERIFY0(dmu_objset_from_ds(ds, &os)); + dsl_pool_config_exit(dp, FTAG); + zfsvfs->z_os = os; + + /* release the VOPs */ + rw_exit(&zfsvfs->z_teardown_inactive_lock); + ZFS_TEARDOWN_EXIT(zfsvfs, FTAG); + + /* + * Try to force unmount this file system. + */ + zfs_vfs_unmount(zfsvfs->z_vfs, 0, NULL); + zfsvfs->z_unmounted = B_TRUE; + return (0); +} + +int +zfs_set_version(zfsvfs_t *zfsvfs, uint64_t newvers) +{ + int error; + objset_t *os = zfsvfs->z_os; + dmu_tx_t *tx; + + if (newvers < ZPL_VERSION_INITIAL || newvers > ZPL_VERSION) + return (SET_ERROR(EINVAL)); + + if (newvers < zfsvfs->z_version) + return (SET_ERROR(EINVAL)); + + if (zfs_spa_version_map(newvers) > + spa_version(dmu_objset_spa(zfsvfs->z_os))) + return (SET_ERROR(ENOTSUP)); + + tx = dmu_tx_create(os); + dmu_tx_hold_zap(tx, MASTER_NODE_OBJ, B_FALSE, ZPL_VERSION_STR); + if (newvers >= ZPL_VERSION_SA && !zfsvfs->z_use_sa) { + dmu_tx_hold_zap(tx, MASTER_NODE_OBJ, B_TRUE, + ZFS_SA_ATTRS); + dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); + } + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + return (error); + } + + error = zap_update(os, MASTER_NODE_OBJ, ZPL_VERSION_STR, + 8, 1, &newvers, tx); + + if (error) { + dmu_tx_commit(tx); + return (error); + } + + if (newvers >= ZPL_VERSION_SA && !zfsvfs->z_use_sa) { + uint64_t sa_obj; + + ASSERT3U(spa_version(dmu_objset_spa(zfsvfs->z_os)), >=, + SPA_VERSION_SA); + sa_obj = zap_create(os, DMU_OT_SA_MASTER_NODE, + DMU_OT_NONE, 0, tx); + + error = zap_add(os, MASTER_NODE_OBJ, + ZFS_SA_ATTRS, 8, 1, &sa_obj, tx); + ASSERT(error == 0); + + VERIFY(0 == sa_set_sa_object(os, sa_obj)); + sa_register_update_callback(os, zfs_sa_upgrade); + } + + spa_history_log_internal(dmu_objset_spa(os), "upgrade", tx, + "oldver=%llu newver=%llu dataset = %llu", zfsvfs->z_version, + newvers, dmu_objset_id(os)); + + dmu_tx_commit(tx); + + zfsvfs->z_version = newvers; + os->os_version = newvers; + + zfs_set_fuid_feature(zfsvfs); + + return (0); +} + +/* + * Return true if the coresponding vfs's unmounted flag is set. + * Otherwise return false. + * If this function returns true we know VFS unmount has been initiated. + */ +boolean_t +zfs_get_vfs_flag_unmounted(objset_t *os) +{ + zfsvfs_t *zfvp; + boolean_t unmounted = B_FALSE; + + ASSERT(dmu_objset_type(os) == DMU_OST_ZFS); + + mutex_enter(&os->os_user_ptr_lock); + zfvp = dmu_objset_get_user(os); + if (zfvp != NULL && zfvp->z_vfs != NULL && + (vfs_isunmount(zfvp->z_vfs))) + unmounted = B_TRUE; + mutex_exit(&os->os_user_ptr_lock); + + return (unmounted); +} + + +ZFS_MODULE_PARAM(zfs, zfs_, vnop_skip_unlinked_drain, + UINT, ZMOD_RW, "Do not call unlinked_drain on import"); diff --git a/module/os/windows/zfs/zfs_vnops_os.c b/module/os/windows/zfs/zfs_vnops_os.c new file mode 100644 index 000000000000..20bd490e4de0 --- /dev/null +++ b/module/os/windows/zfs/zfs_vnops_os.c @@ -0,0 +1,3408 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. + * Copyright (c) 2015 by Chunwei Chen. All rights reserved. + * Copyright 2017 Nexenta Systems, Inc. + * Copyright (c) 2013, 2020 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ + +/* Portions Copyright 2007 Jeremy Teo */ +/* Portions Copyright 2010 Robert Milkowski */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +int zfs_vnop_force_formd_normalized_output = 0; /* disabled by default */ + +/* + * Programming rules. + * + * Each vnode op performs some logical unit of work. To do this, the ZPL must + * properly lock its in-core state, create a DMU transaction, do the work, + * record this work in the intent log (ZIL), commit the DMU transaction, + * and wait for the intent log to commit if it is a synchronous operation. + * Moreover, the vnode ops must work in both normal and log replay context. + * The ordering of events is important to avoid deadlocks and references + * to freed memory. The example below illustrates the following Big Rules: + * + * (1) A check must be made in each zfs thread for a mounted file system. + * This is done avoiding races using zfs_enter(zfsvfs). + * A zfs_exit(zfsvfs) is needed before all returns. Any znodes + * must be checked with zfs_verify_zp(zp). Both of these macros + * can return EIO from the calling function. + * + * (2) zrele() should always be the last thing except for zil_commit() + * (if necessary) and zfs_exit(). This is for 3 reasons: + * First, if it's the last reference, the vnode/znode + * can be freed, so the zp may point to freed memory. Second, the last + * reference will call zfs_zinactive(), which may induce a lot of work -- + * pushing cached pages (which acquires range locks) and syncing out + * cached atime changes. Third, zfs_zinactive() may require a new tx, + * which could deadlock the system if you were already holding one. + * If you must call zrele() within a tx then use zfs_zrele_async(). + * + * (3) All range locks must be grabbed before calling dmu_tx_assign(), + * as they can span dmu_tx_assign() calls. + * + * (4) If ZPL locks are held, pass TXG_NOWAIT as the second argument to + * dmu_tx_assign(). This is critical because we don't want to block + * while holding locks. + * + * If no ZPL locks are held (aside from zfs_enter()), use TXG_WAIT. This + * reduces lock contention and CPU usage when we must wait (note that if + * throughput is constrained by the storage, nearly every transaction + * must wait). + * + * Note, in particular, that if a lock is sometimes acquired before + * the tx assigns, and sometimes after (e.g. z_lock), then failing + * to use a non-blocking assign can deadlock the system. The scenario: + * + * Thread A has grabbed a lock before calling dmu_tx_assign(). + * Thread B is in an already-assigned tx, and blocks for this lock. + * Thread A calls dmu_tx_assign(TXG_WAIT) and blocks in txg_wait_open() + * forever, because the previous txg can't quiesce until B's tx commits. + * + * If dmu_tx_assign() returns ERESTART and zfsvfs->z_assign is TXG_NOWAIT, + * then drop all locks, call dmu_tx_wait(), and try again. On subsequent + * calls to dmu_tx_assign(), pass TXG_NOTHROTTLE in addition to TXG_NOWAIT, + * to indicate that this operation has already called dmu_tx_wait(). + * This will ensure that we don't retry forever, waiting a short bit + * each time. + * + * (5) If the operation succeeded, generate the intent log entry for it + * before dropping locks. This ensures that the ordering of events + * in the intent log matches the order in which they actually occurred. + * During ZIL replay the zfs_log_* functions will update the sequence + * number to indicate the zil transaction has replayed. + * + * (6) At the end of each vnode op, the DMU tx must always commit, + * regardless of whether there were any errors. + * + * (7) After dropping all locks, invoke zil_commit(zilog, foid) + * to ensure that synchronous semantics are provided when necessary. + * + * In general, this is how things should be ordered in each vnode op: + * + * if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + * return (error); // exit if unmounted + * top: + * zfs_dirent_lock(&dl, ...) // lock directory entry (may igrab()) + * rw_enter(...); // grab any other locks you need + * tx = dmu_tx_create(...); // get DMU tx + * dmu_tx_hold_*(); // hold each object you might modify + * error = dmu_tx_assign(tx, (waited ? TXG_NOTHROTTLE : 0) | TXG_NOWAIT); + * if (error) { + * rw_exit(...); // drop locks + * zfs_dirent_unlock(dl); // unlock directory entry + * zrele(...); // release held znodes + * if (error == ERESTART) { + * waited = B_TRUE; + * dmu_tx_wait(tx); + * dmu_tx_abort(tx); + * goto top; + * } + * dmu_tx_abort(tx); // abort DMU tx + * zfs_exit(zfsvfs, FTAG); // finished in zfs + * return (error); // really out of space + * } + * error = do_real_work(); // do whatever this VOP does + * if (error == 0) + * zfs_log_*(...); // on success, make ZIL entry + * dmu_tx_commit(tx); // commit DMU tx -- error or not + * rw_exit(...); // drop locks + * zfs_dirent_unlock(dl); // unlock directory entry + * zrele(...); // release held znodes + * zil_commit(zilog, foid); // synchronous when necessary + * zfs_exit(zfsvfs, FTAG); // finished in zfs + * return (error); // done, report error + */ + +/* + * Virus scanning is unsupported. It would be possible to add a hook + * here to performance the required virus scan. This could be done + * entirely in the kernel or potentially as an update to invoke a + * scanning utility. + */ +static int +zfs_vscan(struct vnode *vp, cred_t *cr, int async) +{ + return (0); +} + +int +zfs_open(struct vnode *vp, int mode, int flag, cred_t *cr) +{ + (void) cr; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = ITOZSB(vp); + int error; + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + + /* Honor ZFS_APPENDONLY file attribute */ + if ((mode & FWRITE) && (zp->z_pflags & ZFS_APPENDONLY) && + ((flag & O_APPEND) == 0)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EPERM)); + } + + /* Virus scan eligible files on open */ + if (!zfs_has_ctldir(zp) && zfsvfs->z_vscan && S_ISREG(zp->z_mode) && + !(zp->z_pflags & ZFS_AV_QUARANTINED) && zp->z_size > 0) { + if (zfs_vscan(vp, cr, 0) != 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EACCES)); + } + } + + /* Keep a count of the synchronous opens in the znode */ + if (flag & (FSYNC | FDSYNC)) + atomic_inc_32(&zp->z_sync_cnt); + + zfs_exit(zfsvfs, FTAG); + return (0); +} + +int +zfs_close(struct vnode *vp, int flag, cred_t *cr) +{ + (void) cr; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = ITOZSB(vp); + int error; + + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + + /* Decrement the synchronous opens in the znode */ + if (flag & (FSYNC | FDSYNC)) + atomic_dec_32(&zp->z_sync_cnt); + + if (!zfs_has_ctldir(zp) && zfsvfs->z_vscan && S_ISREG(zp->z_mode) && + !(zp->z_pflags & ZFS_AV_QUARANTINED) && zp->z_size > 0) + VERIFY(zfs_vscan(vp, cr, 1) == 0); + + zfs_exit(zfsvfs, FTAG); + return (0); +} + +#if defined(_KERNEL) +/* + * When a file is memory mapped, we must keep the IO data synchronized + * between the DMU cache and the memory mapped pages. What this means: + * + * On Write: If we find a memory mapped page, we write to *both* + * the page and the dmu buffer. + */ +void +update_pages(znode_t *zp, int64_t start, int len, + objset_t *os) +{ +} + +int +mappedread(struct znode *zp, int nbytes, zfs_uio_t *uio) +{ + return (0); +} +#endif + +uint64_t zfs_delete_blocks = DMU_MAX_DELETEBLKCNT; + +/* + * Write the bytes to a file. + * + * IN: zp - znode of file to be written to + * data - bytes to write + * len - number of bytes to write + * pos - offset to start writing at + * + * OUT: resid - remaining bytes to write + * + * RETURN: 0 if success + * positive error code if failure + * + * Timestamps: + * zp - ctime|mtime updated if byte count > 0 + */ +int +zfs_write_simple(znode_t *zp, const void *data, size_t len, + loff_t pos, size_t *presid) +{ + int error = 0; + ssize_t resid; + + error = zfs_vn_rdwr(UIO_WRITE, ZTOV(zp), data, len, + pos, UIO_SYSSPACE, 0 /* IO_SYNC */, RLIM64_INFINITY, NULL, &resid); + + if (error) { + return (SET_ERROR(error)); + } else if (presid == NULL) { + if (resid != 0) { + error = SET_ERROR(EIO); + } + } else { + *presid = resid; + } + return (error); +} + +/* + * Drop a reference on the passed inode asynchronously. This ensures + * that the caller will never drop the last reference on an inode in + * the current context. Doing so while holding open a tx could result + * in a deadlock if iput_final() re-enters the filesystem code. + */ +void +zfs_zrele_async(znode_t *zp) +{ + struct vnode *vp = ZTOV(zp); + objset_t *os = ITOZSB(vp)->z_os; + + ASSERT(os != NULL); + + if (vnode_iocount(vp) == 1) + VERIFY(taskq_dispatch(dsl_pool_zrele_taskq(dmu_objset_pool(os)), + (task_func_t *)vnode_put, vp, TQ_SLEEP) != TASKQID_INVALID); + else + zrele(zp); +} + +/* + * Lookup an entry in a directory, or an extended attribute directory. + * If it exists, return a held inode reference for it. + * + * IN: zdp - znode of directory to search. + * nm - name of entry to lookup. + * flags - LOOKUP_XATTR set if looking for an attribute. + * cr - credentials of caller. + * direntflags - directory lookup flags + * realpnp - returned pathname. + * + * OUT: zpp - znode of located entry, NULL if not found. + * + * RETURN: 0 on success, error code on failure. + * + * Timestamps: + * NA + */ +int +zfs_lookup(znode_t *zdp, char *nm, znode_t **zpp, int flags, + cred_t *cr, int *direntflags, struct componentname *realpnp) +{ + zfsvfs_t *zfsvfs = ZTOZSB(zdp); + int error = 0; + + /* .zfs lookups come in here in Windows */ + if (zfsctl_is_node(zdp)) + return (zfsctl_root_lookup(ZTOV(zdp), nm, + zpp, flags, cr, direntflags, realpnp)); + + if ((error = zfs_enter_verify_zp(zfsvfs, zdp, FTAG)) != 0) + return (error); + + *zpp = NULL; + + /* + * Windows has separate vnops for XATTR activity + */ + if (flags & LOOKUP_XATTR) { + /* + * We don't allow recursive attributes.. + * Maybe someday we will. + */ + if (zdp->z_pflags & ZFS_XATTR) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + if ((error = zfs_get_xattrdir(zdp, zpp, cr, flags))) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * Do we have permission to get into attribute directory? + */ + + if ((error = zfs_zaccess(*zpp, ACE_EXECUTE, 0, + B_FALSE, cr, NULL))) { + zrele(*zpp); + *zpp = NULL; + } + + zfs_exit(zfsvfs, FTAG); + return (error); + } + + + if (!S_ISDIR(zdp->z_mode)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(ENOTDIR)); + } + + /* + * Check accessibility of directory. + */ + + if ((error = zfs_zaccess(zdp, ACE_EXECUTE, 0, B_FALSE, cr, NULL))) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + if (zfsvfs->z_utf8 && u8_validate(nm, strlen(nm), + NULL, U8_VALIDATE_ENTIRE, &error) < 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EILSEQ)); + } + + error = zfs_dirlook(zdp, nm, zpp, flags, direntflags, realpnp); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Attempt to create a new entry in a directory. If the entry + * already exists, truncate the file if permissible, else return + * an error. Return the ip of the created or trunc'd file. + * + * IN: dzp - znode of directory to put new file entry in. + * name - name of new file entry. + * vap - attributes of new file. + * excl - flag indicating exclusive or non-exclusive mode. + * mode - mode to open file with. + * cr - credentials of caller. + * flag - file flag. + * vsecp - ACL to be set + * + * OUT: zpp - znode of created or trunc'd entry. + * + * RETURN: 0 on success, error code on failure. + * + * Timestamps: + * dzp - ctime|mtime updated if new entry created + * zp - ctime|mtime always, atime if new + */ + +int +zfs_create(znode_t *dzp, char *name, vattr_t *vap, int excl, + int mode, znode_t **zpp, cred_t *cr, int flag, vsecattr_t *vsecp, + zidmap_t *mnt_ns) +{ + znode_t *zp = NULL; + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + zilog_t *zilog; + objset_t *os; + zfs_dirlock_t *dl; + dmu_tx_t *tx; + int error; + uid_t uid; + gid_t gid; + zfs_acl_ids_t acl_ids; + boolean_t fuid_dirtied; + boolean_t have_acl = B_FALSE; + boolean_t waited = B_FALSE; + + /* + * If we have an ephemeral id, ACL, or XVATTR then + * make sure file system is at proper version + */ + + gid = crgetgid(cr); + uid = crgetuid(cr); + + if (zfsvfs->z_use_fuids == B_FALSE && + (vsecp || IS_EPHEMERAL(uid) || IS_EPHEMERAL(gid))) + return (SET_ERROR(EINVAL)); + + if (name == NULL) + return (SET_ERROR(EINVAL)); + + if ((error = zfs_enter_verify_zp(zfsvfs, dzp, FTAG)) != 0) + return (error); + os = zfsvfs->z_os; + zilog = zfsvfs->z_log; + + if (zfsvfs->z_utf8 && u8_validate(name, strlen(name), + NULL, U8_VALIDATE_ENTIRE, &error) < 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EILSEQ)); + } + + if (vap->va_mask & ATTR_XVATTR) { + if ((error = secpolicy_xvattr(vap, + crgetuid(cr), cr, vap->va_mode)) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + } + +top: + *zpp = NULL; + if (*name == '\0') { + /* + * Null component name refers to the directory itself. + */ + zhold(dzp); + zp = dzp; + dl = NULL; + error = 0; + } else { + /* possible igrab(zp) */ + int zflg = 0; + + if (flag & FIGNORECASE) + zflg |= ZCILOOK; + + error = zfs_dirent_lock(&dl, dzp, name, &zp, zflg, + NULL, NULL); + if (error) { + if (have_acl) + zfs_acl_ids_free(&acl_ids); + if (strcmp(name, "..") == 0) + error = SET_ERROR(EISDIR); + zfs_exit(zfsvfs, FTAG); + return (error); + } + } + + if (zp == NULL) { + uint64_t txtype; + uint64_t projid = ZFS_DEFAULT_PROJID; + + /* + * Create a new file object and update the directory + * to reference it. + */ + if ((error = zfs_zaccess(dzp, ACE_ADD_FILE, 0, B_FALSE, cr, + NULL))) { + if (have_acl) + zfs_acl_ids_free(&acl_ids); + goto out; + } + + /* + * We only support the creation of regular files in + * extended attribute directories. + */ + + if ((dzp->z_pflags & ZFS_XATTR) && !S_ISREG(vap->va_mode)) { + if (have_acl) + zfs_acl_ids_free(&acl_ids); + error = SET_ERROR(EINVAL); + goto out; + } + + if (!have_acl && (error = zfs_acl_ids_create(dzp, 0, vap, + cr, vsecp, &acl_ids, NULL)) != 0) + goto out; + have_acl = B_TRUE; + + if (S_ISREG(vap->va_mode) || S_ISDIR(vap->va_mode)) + projid = zfs_inherit_projid(dzp); + if (zfs_acl_ids_overquota(zfsvfs, &acl_ids, projid)) { + zfs_acl_ids_free(&acl_ids); + error = SET_ERROR(EDQUOT); + goto out; + } + + tx = dmu_tx_create(os); + + dmu_tx_hold_sa_create(tx, acl_ids.z_aclp->z_acl_bytes + + ZFS_SA_BASE_ATTR_SIZE); + + fuid_dirtied = zfsvfs->z_fuid_dirty; + if (fuid_dirtied) + zfs_fuid_txhold(zfsvfs, tx); + dmu_tx_hold_zap(tx, dzp->z_id, TRUE, name); + dmu_tx_hold_sa(tx, dzp->z_sa_hdl, B_FALSE); + if (!zfsvfs->z_use_sa && + acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, + 0, acl_ids.z_aclp->z_acl_bytes); + } + + error = dmu_tx_assign(tx, + (waited ? TXG_NOTHROTTLE : 0) | TXG_NOWAIT); + if (error) { + zfs_dirent_unlock(dl); + if (error == ERESTART) { + waited = B_TRUE; + dmu_tx_wait(tx); + dmu_tx_abort(tx); + goto top; + } + zfs_acl_ids_free(&acl_ids); + dmu_tx_abort(tx); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + zfs_mknode(dzp, vap, tx, cr, 0, &zp, &acl_ids); + + error = zfs_link_create(dl, zp, tx, ZNEW); + if (error != 0) { + /* + * Since, we failed to add the directory entry for it, + * delete the newly created dnode. + */ + zfs_znode_delete(zp, tx); + zfs_acl_ids_free(&acl_ids); + dmu_tx_commit(tx); + + /* + * Failed, have zp but on OsX we don't have a vp, as it + * would have been attached below, and we've cleared out + * zp, signal then not to call zrele() on it. + */ + if (ZTOV(zp) == NULL) { + zfs_znode_free(zp); + zp = NULL; + } + + goto out; + } + + if (fuid_dirtied) + zfs_fuid_sync(zfsvfs, tx); + + txtype = zfs_log_create_txtype(Z_FILE, vsecp, vap); + if (flag & FIGNORECASE) + txtype |= TX_CI; + zfs_log_create(zilog, tx, txtype, dzp, zp, name, + vsecp, acl_ids.z_fuidp, vap); + zfs_acl_ids_free(&acl_ids); + dmu_tx_commit(tx); + + /* + * Windows - attach the vnode _after_ committing the transaction + */ + zfs_znode_getvnode(zp, dzp, zfsvfs); + + } else { + int aflags = (flag & O_APPEND) ? V_APPEND : 0; + + if (have_acl) + zfs_acl_ids_free(&acl_ids); + have_acl = B_FALSE; + + /* + * A directory entry already exists for this name. + */ + /* + * Can't truncate an existing file if in exclusive mode. + */ + if (excl) { + error = SET_ERROR(EEXIST); + goto out; + } + /* + * Can't open a directory for writing. + */ + if (S_ISDIR(zp->z_mode)) { + error = SET_ERROR(EISDIR); + goto out; + } + /* + * Verify requested access to file. + */ + if (mode && (error = zfs_zaccess_rwx(zp, mode, aflags, cr, + NULL))) { + goto out; + } + + mutex_enter(&dzp->z_lock); + dzp->z_seq++; + mutex_exit(&dzp->z_lock); + + /* + * Truncate regular files if requested. + */ + if (S_ISREG(zp->z_mode) && + (vap->va_mask & ATTR_SIZE) && (vap->va_size == 0)) { + /* we can't hold any locks when calling zfs_freesp() */ + if (dl) { + zfs_dirent_unlock(dl); + dl = NULL; + } + error = zfs_freesp(zp, 0, 0, mode, TRUE); + } + } +out: + + if (dl) + zfs_dirent_unlock(dl); + + if (error) { + if (zp) + zrele(zp); + } else { + *zpp = zp; + } + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Remove an entry from a directory. + * + * IN: dzp - znode of directory to remove entry from. + * name - name of entry to remove. + * cr - credentials of caller. + * flags - case flags. + * + * RETURN: 0 if success + * error code if failure + * + * Timestamps: + * dzp - ctime|mtime + * ip - ctime (if nlink > 0) + */ + +static uint64_t null_xattr = 0; + +int +zfs_remove(znode_t *dzp, char *name, cred_t *cr, int flags) +{ + znode_t *zp; + znode_t *xzp; + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + zilog_t *zilog; + uint64_t acl_obj, xattr_obj; + uint64_t xattr_obj_unlinked = 0; + uint64_t obj = 0; + uint64_t links; + zfs_dirlock_t *dl; + dmu_tx_t *tx; + boolean_t may_delete_now, delete_now = FALSE; + boolean_t unlinked, toobig = FALSE; + uint64_t txtype; + struct componentname *realnmp = NULL; + struct componentname realnm = { 0 }; + int error; + int zflg = ZEXISTS; + boolean_t waited = B_FALSE; + + if (name == NULL) + return (SET_ERROR(EINVAL)); + + if ((error = zfs_enter_verify_zp(zfsvfs, dzp, FTAG)) != 0) + return (error); + zilog = zfsvfs->z_log; + + if (flags & FIGNORECASE) { + zflg |= ZCILOOK; + + realnm.cn_nameptr = kmem_zalloc(MAXPATHLEN, KM_SLEEP); + realnm.cn_namelen = MAXPATHLEN; + realnmp = &realnm; + } + +top: + xattr_obj = 0; + xzp = NULL; + /* + * Attempt to lock directory; fail if entry doesn't exist. + */ + if ((error = zfs_dirent_lock(&dl, dzp, name, &zp, zflg, + NULL, realnmp))) { + if (realnmp) + kmem_free(realnm.cn_nameptr, realnm.cn_namelen); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + if ((error = zfs_zaccess_delete(dzp, zp, cr, NULL))) { + goto out; + } + + /* + * Need to use rmdir for removing directories. + */ + if (S_ISDIR(zp->z_mode)) { + error = SET_ERROR(EPERM); + goto out; + } + + mutex_enter(&zp->z_lock); + may_delete_now = vnode_iocount(ZTOV(zp)) == 1 && + !(zp->z_is_mapped); + mutex_exit(&zp->z_lock); + + /* + * We may delete the znode now, or we may put it in the unlinked set; + * it depends on whether we're the last link, and on whether there are + * other holds on the inode. So we dmu_tx_hold() the right things to + * allow for either case. + */ + obj = zp->z_id; + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_zap(tx, dzp->z_id, FALSE, name); + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + zfs_sa_upgrade_txholds(tx, zp); + zfs_sa_upgrade_txholds(tx, dzp); + if (may_delete_now) { + toobig = zp->z_size > zp->z_blksz * zfs_delete_blocks; + /* if the file is too big, only hold_free a token amount */ + dmu_tx_hold_free(tx, zp->z_id, 0, + (toobig ? DMU_MAX_ACCESS : DMU_OBJECT_END)); + } + + /* are there any extended attributes? */ + error = sa_lookup(zp->z_sa_hdl, SA_ZPL_XATTR(zfsvfs), + &xattr_obj, sizeof (xattr_obj)); + if (error == 0 && xattr_obj) { + error = zfs_zget(zfsvfs, xattr_obj, &xzp); + ASSERT0(error); + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_TRUE); + dmu_tx_hold_sa(tx, xzp->z_sa_hdl, B_FALSE); + } + + mutex_enter(&zp->z_lock); + if ((acl_obj = zfs_external_acl(zp)) != 0 && may_delete_now) + dmu_tx_hold_free(tx, acl_obj, 0, DMU_OBJECT_END); + mutex_exit(&zp->z_lock); + + /* charge as an update -- would be nice not to charge at all */ + dmu_tx_hold_zap(tx, zfsvfs->z_unlinkedobj, FALSE, NULL); + + /* + * Mark this transaction as typically resulting in a net free of space + */ + dmu_tx_mark_netfree(tx); + + error = dmu_tx_assign(tx, (waited ? TXG_NOTHROTTLE : 0) | TXG_NOWAIT); + if (error) { + zfs_dirent_unlock(dl); + if (error == ERESTART) { + waited = B_TRUE; + dmu_tx_wait(tx); + dmu_tx_abort(tx); + zrele(zp); + if (xzp) + zrele(xzp); + goto top; + } + if (realnmp) + kmem_free(realnm.cn_nameptr, realnm.cn_namelen); + dmu_tx_abort(tx); + zrele(zp); + if (xzp) + zrele(xzp); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * Remove the directory entry. + */ + error = zfs_link_destroy(dl, zp, tx, zflg, &unlinked); + + if (error) { + dmu_tx_commit(tx); + goto out; + } + + if (unlinked) { + /* + * Hold z_lock so that we can make sure that the ACL obj + * hasn't changed. Could have been deleted due to + * zfs_sa_upgrade(). + */ + mutex_enter(&zp->z_lock); + (void) sa_lookup(zp->z_sa_hdl, SA_ZPL_XATTR(zfsvfs), + &xattr_obj_unlinked, sizeof (xattr_obj_unlinked)); + delete_now = may_delete_now && !toobig && + vnode_iocount(ZTOV(zp)) == 1 && + !(zp->z_is_mapped) && xattr_obj == xattr_obj_unlinked && + zfs_external_acl(zp) == acl_obj; + } + + if (delete_now) { + if (xattr_obj_unlinked) { + mutex_enter(&xzp->z_lock); + xzp->z_unlinked = B_TRUE; + links = 0; + error = sa_update(xzp->z_sa_hdl, SA_ZPL_LINKS(zfsvfs), + &links, sizeof (links), tx); + ASSERT3U(error, ==, 0); + mutex_exit(&xzp->z_lock); + zfs_unlinked_add(xzp, tx); + + if (zp->z_is_sa) + error = sa_remove(zp->z_sa_hdl, + SA_ZPL_XATTR(zfsvfs), tx); + else + error = sa_update(zp->z_sa_hdl, + SA_ZPL_XATTR(zfsvfs), &null_xattr, + sizeof (uint64_t), tx); + ASSERT0(error); + } + /* + * Add to the unlinked set because a new reference could be + * taken concurrently resulting in a deferred destruction. + */ + zfs_unlinked_add(zp, tx); + mutex_exit(&zp->z_lock); + } else if (unlinked) { + mutex_exit(&zp->z_lock); + zfs_unlinked_add(zp, tx); + } + + txtype = TX_REMOVE; + if (flags & FIGNORECASE) + txtype |= TX_CI; + zfs_log_remove(zilog, tx, txtype, dzp, name, obj, unlinked); + + dmu_tx_commit(tx); +out: + if (realnmp) + kmem_free(realnm.cn_nameptr, realnm.cn_namelen); + + zfs_dirent_unlock(dl); + + if (delete_now) { + zrele(zp); + } else { + zfs_zrele_async(zp); + } + if (xzp) + zfs_zrele_async(xzp); + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Create a new directory and insert it into dzp using the name + * provided. Return a pointer to the inserted directory. + * + * IN: dzp - znode of directory to add subdir to. + * dirname - name of new directory. + * vap - attributes of new directory. + * cr - credentials of caller. + * flags - case flags. + * vsecp - ACL to be set + * + * OUT: zpp - znode of created directory. + * + * RETURN: 0 if success + * error code if failure + * + * Timestamps: + * dzp - ctime|mtime updated + * zpp - ctime|mtime|atime updated + */ +int +zfs_mkdir(znode_t *dzp, char *dirname, vattr_t *vap, znode_t **zpp, + cred_t *cr, int flags, vsecattr_t *vsecp, zidmap_t *mnt_ns) +{ + znode_t *zp; + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + zilog_t *zilog; + zfs_dirlock_t *dl; + uint64_t txtype; + dmu_tx_t *tx; + int error; + int zf = ZNEW; + uid_t uid; + gid_t gid = crgetgid(cr); + zfs_acl_ids_t acl_ids; + boolean_t fuid_dirtied; + boolean_t waited = B_FALSE; + + ASSERT(S_ISDIR(vap->va_mode)); + + /* + * If we have an ephemeral id, ACL, or XVATTR then + * make sure file system is at proper version + */ + + uid = crgetuid(cr); + if (zfsvfs->z_use_fuids == B_FALSE && + (vsecp || IS_EPHEMERAL(uid) || IS_EPHEMERAL(gid))) + return (SET_ERROR(EINVAL)); + + if (dirname == NULL) + return (SET_ERROR(EINVAL)); + + // If it is a snapshot/name mkdir, just pretend + if (zfsctl_is_node(dzp)) + return (zfsctl_mkdir(dzp, zpp, dirname)); + + if ((error = zfs_enter_verify_zp(zfsvfs, dzp, FTAG)) != 0) + return (error); + zilog = zfsvfs->z_log; + + if (dzp->z_pflags & ZFS_XATTR) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + if (zfsvfs->z_utf8 && u8_validate(dirname, + strlen(dirname), NULL, U8_VALIDATE_ENTIRE, &error) < 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EILSEQ)); + } + if (flags & FIGNORECASE) + zf |= ZCILOOK; + + if (vap->va_mask & ATTR_XVATTR) { + if ((error = secpolicy_xvattr(vap, + crgetuid(cr), cr, vap->va_mode)) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + } + + if ((error = zfs_acl_ids_create(dzp, 0, vap, cr, + vsecp, &acl_ids, NULL)) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + /* + * First make sure the new directory doesn't exist. + * + * Existence is checked first to make sure we don't return + * EACCES instead of EEXIST which can cause some applications + * to fail. + */ +top: + *zpp = NULL; + + if ((error = zfs_dirent_lock(&dl, dzp, dirname, &zp, zf, + NULL, NULL))) { + zfs_acl_ids_free(&acl_ids); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + if ((error = zfs_zaccess(dzp, ACE_ADD_SUBDIRECTORY, 0, B_FALSE, cr, + NULL))) { + zfs_acl_ids_free(&acl_ids); + zfs_dirent_unlock(dl); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + if (zfs_acl_ids_overquota(zfsvfs, &acl_ids, zfs_inherit_projid(dzp))) { + zfs_acl_ids_free(&acl_ids); + zfs_dirent_unlock(dl); + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EDQUOT)); + } + + /* + * Add a new entry to the directory. + */ + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_zap(tx, dzp->z_id, TRUE, dirname); + dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); + fuid_dirtied = zfsvfs->z_fuid_dirty; + if (fuid_dirtied) + zfs_fuid_txhold(zfsvfs, tx); + if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, + acl_ids.z_aclp->z_acl_bytes); + } + + dmu_tx_hold_sa_create(tx, acl_ids.z_aclp->z_acl_bytes + + ZFS_SA_BASE_ATTR_SIZE); + + error = dmu_tx_assign(tx, (waited ? TXG_NOTHROTTLE : 0) | TXG_NOWAIT); + if (error) { + zfs_dirent_unlock(dl); + if (error == ERESTART) { + waited = B_TRUE; + dmu_tx_wait(tx); + dmu_tx_abort(tx); + goto top; + } + zfs_acl_ids_free(&acl_ids); + dmu_tx_abort(tx); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * Create new node. + */ + zfs_mknode(dzp, vap, tx, cr, 0, &zp, &acl_ids); + + /* + * Now put new name in parent dir. + */ + error = zfs_link_create(dl, zp, tx, ZNEW); + if (error != 0) { + zfs_znode_delete(zp, tx); + goto out; + } + + if (fuid_dirtied) + zfs_fuid_sync(zfsvfs, tx); + + *zpp = zp; + + txtype = zfs_log_create_txtype(Z_DIR, vsecp, vap); + if (flags & FIGNORECASE) + txtype |= TX_CI; + zfs_log_create(zilog, tx, txtype, dzp, zp, dirname, vsecp, + acl_ids.z_fuidp, vap); + +out: + zfs_acl_ids_free(&acl_ids); + + dmu_tx_commit(tx); + /* + * Windows - attach the vnode _after_ committing the transaction + */ + zfs_znode_getvnode(zp, dzp, zfsvfs); + + zfs_dirent_unlock(dl); + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); + + if (error != 0) { + zrele(zp); + } else { + } + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Remove a directory subdir entry. If the current working + * directory is the same as the subdir to be removed, the + * remove will fail. + * + * IN: dzp - znode of directory to remove from. + * name - name of directory to be removed. + * cwd - inode of current working directory. + * cr - credentials of caller. + * flags - case flags + * + * RETURN: 0 on success, error code on failure. + * + * Timestamps: + * dzp - ctime|mtime updated + */ +int +zfs_rmdir(znode_t *dzp, char *name, znode_t *cwd, cred_t *cr, + int flags) +{ + znode_t *zp; + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + zilog_t *zilog; + zfs_dirlock_t *dl; + dmu_tx_t *tx; + int error; + int zflg = ZEXISTS; + boolean_t waited = B_FALSE; + + if (name == NULL) + return (SET_ERROR(EINVAL)); + + if ((error = zfs_enter_verify_zp(zfsvfs, dzp, FTAG)) != 0) + return (error); + zilog = zfsvfs->z_log; + + if (flags & FIGNORECASE) + zflg |= ZCILOOK; +top: + zp = NULL; + + /* + * Attempt to lock directory; fail if entry doesn't exist. + */ + if ((error = zfs_dirent_lock(&dl, dzp, name, &zp, zflg, + NULL, NULL))) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + if ((error = zfs_zaccess_delete(dzp, zp, cr, NULL))) { + goto out; + } + + if (ZTOTYPE(zp) != VDIR) { + error = SET_ERROR(ENOTDIR); + goto out; + } + + if (zp == cwd) { + error = SET_ERROR(EINVAL); + goto out; + } + + /* + * Grab a lock on the directory to make sure that no one is + * trying to add (or lookup) entries while we are removing it. + */ + rw_enter(&zp->z_name_lock, RW_WRITER); + + /* + * Grab a lock on the parent pointer to make sure we play well + * with the treewalk and directory rename code. + */ + rw_enter(&zp->z_parent_lock, RW_WRITER); + + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_zap(tx, dzp->z_id, FALSE, name); + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + dmu_tx_hold_zap(tx, zfsvfs->z_unlinkedobj, FALSE, NULL); + zfs_sa_upgrade_txholds(tx, zp); + zfs_sa_upgrade_txholds(tx, dzp); + dmu_tx_mark_netfree(tx); + error = dmu_tx_assign(tx, (waited ? TXG_NOTHROTTLE : 0) | TXG_NOWAIT); + if (error) { + rw_exit(&zp->z_parent_lock); + rw_exit(&zp->z_name_lock); + zfs_dirent_unlock(dl); + if (error == ERESTART) { + waited = B_TRUE; + dmu_tx_wait(tx); + dmu_tx_abort(tx); + zrele(zp); + goto top; + } + dmu_tx_abort(tx); + zrele(zp); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + error = zfs_link_destroy(dl, zp, tx, zflg, NULL); + + if (error == 0) { + uint64_t txtype = TX_RMDIR; + if (flags & FIGNORECASE) + txtype |= TX_CI; + zfs_log_remove(zilog, tx, txtype, dzp, name, ZFS_NO_OBJECT, + B_FALSE); + } + + dmu_tx_commit(tx); + + rw_exit(&zp->z_parent_lock); + rw_exit(&zp->z_name_lock); +out: + zfs_dirent_unlock(dl); + + zrele(zp); + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Read directory entries from the given directory cursor position and emit + * name and position for each entry. + * + * IN: ip - inode of directory to read. + * ctx - directory entry context. + * cr - credentials of caller. + * + * RETURN: 0 if success + * error code if failure + * + * Timestamps: + * ip - atime updated + * + * Note that the low 4 bits of the cookie returned by zap is always zero. + * This allows us to use the low range for "special" directory entries: + * We use 0 for '.', and 1 for '..'. If this is the root of the filesystem, + * we use the offset 2 for the '.zfs' directory. + */ +int +zfs_readdir(vnode_t *vp, emitdir_ptr_t *ctx, cred_t *cr, zfs_dirlist_t *zccb, + int flags) +{ + int error = 0; + (void) cr; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + objset_t *os; + zap_cursor_t zc; + zap_attribute_t zap; + uint64_t parent; + uint8_t prefetch; + uint8_t type; + int skip_this_entry; + int flag_index_specified = flags & SL_INDEX_SPECIFIED ? 1 : 0; + int flag_restart_scan = flags & SL_RESTART_SCAN ? 1 : 0; + int flag_return_single_entry = flags & SL_RETURN_SINGLE_ENTRY ? 1 : 0; + + dprintf("+zfs_readdir: Index %d, Restart %d, Single %d\n", + flag_index_specified, flag_restart_scan, flag_return_single_entry); + + /* If we are listing a ".zfs" directory, call out */ + if (zfsctl_is_node(zp)) + return (zfsctl_readdir(vp, ctx, cr, zccb, flags)); + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + + if ((error = sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent))) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * Check for valid iov_len. + */ + // if (zfs_uio_resid(uio) <= 0) { + if (ctx->bufsize <= 0) { + zfs_exit(zfsvfs, FTAG); + return ((EINVAL)); + } + + /* + * Quit if directory has been removed (posix) + */ + if (zp->z_unlinked != 0) { + zccb->dir_eof = B_TRUE; + zfs_exit(zfsvfs, FTAG); + return (0); + } + + // Make sure the dirlist type is a valid one + // This isn't needed, just a nicer failure until + // all types are handled. + switch (ctx->dirlisttype) { + case FileFullDirectoryInformation: + case FileIdBothDirectoryInformation: + case FileBothDirectoryInformation: + case FileDirectoryInformation: + case FileNamesInformation: + case FileIdFullDirectoryInformation: + case FileObjectIdInformation: + case FileIdExtdDirectoryInformation: + case FileIdExtdBothDirectoryInformation: + break; + default: + dprintf("%s: ** Directory type %d not handled!\n", + __func__, ctx->dirlisttype); + zfs_exit(zfsvfs, FTAG); + return ((EINVAL)); + } + + error = 0; + os = zfsvfs->z_os; + prefetch = zp->z_zn_prefetch; + + /* + * Initialize the iterator cursor. + */ + if (ctx->offset <= 3) { + /* + * Start iteration from the beginning of the directory. + */ + zap_cursor_init(&zc, os, zp->z_id); + } else { + /* + * The offset is a serialized cursor. + */ + zap_cursor_init_serialized(&zc, os, zp->z_id, ctx->offset); + } + + /* + * Get space to change directory entries into fs independent format. + */ + + /* + * Transform to file-system independent format + */ + // zfsvfs->z_show_ctldir = ZFS_SNAPDIR_VISIBLE; + + while (ctx->outcount < ctx->bufsize) { + ino64_t objnum; + + skip_this_entry = 0; + + + /* + * Special case `.', `..', and `.zfs'. + */ + if (ctx->offset == 0) { + (void) strlcpy(zap.za_name, ".", MAXNAMELEN); + zap.za_normalization_conflict = 0; + objnum = zp->z_id; + type = DT_DIR; + } else if (ctx->offset == 1) { + (void) strlcpy(zap.za_name, "..", MAXNAMELEN); + zap.za_normalization_conflict = 0; + objnum = parent; + type = DT_DIR; +#if 1 + } else if (ctx->offset == 2 && zfs_show_ctldir(zp)) { + (void) strlcpy(zap.za_name, ZFS_CTLDIR_NAME, + MAXNAMELEN); + zap.za_normalization_conflict = 0; + objnum = ZFSCTL_INO_ROOT; + type = DT_DIR; +#endif + } else { + + /* + * Grab next entry. + */ + if ((error = zap_cursor_retrieve(&zc, &zap))) { + if (error == ENOENT) + break; + else + goto update; + } + + if (zap.za_integer_length != 8 || + zap.za_num_integers != 1) { + cmn_err(CE_WARN, "zap_readdir: bad directory " + "entry, obj = %lld, offset = %lld\n", + (u_longlong_t)zp->z_id, + (u_longlong_t)ctx->offset); + error = SET_ERROR(ENXIO); + goto update; + } + + objnum = ZFS_DIRENT_OBJ(zap.za_first_integer); + type = ZFS_DIRENT_TYPE(zap.za_first_integer); + + } + + if (!skip_this_entry) { + + // emit + error = zfs_readdir_emitdir(zfsvfs, zap.za_name, ctx, + zccb, objnum); + + if (error == 0) { + ctx->numdirent++; + } else if (error == ENOSPC) { /* stop iterating */ + break; + } else { + /* other error, skip over entry */ + skip_this_entry = 1; + } + } // !skip_this_entry + + // ASSERT(ctx->outcount <= ctx->bufsize); + + /* + * Move to the next entry, fill in the previous offset. + */ + if (ctx->offset > 2 || (ctx->offset == 2 && + !zfs_show_ctldir(zp))) { + zap_cursor_advance(&zc); + ctx->offset = zap_cursor_serialize(&zc); + } else { + ctx->offset += 1; + } + + if (!skip_this_entry && flag_return_single_entry) break; + } // while + + zfs_readdir_complete(ctx); + + zp->z_zn_prefetch = B_FALSE; /* a lookup will re-enable pre-fetching */ + +update: + zap_cursor_fini(&zc); + + ZFS_ACCESSTIME_STAMP(zfsvfs, zp); + + zfs_exit(zfsvfs, FTAG); + + dprintf("-zfs_readdir: num %d\n", ctx->numdirent); + + return (error); +} + +/* + * Get the requested file attributes and place them in the provided + * vattr structure. + * + * IN: vp - vnode of file. + * vap - va_mask identifies requested attributes. + * If ATTR_XVATTR set, then optional attrs are requested + * flags - ATTR_NOACLCHECK (CIFS server context) + * cr - credentials of caller. + * ct - caller context + * + * OUT: vap - attribute values. + * + * RETURN: 0 (always succeeds) + */ +int +zfs_getattr(vnode_t *vp, vattr_t *vap, int flags, cred_t *cr, + caller_context_t *ct) +{ + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + int error = 0; + uint64_t links; + uint64_t mtime[2], ctime[2], crtime[2], rdev; + xvattr_t *xvap = (xvattr_t *)vap; /* vap may be an xvattr_t * */ + xoptattr_t *xoap = NULL; + boolean_t skipaclchk = /* (flags&ATTR_NOACLCHECK)?B_TRUE: */ B_FALSE; + sa_bulk_attr_t bulk[4]; + int count = 0; + + VERIFY3P(zp->z_zfsvfs, ==, vfs_fsprivate(vnode_mount(vp))); + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + + zfs_fuid_map_ids(zp, cr, &vap->va_uid, &vap->va_gid); + + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MTIME(zfsvfs), NULL, &mtime, 16); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), NULL, &ctime, 16); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CRTIME(zfsvfs), NULL, &crtime, 16); + if (vnode_isblk(vp) || vnode_ischr(vp)) + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_RDEV(zfsvfs), NULL, + &rdev, 8); + + if ((error = sa_bulk_lookup(zp->z_sa_hdl, bulk, count)) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * If ACL is trivial don't bother looking for ACE_READ_ATTRIBUTES. + * Also, if we are the owner don't bother, since owner should + * always be allowed to read basic attributes of file. + */ + if (!(zp->z_pflags & ZFS_ACL_TRIVIAL) && + (vap->va_uid != crgetuid(cr))) { + if ((error = zfs_zaccess(zp, ACE_READ_ATTRIBUTES, 0, + skipaclchk, cr, NULL))) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + } + + /* + * Return all attributes. It's cheaper to provide the answer + * than to determine whether we were asked the question. + */ + + mutex_enter(&zp->z_lock); + vap->va_type = IFTOVT(zp->z_mode); + vap->va_mode = zp->z_mode & ~S_IFMT; + vap->va_nodeid = zp->z_id; + if (vnode_isvroot((vp)) && zfs_show_ctldir(zp)) + links = zp->z_links + 1; + else + links = zp->z_links; + vap->va_nlink = MIN(links, LINK_MAX); /* nlink_t limit! */ + vap->va_size = zp->z_size; + if (vnode_isblk(vp) || vnode_ischr(vp)) + vap->va_rdev = zfs_cmpldev(rdev); + + vap->va_flags = 0; /* FreeBSD: Reset chflags(2) flags. */ + + /* + * Add in any requested optional attributes and the create time. + * Also set the corresponding bits in the returned attribute bitmap. + */ + if ((xoap = xva_getxoptattr(xvap)) != NULL && zfsvfs->z_use_fuids) { + if (XVA_ISSET_REQ(xvap, XAT_ARCHIVE)) { + xoap->xoa_archive = + ((zp->z_pflags & ZFS_ARCHIVE) != 0); + XVA_SET_RTN(xvap, XAT_ARCHIVE); + } + + if (XVA_ISSET_REQ(xvap, XAT_READONLY)) { + xoap->xoa_readonly = + ((zp->z_pflags & ZFS_READONLY) != 0); + XVA_SET_RTN(xvap, XAT_READONLY); + } + + if (XVA_ISSET_REQ(xvap, XAT_SYSTEM)) { + xoap->xoa_system = + ((zp->z_pflags & ZFS_SYSTEM) != 0); + XVA_SET_RTN(xvap, XAT_SYSTEM); + } + + if (XVA_ISSET_REQ(xvap, XAT_HIDDEN)) { + xoap->xoa_hidden = + ((zp->z_pflags & ZFS_HIDDEN) != 0); + XVA_SET_RTN(xvap, XAT_HIDDEN); + } + + if (XVA_ISSET_REQ(xvap, XAT_NOUNLINK)) { + xoap->xoa_nounlink = + ((zp->z_pflags & ZFS_NOUNLINK) != 0); + XVA_SET_RTN(xvap, XAT_NOUNLINK); + } + + if (XVA_ISSET_REQ(xvap, XAT_IMMUTABLE)) { + xoap->xoa_immutable = + ((zp->z_pflags & ZFS_IMMUTABLE) != 0); + XVA_SET_RTN(xvap, XAT_IMMUTABLE); + } + + if (XVA_ISSET_REQ(xvap, XAT_APPENDONLY)) { + xoap->xoa_appendonly = + ((zp->z_pflags & ZFS_APPENDONLY) != 0); + XVA_SET_RTN(xvap, XAT_APPENDONLY); + } + + if (XVA_ISSET_REQ(xvap, XAT_NODUMP)) { + xoap->xoa_nodump = + ((zp->z_pflags & ZFS_NODUMP) != 0); + XVA_SET_RTN(xvap, XAT_NODUMP); + } + + if (XVA_ISSET_REQ(xvap, XAT_OPAQUE)) { + xoap->xoa_opaque = + ((zp->z_pflags & ZFS_OPAQUE) != 0); + XVA_SET_RTN(xvap, XAT_OPAQUE); + } + + if (XVA_ISSET_REQ(xvap, XAT_AV_QUARANTINED)) { + xoap->xoa_av_quarantined = + ((zp->z_pflags & ZFS_AV_QUARANTINED) != 0); + XVA_SET_RTN(xvap, XAT_AV_QUARANTINED); + } + + if (XVA_ISSET_REQ(xvap, XAT_AV_MODIFIED)) { + xoap->xoa_av_modified = + ((zp->z_pflags & ZFS_AV_MODIFIED) != 0); + XVA_SET_RTN(xvap, XAT_AV_MODIFIED); + } + + if (XVA_ISSET_REQ(xvap, XAT_AV_SCANSTAMP) && + vnode_isreg(vp)) { + zfs_sa_get_scanstamp(zp, xvap); + } + if (XVA_ISSET_REQ(xvap, XAT_CREATETIME)) { + uint64_t times[2]; + + (void) sa_lookup(zp->z_sa_hdl, SA_ZPL_CRTIME(zfsvfs), + times, sizeof (times)); + ZFS_TIME_DECODE(&xoap->xoa_createtime, times); + XVA_SET_RTN(xvap, XAT_CREATETIME); + } + + if (XVA_ISSET_REQ(xvap, XAT_REPARSE)) { + xoap->xoa_reparse = ((zp->z_pflags & ZFS_REPARSE) != 0); + XVA_SET_RTN(xvap, XAT_REPARSE); + } + if (XVA_ISSET_REQ(xvap, XAT_GEN)) { + xoap->xoa_generation = zp->z_gen; + XVA_SET_RTN(xvap, XAT_GEN); + } + + if (XVA_ISSET_REQ(xvap, XAT_OFFLINE)) { + xoap->xoa_offline = + ((zp->z_pflags & ZFS_OFFLINE) != 0); + XVA_SET_RTN(xvap, XAT_OFFLINE); + } + + if (XVA_ISSET_REQ(xvap, XAT_SPARSE)) { + xoap->xoa_sparse = + ((zp->z_pflags & ZFS_SPARSE) != 0); + XVA_SET_RTN(xvap, XAT_SPARSE); + } + } + + ZFS_TIME_DECODE(&vap->va_atime, zp->z_atime); + ZFS_TIME_DECODE(&vap->va_mtime, mtime); + ZFS_TIME_DECODE(&vap->va_ctime, ctime); + ZFS_TIME_DECODE(&vap->va_crtime, crtime); + + mutex_exit(&zp->z_lock); + + zfs_exit(zfsvfs, FTAG); + return (0); +} + +void kx_qsort(void* array, size_t nm, size_t member_size, + int (*)(const void *, const void *)); + +/* + * Set the file attributes to the values contained in the + * vattr structure. + * + * IN: zp - znode of file to be modified. + * vap - new attribute values. + * If AT_XVATTR set, then optional attrs are being set + * flags - ATTR_UTIME set if non-default time values provided. + * - ATTR_NOACLCHECK (CIFS context only). + * cr - credentials of caller. + * ct - caller context + * + * RETURN: 0 on success, error code on failure. + * + * Timestamps: + * vp - ctime updated, mtime updated if size changed. + */ +int +zfs_setattr(znode_t *zp, vattr_t *vap, int flags, cred_t *cr, zidmap_t *mnt_ns) +{ + vnode_t *vp = ZTOV(zp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + objset_t *os = zfsvfs->z_os; + zilog_t *zilog; + dmu_tx_t *tx; + vattr_t oldva; + xvattr_t tmpxvattr; + uint_t mask = vap->va_mask; + uint_t saved_mask = 0; + uint64_t saved_mode; + int trim_mask = 0; + uint64_t new_mode; + uint64_t new_uid, new_gid; + uint64_t xattr_obj; + uint64_t mtime[2], ctime[2], crtime[2]; + uint64_t projid = ZFS_INVALID_PROJID; + znode_t *attrzp; + int need_policy = FALSE; + int err, err2; + zfs_fuid_info_t *fuidp = NULL; + xvattr_t *xvap = (xvattr_t *)vap; /* vap may be an xvattr_t * */ + xoptattr_t *xoap; + zfs_acl_t *aclp; + boolean_t skipaclchk = (flags & ATTR_NOACLCHECK) ? B_TRUE : B_FALSE; + boolean_t fuid_dirtied = B_FALSE; + sa_bulk_attr_t bulk[7], xattr_bulk[7]; + int count = 0, xattr_count = 0; + int error = 0; + + if (mask == 0) + return (0); + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + + zilog = zfsvfs->z_log; + + /* + * Make sure that if we have ephemeral uid/gid or xvattr specified + * that file system is at proper version level + */ + + if (zfsvfs->z_use_fuids == B_FALSE && + (((mask & ATTR_UID) && IS_EPHEMERAL(vap->va_uid)) || + ((mask & ATTR_GID) && IS_EPHEMERAL(vap->va_gid)) || + (mask & ATTR_XVATTR))) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + if (mask & ATTR_SIZE && vnode_vtype(vp) == VDIR) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EISDIR)); + } + + if (mask & ATTR_SIZE && vnode_vtype(vp) != VREG && + vnode_vtype(vp) != VFIFO) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + /* + * If this is an xvattr_t, then get a pointer to the structure of + * optional attributes. If this is NULL, then we have a vattr_t. + */ + xoap = xva_getxoptattr(xvap); + + xva_init(&tmpxvattr); + + /* + * Immutable files can only alter immutable bit and atime + */ + if ((zp->z_pflags & ZFS_IMMUTABLE) && + ((mask & (ATTR_SIZE|ATTR_UID|ATTR_GID|ATTR_MTIME|ATTR_MODE)) || + ((mask & ATTR_XVATTR) && XVA_ISSET_REQ(xvap, XAT_CREATETIME)))) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EPERM)); + } + + /* + * Note: ZFS_READONLY is handled in zfs_zaccess_common. + */ + + /* + * Verify timestamps doesn't overflow 32 bits. + * ZFS can handle large timestamps, but 32bit syscalls can't + * handle times greater than 2039. This check should be removed + * once large timestamps are fully supported. + */ +#if 0 + if (mask & (ATTR_ATIME | ATTR_MTIME)) { + if (((mask & ATTR_ATIME) && + TIMESPEC_OVERFLOW(&vap->va_atime)) || + ((mask & ATTR_MTIME) && + TIMESPEC_OVERFLOW(&vap->va_mtime))) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EOVERFLOW)); + } + } +#endif + + if (xoap != NULL && (mask & ATTR_XVATTR)) { + if (XVA_ISSET_REQ(xvap, XAT_PROJID)) { + if (!dmu_objset_projectquota_enabled(os) || + (!S_ISREG(zp->z_mode) && !S_ISDIR(zp->z_mode))) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EOPNOTSUPP)); + } + + projid = xoap->xoa_projid; + if (unlikely(projid == ZFS_INVALID_PROJID)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + if (projid == zp->z_projid && zp->z_pflags & ZFS_PROJID) + projid = ZFS_INVALID_PROJID; + else + need_policy = TRUE; + } + + if (XVA_ISSET_REQ(xvap, XAT_PROJINHERIT) && + (xoap->xoa_projinherit != + ((zp->z_pflags & ZFS_PROJINHERIT) != 0)) && + (!dmu_objset_projectquota_enabled(os) || + (!S_ISREG(zp->z_mode) && !S_ISDIR(zp->z_mode)))) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EOPNOTSUPP)); + } + } + + attrzp = NULL; + aclp = NULL; + + if (zfs_is_readonly(zfsvfs)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EROFS)); + } + + /* + * First validate permissions + */ + + if (mask & ATTR_SIZE) { + /* + * XXX - Note, we are not providing any open + * mode flags here (like FNDELAY), so we may + * block if there are locks present... this + * should be addressed in openat(). + */ + /* XXX - would it be OK to generate a log record here? */ + err = zfs_freesp(zp, vap->va_size, 0, 0, FALSE); + if (err) { + zfs_exit(zfsvfs, FTAG); + return (err); + } + } + + if (mask & (ATTR_ATIME|ATTR_MTIME) || + ((mask & ATTR_XVATTR) && (XVA_ISSET_REQ(xvap, XAT_HIDDEN) || + XVA_ISSET_REQ(xvap, XAT_READONLY) || + XVA_ISSET_REQ(xvap, XAT_ARCHIVE) || + XVA_ISSET_REQ(xvap, XAT_OFFLINE) || + XVA_ISSET_REQ(xvap, XAT_SPARSE) || + XVA_ISSET_REQ(xvap, XAT_CREATETIME) || + XVA_ISSET_REQ(xvap, XAT_SYSTEM)))) { + need_policy = zfs_zaccess(zp, ACE_WRITE_ATTRIBUTES, 0, + skipaclchk, cr, NULL); + } + + if (mask & (ATTR_UID|ATTR_GID)) { + int idmask = (mask & (ATTR_UID|ATTR_GID)); + int take_owner; + int take_group; + + /* + * NOTE: even if a new mode is being set, + * we may clear S_ISUID/S_ISGID bits. + */ + + if (!(mask & ATTR_MODE)) + vap->va_mode = zp->z_mode; + + /* + * Take ownership or chgrp to group we are a member of + */ + + take_owner = (mask & ATTR_UID) && (vap->va_uid == crgetuid(cr)); + take_group = (mask & ATTR_GID) && + zfs_groupmember(zfsvfs, vap->va_gid, cr); + + /* + * If both ATTR_UID and ATTR_GID are set then take_owner and + * take_group must both be set in order to allow taking + * ownership. + * + * Otherwise, send the check through secpolicy_vnode_setattr() + * + */ + + if (((idmask == (ATTR_UID|ATTR_GID)) && take_owner && + take_group) || + ((idmask == ATTR_UID) && take_owner) || + ((idmask == ATTR_GID) && take_group)) { + if (zfs_zaccess(zp, ACE_WRITE_OWNER, 0, + skipaclchk, cr, NULL) == 0) { + /* + * Remove setuid/setgid for non-privileged users + */ + secpolicy_setid_clear(vap, cr); + trim_mask = (mask & (ATTR_UID|ATTR_GID)); + } else { + need_policy = TRUE; + } + } else { + need_policy = TRUE; + } + } + + oldva.va_mode = zp->z_mode; + zfs_fuid_map_ids(zp, cr, &oldva.va_uid, &oldva.va_gid); + if (mask & ATTR_XVATTR) { + /* + * Update xvattr mask to include only those attributes + * that are actually changing. + * + * the bits will be restored prior to actually setting + * the attributes so the caller thinks they were set. + */ + if (XVA_ISSET_REQ(xvap, XAT_APPENDONLY)) { + if (xoap->xoa_appendonly != + ((zp->z_pflags & ZFS_APPENDONLY) != 0)) { + need_policy = TRUE; + } else { + XVA_CLR_REQ(xvap, XAT_APPENDONLY); + XVA_SET_REQ(&tmpxvattr, XAT_APPENDONLY); + } + } + + if (XVA_ISSET_REQ(xvap, XAT_PROJINHERIT)) { + if (xoap->xoa_projinherit != + ((zp->z_pflags & ZFS_PROJINHERIT) != 0)) { + need_policy = TRUE; + } else { + XVA_CLR_REQ(xvap, XAT_PROJINHERIT); + XVA_SET_REQ(&tmpxvattr, XAT_PROJINHERIT); + } + } + + if (XVA_ISSET_REQ(xvap, XAT_NOUNLINK)) { + if (xoap->xoa_nounlink != + ((zp->z_pflags & ZFS_NOUNLINK) != 0)) { + need_policy = TRUE; + } else { + XVA_CLR_REQ(xvap, XAT_NOUNLINK); + XVA_SET_REQ(&tmpxvattr, XAT_NOUNLINK); + } + } + + if (XVA_ISSET_REQ(xvap, XAT_IMMUTABLE)) { + if (xoap->xoa_immutable != + ((zp->z_pflags & ZFS_IMMUTABLE) != 0)) { + need_policy = TRUE; + } else { + XVA_CLR_REQ(xvap, XAT_IMMUTABLE); + XVA_SET_REQ(&tmpxvattr, XAT_IMMUTABLE); + } + } + + if (XVA_ISSET_REQ(xvap, XAT_NODUMP)) { + if (xoap->xoa_nodump != + ((zp->z_pflags & ZFS_NODUMP) != 0)) { + need_policy = TRUE; + } else { + XVA_CLR_REQ(xvap, XAT_NODUMP); + XVA_SET_REQ(&tmpxvattr, XAT_NODUMP); + } + } + + if (XVA_ISSET_REQ(xvap, XAT_AV_MODIFIED)) { + if (xoap->xoa_av_modified != + ((zp->z_pflags & ZFS_AV_MODIFIED) != 0)) { + need_policy = TRUE; + } else { + XVA_CLR_REQ(xvap, XAT_AV_MODIFIED); + XVA_SET_REQ(&tmpxvattr, XAT_AV_MODIFIED); + } + } + + if (XVA_ISSET_REQ(xvap, XAT_AV_QUARANTINED)) { + if ((vnode_vtype(vp) != VREG && + xoap->xoa_av_quarantined) || + xoap->xoa_av_quarantined != + ((zp->z_pflags & ZFS_AV_QUARANTINED) != 0)) { + need_policy = TRUE; + } else { + XVA_CLR_REQ(xvap, XAT_AV_QUARANTINED); + XVA_SET_REQ(&tmpxvattr, XAT_AV_QUARANTINED); + } + } + + if (XVA_ISSET_REQ(xvap, XAT_REPARSE)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EPERM)); + } + + if (need_policy == FALSE && + (XVA_ISSET_REQ(xvap, XAT_AV_SCANSTAMP) || + XVA_ISSET_REQ(xvap, XAT_OPAQUE))) { + need_policy = TRUE; + } + } + + if (mask & ATTR_MODE) { + if (zfs_zaccess(zp, ACE_WRITE_ACL, 0, skipaclchk, cr, + NULL) == 0) { + err = secpolicy_setid_setsticky_clear(vp, vap, + &oldva, cr); + if (err) { + zfs_exit(zfsvfs, FTAG); + return (err); + } + trim_mask |= ATTR_MODE; + } else { + need_policy = TRUE; + } + } + + if (need_policy) { + /* + * If trim_mask is set then take ownership + * has been granted or write_acl is present and user + * has the ability to modify mode. In that case remove + * UID|GID and or MODE from mask so that + * secpolicy_vnode_setattr() doesn't revoke it. + */ + + if (trim_mask) { + saved_mask = vap->va_mask; + vap->va_mask &= ~trim_mask; + if (trim_mask & ATTR_MODE) { + /* + * Save the mode, as secpolicy_vnode_setattr() + * will overwrite it with ova.va_mode. + */ + saved_mode = vap->va_mode; + } + } + err = secpolicy_vnode_setattr(cr, vp, vap, &oldva, flags, + zfs_zaccess_unix, zp); + if (err) { + zfs_exit(zfsvfs, FTAG); + return (err); + } + + if (trim_mask) { + vap->va_mask |= saved_mask; + if (trim_mask & ATTR_MODE) { + /* + * Recover the mode after + * secpolicy_vnode_setattr(). + */ + vap->va_mode = saved_mode; + } + } + } + + /* + * secpolicy_vnode_setattr, or take ownership may have + * changed va_mask + */ + mask = vap->va_mask; + + if ((mask & (ATTR_UID | ATTR_GID)) || projid != ZFS_INVALID_PROJID) { + err = sa_lookup(zp->z_sa_hdl, SA_ZPL_XATTR(zfsvfs), + &xattr_obj, sizeof (xattr_obj)); + + if (err == 0 && xattr_obj) { + err = zfs_zget(zp->z_zfsvfs, xattr_obj, &attrzp); + if (err) + goto out2; + } + if (mask & ATTR_UID) { + new_uid = zfs_fuid_create(zfsvfs, + (uint64_t)vap->va_uid, cr, ZFS_OWNER, &fuidp); + if (new_uid != zp->z_uid && + zfs_id_overquota(zfsvfs, DMU_USERUSED_OBJECT, + new_uid)) { + if (attrzp) + zrele(attrzp); + err = SET_ERROR(EDQUOT); + goto out2; + } + } + + if (mask & ATTR_GID) { + new_gid = zfs_fuid_create(zfsvfs, (uint64_t)vap->va_gid, + cr, ZFS_GROUP, &fuidp); + if (new_gid != zp->z_gid && + zfs_id_overquota(zfsvfs, DMU_GROUPUSED_OBJECT, + new_gid)) { + if (attrzp) + zrele(attrzp); + err = SET_ERROR(EDQUOT); + goto out2; + } + } + + if (projid != ZFS_INVALID_PROJID && + zfs_id_overquota(zfsvfs, DMU_PROJECTUSED_OBJECT, projid)) { + if (attrzp) + zrele(attrzp); + err = SET_ERROR(EDQUOT); + goto out2; + } + } + tx = dmu_tx_create(os); + + if (mask & ATTR_MODE) { + uint64_t pmode = zp->z_mode; + uint64_t acl_obj; + new_mode = (pmode & S_IFMT) | (vap->va_mode & ~S_IFMT); + + if (zp->z_zfsvfs->z_acl_mode == ZFS_ACL_RESTRICTED && + !(zp->z_pflags & ZFS_ACL_TRIVIAL)) { + err = SET_ERROR(EPERM); + goto out; + } + + if ((err = zfs_acl_chmod_setattr(zp, &aclp, new_mode))) + goto out; + + if (!zp->z_is_sa && ((acl_obj = zfs_external_acl(zp)) != 0)) { + /* + * Are we upgrading ACL from old V0 format + * to V1 format? + */ + if (zfsvfs->z_version >= ZPL_VERSION_FUID && + zfs_znode_acl_version(zp) == + ZFS_ACL_VERSION_INITIAL) { + dmu_tx_hold_free(tx, acl_obj, 0, + DMU_OBJECT_END); + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, + 0, aclp->z_acl_bytes); + } else { + dmu_tx_hold_write(tx, acl_obj, 0, + aclp->z_acl_bytes); + } + } else if (!zp->z_is_sa && aclp->z_acl_bytes > ZFS_ACE_SPACE) { + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, + 0, aclp->z_acl_bytes); + } + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_TRUE); + } else { + if (((mask & ATTR_XVATTR) && + XVA_ISSET_REQ(xvap, XAT_AV_SCANSTAMP)) || + (projid != ZFS_INVALID_PROJID && + !(zp->z_pflags & ZFS_PROJID))) + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_TRUE); + else + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + } + + if (attrzp) { + dmu_tx_hold_sa(tx, attrzp->z_sa_hdl, B_FALSE); + } + + fuid_dirtied = zfsvfs->z_fuid_dirty; + if (fuid_dirtied) + zfs_fuid_txhold(zfsvfs, tx); + + zfs_sa_upgrade_txholds(tx, zp); + + err = dmu_tx_assign(tx, TXG_WAIT); + if (err) + goto out; + + count = 0; + /* + * Set each attribute requested. + * We group settings according to the locks they need to acquire. + * + * Note: you cannot set ctime directly, although it will be + * updated as a side-effect of calling this function. + */ + + if (projid != ZFS_INVALID_PROJID && !(zp->z_pflags & ZFS_PROJID)) { + /* + * For the existed object that is upgraded from old system, + * its on-disk layout has no slot for the project ID attribute. + * But quota accounting logic needs to access related slots by + * offset directly. So we need to adjust old objects' layout + * to make the project ID to some unified and fixed offset. + */ + if (attrzp) + err = sa_add_projid(attrzp->z_sa_hdl, tx, projid); + if (err == 0) + err = sa_add_projid(zp->z_sa_hdl, tx, projid); + + if (unlikely(err == EEXIST)) + err = 0; + else if (err != 0) + goto out; + else + projid = ZFS_INVALID_PROJID; + } + + if (mask & (ATTR_UID|ATTR_GID|ATTR_MODE)) + mutex_enter(&zp->z_acl_lock); + mutex_enter(&zp->z_lock); + + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), NULL, + &zp->z_pflags, sizeof (zp->z_pflags)); + + if (attrzp) { + if (mask & (ATTR_UID|ATTR_GID|ATTR_MODE)) + mutex_enter(&attrzp->z_acl_lock); + SA_ADD_BULK_ATTR(xattr_bulk, xattr_count, + SA_ZPL_FLAGS(zfsvfs), NULL, &attrzp->z_pflags, + sizeof (attrzp->z_pflags)); + if (projid != ZFS_INVALID_PROJID) { + attrzp->z_projid = projid; + SA_ADD_BULK_ATTR(xattr_bulk, xattr_count, + SA_ZPL_PROJID(zfsvfs), NULL, &attrzp->z_projid, + sizeof (attrzp->z_projid)); + } + } + + if (mask & (ATTR_UID|ATTR_GID)) { + + if (mask & ATTR_UID) { + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_UID(zfsvfs), NULL, + &new_uid, sizeof (new_uid)); + zp->z_uid = new_uid; + if (attrzp) { + SA_ADD_BULK_ATTR(xattr_bulk, xattr_count, + SA_ZPL_UID(zfsvfs), NULL, &new_uid, + sizeof (new_uid)); + attrzp->z_uid = new_uid; + } + } + + if (mask & ATTR_GID) { + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_GID(zfsvfs), + NULL, &new_gid, sizeof (new_gid)); + zp->z_gid = new_gid; + if (attrzp) { + SA_ADD_BULK_ATTR(xattr_bulk, xattr_count, + SA_ZPL_GID(zfsvfs), NULL, &new_gid, + sizeof (new_gid)); + attrzp->z_gid = new_gid; + } + } + if (!(mask & ATTR_MODE)) { + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MODE(zfsvfs), + NULL, &new_mode, sizeof (new_mode)); + new_mode = zp->z_mode; + } + err = zfs_acl_chown_setattr(zp); + ASSERT(err == 0); + if (attrzp) { + err = zfs_acl_chown_setattr(attrzp); + ASSERT(err == 0); + } + } + + if (mask & ATTR_MODE) { + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MODE(zfsvfs), NULL, + &new_mode, sizeof (new_mode)); + zp->z_mode = new_mode; + ASSERT3U((uintptr_t)aclp, !=, 0); + err = zfs_aclset_common(zp, aclp, cr, tx); + ASSERT0(err); + if (zp->z_acl_cached) + zfs_acl_free(zp->z_acl_cached); + zp->z_acl_cached = aclp; + aclp = NULL; + } + + + if (mask & ATTR_ATIME) { + ZFS_TIME_ENCODE(&vap->va_atime, zp->z_atime); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_ATIME(zfsvfs), NULL, + &zp->z_atime, sizeof (zp->z_atime)); + } + + if (mask & ATTR_MTIME) { + ZFS_TIME_ENCODE(&vap->va_mtime, mtime); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MTIME(zfsvfs), NULL, + mtime, sizeof (mtime)); + } + + if (mask & ATTR_CRTIME) { + ZFS_TIME_ENCODE(&vap->va_crtime, crtime); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CRTIME(zfsvfs), NULL, + crtime, sizeof (crtime)); + } + + if (projid != ZFS_INVALID_PROJID) { + zp->z_projid = projid; + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_PROJID(zfsvfs), NULL, &zp->z_projid, + sizeof (zp->z_projid)); + } + + /* XXX - shouldn't this be done *before* the ATIME/MTIME checks? */ + if (mask & ATTR_SIZE && !(mask & ATTR_MTIME)) { + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MTIME(zfsvfs), + NULL, mtime, sizeof (mtime)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), NULL, + &ctime, sizeof (ctime)); + zfs_tstamp_update_setup(zp, CONTENT_MODIFIED, mtime, ctime); + } else if (mask != 0) { + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), NULL, + &ctime, sizeof (ctime)); + zfs_tstamp_update_setup(zp, STATE_CHANGED, mtime, ctime); + if (attrzp) { + SA_ADD_BULK_ATTR(xattr_bulk, xattr_count, + SA_ZPL_CTIME(zfsvfs), NULL, + &ctime, sizeof (ctime)); + zfs_tstamp_update_setup(attrzp, STATE_CHANGED, + mtime, ctime); + } + } + + /* + * Do this after setting timestamps to prevent timestamp + * update from toggling bit + */ + + if (xoap && (mask & ATTR_XVATTR)) { + + if (XVA_ISSET_REQ(xvap, XAT_CREATETIME)) + xoap->xoa_createtime = vap->va_create_time; + /* + * restore trimmed off masks + * so that return masks can be set for caller. + */ + + if (XVA_ISSET_REQ(&tmpxvattr, XAT_APPENDONLY)) { + XVA_SET_REQ(xvap, XAT_APPENDONLY); + } + if (XVA_ISSET_REQ(&tmpxvattr, XAT_NOUNLINK)) { + XVA_SET_REQ(xvap, XAT_NOUNLINK); + } + if (XVA_ISSET_REQ(&tmpxvattr, XAT_IMMUTABLE)) { + XVA_SET_REQ(xvap, XAT_IMMUTABLE); + } + if (XVA_ISSET_REQ(&tmpxvattr, XAT_NODUMP)) { + XVA_SET_REQ(xvap, XAT_NODUMP); + } + if (XVA_ISSET_REQ(&tmpxvattr, XAT_AV_MODIFIED)) { + XVA_SET_REQ(xvap, XAT_AV_MODIFIED); + } + if (XVA_ISSET_REQ(&tmpxvattr, XAT_AV_QUARANTINED)) { + XVA_SET_REQ(xvap, XAT_AV_QUARANTINED); + } + if (XVA_ISSET_REQ(&tmpxvattr, XAT_PROJINHERIT)) { + XVA_SET_REQ(xvap, XAT_PROJINHERIT); + } + + if (XVA_ISSET_REQ(xvap, XAT_AV_SCANSTAMP)) + ASSERT(vp->v_type == VREG); + + zfs_xvattr_set(zp, xvap, tx); + } + + if (fuid_dirtied) + zfs_fuid_sync(zfsvfs, tx); + + if (mask != 0) + zfs_log_setattr(zilog, tx, TX_SETATTR, zp, vap, mask, fuidp); + + mutex_exit(&zp->z_lock); + if (mask & (ATTR_UID|ATTR_GID|ATTR_MODE)) + mutex_exit(&zp->z_acl_lock); + + if (attrzp) { + if (mask & (ATTR_UID|ATTR_GID|ATTR_MODE)) + mutex_exit(&attrzp->z_acl_lock); + } +out: + if (err == 0 && attrzp) { + err2 = sa_bulk_update(attrzp->z_sa_hdl, xattr_bulk, + xattr_count, tx); + ASSERT(err2 == 0); + } + + if (attrzp) + zrele(attrzp); + + if (aclp) + zfs_acl_free(aclp); + + if (fuidp) { + zfs_fuid_info_free(fuidp); + fuidp = NULL; + } + + if (err) { + dmu_tx_abort(tx); + } else { + err2 = sa_bulk_update(zp->z_sa_hdl, bulk, count, tx); + dmu_tx_commit(tx); + } + +out2: + if (os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); + + zfs_exit(zfsvfs, FTAG); + return (err); +} + +typedef struct zfs_zlock { + krwlock_t *zl_rwlock; /* lock we acquired */ + znode_t *zl_znode; /* znode we held */ + struct zfs_zlock *zl_next; /* next in list */ +} zfs_zlock_t; + +/* + * Drop locks and release vnodes that were held by zfs_rename_lock(). + */ +static void +zfs_rename_unlock(zfs_zlock_t **zlpp) +{ + zfs_zlock_t *zl; + + while ((zl = *zlpp) != NULL) { + if (zl->zl_znode != NULL) + zfs_zrele_async(zl->zl_znode); + rw_exit(zl->zl_rwlock); + *zlpp = zl->zl_next; + kmem_free(zl, sizeof (*zl)); + } +} + +/* + * Search back through the directory tree, using the ".." entries. + * Lock each directory in the chain to prevent concurrent renames. + * Fail any attempt to move a directory into one of its own descendants. + * XXX - z_parent_lock can overlap with map or grow locks + */ +static int +zfs_rename_lock(znode_t *szp, znode_t *tdzp, znode_t *sdzp, zfs_zlock_t **zlpp) +{ + zfs_zlock_t *zl; + znode_t *zp = tdzp; + uint64_t rootid = ZTOZSB(zp)->z_root; + uint64_t oidp = zp->z_id; + krwlock_t *rwlp = &szp->z_parent_lock; + krw_t rw = RW_WRITER; + + /* + * First pass write-locks szp and compares to zp->z_id. + * Later passes read-lock zp and compare to zp->z_parent. + */ + do { + if (!rw_tryenter(rwlp, rw)) { + /* + * Another thread is renaming in this path. + * Note that if we are a WRITER, we don't have any + * parent_locks held yet. + */ + if (rw == RW_READER && zp->z_id > szp->z_id) { + /* + * Drop our locks and restart + */ + zfs_rename_unlock(&zl); + *zlpp = NULL; + zp = tdzp; + oidp = zp->z_id; + rwlp = &szp->z_parent_lock; + rw = RW_WRITER; + continue; + } else { + /* + * Wait for other thread to drop its locks + */ + rw_enter(rwlp, rw); + } + } + + zl = kmem_alloc(sizeof (*zl), KM_SLEEP); + zl->zl_rwlock = rwlp; + zl->zl_znode = NULL; + zl->zl_next = *zlpp; + *zlpp = zl; + + if (oidp == szp->z_id) /* We're a descendant of szp */ + return (SET_ERROR(EINVAL)); + + if (oidp == rootid) /* We've hit the top */ + return (0); + + if (rw == RW_READER) { /* i.e. not the first pass */ + int error = zfs_zget(ZTOZSB(zp), oidp, &zp); + if (error) + return (error); + zl->zl_znode = zp; + } + (void) sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(ZTOZSB(zp)), + &oidp, sizeof (oidp)); + rwlp = &zp->z_parent_lock; + rw = RW_READER; + + } while (zp->z_id != sdzp->z_id); + + return (0); +} + +/* + * Move an entry from the provided source directory to the target + * directory. Change the entry name as indicated. + * + * IN: sdzp - Source directory containing the "old entry". + * snm - Old entry name. + * tdzp - Target directory to contain the "new entry". + * tnm - New entry name. + * cr - credentials of caller. + * flags - case flags + * + * RETURN: 0 on success, error code on failure. + * + * Timestamps: + * sdzp,tdzp - ctime|mtime updated + */ +int +zfs_rename(znode_t *sdzp, char *snm, znode_t *tdzp, char *tnm, + cred_t *cr, int flags, uint64_t rflags, vattr_t *wo_vap, zidmap_t *mnt_ns) +{ + znode_t *szp, *tzp; + zfsvfs_t *zfsvfs = ZTOZSB(sdzp); + zilog_t *zilog; + zfs_dirlock_t *sdl, *tdl; + dmu_tx_t *tx; + zfs_zlock_t *zl; + int cmp, serr, terr; + int error = 0; + int zflg = 0; + boolean_t waited = B_FALSE; + + if (rflags != 0 || wo_vap != NULL) + return (SET_ERROR(EINVAL)); + + if (snm == NULL || tnm == NULL) + return (SET_ERROR(EINVAL)); + + if ((error = zfs_enter_verify_zp(zfsvfs, sdzp, FTAG)) != 0) + return (error); + zilog = zfsvfs->z_log; + + if ((error = zfs_verify_zp(tdzp)) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * We check i_sb because snapshots and the ctldir must have different + * super blocks. + */ + // Can't we use zp->z_zfsvfs in place of zp->vp->v_vfs ? + if (VTOM(ZTOV(tdzp)) != VTOM(ZTOV(sdzp)) || + zfsctl_is_node(tdzp)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EXDEV)); + } + + if (zfsvfs->z_utf8 && u8_validate(tnm, + strlen(tnm), NULL, U8_VALIDATE_ENTIRE, &error) < 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EILSEQ)); + } + + if (flags & FIGNORECASE) + zflg |= ZCILOOK; + +top: + szp = NULL; + tzp = NULL; + zl = NULL; + + /* + * This is to prevent the creation of links into attribute space + * by renaming a linked file into/outof an attribute directory. + * See the comment in zfs_link() for why this is considered bad. + */ + if ((tdzp->z_pflags & ZFS_XATTR) != (sdzp->z_pflags & ZFS_XATTR)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + /* + * Lock source and target directory entries. To prevent deadlock, + * a lock ordering must be defined. We lock the directory with + * the smallest object id first, or if it's a tie, the one with + * the lexically first name. + */ + if (sdzp->z_id < tdzp->z_id) { + cmp = -1; + } else if (sdzp->z_id > tdzp->z_id) { + cmp = 1; + } else { + /* + * First compare the two name arguments without + * considering any case folding. + */ + int nofold = (zfsvfs->z_norm & ~U8_TEXTPREP_TOUPPER); + + cmp = u8_strcmp(snm, tnm, 0, nofold, U8_UNICODE_LATEST, &error); + ASSERT(error == 0 || !zfsvfs->z_utf8); + if (cmp == 0) { + /* + * POSIX: "If the old argument and the new argument + * both refer to links to the same existing file, + * the rename() function shall return successfully + * and perform no other action." + */ + zfs_exit(zfsvfs, FTAG); + return (0); + } + /* + * If the file system is case-folding, then we may + * have some more checking to do. A case-folding file + * system is either supporting mixed case sensitivity + * access or is completely case-insensitive. Note + * that the file system is always case preserving. + * + * In mixed sensitivity mode case sensitive behavior + * is the default. FIGNORECASE must be used to + * explicitly request case insensitive behavior. + * + * If the source and target names provided differ only + * by case (e.g., a request to rename 'tim' to 'Tim'), + * we will treat this as a special case in the + * case-insensitive mode: as long as the source name + * is an exact match, we will allow this to proceed as + * a name-change request. + */ + if ((zfsvfs->z_case == ZFS_CASE_INSENSITIVE || + (zfsvfs->z_case == ZFS_CASE_MIXED && + flags & FIGNORECASE)) && + u8_strcmp(snm, tnm, 0, zfsvfs->z_norm, U8_UNICODE_LATEST, + &error) == 0) { + /* + * case preserving rename request, require exact + * name matches + */ + zflg |= ZCIEXACT; + zflg &= ~ZCILOOK; + } + } + + /* + * If the source and destination directories are the same, we should + * grab the z_name_lock of that directory only once. + */ + if (sdzp == tdzp) { + zflg |= ZHAVELOCK; + rw_enter(&sdzp->z_name_lock, RW_READER); + } + + if (cmp < 0) { + serr = zfs_dirent_lock(&sdl, sdzp, snm, &szp, + ZEXISTS | zflg, NULL, NULL); + terr = zfs_dirent_lock(&tdl, + tdzp, tnm, &tzp, ZRENAMING | zflg, NULL, NULL); + } else { + terr = zfs_dirent_lock(&tdl, + tdzp, tnm, &tzp, zflg, NULL, NULL); + serr = zfs_dirent_lock(&sdl, + sdzp, snm, &szp, ZEXISTS | ZRENAMING | zflg, + NULL, NULL); + } + + if (serr) { + /* + * Source entry invalid or not there. + */ + if (!terr) { + zfs_dirent_unlock(tdl); + if (tzp) + zrele(tzp); + } + + if (sdzp == tdzp) + rw_exit(&sdzp->z_name_lock); + + if (strcmp(snm, ".") == 0 || strcmp(snm, "..") == 0) + serr = EINVAL; + zfs_exit(zfsvfs, FTAG); + return (serr); + } + if (terr) { + zfs_dirent_unlock(sdl); + zrele(szp); + + if (sdzp == tdzp) + rw_exit(&sdzp->z_name_lock); + + if (strcmp(tnm, "..") == 0) + terr = EINVAL; + zfs_exit(zfsvfs, FTAG); + return (terr); + } + + /* + * If we are using project inheritance, means if the directory has + * ZFS_PROJINHERIT set, then its descendant directories will inherit + * not only the project ID, but also the ZFS_PROJINHERIT flag. Under + * such case, we only allow renames into our tree when the project + * IDs are the same. + */ + if (tdzp->z_pflags & ZFS_PROJINHERIT && + tdzp->z_projid != szp->z_projid) { + error = SET_ERROR(EXDEV); + goto out; + } + + /* + * Must have write access at the source to remove the old entry + * and write access at the target to create the new entry. + * Note that if target and source are the same, this can be + * done in a single check. + */ + + if ((error = zfs_zaccess_rename(sdzp, szp, tdzp, tzp, cr, NULL))) + goto out; + + if (S_ISDIR(szp->z_mode)) { + /* + * Check to make sure rename is valid. + * Can't do a move like this: /usr/a/b to /usr/a/b/c/d + */ + if ((error = zfs_rename_lock(szp, tdzp, sdzp, &zl))) + goto out; + } + + /* + * Does target exist? + */ + if (tzp) { + /* + * Source and target must be the same type. + */ + if (S_ISDIR(szp->z_mode)) { + if (!S_ISDIR(tzp->z_mode)) { + error = SET_ERROR(ENOTDIR); + goto out; + } + } else { + if (S_ISDIR(tzp->z_mode)) { + error = SET_ERROR(EISDIR); + goto out; + } + } + /* + * POSIX dictates that when the source and target + * entries refer to the same file object, rename + * must do nothing and exit without error. + */ + if (szp->z_id == tzp->z_id) { + error = 0; + goto out; + } + +#if defined(MAC_OS_X_VERSION_10_12) && \ + (MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_X_VERSION_10_12) + /* If renamex(VFS_RENAME_EXCL) is used, error out */ + if (flags & VFS_RENAME_EXCL) { + error = EEXIST; + goto out; + } +#endif + + } + + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_sa(tx, szp->z_sa_hdl, B_FALSE); + dmu_tx_hold_sa(tx, sdzp->z_sa_hdl, B_FALSE); + dmu_tx_hold_zap(tx, sdzp->z_id, FALSE, snm); + dmu_tx_hold_zap(tx, tdzp->z_id, TRUE, tnm); + if (sdzp != tdzp) { + dmu_tx_hold_sa(tx, tdzp->z_sa_hdl, B_FALSE); + zfs_sa_upgrade_txholds(tx, tdzp); + } + if (tzp) { + dmu_tx_hold_sa(tx, tzp->z_sa_hdl, B_FALSE); + zfs_sa_upgrade_txholds(tx, tzp); + } + + zfs_sa_upgrade_txholds(tx, szp); + dmu_tx_hold_zap(tx, zfsvfs->z_unlinkedobj, FALSE, NULL); + error = dmu_tx_assign(tx, (waited ? TXG_NOTHROTTLE : 0) | TXG_NOWAIT); + if (error) { + if (zl != NULL) + zfs_rename_unlock(&zl); + zfs_dirent_unlock(sdl); + zfs_dirent_unlock(tdl); + + if (sdzp == tdzp) + rw_exit(&sdzp->z_name_lock); + + if (error == ERESTART) { + waited = B_TRUE; + dmu_tx_wait(tx); + dmu_tx_abort(tx); + zrele(szp); + if (tzp) + zrele(tzp); + goto top; + } + dmu_tx_abort(tx); + zrele(szp); + if (tzp) + zrele(tzp); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + if (tzp) /* Attempt to remove the existing target */ + error = zfs_link_destroy(tdl, tzp, tx, zflg, NULL); + + if (error == 0) { + error = zfs_link_create(tdl, szp, tx, ZRENAMING); + if (error == 0) { + szp->z_pflags |= ZFS_AV_MODIFIED; + if (tdzp->z_pflags & ZFS_PROJINHERIT) + szp->z_pflags |= ZFS_PROJINHERIT; + + error = sa_update(szp->z_sa_hdl, SA_ZPL_FLAGS(zfsvfs), + (void *)&szp->z_pflags, sizeof (uint64_t), tx); + ASSERT0(error); + + error = zfs_link_destroy(sdl, szp, tx, ZRENAMING, NULL); + if (error == 0) { + zfs_log_rename(zilog, tx, TX_RENAME | + (flags & FIGNORECASE ? TX_CI : 0), sdzp, + sdl->dl_name, tdzp, tdl->dl_name, szp); + + + } else { + /* + * At this point, we have successfully created + * the target name, but have failed to remove + * the source name. Since the create was done + * with the ZRENAMING flag, there are + * complications; for one, the link count is + * wrong. The easiest way to deal with this + * is to remove the newly created target, and + * return the original error. This must + * succeed; fortunately, it is very unlikely to + * fail, since we just created it. + */ + VERIFY3U(zfs_link_destroy(tdl, szp, tx, + ZRENAMING, NULL), ==, 0); + } + } else { + /* + * If we had removed the existing target, subsequent + * call to zfs_link_create() to add back the same entry + * but, the new dnode (szp) should not fail. + */ + ASSERT(tzp == NULL); + } + } + + dmu_tx_commit(tx); +out: + if (zl != NULL) + zfs_rename_unlock(&zl); + + zfs_dirent_unlock(sdl); + zfs_dirent_unlock(tdl); + + if (sdzp == tdzp) + rw_exit(&sdzp->z_name_lock); + + zrele(szp); + if (tzp) { + zrele(tzp); + } + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Insert the indicated symbolic reference entry into the directory. + * + * IN: dzp - Directory to contain new symbolic link. + * name - Name of directory entry in dip. + * vap - Attributes of new entry. + * link - Name for new symlink entry. + * cr - credentials of caller. + * flags - case flags + * + * OUT: zpp - Znode for new symbolic link. + * + * RETURN: 0 on success, error code on failure. + * + * Timestamps: + * dip - ctime|mtime updated + */ +int +zfs_symlink(znode_t *dzp, char *name, vattr_t *vap, char *link, + znode_t **zpp, cred_t *cr, int flags, zidmap_t *mnt_ns) +{ + znode_t *zp; + zfs_dirlock_t *dl; + dmu_tx_t *tx; + zfsvfs_t *zfsvfs = ZTOZSB(dzp); + zilog_t *zilog; + uint64_t len = strlen(link); + int error; + int zflg = ZNEW; + zfs_acl_ids_t acl_ids; + boolean_t fuid_dirtied; + uint64_t txtype = TX_SYMLINK; + boolean_t waited = B_FALSE; + + ASSERT(S_ISLNK(vap->va_mode)); + + if (name == NULL) + return (SET_ERROR(EINVAL)); + + if ((error = zfs_enter_verify_zp(zfsvfs, dzp, FTAG)) != 0) + return (error); + zilog = zfsvfs->z_log; + + if (zfsvfs->z_utf8 && u8_validate(name, strlen(name), + NULL, U8_VALIDATE_ENTIRE, &error) < 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EILSEQ)); + } + if (flags & FIGNORECASE) + zflg |= ZCILOOK; + + if (len > MAXPATHLEN) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(ENAMETOOLONG)); + } + + if ((error = zfs_acl_ids_create(dzp, 0, + vap, cr, NULL, &acl_ids, NULL)) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } +top: + *zpp = NULL; + + /* + * Attempt to lock directory; fail if entry already exists. + */ + error = zfs_dirent_lock(&dl, dzp, name, &zp, zflg, NULL, NULL); + if (error) { + zfs_acl_ids_free(&acl_ids); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + if ((error = zfs_zaccess(dzp, ACE_ADD_FILE, 0, B_FALSE, cr, NULL))) { + zfs_acl_ids_free(&acl_ids); + zfs_dirent_unlock(dl); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + if (zfs_acl_ids_overquota(zfsvfs, &acl_ids, ZFS_DEFAULT_PROJID)) { + zfs_acl_ids_free(&acl_ids); + zfs_dirent_unlock(dl); + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EDQUOT)); + } + tx = dmu_tx_create(zfsvfs->z_os); + fuid_dirtied = zfsvfs->z_fuid_dirty; + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, MAX(1, len)); + dmu_tx_hold_zap(tx, dzp->z_id, TRUE, name); + dmu_tx_hold_sa_create(tx, acl_ids.z_aclp->z_acl_bytes + + ZFS_SA_BASE_ATTR_SIZE + len); + dmu_tx_hold_sa(tx, dzp->z_sa_hdl, B_FALSE); + if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, + acl_ids.z_aclp->z_acl_bytes); + } + if (fuid_dirtied) + zfs_fuid_txhold(zfsvfs, tx); + error = dmu_tx_assign(tx, (waited ? TXG_NOTHROTTLE : 0) | TXG_NOWAIT); + if (error) { + zfs_dirent_unlock(dl); + if (error == ERESTART) { + waited = B_TRUE; + dmu_tx_wait(tx); + dmu_tx_abort(tx); + goto top; + } + zfs_acl_ids_free(&acl_ids); + dmu_tx_abort(tx); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * Create a new object for the symlink. + * for version 4 ZPL datsets the symlink will be an SA attribute + */ + zfs_mknode(dzp, vap, tx, cr, 0, &zp, &acl_ids); + + if (fuid_dirtied) + zfs_fuid_sync(zfsvfs, tx); + + mutex_enter(&zp->z_lock); + if (zp->z_is_sa) + error = sa_update(zp->z_sa_hdl, SA_ZPL_SYMLINK(zfsvfs), + link, len, tx); + else + zfs_sa_symlink(zp, link, len, tx); + mutex_exit(&zp->z_lock); + + zp->z_size = len; + (void) sa_update(zp->z_sa_hdl, SA_ZPL_SIZE(zfsvfs), + &zp->z_size, sizeof (zp->z_size), tx); + /* + * Insert the new object into the directory. + */ + error = zfs_link_create(dl, zp, tx, ZNEW); + if (error != 0) { + zfs_znode_delete(zp, tx); + } else { + if (flags & FIGNORECASE) + txtype |= TX_CI; + zfs_log_symlink(zilog, tx, txtype, dzp, zp, name, link); + } + + zfs_acl_ids_free(&acl_ids); + + dmu_tx_commit(tx); + + zfs_dirent_unlock(dl); + + /* + * Windows - attach the vnode _after_ committing the transaction + */ + zfs_znode_getvnode(zp, dzp, zfsvfs); + + if (error == 0) { + *zpp = zp; + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); + } else { + zrele(zp); + } + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Return, in the buffer contained in the provided uio structure, + * the symbolic path referred to by ip. + * + * IN: ip - inode of symbolic link + * uio - structure to contain the link path. + * cr - credentials of caller. + * + * RETURN: 0 if success + * error code if failure + * + * Timestamps: + * ip - atime updated + */ +int +zfs_readlink(struct vnode *vp, zfs_uio_t *uio, cred_t *cr) +{ + (void) cr; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = ITOZSB(vp); + int error; + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + + mutex_enter(&zp->z_lock); + if (zp->z_is_sa) + error = sa_lookup_uio(zp->z_sa_hdl, + SA_ZPL_SYMLINK(zfsvfs), uio); + else + error = zfs_sa_readlink(zp, uio); + mutex_exit(&zp->z_lock); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * Insert a new entry into directory tdzp referencing szp. + * + * IN: tdzp - Directory to contain new entry. + * szp - znode of new entry. + * name - name of new entry. + * cr - credentials of caller. + * flags - case flags. + * + * RETURN: 0 if success + * error code if failure + * + * Timestamps: + * tdzp - ctime|mtime updated + * szp - ctime updated + */ +int +zfs_link(znode_t *tdzp, znode_t *szp, char *name, cred_t *cr, + int flags) +{ + struct vnode *svp = ZTOV(szp); + znode_t *tzp; + zfsvfs_t *zfsvfs = ZTOZSB(tdzp); + zilog_t *zilog; + zfs_dirlock_t *dl; + dmu_tx_t *tx; + int error; + int zf = ZNEW; + uint64_t parent; + uid_t owner; + boolean_t waited = B_FALSE; + boolean_t is_tmpfile = 0; + uint64_t txg; + + ASSERT(S_ISDIR(tdzp->z_mode)); + + if (name == NULL) + return (SET_ERROR(EINVAL)); + + if ((error = zfs_enter_verify_zp(zfsvfs, tdzp, FTAG)) != 0) + return (error); + zilog = zfsvfs->z_log; + + if (VTOM(svp) != VTOM(ZTOV(tdzp))) { + zfs_exit(zfsvfs, FTAG); + return (EXDEV); + } + + /* + * POSIX dictates that we return EPERM here. + * Better choices include ENOTSUP or EISDIR. + */ + if (vnode_isdir(svp)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EPERM)); + } + + if ((error = zfs_verify_zp(szp)) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + /* + * If we are using project inheritance, means if the directory has + * ZFS_PROJINHERIT set, then its descendant directories will inherit + * not only the project ID, but also the ZFS_PROJINHERIT flag. Under + * such case, we only allow hard link creation in our tree when the + * project IDs are the same. + */ + if (tdzp->z_pflags & ZFS_PROJINHERIT && + tdzp->z_projid != szp->z_projid) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EXDEV)); + } + + /* Prevent links to .zfs/shares files */ + + if ((error = sa_lookup(szp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (uint64_t))) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + if (parent == zfsvfs->z_shares_dir) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EPERM)); + } + + if (zfsvfs->z_utf8 && u8_validate(name, + strlen(name), NULL, U8_VALIDATE_ENTIRE, &error) < 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EILSEQ)); + } + if (flags & FIGNORECASE) + zf |= ZCILOOK; + + /* + * We do not support links between attributes and non-attributes + * because of the potential security risk of creating links + * into "normal" file space in order to circumvent restrictions + * imposed in attribute space. + */ + if ((szp->z_pflags & ZFS_XATTR) != (tdzp->z_pflags & ZFS_XATTR)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + owner = zfs_fuid_map_id(zfsvfs, KUID_TO_SUID(szp->z_uid), + cr, ZFS_OWNER); + if (owner != crgetuid(cr) && secpolicy_basic_link(cr) != 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EPERM)); + } + + if ((error = zfs_zaccess(tdzp, ACE_ADD_FILE, 0, B_FALSE, cr, NULL))) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + +top: + /* + * Attempt to lock directory; fail if entry already exists. + */ + error = zfs_dirent_lock(&dl, tdzp, name, &tzp, zf, NULL, NULL); + if (error) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_sa(tx, szp->z_sa_hdl, B_FALSE); + dmu_tx_hold_zap(tx, tdzp->z_id, TRUE, name); + if (is_tmpfile) + dmu_tx_hold_zap(tx, zfsvfs->z_unlinkedobj, FALSE, NULL); + + zfs_sa_upgrade_txholds(tx, szp); + zfs_sa_upgrade_txholds(tx, tdzp); + error = dmu_tx_assign(tx, (waited ? TXG_NOTHROTTLE : 0) | TXG_NOWAIT); + if (error) { + zfs_dirent_unlock(dl); + if (error == ERESTART) { + waited = B_TRUE; + dmu_tx_wait(tx); + dmu_tx_abort(tx); + goto top; + } + dmu_tx_abort(tx); + zfs_exit(zfsvfs, FTAG); + return (error); + } + + error = zfs_link_create(dl, szp, tx, 0); + + if (error == 0) { + uint64_t txtype = TX_LINK; + if (flags & FIGNORECASE) + txtype |= TX_CI; + zfs_log_link(zilog, tx, txtype, tdzp, szp, name); + } else if (is_tmpfile) { + /* restore z_unlinked since when linking failed */ + szp->z_unlinked = B_TRUE; + } + txg = dmu_tx_get_txg(tx); + dmu_tx_commit(tx); + + zfs_dirent_unlock(dl); + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zilog, 0); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +void +zfs_inactive(struct vnode *vp) +{ + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = ITOZSB(vp); + int error; + + if (zfsvfs == NULL) + return; + if (zp == NULL) + return; + + rw_enter(&zfsvfs->z_teardown_inactive_lock, RW_READER); + if (zp->z_sa_hdl == NULL) { + /* + * The fs has been unmounted, or we did a + * suspend/resume and this file no longer exists. + */ + rw_exit(&zfsvfs->z_teardown_inactive_lock); + vnode_recycle(vp); + return; + } + + if (zp->z_unlinked) { + /* + * Fast path to recycle a vnode of a removed file. + */ + rw_exit(&zfsvfs->z_teardown_inactive_lock); + vnode_recycle(vp); + return; + } + + if (zp->z_atime_dirty && zp->z_unlinked == 0) { + dmu_tx_t *tx = dmu_tx_create(zfsvfs->z_os); + + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + zfs_sa_upgrade_txholds(tx, zp); + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + } else { + (void) sa_update(zp->z_sa_hdl, SA_ZPL_ATIME(zfsvfs), + (void *)&zp->z_atime, sizeof (zp->z_atime), tx); + zp->z_atime_dirty = 0; + dmu_tx_commit(tx); + } + } + rw_exit(&zfsvfs->z_teardown_inactive_lock); +} + +/* + * Free or allocate space in a file. Currently, this function only + * supports the `F_FREESP' command. However, this command is somewhat + * misnamed, as its functionality includes the ability to allocate as + * well as free space. + * + * IN: zp - znode of file to free data in. + * cmd - action to take (only F_FREESP supported). + * bfp - section of file to free/alloc. + * flag - current file open mode flags. + * offset - current file offset. + * cr - credentials of caller. + * + * RETURN: 0 on success, error code on failure. + * + * Timestamps: + * zp - ctime|mtime updated + */ +int +zfs_space(znode_t *zp, int cmd, flock64_t *bfp, int flag, + offset_t offset, cred_t *cr) +{ + (void) offset; + zfsvfs_t *zfsvfs = ZTOZSB(zp); + uint64_t off, len; + int error; + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + + if (cmd != F_FREESP) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + /* + * Callers might not be able to detect properly that we are read-only, + * so check it explicitly here. + */ + if (zfs_is_readonly(zfsvfs)) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EROFS)); + } + + if (bfp->l_len < 0) { + zfs_exit(zfsvfs, FTAG); + return (SET_ERROR(EINVAL)); + } + + /* + * Permissions aren't checked on Solaris because on this OS + * zfs_space() can only be called with an opened file handle. + * On Linux we can get here through truncate_range() which + * operates directly on inodes, so we need to check access rights. + */ + if ((error = zfs_zaccess(zp, ACE_WRITE_DATA, 0, B_FALSE, cr, NULL))) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + off = bfp->l_start; + len = bfp->l_len; /* 0 means from off to end of file */ + + error = zfs_freesp(zp, off, len, flag, TRUE); + + zfs_exit(zfsvfs, FTAG); + return (error); +} diff --git a/module/os/windows/zfs/zfs_vnops_windows.c b/module/os/windows/zfs/zfs_vnops_windows.c new file mode 100644 index 000000000000..e6fdfb1839a4 --- /dev/null +++ b/module/os/windows/zfs/zfs_vnops_windows.c @@ -0,0 +1,6627 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2017 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ + +#undef _NTDDK_ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// I have no idea what black magic is needed to get ntifs.h to define these + +#ifndef FsRtlEnterFileSystem +#define FsRtlEnterFileSystem() { \ + KeEnterCriticalRegion(); \ +} +#endif +#ifndef FsRtlExitFileSystem +#define FsRtlExitFileSystem() { \ + KeLeaveCriticalRegion(); \ +} +#endif + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +PDEVICE_OBJECT ioctlDeviceObject = NULL; +PDEVICE_OBJECT fsDiskDeviceObject = NULL; +#ifdef DEBUG_IOCOUNT +static kmutex_t GIANT_SERIAL_LOCK; +#endif + +#ifdef _KERNEL + +DRIVER_INITIALIZE DriverEntry; + +unsigned int debug_vnop_osx_printf = 0; +unsigned int zfs_vnop_ignore_negatives = 0; +unsigned int zfs_vnop_ignore_positives = 0; +unsigned int zfs_vnop_create_negatives = 1; + +#endif + +#ifdef ALLOC_PRAGMA +#pragma alloc_text(INIT, DriverEntry) +#endif + +#define DECLARE_CRED(ap) \ + cred_t *cr; +#define DECLARE_CONTEXT(ap) \ + caller_context_t *ct +#define DECLARE_CRED_AND_CONTEXT(ap) \ + DECLARE_CRED(ap); \ + DECLARE_CONTEXT(ap) + + +#ifdef _KERNEL +uint64_t vnop_num_reclaims = 0; +uint64_t vnop_num_vnodes = 0; +uint64_t zfs_disable_wincache = 0; +#endif + +extern void UnlockAndFreeMdl(PMDL); + +BOOLEAN +zfs_AcquireForLazyWrite(void *Context, BOOLEAN Wait) +{ + FILE_OBJECT *fo = Context; + + if (fo == NULL) + return (FALSE); + + struct vnode *vp = fo->FsContext; + dprintf("%s:\n", __func__); + + if (vp == NULL) + return (FALSE); + + if (VN_HOLD(vp) == 0) { + + if (!ExAcquireResourceSharedLite( + vp->FileHeader.PagingIoResource, Wait)) { + dprintf("Failed\n"); + VN_RELE(vp); + return (FALSE); + } + + vnode_ref(vp); + VN_RELE(vp); + return (TRUE); + } + + /* + * There is something wrong (still) with unmounting so + * LazyWriter does not stop (even though volume is gone) + * Presumably we've not correctly told some part of Windows + * that we are unmounted. + * So we have to pretend the lock here went well, and + * ignore the write request later, for it to eventually + * stop. + */ + return (TRUE); + + return (FALSE); +} + +void +zfs_ReleaseFromLazyWrite(void *Context) +{ + FILE_OBJECT *fo = Context; + + if (fo != NULL) { + struct vnode *vp = fo->FsContext; + dprintf("%s:\n", __func__); + if (vp != NULL && VN_HOLD(vp) == 0) { + ExReleaseResourceLite(vp->FileHeader.PagingIoResource); + vnode_rele(vp); + VN_RELE(vp); + } + } +} + +BOOLEAN +zfs_AcquireForReadAhead(void *Context, BOOLEAN Wait) +{ + FILE_OBJECT *fo = Context; + + if (fo == NULL) + return (FALSE); + + struct vnode *vp = fo->FsContext; + dprintf("%s:\n", __func__); + + if (vp == NULL) + return (FALSE); + + if (VN_HOLD(vp) == 0) { + + if (!ExAcquireResourceSharedLite(vp->FileHeader.Resource, + Wait)) { + dprintf("Failed\n"); + VN_RELE(vp); + return (FALSE); + } + + vnode_ref(vp); + VN_RELE(vp); + return (TRUE); + } + + return (FALSE); +} + +void +zfs_ReleaseFromReadAhead(void *Context) +{ + FILE_OBJECT *fo = Context; + + if (fo != NULL) { + struct vnode *vp = fo->FsContext; + dprintf("%s:\n", __func__); + if (vp != NULL && VN_HOLD(vp) == 0) { + ExReleaseResourceLite(vp->FileHeader.Resource); + vnode_rele(vp); + VN_RELE(vp); + } + } +} + +CACHE_MANAGER_CALLBACKS CacheManagerCallbacks = +{ + .AcquireForLazyWrite = zfs_AcquireForLazyWrite, + .ReleaseFromLazyWrite = zfs_ReleaseFromLazyWrite, + .AcquireForReadAhead = zfs_AcquireForReadAhead, + .ReleaseFromReadAhead = zfs_ReleaseFromReadAhead +}; + +int +zfs_init_cache(FILE_OBJECT *fo, struct vnode *vp) +{ + zfs_dirlist_t *zccb = fo->FsContext2; + + try { + if (fo->PrivateCacheMap == NULL) { + + VERIFY3U(zccb->cacheinit, ==, 0); + atomic_inc_64(&zccb->cacheinit); + + CcInitializeCacheMap(fo, + (PCC_FILE_SIZES)&vp->FileHeader.AllocationSize, + FALSE, + &CacheManagerCallbacks, fo); + dprintf("CcInitializeCacheMap called on vp %p\n", vp); + // CcSetAdditionalCacheAttributes(fo, FALSE, FALSE); + // must be FALSE + fo->Flags |= FO_CACHE_SUPPORTED; + dprintf("%s: CcInitializeCacheMap\n", __func__); + } + } except(EXCEPTION_EXECUTE_HANDLER) { + return (GetExceptionCode()); + } + return (0); +} + + +/* + * zfs vfs operations. + */ + +/* + * FileObject->FsContext will point to vnode, many FileObjects can point + * to same vnode. + * FileObject->FsContext2 will point to own "zfs_dirlist_t" and be unique + * to each FileObject. + * - which could also be done with TSD data, but this appears to be + * the Windows norm. + */ +void +zfs_couplefileobject(vnode_t *vp, FILE_OBJECT *fileobject, uint64_t size) +{ + ASSERT3P(fileobject->FsContext2, ==, NULL); + zfs_dirlist_t *zccb = kmem_zalloc(sizeof (zfs_dirlist_t), KM_SLEEP); + zccb->magic = ZFS_DIRLIST_MAGIC; + fileobject->FsContext2 = zccb; + + vnode_couplefileobject(vp, fileobject, size); + + zfs_init_cache(fileobject, vp); +} + +void +zfs_decouplefileobject(vnode_t *vp, FILE_OBJECT *fileobject) +{ + // We release FsContext2 at CLEANUP, but fastfat releases it in + // CLOSE. Does this matter? + zfs_dirlist_t *zccb = fileobject->FsContext2; + + if (zccb != NULL) { + + ASSERT3U(zccb->cacheinit, ==, 1); + zccb->cacheinit = 0; + + if (zccb->searchname.Buffer != NULL) { + kmem_free(zccb->searchname.Buffer, + zccb->searchname.MaximumLength); + zccb->searchname.Buffer = NULL; + zccb->searchname.MaximumLength = 0; + } + + kmem_free(zccb, sizeof (zfs_dirlist_t)); + fileobject->FsContext2 = NULL; + } + + CcUninitializeCacheMap(fileobject, + NULL, NULL); + vnode_decouplefileobject(vp, fileobject); +} + +void +check_and_set_stream_parent(char *stream_name, PFILE_OBJECT FileObject, + uint64_t id) +{ + if (stream_name != NULL && FileObject != NULL && + FileObject->FsContext2 != NULL) { + zfs_dirlist_t *zccb = FileObject->FsContext2; + + zccb->real_file_id = id; + + if (FileObject->FsContext != NULL) { + struct vnode *vp = FileObject->FsContext; + znode_t *dzp; + znode_t *zp = VTOZ(vp); + if (zp != NULL && zp->z_zfsvfs != NULL) { + // Fetch gparent (one above xattr dir) + int error = zfs_zget(zp->z_zfsvfs, id, &dzp); + if (!error) { + zfs_build_path_stream(zp, dzp, + &zp->z_name_cache, &zp->z_name_len, + &zp->z_name_offset, stream_name); + zrele(dzp); + } + } + } + } +} + +/* + * Take filename, look for colons ":". + * No colon, return OK. + * if ends with "::$DATA". Terminate on colon, return OK (regular file open). + * if ends with anything not ":$DATA", return error. + * (we don't handle other types) + * if colon, parse name up until next colon. Assign colonname to + * point to stream name. + */ +int +stream_parse(char *filename, char **streamname) +{ + char *colon, *second; + + // Just a filename, no streams. + colon = strchr(filename, ':'); + if (colon == NULL) + return (0); + + // Regular file, with "::$DATA" end? + if (strcasecmp(colon, "::$DATA") == 0) { + *colon = 0; // Terminate before colon + return (0); + } + + // Look for second colon + second = strchr(&colon[1], ':'); + + // No second colon, just stream name. Validity check? + if (second == NULL) { + *streamname = &colon[1]; + *colon = 0; // Cut off streamname from filename + + // We now ADD ":$DATA" to the stream name. + strcat(*streamname, ":$DATA"); + + goto checkname; + } + + // Have second colon, better be ":$DATA". + if (strcasecmp(second, ":$DATA") == 0) { + + // Terminate at second colon, set streamname + // We now keep the ":$DATA" extension in the xattr name + // *second = 0; + + *streamname = &colon[1]; + *colon = 0; // Cut of streamname from filename + + goto checkname; + } + + // Not $DATA + dprintf("%s: Not handling StreamType '%s'\n", __func__, second); + return (EINVAL); + +checkname: + if (strlen(*streamname) >= 512) + return (STATUS_OBJECT_NAME_INVALID); + + if (strchr(*streamname, '/') || + /* strchr(&colon[2], ':') || there is one at ":$DATA" */ + !strcasecmp("DOSATTRIB:$DATA", *streamname) || + !strcasecmp("EA:$DATA", *streamname) || + !strcasecmp("reparse:$DATA", *streamname) || + !strcasecmp("casesensitive:$DATA", *streamname)) + return (STATUS_OBJECT_NAME_INVALID); + + return (0); +} + +/* + * Attempt to parse 'filename', descending into filesystem. + * If start "dvp" is passed in, it is expected to have a HOLD + * If successful, function will return with: + * - HOLD on dvp + * - HOLD on vp + * - final parsed filename part in 'lastname' (in the case of creating an entry) + */ +int +zfs_find_dvp_vp(zfsvfs_t *zfsvfs, char *filename, int finalpartmaynotexist, + int finalpartmustnotexist, char **lastname, struct vnode **dvpp, + struct vnode **vpp, int flags, ULONG options) +{ + int error = ENOENT; + znode_t *zp; + struct vnode *dvp = NULL; + struct vnode *vp = NULL; + char *word = NULL; + char *brkt = NULL; + struct componentname cn; + int fullstrlen; + char namebuffer[MAXNAMELEN]; + BOOLEAN FileOpenReparsePoint; + + FileOpenReparsePoint = + BooleanFlagOn(options, FILE_OPEN_REPARSE_POINT); + + // Iterate from dvp if given, otherwise root + dvp = *dvpp; + + if (dvp == NULL) { + // Grab a HOLD + error = zfs_zget(zfsvfs, zfsvfs->z_root, &zp); + if (error != 0) + return (ESRCH); // No such dir + dvp = ZTOV(zp); + } else { + // Passed in dvp is already HELD, but grab one now + // since we release dirs as we descend + dprintf("%s: passed in dvp\n", __func__); + if (VN_HOLD(dvp) != 0) + return (ESRCH); + } + + fullstrlen = strlen(filename); + + // Sometimes we are given a path like "\Directory\directory\" + // with the final separator, we want to eat that final character. + if ((fullstrlen > 2) && + (filename[fullstrlen - 1] == '\\')) + filename[--fullstrlen] = 0; + + for (word = strtok_r(filename, "/\\", &brkt); + word; + word = strtok_r(NULL, "/\\", &brkt)) { + int direntflags = 0; + // dprintf("..'%s'..", word); + + // If a component part name is too long + if (strlen(word) > MAXNAMELEN - 1) { + VN_RELE(dvp); + return (STATUS_OBJECT_NAME_INVALID); + } + strlcpy(namebuffer, word, sizeof (namebuffer)); + // Dont forget zfs_lookup() modifies + // "cn" here, so size needs to be max, if + // formD is in effect. + cn.cn_nameiop = LOOKUP; + cn.cn_flags = ISLASTCN; + cn.cn_namelen = strlen(namebuffer); + cn.cn_nameptr = namebuffer; + cn.cn_pnlen = MAXNAMELEN; + cn.cn_pnbuf = namebuffer; + + error = zfs_lookup(VTOZ(dvp), namebuffer, + &zp, flags, NULL, &direntflags, &cn); + + // If snapshot dir and we are pretending it is deleted... + if (error == 0 && zp->z_vnode != NULL && ZTOV(zp)->v_unlink) { + VN_RELE(ZTOV(zp)); + error = ENOENT; + } + if (error != 0) { + // If we are creating a file, or looking up parent, + // allow it not to exist + if (finalpartmaynotexist) + break; + dprintf("failing out here\n"); + // since we weren't successful, release dvp here + VN_RELE(dvp); + dvp = NULL; + break; + } + + // If last lookup hit a non-directory type, we stop + vp = ZTOV(zp); + ASSERT(zp != NULL); + + + /* + * If we come across a REPARSE, we stop processing here + * and pass the "zp" back for caller to do more processing, + * which might include returning "zp" (FILE_OPEN_REPARSE_POINT) + * and ReparseTag. + * But, if IRP->zfsvfs is the same as zp->zfsvfs, the lookup + * was already requested for "us" specifically, so keep going. + * This could fail with nested dirmounts? Only check lowest + * directory to bail. + */ + if ((zp->z_pflags & ZFS_REPARSE) && + (zfsvfs != zp->z_zfsvfs)) { + + // Indicate if reparse was final part + if (lastname) + *lastname = brkt; + + if (dvpp != NULL) + *dvpp = dvp; + if (vpp != NULL) + *vpp = vp; + // VN_RELE(dvp); + return (STATUS_REPARSE); + } + + if (vp->v_type == VDIR) { + // Not reparse + VN_RELE(dvp); + dvp = vp; + vp = NULL; + } else { + // If we aren't the final component, descending dirs, + // and it's a file? + if (brkt != NULL && *brkt != 0) { + dprintf("%s: not a DIR triggered '%s'\n", + __func__, word); + VN_RELE(dvp); + return (ENOTDIR); + } + break; + } // is dir or not + + } // for word + // dprintf("\n"); + + if (dvp != NULL) { + // We return with dvp HELD + // VN_RELE(dvp); + } else { + dprintf("%s: failed to find dvp for '%s' word '%s' err %d\n", + __func__, filename, word?word:"(null)", error); + return (error); + } + + if (error != 0 && !vp && !finalpartmaynotexist) { + VN_RELE(dvp); + return (ENOENT); + } + + if (!word && finalpartmustnotexist && dvp && !vp) { + dprintf("CREATE with existing dir exit?\n"); + VN_RELE(dvp); + + if (zp && ZTOV(zp) && !vnode_isdir(ZTOV(zp))) + return (ENOTDIR); + return (EEXIST); + } + + // If finalpartmaynotexist is TRUE, make sure we are looking at + // the finalpart, and not in the middle of descending + if (finalpartmaynotexist && brkt != NULL && *brkt != 0) { + dprintf("finalpartmaynotexist, but not at finalpart: %s\n", + brkt); + VN_RELE(dvp); + return (ESRCH); + } + + if (lastname) { + + *lastname = word /* ? word : filename */; + + // Skip any leading "\" + while (*lastname != NULL && + (**lastname == '\\' || **lastname == '/')) + (*lastname)++; + + } + + if (dvpp != NULL) + *dvpp = dvp; + if (vpp != NULL) + *vpp = vp; + + return (0); +} + +/* + * In POSIX, the vnop_lookup() would return with iocount still held + * for the caller to issue VN_RELE() on when done. + * The above zfs_find_dvp_vp() behaves a little like that, in that + * if a successful "vp" is returned, it has a iocount lock, and + * is released here when finished. + * zfs_vnop_lookup serves as the bridge between Windows and Unix + * and will assign FileObject->FsContext as appropriate, with usecount set + * when required, but it will not hold iocount. + */ +int +zfs_vnop_lookup_impl(PIRP Irp, PIO_STACK_LOCATION IrpSp, mount_t *zmo, + char *filename, vattr_t *vap) +{ + int error; + cred_t *cr = NULL; + char *finalname = NULL; + PFILE_OBJECT FileObject; + ULONG outlen; + struct vnode *dvp = NULL; + struct vnode *vp = NULL; + znode_t *zp = NULL; + znode_t *dzp = NULL; + ULONG Options; + BOOLEAN CreateDirectory; + BOOLEAN NoIntermediateBuffering; + BOOLEAN OpenDirectory; + BOOLEAN IsPagingFile; + BOOLEAN OpenTargetDirectory; + BOOLEAN DirectoryFile; + BOOLEAN NonDirectoryFile; + BOOLEAN NoEaKnowledge; + BOOLEAN DeleteOnClose; + BOOLEAN OpenRequiringOplock; + BOOLEAN TemporaryFile; + BOOLEAN OpenRoot; + BOOLEAN CreateFile; + BOOLEAN FileOpenByFileId; + BOOLEAN FileOpenReparsePoint; + ULONG CreateDisposition; + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + int flags = 0; + int dvp_no_rele = 0; + char *stream_name = NULL; + boolean_t UndoShareAccess = FALSE; + NTSTATUS Status = STATUS_SUCCESS; + ACCESS_MASK granted_access = 0; + + if (zfsvfs == NULL) + return (STATUS_OBJECT_PATH_NOT_FOUND); + + FileObject = IrpSp->FileObject; + Options = IrpSp->Parameters.Create.Options; + + dprintf("%s: enter\n", __func__); + + if (FileObject->RelatedFileObject != NULL) { + FileObject->Vpb = FileObject->RelatedFileObject->Vpb; + // A relative open must be via a relative path. + if (FileObject->FileName.Length != 0 && + FileObject->FileName.Buffer[0] == L'\\') { + return (STATUS_INVALID_PARAMETER); + } + } else { + FileObject->Vpb = zmo->vpb; + } + + DirectoryFile = + BooleanFlagOn(Options, FILE_DIRECTORY_FILE); + NonDirectoryFile = + BooleanFlagOn(Options, FILE_NON_DIRECTORY_FILE); + NoIntermediateBuffering = + BooleanFlagOn(Options, FILE_NO_INTERMEDIATE_BUFFERING); + NoEaKnowledge = + BooleanFlagOn(Options, FILE_NO_EA_KNOWLEDGE); + DeleteOnClose = + BooleanFlagOn(Options, FILE_DELETE_ON_CLOSE); + FileOpenByFileId = + BooleanFlagOn(Options, FILE_OPEN_BY_FILE_ID); + FileOpenReparsePoint = + BooleanFlagOn(Options, FILE_OPEN_REPARSE_POINT); + + + // Should be passed an 8 byte FileId instead. + if (FileOpenByFileId && FileObject->FileName.Length != + sizeof (ULONGLONG)) + return (STATUS_INVALID_PARAMETER); + + TemporaryFile = BooleanFlagOn(IrpSp->Parameters.Create.FileAttributes, + FILE_ATTRIBUTE_TEMPORARY); + + CreateDisposition = (Options >> 24) & 0x000000ff; + + IsPagingFile = BooleanFlagOn(IrpSp->Flags, SL_OPEN_PAGING_FILE); + ASSERT(!IsPagingFile); + // ASSERT(!OpenRequiringOplock); + // Open the directory instead of the file + OpenTargetDirectory = BooleanFlagOn(IrpSp->Flags, + SL_OPEN_TARGET_DIRECTORY); +/* + * CreateDisposition value Action if file exists + * Action if file does not exist UNIX Perms + * FILE_SUPERSEDE Replace the file. + * Create the file. * Unlink + O_CREAT | O_TRUNC + * FILE_CREATE Return an error. + * Create the file. * O_CREAT | O_EXCL + * FILE_OPEN Open the file. + * Return an error. * 0 + * FILE_OPEN_IF Open the file. + * Create the file. * O_CREAT + * FILE_OVERWRITE Open the file, overwrite it. + * Return an error. * O_TRUNC + * FILE_OVERWRITE_IF Open the file, overwrite it. + * Create the file. * O_CREAT | O_TRUNC + * + * Apparently SUPERSEDE is more or less Unlink entry before recreate, + * so it loses ACLs, XATTRs and NamedStreams. + * + * IoStatus return codes: + * FILE_CREATED + * FILE_OPENED + * FILE_OVERWRITTEN + * FILE_SUPERSEDED + * FILE_EXISTS + * FILE_DOES_NOT_EXIST + * + */ + + // Dir create/open is straight forward, do that here + // Files are harder, do that once we know if it exists. + CreateDirectory = (BOOLEAN)(DirectoryFile && + ((CreateDisposition == FILE_CREATE) || + (CreateDisposition == FILE_OPEN_IF))); + + OpenDirectory = (BOOLEAN)(DirectoryFile && + ((CreateDisposition == FILE_OPEN) || + (CreateDisposition == FILE_OPEN_IF))); + + CreateFile = (BOOLEAN)( + ((CreateDisposition == FILE_CREATE) || + (CreateDisposition == FILE_OPEN_IF) || + (CreateDisposition == FILE_SUPERSEDE) || + (CreateDisposition == FILE_OVERWRITE_IF))); + + // If it is a volumeopen, we just grab rootvp so that directory + // listings work + if (FileObject->FileName.Length == 0 && + FileObject->RelatedFileObject == NULL) { + // If DirectoryFile return STATUS_NOT_A_DIRECTORY + // If OpenTargetDirectory return STATUS_INVALID_PARAMETER + dprintf("Started NULL open, returning root of mount\n"); + error = zfs_zget(zfsvfs, zfsvfs->z_root, &zp); + if (error != 0) + return (FILE_DOES_NOT_EXIST); // No root dir?! + + dvp = ZTOV(zp); + vnode_ref(dvp); // Hold open reference, until CLOSE + + zfs_couplefileobject(dvp, FileObject, 0ULL); + VN_RELE(dvp); + + Irp->IoStatus.Information = FILE_OPENED; + return (STATUS_SUCCESS); + } + + // No name conversion with FileID + + if (!FileOpenByFileId) { + + if (FileObject->FileName.Buffer != NULL && + FileObject->FileName.Length > 0) { + // Convert incoming filename to utf8 + error = RtlUnicodeToUTF8N(filename, PATH_MAX, &outlen, + FileObject->FileName.Buffer, + FileObject->FileName.Length); + + if (error != STATUS_SUCCESS && + error != STATUS_SOME_NOT_MAPPED) { + dprintf("RtlUnicodeToUTF8N returned 0x%x " + "input len %d\n", + error, FileObject->FileName.Length); + return (STATUS_OBJECT_NAME_INVALID); + } + // ASSERT(error != STATUS_SOME_NOT_MAPPED); + // Output string is only null terminated if input is, + // so do so now. + filename[outlen] = 0; + dprintf("%s: converted name is '%s' input len bytes %d " + "(err %d) %s %s\n", __func__, filename, + FileObject->FileName.Length, error, + DeleteOnClose ? "DeleteOnClose" : "", + IrpSp->Flags&SL_CASE_SENSITIVE ? "CaseSensitive" : + "CaseInsensitive"); + + if ((!IrpSp->Flags & SL_CASE_SENSITIVE) && + (zfsvfs->z_case != ZFS_CASE_SENSITIVE)) + flags |= FIGNORECASE; + +#if 0 + if (strcmp( + "\\System Volume Information\\WPSettings.dat", + filename) == 0) + return (STATUS_OBJECT_NAME_INVALID); +#endif + + if (Irp->Overlay.AllocationSize.QuadPart > 0) + dprintf("AllocationSize requested %llu\n", + Irp->Overlay.AllocationSize.QuadPart); + + // Check if we are called as VFS_ROOT(); + OpenRoot = (strncmp("\\", filename, PATH_MAX) == 0 || + strncmp("\\*", filename, PATH_MAX) == 0); + + if (OpenRoot) { + + error = zfs_zget(zfsvfs, zfsvfs->z_root, &zp); + + if (error == 0) { + vp = ZTOV(zp); + zfs_couplefileobject(vp, FileObject, + zp->z_size); + vnode_ref(vp); // Hold ref, until CLOSE + VN_RELE(vp); + + Irp->IoStatus.Information = FILE_OPENED; + return (STATUS_SUCCESS); + } + + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_OBJECT_PATH_NOT_FOUND); + } // OpenRoot + + } else { // If got filename + + // If no filename, we should fail, + // unless related is set. + if (FileObject->RelatedFileObject == NULL) { + // Fail + return (STATUS_OBJECT_NAME_INVALID); + } + // Related set, return it as opened. + dvp = FileObject->RelatedFileObject->FsContext; + zp = VTOZ(dvp); + dprintf("%s: Relative null-name open: '%s'\n", + __func__, zp->z_name_cache); + // Check types + if (NonDirectoryFile && vnode_isdir(dvp)) { + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_FILE_IS_A_DIRECTORY); + } + if (DirectoryFile && !vnode_isdir(dvp)) { + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_NOT_A_DIRECTORY); + } + // Grab vnode to ref + if (VN_HOLD(dvp) == 0) { + vnode_ref(dvp); // Hold ref, until CLOSE + zfs_couplefileobject(dvp, FileObject, 0ULL); + VN_RELE(dvp); + } else { + Irp->IoStatus.Information = 0; + return (STATUS_OBJECT_PATH_NOT_FOUND); + } + Irp->IoStatus.Information = FILE_OPENED; + return (STATUS_SUCCESS); + } + + // We have converted the filename, continue.. + if (FileObject->RelatedFileObject && + FileObject->RelatedFileObject->FsContext) { + dvp = FileObject->RelatedFileObject->FsContext; + + // This branch here, if failure, should not release dvp + dvp_no_rele = 1; + } + +/* + * Here, we want to check for Streams, which come in the syntax + * filename.ext:Stream:Type + * Type: appears optional, or we handle ":DATA". All others will be rejected. + * Stream: name of the stream, we convert this into XATTR named Stream + * It is valid to create a filename containing colons, so who knows what will + * happen here. + */ + error = stream_parse(filename, &stream_name); + if (error) { + Irp->IoStatus.Information = 0; + return (error); + } + if (stream_name != NULL) + dprintf("%s: Parsed out streamname '%s'\n", + __func__, stream_name); + + // There is a special case, where name is just the stream + // ":ZoneIdentifier:$DATA", and + // RelatedFileObject is set to the object. + if (stream_name != NULL && + FileObject->RelatedFileObject && + FileObject->RelatedFileObject->FsContext && + strlen(filename) == 0) { + + // The RelatedFileObject conditional above will + // assign "dvp" - but + // the stream_name check below will expect it in "vp". + // dvp_no_rele is already set. + dvp_no_rele = 1; + vp = FileObject->RelatedFileObject->FsContext; + dvp = NULL; + VERIFY0(VN_HOLD(vp)); + + } else { + + // If we have dvp, it is HELD + error = zfs_find_dvp_vp(zfsvfs, filename, + (CreateFile || OpenTargetDirectory), + (CreateDisposition == FILE_CREATE), + &finalname, &dvp, &vp, flags, Options); + + } + + } else { // Open By File ID + + error = zfs_zget(zfsvfs, + *((uint64_t *)IrpSp->FileObject->FileName.Buffer), &zp); + // Code below assumed dvp is also open + if (error == 0) { + uint64_t parent; + znode_t *dzp; + error = sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)); + if (error == 0) { + error = zfs_zget(zfsvfs, parent, &dzp); + } + if (error != 0) { + VN_RELE(ZTOV(zp)); + return (error); + } // failed to get parentid, or find parent + // Copy over the vp info for below, both are held. + vp = ZTOV(zp); + dvp = ZTOV(dzp); + } + } + + // If successful: + // - vp is HELD + // - dvp is HELD + // we need dvp from here on down. + + // If asked to open reparse point instead of following it, and + // it was the final part of the path, then just open it. + if (error == STATUS_REPARSE && FileOpenReparsePoint && + (!finalname || !*finalname)) + error = STATUS_SUCCESS; + + if (error) { + + /* + * With REPARSE, we are given "zp" to read the ReparseTag, and + * if they asked for it returned, do so, or free it. + */ + if (error == STATUS_REPARSE) { + /* + * How reparse points work from the point of + * view of the filesystem appears to undocumented. + * When returning STATUS_REPARSE, MSDN encourages + * us to return IO_REPARSE in + * Irp->IoStatus.Information, but that means we + * have to do our own translation. If we instead + * return the reparse tag in Information, and + * store a pointer to the reparse data buffer in + * Irp->Tail.Overlay.AuxiliaryBuffer, + * IopSymlinkProcessReparse will do the + * translation for us. + * - maharmstone + */ + zp = VTOZ(vp); + REPARSE_DATA_BUFFER *rpb; + size_t size; + // fix me, direct vp access + size = zfsctl_is_node(zp) ? vp->v_reparse_size : + zp->z_size; + rpb = ExAllocatePoolWithTag(PagedPool, + size, '!FSZ'); + get_reparse_point_impl(zp, rpb, size); + + // Return in Reserved the amount of path + // that was parsed. + /* FileObject->FileName.Length - parsed */ + rpb->Reserved = (outlen - + ((finalname - filename) + + strlen(finalname))) * sizeof (WCHAR); + + dprintf("%s: returning REPARSE\n", __func__); + Irp->IoStatus.Information = rpb->ReparseTag; + Irp->Tail.Overlay.AuxiliaryBuffer = (void *)rpb; + + // should this only work on the final component? +#if 0 + if (Options & FILE_OPEN_REPARSE_POINT) { + // Hold open reference, until CLOSE + vnode_ref(vp); + error = STATUS_SUCCESS; + zfs_couplefileobject(vp, FileObject, + zp ? zp->z_size : 0ULL); + } +#endif + VN_RELE(vp); + if (dvp) + VN_RELE(dvp); + + return (error); // STATUS_REPARSE + } + + if (dvp && !dvp_no_rele) VN_RELE(dvp); + if (vp) VN_RELE(vp); + + if (!dvp && error == ESRCH) { + dprintf("%s: failed to find dvp for '%s' \n", + __func__, filename); + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_OBJECT_PATH_NOT_FOUND); + } + if (error == STATUS_OBJECT_NAME_INVALID) { + dprintf("%s: filename component too long\n", __func__); + return (error); + } + // Open dir with FILE_CREATE but it exists + if (error == EEXIST) { + dprintf("%s: dir exists, wont create\n", __func__); + Irp->IoStatus.Information = FILE_EXISTS; + if (OpenTargetDirectory) + return (STATUS_NOT_A_DIRECTORY); + return (STATUS_FILE_IS_A_DIRECTORY); // 2 + } + if (error == ENOTDIR) { + dprintf("%s: file exists, wont create\n", __func__); + Irp->IoStatus.Information = FILE_EXISTS; + return (STATUS_OBJECT_NAME_COLLISION); // 3 + } + // A directory component did not exist, or was a file + if ((dvp == NULL) || (error == ENOTDIR)) { + dprintf("%s: failed to find dvp - or dvp is a file\n", + __func__); + Irp->IoStatus.Information = 0; + return (STATUS_OBJECT_NAME_NOT_FOUND); + } + dprintf("%s: failed to find vp in dvp\n", __func__); + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_OBJECT_NAME_NOT_FOUND); + } + + + // Streams + // If we opened vp, grab its xattrdir, and try to to locate stream + if (stream_name != NULL && vp != NULL) { + // Here, we will release dvp, and attempt to open the xattr dir. + // xattr dir will be the new dvp. Then we will look for + // streamname in xattrdir, and assign vp. + + VERIFY3P(dvp, !=, vp); + + // Create the xattrdir only if we are to create a new entry + zp = VTOZ(vp); + if ((error = zfs_get_xattrdir(zp, &dzp, cr, + CreateFile ? CREATE_XATTR_DIR : 0))) { + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + VN_RELE(vp); + return (STATUS_OBJECT_NAME_NOT_FOUND); + } + VN_RELE(vp); + if (dvp && !dvp_no_rele) + VN_RELE(dvp); + vp = NULL; + dvp = ZTOV(dzp); + int direntflags = 0; // To detect ED_CASE_CONFLICT + error = zfs_dirlook(dzp, stream_name, &zp, FIGNORECASE, + &direntflags, NULL); + if (!CreateFile && error) { + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_OBJECT_NAME_NOT_FOUND); + } + // Here, it may not exist, as we are to create it. + // If it exists, keep vp, otherwise, it is NULL + if (!error) { + vp = ZTOV(zp); + } // else vp is NULL from above + + finalname = stream_name; + } + + if (OpenTargetDirectory) { + if (dvp) { + +#if 0 + // If we asked for PARENT of a non-existing file, + // do we return error? + if (vp == NULL) { + dprintf("%s: opening PARENT dir, is ENOENT\n", + __func__); + VN_RELE(dvp); + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_OBJECT_NAME_NOT_FOUND); + } +#endif + + dprintf("%s: opening PARENT directory\n", __func__); + zfs_couplefileobject(dvp, FileObject, 0ULL); + vnode_ref(dvp); // Hold open reference, until CLOSE + if (DeleteOnClose) + Status = zfs_setunlink(FileObject, dvp); + if (Status == STATUS_SUCCESS) + Irp->IoStatus.Information = FILE_OPENED; + + if (vp) VN_RELE(vp); + VN_RELE(dvp); + return (Status); + } + ASSERT(vp == NULL); + ASSERT(dvp == NULL); + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_OBJECT_NAME_NOT_FOUND); + } + + // Here we have "dvp" of the directory. + // "vp" if the final part was a file. + + // Don't create if FILE_OPEN_IF (open existing) + if ((CreateDisposition == FILE_OPEN_IF) && (vp != NULL)) + CreateDirectory = 0; + + // Fail if FILE_CREATE but target exist + if ((CreateDisposition == FILE_CREATE) && (vp != NULL)) { + VN_RELE(vp); + VN_RELE(dvp); + Irp->IoStatus.Information = FILE_EXISTS; + if (CreateDirectory && !vnode_isdir(vp)) + return (STATUS_NOT_A_DIRECTORY); + return (STATUS_OBJECT_NAME_COLLISION); // create file error + } + + if (CreateDirectory && finalname) { + + if (TemporaryFile) + return (STATUS_INVALID_PARAMETER); + + if (zfsvfs->z_rdonly || vfs_isrdonly(zfsvfs->z_vfs) || + !spa_writeable(dmu_objset_spa(zfsvfs->z_os))) { + VN_RELE(dvp); + Irp->IoStatus.Information = 0; // ? + return (STATUS_MEDIA_WRITE_PROTECTED); + } + + vap->va_type = VDIR; + // Set default 777 if something else wasn't passed in + if (!(vap->va_mask & ATTR_MODE)) + vap->va_mode = 0777; + vap->va_mode |= S_IFDIR; + vap->va_mask |= (ATTR_MODE | ATTR_TYPE); + + ASSERT(strchr(finalname, '\\') == NULL); + error = zfs_mkdir(VTOZ(dvp), finalname, vap, &zp, NULL, + 0, NULL, NULL); + if (error == 0) { + vp = ZTOV(zp); + zfs_couplefileobject(vp, FileObject, 0ULL); + vnode_ref(vp); // Hold open reference, until CLOSE + if (DeleteOnClose) + Status = zfs_setunlink(FileObject, dvp); + + if (Status == STATUS_SUCCESS) { + Irp->IoStatus.Information = FILE_CREATED; + + // Update pflags, if needed + zfs_setwinflags(zp, + IrpSp->Parameters.Create.FileAttributes); + + IoSetShareAccess( + IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess, + IrpSp->Parameters.Create.ShareAccess, + FileObject, + &vp->share_access); + + zfs_send_notify(zfsvfs, zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_DIR_NAME, + FILE_ACTION_ADDED); + } + VN_RELE(vp); + VN_RELE(dvp); + return (Status); + } + VN_RELE(dvp); + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_OBJECT_PATH_NOT_FOUND); + } + + // If they requested just directory, fail non directories + if (DirectoryFile && vp != NULL && !vnode_isdir(vp)) { + dprintf("%s: asked for directory but found file\n", __func__); + VN_RELE(vp); + VN_RELE(dvp); + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_FILE_IS_A_DIRECTORY); + } + + // Asked for non-directory, but we got directory + if (NonDirectoryFile && !CreateFile && vp == NULL) { + dprintf("%s: asked for file but found directory\n", __func__); + VN_RELE(dvp); + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + return (STATUS_FILE_IS_A_DIRECTORY); + } + + if (vp) { + zp = VTOZ(vp); + } + + // If HIDDEN and SYSTEM are set, then the open of file must also have + // HIDDEN and SYSTEM set. + if ((zp != NULL) && + ((CreateDisposition == FILE_SUPERSEDE) || + (CreateDisposition == FILE_OVERWRITE) || + (CreateDisposition == FILE_OVERWRITE_IF))) { + if (((zp->z_pflags&ZFS_HIDDEN) && + !FlagOn(IrpSp->Parameters.Create.FileAttributes, + FILE_ATTRIBUTE_HIDDEN)) || + ((zp->z_pflags&ZFS_SYSTEM) && + !FlagOn(IrpSp->Parameters.Create.FileAttributes, + FILE_ATTRIBUTE_SYSTEM))) { + VN_RELE(vp); + VN_RELE(dvp); + dprintf("%s: denied due to hidden+system combo\n", + __func__); + return (STATUS_ACCESS_DENIED); + } + } + + // If overwrite, and tagged readonly, fail + // (note, supersede should succeed) + if ((zp != NULL) && + ((CreateDisposition == FILE_OVERWRITE) || + (CreateDisposition == FILE_OVERWRITE_IF))) { + if (zp->z_pflags&ZFS_READONLY) { + VN_RELE(vp); + VN_RELE(dvp); + dprintf("%s: denied due to ZFS_READONLY + OVERWRITE\n", + __func__); + return (STATUS_ACCESS_DENIED); + } + } + + // If flags are readonly, and tries to open with write, fail + if ((zp != NULL) && + (IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess&(FILE_WRITE_DATA | FILE_APPEND_DATA)) && + (zp->z_pflags&ZFS_READONLY)) { + VN_RELE(vp); + VN_RELE(dvp); + dprintf("%s: denied due to ZFS_READONLY + WRITE_DATA\n", + __func__); + return (STATUS_ACCESS_DENIED); + } + + + if (DeleteOnClose && + vp && zp && + dvp && VTOZ(dvp) && + zfs_zaccess_delete(VTOZ(dvp), zp, 0, NULL) > 0) { + VN_RELE(vp); + if (dvp) + VN_RELE(dvp); + + dprintf("%s: denied due to IMMUTABLE+NOUNLINK\n", + __func__); + return (STATUS_ACCESS_DENIED); + } + + + // Some cases we always create the file, and sometimes only if + // it is not there. If the file exists and we are only to create + // the file if it is not there: + if ((CreateDisposition == FILE_OPEN_IF) && (vp != NULL)) + CreateFile = 0; + + + if (vp || CreateFile == 0) { +// NTSTATUS Status; + + // Streams do not call SeAccessCheck? + if (stream_name != NULL && vp != NULL) { + IoSetShareAccess( + IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess, + IrpSp->Parameters.Create.ShareAccess, + FileObject, vp ? &vp->share_access : + &dvp->share_access); + + } else if ( + IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess != 0 && vp) { + + SeLockSubjectContext( + &IrpSp->Parameters.Create.SecurityContext-> + AccessState->SubjectSecurityContext); +#if 1 + if (!FileOpenReparsePoint && + !SeAccessCheck(vnode_security(vp ? vp : dvp), + &IrpSp->Parameters.Create.SecurityContext-> + AccessState->SubjectSecurityContext, + TRUE, + IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess, + 0, NULL, + IoGetFileObjectGenericMapping(), + IrpSp->Flags & SL_FORCE_ACCESS_CHECK ? UserMode : + Irp->RequestorMode, + &granted_access, &Status)) { + SeUnlockSubjectContext( + &IrpSp->Parameters.Create.SecurityContext-> + AccessState->SubjectSecurityContext); + if (vp) VN_RELE(vp); + VN_RELE(dvp); + dprintf("%s: denied due to SeAccessCheck()\n", + __func__); + return (Status); + } +#endif + SeUnlockSubjectContext( + &IrpSp->Parameters.Create.SecurityContext-> + AccessState->SubjectSecurityContext); + } else { + granted_access = 0; + } + + // Io*ShareAccess(): X is not an atomic operation. Therefore, + // drivers calling this routine must protect the shared + // file object + vnode_lock(vp ? vp : dvp); + if (vnode_isinuse(vp ? vp : dvp, 0)) { +// 0 is we are the only (usecount added below), 1+ if already open. + Status = IoCheckShareAccess(granted_access, + IrpSp->Parameters.Create.ShareAccess, FileObject, + vp ? &vp->share_access : &dvp->share_access, FALSE); + if (!NT_SUCCESS(Status)) { + vnode_unlock(vp ? vp : dvp); + if (vp) VN_RELE(vp); + VN_RELE(dvp); + dprintf("%s: denied IoCheckShareAccess\n", + __func__); + return (Status); + } + IoUpdateShareAccess(FileObject, + vp ? &vp->share_access : &dvp->share_access); + } else { + IoSetShareAccess(granted_access, + IrpSp->Parameters.Create.ShareAccess, + FileObject, + vp ? &vp->share_access : &dvp->share_access); + } + // Since we've updated ShareAccess here, if we cancel + // the open we need to undo it. + UndoShareAccess = TRUE; + vnode_unlock(vp ? vp : dvp); + } + +#define UNDO_SHARE_ACCESS(vp) \ + if ((vp) && UndoShareAccess) { \ + vnode_lock((vp)); \ + IoRemoveShareAccess(FileObject, &(vp)->share_access); \ + vnode_unlock((vp)); \ + } + + + // We can not DeleteOnClose if readonly filesystem + if (DeleteOnClose) { + if (zfsvfs->z_rdonly || vfs_isrdonly(zfsvfs->z_vfs) || + !spa_writeable(dmu_objset_spa(zfsvfs->z_os))) { + UNDO_SHARE_ACCESS(vp); + if (vp) VN_RELE(vp); + VN_RELE(dvp); + Irp->IoStatus.Information = 0; // ? + return (STATUS_MEDIA_WRITE_PROTECTED); + } + } + + if (CreateFile && finalname) { + int replacing = 0; + + if (zfsvfs->z_rdonly || vfs_isrdonly(zfsvfs->z_vfs) || + !spa_writeable(dmu_objset_spa(zfsvfs->z_os))) { + UNDO_SHARE_ACCESS(vp); + if (vp) VN_RELE(vp); + VN_RELE(dvp); + Irp->IoStatus.Information = 0; // ? + return (STATUS_MEDIA_WRITE_PROTECTED); + } + + // Would we replace file? + if (vp) { + VN_RELE(vp); + vp = NULL; + replacing = 1; + } + + vap->va_type = VREG; + if (!(vap->va_mask & ATTR_MODE)) + vap->va_mode = 0777 | S_IFREG; + vap->va_mask = (ATTR_MODE | ATTR_TYPE); + + // If O_TRUNC: + switch (CreateDisposition) { + case FILE_SUPERSEDE: + case FILE_OVERWRITE_IF: + case FILE_OVERWRITE: + vap->va_mask |= ATTR_SIZE; + vap->va_size = 0; + break; + } + + // O_EXCL only if FILE_CREATE + error = zfs_create(VTOZ(dvp), finalname, vap, + CreateDisposition == FILE_CREATE, vap->va_mode, + &zp, NULL, 0, NULL, NULL); + if (error == 0) { + boolean_t reenter_for_xattr = B_FALSE; + + // Creating two things? Don't attach until 2nd item. + if (!(zp->z_pflags & ZFS_XATTR) && stream_name != NULL) + reenter_for_xattr = B_TRUE; + + vp = ZTOV(zp); + + if (!reenter_for_xattr) { + zfs_couplefileobject(vp, FileObject, + zp ? zp->z_size : 0ULL); + vnode_ref(vp); + + if (DeleteOnClose) + Status = zfs_setunlink(FileObject, dvp); + } + + if (Status == STATUS_SUCCESS) { + + Irp->IoStatus.Information = replacing ? + CreateDisposition == FILE_SUPERSEDE ? + FILE_SUPERSEDED : FILE_OVERWRITTEN : + FILE_CREATED; + + // Update pflags, if needed + zfs_setwinflags(zp, + IrpSp->Parameters.Create.FileAttributes | + FILE_ATTRIBUTE_ARCHIVE); + + // Did they ask for an AllocationSize + if (Irp->Overlay.AllocationSize.QuadPart > 0) { + uint64_t allocsize = Irp-> + Overlay.AllocationSize.QuadPart; + // zp->z_blksz = + // P2ROUNDUP(allocsize, 512); + } + + vnode_lock(vp); + IoSetShareAccess( + IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess, + IrpSp->Parameters.Create.ShareAccess, + FileObject, + &vp->share_access); + vnode_unlock(vp); + + // Did we create file, or stream? + if (!(zp->z_pflags & ZFS_XATTR)) { + zfs_send_notify(zfsvfs, + zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_FILE_NAME, + FILE_ACTION_ADDED); + } else { + check_and_set_stream_parent(stream_name, + FileObject, + VTOZ(dvp)->z_xattr_parent); + + zfs_send_notify_stream(zfsvfs, // WOOT + zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_STREAM_NAME, + FILE_ACTION_ADDED_STREAM, + NULL); + } + + /* Windows lets you create a file, and stream, in one. */ + /* Call this function again, lets hope, only once */ + if (reenter_for_xattr) { + Status = EAGAIN; + } + + } + VN_RELE(vp); + VN_RELE(dvp); + + return (Status); + } + if (error == EEXIST) + Irp->IoStatus.Information = FILE_EXISTS; + else + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + + UNDO_SHARE_ACCESS(dvp); + VN_RELE(dvp); + return (STATUS_OBJECT_NAME_COLLISION); // create file error + } + + + // Just open it, if the open was to a directory, add ccb + ASSERT(IrpSp->FileObject->FsContext == NULL); + if (vp == NULL) { + zfs_couplefileobject(dvp, FileObject, 0ULL); + vnode_ref(dvp); // Hold open reference, until CLOSE + if (DeleteOnClose) + Status = zfs_setunlink(FileObject, dvp); + + if (Status == STATUS_SUCCESS) { + if (UndoShareAccess == FALSE) { + vnode_lock(dvp); + IoSetShareAccess( + IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess, + IrpSp->Parameters.Create.ShareAccess, + FileObject, + &dvp->share_access); + vnode_unlock(dvp); + } + } else { + UNDO_SHARE_ACCESS(dvp); + } + VN_RELE(dvp); + } else { + // Technically, this should call zfs_open() - + // but zfs_open is mostly empty + zfs_couplefileobject(vp, FileObject, zp->z_size); + vnode_ref(vp); // Hold open reference, until CLOSE + if (DeleteOnClose) + Status = zfs_setunlink(FileObject, dvp); + + if (Status == STATUS_SUCCESS) { + + Irp->IoStatus.Information = FILE_OPENED; + // Did they set the open flags (clearing archive?) + if (IrpSp->Parameters.Create.FileAttributes) + zfs_setwinflags(zp, + IrpSp->Parameters.Create.FileAttributes); + // If we are to truncate the file: + if (CreateDisposition == FILE_OVERWRITE) { + Irp->IoStatus.Information = FILE_OVERWRITTEN; + zp->z_pflags |= ZFS_ARCHIVE; + // zfs_freesp() path uses vnode_pager_setsize() + // so we need to make sure fileobject is set. + zfs_freesp(zp, 0, 0, FWRITE, B_TRUE); + // Did they ask for an AllocationSize + if (Irp->Overlay.AllocationSize.QuadPart > 0) { + uint64_t allocsize = Irp-> + Overlay.AllocationSize.QuadPart; + // zp->z_blksz = + // P2ROUNDUP(allocsize, 512); + } + } + // Update sizes in header. + vp->FileHeader.AllocationSize.QuadPart = + P2ROUNDUP(zp->z_size, zp->z_blksz); + vp->FileHeader.FileSize.QuadPart = zp->z_size; + vp->FileHeader.ValidDataLength.QuadPart = zp->z_size; + // If we created something new, add this permission + if (UndoShareAccess == FALSE) { + vnode_lock(vp); + IoSetShareAccess( + IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess, + IrpSp->Parameters.Create.ShareAccess, + FileObject, + &vp->share_access); + vnode_unlock(vp); + } + } else { + UNDO_SHARE_ACCESS(vp); + } + VN_RELE(vp); + VN_RELE(dvp); + } + + IrpSp->Parameters.Create.SecurityContext->AccessState-> + PreviouslyGrantedAccess |= granted_access; + IrpSp->Parameters.Create.SecurityContext->AccessState-> + RemainingDesiredAccess &= ~(granted_access | MAXIMUM_ALLOWED); + + return (Status); +} + +int +zfs_vnop_lookup(PIRP Irp, PIO_STACK_LOCATION IrpSp, mount_t *zmo) +{ + int status; + char *filename = NULL; + vattr_t vap = { 0 }; + + // Check the EA buffer is good, if supplied. + if (Irp->AssociatedIrp.SystemBuffer != NULL && + IrpSp->Parameters.Create.EaLength > 0) { + ULONG offset; + status = IoCheckEaBufferValidity( + Irp->AssociatedIrp.SystemBuffer, + IrpSp->Parameters.Create.EaLength, &offset); + if (!NT_SUCCESS(status)) { + dprintf("IoCheckEaBufferValidity returned %08x " + "(error at offset %lu)\n", status, offset); + return (status); + } + } + + // Allocate space to hold name, must be freed from here on + filename = kmem_alloc(PATH_MAX, KM_SLEEP); + + // Deal with ExtraCreateParameters +#if defined(NTDDI_WIN10_RS5) && (NTDDI_VERSION >= NTDDI_WIN10_RS5) + /* Check for ExtraCreateParameters */ + PECP_LIST ecp = NULL; + ATOMIC_CREATE_ECP_CONTECT *acec = NULL; + PQUERY_ON_CREATE_ECP_CONTEXT qocContext = NULL; + FsRtlGetEcpListFromIrp(Irp, &ecp); + if (ecp) { + GUID ecpType; + void *ecpContext = NULL; + ULONG ecpContextSize; + while (NT_SUCCESS(FsRtlGetNextExtraCreateParameter(ecp, + ecpContext, &ecpType, &ecpContext, &ecpContextSize))) { + if (IsEqualGUID(&ecpType, &GUID_ECP_ATOMIC_CREATE)) { + dprintf("GUID_ECP_ATOMIC_CREATE\n"); + // More code to come here: + acec = ecpContext; + } else if (IsEqualGUID(&ecpType, + &GUID_ECP_QUERY_ON_CREATE)) { + dprintf("GUID_ECP_QUERY_ON_CREATE\n"); + // It wants a getattr call on success, + // before we finish up + qocContext = + (PQUERY_ON_CREATE_ECP_CONTEXT)ecpContext; + } else if (IsEqualGUID(&ecpType, + &GUID_ECP_CREATE_REDIRECTION)) { + dprintf("GUID_ECP_CREATE_REDIRECTION\n"); + // We get this one a lot. + } else { + dprintf("Other GUID_ECP type\n"); +// IopSymlinkECPGuid "73d5118a-88ba-439f-92f4-46d38952d250" + } + }// while + } // if ecp +#endif + + // The associated buffer on a CreateFile is an EA buffer. + // Already Verified above - do a quickscan of any EAs we + // handle in a special way, before we call zfs_vnop_lookup_impl(). + // We handle the regular EAs afterward. + if (Irp->AssociatedIrp.SystemBuffer != NULL && + IrpSp->Parameters.Create.EaLength > 0) { + PFILE_FULL_EA_INFORMATION ea; + for (ea = + (PFILE_FULL_EA_INFORMATION)Irp->AssociatedIrp.SystemBuffer; + /* empty */; + ea = (PFILE_FULL_EA_INFORMATION)((uint8_t *)ea + + ea->NextEntryOffset)) { + // only parse $LX attrs right now -- things we can store + // before the file gets created. + if (vattr_apply_lx_ea(&vap, ea)) { + dprintf("encountered special attrs EA '%.*s'\n", + ea->EaNameLength, ea->EaName); + } + if (ea->NextEntryOffset == 0) + break; + } + } + + + do { + + // Call ZFS + status = zfs_vnop_lookup_impl(Irp, IrpSp, zmo, filename, &vap); + + } while (status == EAGAIN); + +#if defined(NTDDI_WIN10_RS5) && (NTDDI_VERSION >= NTDDI_WIN10_RS5) + // Did ECP ask for getattr to be returned? None, one or both can be set. + // This requires vnode_couplefileobject() was called + if (NT_SUCCESS(status) && qocContext && IrpSp->FileObject->FsContext) { + + ULONG classes = 0; + + // Handle RS5 >= version < 19H1 when the struct had "Flags". +#if defined(NTDDI_WIN10_19H1) && (NTDDI_VERSION >= NTDDI_WIN10_19H1) + classes = qocContext->RequestedClasses; +#else + classes = qocContext->Flags; +#endif + + if (BooleanFlagOn(classes, QoCFileStatInformation)) { + file_stat_information(IrpSp->DeviceObject, Irp, IrpSp, + &qocContext->StatInformation); + } + if (BooleanFlagOn(classes, QoCFileLxInformation)) { + file_stat_lx_information(IrpSp->DeviceObject, Irp, + IrpSp, &qocContext->LxInformation); + } + if (BooleanFlagOn(classes, QoCFileEaInformation)) { + dprintf("%s: unsupported QoC: QoCFileEaInformation\n"); + } +#if defined(NTDDI_WIN10_19H1) && (NTDDI_VERSION >= NTDDI_WIN10_19H1) + // We should fill this in, right? Only set those we understand. + qocContext->ClassesProcessed = + classes & (QoCFileStatInformation|QoCFileLxInformation); + qocContext->ClassesWithErrors = 0; + qocContext->ClassesWithNoData = 0; +#endif + + FsRtlAcknowledgeEcp(qocContext); + } + + if (NT_SUCCESS(status) && acec && acec-> + InFlags & ATOMIC_CREATE_ECP_IN_FLAG_REPARSE_POINT_SPECIFIED) { + panic("Implement me: atomic reparse point"); + // acec->OutFlags |= + // ATOMIC_CREATE_ECP_OUT_FLAG_REPARSE_POINT_SET; + } +#endif + + // Now handle proper EAs properly + if (NT_SUCCESS(status)) { + if (Irp->AssociatedIrp.SystemBuffer && + IrpSp->FileObject->FsContext) { + // Second pass: this will apply all EAs that are + // not only LX EAs + vnode_apply_eas(IrpSp->FileObject->FsContext, + (PFILE_FULL_EA_INFORMATION) + Irp->AssociatedIrp.SystemBuffer, + IrpSp->Parameters.Create.EaLength, NULL); + } + + if (!BooleanFlagOn(IrpSp->Parameters.Create.Options, + FILE_NO_INTERMEDIATE_BUFFERING)) { + IrpSp->FileObject->Flags |= FO_CACHE_SUPPORTED; + } + } + + // Free filename + kmem_free(filename, PATH_MAX); + + dprintf("%s: %s with %s\n", __func__, + common_status_str(status), + create_reply(status, Irp->IoStatus.Information)); + + return (status); +} + + +/* + * reclaim is called when a vnode is to be terminated, + * VFS (spl-vnode.c) will hold iocount == 1, usecount == 0 + * so release associated ZFS node, and free everything + */ +int +zfs_vnop_reclaim(struct vnode *vp) +{ + znode_t *zp = VTOZ(vp); + if (zp == NULL) { + ASSERT("NULL zp in reclaim?"); + return (0); + } + + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + dprintf(" zfs_vnop_recycle: releasing zp %p and vp %p: '%s'\n", zp, vp, + zp->z_name_cache ? zp->z_name_cache : ""); + + void *sd = vnode_security(vp); + if (sd != NULL) + ExFreePool(sd); + vnode_setsecurity(vp, NULL); + + // Decouple the nodes + ASSERT(ZTOV(zp) != (vnode_t *)0xdeadbeefdeadbeef); + + mutex_enter(&zp->z_lock); + ZTOV(zp) = NULL; + vnode_clearfsnode(vp); /* vp->v_data = NULL */ + mutex_exit(&zp->z_lock); + // vnode_removefsref(vp); /* ADDREF from vnode_create */ + + vp = NULL; + + if (zp->z_name_cache != NULL) + kmem_free(zp->z_name_cache, zp->z_name_len); + zp->z_name_cache = NULL; + zp->z_name_len = 0x12345678; // DBG: show we have been reclaimed + + // Release znode + /* + * This will release as much as it can, based on reclaim_reentry, + * if we are from fastpath, we do not call free here, as zfs_remove + * calls zfs_znode_delete() directly. + * zfs_zinactive() will leave earlier if z_reclaim_reentry is true. + */ + rw_enter(&zfsvfs->z_teardown_inactive_lock, RW_READER); + if (zp->z_sa_hdl == NULL) { + zfs_znode_free(zp); + } else { + zfs_zinactive(zp); + zfs_znode_free(zp); + } + rw_exit(&zfsvfs->z_teardown_inactive_lock); + + atomic_dec_64(&vnop_num_vnodes); + atomic_inc_64(&vnop_num_reclaims); + + if (vnop_num_vnodes % 1000 == 0) + dprintf("%s: num_vnodes %llu\n", __func__, vnop_num_vnodes); + + return (0); +} + + +/* + */ +void +getnewvnode_reserve(int num) +{ +} + +void +getnewvnode_drop_reserve() +{ +} + +/* + * Get new vnode for znode. + * + * This function uses zp->z_zfsvfs, zp->z_mode, zp->z_flags, zp->z_id + * and sets zp->z_vnode and zp->z_vid. + * If given parent, dzp, we can save some hassles. If not, looks it + * up internally. + */ +int +zfs_znode_getvnode(znode_t *zp, znode_t *dzp, zfsvfs_t *zfsvfs) +{ + struct vnode *vp = NULL; + int flags = 0; + // dprintf("getvnode zp %p with vp %p zfsvfs %p vfs %p\n", zp, vp, + // zfsvfs, zfsvfs->z_vfs); + + if (zp->z_vnode) + panic("zp %p vnode already set\n", zp->z_vnode); + + // "root" / mountpoint holds long term ref + if (zp->z_id == zfsvfs->z_root) { + flags |= VNODE_MARKROOT; + } + + /* + * vnode_create() has a habit of calling both vnop_reclaim() and + * vnop_fsync(), which can create havok as we are already holding locks. + */ + vnode_create(zfsvfs->z_vfs, zp, IFTOVT((mode_t)zp->z_mode), flags, &vp); + + atomic_inc_64(&vnop_num_vnodes); + + // dprintf("Assigned zp %p with vp %p\n", zp, vp); + zp->z_vid = vnode_vid(vp); + zp->z_vnode = vp; + + // Build a fullpath string here, for Notifications + // and set_name_information + ASSERT(zp->z_name_cache == NULL); + if (zfs_build_path(zp, dzp, &zp->z_name_cache, &zp->z_name_len, + &zp->z_name_offset) == -1) + dprintf("%s: failed to build fullpath\n", __func__); + + // Assign security here. But, if we are XATTR, we do not? In Windows, + // it refers to Streams and they do not have Security? + if (zp->z_pflags & ZFS_XATTR) + ; + else + zfs_set_security(vp, dzp && ZTOV(dzp) ? ZTOV(dzp) : NULL); + + return (0); +} + + +NTSTATUS +dev_ioctl(PDEVICE_OBJECT DeviceObject, ULONG ControlCode, PVOID InputBuffer, + ULONG InputBufferSize, PVOID OutputBuffer, ULONG OutputBufferSize, + BOOLEAN Override, IO_STATUS_BLOCK* iosb) +{ + PIRP Irp; + KEVENT Event; + NTSTATUS Status; + PIO_STACK_LOCATION Stack; + IO_STATUS_BLOCK IoStatus; + + KeInitializeEvent(&Event, NotificationEvent, FALSE); + + Irp = IoBuildDeviceIoControlRequest(ControlCode, + DeviceObject, + InputBuffer, + InputBufferSize, + OutputBuffer, + OutputBufferSize, + FALSE, + &Event, + &IoStatus); + + if (!Irp) + return (STATUS_INSUFFICIENT_RESOURCES); + + if (Override) { + Stack = IoGetNextIrpStackLocation(Irp); + Stack->Flags |= SL_OVERRIDE_VERIFY_VOLUME; + } + + Status = IoCallDriver(DeviceObject, Irp); + + if (Status == STATUS_PENDING) { + KeWaitForSingleObject(&Event, Executive, KernelMode, + FALSE, NULL); + Status = IoStatus.Status; + } + + if (iosb) + *iosb = IoStatus; + + return (Status); +} + +// THIS IS THE PNP DEVICE ID +NTSTATUS +pnp_query_id(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + mount_t *zmo; + + dprintf("%s: query id type %d\n", __func__, + IrpSp->Parameters.QueryId.IdType); + + zmo = (mount_t *)DeviceObject->DeviceExtension; + + Irp->IoStatus.Information = (ULONG_PTR)ExAllocatePoolWithTag(PagedPool, + zmo->bus_name.Length + sizeof (UNICODE_NULL), '!OIZ'); + if (Irp->IoStatus.Information == 0) + return (STATUS_NO_MEMORY); + + RtlCopyMemory((void *)Irp->IoStatus.Information, zmo->bus_name.Buffer, + zmo->bus_name.Length); + dprintf("replying with '%.*S'\n", (int)zmo->uuid.Length/sizeof (WCHAR), + Irp->IoStatus.Information); + + return (STATUS_SUCCESS); +} + +NTSTATUS +pnp_device_state(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + dprintf("%s:\n", __func__); + + Irp->IoStatus.Information |= PNP_DEVICE_NOT_DISABLEABLE; + + return (STATUS_SUCCESS); +} + +NTSTATUS +query_volume_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status; + Status = STATUS_NOT_IMPLEMENTED; + int space; + int error = 0; + + mount_t *zmo = DeviceObject->DeviceExtension; + if (!zmo || + (zmo->type != MOUNT_TYPE_VCB && + zmo->type != MOUNT_TYPE_DCB)) { + return (STATUS_INVALID_PARAMETER); + } + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); // This returns EIO if fail + + switch (IrpSp->Parameters.QueryVolume.FsInformationClass) { + + case FileFsAttributeInformation: + // + // If overflow, set Information to input_size and NameLength + // to what we fit. + // + + dprintf("* %s: FileFsAttributeInformation\n", __func__); + if (IrpSp->Parameters.QueryVolume.Length < + sizeof (FILE_FS_ATTRIBUTE_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_FS_ATTRIBUTE_INFORMATION); + Status = STATUS_BUFFER_TOO_SMALL; + break; + } + +/* Do not enable until we have implemented FileRenameInformationEx method. */ +#if (NTDDI_VERSION >= NTDDI_WIN10_RS1) +#define ZFS_FS_ATTRIBUTE_POSIX +#endif + +#define ZFS_FS_ATTRIBUTE_CLEANUP_INFO + + FILE_FS_ATTRIBUTE_INFORMATION *ffai = + Irp->AssociatedIrp.SystemBuffer; + ffai->FileSystemAttributes = + FILE_CASE_PRESERVED_NAMES | FILE_NAMED_STREAMS | + FILE_PERSISTENT_ACLS | FILE_SUPPORTS_OBJECT_IDS | + FILE_SUPPORTS_SPARSE_FILES | FILE_VOLUME_QUOTAS | + FILE_SUPPORTS_REPARSE_POINTS | FILE_UNICODE_ON_DISK | + FILE_SUPPORTS_HARD_LINKS | FILE_SUPPORTS_OPEN_BY_FILE_ID | + FILE_SUPPORTS_EXTENDED_ATTRIBUTES | + FILE_CASE_SENSITIVE_SEARCH; +#if defined(ZFS_FS_ATTRIBUTE_POSIX) + ffai->FileSystemAttributes |= FILE_SUPPORTS_POSIX_UNLINK_RENAME; +#endif +#if defined(ZFS_FS_ATTRIBUTE_CLEANUP_INFO) + ffai->FileSystemAttributes |= FILE_RETURNS_CLEANUP_RESULT_INFO; +#endif +#if defined(FILE_SUPPORTS_BLOCK_REFCOUNTING) + /* Block-cloning, from FSCTL_DUPLICATE_EXTENTS */ + ffai->FileSystemAttributes |= FILE_SUPPORTS_BLOCK_REFCOUNTING; +#endif + + /* + * NTFS has these: + * FILE_CASE_SENSITIVE_SEARCH | FILE_FILE_COMPRESSION | + * FILE_RETURNS_CLEANUP_RESULT_INFO | + * FILE_SUPPORTS_POSIX_UNLINK_RENAME | + * FILE_SUPPORTS_ENCRYPTION | FILE_SUPPORTS_TRANSACTIONS | + * FILE_SUPPORTS_USN_JOURNAL; + */ + + if (zfsvfs->z_case == ZFS_CASE_SENSITIVE) + ffai->FileSystemAttributes |= + FILE_CASE_SENSITIVE_SEARCH; + + if (zfsvfs->z_rdonly) { + SetFlag(ffai->FileSystemAttributes, + FILE_READ_ONLY_VOLUME); + } + ffai->MaximumComponentNameLength = MAXNAMELEN - 1; + // ffai->FileSystemAttributes = 0x3E706FF; // ntfs 2023 + + // There is room for one char in the struct + // Alas, many things compare string to "NTFS". + space = IrpSp->Parameters.QueryVolume.Length - + FIELD_OFFSET(FILE_FS_ATTRIBUTE_INFORMATION, FileSystemName); + + UNICODE_STRING name; + RtlInitUnicodeString(&name, L"NTFS"); + + space = MIN(space, name.Length); + ffai->FileSystemNameLength = name.Length; + RtlCopyMemory(ffai->FileSystemName, name.Buffer, space); + Irp->IoStatus.Information = + FIELD_OFFSET(FILE_FS_ATTRIBUTE_INFORMATION, + FileSystemName) + space; + + if (space < name.Length) + Status = STATUS_BUFFER_OVERFLOW; + else + Status = STATUS_SUCCESS; + + ASSERT(Irp->IoStatus.Information <= + IrpSp->Parameters.QueryVolume.Length); + break; + case FileFsControlInformation: + dprintf("* %s: FileFsControlInformation NOT IMPLEMENTED\n", + __func__); + break; + case FileFsDeviceInformation: + dprintf("* %s: FileFsDeviceInformation NOT IMPLEMENTED\n", + __func__); + break; + case FileFsDriverPathInformation: + dprintf("* %s: FileFsDriverPathInformation NOT IMPLEMENTED\n", + __func__); + break; + case FileFsFullSizeInformation: + dprintf("* %s: FileFsFullSizeInformation\n", __func__); + if (IrpSp->Parameters.QueryVolume.Length < + sizeof (FILE_FS_FULL_SIZE_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_FS_FULL_SIZE_INFORMATION); + Status = STATUS_BUFFER_TOO_SMALL; + break; + } + uint64_t refdbytes, availbytes, usedobjs, availobjs; + dmu_objset_space(zfsvfs->z_os, + &refdbytes, &availbytes, &usedobjs, &availobjs); + + FILE_FS_FULL_SIZE_INFORMATION *fffsi = + Irp->AssociatedIrp.SystemBuffer; + fffsi->TotalAllocationUnits.QuadPart = + (refdbytes + availbytes) / 512ULL; + fffsi->ActualAvailableAllocationUnits.QuadPart = + availbytes / 512ULL; + fffsi->CallerAvailableAllocationUnits.QuadPart = + availbytes / 512ULL; + fffsi->BytesPerSector = 512; + fffsi->SectorsPerAllocationUnit = 1; + Irp->IoStatus.Information = + sizeof (FILE_FS_FULL_SIZE_INFORMATION); + Status = STATUS_SUCCESS; + break; + case FileFsObjectIdInformation: + dprintf("* %s: FileFsObjectIdInformation\n", __func__); + FILE_FS_OBJECTID_INFORMATION* ffoi = + Irp->AssociatedIrp.SystemBuffer; + // RtlCopyMemory(ffoi->ObjectId, &Vcb->superblock.uuid.uuid[0], + // sizeof (UCHAR) * 16); + RtlZeroMemory(ffoi->ExtendedInfo, sizeof (ffoi->ExtendedInfo)); + Irp->IoStatus.Information = + sizeof (FILE_FS_OBJECTID_INFORMATION); + Status = STATUS_OBJECT_NAME_NOT_FOUND; // returned by NTFS + break; + case FileFsVolumeInformation: + dprintf("* %s: FileFsVolumeInformation\n", __func__); + if (IrpSp->Parameters.QueryVolume.Length < + sizeof (FILE_FS_VOLUME_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_FS_VOLUME_INFORMATION); + Status = STATUS_BUFFER_TOO_SMALL; + break; + } + FILE_FS_VOLUME_INFORMATION *ffvi = + Irp->AssociatedIrp.SystemBuffer; + TIME_UNIX_TO_WINDOWS_EX(zfsvfs->z_last_unmount_time, 0, + ffvi->VolumeCreationTime.QuadPart); + ffvi->VolumeSerialNumber = 0x19831116; + ffvi->SupportsObjects = TRUE; + ffvi->VolumeLabelLength = + zmo->name.Length; + + int space = + IrpSp->Parameters.QueryFile.Length - + FIELD_OFFSET(FILE_FS_VOLUME_INFORMATION, VolumeLabel); + space = MIN(space, ffvi->VolumeLabelLength); + + /* + * This becomes the name displayed in Explorer, so we return the + * dataset name here, as much as we can + */ + RtlCopyMemory(ffvi->VolumeLabel, zmo->name.Buffer, space); + + Irp->IoStatus.Information = + FIELD_OFFSET(FILE_FS_VOLUME_INFORMATION, + VolumeLabel) + space; + + if (space < ffvi->VolumeLabelLength) + Status = STATUS_BUFFER_OVERFLOW; + else + Status = STATUS_SUCCESS; + + break; + case FileFsSizeInformation: + dprintf("* %s: FileFsSizeInformation\n", __func__); + if (IrpSp->Parameters.QueryVolume.Length < + sizeof (FILE_FS_SIZE_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_FS_SIZE_INFORMATION); + Status = STATUS_BUFFER_TOO_SMALL; + break; + } + + FILE_FS_SIZE_INFORMATION *ffsi = + Irp->AssociatedIrp.SystemBuffer; + ffsi->TotalAllocationUnits.QuadPart = 1024 * 1024 * 1024; + ffsi->AvailableAllocationUnits.QuadPart = 1024 * 1024 * 1024; + ffsi->SectorsPerAllocationUnit = 1; + ffsi->BytesPerSector = 512; + Irp->IoStatus.Information = sizeof (FILE_FS_SIZE_INFORMATION); + Status = STATUS_SUCCESS; + break; + case FileFsSectorSizeInformation: + dprintf("* %s: FileFsSectorSizeInformation\n", __func__); + if (IrpSp->Parameters.QueryVolume.Length < + sizeof (FILE_FS_SECTOR_SIZE_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_FS_SECTOR_SIZE_INFORMATION); + Status = STATUS_BUFFER_TOO_SMALL; + break; + } + FILE_FS_SECTOR_SIZE_INFORMATION *ffssi = + Irp->AssociatedIrp.SystemBuffer; + ffssi->LogicalBytesPerSector = 512; + ffssi->PhysicalBytesPerSectorForAtomicity = 512; + ffssi->PhysicalBytesPerSectorForPerformance = 512; + ffssi->FileSystemEffectivePhysicalBytesPerSectorForAtomicity = + 512; + ffssi->Flags = SSINFO_FLAGS_NO_SEEK_PENALTY; + ffssi->ByteOffsetForSectorAlignment = SSINFO_OFFSET_UNKNOWN; + ffssi->ByteOffsetForPartitionAlignment = SSINFO_OFFSET_UNKNOWN; + Irp->IoStatus.Information = + sizeof (FILE_FS_SECTOR_SIZE_INFORMATION); + Status = STATUS_SUCCESS; + break; + default: + dprintf("* %s: unknown class 0x%x\n", __func__, + IrpSp->Parameters.QueryVolume.FsInformationClass); + Status = STATUS_NOT_IMPLEMENTED; + break; + } + zfs_exit(zfsvfs, FTAG); + return (Status); +} + + +NTSTATUS +lock_control(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_SUCCESS; + + dprintf("%s: FileObject %p flags 0x%x %s %s\n", __func__, + IrpSp->FileObject, IrpSp->Flags, + IrpSp->Flags & SL_EXCLUSIVE_LOCK ? "Exclusive" : "Shared", + IrpSp->Flags & SL_FAIL_IMMEDIATELY ? "Nowait" : "Wait"); + + return (Status); +} + +NTSTATUS +query_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_NOT_IMPLEMENTED; + ULONG usedspace = 0; + struct vnode *vp = NULL; + int normalize = 0; + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + vp = IrpSp->FileObject->FsContext; + if (VN_HOLD(vp) != 0) + return (STATUS_INVALID_PARAMETER); + } + + switch (IrpSp->Parameters.QueryFile.FileInformationClass) { + + case FileAllInformation: + dprintf("%s: FileAllInformation: buffer 0x%lx\n", __func__, + IrpSp->Parameters.QueryFile.Length); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_ALL_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_ALL_INFORMATION); +// We should send Plus Filename here, to be nice, but this doesnt happen + Status = STATUS_BUFFER_TOO_SMALL; + break; + } + FILE_ALL_INFORMATION *all = Irp->AssociatedIrp.SystemBuffer; + + // Even if the name does not fit, the other information + // should be correct + Status = file_basic_information(DeviceObject, Irp, IrpSp, + &all->BasicInformation); + if (Status != STATUS_SUCCESS) + break; + Status = file_standard_information(DeviceObject, Irp, IrpSp, + &all->StandardInformation); + if (Status != STATUS_SUCCESS) + break; + Status = file_position_information(DeviceObject, Irp, IrpSp, + &all->PositionInformation); + if (Status != STATUS_SUCCESS) + break; + Status = file_ea_information(DeviceObject, Irp, IrpSp, + &all->EaInformation); + if (Status != STATUS_SUCCESS) + break; +#if 1 + zfs_dirlist_t *zccb = IrpSp->FileObject->FsContext2; + all->AccessInformation.AccessFlags = + GENERIC_ALL | GENERIC_EXECUTE | + GENERIC_READ | GENERIC_WRITE; + all->ModeInformation.Mode = + zccb && zccb->deleteonclose ? FILE_DELETE_ON_CLOSE : 0; +#endif + Status = file_alignment_information(DeviceObject, Irp, IrpSp, + &all->AlignmentInformation); + if (Status != STATUS_SUCCESS) + break; + + Status = file_internal_information(DeviceObject, Irp, IrpSp, + &all->InternalInformation); + if (Status != STATUS_SUCCESS) + break; + + // First get the Name, to make sure we have room + IrpSp->Parameters.QueryFile.Length -= + offsetof(FILE_ALL_INFORMATION, NameInformation); + Status = file_name_information(DeviceObject, Irp, IrpSp, + &all->NameInformation, &usedspace, 0); + IrpSp->Parameters.QueryFile.Length += + offsetof(FILE_ALL_INFORMATION, NameInformation); + + // file_name_information sets FileNameLength, so update size + // to be ALL struct not NAME struct + // However, there is room for one char in the struct, + // so subtract that from total. + Irp->IoStatus.Information = + FIELD_OFFSET(FILE_ALL_INFORMATION, NameInformation) + + FIELD_OFFSET(FILE_NAME_INFORMATION, FileName) + + usedspace; + // FIELD_OFFSET(FILE_ALL_INFORMATION, NameInformation.FileName) + // + usedspace; + + dprintf("Struct size 0x%x FileNameLen 0x%lx " + "Information retsize 0x%lx\n", + (int)sizeof (FILE_ALL_INFORMATION), + all->NameInformation.FileNameLength, + Irp->IoStatus.Information); + break; + case FileAttributeTagInformation: + Status = file_attribute_tag_information(DeviceObject, Irp, + IrpSp, Irp->AssociatedIrp.SystemBuffer); + break; + case FileBasicInformation: + Status = file_basic_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileCompressionInformation: + Status = file_compression_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileEaInformation: + Status = file_ea_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileInternalInformation: + Status = file_internal_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileNormalizedNameInformation: + dprintf("FileNormalizedNameInformation\n"); + /* + * According to chatGPT, the difference between FileNameInformation + * and FileNormalizedNameInformation is that the latter will + * return a more "portable" name. For example; + * "My Photos (2022)" -> "my_photos_2022", as the FS desires. + * In this example, unified case, no spaces and limited charset. + * + * The complications start when the normalized name is passed to + * lookup (CreateFile->zfs_vnop_lookup()) as it is expected to + * work. Uniqueness would have to be guaranteed (per directory). + * And filename matching would be more complicated. + * + * For now, let's return identical names for Normalized. + * + */ + + normalize = 1; + + zfs_fallthrough; + case FileNameInformation: + // + // If overflow, set Information to input_size and NameLength + // to required size. + // + Status = file_name_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer, &usedspace, normalize); + Irp->IoStatus.Information = + FIELD_OFFSET(FILE_NAME_INFORMATION, FileName) + usedspace; + break; + case FileNetworkOpenInformation: + Status = file_network_open_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FilePositionInformation: + Status = file_position_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileStandardInformation: + Status = file_standard_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileAlignmentInformation: + Status = file_alignment_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileStreamInformation: + Status = file_stream_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileHardLinkInformation: + Status = file_hard_link_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + // Not used - not handled by ntfs either + case FileRemoteProtocolInformation: + dprintf("* %s: FileRemoteProtocolInformation NOT IMPLEMENTED\n", + __func__); +#if 0 + Status = file_remote_protocol_information(DeviceObject, Irp, + IrpSp, Irp->AssociatedIrp.SystemBuffer); +#endif + Status = STATUS_INVALID_PARAMETER; + break; + case FileStandardLinkInformation: + Status = file_standard_link_information(DeviceObject, Irp, + IrpSp, Irp->AssociatedIrp.SystemBuffer); + break; + case FileReparsePointInformation: + break; + case FileIdInformation: + Status = file_id_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + break; + case FileCaseSensitiveInformation: + Status = file_case_sensitive_information(DeviceObject, Irp, + IrpSp, Irp->AssociatedIrp.SystemBuffer); + break; + case FileStatInformation: + // We call these functions from zfs_vnop_lookup, so size + // testing goes here + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_STAT_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_STAT_INFORMATION); + Status = STATUS_BUFFER_TOO_SMALL; + break; + } + Status = file_stat_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + Irp->IoStatus.Information = sizeof (FILE_STAT_INFORMATION); + break; + case FileStatLxInformation: + // We call these functions from zfs_vnop_lookup, so size + // testing goes here + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_STAT_LX_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_STAT_LX_INFORMATION); + Status = STATUS_BUFFER_TOO_SMALL; + break; + } + Status = file_stat_lx_information(DeviceObject, Irp, IrpSp, + Irp->AssociatedIrp.SystemBuffer); + Irp->IoStatus.Information = sizeof (FILE_STAT_LX_INFORMATION); + break; + default: + dprintf("* %s: unknown class 0x%x NOT IMPLEMENTED\n", __func__, + IrpSp->Parameters.QueryFile.FileInformationClass); + break; + } + + if (vp) { + VN_RELE(vp); + vp = NULL; + } + return (Status); +} + +PVOID +MapUserBuffer(IN OUT PIRP Irp) +{ + // + // If there is no Mdl, then we must be in the Fsd, and we can simply + // return the UserBuffer field from the Irp. + // + if (Irp->MdlAddress == NULL) { + return (Irp->UserBuffer); + } else { + PVOID Address = MmGetSystemAddressForMdlSafe(Irp->MdlAddress, + NormalPagePriority | MdlMappingNoExecute); + return (Address); + } +} + +PVOID +BufferUserBuffer(IN OUT PIRP Irp, IN ULONG BufferLength) +{ + PUCHAR UserBuffer; + if (BufferLength == 0) + return (NULL); + + // + // If there is no system buffer we must have been supplied an Mdl + // describing the users input buffer, which we will now snapshot. + // + if (Irp->AssociatedIrp.SystemBuffer == NULL) { + UserBuffer = MapUserBuffer(Irp); + Irp->AssociatedIrp.SystemBuffer = + FsRtlAllocatePoolWithQuotaTag(NonPagedPoolNx, + BufferLength, + 'qtaf'); + // + // Set the flags so that the completion code knows to + // deallocate the buffer. + // + Irp->Flags |= (IRP_BUFFERED_IO | IRP_DEALLOCATE_BUFFER); + + try { + RtlCopyMemory(Irp->AssociatedIrp.SystemBuffer, + UserBuffer, + BufferLength); + } except(EXCEPTION_EXECUTE_HANDLER) { + NTSTATUS Status; + Status = GetExceptionCode(); + } + } + return (Irp->AssociatedIrp.SystemBuffer); +} + +/* + * ** This is how I thought it worked: + * Iterate through the XATTRs of an object, skipping streams. It works + * like readdir, with saving index point, restart_scan and single_entry flags. + * It can optionally supply QueryEa.EaList to query specific set of EAs. + * Each output structure is 4 byte aligned + * + * 1: While EAs fit, including name and value-data, we keep packing them + * in. If we have a non-zero number of valid EAs in the output buffer, + * we return STATUS_SUCCESS, and IoStatus.Information is set to the + * number of bytes in the output buffer. + * + * 2: If we can't fit (the next) EA at all, and there are no prior valid + * EAs (they would handled by 1: above) we return STATUS_BUFFER_OVERFLOW + * and IoStatus.Information is set to what it needs to fit it. + * + * 3: If we finished with all EAs, we return the valid records + * with status STATUS_NO_MORE_EAS. + * + * + * ** But it actually wants everything in one-shot, fit all in the buffer + * or return STATUS_BUFFER_OVERFLOW. + */ +NTSTATUS +query_ea(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_SUCCESS; + + PUCHAR Buffer; + ULONG UserBufferLength; + + PUCHAR UserEaList; + ULONG UserEaListLength; + ULONG UserEaIndex; + BOOLEAN RestartScan; + BOOLEAN ReturnSingleEntry; + BOOLEAN IndexSpecified; + FILE_FULL_EA_INFORMATION *previous_ea = NULL; + uint64_t spaceused = 0; + znode_t *zp = NULL, *xdzp = NULL; + zfsvfs_t *zfsvfs = NULL; + zap_cursor_t zc; + zap_attribute_t za; + int overflow = 0; + + struct vnode *vp = NULL, *xdvp = NULL; + + if (IrpSp->FileObject == NULL) + return (STATUS_INVALID_PARAMETER); + vp = IrpSp->FileObject->FsContext; + if (vp == NULL) + return (STATUS_INVALID_PARAMETER); + + zp = VTOZ(vp); + zfsvfs = zp->z_zfsvfs; + + UserBufferLength = IrpSp->Parameters.QueryEa.Length; + UserEaList = IrpSp->Parameters.QueryEa.EaList; + UserEaListLength = IrpSp->Parameters.QueryEa.EaListLength; + UserEaIndex = IrpSp->Parameters.QueryEa.EaIndex; + RestartScan = BooleanFlagOn(IrpSp->Flags, SL_RESTART_SCAN); + ReturnSingleEntry = BooleanFlagOn(IrpSp->Flags, SL_RETURN_SINGLE_ENTRY); + IndexSpecified = BooleanFlagOn(IrpSp->Flags, SL_INDEX_SPECIFIED); + + dprintf("%s\n", __func__); + + Buffer = MapUserBuffer(Irp); + + if (UserBufferLength < sizeof (FILE_FULL_EA_INFORMATION)) { + + if (UserBufferLength == 0) { + Irp->IoStatus.Information = 0; + return (STATUS_NO_MORE_EAS); + } + + Irp->IoStatus.Information = sizeof (FILE_FULL_EA_INFORMATION); + return (STATUS_BUFFER_OVERFLOW); + // Docs say to return too-small, but some callers get stuck + // calling this in a cpu loop if we return it. + return (STATUS_BUFFER_TOO_SMALL); + } + + znode_t *xzp = NULL; + FILE_GET_EA_INFORMATION *ea; + int error = 0; + + uint64_t start_index = 0; + + zfs_dirlist_t *zccb = IrpSp->FileObject->FsContext2; + + if (RestartScan) { + start_index = 0; + zccb->ea_index = 0; + } else if (IndexSpecified) + start_index = UserEaIndex; + else + start_index = zccb->ea_index; + + struct iovec iov; + iov.iov_base = + (void *)Buffer; + iov.iov_len = UserBufferLength; + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, 0, + UIO_SYSSPACE, UserBufferLength, 0); + + // Pass Flags along for ReturnSingleEntry, so + // lets abuse uio->extflg - no idea what it is for + // it got copied across to Windows so it's there. + uio.uio_extflg = IrpSp->Flags; + + /* ********************** */ + if (UserEaList != NULL) { + + uint64_t offset = 0; + uint64_t current_index = 0; + + do { + /* bounds check: offset is on INPUT list */ + if (offset > UserEaListLength) { + if (xdvp) VN_RELE(xdvp); + return (STATUS_INVALID_PARAMETER); + } + + ea = (FILE_GET_EA_INFORMATION *)&UserEaList[offset]; + + if (offset + ea->EaNameLength > UserEaListLength) { + if (xdvp) VN_RELE(xdvp); + return (STATUS_INVALID_PARAMETER); + } + + /* scan until we get to the index wanted */ + if (current_index >= start_index) { + + error = zpl_xattr_filldir(vp, &uio, ea->EaName, + ea->EaNameLength, &previous_ea); + + if (error == ENOENT) + error = 0; + else if (error != 0) + break; + + if (ReturnSingleEntry) { + current_index++; + break; + } + } + // if (overflow != 0) + // break; + + current_index++; + offset += ea->NextEntryOffset; + + if (ea->NextEntryOffset == 0) + break; + + } while (offset != 0); + + if (current_index >= start_index) + zccb->ea_index = current_index; + + /* ********************** */ + } else { + + zfs_uio_setindex(&uio, start_index); + Status = zpl_xattr_list(vp, &uio, &spaceused, NULL); + zccb->ea_index = zfs_uio_index(&uio); + } + + +out: + + if (xdvp) VN_RELE(xdvp); + + Irp->IoStatus.Information = spaceused; + + // Didn't fit even one + if (overflow) + Status = STATUS_BUFFER_OVERFLOW; + else if (spaceused == 0 && Status == 0) + Status = STATUS_NO_EAS_ON_FILE; + + return (Status); +} + +/* + * Receive an array of structs to set EAs, iterate until Next is null. + */ +NTSTATUS +set_ea(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + uint32_t input_len = IrpSp->Parameters.SetEa.Length; + uint8_t *buffer = NULL; + NTSTATUS Status = STATUS_SUCCESS; + struct vnode *vp = NULL; + + if (IrpSp->FileObject == NULL) + return (STATUS_INVALID_PARAMETER); + + vp = IrpSp->FileObject->FsContext; + if (vp == NULL) + return (STATUS_INVALID_PARAMETER); + + dprintf("%s\n", __func__); + + if (input_len == 0) + return (STATUS_INVALID_PARAMETER); + + mount_t *zmo = DeviceObject->DeviceExtension; + zfsvfs_t *zfsvfs; + if (zmo != NULL && + (zfsvfs = vfs_fsprivate(zmo)) != NULL && + zfsvfs->z_rdonly) + return (STATUS_MEDIA_WRITE_PROTECTED); + + // This magic is straight out of fastfat + buffer = BufferUserBuffer(Irp, input_len); + + ULONG eaErrorOffset = 0; + Status = vnode_apply_eas(vp, (PFILE_FULL_EA_INFORMATION)buffer, + input_len, &eaErrorOffset); + // (Information is ULONG_PTR; as win64 is a LLP64 platform, + // ULONG isn't the right length.) + Irp->IoStatus.Information = 0; + if (!NT_SUCCESS(Status)) { + dprintf("%s: failed vnode_apply_eas: 0x%lx\n", + __func__, Status); + return (Status); + } + return (Status); +} + +size_t +get_reparse_point_impl(znode_t *zp, char *buffer, size_t outlen) +{ + size_t size = 0; + if (zp->z_pflags & ZFS_REPARSE) { + int err; + + if (zfsctl_is_node(zp)) { + REPARSE_DATA_BUFFER *rdb = NULL; + NTSTATUS Status; + Status = zfsctl_get_reparse_point(zp, &rdb, &size); + if (Status == 0) + memcpy(buffer, rdb, size); + } else { + int size = MIN(zp->z_size, outlen); + struct iovec iov; + iov.iov_base = (void *)buffer; + iov.iov_len = size; + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, 0, UIO_SYSSPACE, + size, 0); + err = zfs_readlink(ZTOV(zp), &uio, NULL); + } + } + return (size); +} + +NTSTATUS +get_reparse_point(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_NOT_A_REPARSE_POINT; + PFILE_OBJECT FileObject = IrpSp->FileObject; + DWORD outlen = IrpSp->Parameters.FileSystemControl.OutputBufferLength; + void *buffer = Irp->AssociatedIrp.SystemBuffer; + struct vnode *vp; + + if (FileObject == NULL) + return (STATUS_INVALID_PARAMETER); + + vp = FileObject->FsContext; + + if (vp) { + VN_HOLD(vp); + znode_t *zp = VTOZ(vp); + + if (zp->z_pflags & ZFS_REPARSE) { + int err; + size_t size = 0; + + size = get_reparse_point_impl(zp, buffer, outlen); + Irp->IoStatus.Information = size; + if (outlen < size) + Status = STATUS_BUFFER_OVERFLOW; + else + Status = STATUS_SUCCESS; + } + VN_RELE(vp); + } + dprintf("%s: returning 0x%lx\n", __func__, Status); + return (Status); +} + +NTSTATUS +set_reparse_point(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_SUCCESS; + PFILE_OBJECT FileObject = IrpSp->FileObject; + DWORD inlen = IrpSp->Parameters.DeviceIoControl.InputBufferLength; + void *buffer = Irp->AssociatedIrp.SystemBuffer; + REPARSE_DATA_BUFFER *rdb = buffer; + + if (!FileObject || !IrpSp->FileObject->FsContext) + return (STATUS_INVALID_PARAMETER); + + struct vnode *vp = IrpSp->FileObject->FsContext; + + if (!vp || !VTOZ(vp)) + return (STATUS_INVALID_PARAMETER); + + if (Irp->UserBuffer) + return (STATUS_INVALID_PARAMETER); + + if (inlen < sizeof (ULONG)) { + return (STATUS_INVALID_BUFFER_SIZE); + } + + Status = FsRtlValidateReparsePointBuffer(inlen, rdb); + if (!NT_SUCCESS(Status)) { + dprintf("FsRtlValidateReparsePointBuffer returned %08lx\n", + Status); + return (Status); + } + + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + if (zfsctl_is_node(zp)) + return (zfsctl_set_reparse_point(zp, rdb, inlen)); + + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if (zfsvfs->z_rdonly) + return (STATUS_MEDIA_WRITE_PROTECTED); + + VN_HOLD(vp); + znode_t *dzp = NULL; + uint64_t parent; + int error; + + // Fetch parent + VERIFY(sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)) == 0); + error = zfs_zget(zfsvfs, parent, &dzp); + if (error) { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + + // winbtrfs' test/exe will trigger this, add code here. + // (asked to create reparse point on already reparse point) + if (zp->z_pflags & ZFS_REPARSE) { +// DbgBreakPoint(); + } + // error = zfs_symlink(dzp, , vattr_t * vap, char *link, + // znode_t * *zpp, cred_t * cr, int flags) + + + // Like zfs_symlink, write the data as SA attribute. + dmu_tx_t *tx; + boolean_t fuid_dirtied; + + // Set flags to indicate we are reparse point + zp->z_pflags |= ZFS_REPARSE; + + // Start TX and save FLAGS, SIZE and SYMLINK to disk. + // This code should probably call zfs_symlink() +top: + tx = dmu_tx_create(zfsvfs->z_os); + fuid_dirtied = zfsvfs->z_fuid_dirty; + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, MAX(1, inlen)); + dmu_tx_hold_zap(tx, dzp->z_id, TRUE, NULL); + dmu_tx_hold_sa_create(tx, inlen); + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + if (fuid_dirtied) + zfs_fuid_txhold(zfsvfs, tx); + + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + if (error == ERESTART) + goto top; + goto out; + } + + (void) sa_update(zp->z_sa_hdl, SA_ZPL_FLAGS(zfsvfs), + &zp->z_pflags, sizeof (zp->z_pflags), tx); + + mutex_enter(&zp->z_lock); + if (zp->z_is_sa) + error = sa_update(zp->z_sa_hdl, SA_ZPL_SYMLINK(zfsvfs), + buffer, inlen, tx); + else + zfs_sa_symlink(zp, buffer, inlen, tx); + mutex_exit(&zp->z_lock); + + zp->z_size = inlen; + (void) sa_update(zp->z_sa_hdl, SA_ZPL_SIZE(zfsvfs), + &zp->z_size, sizeof (zp->z_size), tx); + + dmu_tx_commit(tx); + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zfsvfs->z_log, 0); + +out: + if (dzp) + zrele(dzp); + VN_RELE(vp); + + dprintf("%s: returning 0x%lx\n", __func__, Status); + + return (Status); +} + +NTSTATUS +delete_reparse_point(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_SUCCESS; + PFILE_OBJECT FileObject = IrpSp->FileObject; + DWORD inlen = IrpSp->Parameters.DeviceIoControl.InputBufferLength; + void *buffer = Irp->AssociatedIrp.SystemBuffer; + REPARSE_DATA_BUFFER *rdb = buffer; + struct vnode *vp = IrpSp->FileObject->FsContext; + + if (!FileObject) + return (STATUS_INVALID_PARAMETER); + + if (Irp->UserBuffer) + return (STATUS_INVALID_PARAMETER); + + if (inlen < sizeof (ULONG)) { + return (STATUS_INVALID_BUFFER_SIZE); + } + + if (inlen < offsetof(REPARSE_DATA_BUFFER, + GenericReparseBuffer.DataBuffer)) + return (STATUS_INVALID_PARAMETER); + + if (rdb->ReparseDataLength > 0) + return (STATUS_INVALID_PARAMETER); + + if (VN_HOLD(vp) != 0) + return (STATUS_INVALID_PARAMETER); + + znode_t *zp = VTOZ(vp); + + if (zfsctl_is_node(zp)) { + VN_RELE(vp); + return (zfsctl_delete_reparse_point(zp)); + } + // Like zfs_symlink, write the data as SA attribute. + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + znode_t *dzp = NULL; + uint64_t parent; + int error; + + // Fetch parent + VERIFY(sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)) == 0); + error = zfs_zget(zfsvfs, parent, &dzp); + if (error) { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + + dmu_tx_t *tx; + boolean_t fuid_dirtied; + + // Remove flags to indicate we are reparse point + zp->z_pflags &= ~ZFS_REPARSE; + + // Start TX and save FLAGS, SIZE and SYMLINK to disk. + // This code should probably call zfs_symlink() +top: + tx = dmu_tx_create(zfsvfs->z_os); + fuid_dirtied = zfsvfs->z_fuid_dirty; + + dmu_tx_hold_zap(tx, dzp->z_id, FALSE, NULL); // name + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + zfs_sa_upgrade_txholds(tx, zp); + zfs_sa_upgrade_txholds(tx, dzp); + + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + if (error == ERESTART) + goto top; + goto out; + } + + mutex_enter(&zp->z_lock); + + (void) sa_update(zp->z_sa_hdl, SA_ZPL_FLAGS(zfsvfs), + &zp->z_pflags, sizeof (zp->z_pflags), tx); + + if (zp->z_is_sa) + error = sa_remove(zp->z_sa_hdl, SA_ZPL_SYMLINK(zfsvfs), + tx); + else + zfs_sa_symlink(zp, buffer, 0, tx); + + zp->z_size = 0; // If dir size > 2 -> ENOTEMPTY + (void) sa_update(zp->z_sa_hdl, SA_ZPL_SIZE(zfsvfs), + &zp->z_size, sizeof (zp->z_size), tx); + + mutex_exit(&zp->z_lock); + + dmu_tx_commit(tx); + + if (zfsvfs->z_os->os_sync == ZFS_SYNC_ALWAYS) + zil_commit(zfsvfs->z_log, 0); + +out: + if (dzp != NULL) + zrele(dzp); + VN_RELE(vp); + + dprintf("%s: returning 0x%lx\n", __func__, Status); + + return (Status); +} + +NTSTATUS +create_or_get_object_id(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_NOT_IMPLEMENTED; + PFILE_OBJECT FileObject = IrpSp->FileObject; + DWORD inlen = IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + void *buffer = Irp->AssociatedIrp.SystemBuffer; + FILE_OBJECTID_BUFFER *fob = buffer; + + if (!FileObject) + return (STATUS_INVALID_PARAMETER); + + if (!fob || inlen < sizeof (FILE_OBJECTID_BUFFER)) { + Irp->IoStatus.Information = sizeof (FILE_OBJECTID_BUFFER); + return (STATUS_BUFFER_OVERFLOW); + } + + struct vnode *vp = IrpSp->FileObject->FsContext; + VN_HOLD(vp); + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + // ObjectID is 16 bytes to identify the file + // Should we do endian work here? + // znode id + pool guid + RtlCopyMemory(&fob->ObjectId[0], &zp->z_id, sizeof (UINT64)); + uint64_t guid = dmu_objset_fsid_guid(zfsvfs->z_os); + RtlCopyMemory(&fob->ObjectId[sizeof (UINT64)], &guid, sizeof (UINT64)); + + VN_RELE(vp); + + Irp->IoStatus.Information = sizeof (FILE_OBJECTID_BUFFER); + Status = STATUS_SUCCESS; + return (Status); +} + +NTSTATUS +set_sparse(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + uint64_t datalen = + IrpSp->Parameters.FileSystemControl.InputBufferLength; + boolean_t set = B_TRUE; + struct vnode *vp; + znode_t *zp; + + if (!IrpSp->FileObject) + return (STATUS_INVALID_PARAMETER); + + /* Buffer is optional */ + if (Irp->AssociatedIrp.SystemBuffer != NULL && + datalen < sizeof (FILE_SET_SPARSE_BUFFER)) + return (STATUS_INVALID_PARAMETER); + + /* if given */ + FILE_SET_SPARSE_BUFFER *fssb = Irp->AssociatedIrp.SystemBuffer; + + vp = IrpSp->FileObject->FsContext; + if (vp == NULL) + return (STATUS_INVALID_PARAMETER); + + zp = VTOZ(vp); + if (zp == NULL) + return (STATUS_INVALID_PARAMETER); + + if (fssb != NULL) + set = fssb->SetSparse; + + /* We should at least send events */ + + return (STATUS_SUCCESS); +} + +#ifndef FSCTL_GET_INTEGRITY_INFORMATION_BUFFER +typedef struct _FSCTL_GET_INTEGRITY_INFORMATION_BUFFER { + USHORT ChecksumAlgorithm; + USHORT Reserved; + ULONG Flags; + ULONG ChecksumChunkSizeInBytes; + ULONG ClusterSizeInBytes; +} FSCTL_GET_INTEGRITY_INFORMATION_BUFFER, + *PFSCTL_GET_INTEGRITY_INFORMATION_BUFFER; +#endif + +NTSTATUS +get_integrity_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + uint64_t datalen = + IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + FSCTL_GET_INTEGRITY_INFORMATION_BUFFER *fgiib; + + fgiib = (FSCTL_GET_INTEGRITY_INFORMATION_BUFFER *) + Irp->AssociatedIrp.SystemBuffer; + + if (!IrpSp->FileObject) + return (STATUS_INVALID_PARAMETER); + + if (!fgiib || + datalen < sizeof (FSCTL_GET_INTEGRITY_INFORMATION_BUFFER)) + return (STATUS_INVALID_PARAMETER); + + fgiib->ChecksumAlgorithm = 0; + fgiib->Reserved = 0; + fgiib->Flags = 0; + fgiib->ChecksumChunkSizeInBytes = 512; + fgiib->ClusterSizeInBytes = 512; + + Irp->IoStatus.Information = + sizeof (FSCTL_GET_INTEGRITY_INFORMATION_BUFFER); + + return (STATUS_SUCCESS); +} + +#ifndef FSCTL_SET_INTEGRITY_INFORMATION_BUFFER +typedef struct _FSCTL_SET_INTEGRITY_INFORMATION_BUFFER { + USHORT ChecksumAlgorithm; + USHORT Reserved; + ULONG Flags; +} FSCTL_SET_INTEGRITY_INFORMATION_BUFFER, + *PFSCTL_SET_INTEGRITY_INFORMATION_BUFFER; +#endif + +NTSTATUS +set_integrity_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + uint64_t datalen = + IrpSp->Parameters.DeviceIoControl.InputBufferLength; + + if (!IrpSp->FileObject) + return (STATUS_INVALID_PARAMETER); + + if (!Irp->AssociatedIrp.SystemBuffer || + datalen < sizeof (FSCTL_SET_INTEGRITY_INFORMATION_BUFFER)) + return (STATUS_INVALID_PARAMETER); + + return (STATUS_SUCCESS); +} + +NTSTATUS +duplicate_extents_to_file(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, boolean_t extended) +{ + NTSTATUS Status = STATUS_NOT_IMPLEMENTED; + PFILE_OBJECT sourcefo = NULL; + PFILE_OBJECT FileObject = IrpSp->FileObject; + DWORD datalen = IrpSp->Parameters.DeviceIoControl.InputBufferLength; + void *buffer = Irp->AssociatedIrp.SystemBuffer; + struct vnode *outvp = NULL, *invp = NULL; + znode_t *outzp = NULL, *inzp = NULL; + zfsvfs_t *zfsvfs; + uint64_t inoff, outoff; + uint64_t length; + + dprintf("%s\n", extended ? "duplicate_extents_to_file_ex" : + "duplicate_extents_to_file"); + + if (!FileObject) + return (STATUS_INVALID_PARAMETER); + outvp = FileObject->FsContext; + if (outvp == NULL) + return (STATUS_INVALID_PARAMETER); + + outzp = VTOZ(outvp); + if (outzp == NULL) + return (STATUS_INVALID_PARAMETER); + + zfsvfs = outzp->z_zfsvfs; + + if (vfs_isrdonly(zfsvfs->z_vfs)) + return (STATUS_MEDIA_WRITE_PROTECTED); + +#if 0 + if (Irp->RequestorMode == UserMode && + !(ccb->access & FILE_WRITE_DATA)) { + return (STATUS_ACCESS_DENIED); + } +#endif + if (!vnode_isreg(outvp) && !vnode_islnk(outvp)) + return (STATUS_INVALID_PARAMETER); + + + if (extended) { +#ifndef _DUPLICATE_EXTENTS_DATA_EX + typedef struct _DUPLICATE_EXTENTS_DATA_EX { + SIZE_T Size; + HANDLE FileHandle; + LARGE_INTEGER SourceFileOffset; + LARGE_INTEGER TargetFileOffset; + LARGE_INTEGER ByteCount; + ULONG Flags; + } DUPLICATE_EXTENTS_DATA_EX, *PDUPLICATE_EXTENTS_DATA_EX; +#endif + DUPLICATE_EXTENTS_DATA_EX *dede = + (DUPLICATE_EXTENTS_DATA_EX *)buffer; + + if (!buffer || datalen < sizeof (DUPLICATE_EXTENTS_DATA_EX) || + dede->Size != sizeof (DUPLICATE_EXTENTS_DATA_EX)) + return (STATUS_BUFFER_TOO_SMALL); + + if (dede->ByteCount.QuadPart == 0) + return (STATUS_SUCCESS); + + inoff = dede->SourceFileOffset.QuadPart; + outoff = dede->TargetFileOffset.QuadPart; + length = dede->ByteCount.QuadPart; + Status = ObReferenceObjectByHandle(dede->FileHandle, 0, + *IoFileObjectType, Irp->RequestorMode, + (void **)&sourcefo, NULL); + + } else { +#ifndef _DUPLICATE_EXTENTS_DATA + typedef struct _DUPLICATE_EXTENTS_DATA { + HANDLE FileHandle; + LARGE_INTEGER SourceFileOffset; + LARGE_INTEGER TargetFileOffset; + LARGE_INTEGER ByteCount; + } DUPLICATE_EXTENTS_DATA, *PDUPLICATE_EXTENTS_DATA; +#endif + + DUPLICATE_EXTENTS_DATA *ded = + (DUPLICATE_EXTENTS_DATA *)buffer; + if (!buffer || datalen < sizeof (DUPLICATE_EXTENTS_DATA)) + return (STATUS_BUFFER_TOO_SMALL); + if (ded->ByteCount.QuadPart == 0) + return (STATUS_SUCCESS); + + inoff = ded->SourceFileOffset.QuadPart; + outoff = ded->TargetFileOffset.QuadPart; + length = ded->ByteCount.QuadPart; + Status = ObReferenceObjectByHandle(ded->FileHandle, 0, + *IoFileObjectType, Irp->RequestorMode, + (void **)&sourcefo, NULL); + } + + if (!NT_SUCCESS(Status)) { + dprintf("ObReferenceObjectByHandle returned %08lx\n", Status); + return (Status); + } + + invp = sourcefo->FsContext; + if (invp == NULL || VN_HOLD(invp) != 0) { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + /* Holding invp */ + + inzp = VTOZ(invp); + if (inzp == NULL) { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + + /* From here, release sourcefo */ + + /* + * zfs_clone_range(znode_t *inzp, uint64_t *inoffp, znode_t *outzp, + * uint64_t *outoffp, uint64_t *lenp, cred_t *cr) + */ + + Status = zfs_clone_range(inzp, &inoff, outzp, &outoff, + &length, NULL); + +out: + ObDereferenceObject(sourcefo); + if (invp != NULL) + VN_RELE(invp); + return (SET_ERROR(Status)); +} + +/* + * Thought this was needed for clone, but it is not + * but keeping it around in case one day we will need it + */ +#if 0 +#ifndef FILE_REGION_INFO +typedef struct _FILE_REGION_INFO { + LONGLONG FileOffset; + LONGLONG Length; + ULONG Usage; + ULONG Reserved; +} FILE_REGION_INFO, *PFILE_REGION_INFO; +#endif + +#ifndef FILE_REGION_OUTPUT +typedef struct _FILE_REGION_OUTPUT { + ULONG Flags; + ULONG TotalRegionEntryCount; + ULONG RegionEntryCount; + ULONG Reserved; + FILE_REGION_INFO Region[1]; +} FILE_REGION_OUTPUT, *PFILE_REGION_OUTPUT; +#endif + +#ifndef FILE_REGION_USAGE_VALID_CACHED_DATA +#define FILE_REGION_USAGE_VALID_CACHED_DATA 0x00000001 +#endif +#endif + +NTSTATUS +query_file_regions(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ +#if 0 + uint64_t inlen = IrpSp->Parameters.DeviceIoControl.InputBufferLength; + uint64_t outlen = IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + + if (!IrpSp->FileObject) + return (STATUS_INVALID_PARAMETER); + + if (Irp->AssociatedIrp.SystemBuffer != NULL && + inlen < sizeof (FILE_REGION_INFO)) { + return (STATUS_BUFFER_TOO_SMALL); + } + + if (Irp->AssociatedIrp.SystemBuffer == NULL || + outlen < sizeof (FILE_REGION_OUTPUT)) { + Irp->IoStatus.Information = sizeof (FILE_REGION_OUTPUT); + return (STATUS_BUFFER_TOO_SMALL); + } + + struct vnode *vp = IrpSp->FileObject->FsContext; + if (vp == NULL) + return (STATUS_INVALID_PARAMETER); + + znode_t *zp = VTOZ(vp); + if (zp == NULL) + return (STATUS_INVALID_PARAMETER); + + if (inlen == 0) { + + } else { + FILE_REGION_INFO *fri = + (FILE_REGION_INFO *)Irp->AssociatedIrp.SystemBuffer; + if (fri->FileOffset > INT64_MAX || fri->Length > INT64_MAX) + return (STATUS_INVALID_PARAMETER); + if ((fri->FileOffset + fri->Length) > INT64_MAX) + return (STATUS_INVALID_PARAMETER); + if ((fri->Usage & 3) == 0) + return (STATUS_INVALID_PARAMETER); + fri->Reserved = 0; + } + + + FILE_REGION_OUTPUT *fro = + (FILE_REGION_OUTPUT *)Irp->AssociatedIrp.SystemBuffer; + + fro->Flags = 0; + fro->TotalRegionEntryCount = 1; + fro->RegionEntryCount = 0; + fro->Reserved = 0; + + Irp->IoStatus.Information = sizeof (FILE_REGION_OUTPUT); + fro->RegionEntryCount = 1; + fro->Region[0].FileOffset = 0; + fro->Region[0].Length = zp->z_size; + fro->Region[0].Usage = FILE_REGION_USAGE_VALID_CACHED_DATA; + fro->Region[0].Reserved = 0; + + return (STATUS_SUCCESS); +#endif + return (STATUS_INVALID_PARAMETER); +} + +typedef BOOLEAN + (__stdcall *tFsRtlCheckLockForOplockRequest) + (PFILE_LOCK FileLock, PLARGE_INTEGER AllocationSize); +typedef BOOLEAN + (__stdcall *tFsRtlAreThereCurrentOrInProgressFileLocks) + (PFILE_LOCK FileLock); +tFsRtlCheckLockForOplockRequest fFsRtlCheckLockForOplockRequest = NULL; +tFsRtlAreThereCurrentOrInProgressFileLocks \ + fFsRtlAreThereCurrentOrInProgressFileLocks = NULL; + +NTSTATUS +request_oplock(PDEVICE_OBJECT DeviceObject, PIRP *PIrp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = 0; + uint32_t fsctl = IrpSp->Parameters.FileSystemControl.FsControlCode; + PFILE_OBJECT FileObject = IrpSp->FileObject; + PREQUEST_OPLOCK_INPUT_BUFFER buf = NULL; + boolean_t oplock_request = FALSE, oplock_ack = FALSE; + ULONG oplock_count = 0; + PIRP Irp = *PIrp; + int error = 0; + + if (FileObject == NULL) + return (STATUS_INVALID_PARAMETER); + + struct vnode *vp = IrpSp->FileObject->FsContext; + + if (vp == NULL) + return (STATUS_INVALID_PARAMETER); + + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); // This returns EIO if fail + /* HOLD count, no returns from here. */ + + if (VN_HOLD(vp) != 0) { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + + if (!vnode_isreg(vp) && !vnode_isdir(vp)) { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + + if (fsctl == FSCTL_REQUEST_OPLOCK) { + if (IrpSp->Parameters.FileSystemControl.InputBufferLength < + sizeof (REQUEST_OPLOCK_INPUT_BUFFER)) { + Status = STATUS_BUFFER_TOO_SMALL; + goto out; + } + if (IrpSp->Parameters.FileSystemControl.OutputBufferLength < + sizeof (REQUEST_OPLOCK_OUTPUT_BUFFER)) { + Status = STATUS_BUFFER_TOO_SMALL; + goto out; + } + buf = Irp->AssociatedIrp.SystemBuffer; + + // flags are mutually exclusive + if (buf->Flags & REQUEST_OPLOCK_INPUT_FLAG_REQUEST && + buf->Flags & REQUEST_OPLOCK_INPUT_FLAG_ACK) { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + + oplock_request = buf->Flags & REQUEST_OPLOCK_INPUT_FLAG_REQUEST; + oplock_ack = buf->Flags & REQUEST_OPLOCK_INPUT_FLAG_ACK; + + if (!oplock_request && !oplock_ack) { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + } + + boolean_t shared_request = (fsctl == FSCTL_REQUEST_OPLOCK_LEVEL_2) || + (fsctl == FSCTL_REQUEST_OPLOCK && + !(buf->RequestedOplockLevel & OPLOCK_LEVEL_CACHE_WRITE)); + + if (vnode_isdir(vp) && (fsctl != FSCTL_REQUEST_OPLOCK || + !shared_request)) { + dprintf("oplock requests on directories can only be " + "for read or read-handle oplocks\n"); + Status = STATUS_INVALID_PARAMETER; + goto out; + } + + // research this + // ExAcquireResourceSharedLite(&Vcb->tree_lock, true); + + ExAcquireResourceExclusiveLite(vp->FileHeader.Resource, TRUE); + + // Does windows have something like "attribute availability (function)" + // for dynamic checks? + // FastFat example uses #ifdef NTDDI-VERSION which is compile time, + // we should look into + // winbtrfs: + // RtlInitUnicodeString(&name, L"FsRtlCheckLockForOplockRequest"); + // fFsRtlCheckLockForOplockRequest = + // (tFsRtlCheckLockForOplockRequest) + // MmGetSystemRoutineAddress(&name); + // move me to init place + static int firstrun = 1; + if (firstrun) { + UNICODE_STRING name; + RtlInitUnicodeString(&name, L"FsRtlCheckLockForOplockRequest"); + fFsRtlCheckLockForOplockRequest = + (tFsRtlCheckLockForOplockRequest) + MmGetSystemRoutineAddress(&name); + RtlInitUnicodeString(&name, + L"FsRtlAreThereCurrentOrInProgressFileLocks"); + fFsRtlAreThereCurrentOrInProgressFileLocks = + (tFsRtlAreThereCurrentOrInProgressFileLocks) + MmGetSystemRoutineAddress(&name); + firstrun = 0; + } + + if (fsctl == FSCTL_REQUEST_OPLOCK_LEVEL_1 || + fsctl == FSCTL_REQUEST_BATCH_OPLOCK || + fsctl == FSCTL_REQUEST_FILTER_OPLOCK || + fsctl == FSCTL_REQUEST_OPLOCK_LEVEL_2 || oplock_request) { + if (shared_request) { + if (vnode_isreg(vp)) { + if (fFsRtlCheckLockForOplockRequest) + oplock_count = + !fFsRtlCheckLockForOplockRequest( + &vp->lock, + &vp->FileHeader.AllocationSize); + else if + (fFsRtlAreThereCurrentOrInProgressFileLocks) + oplock_count = + fFsRtlAreThereCurrentOrInProgressFileLocks(&vp->lock); + else + oplock_count = + FsRtlAreThereCurrentFileLocks(&vp-> + lock); + } + } else + oplock_count = vnode_iocount(vp); + } + + zfs_dirlist_t *zccb = FileObject->FsContext2; + + if (zccb != NULL && + zccb->magic == ZFS_DIRLIST_MAGIC && + zccb->deleteonclose) { + + if ((fsctl == FSCTL_REQUEST_FILTER_OPLOCK || + fsctl == FSCTL_REQUEST_BATCH_OPLOCK || + (fsctl == FSCTL_REQUEST_OPLOCK && + buf->RequestedOplockLevel & + OPLOCK_LEVEL_CACHE_HANDLE))) { + ExReleaseResourceLite(vp->FileHeader.Resource); + // ExReleaseResourceLite(&Vcb->tree_lock); + Status = STATUS_DELETE_PENDING; + goto out; + } + } + + // This will complete the IRP as well. + // How to stop dispatcher from completing? + Status = FsRtlOplockFsctrl(vp_oplock(vp), Irp, oplock_count); + *PIrp = NULL; // Don't complete. + + // *Pirp = NULL; + + // fcb->Header.IsFastIoPossible = fast_io_possible(fcb); + + ExReleaseResourceLite(vp->FileHeader.Resource); + // ExReleaseResourceLite(&Vcb->tree_lock); + +out: + VN_RELE(vp); + zfs_exit(zfsvfs, FTAG); + + return (Status); +} + +NTSTATUS +user_fs_request(PDEVICE_OBJECT DeviceObject, PIRP *PIrp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_INVALID_DEVICE_REQUEST; + PIRP Irp = *PIrp; + + switch (IrpSp->Parameters.FileSystemControl.FsControlCode) { + case FSCTL_LOCK_VOLUME: + dprintf(" FSCTL_LOCK_VOLUME\n"); + Status = STATUS_SUCCESS; + break; + case FSCTL_UNLOCK_VOLUME: + dprintf(" FSCTL_UNLOCK_VOLUME\n"); + Status = STATUS_SUCCESS; + break; + case FSCTL_DISMOUNT_VOLUME: + dprintf(" FSCTL_DISMOUNT_VOLUME\n"); + break; + case FSCTL_MARK_VOLUME_DIRTY: + dprintf(" FSCTL_MARK_VOLUME_DIRTY\n"); + Status = STATUS_SUCCESS; + break; + case FSCTL_IS_VOLUME_MOUNTED: + dprintf(" FSCTL_IS_VOLUME_MOUNTED\n"); + Status = STATUS_SUCCESS; + { + mount_t *zmo; + zmo = DeviceObject->DeviceExtension; + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs->z_unmounted) + Status = STATUS_VERIFY_REQUIRED; + } + break; + case FSCTL_SET_COMPRESSION: + dprintf(" FSCTL_SET_COMPRESSION\n"); + Status = STATUS_SUCCESS; + break; + case FSCTL_IS_PATHNAME_VALID: + dprintf(" FSCTL_IS_PATHNAME_VALID\n"); + Status = STATUS_SUCCESS; + break; + case FSCTL_GET_RETRIEVAL_POINTERS: + dprintf(" FSCTL_GET_RETRIEVAL_POINTERS\n"); + Status = STATUS_INVALID_PARAMETER; + break; + case FSCTL_IS_VOLUME_DIRTY: + dprintf(" FSCTL_IS_VOLUME_DIRTY\n"); + PULONG VolumeState; + + VolumeState = MapUserBuffer(Irp); + + if (VolumeState == NULL) { + Status = STATUS_INSUFFICIENT_RESOURCES; + break; + } + + if (IrpSp->Parameters.FileSystemControl.OutputBufferLength < + sizeof (ULONG)) { + Status = STATUS_INVALID_PARAMETER; + break; + } + + *VolumeState = 0; + if (0) + SetFlag(*VolumeState, VOLUME_IS_DIRTY); + Irp->IoStatus.Information = sizeof (ULONG); + Status = STATUS_SUCCESS; + break; + case FSCTL_GET_REPARSE_POINT: + dprintf(" FSCTL_GET_REPARSE_POINT\n"); + Status = get_reparse_point(DeviceObject, Irp, IrpSp); + break; + case FSCTL_SET_REPARSE_POINT: + dprintf(" FSCTL_SET_REPARSE_POINT\n"); + Status = set_reparse_point(DeviceObject, Irp, IrpSp); + break; + case FSCTL_DELETE_REPARSE_POINT: + dprintf(" FSCTL_DELETE_REPARSE_POINT\n"); + Status = delete_reparse_point(DeviceObject, Irp, IrpSp); + break; + case FSCTL_CREATE_OR_GET_OBJECT_ID: + dprintf(" FSCTL_CREATE_OR_GET_OBJECT_ID\n"); + Status = create_or_get_object_id(DeviceObject, Irp, IrpSp); + break; + case FSCTL_REQUEST_OPLOCK: + dprintf(" FSCTL_REQUEST_OPLOCK: \n"); + Status = request_oplock(DeviceObject, PIrp, IrpSp); + break; + case FSCTL_FILESYSTEM_GET_STATISTICS: + dprintf(" FSCTL_FILESYSTEM_GET_STATISTICS: \n"); + FILESYSTEM_STATISTICS *fss = Irp->AssociatedIrp.SystemBuffer; + + // btrfs: This is hideously wrong, but at least it stops SMB + // from breaking + + if (IrpSp->Parameters.FileSystemControl.OutputBufferLength < + sizeof (FILESYSTEM_STATISTICS)) + return (STATUS_BUFFER_TOO_SMALL); + + memset(fss, 0, sizeof (FILESYSTEM_STATISTICS)); + + fss->Version = 1; + fss->FileSystemType = FILESYSTEM_STATISTICS_TYPE_NTFS; + fss->SizeOfCompleteStructure = sizeof (FILESYSTEM_STATISTICS); + + Irp->IoStatus.Information = sizeof (FILESYSTEM_STATISTICS); + Status = STATUS_SUCCESS; + break; + case FSCTL_QUERY_DEPENDENT_VOLUME: + dprintf(" FSCTL_QUERY_DEPENDENT_VOLUME: \n"); + STORAGE_QUERY_DEPENDENT_VOLUME_REQUEST *req = + Irp->AssociatedIrp.SystemBuffer; + dprintf("RequestLevel %ld: RequestFlags 0x%lx\n", + req->RequestLevel, req->RequestFlags); +// #define QUERY_DEPENDENT_VOLUME_REQUEST_FLAG_HOST_VOLUMES 0x1 +// #define QUERY_DEPENDENT_VOLUME_REQUEST_FLAG_GUEST_VOLUMES 0x2 + STORAGE_QUERY_DEPENDENT_VOLUME_LEV1_ENTRY *lvl1 = + Irp->AssociatedIrp.SystemBuffer; + STORAGE_QUERY_DEPENDENT_VOLUME_LEV2_ENTRY *lvl2 = + Irp->AssociatedIrp.SystemBuffer; + + switch (req->RequestLevel) { + case 1: + if (IrpSp-> + Parameters.FileSystemControl.OutputBufferLength < + sizeof (STORAGE_QUERY_DEPENDENT_VOLUME_LEV1_ENTRY)) + return (STATUS_BUFFER_TOO_SMALL); + memset(lvl1, 0, + sizeof (STORAGE_QUERY_DEPENDENT_VOLUME_LEV1_ENTRY)); + lvl1->EntryLength = + sizeof (STORAGE_QUERY_DEPENDENT_VOLUME_LEV1_ENTRY); + Irp->IoStatus.Information = + sizeof (STORAGE_QUERY_DEPENDENT_VOLUME_LEV1_ENTRY); + Status = STATUS_SUCCESS; + break; + case 2: + if (IrpSp-> + Parameters.FileSystemControl.OutputBufferLength < + sizeof (STORAGE_QUERY_DEPENDENT_VOLUME_LEV2_ENTRY)) + return (STATUS_BUFFER_TOO_SMALL); + memset(lvl2, 0, + sizeof (STORAGE_QUERY_DEPENDENT_VOLUME_LEV2_ENTRY)); + lvl2->EntryLength = + sizeof (STORAGE_QUERY_DEPENDENT_VOLUME_LEV2_ENTRY); + Irp->IoStatus.Information = + sizeof (STORAGE_QUERY_DEPENDENT_VOLUME_LEV2_ENTRY); + Status = STATUS_SUCCESS; + break; + default: + Status = STATUS_INVALID_PARAMETER; + break; + } + break; + + case FSCTL_SET_SPARSE: + dprintf(" FSCTL_SET_SPARSE\n"); + Status = set_sparse(DeviceObject, Irp, IrpSp); + break; + +#ifndef FSCTL_GET_INTEGRITY_INFORMATION +#define FSCTL_GET_INTEGRITY_INFORMATION CTL_CODE(FILE_DEVICE_FILE_SYSTEM, \ + 159, METHOD_BUFFERED, FILE_ANY_ACCESS) +#endif + +#ifndef FSCTL_SET_INTEGRITY_INFORMATION +#define FSCTL_SET_INTEGRITY_INFORMATION CTL_CODE(FILE_DEVICE_FILE_SYSTEM, \ + 160, METHOD_BUFFERED, FILE_READ_DATA | FILE_WRITE_DATA) +#endif + case FSCTL_GET_INTEGRITY_INFORMATION: + dprintf(" FSCTL_GET_INTEGRITY_INFORMATION_BUFFER\n"); + Status = get_integrity_information(DeviceObject, Irp, IrpSp); + break; + + case FSCTL_SET_INTEGRITY_INFORMATION: + dprintf(" FSCTL_SET_INTEGRITY_INFORMATION_BUFFER\n"); + Status = set_integrity_information(DeviceObject, Irp, IrpSp); + break; + +#ifndef FSCTL_DUPLICATE_EXTENTS_TO_FILE +#define FSCTL_DUPLICATE_EXTENTS_TO_FILE CTL_CODE(FILE_DEVICE_FILE_SYSTEM, \ + 209, METHOD_BUFFERED, FILE_WRITE_DATA) +#endif + case FSCTL_DUPLICATE_EXTENTS_TO_FILE: + dprintf(" FSCTL_DUPLICATE_EXTENTS_TO_FILE\n"); + Status = duplicate_extents_to_file(DeviceObject, Irp, IrpSp, + FALSE); + break; + + case FSCTL_DUPLICATE_EXTENTS_TO_FILE_EX: + dprintf(" FSCTL_DUPLICATE_EXTENTS_TO_FILE_EX\n"); + Status = duplicate_extents_to_file(DeviceObject, Irp, IrpSp, + TRUE); + break; + +#ifndef FSCTL_QUERY_FILE_REGIONS +#define FSCTL_QUERY_FILE_REGIONS CTL_CODE(FILE_DEVICE_FILE_SYSTEM, \ + 161, METHOD_BUFFERED, FILE_ANY_ACCESS) +#endif + case FSCTL_QUERY_FILE_REGIONS: + dprintf(" FSCTL_QUERY_FILE_REGIONS\n"); + Status = query_file_regions(DeviceObject, Irp, IrpSp); + break; + + case FSCTL_ZFS_VOLUME_MOUNTPOINT: + dprintf(" FSCTL_ZFS_VOLUME_MOUNTPOINT\n"); + Status = fsctl_zfs_volume_mountpoint(DeviceObject, Irp, IrpSp); + break; + + case FSCTL_READ_FILE_USN_DATA: + dprintf(" FSCTL_READ_FILE_USN_DATA\n"); + + Status = STATUS_INVALID_DEVICE_REQUEST; + break; + + case FSCTL_QUERY_PERSISTENT_VOLUME_STATE: + dprintf(" FSCTL_QUERY_PERSISTENT_VOLUME_STATE\n"); + PVOID Buffer = Irp->AssociatedIrp.SystemBuffer; + ULONG InputBufferLength = + IrpSp->Parameters.FileSystemControl.InputBufferLength; + ULONG OutputBufferLength = + IrpSp->Parameters.FileSystemControl.OutputBufferLength; + PFILE_FS_PERSISTENT_VOLUME_INFORMATION Info; + + if (NULL == Buffer) + return (STATUS_INVALID_PARAMETER); + + if (sizeof (FILE_FS_PERSISTENT_VOLUME_INFORMATION) > + InputBufferLength || + sizeof (FILE_FS_PERSISTENT_VOLUME_INFORMATION) > + OutputBufferLength) + return (STATUS_BUFFER_TOO_SMALL); + + Info = Buffer; + if (1 != Info->Version || + !FlagOn(Info->FlagMask, + PERSISTENT_VOLUME_STATE_SHORT_NAME_CREATION_DISABLED)) + return (STATUS_INVALID_PARAMETER); + + RtlZeroMemory(Info, + sizeof (FILE_FS_PERSISTENT_VOLUME_INFORMATION)); + Info->VolumeFlags = + PERSISTENT_VOLUME_STATE_SHORT_NAME_CREATION_DISABLED; + Irp->IoStatus.Information = + sizeof (FILE_FS_PERSISTENT_VOLUME_INFORMATION); + + Status = STATUS_SUCCESS; + break; + + case FSCTL_SET_ZERO_DATA: + dprintf(" FSCTL_SET_ZERO_DATA\n"); + Status = fsctl_set_zero_data(DeviceObject, Irp, IrpSp); + break; + + default: + dprintf("* %s: unknown class 0x%lx\n", __func__, + IrpSp->Parameters.FileSystemControl.FsControlCode); + break; + } + + return (Status); +} + +NTSTATUS +query_directory_FileFullDirectoryInformation(PDEVICE_OBJECT DeviceObject, + PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + // FILE_FULL_DIR_INFORMATION *outptr = Irp->UserBuffer; + int flag_index_specified = + IrpSp->Flags & SL_INDEX_SPECIFIED ? 1 : 0; + int flag_restart_scan = + IrpSp->Flags & SL_RESTART_SCAN ? 1 : 0; + int flag_return_single_entry = + IrpSp->Flags & SL_RETURN_SINGLE_ENTRY ? 1 : 0; + int ret; + mount_t *zmo; + zfsvfs_t *zfsvfs; + NTSTATUS Status = STATUS_NO_SUCH_FILE; + + if ((Irp->UserBuffer == NULL && Irp->MdlAddress == NULL) || + IrpSp->Parameters.QueryDirectory.Length <= 0) + return (STATUS_INSUFFICIENT_RESOURCES); + + if (IrpSp->FileObject == NULL || + IrpSp->FileObject->FsContext == NULL || // vnode + IrpSp->FileObject->FsContext2 == NULL) // ccb + return (STATUS_INVALID_PARAMETER); + + struct vnode *dvp = IrpSp->FileObject->FsContext; + zfs_dirlist_t *zccb = IrpSp->FileObject->FsContext2; + + if (zccb->magic != ZFS_DIRLIST_MAGIC) + return (STATUS_INVALID_PARAMETER); + + // Restarting listing? Clear EOF + if (flag_restart_scan) { + zccb->dir_eof = 0; + zccb->uio_offset = 0; + if (zccb->searchname.Buffer != NULL) + kmem_free(zccb->searchname.Buffer, + zccb->searchname.MaximumLength); + zccb->searchname.Buffer = NULL; + zccb->searchname.MaximumLength = 0; + } + + // Did last call complete listing? + if (zccb->dir_eof) + return (STATUS_NO_MORE_FILES); + struct iovec iov; + void *SystemBuffer = MapUserBuffer(Irp); + iov.iov_base = (void *)SystemBuffer; + iov.iov_len = IrpSp->Parameters.QueryDirectory.Length; + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, zccb->uio_offset, UIO_SYSSPACE, + IrpSp->Parameters.QueryDirectory.Length, 0); + + // Grab the root zp + zmo = DeviceObject->DeviceExtension; + ASSERT(zmo->type == MOUNT_TYPE_VCB); + + zfsvfs = vfs_fsprivate(zmo); // or from zp + + if (!zfsvfs) + return (STATUS_INTERNAL_ERROR); + + dprintf("%s: starting vp %p Search pattern '%wZ' type %d: " + "saved search '%wZ'\n", __func__, dvp, + IrpSp->Parameters.QueryDirectory.FileName, + IrpSp->Parameters.QueryDirectory.FileInformationClass, + &zccb->searchname); + + if (IrpSp->Parameters.QueryDirectory.FileName && + IrpSp->Parameters.QueryDirectory.FileName->Buffer && + IrpSp->Parameters.QueryDirectory.FileName->Length != 0 && + wcsncmp(IrpSp->Parameters.QueryDirectory.FileName->Buffer, + L"*", 1) != 0) { + // Save the pattern in the zccb, as it is only given in the + // first call (citation needed) + + // If exists, we should free? + if (zccb->searchname.Buffer != NULL) + kmem_free(zccb->searchname.Buffer, + zccb->searchname.MaximumLength); + + zccb->ContainsWildCards = FsRtlDoesNameContainWildCards( + IrpSp->Parameters.QueryDirectory.FileName); + zccb->searchname.MaximumLength = + IrpSp->Parameters.QueryDirectory.FileName->Length + 2; + zccb->searchname.Length = + IrpSp->Parameters.QueryDirectory.FileName->Length; + zccb->searchname.Buffer = + kmem_alloc(zccb->searchname.MaximumLength, + KM_SLEEP); + if (zccb->ContainsWildCards) { + Status = RtlUpcaseUnicodeString(&zccb->searchname, + IrpSp->Parameters.QueryDirectory.FileName, FALSE); + } else { + RtlCopyMemory(zccb->searchname.Buffer, + IrpSp->Parameters.QueryDirectory.FileName->Buffer, + zccb->searchname.Length); + } + dprintf("%s: setting up search '%wZ' (wildcards: %d) " + "status 0x%lx\n", __func__, + &zccb->searchname, zccb->ContainsWildCards, Status); + } + + emitdir_ptr_t ctx; + ctx.bufsize = (size_t)zfs_uio_resid(&uio); + ctx.alloc_buf = kmem_zalloc(ctx.bufsize, KM_SLEEP); + ctx.bufptr = ctx.alloc_buf; + ctx.outcount = 0; + ctx.next_offset = NULL; + ctx.last_alignment = 0; + ctx.offset = zccb->uio_offset; + ctx.numdirent = 0; + ctx.dirlisttype = IrpSp->Parameters.QueryDirectory.FileInformationClass; + + VN_HOLD(dvp); + ret = zfs_readdir(dvp, &ctx, NULL, zccb, IrpSp->Flags); + VN_RELE(dvp); + + /* finished listing dir? */ + if (ret == ENOENT) { + zccb->dir_eof = 1; + ret = 0; + } else if (ret == ENOSPC) { + + /* + * If we have no "outcount" then buffer is too small + * for the first record. If we do have "outcount", we + * return what we have, and wait to be called again. + */ + if (ctx.outcount > 0) + ret = 0; + else + Status = STATUS_BUFFER_OVERFLOW; + } + + if (ret == 0) { + if (ctx.outcount > 0) { + + if ((ret = zfs_uiomove(ctx.alloc_buf, + (long)ctx.outcount, UIO_READ, &uio))) { + /* + * Reset the pointer, by copying in old value + */ + ctx.offset = zccb->uio_offset; + } + Status = STATUS_SUCCESS; + } else { // outcount == 0 + Status = (zccb->uio_offset == 0) ? STATUS_NO_SUCH_FILE : + STATUS_NO_MORE_FILES; + } + // Set correct buffer size returned. + Irp->IoStatus.Information = ctx.outcount; + // IrpSp->Parameters.QueryDirectory.Length - + // zfs_uio_resid(&uio); + + dprintf("dirlist information in %ld out size %ld\n", + IrpSp->Parameters.QueryDirectory.Length, + Irp->IoStatus.Information); + + // Remember directory index for next time + zccb->uio_offset = ctx.offset; + } + + kmem_free(ctx.alloc_buf, ctx.bufsize); + + return (Status); +} + + +NTSTATUS +query_directory(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_NOT_IMPLEMENTED; + + switch (IrpSp->Parameters.QueryDirectory.FileInformationClass) { + + // The type is now passed into zfs_vnop.c/zfs_readdir() + // so check there for support + case FileBothDirectoryInformation: + case FileDirectoryInformation: + case FileFullDirectoryInformation: // *** + case FileIdBothDirectoryInformation: // *** + case FileIdFullDirectoryInformation: + case FileNamesInformation: + // case FileObjectIdInformation: // Do we need this one? + case FileIdExtdDirectoryInformation: + case FileIdExtdBothDirectoryInformation: + Status = + query_directory_FileFullDirectoryInformation(DeviceObject, + Irp, IrpSp); + break; + case FileQuotaInformation: + dprintf(" %s FileQuotaInformation *NotImplemented\n", + __func__); + break; + break; + case FileReparsePointInformation: + dprintf(" %s FileReparsePointInformation *NotImplemented\n", + __func__); + break; + default: + dprintf(" %s unknown 0x%x *NotImplemented\n", + __func__, + IrpSp->Parameters.QueryDirectory.FileInformationClass); + break; + } + + return (Status); +} + +NTSTATUS +notify_change_directory(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + PFILE_OBJECT fileObject = IrpSp->FileObject; + mount_t *zmo; + + dprintf("%s\n", __func__); + zmo = DeviceObject->DeviceExtension; + ASSERT(zmo != NULL); + if (zmo->type != MOUNT_TYPE_VCB) { + return (STATUS_INVALID_PARAMETER); + } + + struct vnode *vp = fileObject->FsContext; + zfs_dirlist_t *zccb = fileObject->FsContext2; + ASSERT(vp != NULL); + + VN_HOLD(vp); + znode_t *zp = VTOZ(vp); + + if (!vnode_isdir(vp)) { + VN_RELE(vp); + return (STATUS_INVALID_PARAMETER); + } + + if (zccb && zccb->deleteonclose) { + VN_RELE(vp); + return (STATUS_DELETE_PENDING); + } + ASSERT(zmo->NotifySync != NULL); + + dprintf("%s: '%s' for %wZ\n", __func__, + zp&&zp->z_name_cache?zp->z_name_cache:"", &fileObject->FileName); + + FsRtlNotifyFilterChangeDirectory( + zmo->NotifySync, &zmo->DirNotifyList, zccb, + (PSTRING)&fileObject->FileName, + (IrpSp->Flags & SL_WATCH_TREE), FALSE, + IrpSp->Parameters.NotifyDirectory.CompletionFilter, Irp, + NULL, NULL, NULL); + + VN_RELE(vp); + return (STATUS_PENDING); +} + +NTSTATUS +set_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_NOT_IMPLEMENTED; + + Irp->IoStatus.Information = 0; + + switch (IrpSp->Parameters.SetFile.FileInformationClass) { + case FileAllocationInformation: + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + FILE_ALLOCATION_INFORMATION *feofi = + Irp->AssociatedIrp.SystemBuffer; + dprintf("* SET FileAllocationInformation %llu\n", + feofi->AllocationSize.QuadPart); +// This is a noop at the moment. It makes Windows Explorer and apps not crash +// From the documentation, setting the allocation size smaller than EOF +// should shrink it: +// msdn.microsoft.com/en-us/library/windows/desktop/aa364214(v=vs.85).aspx +// However, NTFS doesn't do that! It keeps the size the same. +// Setting a FileAllocationInformation larger than current EOF size does +// not have a observable affect from user space. + if (vnode_isdir(IrpSp->FileObject->FsContext)) + return (STATUS_INVALID_PARAMETER); + + Status = STATUS_SUCCESS; + } + break; + case FileBasicInformation: // chmod + dprintf("* SET FileBasicInformation\n"); + Status = set_file_basic_information(DeviceObject, Irp, IrpSp); + break; + case FileDispositionInformation: // unlink + dprintf("* SET FileDispositionInformation\n"); + Status = set_file_disposition_information(DeviceObject, Irp, + IrpSp, B_FALSE); + break; + case FileDispositionInformationEx: // unlink + dprintf("* SET FileDispositionInformationEx\n"); + Status = set_file_disposition_information(DeviceObject, Irp, + IrpSp, B_TRUE); + break; + case FileEndOfFileInformation: // extend? + Status = set_file_endoffile_information(DeviceObject, Irp, + IrpSp); + break; + case FileLinkInformation: // symlink + Status = set_file_link_information(DeviceObject, Irp, IrpSp); + break; + case FilePositionInformation: // seek + Status = set_file_position_information(DeviceObject, Irp, + IrpSp); + break; + case FileRenameInformation: // vnop_rename + case FileRenameInformationEx: + Status = set_file_rename_information(DeviceObject, Irp, IrpSp); + break; + case FileValidDataLengthInformation: + Status = set_file_valid_data_length_information(DeviceObject, + Irp, IrpSp); + break; + default: + dprintf("* %s: unknown type NOTIMPLEMENTED\n", __func__); + break; + } + + return (Status); +} + + +NTSTATUS +fs_read(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + PFILE_OBJECT fileObject; + ULONG bufferLength; + LARGE_INTEGER byteOffset; + NTSTATUS Status = STATUS_SUCCESS; + int error; + int nocache = Irp->Flags & IRP_NOCACHE; + int pagingio = FlagOn(Irp->Flags, IRP_PAGING_IO); + int releaselock = 0; + + PAGED_CODE(); + + if (FlagOn(IrpSp->MinorFunction, IRP_MN_COMPLETE)) { + dprintf("%s: IRP_MN_COMPLETE\n", __func__); + CcMdlReadComplete(IrpSp->FileObject, Irp->MdlAddress); + // Mdl is now deallocated. + Irp->MdlAddress = NULL; + return (STATUS_SUCCESS); + } + +#if 0 + dprintf(" %s minor type %d flags 0x%x mdl %d System %d " + "User %d paging %d\n", __func__, IrpSp->MinorFunction, + DeviceObject->Flags, (Irp->MdlAddress != 0), + (Irp->AssociatedIrp.SystemBuffer != 0), + (Irp->UserBuffer != 0), + FlagOn(Irp->Flags, IRP_PAGING_IO)); +#endif + if (zfs_disable_wincache) + nocache = 1; + + bufferLength = IrpSp->Parameters.Read.Length; + if (bufferLength == 0) + return (STATUS_SUCCESS); + + fileObject = IrpSp->FileObject; + + // File may have been closed, but CC mgr setting section + // will ask to read + if (fileObject == NULL || fileObject->FsContext == NULL) { + dprintf(" fileObject == NULL\n"); + // ASSERT0("fileobject == NULL"); + return (STATUS_INVALID_PARAMETER); + } + + struct vnode *vp = fileObject->FsContext; + zfs_dirlist_t *zccb = fileObject->FsContext2; + +#if 0 + if (Irp->RequestorMode == UserMode && + !(zccb->access & FILE_READ_DATA)) { + return (STATUS_ACCESS_DENIED); + } +#endif + + VN_HOLD(vp); + znode_t *zp = VTOZ(vp); + + if (IrpSp->Parameters.Read.ByteOffset.LowPart == + FILE_USE_FILE_POINTER_POSITION && + IrpSp->Parameters.Read.ByteOffset.HighPart == -1) { + byteOffset = fileObject->CurrentByteOffset; + } else { + byteOffset = IrpSp->Parameters.Read.ByteOffset; + } + + uint64_t filesize = zp->z_size; + + // If the read starts beyond the End of File, return EOF + // as per fastfat. + if (byteOffset.QuadPart >= filesize) { + Status = STATUS_END_OF_FILE; + goto out; + } + + // Read is beyond file length? shorten + if (byteOffset.QuadPart + bufferLength > filesize) + bufferLength = filesize - byteOffset.QuadPart; + + // nocache transfer, make sure we flush first. + if (!pagingio && !nocache && fileObject->SectionObjectPointer && + (fileObject->SectionObjectPointer->DataSectionObject != NULL)) { +#if 0 + // Sadly this BSODs and I'm not sure why + IO_STATUS_BLOCK IoStatus = { 0 }; + + ExAcquireResourceExclusiveLite(vp->FileHeader.PagingIoResource, + TRUE); + VERIFY3U(zccb->cacheinit, !=, 0); + + VERIFY3P(vnode_sectionpointer(vp), ==, fileObject-> + SectionObjectPointer); + VERIFY(vnode_fileobject_member(vp, fileObject) == 1); + + try { + CcFlushCache(fileObject->SectionObjectPointer, + NULL, // &IrpSp->Parameters.Read.ByteOffset, + 0, // IrpSp->Parameters.Read.Length, + &IoStatus); + } except(EXCEPTION_EXECUTE_HANDLER) { + Status = GetExceptionCode(); + } + + ExReleaseResourceLite(vp->FileHeader.PagingIoResource); + if (!NT_SUCCESS(Status)) + return (Status); +#endif + } + // Grab lock if paging + if (pagingio) { + ExAcquireResourceSharedLite(vp->FileHeader.PagingIoResource, + TRUE); + releaselock = 1; + } + + void *SystemBuffer = MapUserBuffer(Irp); + + if (nocache) { + + } else { + // Cached + +#if 1 + zfs_init_cache(fileObject, vp); +#endif + + // DO A NORMAL CACHED READ, if the MDL bit is not set, + if (!FlagOn(IrpSp->MinorFunction, IRP_MN_MDL)) { + VERIFY3U(zccb->cacheinit, !=, 0); + + vnode_pager_setsize(fileObject, vp, zp->z_size, FALSE); + try { +#if (NTDDI_VERSION >= NTDDI_WIN8) + if (!CcCopyReadEx(fileObject, + &byteOffset, + bufferLength, + TRUE, + SystemBuffer, + &Irp->IoStatus, + Irp->Tail.Overlay.Thread)) { +#else + if (!CcCopyRead(fileObject, + &byteOffset, + bufferLength, + TRUE, + SystemBuffer, + &Irp->IoStatus)) { +#endif + dprintf("CcCopyReadEx error\n"); + } + } except(EXCEPTION_EXECUTE_HANDLER) { + Irp->IoStatus.Status = GetExceptionCode(); + } + Irp->IoStatus.Information = bufferLength; + Status = Irp->IoStatus.Status; + goto out; + + } else { + VERIFY3U(zccb->cacheinit, !=, 0); + + + // MDL read + CcMdlRead(fileObject, + &byteOffset, + bufferLength, + &Irp->MdlAddress, + &Irp->IoStatus); + Status = Irp->IoStatus.Status; + goto out; + } // mdl + + } // !nocache + + + struct iovec iov; + iov.iov_base = (void *)SystemBuffer; + iov.iov_len = bufferLength; + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, byteOffset.QuadPart, UIO_SYSSPACE, + bufferLength, 0); + + dprintf("%s: offset %llx size %lx\n", __func__, + byteOffset.QuadPart, bufferLength); + + error = zfs_read(zp, &uio, 0, NULL); + + // Update bytes read + Irp->IoStatus.Information = bufferLength - zfs_uio_resid(&uio); + + // apparently we dont use EOF +// if (Irp->IoStatus.Information == 0) +// Status = STATUS_END_OF_FILE; + switch (error) { + case 0: + break; + case EISDIR: + Status = STATUS_FILE_IS_A_DIRECTORY; + break; + default: + Status = STATUS_INVALID_PARAMETER; + break; + } + +out: + + VN_RELE(vp); + // Update the file offset + if ((Status == STATUS_SUCCESS) && + (fileObject->Flags & FO_SYNCHRONOUS_IO) && + !(Irp->Flags & IRP_PAGING_IO)) { + // update current byte offset only when synchronous IO + // and not paging IO + fileObject->CurrentByteOffset.QuadPart = + byteOffset.QuadPart + Irp->IoStatus.Information; + } + + if (releaselock) ExReleaseResourceLite(vp->FileHeader.PagingIoResource); + +// dprintf(" FileName: %wZ offset 0x%llx len 0x%lx mdl %p System %p\n", +// &fileObject->FileName, byteOffset.QuadPart, +// bufferLength, Irp->MdlAddress, Irp->AssociatedIrp.SystemBuffer); + + return (Status); +} + + +NTSTATUS +fs_write(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + PFILE_OBJECT fileObject; + ULONG bufferLength; + LARGE_INTEGER byteOffset; + NTSTATUS Status = STATUS_SUCCESS; + int error; + int nocache = Irp->Flags & IRP_NOCACHE; + int pagingio = FlagOn(Irp->Flags, IRP_PAGING_IO); + + if (zfs_disable_wincache) + nocache = 1; + + PAGED_CODE(); + + if (FlagOn(IrpSp->MinorFunction, IRP_MN_COMPLETE)) { + dprintf("%s: IRP_MN_COMPLETE\n", __func__); + CcMdlWriteComplete(IrpSp->FileObject, + &IrpSp->Parameters.Write.ByteOffset, Irp->MdlAddress); + // Mdl is now deallocated. + Irp->MdlAddress = NULL; + return (STATUS_SUCCESS); + } + +#if 0 + dprintf(" %s minor type %d flags 0x%x mdl %d System %d " + "User %d paging %d\n", __func__, IrpSp->MinorFunction, + DeviceObject->Flags, (Irp->MdlAddress != 0), + (Irp->AssociatedIrp.SystemBuffer != 0), + (Irp->UserBuffer != 0), + FlagOn(Irp->Flags, IRP_PAGING_IO)); +#endif + + bufferLength = IrpSp->Parameters.Write.Length; + if (bufferLength == 0) + return (STATUS_SUCCESS); + + fileObject = IrpSp->FileObject; + + if (fileObject == NULL || fileObject->FsContext == NULL) { + dprintf(" fileObject == NULL\n"); + ASSERT0("fileObject == NULL"); + return (STATUS_INVALID_PARAMETER); + } + + struct vnode *vp = fileObject->FsContext; + zfs_dirlist_t *zccb = fileObject->FsContext2; + + VN_HOLD(vp); + znode_t *zp = VTOZ(vp); + ASSERT(ZTOV(zp) == vp); + Irp->IoStatus.Information = 0; + + // Special encoding + byteOffset = IrpSp->Parameters.Write.ByteOffset; + if (IrpSp->Parameters.Write.ByteOffset.HighPart == -1) { + if (IrpSp->Parameters.Write.ByteOffset.LowPart == + FILE_USE_FILE_POINTER_POSITION) { + byteOffset = fileObject->CurrentByteOffset; + } else if (IrpSp->Parameters.Write.ByteOffset.LowPart == + FILE_WRITE_TO_END_OF_FILE) { // APPEND + byteOffset.QuadPart = zp->z_size; + } + } + + if (FlagOn(Irp->Flags, IRP_PAGING_IO)) { + + if (byteOffset.QuadPart >= zp->z_size) { + Status = STATUS_SUCCESS; + goto out; + } + + if (byteOffset.QuadPart + bufferLength > zp->z_size) + bufferLength = zp->z_size - byteOffset.QuadPart; + + // ASSERT(fileObject->PrivateCacheMap != NULL); + } + + if (!nocache && !CcCanIWrite(fileObject, bufferLength, TRUE, FALSE)) { + Status = STATUS_PENDING; + goto out; + } + + if (nocache && !pagingio && fileObject->SectionObjectPointer && + fileObject->SectionObjectPointer->DataSectionObject) { +#if 0 + IO_STATUS_BLOCK iosb; + ExAcquireResourceExclusiveLite(vp->FileHeader.PagingIoResource, + TRUE); + VERIFY3U(zccb->cacheinit, !=, 0); + + try { + CcFlushCache(fileObject->SectionObjectPointer, + &byteOffset, bufferLength, &iosb); + CcPurgeCacheSection(fileObject->SectionObjectPointer, + &byteOffset, bufferLength, FALSE); + } except(EXCEPTION_EXECUTE_HANDLER) { + Status = GetExceptionCode(); + } + + if (!NT_SUCCESS(Status) || !NT_SUCCESS(iosb.Status)) { + ExReleaseResourceLite(vp->FileHeader.PagingIoResource); + if (NT_SUCCESS(Status)) + Status = iosb.Status; + goto out; + } + + ExReleaseResourceLite(vp->FileHeader.PagingIoResource); +#endif + } + + void *SystemBuffer = MapUserBuffer(Irp); + + if (nocache) { + + } else { + + if (fileObject->PrivateCacheMap == NULL) { + vnode_pager_setsize(NULL, vp, zp->z_size, TRUE); + + zfs_init_cache(fileObject, vp); + // CcSetReadAheadGranularity(fileObject, + // READ_AHEAD_GRANULARITY); + } + + // If beyond valid data, zero between to expand + // (this is cachedfile, not paging io, extend ok) + if (byteOffset.QuadPart + bufferLength > zp->z_size) { +#if 0 + LARGE_INTEGER ZeroStart, BeyondZeroEnd; + ZeroStart.QuadPart = zp->z_size; + BeyondZeroEnd.QuadPart = + IrpSp->Parameters.Write.ByteOffset.QuadPart + + IrpSp->Parameters.Write.Length; + dprintf("%s: growing file\n", __func__); + // CACHE_MANAGER(34) + // See the comment for FAT_FILE_SYSTEM(0x23) + if (!CcZeroData(fileObject, + &ZeroStart, &BeyondZeroEnd, + TRUE)) { + dprintf("%s: CcZeroData failed\n", __func__); + } +#endif +// We have written "Length" into the "file" by the way of cache, so we need +// zp->z_size to reflect the new length, so we extend the file on disk, +// even though the actual writes will come later (from CcMgr). + dprintf("%s: growing file\n", __func__); + + // zfs_freesp() calls vnode_pager_setsize(); + error = zfs_freesp(zp, + byteOffset.QuadPart, bufferLength, + FWRITE, B_TRUE); + if (error) + goto fail; + } else { + // vnode_pager_setsize(vp, zp->z_size); + } + + // DO A NORMAL CACHED WRITE, if the MDL bit is not set, + if (!FlagOn(IrpSp->MinorFunction, IRP_MN_MDL)) { + VERIFY3U(zccb->cacheinit, !=, 0); + +// Since we may have grown the filesize, we need to give CcMgr a head's up. + vnode_pager_setsize(fileObject, vp, zp->z_size, FALSE); + + dprintf("CcWrite: offset [ 0x%llx - 0x%llx ] " + "len 0x%lx\n", byteOffset.QuadPart, + byteOffset.QuadPart + bufferLength, + bufferLength); + + Status = STATUS_SUCCESS; + + try { +#if (NTDDI_VERSION >= NTDDI_WIN8) + if (!CcCopyWriteEx(fileObject, + &byteOffset, + bufferLength, + TRUE, + SystemBuffer, + Irp->Tail.Overlay.Thread)) { +#else + if (!CcCopyWrite(fileObject, + &byteOffset, + bufferLength, + TRUE, + SystemBuffer)) { +#endif + dprintf("Could not wait\n"); + ASSERT0("failed copy"); + } + } except(EXCEPTION_EXECUTE_HANDLER) { + Status = GetExceptionCode(); + } + + // Irp->IoStatus.Status = STATUS_SUCCESS; + Irp->IoStatus.Information = bufferLength; + goto out; + } else { + VERIFY3U(zccb->cacheinit, !=, 0); + + // DO AN MDL WRITE + CcPrepareMdlWrite(fileObject, + &byteOffset, + bufferLength, + &Irp->MdlAddress, + &Irp->IoStatus); + + Status = Irp->IoStatus.Status; + goto out; + } + } + + struct iovec iov; + iov.iov_base = (void *)SystemBuffer; + iov.iov_len = bufferLength; + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, byteOffset.QuadPart, UIO_SYSSPACE, + bufferLength, 0); + + // dprintf("%s: offset %llx size %lx\n", __func__, + // byteOffset.QuadPart, bufferLength); + + dprintf("ZfsWrite: offset [ 0x%llx - 0x%llx ] len 0x%lx\n", + byteOffset.QuadPart, byteOffset.QuadPart + bufferLength, + bufferLength); + + if (FlagOn(Irp->Flags, IRP_PAGING_IO)) + // Should we call vnop_pageout instead? + error = zfs_write(zp, &uio, 0, NULL); + else + error = zfs_write(zp, &uio, 0, NULL); +fail: + // if (error == 0) + // zp->z_pflags |= ZFS_ARCHIVE; + switch (error) { + case 0: + break; + case EISDIR: + Status = STATUS_FILE_IS_A_DIRECTORY; + break; + case ENOSPC: + Status = STATUS_DISK_FULL; + break; + default: + Status = STATUS_INVALID_PARAMETER; + break; + } + + // EOF? + if ((bufferLength == zfs_uio_resid(&uio)) && error == ENOSPC) + Status = STATUS_DISK_FULL; + + // Update bytes written + Irp->IoStatus.Information = bufferLength - zfs_uio_resid(&uio); + + + if (Irp->IoStatus.Information) { + zfs_send_notify(zp->z_zfsvfs, zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_SIZE, + FILE_ACTION_MODIFIED); + } + + // Update the file offset +out: + if ((Status == STATUS_SUCCESS) && + (fileObject->Flags & FO_SYNCHRONOUS_IO) && + !(Irp->Flags & IRP_PAGING_IO)) { + fileObject->CurrentByteOffset.QuadPart = + byteOffset.QuadPart + Irp->IoStatus.Information; + } + + VN_RELE(vp); + +// dprintf(" FileName: %wZ offset 0x%llx len 0x%lx mdl %p System %p\n", +// &fileObject->FileName, byteOffset.QuadPart, bufferLength, +// Irp->MdlAddress, Irp->AssociatedIrp.SystemBuffer); + + return (Status); +} + +/* + * The lifetime of a delete. + * 1) If a file open is marked DELETE_ON_CLOSE in zfs_vnop_lookup() we will + * call vnode_setdeleteonclose(vp) to signal the intent. This is so + * file_standard_information can return DeletePending correctly + * (as well as a few more) + * 2) Upon IRP_MJ_CLEANUP (closing a file handle) we are expected to remove + * the file (as tested by IFStest.exe) we will call vnode_setdeleted(vp), + * this will: + * 3) Make zfs_vnop_lookup() return ENOENT when "setdeleted" is set. + * Making it appear as if the file was deleted - but retaining vp and zp + * as required by Windows. + * 4) Eventually IRP_MJ_CLOSE is called, and if final, we can release + * vp and zp, and if "setdeleted" was active, we can finally call + * delete_entry() to remove the file. + */ +NTSTATUS +delete_entry(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + // In Unix, both zfs_unlink and zfs_rmdir expect a filename, + // and we do not have that here + struct vnode *vp = NULL, *dvp = NULL; + int error; + char filename[MAXPATHLEN]; + ULONG outlen; + znode_t *zp = NULL; + mount_t *zmo = DeviceObject->DeviceExtension; + + if (IrpSp->FileObject->FsContext == NULL || + IrpSp->FileObject->FileName.Buffer == NULL || + IrpSp->FileObject->FileName.Length == 0) { + dprintf("%s: called with missing arguments, can't delete\n", + __func__); + return (STATUS_INSTANCE_NOT_AVAILABLE); // FIXME + } + + vp = IrpSp->FileObject->FsContext; + zp = VTOZ(vp); + ASSERT(zp != NULL); + + uint64_t parent = 0; + znode_t *dzp; + + if (zp->z_is_ctldir) + return (STATUS_SUCCESS); + + // No dvp, lookup parent + VERIFY(sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zp->z_zfsvfs), + &parent, sizeof (parent)) == 0); + error = zfs_zget(zp->z_zfsvfs, parent, &dzp); + if (error) + return (STATUS_INSTANCE_NOT_AVAILABLE); // FIXME + dvp = ZTOV(dzp); + + // Unfortunately, filename is littered with "\", clean it up, + // or search based on ID to get name? + dprintf("%s: deleting '%.*S'\n", __func__, + (int)IrpSp->FileObject->FileName.Length / sizeof (WCHAR), + IrpSp->FileObject->FileName.Buffer); + + error = RtlUnicodeToUTF8N(filename, MAXPATHLEN, &outlen, + IrpSp->FileObject->FileName.Buffer, + IrpSp->FileObject->FileName.Length); + + if (error != STATUS_SUCCESS && + error != STATUS_SOME_NOT_MAPPED) { + VN_RELE(dvp); + dprintf("%s: some illegal characters\n", __func__); + return (STATUS_INVALID_PARAMETER); // test.exe + return (STATUS_ILLEGAL_CHARACTER); + } + while (outlen > 0 && filename[outlen - 1] == '\\') outlen--; + filename[outlen] = 0; + + // FIXME, use z_name_cache and offset + char *finalname = NULL; + if ((finalname = strrchr(filename, '\\')) != NULL) + finalname = &finalname[1]; + else + finalname = filename; + + // Release final HOLD on item, ready for deletion + int isdir = vnode_isdir(vp); + + /* ZFS deletes from filename, so RELE last hold on vp. */ + VN_RELE(vp); + vp = NULL; + + if (isdir) { + + error = zfs_rmdir(VTOZ(dvp), finalname, NULL, NULL, 0); + + } else { + + error = zfs_remove(VTOZ(dvp), finalname, NULL, 0); + + } + + if (error == ENOTEMPTY) + error = STATUS_DIRECTORY_NOT_EMPTY; + + // Release parent. + VN_RELE(dvp); + + dprintf("%s: returning %d\n", __func__, error); + return (error); +} + +NTSTATUS +flush_buffers(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + NTSTATUS Status = 0; + + dprintf("%s: \n", __func__); + + if (FileObject == NULL || FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + struct vnode *vp = FileObject->FsContext; + if (VN_HOLD(vp) == 0) { + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + Status = zfs_vnop_ioctl_fullfsync(vp, NULL, zfsvfs); + VN_RELE(vp); + } + return (Status); +} + +NTSTATUS +query_security(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + NTSTATUS Status; + + dprintf("%s: \n", __func__); + + if (FileObject == NULL || FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + void *buf = MapUserBuffer(Irp); + + struct vnode *vp = FileObject->FsContext; + VN_HOLD(vp); + PSECURITY_DESCRIPTOR sd; + sd = vnode_security(vp); + ULONG buflen = IrpSp->Parameters.QuerySecurity.Length; + Status = SeQuerySecurityDescriptorInfo( + &IrpSp->Parameters.QuerySecurity.SecurityInformation, + buf, + &buflen, + &sd); + VN_RELE(vp); + + if (Status == STATUS_BUFFER_TOO_SMALL) { + Status = STATUS_BUFFER_OVERFLOW; + Irp->IoStatus.Information = buflen; + } else if (NT_SUCCESS(Status)) { + Irp->IoStatus.Information = + buflen; + } else { + Irp->IoStatus.Information = 0; + } + + return (Status); +} + +NTSTATUS +set_security(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + NTSTATUS Status = STATUS_SUCCESS; + + dprintf("%s: \n", __func__); + + if (FileObject == NULL || FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + struct vnode *vp = FileObject->FsContext; + VN_HOLD(vp); + PSECURITY_DESCRIPTOR oldsd; + oldsd = vnode_security(vp); + + + // READONLY check here + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + if (vfs_isrdonly(zfsvfs->z_vfs)) { + Status = STATUS_MEDIA_WRITE_PROTECTED; + goto err; + } + + Status = SeSetSecurityDescriptorInfo(NULL, + &IrpSp->Parameters.SetSecurity.SecurityInformation, + IrpSp->Parameters.SetSecurity.SecurityDescriptor, + (void **)&vp->security_descriptor, + PagedPool, + IoGetFileObjectGenericMapping()); + + if (!NT_SUCCESS(Status)) + goto err; + + ExFreePool(oldsd); + + // Now, we might need to update ZFS ondisk information + vattr_t vattr; + vattr.va_mask = 0; + BOOLEAN defaulted; + + if (IrpSp->Parameters.SetSecurity.SecurityInformation & + OWNER_SECURITY_INFORMATION) { + PSID owner; + Status = RtlGetOwnerSecurityDescriptor(vnode_security(vp), + &owner, &defaulted); + if (Status == STATUS_SUCCESS) { + vattr.va_uid = zfs_sid2uid(owner); + vattr.va_mask |= ATTR_UID; + } + } + if (IrpSp->Parameters.SetSecurity.SecurityInformation & + GROUP_SECURITY_INFORMATION) { + PSID group; + Status = RtlGetGroupSecurityDescriptor(vnode_security(vp), + &group, &defaulted); + if (Status == STATUS_SUCCESS) { + // uid/gid reverse is identical + vattr.va_gid = zfs_sid2uid(group); + vattr.va_mask |= ATTR_GID; + } + } + + // Do we need to update ZFS? + if (vattr.va_mask != 0) { + zfs_setattr(zp, &vattr, 0, NULL, NULL); + Status = STATUS_SUCCESS; + } + + Irp->IoStatus.Information = 0; + zfs_send_notify(zfsvfs, zp->z_name_cache, zp->z_name_offset, + FILE_NOTIFY_CHANGE_SECURITY, + FILE_ACTION_MODIFIED); + +err: + VN_RELE(vp); + return (Status); +} + +// #define IOCTL_VOLUME_BASE ((DWORD) 'V') +// #define IOCTL_VOLUME_GET_GPT_ATTRIBUTES +// CTL_CODE(IOCTL_VOLUME_BASE,14,METHOD_BUFFERED,FILE_ANY_ACCESS) + +#define IOCTL_VOLUME_POST_ONLINE \ + CTL_CODE(IOCTL_VOLUME_BASE, 25, METHOD_BUFFERED, \ + FILE_READ_ACCESS | FILE_WRITE_ACCESS) + +NTSTATUS +ioctl_storage_get_device_number(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (STORAGE_DEVICE_NUMBER)) { + Irp->IoStatus.Information = sizeof (STORAGE_DEVICE_NUMBER); + return (STATUS_BUFFER_TOO_SMALL); + } + + PSTORAGE_DEVICE_NUMBER sdn = Irp->AssociatedIrp.SystemBuffer; + sdn->DeviceNumber = 0; + sdn->DeviceType = FILE_DEVICE_VIRTUAL_DISK; + sdn->PartitionNumber = -1; // -1 means can't be partitioned + + Irp->IoStatus.Information = sizeof (STORAGE_DEVICE_NUMBER); + return (STATUS_SUCCESS); +} + + +NTSTATUS +ioctl_volume_get_volume_disk_extents(PDEVICE_OBJECT DeviceObject, + PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + VOLUME_DISK_EXTENTS *vde = Irp->AssociatedIrp.SystemBuffer; + + if (IrpSp->Parameters.QueryFile.Length < sizeof (VOLUME_DISK_EXTENTS)) { + Irp->IoStatus.Information = sizeof (VOLUME_DISK_EXTENTS); + return (STATUS_BUFFER_TOO_SMALL); + } + + Irp->IoStatus.Information = sizeof (VOLUME_DISK_EXTENTS); + RtlZeroMemory(vde, sizeof (VOLUME_DISK_EXTENTS)); + vde->NumberOfDiskExtents = 1; + + return (STATUS_SUCCESS); +} + +NTSTATUS +volume_create(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + mount_t *zmo = DeviceObject->DeviceExtension; + + // This is also called from fsContext when IRP_MJ_CREATE + // FileName is NULL + /* VERIFY(zmo->type == MOUNT_TYPE_DCB); */ + if (zmo->vpb != NULL) + IrpSp->FileObject->Vpb = zmo->vpb; + else + IrpSp->FileObject->Vpb = DeviceObject->Vpb; + + // dprintf("Setting FileObject->Vpb to %p\n", IrpSp->FileObject->Vpb); + // SetFileObjectForVCB(IrpSp->FileObject, zmo); + // IrpSp->FileObject->SectionObjectPointer = + // &zmo->SectionObjectPointers; + // IrpSp->FileObject->FsContext = &zmo->VolumeFileHeader; + +/* + * Check the ShareAccess requested: + * 0 : exclusive + * FILE_SHARE_READ : The file can be opened for read access by other threads + * FILE_SHARE_WRITE : The file can be opened for write access by other threads + * FILE_SHARE_DELETE : The file can be opened for del access by other threads + */ + if ((IrpSp->Parameters.Create.ShareAccess == 0) && + zmo->volume_opens != 0) { + dprintf("%s: sharing violation\n", __func__); + return (STATUS_SHARING_VIOLATION); + } + + atomic_inc_64(&zmo->volume_opens); + Irp->IoStatus.Information = FILE_OPENED; + return (STATUS_SUCCESS); +} + +NTSTATUS +volume_close(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + mount_t *zmo = DeviceObject->DeviceExtension; + VERIFY(zmo->type == MOUNT_TYPE_DCB); + atomic_dec_64(&zmo->volume_opens); + return (STATUS_SUCCESS); +} + +/* + * IRP_MJ_CLEANUP - sent when Windows is done with FileObject HANDLE + * (one of many) + * the vp is not released here, just decrease a count of vp. + */ +int +zfs_fileobject_cleanup(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, vnode_t **hold_vp) +{ + int Status = STATUS_SUCCESS; + mount_t *zmo = NULL; + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + zfs_dirlist_t *zccb = IrpSp->FileObject->FsContext2; + + znode_t *zp = VTOZ(vp); // zp for notify removal + + vnode_rele(vp); // Release longterm hold finally. + + dprintf("IRP_MJ_CLEANUP: '%s' iocount %u usecount %u\n", + zp && zp->z_name_cache ? zp->z_name_cache : "", + vp->v_iocount, vp->v_usecount); + + vnode_lock(vp); + IoRemoveShareAccess(IrpSp->FileObject, &vp->share_access); + vnode_unlock(vp); + + int isdir = vnode_isdir(vp); + + zmo = DeviceObject->DeviceExtension; + VERIFY(zmo->type == MOUNT_TYPE_VCB); + + if (zp != NULL) { + + if (!isdir) { + if (vnode_flushcache(vp, IrpSp->FileObject, + FALSE)) + dprintf( + "cleanup: flushcache said no?\n"); + } + +/* + * Technically, this should only be called on the FileObject which + * opened the file with DELETE_ON_CLOSE - in fastfat, that is stored + * in the ccb (context) set in FsContext2, which holds data for each + * FileObject context. Possibly, we should as well. (We do for dirs) + */ + if (zccb && zccb->deleteonclose) { + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + + zccb->deleteonclose = 0; + + if (zp->z_name_cache != NULL) { + if (isdir) { + dprintf("DIR: FileDelete " + "'%s' name '%s'\n", + zp->z_name_cache, + &zp->z_name_cache[ + zp->z_name_offset]); + zfs_send_notify(zfsvfs, + zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_DIR_NAME, + FILE_ACTION_REMOVED); + } else { + dprintf("FILE: FileDelete " + "'%s' name '%s'\n", + zp->z_name_cache, + &zp->z_name_cache[ + zp->z_name_offset]); + zfs_send_notify(zfsvfs, + zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_FILE_NAME, + FILE_ACTION_REMOVED); + } + } + + // Windows needs us to unlink it now, since CLOSE + // can be delayed and parent deletions might + // fail (ENOTEMPTY). + + // This releases zp! + Status = delete_entry(DeviceObject, Irp, IrpSp); + if (Status != 0) + dprintf("Deletion failed: %d\n", + Status); + + zp = NULL; + + // delete_entry will always consume an IOCOUNT. + *hold_vp = NULL; + + Status = STATUS_SUCCESS; + +// FILE_CLEANUP_UNKNOWN FILE_CLEANUP_WRONG_DEVICE FILE_CLEANUP_FILE_REMAINS +// FILE_CLEANUP_FILE_DELETED FILE_CLEANUP_LINK_DELETED +// FILE_CLEANUP_STREAM_DELETED FILE_CLEANUP_POSIX_STYLE_DELETE +#if defined(ZFS_FS_ATTRIBUTE_CLEANUP_INFO) && defined(ZFS_FS_ATTRIBUTE_POSIX) + Irp->IoStatus.Information = + FILE_CLEANUP_FILE_DELETED | + FILE_CLEANUP_POSIX_STYLE_DELETE; +#elif defined(ZFS_FS_ATTRIBUTE_CLEANUP_INFO) + Irp->IoStatus.Information = + FILE_CLEANUP_FILE_DELETED; +#endif + + } else { + // fastfat zeros end of file here if last + // open closed + } + + } + + if (isdir && IrpSp->FileObject) { + dprintf("Removing all notifications for " + "directory: %p\n", zp); + + FsRtlNotifyCleanup(zmo->NotifySync, &zmo->DirNotifyList, + IrpSp->FileObject->FsContext2); + } + + // dprintf("cleanup: vp %p attempt to ditch CCMgr\n", vp); + // if (vnode_flushcache(vp, IrpSp->FileObject, TRUE) == 1) { + // dprintf("vp %p clearing out FsContext\n", vp); + // IrpSp->FileObject->FsContext = NULL; + // } + // vnode_fileobject_remove(vp, IrpSp->FileObject); + // zfs_decouplefileobject(vp, IrpSp->FileObject); + if (IrpSp->FileObject) + IrpSp->FileObject->Flags |= FO_CLEANUP_COMPLETE; + Status = STATUS_SUCCESS; + } + + return (Status); +} + +/* + * IRP_MJ_CLOSE - sent when Windows is done with FileObject, and we can + * free memory. + */ +int +zfs_fileobject_close(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, vnode_t **hold_vp) +{ + int Status = STATUS_SUCCESS; + + ASSERT(hold_vp != NULL); + + if (IrpSp->FileObject) { + +// Careful not to print something before vnode_fileobject_remove - +// if print is swapped out, we think fileobject is still valid. +// dprintf("IRP_MJ_CLOSE: '%wZ' \n", &IrpSp->FileObject->FileName); + + if (IrpSp->FileObject->FsContext) { +// dprintf("CLOSE clearing FsContext of FO 0x%llx\n", +// IrpSp->FileObject); +// Mark vnode for cleanup, we grab a HOLD to make sure it isn't +// released right here, but marked to be released upon +// reaching 0 count + vnode_t *vp = IrpSp->FileObject->FsContext; + zfs_dirlist_t *zccb = IrpSp->FileObject->FsContext2; + + int isdir = vnode_isdir(vp); + +/* + * First encourage Windows to release the FileObject, CcMgr etc, + * flush everything. + */ + + // FileObject should/could no longer point to vp. + // this also frees zccb + zfs_decouplefileobject(vp, IrpSp->FileObject); + zccb = NULL; + + // vnode_fileobject_remove(vp, IrpSp->FileObject); +// if (isdir) { + + // fastfat also flushes while(parent) dir here, + // if !iocount +// } + +/* + * If we can release now, do so. + * If the reference count for the per-file context structure reaches zero + * and both the ImageSectionObject and DataSectionObject of the + * SectionObjectPointers field from the FILE_OBJECT is zero, the + * filter driver may then delete the per-file context data. + * + */ + + if (!vnode_isvroot(vp)) { + +/* Take hold from dispatcher, try to release in recycle */ + *hold_vp = NULL; + +// Release vp - vnode_recycle expects iocount==1 +// we don't recycle root (unmount does) or RELE on +// recycle error + if (vnode_isvroot(vp) || + (vnode_recycle(vp) != 0)) { +// If recycle failed, manually release dispatcher's HOLD + dprintf("IRP_CLOSE failed to recycle. " + "is_empty %d\n", + vnode_fileobject_empty(vp, 1)); + VN_RELE(vp); + } + + Status = STATUS_SUCCESS; + + vp = NULL; // Paranoia, signal it is gone. + + } else { /* root node */ + Status = STATUS_SUCCESS; + } + } + } + + return (Status); +} + +/* + * We received a long-lived ioctl, so lets setup a taskq to handle it, + * and return pending + * This code was proof-of-concept, and is NOT used. + */ +void +zfsdev_async_thread(void *arg) +{ + NTSTATUS Status; + PIRP Irp; + Irp = (PIRP)arg; + + dprintf("%s: starting ioctl\n", __func__); + + /* Use FKIOCTL to make sure it calls memcpy instead */ + Status = zfsdev_ioctl(NULL, Irp, FKIOCTL); + + dprintf("%s: finished ioctl %ld\n", __func__, Status); + + PMDL mdl = Irp->Tail.Overlay.DriverContext[0]; + if (mdl) { + UnlockAndFreeMdl(mdl); + Irp->Tail.Overlay.DriverContext[0] = NULL; + } + void *fp = Irp->Tail.Overlay.DriverContext[1]; + if (fp) { + ObDereferenceObject(fp); + ZwClose(Irp->Tail.Overlay.DriverContext[2]); + } + + IoCompleteRequest(Irp, + Status == STATUS_SUCCESS ? IO_DISK_INCREMENT : IO_NO_INCREMENT); +} + +/* Not used */ +NTSTATUS +zfsdev_async(PDEVICE_OBJECT DeviceObject, PIRP Irp) +{ + int error; + PMDL mdl = NULL; + PIO_STACK_LOCATION IrpSp; + zfs_cmd_t *zc; + void *fp = NULL; + + IrpSp = IoGetCurrentIrpStackLocation(Irp); + + IoMarkIrpPending(Irp); + + /* + * A separate thread to the one that called us may not access the + * buffer from userland, So we have to map the in/out buffer, + * and put that address in its place. + */ + error = ddi_copysetup( + IrpSp->Parameters.DeviceIoControl.Type3InputBuffer, + sizeof (zfs_cmd_t), + &IrpSp->Parameters.DeviceIoControl.Type3InputBuffer, &mdl); + if (error) + return (error); + + /* Save the MDL so we can free it once done */ + Irp->Tail.Overlay.DriverContext[0] = mdl; + + /* + * We would also need to handle zc->zc_nvlist_src and zc->zc_nvlist_dst + * which is tricker, since they are unpacked into nvlists deep + * in zfsdev_ioctl + * The same problem happens for the filedescriptor from userland, + * also needs to be kernelMode + */ + zc = IrpSp->Parameters.DeviceIoControl.Type3InputBuffer; + + if (zc->zc_cookie) { + error = ObReferenceObjectByHandle((HANDLE)zc->zc_cookie, 0, 0, + KernelMode, &fp, 0); + if (error != STATUS_SUCCESS) + goto out; + Irp->Tail.Overlay.DriverContext[1] = fp; + + HANDLE h = NULL; + error = ObOpenObjectByPointer(fp, + OBJ_FORCE_ACCESS_CHECK | OBJ_KERNEL_HANDLE, NULL, + GENERIC_READ|GENERIC_WRITE, *IoFileObjectType, + KernelMode, &h); + if (error != STATUS_SUCCESS) + goto out; + dprintf("mapped filed is 0x%p\n", h); + zc->zc_cookie = (uint64_t)h; + Irp->Tail.Overlay.DriverContext[2] = h; + } + + taskq_dispatch(system_taskq, zfsdev_async_thread, (void*)Irp, TQ_SLEEP); + return (STATUS_PENDING); + +out: + if (mdl) { + UnlockAndFreeMdl(mdl); + } + if (fp) { + ObDereferenceObject(fp); + } + return (error); +} + +NTSTATUS pnp_query_di(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp); + +/* + * This is the ioctl handler for ioctl done directly on /dev/zfs node. + * This means all the internal ZFS ioctls, like ZFS_IOC_SEND etc. + * But, we will also get general Windows ioctls, not specific to + * volumes, or filesystems. + * Incidentally, cstyle is confused about the function_class + */ +_Function_class_(DRIVER_DISPATCH) + static NTSTATUS + ioctlDispatcher(_In_ PDEVICE_OBJECT DeviceObject, _Inout_ PIRP *PIrp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status; + PIRP Irp = *PIrp; + + PAGED_CODE(); + + dprintf(" %s: enter: major %d: minor %d: %s ioctlDeviceObject\n", + __func__, IrpSp->MajorFunction, IrpSp->MinorFunction, + major2str(IrpSp->MajorFunction, IrpSp->MinorFunction)); + + Status = STATUS_NOT_IMPLEMENTED; + + switch (IrpSp->MajorFunction) { + + case IRP_MJ_CREATE: + dprintf("IRP_MJ_CREATE: zfsdev FileObject %p name '%wZ' " + "length %u flags 0x%x\n", + IrpSp->FileObject, IrpSp->FileObject->FileName, + IrpSp->FileObject->FileName.Length, IrpSp->Flags); + Status = zfsdev_open((dev_t)IrpSp->FileObject, Irp); + break; + case IRP_MJ_CLOSE: + Status = zfsdev_release((dev_t)IrpSp->FileObject, Irp); + + // uninstall + extern kmutex_t zfsdev_state_lock; + if (fsDiskDeviceObject == NULL) { + mutex_enter(&zfsdev_state_lock); + if (ioctlDeviceObject != NULL) { + ObDereferenceObject(ioctlDeviceObject); + IoDeleteDevice(ioctlDeviceObject); + ioctlDeviceObject = NULL; + } + mutex_exit(&zfsdev_state_lock); + } + break; + case IRP_MJ_DEVICE_CONTROL: + { + /* Is it a ZFS ioctl? */ + ulong_t cmd = + IrpSp->Parameters.DeviceIoControl.IoControlCode; + + if ((DEVICE_TYPE_FROM_CTL_CODE(cmd) == ZFSIOCTL_TYPE)) { + ulong_t cmd2; + + cmd2 = DEVICE_FUNCTION_FROM_CTL_CODE(cmd); + if ((cmd2 >= ZFSIOCTL_BASE + ZFS_IOC_FIRST && + cmd2 < ZFSIOCTL_BASE + ZFS_IOC_LAST)) { + + cmd2 -= ZFSIOCTL_BASE; + +/* + * Some IOCTL are very long-living, so we will put them in the + * background and return PENDING. Possibly we should always do + * this logic, but some ioctls are really short lived. + */ + switch (cmd2) { + case ZFS_IOC_UNREGISTER_FS: +// We abuse returnedBytes to send back busy + Irp->IoStatus.Information = + zfs_ioc_unregister_fs(); + Status = STATUS_SUCCESS; + break; +/* + * So to do ioctl in async mode is a hassle, we have to do the copyin/copyout + * MDL work in *this* thread, as the thread we spawn does not have access. + * This would also include zc->zc_nvlist_src / zc->zc_nvlist_dst, so + * zfsdev_ioctl() would need to be changed quite a bit. The file-descriptor + * passed in (zfs send/recv) also needs to be opened for kernel mode. This + * code is left here as an example on how it can be done + * (without zc->zc_nvlist_*) but we currently do not use it. + * Everything is handled synchronously. + * + * case ZFS_IOC_SEND: + * Status = zfsdev_async(DeviceObject, Irp); + * break; + * + */ + default: + Status = zfsdev_ioctl( + DeviceObject, Irp, 0); + } // switch cmd for async + break; + } + } + /* Not ZFS ioctl, handle Windows ones */ + switch (cmd) { + case IOCTL_VOLUME_GET_GPT_ATTRIBUTES: + dprintf("IOCTL_VOLUME_GET_GPT_ATTRIBUTES\n"); + Status = 0; + break; + case IOCTL_MOUNTDEV_QUERY_DEVICE_NAME: + dprintf("IOCTL_MOUNTDEV_QUERY_DEVICE_NAME\n"); + Status = ioctl_query_device_name(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_UNIQUE_ID: + dprintf("IOCTL_MOUNTDEV_QUERY_UNIQUE_ID\n"); + Status = ioctl_query_unique_id(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_STABLE_GUID: + dprintf("IOCTL_MOUNTDEV_QUERY_STABLE_GUID\n"); + Status = ioctl_query_stable_guid(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME: + dprintf("IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME\n"); + break; + case IOCTL_VOLUME_ONLINE: + dprintf("IOCTL_VOLUME_ONLINE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_DISK_IS_WRITABLE: + dprintf("IOCTL_DISK_IS_WRITABLE\n"); + mount_t *zmo = DeviceObject->DeviceExtension; + VERIFY(zmo->type == MOUNT_TYPE_VCB); + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs != NULL && zfsvfs->z_rdonly) + Status = STATUS_MEDIA_WRITE_PROTECTED; + else + Status = STATUS_SUCCESS; + break; + case IOCTL_DISK_MEDIA_REMOVAL: + dprintf("IOCTL_DISK_MEDIA_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_STORAGE_MEDIA_REMOVAL: + dprintf("IOCTL_STORAGE_MEDIA_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_VOLUME_POST_ONLINE: + dprintf("IOCTL_VOLUME_POST_ONLINE\n"); + Status = STATUS_SUCCESS; + break; + /* kstat ioctls */ + case KSTAT_IOC_CHAIN_ID: + dprintf("KSTAT_IOC_CHAIN_ID\n"); + Status = spl_kstat_chain_id(DeviceObject, Irp, + IrpSp); + break; + case KSTAT_IOC_READ: + dprintf("KSTAT_IOC_READ\n"); + Status = spl_kstat_read(DeviceObject, Irp, + IrpSp); + break; + case KSTAT_IOC_WRITE: + dprintf("KSTAT_IOC_WRITE\n"); + Status = spl_kstat_write(DeviceObject, Irp, + IrpSp); + break; + default: + dprintf("**** unknown Windows IOCTL: 0x%lx\n", + cmd); + } + + } + break; + + case IRP_MJ_CLEANUP: + Status = STATUS_SUCCESS; + break; + + case IRP_MJ_FILE_SYSTEM_CONTROL: + switch (IrpSp->MinorFunction) { + case IRP_MN_MOUNT_VOLUME: + dprintf("IRP_MN_MOUNT_VOLUME ioctl\n"); + Status = zfs_vnop_mount(DeviceObject, Irp, IrpSp); + break; + default: + dprintf("IRP_MJ_FILE_SYSTEM_CONTROL default case!\n"); + break; + } + break; + + case IRP_MJ_PNP: + switch (IrpSp->MinorFunction) { + case IRP_MN_QUERY_CAPABILITIES: + Status = QueryCapabilities(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_DEVICE_RELATIONS: + Status = STATUS_NOT_IMPLEMENTED; + break; + case IRP_MN_QUERY_ID: + Status = pnp_query_id(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_PNP_DEVICE_STATE: + Status = pnp_device_state(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_REMOVE_DEVICE: + dprintf("IRP_MN_QUERY_REMOVE_DEVICE\n"); + Status = STATUS_UNSUCCESSFUL; + break; + case IRP_MN_SURPRISE_REMOVAL: + dprintf("IRP_MN_SURPRISE_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_REMOVE_DEVICE: + dprintf("IRP_MN_REMOVE_DEVICE\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_CANCEL_REMOVE_DEVICE: + dprintf("IRP_MN_CANCEL_REMOVE_DEVICE\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_QUERY_INTERFACE: + Status = pnp_query_di(DeviceObject, Irp, IrpSp); + break; + } + break; + + } + + return (Status); +} + +/* + * This is the IOCTL handler for the "virtual" disk volumes we create + * to mount ZFS, and ZVOLs, things like get partitions, and volume size. + * But also open/read/write/close requests of volume access (like dd'ing the + * /dev/diskX node directly). + */ +_Function_class_(DRIVER_DISPATCH) + static NTSTATUS + diskDispatcher( + _In_ PDEVICE_OBJECT DeviceObject, + _Inout_ PIRP *PIrp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status; + PIRP Irp = *PIrp; + + PAGED_CODE(); + + dprintf(" %s: enter: major %d: minor %d: %s diskDeviceObject\n", + __func__, IrpSp->MajorFunction, IrpSp->MinorFunction, + major2str(IrpSp->MajorFunction, IrpSp->MinorFunction)); + + Status = STATUS_NOT_IMPLEMENTED; + + switch (IrpSp->MajorFunction) { + + case IRP_MJ_CREATE: + dprintf("IRP_MJ_CREATE: volume FileObject %p related %p " + "name '%wZ' flags 0x%x\n", + IrpSp->FileObject, + IrpSp->FileObject ? + IrpSp->FileObject->RelatedFileObject : NULL, + IrpSp->FileObject->FileName, IrpSp->Flags); + + Status = volume_create(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_CLOSE: + Status = volume_close(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_DEVICE_CONTROL: + { + ulong_t cmd = IrpSp->Parameters.DeviceIoControl.IoControlCode; + /* Not ZFS ioctl, handle Windows ones */ + switch (cmd) { + case IOCTL_VOLUME_GET_GPT_ATTRIBUTES: + dprintf("IOCTL_VOLUME_GET_GPT_ATTRIBUTES\n"); + Status = ioctl_get_gpt_attributes(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_DEVICE_NAME: + dprintf("IOCTL_MOUNTDEV_QUERY_DEVICE_NAME\n"); + Status = ioctl_query_device_name(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_UNIQUE_ID: + dprintf("IOCTL_MOUNTDEV_QUERY_UNIQUE_ID\n"); + Status = ioctl_query_unique_id(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_STABLE_GUID: + dprintf("IOCTL_MOUNTDEV_QUERY_STABLE_GUID\n"); + Status = ioctl_mountdev_query_stable_guid(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME: + dprintf("IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME\n"); + Status = ioctl_mountdev_query_suggested_link_name( + DeviceObject, Irp, IrpSp); + break; + case IOCTL_VOLUME_ONLINE: + dprintf("IOCTL_VOLUME_ONLINE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_VOLUME_OFFLINE: + case IOCTL_VOLUME_IS_OFFLINE: + dprintf("IOCTL_VOLUME_OFFLINE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_DISK_IS_WRITABLE: + dprintf("IOCTL_DISK_IS_WRITABLE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_DISK_MEDIA_REMOVAL: + dprintf("IOCTL_DISK_MEDIA_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_STORAGE_MEDIA_REMOVAL: + dprintf("IOCTL_STORAGE_MEDIA_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_VOLUME_POST_ONLINE: + dprintf("IOCTL_VOLUME_POST_ONLINE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_STORAGE_GET_HOTPLUG_INFO: + dprintf("IOCTL_STORAGE_GET_HOTPLUG_INFO\n"); + Status = ioctl_storage_get_hotplug_info(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_STORAGE_QUERY_PROPERTY: + dprintf("IOCTL_STORAGE_QUERY_PROPERTY\n"); + Status = ioctl_storage_query_property(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS: + dprintf("IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS\n"); + Status = ioctl_volume_get_volume_disk_extents( + DeviceObject, Irp, IrpSp); + break; + case IOCTL_STORAGE_GET_DEVICE_NUMBER: + dprintf("IOCTL_STORAGE_GET_DEVICE_NUMBER\n"); + Status = ioctl_storage_get_device_number(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_DISK_CHECK_VERIFY: + Status = STATUS_SUCCESS; + break; + case IOCTL_STORAGE_CHECK_VERIFY2: + dprintf("IOCTL_STORAGE_CHECK_VERIFY2\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_VOLUME_IS_DYNAMIC: + { + uint8_t *buf = (UINT8*)Irp->AssociatedIrp.SystemBuffer; + *buf = 1; + Irp->IoStatus.Information = 1; + Status = STATUS_SUCCESS; + break; + } + case IOCTL_MOUNTDEV_LINK_CREATED: + dprintf("IOCTL_MOUNTDEV_LINK_CREATED\n"); + Status = STATUS_SUCCESS; + break; + case 0x4d0010: +// Same as IOCTL_MOUNTDEV_LINK_CREATED but bit 14,15 are 0 (access permissions) + dprintf("IOCTL_MOUNTDEV_LINK_CREATED v2\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_MOUNTDEV_LINK_DELETED: + dprintf("IOCTL_MOUNTDEV_LINK_DELETED\n"); + Status = STATUS_SUCCESS; + break; + case 0x4d0014: +// Same as IOCTL_MOUNTDEV_LINK_DELETED but bit 14,15 are 0 (access permissions) + dprintf("IOCTL_MOUNTDEV_LINK_DELETED v2\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_DISK_GET_PARTITION_INFO_EX: + dprintf("IOCTL_DISK_GET_PARTITION_INFO_EX\n"); + Status = ioctl_disk_get_partition_info_ex(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_DISK_GET_DRIVE_GEOMETRY: + dprintf("IOCTL_DISK_GET_DRIVE_GEOMETRY\n"); + Status = ioctl_disk_get_drive_geometry(DeviceObject, + Irp, IrpSp); + break; + default: + dprintf("**** unknown disk Windows IOCTL: 0x%lx\n", + cmd); + } + + } + break; + + case IRP_MJ_CLEANUP: + Status = STATUS_SUCCESS; + break; + + // Technically we don't really let them read from the virtual + // devices that hold the ZFS filesystem, so we just return all zeros. + case IRP_MJ_READ: + dprintf("disk fake read\n"); + uint64_t bufferLength; + bufferLength = IrpSp->Parameters.Read.Length; + Irp->IoStatus.Information = bufferLength; + Status = STATUS_SUCCESS; + break; + + case IRP_MJ_WRITE: + dprintf("disk fake write\n"); + Irp->IoStatus.Information = IrpSp->Parameters.Write.Length; + Status = STATUS_SUCCESS; + break; + + case IRP_MJ_FILE_SYSTEM_CONTROL: + switch (IrpSp->MinorFunction) { + case IRP_MN_MOUNT_VOLUME: + dprintf("IRP_MN_MOUNT_VOLUME disk\n"); + Status = zfs_vnop_mount(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_USER_FS_REQUEST: + dprintf("IRP_MN_USER_FS_REQUEST: FsControlCode 0lx%x\n", + IrpSp->Parameters.FileSystemControl.FsControlCode); + Status = user_fs_request(DeviceObject, PIrp, IrpSp); + break; + default: + dprintf("IRP_MN_unknown: 0x%x\n", IrpSp->MinorFunction); + break; + } + break; + + case IRP_MJ_QUERY_INFORMATION: + dprintf("volume calling query_information warning\n"); + Status = query_information(DeviceObject, Irp, IrpSp); + break; + + case IRP_MJ_PNP: + switch (IrpSp->MinorFunction) { + case IRP_MN_QUERY_CAPABILITIES: + Status = QueryCapabilities(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_DEVICE_RELATIONS: + Status = STATUS_NOT_IMPLEMENTED; + dprintf("DeviceRelations.Type 0x%x\n", + IrpSp->Parameters.QueryDeviceRelations.Type); + break; + case IRP_MN_QUERY_ID: + Status = pnp_query_id(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_PNP_DEVICE_STATE: + Status = pnp_device_state(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_REMOVE_DEVICE: + dprintf("IRP_MN_QUERY_REMOVE_DEVICE\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_SURPRISE_REMOVAL: + dprintf("IRP_MN_SURPRISE_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_REMOVE_DEVICE: + dprintf("IRP_MN_REMOVE_DEVICE\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_CANCEL_REMOVE_DEVICE: + dprintf("IRP_MN_CANCEL_REMOVE_DEVICE\n"); + Status = STATUS_SUCCESS; + break; + } + break; + + } + + return (Status); +} + +/* + * This is the main FileSystem IOCTL handler. This is where the filesystem + * vnops happen and we handle everything with files and directories in ZFS. + */ +_Function_class_(DRIVER_DISPATCH) + static NTSTATUS + fsDispatcher( + _In_ PDEVICE_OBJECT DeviceObject, + _Inout_ PIRP *PIrp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status; + struct vnode *hold_vp = NULL; + PIRP Irp = *PIrp; + + PAGED_CODE(); + + dprintf(" %s: enter: major %d: minor %d: %s fsDeviceObject\n", + __func__, IrpSp->MajorFunction, IrpSp->MinorFunction, + major2str(IrpSp->MajorFunction, IrpSp->MinorFunction)); + +#ifdef DEBUG_IOCOUNT + int skiplock = 0; +/* + * Watch out for re-entrant calls! MJ_READ, can call CCMGR, which calls + * MJ_READ! + * This could be a bigger issue, in that any mutex calls in the + * zfs_read()/zfs_write stack could die - including rangelocks, + * rwlocks. Investigate. (+zfs_freesp/zfs_trunc) + */ + if (mutex_owned(&GIANT_SERIAL_LOCK)) + skiplock = 1; + else + mutex_enter(&GIANT_SERIAL_LOCK); + + zfsvfs_t *zfsvfs = NULL; +#endif + +/* + * Like VFS layer in upstream, we hold the "vp" here before calling into + * the VNOP handlers. + * There is one special case, IRP_MJ_CREATE / zfs_vnop_lookup, which has + * no vp to start, + * and assigns the vp on success (held). + * We also pass "hold_vp" down to delete_entry, so it can release the + * last hold to delete + */ + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + hold_vp = IrpSp->FileObject->FsContext; + if (VN_HOLD(hold_vp) != 0) { + + // If we were given a vp, but can't hold the vp, + // we should fail this OP. + Irp->IoStatus.Information = 0; + hold_vp = NULL; + return (STATUS_INVALID_PARAMETER); + + } else { + + // Add FO to vp, if this is the first we've heard of it + vnode_fileobject_add(IrpSp->FileObject->FsContext, + IrpSp->FileObject); + + // This is useful if you have iocount leaks, and do + // only single-threaded operations +#ifdef DEBUG_IOCOUNT + if (!vnode_isvroot(hold_vp) && vnode_isdir(hold_vp)) + ASSERT(hold_vp->v_iocount == 1); + zfsvfs = VTOZ(hold_vp)->z_zfsvfs; +#endif + } + } +/* + * Inside VNOP handlers, we no longer need to call VN_HOLD() on *this* vp + * (but might for dvp etc) and eventually that code will be removed, if this + * style works out. + */ + + Status = STATUS_NOT_IMPLEMENTED; + + switch (IrpSp->MajorFunction) { + + case IRP_MJ_CREATE: + if (IrpSp->Parameters.Create.Options & FILE_OPEN_BY_FILE_ID) + dprintf("IRP_MJ_CREATE: FileObject %p related %p " + "FileID 0x%llx flags 0x%x sharing 0x%x options " + "0x%lx\n", + IrpSp->FileObject, + IrpSp->FileObject ? + IrpSp->FileObject->RelatedFileObject : + NULL, + *((uint64_t *)IrpSp->FileObject->FileName.Buffer), + IrpSp->Flags, IrpSp->Parameters.Create.ShareAccess, + IrpSp->Parameters.Create.Options); + else + dprintf("IRP_MJ_CREATE: FileObject %p related %p " + "name '%wZ' flags 0x%x sharing 0x%x options " + "%s attr 0x%x DesAcc 0x%lx\n", + IrpSp->FileObject, + IrpSp->FileObject ? + IrpSp->FileObject->RelatedFileObject : + NULL, + IrpSp->FileObject->FileName, IrpSp->Flags, + IrpSp->Parameters.Create.ShareAccess, + create_options(IrpSp->Parameters.Create.Options), + IrpSp->Parameters.Create.FileAttributes, + IrpSp->Parameters.Create.SecurityContext-> + DesiredAccess); + + Irp->IoStatus.Information = FILE_OPENED; + Status = STATUS_SUCCESS; + +#if 0 + // Disallow autorun.inf for now + if (IrpSp && IrpSp->FileObject && + IrpSp->FileObject->FileName.Buffer && + _wcsicmp(IrpSp->FileObject->FileName.Buffer, + L"\\autorun.inf") == 0) { + Irp->IoStatus.Information = FILE_DOES_NOT_EXIST; + Status = STATUS_OBJECT_NAME_NOT_FOUND; + break; + } +#endif + + mount_t *zmo = DeviceObject->DeviceExtension; + VERIFY(zmo->type == MOUNT_TYPE_VCB); + + // + // Check if we are opening the volume and not a file/directory. + // We are opening the volume if the name is empty and there + // isn't a related file object. If there is a related + // file object then it is the Vcb itself. + // + + // We have a name, so we are looking for something specific + // Attempt to find the requested object + if (IrpSp && IrpSp->FileObject && + /* IrpSp->FileObject->FileName.Buffer && */ + zmo) { + + Status = zfs_vnop_lookup(Irp, IrpSp, zmo); + + if (Status == EROFS) + Status = STATUS_MEDIA_WRITE_PROTECTED; + } + break; + +/* + * CLEANUP comes before CLOSE. The IFSTEST.EXE on notifications + * require them to arrive at CLEANUP time, and deemed too late + * to be sent from CLOSE. It is required we act on DELETE_ON_CLOSE + * in CLEANUP, which means we have to call delete here. + * fastfat: + * Close is invoked whenever the last reference to a file object is deleted. + * Cleanup is invoked when the last handle to a file object is closed, and + * is called before close. + * The function of close is to completely tear down and remove the fcb/dcb/ccb + * structures associated with the file object. + * So for ZFS, CLEANUP will leave FsContext=vp around - to have it be freed in + * CLOSE. + */ + case IRP_MJ_CLEANUP: + Status = zfs_fileobject_cleanup(DeviceObject, Irp, IrpSp, + &hold_vp); + break; + + case IRP_MJ_CLOSE: + Status = zfs_fileobject_close(DeviceObject, Irp, IrpSp, + &hold_vp); + break; + + case IRP_MJ_DEVICE_CONTROL: + { + ulong_t cmd = IrpSp->Parameters.DeviceIoControl.IoControlCode; + /* Not ZFS ioctl, handle Windows ones */ + switch (cmd) { + case IOCTL_VOLUME_GET_GPT_ATTRIBUTES: + dprintf("IOCTL_VOLUME_GET_GPT_ATTRIBUTES\n"); + Status = 0; + break; + case IOCTL_MOUNTDEV_QUERY_DEVICE_NAME: + dprintf("IOCTL_MOUNTDEV_QUERY_DEVICE_NAME\n"); + Status = ioctl_query_device_name(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_UNIQUE_ID: + dprintf("IOCTL_MOUNTDEV_QUERY_UNIQUE_ID\n"); + Status = ioctl_query_unique_id(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_STABLE_GUID: + dprintf("IOCTL_MOUNTDEV_QUERY_STABLE_GUID\n"); + Status = ioctl_query_stable_guid(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME: + dprintf("IOCTL_MOUNTDEV_QUERY_SUGGESTED_LINK_NAME\n"); + Status = ioctl_mountdev_query_suggested_link_name( + DeviceObject, Irp, IrpSp); + break; + case IOCTL_VOLUME_ONLINE: + dprintf("IOCTL_VOLUME_ONLINE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_VOLUME_OFFLINE: + dprintf("IOCTL_VOLUME_OFFLINE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_DISK_IS_WRITABLE: + dprintf("IOCTL_DISK_IS_WRITABLE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_DISK_MEDIA_REMOVAL: + dprintf("IOCTL_DISK_MEDIA_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_STORAGE_MEDIA_REMOVAL: + dprintf("IOCTL_STORAGE_MEDIA_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_VOLUME_POST_ONLINE: + dprintf("IOCTL_VOLUME_POST_ONLINE\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_STORAGE_CHECK_VERIFY: + dprintf("IOCTL_STORAGE_CHECK_VERIFY\n"); + Status = STATUS_SUCCESS; + break; + case IOCTL_DISK_GET_DRIVE_GEOMETRY: + dprintf("IOCTL_DISK_GET_DRIVE_GEOMETRY\n"); + Status = ioctl_disk_get_drive_geometry(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_DISK_GET_DRIVE_GEOMETRY_EX: + dprintf("IOCTL_DISK_GET_DRIVE_GEOMETRY_EX\n"); + Status = ioctl_disk_get_drive_geometry_ex(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_DISK_GET_PARTITION_INFO: + dprintf("IOCTL_DISK_GET_PARTITION_INFO\n"); + Status = ioctl_disk_get_partition_info(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_DISK_GET_PARTITION_INFO_EX: + dprintf("IOCTL_DISK_GET_PARTITION_INFO_EX\n"); + Status = ioctl_disk_get_partition_info_ex(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_VOLUME_IS_IO_CAPABLE: + dprintf("IOCTL_VOLUME_IS_IO_CAPABLE\n"); + Status = ioctl_volume_is_io_capable(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_STORAGE_GET_HOTPLUG_INFO: + dprintf("IOCTL_STORAGE_GET_HOTPLUG_INFO\n"); + Status = ioctl_storage_get_hotplug_info(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS: + dprintf("IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS\n"); + Status = ioctl_volume_get_volume_disk_extents( + DeviceObject, Irp, IrpSp); + break; + case IOCTL_DISK_GET_LENGTH_INFO: + dprintf("IOCTL_DISK_GET_LENGTH_INFO\n"); + Status = ioctl_disk_get_length_info(DeviceObject, Irp, + IrpSp); + break; + case IOCTL_STORAGE_GET_DEVICE_NUMBER: + dprintf("IOCTL_STORAGE_GET_DEVICE_NUMBER\n"); + Status = ioctl_storage_get_device_number(DeviceObject, + Irp, IrpSp); + break; + case IOCTL_STORAGE_QUERY_PROPERTY: + dprintf("IOCTL_STORAGE_QUERY_PROPERTY\n"); + Status = ioctl_storage_query_property(DeviceObject, Irp, + IrpSp); + break; + + case FSCTL_DISMOUNT_VOLUME: + dprintf("FSCTL_DISMOUNT_VOLUME\n"); + Status = 0; + break; + case FSCTL_LOCK_VOLUME: + dprintf("FSCTL_LOCK_VOLUME\n"); + Status = 0; + break; + + + default: + dprintf("**** unknown fsWindows IOCTL: 0x%lx\n", cmd); + } + + } + break; + + case IRP_MJ_FILE_SYSTEM_CONTROL: + switch (IrpSp->MinorFunction) { + case IRP_MN_MOUNT_VOLUME: + dprintf("IRP_MN_MOUNT_VOLUME fs\n"); + Status = zfs_vnop_mount(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_USER_FS_REQUEST: + Status = user_fs_request(DeviceObject, PIrp, IrpSp); + break; + // FSCTL_QUERY_VOLUME_CONTAINER_STATE 0x90930 + case IRP_MN_KERNEL_CALL: + dprintf("IRP_MN_KERNEL_CALL: unknown 0x%x\n", + IrpSp->Parameters.FileSystemControl.FsControlCode); + Status = STATUS_INVALID_DEVICE_REQUEST; + break; + default: + dprintf("IRP_MJ_FILE_SYSTEM_CONTROL: unknown 0x%x\n", + IrpSp->MinorFunction); + Status = STATUS_INVALID_DEVICE_REQUEST; + } + break; + + case IRP_MJ_PNP: + switch (IrpSp->MinorFunction) { + case IRP_MN_QUERY_CAPABILITIES: + Status = QueryCapabilities(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_DEVICE_RELATIONS: + Status = STATUS_NOT_IMPLEMENTED; + + if (IrpSp->Parameters.QueryDeviceRelations.Type == + TargetDeviceRelation) { + PDEVICE_RELATIONS DeviceRelations; + DeviceRelations = + (PDEVICE_RELATIONS)ExAllocatePool(PagedPool, + sizeof (DEVICE_RELATIONS)); + if (!DeviceRelations) { + dprintf("enomem DeviceRelations\n"); + Status = STATUS_INSUFFICIENT_RESOURCES; + break; + } + + dprintf("TargetDeviceRelations\n"); + +/* The PnP manager will remove this when it is done with device */ + ObReferenceObject(DeviceObject); + + DeviceRelations->Count = 1; + DeviceRelations->Objects[0] = DeviceObject; + Irp->IoStatus.Information = + (ULONG_PTR)DeviceRelations; + + Status = STATUS_SUCCESS; + break; + } + + dprintf("DeviceRelations.Type 0x%x\n", + IrpSp->Parameters.QueryDeviceRelations.Type); + break; + case IRP_MN_QUERY_ID: + Status = pnp_query_id(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_PNP_DEVICE_STATE: + Status = pnp_device_state(DeviceObject, Irp, IrpSp); + break; + case IRP_MN_QUERY_REMOVE_DEVICE: + dprintf("IRP_MN_QUERY_REMOVE_DEVICE\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_SURPRISE_REMOVAL: + dprintf("IRP_MN_SURPRISE_REMOVAL\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_REMOVE_DEVICE: + dprintf("IRP_MN_REMOVE_DEVICE\n"); + Status = STATUS_SUCCESS; + break; + case IRP_MN_CANCEL_REMOVE_DEVICE: + dprintf("IRP_MN_CANCEL_REMOVE_DEVICE\n"); + Status = STATUS_SUCCESS; + break; + } + break; + case IRP_MJ_QUERY_VOLUME_INFORMATION: + Status = query_volume_information(DeviceObject, Irp, IrpSp); + break; + + case IRP_MJ_LOCK_CONTROL: + Status = lock_control(DeviceObject, Irp, IrpSp); + break; + + case IRP_MJ_QUERY_INFORMATION: + Status = query_information(DeviceObject, Irp, IrpSp); + break; + + case IRP_MJ_DIRECTORY_CONTROL: + switch (IrpSp->MinorFunction) { + case IRP_MN_NOTIFY_CHANGE_DIRECTORY: + Status = notify_change_directory(DeviceObject, Irp, + IrpSp); + break; + case IRP_MN_QUERY_DIRECTORY: + Status = query_directory(DeviceObject, Irp, IrpSp); + break; + } + break; + case IRP_MJ_SET_INFORMATION: + Status = set_information(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_READ: + Status = fs_read(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_WRITE: + Status = fs_write(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_FLUSH_BUFFERS: + Status = flush_buffers(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_QUERY_SECURITY: + Status = query_security(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_SET_SECURITY: + Status = set_security(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_QUERY_EA: + Status = query_ea(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_SET_EA: + Status = set_ea(DeviceObject, Irp, IrpSp); + break; + case IRP_MJ_SHUTDOWN: + dprintf("IRP_MJ_SHUTDOWN\n"); + Status = STATUS_SUCCESS; + break; + } + + /* + * Re-check (since MJ_CREATE/vnop_lookup might have set it) vp here, + * to see if we should call setsize + */ + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + + /* + * vp "might" be held above, or not (vnop_lookup) so grab + * another just in case + */ + if (vp && vnode_sizechange(vp) && + VN_HOLD(vp) == 0) { + if (CcIsFileCached(IrpSp->FileObject)) { + znode_t *zp = VTOZ(vp); + vnode_pager_setsize(IrpSp->FileObject, vp, + zp->z_size, FALSE); + dprintf("sizechanged, updated to %llx\n", + vp->FileHeader.FileSize); + } + VN_RELE(vp); + } + } + + /* If we held the vp above, release it now. */ + if (hold_vp != NULL) { + VN_RELE(hold_vp); + } + +#ifdef DEBUG_IOCOUNT + // Since we have serialised all fsdispatch() calls, and we are + // about to leave - all iocounts should be zero, check that is true. + // ZFSCallbackAcquireForCreateSection() can give false positives, as + // they are called async, outside IRP dispatcher. + if (!skiplock) { + // Wait for all async_rele to finish + if (zfsvfs) + taskq_wait(dsl_pool_vnrele_taskq(dmu_objset_pool( + zfsvfs->z_os))); + vnode_check_iocount(); + mutex_exit(&GIANT_SERIAL_LOCK); + } +#endif + + return (Status); +} + +/* + * ALL ioctl requests come in here, and we do the Windows specific + * work to handle IRPs then we sort out the type of request + * (ioctl, volume, filesystem) and call each respective handler. + */ +_Function_class_(DRIVER_DISPATCH) + NTSTATUS + dispatcher( + _In_ PDEVICE_OBJECT DeviceObject, + _Inout_ PIRP Irp) +{ + BOOLEAN TopLevel = FALSE; + BOOLEAN AtIrqlPassiveLevel; + PIO_STACK_LOCATION IrpSp; + NTSTATUS Status = STATUS_NOT_IMPLEMENTED; + + // Storport can call itself (and hence, ourselves) so this isn't + // always true. + // PAGED_CODE(); + + // dprintf("%s: enter\n", __func__); + + // If we were called with our file system device object instead of a + // volume device object, just complete this request with STATUS_SUCCESS +#if 0 + if (vnop_deviceObject == VolumeDeviceObject) { + dprintf("%s: own object\n", __func__); + Irp->IoStatus.Status = STATUS_SUCCESS; + Irp->IoStatus.Information = FILE_OPENED; + IoCompleteRequest(Irp, IO_DISK_INCREMENT); + return (STATUS_SUCCESS); + } +#endif + IrpSp = IoGetCurrentIrpStackLocation(Irp); + + dprintf("%s: enter: major %d: minor %d: %s: type 0x%x: fo %p\n", + __func__, IrpSp->MajorFunction, IrpSp->MinorFunction, + major2str(IrpSp->MajorFunction, IrpSp->MinorFunction), + Irp->Type, IrpSp->FileObject); + + KIRQL saveIRQL; + saveIRQL = KeGetCurrentIrql(); + + + AtIrqlPassiveLevel = (KeGetCurrentIrql() == PASSIVE_LEVEL); + if (AtIrqlPassiveLevel) { + FsRtlEnterFileSystem(); + } + if (IoGetTopLevelIrp() == NULL) { + IoSetTopLevelIrp(Irp); + TopLevel = TRUE; + } + + if (DeviceObject == ioctlDeviceObject) + Status = ioctlDispatcher(DeviceObject, &Irp, IrpSp); + else { + mount_t *zmo = DeviceObject->DeviceExtension; + if (zmo && zmo->type == MOUNT_TYPE_DCB) + Status = diskDispatcher(DeviceObject, &Irp, IrpSp); + else if (zmo && zmo->type == MOUNT_TYPE_VCB) + Status = fsDispatcher(DeviceObject, &Irp, IrpSp); + else { + extern PDRIVER_DISPATCH + STOR_MajorFunction[IRP_MJ_MAXIMUM_FUNCTION + 1]; + if (STOR_MajorFunction[IrpSp->MajorFunction] != NULL) { + if (TopLevel) { + IoSetTopLevelIrp(NULL); + } + if (AtIrqlPassiveLevel) { + FsRtlExitFileSystem(); + } + // dprintf("Relaying IRP to STORport\n"); + return (STOR_MajorFunction[IrpSp->MajorFunction] + (DeviceObject, Irp)); + } + + // Got a request we don't care about? + Status = STATUS_INVALID_DEVICE_REQUEST; + Irp->IoStatus.Information = 0; + } + } + + if (AtIrqlPassiveLevel) { + FsRtlExitFileSystem(); + } + if (TopLevel) { + IoSetTopLevelIrp(NULL); + } + + switch (Status) { + case STATUS_SUCCESS: + case STATUS_BUFFER_OVERFLOW: + case STATUS_PENDING: + break; + default: + dprintf("%s: exit: 0x%x %s Information 0x%x : %s\n", + __func__, Status, + common_status_str(Status), + Irp ? Irp->IoStatus.Information : 0, + major2str(IrpSp->MajorFunction, IrpSp->MinorFunction)); + } + + // Complete the request if it isn't pending (ie, we + // called zfsdev_async()) + if (Status != STATUS_PENDING && Irp != NULL) { + // IOCTL_STORAGE_GET_HOTPLUG_INFO + // IOCTL_DISK_CHECK_VERIFY + // IOCTL_STORAGE_QUERY_PROPERTY + Irp->IoStatus.Status = Status; + + IoCompleteRequest(Irp, + Status == STATUS_SUCCESS ? IO_DISK_INCREMENT : + IO_NO_INCREMENT); + } + + VERIFY3U(saveIRQL, ==, KeGetCurrentIrql()); + + + return (Status); +} + +NTSTATUS +ZFSCallbackAcquireForCreateSection( + IN PFS_FILTER_CALLBACK_DATA CallbackData, + OUT PVOID *CompletionContext) +{ + ASSERT(CallbackData->SizeOfFsFilterCallbackData == + sizeof (FS_FILTER_CALLBACK_DATA)); + + dprintf("%s: Operation 0x%x \n", __func__, + CallbackData->Operation); + + struct vnode *vp; + vp = CallbackData->FileObject->FsContext; + + ASSERT(vp != NULL); + if (vp == NULL) + return (STATUS_INVALID_PARAMETER); + +#ifdef DEBUG_IOCOUNT + int nolock = 0; + if (mutex_owned(&GIANT_SERIAL_LOCK)) + nolock = 1; + else + mutex_enter(&GIANT_SERIAL_LOCK); +#endif + if (VN_HOLD(vp) == 0) { + dprintf("%s: locked: %p\n", __func__, vp->FileHeader.Resource); + ExAcquireResourceExclusiveLite(vp->FileHeader.Resource, TRUE); + vnode_ref(vp); + VN_RELE(vp); + } else { +#ifdef DEBUG_IOCOUNT + if (!nolock) + mutex_exit(&GIANT_SERIAL_LOCK); +#endif + return (STATUS_INVALID_PARAMETER); + } +#ifdef DEBUG_IOCOUNT + if (!nolock) + mutex_exit(&GIANT_SERIAL_LOCK); +#endif + + if (CallbackData-> + Parameters.AcquireForSectionSynchronization.SyncType != + SyncTypeCreateSection) { + return (STATUS_FSFILTER_OP_COMPLETED_SUCCESSFULLY); + } else if (vp->share_access.Writers == 0) { + return (STATUS_FILE_LOCKED_WITH_ONLY_READERS); + } else { + return (STATUS_FILE_LOCKED_WITH_WRITERS); + } + +} + +NTSTATUS +ZFSCallbackReleaseForCreateSection( + IN PFS_FILTER_CALLBACK_DATA CallbackData, + OUT PVOID *CompletionContext) +{ + struct vnode *vp; + vp = CallbackData->FileObject->FsContext; + + dprintf("%s: vp %p\n", __func__, vp); + + ASSERT(vp != NULL); + if (vp == NULL) + return (STATUS_INVALID_PARAMETER); + + if (vp->FileHeader.Resource) { + dprintf("%s: unlocked: %p\n", + __func__, vp->FileHeader.Resource); + ExReleaseResourceLite(vp->FileHeader.Resource); +#ifdef DEBUG_IOCOUNT + int nolock = 0; + if (mutex_owned(&GIANT_SERIAL_LOCK)) + nolock = 1; + else + mutex_enter(&GIANT_SERIAL_LOCK); +#endif + if (VN_HOLD(vp) == 0) { + vnode_rele(vp); + VN_RELE(vp); + } +#ifdef DEBUG_IOCOUNT + if (!nolock) + mutex_exit(&GIANT_SERIAL_LOCK); +#endif + } + + return (STATUS_FSFILTER_OP_COMPLETED_SUCCESSFULLY); +} + +void +zfs_windows_vnops_callback(PDEVICE_OBJECT deviceObject) +{ + +} + +int +zfs_vfsops_init(void) +{ + +#ifdef DEBUG_IOCOUNT + mutex_init(&GIANT_SERIAL_LOCK, NULL, MUTEX_DEFAULT, NULL); +#endif + return (0); +} + +int +zfs_vfsops_fini(void) +{ + +#ifdef DEBUG_IOCOUNT + mutex_destroy(&GIANT_SERIAL_LOCK); +#endif + return (0); +} + +NTSTATUS +pnp_query_di(PDEVICE_OBJECT DeviceObject, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS status; + if (IsEqualGUID(IrpSp->Parameters.QueryInterface.InterfaceType, + &ZFSZVOLDI_GUID)) { + if (IrpSp->Parameters.QueryInterface.Version < 1) + status = STATUS_NOT_SUPPORTED; + else if (IrpSp->Parameters.QueryInterface.Size < + sizeof (zfsdizvol_t)) + status = STATUS_BUFFER_TOO_SMALL; + else if ((IrpSp-> + Parameters.QueryInterface.InterfaceSpecificData == + NULL) || strlen(IrpSp-> + Parameters.QueryInterface.InterfaceSpecificData) <= 8) + status = STATUS_INVALID_PARAMETER; + else { + PVOID zv; // zvol_state_t*, opaque here + uint32_t openCount; + extern PVOID zvol_name2zvolState(const char *name, + uint32_t *openCount); + PCHAR vendorUniqueId = (PCHAR)IrpSp-> + Parameters.QueryInterface.InterfaceSpecificData; + zv = zvol_name2zvolState(&vendorUniqueId[8], + &openCount); + // check that the minor number is non-zero: that + // signifies the zvol has fully completed its + // bringup phase. + if (zv && openCount) { + extern void IncZvolRef(PVOID Context); + extern void DecZvolRef(PVOID Context); + extern NTSTATUS ZvolDiRead(PVOID Context, + zfsiodesc_t *pIo); + extern NTSTATUS ZvolDiWrite(PVOID Context, + zfsiodesc_t *pIo); + // lock in an extra reference on the zvol + IncZvolRef(zv); + zfsdizvol_t *pDI = (zfsdizvol_t *)IrpSp-> + Parameters.QueryInterface.Interface; + pDI->header.Size = sizeof (zfsdizvol_t); + pDI->header.Version = ZFSZVOLDI_VERSION; + pDI->header.Context = zv; + pDI->header.InterfaceReference = IncZvolRef; + pDI->header.InterfaceDereference = DecZvolRef; + pDI->Read = ZvolDiRead; + pDI->Write = ZvolDiWrite; + Irp->IoStatus.Information = 0; + status = STATUS_SUCCESS; + } + else + status = STATUS_NOT_FOUND; + } + } + else + status = STATUS_NOT_IMPLEMENTED; + return (status); +} diff --git a/module/os/windows/zfs/zfs_vnops_windows_lib.c b/module/os/windows/zfs/zfs_vnops_windows_lib.c new file mode 100644 index 000000000000..c3c79ee02153 --- /dev/null +++ b/module/os/windows/zfs/zfs_vnops_windows_lib.c @@ -0,0 +1,5287 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2017 Jorgen Lundman + * Portions Copyright 2022 Andrew Innes + */ +#define INITGUID +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include + + +#include +#include +#include +#include + +#undef _NTDDK_ + +typedef struct { + UCHAR revision; + UCHAR elements; + UCHAR auth[6]; + UINT32 nums[8]; +} sid_header; + +// BUILTIN\Administrators +static sid_header sid_BA = { 1, 2, SECURITY_NT_AUTHORITY, {32, 544} }; +// NT AUTHORITY\SYSTEM +static sid_header sid_SY = { 1, 1, SECURITY_NT_AUTHORITY, {18} }; +// BUILTIN\Users +static sid_header sid_BU = { 1, 2, SECURITY_NT_AUTHORITY, {32, 545} }; +// NT AUTHORITY\Authenticated Users +static sid_header sid_AU = { 1, 1, SECURITY_NT_AUTHORITY, {11} }; + +// MandatoryLevel\High +static sid_header sid_MH = + { 1, 1, SECURITY_MANDATORY_LABEL_AUTHORITY, {12288} }; +// MandatoryLevel\Low +static sid_header sid_ML = + { 1, 1, SECURITY_MANDATORY_LABEL_AUTHORITY, {4096} }; + +typedef struct { + UCHAR flags; + ACCESS_MASK mask; + sid_header* sid; +} dacl; + +/* + * + * Brand new ntfs: + * F:\ BUILTIN\Administrators:(F) + * BUILTIN\Administrators:(OI)(CI)(IO)(F) + * NT AUTHORITY\SYSTEM:(F) + * NT AUTHORITY\SYSTEM:(OI)(CI)(IO)(F) + * NT AUTHORITY\Authenticated Users:(M) + * NT AUTHORITY\Authenticated Users:(OI)(CI)(IO)(M) + * BUILTIN\Users:(RX) + * BUILTIN\Users:(OI)(CI)(IO)(GR,GE) + */ +static dacl def_dacls[] = { + // BUILTIN\Administrators:(F) + { 0, FILE_ALL_ACCESS, &sid_BA }, + // BUILTIN\Administrators:(OI)(CI)(IO)(F) + { OBJECT_INHERIT_ACE | CONTAINER_INHERIT_ACE | INHERIT_ONLY_ACE, + FILE_ALL_ACCESS, &sid_BA }, + // NT AUTHORITY\SYSTEM:(F) + { 0, FILE_ALL_ACCESS, &sid_SY }, + // NT AUTHORITY\SYSTEM:(OI)(CI)(IO)(F) + { OBJECT_INHERIT_ACE | CONTAINER_INHERIT_ACE | INHERIT_ONLY_ACE, + FILE_ALL_ACCESS, &sid_SY }, + // NT AUTHORITY\Authenticated Users:(M) + { 0, FILE_GENERIC_READ | FILE_GENERIC_WRITE | DELETE | + FILE_GENERIC_EXECUTE, &sid_AU }, + // NT AUTHORITY\Authenticated Users:(OI)(CI)(IO)(M) + { OBJECT_INHERIT_ACE | CONTAINER_INHERIT_ACE | INHERIT_ONLY_ACE, + FILE_GENERIC_READ | FILE_GENERIC_WRITE | DELETE | + FILE_GENERIC_EXECUTE, &sid_AU }, + // BUILTIN\Users:(RX) + { 0, FILE_GENERIC_READ | FILE_GENERIC_EXECUTE, &sid_BU }, + // BUILTIN\Users:(OI)(CI)(IO)(GR,GE) + { OBJECT_INHERIT_ACE | CONTAINER_INHERIT_ACE | INHERIT_ONLY_ACE, + GENERIC_READ | GENERIC_EXECUTE, &sid_BU }, +#if 0 // C: only? + // Mandatory Label\High Mandatory Level:(OI)(NP)(IO)(NW) + { OBJECT_INHERIT_ACE | NO_PROPAGATE_INHERIT_ACE | INHERIT_ONLY_ACE, + SYSTEM_MANDATORY_LABEL_NO_WRITE_UP, &sid_MH }, +#endif + // END + { 0, 0, NULL } +}; + +// #define USE_RECYCLE_ACL +#ifdef USE_RECYCLE_ACL +/* + * Brand new $Recycle.bin + * + * Owner: WDKRemoteUser + * Group: None + * + * F : \$Recycle.bin BUILTIN\Administrators:(I)(F) + * NT AUTHORITY\SYSTEM : (I)(F) + * NT AUTHORITY\Authenticated Users : (I)(M) + * BUILTIN\Users : (I)(RX) + */ +static dacl recycle_dacls[] = { + // BUILTIN\Administrators:(I)(F) + { INHERITED_ACE, FILE_ALL_ACCESS, &sid_BA }, + // NT AUTHORITY\SYSTEM : (I)(F) + { INHERITED_ACE, FILE_ALL_ACCESS, &sid_SY }, + // NT AUTHORITY\Authenticated Users : (I)(M) + { INHERITED_ACE, FILE_GENERIC_READ | FILE_GENERIC_WRITE | DELETE + | FILE_GENERIC_EXECUTE, &sid_AU }, + // BUILTIN\Users : (I)(RX) + { INHERITED_ACE, FILE_GENERIC_READ | FILE_GENERIC_EXECUTE, &sid_BU }, + // END + { 0, 0, NULL } +}; +#endif + +char * +major2str(int major, int minor) +{ + switch (major) { + case IRP_MJ_CREATE: + return ("IRP_MJ_CREATE"); + case IRP_MJ_CREATE_NAMED_PIPE: + return ("IRP_MJ_CREATE_NAMED_PIPE"); + case IRP_MJ_CLOSE: + return ("IRP_MJ_CLOSE"); + case IRP_MJ_READ: + return ("IRP_MJ_READ"); + case IRP_MJ_WRITE: + return ("IRP_MJ_WRITE"); + case IRP_MJ_QUERY_INFORMATION: + return ("IRP_MJ_QUERY_INFORMATION"); + case IRP_MJ_SET_INFORMATION: + return ("IRP_MJ_SET_INFORMATION"); + case IRP_MJ_QUERY_EA: + return ("IRP_MJ_QUERY_EA"); + case IRP_MJ_SET_EA: + return ("IRP_MJ_SET_EA"); + case IRP_MJ_FLUSH_BUFFERS: + return ("IRP_MJ_FLUSH_BUFFERS"); + case IRP_MJ_QUERY_VOLUME_INFORMATION: + return ("IRP_MJ_QUERY_VOLUME_INFORMATION"); + case IRP_MJ_SET_VOLUME_INFORMATION: + return ("IRP_MJ_SET_VOLUME_INFORMATION"); + case IRP_MJ_DIRECTORY_CONTROL: + switch (minor) { + case IRP_MN_NOTIFY_CHANGE_DIRECTORY: + return ("IRP_MJ_DIRECTORY_CONTROL(IRP_MN_NOTIFY_CHANGE_DIRECTORY)"); + case IRP_MN_QUERY_DIRECTORY: + return ("IRP_MJ_DIRECTORY_CONTROL(IRP_MN_QUERY_DIRECTORY)"); + } + return ("IRP_MJ_DIRECTORY_CONTROL"); + case IRP_MJ_FILE_SYSTEM_CONTROL: + switch (minor) { + case IRP_MN_KERNEL_CALL: + return ("IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_KERNEL_CALL)"); + case IRP_MN_MOUNT_VOLUME: + return ("IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_MOUNT_VOLUME)"); + case IRP_MN_USER_FS_REQUEST: + return ("IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_USER_FS_REQUEST)"); + case IRP_MN_VERIFY_VOLUME: + return ("IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_VERIFY_VOLUME)"); + case IRP_MN_LOAD_FILE_SYSTEM: + return ("IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_LOAD_FILE_SYSTEM)"); + } + return ("IRP_MJ_FILE_SYSTEM_CONTROL"); + case IRP_MJ_DEVICE_CONTROL: + return ("IRP_MJ_DEVICE_CONTROL"); + case IRP_MJ_INTERNAL_DEVICE_CONTROL: + return ("IRP_MJ_INTERNAL_DEVICE_CONTROL"); + case IRP_MJ_SHUTDOWN: + return ("IRP_MJ_SHUTDOWN"); + case IRP_MJ_LOCK_CONTROL: + switch (minor) { + case IRP_MN_LOCK: + return ("IRP_MJ_LOCK_CONTROL(IRP_MN_LOCK)"); + case IRP_MN_UNLOCK_ALL: + return ("IRP_MJ_LOCK_CONTROL(IRP_MN_UNLOCK_ALL)"); + case IRP_MN_UNLOCK_ALL_BY_KEY: + return ("IRP_MJ_LOCK_CONTROL(IRP_MN_UNLOCK_ALL_BY_KEY)"); + case IRP_MN_UNLOCK_SINGLE: + return ("IRP_MJ_LOCK_CONTROL(IRP_MN_UNLOCK_SINGLE)"); + } + return ("IRP_MJ_LOCK_CONTROL"); + case IRP_MJ_CLEANUP: + return ("IRP_MJ_CLEANUP"); + case IRP_MJ_CREATE_MAILSLOT: + return ("IRP_MJ_CREATE_MAILSLOT"); + case IRP_MJ_QUERY_SECURITY: + return ("IRP_MJ_QUERY_SECURITY"); + case IRP_MJ_SET_SECURITY: + return ("IRP_MJ_SET_SECURITY"); + case IRP_MJ_POWER: + return ("IRP_MJ_POWER"); + case IRP_MJ_SYSTEM_CONTROL: + return ("IRP_MJ_SYSTEM_CONTROL"); + case IRP_MJ_DEVICE_CHANGE: + return ("IRP_MJ_DEVICE_CHANGE"); + case IRP_MJ_QUERY_QUOTA: + return ("IRP_MJ_QUERY_QUOTA"); + case IRP_MJ_SET_QUOTA: + return ("IRP_MJ_SET_QUOTA"); + case IRP_MJ_PNP: + switch (minor) { + case IRP_MN_START_DEVICE: + return ("IRP_MJ_PNP(IRP_MN_START_DEVICE)"); + case IRP_MN_QUERY_REMOVE_DEVICE: + return ("IRP_MJ_PNP(IRP_MN_QUERY_REMOVE_DEVICE)"); + case IRP_MN_REMOVE_DEVICE: + return ("IRP_MJ_PNP(IRP_MN_REMOVE_DEVICE)"); + case IRP_MN_CANCEL_REMOVE_DEVICE: + return ("IRP_MJ_PNP(IRP_MN_CANCEL_REMOVE_DEVICE)"); + case IRP_MN_STOP_DEVICE: + return ("IRP_MJ_PNP(IRP_MN_STOP_DEVICE)"); + case IRP_MN_QUERY_STOP_DEVICE: + return ("IRP_MJ_PNP(IRP_MN_QUERY_STOP_DEVICE)"); + case IRP_MN_CANCEL_STOP_DEVICE: + return ("IRP_MJ_PNP(IRP_MN_CANCEL_STOP_DEVICE)"); + case IRP_MN_QUERY_DEVICE_RELATIONS: + return ("IRP_MJ_PNP(IRP_MN_QUERY_DEVICE_RELATIONS)"); + case IRP_MN_QUERY_INTERFACE: + return ("IRP_MJ_PNP(IRP_MN_QUERY_INTERFACE)"); + case IRP_MN_QUERY_RESOURCES: + return ("IRP_MJ_PNP(IRP_MN_QUERY_RESOURCES)"); + case IRP_MN_QUERY_RESOURCE_REQUIREMENTS: + return ("IRP_MJ_PNP(IRP_MN_QUERY_RESOURCE_REQUIREMENTS)"); + case IRP_MN_QUERY_CAPABILITIES: + return ("IRP_MJ_PNP(IRP_MN_QUERY_CAPABILITIES)"); + case IRP_MN_QUERY_DEVICE_TEXT: + return ("IRP_MJ_PNP(IRP_MN_QUERY_DEVICE_TEXT)"); + case IRP_MN_FILTER_RESOURCE_REQUIREMENTS: + return ("IRP_MJ_PNP(IRP_MN_FILTER_RESOURCE_REQUIREMENTS)"); + case IRP_MN_READ_CONFIG: + return ("IRP_MJ_PNP(IRP_MN_READ_CONFIG)"); + case IRP_MN_WRITE_CONFIG: + return ("IRP_MJ_PNP(IRP_MN_WRITE_CONFIG)"); + case IRP_MN_EJECT: + return ("IRP_MJ_PNP(IRP_MN_EJECT)"); + case IRP_MN_SET_LOCK: + return ("IRP_MJ_PNP(IRP_MN_SET_LOCK)"); + case IRP_MN_QUERY_ID: + return ("IRP_MJ_PNP(IRP_MN_QUERY_ID)"); + case IRP_MN_QUERY_PNP_DEVICE_STATE: + return ("IRP_MJ_PNP(IRP_MN_QUERY_PNP_DEVICE_STATE)"); + case IRP_MN_QUERY_BUS_INFORMATION: + return ("IRP_MJ_PNP(IRP_MN_QUERY_BUS_INFORMATION)"); + case IRP_MN_DEVICE_USAGE_NOTIFICATION: + return ("IRP_MJ_PNP(IRP_MN_DEVICE_USAGE_NOTIFICATION)"); + case IRP_MN_SURPRISE_REMOVAL: // SUPPLIES! + return ("IRP_MJ_PNP(IRP_MN_SURPRISE_REMOVAL)"); + } + return ("IRP_MJ_PNP"); + default: + break; + } + return ("Unknown"); +} + +char * +common_status_str(NTSTATUS Status) +{ + switch (Status) { + case STATUS_SUCCESS: + return ("OK"); + case STATUS_BUFFER_OVERFLOW: + return ("Overflow"); + case STATUS_BUFFER_TOO_SMALL: + return ("BufferTooSmall"); + case STATUS_END_OF_FILE: + return ("EOF"); + case STATUS_NO_MORE_FILES: + return ("NoMoreFiles"); + case STATUS_OBJECT_PATH_NOT_FOUND: + return ("ObjectPathNotFound"); + case STATUS_NO_SUCH_FILE: + return ("NoSuchFile"); + case STATUS_ACCESS_DENIED: + return ("AccessDenied"); + case STATUS_NOT_IMPLEMENTED: + return ("NotImplemented"); + case STATUS_PENDING: + return ("STATUS_PENDING"); + case STATUS_INVALID_PARAMETER: + return ("STATUS_INVALID_PARAMETER"); + case STATUS_OBJECT_NAME_NOT_FOUND: + return ("STATUS_OBJECT_NAME_NOT_FOUND"); + case STATUS_OBJECT_NAME_COLLISION: + return ("STATUS_OBJECT_NAME_COLLISION"); + case STATUS_FILE_IS_A_DIRECTORY: + return ("STATUS_FILE_IS_A_DIRECTORY"); + case STATUS_NOT_A_REPARSE_POINT: + return ("STATUS_NOT_A_REPARSE_POINT"); + case STATUS_NOT_FOUND: + return ("STATUS_NOT_FOUND"); + case STATUS_NO_MORE_EAS: + return ("STATUS_NO_MORE_EAS"); + case STATUS_NO_EAS_ON_FILE: + return ("STATUS_NO_EAS_ON_FILE"); + case 0xa0000003: + return ("STATUS_REPARSE_POINT"); + case STATUS_DIRECTORY_IS_A_REPARSE_POINT: + return ("STATUS_DIRECTORY_IS_A_REPARSE_POINT"); + case STATUS_REPARSE: + return ("STATUS_REPARSE"); + default: + return ("<*****>"); + } +} + +void +strupper(char *s, size_t max) +{ + while ((max > 0) && *s) { + *s = toupper(*s); + s++; + max--; + } +} + +char * +create_options(ULONG Options) +{ + static char out[256]; + + BOOLEAN CreateDirectory; + BOOLEAN OpenDirectory; + BOOLEAN CreateFile; + ULONG CreateDisposition; + + out[0] = 0; + + BOOLEAN DirectoryFile; + DirectoryFile = BooleanFlagOn(Options, FILE_DIRECTORY_FILE); + + if (BooleanFlagOn(Options, FILE_DIRECTORY_FILE)) + strlcat(out, "DirectoryFile ", sizeof (out)); + if (BooleanFlagOn(Options, FILE_NON_DIRECTORY_FILE)) + strlcat(out, "NonDirectoryFile ", sizeof (out)); + if (BooleanFlagOn(Options, FILE_NO_INTERMEDIATE_BUFFERING)) + strlcat(out, "NoIntermediateBuffering ", sizeof (out)); + if (BooleanFlagOn(Options, FILE_NO_EA_KNOWLEDGE)) + strlcat(out, "NoEaKnowledge ", sizeof (out)); + if (BooleanFlagOn(Options, FILE_DELETE_ON_CLOSE)) + strlcat(out, "DeleteOnClose ", sizeof (out)); + if (BooleanFlagOn(Options, FILE_OPEN_BY_FILE_ID)) + strlcat(out, "FileOpenByFileId ", sizeof (out)); + + CreateDisposition = (Options >> 24) & 0x000000ff; + + switch (CreateDisposition) { + case FILE_SUPERSEDE: + strlcat(out, "@FILE_SUPERSEDE ", sizeof (out)); + break; + case FILE_CREATE: + strlcat(out, "@FILE_CREATE ", sizeof (out)); + break; + case FILE_OPEN: + strlcat(out, "@FILE_OPEN ", sizeof (out)); + break; + case FILE_OPEN_IF: + strlcat(out, "@FILE_OPEN_IF ", sizeof (out)); + break; + case FILE_OVERWRITE: + strlcat(out, "@FILE_OVERWRITE ", sizeof (out)); + break; + case FILE_OVERWRITE_IF: + strlcat(out, "@FILE_OVERWRITE_IF ", sizeof (out)); + break; + } + + CreateDirectory = (BOOLEAN)(DirectoryFile && + ((CreateDisposition == FILE_CREATE) || + (CreateDisposition == FILE_OPEN_IF))); + + OpenDirectory = (BOOLEAN)(DirectoryFile && + ((CreateDisposition == FILE_OPEN) || + (CreateDisposition == FILE_OPEN_IF))); + + CreateFile = (BOOLEAN)( + ((CreateDisposition == FILE_CREATE) || + (CreateDisposition == FILE_OPEN_IF) || + (CreateDisposition == FILE_SUPERSEDE) || + (CreateDisposition == FILE_OVERWRITE_IF))); + if (CreateDirectory) + strlcat(out, "#CreateDirectory ", sizeof (out)); + if (OpenDirectory) + strlcat(out, "#OpenDirectory ", sizeof (out)); + if (CreateFile) + strlcat(out, "#CreateFile ", sizeof (out)); + + return (out); +} + +char * +create_reply(NTSTATUS status, ULONG reply) +{ + switch (reply) { + case FILE_SUPERSEDED: + return ("FILE_SUPERSEDED"); + case FILE_OPENED: + return ("FILE_OPENED"); + case FILE_CREATED: + return ("FILE_CREATED"); + case FILE_OVERWRITTEN: + return ("FILE_OVERWRITTEN"); + case FILE_EXISTS: + return ("FILE_EXISTS"); + case FILE_DOES_NOT_EXIST: + return ("FILE_DOES_NOT_EXIST"); + default: + if (status == STATUS_REPARSE) + return ("ReparseTag"); + return ("FileUnknown"); + } +} + +int +AsciiStringToUnicodeString(char *in, PUNICODE_STRING out) +{ + ANSI_STRING conv; + if (in == NULL) { + memset(out, 0, sizeof (UNICODE_STRING)); + return (0); + } + conv.Buffer = in; + conv.Length = strlen(in); + conv.MaximumLength = PATH_MAX; + return (RtlAnsiStringToUnicodeString(out, &conv, TRUE)); +} + +void +FreeUnicodeString(PUNICODE_STRING s) +{ + if (s->Buffer) ExFreePool(s->Buffer); + s->Buffer = NULL; +} + +int +zfs_vnop_ioctl_fullfsync(struct vnode *vp, vfs_context_t *ct, zfsvfs_t *zfsvfs) +{ + int error = 0; + + // error = zfs_fsync(VTOZ(vp), /* syncflag */ 0, NULL); + return (error); +} + +uint32_t +zfs_getwinflags(znode_t *zp) +{ + uint32_t winflags = 0; + uint64_t zflags = zp->z_pflags; + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + if (zflags & ZFS_HIDDEN) + winflags |= FILE_ATTRIBUTE_HIDDEN; + if (zflags & ZFS_SYSTEM) + winflags |= FILE_ATTRIBUTE_SYSTEM; + if (zflags & ZFS_ARCHIVE) + winflags |= FILE_ATTRIBUTE_ARCHIVE; + if (zflags & ZFS_READONLY || zfsvfs->z_rdonly) + winflags |= FILE_ATTRIBUTE_READONLY; + if (zflags & ZFS_REPARSE) + winflags |= FILE_ATTRIBUTE_REPARSE_POINT; + + if (S_ISDIR(zp->z_mode)) { + winflags |= FILE_ATTRIBUTE_DIRECTORY; + winflags &= ~FILE_ATTRIBUTE_ARCHIVE; + } + + if (winflags == 0) + winflags = FILE_ATTRIBUTE_NORMAL; + + dprintf("%s: changing zfs 0x%08llx to win 0x%08lx\n", __func__, + zflags, winflags); + return (winflags); +} + +int +zfs_setwinflags(znode_t *zp, uint32_t winflags) +{ + uint64_t zflags = 0; + + zflags = zp->z_pflags; + + if (winflags & FILE_ATTRIBUTE_HIDDEN) + zflags |= ZFS_HIDDEN; + else + zflags &= ~ZFS_HIDDEN; + + if (winflags & FILE_ATTRIBUTE_SYSTEM) + zflags |= ZFS_SYSTEM; + else + zflags &= ~ZFS_SYSTEM; + + if (winflags & FILE_ATTRIBUTE_ARCHIVE) + zflags |= ZFS_ARCHIVE; + else + zflags &= ~ZFS_ARCHIVE; + + if (winflags & FILE_ATTRIBUTE_READONLY) + zflags |= ZFS_READONLY; + else + zflags &= ~ZFS_READONLY; + + if (zp->z_pflags != zflags) { + zp->z_pflags = zflags; + dprintf("%s changing win 0x%08lx to zfs 0x%08llx\n", __func__, + winflags, zflags); + return (1); + } + + return (0); +} + +// WSL uses special EAs to interact with uid/gid/mode/device major/minor +// Returns: TRUE if the EA was stored in the vattr. +BOOLEAN +vattr_apply_lx_ea(vattr_t *vap, PFILE_FULL_EA_INFORMATION ea) +{ + BOOLEAN setVap = FALSE; + + if (ea->EaNameLength != 6 || strncmp(ea->EaName, "$LX", 3) != 0) + return (FALSE); + + void *eaValue = &ea->EaName[0] + ea->EaNameLength + 1; + if (strncmp(ea->EaName, LX_FILE_METADATA_UID_EA_NAME, + ea->EaNameLength) == 0) { + vap->va_uid = *(PUINT32)eaValue; + vap->va_active |= ATTR_UID; + setVap = TRUE; + } else if (strncmp(ea->EaName, LX_FILE_METADATA_GID_EA_NAME, + ea->EaNameLength) == 0) { + vap->va_gid = *(PUINT32)eaValue; + vap->va_active |= ATTR_GID; + setVap = TRUE; + } else if (strncmp(ea->EaName, LX_FILE_METADATA_MODE_EA_NAME, + ea->EaNameLength) == 0) { + vap->va_mode = *(PUINT32)eaValue; + vap->va_active |= ATTR_MODE; + setVap = TRUE; + } else if (strncmp(ea->EaName, LX_FILE_METADATA_DEVICE_ID_EA_NAME, + ea->EaNameLength) == 0) { + UINT32 *vu32 = (UINT32*)eaValue; + vap->va_rdev = makedev(vu32[0], vu32[1]); + vap->va_active |= VNODE_ATTR_va_rdev; + setVap = TRUE; + } + return (setVap); +} + +static int +vnode_apply_single_ea(struct vnode *vp, struct vnode *xdvp, + FILE_FULL_EA_INFORMATION *ea) +{ + int error; + znode_t *xzp = NULL; + vnode_t *xvp = NULL; + dprintf("%s: xattr '%.*s' valuelen %u\n", __func__, + ea->EaNameLength, ea->EaName, ea->EaValueLength); + + if (ea->EaValueLength > 0) { + /* Write data */ + struct iovec iov; + iov.iov_base = (void *)(ea->EaName + ea->EaNameLength + 1); + iov.iov_len = ea->EaValueLength; + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, 0, UIO_SYSSPACE, + ea->EaValueLength, 0); + + error = zpl_xattr_set(vp, ea->EaName, &uio, 0, NULL); + } else { + error = zpl_xattr_set(vp, ea->EaName, NULL, 0, NULL); + } + return (error); +} + + +/* + * Apply a set of EAs to a vnode, while handling special Windows EAs that + * set UID/GID/Mode/rdev. + */ +NTSTATUS +vnode_apply_eas(struct vnode *vp, PFILE_FULL_EA_INFORMATION eas, + ULONG eaLength, PULONG pEaErrorOffset) +{ + NTSTATUS Status = STATUS_SUCCESS; + + if (vp == NULL || eas == NULL) + return (STATUS_INVALID_PARAMETER); + + // Optional: Check for validity if the caller wants it. + if (pEaErrorOffset != NULL) { + Status = IoCheckEaBufferValidity(eas, eaLength, pEaErrorOffset); + if (!NT_SUCCESS(Status)) { + dprintf("%s: failed validity: 0x%x\n", + __func__, Status); + return (Status); + } + } + + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + // We can land here without a sa_hdl, for example .zfs + if (zp->z_sa_hdl == NULL) + return (Status); + + struct vnode *xdvp = NULL; + znode_t *xdzp = NULL; + vattr_t vap = { 0 }; + int error; + PFILE_FULL_EA_INFORMATION ea; + for (ea = eas; /* empty */; + ea = (PFILE_FULL_EA_INFORMATION) + ((uint8_t *)ea + ea->NextEntryOffset)) { + if (vattr_apply_lx_ea(&vap, ea)) { + dprintf(" encountered special attrs EA '%.*s'\n", + ea->EaNameLength, ea->EaName); + } else { + + error = vnode_apply_single_ea(vp, xdvp, ea); + if (error != 0) + dprintf("failed to process xattr: %d\n", error); + } + + if (ea->NextEntryOffset == 0) + break; + } + + // We should perhaps translate some of the "error" codes we can + // get here, into Status return values. Currently, all errors are + // masked, and we always return OK. + + // Update zp based on LX eas. + if (vap.va_active != 0) + zfs_setattr(zp, &vap, 0, NULL, NULL); + + zfs_send_notify(zfsvfs, zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_EA, + FILE_ACTION_MODIFIED); + +out: + + vnode_clear_easize(vp); + + return (Status); +} + + +extern int zfs_vnop_force_formd_normalized_output; + +void +zfs_readdir_complete(emitdir_ptr_t *ctx) +{ + // The last eodp should have Next offset of 0 + // This assumes NextEntryOffset is the FIRST entry in all structs + if (ctx->next_offset != NULL) + *ctx->next_offset = 0; + + // The outcout += reclen; above unfortunately adds the possibly + // aligned (to 8 bytes) length. But the last entry should not + // be rounded-up. + if ((ctx->outcount > ctx->last_alignment) && + (ctx->last_alignment > 0)) { + ctx->outcount -= ctx->last_alignment; + } +} + +/* + * Put out one directory entry to the output buffer, using + * whatever struct specified in ctx->dirlisttype. + * Return: + * 0 : keep iterating + * ESRC : search-pattern in use, and didn't match (keep iterating) + * ENOSPC : no more room in buffer (but more to come - stop) + */ +int +zfs_readdir_emitdir(zfsvfs_t *zfsvfs, const char *name, emitdir_ptr_t *ctx, + zfs_dirlist_t *zccb, ino64_t objnum) +{ + znode_t *tzp = NULL; + int structsize = 0; + void *nameptr = NULL; + ULONG namelenholder = 0; + int get_zp = ENOENT; + size_t namelen; + int error; + int force_formd_normalized_output = 0; + ushort_t reclen, rawsize; + ULONG *next_offset; + uint64_t guid; + + // Windows combines vnop_readdir and vnop_getattr, + // so we need to lookup a bunch of values, we try + // to do that as lightweight as possible. + + if ((zfsvfs->z_ctldir != NULL) && + (objnum == ZFSCTL_INO_ROOT) || + (objnum == ZFSCTL_INO_SNAPDIR) || + ((objnum >= zfsvfs->z_ctldir_startid) && + (objnum <= ZFSCTL_INO_SNAPDIRS))) { + struct vnode *vp; + + get_zp = zfs_vfs_vget(zfsvfs->z_vfs, objnum, &vp, NULL); + if (get_zp == 0) + tzp = VTOZ(vp); + + } else { + get_zp = zfs_zget_ext(zfsvfs, + objnum, &tzp, + ZGET_FLAG_UNLINKED); + } + + /* + * Could not find it, error out ? print name ? + * Can't zget the .zfs dir etc, so we need a dummy + * node, so we grab root node instead. + */ + if (get_zp != 0 && tzp == NULL) { + get_zp = zfs_zget_ext(zfsvfs, + zfsvfs->z_root, &tzp, + ZGET_FLAG_UNLINKED); + } + if (get_zp != 0 && tzp == NULL) { + return (get_zp); + } + + /* + * Check if name will fit. + * + * Note: non-ascii names may expand (up to 3x) when converted + * to NFD + */ + namelen = strlen(name); + + /* sysctl to force formD normalization of vnop output */ + if (zfs_vnop_force_formd_normalized_output && + !is_ascii_str(name)) + force_formd_normalized_output = 1; + else + force_formd_normalized_output = 0; + + if (force_formd_normalized_output) + namelen = MIN(MAXNAMLEN, namelen * 3); + + /* + * Fetch filename conversion length + */ + + error = RtlUTF8ToUnicodeN(NULL, 0, &namelenholder, + name, namelen); + + // We need to fill in more fields, for getattr + uint64_t mtime[2] = { 0 }; + uint64_t ctime[2] = { 0 }; + uint64_t crtime[2] = { 0 }; + if (tzp->z_is_sa && tzp->z_sa_hdl != NULL) { + /* dummy_zp wont have sa_hdl */ + sa_bulk_attr_t bulk[3]; + int count = 0; + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_MTIME(zfsvfs), NULL, &mtime, 16); + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_CTIME(zfsvfs), NULL, &ctime, 16); + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_CRTIME(zfsvfs), NULL, &crtime, 16); + sa_bulk_lookup(tzp->z_sa_hdl, bulk, count); + // Is it worth warning about failed lookup here? + } + + structsize = 0; /* size of win struct desired */ + /* bufptr : output memory area, incrementing */ + /* outcount : amount written to output, incrementing */ + /* bufsize : size of output area - static */ + + /* Fill in struct based on desired type. */ + switch (ctx->dirlisttype) { + + case FileFullDirectoryInformation: + structsize = FIELD_OFFSET( + FILE_FULL_DIR_INFORMATION, + FileName[0]); + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + break; + FILE_FULL_DIR_INFORMATION *eodp = + (FILE_FULL_DIR_INFORMATION *)ctx->bufptr; + next_offset = &eodp->NextEntryOffset; + + eodp->FileIndex = ctx->offset; + eodp->AllocationSize.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + P2ROUNDUP(tzp->z_size, + zfs_blksz(tzp)); + eodp->EndOfFile.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + tzp->z_size; + TIME_UNIX_TO_WINDOWS(mtime, + eodp->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + eodp->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + eodp->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(tzp->z_atime, + eodp->LastAccessTime.QuadPart); + // Magic code to change dir icon to link + eodp->EaSize = + tzp->z_pflags & ZFS_REPARSE ? + 0xa0000003 : + xattr_getsize(ZTOV(tzp)); + eodp->FileAttributes = + zfs_getwinflags(tzp); + nameptr = eodp->FileName; + eodp->FileNameLength = namelenholder; + + break; + + case FileIdBothDirectoryInformation: + structsize = FIELD_OFFSET( + FILE_ID_BOTH_DIR_INFORMATION, + FileName[0]); + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + break; + FILE_ID_BOTH_DIR_INFORMATION *fibdi; + fibdi = (FILE_ID_BOTH_DIR_INFORMATION *) + ctx->bufptr; + next_offset = &fibdi->NextEntryOffset; + + fibdi->AllocationSize.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + P2ROUNDUP(tzp->z_size, + zfs_blksz(tzp)); + fibdi->EndOfFile.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + tzp->z_size; + TIME_UNIX_TO_WINDOWS(mtime, + fibdi->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + fibdi->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + fibdi->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(tzp->z_atime, + fibdi->LastAccessTime.QuadPart); + fibdi->EaSize = + tzp->z_pflags & ZFS_REPARSE ? + 0xa0000003 : + xattr_getsize(ZTOV(tzp)); + fibdi->FileAttributes = + zfs_getwinflags(tzp); + fibdi->FileId.QuadPart = objnum; + fibdi->FileIndex = ctx->offset; + fibdi->ShortNameLength = 0; + nameptr = fibdi->FileName; + fibdi->FileNameLength = namelenholder; + + break; + + case FileBothDirectoryInformation: + structsize = FIELD_OFFSET( + FILE_BOTH_DIR_INFORMATION, + FileName[0]); + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + break; + FILE_BOTH_DIR_INFORMATION *fbdi = + (FILE_BOTH_DIR_INFORMATION *)ctx->bufptr; + next_offset = &fbdi->NextEntryOffset; + + fbdi->AllocationSize.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + P2ROUNDUP(tzp->z_size, + zfs_blksz(tzp)); + fbdi->EndOfFile.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + tzp->z_size; + TIME_UNIX_TO_WINDOWS(mtime, + fbdi->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + fbdi->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + fbdi->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(tzp->z_atime, + fbdi->LastAccessTime.QuadPart); + fbdi->EaSize = + tzp->z_pflags & ZFS_REPARSE ? + 0xa0000003 : + xattr_getsize(ZTOV(tzp)); + fbdi->FileAttributes = + zfs_getwinflags(tzp); + fbdi->FileIndex = ctx->offset; + fbdi->ShortNameLength = 0; + nameptr = fbdi->FileName; + fbdi->FileNameLength = namelenholder; + + break; + + case FileDirectoryInformation: + structsize = FIELD_OFFSET( + FILE_DIRECTORY_INFORMATION, + FileName[0]); + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + break; + FILE_DIRECTORY_INFORMATION *fdi = + (FILE_DIRECTORY_INFORMATION *) + ctx->bufptr; + next_offset = &fdi->NextEntryOffset; + + fdi->AllocationSize.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + P2ROUNDUP(tzp->z_size, + zfs_blksz(tzp)); + fdi->EndOfFile.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + tzp->z_size; + TIME_UNIX_TO_WINDOWS(mtime, + fdi->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + fdi->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + fdi->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(tzp->z_atime, + fdi->LastAccessTime.QuadPart); + fdi->FileAttributes = + zfs_getwinflags(tzp); + fdi->FileIndex = ctx->offset; + nameptr = fdi->FileName; + fdi->FileNameLength = namelenholder; + break; + + case FileNamesInformation: + structsize = + FIELD_OFFSET(FILE_NAMES_INFORMATION, + FileName[0]); + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + break; + FILE_NAMES_INFORMATION *fni = + (FILE_NAMES_INFORMATION *)ctx->bufptr; + next_offset = &fni->NextEntryOffset; + + fni->FileIndex = ctx->offset; + nameptr = fni->FileName; + fni->FileNameLength = namelenholder; + break; + + case FileIdFullDirectoryInformation: + structsize = FIELD_OFFSET( + FILE_ID_FULL_DIR_INFORMATION, + FileName[0]); + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + break; + FILE_ID_FULL_DIR_INFORMATION *fifdi = + (FILE_ID_FULL_DIR_INFORMATION *) + ctx->bufptr; + next_offset = &fifdi->NextEntryOffset; + + fifdi->FileIndex = ctx->offset; + fifdi->AllocationSize.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + P2ROUNDUP(tzp->z_size, + zfs_blksz(tzp)); + fifdi->EndOfFile.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + tzp->z_size; + TIME_UNIX_TO_WINDOWS(mtime, + fifdi->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + fifdi->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + fifdi->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(tzp->z_atime, + fifdi->LastAccessTime.QuadPart); + fifdi->EaSize = + tzp->z_pflags & ZFS_REPARSE ? + 0xa0000003 : + xattr_getsize(ZTOV(tzp)); + fifdi->FileAttributes = + zfs_getwinflags(tzp); + fifdi->FileId.QuadPart = tzp->z_id; + nameptr = fifdi->FileName; + fifdi->FileNameLength = namelenholder; + break; + + case FileIdExtdDirectoryInformation: + structsize = FIELD_OFFSET( + FILE_ID_EXTD_DIR_INFORMATION, + FileName[0]); + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + break; + FILE_ID_EXTD_DIR_INFORMATION *fiedi = + (FILE_ID_EXTD_DIR_INFORMATION *) + ctx->bufptr; + next_offset = &fiedi->NextEntryOffset; + + fiedi->FileIndex = ctx->offset; + fiedi->AllocationSize.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + P2ROUNDUP(tzp->z_size, + zfs_blksz(tzp)); + fiedi->EndOfFile.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + tzp->z_size; + TIME_UNIX_TO_WINDOWS(mtime, + fiedi->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + fiedi->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + fiedi->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(tzp->z_atime, + fiedi->LastAccessTime.QuadPart); + fiedi->EaSize = + tzp->z_pflags & ZFS_REPARSE ? + 0xa0000003 : + xattr_getsize(ZTOV(tzp)); + fiedi->FileAttributes = + zfs_getwinflags(tzp); + RtlCopyMemory(&fiedi->FileId.Identifier[0], &tzp->z_id, + sizeof (UINT64)); + guid = dmu_objset_fsid_guid(zfsvfs->z_os); + RtlCopyMemory(&fiedi->FileId.Identifier[sizeof (UINT64)], + &guid, sizeof (UINT64)); + nameptr = fiedi->FileName; + fiedi->FileNameLength = namelenholder; + break; + + case FileIdExtdBothDirectoryInformation: + structsize = FIELD_OFFSET( + FILE_ID_EXTD_BOTH_DIR_INFORMATION, + FileName[0]); + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + break; + FILE_ID_EXTD_BOTH_DIR_INFORMATION *fiebdi = + (FILE_ID_EXTD_BOTH_DIR_INFORMATION *) + ctx->bufptr; + next_offset = &fiebdi->NextEntryOffset; + + fiebdi->FileIndex = ctx->offset; + fiebdi->AllocationSize.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + P2ROUNDUP(tzp->z_size, + zfs_blksz(tzp)); + fiebdi->EndOfFile.QuadPart = + S_ISDIR(tzp->z_mode) ? 0 : + tzp->z_size; + TIME_UNIX_TO_WINDOWS(mtime, + fiebdi->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + fiebdi->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + fiebdi->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(tzp->z_atime, + fiebdi->LastAccessTime.QuadPart); + fiebdi->EaSize = + xattr_getsize(ZTOV(tzp)); + fiebdi->ReparsePointTag = + tzp->z_pflags & ZFS_REPARSE ? + get_reparse_tag(tzp) : 0; + fiebdi->FileAttributes = + zfs_getwinflags(tzp); + fiebdi->ShortNameLength = 0; + RtlCopyMemory(&fiebdi->FileId.Identifier[0], &tzp->z_id, + sizeof (UINT64)); + guid = dmu_objset_fsid_guid(zfsvfs->z_os); + RtlCopyMemory(&fiebdi->FileId.Identifier[sizeof (UINT64)], + &guid, sizeof (UINT64)); + nameptr = fiebdi->FileName; + fiebdi->FileNameLength = namelenholder; + break; + + default: + panic("%s unknown listing type %d\n", + __func__, ctx->dirlisttype); + } + + // Release the zp + if (get_zp == 0 && tzp != NULL) { + VN_RELE(ZTOV(tzp)); + } + + // If know we can't fit struct, just leave + if (ctx->outcount + structsize + + namelenholder > ctx->bufsize) + return (ENOSPC); + + rawsize = structsize + namelenholder; + reclen = DIRENT_RECLEN(rawsize); /* align to 8 */ + + /* + * Will this entry fit in the buffer? + * This time with alignment + */ + if (ctx->outcount + rawsize > ctx->bufsize) { + return (ENOSPC); + } + + // If it is going to fit, compute alignment, + // in case this dir entry is the last one, + // we don't align last one. + ctx->last_alignment = reclen - rawsize; + + // Convert the filename over, or as much + // as we can fit + ULONG namelenholder2 = 0; + error = RtlUTF8ToUnicodeN(nameptr, + namelenholder, &namelenholder2, + name, namelen); + ASSERT(namelenholder == namelenholder2); +#if 0 + dprintf("%s: '%.*S' -> '%s' (namelen %d bytes: " + "structsize %d)\n", __func__, + namelenholder / sizeof (WCHAR), nameptr, + name, namelenholder, structsize); +#endif + + /* SEARCH PATTERN */ + if (zccb->searchname.Buffer && zccb->searchname.Length) { + UNICODE_STRING thisname; + // dprintf("%s: '%.*S' -> '%s'\n", __func__, + // tmpnamelen / sizeof(WCHAR), tmpname, zap.za_name); + + thisname.Buffer = nameptr; + thisname.Length = thisname.MaximumLength = namelenholder2; + // wildcard? + if (zccb->ContainsWildCards) { + if (!FsRtlIsNameInExpression(&zccb->searchname, + &thisname, + !(zfsvfs->z_case == ZFS_CASE_SENSITIVE), + NULL)) + return (ESRCH); + } else { + if (!FsRtlAreNamesEqual(&thisname, + &zccb->searchname, + !(zfsvfs->z_case == ZFS_CASE_SENSITIVE), + NULL)) + return (ESRCH); + } +#if 0 + dprintf("comparing names '%.*S' == '%.*S' skip %d\n", + thisname.Length / sizeof (WCHAR), thisname.Buffer, + zccb->searchname.Length / sizeof (WCHAR), + zccb->searchname.Buffer, + skip_this_entry); +#endif + } + /* SEARCH PATTERN */ + + + + + // If we aren't to skip, advance all pointers + VERIFY3P(next_offset, !=, NULL); + ctx->next_offset = next_offset; + *ctx->next_offset = reclen; + + ctx->outcount += reclen; + ctx->bufptr += reclen; + return (0); +} + +/* + * Lookup/Create an extended attribute entry. + * + * Input arguments: + * dzp - znode for hidden attribute directory + * name - name of attribute + * flag - ZNEW: if the entry already exists, fail with EEXIST. + * ZEXISTS: if the entry does not exist, fail with ENOENT. + * + * Output arguments: + * vpp - pointer to the vnode for the entry (NULL if there isn't one) + * + * Return value: 0 on success or errno value on failure. + */ +int +zfs_obtain_xattr(znode_t *dzp, const char *name, mode_t mode, cred_t *cr, + vnode_t **vpp, int flag) +{ + int error = 0; + znode_t *xzp = NULL; + zfsvfs_t *zfsvfs = dzp->z_zfsvfs; + zilog_t *zilog; + zfs_dirlock_t *dl; + dmu_tx_t *tx; + struct vnode_attr vattr = { 0 }; + struct componentname cn = { 0 }; + zfs_acl_ids_t acl_ids; + + /* zfs_dirent_lock() expects a component name */ + + if ((error = zfs_enter_verify_zp(zfsvfs, dzp, FTAG)) != 0) + return (error); + zilog = zfsvfs->z_log; + + vattr.va_type = VREG; + vattr.va_mode = mode & ~S_IFMT; + vattr.va_mask = ATTR_TYPE | ATTR_MODE; + + if ((error = zfs_acl_ids_create(dzp, 0, + &vattr, cr, NULL, &acl_ids, NULL)) != 0) { + zfs_exit(zfsvfs, FTAG); + return (error); + } + + cn.cn_namelen = cn.cn_pnlen = strlen(name)+1; + cn.cn_nameptr = cn.cn_pnbuf = (char *)kmem_zalloc(cn.cn_pnlen, + KM_SLEEP); + +top: + /* Lock the attribute entry name. */ + if ((error = zfs_dirent_lock(&dl, dzp, (char *)name, &xzp, flag, + NULL, &cn))) { + goto out; + } + /* If the name already exists, we're done. */ + if (xzp != NULL) { + zfs_dirent_unlock(dl); + goto out; + } + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_sa(tx, dzp->z_sa_hdl, B_FALSE); + dmu_tx_hold_zap(tx, dzp->z_id, TRUE, (char *)name); + dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); + +#if 1 // FIXME + if (dzp->z_pflags & ZFS_INHERIT_ACE) { + dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, SPA_MAXBLOCKSIZE); + } +#endif + zfs_sa_upgrade_txholds(tx, dzp); + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + zfs_dirent_unlock(dl); + if (error == ERESTART) { + dmu_tx_wait(tx); + dmu_tx_abort(tx); + goto top; + } + dmu_tx_abort(tx); + goto out; + } + + zfs_mknode(dzp, &vattr, tx, cr, 0, &xzp, &acl_ids); + + /* + * ASSERT(xzp->z_id == zoid); + */ + (void) zfs_link_create(dl, xzp, tx, ZNEW); + zfs_log_create(zilog, tx, TX_CREATE, dzp, xzp, (char *)name, + NULL /* vsecp */, 0 /* acl_ids.z_fuidp */, &vattr); + dmu_tx_commit(tx); + + /* + * OS X - attach the vnode _after_ committing the transaction + */ + zfs_znode_getvnode(xzp, dzp, zfsvfs); + + zfs_dirent_unlock(dl); +out: + zfs_acl_ids_free(&acl_ids); + if (cn.cn_pnbuf) + kmem_free(cn.cn_pnbuf, cn.cn_pnlen); + + /* The REPLACE error if doesn't exist is ENOATTR */ + if ((flag & ZEXISTS) && (error == ENOENT)) + error = STATUS_NO_EAS_ON_FILE; + + if (xzp) + *vpp = ZTOV(xzp); + + zfs_exit(zfsvfs, FTAG); + return (error); +} + +/* + * ace_trivial: + * determine whether an ace_t acl is trivial + * + * Trivialness implies that the acl is composed of only + * owner, group, everyone entries. ACL can't + * have read_acl denied, and write_owner/write_acl/write_attributes + * can only be owner@ entry. + */ +int +ace_trivial_common(void *acep, int aclcnt, + uintptr_t (*walk)(void *, uintptr_t, int, + uint16_t *, uint16_t *, uint32_t *)) +{ + uint16_t flags; + uint32_t mask; + uint16_t type; + uint64_t cookie = 0; + + while ((cookie = walk(acep, cookie, aclcnt, &flags, &type, &mask))) { + switch (flags & ACE_TYPE_FLAGS) { + case ACE_OWNER: + case ACE_GROUP|ACE_IDENTIFIER_GROUP: + case ACE_EVERYONE: + break; + default: + return (1); + + } + + if (flags & (ACE_FILE_INHERIT_ACE| + ACE_DIRECTORY_INHERIT_ACE|ACE_NO_PROPAGATE_INHERIT_ACE| + ACE_INHERIT_ONLY_ACE)) + return (1); + + /* + * Special check for some special bits + * + * Don't allow anybody to deny reading basic + * attributes or a files ACL. + */ + if ((mask & (ACE_READ_ACL|ACE_READ_ATTRIBUTES)) && + (type == ACE_ACCESS_DENIED_ACE_TYPE)) + return (1); + + /* + * Delete permission is never set by default + */ + if (mask & ACE_DELETE) + return (1); + + /* + * Child delete permission should be accompanied by write + */ + if ((mask & ACE_DELETE_CHILD) && !(mask & ACE_WRITE_DATA)) + return (1); + /* + * only allow owner@ to have + * write_acl/write_owner/write_attributes/write_xattr/ + */ + + if (type == ACE_ACCESS_ALLOWED_ACE_TYPE && + (!(flags & ACE_OWNER) && (mask & + (ACE_WRITE_OWNER|ACE_WRITE_ACL| ACE_WRITE_ATTRIBUTES| + ACE_WRITE_NAMED_ATTRS)))) + return (1); + + } + + return (0); +} + + +void +acl_trivial_access_masks(mode_t mode, boolean_t isdir, trivial_acl_t *masks) +{ + uint32_t read_mask = ACE_READ_DATA; + uint32_t write_mask = ACE_WRITE_DATA|ACE_APPEND_DATA; + uint32_t execute_mask = ACE_EXECUTE; + + if (isdir) + write_mask |= ACE_DELETE_CHILD; + + masks->deny1 = 0; + if (!(mode & S_IRUSR) && (mode & (S_IRGRP|S_IROTH))) + masks->deny1 |= read_mask; + if (!(mode & S_IWUSR) && (mode & (S_IWGRP|S_IWOTH))) + masks->deny1 |= write_mask; + if (!(mode & S_IXUSR) && (mode & (S_IXGRP|S_IXOTH))) + masks->deny1 |= execute_mask; + + masks->deny2 = 0; + if (!(mode & S_IRGRP) && (mode & S_IROTH)) + masks->deny2 |= read_mask; + if (!(mode & S_IWGRP) && (mode & S_IWOTH)) + masks->deny2 |= write_mask; + if (!(mode & S_IXGRP) && (mode & S_IXOTH)) + masks->deny2 |= execute_mask; + + masks->allow0 = 0; + if ((mode & S_IRUSR) && (!(mode & S_IRGRP) && (mode & S_IROTH))) + masks->allow0 |= read_mask; + if ((mode & S_IWUSR) && (!(mode & S_IWGRP) && (mode & S_IWOTH))) + masks->allow0 |= write_mask; + if ((mode & S_IXUSR) && (!(mode & S_IXGRP) && (mode & S_IXOTH))) + masks->allow0 |= execute_mask; + + masks->owner = ACE_WRITE_ATTRIBUTES|ACE_WRITE_OWNER|ACE_WRITE_ACL| + ACE_WRITE_NAMED_ATTRS|ACE_READ_ACL|ACE_READ_ATTRIBUTES| + ACE_READ_NAMED_ATTRS|ACE_SYNCHRONIZE; + if (mode & S_IRUSR) + masks->owner |= read_mask; + if (mode & S_IWUSR) + masks->owner |= write_mask; + if (mode & S_IXUSR) + masks->owner |= execute_mask; + + masks->group = ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATTRS| + ACE_SYNCHRONIZE; + if (mode & S_IRGRP) + masks->group |= read_mask; + if (mode & S_IWGRP) + masks->group |= write_mask; + if (mode & S_IXGRP) + masks->group |= execute_mask; + + masks->everyone = ACE_READ_ACL|ACE_READ_ATTRIBUTES|ACE_READ_NAMED_ATTRS| + ACE_SYNCHRONIZE; + if (mode & S_IROTH) + masks->everyone |= read_mask; + if (mode & S_IWOTH) + masks->everyone |= write_mask; + if (mode & S_IXOTH) + masks->everyone |= execute_mask; +} + + + +#define KAUTH_DIR_WRITE (KAUTH_VNODE_ACCESS | KAUTH_VNODE_ADD_FILE | \ + KAUTH_VNODE_ADD_SUBDIRECTORY | \ + KAUTH_VNODE_DELETE_CHILD) + +#define KAUTH_DIR_READ (KAUTH_VNODE_ACCESS | KAUTH_VNODE_LIST_DIRECTORY) + +#define KAUTH_DIR_EXECUTE (KAUTH_VNODE_ACCESS | KAUTH_VNODE_SEARCH) + +#define KAUTH_FILE_WRITE (KAUTH_VNODE_ACCESS | KAUTH_VNODE_WRITE_DATA) + +#define KAUTH_FILE_READ (KAUTH_VNODE_ACCESS | KAUTH_VNODE_READ_DATA) + +#define KAUTH_FILE_EXECUTE (KAUTH_VNODE_ACCESS | KAUTH_VNODE_EXECUTE) + +/* + * Compute the same user access value as getattrlist(2) + */ +uint32_t +getuseraccess(znode_t *zp, vfs_context_t ctx) +{ + uint32_t user_access = 0; +#if 0 + vnode_t *vp; + int error = 0; + zfs_acl_phys_t acl_phys; + /* Only take the expensive vnode_authorize path when we have an ACL */ + + error = sa_lookup(zp->z_sa_hdl, SA_ZPL_ZNODE_ACL(zp->z_zfsvfs), + &acl_phys, sizeof (acl_phys)); + + if (error || acl_phys.z_acl_count == 0) { + kauth_cred_t cred = vfs_context_ucred(ctx); + uint64_t obj_uid; + uint64_t obj_mode; + + /* User id 0 (root) always gets access. */ + if (!vfs_context_suser(ctx)) { + return (R_OK | W_OK | X_OK); + } + + sa_lookup(zp->z_sa_hdl, SA_ZPL_UID(zp->z_zfsvfs), + &obj_uid, sizeof (obj_uid)); + sa_lookup(zp->z_sa_hdl, SA_ZPL_MODE(zp->z_zfsvfs), + &obj_mode, sizeof (obj_mode)); + + // obj_uid = pzp->zp_uid; + obj_mode = obj_mode & MODEMASK; + if (obj_uid == UNKNOWNUID) { + obj_uid = kauth_cred_getuid(cred); + } + if ((obj_uid == kauth_cred_getuid(cred)) || + (obj_uid == UNKNOWNUID)) { + return (((u_int32_t)obj_mode & S_IRWXU) >> 6); + } + /* Otherwise, settle for 'others' access. */ + return ((u_int32_t)obj_mode & S_IRWXO); + } + vp = ZTOV(zp); + if (vnode_isdir(vp)) { + if (vnode_authorize(vp, NULLVP, KAUTH_DIR_WRITE, ctx) == 0) + user_access |= W_OK; + if (vnode_authorize(vp, NULLVP, KAUTH_DIR_READ, ctx) == 0) + user_access |= R_OK; + if (vnode_authorize(vp, NULLVP, KAUTH_DIR_EXECUTE, ctx) == 0) + user_access |= X_OK; + } else { + if (vnode_authorize(vp, NULLVP, KAUTH_FILE_WRITE, ctx) == 0) + user_access |= W_OK; + if (vnode_authorize(vp, NULLVP, KAUTH_FILE_READ, ctx) == 0) + user_access |= R_OK; + if (vnode_authorize(vp, NULLVP, KAUTH_FILE_EXECUTE, ctx) == 0) + user_access |= X_OK; + } +#endif + return (user_access); +} + +#define KAUTH_WKG_NOT 0 /* not a well-known GUID */ +#define KAUTH_WKG_OWNER 1 +#define KAUTH_WKG_GROUP 2 +#define KAUTH_WKG_NOBODY 3 +#define KAUTH_WKG_EVERYBODY 4 + + +static unsigned char fingerprint[] = {0xab, 0xcd, 0xef, 0xab, 0xcd, 0xef, + 0xab, 0xcd, 0xef, 0xab, 0xcd, 0xef}; + +/* + * Convert "Well Known" GUID to enum type. + */ +int +kauth_wellknown_guid(guid_t *guid) +{ + uint32_t last = 0; + + if (memcmp(fingerprint, guid->g_guid, sizeof (fingerprint))) + return (KAUTH_WKG_NOT); + + last = BE_32(*((uint32_t *)&guid->g_guid[12])); + + switch (last) { + case 0x0c: + return (KAUTH_WKG_EVERYBODY); + case 0x0a: + return (KAUTH_WKG_OWNER); + case 0x10: + return (KAUTH_WKG_GROUP); + case 0xFFFFFFFE: + return (KAUTH_WKG_NOBODY); + } + + return (KAUTH_WKG_NOT); +} + + +/* + * Set GUID to "well known" guid, based on enum type + */ +void +nfsacl_set_wellknown(int wkg, guid_t *guid) +{ + /* + * All WKGs begin with the same 12 bytes. + */ + memcpy((void *)guid, fingerprint, 12); + /* + * The final 4 bytes are our code (in network byte order). + */ + switch (wkg) { + case 4: + *((uint32_t *)&guid->g_guid[12]) = BE_32(0x0000000c); + break; + case 3: + *((uint32_t *)&guid->g_guid[12]) = BE_32(0xfffffffe); + break; + case 1: + *((uint32_t *)&guid->g_guid[12]) = BE_32(0x0000000a); + break; + case 2: + *((uint32_t *)&guid->g_guid[12]) = BE_32(0x00000010); + }; +} + + +/* + * Convert Darwin ACL list, into ZFS ACL "aces" list. + */ +void +aces_from_acl(ace_t *aces, int *nentries, struct kauth_acl *k_acl, + int *seen_type) +{ +#if 0 + int i; + ace_t *ace; + guid_t *guidp; + kauth_ace_rights_t *ace_rights; + uid_t who; + uint32_t mask = 0; + uint16_t flags = 0; + uint16_t type = 0; + uint32_t ace_flags; + int wkg; + int err = 0; + + *nentries = k_acl->acl_entrycount; + + // memset(aces, 0, sizeof (*aces) * *nentries); + + // *nentries = aclp->acl_cnt; + for (i = 0; i < *nentries; i++) { + // entry = &(aclp->acl_entry[i]); + + flags = 0; + mask = 0; + + ace = &(aces[i]); + + /* Note Mac OS X GUID is a 128-bit identifier */ + guidp = &k_acl->acl_ace[i].ace_applicable; + + who = -1; + wkg = kauth_wellknown_guid(guidp); + + switch (wkg) { + case KAUTH_WKG_OWNER: + flags |= ACE_OWNER; + if (seen_type) *seen_type |= ACE_OWNER; + break; + case KAUTH_WKG_GROUP: + flags |= ACE_GROUP|ACE_IDENTIFIER_GROUP; + if (seen_type) *seen_type |= ACE_GROUP; + break; + case KAUTH_WKG_EVERYBODY: + flags |= ACE_EVERYONE; + if (seen_type) *seen_type |= ACE_EVERYONE; + break; + + case KAUTH_WKG_NOBODY: + default: + /* Try to get a uid from supplied guid */ + err = kauth_cred_guid2uid(guidp, &who); + if (err) { + err = kauth_cred_guid2gid(guidp, &who); + if (!err) { + flags |= ACE_IDENTIFIER_GROUP; + } + } + if (err) { + *nentries = 0; + dprintf("ZFS: return to guid2gid\n"); + return; + } + } // switch + ace->a_who = who; + + ace_rights = k_acl->acl_ace[i].ace_rights; + if (ace_rights & KAUTH_VNODE_READ_DATA) + mask |= ACE_READ_DATA; + if (ace_rights & KAUTH_VNODE_WRITE_DATA) + mask |= ACE_WRITE_DATA; + if (ace_rights & KAUTH_VNODE_APPEND_DATA) + mask |= ACE_APPEND_DATA; + if (ace_rights & KAUTH_VNODE_READ_EXTATTRIBUTES) + mask |= ACE_READ_NAMED_ATTRS; + if (ace_rights & KAUTH_VNODE_WRITE_EXTATTRIBUTES) + mask |= ACE_WRITE_NAMED_ATTRS; + if (ace_rights & KAUTH_VNODE_EXECUTE) + mask |= ACE_EXECUTE; + if (ace_rights & KAUTH_VNODE_DELETE_CHILD) + mask |= ACE_DELETE_CHILD; + if (ace_rights & KAUTH_VNODE_READ_ATTRIBUTES) + mask |= ACE_READ_ATTRIBUTES; + if (ace_rights & KAUTH_VNODE_WRITE_ATTRIBUTES) + mask |= ACE_WRITE_ATTRIBUTES; + if (ace_rights & KAUTH_VNODE_DELETE) + mask |= ACE_DELETE; + if (ace_rights & KAUTH_VNODE_READ_SECURITY) + mask |= ACE_READ_ACL; + if (ace_rights & KAUTH_VNODE_WRITE_SECURITY) + mask |= ACE_WRITE_ACL; + if (ace_rights & KAUTH_VNODE_TAKE_OWNERSHIP) + mask |= ACE_WRITE_OWNER; + if (ace_rights & KAUTH_VNODE_SYNCHRONIZE) + mask |= ACE_SYNCHRONIZE; + ace->a_access_mask = mask; + + ace_flags = k_acl->acl_ace[i].ace_flags; + if (ace_flags & KAUTH_ACE_FILE_INHERIT) + flags |= ACE_FILE_INHERIT_ACE; + if (ace_flags & KAUTH_ACE_DIRECTORY_INHERIT) + flags |= ACE_DIRECTORY_INHERIT_ACE; + if (ace_flags & KAUTH_ACE_LIMIT_INHERIT) + flags |= ACE_NO_PROPAGATE_INHERIT_ACE; + if (ace_flags & KAUTH_ACE_ONLY_INHERIT) + flags |= ACE_INHERIT_ONLY_ACE; + ace->a_flags = flags; + + switch (ace_flags & KAUTH_ACE_KINDMASK) { + case KAUTH_ACE_PERMIT: + type = ACE_ACCESS_ALLOWED_ACE_TYPE; + break; + case KAUTH_ACE_DENY: + type = ACE_ACCESS_DENIED_ACE_TYPE; + break; + case KAUTH_ACE_AUDIT: + type = ACE_SYSTEM_AUDIT_ACE_TYPE; + break; + case KAUTH_ACE_ALARM: + type = ACE_SYSTEM_ALARM_ACE_TYPE; + break; + } + ace->a_type = type; + dprintf(" ACL: %d type %04x, mask %04x, flags %04x, who %d\n", + i, type, mask, flags, who); + } +#endif +} + + + +int +zpl_xattr_set_sa(struct vnode *vp, const char *name, const void *value, + size_t size, int flags, cred_t *cr) +{ + znode_t *zp = VTOZ(vp); + nvlist_t *nvl; + size_t sa_size; + int error; + + ASSERT(zp->z_xattr_cached); + nvl = zp->z_xattr_cached; + + if (value == NULL) { + error = -nvlist_remove(nvl, name, DATA_TYPE_BYTE_ARRAY); + if (error == -ENOENT) + return (error); + // error = zpl_xattr_set_dir(vp, name, NULL, 0, flags, cr); + } else { + /* Limited to 32k to keep nvpair memory allocations small */ + if (size > DXATTR_MAX_ENTRY_SIZE) + return (-EFBIG); + + /* Prevent the DXATTR SA from consuming the entire SA region */ + error = -nvlist_size(nvl, &sa_size, NV_ENCODE_XDR); + if (error) + return (error); + + if (sa_size > DXATTR_MAX_SA_SIZE) + return (-EFBIG); + error = -nvlist_add_byte_array(nvl, name, + (uchar_t *)value, size); + if (error) + return (error); + } + + /* Update the SA for additions, modifications, and removals. */ + if (!error) + error = -zfs_sa_set_xattr(zp, name, value, size); + + ASSERT3S(error, <=, 0); + + return (error); +} + +int +zpl_xattr_get_sa(struct vnode *vp, const char *name, void *value, size_t size) +{ + znode_t *zp = VTOZ(vp); + uchar_t *nv_value; + uint_t nv_size; + int error = 0; + +#ifdef __LINUX__ + ASSERT(RW_LOCK_HELD(&zp->z_xattr_lock)); +#endif + + mutex_enter(&zp->z_lock); + if (zp->z_xattr_cached == NULL) + error = -zfs_sa_get_xattr(zp); + mutex_exit(&zp->z_lock); + + if (error) + return (error); + + ASSERT(zp->z_xattr_cached); + error = -nvlist_lookup_byte_array(zp->z_xattr_cached, name, + &nv_value, &nv_size); + if (error) + return (error); + + if (!size) + return (nv_size); + if (size < nv_size) + return (-ERANGE); + + memcpy(value, nv_value, nv_size); + + return (nv_size); +} + +/* dst buffer must be at least UUID_PRINTABLE_STRING_LENGTH bytes */ +int +zfs_vfs_uuid_unparse(uuid_t uuid, char *dst) +{ + if (!uuid || !dst) { + dprintf("%s missing argument\n", __func__); + return (EINVAL); + } + + snprintf(dst, UUID_PRINTABLE_STRING_LENGTH, "%02x%02x%02x%02x-" + "%02x%02x-%02x%02x-%02x%02x-%02x%02x%02x%02x%02x%02x", + uuid[0], uuid[1], uuid[2], uuid[3], + uuid[4], uuid[5], uuid[6], uuid[7], + uuid[8], uuid[9], uuid[10], uuid[11], + uuid[12], uuid[13], uuid[14], uuid[15]); + + return (0); +} + +#include +int +zfs_vfs_uuid_gen(const char *osname, uuid_t uuid) +{ +#if 1 + MD5_CTX md5c; + /* namespace (generated by uuidgen) */ + /* 50670853-FBD2-4EC3-9802-73D847BF7E62 */ + char namespace[16] = {0x50, 0x67, 0x08, 0x53, /* - */ + 0xfb, 0xd2, /* - */ 0x4e, 0xc3, /* - */ + 0x98, 0x02, /* - */ + 0x73, 0xd8, 0x47, 0xbf, 0x7e, 0x62}; + + /* Validate arguments */ + if (!osname || !uuid || strlen(osname) == 0) { + dprintf("%s missing argument\n", __func__); + return (EINVAL); + } + + /* + * UUID version 3 (MD5) namespace variant: + * hash namespace (uuid) together with name + */ + MD5Init(&md5c); + MD5Update(&md5c, &namespace, sizeof (namespace)); + MD5Update(&md5c, osname, strlen(osname)); + MD5Final(uuid, &md5c); + + /* + * To make UUID version 3, twiddle a few bits: + * xxxxxxxx-xxxx-Mxxx-Nxxx-xxxxxxxxxxxx + * [uint32]-[uin-t32]-[uin-t32][uint32] + * M should be 0x3 to indicate uuid v3 + * N should be 0x8, 0x9, 0xa, or 0xb + */ + uuid[6] = (uuid[6] & 0x0F) | 0x30; + uuid[8] = (uuid[8] & 0x3F) | 0x80; + + /* Print all caps */ + // dprintf("%s UUIDgen: [%s](%ld)->" + dprintf("%s UUIDgen: [%s](%ld) -> " + "[%02X%02X%02X%02X-%02X%02X-%02X%02X-" + "%02X%02X-%02X%02X%02X%02X%02X%02X]\n", + __func__, osname, strlen(osname), + uuid[0], uuid[1], uuid[2], uuid[3], + uuid[4], uuid[5], uuid[6], uuid[7], + uuid[8], uuid[9], uuid[10], uuid[11], + uuid[12], uuid[13], uuid[14], uuid[15]); +#endif + return (0); +} + + +/* + * Attempt to build a full path from a zp, traversing up through parents. + * start_zp should already be held (VN_HOLD()) and if parent_zp is + * not NULL, it too should be held. + * Returned is an allocated string (kmem_alloc) which should be freed + * by caller (kmem_free(fullpath, returnsize)). + * If supplied, start_zp_offset, is the index into fullpath where the + * start_zp component name starts. (Point between start_parent/start_zp). + * returnsize includes the final NULL, so it is strlen(fullpath)+1 + */ +int +zfs_build_path(znode_t *start_zp, znode_t *start_parent, char **fullpath, + uint32_t *returnsize, uint32_t *start_zp_offset) +{ + char *work; + int index, size, part, error = 0; + znode_t *zp = NULL; + znode_t *dzp = NULL; + uint64_t parent; + zfsvfs_t *zfsvfs; + char name[MAXPATHLEN]; + // No output? nothing to do + if (!fullpath) + return (EINVAL); + // No input? nothing to do + if (!start_zp) + return (EINVAL); + + zfsvfs = start_zp->z_zfsvfs; + zp = start_zp; + + VN_HOLD(ZTOV(zp)); + + work = kmem_alloc(MAXPATHLEN * 2, KM_SLEEP); + index = MAXPATHLEN * 2 - 1; + + work[--index] = 0; + size = 1; + + while (1) { + + // Fetch parent + if (start_parent) { + dzp = start_parent; + VN_HOLD(ZTOV(dzp)); + parent = dzp->z_id; + start_parent = NULL; + } else if (zp->z_sa_hdl != NULL) { + VERIFY(sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)) == 0); + error = zfs_zget(zfsvfs, parent, &dzp); + if (error) { + dprintf("%s: zget failed %d\n", + __func__, error); + goto failed; + } + } else if (zfsctl_is_node(zp)) { + struct vnode *vp = NULL; + vp = zfs_root_dotdot(ZTOV(zp)); + // .zfs/snapshot/$name - parent is snapshot + if (vp == NULL) { + dprintf("%s: snapshot dotdot failed %d\n", + __func__, error); + goto failed; + } + dzp = VTOZ(vp); + } + // dzp held from here. + + // Find name + if (zp->z_id == zfsvfs->z_root) + strlcpy(name, "", MAXPATHLEN); + else if (zp->z_id == ZFSCTL_INO_ROOT) + strlcpy(name, ZFS_CTLDIR_NAME, MAXPATHLEN); + else if (zp->z_id == ZFSCTL_INO_SNAPDIR) + strlcpy(name, ZFS_SNAPDIR_NAME, MAXPATHLEN); + else if (zfsctl_is_leafnode(zp)) { + while (error == 0) { + uint64_t id, pos = 0; + boolean_t case_conflict; + dsl_pool_config_enter( + dmu_objset_pool(zfsvfs->z_os), FTAG); + error = dmu_snapshot_list_next(zfsvfs->z_os, + MAXPATHLEN, name, &id, &pos, + &case_conflict); + dsl_pool_config_exit( + dmu_objset_pool(zfsvfs->z_os), FTAG); + if (error == 0 && + (ZFSCTL_INO_SNAPDIRS - id) == zp->z_id) + break; + } + if (error != 0) { + dprintf("%s: snapshot search failed %d\n", + __func__, error); + goto failed; + } + } else + if ((error = zap_value_search(zfsvfs->z_os, parent, + zp->z_id, ZFS_DIRENT_OBJ(-1ULL), name)) != 0) { + dprintf("%s: zap_value_search failed %d\n", + __func__, error); + goto failed; + } + // Copy in name. + part = strlen(name); + // Check there is room + if (part + 1 > index) { + dprintf("%s: out of space\n", __func__); + goto failed; + } + + index -= part; + memcpy(&work[index], name, part); + + // If start_zp, remember index (to be adjusted) + if (zp == start_zp && start_zp_offset) + *start_zp_offset = index; + + // Prepend "/" + work[--index] = '\\'; + size += part + 1; + + // Swap dzp and zp to "go up one". + VN_RELE(ZTOV(zp)); // we are done with zp. + zp = dzp; // Now focus on parent + dzp = NULL; + + if (zp == NULL) // No parent + break; + + // If parent, stop, "/" is already copied in. + if (zp->z_id == zfsvfs->z_root) + break; + } + + // Release "parent" if it was held, now called zp. + if (zp != NULL) + VN_RELE(ZTOV(zp)); + + // Correct index + if (start_zp_offset) + *start_zp_offset = *start_zp_offset - index; + if (returnsize) + *returnsize = size; + ASSERT(size != 0); + *fullpath = kmem_alloc(size, KM_SLEEP); + memmove(*fullpath, &work[index], size); + kmem_free(work, MAXPATHLEN * 2); + + // If "/" we don't want offset to be "1", but "0". + if ((*fullpath)[0] == '\\' && + (*fullpath)[1] == 0 && + start_zp_offset) + *start_zp_offset = 0; + + dprintf("%s: set '%s' as name\n", __func__, *fullpath); + return (0); + +failed: + if (zp != NULL) + VN_RELE(ZTOV(zp)); + if (dzp != NULL) + VN_RELE(ZTOV(dzp)); + kmem_free(work, MAXPATHLEN * 2); + return (-1); +} + +/* + * Eventually, build_path above could handle streams, but for now lets + * just set the stream name. + * Using FileTest on NTFS file:Zone.Identifier:$DATA returns the + * name "/src/openzfs/zpool.exe:Zone.Identifier" + */ +int +zfs_build_path_stream(znode_t *start_zp, znode_t *start_parent, char **fullpath, + uint32_t *returnsize, uint32_t *start_zp_offset, char *stream) +{ + + if (start_zp == NULL) + return (EINVAL); + + if (stream == NULL) + return (EINVAL); + + if (start_zp->z_name_cache != NULL) { + kmem_free(start_zp->z_name_cache, start_zp->z_name_len); + start_zp->z_name_cache = NULL; + start_zp->z_name_len = 0; + } + + // start_parent->name + ":" + streamname + null + start_zp->z_name_cache = kmem_asprintf("%s:%s", + start_parent->z_name_cache, stream); + start_zp->z_name_len = strlen(start_zp->z_name_cache); + // start_zp->z_name_offset = start_parent->z_name_len + 1; + start_zp->z_name_offset = start_parent->z_name_offset; + + return (0); +} + +/* + * This is connected to IRP_MN_NOTIFY_DIRECTORY_CHANGE + * and sending the notifications of changes + * + * Should be sent as "file0:streamname" + */ +void +zfs_send_notify_stream(zfsvfs_t *zfsvfs, char *name, int nameoffset, + ULONG FilterMatch, ULONG Action, char *stream) +{ + mount_t *zmo; + zmo = zfsvfs->z_vfs; + UNICODE_STRING ustr; + UNICODE_STRING ustream; + + if (name == NULL) + return; + + AsciiStringToUnicodeString(name, &ustr); + + dprintf("%s: '%wZ' part '%S' %lu %u\n", __func__, &ustr, + /* &name[nameoffset], */ &ustr.Buffer[nameoffset], + FilterMatch, Action); + + if (stream != NULL) { + AsciiStringToUnicodeString(stream, &ustream); + dprintf("%s: with stream '%wZ'\n", __func__, &ustream); + } + + /* Is nameoffset in bytes, or in characters? */ + FsRtlNotifyFilterReportChange(zmo->NotifySync, &zmo->DirNotifyList, + (PSTRING)&ustr, + nameoffset * sizeof (WCHAR), + stream == NULL ? NULL : (PSTRING)&ustream, // StreamName + NULL, FilterMatch, Action, NULL, NULL); + + FreeUnicodeString(&ustr); + if (stream != NULL) + FreeUnicodeString(&ustream); +} + +// Filenames should be "/dir/filename:streamname" +// currently it is "streamname:$DATA" +void +zfs_send_notify(zfsvfs_t *zfsvfs, char *name, int nameoffset, + ULONG FilterMatch, ULONG Action) +{ + zfs_send_notify_stream(zfsvfs, name, nameoffset, FilterMatch, + Action, NULL); +} + + +void +zfs_uid2sid(uint64_t uid, SID **sid) +{ + int num; + SID *tmp; + + ASSERT(sid != NULL); + + // Root? + num = (uid == 0) ? 1 : 2; + + tmp = ExAllocatePoolWithTag(PagedPool, + offsetof(SID, SubAuthority) + (num * sizeof (ULONG)), 'zsid'); + + tmp->Revision = 1; + tmp->SubAuthorityCount = num; + tmp->IdentifierAuthority.Value[0] = 0; + tmp->IdentifierAuthority.Value[1] = 0; + tmp->IdentifierAuthority.Value[2] = 0; + tmp->IdentifierAuthority.Value[3] = 0; + tmp->IdentifierAuthority.Value[4] = 0; + + if (uid == 0) { + tmp->IdentifierAuthority.Value[5] = 5; + tmp->SubAuthority[0] = 18; + } else { + tmp->IdentifierAuthority.Value[5] = 22; + tmp->SubAuthority[0] = 1; + tmp->SubAuthority[1] = uid; // bits truncation? + } + + *sid = tmp; +} + +uint64_t +zfs_sid2uid(SID *sid) +{ + // Root + if (sid->Revision == 1 && sid->SubAuthorityCount == 1 && + sid->IdentifierAuthority.Value[0] == 0 && + sid->IdentifierAuthority.Value[1] == 0 && + sid->IdentifierAuthority.Value[2] == 0 && + sid->IdentifierAuthority.Value[3] == 0 && + sid->IdentifierAuthority.Value[4] == 0 && + sid->IdentifierAuthority.Value[5] == 18) + return (0); + + // Samba's SID scheme: S-1-22-1-X + if (sid->Revision == 1 && sid->SubAuthorityCount == 2 && + sid->IdentifierAuthority.Value[0] == 0 && + sid->IdentifierAuthority.Value[1] == 0 && + sid->IdentifierAuthority.Value[2] == 0 && + sid->IdentifierAuthority.Value[3] == 0 && + sid->IdentifierAuthority.Value[4] == 0 && + sid->IdentifierAuthority.Value[5] == 22 && + sid->SubAuthority[0] == 1) + return (sid->SubAuthority[1]); + + return (UID_NOBODY); +} + + +void +zfs_gid2sid(uint64_t gid, SID **sid) +{ + int num = 2; + SID *tmp; + + ASSERT(sid != NULL); + + tmp = ExAllocatePoolWithTag(PagedPool, + offsetof(SID, SubAuthority) + (num * sizeof (ULONG)), 'zsid'); + + tmp->Revision = 1; + tmp->SubAuthorityCount = num; + tmp->IdentifierAuthority.Value[0] = 0; + tmp->IdentifierAuthority.Value[1] = 0; + tmp->IdentifierAuthority.Value[2] = 0; + tmp->IdentifierAuthority.Value[3] = 0; + tmp->IdentifierAuthority.Value[4] = 0; + + tmp->IdentifierAuthority.Value[5] = 22; + tmp->SubAuthority[0] = 2; + tmp->SubAuthority[1] = gid; // bits truncation? + + *sid = tmp; +} + +void +zfs_freesid(SID *sid) +{ + ASSERT(sid != NULL); + ExFreePool(sid); +} + + +static ACL * +zfs_set_acl(dacl *dacls) +{ + int size, i; + ACL *acl = NULL; + ACCESS_ALLOWED_ACE *aaa; + + size = sizeof (ACL); + i = 0; + while (dacls[i].sid) { + size += sizeof (ACCESS_ALLOWED_ACE); + size += 8 + (dacls[i].sid->elements * sizeof (UINT32)) - + sizeof (ULONG); + i++; + } + + acl = ExAllocatePoolWithTag(PagedPool, size, 'zacl'); + if (!acl) + return (NULL); + + acl->AclRevision = ACL_REVISION; + acl->Sbz1 = 0; + acl->AclSize = size; + acl->AceCount = i; + acl->Sbz2 = 0; + + aaa = (ACCESS_ALLOWED_ACE*)&acl[1]; + i = 0; + while (dacls[i].sid) { + aaa->Header.AceType = ACCESS_ALLOWED_ACE_TYPE; + aaa->Header.AceFlags = dacls[i].flags; + aaa->Header.AceSize = sizeof (ACCESS_ALLOWED_ACE) - + sizeof (ULONG) + 8 + + (dacls[i].sid->elements * sizeof (UINT32)); + aaa->Mask = dacls[i].mask; + + RtlCopyMemory(&aaa->SidStart, dacls[i].sid, + 8 + (dacls[i].sid->elements * sizeof (UINT32))); + + aaa = (ACCESS_ALLOWED_ACE*)((UINT8*)aaa + aaa->Header.AceSize); + i++; + } + + return (acl); +} + + +void +zfs_set_security_root(struct vnode *vp) +{ + SECURITY_DESCRIPTOR sd; + SID *usersid = NULL, *groupsid = NULL; + znode_t *zp = VTOZ(vp); + NTSTATUS Status; + ACL *acl = NULL; + + Status = RtlCreateSecurityDescriptor(&sd, SECURITY_DESCRIPTOR_REVISION); + if (Status != STATUS_SUCCESS) + goto err; + + acl = def_dacls; + + zfs_uid2sid(zp->z_uid, &usersid); + zfs_gid2sid(zp->z_gid, &groupsid); + + RtlSetOwnerSecurityDescriptor(&sd, usersid, FALSE); + RtlSetGroupSecurityDescriptor(&sd, groupsid, FALSE); + + acl = zfs_set_acl(acl); + + if (acl) + Status = RtlSetDaclSecurityDescriptor(&sd, TRUE, acl, FALSE); + + ULONG buflen = 0; + Status = RtlAbsoluteToSelfRelativeSD(&sd, NULL, &buflen); + if (Status != STATUS_SUCCESS && + Status != STATUS_BUFFER_TOO_SMALL) + goto err; + + ASSERT(buflen != 0); + + void *tmp = ExAllocatePoolWithTag(PagedPool, buflen, 'ZSEC'); + if (tmp == NULL) + goto err; + + Status = RtlAbsoluteToSelfRelativeSD(&sd, tmp, &buflen); + + vnode_setsecurity(vp, tmp); + +err: + if (acl) + ExFreePool(acl); + if (usersid != NULL) + zfs_freesid(usersid); + if (groupsid != NULL) + zfs_freesid(groupsid); +} + +int +zfs_set_security(struct vnode *vp, struct vnode *dvp) +{ + SECURITY_SUBJECT_CONTEXT subjcont; + NTSTATUS Status; + SID *usersid = NULL, *groupsid = NULL; + int error = 0; + + if (vp == NULL) + return (0); + + if (vp->security_descriptor != NULL) + return (0); + + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + // If we are the rootvp, we don't have a parent, so do different setup + if (zp->z_id == zfsvfs->z_root || + zp->z_id == ZFSCTL_INO_ROOT) { + zfs_set_security_root(vp); + return (0); + } + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + // If no parent, find it. This will take one hold on + // dvp, either directly or from zget(). + znode_t *dzp = NULL; + if (dvp == NULL) { + if (zp->z_sa_hdl != NULL) { + uint64_t parent; + if (sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)) != 0) { + goto err; + } + if (zfs_zget(zfsvfs, parent, &dzp)) { + dvp = NULL; + goto err; + } + dvp = ZTOV(dzp); + + } else { // What to do if no sa_hdl ? + goto err; + } + } else { + VN_HOLD(dvp); + dzp = VTOZ(dvp); + } + + if (vnode_security(dvp) == NULL) + zfs_set_security(dvp, NULL); + + // We can fail here, if we are processing unlinked-list + if (vnode_security(dvp) == NULL) + goto err; + + ASSERT(dvp != NULL); + ASSERT(dzp != NULL); + ASSERT(vnode_security(dvp) != NULL); + + SeCaptureSubjectContext(&subjcont); + void *sd = NULL; + Status = SeAssignSecurityEx(vnode_security(dvp), NULL, (void**)&sd, + NULL, vnode_isdir(vp)?TRUE:FALSE, SEF_DACL_AUTO_INHERIT, + &subjcont, IoGetFileObjectGenericMapping(), PagedPool); + + if (Status != STATUS_SUCCESS) + goto err; + + vnode_setsecurity(vp, sd); + + zfs_uid2sid(zp->z_uid, &usersid); + RtlSetOwnerSecurityDescriptor(&sd, usersid, FALSE); + + zfs_gid2sid(zp->z_gid, &groupsid); + RtlSetGroupSecurityDescriptor(&sd, groupsid, FALSE); + +err: + if (dvp) + VN_RELE(dvp); + zfs_exit(zfsvfs, FTAG); + + if (usersid != NULL) + zfs_freesid(usersid); + if (groupsid != NULL) + zfs_freesid(groupsid); + return (0); +} + +// return true if a XATTR name should be skipped +int +xattr_protected(char *name) +{ + return (0); +} + +// return true if xattr is a stream (name ends with ":$DATA") +int +xattr_stream(char *name) +{ + char tail[] = ":$DATA"; + int taillen = sizeof (tail); + int len; + + if (name == NULL) + return (0); + len = strlen(name); + if (len < taillen) + return (0); + + if (strcmp(&name[len - taillen + 1], tail) == 0) + return (1); + + return (0); +} + +// Get the size needed for EA, check first if it is +// cached in vnode. Otherwise, compute it and set. +uint64_t +xattr_getsize(struct vnode *vp) +{ + znode_t *zp; + zfsvfs_t *zfsvfs; + ssize_t retsize = 0; + + if (vp == NULL) + return (0); + +#if 0 + boolean_t cached = B_FALSE; + uint64_t cached_size = 0ULL; + // To test the caching is correct + cached = vnode_easize(vp, &cached_size); +#else + // Cached? Easy, use it + if (vnode_easize(vp, &retsize)) + return (retsize); +#endif + + zp = VTOZ(vp); + zfsvfs = zp->z_zfsvfs; + + if (!zp->z_is_sa || zp->z_sa_hdl == NULL) + return (0); + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, NULL, 0, 0, + UIO_SYSSPACE, 0, 0); + + zpl_xattr_list(vp, &uio, &retsize, NULL); + + // It appears I should round it up here: + retsize += (((retsize)+3) & ~3) - (retsize); + + // Cache result, even if failure (cached as 0). + vnode_set_easize(vp, retsize); + + return (retsize); +} + +/* + * Call vnode_setunlink if zfs_zaccess_delete() allows it + * TODO: provide credentials + */ +NTSTATUS +zfs_setunlink(FILE_OBJECT *fo, vnode_t *dvp) +{ + vnode_t *vp = NULL; + NTSTATUS Status = STATUS_UNSUCCESSFUL; + + if (fo == NULL) { + Status = STATUS_INVALID_PARAMETER; + goto err; + } + + vp = fo->FsContext; + + if (vp == NULL) { + Status = STATUS_INVALID_PARAMETER; + goto err; + } + + znode_t *zp = NULL; + znode_t *dzp = NULL; + zfs_dirlist_t *zccb = fo->FsContext2; + + zfsvfs_t *zfsvfs; + + VN_HOLD(vp); + zp = VTOZ(vp); + + // Holding vp, not dvp, use "out:" to leave + + if (vp && zp) { + zfsvfs = zp->z_zfsvfs; + } else { + Status = STATUS_INVALID_PARAMETER; + goto out; + } + + // If it belongs in .zfs, just reply OK. + // mounting will attempted to delete directory + // to replace with reparse point. + if (zfsctl_is_node(zp)) { + if (zfsctl_is_leafnode(zp)) { + fo->DeletePending = TRUE; + ASSERT3P(zccb, !=, NULL); + zccb->deleteonclose = 1; + // We no longer use v_unlink so lets abuse + // it here until we decide we like it + vp->v_unlink = 1; + Status = STATUS_SUCCESS; + goto out; + } + Status = STATUS_CANNOT_DELETE; + goto out; + } + + if (zfsvfs->z_rdonly || vfs_isrdonly(zfsvfs->z_vfs) || + !spa_writeable(dmu_objset_spa(zfsvfs->z_os))) { + Status = STATUS_MEDIA_WRITE_PROTECTED; + goto out; + } + + // Cannot delete a user mapped image. + if (!MmFlushImageSection(&vp->SectionObjectPointers, + MmFlushForDelete)) { + Status = STATUS_CANNOT_DELETE; + goto out; + } + + // if dvp == null, find it + + if (dvp == NULL) { + uint64_t parent; + + if (sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)) != 0) { + goto err; + } + if (zfs_zget(zfsvfs, parent, &dzp)) { + dvp = NULL; + goto err; + } + dvp = ZTOV(dzp); + } else { + dzp = VTOZ(dvp); + VN_HOLD(dvp); + } + + + // Holding dvp, use "err:" to leave. + + // If we are root + if (zp->z_id == zfsvfs->z_root) { + Status = STATUS_CANNOT_DELETE; + goto err; + } + + // If we are a dir, and have more than "." and "..", we + // are not empty. + if (S_ISDIR(zp->z_mode)) { + + if (zp->z_size > 2) { + Status = STATUS_DIRECTORY_NOT_EMPTY; + goto err; + } + } + + int error = zfs_zaccess_delete(dzp, zp, 0, NULL); + + if (error == 0) { + ASSERT3P(zccb, !=, NULL); + zccb->deleteonclose = 1; + fo->DeletePending = TRUE; + Status = STATUS_SUCCESS; + } else { + Status = STATUS_ACCESS_DENIED; + } + +err: + if (dvp) { + VN_RELE(dvp); + dvp = NULL; + } + +out: + if (vp) { + VN_RELE(vp); + vp = NULL; + } + + return (Status); +} + +int +uio_prefaultpages(ssize_t n, struct uio *uio) +{ + return (0); +} + +/* No #pragma weaks here! */ +void +dmu_buf_add_ref(dmu_buf_t *db, const void *tag) +{ + dbuf_add_ref((dmu_buf_impl_t *)db, tag); +} + +boolean_t +dmu_buf_try_add_ref(dmu_buf_t *db, objset_t *os, uint64_t object, + uint64_t blkid, const void *tag) +{ + return (dbuf_try_add_ref(db, os, object, blkid, tag)); +} + +/* IRP_MJ_SET_INFORMATION helpers */ + + +NTSTATUS +set_file_basic_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_INVALID_PARAMETER; + + if (IrpSp->FileObject == NULL || IrpSp->FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + PFILE_OBJECT FileObject = IrpSp->FileObject; + struct vnode *vp = FileObject->FsContext; + mount_t *zmo = DeviceObject->DeviceExtension; + ULONG NotifyFilter = 0; + + zfsvfs_t *zfsvfs = NULL; + if (zmo != NULL && + (zfsvfs = vfs_fsprivate(zmo)) != NULL && + zfsvfs->z_rdonly) + return (STATUS_MEDIA_WRITE_PROTECTED); + + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if (VN_HOLD(vp) == 0 && VTOZ(vp) != NULL) { + FILE_BASIC_INFORMATION *fbi = + Irp->AssociatedIrp.SystemBuffer; + vattr_t va = { 0 }; + uint64_t unixtime[2] = { 0 }; + znode_t *zp = VTOZ(vp); + +// can request that the file system not update .. LastAccessTime, +// LastWriteTime, and ChangeTime .. setting the appropriate members to -1. +// ie, LastAccessTime = -1 -> atime = disabled - not implemented +// LastAccessTime = -2 -> cancel the disable (-1), return to normal. +// a value of "0" means to keep existing value. + if (fbi->ChangeTime.QuadPart > 0) { + TIME_WINDOWS_TO_UNIX(fbi->ChangeTime.QuadPart, + unixtime); + va.va_change_time.tv_sec = unixtime[0]; + va.va_change_time.tv_nsec = unixtime[1]; + va.va_active |= ATTR_CTIME; + } + if (fbi->LastWriteTime.QuadPart > 0) { + TIME_WINDOWS_TO_UNIX( + fbi->LastWriteTime.QuadPart, + unixtime); + va.va_modify_time.tv_sec = unixtime[0]; + va.va_modify_time.tv_nsec = unixtime[1]; + va.va_active |= ATTR_MTIME; + NotifyFilter |= FILE_NOTIFY_CHANGE_LAST_WRITE; + } + if (fbi->CreationTime.QuadPart > 0) { + TIME_WINDOWS_TO_UNIX(fbi->CreationTime.QuadPart, + unixtime); + va.va_create_time.tv_sec = unixtime[0]; + va.va_create_time.tv_nsec = unixtime[1]; + va.va_active |= ATTR_CRTIME; // ATTR_CRTIME + NotifyFilter |= FILE_NOTIFY_CHANGE_CREATION; + } + if (fbi->LastAccessTime.QuadPart > 0) { + TIME_WINDOWS_TO_UNIX( + fbi->LastAccessTime.QuadPart, + zp->z_atime); + NotifyFilter |= FILE_NOTIFY_CHANGE_LAST_ACCESS; + } + if (fbi->FileAttributes) + if (zfs_setwinflags(VTOZ(vp), + fbi->FileAttributes)) { + va.va_active |= ATTR_MODE; + NotifyFilter |= FILE_NOTIFY_CHANGE_ATTRIBUTES; + } + Status = zfs_setattr(zp, &va, 0, NULL, NULL); + + // zfs_setattr will turn ARCHIVE back on, when perhaps + // it is set off by this call + if (fbi->FileAttributes) + zfs_setwinflags(zp, fbi->FileAttributes); + + if (NotifyFilter != 0) + zfs_send_notify(zp->z_zfsvfs, zp->z_name_cache, + zp->z_name_offset, + NotifyFilter, + FILE_ACTION_MODIFIED); + + + VN_RELE(vp); + } + + return (Status); +} + +NTSTATUS +set_file_disposition_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, boolean_t ex) +{ + NTSTATUS Status = STATUS_INVALID_PARAMETER; + + if (IrpSp->FileObject == NULL || IrpSp->FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + PFILE_OBJECT FileObject = IrpSp->FileObject; + struct vnode *vp = FileObject->FsContext; + zfs_dirlist_t *zccb = FileObject->FsContext2; + mount_t *zmo = DeviceObject->DeviceExtension; + ULONG flags = 0; + + zfsvfs_t *zfsvfs = NULL; + if (zmo != NULL && + (zfsvfs = vfs_fsprivate(zmo)) != NULL && + zfsvfs->z_rdonly) + return (STATUS_MEDIA_WRITE_PROTECTED); + + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if (VN_HOLD(vp) == 0 && VTOZ(vp) != NULL) { + + if (ex) { + FILE_DISPOSITION_INFORMATION_EX *fdie = + Irp->AssociatedIrp.SystemBuffer; + flags = fdie->Flags; + } else { + FILE_DISPOSITION_INFORMATION *fdi = + Irp->AssociatedIrp.SystemBuffer; + flags = fdi->DeleteFile ? FILE_DISPOSITION_DELETE : 0; + } + + dprintf("Deletion %s on '%wZ'\n", + flags & FILE_DISPOSITION_DELETE ? "set" : "unset", + IrpSp->FileObject->FileName); + Status = STATUS_SUCCESS; + if (flags & FILE_DISPOSITION_DELETE) { + Status = zfs_setunlink(IrpSp->FileObject, NULL); + } else { + if (zccb) zccb->deleteonclose = 0; + FileObject->DeletePending = FALSE; + } + // Dirs marked for Deletion should release all + // pending Notify events + if (Status == STATUS_SUCCESS && + (flags & FILE_DISPOSITION_DELETE)) { +// FsRtlNotifyCleanup(zmo->NotifySync, +// &zmo->DirNotifyList, vp); + + FsRtlNotifyFullChangeDirectory(zmo->NotifySync, + &zmo->DirNotifyList, + FileObject->FsContext2, + NULL, + FALSE, + FALSE, + 0, + NULL, + NULL, + NULL); + } + + VN_RELE(vp); + } + return (Status); +} + +NTSTATUS +set_file_endoffile_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status = STATUS_SUCCESS; + + if (IrpSp->FileObject == NULL || IrpSp->FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + PFILE_OBJECT FileObject = IrpSp->FileObject; + struct vnode *vp = FileObject->FsContext; + zfs_dirlist_t *zccb = FileObject->FsContext2; + FILE_END_OF_FILE_INFORMATION *feofi = Irp->AssociatedIrp.SystemBuffer; + int changed = 0; + int error = 0; + mount_t *zmo = DeviceObject->DeviceExtension; + + zfsvfs_t *zfsvfs = NULL; + if (zmo != NULL && + (zfsvfs = vfs_fsprivate(zmo)) != NULL && + zfsvfs->z_rdonly) + return (STATUS_MEDIA_WRITE_PROTECTED); + + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if (vnode_isdir(vp)) + return (STATUS_INVALID_PARAMETER); + + dprintf("* File_EndOfFile_Information:\n"); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + if (VN_HOLD(vp) != 0) { + zfs_exit(zfsvfs, FTAG); + return (STATUS_INVALID_PARAMETER); + } + + znode_t *zp = VTOZ(vp); + + // From FASTFAT + // This is kinda gross, but if the file is not cached, but there is + // a data section, we have to cache the file to avoid a bunch of + // extra work. + BOOLEAN CacheMapInitialized = FALSE; + if (FileObject && FileObject->SectionObjectPointer && + (FileObject->SectionObjectPointer->DataSectionObject != NULL) && + (FileObject->SectionObjectPointer->SharedCacheMap == NULL) && + !FlagOn(Irp->Flags, IRP_PAGING_IO)) { + + vnode_pager_setsize(NULL, vp, zp->z_size, TRUE); + + CcInitializeCacheMap(FileObject, + (PCC_FILE_SIZES)&vp->FileHeader.AllocationSize, + FALSE, + &CacheManagerCallbacks, vp); + + // CcSetAdditionalCacheAttributes(FileObject, FALSE, FALSE); + CacheMapInitialized = TRUE; + } + + if (!zfsvfs->z_unmounted) { + + // DeleteOnClose just returns OK. + if (zccb && zccb->deleteonclose) { + Status = STATUS_SUCCESS; + goto out; + } + + // Advance only? + if (IrpSp->Parameters.SetFile.AdvanceOnly) { + if (feofi->EndOfFile.QuadPart > zp->z_size) { + + Status = zfs_freesp(zp, + feofi->EndOfFile.QuadPart, + 0, 0, TRUE); + changed = 1; + } + dprintf("%s: AdvanceOnly\n", __func__); + goto out; + } + // Truncation? + if (zp->z_size > feofi->EndOfFile.QuadPart) { + // Are we able to truncate? + if (FileObject->SectionObjectPointer && + !MmCanFileBeTruncated( + FileObject->SectionObjectPointer, + &feofi->EndOfFile)) { + Status = STATUS_USER_MAPPED_FILE; + goto out; + } + dprintf("%s: CanTruncate\n", __func__); + } + + // Set new size + Status = zfs_freesp(zp, feofi->EndOfFile.QuadPart, + 0, 0, TRUE); // Len = 0 is truncate + changed = 1; + } + +out: + + if (NT_SUCCESS(Status) && changed) { + + dprintf("%s: new size 0x%llx set\n", __func__, zp->z_size); + + // zfs_freesp() calls vnode_paget_setsize(), but we need + // xto update it here. + if (FileObject->SectionObjectPointer) + vnode_pager_setsize(FileObject, vp, zp->z_size, FALSE); + + // No notify for XATTR/Stream for now + if (!(zp->z_pflags & ZFS_XATTR)) { + zfs_send_notify(zfsvfs, zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_SIZE, + FILE_ACTION_MODIFIED); + } + } + + if (CacheMapInitialized) { + CcUninitializeCacheMap(FileObject, NULL, NULL); + } + + // We handled setsize in here. + vnode_setsizechange(vp, 0); + + VN_RELE(vp); + zfs_exit(zfsvfs, FTAG); + return (Status); +} + +// create hardlink by calling zfs_create +NTSTATUS +set_file_link_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + /* + * typedef struct _FILE_LINK_INFORMATION { + * BOOLEAN ReplaceIfExists; + * HANDLE RootDirectory; + * ULONG FileNameLength; + * WCHAR FileName[1]; + * } FILE_LINK_INFORMATION, *PFILE_LINK_INFORMATION; + */ + + FILE_LINK_INFORMATION *link = Irp->AssociatedIrp.SystemBuffer; + dprintf("* FileLinkInformation: %.*S\n", + (int)link->FileNameLength / sizeof (WCHAR), link->FileName); + + // So, use FileObject to get VP. + // Use VP to lookup parent. + // Use Filename to find destonation dvp, and vp if it exists. + if (IrpSp->FileObject == NULL || IrpSp->FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + mount_t *zmo = DeviceObject->DeviceExtension; + + zfsvfs_t *zfsvfs = NULL; + if (zmo != NULL && + (zfsvfs = vfs_fsprivate(zmo)) != NULL && + zfsvfs->z_rdonly) + return (STATUS_MEDIA_WRITE_PROTECTED); + + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + FILE_OBJECT *RootFileObject = NULL; + PFILE_OBJECT FileObject = IrpSp->FileObject; + struct vnode *fvp = FileObject->FsContext; + znode_t *zp = VTOZ(fvp); + znode_t *dzp = NULL; + int error; + ULONG outlen; + char *remainder = NULL; + char buffer[MAXNAMELEN], *filename; + struct vnode *tdvp = NULL, *tvp = NULL, *fdvp = NULL; + uint64_t parent = 0; + + // If given a RootDirectory Handle, lookup tdvp + if (link->RootDirectory != 0) { + if (ObReferenceObjectByHandle(link->RootDirectory, + GENERIC_READ, *IoFileObjectType, KernelMode, + &RootFileObject, NULL) != STATUS_SUCCESS) { + return (STATUS_INVALID_PARAMETER); + } + tdvp = RootFileObject->FsContext; + VN_HOLD(tdvp); + } else { + // Name can be absolute, if so use name, otherwise, + // use vp's parent. + } + + // Convert incoming filename to utf8 + error = RtlUnicodeToUTF8N(buffer, MAXNAMELEN, &outlen, + link->FileName, link->FileNameLength); + + if (error != STATUS_SUCCESS && + error != STATUS_SOME_NOT_MAPPED) { + if (tdvp) VN_RELE(tdvp); + if (RootFileObject) ObDereferenceObject(RootFileObject); + return (STATUS_ILLEGAL_CHARACTER); + } + + // Output string is only null terminated if input is, so do so now. + buffer[outlen] = 0; + filename = buffer; + + // Filename is often "\??\E:\name" so we want to eat everything + // up to the "\name" + if ((filename[0] == '\\') && + (filename[1] == '?') && + (filename[2] == '?') && + (filename[3] == '\\') && + /* [4] drive letter */ + (filename[5] == ':') && + (filename[6] == '\\')) + filename = &filename[6]; + + error = zfs_find_dvp_vp(zfsvfs, filename, 1, 0, &remainder, &tdvp, + &tvp, 0, 0); + if (error) { + if (tdvp) VN_RELE(tdvp); + if (RootFileObject) ObDereferenceObject(RootFileObject); + return (STATUS_OBJECTID_NOT_FOUND); + } + + // Fetch parent + VERIFY(sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)) == 0); + + // Fetch fdvp + error = zfs_zget(zfsvfs, parent, &dzp); + if (error) { + error = STATUS_OBJECTID_NOT_FOUND; + goto out; + } + + // Lookup name + if (zp->z_name_cache == NULL) { + error = STATUS_OBJECTID_NOT_FOUND; + goto out; + } + + fdvp = ZTOV(dzp); + VN_HOLD(fvp); + // "tvp"(if not NULL) and "tdvp" is held by zfs_find_dvp_vp + + // What about link->ReplaceIfExist ? + + error = zfs_link(VTOZ(tdvp), VTOZ(fvp), + remainder ? remainder : filename, NULL, 0); + + if (error == 0) { + + // FIXME, zget to get name? +#if 0 + // Release fromname, and lookup new name + kmem_free(zp->z_name_cache, zp->z_name_len); + zp->z_name_cache = NULL; + if (zfs_build_path(zp, VTOZ(tdvp), &zp->z_name_cache, + &zp->z_name_len, &zp->z_name_offset) == 0) { + zfs_send_notify(zfsvfs, zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_CREATION, + FILE_ACTION_ADDED); + } +#endif + } + // Release all holds +out: + if (RootFileObject) ObDereferenceObject(RootFileObject); + if (tdvp) VN_RELE(tdvp); + if (fdvp) VN_RELE(fdvp); + if (fvp) VN_RELE(fvp); + if (tvp) VN_RELE(tvp); + + return (error); +} + +NTSTATUS +set_file_rename_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status; + boolean_t ExVariant = + IrpSp->Parameters.SetFile.FileInformationClass == + FileRenameInformationEx; +/* + * The file name string in the FileName member must be specified in + * one of the following forms. + * A simple file name. (The RootDirectory member is NULL.) In this case, + * the file is simply renamed within the same directory. + * That is, the rename operation changes the name of the file but not its + * location. + * + * A fully qualified file name. (The RootDirectory member is NULL.) + * In this case, the rename operation changes the name and location + * of the file. + * + * A relative file name. In this case, the RootDirectory member contains + * a handle to the target directory for the rename operation. The file + * name itself must be a simple file name. + * + * NOTE: The RootDirectory handle thing never happens, and no sample + * source (including fastfat) handles it. + */ + + FILE_RENAME_INFORMATION *ren = Irp->AssociatedIrp.SystemBuffer; + dprintf("* FileRenameInformation: %.*S\n", + (int)ren->FileNameLength / sizeof (WCHAR), ren->FileName); + + // ASSERT(ren->RootDirectory == NULL); + + // So, use FileObject to get VP. + // Use VP to lookup parent. + // Use Filename to find destonation dvp, and vp if it exists. + if (IrpSp->FileObject == NULL || IrpSp->FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + mount_t *zmo = DeviceObject->DeviceExtension; + + zfsvfs_t *zfsvfs = NULL; + if (zmo != NULL && + (zfsvfs = vfs_fsprivate(zmo)) != NULL && + zfsvfs->z_rdonly) + return (STATUS_MEDIA_WRITE_PROTECTED); + + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + PFILE_OBJECT FileObject = IrpSp->FileObject; + struct vnode *fvp = FileObject->FsContext; + znode_t *zp = VTOZ(fvp); + znode_t *dzp = NULL; + int error; + ULONG outlen; + char *remainder = NULL; + char buffer[MAXNAMELEN], *filename; + struct vnode *tdvp = NULL, *tvp = NULL, *fdvp = NULL; + uint64_t parent = 0; + PFILE_OBJECT dFileObject = NULL; + HANDLE destParentHandle = 0; + int use_fdvp_for_tdvp = 0; + + // Convert incoming filename to utf8 + error = RtlUnicodeToUTF8N(buffer, MAXNAMELEN, &outlen, + ren->FileName, ren->FileNameLength); + + if (error != STATUS_SUCCESS && + error != STATUS_SOME_NOT_MAPPED) { + return (STATUS_ILLEGAL_CHARACTER); + } + SetFlag(FileObject->Flags, FO_FILE_MODIFIED); + // Output string is only null terminated if input is, so do so now. + buffer[outlen] = 0; + filename = buffer; + + // Filename is often "\??\E:\lower\name" - and "/lower" might be + // another dataset so we need to drive a lookup, with + // SL_OPEN_TARGET_DIRECTORY set so we get the parent of where + // we are renaming to. This will give us "tdvp", and + // possibly "tvp" is we are to rename over an item. +#if 0 + if ((filename[0] == '\\') && + (filename[1] == '?') && + (filename[2] == '?') && + (filename[3] == '\\') && + /* [4] drive letter */ + (filename[5] == ':') && + (filename[6] == '\\')) + filename = &filename[6]; +#endif + + // If it starts with "\" drive the lookup, if it is just a name + // like "HEAD", assume tdvp is same as fdvp. + if (filename[0] == '\\') { + OBJECT_ATTRIBUTES oa; + IO_STATUS_BLOCK ioStatus; + UNICODE_STRING uFileName; + // RtlInitEmptyUnicodeString(&uFileName, ren->FileName, + // ren->FileNameLength); // doesn't set length + // Is there really no offical wrapper to do this? + uFileName.Length = uFileName.MaximumLength = + ren->FileNameLength; + uFileName.Buffer = ren->FileName; + + InitializeObjectAttributes(&oa, &uFileName, + OBJ_CASE_INSENSITIVE | OBJ_KERNEL_HANDLE, + NULL, NULL); + + Status = IoCreateFile( + &destParentHandle, + FILE_READ_DATA, + &oa, + &ioStatus, + NULL, + 0, + FILE_SHARE_READ, + FILE_OPEN, + FILE_OPEN_FOR_BACKUP_INTENT, + NULL, + 0, + CreateFileTypeNone, + NULL, + IO_FORCE_ACCESS_CHECK | IO_OPEN_TARGET_DIRECTORY | + IO_NO_PARAMETER_CHECKING); + + if (!NT_SUCCESS(Status)) + return (STATUS_INVALID_PARAMETER); + + // We have the targetdirectoryparent - get FileObject. + Status = ObReferenceObjectByHandle(destParentHandle, + STANDARD_RIGHTS_REQUIRED, + *IoFileObjectType, + KernelMode, + &dFileObject, + NULL); + if (!NT_SUCCESS(Status)) { + ZwClose(destParentHandle); + return (STATUS_INVALID_PARAMETER); + } + + // All exits need to go through "out:" at this point on. + + // Assign tdvp + tdvp = dFileObject->FsContext; + + + // Hold it + VERIFY0(VN_HOLD(tdvp)); + + // Filename is '\??\E:\dir\dir\file' and we only care about + // the last part. + char *r = strrchr(filename, '\\'); + if (r == NULL) + r = strrchr(filename, '/'); + if (r != NULL) { + r++; + filename = r; + } + + error = zfs_find_dvp_vp(zfsvfs, filename, 1, 0, &remainder, + &tdvp, &tvp, 0, 0); + if (error) { + Status = STATUS_OBJECTID_NOT_FOUND; + goto out; + } + } else { + // Name might be just "HEAD" so use fdvp + use_fdvp_for_tdvp = 1; + } + + // Goto out will release this + VN_HOLD(fvp); + + // If we have a "tvp" here, then something exists where we are to rename + if (tvp && !ExVariant && !ren->ReplaceIfExists) { + error = STATUS_OBJECT_NAME_COLLISION; + goto out; + } + if (tvp && ExVariant && !(ren->Flags&FILE_RENAME_REPLACE_IF_EXISTS)) { + error = STATUS_OBJECT_NAME_COLLISION; + goto out; + } + + VERIFY(sa_lookup(zp->z_sa_hdl, SA_ZPL_PARENT(zfsvfs), + &parent, sizeof (parent)) == 0); + + // Fetch fdvp + error = zfs_zget(zfsvfs, parent, &dzp); + if (error) { + error = STATUS_OBJECTID_NOT_FOUND; + goto out; + } + + // Lookup name + if (zp->z_name_cache == NULL) { + error = STATUS_OBJECTID_NOT_FOUND; + goto out; + } + + fdvp = ZTOV(dzp); + // "tvp" (if not NULL) and "tdvp" is held by zfs_find_dvp_vp + + if (use_fdvp_for_tdvp) { + tdvp = fdvp; + VERIFY0(VN_HOLD(tdvp)); + } + + + error = zfs_rename(VTOZ(fdvp), &zp->z_name_cache[zp->z_name_offset], + VTOZ(tdvp), remainder ? remainder : filename, NULL, 0, 0, NULL, + NULL); + + if (error == 0) { + // rename file in same directory: + // sned dir modified, send OLD_NAME, NEW_NAME + // Moving to different volume: + // FILE_ACTION_REMOVED, FILE_ACTION_ADDED + // send CHANGE_LAST_WRITE + znode_t *tdzp = VTOZ(tdvp); + zfs_send_notify(zfsvfs, tdzp->z_name_cache, tdzp->z_name_offset, + FILE_NOTIFY_CHANGE_LAST_WRITE, + FILE_ACTION_MODIFIED); + + zfs_send_notify(zfsvfs, zp->z_name_cache, zp->z_name_offset, + vnode_isdir(fvp) ? + FILE_NOTIFY_CHANGE_DIR_NAME : + FILE_NOTIFY_CHANGE_FILE_NAME, + FILE_ACTION_RENAMED_OLD_NAME); + + // Release fromname, and lookup new name + kmem_free(zp->z_name_cache, zp->z_name_len); + zp->z_name_cache = NULL; + + if (zfs_build_path(zp, tdzp, &zp->z_name_cache, + &zp->z_name_len, &zp->z_name_offset) == 0) { + zfs_send_notify(zfsvfs, zp->z_name_cache, + zp->z_name_offset, + vnode_isdir(fvp) ? + FILE_NOTIFY_CHANGE_DIR_NAME : + FILE_NOTIFY_CHANGE_FILE_NAME, + FILE_ACTION_RENAMED_NEW_NAME); + } + } + // Release all holds + if (error == EBUSY) + error = STATUS_ACCESS_DENIED; +out: + if (destParentHandle != 0) + ZwClose(destParentHandle); + if (dFileObject) + ObDereferenceObject(dFileObject); + if (tdvp) VN_RELE(tdvp); + if (fdvp) VN_RELE(fdvp); + if (fvp) VN_RELE(fvp); + if (tvp) VN_RELE(tvp); + + return (error); +} + +NTSTATUS +set_file_valid_data_length_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status; + FILE_VALID_DATA_LENGTH_INFORMATION *fvdli = + Irp->AssociatedIrp.SystemBuffer; + dprintf("* FileValidDataLengthInformation: \n"); + mount_t *zmo = DeviceObject->DeviceExtension; + int error; + + if (IrpSp->Parameters.SetFile.Length < + sizeof (FILE_VALID_DATA_LENGTH_INFORMATION)) + return (STATUS_INVALID_PARAMETER); + + if (IrpSp->FileObject == NULL || + IrpSp->FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + struct vnode *vp = IrpSp->FileObject->FsContext; + znode_t *zp = VTOZ(vp); + + if (zmo == NULL || zp == NULL) + return (STATUS_INVALID_PARAMETER); + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + if (fvdli->ValidDataLength.QuadPart <= + vp->FileHeader.ValidDataLength.QuadPart || + fvdli->ValidDataLength.QuadPart > + vp->FileHeader.FileSize.QuadPart) { + dprintf("invalid VDL of %I64u (%I64u, file %I64u)\n", + fvdli->ValidDataLength.QuadPart, + vp->FileHeader.ValidDataLength.QuadPart, + vp->FileHeader.FileSize.QuadPart); + Status = STATUS_INVALID_PARAMETER; + goto end; + } + + vp->FileHeader.ValidDataLength = fvdli->ValidDataLength; + vnode_setsizechange(vp, 1); + + zfs_send_notify(zp->z_zfsvfs, zp->z_name_cache, + zp->z_name_offset, + FILE_NOTIFY_CHANGE_SIZE, + FILE_ACTION_MODIFIED); + + Status = STATUS_SUCCESS; + +end: + zfs_exit(zfsvfs, FTAG); + + return (Status); +} + +NTSTATUS +set_file_position_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + FILE_POSITION_INFORMATION *fpi = Irp->AssociatedIrp.SystemBuffer; + dprintf("* FilePositionInformation: \n"); + + if (IrpSp->FileObject == NULL || IrpSp->FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + if (IrpSp->Parameters.SetFile.Length < + sizeof (FILE_POSITION_INFORMATION)) + return (STATUS_INVALID_PARAMETER); + + IrpSp->FileObject->CurrentByteOffset = fpi->CurrentByteOffset; + return (STATUS_SUCCESS); +} + +/* IRP_MJ_QUERY_INFORMATION helpers */ +ULONG +get_reparse_tag(znode_t *zp) +{ + if (!(zp->z_pflags & ZFS_REPARSE)) + return (0); + + if (zfsctl_is_node(zp)) + return (zfsctl_get_reparse_tag(zp)); + + int err; + REPARSE_DATA_BUFFER tagdata; + struct iovec iov; + iov.iov_base = (void *)&tagdata; + iov.iov_len = sizeof (tagdata); + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, 0, UIO_SYSSPACE, + sizeof (tagdata), 0); + err = zfs_readlink(ZTOV(zp), &uio, NULL); + return (tagdata.ReparseTag); +} + +NTSTATUS +file_attribute_tag_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_ATTRIBUTE_TAG_INFORMATION *tag) +{ + dprintf(" %s\n", __func__); + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_ATTRIBUTE_TAG_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_ATTRIBUTE_TAG_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + tag->FileAttributes = zfs_getwinflags(zp); + tag->ReparseTag = get_reparse_tag(zp); + Irp->IoStatus.Information = + sizeof (FILE_ATTRIBUTE_TAG_INFORMATION); + ASSERT(tag->FileAttributes != 0); + return (STATUS_SUCCESS); + } + return (STATUS_INVALID_PARAMETER); +} + +NTSTATUS +file_internal_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_INTERNAL_INFORMATION *infernal) +{ + dprintf(" %s\n", __func__); + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_INTERNAL_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_INTERNAL_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + zfs_dirlist_t *zccb = IrpSp->FileObject->FsContext2; + znode_t *zp = VTOZ(vp); + /* For streams, we need to reply with parent file */ + if (zccb && zp->z_pflags & ZFS_XATTR) + infernal->IndexNumber.QuadPart = zccb->real_file_id; + else + infernal->IndexNumber.QuadPart = zp->z_id; + Irp->IoStatus.Information = sizeof (FILE_INTERNAL_INFORMATION); + return (STATUS_SUCCESS); + } + + return (STATUS_NO_SUCH_FILE); +} + +NTSTATUS +file_basic_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_BASIC_INFORMATION *basic) +{ + dprintf(" %s\n", __func__); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_BASIC_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_BASIC_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + if (VN_HOLD(vp) == 0) { + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + if (zp->z_is_sa) { + sa_bulk_attr_t bulk[3]; + int count = 0; + uint64_t mtime[2]; + uint64_t ctime[2]; + uint64_t crtime[2]; + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_MTIME(zfsvfs), + NULL, &mtime, 16); + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_CTIME(zfsvfs), + NULL, &ctime, 16); + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_CRTIME(zfsvfs), + NULL, &crtime, 16); + sa_bulk_lookup(zp->z_sa_hdl, bulk, count); + + TIME_UNIX_TO_WINDOWS(mtime, + basic->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + basic->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + basic->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(zp->z_atime, + basic->LastAccessTime.QuadPart); + } + // FileAttributes == 0 means don't set + // - undocumented, but seen in fastfat + // if (basic->FileAttributes != 0) + basic->FileAttributes = zfs_getwinflags(zp); + + VN_RELE(vp); + } + Irp->IoStatus.Information = sizeof (FILE_BASIC_INFORMATION); + return (STATUS_SUCCESS); + } + + // This can be called from diskDispatcher, referring to the volume. + // if so, make something up. Is this the right thing to do? + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext == NULL) { + mount_t *zmo = DeviceObject->DeviceExtension; + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + + LARGE_INTEGER JanOne1980 = { 0xe1d58000, 0x01a8e79f }; + ExLocalTimeToSystemTime(&JanOne1980, + &basic->LastWriteTime); + basic->CreationTime = basic->LastAccessTime = + basic->LastWriteTime; + basic->FileAttributes = FILE_ATTRIBUTE_DIRECTORY; + if (zfsvfs->z_rdonly) + basic->FileAttributes |= FILE_ATTRIBUTE_READONLY; + Irp->IoStatus.Information = sizeof (FILE_BASIC_INFORMATION); + return (STATUS_SUCCESS); + } + + dprintf(" %s failing\n", __func__); + return (STATUS_OBJECT_NAME_NOT_FOUND); +} +NTSTATUS +file_compression_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_COMPRESSION_INFORMATION *fci) +{ + dprintf(" %s\n", __func__); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_COMPRESSION_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_COMPRESSION_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + if (VN_HOLD(vp) == 0) { + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + memset(fci, 0, sizeof (FILE_COMPRESSION_INFORMATION)); + + // Deal with ads here, and send adsdata.length + if (vnode_isdir(vp)) + fci->CompressedFileSize.QuadPart = zp->z_size; + + VN_RELE(vp); + } + Irp->IoStatus.Information = + sizeof (FILE_COMPRESSION_INFORMATION); + return (STATUS_SUCCESS); + } + + return (STATUS_INVALID_PARAMETER); +} + +uint64_t +zfs_blksz(znode_t *zp) +{ + if (zp->z_blksz) + return (zp->z_blksz); + if (zp->z_sa_hdl) { + uint32_t blksize; + uint64_t nblks; + sa_object_size(zp->z_sa_hdl, &blksize, &nblks); + if (blksize) + return ((uint64_t)blksize); + } + + if (zp->z_zfsvfs->z_max_blksz) + return (zp->z_zfsvfs->z_max_blksz); + return (512ULL); +} + +NTSTATUS +file_standard_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_STANDARD_INFORMATION *standard) +{ + dprintf(" %s\n", __func__); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_STANDARD_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_STANDARD_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + standard->Directory = TRUE; + standard->AllocationSize.QuadPart = 512; + standard->EndOfFile.QuadPart = 512; + standard->DeletePending = FALSE; + standard->NumberOfLinks = 1; + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + zfs_dirlist_t *zccb = IrpSp->FileObject->FsContext2; + VN_HOLD(vp); + znode_t *zp = VTOZ(vp); + standard->Directory = vnode_isdir(vp) ? TRUE : FALSE; + // sa_object_size(zp->z_sa_hdl, &blksize, &nblks); + uint64_t blk = zfs_blksz(zp); + // space taken on disk, multiples of block size + + standard->AllocationSize.QuadPart = allocationsize(zp); + standard->EndOfFile.QuadPart = vnode_isdir(vp) ? 0 : zp->z_size; + standard->NumberOfLinks = zp->z_links; + standard->DeletePending = zccb && + zccb->deleteonclose ? TRUE : FALSE; + Irp->IoStatus.Information = sizeof (FILE_STANDARD_INFORMATION); + +#ifndef FILE_STANDARD_INFORMATION_EX + typedef struct _FILE_STANDARD_INFORMATION_EX { + LARGE_INTEGER AllocationSize; + LARGE_INTEGER EndOfFile; + ULONG NumberOfLinks; + BOOLEAN DeletePending; + BOOLEAN Directory; + BOOLEAN AlternateStream; + BOOLEAN MetadataAttribute; + } FILE_STANDARD_INFORMATION_EX, *PFILE_STANDARD_INFORMATION_EX; +#endif + if (IrpSp->Parameters.QueryFile.Length >= + sizeof (FILE_STANDARD_INFORMATION_EX)) { + FILE_STANDARD_INFORMATION_EX *estandard; + estandard = (FILE_STANDARD_INFORMATION_EX *)standard; + estandard->AlternateStream = zp->z_pflags & ZFS_XATTR; + estandard->MetadataAttribute = FALSE; + Irp->IoStatus.Information = + sizeof (FILE_STANDARD_INFORMATION_EX); + } + + VN_RELE(vp); + dprintf("Returning size %llu and allocsize %llu\n", + standard->EndOfFile.QuadPart, + standard->AllocationSize.QuadPart); + + return (STATUS_SUCCESS); + } + return (STATUS_OBJECT_NAME_NOT_FOUND); +} + +NTSTATUS +file_position_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_POSITION_INFORMATION *position) +{ + dprintf(" %s\n", __func__); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_POSITION_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_POSITION_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + if (IrpSp->FileObject) + position->CurrentByteOffset.QuadPart = + IrpSp->FileObject->CurrentByteOffset.QuadPart; + + Irp->IoStatus.Information = sizeof (FILE_POSITION_INFORMATION); + return (STATUS_SUCCESS); +} + +NTSTATUS +file_ea_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_EA_INFORMATION *ea) +{ + NTSTATUS Status = STATUS_INVALID_PARAMETER; + + dprintf(" %s\n", __func__); + if (IrpSp->Parameters.QueryFile.Length < sizeof (FILE_EA_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_EA_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + ea->EaSize = 0; + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + + ea->EaSize = xattr_getsize(vp); + + dprintf("%s: returning size %d / 0x%x\n", __func__, + ea->EaSize, ea->EaSize); + + Irp->IoStatus.Information = sizeof (FILE_EA_INFORMATION); + return (STATUS_SUCCESS); + } + + return (STATUS_INVALID_PARAMETER); +} + +NTSTATUS +file_alignment_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_ALIGNMENT_INFORMATION *fai) +{ + dprintf(" %s\n", __func__); + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_ALIGNMENT_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_ALIGNMENT_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + fai->AlignmentRequirement = 0; /* FILE_WORD_ALIGNMENT; */ + return (STATUS_SUCCESS); +} + +NTSTATUS +file_network_open_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_NETWORK_OPEN_INFORMATION *netopen) +{ + dprintf(" %s\n", __func__); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_NETWORK_OPEN_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_NETWORK_OPEN_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + if (zp->z_is_sa) { + sa_bulk_attr_t bulk[3]; + int count = 0; + uint64_t mtime[2]; + uint64_t ctime[2]; + uint64_t crtime[2]; + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_MTIME(zfsvfs), NULL, + &mtime, 16); + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_CTIME(zfsvfs), NULL, + &ctime, 16); + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_CRTIME(zfsvfs), NULL, + &crtime, 16); + sa_bulk_lookup(zp->z_sa_hdl, bulk, count); + + TIME_UNIX_TO_WINDOWS(mtime, + netopen->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + netopen->ChangeTime.QuadPart); + TIME_UNIX_TO_WINDOWS(crtime, + netopen->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(zp->z_atime, + netopen->LastAccessTime.QuadPart); + } + netopen->AllocationSize.QuadPart = + P2ROUNDUP(zp->z_size, zfs_blksz(zp)); + netopen->EndOfFile.QuadPart = vnode_isdir(vp) ? 0 : zp->z_size; + netopen->FileAttributes = zfs_getwinflags(zp); + Irp->IoStatus.Information = + sizeof (FILE_NETWORK_OPEN_INFORMATION); + return (STATUS_SUCCESS); + } + + return (STATUS_OBJECT_PATH_NOT_FOUND); +} + +NTSTATUS +file_standard_link_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_STANDARD_LINK_INFORMATION *fsli) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + + dprintf(" %s\n", __func__); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_STANDARD_LINK_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_STANDARD_LINK_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + if (IrpSp->FileObject && IrpSp->FileObject->FsContext) { + struct vnode *vp = IrpSp->FileObject->FsContext; + zfs_dirlist_t *zccb = IrpSp->FileObject->FsContext2; + + znode_t *zp = VTOZ(vp); + + fsli->NumberOfAccessibleLinks = zp->z_links; + fsli->TotalNumberOfLinks = zp->z_links; + fsli->DeletePending = zccb && + zccb->deleteonclose ? TRUE : FALSE; + fsli->Directory = S_ISDIR(zp->z_mode); + } + + Irp->IoStatus.Information = sizeof (FILE_STANDARD_LINK_INFORMATION); + return (STATUS_SUCCESS); +} + +NTSTATUS +file_id_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_ID_INFORMATION *fii) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + + dprintf(" %s\n", __func__); + if (IrpSp->Parameters.QueryFile.Length < sizeof (FILE_ID_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_ID_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + struct vnode *vp = FileObject->FsContext; + + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + fii->VolumeSerialNumber = 0x19831116; + + RtlCopyMemory(&fii->FileId.Identifier[0], &zp->z_id, sizeof (UINT64)); + uint64_t guid = dmu_objset_fsid_guid(zfsvfs->z_os); + RtlCopyMemory(&fii->FileId.Identifier[sizeof (UINT64)], + &guid, sizeof (UINT64)); + + Irp->IoStatus.Information = sizeof (FILE_ID_INFORMATION); + return (STATUS_SUCCESS); +} + +NTSTATUS +file_case_sensitive_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_CASE_SENSITIVE_INFORMATION *fcsi) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + + dprintf(" %s\n", __func__); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_CASE_SENSITIVE_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_CASE_SENSITIVE_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + fcsi->Flags = 0; + + struct vnode *vp = FileObject->FsContext; + if (vp != NULL) { + znode_t *zp = VTOZ(vp); + if (zp != NULL) { + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + if (zfsvfs->z_case == ZFS_CASE_SENSITIVE) + fcsi->Flags |= FILE_CS_FLAG_CASE_SENSITIVE_DIR; + + } + } + + Irp->IoStatus.Information = sizeof (FILE_CASE_SENSITIVE_INFORMATION); + return (STATUS_SUCCESS); +} + +NTSTATUS +file_stat_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_STAT_INFORMATION *fsi) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + + dprintf(" %s\n", __func__); + + /* vp is already help in query_information */ + struct vnode *vp = FileObject->FsContext; + + if (vp) { + + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + if (zp->z_is_sa) { + sa_bulk_attr_t bulk[3]; + int count = 0; + uint64_t mtime[2]; + uint64_t ctime[2]; + uint64_t crtime[2]; + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_MTIME(zfsvfs), NULL, + &mtime, 16); + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_CTIME(zfsvfs), NULL, + &ctime, 16); + SA_ADD_BULK_ATTR(bulk, count, + SA_ZPL_CRTIME(zfsvfs), NULL, + &crtime, 16); + sa_bulk_lookup(zp->z_sa_hdl, bulk, count); + + TIME_UNIX_TO_WINDOWS(crtime, + fsi->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(zp->z_atime, + fsi->LastAccessTime.QuadPart); + TIME_UNIX_TO_WINDOWS(mtime, + fsi->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + fsi->ChangeTime.QuadPart); + } + fsi->FileId.QuadPart = zp->z_id; + fsi->AllocationSize.QuadPart = + P2ROUNDUP(zp->z_size, zfs_blksz(zp)); + fsi->EndOfFile.QuadPart = zp->z_size; + fsi->FileAttributes = zfs_getwinflags(zp); + fsi->ReparseTag = get_reparse_tag(zp); + fsi->NumberOfLinks = zp->z_links; + fsi->EffectiveAccess = GENERIC_ALL; + } + + return (STATUS_SUCCESS); +} + +// Convert ZFS (Unix) mode to Windows mode. +ULONG +ZMODE2WMODE(mode_t z) +{ + ULONG w = 0; + + if (S_ISDIR(z)) w |= 0x4000; // _S_IFDIR + if (S_ISREG(z)) w |= 0x8000; // _S_IFREG + if (S_ISCHR(z)) w |= 0x2000; // _S_IFCHR + if (S_ISFIFO(z)) w |= 0x1000; // _S_IFIFO + if ((z&S_IRUSR) == S_IRUSR) w |= 0x0100; // _S_IREAD + if ((z&S_IWUSR) == S_IWUSR) w |= 0x0080; // _S_IWRITE + if ((z&S_IXUSR) == S_IXUSR) w |= 0x0040; // _S_IEXEC + // Couldn't find documentation for the following, but + // tested in lx/ubuntu to be correct. + if ((z&S_IRGRP) == S_IRGRP) w |= 0x0020; // + if ((z&S_IWGRP) == S_IWGRP) w |= 0x0010; // + if ((z&S_IXGRP) == S_IXGRP) w |= 0x0008; // + if ((z&S_IROTH) == S_IROTH) w |= 0x0004; // + if ((z&S_IWOTH) == S_IWOTH) w |= 0x0002; // + if ((z&S_IXOTH) == S_IXOTH) w |= 0x0001; // + return (w); +} + +NTSTATUS +file_stat_lx_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_STAT_LX_INFORMATION *fsli) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + + dprintf(" %s\n", __func__); + + /* vp is already help in query_information */ + struct vnode *vp = FileObject->FsContext; + + if (vp) { + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + if (zp->z_is_sa) { + sa_bulk_attr_t bulk[3]; + int count = 0; + uint64_t mtime[2]; + uint64_t ctime[2]; + uint64_t crtime[2]; + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MTIME(zfsvfs), + NULL, &mtime, 16); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), + NULL, &ctime, 16); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CRTIME(zfsvfs), + NULL, &crtime, 16); + sa_bulk_lookup(zp->z_sa_hdl, bulk, count); + + TIME_UNIX_TO_WINDOWS(crtime, + fsli->CreationTime.QuadPart); + TIME_UNIX_TO_WINDOWS(zp->z_atime, + fsli->LastAccessTime.QuadPart); + TIME_UNIX_TO_WINDOWS(mtime, + fsli->LastWriteTime.QuadPart); + TIME_UNIX_TO_WINDOWS(ctime, + fsli->ChangeTime.QuadPart); + } + fsli->FileId.QuadPart = zp->z_id; + fsli->AllocationSize.QuadPart = + P2ROUNDUP(zp->z_size, zfs_blksz(zp)); + fsli->EndOfFile.QuadPart = zp->z_size; + fsli->FileAttributes = zfs_getwinflags(zp); + fsli->ReparseTag = get_reparse_tag(zp); + fsli->NumberOfLinks = zp->z_links; + fsli->EffectiveAccess = + SPECIFIC_RIGHTS_ALL | ACCESS_SYSTEM_SECURITY; + fsli->LxFlags = LX_FILE_METADATA_HAS_UID | + LX_FILE_METADATA_HAS_GID | LX_FILE_METADATA_HAS_MODE; + if (zfsvfs->z_case == ZFS_CASE_SENSITIVE) + fsli->LxFlags |= LX_FILE_CASE_SENSITIVE_DIR; + fsli->LxUid = zp->z_uid; + fsli->LxGid = zp->z_gid; + fsli->LxMode = ZMODE2WMODE(zp->z_mode); + fsli->LxDeviceIdMajor = 0; + fsli->LxDeviceIdMinor = 0; + } + return (STATUS_SUCCESS); +} + +// +// If overflow, set Information to input_size and NameLength to required size. +// +NTSTATUS +file_name_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_NAME_INFORMATION *name, + PULONG usedspace, int normalize) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + + dprintf("* %s: (normalize %d)\n", __func__, normalize); + + if (FileObject == NULL || FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + if (IrpSp->Parameters.QueryFile.Length < + (ULONG)FIELD_OFFSET(FILE_NAME_INFORMATION, FileName[0])) { + Irp->IoStatus.Information = sizeof (FILE_NAME_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + struct vnode *vp = FileObject->FsContext; + znode_t *zp = VTOZ(vp); + char strname[MAXPATHLEN + 2]; + int error = 0; + uint64_t parent = 0; + + ASSERT(zp != NULL); + + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + NTSTATUS Status = STATUS_SUCCESS; + + VN_HOLD(vp); + + if (zp->z_id == zfsvfs->z_root) { + strlcpy(strname, "\\", MAXPATHLEN); + } else { + + // Should never be unset! + if (zp->z_name_cache == NULL) { + dprintf("%s: name not set path taken\n", __func__); + if (zfs_build_path(zp, NULL, &zp->z_name_cache, + &zp->z_name_len, &zp->z_name_offset) == -1) { + dprintf("%s: failed to build fullpath\n", + __func__); + return (STATUS_OBJECT_PATH_NOT_FOUND); + } + } + + // Safety + if (zp->z_name_cache != NULL) { +#if 0 + // Just name + strlcpy(strname, &zp->z_name_cache[zp->z_name_offset], + MAXPATHLEN); +#else + // Full path name + strlcpy(strname, zp->z_name_cache, + MAXPATHLEN); +#endif + // If it is a DIR, make sure it ends with "\", + // except for root, that is just "\" + if (S_ISDIR(zp->z_mode)) + strlcat(strname, "\\", + MAXPATHLEN); + } + } + VN_RELE(vp); + + // Convert name, setting FileNameLength to how much we need + error = RtlUTF8ToUnicodeN(NULL, 0, &name->FileNameLength, + strname, strlen(strname)); + + dprintf("%s: remaining space %d str.len %d struct size %d\n", + __func__, IrpSp->Parameters.QueryFile.Length, + name->FileNameLength, sizeof (FILE_NAME_INFORMATION)); + // CHECK ERROR here. + // Calculate how much room there is for filename, after + // the struct and its first wchar + int space = IrpSp->Parameters.QueryFile.Length - + FIELD_OFFSET(FILE_NAME_INFORMATION, FileName); + space = MIN(space, name->FileNameLength); + + ASSERT(space >= 0); + + // Copy over as much as we can, including the first wchar + error = RtlUTF8ToUnicodeN(name->FileName, + space /* + sizeof (name->FileName) */, + NULL, strname, strlen(strname)); + + if (space < name->FileNameLength) + Status = STATUS_BUFFER_OVERFLOW; + else + Status = STATUS_SUCCESS; + + // name->FileNameLength holds how much is actually there + // and usedspace how much we needed to have + + // Would this go one byte too far? + // name->FileName[name->FileNameLength / sizeof (name->FileName)] = 0; + + // Return how much of the filename we copied after the first wchar + // which is used with sizeof (struct) to work out how much + // bigger the return is. + if (usedspace) *usedspace = space; + // Space will always be 2 or more, since struct has room for 1 wchar + + dprintf("* %s: %s name of '%.*S' struct size 0x%x and " + "FileNameLength 0x%x Usedspace 0x%x\n", __func__, + Status == STATUS_BUFFER_OVERFLOW ? "partial" : "", + space / 2, name->FileName, + sizeof (FILE_NAME_INFORMATION), name->FileNameLength, space); + + return (Status); +} + +// This function is not used - left in as example. If you think +// something is not working due to missing FileRemoteProtocolInformation +// then think again. This is not the problem. +NTSTATUS +file_remote_protocol_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_REMOTE_PROTOCOL_INFORMATION *frpi) +{ + dprintf(" %s\n", __func__); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_REMOTE_PROTOCOL_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (FILE_REMOTE_PROTOCOL_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + frpi->StructureVersion = 4; + frpi->StructureSize = sizeof (FILE_REMOTE_PROTOCOL_INFORMATION); + frpi->Protocol = WNNC_NET_GOOGLE; + frpi->ProtocolMajorVersion = 1; + frpi->ProtocolMinorVersion = 0; + frpi->ProtocolRevision = 3; + frpi->Flags = REMOTE_PROTOCOL_FLAG_LOOPBACK; + Irp->IoStatus.Information = sizeof (FILE_REMOTE_PROTOCOL_INFORMATION); + return (STATUS_SUCCESS); +} + +// Insert a streamname into an output buffer, if there is room, +// StreamNameLength is always the FULL name length, even when we only +// fit partial. +// Return 0 for OK, 1 for overflow. +// ADS are returned as ":Zone.Identifier:$DATA". +// EAs are returned as "Zone.Identifier". +// OK, this should only return Streams, but keeping +// the EA code around in case.. +int +zfswin_insert_streamname(char *streamname, uint8_t *outbuffer, + FILE_STREAM_INFORMATION **previous_stream, uint64_t availablebytes, + uint64_t *spaceused, uint64_t streamsize) +{ + /* + * typedef struct _FILE_STREAM_INFO { + * DWORD NextEntryOffset; + * DWORD StreamNameLength; + * LARGE_INTEGER StreamSize; + * LARGE_INTEGER StreamAllocationSize; + * WCHAR StreamName[1]; + * } FILE_STREAM_INFO, *PFILE_STREAM_INFO; + */ + // The first stream struct we assume is already aligned, + // but further ones should be padded here. + FILE_STREAM_INFORMATION *stream = NULL; + int overflow = 0; + boolean_t isADS = B_FALSE; + + int len = strlen(streamname); + if ((toupper(streamname[len - 6]) == ':') && + (toupper(streamname[len - 5]) == '$') && + (toupper(streamname[len - 4]) == 'D') && + (toupper(streamname[len - 3]) == 'A') && + (toupper(streamname[len - 2]) == 'T') && + (toupper(streamname[len - 1]) == 'A')) + isADS = B_TRUE; + + // If not first struct, align outsize to 8 byte - 0 aligns to 0. + *spaceused = (((*spaceused) + 7) & ~7); + + // Convert filename, to get space required. + ULONG needed_streamnamelen; + int error; + + // Check error? Do we care about convertion errors? + error = RtlUTF8ToUnicodeN(NULL, 0, &needed_streamnamelen, + streamname, strlen(streamname)); + + // Is there room? We have to add the struct if there is room for it + // and fill it out as much as possible, and copy in as much of the name + // as we can. + + if (*spaceused + sizeof (FILE_STREAM_INFORMATION) <= availablebytes) { + + stream = (FILE_STREAM_INFORMATION *)&outbuffer[*spaceused]; + + // Room for one more struct, update previous' next ptr + if (*previous_stream != NULL) { + // Update previous structure to point to this one. + // It is not offset from the buffer, but offset from + // last "stream" struct. + // **lastNextEntryOffset = (DWORD)*spaceused; + (*previous_stream)->NextEntryOffset = + (uintptr_t)stream - (uintptr_t)(*previous_stream); + } + + + // Directly set next to 0, assuming this will be last record + stream->NextEntryOffset = 0; + + // remember this struct's NextEntry, so the next one can + // fill it in. + *previous_stream = stream; + + // Set all the fields now + stream->StreamSize.QuadPart = streamsize; + stream->StreamAllocationSize.QuadPart = + P2ROUNDUP(streamsize, 512); + + // Return the total name length, "needed" is in bytes, + // so add 2 to fit the ":" + stream->StreamNameLength = + needed_streamnamelen; + if (isADS) // + ":" + stream->StreamNameLength += sizeof (WCHAR); + + // Consume the space of the struct + *spaceused += FIELD_OFFSET(FILE_STREAM_INFORMATION, StreamName); + + uint64_t roomforname; + if (*spaceused + stream->StreamNameLength <= availablebytes) { + roomforname = stream->StreamNameLength; + } else { + roomforname = availablebytes - *spaceused; + overflow = 1; + } + + // Consume the space of (partial?) filename + *spaceused += roomforname; + + // Now copy out as much of the filename as can fit. + // We need to real full length in StreamNameLength + // There is always room for 1 char + PWSTR out = &stream->StreamName[0]; + + if (isADS) { + *out = L':'; + out++; + roomforname -= sizeof (WCHAR); + } + + // Convert as much as we can, accounting for the start ":" + error = RtlUTF8ToUnicodeN(out, roomforname, + NULL, streamname, strlen(streamname)); + + dprintf("%s: added %s streamname '%s'\n", __func__, + overflow ? "(partial)" : "", streamname); + } else { + dprintf("%s: no room for '%s'\n", __func__, streamname); + overflow = 1; + } + + return (overflow); +} + +// +// If overflow, set Information to input_size and NameLength to required size. +// +NTSTATUS +file_stream_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_STREAM_INFORMATION *stream) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + NTSTATUS Status; + void *outbuffer = Irp->AssociatedIrp.SystemBuffer; + uint64_t availablebytes = IrpSp->Parameters.QueryFile.Length; + FILE_STREAM_INFORMATION *previous_stream = NULL; + int overflow = 0; + int error = 0; + + dprintf("%s: \n", __func__); + + if (FileObject == NULL || FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_STREAM_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_STREAM_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + struct vnode *vp = FileObject->FsContext; + zfs_dirlist_t *zccb = FileObject->FsContext2; + znode_t *zp = VTOZ(vp), *xzp = NULL; + znode_t *xdzp = NULL; + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + // This exits when unmounting + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); + + struct vnode *xdvp = NULL; + void *cr = NULL; + uint64_t spaceused = 0; + zap_cursor_t zc; + objset_t *os; + zap_attribute_t za; + + // Iterate the xattrs. + // Windows can call this on a stream zp, in this case, we + // need to go find the real parent, and iterate on that. + if (zccb && zp->z_pflags & ZFS_XATTR) { + + error = zfs_zget(zfsvfs, zccb->real_file_id, &zp); + if (error) + goto out; + } else { + VN_HOLD(vp); + } + + // Add a record for this name, if there is room. Keep a + // count of how much space would need. + // insert_xattrname adds first ":" and ":$DATA" + if (vnode_isdir(vp)) + overflow = zfswin_insert_streamname("", outbuffer, + &previous_stream, availablebytes, &spaceused, + vnode_isdir(vp) ? 0 : zp->z_size); + else + overflow = zfswin_insert_streamname(":$DATA", outbuffer, + &previous_stream, availablebytes, &spaceused, + vnode_isdir(vp) ? 0 : zp->z_size); + + /* Grab the hidden attribute directory vnode. */ + if (zfs_get_xattrdir(zp, &xdzp, cr, 0) != 0) { + goto out; + } + + xdvp = ZTOV(xdzp); + os = zfsvfs->z_os; + + stream = (FILE_STREAM_INFORMATION *)outbuffer; + + for (zap_cursor_init(&zc, os, VTOZ(xdvp)->z_id); + zap_cursor_retrieve(&zc, &za) == 0; zap_cursor_advance(&zc)) { + + if (!xattr_stream(za.za_name)) + continue; /* skip */ + + // We need to lookup the size of the xattr. + int error = zfs_dirlook(VTOZ(xdvp), za.za_name, &xzp, 0, + NULL, NULL); + + overflow += zfswin_insert_streamname(za.za_name, outbuffer, + &previous_stream, availablebytes, &spaceused, + xzp ? xzp->z_size : 0); + + if (error == 0) + zrele(xzp); + + } + + zap_cursor_fini(&zc); + +out: + if (xdvp) { + VN_RELE(xdvp); + } + + zrele(zp); + + zfs_exit(zfsvfs, FTAG); + + if (overflow > 0) + Status = STATUS_BUFFER_OVERFLOW; + else + Status = STATUS_SUCCESS; + + // Set to how space we used. + Irp->IoStatus.Information = spaceused; + + return (Status); +} + +NTSTATUS +file_hard_link_information(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp, FILE_LINKS_INFORMATION *fhli) +{ + PFILE_OBJECT FileObject = IrpSp->FileObject; + NTSTATUS Status; + uint64_t availablebytes = IrpSp->Parameters.QueryFile.Length; + + dprintf("%s: \n", __func__); + + if (FileObject == NULL || FileObject->FsContext == NULL) + return (STATUS_INVALID_PARAMETER); + + if (IrpSp->Parameters.QueryFile.Length < + sizeof (FILE_LINKS_INFORMATION)) { + Irp->IoStatus.Information = sizeof (FILE_LINKS_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + struct vnode *vp = FileObject->FsContext; + zfs_dirlist_t *zccb = FileObject->FsContext2; + + fhli->EntriesReturned = 0; + fhli->BytesNeeded = sizeof (FILE_LINKS_INFORMATION); + + Irp->IoStatus.Information = sizeof (FILE_LINKS_INFORMATION); + + return (STATUS_SUCCESS); +} + +/* IRP_MJ_DEVICE_CONTROL helpers */ + +NTSTATUS +QueryCapabilities(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS Status; + PDEVICE_CAPABILITIES DeviceCapabilities; + DeviceCapabilities = IrpSp->Parameters.DeviceCapabilities.Capabilities; + DeviceCapabilities->SurpriseRemovalOK = TRUE; + DeviceCapabilities->LockSupported = TRUE; + DeviceCapabilities->EjectSupported = TRUE; + DeviceCapabilities->Removable = FALSE; // XX + DeviceCapabilities->DockDevice = FALSE; + DeviceCapabilities->D1Latency = + DeviceCapabilities->D2Latency = + DeviceCapabilities->D3Latency = 0; + DeviceCapabilities->NoDisplayInUI = 0; + Irp->IoStatus.Information = sizeof (DEVICE_CAPABILITIES); + + return (STATUS_SUCCESS); +} + +NTSTATUS +ioctl_get_gpt_attributes(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + mount_t *zmo; + NTSTATUS Status; + VOLUME_GET_GPT_ATTRIBUTES_INFORMATION *vggai; + + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength < + sizeof (VOLUME_GET_GPT_ATTRIBUTES_INFORMATION)) { + Irp->IoStatus.Information = + sizeof (VOLUME_GET_GPT_ATTRIBUTES_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + zmo = (mount_t *)DeviceObject->DeviceExtension; + vggai = (VOLUME_GET_GPT_ATTRIBUTES_INFORMATION *) + Irp->AssociatedIrp.SystemBuffer; + + if (!zmo || + (zmo->type != MOUNT_TYPE_VCB && + zmo->type != MOUNT_TYPE_DCB)) { + return (STATUS_INVALID_PARAMETER); + } + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + Irp->IoStatus.Information = + sizeof (VOLUME_GET_GPT_ATTRIBUTES_INFORMATION); + + if (zfsvfs->z_rdonly) + vggai->GptAttributes = + GPT_BASIC_DATA_ATTRIBUTE_READ_ONLY; + else + vggai->GptAttributes = 0; + + return (STATUS_SUCCESS); +} + +// +// If overflow, set Information to sizeof (MOUNTDEV_NAME), and +// NameLength to required size. +// +NTSTATUS +ioctl_query_device_name(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + // Return name in MOUNTDEV_NAME + PMOUNTDEV_NAME name; + mount_t *zmo; + NTSTATUS Status; + + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength < + sizeof (MOUNTDEV_NAME)) { + Irp->IoStatus.Information = sizeof (MOUNTDEV_NAME); + return (STATUS_BUFFER_TOO_SMALL); + } + + zmo = (mount_t *)DeviceObject->DeviceExtension; + + /* If given a file, it must be root */ + if (IrpSp->FileObject != NULL && IrpSp->FileObject->FsContext != NULL) { + struct vnode *vp = IrpSp->FileObject->FsContext; + if (vp != NULL) { + znode_t *zp = VTOZ(vp); + if (zp != NULL) { + if (zp->z_id != zp->z_zfsvfs->z_root) { + dprintf("%s on file which isn't root\n", + __func__); + return (STATUS_INVALID_PARAMETER); + } + } + } + } + + name = Irp->AssociatedIrp.SystemBuffer; + + int space = IrpSp->Parameters.DeviceIoControl.OutputBufferLength - + sizeof (MOUNTDEV_NAME); +#if 1 + space = MIN(space, zmo->device_name.Length); + name->NameLength = zmo->device_name.Length; + RtlCopyMemory(name->Name, zmo->device_name.Buffer, + space + sizeof (name->Name)); + Irp->IoStatus.Information = sizeof (MOUNTDEV_NAME) + space; + + if (space < zmo->device_name.Length - sizeof (name->Name)) + Status = STATUS_BUFFER_OVERFLOW; + else + Status = STATUS_SUCCESS; +#else + if (zmo->parent_device != NULL) { + DeviceObject = zmo->parent_device; + zmo = (mount_t *)DeviceObject->DeviceExtension; + } + + space = MIN(space, zmo->device_name.Length); + name->NameLength = zmo->device_name.Length; + RtlCopyMemory(name->Name, zmo->device_name.Buffer, + space + sizeof (name->Name)); + Irp->IoStatus.Information = sizeof (MOUNTDEV_NAME) + space; + + if (space < zmo->device_name.Length - sizeof (name->Name)) + Status = STATUS_BUFFER_OVERFLOW; + else + Status = STATUS_SUCCESS; +#endif + + ASSERT(Irp->IoStatus.Information <= + IrpSp->Parameters.DeviceIoControl.OutputBufferLength); + + dprintf("replying with '%.*S'\n", + space + sizeof (name->Name) / sizeof (WCHAR), name->Name); + + return (Status); +} + +NTSTATUS +ioctl_disk_get_drive_geometry(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + int error = 0; + dprintf("%s: \n", __func__); + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength < + sizeof (DISK_GEOMETRY)) { + Irp->IoStatus.Information = sizeof (DISK_GEOMETRY); + return (STATUS_BUFFER_TOO_SMALL); + } + + mount_t *zmo = DeviceObject->DeviceExtension; + if (!zmo || + (zmo->type != MOUNT_TYPE_VCB && + zmo->type != MOUNT_TYPE_DCB)) { + return (STATUS_INVALID_PARAMETER); + } + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); // This returns EIO if fail + + uint64_t refdbytes, availbytes, usedobjs, availobjs; + dmu_objset_space(zfsvfs->z_os, + &refdbytes, &availbytes, &usedobjs, &availobjs); + + DISK_GEOMETRY *geom = Irp->AssociatedIrp.SystemBuffer; + + geom->BytesPerSector = 512; + geom->SectorsPerTrack = 1; + geom->TracksPerCylinder = 1; + geom->Cylinders.QuadPart = (availbytes + refdbytes) / 512; + geom->MediaType = FixedMedia; + zfs_exit(zfsvfs, FTAG); + + Irp->IoStatus.Information = sizeof (DISK_GEOMETRY); + return (STATUS_SUCCESS); +} + +// This is how Windows Samples handle it +typedef struct _DISK_GEOMETRY_EX_INTERNAL { + DISK_GEOMETRY Geometry; + LARGE_INTEGER DiskSize; + DISK_PARTITION_INFO Partition; + DISK_DETECTION_INFO Detection; +} DISK_GEOMETRY_EX_INTERNAL, *PDISK_GEOMETRY_EX_INTERNAL; + +NTSTATUS +ioctl_disk_get_drive_geometry_ex(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + int error = 0; + dprintf("%s: \n", __func__); + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength < + FIELD_OFFSET(DISK_GEOMETRY_EX, Data)) { + Irp->IoStatus.Information = sizeof (DISK_GEOMETRY_EX); + return (STATUS_BUFFER_TOO_SMALL); + } + + mount_t *zmo = DeviceObject->DeviceExtension; + if (!zmo || + (zmo->type != MOUNT_TYPE_VCB && + zmo->type != MOUNT_TYPE_DCB)) { + return (STATUS_INVALID_PARAMETER); + } + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); // This returns EIO if fail + + uint64_t refdbytes, availbytes, usedobjs, availobjs; + dmu_objset_space(zfsvfs->z_os, + &refdbytes, &availbytes, &usedobjs, &availobjs); + + + DISK_GEOMETRY_EX_INTERNAL *geom = Irp->AssociatedIrp.SystemBuffer; + geom->DiskSize.QuadPart = availbytes + refdbytes; + geom->Geometry.BytesPerSector = 512; + geom->Geometry.MediaType = FixedMedia; + + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength >= + FIELD_OFFSET(DISK_GEOMETRY_EX_INTERNAL, Detection)) { + geom->Partition.SizeOfPartitionInfo = sizeof (geom->Partition); + geom->Partition.PartitionStyle = PARTITION_STYLE_GPT; + // geom->Partition.Gpt.DiskId = 0; + } + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength >= + sizeof (DISK_GEOMETRY_EX_INTERNAL)) { + geom->Detection.SizeOfDetectInfo = sizeof (geom->Detection); + } + zfs_exit(zfsvfs, FTAG); + + Irp->IoStatus.Information = + MIN(IrpSp->Parameters.DeviceIoControl.OutputBufferLength, + sizeof (DISK_GEOMETRY_EX_INTERNAL)); + return (STATUS_SUCCESS); +} + +NTSTATUS +ioctl_disk_get_partition_info(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + int error = 0; + dprintf("%s: \n", __func__); + + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength < + sizeof (PARTITION_INFORMATION)) { + Irp->IoStatus.Information = sizeof (PARTITION_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + mount_t *zmo = DeviceObject->DeviceExtension; + if (!zmo || + (zmo->type != MOUNT_TYPE_VCB && + zmo->type != MOUNT_TYPE_DCB)) { + return (STATUS_INVALID_PARAMETER); + } + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); // This returns EIO if fail + + uint64_t refdbytes, availbytes, usedobjs, availobjs; + dmu_objset_space(zfsvfs->z_os, + &refdbytes, &availbytes, &usedobjs, &availobjs); + + PARTITION_INFORMATION *part = Irp->AssociatedIrp.SystemBuffer; + + part->PartitionLength.QuadPart = availbytes + refdbytes; + part->StartingOffset.QuadPart = 0; + part->BootIndicator = FALSE; + part->PartitionNumber = (ULONG)(-1L); + part->HiddenSectors = (ULONG)(1L); + part->RecognizedPartition = TRUE; + part->RewritePartition = FALSE; + part->PartitionType = 'ZFS'; + + zfs_exit(zfsvfs, FTAG); + + Irp->IoStatus.Information = sizeof (PARTITION_INFORMATION); + + return (STATUS_SUCCESS); +} + +NTSTATUS +ioctl_disk_get_partition_info_ex(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + int error = 0; + dprintf("%s: \n", __func__); + + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength < + sizeof (PARTITION_INFORMATION_EX)) { + Irp->IoStatus.Information = sizeof (PARTITION_INFORMATION_EX); + return (STATUS_BUFFER_TOO_SMALL); + } + + mount_t *zmo = DeviceObject->DeviceExtension; + if (!zmo || + (zmo->type != MOUNT_TYPE_VCB && + zmo->type != MOUNT_TYPE_DCB)) { + return (STATUS_INVALID_PARAMETER); + } + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); // This returns EIO if fail + + uint64_t refdbytes, availbytes, usedobjs, availobjs; + dmu_objset_space(zfsvfs->z_os, + &refdbytes, &availbytes, &usedobjs, &availobjs); + + PARTITION_INFORMATION_EX *part = Irp->AssociatedIrp.SystemBuffer; + + part->PartitionStyle = PARTITION_STYLE_MBR; + part->RewritePartition = FALSE; + part->Mbr.RecognizedPartition = FALSE; + part->Mbr.PartitionType = PARTITION_ENTRY_UNUSED; + part->Mbr.BootIndicator = FALSE; + part->Mbr.HiddenSectors = 0; + part->StartingOffset.QuadPart = 0; + part->PartitionLength.QuadPart = availbytes + refdbytes; + part->PartitionNumber = 0; + + zfs_exit(zfsvfs, FTAG); + + Irp->IoStatus.Information = sizeof (PARTITION_INFORMATION_EX); + + return (STATUS_SUCCESS); +} + +NTSTATUS +ioctl_disk_get_length_info(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + int error = 0; + dprintf("%s: \n", __func__); + + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength < + sizeof (GET_LENGTH_INFORMATION)) { + Irp->IoStatus.Information = sizeof (GET_LENGTH_INFORMATION); + return (STATUS_BUFFER_TOO_SMALL); + } + + mount_t *zmo = DeviceObject->DeviceExtension; + if (!zmo || + (zmo->type != MOUNT_TYPE_VCB && + zmo->type != MOUNT_TYPE_DCB)) { + return (STATUS_INVALID_PARAMETER); + } + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + if ((error = zfs_enter(zfsvfs, FTAG)) != 0) + return (error); // This returns EIO if fail + + uint64_t refdbytes, availbytes, usedobjs, availobjs; + dmu_objset_space(zfsvfs->z_os, + &refdbytes, &availbytes, &usedobjs, &availobjs); + + GET_LENGTH_INFORMATION *gli = Irp->AssociatedIrp.SystemBuffer; + gli->Length.QuadPart = availbytes + refdbytes; + + zfs_exit(zfsvfs, FTAG); + + Irp->IoStatus.Information = sizeof (GET_LENGTH_INFORMATION); + + return (STATUS_SUCCESS); +} + +NTSTATUS +ioctl_volume_is_io_capable(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + dprintf("%s: \n", __func__); + return (STATUS_SUCCESS); +} + +NTSTATUS +ioctl_storage_get_hotplug_info(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + dprintf("%s: \n", __func__); + + if (IrpSp->Parameters.DeviceIoControl.OutputBufferLength < + sizeof (STORAGE_HOTPLUG_INFO)) { + Irp->IoStatus.Information = sizeof (STORAGE_HOTPLUG_INFO); + return (STATUS_BUFFER_TOO_SMALL); + } + + STORAGE_HOTPLUG_INFO *hot = Irp->AssociatedIrp.SystemBuffer; + hot->Size = sizeof (STORAGE_HOTPLUG_INFO); + hot->MediaRemovable = FALSE; // XX + hot->DeviceHotplug = TRUE; + hot->MediaHotplug = FALSE; + hot->WriteCacheEnableOverride = FALSE; + + Irp->IoStatus.Information = sizeof (STORAGE_HOTPLUG_INFO); + return (STATUS_SUCCESS); +} + +NTSTATUS +ioctl_storage_query_property(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + NTSTATUS status; + ULONG outputLength; + + dprintf("%s: \n", __func__); + + outputLength = IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + if (outputLength < sizeof (STORAGE_PROPERTY_QUERY)) { + Irp->IoStatus.Information = sizeof (STORAGE_PROPERTY_QUERY); + return (STATUS_BUFFER_TOO_SMALL); + } + + STORAGE_PROPERTY_QUERY *spq = Irp->AssociatedIrp.SystemBuffer; + + switch (spq->QueryType) { + + case PropertyExistsQuery: + + // ExistsQuery: return OK if exists. + Irp->IoStatus.Information = 0; + + switch (spq->PropertyId) { + case StorageDeviceUniqueIdProperty: +dprintf(" PropertyExistsQuery StorageDeviceUniqueIdProperty\n"); + status = STATUS_SUCCESS; + break; + case StorageDeviceWriteCacheProperty: + case StorageAdapterProperty: +dprintf(" PropertyExistsQuery Not implemented 0x%x\n", spq->PropertyId); + status = STATUS_NOT_IMPLEMENTED; + break; + case StorageDeviceAttributesProperty: +dprintf(" PropertyExistsQuery StorageDeviceAttributesProperty\n"); + status = STATUS_SUCCESS; + break; + default: +dprintf(" PropertyExistsQuery unknown 0x%x\n", spq->PropertyId); + status = STATUS_NOT_IMPLEMENTED; + break; + } // switch PropertyId + break; + + // Query property, check input buffer size. + case PropertyStandardQuery: + + switch (spq->PropertyId) { + case StorageDeviceProperty: +dprintf(" PropertyStandardQuery StorageDeviceProperty\n"); + Irp->IoStatus.Information = + sizeof (STORAGE_DEVICE_DESCRIPTOR); + if (outputLength < sizeof (STORAGE_DEVICE_DESCRIPTOR)) { + status = STATUS_BUFFER_TOO_SMALL; + break; + } + PSTORAGE_DEVICE_DESCRIPTOR storage; + storage = Irp->AssociatedIrp.SystemBuffer; + status = STATUS_SUCCESS; + break; + case StorageAdapterProperty: +dprintf(" PropertyStandardQuery Not implemented 0x%x\n", spq->PropertyId); + status = STATUS_NOT_IMPLEMENTED; + break; + case StorageDeviceAttributesProperty: +dprintf(" PropertyStandardQuery StorageDeviceAttributesProperty\n"); + Irp->IoStatus.Information = + sizeof (STORAGE_DEVICE_ATTRIBUTES_DESCRIPTOR); + if (outputLength < + sizeof (STORAGE_DEVICE_ATTRIBUTES_DESCRIPTOR)) { + status = STATUS_BUFFER_TOO_SMALL; + break; + } + STORAGE_DEVICE_ATTRIBUTES_DESCRIPTOR *sdad; + sdad = Irp->AssociatedIrp.SystemBuffer; + sdad->Version = 1; + sdad->Size = + sizeof (STORAGE_DEVICE_ATTRIBUTES_DESCRIPTOR); + sdad->Attributes = + STORAGE_ATTRIBUTE_BYTE_ADDRESSABLE_IO; + status = STATUS_SUCCESS; + break; + default: + dprintf(" PropertyStandardQuery unknown 0x%x\n", + spq->PropertyId); + status = STATUS_NOT_IMPLEMENTED; + break; + } // switch propertyId + break; + + default: + dprintf("%s: unknown Querytype: 0x%x\n", + __func__, spq->QueryType); + status = STATUS_NOT_IMPLEMENTED; + break; + } + + Irp->IoStatus.Information = sizeof (STORAGE_PROPERTY_QUERY); + return (status); +} + +// Query Unique id uses 1 byte chars. +// If overflow, set Information to sizeof (MOUNTDEV_UNIQUE_ID), +// and NameLength to required size. +// +NTSTATUS +ioctl_query_unique_id(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + PMOUNTDEV_UNIQUE_ID uniqueId; + ULONG bufferLength = + IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + mount_t *zmo; + char osname[MAXNAMELEN]; + ULONG len; + + dprintf("%s: \n", __func__); + + zmo = (mount_t *)DeviceObject->DeviceExtension; + + if (bufferLength < sizeof (MOUNTDEV_UNIQUE_ID)) { + Irp->IoStatus.Information = sizeof (MOUNTDEV_UNIQUE_ID); + return (STATUS_BUFFER_TOO_SMALL); + } + + RtlUnicodeToUTF8N(osname, MAXPATHLEN, &len, zmo->name.Buffer, + zmo->name.Length); + osname[len] = 0; + + // uniqueId appears to be CHARS not WCHARS, + // so this might need correcting? + uniqueId = (PMOUNTDEV_UNIQUE_ID)Irp->AssociatedIrp.SystemBuffer; + + uniqueId->UniqueIdLength = strlen(osname); + + if (sizeof (USHORT) + uniqueId->UniqueIdLength <= bufferLength) { + RtlCopyMemory((PCHAR)uniqueId->UniqueId, osname, + uniqueId->UniqueIdLength); + Irp->IoStatus.Information = + FIELD_OFFSET(MOUNTDEV_UNIQUE_ID, UniqueId[0]) + + uniqueId->UniqueIdLength; + dprintf("replying with '%.*s'\n", + uniqueId->UniqueIdLength, uniqueId->UniqueId); + return (STATUS_SUCCESS); + } else { + Irp->IoStatus.Information = sizeof (MOUNTDEV_UNIQUE_ID); + return (STATUS_BUFFER_OVERFLOW); + } +} + +NTSTATUS +ioctl_query_stable_guid(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + PMOUNTDEV_STABLE_GUID mountGuid; + ULONG bufferLength = + IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + mount_t *zmo; + + dprintf("%s: \n", __func__); + + zmo = (mount_t *)DeviceObject->DeviceExtension; + + if (bufferLength < sizeof (MOUNTDEV_STABLE_GUID)) { + Irp->IoStatus.Information = sizeof (MOUNTDEV_STABLE_GUID); + return (STATUS_BUFFER_TOO_SMALL); + } + + mountGuid = (PMOUNTDEV_STABLE_GUID)Irp->AssociatedIrp.SystemBuffer; + RtlZeroMemory(&mountGuid->StableGuid, sizeof (mountGuid->StableGuid)); + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs) { + uint64_t guid = dmu_objset_fsid_guid(zfsvfs->z_os); + RtlCopyMemory(&mountGuid->StableGuid, &guid, sizeof (guid)); + Irp->IoStatus.Information = sizeof (MOUNTDEV_STABLE_GUID); + return (STATUS_SUCCESS); + } + return (STATUS_NOT_FOUND); +} + +NTSTATUS +ioctl_mountdev_query_suggested_link_name(PDEVICE_OBJECT DeviceObject, + PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + MOUNTDEV_SUGGESTED_LINK_NAME *linkName; + ULONG bufferLength = + IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + // UNICODE_STRING MountPoint; + mount_t *zmo = (mount_t *)DeviceObject->DeviceExtension; + + dprintf("%s: \n", __func__); + + if (bufferLength < sizeof (MOUNTDEV_SUGGESTED_LINK_NAME)) { + Irp->IoStatus.Information = + sizeof (MOUNTDEV_SUGGESTED_LINK_NAME); + return (STATUS_BUFFER_TOO_SMALL); + } + + // We only reply to strict driveletter mounts, not paths... + if (!zmo->justDriveLetter) + return (STATUS_NOT_FOUND); + + // If "?:" then just let windows pick drive letter + if (zmo->mountpoint.Buffer[4] == L'?') + return (STATUS_NOT_FOUND); + + // This code works, for driveletters. + // The mountpoint string is "\\??\\f:" so change + // that to DosDevicesF: + + DECLARE_UNICODE_STRING_SIZE(MountPoint, + ZFS_MAX_DATASET_NAME_LEN); // 36(uuid) + 6 (punct) + 6 (Volume) + RtlUnicodeStringPrintf(&MountPoint, L"\\DosDevices\\%wc:", + towupper(zmo->mountpoint.Buffer[4])); // "\??\F:" + + // RtlInitUnicodeString(&MountPoint, L"\\DosDevices\\G:"); + + linkName = + (PMOUNTDEV_SUGGESTED_LINK_NAME)Irp->AssociatedIrp.SystemBuffer; + + linkName->UseOnlyIfThereAreNoOtherLinks = FALSE; + linkName->NameLength = MountPoint.Length; + + if (sizeof (USHORT) + linkName->NameLength <= bufferLength) { + RtlCopyMemory((PCHAR)linkName->Name, MountPoint.Buffer, + linkName->NameLength); + Irp->IoStatus.Information = + FIELD_OFFSET(MOUNTDEV_SUGGESTED_LINK_NAME, Name[0]) + + linkName->NameLength; + dprintf(" LinkName %wZ (%d)\n", MountPoint, MountPoint.Length); + return (STATUS_SUCCESS); + } + + Irp->IoStatus.Information = sizeof (MOUNTDEV_SUGGESTED_LINK_NAME); + return (STATUS_BUFFER_OVERFLOW); + +} + +NTSTATUS +ioctl_mountdev_query_stable_guid(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + MOUNTDEV_STABLE_GUID *guid = Irp->UserBuffer; + ULONG bufferLength = + IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + mount_t *zmo = (mount_t *)DeviceObject->DeviceExtension; + + dprintf("%s: \n", __func__); + + if (bufferLength < sizeof (MOUNTDEV_STABLE_GUID)) { + Irp->IoStatus.Information = sizeof (MOUNTDEV_STABLE_GUID); + return (STATUS_BUFFER_TOO_SMALL); + } + + zfsvfs_t *zfsvfs = vfs_fsprivate(zmo); + if (zfsvfs == NULL) + return (STATUS_INVALID_PARAMETER); + + extern int zfs_vfs_uuid_gen(const char *osname, uuid_t uuid); + + // A bit naughty + zfs_vfs_uuid_gen(spa_name(dmu_objset_spa(zfsvfs->z_os)), + (char *)&guid->StableGuid); + + Irp->IoStatus.Information = sizeof (MOUNTDEV_STABLE_GUID); + return (STATUS_SUCCESS); +} + +NTSTATUS +fsctl_zfs_volume_mountpoint(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + mount_t *zmo = (mount_t *)DeviceObject->DeviceExtension; + + ULONG bufferLength = + IrpSp->Parameters.DeviceIoControl.OutputBufferLength; + + if (bufferLength < sizeof (fsctl_zfs_volume_mountpoint_t) + + zmo->mountpoint.Length) { + Irp->IoStatus.Information = + sizeof (fsctl_zfs_volume_mountpoint_t) + + zmo->mountpoint.Length; + return (STATUS_BUFFER_TOO_SMALL); + } + + fsctl_zfs_volume_mountpoint_t *fzvm = + (fsctl_zfs_volume_mountpoint_t *)Irp->AssociatedIrp.SystemBuffer; + + fzvm->len = zmo->mountpoint.Length; + memcpy(fzvm->buffer, zmo->mountpoint.Buffer, fzvm->len); + Irp->IoStatus.Information = + sizeof (fsctl_zfs_volume_mountpoint_t) + + zmo->mountpoint.Length; + return (STATUS_SUCCESS); +} + +NTSTATUS +fsctl_set_zero_data(PDEVICE_OBJECT DeviceObject, PIRP Irp, + PIO_STACK_LOCATION IrpSp) +{ + FILE_ZERO_DATA_INFORMATION *fzdi = Irp->AssociatedIrp.SystemBuffer; + ULONG length = IrpSp->Parameters.FileSystemControl.InputBufferLength; + PFILE_OBJECT FileObject = IrpSp->FileObject; + NTSTATUS Status; + LARGE_INTEGER time; + uint64_t start, end; + IO_STATUS_BLOCK iosb; + zfs_dirlist_t *zccb; + + if (!fzdi || length < sizeof (FILE_ZERO_DATA_INFORMATION)) + return (STATUS_INVALID_PARAMETER); + + if (!FileObject) + return (STATUS_INVALID_PARAMETER); + + if (fzdi->BeyondFinalZero.QuadPart <= fzdi->FileOffset.QuadPart) { + dprintf("BeyondFinalZero was <= to Offset (%I64x <= %I64x)\n", + fzdi->BeyondFinalZero.QuadPart, fzdi->FileOffset.QuadPart); + return (STATUS_INVALID_PARAMETER); + } + + struct vnode *vp = FileObject->FsContext; + + if (!vp) + return (STATUS_INVALID_PARAMETER); + + zccb = FileObject->FsContext2; + + if (!zccb) + return (STATUS_INVALID_PARAMETER); + +// if (Irp->RequestorMode == UserMode && +// !(ccb->access & FILE_WRITE_DATA)) { +// WARN("insufficient privileges\n"); +// return STATUS_ACCESS_DENIED; +// } + + znode_t *zp = VTOZ(vp); + + // ExAcquireResourceSharedLite(&zmo->tree_lock, true); + ExAcquireResourceExclusiveLite(vp->FileHeader.Resource, TRUE); + + CcFlushCache(FileObject->SectionObjectPointer, NULL, 0, &iosb); + + if (!vnode_isreg(vp)) { + dprintf("FileObject did not point to a file\n"); + Status = STATUS_INVALID_PARAMETER; + goto end; + } + + /* + * btrfs has this test, but MS "test.exe streams" tests that this + * works, so we will leave it in. + */ +#if 0 + if (zp->z_pflags & ZFS_XATTR) { + dprintf("FileObject is stream\n"); + Status = STATUS_INVALID_PARAMETER; + goto end; + } +#endif + + if ((uint64_t)fzdi->FileOffset.QuadPart >= zp->z_size) { + Status = STATUS_SUCCESS; + goto end; + } + + Status = zfs_freesp(zp, fzdi->FileOffset.QuadPart, + fzdi->BeyondFinalZero.QuadPart - fzdi->FileOffset.QuadPart, + O_RDWR, TRUE); + + CcPurgeCacheSection(FileObject->SectionObjectPointer, + &fzdi->FileOffset, + (ULONG)(fzdi->BeyondFinalZero.QuadPart - fzdi->FileOffset.QuadPart), + FALSE); + + Status = STATUS_SUCCESS; + +end: + + ExReleaseResourceLite(vp->FileHeader.Resource); + // ExReleaseResourceLite(&Vcb->tree_lock); + + return (Status); +} diff --git a/module/os/windows/zfs/zfs_vnops_windows_mount.c b/module/os/windows/zfs/zfs_vnops_windows_mount.c new file mode 100644 index 000000000000..071328bdc6f5 --- /dev/null +++ b/module/os/windows/zfs/zfs_vnops_windows_mount.c @@ -0,0 +1,1593 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2019 Jorgen Lundman + */ +#define INITGUID +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include + +#undef _NTDDK_ + +#include +#pragma comment(lib, "wdmsec.lib") + + +extern int getzfsvfs(const char *dsname, zfsvfs_t **zfvp); + +uint64_t zfs_disable_removablemedia = 0; + + +/* + * Jump through the hoops needed to make a mount happen. + * + * Create a new Volume name + * Register a new unknown device + * Assign volume name + * Register device as disk + * fill in disk information + * broadcast information + */ + +NTSTATUS +mountmgr_add_drive_letter(PDEVICE_OBJECT mountmgr, PUNICODE_STRING devpath) +{ + NTSTATUS Status; + ULONG mmdltsize; + MOUNTMGR_DRIVE_LETTER_TARGET* mmdlt; + MOUNTMGR_DRIVE_LETTER_INFORMATION mmdli; + + mmdltsize = offsetof(MOUNTMGR_DRIVE_LETTER_TARGET, DeviceName[0]) + + devpath->Length; + + mmdlt = kmem_alloc(mmdltsize, KM_SLEEP); + + mmdlt->DeviceNameLength = devpath->Length; + RtlCopyMemory(&mmdlt->DeviceName, devpath->Buffer, devpath->Length); + dprintf("mmdlt = %.*S\n", + mmdlt->DeviceNameLength / sizeof (WCHAR), mmdlt->DeviceName); + + Status = dev_ioctl(mountmgr, IOCTL_MOUNTMGR_NEXT_DRIVE_LETTER, + mmdlt, mmdltsize, &mmdli, + sizeof (MOUNTMGR_DRIVE_LETTER_INFORMATION), FALSE, NULL); + + if (!NT_SUCCESS(Status)) + dprintf("IOCTL_MOUNTMGR_NEXT_DRIVE_LETTER returned %08x\n", + Status); + else + dprintf("DriveLetterWasAssigned = %u, " + "CurrentDriveLetter = %c\n", + mmdli.DriveLetterWasAssigned, mmdli.CurrentDriveLetter); + + kmem_free(mmdlt, mmdltsize); + + return (Status); +} + +/* + * check if valid mountpoint, like \DosDevices\X: + */ +BOOLEAN +MOUNTMGR_IS_DRIVE_LETTER_A(char *mountpoint) +{ + UNICODE_STRING wc_mpt; + wchar_t buf[PATH_MAX]; + mbstowcs(buf, mountpoint, sizeof (buf)); + RtlInitUnicodeString(&wc_mpt, buf); + return (MOUNTMGR_IS_DRIVE_LETTER(&wc_mpt)); +} + +/* + * check if valid mountpoint, like \??\Volume{abc} + */ +BOOLEAN +MOUNTMGR_IS_VOLUME_NAME_A(char *mountpoint) +{ + UNICODE_STRING wc_mpt; + wchar_t buf[PATH_MAX]; + mbstowcs(buf, mountpoint, sizeof (buf)); + RtlInitUnicodeString(&wc_mpt, buf); + return (MOUNTMGR_IS_VOLUME_NAME(&wc_mpt)); +} + +/* + * Returns the last mountpoint for the device (devpath) (unfiltered) + * This is either \DosDevices\X: or \??\Volume{abc} in most cases + * If only_driveletter or only_volume_name is set TRUE, + * every mountpoint will be checked with MOUNTMGR_IS_DRIVE_LETTER or + * MOUNTMGR_IS_VOLUME_NAME and discarded if not valid + * only_driveletter and only_volume_name are mutual exclusive + */ +NTSTATUS +mountmgr_get_mountpoint(PDEVICE_OBJECT mountmgr, + PUNICODE_STRING devpath, PUNICODE_STRING savename, BOOLEAN only_driveletter, + BOOLEAN only_volume_name) +{ + MOUNTMGR_MOUNT_POINT point = { 0 }; + MOUNTMGR_MOUNT_POINTS points; + PMOUNTMGR_MOUNT_POINTS ppoints = NULL; + int len; + NTSTATUS Status; + + if (only_driveletter && only_volume_name) + return (STATUS_INVALID_PARAMETER); + + ppoints = &points; + Status = dev_ioctl(mountmgr, IOCTL_MOUNTMGR_QUERY_POINTS, + &point, sizeof (MOUNTMGR_MOUNT_POINT), ppoints, + sizeof (MOUNTMGR_MOUNT_POINTS), FALSE, NULL); + + if (Status == STATUS_BUFFER_OVERFLOW) { + len = points.Size; + ppoints = kmem_alloc(len, KM_SLEEP); + Status = dev_ioctl(mountmgr, IOCTL_MOUNTMGR_QUERY_POINTS, + &point, sizeof (MOUNTMGR_MOUNT_POINT), + ppoints, len, FALSE, NULL); + + } + dprintf("IOCTL_MOUNTMGR_QUERY_POINTS return %x - looking for '%wZ'\n", + Status, devpath); + if (Status == STATUS_SUCCESS) { + for (int Index = 0; + Index < ppoints->NumberOfMountPoints; + Index++) { + PMOUNTMGR_MOUNT_POINT ipoint = + ppoints->MountPoints + Index; + PWCHAR DeviceName = + (PWCHAR)((PUCHAR)ppoints + + ipoint->DeviceNameOffset); + PWCHAR SymbolicLinkName = + (PWCHAR)((PUCHAR)ppoints + + ipoint->SymbolicLinkNameOffset); + + // Why is this hackery needed, we should be able + // to lookup the drive letter from volume name + dprintf(" point %d: '%.*S' '%.*S'\n", Index, + ipoint->DeviceNameLength / sizeof (WCHAR), + DeviceName, + ipoint->SymbolicLinkNameLength / sizeof (WCHAR), + SymbolicLinkName); + if (wcsncmp(DeviceName, devpath->Buffer, + ipoint->DeviceNameLength / sizeof (WCHAR)) == 0) { + + RtlUnicodeStringCbCopyStringN(savename, + SymbolicLinkName, + ipoint->SymbolicLinkNameLength); + // Might as well null terminate. + savename->Buffer[ + ipoint->SymbolicLinkNameLength / + sizeof (WCHAR)] = 0; + + if (only_driveletter && + !MOUNTMGR_IS_DRIVE_LETTER(savename)) + savename->Length = 0; + else if (only_volume_name && + !MOUNTMGR_IS_VOLUME_NAME(savename)) + savename->Length = 0; + + if (MOUNTMGR_IS_DRIVE_LETTER(savename) || + MOUNTMGR_IS_VOLUME_NAME(savename)) + break; + } + } + } + + if (ppoints != NULL) kmem_free(ppoints, len); + return (STATUS_SUCCESS); +} + +/* + * Returns the last valid mountpoint of the device according + * to MOUNTMGR_IS_DRIVE_LETTER() + */ +NTSTATUS +mountmgr_get_drive_letter(DEVICE_OBJECT *mountmgr, + PUNICODE_STRING devpath, PUNICODE_STRING savename) +{ + return (mountmgr_get_mountpoint(mountmgr, devpath, savename, + TRUE, FALSE)); +} + +/* + * Returns the last valid mountpoint of the device according + * to MOUNTMGR_IS_VOLUME_NAME() + */ +NTSTATUS +mountmgr_get_volume_name_mountpoint(PDEVICE_OBJECT mountmgr, + PUNICODE_STRING devpath, PUNICODE_STRING savename) +{ + return (mountmgr_get_mountpoint(mountmgr, devpath, savename, + FALSE, TRUE)); +} + +NTSTATUS +SendIoctlToMountManager(__in ULONG IoControlCode, __in PVOID InputBuffer, + __in ULONG Length, __out PVOID OutputBuffer, + __in ULONG OutputLength) +{ + NTSTATUS status; + UNICODE_STRING mountManagerName; + PFILE_OBJECT mountFileObject; + PDEVICE_OBJECT mountDeviceObject; + PIRP irp; + KEVENT driverEvent; + IO_STATUS_BLOCK iosb; + + RtlInitUnicodeString(&mountManagerName, MOUNTMGR_DEVICE_NAME); + + status = IoGetDeviceObjectPointer(&mountManagerName, + FILE_READ_ATTRIBUTES, + &mountFileObject, &mountDeviceObject); + + if (!NT_SUCCESS(status)) { + dprintf(" IoGetDeviceObjectPointer failed: 0x%x\n", status); + return (status); + } + + KeInitializeEvent(&driverEvent, NotificationEvent, FALSE); + + irp = IoBuildDeviceIoControlRequest(IoControlCode, mountDeviceObject, + InputBuffer, Length, OutputBuffer, + OutputLength, FALSE, &driverEvent, &iosb); + + if (irp == NULL) { + dprintf(" IoBuildDeviceIoControlRequest failed\n"); + return (STATUS_INSUFFICIENT_RESOURCES); + } + + status = IoCallDriver(mountDeviceObject, irp); + + if (status == STATUS_PENDING) { + KeWaitForSingleObject(&driverEvent, Executive, KernelMode, + FALSE, NULL); + } + status = iosb.Status; + + ObDereferenceObject(mountFileObject); + // Don't dereference mountDeviceObject, mountFileObject is enough + + if (NT_SUCCESS(status)) { + dprintf(" IoCallDriver success\n"); + } else { + dprintf(" IoCallDriver failed: 0x%x\n", status); + } + + return (status); +} + +NTSTATUS +MountMgrChangeNotify(void) +{ + NTSTATUS status; + ULONG length; + MOUNTMGR_CHANGE_NOTIFY_INFO chinfo_in; + MOUNTMGR_CHANGE_NOTIFY_INFO chinfo_out; + + dprintf("=> MountMgrChangeNotify\n"); + + length = sizeof (MOUNTMGR_CHANGE_NOTIFY_INFO); + + status = SendIoctlToMountManager(IOCTL_MOUNTMGR_CHANGE_NOTIFY, + &chinfo_in, length, &chinfo_out, length); + + if (NT_SUCCESS(status)) + dprintf(" IoCallDriver success\n"); + else + dprintf(" IoCallDriver failed: 0x%x\n", status); + + dprintf("<= MountMgrChangeNotify\n"); + + return (status); +} + +NTSTATUS +SendVolumeArrivalNotification(PUNICODE_STRING DeviceName) +{ + NTSTATUS status; + PMOUNTMGR_TARGET_NAME targetName; + ULONG length; + + dprintf("=> SendVolumeArrivalNotification: '%wZ'\n", DeviceName); + + length = sizeof (MOUNTMGR_TARGET_NAME) + DeviceName->Length - 1; + targetName = ExAllocatePool(PagedPool, length); + + if (targetName == NULL) { + dprintf(" can't allocate MOUNTMGR_TARGET_NAME\n"); + return (STATUS_INSUFFICIENT_RESOURCES); + } + + RtlZeroMemory(targetName, length); + + targetName->DeviceNameLength = DeviceName->Length; + RtlCopyMemory(targetName->DeviceName, DeviceName->Buffer, + DeviceName->Length); + + status = SendIoctlToMountManager( + IOCTL_MOUNTMGR_VOLUME_ARRIVAL_NOTIFICATION, + targetName, length, NULL, 0); + + if (NT_SUCCESS(status)) { + dprintf(" IoCallDriver success\n"); + } else { + dprintf(" IoCallDriver failed: 0x%x\n", status); + } + + ExFreePool(targetName); + + dprintf("<= SendVolumeArrivalNotification\n"); + + return (status); +} + +NTSTATUS +SendVolumeRemovalNotification(PUNICODE_STRING DeviceName) +{ + NTSTATUS status; + PMOUNTMGR_TARGET_NAME targetName; + ULONG length; + + dprintf("=> SendVolumeRemovalNotification: '%wZ'\n", DeviceName); + + length = sizeof (MOUNTMGR_TARGET_NAME) + DeviceName->Length - 1; + targetName = ExAllocatePool(PagedPool, length); + + if (targetName == NULL) { + dprintf(" can't allocate MOUNTMGR_TARGET_NAME\n"); + return (STATUS_INSUFFICIENT_RESOURCES); + } + + RtlZeroMemory(targetName, length); + + targetName->DeviceNameLength = DeviceName->Length; + RtlCopyMemory(targetName->DeviceName, DeviceName->Buffer, + DeviceName->Length); + +#ifndef IOCTL_MOUNTMGR_VOLUME_REMOVAL_NOTIFICATION +#define IOCTL_MOUNTMGR_VOLUME_REMOVAL_NOTIFICATION \ + CTL_CODE(MOUNTMGRCONTROLTYPE, 22, METHOD_BUFFERED, FILE_READ_ACCESS) +#endif + + status = SendIoctlToMountManager( + IOCTL_MOUNTMGR_VOLUME_REMOVAL_NOTIFICATION, + targetName, length, NULL, 0); + + if (NT_SUCCESS(status)) { + dprintf(" IoCallDriver success\n"); + } else { + dprintf(" IoCallDriver failed: 0x%x\n", status); + } + + ExFreePool(targetName); + + dprintf("<= SendVolumeArrivalNotification\n"); + + return (status); +} + +NTSTATUS +RegisterDeviceInterface(__in PDRIVER_OBJECT DriverObject, + __in PDEVICE_OBJECT DeviceObject, __in mount_t *Dcb) +{ + PDEVICE_OBJECT pnpDeviceObject = NULL; + NTSTATUS status; + + status = IoReportDetectedDevice( + DriverObject, + InterfaceTypeUndefined, + 0xFFFFFFFF, // 0 + 0xFFFFFFFF, // 0 + NULL, + NULL, + FALSE, + &pnpDeviceObject); + + if (NT_SUCCESS(status)) { + dprintf(" IoReportDetectedDevice success\n"); + } else { + dprintf(" IoReportDetectedDevice failed: 0x%x\n", status); + return (status); + } + + if (IoAttachDeviceToDeviceStack(pnpDeviceObject, + DeviceObject) != NULL) { + dprintf(" IoAttachDeviceToDeviceStack success\n"); + } else { + dprintf(" IoAttachDeviceToDeviceStack failed\n"); + } + + status = IoRegisterDeviceInterface( + pnpDeviceObject, + &GUID_DEVINTERFACE_DISK, + NULL, + &Dcb->device_name); // device_name checks out + + if (NT_SUCCESS(status)) { + dprintf(" IoRegisterDeviceInterface success: %wZ\n", + &Dcb->device_name); + } else { + dprintf(" IoRegisterDeviceInterface failed: 0x%x\n", status); + return (status); + } + + status = IoSetDeviceInterfaceState(&Dcb->device_name, TRUE); + + if (NT_SUCCESS(status)) { + dprintf(" IoSetDeviceInterfaceState success\n"); + } else { + dprintf(" IoSetDeviceInterfaceState failed: 0x%x\n", status); + return (status); + } + + status = IoRegisterDeviceInterface( + pnpDeviceObject, + &MOUNTDEV_MOUNTED_DEVICE_GUID, + NULL, + &Dcb->fs_name); + + if (NT_SUCCESS(status)) { + dprintf(" IoRegisterDeviceInterface success: %wZ\n", + &Dcb->fs_name); + } else { + dprintf(" IoRegisterDeviceInterface failed: 0x%x\n", status); + return (status); + } + + status = IoSetDeviceInterfaceState(&Dcb->fs_name, TRUE); + + if (NT_SUCCESS(status)) { + dprintf(" IoSetDeviceInterfaceState success\n"); + } else { + dprintf(" IoSetDeviceInterfaceState failed: 0x%x\n", status); + return (status); + } + + return (status); +} + +NTSTATUS +SendVolumeCreatePoint(__in PUNICODE_STRING DeviceName, + __in PUNICODE_STRING MountPoint) +{ + NTSTATUS status; + PMOUNTMGR_CREATE_POINT_INPUT point; + ULONG length; + + dprintf("=> SendVolumeCreatePoint\n"); + + length = sizeof (MOUNTMGR_CREATE_POINT_INPUT) + MountPoint->Length + + DeviceName->Length; + point = ExAllocatePool(PagedPool, length); + + if (point == NULL) { + dprintf(" can't allocate MOUNTMGR_CREATE_POINT_INPUT\n"); + return (STATUS_INSUFFICIENT_RESOURCES); + } + + RtlZeroMemory(point, length); + + dprintf(" DeviceName: %wZ\n", DeviceName); + point->DeviceNameOffset = sizeof (MOUNTMGR_CREATE_POINT_INPUT); + point->DeviceNameLength = DeviceName->Length; + RtlCopyMemory((PCHAR)point + point->DeviceNameOffset, + DeviceName->Buffer, + DeviceName->Length); + + dprintf(" MountPoint: %wZ\n", MountPoint); + point->SymbolicLinkNameOffset = + point->DeviceNameOffset + point->DeviceNameLength; + point->SymbolicLinkNameLength = MountPoint->Length; + RtlCopyMemory((PCHAR)point + point->SymbolicLinkNameOffset, + MountPoint->Buffer, MountPoint->Length); + + status = SendIoctlToMountManager(IOCTL_MOUNTMGR_CREATE_POINT, point, + length, NULL, 0); + + if (NT_SUCCESS(status)) { + dprintf(" IoCallDriver success\n"); + } else { + dprintf(" IoCallDriver failed: 0x%x\n", status); + } + + ExFreePool(point); + + dprintf("<= SendVolumeCreatePoint\n"); + + return (status); +} + +NTSTATUS +SendVolumeCreatePointX(__in PUNICODE_STRING DeviceName, + __in PUNICODE_STRING MountPoint) +{ + NTSTATUS status; + PMOUNTMGR_VOLUME_MOUNT_POINT point; + ULONG length; + + dprintf("=> SendVolumeCreatePointX\n"); + + length = sizeof (MOUNTMGR_VOLUME_MOUNT_POINT) + MountPoint->Length + + DeviceName->Length; + point = ExAllocatePool(PagedPool, length); + + if (point == NULL) { + dprintf(" can't allocate MOUNTMGR_VOLUME_MOUNT_POINT\n"); + return (STATUS_INSUFFICIENT_RESOURCES); + } + + RtlZeroMemory(point, length); + + dprintf(" DeviceName: %wZ\n", DeviceName); + point->TargetVolumeNameOffset = sizeof (MOUNTMGR_VOLUME_MOUNT_POINT); + point->TargetVolumeNameLength = DeviceName->Length; + RtlCopyMemory((PCHAR)point + point->TargetVolumeNameOffset, + DeviceName->Buffer, + DeviceName->Length); + + dprintf(" MountPoint: %wZ\n", MountPoint); + point->SourceVolumeNameOffset = + point->TargetVolumeNameOffset + point->TargetVolumeNameLength; + point->SourceVolumeNameLength = MountPoint->Length; + RtlCopyMemory((PCHAR)point + point->SourceVolumeNameOffset, + MountPoint->Buffer, MountPoint->Length); + + status = SendIoctlToMountManager( + IOCTL_MOUNTMGR_VOLUME_MOUNT_POINT_CREATED, + point, length, NULL, 0); + + if (NT_SUCCESS(status)) { + dprintf(" IoCallDriver success\n"); + } else { + dprintf(" IoCallDriver failed: 0x%x\n", status); + } + + ExFreePool(point); + + dprintf("<= SendVolumeCreatePointX\n"); + + return (status); +} + +NTSTATUS +SendVolumeDeletePoints(__in PUNICODE_STRING MountPoint, + __in PUNICODE_STRING DeviceName) +{ + NTSTATUS status; + PMOUNTMGR_MOUNT_POINT point; + PMOUNTMGR_MOUNT_POINTS deletedPoints; + ULONG length; + ULONG olength; + + dprintf("=> SendVolumeDeletePoints: '%wZ'\n", DeviceName); + + if (_wcsnicmp(L"\\DosDevices\\", MountPoint->Buffer, 12)) { + dprintf("Not a drive letter, skipping\n"); + return (STATUS_SUCCESS); + } + + length = sizeof (MOUNTMGR_MOUNT_POINT) + MountPoint->Length; + if (DeviceName != NULL) { + length += DeviceName->Length; + } + point = kmem_alloc(length, KM_SLEEP); + + if (point == NULL) { + dprintf(" can't allocate MOUNTMGR_CREATE_POINT_INPUT\n"); + return (STATUS_INSUFFICIENT_RESOURCES); + } + + olength = sizeof (MOUNTMGR_MOUNT_POINTS) + 1024; + deletedPoints = kmem_alloc(olength, KM_SLEEP); + if (deletedPoints == NULL) { + dprintf(" can't allocate PMOUNTMGR_MOUNT_POINTS\n"); + kmem_free(point, length); + return (STATUS_INSUFFICIENT_RESOURCES); + } + + RtlZeroMemory(point, length); // kmem_zalloc + RtlZeroMemory(deletedPoints, olength); + + dprintf(" MountPoint: %wZ\n", MountPoint); + point->SymbolicLinkNameOffset = sizeof (MOUNTMGR_MOUNT_POINT); + point->SymbolicLinkNameLength = MountPoint->Length; + RtlCopyMemory((PCHAR)point + point->SymbolicLinkNameOffset, + MountPoint->Buffer, MountPoint->Length); + if (DeviceName != NULL) { + dprintf(" DeviceName: %wZ\n", DeviceName); + point->DeviceNameOffset = + point->SymbolicLinkNameOffset + + point->SymbolicLinkNameLength; + point->DeviceNameLength = DeviceName->Length; + RtlCopyMemory((PCHAR)point + point->DeviceNameOffset, + DeviceName->Buffer, DeviceName->Length); + } + + // Only symbolic link can be deleted with IOCTL_MOUNTMGR_DELETE_POINTS. + // If any other entry is specified, the mount manager will ignore + // subsequent IOCTL_MOUNTMGR_VOLUME_ARRIVAL_NOTIFICATION for the + // same volume ID. + status = SendIoctlToMountManager(IOCTL_MOUNTMGR_DELETE_POINTS, point, + length, deletedPoints, olength); + + if (NT_SUCCESS(status)) { + dprintf(" IoCallDriver success, %d mount points deleted.\n", + deletedPoints->NumberOfMountPoints); + } else { + dprintf(" IoCallDriver failed: 0x%x\n", status); + } + + kmem_free(point, length); + kmem_free(deletedPoints, olength); + + dprintf("<= SendVolumeDeletePoints\n"); + + return (status); +} + +void +zfs_release_mount(mount_t *zmo) +{ + FreeUnicodeString(&zmo->symlink_name); + FreeUnicodeString(&zmo->device_name); + FreeUnicodeString(&zmo->fs_name); + FreeUnicodeString(&zmo->uuid); + FreeUnicodeString(&zmo->mountpoint); + + if (zmo->vpb) { + zmo->vpb->DeviceObject = NULL; + zmo->vpb->RealDevice = NULL; + zmo->vpb->Flags = 0; + } +} + +int +zfs_windows_mount(zfs_cmd_t *zc) +{ + dprintf("%s: '%s' '%s'\n", __func__, zc->zc_name, zc->zc_value); + NTSTATUS status; + uuid_t uuid; + char uuid_a[UUID_PRINTABLE_STRING_LENGTH]; + PDEVICE_OBJECT pdo = NULL; + PDEVICE_OBJECT diskDeviceObject = NULL; + PDEVICE_OBJECT fsDeviceObject = NULL; + + /* + * We expect mountpath (zv_value) to be already sanitised, ie, Windows + * translated paths. So it should be on this style: + * "\\??\\c:" mount as drive letter C: + * "\\??\\?:" mount as first available drive letter + * "\\??\\c:\\BOOM" mount as drive letter C:\BOOM + */ + int mplen = strlen(zc->zc_value); + if ((mplen < 6) || + strncmp("\\??\\", zc->zc_value, 4)) { + dprintf("%s: mountpoint '%s' does not start with \\??\\x:", + __func__, zc->zc_value); + return (EINVAL); + } + + zfs_vfs_uuid_gen(zc->zc_name, uuid); + zfs_vfs_uuid_unparse(uuid, uuid_a); + + char buf[PATH_MAX]; + // snprintf(buf, sizeof (buf), "\\Device\\ZFS{%s}", uuid_a); + WCHAR diskDeviceNameBuf[MAXIMUM_FILENAME_LENGTH]; // L"\\Device\\Volume" + WCHAR fsDeviceNameBuf[MAXIMUM_FILENAME_LENGTH]; // L"\\Device\\ZFS" + // L"\\DosDevices\\Global\\Volume" + WCHAR symbolicLinkNameBuf[MAXIMUM_FILENAME_LENGTH]; + UNICODE_STRING diskDeviceName; + UNICODE_STRING fsDeviceName; + UNICODE_STRING symbolicLinkTarget; + + ANSI_STRING pants; + ULONG deviceCharacteristics; + deviceCharacteristics = 0; // FILE_DEVICE_IS_MOUNTED; + /* Allow $recycle.bin - don't set removable. */ + if (!zfs_disable_removablemedia) + deviceCharacteristics |= FILE_REMOVABLE_MEDIA; + + snprintf(buf, sizeof (buf), "\\Device\\Volume{%s}", uuid_a); + // snprintf(buf, sizeof (buf), "\\Device\\ZFS_%s", zc->zc_name); + pants.Buffer = buf; + pants.Length = strlen(buf); + pants.MaximumLength = PATH_MAX; + status = RtlAnsiStringToUnicodeString(&diskDeviceName, &pants, TRUE); + dprintf("%s: new devstring '%wZ'\n", __func__, &diskDeviceName); + + status = IoCreateDeviceSecure(WIN_DriverObject, + sizeof (mount_t), + &diskDeviceName, + FILE_DEVICE_DISK, + deviceCharacteristics, + FALSE, + &SDDL_DEVOBJ_SYS_ALL_ADM_RWX_WORLD_RW_RES_R, + NULL, + &diskDeviceObject); + + if (status != STATUS_SUCCESS) { + dprintf("IoCreateDeviceSecure returned %08x\n", status); + return (status); + } + + diskDeviceObject->Flags |= DO_BUS_ENUMERATED_DEVICE; + + mount_t *zmo_dcb = diskDeviceObject->DeviceExtension; + zmo_dcb->type = MOUNT_TYPE_DCB; + zmo_dcb->size = sizeof (mount_t); + vfs_setfsprivate(zmo_dcb, NULL); + dprintf("%s: created dcb at %p asked for size %d\n", + __func__, zmo_dcb, sizeof (mount_t)); + AsciiStringToUnicodeString(uuid_a, &zmo_dcb->uuid); + // Should we keep the name with slashes like "BOOM/lower" or + // just "lower". Turns out the name in Explorer only + // works for 4 chars or lower. Why? + AsciiStringToUnicodeString(zc->zc_name, &zmo_dcb->name); + AsciiStringToUnicodeString(buf, &zmo_dcb->device_name); + // strlcpy(zc->zc_value, buf, sizeof (zc->zc_value)); + zmo_dcb->deviceObject = diskDeviceObject; + dprintf("New device %p has extension %p\n", + diskDeviceObject, zmo_dcb); + + snprintf(buf, sizeof (buf), "\\DosDevices\\Global\\Volume{%s}", uuid_a); + pants.Buffer = buf; + pants.Length = strlen(buf); + pants.MaximumLength = PATH_MAX; + status = RtlAnsiStringToUnicodeString(&symbolicLinkTarget, &pants, + TRUE); + dprintf("%s: new symlink '%wZ'\n", __func__, &symbolicLinkTarget); + AsciiStringToUnicodeString(buf, &zmo_dcb->symlink_name); + + snprintf(buf, sizeof (buf), "\\Device\\ZFS{%s}", uuid_a); + pants.Buffer = buf; + pants.Length = strlen(buf); + pants.MaximumLength = PATH_MAX; + status = RtlAnsiStringToUnicodeString(&fsDeviceName, &pants, TRUE); + dprintf("%s: new fsname '%wZ'\n", __func__, &fsDeviceName); + AsciiStringToUnicodeString(buf, &zmo_dcb->fs_name); + + diskDeviceObject->Flags |= DO_DIRECT_IO; + + + status = IoCreateSymbolicLink(&symbolicLinkTarget, &diskDeviceName); + + if (!NT_SUCCESS(status)) { + IoDeleteDevice(diskDeviceObject); + dprintf(" IoCreateSymbolicLink returned 0x%x\n", status); + return (status); + } + + // Call ZFS and have it setup a mount "zfsvfs" + // we don't have the vcb yet, but we want to find out mount + // problems early. + struct zfs_mount_args mnt_args; + mnt_args.struct_size = sizeof (struct zfs_mount_args); + mnt_args.optlen = 0; + mnt_args.mflag = 0; // Set flags + mnt_args.fspec = zc->zc_name; + + // zc_cleanup_fd carrier mount flags for now. + if (zc->zc_cleanup_fd & MNT_RDONLY) + vfs_setrdonly(zmo_dcb); + + // Mount will temporarily be pointing to "dcb" until the + // zfs_vnop_mount() below corrects it to "vcb". + status = zfs_vfs_mount(zmo_dcb, NULL, (user_addr_t)&mnt_args, NULL); + dprintf("%s: zfs_vfs_mount() returns %d\n", __func__, status); + + if (status) { + IoDeleteDevice(diskDeviceObject); + return (status); + } + + // Check if we are to mount with driveletter, or path + // We already check that path is "\\??\\" above, and + // at least 6 chars. Seventh char can be zero, or "/" + // then zero, for drive only mount. + if ((zc->zc_value[6] == 0) || + ((zc->zc_value[6] == '/') && + (zc->zc_value[7] == 0))) { + zmo_dcb->justDriveLetter = B_TRUE; + } else { + zmo_dcb->justDriveLetter = B_FALSE; + } + + // Remember mountpoint path + AsciiStringToUnicodeString(zc->zc_value, &zmo_dcb->mountpoint); + + dprintf("%s: driveletter %d '%wZ'\n", + __func__, zmo_dcb->justDriveLetter, &zmo_dcb->mountpoint); + + // Return volume name to userland + snprintf(zc->zc_value, sizeof (zc->zc_value), + "\\DosDevices\\Global\\Volume{%s}", uuid_a); + + // Mark devices as initialized + diskDeviceObject->Flags &= ~DO_DEVICE_INITIALIZING; + ObReferenceObject(diskDeviceObject); + + dprintf("Verify Volume\n"); + IoVerifyVolume(diskDeviceObject, FALSE); + + status = STATUS_SUCCESS; + return (status); +} + +void +InitVpb(__in PVPB Vpb, __in PDEVICE_OBJECT VolumeDevice) +{ + if (Vpb != NULL) { + Vpb->DeviceObject = VolumeDevice; + Vpb->VolumeLabelLength = + (USHORT)wcslen(VOLUME_LABEL) * sizeof (WCHAR); + RtlStringCchCopyW(Vpb->VolumeLabel, + sizeof (Vpb->VolumeLabel) / sizeof (WCHAR), VOLUME_LABEL); + Vpb->SerialNumber = 0x19831116; + Vpb->Flags |= VPB_MOUNTED; + } +} + +NTSTATUS +CreateReparsePoint(POBJECT_ATTRIBUTES poa, PCUNICODE_STRING SubstituteName, + PCUNICODE_STRING PrintName) +{ + HANDLE hFile; + IO_STATUS_BLOCK iosb; + NTSTATUS status; + + dprintf("%s: \n", __func__); + + // this is stalled forever waiting for event of deletion - + // possibly ZFS doesnt send event? + status = ZwDeleteFile(poa); + if (status != STATUS_SUCCESS) + dprintf("pre-rmdir failed 0x%x\n", status); + status = ZwCreateFile(&hFile, FILE_ALL_ACCESS, poa, &iosb, 0, 0, 0, + FILE_CREATE, FILE_DIRECTORY_FILE | FILE_SYNCHRONOUS_IO_NONALERT, + 0, 0); + if (0 > status) + return (status); + dprintf("%s: create ok\n", __func__); + + USHORT cb = 2 * sizeof (WCHAR) + + FIELD_OFFSET(REPARSE_DATA_BUFFER, + MountPointReparseBuffer.PathBuffer) + + SubstituteName->Length + PrintName->Length; + PREPARSE_DATA_BUFFER prdb = (PREPARSE_DATA_BUFFER)alloca(cb); + RtlZeroMemory(prdb, cb); + prdb->ReparseTag = IO_REPARSE_TAG_MOUNT_POINT; + prdb->ReparseDataLength = cb - REPARSE_DATA_BUFFER_HEADER_SIZE; + prdb->MountPointReparseBuffer.SubstituteNameLength = + SubstituteName->Length; + prdb->MountPointReparseBuffer.PrintNameLength = PrintName->Length; + prdb->MountPointReparseBuffer.PrintNameOffset = + SubstituteName->Length + sizeof (WCHAR); + memcpy(prdb->MountPointReparseBuffer.PathBuffer, + SubstituteName->Buffer, SubstituteName->Length); + memcpy(RtlOffsetToPointer(prdb->MountPointReparseBuffer.PathBuffer, + SubstituteName->Length + sizeof (WCHAR)), + PrintName->Buffer, PrintName->Length); + status = ZwFsControlFile(hFile, 0, 0, 0, &iosb, + FSCTL_SET_REPARSE_POINT, prdb, cb, 0, 0); + dprintf("%s: ControlFile %d / 0x%x\n", __func__, status, status); + + if (0 > status) { + static FILE_DISPOSITION_INFORMATION fdi = { TRUE }; + ZwSetInformationFile(hFile, &iosb, &fdi, + sizeof (fdi), FileDispositionInformation); + } + ZwClose(hFile); + return (status); +} + +NTSTATUS +DeleteReparsePoint(POBJECT_ATTRIBUTES poa) +{ + HANDLE hFile; + IO_STATUS_BLOCK iosb; + NTSTATUS status; + REPARSE_DATA_BUFFER ReparseData; + DWORD BytesReturned = 0; + + dprintf("%s: \n", __func__); + + status = ZwCreateFile(&hFile, FILE_ALL_ACCESS, poa, &iosb, 0, 0, 0, + FILE_OPEN_IF, FILE_DIRECTORY_FILE | FILE_SYNCHRONOUS_IO_NONALERT | + FILE_OPEN_FOR_BACKUP_INTENT | FILE_OPEN_REPARSE_POINT, + 0, 0); + if (0 > status) + return (status); + dprintf("%s: create ok\n", __func__); + + memset(&ReparseData, 0, REPARSE_DATA_BUFFER_HEADER_SIZE); + ReparseData.ReparseTag = IO_REPARSE_TAG_MOUNT_POINT; + + status = ZwFsControlFile(hFile, 0, 0, 0, &iosb, + FSCTL_DELETE_REPARSE_POINT, &ReparseData, + REPARSE_DATA_BUFFER_HEADER_SIZE, NULL, 0); + + ZwClose(hFile); + return (status); +} + +/* + * go through all mointpoints (IOCTL_MOUNTMGR_QUERY_POINTS) + * and check if our driveletter is in the list + * return 1 if yes, otherwise 0 + */ +NTSTATUS +mountmgr_is_driveletter_assigned(PDEVICE_OBJECT mountmgr, + wchar_t driveletter, BOOLEAN *ret) +{ + MOUNTMGR_MOUNT_POINT point = { 0 }; + MOUNTMGR_MOUNT_POINTS points; + PMOUNTMGR_MOUNT_POINTS ppoints = NULL; + int len; + *ret = 0; + NTSTATUS Status; + + ppoints = &points; + Status = dev_ioctl(mountmgr, IOCTL_MOUNTMGR_QUERY_POINTS, &point, + sizeof (MOUNTMGR_MOUNT_POINT), ppoints, + sizeof (MOUNTMGR_MOUNT_POINTS), FALSE, NULL); + + if (Status == STATUS_BUFFER_OVERFLOW) { + len = points.Size; + ppoints = kmem_alloc(len, KM_SLEEP); + Status = dev_ioctl(mountmgr, IOCTL_MOUNTMGR_QUERY_POINTS, + &point, sizeof (MOUNTMGR_MOUNT_POINT), ppoints, + len, FALSE, NULL); + } + dprintf("IOCTL_MOUNTMGR_QUERY_POINTS return %x - " + "looking for driveletter '%c'\n", + Status, driveletter); + if (Status == STATUS_SUCCESS) { + char mpt_name[PATH_MAX] = { 0 }; + for (int Index = 0; + Index < ppoints->NumberOfMountPoints; + Index++) { + PMOUNTMGR_MOUNT_POINT ipoint = + ppoints->MountPoints + Index; + PWCHAR DeviceName = + (PWCHAR)((PUCHAR)ppoints + + ipoint->DeviceNameOffset); + PWCHAR SymbolicLinkName = + (PWCHAR)((PUCHAR)ppoints + + ipoint->SymbolicLinkNameOffset); + + dprintf(" point %d: '%.*S' '%.*S'\n", Index, + ipoint->DeviceNameLength / sizeof (WCHAR), + DeviceName, + ipoint->SymbolicLinkNameLength / + sizeof (WCHAR), + SymbolicLinkName); + + ULONG length = 0; + RtlUnicodeToUTF8N(mpt_name, MAXPATHLEN, &length, + SymbolicLinkName, + ipoint->SymbolicLinkNameLength); + mpt_name[length] = 0; + char c_driveletter; + wctomb(&c_driveletter, driveletter); + if (MOUNTMGR_IS_DRIVE_LETTER_A(mpt_name) && + mpt_name[12] == c_driveletter) { + *ret = 1; + if (ppoints != NULL) kmem_free(ppoints, len); + return (STATUS_SUCCESS); + } + } + } + + if (ppoints != NULL) kmem_free(ppoints, len); + return (Status); +} + +/* + * assign driveletter with IOCTL_MOUNTMGR_CREATE_POINT + */ +NTSTATUS +mountmgr_assign_driveletter(PUNICODE_STRING device_name, + wchar_t driveletter) +{ + DECLARE_UNICODE_STRING_SIZE(mpt, 16); + RtlUnicodeStringPrintf(&mpt, L"\\DosDevices\\%c:", driveletter); + return (SendVolumeCreatePoint(device_name, &mpt)); +} + + +/* + * assign next free driveletter (D..Z) if mountmgr is offended + * and refuses to do it + */ +NTSTATUS +SetNextDriveletterManually(PDEVICE_OBJECT mountmgr, + PUNICODE_STRING device_name) +{ + NTSTATUS status; + for (wchar_t c = 'D'; c <= 'Z'; c++) { + BOOLEAN ret; + status = mountmgr_is_driveletter_assigned(mountmgr, c, &ret); + if (status == STATUS_SUCCESS && ret == 0) { + status = mountmgr_assign_driveletter(device_name, c); + + if (status == STATUS_SUCCESS) { + // prove it + status = + mountmgr_is_driveletter_assigned(mountmgr, + c, &ret); + if (status == STATUS_SUCCESS) { + if (ret == 1) + return (STATUS_SUCCESS); + else + return + (STATUS_VOLUME_DISMOUNTED); + } else { + return (status); + } + } + } + } + return (status); +} + + + +void +generateGUID(char *pguid) +{ + char *uuid_format = "xxxxxxxx-xxxx-4xxx-Nxxx-xxxxxxxxxxxx"; + char *szHex = "0123456789ABCDEF-"; + int len = strlen(uuid_format); + + for (int i = 0; i < len + 1; i++) { + int r = rand() % 16; + char c = ' '; + + switch (uuid_format[i]) { + case 'x': { c = szHex[r]; } + break; + case 'N': { c = szHex[r & 0x03 | 0x08]; } + break; + case '-': { c = '-'; } + break; + case '4': { c = '4'; } + break; + } + + pguid[i] = (i < len) ? c : 0x00; + } +} + + +void +generateVolumeNameMountpoint(wchar_t *vol_mpt) +{ + char GUID[50]; + wchar_t wc_guid[50]; + generateGUID(&GUID); + mbstowcs(&wc_guid, GUID, 50); + int len = _snwprintf(vol_mpt, 50, L"\\??\\Volume{%s}", wc_guid); +} + +int +zfs_vnop_mount(PDEVICE_OBJECT DiskDevice, PIRP Irp, PIO_STACK_LOCATION IrpSp) +{ + PDRIVER_OBJECT DriverObject = DiskDevice->DriverObject; + PDEVICE_OBJECT volDeviceObject; + NTSTATUS status; + PDEVICE_OBJECT DeviceToMount; + + dprintf("%s\n", __func__); + + if (IrpSp->Parameters.MountVolume.DeviceObject == NULL) { + dprintf("%s: MountVolume is NULL\n", __func__); + return (STATUS_UNRECOGNIZED_VOLUME); + } + + DeviceToMount = IoGetDeviceAttachmentBaseRef(IrpSp-> + Parameters.MountVolume.DeviceObject); + dprintf("*** mount request for %p : minor\n", DeviceToMount); + + if (DeviceToMount == NULL) { + dprintf("%s: DeviceToMount is NULL\n", __func__); + return (STATUS_UNRECOGNIZED_VOLUME); + } + + // DeviceToMount must be released from here down + + if (DeviceToMount->DriverObject == WIN_DriverObject) { + dprintf("*** The device belong to us\n"); + } else { + dprintf("*** The device does NOT belong to us\n"); + status = STATUS_UNRECOGNIZED_VOLUME; + goto out; + } + + mount_t *dcb = DeviceToMount->DeviceExtension; + if (dcb == NULL) { + dprintf("%s: Not a ZFS dataset -- ignoring\n", __func__); + status = STATUS_UNRECOGNIZED_VOLUME; + goto out; + } + + if ((dcb->type != MOUNT_TYPE_DCB) || + (dcb->size != sizeof (mount_t))) { + dprintf("%s: Not a ZFS dataset -- dcb %p ignoring: " + "type 0x%x != 0x%x, size %d != %d\n", + __func__, dcb, + dcb->type, MOUNT_TYPE_DCB, dcb->size, sizeof (mount_t)); + status = STATUS_UNRECOGNIZED_VOLUME; + goto out; + } + + zfsvfs_t *xzfsvfs = vfs_fsprivate(dcb); + + if (xzfsvfs && xzfsvfs->z_unmounted) { + dprintf("%s: Is a ZFS dataset -- unmounted. dcb %p ignoring: " + "type 0x%x != 0x%x, size %d != %d\n", + __func__, dcb, + dcb->type, MOUNT_TYPE_DCB, dcb->size, sizeof (mount_t)); + status = STATUS_UNRECOGNIZED_VOLUME; + goto out; + } + + // ZFS Dataset being mounted: + // dprintf("%s: mounting '%wZ'\n", __func__, dcb->name); + + // We created a DISK before, now we create a VOLUME + ULONG deviceCharacteristics; + deviceCharacteristics = 0; // FILE_DEVICE_IS_MOUNTED; + /* Allow $recycle.bin - don't set removable. */ + if (!zfs_disable_removablemedia) + deviceCharacteristics |= FILE_REMOVABLE_MEDIA; + + if (dcb->mountflags & MNT_RDONLY) + deviceCharacteristics |= FILE_READ_ONLY_DEVICE; + + status = IoCreateDevice(DriverObject, + sizeof (mount_t), + NULL, +// FILE_DEVICE_DISK, + FILE_DEVICE_DISK_FILE_SYSTEM, + deviceCharacteristics, + FALSE, + &volDeviceObject); + + if (!NT_SUCCESS(status)) { + dprintf("%s: IoCreateDevice failed: 0x%x\n", __func__, status); + goto out; + } + + mount_t *vcb = volDeviceObject->DeviceExtension; + vcb->type = MOUNT_TYPE_VCB; + vcb->size = sizeof (mount_t); + + volDeviceObject->Flags |= DO_BUS_ENUMERATED_DEVICE; + + zfsvfs_t *zfsvfs = vfs_fsprivate(dcb); + int giveup = 0; + while (zfsvfs == NULL) { + delay(hz / 10); + dprintf("zfsvfs not resolved yet\n"); + zfsvfs = vfs_fsprivate(dcb); + if (giveup++ > 50) + return (STATUS_UNRECOGNIZED_VOLUME); + } + zfsvfs->z_vfs = vcb; + vfs_setfsprivate(vcb, zfsvfs); + // a bit hacky this bit, but we created some vnodes under + // dcb during this mount hand over, make them be owned by + // vcb + vfs_changeowner(dcb, vcb); + + // Remember the parent device, so during unmount we can free both. + vcb->parent_device = dcb; + + // vcb is the ptr used in unmount, so set both devices here. + // vcb->diskDeviceObject = dcb->deviceObject; + vcb->deviceObject = volDeviceObject; + + RtlDuplicateUnicodeString(0, &dcb->fs_name, &vcb->fs_name); + RtlDuplicateUnicodeString(0, &dcb->name, &vcb->name); + RtlDuplicateUnicodeString(0, &dcb->device_name, &vcb->device_name); + RtlDuplicateUnicodeString(0, &dcb->symlink_name, &vcb->symlink_name); + RtlDuplicateUnicodeString(0, &dcb->uuid, &vcb->uuid); + RtlDuplicateUnicodeString(0, &dcb->mountpoint, &vcb->mountpoint); + + vcb->mountflags = dcb->mountflags; + if (vfs_isrdonly(dcb)) + vfs_setrdonly(vcb); + + // Directory notification + InitializeListHead(&vcb->DirNotifyList); + FsRtlNotifyInitializeSync(&vcb->NotifySync); + + PVPB vpb = NULL; + vpb = IrpSp->Parameters.MountVolume.Vpb; + InitVpb(vpb, volDeviceObject); + vcb->vpb = vpb; + dcb->vpb = vpb; + + volDeviceObject->Flags |= DO_DIRECT_IO; + volDeviceObject->Flags &= ~DO_DEVICE_INITIALIZING; + // SetLongFlag(vcb->Flags, VCB_MOUNTED); + + ObReferenceObject(volDeviceObject); + + status = SendVolumeArrivalNotification(&dcb->device_name); + if (!NT_SUCCESS(status)) { + dprintf(" SendVolumeArrivalNotification failed: 0x%x\n", + status); + } + + UNICODE_STRING name; + PFILE_OBJECT fileObject; + PDEVICE_OBJECT mountmgr; + + // Query MntMgr for points, just informative + RtlInitUnicodeString(&name, MOUNTMGR_DEVICE_NAME); + status = IoGetDeviceObjectPointer(&name, FILE_READ_ATTRIBUTES, + &fileObject, &mountmgr); + DECLARE_UNICODE_STRING_SIZE(mountpath, PATH_MAX); + status = mountmgr_get_drive_letter(mountmgr, &dcb->device_name, + &mountpath); + + // Check if we are to mount as path or just drive letter + if (dcb->justDriveLetter) { + + // If SendVolumeArrival was executed successfully we should + // have two mountpoints + // point 1: \Device\Volumes{abc} \DosDevices\X: + // point 2: \Device\Volumes{abc} \??\Volume{xyz} + // but if we are in remount and removed the mountpoints + // for this volume manually before + // they won't get assigned by mountmgr automatically anymore. + // So at least check if we got them and if not, try to create. + + if (!MOUNTMGR_IS_DRIVE_LETTER(&mountpath)) { + DECLARE_UNICODE_STRING_SIZE(mountpoint, PATH_MAX); + status = mountmgr_get_volume_name_mountpoint(mountmgr, + &dcb->device_name, &mountpoint); + if (!MOUNTMGR_IS_VOLUME_NAME(&mountpoint)) { + // We have no volume name mountpoint for our + // device, so generate a valid GUID and mount + // the device + UNICODE_STRING vol_mpt; + wchar_t buf[50]; + generateVolumeNameMountpoint(&buf); + RtlInitUnicodeString(&vol_mpt, buf); + status = + SendVolumeCreatePoint(&dcb->device_name, + &vol_mpt); + } + + // If driveletter was provided, try to add it + // as mountpoint + if (dcb && dcb->mountpoint.Length > 0 && + dcb->mountpoint.Buffer[4] != '?') { + // check if driveletter is unassigned + BOOLEAN ret; + status = mountmgr_is_driveletter_assigned( + mountmgr, + dcb->mountpoint.Buffer[4], &ret); + + if (status == STATUS_SUCCESS && ret == 0) { + // driveletter is unassigned, try to + // add mountpoint + status = mountmgr_assign_driveletter( + &dcb->device_name, + dcb->mountpoint.Buffer[4]); + } else { + // driveletter already assigned, + // find another one + SetNextDriveletterManually(mountmgr, + &dcb->device_name); + } + } else { + // user provided no driveletter, + // find one on our own + SetNextDriveletterManually(mountmgr, + &dcb->device_name); + } + } // !MOUNTMGR_IS_DRIVE_LETTER(&actualDriveletter) + } else { + OBJECT_ATTRIBUTES poa; + // 36(uuid) + 6 (punct) + 6 (Volume) + DECLARE_UNICODE_STRING_SIZE(volStr, + ZFS_MAX_DATASET_NAME_LEN); + // "\??\Volume{0b1bb601-af0b-32e8-a1d2-54c167af6277}" + RtlUnicodeStringPrintf(&volStr, + L"\\??\\Volume{%wZ}", + vcb->uuid); + InitializeObjectAttributes(&poa, + &dcb->mountpoint, OBJ_KERNEL_HANDLE, NULL, NULL); + dprintf("Creating reparse mountpoint on '%wZ' for " + "volume '%wZ'\n", + &dcb->mountpoint, &volStr); + // 3rd arg is visible in DOS box + CreateReparsePoint(&poa, &volStr, &vcb->name); + + // Remove drive letter? + // RtlUnicodeStringPrintf(&volStr, L"\\DosDevices\\E:"); + // RtlUnicodeStringPrintf(&volStr, L"%s", namex); + // "\??\Volume{0b1bb601-af0b-32e8-a1d2-54c167af6277}" + + status = SendVolumeDeletePoints(&mountpath, &dcb->device_name); + + // Must start with "\DosDevices\X:" + // mountpoint = "\\??\\x:" + DECLARE_UNICODE_STRING_SIZE(mpoint, 128); + + status = RtlUnicodeStringPrintf( + &mpoint, L"\\DosDevices\\%ws", + &dcb->mountpoint.Buffer[4]); + + status = + SendVolumeCreatePointX( + &dcb->device_name, &mpoint); + + } + + // match IoGetDeviceAttachmentBaseRef() + ObDereferenceObject(fileObject); + + + // It seems likely we should announce our new filesystem, but when + // we do it stops working in explorer with "invalid function". + // But if we can set this, we can't call + // FSRTL_VOLUME_MOUNT below it, and more importantly, + // FSRTL_VOLUME_DISMOUNT + // before we umount. Need to figure out why. + +out: + ObDereferenceObject(DeviceToMount); + dprintf("%s: exit: 0x%x\n", __func__, status); + return (status); +} + +int +zfs_remove_driveletter(mount_t *zmo) +{ + UNICODE_STRING name; + PFILE_OBJECT fileObject; + PDEVICE_OBJECT mountmgr; + NTSTATUS Status; + + dprintf("%s: removing driveletter for '%wZ'\n", __func__, &zmo->name); + + // Query MntMgr for points, just informative + RtlInitUnicodeString(&name, MOUNTMGR_DEVICE_NAME); + Status = IoGetDeviceObjectPointer(&name, FILE_READ_ATTRIBUTES, + &fileObject, &mountmgr); + + MOUNTMGR_MOUNT_POINT *mmp = NULL; + ULONG mmpsize; + MOUNTMGR_MOUNT_POINTS mmps1, *mmps2 = NULL; + + mmpsize = sizeof (MOUNTMGR_MOUNT_POINT) + zmo->device_name.Length; + + mmp = kmem_zalloc(mmpsize, KM_SLEEP); + + mmp->DeviceNameOffset = sizeof (MOUNTMGR_MOUNT_POINT); + mmp->DeviceNameLength = zmo->device_name.Length; + RtlCopyMemory(&mmp[1], zmo->device_name.Buffer, + zmo->device_name.Length); + + Status = dev_ioctl(mountmgr, IOCTL_MOUNTMGR_DELETE_POINTS, + mmp, mmpsize, &mmps1, sizeof (MOUNTMGR_MOUNT_POINTS), FALSE, NULL); + + if (!NT_SUCCESS(Status) && Status != STATUS_BUFFER_OVERFLOW) { + goto out; + } + + if (Status != STATUS_BUFFER_OVERFLOW || mmps1.Size == 0) { + Status = STATUS_NOT_FOUND; + goto out; + } + + mmps2 = kmem_zalloc(mmps1.Size, KM_SLEEP); + + Status = dev_ioctl(mountmgr, IOCTL_MOUNTMGR_DELETE_POINTS, + mmp, mmpsize, mmps2, mmps1.Size, FALSE, NULL); + + // if (!NT_SUCCESS(Status)) + // ERR("IOCTL_MOUNTMGR_DELETE_POINTS 2 returned %08x\n", Status); + +out: + dprintf("%s: removing driveletter returns 0x%x\n", __func__, Status); + + if (mmps2) + kmem_free(mmps2, mmps1.Size); + if (mmp) + kmem_free(mmp, mmpsize); + + ObDereferenceObject(fileObject); + return (Status); +} + +int +zfs_windows_unmount(zfs_cmd_t *zc) +{ + // IRP_MN_QUERY_REMOVE_DEVICE + // IRP_MN_REMOVE_DEVICE + // FsRtlNotifyVolumeEvent(, FSRTL_VOLUME_DISMOUNT); + + // Use name, lookup zfsvfs + // use zfsvfs to get mount_t + // mount_t has deviceObject, names etc. + mount_t *zmo; + mount_t *zmo_dcb = NULL; + zfsvfs_t *zfsvfs; + int error = EBUSY; + znode_t *zp; + // int rdonly; + if (getzfsvfs(zc->zc_name, &zfsvfs) == 0) { + + zmo = zfsvfs->z_vfs; + NTSTATUS ntstatus; + ASSERT(zmo->type == MOUNT_TYPE_VCB); + + // Try issuing DISMOUNT ... this wont work unless + // "attached" in RegisterDeviceInterface() + FILE_OBJECT *root_file; + root_file = IoCreateStreamFileObject(NULL, + zmo->deviceObject); + ntstatus = FsRtlNotifyVolumeEvent(root_file, + FSRTL_VOLUME_DISMOUNT); + ObDereferenceObject(root_file); + + UNICODE_STRING name; + PFILE_OBJECT fileObject; + PDEVICE_OBJECT mountmgr; + + // Query MntMgr for points, just informative + RtlInitUnicodeString(&name, MOUNTMGR_DEVICE_NAME); + NTSTATUS status = IoGetDeviceObjectPointer(&name, + FILE_READ_ATTRIBUTES, &fileObject, &mountmgr); + DECLARE_UNICODE_STRING_SIZE(mountpath, PATH_MAX); + status = mountmgr_get_drive_letter(mountmgr, + &zmo->device_name, &mountpath); + + // Save the parent device + zmo_dcb = zmo->parent_device; + + // Flush volume + // rdonly = !spa_writeable(dmu_objset_spa(zfsvfs->z_os)); + + // Delete mountpoints for our volume manually + // Query the mountmgr for mountpoints and delete them + // until no mountpoint is left Because we are not satisfied + // with mountmgrs work, it gets offended and doesn't + // automatically create mointpoints for our volume after we + // deleted them manually But as long as we recheck that in + // mount and create points manually (if necessary), + // that should be ok hopefully + + // We used to loop here and keep deleting anything we find, + // but we are only allowed to remove symlinks, anything + // else and MountMgr ignores the device. + ObDereferenceObject(fileObject); + + if (MOUNTMGR_IS_DRIVE_LETTER(&mountpath)) { + + zfs_remove_driveletter(zmo); + + } else { + // If mount uses reparsepoint (not driveletter) + OBJECT_ATTRIBUTES poa; + + InitializeObjectAttributes(&poa, + &zmo_dcb->mountpoint, OBJ_KERNEL_HANDLE, NULL, + NULL); + dprintf("Deleting reparse mountpoint '%wZ'\n", + &zmo_dcb->mountpoint); + DeleteReparsePoint(&poa); + + // Remove directory, only for !driveletter + ZwDeleteFile(&poa); + } + + KIRQL irql; + IoAcquireVpbSpinLock(&irql); + zmo->vpb->Flags &= ~VPB_MOUNTED; + zmo_dcb->vpb->Flags &= ~VPB_MOUNTED; + zmo->vpb->Flags |= VPB_DIRECT_WRITES_ALLOWED; + zmo->vpb->DeviceObject = NULL; + IoReleaseVpbSpinLock(irql); + + // Release any notifications +#if (NTDDI_VERSION >= NTDDI_VISTA) + FsRtlNotifyCleanupAll(zmo->NotifySync, &zmo->DirNotifyList); +#endif + + // This will make it try to mount again, so make sure we dont + + status = SendVolumeRemovalNotification(&zmo_dcb->device_name); + + /* + * We call mount on DCB, but shouldn't it be VCB? We + * match unmount on DCB here so vflush can compare. + * DCB and VCB do have almost the same information, but + * it is probably more correct to change mount to use VCB. + */ + error = zfs_vfs_unmount(zmo, 0, NULL); + dprintf("%s: zfs_vfs_unmount %d\n", __func__, error); + if (error) + goto out_unlock; + + // Release devices + IoDeleteSymbolicLink(&zmo->symlink_name); + + // fsDeviceObject + if (zmo->deviceObject) { + // For some reason IODetachDevice can cause BSOD + // IoDetachDevice(zmo->deviceObject); + IoDeleteDevice(zmo->deviceObject); + } + + // diskDeviceObject + if (zmo->diskDeviceObject) + IoDeleteDevice(zmo->diskDeviceObject); + + zfs_release_mount(zmo); + + // There should also be a diskDevice above us to release. + if (zmo_dcb != NULL) { + if (zmo_dcb->deviceObject) + IoDeleteDevice(zmo_dcb->deviceObject); + if (zmo_dcb->diskDeviceObject) + IoDeleteDevice(zmo_dcb->diskDeviceObject); + zfs_release_mount(zmo_dcb); + } + + + error = 0; + +out_unlock: + // counter to getzfvfs + vfs_unbusy(zfsvfs->z_vfs); + } + return (error); +} diff --git a/module/os/windows/zfs/zfs_vnops_windows_xattr.c b/module/os/windows/zfs/zfs_vnops_windows_xattr.c new file mode 100644 index 000000000000..d16e184d53a6 --- /dev/null +++ b/module/os/windows/zfs/zfs_vnops_windows_xattr.c @@ -0,0 +1,1058 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2011, Lawrence Livermore National Security, LLC. + * + * Extended attributes (xattr) on Solaris are implemented as files + * which exist in a hidden xattr directory. These extended attributes + * can be accessed using the attropen() system call which opens + * the extended attribute. It can then be manipulated just like + * a standard file descriptor. This has a couple advantages such + * as practically no size limit on the file, and the extended + * attributes permissions may differ from those of the parent file. + * This interface is really quite clever, but it's also completely + * different than what is supported on Linux. It also comes with a + * steep performance penalty when accessing small xattrs because they + * are not stored with the parent file. + * + * Under Linux extended attributes are manipulated by the system + * calls getxattr(2), setxattr(2), and listxattr(2). They consider + * extended attributes to be name/value pairs where the name is a + * NULL terminated string. The name must also include one of the + * following namespace prefixes: + * + * user - No restrictions and is available to user applications. + * trusted - Restricted to kernel and root (CAP_SYS_ADMIN) use. + * system - Used for access control lists (system.nfs4_acl, etc). + * security - Used by SELinux to store a files security context. + * + * The value under Linux to limited to 65536 bytes of binary data. + * In practice, individual xattrs tend to be much smaller than this + * and are typically less than 100 bytes. A good example of this + * are the security.selinux xattrs which are less than 100 bytes and + * exist for every file when xattr labeling is enabled. + * + * The Linux xattr implementation has been written to take advantage of + * this typical usage. When the dataset property 'xattr=sa' is set, + * then xattrs will be preferentially stored as System Attributes (SA). + * This allows tiny xattrs (~100 bytes) to be stored with the dnode and + * up to 64k of xattrs to be stored in the spill block. If additional + * xattr space is required, which is unlikely under Linux, they will + * be stored using the traditional directory approach. + * + * This optimization results in roughly a 3x performance improvement + * when accessing xattrs because it avoids the need to perform a seek + * for every xattr value. When multiple xattrs are stored per-file + * the performance improvements are even greater because all of the + * xattrs stored in the spill block will be cached. + * + * However, by default SA based xattrs are disabled in the Linux port + * to maximize compatibility with other implementations. If you do + * enable SA based xattrs then they will not be visible on platforms + * which do not support this feature. + * + * NOTE: One additional consequence of the xattr directory implementation + * is that when an extended attribute is manipulated an inode is created. + * This inode will exist in the Linux inode cache but there will be no + * associated entry in the dentry cache which references it. This is + * safe but it may result in some confusion. Enabling SA based xattrs + * largely avoids the issue except in the overflow case. + */ +/* + * Large Block Coment about differences on Windows goes here, etc. + */ + +#include +#include +#include +#include +#include +#include +#include +// #include + +// Windows has no concept of these, it will always replace +// existing xattrs. Callers will need to check for existence +// by hand. +#define XATTR_CREATE 0 +#define XATTR_REPLACE 0 + +#define XATTR_USER_PREFIX "windows:" +#define XATTR_USER_PREFIX_LEN strlen("windows:") + +void +strlower(char *s) +{ + while (*s != 0) + *s++ = tolower(*s); +} + +enum xattr_permission { + XAPERM_DENY, + XAPERM_ALLOW, + XAPERM_COMPAT, +}; + +static unsigned int zfs_xattr_compat = 0; + +static enum xattr_permission +zpl_xattr_permission(struct vnode *dvp, zfs_uio_t *uio, const char *name, + int name_len) +{ + if (xattr_protected(name)) + return (XAPERM_DENY); + if (xattr_stream(name)) + return (XAPERM_DENY); + + return (zfs_xattr_compat ? XAPERM_COMPAT : XAPERM_ALLOW); +} + + + +/* + * Insert an EA into an output buffer, if there is room, + * EAName is always the FULL name length, even when we only + * fit partial. + * Return 0 for OK, 1 for overflow. + * + * Windows can not just return a list of EAs, the caller will + * always get the EA values as well. + * + * Windows fills in previous EA record with the offset to this + * entry, so we pass previous_ea PTR along. + * + * This function is not static here, as it is called from other + * files, in particular as the QUERY_EAS can be called with a + * list of EAs to process. + * + * Windows will also supply the starting_index when resuming a + * listing process. + * + * This can be called with vp == NULL, be mindful. + * + */ +int +zpl_xattr_filldir(struct vnode *dvp, zfs_uio_t *uio, const char *name, + int name_len, FILE_FULL_EA_INFORMATION **previous_ea) +{ + // The first xattr struct we assume is already aligned, but further ones + // should be padded here. + FILE_FULL_EA_INFORMATION *ea = NULL; + int overflow = 0; + uint64_t needed_total = 0; + int error = 0; + ssize_t retsize; + + enum xattr_permission perm; + + /* Check permissions using the per-namespace list xattr handler. */ + perm = zpl_xattr_permission(dvp, uio, name, name_len); + if (perm == XAPERM_DENY) + return (0); + + /* If it starts with "windows:", skip past it. */ + if (perm != XAPERM_COMPAT) { + if (name_len >= XATTR_USER_PREFIX_LEN && + strncmp(XATTR_USER_PREFIX, name, + XATTR_USER_PREFIX_LEN) == 0) { + name += XATTR_USER_PREFIX_LEN; + name_len -= XATTR_USER_PREFIX_LEN; + } + } + + // If not first struct, align outsize to 4 bytes - 0 aligns to 0. + uint64_t alignment; + uint64_t spaceused; + spaceused = zfs_uio_offset(uio); + alignment = (((spaceused) + 3) & ~3) - (spaceused); + + // EaValueLength is a USHORT, so cap xattrs (in this IRP) + // to that max. + uint32_t readmax; + readmax = 0; + + // Get size of EA, if we can + if (dvp && zpl_xattr_get(dvp, name, NULL, + &retsize, NULL) == 0) + readmax = MIN(retsize, 0x0000ffff); + + // Since we would only return whole records, work out how much space + // we need, so we can return it, and if things still fit, fill it in + needed_total = + alignment + + FIELD_OFFSET(FILE_FULL_EA_INFORMATION, EaName) + + name_len + 1; + // + readmax; + + /* When resid is 0 only calculate the required size. */ + if (uio == NULL || zfs_uio_resid(uio) == 0) { + zfs_uio_setoffset(uio, zfs_uio_offset(uio) + + needed_total + readmax); + return (0); + } + + /* Will it fit */ + if (needed_total + readmax > zfs_uio_resid(uio)) + return (STATUS_BUFFER_OVERFLOW); + + unsigned char *outbuffer; + outbuffer = zfs_uio_iovbase(uio, 0); + + // the data should now fit. + ea = &outbuffer[zfs_uio_offset(uio) + alignment]; + + // Room for one more struct, update previous's next ptr + if (*previous_ea != NULL) { + // Update previous structure to point to this one. + (*previous_ea)->NextEntryOffset = + (uintptr_t)ea - (uintptr_t)(*previous_ea); + } + + // Directly set next to 0, assuming this will be last record + ea->NextEntryOffset = 0; + ea->Flags = 0; // Fix me? + ea->EaValueLength = 0; + + // remember this EA, so the next one + // can fill in the offset. + *previous_ea = ea; + + // Return the total name length not counting null + ea->EaNameLength = name_len; + + // Now copy out as much of the filename as can fit. + // We need to real full length in StreamNameLength + // There is always room for 1 char + strlcpy(ea->EaName, name, name_len + 1); + + // Windows test.exe require uppercase, after the lookups + if (dvp && VTOZ(dvp)->z_zfsvfs->z_case == ZFS_CASE_INSENSITIVE) + strupper(ea->EaName, name_len + 1); + + // We need to move uio forward by the amount that ea and name takes up. + zfs_uio_advance(uio, needed_total); + + // Now add the value, if there is one. + if (dvp != NULL) { + + // MSN: value(s) associated with each entry follows + // the EaName array. + // That is, an EA's values are located at + // EaName + (EaNameLength + 1). + + // zfs_read(VTOZ(vp), &uio, 0, NULL); + + error = zpl_xattr_get(dvp, name, uio, + &retsize, NULL); + + // Set the valuelen, should this be the full + // value or what we would need? + // That is how the names work. + // ea->EaValueLength = VTOZ(vp)->z_size; + ea->EaValueLength = retsize; + + } + + dprintf("%s: added xattrname '%s'\n", __func__, + name); + + return (error); +} + +/* + * Read as many directory entry names as will fit in to the provided buffer, + * or when no buffer is provided calculate the required buffer size. + */ +static int +zpl_xattr_readdir(struct vnode *dxip, struct vnode *dvp, zfs_uio_t *uio) +{ + zap_cursor_t zc; + zap_attribute_t zap; + int error; + FILE_FULL_EA_INFORMATION *previous_ea = NULL; + + int start_index = zfs_uio_index(uio); + + // For some reason zap_cursor_init_serialized doesnt work +// if (start_index == 0) + zap_cursor_init(&zc, ITOZSB(dxip)->z_os, ITOZ(dxip)->z_id); +// else +// zap_cursor_init_serialized(&zc, +// ITOZSB(dxip)->z_os, ITOZ(dxip)->z_id, +// start_index); + int current_index = 0; + + while ((error = zap_cursor_retrieve(&zc, &zap)) == 0) { + + if (zap.za_integer_length != 8 || zap.za_num_integers != 1) { + error = STATUS_EA_CORRUPT_ERROR; + break; + } + + if (current_index >= start_index) { + error = zpl_xattr_filldir(dvp, uio, + zap.za_name, strlen(zap.za_name), &previous_ea); + + if (error) + break; + + if (BooleanFlagOn(uio->uio_extflg, + SL_RETURN_SINGLE_ENTRY)) { + current_index++; + break; + } + } + + current_index++; + zap_cursor_advance(&zc); + } + + zap_cursor_fini(&zc); + + if (error == ENOENT) + error = 0; + + if (current_index >= start_index) + zfs_uio_setindex(uio, current_index); + return (error); +} + +static ssize_t +zpl_xattr_list_dir(struct vnode *dvp, zfs_uio_t *uio, cred_t *cr) +{ + struct vnode *dxip = NULL; + znode_t *dxzp; + int error; + + /* Lookup the xattr directory */ + error = zfs_lookup(ITOZ(dvp), NULL, &dxzp, LOOKUP_XATTR, + cr, NULL, NULL); + if (error) { + if (error == ENOENT) + error = 0; + return (error); + } + + dxip = ZTOI(dxzp); + error = zpl_xattr_readdir(dxip, dvp, uio); + VN_RELE(dxip); + + return (error); +} + +static ssize_t +zpl_xattr_list_sa(struct vnode *dvp, zfs_uio_t *uio) +{ + znode_t *zp = ITOZ(dvp); + nvpair_t *nvp = NULL; + int error = 0; + FILE_FULL_EA_INFORMATION *previous_ea = NULL; + + mutex_enter(&zp->z_lock); + if (zp->z_xattr_cached == NULL) + error = zfs_sa_get_xattr(zp); + mutex_exit(&zp->z_lock); + + if (error) + return (error); + + int start_index = zfs_uio_index(uio); + + ASSERT(zp->z_xattr_cached); + + int current_index = 0; + + while ((nvp = nvlist_next_nvpair(zp->z_xattr_cached, nvp)) != NULL) { + ASSERT3U(nvpair_type(nvp), ==, DATA_TYPE_BYTE_ARRAY); + + if (current_index++ < start_index) + continue; + + error = zpl_xattr_filldir(dvp, uio, nvpair_name(nvp), + strlen(nvpair_name(nvp)), &previous_ea); + if (error) + return (error); + + if (BooleanFlagOn(uio->uio_extflg, SL_RETURN_SINGLE_ENTRY)) + break; + } + + if (current_index > start_index) + zfs_uio_setindex(uio, current_index); + + return (0); +} + +int +zpl_xattr_list(struct vnode *dvp, zfs_uio_t *uio, ssize_t *size, cred_t *cr) +{ + znode_t *zp = ITOZ(dvp); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + int error = 0; + int start_index = 0; + + if (uio != NULL) + start_index = zfs_uio_index(uio); + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + rw_enter(&zp->z_xattr_lock, RW_READER); + + if (zfsvfs->z_use_sa && zp->z_is_sa) { + error = zpl_xattr_list_sa(dvp, uio); + if (error) + goto out; + if (uio && zfs_uio_index(uio) != start_index) { + zfs_uio_setindex(uio, 0); + } else { + start_index = 0; + } + } + + error = zpl_xattr_list_dir(dvp, uio, cr); + if (error) + goto out; + + // Add up the index of _sa and _dir + if (uio != NULL) + zfs_uio_setindex(uio, start_index + zfs_uio_index(uio)); + + if (size) + *size = zfs_uio_offset(uio); + +out: + + rw_exit(&zp->z_xattr_lock); + zfs_exit(zfsvfs, FTAG); + + return (error); +} + +static int +zpl_xattr_get_dir(struct vnode *ip, const char *name, zfs_uio_t *uio, + ssize_t *size, cred_t *cr) +{ + znode_t *dxzp = NULL; + znode_t *xzp = NULL; + int error; + + /* Lookup the xattr directory */ + error = zfs_lookup(ITOZ(ip), NULL, &dxzp, LOOKUP_XATTR, + cr, NULL, NULL); + if (error) + goto out; + + if (size) + *size = 0; // NA + + /* Lookup a specific xattr name in the directory */ + char namebuffer[MAXNAMELEN]; + strlcpy(namebuffer, name, sizeof (namebuffer)); + // Dont forget zfs_lookup() modifies + // "cn" here, so size needs to be max, if + // formD is in effect. + // FIGNORECASE needs "cn" to work, and Windows + // expects case-insensitive + struct componentname cn; + cn.cn_nameiop = LOOKUP; + cn.cn_flags = ISLASTCN; + cn.cn_namelen = strlen(namebuffer); + cn.cn_nameptr = namebuffer; + cn.cn_pnlen = MAXNAMELEN; + cn.cn_pnbuf = namebuffer; + int flags = 0; + + if (ip && VTOZ(ip)->z_zfsvfs->z_case == ZFS_CASE_INSENSITIVE) + flags = FIGNORECASE; + + error = zfs_lookup(dxzp, (char *)name, &xzp, flags, cr, NULL, &cn); + if (error) + goto out; + + if (size) + *size = xzp->z_size; + + if (uio == NULL || zfs_uio_resid(uio) == 0) + goto out; + + if (zfs_uio_resid(uio) < xzp->z_size) { + error = STATUS_BUFFER_OVERFLOW; + goto out; + } + + uint64_t resid_before = zfs_uio_resid(uio); + + // This advanced uio for us. zfsread() can not + // handle offset != 0, so setup a new uio + if (zfs_uio_offset(uio) != 0ULL) { + struct iovec iov; + iov.iov_base = + (void *)(((uint64_t)zfs_uio_iovbase(uio, 0)) + + zfs_uio_offset(uio)); + iov.iov_len = xzp->z_size; + zfs_uio_t tuio; + zfs_uio_iovec_init(&tuio, &iov, 1, 0, + UIO_SYSSPACE, xzp->z_size, 0); + + error = zfs_read(xzp, &tuio, 0, cr); + + zfs_uio_advance(uio, xzp->z_size - zfs_uio_resid(&tuio)); + + } else { + + error = zfs_read(xzp, uio, 0, cr); + } + + if (size) + *size = (resid_before - zfs_uio_resid(uio)); + +out: + if (xzp) + zrele(xzp); + + if (dxzp) + zrele(dxzp); + + return (error); +} + +static int +zpl_xattr_get_sa(struct vnode *ip, const char *name, zfs_uio_t *uio, + ssize_t *size) +{ + znode_t *zp = ITOZ(ip); + uchar_t *nv_value; + uint_t nv_size; + int error = 0; + char *lowername = NULL; + + ASSERT(RW_LOCK_HELD(&zp->z_xattr_lock)); + + if (size) + *size = 0; // NA + + mutex_enter(&zp->z_lock); + if (zp->z_xattr_cached == NULL) + error = zfs_sa_get_xattr(zp); + mutex_exit(&zp->z_lock); + + if (error) + return (error); + + /* nvlist has no FIGNORECASE */ + if (zp->z_zfsvfs->z_case == ZFS_CASE_INSENSITIVE) { + lowername = spa_strdup(name); + strlower(lowername); + name = lowername; + } + + ASSERT(zp->z_xattr_cached); + error = nvlist_lookup_byte_array(zp->z_xattr_cached, name, + &nv_value, &nv_size); + + if (lowername) + spa_strfree(lowername); + + if (error) + return (error); + + if (size) + *size = nv_size; + + if (uio == NULL || zfs_uio_resid(uio) == 0) + return (0); + + if (zfs_uio_resid(uio) < nv_size) + return (STATUS_BUFFER_OVERFLOW); + + // uiomove uses skip and not offset to start. + zfs_uio_setskip(uio, zfs_uio_offset(uio)); + zfs_uiomove(nv_value, nv_size, UIO_READ, uio); + zfs_uio_setskip(uio, 0); + + if (size) + *size = nv_size; + + return (0); +} + +static int +__zpl_xattr_get(struct vnode *ip, const char *name, zfs_uio_t *uio, + ssize_t *retsize, cred_t *cr) +{ + znode_t *zp = ITOZ(ip); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + int error; + + ASSERT(RW_LOCK_HELD(&zp->z_xattr_lock)); + + if (zfsvfs->z_use_sa && zp->z_is_sa) { + error = zpl_xattr_get_sa(ip, name, uio, retsize); + if (error != ENOENT) + goto out; + } + + error = zpl_xattr_get_dir(ip, name, uio, retsize, cr); + +out: + if (error == ENOENT) + error = STATUS_NO_EAS_ON_FILE; + + return (error); +} + +#define XATTR_NOENT 0x0 +#define XATTR_IN_SA 0x1 +#define XATTR_IN_DIR 0x2 +/* check where the xattr resides */ +static int +__zpl_xattr_where(struct vnode *ip, const char *name, int *where, cred_t *cr) +{ + znode_t *zp = ITOZ(ip); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + int error; + ssize_t retsize; + + ASSERT(where); + ASSERT(RW_LOCK_HELD(&zp->z_xattr_lock)); + + *where = XATTR_NOENT; + if (zfsvfs->z_use_sa && zp->z_is_sa) { + error = zpl_xattr_get_sa(ip, name, NULL, &retsize); + if (error == 0) + *where |= XATTR_IN_SA; + else if (error != ENOENT) + return (error); + } + + error = zpl_xattr_get_dir(ip, name, NULL, &retsize, cr); + if (error == 0) + *where |= XATTR_IN_DIR; + else if (error != ENOENT) + return (error); + + if (*where == (XATTR_IN_SA|XATTR_IN_DIR)) + cmn_err(CE_WARN, "ZFS: inode %p has xattr \"%s\"" + " in both SA and dir", ip, name); + if (*where == XATTR_NOENT) + error = STATUS_NO_EAS_ON_FILE; + else + error = 0; + + return (error); +} + +int +zpl_xattr_get(struct vnode *ip, const char *name, zfs_uio_t *uio, + ssize_t *retsize, cred_t *cr) +{ + znode_t *zp = ITOZ(ip); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + int error; + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + return (error); + + rw_enter(&zp->z_xattr_lock, RW_READER); + /* + * Try to look up the name with the namespace prefix first for + * compatibility with xattrs from this platform. If that fails, + * try again without the namespace prefix for compatibility with + * other platforms. + */ + char *xattr_name = kmem_asprintf("%s%s", XATTR_USER_PREFIX, name); + error = __zpl_xattr_get(ip, xattr_name, uio, retsize, cr); + kmem_strfree(xattr_name); + + if (error == STATUS_NO_EAS_ON_FILE) + error = __zpl_xattr_get(ip, name, uio, retsize, cr); + + rw_exit(&zp->z_xattr_lock); + zfs_exit(zfsvfs, FTAG); + + return (error); +} + +static int +zpl_xattr_set_dir(struct vnode *ip, const char *name, zfs_uio_t *uio, + int flags, cred_t *cr) +{ + znode_t *dxzp = NULL; + znode_t *xzp = NULL; + int lookup_flags, error; + const int xattr_mode = S_IFREG | 0644; + + /* + * Lookup the xattr directory. When we're adding an entry pass + * CREATE_XATTR_DIR to ensure the xattr directory is created. + * When removing an entry this flag is not passed to avoid + * unnecessarily creating a new xattr directory. + */ + lookup_flags = LOOKUP_XATTR; + if (uio != NULL) + lookup_flags |= CREATE_XATTR_DIR; + + error = zfs_lookup(ITOZ(ip), NULL, &dxzp, lookup_flags, + cr, NULL, NULL); + if (error) + goto out; + + /* Lookup a specific xattr name in the directory */ + error = zfs_lookup(dxzp, (char *)name, &xzp, 0, cr, NULL, NULL); + + if (error && (error != ENOENT)) + goto out; + + error = 0; + + /* Remove a specific name xattr when value is set to NULL. */ + if (uio == NULL) { + if (xzp) + error = zfs_remove(dxzp, (char *)name, cr, 0); + + goto out; + } + + /* Lookup failed create a new xattr. */ + if (xzp == NULL) { + + /* Set va_size and 0 - skip zfs_freesp below? */ + vattr_t vattr = { 0 }; + vattr.va_type = VREG; + vattr.va_mode = xattr_mode; + // vattr.va_uid = crgetfsuid(cr); + // vattr.va_gid = crgetfsgid(cr); + vattr.va_mask = ATTR_TYPE | ATTR_MODE; + + error = zfs_create(dxzp, (char *)name, &vattr, 0, + xattr_mode, &xzp, cr, 0, NULL, NULL); + if (error) + goto out; + } + + ASSERT(xzp != NULL); + + error = zfs_freesp(xzp, 0, 0, xattr_mode, TRUE); + if (error) + goto out; +#if 0 + /* + * For some reason our zfs_read() and zfs_write() + * do not handle uio_offset, or rather, it means + * something else. + */ + struct iovec iov; + iov.iov_base = + (void *)(((uint64_t)zfs_uio_iovbase(uio, 0)) + zfs_uio_offset(uio)); + iov.iov_len = zfs_uio_resid(uio); + + zfs_uio_t tuio; + zfs_uio_iovec_init(&tuio, &iov, 1, 0, + UIO_SYSSPACE, zfs_uio_resid(uio), 0); + + error = zfs_write(xzp, &tuio, 0, cr); +#else + error = zfs_write(xzp, uio, 0, cr); +#endif + +out: + if (error == 0) { + /* + * ip->z_ctime = current_time(ip); + * zfs_mark_inode_dirty(ip); + */ + } + + if (xzp) + zrele(xzp); + + if (dxzp) + zrele(dxzp); + + if (error == ENOENT) + error = STATUS_NO_EAS_ON_FILE; + + return (error); +} + +static int +zpl_xattr_set_sa(struct vnode *ip, const char *name, zfs_uio_t *uio, + int flags, cred_t *cr) +{ + znode_t *zp = ITOZ(ip); + nvlist_t *nvl; + size_t sa_size; + int error = 0; + void *buf = NULL; + int len = 0; + size_t used; + int allocated = 0; + char *lowername = NULL; + + mutex_enter(&zp->z_lock); + if (zp->z_xattr_cached == NULL) + error = zfs_sa_get_xattr(zp); + mutex_exit(&zp->z_lock); + + if (error) + return (error); + + /* + * We have to be careful not to "consume" the uio, + * in the error cases, as it is to be used next in + * xattr=dir. + * This only supports "one iovec" for the data + */ + if (uio != NULL) { + buf = zfs_uio_iovbase(uio, 0); + len = zfs_uio_iovlen(uio, 0); + } + + ASSERT(zp->z_xattr_cached); + nvl = zp->z_xattr_cached; + + /* nvlist has no FIGNORECASE */ + if (zp->z_zfsvfs->z_case == ZFS_CASE_INSENSITIVE) { + lowername = spa_strdup(name); + strlower(lowername); + name = lowername; + } + + if (uio == NULL) { + error = nvlist_remove(nvl, name, DATA_TYPE_BYTE_ARRAY); + if (error == ENOENT) + error = zpl_xattr_set_dir(ip, name, NULL, flags, cr); + } else { + /* Limited to 32k to keep nvpair memory allocations small */ + if (zfs_uio_resid(uio) > DXATTR_MAX_ENTRY_SIZE) { + if (lowername) + spa_strfree(lowername); + return (STATUS_EA_TOO_LARGE); + } + + /* Prevent the DXATTR SA from consuming the entire SA region */ + error = nvlist_size(nvl, &sa_size, NV_ENCODE_XDR); + if (error) { + if (lowername) + spa_strfree(lowername); + return (error); + } + + if (sa_size > DXATTR_MAX_SA_SIZE) { + if (lowername) + spa_strfree(lowername); + return (STATUS_EA_TOO_LARGE); + } + + /* + * Allocate memory to copyin, which is a shame as nvlist + * will also allocate memory to hold it. Could consider a + * nvlist_add_byte_array_uio() so the memcpy(KM_SLEEP); + * zfs_uiomove(buf, ) uses uiomove() + * instead. + */ + if (zfs_uio_segflg(uio) != UIO_SYSSPACE) { + allocated = 1; + buf = kmem_alloc(len, KM_SLEEP); + /* Don't consume uio yet; uiocopy, not uiomove */ + zfs_uiocopy(buf, len, UIO_WRITE, uio, &used); + } + + error = nvlist_add_byte_array(nvl, name, + (uchar_t *)buf, len); + + /* Free this after zfs_sa_set_xattr() */ + + } + + /* + * Update the SA for additions, modifications, and removals. On + * error drop the inconsistent cached version of the nvlist, it + * will be reconstructed from the ARC when next accessed. + */ + if (error == 0) { + error = zfs_sa_set_xattr(zp, name, buf, len); + } + + if (lowername) + spa_strfree(lowername); + + if (allocated == 1) + kmem_free(buf, len); + + if (error) { + nvlist_free(nvl); + zp->z_xattr_cached = NULL; + } else if (uio != NULL) { + /* Finally consume uio */ + zfs_uio_advance(uio, len); + } + + return (error); +} + +static int +_zpl_xattr_set(struct vnode *ip, const char *name, zfs_uio_t *uio, int flags, + cred_t *cr) +{ + znode_t *zp = ITOZ(ip); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + int where; + int error; + + if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) + goto out1; + rw_enter(&zp->z_xattr_lock, RW_WRITER); + + /* + * Before setting the xattr check to see if it already exists. + * This is done to ensure the following optional flags are honored. + * + * XATTR_CREATE: fail if xattr already exists + * XATTR_REPLACE: fail if xattr does not exist + * + * We also want to know if it resides in sa or dir, so we can make + * sure we don't end up with duplicate in both places. + */ + + error = __zpl_xattr_where(ip, name, &where, cr); + if (error != 0) { + if (error != STATUS_NO_EAS_ON_FILE) + goto out; + if (flags & XATTR_REPLACE) + goto out; + + /* The xattr to be removed already doesn't exist */ + error = 0; + } else { + error = EEXIST; + if (flags & XATTR_CREATE) + goto out; + } + + /* + * Preferentially store the xattr as a SA for better performance + * We must not do this for "com.apple.ResourceFork" as it can + * be opened as a vnode. + */ + if (zfsvfs->z_use_sa && zp->z_is_sa && + (zfsvfs->z_xattr_sa || + (uio == NULL && where & XATTR_IN_SA))) { + error = zpl_xattr_set_sa(ip, name, uio, flags, cr); + if (error == 0) { + /* + * Successfully put into SA, we need to clear the one + * in dir. + */ + if (where & XATTR_IN_DIR) + zpl_xattr_set_dir(ip, name, NULL, 0, cr); + goto out; + } + } + + error = zpl_xattr_set_dir(ip, name, uio, flags, cr); + + /* + * Successfully put into dir, we need to clear the one in SA. + */ + if (error == 0 && (where & XATTR_IN_SA)) + zpl_xattr_set_sa(ip, name, NULL, 0, cr); +out: + rw_exit(&zp->z_xattr_lock); + zfs_exit(zfsvfs, FTAG); +out1: + + return (error); +} + +/* + * Return an allocated (caller free()s) string potentially prefixed + * based on the xattr_compat tunable. + */ +const char * +zpl_xattr_prefixname(const char *name) +{ + if (zfs_xattr_compat) { + return (spa_strdup(name)); + } else { + return (kmem_asprintf("%s%s", XATTR_USER_PREFIX, name)); + } +} + +int +zpl_xattr_set(struct vnode *ip, const char *name, zfs_uio_t *uio, int flags, + cred_t *cr) +{ + int error; + + /* + * Remove alternate compat version of the xattr so we only set the + * version specified by the zfs_xattr_compat tunable. + * + * The following flags must be handled correctly: + * + * XATTR_CREATE: fail if xattr already exists + * XATTR_REPLACE: fail if xattr does not exist + */ + + char *prefixed_name = kmem_asprintf("%s%s", XATTR_USER_PREFIX, name); + const char *clear_name, *set_name; + if (zfs_xattr_compat) { + clear_name = prefixed_name; + set_name = name; + } else { + clear_name = name; + set_name = prefixed_name; + } + + /* + * Clear the old value with the alternative name format, if it exists. + */ + error = _zpl_xattr_set(ip, clear_name, NULL, flags, cr); + + /* + * XATTR_CREATE was specified and we failed to clear the xattr + * because it already exists. Stop here. + */ + if (error == EEXIST) + goto out; + + /* + * If XATTR_REPLACE was specified and we succeeded to clear + * an xattr, we don't need to replace anything when setting + * the new value. If we failed with -ENODATA that's fine, + * there was nothing to be cleared and we can ignore the error. + */ + if (error == 0) + flags &= ~XATTR_REPLACE; + + /* + * Set the new value with the configured name format. + */ + error = _zpl_xattr_set(ip, set_name, uio, flags, cr); +out: + kmem_strfree(prefixed_name); + return (error); +} + + +ZFS_MODULE_PARAM(zfs, zfs_, xattr_compat, UINT, ZMOD_RW, + "Use legacy ZFS xattr naming for writing new user namespace xattrs"); diff --git a/module/os/windows/zfs/zfs_windows_zvol.c b/module/os/windows/zfs/zfs_windows_zvol.c new file mode 100644 index 000000000000..d428c5cfb850 --- /dev/null +++ b/module/os/windows/zfs/zfs_windows_zvol.c @@ -0,0 +1,1095 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2019 Jorgen Lundman + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +extern PDRIVER_OBJECT WIN_DriverObject; +static pHW_HBA_EXT STOR_HBAExt = NULL; + + +// Verbose + +int +zvol_start(PDRIVER_OBJECT DriverObject, PUNICODE_STRING pRegistryPath) +{ + pwzvolDriverInfo pwzvolDrvInfo; + NTSTATUS status; + VIRTUAL_HW_INITIALIZATION_DATA hwInitData; + + RtlZeroMemory(&STOR_wzvolDriverInfo, sizeof (STOR_wzvolDriverInfo)); + pwzvolDrvInfo = &STOR_wzvolDriverInfo; + + RtlZeroMemory(pwzvolDrvInfo, sizeof (wzvolDriverInfo)); + pwzvolDrvInfo->pDriverObj = DriverObject; + + KeInitializeSpinLock(&pwzvolDrvInfo->DrvInfoLock); + KeInitializeSpinLock(&pwzvolDrvInfo->MPIOExtLock); + KeInitializeSpinLock(&pwzvolDrvInfo->SrbExtLock); + + InitializeListHead(&pwzvolDrvInfo->ListMPHBAObj); + InitializeListHead(&pwzvolDrvInfo->ListMPIOExt); + InitializeListHead(&pwzvolDrvInfo->ListSrbExt); + + pwzvolDrvInfo->wzvolRegInfo.BreakOnEntry = DEFAULT_BREAK_ON_ENTRY; + pwzvolDrvInfo->wzvolRegInfo.DebugLevel = DEFAULT_DEBUG_LEVEL; + pwzvolDrvInfo->wzvolRegInfo.InitiatorID = DEFAULT_INITIATOR_ID; + pwzvolDrvInfo->wzvolRegInfo.PhysicalDiskSize = + DEFAULT_PHYSICAL_DISK_SIZE; + pwzvolDrvInfo->wzvolRegInfo.VirtualDiskSize = DEFAULT_VIRTUAL_DISK_SIZE; + pwzvolDrvInfo->wzvolRegInfo.NbrVirtDisks = DEFAULT_NbrVirtDisks; + + pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperHBA = DEFAULT_NbrLUNsperHBA; + pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperTarget = DEFAULT_NbrLUNsperTarget; + pwzvolDrvInfo->wzvolRegInfo.bCombineVirtDisks = + DEFAULT_bCombineVirtDisks; + + RtlInitUnicodeString(&pwzvolDrvInfo->wzvolRegInfo.VendorId, VENDOR_ID); + RtlInitUnicodeString(&pwzvolDrvInfo->wzvolRegInfo.ProductId, + PRODUCT_ID); + RtlInitUnicodeString(&pwzvolDrvInfo->wzvolRegInfo.ProductRevision, + PRODUCT_REV); + + // Calculate the combination of busses, targets and Luns to fit + // the NbrLUNsperHBA requirement. We privilege the maximum amount + // of targets vs. luns so TARGET RESETs don't affect a bunch of LUNs + // + if ((pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperHBA / + pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperTarget) > + SCSI_MAXIMUM_TARGETS_PER_BUS) + pwzvolDrvInfo->MaximumNumberOfTargets = + SCSI_MAXIMUM_TARGETS_PER_BUS; + else + pwzvolDrvInfo->MaximumNumberOfTargets = + (pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperHBA / + pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperTarget) + + (pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperHBA % + pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperTarget ? 1 : 0); + + // pwzvolDrvInfo->MaximumNumberOfTargets = + // (pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperHBA <= + // SCSI_MAXIMUM_TARGETS_PER_BUS ? + // pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperHBA : + // SCSI_MAXIMUM_TARGETS_PER_BUS); + pwzvolDrvInfo->MaximumNumberOfLogicalUnits = + (pwzvolDrvInfo->wzvolRegInfo.NbrLUNsperHBA / + pwzvolDrvInfo->MaximumNumberOfTargets) + 1; + // supporting more would mean bigger changes in the zv_targets + // array. now we can go up to 32,640 zvols. + pwzvolDrvInfo->NumberOfBuses = 1; + pwzvolDrvInfo->zvContextArray = + (wzvolContext*)ExAllocatePoolWithTag(NonPagedPoolNx, + ((SIZE_T)pwzvolDrvInfo->MaximumNumberOfTargets * + pwzvolDrvInfo->MaximumNumberOfLogicalUnits * + sizeof (wzvolContext)), MP_TAG_GENERAL); + if (pwzvolDrvInfo->zvContextArray == NULL) + return (STATUS_NO_MEMORY); + + RtlZeroMemory(pwzvolDrvInfo->zvContextArray, + ((SIZE_T)pwzvolDrvInfo->MaximumNumberOfTargets * + pwzvolDrvInfo->MaximumNumberOfLogicalUnits * + (sizeof (wzvolContext)))); + + RtlZeroMemory(&hwInitData, sizeof (VIRTUAL_HW_INITIALIZATION_DATA)); + + hwInitData.HwInitializationDataSize = + sizeof (VIRTUAL_HW_INITIALIZATION_DATA); + + hwInitData.HwInitialize = wzvol_HwInitialize; + hwInitData.HwStartIo = wzvol_HwStartIo; + hwInitData.HwFindAdapter = wzvol_HwFindAdapter; + hwInitData.HwResetBus = wzvol_HwResetBus; + hwInitData.HwAdapterControl = wzvol_HwAdapterControl; + hwInitData.HwFreeAdapterResources = wzvol_HwFreeAdapterResources; + hwInitData.HwInitializeTracing = wzvol_TracingInit; + hwInitData.HwCleanupTracing = wzvol_TracingCleanup; + hwInitData.HwProcessServiceRequest = wzvol_ProcServReq; + hwInitData.HwCompleteServiceIrp = wzvol_CompServReq; + + hwInitData.AdapterInterfaceType = Internal; + + hwInitData.DeviceExtensionSize = sizeof (HW_HBA_EXT); + hwInitData.SpecificLuExtensionSize = sizeof (HW_LU_EXTENSION); + hwInitData.SrbExtensionSize = + sizeof (HW_SRB_EXTENSION) + IoSizeofWorkItem(); + + hwInitData.TaggedQueuing = TRUE; + hwInitData.AutoRequestSense = TRUE; + hwInitData.MultipleRequestPerLu = TRUE; + hwInitData.ReceiveEvent = TRUE; + + status = StorPortInitialize( + DriverObject, + pRegistryPath, + (PHW_INITIALIZATION_DATA)&hwInitData, + NULL); + + return (status); +} + +BOOLEAN +wzvol_HwInitialize(__in pHW_HBA_EXT pHBAExt) +{ + UNREFERENCED_PARAMETER(pHBAExt); + + dprintf("%s: entry\n", __func__); + return (TRUE); +} + +ULONG +wzvol_HwFindAdapter( + __in pHW_HBA_EXT pHBAExt, + __in PVOID pHwContext, + __in PVOID pBusInformation, + __in PVOID pLowerDO, + __in PCHAR pArgumentString, + __in __out PPORT_CONFIGURATION_INFORMATION pConfigInfo, + __in PBOOLEAN pBAgain) +{ + ULONG i, len, status = SP_RETURN_FOUND; + PCHAR pChar; + + dprintf("%s: entry\n", __func__); + +#if defined(_AMD64_) + + KLOCK_QUEUE_HANDLE LockHandle; + +#else + + KIRQL SaveIrql; + +#endif + + UNREFERENCED_PARAMETER(pHwContext); + UNREFERENCED_PARAMETER(pBusInformation); + UNREFERENCED_PARAMETER(pLowerDO); + UNREFERENCED_PARAMETER(pArgumentString); + + dprintf("%s: pHBAExt = 0x%p, pConfigInfo = 0x%p\n", __func__, + pHBAExt, pConfigInfo); + + // Copy master object from static variable. + pHBAExt->pwzvolDrvObj = &STOR_wzvolDriverInfo; + + if (STOR_HBAExt == NULL) { + // We save the first adapter only to announce. + STOR_HBAExt = pHBAExt; // So we can announce + pHBAExt->bDontReport = FALSE; + } else { + // If MPIO support is not requested we won't present + // the LUNs through other found adapters. + pHBAExt->bDontReport = + !STOR_wzvolDriverInfo.wzvolRegInfo.bCombineVirtDisks; + } + + InitializeListHead(&pHBAExt->MPIOLunList); + InitializeListHead(&pHBAExt->LUList); + + KeInitializeSpinLock(&pHBAExt->WkItemsLock); + KeInitializeSpinLock(&pHBAExt->WkRoutinesLock); + KeInitializeSpinLock(&pHBAExt->MPHBAObjLock); + KeInitializeSpinLock(&pHBAExt->LUListLock); + + pHBAExt->HostTargetId = + (UCHAR)pHBAExt->pwzvolDrvObj->wzvolRegInfo.InitiatorID; + + pHBAExt->pDrvObj = pHBAExt->pwzvolDrvObj->pDriverObj; + + pHBAExt->NbrLUNsperHBA = + pHBAExt->pwzvolDrvObj->wzvolRegInfo.NbrLUNsperHBA; + + pConfigInfo->VirtualDevice = TRUE; + pConfigInfo->WmiDataProvider = TRUE; + pConfigInfo->MaximumTransferLength = SP_UNINITIALIZED_VALUE; + pConfigInfo->NumberOfPhysicalBreaks = 0x21; // 128K IO size + pConfigInfo->AlignmentMask = 0x3; + pConfigInfo->CachesData = FALSE; + pConfigInfo->ScatterGather = TRUE; + pConfigInfo->MapBuffers = STOR_MAP_ALL_BUFFERS_INCLUDING_READ_WRITE; + pConfigInfo->SynchronizationModel = StorSynchronizeFullDuplex; + pConfigInfo->MaximumNumberOfLogicalUnits = + pHBAExt->pwzvolDrvObj->MaximumNumberOfLogicalUnits; + pConfigInfo->MaximumNumberOfTargets = + pHBAExt->pwzvolDrvObj->MaximumNumberOfTargets; + pConfigInfo->NumberOfBuses = pHBAExt->pwzvolDrvObj->NumberOfBuses; + + dprintf("%s: pHBAExt = 0x%p, NbBuses/MaxTargets/MaxLuns=%d/%d/%d.\n", + __func__, + pHBAExt, pConfigInfo->NumberOfBuses, + pConfigInfo->MaximumNumberOfTargets, + pConfigInfo->MaximumNumberOfLogicalUnits); + + // Save Vendor Id, Product Id, Revision in device extension. + + pChar = (PCHAR)pHBAExt->pwzvolDrvObj->wzvolRegInfo.VendorId.Buffer; + len = min(8, (pHBAExt->pwzvolDrvObj->wzvolRegInfo.VendorId.Length / 2)); + for (i = 0; i < len; i++, pChar += 2) + pHBAExt->VendorId[i] = *pChar; + + pChar = (PCHAR)pHBAExt->pwzvolDrvObj->wzvolRegInfo.ProductId.Buffer; + len = min(16, + (pHBAExt->pwzvolDrvObj->wzvolRegInfo.ProductId.Length / 2)); + for (i = 0; i < len; i++, pChar += 2) + pHBAExt->ProductId[i] = *pChar; + + pChar = (PCHAR)pHBAExt->pwzvolDrvObj-> + wzvolRegInfo.ProductRevision.Buffer; + len = min(4, + (pHBAExt->pwzvolDrvObj->wzvolRegInfo.ProductRevision.Length / 2)); + for (i = 0; i < len; i++, pChar += 2) + pHBAExt->ProductRevision[i] = *pChar; + + // Add HBA extension to master driver object's linked list. + +#if defined(_AMD64_) + KeAcquireInStackQueuedSpinLock(&pHBAExt->pwzvolDrvObj->DrvInfoLock, + &LockHandle); +#else + KeAcquireSpinLock(&pHBAExt->pwzvolDrvObj->DrvInfoLock, &SaveIrql); +#endif + InsertTailList(&pHBAExt->pwzvolDrvObj->ListMPHBAObj, &pHBAExt->List); + pHBAExt->pwzvolDrvObj->DrvInfoNbrMPHBAObj++; +#if defined(_AMD64_) + KeReleaseInStackQueuedSpinLock(&LockHandle); +#else + KeReleaseSpinLock(&pHBAExt->pwzvolDrvObj->DrvInfoLock, SaveIrql); +#endif + + InitializeWmiContext(pHBAExt); + + // *pBAgain = FALSE; // should not touch this. + + return (status); +} + + + +// Maximum WMIEvent size StorPort will support. +#define StorPortMaxWMIEventSize 0x80 +#define InstName L"ZVOL" + +void +wzvol_HwReportAdapter(__in pHW_HBA_EXT pHBAExt) +{ + dprintf("%s: entry\n", __func__); + +#if 1 + NTSTATUS status; + PWNODE_SINGLE_INSTANCE pWnode; + ULONG WnodeSize, + WnodeSizeInstanceName, + WnodeSizeDataBlock, + length, + size; + GUID lclGuid = MSFC_AdapterEvent_GUID; + UNICODE_STRING lclInstanceName; + UCHAR myPortWWN[8] = { 1, 2, 3, 4, 5, 6, 7, 8 }; + PMSFC_AdapterEvent pAdapterArr; + + // With the instance name used here and with the rounding-up + // to 4-byte alignment of the data portion used here, + // 0x34 (52) bytes are available for the actual data of the + // WMI event. (The 0x34 bytes result from the fact that StorPort + // at present (August 2008) allows 0x80 bytes for the entire + // WMIEvent (header, instance name and data); the header is 0x40 + // bytes; the instance name used here results in 0xA bytes, + // and the rounding up consumes 2 bytes; in other words, + // 0x80 - (0x40 + 0x0A + 0x02)). + + RtlInitUnicodeString(&lclInstanceName, InstName); + + WnodeSize = sizeof (WNODE_SINGLE_INSTANCE); + + // Because the first field in the data block, EventType, is a ULONG, + // ensure that the data block begins on a + // 4-byte boundary (as will be calculated in DataBlockOffset). + + // Size of USHORT at beginning plus + WnodeSizeInstanceName = sizeof (USHORT) + + lclInstanceName.Length; // size of instance name. + // Round length up to multiple of 4 (if needed). + WnodeSizeInstanceName = + (ULONG)WDF_ALIGN_SIZE_UP(WnodeSizeInstanceName, sizeof (ULONG)); + + WnodeSizeDataBlock = MSFC_AdapterEvent_SIZE; // Size of data block. + + size = WnodeSize + // Size of WMIEvent. + WnodeSizeInstanceName + + WnodeSizeDataBlock; + + pWnode = ExAllocatePoolWithTag(NonPagedPoolNx, size, MP_TAG_GENERAL); + + if (NULL != pWnode) { + RtlZeroMemory(pWnode, size); + + // Fill out most of header. StorPort will set the + // ProviderId and TimeStamp in the header. + + pWnode->WnodeHeader.BufferSize = size; + pWnode->WnodeHeader.Version = 1; + RtlCopyMemory(&pWnode->WnodeHeader.Guid, &lclGuid, + sizeof (lclGuid)); + pWnode->WnodeHeader.Flags = WNODE_FLAG_EVENT_ITEM | + WNODE_FLAG_SINGLE_INSTANCE; + + // Say where to find instance name and the data block + // and what is the data block's size. + + pWnode->OffsetInstanceName = WnodeSize; + pWnode->DataBlockOffset = WnodeSize + WnodeSizeInstanceName; + pWnode->SizeDataBlock = WnodeSizeDataBlock; + + // Copy the instance name. + + // Length remaining and available. + size -= WnodeSize; + // Copy WCHAR string, preceded by its size. + status = WDF_WMI_BUFFER_APPEND_STRING( + WDF_PTR_ADD_OFFSET(pWnode, pWnode->OffsetInstanceName), + size, + &lclInstanceName, + &length); + + if (STATUS_SUCCESS != status) { + ASSERT(FALSE); + } + + pAdapterArr = + WDF_PTR_ADD_OFFSET(pWnode, pWnode->DataBlockOffset); + + // Copy event code and WWN. + + pAdapterArr->EventType = HBA_EVENT_ADAPTER_ADD; + + RtlCopyMemory(pAdapterArr->PortWWN, myPortWWN, + sizeof (myPortWWN)); + + // Ask StorPort to announce the event. + + StorPortNotification(WMIEvent, + pHBAExt, + pWnode, + 0xFF); // Notification pertains to an HBA. + + ExFreePoolWithTag(pWnode, MP_TAG_GENERAL); + } else { + } +#endif +} + +void +wzvol_HwReportLink(__in pHW_HBA_EXT pHBAExt) +{ + dprintf("%s: entry\n", __func__); + +#if 1 + NTSTATUS status; + PWNODE_SINGLE_INSTANCE pWnode; + PMSFC_LinkEvent pLinkEvent; + ULONG WnodeSize, + WnodeSizeInstanceName, + WnodeSizeDataBlock, + length, + size; + GUID lclGuid = MSFC_LinkEvent_GUID; + UNICODE_STRING lclInstanceName; + + // Define 16 entries in MSFC_LinkEvent.RLIRBuffer[]. +#define RLIRBufferArraySize 0x10 + + UCHAR myAdapterWWN[8] = { 1, 2, 3, 4, 5, 6, 7, 8 }; + UCHAR myRLIRBuffer[RLIRBufferArraySize] = + { 10, 11, 12, 13, 14, 15, 16, 17, 20, 21, 22, 23, + 24, 25, 26, 0xFF }; + + RtlInitUnicodeString(&lclInstanceName, InstName); + + WnodeSize = sizeof (WNODE_SINGLE_INSTANCE); + WnodeSizeInstanceName = sizeof (USHORT) + + lclInstanceName.Length; + WnodeSizeInstanceName = + (ULONG)WDF_ALIGN_SIZE_UP(WnodeSizeInstanceName, sizeof (ULONG)); + WnodeSizeDataBlock = + FIELD_OFFSET(MSFC_LinkEvent, RLIRBuffer) + + sizeof (myRLIRBuffer); + + size = WnodeSize + + WnodeSizeInstanceName + + WnodeSizeDataBlock; + + pWnode = ExAllocatePoolWithTag(NonPagedPoolNx, size, MP_TAG_GENERAL); + + if (NULL != pWnode) { + RtlZeroMemory(pWnode, size); + + // Fill out most of header. StorPort will set the + // ProviderId and TimeStamp in the header. + + pWnode->WnodeHeader.BufferSize = size; + pWnode->WnodeHeader.Version = 1; + RtlCopyMemory(&pWnode->WnodeHeader.Guid, &lclGuid, + sizeof (lclGuid)); + pWnode->WnodeHeader.Flags = WNODE_FLAG_EVENT_ITEM | + WNODE_FLAG_SINGLE_INSTANCE; + + // Say where to find instance name and the data block + // and what is the data block's size. + + pWnode->OffsetInstanceName = WnodeSize; + pWnode->DataBlockOffset = WnodeSize + WnodeSizeInstanceName; + pWnode->SizeDataBlock = WnodeSizeDataBlock; + + // Copy the instance name. + + size -= WnodeSize; + status = WDF_WMI_BUFFER_APPEND_STRING( + WDF_PTR_ADD_OFFSET(pWnode, pWnode->OffsetInstanceName), + size, + &lclInstanceName, + &length); + + if (STATUS_SUCCESS != status) { + ASSERT(FALSE); + } + + pLinkEvent = + WDF_PTR_ADD_OFFSET(pWnode, pWnode->DataBlockOffset); + + // Copy event code, WWN, buffer size and buffer contents. + + pLinkEvent->EventType = HBA_EVENT_LINK_INCIDENT; + + RtlCopyMemory(pLinkEvent->AdapterWWN, myAdapterWWN, + sizeof (myAdapterWWN)); + + pLinkEvent->RLIRBufferSize = sizeof (myRLIRBuffer); + + RtlCopyMemory(pLinkEvent->RLIRBuffer, myRLIRBuffer, + sizeof (myRLIRBuffer)); + + StorPortNotification(WMIEvent, + pHBAExt, + pWnode, + 0xFF); + + ExFreePoolWithTag(pWnode, MP_TAG_GENERAL); + } else { + } +#endif +} + +void +wzvol_HwReportLog(__in pHW_HBA_EXT pHBAExt) +{ + dprintf("%s: entry\n", __func__); + +#if 1 + NTSTATUS status; + PWNODE_SINGLE_INSTANCE pWnode; + ULONG WnodeSize, + WnodeSizeInstanceName, + WnodeSizeDataBlock, + length, + size; + UNICODE_STRING lclInstanceName; + PIO_ERROR_LOG_PACKET pLogError; + + RtlInitUnicodeString(&lclInstanceName, InstName); + + WnodeSize = sizeof (WNODE_SINGLE_INSTANCE); + WnodeSizeInstanceName = sizeof (USHORT) + + lclInstanceName.Length; + WnodeSizeInstanceName = + (ULONG)WDF_ALIGN_SIZE_UP(WnodeSizeInstanceName, sizeof (ULONG)); + WnodeSizeDataBlock = sizeof (IO_ERROR_LOG_PACKET); + + size = WnodeSize + + WnodeSizeInstanceName + + WnodeSizeDataBlock; + + pWnode = ExAllocatePoolWithTag(NonPagedPoolNx, size, MP_TAG_GENERAL); + + if (NULL != pWnode) { + RtlZeroMemory(pWnode, size); + + // Fill out most of header. StorPort will set the + // ProviderId and TimeStamp in the header. + + pWnode->WnodeHeader.BufferSize = size; + pWnode->WnodeHeader.Version = 1; + pWnode->WnodeHeader.Flags = WNODE_FLAG_EVENT_ITEM | + WNODE_FLAG_LOG_WNODE; + + pWnode->WnodeHeader.HistoricalContext = 9; + + // Say where to find instance name and the data + // block and what is the data block's size. + + pWnode->OffsetInstanceName = WnodeSize; + pWnode->DataBlockOffset = WnodeSize + WnodeSizeInstanceName; + pWnode->SizeDataBlock = WnodeSizeDataBlock; + + // Copy the instance name. + + size -= WnodeSize; + status = WDF_WMI_BUFFER_APPEND_STRING( + WDF_PTR_ADD_OFFSET(pWnode, pWnode->OffsetInstanceName), + size, + &lclInstanceName, + &length); + + if (STATUS_SUCCESS != status) { + ASSERT(FALSE); + } + + pLogError = + WDF_PTR_ADD_OFFSET(pWnode, pWnode->DataBlockOffset); + + pLogError->UniqueErrorValue = 0x40; + pLogError->FinalStatus = 0x41; + pLogError->ErrorCode = 0x42; + + StorPortNotification(WMIEvent, + pHBAExt, + pWnode, + 0xFF); + + ExFreePoolWithTag(pWnode, MP_TAG_GENERAL); + } else { + } +#endif +} + +BOOLEAN +wzvol_HwResetBus( + __in pHW_HBA_EXT pHBAExt, + __in ULONG BusId) +{ + UNREFERENCED_PARAMETER(pHBAExt); + UNREFERENCED_PARAMETER(BusId); + + // To do: At some future point, it may be worthwhile to ensure + // that any SRBs being handled be completed at once. Practically + // speaking, however, it seems that the only SRBs that would not + // be completed very quickly would be those handled by the worker + // thread. In the future, therefore, there might be a global flag + // set here to instruct the thread to complete outstanding I/Os + // as they appear; but a period for that happening would have to + // be devised (such completion shouldn't be unbounded). + + return (TRUE); +} + +NTSTATUS +wzvol_HandleRemoveDevice( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_PNP_REQUEST_BLOCK pSrb) +{ + UNREFERENCED_PARAMETER(pHBAExt); + + pSrb->SrbStatus = SRB_STATUS_BAD_FUNCTION; + + return (STATUS_UNSUCCESSFUL); +} + +NTSTATUS +wzvol_HandleQueryCapabilities( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_PNP_REQUEST_BLOCK pSrb) +{ + NTSTATUS status = STATUS_SUCCESS; + PSTOR_DEVICE_CAPABILITIES pStorageCapabilities = + (PSTOR_DEVICE_CAPABILITIES)pSrb->DataBuffer; + + UNREFERENCED_PARAMETER(pHBAExt); + + dprintf("%s: entry\n", __func__); + + RtlZeroMemory(pStorageCapabilities, pSrb->DataTransferLength); + + pStorageCapabilities->Removable = FALSE; + pStorageCapabilities->SurpriseRemovalOK = FALSE; + + pSrb->SrbStatus = SRB_STATUS_SUCCESS; + + return (status); +} + +NTSTATUS +wzvol_HwHandlePnP( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_PNP_REQUEST_BLOCK pSrb) +{ + NTSTATUS status = STATUS_SUCCESS; + dprintf("%s: entry\n", __func__); + +#if 1 + switch (pSrb->PnPAction) { + + case StorRemoveDevice: + status = wzvol_HandleRemoveDevice(pHBAExt, pSrb); + break; + + case StorQueryCapabilities: + status = wzvol_HandleQueryCapabilities(pHBAExt, pSrb); + break; + + default: + pSrb->SrbStatus = SRB_STATUS_SUCCESS; + } + + if (STATUS_SUCCESS != status) { + } +#endif + return (status); +} + +BOOLEAN +wzvol_HwStartIo( + __in pHW_HBA_EXT pHBAExt, + __in __out PSCSI_REQUEST_BLOCK pSrb) +{ + +/* + * This function cal be called as DPC, so we can not call into ZFS + * (mutex etc) and kmem, including dprintf + */ + + UCHAR srbStatus = SRB_STATUS_INVALID_REQUEST; + BOOLEAN bFlag; + NTSTATUS status; + UCHAR Result = ResultDone; + + + _InterlockedExchangeAdd((volatile LONG *)&pHBAExt->SRBsSeen, 1); + + // Next, if true, will cause StorPort to remove the associated LUNs + // if, for example, devmgmt.msc is asked "scan for hardware changes." + + + switch (pSrb->Function) { + + case SRB_FUNCTION_EXECUTE_SCSI: + srbStatus = ScsiExecuteMain(pHBAExt, pSrb, &Result); + break; + + case SRB_FUNCTION_WMI: + _InterlockedExchangeAdd((volatile LONG *)&pHBAExt->WMISRBsSeen, + 1); + bFlag = HandleWmiSrb(pHBAExt, (PSCSI_WMI_REQUEST_BLOCK)pSrb); + srbStatus = TRUE == bFlag ? SRB_STATUS_SUCCESS : + SRB_STATUS_INVALID_REQUEST; + break; + + case SRB_FUNCTION_RESET_BUS: + case SRB_FUNCTION_RESET_DEVICE: + case SRB_FUNCTION_RESET_LOGICAL_UNIT: + { + // Set as cancelled all queued SRBs that match the criteria. + KIRQL oldIrql; + PLIST_ENTRY pNextEntry; + PHW_SRB_EXTENSION pSrbExt; + KeAcquireSpinLock(&pHBAExt->pwzvolDrvObj->SrbExtLock, &oldIrql); + for (pNextEntry = pHBAExt->pwzvolDrvObj->ListSrbExt.Flink; + pNextEntry != &pHBAExt->pwzvolDrvObj->ListSrbExt; + pNextEntry = pNextEntry->Flink) { + pSrbExt = CONTAINING_RECORD(pNextEntry, + HW_SRB_EXTENSION, QueuedForProcessing); + if ((pSrbExt->pSrbBackPtr->PathId == pSrb->PathId) && + (pSrb->Function == SRB_FUNCTION_RESET_BUS ? TRUE : + pSrbExt->pSrbBackPtr->TargetId == pSrb->TargetId) && + (pSrb->Function == SRB_FUNCTION_RESET_BUS || + pSrb->Function == SRB_FUNCTION_RESET_DEVICE ? TRUE : + pSrbExt->pSrbBackPtr->Lun == pSrb->Lun)) + pSrbExt->Cancelled = 1; + } + KeReleaseSpinLock(&pHBAExt->pwzvolDrvObj->SrbExtLock, oldIrql); + srbStatus = SRB_STATUS_SUCCESS; + } + break; + case SRB_FUNCTION_PNP: + status = wzvol_HwHandlePnP(pHBAExt, + (PSCSI_PNP_REQUEST_BLOCK)pSrb); + srbStatus = pSrb->SrbStatus; + + break; + + case SRB_FUNCTION_POWER: + // Do nothing. + srbStatus = SRB_STATUS_SUCCESS; + + break; + + case SRB_FUNCTION_SHUTDOWN: + // Do nothing. + srbStatus = SRB_STATUS_SUCCESS; + + break; + + default: + srbStatus = SRB_STATUS_INVALID_REQUEST; + break; + + } // switch (pSrb->Function) + + if (ResultDone == Result) { + pSrb->SrbStatus = srbStatus; + + // Note: A miniport with real hardware would not always + // be calling RequestComplete from HwStorStartIo. Rather, + // the miniport would typically be doing real I/O and would + // call RequestComplete only at the end of that + // real I/O, in its HwStorInterrupt or in a DPC routine. + + StorPortNotification(RequestComplete, pHBAExt, pSrb); + } + + return (TRUE); +} + +SCSI_ADAPTER_CONTROL_STATUS +wzvol_HwAdapterControl( + __in pHW_HBA_EXT pHBAExt, + __in SCSI_ADAPTER_CONTROL_TYPE ControlType, + __in PVOID pParameters) +{ + PSCSI_SUPPORTED_CONTROL_TYPE_LIST pCtlTypList; + ULONG i; + + dprintf("MpHwAdapterControl: ControlType = %d\n", ControlType); + + pHBAExt->AdapterState = ControlType; + + switch (ControlType) { + case ScsiQuerySupportedControlTypes: + dprintf("MpHwAdapterControl: ScsiQuerySupportedControlTypes\n"); + + // Ggt pointer to control type list + pCtlTypList = (PSCSI_SUPPORTED_CONTROL_TYPE_LIST)pParameters; + + // Cycle through list to set TRUE for each type supported + // making sure not to go past the MaxControlType + for (i = 0; i < pCtlTypList->MaxControlType; i++) + if (i == ScsiQuerySupportedControlTypes || + i == ScsiStopAdapter || i == ScsiRestartAdapter || + i == ScsiSetBootConfig || + i == ScsiSetRunningConfig) { + pCtlTypList->SupportedTypeList[i] = TRUE; + } + break; + + case ScsiStopAdapter: + dprintf("MpHwAdapterControl: ScsiStopAdapter\n"); + + // Free memory allocated for disk + wzvol_StopAdapter(pHBAExt); + + break; + + case ScsiRestartAdapter: + dprintf("MpHwAdapterControl: ScsiRestartAdapter\n"); + + /* To Do: Add some function. */ + + break; + + case ScsiSetBootConfig: + dprintf("MpHwAdapterControl: ScsiSetBootConfig\n"); + + break; + + case ScsiSetRunningConfig: + dprintf("MpHwAdapterControl: ScsiSetRunningConfig\n"); + + break; + + default: + dprintf("MpHwAdapterControl: UNKNOWN\n"); + + break; + } + + dprintf("MpHwAdapterControl - OUT\n"); + + return (ScsiAdapterControlSuccess); +} + +void +wzvol_StopAdapter(__in pHW_HBA_EXT pHBAExt) +{ + pHW_LU_EXTENSION pLUExt, pLUExt2; + PLIST_ENTRY pNextEntry, pNextEntry2; + pwzvolDriverInfo pwzvolDrvInfo = pHBAExt->pwzvolDrvObj; + pHW_LU_EXTENSION_MPIO pLUMPIOExt = NULL; + +#if defined(_AMD64_) + KLOCK_QUEUE_HANDLE LockHandle; +#else + KIRQL SaveIrql; +#endif + + dprintf("%s: entry\n", __func__); + + // Clean up the "disk buffers." + + for (pNextEntry = pHBAExt->LUList.Flink; + pNextEntry != &pHBAExt->LUList; + pNextEntry = pNextEntry->Flink) { + pLUExt = CONTAINING_RECORD(pNextEntry, HW_LU_EXTENSION, List); + + if (pwzvolDrvInfo->wzvolRegInfo.bCombineVirtDisks) { + pLUMPIOExt = pLUExt->pLUMPIOExt; + + if (!pLUMPIOExt) { + break; + } + +#if defined(_AMD64_) + KeAcquireInStackQueuedSpinLock( + &pLUMPIOExt->LUExtMPIOLock, + &LockHandle); +#else + KeAcquireSpinLock(&pLUMPIOExt->LUExtMPIOLock, + &SaveIrql); +#endif + + for (pNextEntry2 = pLUMPIOExt->LUExtList.Flink; + pNextEntry2 != &pLUMPIOExt->LUExtList; + pNextEntry2 = pNextEntry2->Flink) { + pLUExt2 = CONTAINING_RECORD(pNextEntry2, + HW_LU_EXTENSION, MPIOList); + + if (pLUExt2 == pLUExt) { + break; + } + } + + if (pNextEntry2 != &pLUMPIOExt->LUExtList) { + RemoveEntryList(pNextEntry2); + + pLUMPIOExt->NbrRealLUNs--; + + if (0 == pLUMPIOExt->NbrRealLUNs) { + ExFreePool(pLUExt->pDiskBuf); + } + } + +#if defined(_AMD64_) + KeReleaseInStackQueuedSpinLock(&LockHandle); +#else + KeReleaseSpinLock(&pLUMPIOExt->LUExtMPIOLock, SaveIrql); +#endif + } else { + ExFreePool(pLUExt->pDiskBuf); + } + } + + // Clean up the linked list of MPIO collector objects, if needed. + + if (pwzvolDrvInfo->wzvolRegInfo.bCombineVirtDisks) { +#if defined(_AMD64_) + KeAcquireInStackQueuedSpinLock( + &pwzvolDrvInfo->MPIOExtLock, &LockHandle); +#else + KeAcquireSpinLock(&pwzvolDrvInfo->MPIOExtLock, &SaveIrql); +#endif + + for (pNextEntry = pwzvolDrvInfo->ListMPIOExt.Flink; + pNextEntry != &pwzvolDrvInfo->ListMPIOExt; + pNextEntry = pNextEntry2) { + pLUMPIOExt = CONTAINING_RECORD(pNextEntry, + HW_LU_EXTENSION_MPIO, List); + + if (!pLUMPIOExt) { + break; + } + + pNextEntry2 = pNextEntry->Flink; + + if (0 == pLUMPIOExt->NbrRealLUNs) { + RemoveEntryList(pNextEntry); + ExFreePoolWithTag(pLUMPIOExt, MP_TAG_GENERAL); + } + } + +#if defined(_AMD64_) + KeReleaseInStackQueuedSpinLock(&LockHandle); +#else + KeReleaseSpinLock(&pwzvolDrvInfo->MPIOExtLock, SaveIrql); +#endif + } +} + +void +wzvol_TracingInit( + __in PVOID pArg1, + __in PVOID pArg2) +{ +// WPP_INIT_TRACING(pArg1, pArg2); +} + +void +wzvol_TracingCleanup(__in PVOID pArg1) +{ +#if 1 + dprintf("MPTracingCleanUp entered\n"); + + // WPP_CLEANUP(pArg1); +#endif +} + +void +wzvol_HwFreeAdapterResources(__in pHW_HBA_EXT pHBAExt) +{ +#if 1 + PLIST_ENTRY pNextEntry; + pHW_HBA_EXT pLclHBAExt; +#if defined(_AMD64_) + KLOCK_QUEUE_HANDLE LockHandle; +#else + KIRQL SaveIrql; +#endif + + dprintf("MpHwFreeAdapterResources entered, pHBAExt = 0x%p\n", pHBAExt); + +#if defined(_AMD64_) + KeAcquireInStackQueuedSpinLock(&pHBAExt->pwzvolDrvObj->DrvInfoLock, + &LockHandle); +#else + KeAcquireSpinLock(&pHBAExt->pwzvolDrvObj->DrvInfoLock, &SaveIrql); +#endif + + for (pNextEntry = pHBAExt->pwzvolDrvObj->ListMPHBAObj.Flink; + pNextEntry != &pHBAExt->pwzvolDrvObj->ListMPHBAObj; + pNextEntry = pNextEntry->Flink) { + pLclHBAExt = CONTAINING_RECORD(pNextEntry, HW_HBA_EXT, List); + + if (pLclHBAExt == pHBAExt) { + RemoveEntryList(pNextEntry); + pHBAExt->pwzvolDrvObj->DrvInfoNbrMPHBAObj--; + break; + } + } + +#if defined(_AMD64_) + KeReleaseInStackQueuedSpinLock(&LockHandle); +#else + KeReleaseSpinLock(&pHBAExt->pwzvolDrvObj->DrvInfoLock, SaveIrql); +#endif + +#endif + + if (STOR_HBAExt == pHBAExt) + STOR_HBAExt = NULL; +} + +void +wzvol_CompleteIrp( + __in pHW_HBA_EXT pHBAExt, + __in PIRP pIrp) +{ + dprintf("MpCompleteIrp entered\n"); + + if (NULL != pIrp) { + NTSTATUS Status; + PIO_STACK_LOCATION pIrpStack = + IoGetCurrentIrpStackLocation(pIrp); + + switch (pIrpStack->Parameters.DeviceIoControl.IoControlCode) { + case IOCTL_MINIPORT_PROCESS_SERVICE_IRP: + Status = STATUS_SUCCESS; + break; + default: + Status = STATUS_INVALID_DEVICE_REQUEST; + break; + } + + pIrp->IoStatus.Status = Status; + if (NT_SUCCESS(Status)) + pIrp->IoStatus.Information = + pIrpStack-> + Parameters.DeviceIoControl.OutputBufferLength; + else + pIrp->IoStatus.Information = 0; + + StorPortCompleteServiceIrp(pHBAExt, pIrp); + } +} + +void +wzvol_QueueServiceIrp( + __in pHW_HBA_EXT pHBAExt, + __in PIRP pIrp) +{ + +#if 1 + PIRP pOldIrp; + + dprintf("MpQueueServiceIrp entered\n"); + + pOldIrp = InterlockedExchangePointer(&pHBAExt->pReverseCallIrp, pIrp); + if (NULL != pOldIrp) { + wzvol_CompleteIrp(pHBAExt, pOldIrp); +} +#endif +} + +void +wzvol_ProcServReq( + __in pHW_HBA_EXT pHBAExt, + __in PIRP pIrp) +{ + +#if 1 + dprintf("MpProcServReq entered\n"); + + wzvol_QueueServiceIrp(pHBAExt, pIrp); +#endif +} + +void +wzvol_CompServReq(__in pHW_HBA_EXT pHBAExt) +{ + +#if 1 + dprintf("MpHwCompServReq entered\n"); + + wzvol_QueueServiceIrp(pHBAExt, NULL); +#endif +} + +void +wzvol_announce_buschange(void) +{ + dprintf("%s: \n", __func__); + if (STOR_HBAExt != NULL) + StorPortNotification(BusChangeDetected, STOR_HBAExt, 0); +} diff --git a/module/os/windows/zfs/zfs_windows_zvol_scsi.c b/module/os/windows/zfs/zfs_windows_zvol_scsi.c new file mode 100644 index 000000000000..7f6b6f3839a3 --- /dev/null +++ b/module/os/windows/zfs/zfs_windows_zvol_scsi.c @@ -0,0 +1,1124 @@ +/* + * Module Name: scsi.c + * Project: CppWDKStorPortVirtualMiniport + * + * Copyright (c) Microsoft Corporation. + * + * a. ScsiExecuteMain() + * Handles SCSI SRBs with opcodes needed to support file system operations by + * calling subroutines. Fails SRBs with other opcodes. + * Note: In a real-world virtual miniport, it may be necessary to handle + * other opcodes. + * + * b. ScsiOpInquiry() + * Handles Inquiry, including creating a new LUN as needed. + * + * c. ScsiOpVPD() + * Handles Vital Product Data. + * + * d. ScsiOpRead() + * Beginning of a SCSI Read operation. + * + * e. ScsiOpWrite() + * Beginning of a SCSI Write operation. + * + * f. ScsiReadWriteSetup() + * Sets up a work element for SCSI Read or Write and enqueues the element. + * + * g. ScsiOpReportLuns() + * Handles Report LUNs. + * + * + * This source is subject to the Microsoft Public License. + * See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL. + * All other rights reserved. + * + * THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, + * EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE. + */ + +#include +#include +#include + +#include +#include + +#pragma warning(push) +#pragma warning(disable : 4204) +#include +#pragma warning(pop) + +#include +#include +#include +#include +#include +#include +#include + +/* + * We have a list of ZVOLs, and we receive incoming (Target, Lun) + * requests that needs to be mapped to the correct "zv" ptr. + * + * ssv-18807: fixed the race condition in the zvol destroy processing + * by adding remove lock logic to ensure no new I/O can be processed + * from the front end (StorPort) and all outstanding host I/Os have + * left the pipeline. + * + * The zv control block starts to get protected in wzvol_assign_targetid() + * and this until wzvol_clear_targetid() is called. + * + * Once wzvol_find_target(t,l) returns a valid pointer to the zv, + * that zv is protected via an extra reference on its remove lock so + * it can't be freed unless all references on it are cleared. + * It is the caller's responsibility to clear the extra reference it got + * by calling wzvol_unlock_target(zv). + * + * wzvol_find_target(t,l) will take an extra reference each time its + * called so each of those will need their wzvol_unlock_target(zv) + * counterpart call. + * + * The wzvol_lock_target(zv) call is commented out because not used yet + * but its purpose is for when nested extra references need to be taken + * on the zv after wzvol_find_target(t,l) was called. That can be useful + * for when asynchronous processing (queueing) involving the zv control + * block need to make sure that zv stays allocated. + * + * When the zvol is destroyed the wzvol_clear_targetid(t,l,zv) will + * actively wait for all references to be released and no new one can + * be taken. + * + * programming notes: the remove lock must be dynamically allocated + * because it cannot be reinitialized. An interlocked refcnt variable + * is also necessary to protect the remove lock control block's allocation. + * when the refcnt reaches 0 it is safe to free the remove lock cb. + */ +extern wzvolDriverInfo STOR_wzvolDriverInfo; +extern taskq_t *zvol_taskq; + +inline int +resolveArrayIndex(int t, int l, int nbL) +{ + return ((t * nbL) + l); +} + +static inline void +wzvol_decref_target(wzvolContext* zvc) +{ + if (atomic_dec_64_nv(&zvc->refCnt) == 0) { + PIO_REMOVE_LOCK pIoRemLock = zvc->pIoRemLock; + ASSERT(pIoRemLock != NULL); + // when refCnt is 0 we can free the remove lock block. + // All IoReleaseRemoveLock have been called. + atomic_cas_ptr(&zvc->pIoRemLock, pIoRemLock, NULL); + kmem_free(pIoRemLock, sizeof (*pIoRemLock)); + } +} + +/* + * not used now but left for completeness in case we need to have + * an extra reference after calling find_targetid + */ +static inline BOOLEAN +wzvol_lock_target(zvol_state_t *zv) +{ + wzvolContext *zvc = (pwzvolContext)zv->zv_zso->zso_target_context; + if (zvc) { + if (atomic_inc_64_nv(&zvc->refCnt) > 1) { + // safe to access the remove lock. + // Make sure we are on the same zv. + if (zvc->zv == zv) { + if (STATUS_SUCCESS == + IoAcquireRemoveLock(zvc->pIoRemLock, zv)) + return (TRUE); + else + wzvol_decref_target(zvc); + } else + wzvol_decref_target(zvc); + } else + atomic_dec_64_nv(&zvc->refCnt); + } + return (FALSE); +} + +static inline void +wzvol_unlock_target(zvol_state_t *zv) +{ + wzvolContext* zvc = (pwzvolContext)zv->zv_zso->zso_target_context; + IoReleaseRemoveLock(zvc->pIoRemLock, zv); + wzvol_decref_target(zvc); +} + +int +wzvol_assign_targetid(zvol_state_t *zv) +{ + wzvolContext* zv_targets = STOR_wzvolDriverInfo.zvContextArray; + ASSERT(zv->zv_zso->zso_target_context == NULL); + PIO_REMOVE_LOCK pIoRemLock = kmem_zalloc(sizeof (*pIoRemLock), + KM_SLEEP); + if (!pIoRemLock) { + dprintf("ZFS: Unable to assign targetid - out of memory.\n"); + ASSERT("Unable to assign targetid - out of memory."); + return (0); + } + + IoInitializeRemoveLock(pIoRemLock, 'KLRZ', 0, 0); + + if (STATUS_SUCCESS != IoAcquireRemoveLock(pIoRemLock, zv)) { + dprintf("ZFS: Unable to assign targetid - can't acquire " + "the remlock.\n"); + ASSERT("Unable to assign targetid - can't acquire remlock."); + } else { + for (uint8_t l = 0; + l < STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits; + l++) { + for (uint8_t t = 0; + t < STOR_wzvolDriverInfo.MaximumNumberOfTargets; + t++) { + int zvidx = resolveArrayIndex(t, l, + STOR_wzvolDriverInfo. + MaximumNumberOfLogicalUnits); + + if (zv_targets[zvidx].zv == NULL && + zv_targets[zvidx].pIoRemLock == NULL) { + if (atomic_inc_64_nv( + &zv_targets[zvidx].refCnt) == 1) { + // brand new entry - got it. + ASSERT( + zv_targets[zvidx].pIoRemLock + == NULL); + zv->zv_zso->zso_target_id = t; + zv->zv_zso->zso_lun_id = l; + zv->zv_zso->zso_target_context = + &zv_targets[zvidx]; + zv_targets[zvidx].pIoRemLock = + pIoRemLock; + atomic_cas_ptr( + &zv_targets[zvidx].zv, + NULL, zv); + // zv is now searchable + return (1); + } else { + // assign_targetid collision + // (very rare) + wzvol_decref_target( + &zv_targets[zvidx]); + } + } + } + } + IoReleaseRemoveLock(pIoRemLock, zv); + } + + kmem_free(pIoRemLock, sizeof (*pIoRemLock)); + dprintf("ZFS: Unable to assign targetid - out of room.\n"); + ASSERT("Unable to assign targetid - out of room."); + return (0); +} + +/* + * note: find_target will lock the zv's remove lock. caller + * is responsible to unlock_target if a non-NULL zv pointer is returned + */ +static inline zvol_state_t * +wzvol_find_target(uint8_t targetid, uint8_t lun) +{ + wzvolContext *zv_targets = STOR_wzvolDriverInfo.zvContextArray; + ASSERT(targetid < STOR_wzvolDriverInfo.MaximumNumberOfTargets); + ASSERT(lun < STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits); + if (targetid < STOR_wzvolDriverInfo.MaximumNumberOfTargets && + lun < STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits) { + int zvidx = resolveArrayIndex(targetid, lun, + STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits); + zvol_state_t *zv = zv_targets[zvidx].zv; + if (zv) { + if (atomic_inc_64_nv(&zv_targets[zvidx].refCnt) > 1) { + // safe to access the remove lock + if (STATUS_SUCCESS == + IoAcquireRemoveLock( + zv_targets[zvidx].pIoRemLock, + zv)) + return ((zvol_state_t *) + zv_targets[zvidx].zv); + else + wzvol_decref_target(&zv_targets[zvidx]); + } else + atomic_dec_64_nv(&zv_targets[zvidx].refCnt); + } // nothing in that t-l + } + return (NULL); +} + + +void +wzvol_clear_targetid(uint8_t targetid, uint8_t lun, zvol_state_t *zv) +{ + wzvolContext *zvc = (pwzvolContext)zv->zv_zso->zso_target_context; + + ASSERT(KeGetCurrentIrql() < DISPATCH_LEVEL); + ASSERT(targetid < STOR_wzvolDriverInfo.MaximumNumberOfTargets); + ASSERT(lun < STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits); + if (targetid < STOR_wzvolDriverInfo.MaximumNumberOfTargets && + lun < STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits) { + /* + * make sure no new I/O can enter the front-end + all + * outstanding I/Os are completed (ssv-18807). + */ + if (atomic_cas_ptr( + &STOR_wzvolDriverInfo.zvContextArray[resolveArrayIndex( + targetid, lun, + STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits)].zv, + zv, NULL) == zv) { + IoReleaseRemoveLockAndWait(zvc->pIoRemLock, zv); + wzvol_decref_target(zvc); + } + } +} + +UCHAR +ScsiExecuteMain( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb, + __in PUCHAR pResult) +{ + UCHAR status = SRB_STATUS_INVALID_REQUEST; + + TraceEvent(TRACE_VERBOSE, "%s:%d: ScsiExecute: pSrb = 0x%p, CDB = 0x%x" + " Path: %x TID: %x Lun: %x\n", __func__, __LINE__, pSrb, + pSrb->Cdb[0], pSrb->PathId, pSrb->TargetId, pSrb->Lun); + *pResult = ResultDone; + + // Verify that the B/T/L is not out of bound. + if (pSrb->PathId > 0) { + status = SRB_STATUS_INVALID_PATH_ID; + goto Done; + } else if (pSrb->TargetId >= + STOR_wzvolDriverInfo.MaximumNumberOfTargets) { + status = SRB_STATUS_INVALID_TARGET_ID; + goto Done; + } else if (pSrb->Lun >= + STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits) { + status = SRB_STATUS_INVALID_LUN; + goto Done; + } + + // Handle sufficient opcodes to support a LUN suitable + // for a file system. Other opcodes are failed. + switch (pSrb->Cdb[0]) { + + case SCSIOP_TEST_UNIT_READY: + case SCSIOP_SYNCHRONIZE_CACHE: + case SCSIOP_START_STOP_UNIT: + case SCSIOP_VERIFY: + status = SRB_STATUS_SUCCESS; + break; + + case SCSIOP_INQUIRY: + status = ScsiOpInquiry(pHBAExt, pSrb); + break; + + case SCSIOP_READ_CAPACITY: + status = ScsiOpReadCapacity(pHBAExt, pSrb); + break; + + case SCSIOP_READ_CAPACITY16: + status = ScsiOpReadCapacity16(pHBAExt, pSrb); + break; + + case SCSIOP_READ: + case SCSIOP_READ16: + status = ScsiOpRead(pHBAExt, pSrb, pResult); + break; + + case SCSIOP_WRITE: + case SCSIOP_WRITE16: + status = ScsiOpWrite(pHBAExt, pSrb, pResult); + break; + + case SCSIOP_MODE_SENSE: + status = ScsiOpModeSense(pHBAExt, pSrb); + break; + + case SCSIOP_REPORT_LUNS: + status = ScsiOpReportLuns(pHBAExt, pSrb); + break; + + case SCSIOP_UNMAP: + status = ScsiOpUnmap(pHBAExt, pSrb, pResult); + break; + + default: + status = SRB_STATUS_INVALID_REQUEST; + break; + + } // switch (pSrb->Cdb[0]) + +Done: + return (status); +} + +pHW_LU_EXTENSION_MPIO +ScsiGetMPIOExt( + __in pHW_HBA_EXT pHBAExt, + __in pHW_LU_EXTENSION pLUExt, + __in PSCSI_REQUEST_BLOCK pSrb) +{ + pHW_LU_EXTENSION_MPIO pLUMPIOExt = NULL; +#if defined(_AMD64_) + KLOCK_QUEUE_HANDLE LockHandle, + LockHandle2; +#else + KIRQL SaveIrql, + SaveIrql2; +#endif + PLIST_ENTRY pNextEntry; + +#if defined(_AMD64_) + KeAcquireInStackQueuedSpinLock(&pHBAExt->pwzvolDrvObj->MPIOExtLock, + &LockHandle); +#else + KeAcquireSpinLock(&pHBAExt->pwzvolDrvObj->MPIOExtLock, &SaveIrql); +#endif + + for (pNextEntry = pHBAExt->pwzvolDrvObj->ListMPIOExt.Flink; + pNextEntry != &pHBAExt->pwzvolDrvObj->ListMPIOExt; + pNextEntry = pNextEntry->Flink) { + pLUMPIOExt = CONTAINING_RECORD(pNextEntry, + HW_LU_EXTENSION_MPIO, List); + + if (pSrb->PathId == pLUMPIOExt->ScsiAddr.PathId && + pSrb->TargetId == pLUMPIOExt->ScsiAddr.TargetId && + pSrb->Lun == pLUMPIOExt->ScsiAddr.Lun) { + break; + } + } + + if (pNextEntry == &pHBAExt->pwzvolDrvObj->ListMPIOExt) { + pLUMPIOExt = ExAllocatePoolWithTag(NonPagedPoolNx, + sizeof (HW_LU_EXTENSION_MPIO), MP_TAG_GENERAL); + + if (!pLUMPIOExt) { + dprintf("Failed to allocate HW_LU_EXTENSION_MPIO\n"); + goto Done; + } + + RtlZeroMemory(pLUMPIOExt, sizeof (HW_LU_EXTENSION_MPIO)); + + pLUMPIOExt->ScsiAddr.PathId = pSrb->PathId; + pLUMPIOExt->ScsiAddr.TargetId = pSrb->TargetId; + pLUMPIOExt->ScsiAddr.Lun = pSrb->Lun; + + KeInitializeSpinLock(&pLUMPIOExt->LUExtMPIOLock); + + InitializeListHead(&pLUMPIOExt->LUExtList); + + // ScsiAllocDiskBuf(pHBAExt, + // &pLUMPIOExt->pDiskBuf, &pLUExt->MaxBlocks); + + if (!pLUMPIOExt->pDiskBuf) { + dprintf("Failed to allocate DiskBuf\n"); + ExFreePoolWithTag(pLUMPIOExt, MP_TAG_GENERAL); + pLUMPIOExt = NULL; + + goto Done; + } + + InsertTailList(&pHBAExt->pwzvolDrvObj->ListMPIOExt, + &pLUMPIOExt->List); + + pHBAExt->pwzvolDrvObj->DrvInfoNbrMPIOExtObj++; + } else { + pLUExt->MaxBlocks = + (USHORT)(pHBAExt->pwzvolDrvObj-> + wzvolRegInfo.PhysicalDiskSize / MP_BLOCK_SIZE); + } + +Done: + if (pLUMPIOExt) { + +#if defined(_AMD64_) + KeAcquireInStackQueuedSpinLock(&pLUMPIOExt->LUExtMPIOLock, + &LockHandle2); +#else + KeAcquireSpinLock(&pLUMPIOExt->LUExtMPIOLock, &SaveIrql2); +#endif + + pLUExt->pLUMPIOExt = pLUMPIOExt; + pLUExt->pDiskBuf = pLUMPIOExt->pDiskBuf; + + InsertTailList(&pLUMPIOExt->LUExtList, &pLUExt->MPIOList); + pLUMPIOExt->NbrRealLUNs++; + +#if defined(_AMD64_) + KeReleaseInStackQueuedSpinLock(&LockHandle2); +#else + KeReleaseSpinLock(&pLUMPIOExt->LUExtMPIOLock, SaveIrql2); +#endif + } + +#if defined(_AMD64_) + KeReleaseInStackQueuedSpinLock(&LockHandle); +#else + KeReleaseSpinLock(&pHBAExt->pwzvolDrvObj->MPIOExtLock, SaveIrql); +#endif + + return (pLUMPIOExt); +} + +UCHAR +ScsiOpInquiry( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb) +{ + UCHAR status = SRB_STATUS_SUCCESS; + zvol_state_t *zv = NULL; + + if (pHBAExt->bDontReport) { + status = SRB_STATUS_NO_DEVICE; + goto out; + } + + zv = wzvol_find_target(pSrb->TargetId, pSrb->Lun); + if (NULL == zv) { + dprintf("Unable to get zv context for device %d:%d:%d\n", + pSrb->PathId, pSrb->TargetId, pSrb->Lun); + status = SRB_STATUS_NO_DEVICE; + goto out; + } + + ASSERT(pSrb->DataTransferLength > 0); + if (0 == pSrb->DataTransferLength) { + status = SRB_STATUS_DATA_OVERRUN; + goto out; + } + + RtlZeroMemory((PUCHAR)pSrb->DataBuffer, pSrb->DataTransferLength); + + if (1 == ((PCDB)pSrb->Cdb)->CDB6INQUIRY3.EnableVitalProductData) { + status = ScsiOpVPD(pHBAExt, pSrb, zv); + } else { + PINQUIRYDATA pInqData = pSrb->DataBuffer; + // Claim SCSI-3 commands support + pInqData->DeviceType = DISK_DEVICE; + pInqData->DeviceTypeQualifier = DEVICE_CONNECTED; + pInqData->ResponseDataFormat = 2; + pInqData->Versions = 5; + pInqData->RemovableMedia = FALSE; + pInqData->CommandQueue = TRUE; + + RtlMoveMemory(pInqData->VendorId, pHBAExt->VendorId, 8); + RtlMoveMemory(pInqData->ProductId, pHBAExt->ProductId, 16); + RtlMoveMemory(pInqData->ProductRevisionLevel, + pHBAExt->ProductRevision, 4); + memset((PCHAR)pInqData->VendorSpecific, ' ', + sizeof (pInqData->VendorSpecific)); + sprintf(pInqData->VendorSpecific, "%.04d-%.04d-%.04d", + pSrb->PathId, pSrb->TargetId, pSrb->Lun); + pInqData->VendorSpecific[strlen(pInqData->VendorSpecific)] = + ' '; + + pInqData->AdditionalLength = sizeof (*pInqData) - 4; + } + +out: + if (zv) + wzvol_unlock_target(zv); + return (status); +} + +UCHAR +ScsiOpVPD( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb, + __in PVOID zvContext) +{ + UCHAR status = SRB_STATUS_SUCCESS; + ULONG len = 0; + zvol_state_t *zv = (zvol_state_t *)zvContext; + + switch (((struct _CDB6INQUIRY3 *)&pSrb->Cdb)->PageCode) { + case VPD_SUPPORTED_PAGES: + { + PVPD_SUPPORTED_PAGES_PAGE pPage = pSrb->DataBuffer; + // 0x00 + 0x80 + 0x83 + len = sizeof (VPD_SUPPORTED_PAGES_PAGE) + 3; + if (pSrb->DataTransferLength < len) { + status = SRB_STATUS_DATA_OVERRUN; + goto ScsiOpVPD_done; + } + + pPage->DeviceType = DIRECT_ACCESS_DEVICE; + pPage->DeviceTypeQualifier = DEVICE_CONNECTED; + pPage->PageCode = VPD_SUPPORTED_PAGES; + pPage->PageLength = 3; + pPage->SupportedPageList[0] = VPD_SUPPORTED_PAGES; + pPage->SupportedPageList[1] = VPD_SERIAL_NUMBER; + pPage->SupportedPageList[2] = VPD_DEVICE_IDENTIFIERS; + } + break; + case VPD_SERIAL_NUMBER: + { + PVPD_SERIAL_NUMBER_PAGE pPage = pSrb->DataBuffer; + len = sizeof (VPD_SERIAL_NUMBER_PAGE) + strlen(zv->zv_name); + if (pSrb->DataTransferLength < len) { + status = SRB_STATUS_DATA_OVERRUN; + goto ScsiOpVPD_done; + } + + pPage->DeviceType = DIRECT_ACCESS_DEVICE; + pPage->DeviceTypeQualifier = DEVICE_CONNECTED; + pPage->PageCode = VPD_SERIAL_NUMBER; + pPage->PageLength = strlen(zv->zv_name); + memcpy(&pPage->SerialNumber[0], zv->zv_name, + strlen(zv->zv_name)); + + dprintf("ScsiOpVPD: VPD Page: %d Serial No.: %s", + pPage->PageCode, (const char *)pPage->SerialNumber); + } + break; + case VPD_DEVICE_IDENTIFIERS: + { + PVPD_IDENTIFICATION_PAGE pPage = pSrb->DataBuffer; + PVPD_IDENTIFICATION_DESCRIPTOR pDesc = + (PVPD_IDENTIFICATION_DESCRIPTOR)&pPage->Descriptors[0]; + + len = sizeof (VPD_IDENTIFICATION_PAGE) + + sizeof (VPD_IDENTIFICATION_DESCRIPTOR) + + strlen(VENDOR_ID_ascii) + strlen(zv->zv_name); + if (pSrb->DataTransferLength < len) { + status = SRB_STATUS_DATA_OVERRUN; + goto ScsiOpVPD_done; + } + + pPage->PageCode = VPD_DEVICE_IDENTIFIERS; + // Only descriptor is the vendor T10 for now: + // VendorId:Poolname/Zvolname + // NAA can't be done as OpenZFS is not IEEE registered for NAA. + pDesc->CodeSet = VpdCodeSetAscii; + pDesc->IdentifierType = VpdIdentifierTypeVendorId; + pDesc->Association = VpdAssocDevice; + pDesc->IdentifierLength = strlen(VENDOR_ID_ascii) + + strlen(zv->zv_name); + memcpy(&pDesc->Identifier[0], VENDOR_ID_ascii, + strlen(VENDOR_ID_ascii)); + memcpy(&pDesc->Identifier[strlen(VENDOR_ID_ascii)], + zv->zv_name, strlen(zv->zv_name)); + pPage->PageLength = + FIELD_OFFSET(VPD_IDENTIFICATION_DESCRIPTOR, + Identifier) + pDesc->IdentifierLength; + } + break; + default: + status = SRB_STATUS_INVALID_REQUEST; + len = 0; + break; + } // endswitch + +ScsiOpVPD_done: + pSrb->DataTransferLength = len; + return (status); +} + +UCHAR +ScsiOpReadCapacity( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb) +{ + UNREFERENCED_PARAMETER(pHBAExt); + PREAD_CAPACITY_DATA readCapacity = pSrb->DataBuffer; + ULONG maxBlocks, + blockSize; + zvol_state_t *zv = wzvol_find_target(pSrb->TargetId, pSrb->Lun); + if (NULL == zv) { + dprintf("Unable to get zv context for device %d:%d:%d\n", + pSrb->PathId, pSrb->TargetId, pSrb->Lun); + pSrb->DataTransferLength = 0; + return (SRB_STATUS_NO_DEVICE); + } + + RtlZeroMemory((PUCHAR)pSrb->DataBuffer, pSrb->DataTransferLength); + + /* + * fake maxBlocks to ULONG_MAX so that Windows calls with + * SCSIOP_READ_CAPACITY16. + * This would help specify non-zero LogicalPerPhysicalExponent + * that makes logical and physical sector size of a zvol different, + * kind of 512e disk! + */ + maxBlocks = ULONG_MAX; + blockSize = MP_BLOCK_SIZE; + + dprintf("Block Size: 0x%x Total Blocks: 0x%x\n", blockSize, maxBlocks); + REVERSE_BYTES(&readCapacity->BytesPerBlock, &blockSize); + REVERSE_BYTES(&readCapacity->LogicalBlockAddress, &maxBlocks); + + wzvol_unlock_target(zv); + return (SRB_STATUS_SUCCESS); +} + +UCHAR +ScsiOpReadCapacity16( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb) +{ + PREAD_CAPACITY16_DATA readCapacity = pSrb->DataBuffer; + ULONGLONG maxBlocks = 0; + ULONG blockSize = 0; + UCHAR lppExponent = 0; + ULONG lppFactor; + UNREFERENCED_PARAMETER(pHBAExt); + + zvol_state_t *zv = wzvol_find_target(pSrb->TargetId, pSrb->Lun); + if (NULL == zv) { + dprintf("Unable to get zv context for device %d:%d:%d\n", + pSrb->PathId, pSrb->TargetId, pSrb->Lun); + pSrb->DataTransferLength = 0; + return (SRB_STATUS_NO_DEVICE); + } + RtlZeroMemory((PUCHAR)pSrb->DataBuffer, pSrb->DataTransferLength); + blockSize = MP_BLOCK_SIZE; + maxBlocks = (zv->zv_volsize / blockSize) - 1; + + dprintf("%s:%d Block Size: 0x%x Total Blocks: 0x%llx targetid:%d " + "lun:%d, volname:%s, zv_volsize=%llu\n", __func__, __LINE__, + blockSize, maxBlocks, pSrb->TargetId, pSrb->Lun, zv->zv_name, + zv->zv_volsize); + REVERSE_BYTES(&readCapacity->BytesPerBlock, &blockSize); + REVERSE_BYTES_QUAD(&readCapacity->LogicalBlockAddress.QuadPart, + &maxBlocks); + lppFactor = zv->zv_volblocksize / MP_BLOCK_SIZE; + ASSERT((lppFactor & (lppFactor - 1)) == 0); + while (lppFactor >>= 1) + lppExponent++; + readCapacity->LogicalPerPhysicalExponent = lppExponent; + wzvol_unlock_target(zv); + return (SRB_STATUS_SUCCESS); +} + +UCHAR +ScsiOpRead( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb, + __in PUCHAR pResult) +{ + UCHAR status; + + status = ScsiReadWriteSetup(pHBAExt, pSrb, ActionRead, pResult); + + return (status); +} + +UCHAR +ScsiOpWrite( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb, + __in PUCHAR pResult) +{ + UCHAR status; + + status = ScsiReadWriteSetup(pHBAExt, pSrb, ActionWrite, pResult); + + return (status); +} + +UCHAR +ScsiOpUnmap( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb, + __in PUCHAR pResult) +{ + UCHAR status; + + status = ScsiReadWriteSetup(pHBAExt, pSrb, ActionUnmap, pResult); + + return (status); +} + +UCHAR +ScsiOpUnmap_impl( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb, + __in zvol_state_t *zv) +{ + PUNMAP_LIST_HEADER DataBuffer = pSrb->DataBuffer; + ULONG DataTransferLength = pSrb->DataTransferLength; + ULONG DataLength; + int ret; + + if (0 == DataBuffer || + DataTransferLength < sizeof (UNMAP_LIST_HEADER) || + DataTransferLength < sizeof (UNMAP_LIST_HEADER) + (DataLength = + ((ULONG)DataBuffer->BlockDescrDataLength[0] << 8) | + (ULONG)DataBuffer->BlockDescrDataLength[1])) + return (SRB_STATUS_INTERNAL_ERROR); + + // Test for DataLength > MaxTransferLength ? + + // Fasttrack 0 length + if (DataLength == 0) + return (SRB_STATUS_SUCCESS); + + // Loop all the unmap ranges + for (ULONG i = 0, n = DataLength / sizeof (UNMAP_BLOCK_DESCRIPTOR); + n > i; i++) { + PUNMAP_BLOCK_DESCRIPTOR Src = &DataBuffer->Descriptors[i]; + UINT64 BlockAddress = + ((UINT64)Src->StartingLba[0] << 56) | + ((UINT64)Src->StartingLba[1] << 48) | + ((UINT64)Src->StartingLba[2] << 40) | + ((UINT64)Src->StartingLba[3] << 32) | + ((UINT64)Src->StartingLba[4] << 24) | + ((UINT64)Src->StartingLba[5] << 16) | + ((UINT64)Src->StartingLba[6] << 8) | + ((UINT64)Src->StartingLba[7]); + UINT32 BlockCount = + ((UINT32)Src->LbaCount[0] << 24) | + ((UINT32)Src->LbaCount[1] << 16) | + ((UINT32)Src->LbaCount[2] << 8) | + ((UINT32)Src->LbaCount[3]); + + ret = zvol_os_unmap(zv, + BlockAddress * zv->zv_volblocksize, + BlockCount * zv->zv_volblocksize); + if (ret != 0) + break; + } + + if (ret != 0) + return (SCSI_SENSE_ILLEGAL_REQUEST); + + return (SRB_STATUS_SUCCESS); +} + +UCHAR +ScsiReadWriteSetup( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb, + __in MpWkRtnAction WkRtnAction, + __in PUCHAR pResult) +{ + PHW_SRB_EXTENSION pSrbExt = pSrb->SrbExtension; + pMP_WorkRtnParms pWkRtnParms = &pSrbExt->WkRtnParms; + + ASSERT(pSrb->DataBuffer != NULL); + + *pResult = ResultDone; + + RtlZeroMemory(pWkRtnParms, sizeof (MP_WorkRtnParms)); + + pWkRtnParms->pHBAExt = pHBAExt; + pWkRtnParms->pSrb = pSrb; + pWkRtnParms->Action = WkRtnAction; + + IoInitializeWorkItem((PDEVICE_OBJECT)pHBAExt->pDrvObj, + (PIO_WORKITEM)pWkRtnParms->pQueueWorkItem); + + // Save the SRB in a list allowing cancellation via + // SRB_FUNCTION_RESET_xxx + pSrbExt->pSrbBackPtr = pSrb; + pSrbExt->Cancelled = 0; + KIRQL oldIrql; + KeAcquireSpinLock(&pHBAExt->pwzvolDrvObj->SrbExtLock, &oldIrql); + InsertTailList(&pHBAExt->pwzvolDrvObj->ListSrbExt, + &pSrbExt->QueuedForProcessing); + KeReleaseSpinLock(&pHBAExt->pwzvolDrvObj->SrbExtLock, oldIrql); + + // Queue work item, which will run in the System process. + + IoQueueWorkItem((PIO_WORKITEM)pWkRtnParms->pQueueWorkItem, + wzvol_GeneralWkRtn, DelayedWorkQueue, pWkRtnParms); + + *pResult = ResultQueued; + return (SRB_STATUS_SUCCESS); +} + +UCHAR +ScsiOpModeSense( + __in pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb) +{ + UNREFERENCED_PARAMETER(pHBAExt); + + RtlZeroMemory((PUCHAR)pSrb->DataBuffer, pSrb->DataTransferLength); + + return (SRB_STATUS_SUCCESS); +} + +UCHAR +ScsiOpReportLuns( + __in __out pHW_HBA_EXT pHBAExt, + __in PSCSI_REQUEST_BLOCK pSrb) +{ + UCHAR status = SRB_STATUS_SUCCESS; + PLUN_LIST pLunList = (PLUN_LIST)pSrb->DataBuffer; // Point to LUN list. + uint8_t GoodLunIdx = 0; + uint8_t totalLun = 0; + zvol_state_t *zv; + + if (FALSE == pHBAExt->bReportAdapterDone) { + wzvol_HwReportAdapter(pHBAExt); + wzvol_HwReportLink(pHBAExt); + wzvol_HwReportLog(pHBAExt); + pHBAExt->bReportAdapterDone = TRUE; + } + + RtlZeroMemory((PUCHAR)pSrb->DataBuffer, pSrb->DataTransferLength); + if (!pHBAExt->bDontReport) { + for (uint8_t i = 0; + i < STOR_wzvolDriverInfo.MaximumNumberOfLogicalUnits; + i++) { + // make sure we have the space for 1 more LUN each time. + if ((zv = wzvol_find_target(pSrb->TargetId, i)) != + NULL) { + totalLun++; + if (pSrb->DataTransferLength >= + FIELD_OFFSET(LUN_LIST, Lun) + + (GoodLunIdx * sizeof (pLunList->Lun[0])) + + sizeof (pLunList->Lun[0])) { + pLunList->Lun[GoodLunIdx][1] = (UCHAR)i; + GoodLunIdx++; + } + wzvol_unlock_target(zv); + } + } + } + + *((ULONG*)&pLunList->LunListLength) = + RtlUlongByteSwap(totalLun * sizeof (pLunList->Lun[0])); + pSrb->DataTransferLength = FIELD_OFFSET(LUN_LIST, Lun) + + (GoodLunIdx * sizeof (pLunList->Lun[0])); + + return (status); +} + +void +wzvol_WkRtn(__in PVOID pWkParms) +{ + pMP_WorkRtnParms pWkRtnParms = (pMP_WorkRtnParms)pWkParms; + pHW_HBA_EXT pHBAExt = pWkRtnParms->pHBAExt; + PSCSI_REQUEST_BLOCK pSrb = pWkRtnParms->pSrb; + PCDB pCdb = (PCDB)pSrb->Cdb; + PHW_SRB_EXTENSION pSrbExt = (PHW_SRB_EXTENSION)pSrb->SrbExtension; + ULONGLONG startingSector = 0ULL, sectorOffset = 0ULL; + UCHAR status; + int flags = 0; + zvol_state_t *zv = NULL; + + // Find out if that SRB has been cancelled and busy it back if it was. + KIRQL oldIrql; + KeAcquireSpinLock(&pHBAExt->pwzvolDrvObj->SrbExtLock, &oldIrql); + RemoveEntryList(&pSrbExt->QueuedForProcessing); + KeReleaseSpinLock(&pHBAExt->pwzvolDrvObj->SrbExtLock, oldIrql); + if (pSrbExt->Cancelled) { + status = SRB_STATUS_BUSY; + goto Done; + } + ASSERT(pSrb->DataBuffer != NULL); + + zv = wzvol_find_target(pSrb->TargetId, pSrb->Lun); + if (zv == NULL) { + status = SRB_STATUS_NO_DEVICE; + goto Done; + } + + if (ActionUnmap == pWkRtnParms->Action) { + status = ScsiOpUnmap_impl(pHBAExt, pSrb, zv); + goto Done; + } + + if (pSrb->CdbLength == 10) { + startingSector = (ULONG)pCdb->CDB10.LogicalBlockByte3 | + pCdb->CDB10.LogicalBlockByte2 << 8 | + pCdb->CDB10.LogicalBlockByte1 << 16 | + pCdb->CDB10.LogicalBlockByte0 << 24; + if (pCdb->CDB10.ForceUnitAccess) + flags |= ZVOL_WRITE_SYNC; + } else if (pSrb->CdbLength == 16) { + REVERSE_BYTES_QUAD(&startingSector, pCdb->CDB16.LogicalBlock); + if (pCdb->CDB16.ForceUnitAccess) + flags |= ZVOL_WRITE_SYNC; + } else { + status = SRB_STATUS_ERROR; + goto Done; + } + + sectorOffset = startingSector * MP_BLOCK_SIZE; + + TraceEvent(TRACE_VERBOSE, "%s:%d: MpWkRtn Action: %X, starting sector:" + " 0x%llX, sector offset: 0x%llX\n", __func__, __LINE__, + pWkRtnParms->Action, startingSector, sectorOffset); + TraceEvent(TRACE_VERBOSE, "%s:%d: MpWkRtn pSrb: 0x%p, pSrb->DataBuffer" + ": 0x%p\n", __func__, __LINE__, pSrb, pSrb->DataBuffer); + + if (sectorOffset >= zv->zv_volsize) { + dprintf("%s:%d invalid starting sector: %d for zvol:%s, " + "volsize=%llu\n", __func__, __LINE__, startingSector, + zv->zv_name, zv->zv_volsize); + status = SRB_STATUS_INVALID_REQUEST; + goto Done; + } + + // Create an uio for the IO. If we can possibly embed + // the uio in some Extension to this IO, we could + // save the allocation here. + struct iovec iov; + iov.iov_base = (void *)pSrb->DataBuffer; + iov.iov_len = pSrb->DataTransferLength; + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, sectorOffset, UIO_SYSSPACE, + pSrb->DataTransferLength, 0); + // ActionRead == pWkRtnParms->Action ? UIO_READ : UIO_WRITE); + + /* Call ZFS to read/write data */ + if (ActionRead == pWkRtnParms->Action) { + status = zvol_os_read_zv(zv, &uio, flags); + } else { + status = zvol_os_write_zv(zv, &uio, flags); + } + + if (status == 0) + status = SRB_STATUS_SUCCESS; + +Done: + if (zv) + wzvol_unlock_target(zv); + + pSrb->SrbStatus = status; + + // Tell StorPort this action has been completed. + StorPortNotification(RequestComplete, pHBAExt, pSrb); +} + +void +wzvol_GeneralWkRtn( + __in PVOID pDummy, + __in PVOID pWkParms) +{ + pMP_WorkRtnParms pWkRtnParms = (pMP_WorkRtnParms)pWkParms; + + UNREFERENCED_PARAMETER(pDummy); + IoUninitializeWorkItem((PIO_WORKITEM)pWkRtnParms->pQueueWorkItem); + + // If the next starts, it has to be stopped by a kernel debugger. + + while (pWkRtnParms->SecondsToDelay) { + LARGE_INTEGER delay; + + delay.QuadPart = + -10 * 1000 * 1000 * pWkRtnParms->SecondsToDelay; + + KeDelayExecutionThread(KernelMode, TRUE, &delay); + } + + wzvol_WkRtn(pWkParms); + +} + +/* + * ZFS ZVOLDI + * ZVOL Direct Interface + */ +void +bzvol_ReadWriteTaskRtn(__in PVOID pWkParms) +{ + NTSTATUS Status; + pMP_WorkRtnParms pWkRtnParms = (pMP_WorkRtnParms)pWkParms; + zfsiodesc_t *pIo = &pWkRtnParms->ioDesc; + int iores; + + struct iovec iov; + iov.iov_base = (void*)pIo->Buffer; + iov.iov_len = pIo->Length; + + zfs_uio_t uio; + zfs_uio_iovec_init(&uio, &iov, 1, pIo->ByteOffset, UIO_SYSSPACE, + pIo->Length, 0); + + /* Call ZFS to read/write data */ + + if (ActionRead == pWkRtnParms->Action) { + iores = zvol_os_read_zv(pWkRtnParms->zv, &uio, 0); + } else { + /* TODO add flag if FUA */ + iores = zvol_os_write_zv(pWkRtnParms->zv, &uio, 0); + } + + if (pIo->Cb) { + pIo->Cb(pIo, iores == 0 ? STATUS_SUCCESS : STATUS_UNSUCCESSFUL, + TRUE); + } + + ExFreePoolWithTag(pWkRtnParms, MP_TAG_GENERAL); +} + +void +bzvol_TaskQueuingWkRtn(__in PVOID pDummy, __in PVOID pWkParms) +{ + pMP_WorkRtnParms pWkRtnParms = (pMP_WorkRtnParms)pWkParms; + zfsiodesc_t *pIo = &pWkRtnParms->ioDesc; + PIO_WORKITEM pWI = (PIO_WORKITEM)ALIGN_UP_POINTER_BY( + pWkRtnParms->pQueueWorkItem, 16); + + UNREFERENCED_PARAMETER(pDummy); + IoUninitializeWorkItem(pWI); + + if (pIo->Flags & ZFSZVOLFG_AlwaysPend) { + taskq_init_ent(&pWkRtnParms->ent); + taskq_dispatch_ent(zvol_taskq, bzvol_ReadWriteTaskRtn, + pWkRtnParms, 0, &pWkRtnParms->ent); + } else { + // bypass the taskq and do everything under workitem thread + // context. + bzvol_ReadWriteTaskRtn(pWkRtnParms); + } + // workitem freed inside bzvol_ReadWriteTaskRtn +} + +NTSTATUS +DiReadWriteSetup(zvol_state_t *zv, MpWkRtnAction action, zfsiodesc_t *pIo) +{ + // cannot use kmem_alloc with sleep if IRQL dispatch so get straight + // from NP pool. + pMP_WorkRtnParms pWkRtnParms = (pMP_WorkRtnParms)ExAllocatePoolWithTag( + NonPagedPoolNx, ALIGN_UP_BY(sizeof (MP_WorkRtnParms), 16) + + IoSizeofWorkItem(), MP_TAG_GENERAL); + if (NULL == pWkRtnParms) { + if (pIo->Cb) { + pIo->Cb(pIo, STATUS_INSUFFICIENT_RESOURCES, FALSE); + } + return (STATUS_INSUFFICIENT_RESOURCES); + } + RtlZeroMemory(pWkRtnParms, sizeof (MP_WorkRtnParms)); + pWkRtnParms->ioDesc = *pIo; + pWkRtnParms->zv = zv; + pWkRtnParms->Action = action; + + /* + * SSV-19147: cannot use taskq queuing at dispatch. must queue a work + * item to do it. Since taskq queueing involves a possibly waiting mutex + * we do not want to slow down the caller so perform taskq queuing + * always in the workitem. + */ + extern PDEVICE_OBJECT ioctlDeviceObject; + PIO_WORKITEM pWI = (PIO_WORKITEM)ALIGN_UP_POINTER_BY( + pWkRtnParms->pQueueWorkItem, 16); + IoInitializeWorkItem(ioctlDeviceObject, pWI); + IoQueueWorkItem(pWI, bzvol_TaskQueuingWkRtn, DelayedWorkQueue, + pWkRtnParms); + return (STATUS_PENDING); // queued up. +} + +NTSTATUS +ZvolDiRead(PVOID Context, zfsiodesc_t *pIo) +{ + return (DiReadWriteSetup((zvol_state_t *)Context, ActionRead, pIo)); +} + +NTSTATUS +ZvolDiWrite(PVOID Context, zfsiodesc_t *pIo) +{ + return (DiReadWriteSetup((zvol_state_t *)Context, ActionWrite, pIo)); +} diff --git a/module/os/windows/zfs/zfs_windows_zvol_wmi.c b/module/os/windows/zfs/zfs_windows_zvol_wmi.c new file mode 100644 index 000000000000..927a77b1944d --- /dev/null +++ b/module/os/windows/zfs/zfs_windows_zvol_wmi.c @@ -0,0 +1,1172 @@ +/* + * Module Name: wmi.c + * Project: CppWDKStorPortVirtualMiniport + * + * Copyright (c) Microsoft Corporation. + * + * a. HandleWmiSrb() + * Handles WMI SRBs, possibly by calling a subroutine. + * + * b. QueryWmiDataBlock() + * Supports WMI Query Data Block. + * + * c. ExecuteWmiMethod() + * Supports WMI Execute Method. + * + * This source is subject to the Microsoft Public License. + * See http://www.microsoft.com/opensource/licenses.mspx#Ms-PL. + * All other rights reserved. + * + * THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, + * EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// #include "trace.h" +// #include "wmi.tmh" + +// Paper about FC WMI classes, including C++ and VBS examples: +// +// http:/ /download.microsoft.com/download/9/c/5/ +// 9c5b2167-8017-4bae-9fde-d599bac8184a/FCTopology.doc +// + +#define IdxGmDemoDriver_GUID 0 +#define IdxGmDemoDriver2_GUID 1 +#define IdxGmDemoDriverSrbActivity_GUID 2 +#define IdxGmDrvDrvMethodGuid 3 +#define IdxMSFC_AdapterEvent_GUID 4 +#define IdxMSFC_LinkEvent_GUID 5 +#define IdxMSFC_FibrePortHBAStatistics_GUID 6 +#define IdxMSFC_FibrePortHBAAttributes_GUID 7 +#define IdxMSFC_FCAdapterHBAAttributes_GUID 8 +#define IdxMSFC_HBAFCPInfo_GUID 9 +#define IdxMSFC_FibrePortHBAMethods_GUID 10 +#define IdxMSFC_HBAAdapterMethods_GUID 11 +#define IdxMSFC_HBAPortStatistics_GUID 12 + +#define NUMBEROFPORTS 1 + +SCSIWMIGUIDREGINFO WmiGuidList[] = // GUIDs supported. +{ + {&GmDemoDriver_GUID, 1, WMIREG_FLAG_INSTANCE_PDO}, + {&GmDemoDriver2_GUID, 1, WMIREG_FLAG_INSTANCE_PDO}, + {&GmDemoDriverSrbActivity_GUID, 1, WMIREG_FLAG_INSTANCE_PDO}, + {&GmDrvDrvMethod_GUID, 1, 0x0}, + {&MSFC_AdapterEvent_GUID, NUMBEROFPORTS, 0x0}, + {&MSFC_LinkEvent_GUID, NUMBEROFPORTS, 0x0}, + {&MSFC_FibrePortHBAStatistics_GUID, NUMBEROFPORTS, 0x0}, + {&MSFC_FibrePortHBAAttributes_GUID, NUMBEROFPORTS, 0x0}, + {&MSFC_FCAdapterHBAAttributes_GUID, NUMBEROFPORTS, 0x0}, + {&MSFC_HBAFCPInfo_GUID, NUMBEROFPORTS, 0x0}, + {&MSFC_FibrePortHBAMethods_GUID, NUMBEROFPORTS, 0x0}, + {&MSFC_HBAAdapterMethods_GUID, NUMBEROFPORTS, 0x0}, + {&MSFC_HBAPortStatistics_GUID, NUMBEROFPORTS, 0x0}, +}; + +#define WmiGuidCount (sizeof (WmiGuidList) / sizeof (SCSIWMIGUIDREGINFO)) + +UCHAR +QueryWmiRegInfo(IN PVOID pContext, + IN PSCSIWMI_REQUEST_CONTEXT pRequestContext, + OUT PWCHAR *pMofResourceName); + +BOOLEAN +QueryWmiDataBlock( + IN PVOID pContext, + IN PSCSIWMI_REQUEST_CONTEXT pDispatchContext, + IN ULONG GuidIndex, + IN ULONG InstanceIndex, + IN ULONG InstanceCount, + IN OUT PULONG pInstanceLengthArray, + IN ULONG BufferAvail, + OUT PUCHAR); + +UCHAR +SetWmiDataBlock( + IN PVOID pContext, + IN PSCSIWMI_REQUEST_CONTEXT pDispatchContext, + IN ULONG GuidIndex, + IN ULONG InstanceIndex, + IN ULONG BufferSize, + IN PUCHAR pBuffer); + +UCHAR +ExecuteWmiMethod( + IN PVOID pContext, + IN PSCSIWMI_REQUEST_CONTEXT pDispatchContext, + IN ULONG GuidIndex, + IN ULONG InstanceIndex, + IN ULONG MethodId, + IN ULONG InBufferSize, + IN ULONG OutBufferSize, + IN OUT PUCHAR pBuffer); + +void +SpUpdateWmiRequest( + pHW_HBA_EXT pHbaExtension, + PSCSI_WMI_REQUEST_BLOCK pSrb, + PSCSIWMI_REQUEST_CONTEXT pDispatchContext, + UCHAR Status, + ULONG SizeNeeded); + +void +InitializeWmiContext(__in pHW_HBA_EXT pHbaExtension) +{ + PSCSI_WMILIB_CONTEXT pWmiLibContext = + &(pHbaExtension->WmiLibContext); + + RtlZeroMemory(pWmiLibContext, + sizeof (SCSI_WMILIB_CONTEXT)); + + pWmiLibContext->GuidCount = WmiGuidCount; + pWmiLibContext->GuidList = WmiGuidList; + + // + // Point to WMI callback routines. + // + pWmiLibContext->QueryWmiRegInfo = QueryWmiRegInfo; + pWmiLibContext->QueryWmiDataBlock = QueryWmiDataBlock; + pWmiLibContext->SetWmiDataBlock = SetWmiDataBlock; + pWmiLibContext->ExecuteWmiMethod = ExecuteWmiMethod; + pWmiLibContext->WmiFunctionControl = NULL; +} + +BOOLEAN +HandleWmiSrb( + __in pHW_HBA_EXT pHbaExtension, + __in __out PSCSI_WMI_REQUEST_BLOCK pSrb) +{ + PSCSIWMI_REQUEST_CONTEXT pRequestContext; + PHW_SRB_EXTENSION pSrbExtension; + BOOLEAN bPending = FALSE; + + // + // Validate our assumptions. + // + ASSERT(pSrb->Function == SRB_FUNCTION_WMI); + ASSERT(pSrb->Length == sizeof (SCSI_WMI_REQUEST_BLOCK)); + + if (!(pSrb->WMIFlags & SRB_WMI_FLAGS_ADAPTER_REQUEST)) { + + // + // This is targetted to one of the disks, since there is + // no per-disk WMI information we return an error. + // Note that if there was per-disk information, then you'd + // likely have a different WmiLibContext and a different + // set of GUIDs. + // + pSrb->DataTransferLength = 0; + pSrb->SrbStatus = SRB_STATUS_NO_DEVICE; + + } else { + if (IRP_MN_ENABLE_EVENTS == pSrb->WMISubFunction) { + } + + pSrbExtension = (PHW_SRB_EXTENSION)pSrb->SrbExtension; + pRequestContext = &(pSrbExtension->WmiRequestContext); + + // + // Save the pointer to the SRB in UserContext of + // SCSIWMI_REQUEST_CONTEXT + // + pRequestContext->UserContext = pSrb; + + // + // Process the incoming WMI request. + // + + bPending = ScsiPortWmiDispatchFunction( + &pHbaExtension->WmiLibContext, + pSrb->WMISubFunction, + pHbaExtension, + pRequestContext, + pSrb->DataPath, + pSrb->DataTransferLength, + pSrb->DataBuffer); + + // If the request is complete, status and transfer + // length aren't ever going to be set. + + if (FALSE == bPending) { + pSrb->DataTransferLength = + ScsiPortWmiGetReturnSize(pRequestContext); + pSrb->SrbStatus = + ScsiPortWmiGetReturnStatus(pRequestContext); + } + } + + return (TRUE); +} + +UCHAR +QueryWmiRegInfo( + __in PVOID pContext, + __in PSCSIWMI_REQUEST_CONTEXT pRequestContext, + __out PWCHAR *pMofResourceName) +{ + KIRQL saveIRQL; + UNREFERENCED_PARAMETER(pContext); + UNREFERENCED_PARAMETER(pRequestContext); + + saveIRQL = KeGetCurrentIrql(); + + *pMofResourceName = L"MofResource"; + + return (SRB_STATUS_SUCCESS); +} + +BOOLEAN +QueryWmiDataBlock( + __in PVOID pContext, + __in PSCSIWMI_REQUEST_CONTEXT pDispatchContext, + __in ULONG GuidIndex, + __in ULONG InstanceIndex, + __in ULONG InstanceCount, + __in __out PULONG pInstanceLenArr, + __in ULONG BufferAvail, + __out PUCHAR pBuffer) +{ + pHW_HBA_EXT pHbaExtension = (pHW_HBA_EXT)pContext; + PSCSI_WMI_REQUEST_BLOCK pSrb = + (PSCSI_WMI_REQUEST_BLOCK)pDispatchContext->UserContext; + ULONG sizeNeeded = 0, + i, + IdxLim = InstanceIndex + InstanceCount, + ulBfrUsed = 0, + LastIndex, + InstanceSize; + UCHAR status = SRB_STATUS_SUCCESS; + PWCHAR pBfrX; + + switch (GuidIndex) { + case IdxGmDemoDriver_GUID: + + // Demo index. + + sizeNeeded = GmDemoDriver_SIZE; + + if (BufferAvail < sizeNeeded) { + status = SRB_STATUS_DATA_OVERRUN; + break; + } + + for (i = InstanceIndex; i < IdxLim; i++) { + PGmDemoDriver pOut = + (PGmDemoDriver) + ((PUCHAR)pBuffer + ulBfrUsed); + + pOut->TheAnswer = 22; + pOut->TheNextAnswer = 23; + pOut->SRBsSeen = pHbaExtension->SRBsSeen; + pOut->WMISRBsSeen = pHbaExtension->WMISRBsSeen; + + pInstanceLenArr[i] = sizeNeeded; + ulBfrUsed += pInstanceLenArr[i]; + } + + break; + + case IdxGmDemoDriver2_GUID: + sizeNeeded = sizeof (ULONG); + + if (BufferAvail < sizeNeeded) { + status = SRB_STATUS_DATA_OVERRUN; + break; + } + + status = SRB_STATUS_INVALID_REQUEST; + break; + + case IdxGmDemoDriverSrbActivity_GUID: + sizeNeeded = 0; + break; + + case IdxGmDrvDrvMethodGuid: + sizeNeeded = sizeof (ULONG); + + if (BufferAvail < sizeNeeded) { + status = SRB_STATUS_DATA_OVERRUN; + break; + } + break; + + case IdxMSFC_FibrePortHBAStatistics_GUID: { + PMSFC_FibrePortHBAStatistics pPortStats; + PMSFC_HBAPortStatistics pHBAPortStats; + + // Verify there is enough room in the output buffer + // to return all data requested. + // Calculate size, rounding up to a multiple of + // 8 if needed. + InstanceSize = + (sizeof (MSFC_FibrePortHBAStatistics)+7) & ~7; + sizeNeeded = InstanceCount * InstanceSize; + + if (BufferAvail >= sizeNeeded) { + LastIndex = InstanceIndex + InstanceCount; + + for (i = InstanceIndex, + pPortStats = + (PMSFC_FibrePortHBAStatistics)pBuffer; + i < LastIndex; + i++, pPortStats++) { + // Set a unique value for the port. + + pPortStats->UniquePortId = + (ULONGLONG)pHbaExtension + i; + + pPortStats->HBAStatus = HBA_STATUS_OK; + + pHBAPortStats = + &pPortStats->Statistics; + + pHBAPortStats->SecondsSinceLastReset = + 10; + pHBAPortStats->TxFrames = 11; + pHBAPortStats->TxWords = 12; + pHBAPortStats->RxFrames = 13; + pHBAPortStats->RxWords = 14; + pHBAPortStats->LIPCount = 15; + pHBAPortStats->NOSCount = 16; + pHBAPortStats->ErrorFrames = 17; + pHBAPortStats->DumpedFrames = 18; + pHBAPortStats->LinkFailureCount = 19; + pHBAPortStats->LossOfSyncCount = 20; + pHBAPortStats->LossOfSignalCount = 21; + pHBAPortStats-> + PrimitiveSeqProtocolErrCount = 22; + pHBAPortStats->InvalidTxWordCount = 23; + pHBAPortStats->InvalidCRCCount = 24; + + *pInstanceLenArr++ = + sizeof (MSFC_FibrePortHBAStatistics); + } + } else { + status = SRB_STATUS_DATA_OVERRUN; + } + + break; + } + + case IdxMSFC_HBAPortStatistics_GUID: { + PMSFC_HBAPortStatistics pHBAPortStats; + PUCHAR pBuffer2 = pBuffer; + + InstanceSize = (sizeof (MSFC_HBAPortStatistics)+7) & ~7; + sizeNeeded = InstanceCount * InstanceSize; + + if (BufferAvail >= sizeNeeded) { + LastIndex = InstanceIndex + InstanceCount; + + for (i = InstanceIndex; i < LastIndex; i++) { + pHBAPortStats = + (PMSFC_HBAPortStatistics)pBuffer2; + + memset(pBuffer2, 0, InstanceSize); + + pHBAPortStats->SecondsSinceLastReset = + 0; + pHBAPortStats->TxFrames = 1; + pHBAPortStats->TxWords = 2; + pHBAPortStats->RxFrames = 3; + pHBAPortStats->RxWords = 4; + pHBAPortStats->LIPCount = 5; + pHBAPortStats->NOSCount = 6; + pHBAPortStats->ErrorFrames = 7; + pHBAPortStats->DumpedFrames = 8; + pHBAPortStats->LinkFailureCount = 9; + pHBAPortStats->LossOfSyncCount = 10; + pHBAPortStats->LossOfSignalCount = 11; + pHBAPortStats-> + PrimitiveSeqProtocolErrCount = 12; + pHBAPortStats->InvalidTxWordCount = 13; + pHBAPortStats->InvalidCRCCount = 14; + + pBuffer2 += InstanceSize; + *pInstanceLenArr++ = + sizeof (MSFC_HBAPortStatistics); + } + } else { + status = SRB_STATUS_DATA_OVERRUN; + } + + break; + } + + case IdxMSFC_FibrePortHBAAttributes_GUID: { + PUCHAR pBuffer2 = pBuffer; + +#define FibrePortHBAAttributesNODEWWN "VM123456" +/* Will appear as 56:4D:32:33:34:35:36:37 */ +#define FibrePortHBAAttributesPortWWN "VM234567" +#define FibrePortHBAAttributesPortType 0x99 +#define FibrePortHBAAttributesPortSupportedFc4Types "VM345678" +#define FibrePortHBAAttributesPortActiveFc4Types "VM456789" +#define FibrePortHBAAttributesFabricName "VM56789A" + + InstanceSize = + (sizeof (MSFC_FibrePortHBAAttributes)+7)&~7; + sizeNeeded = InstanceCount * InstanceSize; + + if (BufferAvail >= sizeNeeded) { + LastIndex = InstanceIndex + InstanceCount; + + for (i = InstanceIndex; i < LastIndex; i++) { + PMSFC_FibrePortHBAAttributes + pFibrePortHBAAttributes = + (PMSFC_FibrePortHBAAttributes) + pBuffer2; + + memset(pBuffer2, 0, InstanceSize); + + pFibrePortHBAAttributes->UniquePortId = + ((ULONGLONG)pHbaExtension) + i; + pFibrePortHBAAttributes->HBAStatus = + HBA_STATUS_OK; + + memcpy(pFibrePortHBAAttributes->Attributes.NodeWWN, + FibrePortHBAAttributesNODEWWN, + sizeof (pFibrePortHBAAttributes->Attributes.NodeWWN)); + + memcpy(pFibrePortHBAAttributes->Attributes.PortWWN, + FibrePortHBAAttributesPortWWN, + sizeof (pFibrePortHBAAttributes->Attributes.PortWWN)); + + pFibrePortHBAAttributes->Attributes. + PortFcId = i + 0x100; + pFibrePortHBAAttributes->Attributes. + PortType = + FibrePortHBAAttributesPortType + i; + pFibrePortHBAAttributes->Attributes. + PortState = i; + pFibrePortHBAAttributes->Attributes. + PortSupportedClassofService = i; + + memcpy(pFibrePortHBAAttributes->Attributes.PortSupportedFc4Types, + FibrePortHBAAttributesPortSupportedFc4Types, + sizeof (pFibrePortHBAAttributes->Attributes.PortSupportedFc4Types)); + + memcpy(pFibrePortHBAAttributes->Attributes.PortActiveFc4Types, + FibrePortHBAAttributesPortActiveFc4Types, + sizeof (pFibrePortHBAAttributes->Attributes.PortActiveFc4Types)); + + pFibrePortHBAAttributes->Attributes. + PortSupportedSpeed = i * 2; + pFibrePortHBAAttributes->Attributes. + PortSpeed = i; + pFibrePortHBAAttributes->Attributes. + PortMaxFrameSize = i * 4; + + memcpy(pFibrePortHBAAttributes->Attributes.FabricName, + FibrePortHBAAttributesFabricName, + sizeof (pFibrePortHBAAttributes->Attributes.FabricName)); + + pFibrePortHBAAttributes->Attributes. + NumberofDiscoveredPorts = 1; + + pBuffer2 += InstanceSize; + *pInstanceLenArr++ = + sizeof (MSFC_FibrePortHBAAttributes); + } + } else { + status = SRB_STATUS_DATA_OVERRUN; + } + break; + } + +#define CopyWMIString(_pDest, _pSrc, _maxlength) \ + { \ + PUSHORT _pDestTemp = _pDest; \ + USHORT _length = _maxlength - sizeof (USHORT); \ + *_pDestTemp++ = _length;\ + _length = (USHORT)min(wcslen(_pSrc)*sizeof (WCHAR), _length); \ + memcpy(_pDestTemp, _pSrc, _length);\ + } + + case IdxMSFC_FCAdapterHBAAttributes_GUID: + { + PMSFC_FCAdapterHBAAttributes pFCAdapterHBAAttributes; + + // + // First thing to do is verify if there is enough + // room in the output buffer to return all data + // requested + // + sizeNeeded = (sizeof (MSFC_FCAdapterHBAAttributes)); + + if (BufferAvail >= sizeNeeded) { +#define FCAdapterHBAAttributesNODEWWN "12345678" +#define VENDORID 0x1234 +#define PRODUCTID 0x5678 +#define MANUFACTURER L"OpenZFS" +#define SERIALNUMBER L"ZVOL SerialNumber" +#define MODEL L"ZVOL Model" +#define MODELDESCRIPTION L"ZVOL ModelDescription" +#define NODESYMBOLICNAME L"ZVOL NodeSymbolicName" +#define HARDWAREVERSION L"ZVOL HardwareVersion" +#define DRIVERVERSION L"ZVOL DriverVersion" +#define OPTIONROMVERSION L"ZVOL OptionROMVersion" +#define DRIVERNAME L"ZVOL DriverName" +#define FIRMWAREVERSION L"ZVOL FirmwareVersion" +#define MFRDOMAIN L"ZVOL MfrDomain" + + // + // We know there is always only 1 instance + // for this guid + // + pFCAdapterHBAAttributes = + (PMSFC_FCAdapterHBAAttributes)pBuffer; + memset(pBuffer, 0, sizeNeeded); + pFCAdapterHBAAttributes->UniqueAdapterId = + (ULONGLONG)pHbaExtension; + + pFCAdapterHBAAttributes->HBAStatus = + HBA_STATUS_OK; + + memcpy(pFCAdapterHBAAttributes->NodeWWN, + FCAdapterHBAAttributesNODEWWN, + sizeof (pFCAdapterHBAAttributes->NodeWWN)); + + pFCAdapterHBAAttributes->VendorSpecificID = + VENDORID | (PRODUCTID<<16); + + pFCAdapterHBAAttributes->NumberOfPorts = + NUMBEROFPORTS; + + pBfrX = pFCAdapterHBAAttributes->Manufacturer; + + CopyWMIString(pFCAdapterHBAAttributes->Manufacturer, + MANUFACTURER, + sizeof (pFCAdapterHBAAttributes->Manufacturer)); + + CopyWMIString(pFCAdapterHBAAttributes->SerialNumber, + SERIALNUMBER, + sizeof (pFCAdapterHBAAttributes->SerialNumber)); + + CopyWMIString(pFCAdapterHBAAttributes->Model, + MODEL, + sizeof (pFCAdapterHBAAttributes->Model)); + + CopyWMIString(pFCAdapterHBAAttributes->ModelDescription, + MODELDESCRIPTION, + sizeof (pFCAdapterHBAAttributes->ModelDescription)); + + CopyWMIString(pFCAdapterHBAAttributes->NodeSymbolicName, + NODESYMBOLICNAME, + sizeof (pFCAdapterHBAAttributes->NodeSymbolicName)); + + CopyWMIString(pFCAdapterHBAAttributes->HardwareVersion, + HARDWAREVERSION, + sizeof (pFCAdapterHBAAttributes->HardwareVersion)); + + CopyWMIString(pFCAdapterHBAAttributes->DriverVersion, + DRIVERVERSION, + sizeof (pFCAdapterHBAAttributes->DriverVersion)); + + CopyWMIString(pFCAdapterHBAAttributes->OptionROMVersion, + OPTIONROMVERSION, + sizeof (pFCAdapterHBAAttributes->OptionROMVersion)); + + CopyWMIString(pFCAdapterHBAAttributes->FirmwareVersion, + FIRMWAREVERSION, + sizeof (pFCAdapterHBAAttributes->FirmwareVersion)); + + CopyWMIString(pFCAdapterHBAAttributes->DriverName, + DRIVERNAME, + sizeof (pFCAdapterHBAAttributes->DriverName)); + + CopyWMIString(pFCAdapterHBAAttributes->MfgDomain, + MFRDOMAIN, + sizeof (pFCAdapterHBAAttributes->MfgDomain)); + + pInstanceLenArr[0] = + sizeof (MSFC_FCAdapterHBAAttributes); + + } else { + status = SRB_STATUS_DATA_OVERRUN; + } + break; + } + + case IdxMSFC_HBAFCPInfo_GUID: + case IdxMSFC_FibrePortHBAMethods_GUID: + case IdxMSFC_HBAAdapterMethods_GUID: + { + // + // Methods don't return data per se, but must respond to + // queries with an empty data block. We know that all of + // these method guids only have one instance + // + sizeNeeded = sizeof (ULONG); + + if (BufferAvail >= sizeNeeded) { + pInstanceLenArr[0] = sizeNeeded; + } else { + status = SRB_STATUS_DATA_OVERRUN; + } + break; + } + + default: + status = SRB_STATUS_ERROR; + break; + } + + SpUpdateWmiRequest(pHbaExtension, pSrb, pDispatchContext, + status, sizeNeeded); + + return (SRB_STATUS_PENDING); +} + +UCHAR +SetWmiDataBlock( + __in PVOID pContext, + __in PSCSIWMI_REQUEST_CONTEXT pDispatchContext, + __in ULONG GuidIndex, + __in ULONG InstanceIndex, + __in ULONG BufferSize, + __in PUCHAR pBuffer) +{ + pHW_HBA_EXT pHbaExtension = (pHW_HBA_EXT)pContext; + PSCSI_WMI_REQUEST_BLOCK pSrb = + (PSCSI_WMI_REQUEST_BLOCK)pDispatchContext->UserContext; + UCHAR status = SRB_STATUS_SUCCESS; + ULONG sizeNeeded = 0; + + UNREFERENCED_PARAMETER(InstanceIndex); + UNREFERENCED_PARAMETER(BufferSize); + UNREFERENCED_PARAMETER(pBuffer); + + switch (GuidIndex) { + + default: + + status = SRB_STATUS_INVALID_REQUEST; + break; + } + + SpUpdateWmiRequest(pHbaExtension, pSrb, pDispatchContext, + status, sizeNeeded); + + return (SRB_STATUS_PENDING); +} + +UCHAR +ExecuteWmiMethod( + __in PVOID pContext, + __in PSCSIWMI_REQUEST_CONTEXT pDispatchContext, + __in ULONG GuidIndex, + __in ULONG InstanceIndex, + __in ULONG MethodId, + __in ULONG InBufferSize, + __in ULONG OutBufferSize, + __in __out PUCHAR pBuffer) +{ + pHW_HBA_EXT pHbaExtension = (pHW_HBA_EXT)pContext; + PSCSI_WMI_REQUEST_BLOCK pSrb = + (PSCSI_WMI_REQUEST_BLOCK)pDispatchContext->UserContext; + ULONG sizeNeeded = 0, + i; + UCHAR status = SRB_STATUS_SUCCESS; + + UNREFERENCED_PARAMETER(InstanceIndex); + + switch (GuidIndex) { + + case IdxMSFC_HBAFCPInfo_GUID: + + switch (MethodId) { + + // The next is the source of LUN that FCInfo + // gets (via HBAAPI!HbapGetFcpTargets). + + case GetFcpTargetMapping: { +#define PORTNAME L"VirtMini FibrePortName" +#define OSDEVICENAME L"VirtMini OsDeviceName" +#define LuidNAME "VirtMini Dummy LUID" + +#define FcpIdNODEWWN "23456789" +#define FcpIdPORTWWN "3456789A" + + sizeNeeded = FIELD_OFFSET( + GetFcpTargetMapping_OUT, + Entry) + + (pHbaExtension->NbrLUNsperHBA * + sizeof (HBAFCPScsiEntry)); + + if (OutBufferSize >= sizeNeeded) { + PGetFcpTargetMapping_IN pIn = + (PGetFcpTargetMapping_IN) + pBuffer; + PGetFcpTargetMapping_OUT pOut = + (PGetFcpTargetMapping_OUT) + pBuffer; + + pIn; + + RtlZeroMemory(pOut, OutBufferSize); + + pOut->HBAStatus = HBA_STATUS_OK; + + pOut->TotalEntryCount = + pHbaExtension->NbrLUNsperHBA; + + pOut->OutEntryCount = + pHbaExtension->NbrLUNsperHBA; + + for (i = 0; i < pHbaExtension->NbrLUNsperHBA; + i++) { + // Construct the present HBAFCPID. + + pOut->Entry[i].FCPId.Fcid = i; + + memcpy(pOut->Entry[i].FCPId.NodeWWN, + FcpIdNODEWWN, + sizeof (pOut->Entry[i].FCPId.NodeWWN)); + + memcpy(pOut->Entry[i].FCPId.PortWWN, + FcpIdPORTWWN, + sizeof (pOut->Entry[i].FCPId.PortWWN)); + + pOut->Entry[i].FCPId.FcpLun = i; + + // Construct Luid. + + memcpy(pOut->Entry[i].Luid, LuidNAME, + min(sizeof (pOut->Entry[i].Luid), + sizeof (LuidNAME)-1)); + + // Construct the present HBAScsiID. + + pOut->Entry[i].ScsiId.ScsiBusNumber = 0; + pOut->Entry[i].ScsiId.ScsiTargetNumber = + 0; + pOut->Entry[i].ScsiId.ScsiOSLun = i; + } + } else { + status = SRB_STATUS_DATA_OVERRUN; + } + + break; + } + + default: + + __try { + DbgBreakPoint(); + } + __except(EXCEPTION_EXECUTE_HANDLER) { + } + + status = SRB_STATUS_INVALID_REQUEST; + break; + } + + break; + + case IdxMSFC_HBAAdapterMethods_GUID: + + switch (MethodId) { + + case GetDiscoveredPortAttributes: { + PGetDiscoveredPortAttributes_OUT pOut = + (PGetDiscoveredPortAttributes_OUT) + pBuffer; + sizeNeeded = + GetDiscoveredPortAttributes_OUT_SIZE; + + if (OutBufferSize >= sizeNeeded) { + memset(pOut, 0, sizeNeeded); + + // since this is a virtual + // driver with no discovered + // ports,always return an error + pOut->HBAStatus = + HBA_STATUS_ERROR_ILLEGAL_INDEX; + } else { + status = + SRB_STATUS_DATA_OVERRUN; + } + + break; + } + + case RefreshInformation: { + break; + } + + case ScsiInquiry: { + PScsiInquiry_OUT pOut = + (PScsiInquiry_OUT)pBuffer; + PScsiInquiry_IN pIn = + (PScsiInquiry_IN)pBuffer; + PINQUIRYDATA pInqData = + (PINQUIRYDATA)pOut->ResponseBuffer; + struct _CDB6INQUIRY3 *pCdbInq = + (struct _CDB6INQUIRY3 *)pBuffer; + + sizeNeeded = + FIELD_OFFSET(ScsiInquiry_OUT, + ResponseBuffer) + + sizeof (INQUIRYDATA); + + if (OutBufferSize < sizeNeeded) { + status = + SRB_STATUS_DATA_OVERRUN; +// Taken from \\tkcsssrcidx01\windows2008r2-windows7\rtm\drivers\ +// oem\src\storage\elxstor\elxstorwmi.c, +// ElxWmiScsiOperations(). + pIn = NULL; + pCdbInq = NULL; + + break; + } + + memset(pOut, 0, OutBufferSize); + + pOut->HBAStatus = HBA_STATUS_OK; + pOut->ResponseBufferSize = + sizeof (INQUIRYDATA); + pOut->SenseBufferSize = 0; + pOut->ScsiStatus = 0; + + pInqData->DeviceType = DISK_DEVICE; + pInqData->RemovableMedia = FALSE; + pInqData->CommandQueue = TRUE; + + RtlMoveMemory(pInqData->VendorId, + VENDOR_ID_ascii, + sizeof (pInqData->VendorId)); + RtlMoveMemory(pInqData->ProductId, + PRODUCT_ID_ascii, + sizeof (pInqData->ProductId)); + RtlMoveMemory(pInqData->ProductRevisionLevel, + PRODUCT_REV_ascii, + sizeof (pInqData->ProductRevisionLevel)); + + break; + } + + case SendCTPassThru: { + +#define minSizeNeeded 0x1000 + +// +// Derived from \\tkcsssrcidx01\windows2008r2-windows7\rtm\drivers\ +// storage\hbaapi\dll\ctpass.h +// + +// Response codes, see 4.3.1.6 FC-GS3. Note they are reversed since +// they are little endian in the packet. + +#define CTREJECT 0x0180 +#define CTACCESPT 0x0280 + +// CT Passthru definitions. See section 4.3 in FC-GS3 + + typedef struct _CTPREAMBLE { + UCHAR Revision; + UCHAR IN_ID[3]; + UCHAR GS_Type; + UCHAR GS_SubType; + UCHAR Options; + UCHAR Reserved1; + USHORT CommandResponse; + union { + USHORT MaxResidualSize; + USHORT MaxSize; + } _xx; + UCHAR Reserved2; + UCHAR Reason; + UCHAR ReasonExplaination; + UCHAR VendorSpecific; + } CTPREAMBLE, *PCTPREAMBLE; + + typedef struct _CTPASSTHRU_GSPN_ID_ACCEPT { + CTPREAMBLE Preamble; + UCHAR SymbolicNameLen; + UCHAR SymbolicName[1]; // symbolic name + } CTPASSTHRU_GSPN_ID_ACCEPT, *PCTPASSTHRU_GSPN_ID_ACCEPT; + + PSendCTPassThru_IN pIn; + PSendCTPassThru_OUT pOut; + ULONG RequestCount, + ResponseCount; + + if (InBufferSize >= sizeof (ULONG)) { + pIn = (PSendCTPassThru_IN)pBuffer; + + RequestCount = pIn->RequestBufferCount; + sizeNeeded = FIELD_OFFSET(SendCTPassThru_IN, + RequestBuffer) + RequestCount; + + if (InBufferSize >= sizeNeeded) { +#define RESPONSE_BUFFER_SIZE 0x1000 + + ResponseCount = RESPONSE_BUFFER_SIZE; + sizeNeeded = FIELD_OFFSET(SendCTPassThru_OUT, + ResponseBuffer) + ResponseCount; + + if (OutBufferSize >= sizeNeeded) { + PCTPASSTHRU_GSPN_ID_ACCEPT pRespBfr; +#define SYMBOLICNAME "VMSymName" + + pOut = (PSendCTPassThru_OUT)pBuffer; + pOut->HBAStatus = HBA_STATUS_OK; + pOut->TotalResponseBufferCount = + ResponseCount; + pOut->ActualResponseBufferCount = + ResponseCount; + + pRespBfr = + (PCTPASSTHRU_GSPN_ID_ACCEPT) + pOut->ResponseBuffer; + + memset(pRespBfr, 0, ResponseCount); + + pRespBfr->Preamble.CommandResponse = + CTACCESPT; + pRespBfr->SymbolicNameLen = + sizeof (SYMBOLICNAME)-1; + memcpy(pRespBfr->SymbolicName, + SYMBOLICNAME, + pRespBfr->SymbolicNameLen); + } else { + status = SRB_STATUS_DATA_OVERRUN; + } + } else { + status = SRB_STATUS_ERROR; + } + } else { + sizeNeeded = minSizeNeeded; + status = SRB_STATUS_ERROR; + } + + break; + } + + case ScsiReadCapacity: { + PScsiReadCapacity_IN pIn; + PScsiReadCapacity_OUT pOut; + PREAD_CAPACITY_DATA pReadCapData; + + if (InBufferSize >= sizeof (ULONG)) { + pIn = (PScsiReadCapacity_IN)pBuffer; + sizeNeeded = sizeof (ScsiReadCapacity_IN); + if (InBufferSize >= sizeNeeded) { + sizeNeeded = FIELD_OFFSET(ScsiReadCapacity_OUT, + ResponseBuffer) + + ScsiReadCapacity_OUT_ResponseBuffer_SIZE_HINT; + + if (OutBufferSize >= sizeNeeded) { + pOut = (PScsiReadCapacity_OUT) + pBuffer; + pOut->HBAStatus = HBA_STATUS_OK; + pOut->ResponseBufferSize = + sizeNeeded; + pOut->SenseBufferSize = 0; + + pReadCapData = + (PREAD_CAPACITY_DATA) + pOut->ResponseBuffer; + pReadCapData-> + LogicalBlockAddress = 0; + pReadCapData->BytesPerBlock = + MP_BLOCK_SIZE; + } else { + status = + SRB_STATUS_DATA_OVERRUN; + } + } else { + status = SRB_STATUS_ERROR; + } + } else { + sizeNeeded = minSizeNeeded; + status = SRB_STATUS_ERROR; + } + break; + } + case SendRNID: { + PSendRNID_IN pIn; + PSendRNID_OUT pOut; + + if (InBufferSize >= sizeof (ULONG)) { + pIn = (PSendRNID_IN)pBuffer; + + sizeNeeded = sizeof (SendRNID_IN); + + if (InBufferSize >= sizeNeeded) { + sizeNeeded = FIELD_OFFSET( + SendRNID_OUT, ResponseBuffer) + + SendRNID_OUT_ResponseBuffer_SIZE_HINT; + + if (OutBufferSize >= sizeNeeded) { + pOut = (PSendRNID_OUT)pBuffer; + + pOut->HBAStatus = HBA_STATUS_OK; + pOut->ResponseBufferCount = + SendRNID_OUT_ResponseBuffer_SIZE_HINT; + + memset(pOut->ResponseBuffer, + 0xFF, + pOut->ResponseBufferCount); + } else { + status = + SRB_STATUS_DATA_OVERRUN; + } + } else { + status = SRB_STATUS_ERROR; + } + } else { + sizeNeeded = minSizeNeeded; + status = SRB_STATUS_ERROR; + } + break; + } + + default: + __try { + DbgBreakPoint(); + } + __except(EXCEPTION_EXECUTE_HANDLER) { + } + + status = SRB_STATUS_INVALID_REQUEST; + break; + } + break; + + case IdxGmDrvDrvMethodGuid: + + switch (MethodId) { + + case GmDrvDemoMethod1: { + PGmDrvDemoMethod1_IN pInBfr = + (PGmDrvDemoMethod1_IN)pBuffer; + PGmDrvDemoMethod1_OUT pOutBfr = + (PGmDrvDemoMethod1_OUT)pBuffer; + + sizeNeeded = GmDrvDemoMethod1_OUT_SIZE; + + if (OutBufferSize < sizeNeeded) { + status = + SRB_STATUS_DATA_OVERRUN; + break; + } + + if (InBufferSize < GmDrvDemoMethod1_IN_SIZE) { + status = SRB_STATUS_BAD_FUNCTION; + break; + } + + pOutBfr->outDatum = pInBfr->inDatum + 1; + break; + + } + + case GmDrvDemoMethod2: { + PGmDrvDemoMethod2_IN pInBfr = + (PGmDrvDemoMethod2_IN)pBuffer; + PGmDrvDemoMethod2_OUT pOutBfr = + (PGmDrvDemoMethod2_OUT)pBuffer; + + sizeNeeded = GmDrvDemoMethod2_OUT_SIZE; + + if (OutBufferSize < sizeNeeded) { + status = SRB_STATUS_DATA_OVERRUN; + break; + } + + if (InBufferSize < GmDrvDemoMethod2_IN_SIZE) { + status = SRB_STATUS_BAD_FUNCTION; + break; + } + + pOutBfr->outDatum1 = pInBfr->inDatum1 + + pInBfr->inDatum2 + 1; + break; + + } + + case GmDrvDemoMethod3: { + ULONG x1, x2; + PGmDrvDemoMethod3_IN pInBfr = + (PGmDrvDemoMethod3_IN)pBuffer; + PGmDrvDemoMethod3_OUT pOutBfr = + (PGmDrvDemoMethod3_OUT)pBuffer; + + sizeNeeded = GmDrvDemoMethod3_OUT_SIZE; + + if (OutBufferSize < sizeNeeded) { + status = SRB_STATUS_DATA_OVERRUN; + break; + } + + if (InBufferSize < GmDrvDemoMethod3_IN_SIZE) { + status = SRB_STATUS_BAD_FUNCTION; + break; + } + + x1 = pInBfr->inDatum1 + 1; + x2 = pInBfr->inDatum2 + 1; + + pOutBfr->outDatum1 = x1; + pOutBfr->outDatum2 = x2; + + break; + } + + default: + status = SRB_STATUS_INVALID_REQUEST; + break; + } + break; + + default: + status = SRB_STATUS_INVALID_REQUEST; + break; + } + + SpUpdateWmiRequest(pHbaExtension, pSrb, pDispatchContext, + status, sizeNeeded); + + return (SRB_STATUS_PENDING); +} + +void +SpUpdateWmiRequest( + __in pHW_HBA_EXT pHbaExtension, + __in PSCSI_WMI_REQUEST_BLOCK pSrb, + __in PSCSIWMI_REQUEST_CONTEXT pDispatchContext, + __in UCHAR Status, + __in ULONG SizeNeeded) +{ + UNREFERENCED_PARAMETER(pHbaExtension); + + // Update the request if the status is NOT pending or NOT + // already completed within the callback. + + if (SRB_STATUS_PENDING != Status) { + + // + // Request completed successfully or there was an error. + // + + ScsiPortWmiPostProcess(pDispatchContext, Status, SizeNeeded); + + pSrb->SrbStatus = ScsiPortWmiGetReturnStatus(pDispatchContext); + pSrb->DataTransferLength = + ScsiPortWmiGetReturnSize(pDispatchContext); + } +} diff --git a/module/os/windows/zfs/zfs_znode.c b/module/os/windows/zfs/zfs_znode.c new file mode 100644 index 000000000000..58c79bab42b5 --- /dev/null +++ b/module/os/windows/zfs/zfs_znode.c @@ -0,0 +1,2413 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. + * Portions Copyright 2007-2009 Apple Inc. All rights reserved. + * Use is subject to license terms. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. + */ + +/* Portions Copyright 2007 Jeremy Teo */ +/* Portions Copyright 2011 Martin Matuska */ +/* Portions Copyright 2013 Jorgen Lundman */ +/* Portions Copyright 2022 Andrew Innes */ + +#ifdef _KERNEL +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#endif /* _KERNEL */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "zfs_prop.h" +#include "zfs_comutil.h" + +/* + * Functions needed for userland (ie: libzpool) are not put under + * #ifdef_KERNEL; the rest of the functions have dependencies + * (such as VFS logic) that will not compile easily in userland. + */ +#ifdef _KERNEL +/* + * This is used by the test suite so that it can delay znodes from being + * freed in order to inspect the unlinked set. + */ +int zfs_unlink_suspend_progress = 0; + +/* + * This callback is invoked when acquiring a RL_WRITER or RL_APPEND lock on + * z_rangelock. It will modify the offset and length of the lock to reflect + * znode-specific information, and convert RL_APPEND to RL_WRITER. This is + * called with the rangelock_t's rl_lock held, which avoids races. + */ + +kmem_cache_t *znode_cache = NULL; +static kmem_cache_t *znode_hold_cache = NULL; +unsigned int zfs_object_mutex_size = ZFS_OBJ_MTX_SZ; + +/* + * This callback is invoked when acquiring a RL_WRITER or RL_APPEND lock on + * z_rangelock. It will modify the offset and length of the lock to reflect + * znode-specific information, and convert RL_APPEND to RL_WRITER. This is + * called with the rangelock_t's rl_lock held, which avoids races. + */ +static void +zfs_rangelock_cb(zfs_locked_range_t *new, void *arg) +{ + znode_t *zp = arg; + + /* + * If in append mode, convert to writer and lock starting at the + * current end of file. + */ + if (new->lr_type == RL_APPEND) { + new->lr_offset = zp->z_size; + new->lr_type = RL_WRITER; + } + + /* + * If we need to grow the block size then lock the whole file range. + */ + uint64_t end_size = MAX(zp->z_size, new->lr_offset + new->lr_length); + if (end_size > zp->z_blksz && (!ISP2(zp->z_blksz) || + zp->z_blksz < zp->z_zfsvfs->z_max_blksz)) { + new->lr_offset = 0; + new->lr_length = UINT64_MAX; + } +} + + +#if 0 // unused function +static void +znode_evict_error(dmu_buf_t *dbuf, void *user_ptr) +{ + /* + * We should never drop all dbuf refs without first clearing + * the eviction callback. + */ + panic("evicting znode %p\n", user_ptr); +} +#endif + +extern struct vop_vector zfs_vnodeops; +extern struct vop_vector zfs_fifoops; +extern struct vop_vector zfs_shareops; + +/* + * XXX: We cannot use this function as a cache constructor, because + * there is one global cache for all file systems and we need + * to pass vfsp here, which is not possible, because argument + * 'cdrarg' is defined at kmem_cache_create() time. + */ +static int +zfs_znode_cache_constructor(void *buf, void *arg, int kmflags) +{ + (void) arg, (void) kmflags; + znode_t *zp = buf; + + list_link_init(&zp->z_link_node); + + mutex_init(&zp->z_lock, NULL, MUTEX_DEFAULT, NULL); + rw_init(&zp->z_map_lock, NULL, RW_DEFAULT, NULL); + rw_init(&zp->z_parent_lock, NULL, RW_DEFAULT, NULL); + rw_init(&zp->z_name_lock, NULL, RW_DEFAULT, NULL); + mutex_init(&zp->z_acl_lock, NULL, MUTEX_DEFAULT, NULL); + rw_init(&zp->z_xattr_lock, NULL, RW_DEFAULT, NULL); + zfs_rangelock_init(&zp->z_rangelock, zfs_rangelock_cb, zp); + + mutex_init(&zp->z_attach_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&zp->z_attach_cv, NULL, CV_DEFAULT, NULL); + + zp->z_dirlocks = NULL; + zp->z_acl_cached = NULL; + zp->z_xattr_cached = NULL; + zp->z_xattr_parent = 0; + return (0); +} + +static void +zfs_znode_cache_destructor(void *buf, void *arg) +{ + (void) arg; + znode_t *zp = buf; + + ASSERT(ZTOV(zp) == NULL); + ASSERT(!list_link_active(&zp->z_link_node)); + mutex_destroy(&zp->z_lock); + rw_destroy(&zp->z_map_lock); + rw_destroy(&zp->z_parent_lock); + rw_destroy(&zp->z_name_lock); + mutex_destroy(&zp->z_acl_lock); + rw_destroy(&zp->z_xattr_lock); + zfs_rangelock_fini(&zp->z_rangelock); + mutex_destroy(&zp->z_attach_lock); + cv_destroy(&zp->z_attach_cv); + + ASSERT(zp->z_dirlocks == NULL); + ASSERT(zp->z_acl_cached == NULL); + ASSERT(zp->z_xattr_cached == NULL); +} + +static int +zfs_znode_hold_cache_constructor(void *buf, void *arg, int kmflags) +{ + znode_hold_t *zh = buf; + + mutex_init(&zh->zh_lock, NULL, MUTEX_DEFAULT, NULL); + zh->zh_refcount = 0; + + return (0); +} + +static void +zfs_znode_hold_cache_destructor(void *buf, void *arg) +{ + znode_hold_t *zh = buf; + + mutex_destroy(&zh->zh_lock); +} + +void +zfs_znode_init(void) +{ + /* + * Initialize zcache. The KMC_SLAB hint is used in order that it be + * backed by kmalloc() when on the Linux slab in order that any + * wait_on_bit() operations on the related inode operate properly. + */ + ASSERT(znode_cache == NULL); + znode_cache = kmem_cache_create("zfs_znode_cache", + sizeof (znode_t), 0, + zfs_znode_cache_constructor, + zfs_znode_cache_destructor, NULL, NULL, + NULL, 0); + + ASSERT(znode_hold_cache == NULL); + znode_hold_cache = kmem_cache_create("zfs_znode_hold_cache", + sizeof (znode_hold_t), 0, zfs_znode_hold_cache_constructor, + zfs_znode_hold_cache_destructor, NULL, NULL, NULL, 0); +} + +void +zfs_znode_fini(void) +{ + /* + * Cleanup zcache + */ + if (znode_cache) + kmem_cache_destroy(znode_cache); + znode_cache = NULL; + + if (znode_hold_cache) + kmem_cache_destroy(znode_hold_cache); + znode_hold_cache = NULL; +} + +/* + * The zfs_znode_hold_enter() / zfs_znode_hold_exit() functions are used to + * serialize access to a znode and its SA buffer while the object is being + * created or destroyed. This kind of locking would normally reside in the + * znode itself but in this case that's impossible because the znode and SA + * buffer may not yet exist. Therefore the locking is handled externally + * with an array of mutexs and AVLs trees which contain per-object locks. + * + * In zfs_znode_hold_enter() a per-object lock is created as needed, inserted + * in to the correct AVL tree and finally the per-object lock is held. In + * zfs_znode_hold_exit() the process is reversed. The per-object lock is + * released, removed from the AVL tree and destroyed if there are no waiters. + * + * This scheme has two important properties: + * + * 1) No memory allocations are performed while holding one of the z_hold_locks. + * This ensures evict(), which can be called from direct memory reclaim, will + * never block waiting on a z_hold_locks which just happens to have hashed + * to the same index. + * + * 2) All locks used to serialize access to an object are per-object and never + * shared. This minimizes lock contention without creating a large number + * of dedicated locks. + * + * On the downside it does require znode_lock_t structures to be frequently + * allocated and freed. However, because these are backed by a kmem cache + * and very short lived this cost is minimal. + */ +int +zfs_znode_hold_compare(const void *a, const void *b) +{ + const znode_hold_t *zh_a = (const znode_hold_t *)a; + const znode_hold_t *zh_b = (const znode_hold_t *)b; + + return (TREE_CMP(zh_a->zh_obj, zh_b->zh_obj)); +} + +boolean_t +zfs_znode_held(zfsvfs_t *zfsvfs, uint64_t obj) +{ + znode_hold_t *zh, search; + int i = ZFS_OBJ_HASH(zfsvfs, obj); + boolean_t held; + + search.zh_obj = obj; + + mutex_enter(&zfsvfs->z_hold_locks[i]); + zh = avl_find(&zfsvfs->z_hold_trees[i], &search, NULL); + held = (zh && MUTEX_HELD(&zh->zh_lock)) ? B_TRUE : B_FALSE; + mutex_exit(&zfsvfs->z_hold_locks[i]); + + return (held); +} + +static znode_hold_t * +zfs_znode_hold_enter(zfsvfs_t *zfsvfs, uint64_t obj) +{ + znode_hold_t *zh, *zh_new, search; + int i = ZFS_OBJ_HASH(zfsvfs, obj); + boolean_t found = B_FALSE; + + zh_new = kmem_cache_alloc(znode_hold_cache, KM_SLEEP); + search.zh_obj = obj; + + mutex_enter(&zfsvfs->z_hold_locks[i]); + zh = avl_find(&zfsvfs->z_hold_trees[i], &search, NULL); + if (likely(zh == NULL)) { + zh = zh_new; + zh->zh_obj = obj; + avl_add(&zfsvfs->z_hold_trees[i], zh); + } else { + ASSERT3U(zh->zh_obj, ==, obj); + found = B_TRUE; + } + zh->zh_refcount++; + ASSERT3S(zh->zh_refcount, >, 0); + mutex_exit(&zfsvfs->z_hold_locks[i]); + + if (found == B_TRUE) + kmem_cache_free(znode_hold_cache, zh_new); + + ASSERT(MUTEX_NOT_HELD(&zh->zh_lock)); + mutex_enter(&zh->zh_lock); + + return (zh); +} + +static void +zfs_znode_hold_exit(zfsvfs_t *zfsvfs, znode_hold_t *zh) +{ + int i = ZFS_OBJ_HASH(zfsvfs, zh->zh_obj); + boolean_t remove = B_FALSE; + + ASSERT(zfs_znode_held(zfsvfs, zh->zh_obj)); + mutex_exit(&zh->zh_lock); + + mutex_enter(&zfsvfs->z_hold_locks[i]); + ASSERT3S(zh->zh_refcount, >, 0); + if (--zh->zh_refcount == 0) { + avl_remove(&zfsvfs->z_hold_trees[i], zh); + remove = B_TRUE; + } + mutex_exit(&zfsvfs->z_hold_locks[i]); + + if (remove == B_TRUE) + kmem_cache_free(znode_hold_cache, zh); +} + +int +zfs_create_share_dir(zfsvfs_t *zfsvfs, dmu_tx_t *tx) +{ + int error = 0; +#if 0 // FIXME, uses vnode struct, not ptr + zfs_acl_ids_t acl_ids; + vattr_t vattr; + znode_t *sharezp; + struct vnode *vp, *vnode; + znode_t *zp; + + vattr.va_mask = ATTR_MODE|ATTR_UID|ATTR_GID|ATTR_TYPE; + vattr.va_type = VDIR; + vattr.va_mode = S_IFDIR|0555; + vattr.va_uid = crgetuid(kcred); + vattr.va_gid = crgetgid(kcred); + + sharezp = kmem_cache_alloc(znode_cache, KM_SLEEP); + sharezp->z_moved = 0; + sharezp->z_unlinked = 0; + sharezp->z_atime_dirty = 0; + sharezp->z_zfsvfs = zfsvfs; + sharezp->z_is_sa = zfsvfs->z_use_sa; + + sharezp->z_vnode = vnode; + vnode.v_data = sharezp; + + vp = ZTOV(sharezp); + vp->v_type = VDIR; + + VERIFY(0 == zfs_acl_ids_create(sharezp, IS_ROOT_NODE, &vattr, + kcred, NULL, &acl_ids, NULL)); + zfs_mknode(sharezp, &vattr, tx, kcred, IS_ROOT_NODE, &zp, &acl_ids); + ASSERT3P(zp, ==, sharezp); + POINTER_INVALIDATE(&sharezp->z_zfsvfs); + error = zap_add(zfsvfs->z_os, MASTER_NODE_OBJ, + ZFS_SHARES_DIR, 8, 1, &sharezp->z_id, tx); + zfsvfs->z_shares_dir = sharezp->z_id; + + zfs_acl_ids_free(&acl_ids); + ZTOV(sharezp)->v_data = NULL; + ZTOV(sharezp)->v_count = 0; + ZTOV(sharezp)->v_holdcnt = 0; + zp->z_vnode = NULL; + sa_handle_destroy(sharezp->z_sa_hdl); + sharezp->z_vnode = NULL; + kmem_cache_free(znode_cache, sharezp); +#endif + return (error); +} + +/* + * define a couple of values we need available + * for both 64 and 32 bit environments. + */ +#ifndef NBITSMINOR64 +#define NBITSMINOR64 32 +#endif +#ifndef MAXMAJ64 +#define MAXMAJ64 0xffffffffUL +#endif +#ifndef MAXMIN64 +#define MAXMIN64 0xffffffffUL +#endif + +/* + * Create special expldev for ZFS private use. + * Can't use standard expldev since it doesn't do + * what we want. The standard expldev() takes a + * dev32_t in LP64 and expands it to a long dev_t. + * We need an interface that takes a dev32_t in ILP32 + * and expands it to a long dev_t. + */ +static uint64_t +zfs_expldev(dev_t dev) +{ + return (((uint64_t)major(dev) << NBITSMINOR64) | minor(dev)); +} +/* + * Special cmpldev for ZFS private use. + * Can't use standard cmpldev since it takes + * a long dev_t and compresses it to dev32_t in + * LP64. We need to do a compaction of a long dev_t + * to a dev32_t in ILP32. + */ +dev_t +zfs_cmpldev(uint64_t dev) +{ + return (makedev((dev >> NBITSMINOR64), (dev & MAXMIN64))); +} + +static void +zfs_znode_sa_init(zfsvfs_t *zfsvfs, znode_t *zp, + dmu_buf_t *db, dmu_object_type_t obj_type, + sa_handle_t *sa_hdl) +{ + ASSERT(zfs_znode_held(zfsvfs, zp->z_id)); + + mutex_enter(&zp->z_lock); + + ASSERT(zp->z_sa_hdl == NULL); + ASSERT(zp->z_acl_cached == NULL); + if (sa_hdl == NULL) { + VERIFY(0 == sa_handle_get_from_db(zfsvfs->z_os, db, zp, + SA_HDL_SHARED, &zp->z_sa_hdl)); + } else { + zp->z_sa_hdl = sa_hdl; + sa_set_userp(sa_hdl, zp); + } + + zp->z_is_sa = (obj_type == DMU_OT_SA) ? B_TRUE : B_FALSE; + + mutex_exit(&zp->z_lock); +} + +void +zfs_znode_dmu_fini(znode_t *zp) +{ + ASSERT(zfs_znode_held(ZTOZSB(zp), zp->z_id) || zp->z_unlinked || + RW_WRITE_HELD(&ZTOZSB(zp)->z_teardown_inactive_lock)); + + sa_handle_destroy(zp->z_sa_hdl); + zp->z_sa_hdl = NULL; +} + +#if 0 // Until we need it ? +static void +zfs_vnode_destroy(struct vnode *vp) +{ + znode_t *zp = VTOZ(vp); + zfsvfs_t *zfsvfs = ZTOZSB(zp); + + if (vp != NULL) { + znode_t *zp = VTOZ(vp); + + if (zp != NULL) { + mutex_enter(&zfsvfs->z_znodes_lock); + if (list_link_active(&zp->z_link_node)) { + list_remove(&zfsvfs->z_all_znodes, zp); + } + mutex_exit(&zfsvfs->z_znodes_lock); + + if (zp->z_acl_cached) { + zfs_acl_free(zp->z_acl_cached); + zp->z_acl_cached = NULL; + } + + if (zp->z_xattr_cached) { + nvlist_free(zp->z_xattr_cached); + zp->z_xattr_cached = NULL; + } + + kmem_cache_free(znode_cache, zp); + } + + vnode_clearfsnode(vp); + vnode_put(vp); + vnode_recycle(vp); + } +} +#endif + +/* + * Construct a new znode/vnode and intialize. + * + * This does not do a call to dmu_set_user() that is + * up to the caller to do, in case you don't want to + * return the znode + */ +static znode_t * +zfs_znode_alloc(zfsvfs_t *zfsvfs, dmu_buf_t *db, int blksz, + dmu_object_type_t obj_type, sa_handle_t *hdl) +{ + znode_t *zp; + struct vnode *vp; + uint64_t mode; + uint64_t parent; + sa_bulk_attr_t bulk[11]; + int count = 0; + uint64_t projid = ZFS_DEFAULT_PROJID; + + zp = kmem_cache_alloc(znode_cache, KM_SLEEP); + + ASSERT(zp->z_dirlocks == NULL); + ASSERT(!POINTER_IS_VALID(zp->z_zfsvfs)); + + /* + * Defer setting z_zfsvfs until the znode is ready to be a candidate for + * the zfs_znode_move() callback. + */ + zp->z_vnode = NULL; + zp->z_sa_hdl = NULL; + zp->z_unlinked = 0; + zp->z_atime_dirty = 0; + zp->z_mapcnt = 0; + zp->z_id = db->db_object; + zp->z_blksz = blksz; + zp->z_seq = 0x7A4653; + zp->z_sync_cnt = 0; + + zp->z_is_mapped = 0; + zp->z_is_ctldir = 0; + zp->z_vid = 0; + zp->z_uid = 0; + zp->z_gid = 0; + zp->z_size = 0; + zp->z_name_cache = NULL; + zp->z_name_len = 0; + zp->z_name_offset = 0; + + taskq_init_ent(&zp->z_attach_taskq); + + vp = ZTOV(zp); /* Does nothing in OSX */ + + zfs_znode_sa_init(zfsvfs, zp, db, obj_type, hdl); + + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MODE(zfsvfs), NULL, &mode, 8); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_GEN(zfsvfs), NULL, &zp->z_gen, 8); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_SIZE(zfsvfs), NULL, + &zp->z_size, 8); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_LINKS(zfsvfs), NULL, + &zp->z_links, 8); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), NULL, + &zp->z_pflags, 8); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_PARENT(zfsvfs), NULL, &parent, 8); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_ATIME(zfsvfs), NULL, + &zp->z_atime, 16); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_UID(zfsvfs), NULL, + &zp->z_uid, 8); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_GID(zfsvfs), NULL, + &zp->z_gid, 8); + + if (sa_bulk_lookup(zp->z_sa_hdl, bulk, count) != 0 || zp->z_gen == 0 || + (dmu_objset_projectquota_enabled(zfsvfs->z_os) && + (zp->z_pflags & ZFS_PROJID) && + sa_lookup(zp->z_sa_hdl, SA_ZPL_PROJID(zfsvfs), &projid, 8) != 0)) { + if (hdl == NULL) + sa_handle_destroy(zp->z_sa_hdl); + zp->z_sa_hdl = NULL; + dprintf("znode_alloc: sa_bulk_lookup failed - aborting\n"); + kmem_cache_free(znode_cache, zp); + return (NULL); + } + + zp->z_projid = projid; + zp->z_mode = mode; + /* Cache the xattr parent id */ + if (zp->z_pflags & ZFS_XATTR) + zp->z_xattr_parent = parent; + + mutex_enter(&zfsvfs->z_znodes_lock); + list_insert_tail(&zfsvfs->z_all_znodes, zp); + membar_producer(); + /* + * Everything else must be valid before assigning z_zfsvfs makes the + * znode eligible for zfs_znode_move(). + */ + zp->z_zfsvfs = zfsvfs; + mutex_exit(&zfsvfs->z_znodes_lock); + + return (zp); +} + + +static uint64_t empty_xattr; +static uint64_t pad[4]; +static zfs_acl_phys_t acl_phys; +/* + * Create a new DMU object to hold a zfs znode. + * + * IN: dzp - parent directory for new znode + * vap - file attributes for new znode + * tx - dmu transaction id for zap operations + * cr - credentials of caller + * flag - flags: + * IS_ROOT_NODE - new object will be root + * IS_XATTR - new object is an attribute + * bonuslen - length of bonus buffer + * setaclp - File/Dir initial ACL + * fuidp - Tracks fuid allocation. + * + * OUT: zpp - allocated znode + * + * OS X implementation notes: + * + * The caller of zfs_mknode() is expected to call zfs_znode_getvnode() + * AFTER the dmu_tx_commit() is performed. This prevents deadlocks + * since vnode_create can indirectly attempt to clean a dirty vnode. + * + * The current list of callers includes: + * zfs_vnop_create + * zfs_vnop_mkdir + * zfs_vnop_symlink + * zfs_obtain_xattr + * zfs_make_xattrdir + */ +void +zfs_mknode(znode_t *dzp, vattr_t *vap, dmu_tx_t *tx, cred_t *cr, + uint_t flag, znode_t **zpp, zfs_acl_ids_t *acl_ids) +{ + uint64_t crtime[2], atime[2], mtime[2], ctime[2]; + uint64_t mode, size, links, parent, pflags; + uint64_t projid = ZFS_DEFAULT_PROJID; + uint64_t dzp_pflags = 0; + uint64_t rdev = 0; + zfsvfs_t *zfsvfs = dzp->z_zfsvfs; + dmu_buf_t *db; + timestruc_t now; + uint64_t gen, obj; + int bonuslen; + int dnodesize; + sa_handle_t *sa_hdl; + dmu_object_type_t obj_type; + sa_bulk_attr_t *sa_attrs; + int cnt = 0; + zfs_acl_locator_cb_t locate = { 0 }; + int err = 0; + znode_hold_t *zh; + + ASSERT(vap && (vap->va_mask & (ATTR_TYPE|ATTR_MODE)) == + (ATTR_TYPE|ATTR_MODE)); + + if (zfsvfs->z_replay) { + obj = vap->va_nodeid; + now = vap->va_ctime; /* see zfs_replay_create() */ + gen = vap->va_nblocks; /* ditto */ + dnodesize = vap->va_fsid; /* ditto */ + } else { + obj = 0; + gethrestime(&now); + gen = dmu_tx_get_txg(tx); + dnodesize = dmu_objset_dnodesize(zfsvfs->z_os); + } + + if (dnodesize == 0) + dnodesize = DNODE_MIN_SIZE; + + obj_type = zfsvfs->z_use_sa ? DMU_OT_SA : DMU_OT_ZNODE; + bonuslen = (obj_type == DMU_OT_SA) ? + DN_BONUS_SIZE(dnodesize) : ZFS_OLD_ZNODE_PHYS_SIZE; + + /* + * Create a new DMU object. + */ + /* + * There's currently no mechanism for pre-reading the blocks that will + * be needed to allocate a new object, so we accept the small chance + * that there will be an i/o error and we will fail one of the + * assertions below. + */ + if (vap->va_type == VDIR) { + if (zfsvfs->z_replay) { + VERIFY0(zap_create_claim_norm_dnsize(zfsvfs->z_os, obj, + zfsvfs->z_norm, DMU_OT_DIRECTORY_CONTENTS, + obj_type, bonuslen, dnodesize, tx)); + } else { + obj = zap_create_norm_dnsize(zfsvfs->z_os, + zfsvfs->z_norm, DMU_OT_DIRECTORY_CONTENTS, + obj_type, bonuslen, dnodesize, tx); + } + } else { + if (zfsvfs->z_replay) { + VERIFY0(dmu_object_claim_dnsize(zfsvfs->z_os, obj, + DMU_OT_PLAIN_FILE_CONTENTS, 0, + obj_type, bonuslen, dnodesize, tx)); + } else { + obj = dmu_object_alloc_dnsize(zfsvfs->z_os, + DMU_OT_PLAIN_FILE_CONTENTS, 0, + obj_type, bonuslen, dnodesize, tx); + } + } + + zh = zfs_znode_hold_enter(zfsvfs, obj); + VERIFY0(sa_buf_hold(zfsvfs->z_os, obj, NULL, &db)); + + /* + * If this is the root, fix up the half-initialized parent pointer + * to reference the just-allocated physical data area. + */ + if (flag & IS_ROOT_NODE) { + dzp->z_id = obj; + } else { + dzp_pflags = dzp->z_pflags; + } + + /* + * If parent is an xattr, so am I. + */ + if (dzp_pflags & ZFS_XATTR) { + flag |= IS_XATTR; + } + + if (zfsvfs->z_use_fuids) + pflags = ZFS_ARCHIVE | ZFS_AV_MODIFIED; + else + pflags = 0; + + if (vap->va_type == VDIR) { + size = 2; /* contents ("." and "..") */ + links = (flag & (IS_ROOT_NODE | IS_XATTR)) ? 2 : 1; + } else { + size = links = 0; + } + + if (vap->va_type == VBLK || vap->va_type == VCHR) { + rdev = zfs_expldev(vap->va_rdev); + } + + parent = dzp->z_id; + mode = acl_ids->z_mode; + if (flag & IS_XATTR) + pflags |= ZFS_XATTR; + + if (S_ISREG(vap->va_mode) || S_ISDIR(vap->va_mode)) { + /* + * With ZFS_PROJID flag, we can easily know whether there is + * project ID stored on disk or not. See zfs_space_delta_cb(). + */ + if (obj_type != DMU_OT_ZNODE && + dmu_objset_projectquota_enabled(zfsvfs->z_os)) + pflags |= ZFS_PROJID; + + /* + * Inherit project ID from parent if required. + */ + projid = zfs_inherit_projid(dzp); + if (dzp->z_pflags & ZFS_PROJINHERIT) + pflags |= ZFS_PROJINHERIT; + } + + /* + * No execs denied will be deterimed when zfs_mode_compute() is called. + */ + pflags |= acl_ids->z_aclp->z_hints & + (ZFS_ACL_TRIVIAL|ZFS_INHERIT_ACE|ZFS_ACL_AUTO_INHERIT| + ZFS_ACL_DEFAULTED|ZFS_ACL_PROTECTED); + + ZFS_TIME_ENCODE(&now, crtime); + ZFS_TIME_ENCODE(&now, ctime); + + if (vap->va_mask & ATTR_ATIME) { + ZFS_TIME_ENCODE(&vap->va_atime, atime); + } else { + ZFS_TIME_ENCODE(&now, atime); + } + + if (vap->va_mask & ATTR_MTIME) { + ZFS_TIME_ENCODE(&vap->va_mtime, mtime); + } else { + ZFS_TIME_ENCODE(&now, mtime); + } + + /* Now add in all of the "SA" attributes */ + VERIFY(0 == sa_handle_get_from_db(zfsvfs->z_os, db, NULL, SA_HDL_SHARED, + &sa_hdl)); + + /* + * Setup the array of attributes to be replaced/set on the new file + * + * order for DMU_OT_ZNODE is critical since it needs to be constructed + * in the old znode_phys_t format. Don't change this ordering + */ + sa_attrs = kmem_alloc(sizeof (sa_bulk_attr_t) * ZPL_END, KM_SLEEP); + + if (obj_type == DMU_OT_ZNODE) { + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_ATIME(zfsvfs), + NULL, &atime, 16); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_MTIME(zfsvfs), + NULL, &mtime, 16); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_CTIME(zfsvfs), + NULL, &ctime, 16); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_CRTIME(zfsvfs), + NULL, &crtime, 16); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_GEN(zfsvfs), + NULL, &gen, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_MODE(zfsvfs), + NULL, &mode, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_SIZE(zfsvfs), + NULL, &size, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_PARENT(zfsvfs), + NULL, &parent, 8); + } else { + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_MODE(zfsvfs), + NULL, &mode, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_SIZE(zfsvfs), + NULL, &size, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_GEN(zfsvfs), + NULL, &gen, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_UID(zfsvfs), + NULL, &acl_ids->z_fuid, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_GID(zfsvfs), + NULL, &acl_ids->z_fgid, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_PARENT(zfsvfs), + NULL, &parent, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_FLAGS(zfsvfs), + NULL, &pflags, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_ATIME(zfsvfs), + NULL, &atime, 16); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_MTIME(zfsvfs), + NULL, &mtime, 16); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_CTIME(zfsvfs), + NULL, &ctime, 16); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_CRTIME(zfsvfs), + NULL, &crtime, 16); + } + + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_LINKS(zfsvfs), NULL, &links, 8); + + if (obj_type == DMU_OT_ZNODE) { + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_XATTR(zfsvfs), NULL, + &empty_xattr, 8); + } else if (dmu_objset_projectquota_enabled(zfsvfs->z_os) && + pflags & ZFS_PROJID) { + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_PROJID(zfsvfs), + NULL, &projid, 8); + } + if (obj_type == DMU_OT_ZNODE || + (vap->va_type == VBLK || vap->va_type == VCHR)) { + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_RDEV(zfsvfs), + NULL, &rdev, 8); + + } + if (obj_type == DMU_OT_ZNODE) { + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_FLAGS(zfsvfs), + NULL, &pflags, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_UID(zfsvfs), NULL, + &acl_ids->z_fuid, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_GID(zfsvfs), NULL, + &acl_ids->z_fgid, 8); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_PAD(zfsvfs), NULL, pad, + sizeof (uint64_t) * 4); + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_ZNODE_ACL(zfsvfs), NULL, + &acl_phys, sizeof (zfs_acl_phys_t)); + } else if (acl_ids->z_aclp->z_version >= ZFS_ACL_VERSION_FUID) { + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_DACL_COUNT(zfsvfs), NULL, + &acl_ids->z_aclp->z_acl_count, 8); + locate.cb_aclp = acl_ids->z_aclp; + SA_ADD_BULK_ATTR(sa_attrs, cnt, SA_ZPL_DACL_ACES(zfsvfs), + zfs_acl_data_locator, &locate, + acl_ids->z_aclp->z_acl_bytes); + mode = zfs_mode_compute(mode, acl_ids->z_aclp, &pflags, + acl_ids->z_fuid, acl_ids->z_fgid); + } + + VERIFY(sa_replace_all_by_template(sa_hdl, sa_attrs, cnt, tx) == 0); + + if (!(flag & IS_ROOT_NODE)) { + /* + * We must not hold any locks while calling vnode_create inside + * zfs_znode_alloc(), as it may call either of vnop_reclaim, or + * vnop_fsync. If it is not enough to just release ZFS_OBJ_HOLD + * we will have to attach the vnode after the dmu_commit like + * maczfs does, in each vnop caller. + */ + do { + *zpp = zfs_znode_alloc(zfsvfs, db, 0, obj_type, sa_hdl); + } while (*zpp == NULL); + + VERIFY(*zpp != NULL); + VERIFY(dzp != NULL); + } else { + /* + * If we are creating the root node, the "parent" we + * passed in is the znode for the root. + */ + *zpp = dzp; + + (*zpp)->z_sa_hdl = sa_hdl; + } + + (*zpp)->z_pflags = pflags; + (*zpp)->z_mode = mode; + (*zpp)->z_dnodesize = dnodesize; + (*zpp)->z_projid = projid; + + if (vap->va_mask & ATTR_XVATTR) + zfs_xvattr_set(*zpp, (xvattr_t *)vap, tx); + + if (obj_type == DMU_OT_ZNODE || + acl_ids->z_aclp->z_version < ZFS_ACL_VERSION_FUID) { + err = zfs_aclset_common(*zpp, acl_ids->z_aclp, cr, tx); + ASSERT(err == 0); + } + + kmem_free(sa_attrs, sizeof (sa_bulk_attr_t) * ZPL_END); + zfs_znode_hold_exit(zfsvfs, zh); +} + +/* + * Update in-core attributes. It is assumed the caller will be doing an + * sa_bulk_update to push the changes out. + */ +void +zfs_xvattr_set(znode_t *zp, xvattr_t *xvap, dmu_tx_t *tx) +{ + xoptattr_t *xoap; + + xoap = xva_getxoptattr(xvap); + ASSERT(xoap); + + if (XVA_ISSET_REQ(xvap, XAT_CREATETIME)) { + uint64_t times[2]; + ZFS_TIME_ENCODE(&xoap->xoa_createtime, times); + (void) sa_update(zp->z_sa_hdl, SA_ZPL_CRTIME(zp->z_zfsvfs), + ×, sizeof (times), tx); + XVA_SET_RTN(xvap, XAT_CREATETIME); + } + if (XVA_ISSET_REQ(xvap, XAT_READONLY)) { + ZFS_ATTR_SET(zp, ZFS_READONLY, xoap->xoa_readonly, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_READONLY); + } + if (XVA_ISSET_REQ(xvap, XAT_HIDDEN)) { + ZFS_ATTR_SET(zp, ZFS_HIDDEN, xoap->xoa_hidden, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_HIDDEN); + } + if (XVA_ISSET_REQ(xvap, XAT_SYSTEM)) { + ZFS_ATTR_SET(zp, ZFS_SYSTEM, xoap->xoa_system, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_SYSTEM); + } + if (XVA_ISSET_REQ(xvap, XAT_ARCHIVE)) { + ZFS_ATTR_SET(zp, ZFS_ARCHIVE, xoap->xoa_archive, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_ARCHIVE); + } + if (XVA_ISSET_REQ(xvap, XAT_IMMUTABLE)) { + ZFS_ATTR_SET(zp, ZFS_IMMUTABLE, xoap->xoa_immutable, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_IMMUTABLE); + } + if (XVA_ISSET_REQ(xvap, XAT_NOUNLINK)) { + ZFS_ATTR_SET(zp, ZFS_NOUNLINK, xoap->xoa_nounlink, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_NOUNLINK); + } + if (XVA_ISSET_REQ(xvap, XAT_APPENDONLY)) { + ZFS_ATTR_SET(zp, ZFS_APPENDONLY, xoap->xoa_appendonly, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_APPENDONLY); + } + if (XVA_ISSET_REQ(xvap, XAT_NODUMP)) { + ZFS_ATTR_SET(zp, ZFS_NODUMP, xoap->xoa_nodump, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_NODUMP); + } + if (XVA_ISSET_REQ(xvap, XAT_OPAQUE)) { + ZFS_ATTR_SET(zp, ZFS_OPAQUE, xoap->xoa_opaque, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_OPAQUE); + } + if (XVA_ISSET_REQ(xvap, XAT_AV_QUARANTINED)) { + ZFS_ATTR_SET(zp, ZFS_AV_QUARANTINED, + xoap->xoa_av_quarantined, zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_AV_QUARANTINED); + } + if (XVA_ISSET_REQ(xvap, XAT_AV_MODIFIED)) { + ZFS_ATTR_SET(zp, ZFS_AV_MODIFIED, xoap->xoa_av_modified, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_AV_MODIFIED); + } + if (XVA_ISSET_REQ(xvap, XAT_AV_SCANSTAMP)) { + zfs_sa_set_scanstamp(zp, xvap, tx); + XVA_SET_RTN(xvap, XAT_AV_SCANSTAMP); + } + if (XVA_ISSET_REQ(xvap, XAT_REPARSE)) { + ZFS_ATTR_SET(zp, ZFS_REPARSE, xoap->xoa_reparse, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_REPARSE); + } + if (XVA_ISSET_REQ(xvap, XAT_OFFLINE)) { + ZFS_ATTR_SET(zp, ZFS_OFFLINE, xoap->xoa_offline, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_OFFLINE); + } + if (XVA_ISSET_REQ(xvap, XAT_SPARSE)) { + ZFS_ATTR_SET(zp, ZFS_SPARSE, xoap->xoa_sparse, + zp->z_pflags, tx); + XVA_SET_RTN(xvap, XAT_SPARSE); + } +} + +int +zfs_zget(zfsvfs_t *zfsvfs, uint64_t obj_num, znode_t **zpp) +{ + return (zfs_zget_ext(zfsvfs, obj_num, zpp, 0)); +} + +int +zfs_zget_ext(zfsvfs_t *zfsvfs, uint64_t obj_num, znode_t **zpp, + int flags) +{ + dmu_object_info_t doi; + dmu_buf_t *db; + znode_t *zp; + znode_hold_t *zh; + struct vnode *vp = NULL; + sa_handle_t *hdl; + uint32_t vid; + int err; + + dprintf("+zget %llu\n", obj_num); + + *zpp = NULL; + +again: + zh = zfs_znode_hold_enter(zfsvfs, obj_num); + + err = sa_buf_hold(zfsvfs->z_os, obj_num, NULL, &db); + if (err) { + zfs_znode_hold_exit(zfsvfs, zh); + return (err); + } + + dmu_object_info_from_db(db, &doi); + if (doi.doi_bonus_type != DMU_OT_SA && + (doi.doi_bonus_type != DMU_OT_ZNODE || + (doi.doi_bonus_type == DMU_OT_ZNODE && + doi.doi_bonus_size < sizeof (znode_phys_t)))) { + sa_buf_rele(db, NULL); + zfs_znode_hold_exit(zfsvfs, zh); + return (SET_ERROR(EINVAL)); + } + + hdl = dmu_buf_get_user(db); + if (hdl != NULL) { + zp = sa_get_userdata(hdl); + + + /* + * Since "SA" does immediate eviction we + * should never find a sa handle that doesn't + * know about the znode. + */ + ASSERT3P(zp, !=, NULL); + + mutex_enter(&zp->z_lock); + + /* + * Since zp may disappear after we unlock below, + * we save a copy of vp and it's vid + */ + vid = zp->z_vid; + vp = ZTOV(zp); + + /* + * Since we do immediate eviction of the z_dbuf, we + * should never find a dbuf with a znode that doesn't + * know about the dbuf. + */ + ASSERT3U(zp->z_id, ==, obj_num); + + /* + * OS X can return the znode when the file is unlinked + * in order to support the sync of open-unlinked files + */ + if (!(flags & ZGET_FLAG_UNLINKED) && zp->z_unlinked) { + mutex_exit(&zp->z_lock); + sa_buf_rele(db, NULL); + zfs_znode_hold_exit(zfsvfs, zh); + return (ENOENT); + } + + mutex_exit(&zp->z_lock); + sa_buf_rele(db, NULL); + zfs_znode_hold_exit(zfsvfs, zh); + + /* + * We are racing zfs_znode_getvnode() and we got here first, we + * need to let it get ahead + */ + if (!vp) { + + dprintf("%s: async racing attach\n", __func__); + // Could be zp is being torn down, idle a bit, + // and retry. This branch is rarely executed. + kpreempt(KPREEMPT_SYNC); + goto again; + } + + /* + * Due to vnode_create() -> zfs_fsync() -> zil_commit() -> + * zget() -> vnode_getwithvid() -> deadlock. Unsure why + * vnode_getwithvid() ends up sleeping in msleep() but + * vnode_get() does not. + * As we can deadlock here using vnode_getwithvid() we will use + * the simpler vnode_get() in the ASYNC cases. We verify the + * vids match below. + */ + err = vnode_getwithref(vp); + + if (err != 0) { + dprintf("ZFS: vnode_get() returned %d\n", err); + kpreempt(KPREEMPT_SYNC); + goto again; + } + + /* + * Since we had to drop all of our locks above, make sure + * that we have the vnode and znode we had before. + */ + mutex_enter(&zp->z_lock); + if ((vid != zp->z_vid) || (vp != ZTOV(zp))) { + mutex_exit(&zp->z_lock); + /* + * Release the wrong vp from vnode_getwithvid(). + */ + VN_RELE(vp); + dprintf("ZFS: the vids do not match part 1\n"); + goto again; + } + if (vnode_vid(vp) != zp->z_vid) + dprintf("ZFS: the vids do not match\n"); + mutex_exit(&zp->z_lock); + + *zpp = zp; + + return (0); + } // if vnode != NULL + + /* + * Not found create new znode/vnode + * but only if file exists. + * + * There is a small window where zfs_vget() could + * find this object while a file create is still in + * progress. This is checked for in zfs_znode_alloc() + * + * if zfs_znode_alloc() fails it will drop the hold on the + * bonus buffer. + */ + + zp = NULL; + zp = zfs_znode_alloc(zfsvfs, db, doi.doi_data_block_size, + doi.doi_bonus_type, NULL); + if (zp == NULL) { + err = SET_ERROR(ENOENT); + zfs_znode_hold_exit(zfsvfs, zh); + dprintf("zget returning %d\n", err); + return (err); + } + + dprintf("zget create: %llu setting to %p\n", obj_num, zp); + *zpp = zp; + + zfs_znode_hold_exit(zfsvfs, zh); + + /* Attach a vnode to our new znode */ + zfs_znode_getvnode(zp, NULL, zfsvfs); + + dprintf("zget returning %d\n", err); + return (err); +} + + +int +zfs_rezget(znode_t *zp) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + dmu_object_info_t doi; + dmu_buf_t *db; + struct vnode *vp; + uint64_t obj_num = zp->z_id; + uint64_t mode, size; + sa_bulk_attr_t bulk[8]; + int err; + int count = 0; + uint64_t gen; + uint64_t projid = ZFS_DEFAULT_PROJID; + znode_hold_t *zh; + + if (zp->z_is_ctldir) + return (0); + + zh = zfs_znode_hold_enter(zfsvfs, obj_num); + + mutex_enter(&zp->z_acl_lock); + if (zp->z_acl_cached) { + zfs_acl_free(zp->z_acl_cached); + zp->z_acl_cached = NULL; + } + mutex_exit(&zp->z_acl_lock); + + rw_enter(&zp->z_xattr_lock, RW_WRITER); + if (zp->z_xattr_cached) { + nvlist_free(zp->z_xattr_cached); + zp->z_xattr_cached = NULL; + } + + rw_exit(&zp->z_xattr_lock); + + ASSERT(zp->z_sa_hdl == NULL); + err = sa_buf_hold(zfsvfs->z_os, obj_num, NULL, &db); + if (err) { + zfs_znode_hold_exit(zfsvfs, zh); + return (err); + } + + dmu_object_info_from_db(db, &doi); + if (doi.doi_bonus_type != DMU_OT_SA && + (doi.doi_bonus_type != DMU_OT_ZNODE || + (doi.doi_bonus_type == DMU_OT_ZNODE && + doi.doi_bonus_size < sizeof (znode_phys_t)))) { + sa_buf_rele(db, NULL); + zfs_znode_hold_exit(zfsvfs, zh); + return (SET_ERROR(EINVAL)); + } + + zfs_znode_sa_init(zfsvfs, zp, db, doi.doi_bonus_type, NULL); + size = zp->z_size; + + /* reload cached values */ + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_GEN(zfsvfs), NULL, + &gen, sizeof (gen)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_SIZE(zfsvfs), NULL, + &zp->z_size, sizeof (zp->z_size)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_LINKS(zfsvfs), NULL, + &zp->z_links, sizeof (zp->z_links)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), NULL, + &zp->z_pflags, + sizeof (zp->z_pflags)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_ATIME(zfsvfs), NULL, + &zp->z_atime, sizeof (zp->z_atime)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_UID(zfsvfs), NULL, + &zp->z_uid, sizeof (zp->z_uid)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_GID(zfsvfs), NULL, + &zp->z_gid, sizeof (zp->z_gid)); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MODE(zfsvfs), NULL, + &mode, sizeof (mode)); + + if (sa_bulk_lookup(zp->z_sa_hdl, bulk, count)) { + zfs_znode_dmu_fini(zp); + zfs_znode_hold_exit(zfsvfs, zh); + return (SET_ERROR(EIO)); + } + + if (dmu_objset_projectquota_enabled(zfsvfs->z_os)) { + err = sa_lookup(zp->z_sa_hdl, SA_ZPL_PROJID(zfsvfs), + &projid, 8); + if (err != 0 && err != ENOENT) { + zfs_znode_dmu_fini(zp); + zfs_znode_hold_exit(zfsvfs, zh); + return (SET_ERROR(err)); + } + } + + zp->z_projid = projid; + zp->z_mode = mode; + + if (gen != zp->z_gen) { + zfs_znode_dmu_fini(zp); + zfs_znode_hold_exit(zfsvfs, zh); + return (SET_ERROR(EIO)); + } + + /* + * XXXPJD: Not sure how is that possible, but under heavy + * zfs recv -F load it happens that z_gen is the same, but + * vnode type is different than znode type. This would mean + * that for example regular file was replaced with directory + * which has the same object number. + */ + vp = ZTOV(zp); + if (vp != NULL && + vnode_vtype(vp) != IFTOVT((mode_t)zp->z_mode)) { + zfs_znode_dmu_fini(zp); + zfs_znode_hold_exit(zfsvfs, zh); + return (EIO); + } + + zp->z_blksz = doi.doi_data_block_size; + if (vp != NULL) { + vn_pages_remove(vp, 0, 0); + if (zp->z_size != size) + vnode_pager_setsize(NULL, vp, zp->z_size, TRUE); + } + + /* + * If the file has zero links, then it has been unlinked on the send + * side and it must be in the received unlinked set. + * We call zfs_znode_dmu_fini() now to prevent any accesses to the + * stale data and to prevent automatical removal of the file in + * zfs_zinactive(). The file will be removed either when it is removed + * on the send side and the next incremental stream is received or + * when the unlinked set gets processed. + */ + zp->z_unlinked = (zp->z_links == 0); + if (zp->z_unlinked) + zfs_znode_dmu_fini(zp); + + zfs_znode_hold_exit(zfsvfs, zh); + + return (0); +} + +void +zfs_znode_delete(znode_t *zp, dmu_tx_t *tx) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + objset_t *os = zfsvfs->z_os; + uint64_t obj = zp->z_id; + uint64_t acl_obj = zfs_external_acl(zp); + znode_hold_t *zh; + + zh = zfs_znode_hold_enter(zfsvfs, obj); + if (acl_obj) { + VERIFY(!zp->z_is_sa); + VERIFY(0 == dmu_object_free(os, acl_obj, tx)); + } + VERIFY(0 == dmu_object_free(os, obj, tx)); + zfs_znode_dmu_fini(zp); + zfs_znode_hold_exit(zfsvfs, zh); +} + +void +zfs_zinactive(znode_t *zp) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + uint64_t z_id = zp->z_id; + znode_hold_t *zh; + + ASSERT(zp->z_sa_hdl); + + /* + * Don't allow a zfs_zget() while were trying to release this znode + */ + zh = zfs_znode_hold_enter(zfsvfs, z_id); + + mutex_enter(&zp->z_lock); + + /* + * If this was the last reference to a file with no links, remove + * the file from the file system unless the file system is mounted + * read-only. That can happen, for example, if the file system was + * originally read-write, the file was opened, then unlinked and + * the file system was made read-only before the file was finally + * closed. The file will remain in the unlinked set. + */ + if (zp->z_unlinked) { + ASSERT(!zfsvfs->z_issnap); + + if (!(vfs_isrdonly(zfsvfs->z_vfs)) && + !zfs_unlink_suspend_progress) { + mutex_exit(&zp->z_lock); + zfs_znode_hold_exit(zfsvfs, zh); + zfs_rmnode(zp); + return; + } + } + + mutex_exit(&zp->z_lock); + zfs_znode_dmu_fini(zp); + + zfs_znode_hold_exit(zfsvfs, zh); +} + +void +zfs_znode_free(znode_t *zp) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + + mutex_enter(&zfsvfs->z_znodes_lock); + zp->z_vnode = NULL; + zp->z_zfsvfs = NULL; + POINTER_INVALIDATE(&zp->z_zfsvfs); + list_remove(&zfsvfs->z_all_znodes, zp); /* XXX */ + mutex_exit(&zfsvfs->z_znodes_lock); + + if (zp->z_acl_cached) { + zfs_acl_free(zp->z_acl_cached); + zp->z_acl_cached = NULL; + } + + if (zp->z_xattr_cached) { + nvlist_free(zp->z_xattr_cached); + zp->z_xattr_cached = NULL; + } + + if (zp->z_name_cache != NULL) { + kmem_free(zp->z_name_cache, zp->z_name_len); + zp->z_name_cache = NULL; + } + + ASSERT(zp->z_sa_hdl == NULL); + + kmem_cache_free(znode_cache, zp); +} + + +/* + * Prepare to update znode time stamps. + * + * IN: zp - znode requiring timestamp update + * flag - ATTR_MTIME, ATTR_CTIME, ATTR_ATIME flags + * have_tx - true of caller is creating a new txg + * + * OUT: zp - new atime (via underlying inode's i_atime) + * mtime - new mtime + * ctime - new ctime + * + * NOTE: The arguments are somewhat redundant. The following condition + * is always true: + * + * have_tx == !(flag & ATTR_ATIME) + */ +void +zfs_tstamp_update_setup_ext(znode_t *zp, uint_t flag, uint64_t mtime[2], + uint64_t ctime[2], boolean_t have_tx) +{ + timestruc_t now; + + gethrestime(&now); + + if (have_tx) { /* will sa_bulk_update happen really soon? */ + zp->z_atime_dirty = 0; + zp->z_seq++; + } else { + zp->z_atime_dirty = 1; + } + + if (flag & ATTR_ATIME) { + ZFS_TIME_ENCODE(&now, zp->z_atime); + } + + if (flag & ATTR_MTIME) { + ZFS_TIME_ENCODE(&now, mtime); + if (zp->z_zfsvfs->z_use_fuids) { + zp->z_pflags |= (ZFS_ARCHIVE | + ZFS_AV_MODIFIED); + } + } + + if (flag & ATTR_CTIME) { + ZFS_TIME_ENCODE(&now, ctime); + if (zp->z_zfsvfs->z_use_fuids) + zp->z_pflags |= ZFS_ARCHIVE; + } +} + +void +zfs_tstamp_update_setup(znode_t *zp, uint_t flag, uint64_t mtime[2], + uint64_t ctime[2]) +{ + zfs_tstamp_update_setup_ext(zp, flag, mtime, ctime, B_TRUE); +} + +/* + * Grow the block size for a file. + * + * IN: zp - znode of file to free data in. + * size - requested block size + * tx - open transaction. + * + * NOTE: this function assumes that the znode is write locked. + */ +void +zfs_grow_blocksize(znode_t *zp, uint64_t size, dmu_tx_t *tx) +{ + int error; + u_longlong_t dummy; + + if (size <= zp->z_blksz) + return; + /* + * If the file size is already greater than the current blocksize, + * we will not grow. If there is more than one block in a file, + * the blocksize cannot change. + */ + if (zp->z_blksz && zp->z_size > zp->z_blksz) + return; + + error = dmu_object_set_blocksize(zp->z_zfsvfs->z_os, + zp->z_id, + size, 0, tx); + + if (error == ENOTSUP) + return; + ASSERT(error == 0); + + /* What blocksize did we actually get? */ + dmu_object_size_from_db(sa_get_db(zp->z_sa_hdl), &zp->z_blksz, &dummy); +} + +#ifdef sun +/* + * This is a dummy interface used when pvn_vplist_dirty() should *not* + * be calling back into the fs for a putpage(). E.g.: when truncating + * a file, the pages being "thrown away* don't need to be written out. + */ + +static int +zfs_no_putpage(struct vnode *vp, page_t *pp, u_offset_t *offp, size_t *lenp, + int flags, cred_t *cr) +{ + ASSERT(0); + return (0); +} +#endif /* sun */ + +/* + * Increase the file length + * + * IN: zp - znode of file to free data in. + * end - new end-of-file + * + * RETURN: 0 on success, error code on failure + */ +static int +zfs_extend(znode_t *zp, uint64_t end) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + dmu_tx_t *tx; + zfs_locked_range_t *lr; + uint64_t newblksz; + int error; + + /* + * We will change zp_size, lock the whole file. + */ + lr = zfs_rangelock_enter(&zp->z_rangelock, 0, UINT64_MAX, RL_WRITER); + + /* + * Nothing to do if file already at desired length. + */ + if (end <= zp->z_size) { + zfs_rangelock_exit(lr); + return (0); + } + + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + zfs_sa_upgrade_txholds(tx, zp); + if (end > zp->z_blksz && + (!ISP2(zp->z_blksz) || zp->z_blksz < zfsvfs->z_max_blksz)) { + /* + * We are growing the file past the current block size. + */ + if (zp->z_blksz > zp->z_zfsvfs->z_max_blksz) { + /* + * File's blocksize is already larger than the + * "recordsize" property. Only let it grow to + * the next power of 2. + */ + ASSERT(!ISP2(zp->z_blksz)); + newblksz = MIN(end, 1 << highbit64(zp->z_blksz)); + } else { + newblksz = MIN(end, zp->z_zfsvfs->z_max_blksz); + } + dmu_tx_hold_write(tx, zp->z_id, 0, newblksz); + } else { + newblksz = 0; + } + + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + zfs_rangelock_exit(lr); + return (error); + } + + if (newblksz) + zfs_grow_blocksize(zp, newblksz, tx); + + zp->z_size = end; + + VERIFY(0 == sa_update(zp->z_sa_hdl, SA_ZPL_SIZE(zp->z_zfsvfs), + &zp->z_size, + sizeof (zp->z_size), tx)); + + vnode_pager_setsize(NULL, ZTOV(zp), end, TRUE); + + zfs_rangelock_exit(lr); + + dmu_tx_commit(tx); + + return (0); +} + + +/* + * Free space in a file. + * + * IN: zp - znode of file to free data in. + * off - start of section to free. + * len - length of section to free. + * + * RETURN: 0 on success, error code on failure + */ +static int +zfs_free_range(znode_t *zp, uint64_t off, uint64_t len) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + zfs_locked_range_t *lr; + int error; + + /* + * Lock the range being freed. + */ + lr = zfs_rangelock_enter(&zp->z_rangelock, off, len, RL_WRITER); + + /* + * Nothing to do if file already at desired length. + */ + if (off >= zp->z_size) { + zfs_rangelock_exit(lr); + return (0); + } + + if (off + len > zp->z_size) + len = zp->z_size - off; + + error = dmu_free_long_range(zfsvfs->z_os, zp->z_id, off, len); + + if (error == 0) { + /* + * In FreeBSD we cannot free block in the middle of a file, + * but only at the end of a file, so this code path should + * never happen. + */ + vnode_pager_setsize(NULL, ZTOV(zp), off, TRUE); + } + +#ifdef _LINUX + /* + * Zero partial page cache entries. This must be done under a + * range lock in order to keep the ARC and page cache in sync. + */ + if (zp->z_is_mapped) { + loff_t first_page, last_page, page_len; + loff_t first_page_offset, last_page_offset; + + /* first possible full page in hole */ + first_page = (off + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT; + /* last page of hole */ + last_page = (off + len) >> PAGE_CACHE_SHIFT; + + /* offset of first_page */ + first_page_offset = first_page << PAGE_CACHE_SHIFT; + /* offset of last_page */ + last_page_offset = last_page << PAGE_CACHE_SHIFT; + + /* truncate whole pages */ + if (last_page_offset > first_page_offset) { + truncate_inode_pages_range(ZTOI(zp)->i_mapping, + first_page_offset, last_page_offset - 1); + } + + /* truncate sub-page ranges */ + if (first_page > last_page) { + /* entire punched area within a single page */ + zfs_zero_partial_page(zp, off, len); + } else { + /* beginning of punched area at the end of a page */ + page_len = first_page_offset - off; + if (page_len > 0) + zfs_zero_partial_page(zp, off, page_len); + + /* end of punched area at the beginning of a page */ + page_len = off + len - last_page_offset; + if (page_len > 0) + zfs_zero_partial_page(zp, last_page_offset, + page_len); + } + } +#endif + zfs_rangelock_exit(lr); + + return (error); +} + +/* + * Truncate a file + * + * IN: zp - znode of file to free data in. + * end - new end-of-file. + * + * RETURN: 0 on success, error code on failure + */ +static int +zfs_trunc(znode_t *zp, uint64_t end) +{ + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + struct vnode *vp = ZTOV(zp); + dmu_tx_t *tx; + zfs_locked_range_t *lr; + int error; + sa_bulk_attr_t bulk[2]; + int count = 0; + /* + * We will change zp_size, lock the whole file. + */ + lr = zfs_rangelock_enter(&zp->z_rangelock, 0, UINT64_MAX, RL_WRITER); + + /* + * Nothing to do if file already at desired length. + */ + if (end >= zp->z_size) { + zfs_rangelock_exit(lr); + return (0); + } + + error = dmu_free_long_range(zfsvfs->z_os, zp->z_id, end, + DMU_OBJECT_END); + if (error) { + zfs_rangelock_exit(lr); + return (error); + } + + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + zfs_sa_upgrade_txholds(tx, zp); + dmu_tx_mark_netfree(tx); + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + zfs_rangelock_exit(lr); + return (error); + } + + zp->z_size = end; + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_SIZE(zfsvfs), + NULL, &zp->z_size, sizeof (zp->z_size)); + + if (end == 0) { + zp->z_pflags &= ~ZFS_SPARSE; + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), + NULL, &zp->z_pflags, 8); + } + VERIFY(sa_bulk_update(zp->z_sa_hdl, bulk, count, tx) == 0); + + dmu_tx_commit(tx); + + /* + * Clear any mapped pages in the truncated region. This has to + * happen outside of the transaction to avoid the possibility of + * a deadlock with someone trying to push a page that we are + * about to invalidate. + */ + vnode_pager_setsize(NULL, vp, end, TRUE); + + zfs_rangelock_exit(lr); + + return (0); +} + +/* + * Free space in a file + * + * IN: zp - znode of file to free data in. + * off - start of range + * len - end of range (0 => EOF) + * flag - current file open mode flags. + * log - TRUE if this action should be logged + * + * RETURN: 0 on success, error code on failure + */ +int +zfs_freesp(znode_t *zp, uint64_t off, uint64_t len, int flag, boolean_t log) +{ +// struct vnode *vp = ZTOV(zp); + dmu_tx_t *tx; + zfsvfs_t *zfsvfs = zp->z_zfsvfs; + zilog_t *zilog = zfsvfs->z_log; + uint64_t mode; + uint64_t mtime[2], ctime[2]; + sa_bulk_attr_t bulk[3]; + int count = 0; + int error; + + if (vnode_isfifo(ZTOV(zp))) + return (0); + + if ((error = sa_lookup(zp->z_sa_hdl, SA_ZPL_MODE(zfsvfs), &mode, + sizeof (mode))) != 0) + return (error); + + if (off > zp->z_size) { + error = zfs_extend(zp, off+len); + if (error == 0 && log) + goto log; + goto out; + } + + if (len == 0) { + error = zfs_trunc(zp, off); + } else { + if ((error = zfs_free_range(zp, off, len)) == 0 && + off + len > zp->z_size) + error = zfs_extend(zp, off+len); + } + if (error || !log) + goto out; +log: + tx = dmu_tx_create(zfsvfs->z_os); + dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_FALSE); + zfs_sa_upgrade_txholds(tx, zp); + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + goto out; + } + + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_MTIME(zfsvfs), NULL, mtime, 16); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_CTIME(zfsvfs), NULL, ctime, 16); + SA_ADD_BULK_ATTR(bulk, count, SA_ZPL_FLAGS(zfsvfs), + NULL, &zp->z_pflags, 8); + zfs_tstamp_update_setup(zp, CONTENT_MODIFIED, mtime, ctime); + error = sa_bulk_update(zp->z_sa_hdl, bulk, count, tx); + ASSERT(error == 0); + + zfs_log_truncate(zilog, tx, TX_TRUNCATE, zp, off, len); + + dmu_tx_commit(tx); + + error = 0; + +out: + + return (error); +} + +void +zfs_create_fs(objset_t *os, cred_t *cr, nvlist_t *zplprops, dmu_tx_t *tx) +{ + zfsvfs_t *zfsvfs; + uint64_t moid, obj, sa_obj, version; + uint64_t sense = ZFS_CASE_SENSITIVE; + uint64_t norm = 0; + nvpair_t *elem; + int size; + int error; + int i; + znode_t *rootzp = NULL; + vattr_t vattr; + znode_t *zp; + zfs_acl_ids_t acl_ids; + + /* + * First attempt to create master node. + */ + /* + * In an empty objset, there are no blocks to read and thus + * there can be no i/o errors (which we assert below). + */ + moid = MASTER_NODE_OBJ; + error = zap_create_claim(os, moid, DMU_OT_MASTER_NODE, + DMU_OT_NONE, 0, tx); + ASSERT(error == 0); + + /* + * Set starting attributes. + */ + version = zfs_zpl_version_map(spa_version(dmu_objset_spa(os))); + elem = NULL; + while ((elem = nvlist_next_nvpair(zplprops, elem)) != NULL) { + /* For the moment we expect all zpl props to be uint64_ts */ + uint64_t val; + char *name; + + ASSERT(nvpair_type(elem) == DATA_TYPE_UINT64); + VERIFY(nvpair_value_uint64(elem, &val) == 0); + name = nvpair_name(elem); + if (strcmp(name, zfs_prop_to_name(ZFS_PROP_VERSION)) == 0) { + if (val < version) + version = val; + } else { + error = zap_update(os, moid, name, 8, 1, &val, tx); + } + ASSERT(error == 0); + if (strcmp(name, zfs_prop_to_name(ZFS_PROP_NORMALIZE)) == 0) + norm = val; + else if (strcmp(name, zfs_prop_to_name(ZFS_PROP_CASE)) == 0) + sense = val; + } + ASSERT(version != 0); + error = zap_update(os, moid, ZPL_VERSION_STR, 8, 1, &version, tx); + + /* + * Create zap object used for SA attribute registration + */ + + if (version >= ZPL_VERSION_SA) { + sa_obj = zap_create(os, DMU_OT_SA_MASTER_NODE, + DMU_OT_NONE, 0, tx); + error = zap_add(os, moid, ZFS_SA_ATTRS, 8, 1, &sa_obj, tx); + ASSERT(error == 0); + } else { + sa_obj = 0; + } + /* + * Create a delete queue. + */ + obj = zap_create(os, DMU_OT_UNLINKED_SET, DMU_OT_NONE, 0, tx); + + error = zap_add(os, moid, ZFS_UNLINKED_SET, 8, 1, &obj, tx); + ASSERT(error == 0); + + /* + * Create root znode. Create minimal znode/vnode/zfsvfs + * to allow zfs_mknode to work. + */ + VATTR_NULL(&vattr); + vattr.va_mask = ATTR_MODE|ATTR_UID|ATTR_GID|ATTR_TYPE; + vattr.va_type = VDIR; + vattr.va_mode = S_IFDIR|0755; + vattr.va_uid = crgetuid(cr); + vattr.va_gid = crgetgid(cr); + + rootzp = kmem_cache_alloc(znode_cache, KM_SLEEP); + ASSERT(!POINTER_IS_VALID(rootzp->z_zfsvfs)); + rootzp->z_unlinked = 0; + rootzp->z_atime_dirty = 0; + rootzp->z_is_sa = USE_SA(version, os); + + rootzp->z_vnode = NULL; + + zfsvfs = kmem_alloc(sizeof (zfsvfs_t), KM_SLEEP); + + memset(zfsvfs, 0, sizeof (zfsvfs_t)); + + zfsvfs->z_os = os; + zfsvfs->z_parent = zfsvfs; + zfsvfs->z_version = version; + zfsvfs->z_use_fuids = USE_FUIDS(version, os); + zfsvfs->z_use_sa = USE_SA(version, os); + zfsvfs->z_norm = norm; + + error = sa_setup(os, sa_obj, zfs_attr_table, ZPL_END, + &zfsvfs->z_attr_table); + + ASSERT(error == 0); + + /* + * Fold case on file systems that are always or sometimes case + * insensitive. + */ + if (sense == ZFS_CASE_INSENSITIVE || sense == ZFS_CASE_MIXED) + zfsvfs->z_norm |= U8_TEXTPREP_TOUPPER; + + mutex_init(&zfsvfs->z_znodes_lock, NULL, MUTEX_DEFAULT, NULL); + list_create(&zfsvfs->z_all_znodes, sizeof (znode_t), + offsetof(znode_t, z_link_node)); + + size = MIN(1 << (highbit64(zfs_object_mutex_size)-1), ZFS_OBJ_MTX_MAX); + zfsvfs->z_hold_size = size; + zfsvfs->z_hold_trees = kmem_zalloc(sizeof (avl_tree_t) * size, + KM_SLEEP); + zfsvfs->z_hold_locks = kmem_zalloc(sizeof (kmutex_t) * size, KM_SLEEP); + for (i = 0; i != size; i++) { + avl_create(&zfsvfs->z_hold_trees[i], zfs_znode_hold_compare, + sizeof (znode_hold_t), offsetof(znode_hold_t, zh_node)); + mutex_init(&zfsvfs->z_hold_locks[i], NULL, MUTEX_DEFAULT, NULL); + } + + rootzp->z_zfsvfs = zfsvfs; + VERIFY(0 == zfs_acl_ids_create(rootzp, IS_ROOT_NODE, &vattr, + cr, NULL, &acl_ids, NULL)); + zfs_mknode(rootzp, &vattr, tx, cr, IS_ROOT_NODE, &zp, &acl_ids); + ASSERT3P(zp, ==, rootzp); + error = zap_add(os, moid, ZFS_ROOT_OBJ, 8, 1, &rootzp->z_id, tx); + ASSERT(error == 0); + zfs_acl_ids_free(&acl_ids); + POINTER_INVALIDATE(&rootzp->z_zfsvfs); + + sa_handle_destroy(rootzp->z_sa_hdl); + rootzp->z_sa_hdl = NULL; + rootzp->z_vnode = NULL; + kmem_cache_free(znode_cache, rootzp); + + for (i = 0; i != size; i++) { + avl_destroy(&zfsvfs->z_hold_trees[i]); + mutex_destroy(&zfsvfs->z_hold_locks[i]); + } + + /* + * Create shares directory + */ + + error = zfs_create_share_dir(zfsvfs, tx); + + ASSERT(error == 0); + + list_destroy(&zfsvfs->z_all_znodes); + mutex_destroy(&zfsvfs->z_znodes_lock); + + kmem_free(zfsvfs->z_hold_trees, sizeof (avl_tree_t) * size); + kmem_free(zfsvfs->z_hold_locks, sizeof (kmutex_t) * size); + + kmem_free(zfsvfs, sizeof (zfsvfs_t)); +} + +void +zfs_znode_update_vfs(znode_t *zp) +{ + vnode_pager_setsize(NULL, ZTOV(zp), zp->z_size, TRUE); +} + +#endif /* _KERNEL */ + +static int +zfs_sa_setup(objset_t *osp, sa_attr_type_t **sa_table) +{ + uint64_t sa_obj = 0; + int error; + + error = zap_lookup(osp, MASTER_NODE_OBJ, ZFS_SA_ATTRS, 8, 1, &sa_obj); + if (error != 0 && error != ENOENT) + return (error); + + error = sa_setup(osp, sa_obj, zfs_attr_table, ZPL_END, sa_table); + return (error); +} +static int +zfs_grab_sa_handle(objset_t *osp, uint64_t obj, sa_handle_t **hdlp, + dmu_buf_t **db, void *tag) +{ + dmu_object_info_t doi; + int error; + + if ((error = sa_buf_hold(osp, obj, tag, db)) != 0) + return (error); + + dmu_object_info_from_db(*db, &doi); + if (((doi.doi_bonus_type != DMU_OT_SA) && + (doi.doi_bonus_type != DMU_OT_ZNODE)) || + ((doi.doi_bonus_type == DMU_OT_ZNODE) && + (doi.doi_bonus_size < sizeof (znode_phys_t)))) { + sa_buf_rele(*db, tag); + return (SET_ERROR(ENOTSUP)); + } + + error = sa_handle_get(osp, obj, NULL, SA_HDL_PRIVATE, hdlp); + if (error != 0) { + sa_buf_rele(*db, tag); + return (error); + } + return (0); +} + +void +zfs_release_sa_handle(sa_handle_t *hdl, dmu_buf_t *db, void *tag) +{ + sa_handle_destroy(hdl); + sa_buf_rele(db, tag); +} + +/* + * Given an object number, return its parent object number and whether + * or not the object is an extended attribute directory. + */ +static int +zfs_obj_to_pobj(objset_t *osp, sa_handle_t *hdl, sa_attr_type_t *sa_table, + uint64_t *pobjp, int *is_xattrdir) +{ + uint64_t parent; + uint64_t pflags; + uint64_t mode; + uint64_t parent_mode; + sa_bulk_attr_t bulk[3]; + sa_handle_t *sa_hdl; + dmu_buf_t *sa_db; + int count = 0; + int error; + + SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_PARENT], NULL, + &parent, sizeof (parent)); + SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_FLAGS], NULL, + &pflags, sizeof (pflags)); + SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_MODE], NULL, + &mode, sizeof (mode)); + + if ((error = sa_bulk_lookup(hdl, bulk, count)) != 0) + return (error); + + /* + * When a link is removed its parent pointer is not changed and will + * be invalid. There are two cases where a link is removed but the + * file stays around, when it goes to the delete queue and when there + * are additional links. + */ + error = zfs_grab_sa_handle(osp, parent, &sa_hdl, &sa_db, FTAG); + if (error != 0) + return (error); + + error = sa_lookup(sa_hdl, ZPL_MODE, &parent_mode, sizeof (parent_mode)); + zfs_release_sa_handle(sa_hdl, sa_db, FTAG); + if (error != 0) + return (error); + + *is_xattrdir = ((pflags & ZFS_XATTR) != 0) && S_ISDIR(mode); + + /* + * Extended attributes can be applied to files, directories, etc. + * Otherwise the parent must be a directory. + */ + if (!*is_xattrdir && !S_ISDIR(parent_mode)) + return ((EINVAL)); + + *pobjp = parent; + + return (0); +} + +/* + * Given an object number, return some zpl level statistics + */ +static int +zfs_obj_to_stats_impl(sa_handle_t *hdl, sa_attr_type_t *sa_table, + zfs_stat_t *sb) +{ + sa_bulk_attr_t bulk[4]; + int count = 0; + + SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_MODE], NULL, + &sb->zs_mode, sizeof (sb->zs_mode)); + SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_GEN], NULL, + &sb->zs_gen, sizeof (sb->zs_gen)); + SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_LINKS], NULL, + &sb->zs_links, sizeof (sb->zs_links)); + SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_CTIME], NULL, + &sb->zs_ctime, sizeof (sb->zs_ctime)); + + return (sa_bulk_lookup(hdl, bulk, count)); +} + +static int +zfs_obj_to_path_impl(objset_t *osp, uint64_t obj, sa_handle_t *hdl, + sa_attr_type_t *sa_table, char *buf, int len) +{ + sa_handle_t *sa_hdl; + sa_handle_t *prevhdl = NULL; + dmu_buf_t *prevdb = NULL; + dmu_buf_t *sa_db = NULL; + char *path = buf + len - 1; + int error; + + *path = '\0'; + sa_hdl = hdl; + + uint64_t deleteq_obj; + VERIFY0(zap_lookup(osp, MASTER_NODE_OBJ, + ZFS_UNLINKED_SET, sizeof (uint64_t), 1, &deleteq_obj)); + error = zap_lookup_int(osp, deleteq_obj, obj); + if (error == 0) { + return (ESTALE); + } else if (error != ENOENT) { + return (error); + } + error = 0; + + for (;;) { + uint64_t pobj = 0; + char component[MAXNAMELEN + 2]; + size_t complen; + int is_xattrdir = 0; + + if (prevdb) + zfs_release_sa_handle(prevhdl, prevdb, FTAG); + + if ((error = zfs_obj_to_pobj(osp, sa_hdl, sa_table, &pobj, + &is_xattrdir)) != 0) + break; + + if (pobj == obj) { + if (path[0] != '/') + *--path = '/'; + break; + } + + component[0] = '/'; + if (is_xattrdir) { + (void) snprintf(component + 1, MAXNAMELEN+1, + ""); + } else { + error = zap_value_search(osp, pobj, obj, + ZFS_DIRENT_OBJ(-1ULL), + component + 1); + if (error != 0) + break; + } + + complen = strlen(component); + path -= complen; + ASSERT(path >= buf); + memcpy(path, component, complen); + obj = pobj; + + if (sa_hdl != hdl) { + prevhdl = sa_hdl; + prevdb = sa_db; + } + error = zfs_grab_sa_handle(osp, obj, &sa_hdl, &sa_db, FTAG); + if (error != 0) { + sa_hdl = prevhdl; + sa_db = prevdb; + break; + } + } + + if (sa_hdl != NULL && sa_hdl != hdl) { + ASSERT(sa_db != NULL); + zfs_release_sa_handle(sa_hdl, sa_db, FTAG); + } + + if (error == 0) + (void) memmove(buf, path, buf + len - path); + + return (error); +} + +int +zfs_obj_to_path(objset_t *osp, uint64_t obj, char *buf, int len) +{ + sa_attr_type_t *sa_table; + sa_handle_t *hdl; + dmu_buf_t *db; + int error; + + error = zfs_sa_setup(osp, &sa_table); + if (error != 0) + return (error); + + error = zfs_grab_sa_handle(osp, obj, &hdl, &db, FTAG); + if (error != 0) + return (error); + + error = zfs_obj_to_path_impl(osp, obj, hdl, sa_table, buf, len); + + zfs_release_sa_handle(hdl, db, FTAG); + return (error); +} + +int +zfs_obj_to_stats(objset_t *osp, uint64_t obj, zfs_stat_t *sb, + char *buf, int len) +{ + char *path = buf + len - 1; + sa_attr_type_t *sa_table; + sa_handle_t *hdl; + dmu_buf_t *db; + int error; + + *path = '\0'; + + error = zfs_sa_setup(osp, &sa_table); + if (error != 0) + return (error); + + error = zfs_grab_sa_handle(osp, obj, &hdl, &db, FTAG); + if (error != 0) + return (error); + + error = zfs_obj_to_stats_impl(hdl, sa_table, sb); + if (error != 0) { + zfs_release_sa_handle(hdl, db, FTAG); + return (error); + } + + error = zfs_obj_to_path_impl(osp, obj, hdl, sa_table, buf, len); + + zfs_release_sa_handle(hdl, db, FTAG); + return (error); +} + +/* + * Read a property stored within the master node. + */ +int +zfs_get_zplprop(objset_t *os, zfs_prop_t prop, uint64_t *value) +{ + uint64_t *cached_copy = NULL; + + /* + * Figure out where in the objset_t the cached copy would live, if it + * is available for the requested property. + */ + if (os != NULL) { + switch (prop) { + case ZFS_PROP_VERSION: + cached_copy = &os->os_version; + break; + case ZFS_PROP_NORMALIZE: + cached_copy = &os->os_normalization; + break; + case ZFS_PROP_UTF8ONLY: + cached_copy = &os->os_utf8only; + break; + case ZFS_PROP_CASE: + cached_copy = &os->os_casesensitivity; + break; + default: + break; + } + } + if (cached_copy != NULL && *cached_copy != OBJSET_PROP_UNINITIALIZED) { + *value = *cached_copy; + return (0); + } + + /* + * If the property wasn't cached, look up the file system's value for + * the property. For the version property, we look up a slightly + * different string. + */ + const char *pname; + int error = ENOENT; + if (prop == ZFS_PROP_VERSION) { + pname = ZPL_VERSION_STR; + } else { + pname = zfs_prop_to_name(prop); + } + + if (os != NULL) { + ASSERT3U(os->os_phys->os_type, ==, DMU_OST_ZFS); + error = zap_lookup(os, MASTER_NODE_OBJ, pname, 8, 1, value); + } + + if (error == ENOENT) { + /* No value set, use the default value */ + switch (prop) { + case ZFS_PROP_VERSION: + *value = ZPL_VERSION; + break; + case ZFS_PROP_NORMALIZE: + case ZFS_PROP_UTF8ONLY: + *value = 0; + break; + case ZFS_PROP_CASE: + *value = ZFS_CASE_SENSITIVE; + break; + case ZFS_PROP_ACLMODE: + *value = ZFS_ACLTYPE_OFF; + break; + default: + return (error); + } + error = 0; + } + + /* + * If one of the methods for getting the property value above worked, + * copy it into the objset_t's cache. + */ + if (error == 0 && cached_copy != NULL) { + *cached_copy = *value; + } + + return (error); +} \ No newline at end of file diff --git a/module/os/windows/zfs/zio_crypt.c b/module/os/windows/zfs/zio_crypt.c new file mode 100644 index 000000000000..e2cc37e1e766 --- /dev/null +++ b/module/os/windows/zfs/zio_crypt.c @@ -0,0 +1,2049 @@ +/* + * CDDL HEADER START + * + * This file and its contents are supplied under the terms of the + * Common Development and Distribution License ("CDDL"), version 1.0. + * You may only use this file in accordance with the terms of version + * 1.0 of the CDDL. + * + * A full copy of the text of the CDDL should have accompanied this + * source. A copy of the CDDL is also available via the Internet at + * http://www.illumos.org/license/CDDL. + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2017, Datto, Inc. All rights reserved. + * Portions Copyright 2022 Andrew Innes + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * This file is responsible for handling all of the details of generating + * encryption parameters and performing encryption and authentication. + * + * BLOCK ENCRYPTION PARAMETERS: + * Encryption /Authentication Algorithm Suite (crypt): + * The encryption algorithm, mode, and key length we are going to use. We + * currently support AES in either GCM or CCM modes with 128, 192, and 256 bit + * keys. All authentication is currently done with SHA512-HMAC. + * + * Plaintext: + * The unencrypted data that we want to encrypt. + * + * Initialization Vector (IV): + * An initialization vector for the encryption algorithms. This is used to + * "tweak" the encryption algorithms so that two blocks of the same data are + * encrypted into different ciphertext outputs, thus obfuscating block patterns. + * The supported encryption modes (AES-GCM and AES-CCM) require that an IV is + * never reused with the same encryption key. This value is stored unencrypted + * and must simply be provided to the decryption function. We use a 96 bit IV + * (as recommended by NIST) for all block encryption. For non-dedup blocks we + * derive the IV randomly. The first 64 bits of the IV are stored in the second + * word of DVA[2] and the remaining 32 bits are stored in the upper 32 bits of + * blk_fill. This is safe because encrypted blocks can't use the upper 32 bits + * of blk_fill. We only encrypt level 0 blocks, which normally have a fill count + * of 1. The only exception is for DMU_OT_DNODE objects, where the fill count of + * level 0 blocks is the number of allocated dnodes in that block. The on-disk + * format supports at most 2^15 slots per L0 dnode block, because the maximum + * block size is 16MB (2^24). In either case, for level 0 blocks this number + * will still be smaller than UINT32_MAX so it is safe to store the IV in the + * top 32 bits of blk_fill, while leaving the bottom 32 bits of the fill count + * for the dnode code. + * + * Master key: + * This is the most important secret data of an encrypted dataset. It is used + * along with the salt to generate that actual encryption keys via HKDF. We + * do not use the master key to directly encrypt any data because there are + * theoretical limits on how much data can actually be safely encrypted with + * any encryption mode. The master key is stored encrypted on disk with the + * user's wrapping key. Its length is determined by the encryption algorithm. + * For details on how this is stored see the block comment in dsl_crypt.c + * + * Salt: + * Used as an input to the HKDF function, along with the master key. We use a + * 64 bit salt, stored unencrypted in the first word of DVA[2]. Any given salt + * can be used for encrypting many blocks, so we cache the current salt and the + * associated derived key in zio_crypt_t so we do not need to derive it again + * needlessly. + * + * Encryption Key: + * A secret binary key, generated from an HKDF function used to encrypt and + * decrypt data. + * + * Message Authentication Code (MAC) + * The MAC is an output of authenticated encryption modes such as AES-GCM and + * AES-CCM. Its purpose is to ensure that an attacker cannot modify encrypted + * data on disk and return garbage to the application. Effectively, it is a + * checksum that can not be reproduced by an attacker. We store the MAC in the + * second 128 bits of blk_cksum, leaving the first 128 bits for a truncated + * regular checksum of the ciphertext which can be used for scrubbing. + * + * OBJECT AUTHENTICATION: + * Some object types, such as DMU_OT_MASTER_NODE cannot be encrypted because + * they contain some info that always needs to be readable. To prevent this + * data from being altered, we authenticate this data using SHA512-HMAC. This + * will produce a MAC (similar to the one produced via encryption) which can + * be used to verify the object was not modified. HMACs do not require key + * rotation or IVs, so we can keep up to the full 3 copies of authenticated + * data. + * + * ZIL ENCRYPTION: + * ZIL blocks have their bp written to disk ahead of the associated data, so we + * cannot store the MAC there as we normally do. For these blocks the MAC is + * stored in the embedded checksum within the zil_chain_t header. The salt and + * IV are generated for the block on bp allocation instead of at encryption + * time. In addition, ZIL blocks have some pieces that must be left in plaintext + * for claiming even though all of the sensitive user data still needs to be + * encrypted. The function zio_crypt_init_uios_zil() handles parsing which + * pieces of the block need to be encrypted. All data that is not encrypted is + * authenticated using the AAD mechanisms that the supported encryption modes + * provide for. In order to preserve the semantics of the ZIL for encrypted + * datasets, the ZIL is not protected at the objset level as described below. + * + * DNODE ENCRYPTION: + * Similarly to ZIL blocks, the core part of each dnode_phys_t needs to be left + * in plaintext for scrubbing and claiming, but the bonus buffers might contain + * sensitive user data. The function zio_crypt_init_uios_dnode() handles parsing + * which pieces of the block need to be encrypted. For more details about + * dnode authentication and encryption, see zio_crypt_init_uios_dnode(). + * + * OBJECT SET AUTHENTICATION: + * Up to this point, everything we have encrypted and authenticated has been + * at level 0 (or -2 for the ZIL). If we did not do any further work the + * on-disk format would be susceptible to attacks that deleted or rearranged + * the order of level 0 blocks. Ideally, the cleanest solution would be to + * maintain a tree of authentication MACs going up the bp tree. However, this + * presents a problem for raw sends. Send files do not send information about + * indirect blocks so there would be no convenient way to transfer the MACs and + * they cannot be recalculated on the receive side without the master key which + * would defeat one of the purposes of raw sends in the first place. Instead, + * for the indirect levels of the bp tree, we use a regular SHA512 of the MACs + * from the level below. We also include some portable fields from blk_prop such + * as the lsize and compression algorithm to prevent the data from being + * misinterpreted. + * + * At the objset level, we maintain 2 separate 256 bit MACs in the + * objset_phys_t. The first one is "portable" and is the logical root of the + * MAC tree maintained in the metadnode's bps. The second, is "local" and is + * used as the root MAC for the user accounting objects, which are also not + * transferred via "zfs send". The portable MAC is sent in the DRR_BEGIN payload + * of the send file. The useraccounting code ensures that the useraccounting + * info is not present upon a receive, so the local MAC can simply be cleared + * out at that time. For more info about objset_phys_t authentication, see + * zio_crypt_do_objset_hmacs(). + * + * CONSIDERATIONS FOR DEDUP: + * In order for dedup to work, blocks that we want to dedup with one another + * need to use the same IV and encryption key, so that they will have the same + * ciphertext. Normally, one should never reuse an IV with the same encryption + * key or else AES-GCM and AES-CCM can both actually leak the plaintext of both + * blocks. In this case, however, since we are using the same plaintext as + * well all that we end up with is a duplicate of the original ciphertext we + * already had. As a result, an attacker with read access to the raw disk will + * be able to tell which blocks are the same but this information is given away + * by dedup anyway. In order to get the same IVs and encryption keys for + * equivalent blocks of data we use an HMAC of the plaintext. We use an HMAC + * here so that a reproducible checksum of the plaintext is never available to + * the attacker. The HMAC key is kept alongside the master key, encrypted on + * disk. The first 64 bits of the HMAC are used in place of the random salt, and + * the next 96 bits are used as the IV. As a result of this mechanism, dedup + * will only work within a clone family since encrypted dedup requires use of + * the same master and HMAC keys. + */ + +/* + * After encrypting many blocks with the same key we may start to run up + * against the theoretical limits of how much data can securely be encrypted + * with a single key using the supported encryption modes. The most obvious + * limitation is that our risk of generating 2 equivalent 96 bit IVs increases + * the more IVs we generate (which both GCM and CCM modes strictly forbid). + * This risk actually grows surprisingly quickly over time according to the + * Birthday Problem. With a total IV space of 2^(96 bits), and assuming we have + * generated n IVs with a cryptographically secure RNG, the approximate + * probability p(n) of a collision is given as: + * + * p(n) ~= e^(-n*(n-1)/(2*(2^96))) + * + * [http://www.math.cornell.edu/~mec/2008-2009/TianyiZheng/Birthday.html] + * + * Assuming that we want to ensure that p(n) never goes over 1 / 1 trillion + * we must not write more than 398,065,730 blocks with the same encryption key. + * Therefore, we rotate our keys after 400,000,000 blocks have been written by + * generating a new random 64 bit salt for our HKDF encryption key generation + * function. + */ +#define ZFS_KEY_MAX_SALT_USES_DEFAULT 400000000 +#define ZFS_CURRENT_MAX_SALT_USES \ + (MIN(zfs_key_max_salt_uses, ZFS_KEY_MAX_SALT_USES_DEFAULT)) +unsigned long zfs_key_max_salt_uses = ZFS_KEY_MAX_SALT_USES_DEFAULT; + +typedef struct blkptr_auth_buf { + uint64_t bab_prop; /* blk_prop - portable mask */ + uint8_t bab_mac[ZIO_DATA_MAC_LEN]; /* MAC from blk_cksum */ + uint64_t bab_pad; /* reserved for future use */ +} blkptr_auth_buf_t; + +const zio_crypt_info_t zio_crypt_table[ZIO_CRYPT_FUNCTIONS] = { + {"", ZC_TYPE_NONE, 0, "inherit"}, + {"", ZC_TYPE_NONE, 0, "on"}, + {"", ZC_TYPE_NONE, 0, "off"}, + {SUN_CKM_AES_CCM, ZC_TYPE_CCM, 16, "aes-128-ccm"}, + {SUN_CKM_AES_CCM, ZC_TYPE_CCM, 24, "aes-192-ccm"}, + {SUN_CKM_AES_CCM, ZC_TYPE_CCM, 32, "aes-256-ccm"}, + {SUN_CKM_AES_GCM, ZC_TYPE_GCM, 16, "aes-128-gcm"}, + {SUN_CKM_AES_GCM, ZC_TYPE_GCM, 24, "aes-192-gcm"}, + {SUN_CKM_AES_GCM, ZC_TYPE_GCM, 32, "aes-256-gcm"} +}; + +void +zio_crypt_key_destroy(zio_crypt_key_t *key) +{ + rw_destroy(&key->zk_salt_lock); + + /* free crypto templates */ + crypto_destroy_ctx_template(key->zk_current_tmpl); + crypto_destroy_ctx_template(key->zk_hmac_tmpl); + + /* zero out sensitive data */ + memset(key, 0, sizeof (zio_crypt_key_t)); +} + +int +zio_crypt_key_init(uint64_t crypt, zio_crypt_key_t *key) +{ + int ret; + crypto_mechanism_t mech; + uint_t keydata_len; + + ASSERT(key != NULL); + ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); + + keydata_len = zio_crypt_table[crypt].ci_keylen; + memset(key, 0, sizeof (zio_crypt_key_t)); + + /* fill keydata buffers and salt with random data */ + ret = random_get_bytes((uint8_t *)&key->zk_guid, sizeof (uint64_t)); + if (ret != 0) + goto error; + + ret = random_get_bytes(key->zk_master_keydata, keydata_len); + if (ret != 0) + goto error; + + ret = random_get_bytes(key->zk_hmac_keydata, SHA512_HMAC_KEYLEN); + if (ret != 0) + goto error; + + ret = random_get_bytes(key->zk_salt, ZIO_DATA_SALT_LEN); + if (ret != 0) + goto error; + + /* derive the current key from the master key */ + ret = hkdf_sha512(key->zk_master_keydata, keydata_len, NULL, 0, + key->zk_salt, ZIO_DATA_SALT_LEN, key->zk_current_keydata, + keydata_len); + if (ret != 0) + goto error; + + /* initialize keys for the ICP */ + key->zk_current_key.ck_data = key->zk_current_keydata; + key->zk_current_key.ck_length = CRYPTO_BYTES2BITS(keydata_len); + + key->zk_hmac_key.ck_data = &key->zk_hmac_key; + key->zk_hmac_key.ck_length = CRYPTO_BYTES2BITS(SHA512_HMAC_KEYLEN); + + /* + * Initialize the crypto templates. It's ok if this fails because + * this is just an optimization. + */ + mech.cm_type = crypto_mech2id(zio_crypt_table[crypt].ci_mechname); + ret = crypto_create_ctx_template(&mech, &key->zk_current_key, + &key->zk_current_tmpl); + if (ret != CRYPTO_SUCCESS) + key->zk_current_tmpl = NULL; + + mech.cm_type = crypto_mech2id(SUN_CKM_SHA512_HMAC); + ret = crypto_create_ctx_template(&mech, &key->zk_hmac_key, + &key->zk_hmac_tmpl); + if (ret != CRYPTO_SUCCESS) + key->zk_hmac_tmpl = NULL; + + key->zk_crypt = crypt; + key->zk_version = ZIO_CRYPT_KEY_CURRENT_VERSION; + key->zk_salt_count = 0; + rw_init(&key->zk_salt_lock, NULL, RW_DEFAULT, NULL); + + return (0); + +error: + zio_crypt_key_destroy(key); + return (ret); +} + +static int +zio_crypt_key_change_salt(zio_crypt_key_t *key) +{ + int ret = 0; + uint8_t salt[ZIO_DATA_SALT_LEN]; + crypto_mechanism_t mech; + uint_t keydata_len = zio_crypt_table[key->zk_crypt].ci_keylen; + + /* generate a new salt */ + ret = random_get_bytes(salt, ZIO_DATA_SALT_LEN); + if (ret != 0) + goto error; + + rw_enter(&key->zk_salt_lock, RW_WRITER); + + /* someone beat us to the salt rotation, just unlock and return */ + if (key->zk_salt_count < ZFS_CURRENT_MAX_SALT_USES) + goto out_unlock; + + /* derive the current key from the master key and the new salt */ + ret = hkdf_sha512(key->zk_master_keydata, keydata_len, NULL, 0, + salt, ZIO_DATA_SALT_LEN, key->zk_current_keydata, keydata_len); + if (ret != 0) + goto out_unlock; + + /* assign the salt and reset the usage count */ + memcpy(key->zk_salt, salt, ZIO_DATA_SALT_LEN); + key->zk_salt_count = 0; + + /* destroy the old context template and create the new one */ + crypto_destroy_ctx_template(key->zk_current_tmpl); + ret = crypto_create_ctx_template(&mech, &key->zk_current_key, + &key->zk_current_tmpl); + if (ret != CRYPTO_SUCCESS) + key->zk_current_tmpl = NULL; + + rw_exit(&key->zk_salt_lock); + + return (0); + +out_unlock: + rw_exit(&key->zk_salt_lock); +error: + return (ret); +} + +/* See comment above zfs_key_max_salt_uses definition for details */ +int +zio_crypt_key_get_salt(zio_crypt_key_t *key, uint8_t *salt) +{ + int ret; + boolean_t salt_change; + + rw_enter(&key->zk_salt_lock, RW_READER); + + memcpy(salt, key->zk_salt, ZIO_DATA_SALT_LEN); + salt_change = (atomic_inc_64_nv(&key->zk_salt_count) >= + ZFS_CURRENT_MAX_SALT_USES); + + rw_exit(&key->zk_salt_lock); + + if (salt_change) { + ret = zio_crypt_key_change_salt(key); + if (ret != 0) + goto error; + } + + return (0); + +error: + return (ret); +} + +/* + * This function handles all encryption and decryption in zfs. When + * encrypting it expects puio to reference the plaintext and cuio to + * reference the ciphertext. cuio must have enough space for the + * ciphertext + room for a MAC. datalen should be the length of the + * plaintext / ciphertext alone. + */ +static int +zio_do_crypt_uio(boolean_t encrypt, uint64_t crypt, crypto_key_t *key, + crypto_ctx_template_t tmpl, uint8_t *ivbuf, uint_t datalen, + zfs_uio_t *puio, zfs_uio_t *cuio, uint8_t *authbuf, uint_t auth_len) +{ + int ret; + crypto_data_t plaindata, cipherdata; + CK_AES_CCM_PARAMS ccmp; + CK_AES_GCM_PARAMS gcmp; + crypto_mechanism_t mech; + zio_crypt_info_t crypt_info; + uint_t plain_full_len, maclen; + + ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); + + /* lookup the encryption info */ + crypt_info = zio_crypt_table[crypt]; + + /* the mac will always be the last iovec_t in the cipher uio */ + maclen = cuio->uio_iov[cuio->uio_iovcnt - 1].iov_len; + + ASSERT(maclen <= ZIO_DATA_MAC_LEN); + + /* setup encryption mechanism (same as crypt) */ + mech.cm_type = crypto_mech2id(crypt_info.ci_mechname); + + /* + * Strangely, the ICP requires that plain_full_len must include + * the MAC length when decrypting, even though the UIO does not + * need to have the extra space allocated. + */ + if (encrypt) { + plain_full_len = datalen; + } else { + plain_full_len = datalen + maclen; + } + + /* + * setup encryption params (currently only AES CCM and AES GCM + * are supported) + */ + if (crypt_info.ci_crypt_type == ZC_TYPE_CCM) { + ccmp.ulNonceSize = ZIO_DATA_IV_LEN; + ccmp.ulAuthDataSize = auth_len; + ccmp.authData = authbuf; + ccmp.ulMACSize = maclen; + ccmp.nonce = ivbuf; + ccmp.ulDataSize = plain_full_len; + + mech.cm_param = (char *)(&ccmp); + mech.cm_param_len = sizeof (CK_AES_CCM_PARAMS); + } else { + gcmp.ulIvLen = ZIO_DATA_IV_LEN; + gcmp.ulIvBits = CRYPTO_BYTES2BITS(ZIO_DATA_IV_LEN); + gcmp.ulAADLen = auth_len; + gcmp.pAAD = authbuf; + gcmp.ulTagBits = CRYPTO_BYTES2BITS(maclen); + gcmp.pIv = ivbuf; + + mech.cm_param = (char *)(&gcmp); + mech.cm_param_len = sizeof (CK_AES_GCM_PARAMS); + } + + /* populate the cipher and plain data structs. */ + plaindata.cd_format = CRYPTO_DATA_UIO; + plaindata.cd_offset = 0; + plaindata.cd_uio = puio; + plaindata.cd_length = plain_full_len; + + cipherdata.cd_format = CRYPTO_DATA_UIO; + cipherdata.cd_offset = 0; + cipherdata.cd_uio = cuio; + cipherdata.cd_length = datalen + maclen; + + /* perform the actual encryption */ + if (encrypt) { + ret = crypto_encrypt(&mech, &plaindata, key, tmpl, &cipherdata); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + } else { + ret = crypto_decrypt(&mech, &cipherdata, key, tmpl, &plaindata); + if (ret != CRYPTO_SUCCESS) { + ASSERT3U(ret, ==, CRYPTO_INVALID_MAC); + ret = SET_ERROR(ECKSUM); + goto error; + } + } + + return (0); + +error: + return (ret); +} + +int +zio_crypt_key_wrap(crypto_key_t *cwkey, zio_crypt_key_t *key, uint8_t *iv, + uint8_t *mac, uint8_t *keydata_out, uint8_t *hmac_keydata_out) +{ + int ret; + zfs_uio_t puio, cuio; + uint64_t aad[3]; + iovec_t plain_iovecs[2], cipher_iovecs[3]; + uint64_t crypt = key->zk_crypt; + uint_t enc_len, keydata_len, aad_len; + + ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); + + keydata_len = zio_crypt_table[crypt].ci_keylen; + + /* generate iv for wrapping the master and hmac key */ + ret = random_get_pseudo_bytes(iv, WRAPPING_IV_LEN); + if (ret != 0) + goto error; + + /* initialize zfs_uio_ts */ + plain_iovecs[0].iov_base = key->zk_master_keydata; + plain_iovecs[0].iov_len = keydata_len; + plain_iovecs[1].iov_base = key->zk_hmac_keydata; + plain_iovecs[1].iov_len = SHA512_HMAC_KEYLEN; + + cipher_iovecs[0].iov_base = keydata_out; + cipher_iovecs[0].iov_len = keydata_len; + cipher_iovecs[1].iov_base = hmac_keydata_out; + cipher_iovecs[1].iov_len = SHA512_HMAC_KEYLEN; + cipher_iovecs[2].iov_base = mac; + cipher_iovecs[2].iov_len = WRAPPING_MAC_LEN; + + /* + * Although we don't support writing to the old format, we do + * support rewrapping the key so that the user can move and + * quarantine datasets on the old format. + */ + if (key->zk_version == 0) { + aad_len = sizeof (uint64_t); + aad[0] = LE_64(key->zk_guid); + } else { + ASSERT3U(key->zk_version, ==, ZIO_CRYPT_KEY_CURRENT_VERSION); + aad_len = sizeof (uint64_t) * 3; + aad[0] = LE_64(key->zk_guid); + aad[1] = LE_64(crypt); + aad[2] = LE_64(key->zk_version); + } + + enc_len = zio_crypt_table[crypt].ci_keylen + SHA512_HMAC_KEYLEN; + puio.uio_iov = plain_iovecs; + puio.uio_iovcnt = 2; + puio.uio_segflg = UIO_SYSSPACE; + cuio.uio_iov = cipher_iovecs; + cuio.uio_iovcnt = 3; + cuio.uio_segflg = UIO_SYSSPACE; + + /* encrypt the keys and store the resulting ciphertext and mac */ + ret = zio_do_crypt_uio(B_TRUE, crypt, cwkey, NULL, iv, enc_len, + &puio, &cuio, (uint8_t *)aad, aad_len); + if (ret != 0) + goto error; + + return (0); + +error: + return (ret); +} + +int +zio_crypt_key_unwrap(crypto_key_t *cwkey, uint64_t crypt, uint64_t version, + uint64_t guid, uint8_t *keydata, uint8_t *hmac_keydata, uint8_t *iv, + uint8_t *mac, zio_crypt_key_t *key) +{ + crypto_mechanism_t mech; + zfs_uio_t puio, cuio; + uint64_t aad[3]; + iovec_t plain_iovecs[2], cipher_iovecs[3]; + uint_t enc_len, keydata_len, aad_len; + int ret; + + ASSERT3U(crypt, <, ZIO_CRYPT_FUNCTIONS); + + rw_init(&key->zk_salt_lock, NULL, RW_DEFAULT, NULL); + + keydata_len = zio_crypt_table[crypt].ci_keylen; + + /* initialize zfs_uio_ts */ + plain_iovecs[0].iov_base = key->zk_master_keydata; + plain_iovecs[0].iov_len = keydata_len; + plain_iovecs[1].iov_base = key->zk_hmac_keydata; + plain_iovecs[1].iov_len = SHA512_HMAC_KEYLEN; + + cipher_iovecs[0].iov_base = keydata; + cipher_iovecs[0].iov_len = keydata_len; + cipher_iovecs[1].iov_base = hmac_keydata; + cipher_iovecs[1].iov_len = SHA512_HMAC_KEYLEN; + cipher_iovecs[2].iov_base = mac; + cipher_iovecs[2].iov_len = WRAPPING_MAC_LEN; + + if (version == 0) { + aad_len = sizeof (uint64_t); + aad[0] = LE_64(guid); + } else { + ASSERT3U(version, ==, ZIO_CRYPT_KEY_CURRENT_VERSION); + aad_len = sizeof (uint64_t) * 3; + aad[0] = LE_64(guid); + aad[1] = LE_64(crypt); + aad[2] = LE_64(version); + } + + enc_len = keydata_len + SHA512_HMAC_KEYLEN; + puio.uio_iov = plain_iovecs; + puio.uio_segflg = UIO_SYSSPACE; + puio.uio_iovcnt = 2; + cuio.uio_iov = cipher_iovecs; + cuio.uio_iovcnt = 3; + cuio.uio_segflg = UIO_SYSSPACE; + + /* decrypt the keys and store the result in the output buffers */ + ret = zio_do_crypt_uio(B_FALSE, crypt, cwkey, NULL, iv, enc_len, + &puio, &cuio, (uint8_t *)aad, aad_len); + if (ret != 0) + goto error; + + /* generate a fresh salt */ + ret = random_get_bytes(key->zk_salt, ZIO_DATA_SALT_LEN); + if (ret != 0) + goto error; + + /* derive the current key from the master key */ + ret = hkdf_sha512(key->zk_master_keydata, keydata_len, NULL, 0, + key->zk_salt, ZIO_DATA_SALT_LEN, key->zk_current_keydata, + keydata_len); + if (ret != 0) + goto error; + + /* initialize keys for ICP */ + key->zk_current_key.ck_data = key->zk_current_keydata; + key->zk_current_key.ck_length = CRYPTO_BYTES2BITS(keydata_len); + + key->zk_hmac_key.ck_data = key->zk_hmac_keydata; + key->zk_hmac_key.ck_length = CRYPTO_BYTES2BITS(SHA512_HMAC_KEYLEN); + + /* + * Initialize the crypto templates. It's ok if this fails because + * this is just an optimization. + */ + mech.cm_type = crypto_mech2id(zio_crypt_table[crypt].ci_mechname); + ret = crypto_create_ctx_template(&mech, &key->zk_current_key, + &key->zk_current_tmpl); + if (ret != CRYPTO_SUCCESS) + key->zk_current_tmpl = NULL; + + mech.cm_type = crypto_mech2id(SUN_CKM_SHA512_HMAC); + ret = crypto_create_ctx_template(&mech, &key->zk_hmac_key, + &key->zk_hmac_tmpl); + if (ret != CRYPTO_SUCCESS) + key->zk_hmac_tmpl = NULL; + + key->zk_crypt = crypt; + key->zk_version = version; + key->zk_guid = guid; + key->zk_salt_count = 0; + + return (0); + +error: + zio_crypt_key_destroy(key); + return (ret); +} + +int +zio_crypt_generate_iv(uint8_t *ivbuf) +{ + int ret; + + /* randomly generate the IV */ + ret = random_get_pseudo_bytes(ivbuf, ZIO_DATA_IV_LEN); + if (ret != 0) + goto error; + + return (0); + +error: + memset(ivbuf, 0, ZIO_DATA_IV_LEN); + return (ret); +} + +int +zio_crypt_do_hmac(zio_crypt_key_t *key, uint8_t *data, uint_t datalen, + uint8_t *digestbuf, uint_t digestlen) +{ + int ret; + crypto_mechanism_t mech; + crypto_data_t in_data, digest_data; + uint8_t raw_digestbuf[SHA512_DIGEST_LENGTH]; + + ASSERT3U(digestlen, <=, SHA512_DIGEST_LENGTH); + + /* initialize sha512-hmac mechanism and crypto data */ + mech.cm_type = crypto_mech2id(SUN_CKM_SHA512_HMAC); + mech.cm_param = NULL; + mech.cm_param_len = 0; + + /* initialize the crypto data */ + in_data.cd_format = CRYPTO_DATA_RAW; + in_data.cd_offset = 0; + in_data.cd_length = datalen; + in_data.cd_raw.iov_base = (char *)data; + in_data.cd_raw.iov_len = in_data.cd_length; + + digest_data.cd_format = CRYPTO_DATA_RAW; + digest_data.cd_offset = 0; + digest_data.cd_length = SHA512_DIGEST_LENGTH; + digest_data.cd_raw.iov_base = (char *)raw_digestbuf; + digest_data.cd_raw.iov_len = digest_data.cd_length; + + /* generate the hmac */ + ret = crypto_mac(&mech, &in_data, &key->zk_hmac_key, key->zk_hmac_tmpl, + &digest_data); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + memcpy(digestbuf, raw_digestbuf, digestlen); + + return (0); + +error: + memset(digestbuf, 0, digestlen); + return (ret); +} + +int +zio_crypt_generate_iv_salt_dedup(zio_crypt_key_t *key, uint8_t *data, + uint_t datalen, uint8_t *ivbuf, uint8_t *salt) +{ + int ret; + uint8_t digestbuf[SHA512_DIGEST_LENGTH]; + + ret = zio_crypt_do_hmac(key, data, datalen, + digestbuf, SHA512_DIGEST_LENGTH); + if (ret != 0) + return (ret); + + memcpy(salt, digestbuf, ZIO_DATA_SALT_LEN); + memcpy(ivbuf, digestbuf + ZIO_DATA_SALT_LEN, ZIO_DATA_IV_LEN); + + return (0); +} + +/* + * The following functions are used to encode and decode encryption parameters + * into blkptr_t and zil_header_t. The ICP wants to use these parameters as + * byte strings, which normally means that these strings would not need to deal + * with byteswapping at all. However, both blkptr_t and zil_header_t may be + * byteswapped by lower layers and so we must "undo" that byteswap here upon + * decoding and encoding in a non-native byteorder. These functions require + * that the byteorder bit is correct before being called. + */ +void +zio_crypt_encode_params_bp(blkptr_t *bp, uint8_t *salt, uint8_t *iv) +{ + uint64_t val64; + uint32_t val32; + + ASSERT(BP_IS_ENCRYPTED(bp)); + + if (!BP_SHOULD_BYTESWAP(bp)) { + memcpy(&bp->blk_dva[2].dva_word[0], salt, sizeof (uint64_t)); + memcpy(&bp->blk_dva[2].dva_word[1], iv, sizeof (uint64_t)); + memcpy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); + BP_SET_IV2(bp, val32); + } else { + memcpy(&val64, salt, sizeof (uint64_t)); + bp->blk_dva[2].dva_word[0] = BSWAP_64(val64); + + memcpy(&val64, iv, sizeof (uint64_t)); + bp->blk_dva[2].dva_word[1] = BSWAP_64(val64); + + memcpy(&val32, iv + sizeof (uint64_t), sizeof (uint32_t)); + BP_SET_IV2(bp, BSWAP_32(val32)); + } +} + +void +zio_crypt_decode_params_bp(const blkptr_t *bp, uint8_t *salt, uint8_t *iv) +{ + uint64_t val64; + uint32_t val32; + + ASSERT(BP_IS_PROTECTED(bp)); + + /* for convenience, so callers don't need to check */ + if (BP_IS_AUTHENTICATED(bp)) { + memset(salt, 0, ZIO_DATA_SALT_LEN); + memset(iv, 0, ZIO_DATA_IV_LEN); + return; + } + + if (!BP_SHOULD_BYTESWAP(bp)) { + memcpy(salt, &bp->blk_dva[2].dva_word[0], sizeof (uint64_t)); + memcpy(iv, &bp->blk_dva[2].dva_word[1], sizeof (uint64_t)); + + val32 = (uint32_t)BP_GET_IV2(bp); + memcpy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); + } else { + val64 = BSWAP_64(bp->blk_dva[2].dva_word[0]); + memcpy(salt, &val64, sizeof (uint64_t)); + + val64 = BSWAP_64(bp->blk_dva[2].dva_word[1]); + memcpy(iv, &val64, sizeof (uint64_t)); + + val32 = BSWAP_32((uint32_t)BP_GET_IV2(bp)); + memcpy(iv + sizeof (uint64_t), &val32, sizeof (uint32_t)); + } +} + +void +zio_crypt_encode_mac_bp(blkptr_t *bp, uint8_t *mac) +{ + uint64_t val64; + + ASSERT(BP_USES_CRYPT(bp)); + ASSERT3U(BP_GET_TYPE(bp), !=, DMU_OT_OBJSET); + + if (!BP_SHOULD_BYTESWAP(bp)) { + memcpy(&bp->blk_cksum.zc_word[2], mac, sizeof (uint64_t)); + memcpy(&bp->blk_cksum.zc_word[3], mac + sizeof (uint64_t), + sizeof (uint64_t)); + } else { + memcpy(&val64, mac, sizeof (uint64_t)); + bp->blk_cksum.zc_word[2] = BSWAP_64(val64); + + memcpy(&val64, mac + sizeof (uint64_t), sizeof (uint64_t)); + bp->blk_cksum.zc_word[3] = BSWAP_64(val64); + } +} + +void +zio_crypt_decode_mac_bp(const blkptr_t *bp, uint8_t *mac) +{ + uint64_t val64; + + ASSERT(BP_USES_CRYPT(bp) || BP_IS_HOLE(bp)); + + /* for convenience, so callers don't need to check */ + if (BP_GET_TYPE(bp) == DMU_OT_OBJSET) { + memset(mac, 0, ZIO_DATA_MAC_LEN); + return; + } + + if (!BP_SHOULD_BYTESWAP(bp)) { + memcpy(mac, &bp->blk_cksum.zc_word[2], sizeof (uint64_t)); + memcpy(mac + sizeof (uint64_t), &bp->blk_cksum.zc_word[3], + sizeof (uint64_t)); + } else { + val64 = BSWAP_64(bp->blk_cksum.zc_word[2]); + memcpy(mac, &val64, sizeof (uint64_t)); + + val64 = BSWAP_64(bp->blk_cksum.zc_word[3]); + memcpy(mac + sizeof (uint64_t), &val64, sizeof (uint64_t)); + } +} + +void +zio_crypt_encode_mac_zil(void *data, uint8_t *mac) +{ + zil_chain_t *zilc = data; + + memcpy(&zilc->zc_eck.zec_cksum.zc_word[2], mac, sizeof (uint64_t)); + memcpy(&zilc->zc_eck.zec_cksum.zc_word[3], mac + sizeof (uint64_t), + sizeof (uint64_t)); +} + +void +zio_crypt_decode_mac_zil(const void *data, uint8_t *mac) +{ + /* + * The ZIL MAC is embedded in the block it protects, which will + * not have been byteswapped by the time this function has been called. + * As a result, we don't need to worry about byteswapping the MAC. + */ + const zil_chain_t *zilc = data; + + memcpy(mac, &zilc->zc_eck.zec_cksum.zc_word[2], sizeof (uint64_t)); + memcpy(mac + sizeof (uint64_t), &zilc->zc_eck.zec_cksum.zc_word[3], + sizeof (uint64_t)); +} + +/* + * This routine takes a block of dnodes (src_abd) and copies only the bonus + * buffers to the same offsets in the dst buffer. datalen should be the size + * of both the src_abd and the dst buffer (not just the length of the bonus + * buffers). + */ +void +zio_crypt_copy_dnode_bonus(abd_t *src_abd, uint8_t *dst, uint_t datalen) +{ + uint_t i, max_dnp = datalen >> DNODE_SHIFT; + uint8_t *src; + dnode_phys_t *dnp, *sdnp, *ddnp; + + src = abd_borrow_buf_copy(src_abd, datalen); + + sdnp = (dnode_phys_t *)src; + ddnp = (dnode_phys_t *)dst; + + for (i = 0; i < max_dnp; i += sdnp[i].dn_extra_slots + 1) { + dnp = &sdnp[i]; + if (dnp->dn_type != DMU_OT_NONE && + DMU_OT_IS_ENCRYPTED(dnp->dn_bonustype) && + dnp->dn_bonuslen != 0) { + memcpy(DN_BONUS(&ddnp[i]), DN_BONUS(dnp), + DN_MAX_BONUS_LEN(dnp)); + } + } + + abd_return_buf(src_abd, src, datalen); +} + +/* + * This function decides what fields from blk_prop are included in + * the on-disk various MAC algorithms. + */ +static void +zio_crypt_bp_zero_nonportable_blkprop(blkptr_t *bp, uint64_t version) +{ + /* + * Version 0 did not properly zero out all non-portable fields + * as it should have done. We maintain this code so that we can + * do read-only imports of pools on this version. + */ + if (version == 0) { + BP_SET_DEDUP(bp, 0); + BP_SET_CHECKSUM(bp, 0); + BP_SET_PSIZE(bp, SPA_MINBLOCKSIZE); + return; + } + + ASSERT3U(version, ==, ZIO_CRYPT_KEY_CURRENT_VERSION); + + /* + * The hole_birth feature might set these fields even if this bp + * is a hole. We zero them out here to guarantee that raw sends + * will function with or without the feature. + */ + if (BP_IS_HOLE(bp)) { + bp->blk_prop = 0ULL; + return; + } + + /* + * At L0 we want to verify these fields to ensure that data blocks + * can not be reinterpreted. For instance, we do not want an attacker + * to trick us into returning raw lz4 compressed data to the user + * by modifying the compression bits. At higher levels, we cannot + * enforce this policy since raw sends do not convey any information + * about indirect blocks, so these values might be different on the + * receive side. Fortunately, this does not open any new attack + * vectors, since any alterations that can be made to a higher level + * bp must still verify the correct order of the layer below it. + */ + if (BP_GET_LEVEL(bp) != 0) { + BP_SET_BYTEORDER(bp, 0); + BP_SET_COMPRESS(bp, 0); + + /* + * psize cannot be set to zero or it will trigger + * asserts, but the value doesn't really matter as + * long as it is constant. + */ + BP_SET_PSIZE(bp, SPA_MINBLOCKSIZE); + } + + BP_SET_DEDUP(bp, 0); + BP_SET_CHECKSUM(bp, 0); +} + +static void +zio_crypt_bp_auth_init(uint64_t version, boolean_t should_bswap, blkptr_t *bp, + blkptr_auth_buf_t *bab, uint_t *bab_len) +{ + blkptr_t tmpbp = *bp; + + if (should_bswap) + byteswap_uint64_array(&tmpbp, sizeof (blkptr_t)); + + ASSERT(BP_USES_CRYPT(&tmpbp) || BP_IS_HOLE(&tmpbp)); + ASSERT0(BP_IS_EMBEDDED(&tmpbp)); + + zio_crypt_decode_mac_bp(&tmpbp, bab->bab_mac); + + /* + * We always MAC blk_prop in LE to ensure portability. This + * must be done after decoding the mac, since the endianness + * will get zero'd out here. + */ + zio_crypt_bp_zero_nonportable_blkprop(&tmpbp, version); + bab->bab_prop = LE_64(tmpbp.blk_prop); + bab->bab_pad = 0ULL; + + /* version 0 did not include the padding */ + *bab_len = sizeof (blkptr_auth_buf_t); + if (version == 0) + *bab_len -= sizeof (uint64_t); +} + +static int +zio_crypt_bp_do_hmac_updates(crypto_context_t ctx, uint64_t version, + boolean_t should_bswap, blkptr_t *bp) +{ + int ret; + uint_t bab_len; + blkptr_auth_buf_t bab; + crypto_data_t cd; + + zio_crypt_bp_auth_init(version, should_bswap, bp, &bab, &bab_len); + cd.cd_format = CRYPTO_DATA_RAW; + cd.cd_offset = 0; + cd.cd_length = bab_len; + cd.cd_raw.iov_base = (char *)&bab; + cd.cd_raw.iov_len = cd.cd_length; + + ret = crypto_mac_update(ctx, &cd); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + return (0); + +error: + return (ret); +} + +static void +zio_crypt_bp_do_indrect_checksum_updates(SHA2_CTX *ctx, uint64_t version, + boolean_t should_bswap, blkptr_t *bp) +{ + uint_t bab_len; + blkptr_auth_buf_t bab; + + zio_crypt_bp_auth_init(version, should_bswap, bp, &bab, &bab_len); + SHA2Update(ctx, &bab, bab_len); +} + +static void +zio_crypt_bp_do_aad_updates(uint8_t **aadp, uint_t *aad_len, uint64_t version, + boolean_t should_bswap, blkptr_t *bp) +{ + uint_t bab_len; + blkptr_auth_buf_t bab; + + zio_crypt_bp_auth_init(version, should_bswap, bp, &bab, &bab_len); + memcpy(*aadp, &bab, bab_len); + *aadp += bab_len; + *aad_len += bab_len; +} + +static int +zio_crypt_do_dnode_hmac_updates(crypto_context_t ctx, uint64_t version, + boolean_t should_bswap, dnode_phys_t *dnp) +{ + int ret, i; + dnode_phys_t *adnp, tmp_dncore; + size_t dn_core_size = offsetof(dnode_phys_t, dn_blkptr); + boolean_t le_bswap = (should_bswap == ZFS_HOST_BYTEORDER); + crypto_data_t cd; + + cd.cd_format = CRYPTO_DATA_RAW; + cd.cd_offset = 0; + + /* + * Authenticate the core dnode (masking out non-portable bits). + * We only copy the first 64 bytes we operate on to avoid the overhead + * of copying 512-64 unneeded bytes. The compiler seems to be fine + * with that. + */ + memcpy(&tmp_dncore, dnp, dn_core_size); + adnp = &tmp_dncore; + + if (le_bswap) { + adnp->dn_datablkszsec = BSWAP_16(adnp->dn_datablkszsec); + adnp->dn_bonuslen = BSWAP_16(adnp->dn_bonuslen); + adnp->dn_maxblkid = BSWAP_64(adnp->dn_maxblkid); + adnp->dn_used = BSWAP_64(adnp->dn_used); + } + adnp->dn_flags &= DNODE_CRYPT_PORTABLE_FLAGS_MASK; + adnp->dn_used = 0; + + cd.cd_length = dn_core_size; + cd.cd_raw.iov_base = (char *)adnp; + cd.cd_raw.iov_len = cd.cd_length; + + ret = crypto_mac_update(ctx, &cd); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + for (i = 0; i < dnp->dn_nblkptr; i++) { + ret = zio_crypt_bp_do_hmac_updates(ctx, version, + should_bswap, &dnp->dn_blkptr[i]); + if (ret != 0) + goto error; + } + + if (dnp->dn_flags & DNODE_FLAG_SPILL_BLKPTR) { + ret = zio_crypt_bp_do_hmac_updates(ctx, version, + should_bswap, DN_SPILL_BLKPTR(dnp)); + if (ret != 0) + goto error; + } + + return (0); + +error: + return (ret); +} + +/* + * objset_phys_t blocks introduce a number of exceptions to the normal + * authentication process. objset_phys_t's contain 2 separate HMACS for + * protecting the integrity of their data. The portable_mac protects the + * metadnode. This MAC can be sent with a raw send and protects against + * reordering of data within the metadnode. The local_mac protects the user + * accounting objects which are not sent from one system to another. + * + * In addition, objset blocks are the only blocks that can be modified and + * written to disk without the key loaded under certain circumstances. During + * zil_claim() we need to be able to update the zil_header_t to complete + * claiming log blocks and during raw receives we need to write out the + * portable_mac from the send file. Both of these actions are possible + * because these fields are not protected by either MAC so neither one will + * need to modify the MACs without the key. However, when the modified blocks + * are written out they will be byteswapped into the host machine's native + * endianness which will modify fields protected by the MAC. As a result, MAC + * calculation for objset blocks works slightly differently from other block + * types. Where other block types MAC the data in whatever endianness is + * written to disk, objset blocks always MAC little endian version of their + * values. In the code, should_bswap is the value from BP_SHOULD_BYTESWAP() + * and le_bswap indicates whether a byteswap is needed to get this block + * into little endian format. + */ +static int +zio_crypt_do_objset_hmacs(zio_crypt_key_t *key, void *data, + uint_t datalen, boolean_t should_bswap, uint8_t *portable_mac, + uint8_t *local_mac) +{ + int ret; + crypto_mechanism_t mech; + crypto_context_t ctx; + crypto_data_t cd; + objset_phys_t *osp = data; + uint64_t intval; + boolean_t le_bswap = (should_bswap == ZFS_HOST_BYTEORDER); + uint8_t raw_portable_mac[SHA512_DIGEST_LENGTH]; + uint8_t raw_local_mac[SHA512_DIGEST_LENGTH]; + + /* initialize HMAC mechanism */ + mech.cm_type = crypto_mech2id(SUN_CKM_SHA512_HMAC); + mech.cm_param = NULL; + mech.cm_param_len = 0; + + cd.cd_format = CRYPTO_DATA_RAW; + cd.cd_offset = 0; + + /* calculate the portable MAC from the portable fields and metadnode */ + ret = crypto_mac_init(&mech, &key->zk_hmac_key, NULL, &ctx); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + /* add in the os_type */ + intval = (le_bswap) ? osp->os_type : BSWAP_64(osp->os_type); + cd.cd_length = sizeof (uint64_t); + cd.cd_raw.iov_base = (char *)&intval; + cd.cd_raw.iov_len = cd.cd_length; + + ret = crypto_mac_update(ctx, &cd); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + /* add in the portable os_flags */ + intval = osp->os_flags; + if (should_bswap) + intval = BSWAP_64(intval); + intval &= OBJSET_CRYPT_PORTABLE_FLAGS_MASK; + if (!ZFS_HOST_BYTEORDER) + intval = BSWAP_64(intval); + + cd.cd_length = sizeof (uint64_t); + cd.cd_raw.iov_base = (char *)&intval; + cd.cd_raw.iov_len = cd.cd_length; + + ret = crypto_mac_update(ctx, &cd); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + /* add in fields from the metadnode */ + ret = zio_crypt_do_dnode_hmac_updates(ctx, key->zk_version, + should_bswap, &osp->os_meta_dnode); + if (ret) + goto error; + + /* store the final digest in a temporary buffer and copy what we need */ + cd.cd_length = SHA512_DIGEST_LENGTH; + cd.cd_raw.iov_base = (char *)raw_portable_mac; + cd.cd_raw.iov_len = cd.cd_length; + + ret = crypto_mac_final(ctx, &cd); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + memcpy(portable_mac, raw_portable_mac, ZIO_OBJSET_MAC_LEN); + /* + * This is necessary here as we check next whether + * OBJSET_FLAG_USERACCOUNTING_COMPLETE is set in order to + * decide if the local_mac should be zeroed out. That flag will always + * be set by dmu_objset_id_quota_upgrade_cb() and + * dmu_objset_userspace_upgrade_cb() if useraccounting has been + * completed. + */ + intval = osp->os_flags; + if (should_bswap) + intval = BSWAP_64(intval); + boolean_t uacct_incomplete = + !(intval & OBJSET_FLAG_USERACCOUNTING_COMPLETE); + + /* + * The local MAC protects the user, group and project accounting. + * If these objects are not present, the local MAC is zeroed out. + */ + if (uacct_incomplete || + (datalen >= OBJSET_PHYS_SIZE_V3 && + osp->os_userused_dnode.dn_type == DMU_OT_NONE && + osp->os_groupused_dnode.dn_type == DMU_OT_NONE && + osp->os_projectused_dnode.dn_type == DMU_OT_NONE) || + (datalen >= OBJSET_PHYS_SIZE_V2 && + osp->os_userused_dnode.dn_type == DMU_OT_NONE && + osp->os_groupused_dnode.dn_type == DMU_OT_NONE) || + (datalen <= OBJSET_PHYS_SIZE_V1)) { + memset(local_mac, 0, ZIO_OBJSET_MAC_LEN); + return (0); + } + + /* calculate the local MAC from the userused and groupused dnodes */ + ret = crypto_mac_init(&mech, &key->zk_hmac_key, NULL, &ctx); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + /* add in the non-portable os_flags */ + intval = osp->os_flags; + if (should_bswap) + intval = BSWAP_64(intval); + intval &= ~OBJSET_CRYPT_PORTABLE_FLAGS_MASK; + if (!ZFS_HOST_BYTEORDER) + intval = BSWAP_64(intval); + + cd.cd_length = sizeof (uint64_t); + cd.cd_raw.iov_base = (char *)&intval; + cd.cd_raw.iov_len = cd.cd_length; + + ret = crypto_mac_update(ctx, &cd); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + /* add in fields from the user accounting dnodes */ + if (osp->os_userused_dnode.dn_type != DMU_OT_NONE) { + ret = zio_crypt_do_dnode_hmac_updates(ctx, key->zk_version, + should_bswap, &osp->os_userused_dnode); + if (ret) + goto error; + } + + if (osp->os_groupused_dnode.dn_type != DMU_OT_NONE) { + ret = zio_crypt_do_dnode_hmac_updates(ctx, key->zk_version, + should_bswap, &osp->os_groupused_dnode); + if (ret) + goto error; + } + + if (osp->os_projectused_dnode.dn_type != DMU_OT_NONE && + datalen >= OBJSET_PHYS_SIZE_V3) { + ret = zio_crypt_do_dnode_hmac_updates(ctx, key->zk_version, + should_bswap, &osp->os_projectused_dnode); + if (ret) + goto error; + } + + /* store the final digest in a temporary buffer and copy what we need */ + cd.cd_length = SHA512_DIGEST_LENGTH; + cd.cd_raw.iov_base = (char *)raw_local_mac; + cd.cd_raw.iov_len = cd.cd_length; + + ret = crypto_mac_final(ctx, &cd); + if (ret != CRYPTO_SUCCESS) { + ret = SET_ERROR(EIO); + goto error; + } + + memcpy(local_mac, raw_local_mac, ZIO_OBJSET_MAC_LEN); + + return (0); + +error: + memset(portable_mac, 0, ZIO_OBJSET_MAC_LEN); + memset(local_mac, 0, ZIO_OBJSET_MAC_LEN); + return (ret); +} + +static void +zio_crypt_destroy_uio(zfs_uio_t *uio) +{ + if (uio->uio_iov) + kmem_free((void *)uio->uio_iov, + uio->uio_iovcnt * sizeof (iovec_t)); +} + +/* + * This function parses an uncompressed indirect block and returns a checksum + * of all the portable fields from all of the contained bps. The portable + * fields are the MAC and all of the fields from blk_prop except for the dedup, + * checksum, and psize bits. For an explanation of the purpose of this, see + * the comment block on object set authentication. + */ +static int +zio_crypt_do_indirect_mac_checksum_impl(boolean_t generate, void *buf, + uint_t datalen, uint64_t version, boolean_t byteswap, uint8_t *cksum) +{ + blkptr_t *bp; + int i, epb = datalen >> SPA_BLKPTRSHIFT; + SHA2_CTX ctx; + uint8_t digestbuf[SHA512_DIGEST_LENGTH]; + + /* checksum all of the MACs from the layer below */ + SHA2Init(SHA512, &ctx); + for (i = 0, bp = buf; i < epb; i++, bp++) { + zio_crypt_bp_do_indrect_checksum_updates(&ctx, version, + byteswap, bp); + } + SHA2Final(digestbuf, &ctx); + + if (generate) { + memcpy(cksum, digestbuf, ZIO_DATA_MAC_LEN); + return (0); + } + + if (memcmp(digestbuf, cksum, ZIO_DATA_MAC_LEN) != 0) + return (SET_ERROR(ECKSUM)); + + return (0); +} + +int +zio_crypt_do_indirect_mac_checksum(boolean_t generate, void *buf, + uint_t datalen, boolean_t byteswap, uint8_t *cksum) +{ + int ret; + + /* + * Unfortunately, callers of this function will not always have + * easy access to the on-disk format version. This info is + * normally found in the DSL Crypto Key, but the checksum-of-MACs + * is expected to be verifiable even when the key isn't loaded. + * Here, instead of doing a ZAP lookup for the version for each + * zio, we simply try both existing formats. + */ + ret = zio_crypt_do_indirect_mac_checksum_impl(generate, buf, + datalen, ZIO_CRYPT_KEY_CURRENT_VERSION, byteswap, cksum); + if (ret == ECKSUM) { + ASSERT(!generate); + ret = zio_crypt_do_indirect_mac_checksum_impl(generate, + buf, datalen, 0, byteswap, cksum); + } + + return (ret); +} + +int +zio_crypt_do_indirect_mac_checksum_abd(boolean_t generate, abd_t *abd, + uint_t datalen, boolean_t byteswap, uint8_t *cksum) +{ + int ret; + void *buf; + + buf = abd_borrow_buf_copy(abd, datalen); + ret = zio_crypt_do_indirect_mac_checksum(generate, buf, datalen, + byteswap, cksum); + abd_return_buf(abd, buf, datalen); + + return (ret); +} + +/* + * Special case handling routine for encrypting / decrypting ZIL blocks. + * We do not check for the older ZIL chain because the encryption feature + * was not available before the newer ZIL chain was introduced. The goal + * here is to encrypt everything except the blkptr_t of a lr_write_t and + * the zil_chain_t header. Everything that is not encrypted is authenticated. + */ +static int +zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, + uint8_t *cipherbuf, uint_t datalen, boolean_t byteswap, zfs_uio_t *puio, + zfs_uio_t *cuio, uint_t *enc_len, uint8_t **authbuf, uint_t *auth_len, + boolean_t *no_crypt) +{ + int ret; + uint64_t txtype, lr_len; + uint_t nr_src, nr_dst, crypt_len; + uint_t aad_len = 0, nr_iovecs = 0, total_len = 0; + iovec_t *src_iovecs = NULL, *dst_iovecs = NULL; + uint8_t *src, *dst, *slrp, *dlrp, *blkend, *aadp; + zil_chain_t *zilc; + lr_t *lr; + uint8_t *aadbuf = zio_buf_alloc(datalen); + + /* cipherbuf always needs an extra iovec for the MAC */ + if (encrypt) { + src = plainbuf; + dst = cipherbuf; + nr_src = 0; + nr_dst = 1; + } else { + src = cipherbuf; + dst = plainbuf; + nr_src = 1; + nr_dst = 0; + } + + /* find the start and end record of the log block */ + zilc = (zil_chain_t *)src; + slrp = src + sizeof (zil_chain_t); + aadp = aadbuf; + blkend = src + ((byteswap) ? BSWAP_64(zilc->zc_nused) : zilc->zc_nused); + + /* calculate the number of encrypted iovecs we will need */ + for (; slrp < blkend; slrp += lr_len) { + lr = (lr_t *)slrp; + + if (!byteswap) { + txtype = lr->lrc_txtype; + lr_len = lr->lrc_reclen; + } else { + txtype = BSWAP_64(lr->lrc_txtype); + lr_len = BSWAP_64(lr->lrc_reclen); + } + + nr_iovecs++; + if (txtype == TX_WRITE && lr_len != sizeof (lr_write_t)) + nr_iovecs++; + } + + nr_src += nr_iovecs; + nr_dst += nr_iovecs; + + /* allocate the iovec arrays */ + if (nr_src != 0) { + src_iovecs = kmem_alloc(nr_src * sizeof (iovec_t), KM_SLEEP); + if (src_iovecs == NULL) { + ret = SET_ERROR(ENOMEM); + goto error; + } + } + + if (nr_dst != 0) { + dst_iovecs = kmem_alloc(nr_dst * sizeof (iovec_t), KM_SLEEP); + if (dst_iovecs == NULL) { + ret = SET_ERROR(ENOMEM); + goto error; + } + } + + /* + * Copy the plain zil header over and authenticate everything except + * the checksum that will store our MAC. If we are writing the data + * the embedded checksum will not have been calculated yet, so we don't + * authenticate that. + */ + memcpy(dst, src, sizeof (zil_chain_t)); + memcpy(aadp, src, sizeof (zil_chain_t) - sizeof (zio_eck_t)); + aadp += sizeof (zil_chain_t) - sizeof (zio_eck_t); + aad_len += sizeof (zil_chain_t) - sizeof (zio_eck_t); + + /* loop over records again, filling in iovecs */ + nr_iovecs = 0; + slrp = src + sizeof (zil_chain_t); + dlrp = dst + sizeof (zil_chain_t); + + for (; slrp < blkend; slrp += lr_len, dlrp += lr_len) { + lr = (lr_t *)slrp; + + if (!byteswap) { + txtype = lr->lrc_txtype; + lr_len = lr->lrc_reclen; + } else { + txtype = BSWAP_64(lr->lrc_txtype); + lr_len = BSWAP_64(lr->lrc_reclen); + } + + /* copy the common lr_t */ + memcpy(dlrp, slrp, sizeof (lr_t)); + memcpy(aadp, slrp, sizeof (lr_t)); + aadp += sizeof (lr_t); + aad_len += sizeof (lr_t); + + ASSERT3P(src_iovecs, !=, NULL); + ASSERT3P(dst_iovecs, !=, NULL); + + /* + * If this is a TX_WRITE record we want to encrypt everything + * except the bp if exists. If the bp does exist we want to + * authenticate it. + */ + if (txtype == TX_WRITE) { + crypt_len = sizeof (lr_write_t) - + sizeof (lr_t) - sizeof (blkptr_t); + src_iovecs[nr_iovecs].iov_base = slrp + sizeof (lr_t); + src_iovecs[nr_iovecs].iov_len = crypt_len; + dst_iovecs[nr_iovecs].iov_base = dlrp + sizeof (lr_t); + dst_iovecs[nr_iovecs].iov_len = crypt_len; + + /* copy the bp now since it will not be encrypted */ + memcpy(dlrp + sizeof (lr_write_t) - sizeof (blkptr_t), + slrp + sizeof (lr_write_t) - sizeof (blkptr_t), + sizeof (blkptr_t)); + memcpy(aadp, + slrp + sizeof (lr_write_t) - sizeof (blkptr_t), + sizeof (blkptr_t)); + aadp += sizeof (blkptr_t); + aad_len += sizeof (blkptr_t); + nr_iovecs++; + total_len += crypt_len; + + if (lr_len != sizeof (lr_write_t)) { + crypt_len = lr_len - sizeof (lr_write_t); + src_iovecs[nr_iovecs].iov_base = + slrp + sizeof (lr_write_t); + src_iovecs[nr_iovecs].iov_len = crypt_len; + dst_iovecs[nr_iovecs].iov_base = + dlrp + sizeof (lr_write_t); + dst_iovecs[nr_iovecs].iov_len = crypt_len; + nr_iovecs++; + total_len += crypt_len; + } + } else { + crypt_len = lr_len - sizeof (lr_t); + src_iovecs[nr_iovecs].iov_base = slrp + sizeof (lr_t); + src_iovecs[nr_iovecs].iov_len = crypt_len; + dst_iovecs[nr_iovecs].iov_base = dlrp + sizeof (lr_t); + dst_iovecs[nr_iovecs].iov_len = crypt_len; + nr_iovecs++; + total_len += crypt_len; + } + } + + *no_crypt = (nr_iovecs == 0); + *enc_len = total_len; + *authbuf = aadbuf; + *auth_len = aad_len; + + if (encrypt) { + puio->uio_iov = src_iovecs; + puio->uio_iovcnt = nr_src; + cuio->uio_iov = dst_iovecs; + cuio->uio_iovcnt = nr_dst; + } else { + puio->uio_iov = dst_iovecs; + puio->uio_iovcnt = nr_dst; + cuio->uio_iov = src_iovecs; + cuio->uio_iovcnt = nr_src; + } + + return (0); + +error: + zio_buf_free(aadbuf, datalen); + if (src_iovecs != NULL) + kmem_free(src_iovecs, nr_src * sizeof (iovec_t)); + if (dst_iovecs != NULL) + kmem_free(dst_iovecs, nr_dst * sizeof (iovec_t)); + + *enc_len = 0; + *authbuf = NULL; + *auth_len = 0; + *no_crypt = B_FALSE; + puio->uio_iov = NULL; + puio->uio_iovcnt = 0; + cuio->uio_iov = NULL; + cuio->uio_iovcnt = 0; + return (ret); +} + +/* + * Special case handling routine for encrypting / decrypting dnode blocks. + */ +static int +zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, + uint8_t *plainbuf, uint8_t *cipherbuf, uint_t datalen, boolean_t byteswap, + zfs_uio_t *puio, zfs_uio_t *cuio, uint_t *enc_len, uint8_t **authbuf, + uint_t *auth_len, boolean_t *no_crypt) +{ + int ret; + uint_t nr_src, nr_dst, crypt_len; + uint_t aad_len = 0, nr_iovecs = 0, total_len = 0; + uint_t i, j, max_dnp = datalen >> DNODE_SHIFT; + iovec_t *src_iovecs = NULL, *dst_iovecs = NULL; + uint8_t *src, *dst, *aadp; + dnode_phys_t *dnp, *adnp, *sdnp, *ddnp; + uint8_t *aadbuf = zio_buf_alloc(datalen); + + if (encrypt) { + src = plainbuf; + dst = cipherbuf; + nr_src = 0; + nr_dst = 1; + } else { + src = cipherbuf; + dst = plainbuf; + nr_src = 1; + nr_dst = 0; + } + + sdnp = (dnode_phys_t *)src; + ddnp = (dnode_phys_t *)dst; + aadp = aadbuf; + + /* + * Count the number of iovecs we will need to do the encryption by + * counting the number of bonus buffers that need to be encrypted. + */ + for (i = 0; i < max_dnp; i += sdnp[i].dn_extra_slots + 1) { + /* + * This block may still be byteswapped. However, all of the + * values we use are either uint8_t's (for which byteswapping + * is a noop) or a * != 0 check, which will work regardless + * of whether or not we byteswap. + */ + if (sdnp[i].dn_type != DMU_OT_NONE && + DMU_OT_IS_ENCRYPTED(sdnp[i].dn_bonustype) && + sdnp[i].dn_bonuslen != 0) { + nr_iovecs++; + } + } + + nr_src += nr_iovecs; + nr_dst += nr_iovecs; + + if (nr_src != 0) { + src_iovecs = kmem_alloc(nr_src * sizeof (iovec_t), KM_SLEEP); + if (src_iovecs == NULL) { + ret = SET_ERROR(ENOMEM); + goto error; + } + } + + if (nr_dst != 0) { + dst_iovecs = kmem_alloc(nr_dst * sizeof (iovec_t), KM_SLEEP); + if (dst_iovecs == NULL) { + ret = SET_ERROR(ENOMEM); + goto error; + } + } + + nr_iovecs = 0; + + /* + * Iterate through the dnodes again, this time filling in the uios + * we allocated earlier. We also concatenate any data we want to + * authenticate onto aadbuf. + */ + for (i = 0; i < max_dnp; i += sdnp[i].dn_extra_slots + 1) { + dnp = &sdnp[i]; + + /* copy over the core fields and blkptrs (kept as plaintext) */ + memcpy(&ddnp[i], dnp, + (uint8_t *)DN_BONUS(dnp) - (uint8_t *)dnp); + + if (dnp->dn_flags & DNODE_FLAG_SPILL_BLKPTR) { + memcpy(DN_SPILL_BLKPTR(&ddnp[i]), DN_SPILL_BLKPTR(dnp), + sizeof (blkptr_t)); + } + + /* + * Handle authenticated data. We authenticate everything in + * the dnode that can be brought over when we do a raw send. + * This includes all of the core fields as well as the MACs + * stored in the bp checksums and all of the portable bits + * from blk_prop. We include the dnode padding here in case it + * ever gets used in the future. Some dn_flags and dn_used are + * not portable so we mask those out values out of the + * authenticated data. + */ + crypt_len = offsetof(dnode_phys_t, dn_blkptr); + memcpy(aadp, dnp, crypt_len); + adnp = (dnode_phys_t *)aadp; + adnp->dn_flags &= DNODE_CRYPT_PORTABLE_FLAGS_MASK; + adnp->dn_used = 0; + aadp += crypt_len; + aad_len += crypt_len; + + for (j = 0; j < dnp->dn_nblkptr; j++) { + zio_crypt_bp_do_aad_updates(&aadp, &aad_len, + version, byteswap, &dnp->dn_blkptr[j]); + } + + if (dnp->dn_flags & DNODE_FLAG_SPILL_BLKPTR) { + zio_crypt_bp_do_aad_updates(&aadp, &aad_len, + version, byteswap, DN_SPILL_BLKPTR(dnp)); + } + + /* + * If this bonus buffer needs to be encrypted, we prepare an + * iovec_t. The encryption / decryption functions will fill + * this in for us with the encrypted or decrypted data. + * Otherwise we add the bonus buffer to the authenticated + * data buffer and copy it over to the destination. The + * encrypted iovec extends to DN_MAX_BONUS_LEN(dnp) so that + * we can guarantee alignment with the AES block size + * (128 bits). + */ + crypt_len = DN_MAX_BONUS_LEN(dnp); + if (dnp->dn_type != DMU_OT_NONE && + DMU_OT_IS_ENCRYPTED(dnp->dn_bonustype) && + dnp->dn_bonuslen != 0) { + ASSERT3U(nr_iovecs, <, nr_src); + ASSERT3U(nr_iovecs, <, nr_dst); + ASSERT3P(src_iovecs, !=, NULL); + ASSERT3P(dst_iovecs, !=, NULL); + src_iovecs[nr_iovecs].iov_base = DN_BONUS(dnp); + src_iovecs[nr_iovecs].iov_len = crypt_len; + dst_iovecs[nr_iovecs].iov_base = DN_BONUS(&ddnp[i]); + dst_iovecs[nr_iovecs].iov_len = crypt_len; + + nr_iovecs++; + total_len += crypt_len; + } else { + memcpy(DN_BONUS(&ddnp[i]), DN_BONUS(dnp), crypt_len); + memcpy(aadp, DN_BONUS(dnp), crypt_len); + aadp += crypt_len; + aad_len += crypt_len; + } + } + + *no_crypt = (nr_iovecs == 0); + *enc_len = total_len; + *authbuf = aadbuf; + *auth_len = aad_len; + + if (encrypt) { + puio->uio_iov = src_iovecs; + puio->uio_iovcnt = nr_src; + cuio->uio_iov = dst_iovecs; + cuio->uio_iovcnt = nr_dst; + } else { + puio->uio_iov = dst_iovecs; + puio->uio_iovcnt = nr_dst; + cuio->uio_iov = src_iovecs; + cuio->uio_iovcnt = nr_src; + } + + return (0); + +error: + zio_buf_free(aadbuf, datalen); + if (src_iovecs != NULL) + kmem_free(src_iovecs, nr_src * sizeof (iovec_t)); + if (dst_iovecs != NULL) + kmem_free(dst_iovecs, nr_dst * sizeof (iovec_t)); + + *enc_len = 0; + *authbuf = NULL; + *auth_len = 0; + *no_crypt = B_FALSE; + puio->uio_iov = NULL; + puio->uio_iovcnt = 0; + cuio->uio_iov = NULL; + cuio->uio_iovcnt = 0; + return (ret); +} + +static int +zio_crypt_init_uios_normal(boolean_t encrypt, uint8_t *plainbuf, + uint8_t *cipherbuf, uint_t datalen, zfs_uio_t *puio, zfs_uio_t *cuio, + uint_t *enc_len) +{ + int ret; + uint_t nr_plain = 1, nr_cipher = 2; + iovec_t *plain_iovecs = NULL, *cipher_iovecs = NULL; + + /* allocate the iovecs for the plain and cipher data */ + plain_iovecs = kmem_alloc(nr_plain * sizeof (iovec_t), + KM_SLEEP); + if (!plain_iovecs) { + ret = SET_ERROR(ENOMEM); + goto error; + } + + cipher_iovecs = kmem_alloc(nr_cipher * sizeof (iovec_t), + KM_SLEEP); + if (!cipher_iovecs) { + ret = SET_ERROR(ENOMEM); + goto error; + } + + plain_iovecs[0].iov_base = plainbuf; + plain_iovecs[0].iov_len = datalen; + cipher_iovecs[0].iov_base = cipherbuf; + cipher_iovecs[0].iov_len = datalen; + + *enc_len = datalen; + puio->uio_iov = plain_iovecs; + puio->uio_iovcnt = nr_plain; + cuio->uio_iov = cipher_iovecs; + cuio->uio_iovcnt = nr_cipher; + + return (0); + +error: + if (plain_iovecs != NULL) + kmem_free(plain_iovecs, nr_plain * sizeof (iovec_t)); + if (cipher_iovecs != NULL) + kmem_free(cipher_iovecs, nr_cipher * sizeof (iovec_t)); + + *enc_len = 0; + puio->uio_iov = NULL; + puio->uio_iovcnt = 0; + cuio->uio_iov = NULL; + cuio->uio_iovcnt = 0; + return (ret); +} + +/* + * This function builds up the plaintext (puio) and ciphertext (cuio) uios so + * that they can be used for encryption and decryption by zio_do_crypt_uio(). + * Most blocks will use zio_crypt_init_uios_normal(), with ZIL and dnode blocks + * requiring special handling to parse out pieces that are to be encrypted. The + * authbuf is used by these special cases to store additional authenticated + * data (AAD) for the encryption modes. + */ +static int +zio_crypt_init_uios(boolean_t encrypt, uint64_t version, dmu_object_type_t ot, + uint8_t *plainbuf, uint8_t *cipherbuf, uint_t datalen, boolean_t byteswap, + uint8_t *mac, zfs_uio_t *puio, zfs_uio_t *cuio, uint_t *enc_len, + uint8_t **authbuf, uint_t *auth_len, boolean_t *no_crypt) +{ + int ret; + iovec_t *mac_iov; + + ASSERT(DMU_OT_IS_ENCRYPTED(ot) || ot == DMU_OT_NONE); + + /* route to handler */ + switch (ot) { + case DMU_OT_INTENT_LOG: + ret = zio_crypt_init_uios_zil(encrypt, plainbuf, cipherbuf, + datalen, byteswap, puio, cuio, enc_len, authbuf, auth_len, + no_crypt); + break; + case DMU_OT_DNODE: + ret = zio_crypt_init_uios_dnode(encrypt, version, plainbuf, + cipherbuf, datalen, byteswap, puio, cuio, enc_len, authbuf, + auth_len, no_crypt); + break; + default: + ret = zio_crypt_init_uios_normal(encrypt, plainbuf, cipherbuf, + datalen, puio, cuio, enc_len); + *authbuf = NULL; + *auth_len = 0; + *no_crypt = B_FALSE; + break; + } + + if (ret != 0) + goto error; + + /* populate the uios */ + puio->uio_segflg = UIO_SYSSPACE; + cuio->uio_segflg = UIO_SYSSPACE; + + mac_iov = ((iovec_t *)&cuio->uio_iov[cuio->uio_iovcnt - 1]); + mac_iov->iov_base = mac; + mac_iov->iov_len = ZIO_DATA_MAC_LEN; + + return (0); + +error: + return (ret); +} + +/* + * Primary encryption / decryption entrypoint for zio data. + */ +int +zio_do_crypt_data(boolean_t encrypt, zio_crypt_key_t *key, + dmu_object_type_t ot, boolean_t byteswap, uint8_t *salt, uint8_t *iv, + uint8_t *mac, uint_t datalen, uint8_t *plainbuf, uint8_t *cipherbuf, + boolean_t *no_crypt) +{ + int ret; + boolean_t locked = B_FALSE; + uint64_t crypt = key->zk_crypt; + uint_t keydata_len = zio_crypt_table[crypt].ci_keylen; + uint_t enc_len, auth_len; + zfs_uio_t puio, cuio; + uint8_t enc_keydata[MASTER_KEY_MAX_LEN]; + crypto_key_t tmp_ckey, *ckey = NULL; + crypto_ctx_template_t tmpl; + uint8_t *authbuf = NULL; + + memset(&puio, 0, sizeof (puio)); + memset(&cuio, 0, sizeof (cuio)); + + /* + * If the needed key is the current one, just use it. Otherwise we + * need to generate a temporary one from the given salt + master key. + * If we are encrypting, we must return a copy of the current salt + * so that it can be stored in the blkptr_t. + */ + rw_enter(&key->zk_salt_lock, RW_READER); + locked = B_TRUE; + + if (memcmp(salt, key->zk_salt, ZIO_DATA_SALT_LEN) == 0) { + ckey = &key->zk_current_key; + tmpl = key->zk_current_tmpl; + } else { + rw_exit(&key->zk_salt_lock); + locked = B_FALSE; + + ret = hkdf_sha512(key->zk_master_keydata, keydata_len, NULL, 0, + salt, ZIO_DATA_SALT_LEN, enc_keydata, keydata_len); + if (ret != 0) + goto error; + + tmp_ckey.ck_data = enc_keydata; + tmp_ckey.ck_length = CRYPTO_BYTES2BITS(keydata_len); + + ckey = &tmp_ckey; + tmpl = NULL; + } + + /* + * Attempt to use QAT acceleration if we can. We currently don't + * do this for metadnode and ZIL blocks, since they have a much + * more involved buffer layout and the qat_crypt() function only + * works in-place. + */ + if (qat_crypt_use_accel(datalen) && + ot != DMU_OT_INTENT_LOG && ot != DMU_OT_DNODE) { + uint8_t *srcbuf, *dstbuf; + + if (encrypt) { + srcbuf = plainbuf; + dstbuf = cipherbuf; + } else { + srcbuf = cipherbuf; + dstbuf = plainbuf; + } + + ret = qat_crypt((encrypt) ? QAT_ENCRYPT : QAT_DECRYPT, srcbuf, + dstbuf, NULL, 0, iv, mac, ckey, key->zk_crypt, datalen); + if (ret == CPA_STATUS_SUCCESS) { + if (locked) { + rw_exit(&key->zk_salt_lock); + locked = B_FALSE; + } + + return (0); + } + /* If the hardware implementation fails fall back to software */ + } + + /* create uios for encryption */ + ret = zio_crypt_init_uios(encrypt, key->zk_version, ot, plainbuf, + cipherbuf, datalen, byteswap, mac, &puio, &cuio, &enc_len, + &authbuf, &auth_len, no_crypt); + if (ret != 0) + goto error; + + /* perform the encryption / decryption in software */ + ret = zio_do_crypt_uio(encrypt, key->zk_crypt, ckey, tmpl, iv, enc_len, + &puio, &cuio, authbuf, auth_len); + if (ret != 0) + goto error; + + if (locked) { + rw_exit(&key->zk_salt_lock); + locked = B_FALSE; + } + + if (authbuf != NULL) + zio_buf_free(authbuf, datalen); + if (ckey == &tmp_ckey) + memset(enc_keydata, 0, keydata_len); + zio_crypt_destroy_uio(&puio); + zio_crypt_destroy_uio(&cuio); + + return (0); + +error: + if (locked) + rw_exit(&key->zk_salt_lock); + if (authbuf != NULL) + zio_buf_free(authbuf, datalen); + if (ckey == &tmp_ckey) + memset(enc_keydata, 0, keydata_len); + zio_crypt_destroy_uio(&puio); + zio_crypt_destroy_uio(&cuio); + + return (ret); +} + +/* + * Simple wrapper around zio_do_crypt_data() to work with abd's instead of + * linear buffers. + */ +int +zio_do_crypt_abd(boolean_t encrypt, zio_crypt_key_t *key, dmu_object_type_t ot, + boolean_t byteswap, uint8_t *salt, uint8_t *iv, uint8_t *mac, + uint_t datalen, abd_t *pabd, abd_t *cabd, boolean_t *no_crypt) +{ + int ret; + void *ptmp, *ctmp; + + if (encrypt) { + ptmp = abd_borrow_buf_copy(pabd, datalen); + ctmp = abd_borrow_buf(cabd, datalen); + } else { + ptmp = abd_borrow_buf(pabd, datalen); + ctmp = abd_borrow_buf_copy(cabd, datalen); + } + + ret = zio_do_crypt_data(encrypt, key, ot, byteswap, salt, iv, mac, + datalen, ptmp, ctmp, no_crypt); + if (ret != 0) + goto error; + + if (encrypt) { + abd_return_buf(pabd, ptmp, datalen); + abd_return_buf_copy(cabd, ctmp, datalen); + } else { + abd_return_buf_copy(pabd, ptmp, datalen); + abd_return_buf(cabd, ctmp, datalen); + } + + return (0); + +error: + if (encrypt) { + abd_return_buf(pabd, ptmp, datalen); + abd_return_buf_copy(cabd, ctmp, datalen); + } else { + abd_return_buf_copy(pabd, ptmp, datalen); + abd_return_buf(cabd, ctmp, datalen); + } + + return (ret); +} + +#if defined(_KERNEL) +/* BEGIN CSTYLED */ +module_param(zfs_key_max_salt_uses, ulong, 0644); +MODULE_PARM_DESC(zfs_key_max_salt_uses, "Max number of times a salt value " + "can be used for generating encryption keys before it is rotated"); +/* END CSTYLED */ +#endif diff --git a/module/os/windows/zfs/zvol_os.c b/module/os/windows/zfs/zvol_os.c new file mode 100644 index 000000000000..a7c80b81a0d2 --- /dev/null +++ b/module/os/windows/zfs/zvol_os.c @@ -0,0 +1,1085 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2020 by Jorgen Lundman. All rights reserved. + * Portions Copyright 2022 Andrew Innes + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static uint32_t zvol_major = ZVOL_MAJOR; + +unsigned int zvol_request_sync = 0; +unsigned int zvol_prefetch_bytes = (128 * 1024); +unsigned long zvol_max_discard_blocks = 16384; +unsigned int zvol_threads = 32; + +taskq_t *zvol_taskq; + +extern void wzvol_clear_targetid(uint8_t targetid, uint8_t lun, + zvol_state_t *zv); +extern void wzvol_announce_buschange(void); +extern int wzvol_assign_targetid(zvol_state_t *zv); +extern list_t zvol_state_list; + +_Atomic uint64_t spl_lowest_zvol_stack_remaining = 0; + +typedef struct zv_request { + zvol_state_t *zv; + + void (*zv_func)(void *); + void *zv_arg; + + taskq_ent_t ent; +} zv_request_t; + +#define ZVOL_LOCK_HELD (1<<0) +#define ZVOL_LOCK_SPA (1<<1) +#define ZVOL_LOCK_SUSPEND (1<<2) + +static void +zvol_os_spawn_cb(void *param) +{ + zv_request_t *zvr = (zv_request_t *)param; + + zvr->zv_func(zvr->zv_arg); + + kmem_free(zvr, sizeof (zv_request_t)); +} + +static void +zvol_os_spawn(void (*func)(void *), void *arg) +{ + zv_request_t *zvr; + zvr = kmem_alloc(sizeof (zv_request_t), KM_SLEEP); + zvr->zv_arg = arg; + zvr->zv_func = func; + + taskq_init_ent(&zvr->ent); + + taskq_dispatch_ent(zvol_taskq, + zvol_os_spawn_cb, zvr, 0, &zvr->ent); +} + +/* + * Given a path, return TRUE if path is a ZVOL. + * Implement me when it is time for zpools-is-zvols. + * Windows. + * Returning FALSE makes caller process everything async, + * which will deadlock if zpool-in-zvol exists. + * Returning TRUE goes into slow, but safe, path. + */ +static boolean_t +zvol_os_is_zvol(const char *device) +{ + return (B_FALSE); +} + +/* + * Make sure zv is still in the list (not freed) and if it is + * grab the locks in the correct order. + * Can we rely on list_link_active() instead of looping list? + * Return value: + * 0 : not found. No locks. + * ZVOL_LOCK_HELD : found and zv->zv_state_lock held + * |ZVOL_LOCK_SPA : spa_namespace_lock held + * |ZVOL_LOCK_SUSPEND : zv->zv_state_lock held + * call zvol_os_verify_lock_exit() to release + */ +static int +zvol_os_verify_and_lock(zvol_state_t *node, boolean_t takesuspend) +{ + zvol_state_t *zv; + int ret = ZVOL_LOCK_HELD; + +retry: + rw_enter(&zvol_state_lock, RW_READER); + for (zv = list_head(&zvol_state_list); zv != NULL; + zv = list_next(&zvol_state_list, zv)) { + + /* Until we find the node ... */ + if (zv != node) + continue; + + /* If this is to be first open, deal with spa_namespace */ + if (zv->zv_open_count == 0 && + !mutex_owned(&spa_namespace_lock)) { + /* + * We need to guarantee that the namespace lock is held + * to avoid spurious failures in zvol_first_open. + */ + ret |= ZVOL_LOCK_SPA; + if (!mutex_tryenter(&spa_namespace_lock)) { + rw_exit(&zvol_state_lock); + mutex_enter(&spa_namespace_lock); + /* Sadly, this will restart for loop */ + goto retry; + } + } + + mutex_enter(&zv->zv_state_lock); + + /* + * make sure zvol is not suspended during first open + * (hold zv_suspend_lock) and respect proper lock acquisition + * ordering - zv_suspend_lock before zv_state_lock + */ + if (zv->zv_open_count == 0 || takesuspend) { + ret |= ZVOL_LOCK_SUSPEND; + if (!rw_tryenter(&zv->zv_suspend_lock, RW_READER)) { + mutex_exit(&zv->zv_state_lock); + + /* If we hold spa_namespace, we can deadlock */ + if (ret & ZVOL_LOCK_SPA) { + rw_exit(&zvol_state_lock); + mutex_exit(&spa_namespace_lock); + ret &= ~ZVOL_LOCK_SPA; + dprintf("%s: spa_namespace loop\n", + __func__); + /* Let's not busy loop */ + delay(hz>>2); + goto retry; + } + rw_enter(&zv->zv_suspend_lock, RW_READER); + mutex_enter(&zv->zv_state_lock); + /* check to see if zv_suspend_lock is needed */ + if (zv->zv_open_count != 0) { + rw_exit(&zv->zv_suspend_lock); + ret &= ~ZVOL_LOCK_SUSPEND; + } + } + } + rw_exit(&zvol_state_lock); + + /* Success */ + return (ret); + + } /* for */ + + /* Not found */ + rw_exit(&zvol_state_lock); + + /* It's possible we grabbed spa, but then didn't re-find zv */ + if (ret & ZVOL_LOCK_SPA) + mutex_exit(&spa_namespace_lock); + return (0); +} + +static void +zvol_os_verify_lock_exit(zvol_state_t *zv, int locks) +{ + if (locks & ZVOL_LOCK_SPA) + mutex_exit(&spa_namespace_lock); + mutex_exit(&zv->zv_state_lock); + if (locks & ZVOL_LOCK_SUSPEND) + rw_exit(&zv->zv_suspend_lock); +} + +static void +zvol_os_register_device_cb(void *param) +{ + zvol_state_t *zv = (zvol_state_t *)param; + int locks; + + if ((locks = zvol_os_verify_and_lock(zv, zv->zv_open_count == 0)) == 0) + return; + + zvol_os_verify_lock_exit(zv, locks); +} + +int +zvol_os_write(dev_t dev, zfs_uio_t *uio, int p) +{ + return (ENOTSUP); +} + +int +zvol_os_read(dev_t dev, zfs_uio_t *uio, int p) +{ + return (ENOTSUP); +} + +int +zvol_os_read_zv(zvol_state_t *zv, zfs_uio_t *uio, int flags) +{ + zfs_locked_range_t *lr; + int error = 0; + uint64_t offset = 0; + + const ULONG_PTR r = IoGetRemainingStackSize(); + + if (spl_lowest_zvol_stack_remaining == 0) { + spl_lowest_zvol_stack_remaining = r; + } else if (spl_lowest_zvol_stack_remaining > r) { + spl_lowest_zvol_stack_remaining = r; + } + + if (zv == NULL || zv->zv_dn == NULL) + return (ENXIO); + + uint64_t volsize = zv->zv_volsize; + if (zfs_uio_offset(uio) >= volsize) + return (EIO); + + rw_enter(&zv->zv_suspend_lock, RW_READER); + + lr = zfs_rangelock_enter(&zv->zv_rangelock, + zfs_uio_offset(uio), zfs_uio_resid(uio), RL_READER); + + while (zfs_uio_resid(uio) > 0 && zfs_uio_offset(uio) < volsize) { + uint64_t bytes = MIN(zfs_uio_resid(uio), DMU_MAX_ACCESS >> 1); + + /* don't read past the end */ + if (bytes > volsize - zfs_uio_offset(uio)) + bytes = volsize - zfs_uio_offset(uio); + + TraceEvent(TRACE_VERBOSE, "%s:%d: %s %llu len %llu bytes" + " %llu\n", __func__, __LINE__, "zvol_read_iokit: position", + zfs_uio_offset(uio), zfs_uio_resid(uio), bytes); + + error = dmu_read_uio_dnode(zv->zv_dn, uio, bytes); + + if (error) { + /* convert checksum errors into IO errors */ + if (error == ECKSUM) + error = EIO; + break; + } + } + zfs_rangelock_exit(lr); + + dataset_kstats_update_read_kstats(&zv->zv_kstat, offset); + + rw_exit(&zv->zv_suspend_lock); + return (error); +} + + +int +zvol_os_write_zv(zvol_state_t *zv, zfs_uio_t *uio, int flags) +{ + uint64_t volsize; + zfs_locked_range_t *lr; + int error = 0; + boolean_t sync; + uint64_t offset = 0; + uint64_t bytes = 0; + uint64_t off; + + const ULONG_PTR r = IoGetRemainingStackSize(); + + if (spl_lowest_zvol_stack_remaining == 0) { + spl_lowest_zvol_stack_remaining = r; + } else if (spl_lowest_zvol_stack_remaining > r) { + spl_lowest_zvol_stack_remaining = r; + } + + + if (zv == NULL) + return (ENXIO); + + /* Some requests are just for flush and nothing else. */ + if (zfs_uio_resid(uio) == 0) + return (0); + + volsize = zv->zv_volsize; + if (zfs_uio_offset(uio) >= volsize) + return (EIO); + + rw_enter(&zv->zv_suspend_lock, RW_READER); + + /* + * Open a ZIL if this is the first time we have written to this + * zvol. We protect zv->zv_zilog with zv_suspend_lock rather + * than zv_state_lock so that we don't need to acquire an + * additional lock in this path. + */ + if (zv->zv_zilog == NULL) { + rw_exit(&zv->zv_suspend_lock); + rw_enter(&zv->zv_suspend_lock, RW_WRITER); + if (zv->zv_zilog == NULL) { + zv->zv_zilog = zil_open(zv->zv_objset, + zvol_get_data, NULL); + zv->zv_flags |= ZVOL_WRITTEN_TO; + } + rw_downgrade(&zv->zv_suspend_lock); + } + + TraceEvent(TRACE_VERBOSE, "%s:%d: zvol_write_iokit(offset " + "0x%llx bytes 0x%llx)\n", __func__, __LINE__, + zfs_uio_offset(uio), zfs_uio_resid(uio), bytes); + + sync = (zv->zv_objset->os_sync == ZFS_SYNC_ALWAYS); + + /* Lock the entire range */ + lr = zfs_rangelock_enter(&zv->zv_rangelock, zfs_uio_offset(uio), + zfs_uio_resid(uio), RL_WRITER); + + /* Iterate over (DMU_MAX_ACCESS/2) segments */ + while (zfs_uio_resid(uio) > 0 && zfs_uio_offset(uio) < volsize) { + uint64_t bytes = MIN(zfs_uio_resid(uio), DMU_MAX_ACCESS >> 1); + uint64_t off = zfs_uio_offset(uio); + dmu_tx_t *tx = dmu_tx_create(zv->zv_objset); + + /* don't write past the end */ + if (bytes > volsize - off) + bytes = volsize - off; + + dmu_tx_hold_write_by_dnode(tx, zv->zv_dn, off, bytes); + error = dmu_tx_assign(tx, TXG_WAIT); + if (error) { + dmu_tx_abort(tx); + break; + } + + error = dmu_write_uio_dnode(zv->zv_dn, uio, + bytes, tx); + + if (error == 0) { + zvol_log_write(zv, tx, offset, bytes, sync); + } + dmu_tx_commit(tx); + + if (error) + break; + } + zfs_rangelock_exit(lr); + + dataset_kstats_update_write_kstats(&zv->zv_kstat, offset); + + if (sync) + zil_commit(zv->zv_zilog, ZVOL_OBJ); + + rw_exit(&zv->zv_suspend_lock); + + return (error); +} + +int +zvol_os_unmap(zvol_state_t *zv, uint64_t off, uint64_t bytes) +{ + zfs_locked_range_t *lr = NULL; + dmu_tx_t *tx = NULL; + int error = 0; + uint64_t end = off + bytes; + + if (zv == NULL) + return (ENXIO); + + /* + * XNU's wipefs_wipe() will issue one giant unmap for the entire + * device; + * zfs create -V 8g BOOM/vol + * zvolIO doDiscard calling zvol_unmap with offset, bytes (0, 858992) + * Which will both take too long, and is uneccessary. We will ignore + * any unmaps deemed "too large". + * This is APPLE, check if Windows does the same. + */ + if ((off == 0ULL) && + (zv->zv_volsize > (1ULL << 24)) && /* 16Mb slop */ + (bytes >= (zv->zv_volsize - (1ULL << 24)))) + return (0); + + rw_enter(&zv->zv_suspend_lock, RW_READER); + + /* + * Open a ZIL if this is the first time we have written to this + * zvol. We protect zv->zv_zilog with zv_suspend_lock rather + * than zv_state_lock so that we don't need to acquire an + * additional lock in this path. + */ + if (zv->zv_zilog == NULL) { + rw_exit(&zv->zv_suspend_lock); + rw_enter(&zv->zv_suspend_lock, RW_WRITER); + if (zv->zv_zilog == NULL) { + zv->zv_zilog = zil_open(zv->zv_objset, + zvol_get_data, NULL); + zv->zv_flags |= ZVOL_WRITTEN_TO; + } + rw_downgrade(&zv->zv_suspend_lock); + } + + off = P2ROUNDUP(off, zv->zv_volblocksize); + end = P2ALIGN(end, zv->zv_volblocksize); + + if (end > zv->zv_volsize) /* don't write past the end */ + end = zv->zv_volsize; + + if (off >= end) { + /* Return success- caller does not need to know */ + goto out; + } + + bytes = end - off; + lr = zfs_rangelock_enter(&zv->zv_rangelock, off, bytes, RL_WRITER); + + tx = dmu_tx_create(zv->zv_objset); + + dmu_tx_mark_netfree(tx); + + error = dmu_tx_assign(tx, TXG_WAIT); + + if (error) { + dmu_tx_abort(tx); + } else { + + zvol_log_truncate(zv, tx, off, bytes, B_TRUE); + + dmu_tx_commit(tx); + + error = dmu_free_long_range(zv->zv_objset, + ZVOL_OBJ, off, bytes); + } + + zfs_rangelock_exit(lr); + + if (error == 0) { + /* + * If the 'sync' property is set to 'always' then + * treat this as a synchronous operation + * (i.e. commit to zil). + */ + if (zv->zv_objset->os_sync == ZFS_SYNC_ALWAYS) { + zil_commit(zv->zv_zilog, ZVOL_OBJ); + } + } + +out: + rw_exit(&zv->zv_suspend_lock); + return (error); +} + +int +zvol_os_update_volsize(zvol_state_t *zv, uint64_t volsize) +{ + zv->zv_volsize = volsize; + return (0); +} + +static void +zvol_os_clear_private(zvol_state_t *zv) +{ +#if 0 + // Close the Storport half open + if (zv->zv_open_count == 0) { + wzvol_clear_targetid(zv->zv_zso->zso_target_id, + zv->zv_zso->zso_lun_id, zv); + wzvol_announce_buschange(); + } +#endif +} + +/* + * Find a zvol_state_t given the full major+minor dev_t. If found, + * return with zv_state_lock taken, otherwise, return (NULL) without + * taking zv_state_lock. + */ +static zvol_state_t * +zvol_os_find_by_dev(dev_t dev) +{ + zvol_state_t *zv; + + dprintf("%s\n", __func__); + + rw_enter(&zvol_state_lock, RW_READER); + for (zv = list_head(&zvol_state_list); zv != NULL; + zv = list_next(&zvol_state_list, zv)) { + mutex_enter(&zv->zv_state_lock); + if (zv->zv_zso->zso_dev == dev) { + rw_exit(&zvol_state_lock); + return (zv); + } + mutex_exit(&zv->zv_state_lock); + } + rw_exit(&zvol_state_lock); + + return (NULL); +} + +zvol_state_t * +zvol_os_targetlun_lookup(uint8_t target, uint8_t lun) +{ + zvol_state_t *zv; + + dprintf("%s\n", __func__); + + rw_enter(&zvol_state_lock, RW_READER); + for (zv = list_head(&zvol_state_list); zv != NULL; + zv = list_next(&zvol_state_list, zv)) { + mutex_enter(&zv->zv_state_lock); + if (zv->zv_zso->zso_target_id == target && + zv->zv_zso->zso_lun_id == lun) { + rw_exit(&zvol_state_lock); + return (zv); + } + mutex_exit(&zv->zv_state_lock); + } + rw_exit(&zvol_state_lock); + return (NULL); +} + + +void +zvol_os_validate_dev(zvol_state_t *zv) +{ + ASSERT3U(minor(zv->zv_zso->zso_dev) & ZVOL_MINOR_MASK, ==, 0); +} + +/* + * Allocate memory for a new zvol_state_t and setup the required + * request queue and generic disk structures for the block device. + */ +static zvol_state_t * +zvol_os_alloc(dev_t dev, const char *name) +{ + zvol_state_t *zv; + struct zvol_state_os *zso; + uint64_t volmode; + + if (dsl_prop_get_integer(name, "volmode", &volmode, NULL) != 0) + return (NULL); + + if (volmode == ZFS_VOLMODE_DEFAULT) + volmode = zvol_volmode; + + if (volmode == ZFS_VOLMODE_NONE) + return (NULL); + + zv = kmem_zalloc(sizeof (zvol_state_t), KM_SLEEP); + zso = kmem_zalloc(sizeof (struct zvol_state_os), KM_SLEEP); + zv->zv_zso = zso; + + list_link_init(&zv->zv_next); + mutex_init(&zv->zv_state_lock, NULL, MUTEX_DEFAULT, NULL); + + zv->zv_open_count = 0; + strlcpy(zv->zv_name, name, MAXNAMELEN); + + zfs_rangelock_init(&zv->zv_rangelock, NULL, NULL); + rw_init(&zv->zv_suspend_lock, NULL, RW_DEFAULT, NULL); + + return (zv); +#if 0 +out_kmem: + kmem_free(zso, sizeof (struct zvol_state_os)); + kmem_free(zv, sizeof (zvol_state_t)); + return (NULL); +#endif +} + +/* + * Cleanup then free a zvol_state_t which was created by zvol_alloc(). + * At this time, the structure is not opened by anyone, is taken off + * the zvol_state_list, and has its private data set to NULL. + * The zvol_state_lock is dropped. + * + */ +static void +zvol_os_free(zvol_state_t *zv) +{ + dprintf("%s\n", __func__); +#if 0 + rw_enter(&zv->zv_suspend_lock, RW_READER); + mutex_enter(&zv->zv_state_lock); + zv->zv_zso->zso_open_count--; + zv->zv_open_count++; + zvol_last_close(zv); + zv->zv_open_count--; + mutex_exit(&zv->zv_state_lock); + rw_exit(&zv->zv_suspend_lock); +#endif + + ASSERT(!RW_LOCK_HELD(&zv->zv_suspend_lock)); + ASSERT(!MUTEX_HELD(&zv->zv_state_lock)); + ASSERT(zv->zv_open_count == 0); + + rw_destroy(&zv->zv_suspend_lock); + zfs_rangelock_fini(&zv->zv_rangelock); + + mutex_destroy(&zv->zv_state_lock); + dataset_kstats_destroy(&zv->zv_kstat); + + kmem_free(zv->zv_zso, sizeof (struct zvol_state_os)); + kmem_free(zv, sizeof (zvol_state_t)); +} + +void +zvol_os_attach(char *name) +{ + zvol_state_t *zv; + uint64_t hash = zvol_name_hash(name); + int error = 0; + + dprintf("%s\n", __func__); + + zv = zvol_find_by_name_hash(name, hash, RW_NONE); + if (zv != NULL) { + mutex_exit(&zv->zv_state_lock); + error = zvol_os_open_zv(zv, zv->zv_flags & ZVOL_RDONLY ? + FREAD : FWRITE, 0, NULL); // readonly? + // Assign new TargetId and Lun + if (error == 0) { + wzvol_assign_targetid(zv); + wzvol_announce_buschange(); + } + } +} + +void +zvol_os_detach_zv(zvol_state_t *zv) +{ + if (zv != NULL) { + wzvol_clear_targetid(zv->zv_zso->zso_target_id, + zv->zv_zso->zso_lun_id, zv); + /* Last close needs suspect lock, give it a try */ + if (zv->zv_open_count == 1) { + if (rw_tryenter(&zv->zv_suspend_lock, RW_READER)) { + zvol_last_close(zv); + zv->zv_open_count--; + rw_exit(&zv->zv_suspend_lock); + } + } + wzvol_announce_buschange(); + } +} + +void +zvol_os_detach(char *name) +{ + zvol_state_t *zv; + uint64_t hash = zvol_name_hash(name); + + dprintf("%s\n", __func__); + + zv = zvol_find_by_name_hash(name, hash, RW_NONE); + if (zv != NULL) { + zvol_os_detach_zv(zv); + mutex_exit(&zv->zv_state_lock); + wzvol_announce_buschange(); + } +} + +void +zvol_wait_close(zvol_state_t *zv) +{ +} + +/* + * Create a block device minor node and setup the linkage between it + * and the specified volume. Once this function returns the block + * device is live and ready for use. + */ +static int +zvol_os_create_minor(const char *name) +{ + zvol_state_t *zv; + objset_t *os; + dmu_object_info_t *doi; + uint64_t volsize; + unsigned minor = 0; + int error = 0; + uint64_t hash = zvol_name_hash(name); + boolean_t replayed_zil = B_FALSE; + + dprintf("%s\n", __func__); + + if (zvol_inhibit_dev) + return (0); + + // minor? + zv = zvol_find_by_name_hash(name, hash, RW_NONE); + if (zv) { + ASSERT(MUTEX_HELD(&zv->zv_state_lock)); + mutex_exit(&zv->zv_state_lock); + return (SET_ERROR(EEXIST)); + } + + doi = kmem_alloc(sizeof (dmu_object_info_t), KM_SLEEP); + + error = dmu_objset_own(name, DMU_OST_ZVOL, B_TRUE, B_TRUE, FTAG, &os); + if (error) + goto out_doi; + + error = dmu_object_info(os, ZVOL_OBJ, doi); + if (error) + goto out_dmu_objset_disown; + + error = zap_lookup(os, ZVOL_ZAP_OBJ, "size", 8, 1, &volsize); + if (error) + goto out_dmu_objset_disown; + + zv = zvol_os_alloc(makedevice(zvol_major, minor), name); + if (zv == NULL) { + error = SET_ERROR(EAGAIN); + goto out_dmu_objset_disown; + } + zv->zv_hash = hash; + + if (dmu_objset_is_snapshot(os)) + zv->zv_flags |= ZVOL_RDONLY; + + zv->zv_volblocksize = doi->doi_data_block_size; + zv->zv_volsize = volsize; + zv->zv_objset = os; + + // set_capacity(zv->zv_zso->zvo_disk, zv->zv_volsize >> 9); + ASSERT3P(zv->zv_zilog, ==, NULL); + zv->zv_zilog = zil_open(os, zvol_get_data, NULL); + if (spa_writeable(dmu_objset_spa(os))) { + if (zil_replay_disable) + replayed_zil = zil_destroy(zv->zv_zilog, B_FALSE); + else + replayed_zil = zil_replay(os, zv, zvol_replay_vector); + } + if (replayed_zil) + zil_close(zv->zv_zilog); + zv->zv_zilog = NULL; + + dataset_kstats_create(&zv->zv_kstat, zv->zv_objset); + + rw_enter(&zvol_state_lock, RW_WRITER); + zvol_insert(zv); + rw_exit(&zvol_state_lock); + + /* + * Here is where we differ to upstream. They will call open and + * close, as userland opens the /dev/disk node. Once opened, it + * has the open_count, and long_holds held, which is used in + * read/write. Once closed, everything is released. So when it + * comes to export/unmount/destroy of the ZVOL, it checks for + * opencount==0 and longholds==0. They should allow for == 1 + * for Windows. + * However, in Windows there is no open/close devnode, but rather, + * we assign the zvol to the storport API, to expose the device. + * Really, the zvol needs to be "open" the entire time that storport + * has it. If we leave zvol "open" it will fail the checks for "==0". + * So we steal an opencount, and remember it privately. We could also + * change zvol.c to special-case it for Windows. + */ +#if 0 + if (error == 0) { + error = dnode_hold(os, ZVOL_OBJ, FTAG, &zv->zv_zso->zso_dn); + if (error == 0) { + zfs_rangelock_init(&zv->zv_zso->zso_rangelock, NULL, + NULL); + wzvol_announce_buschange(); + } + // zvol_os_close_zv(zv, FWRITE, 0, NULL); + } +// error = dmu_objset_own(zv->zv_name, DMU_OST_ZVOL, ro, B_TRUE, zv, &os); + + if (error == 0) { + // We keep zv_objset / dmu_objset_own() open for storport + goto out_doi; + } +#endif + + // About to disown + zv->zv_objset = NULL; +out_dmu_objset_disown: + dmu_objset_disown(os, B_TRUE, FTAG); +out_doi: + kmem_free(doi, sizeof (dmu_object_info_t)); + + if (error == 0) { + zvol_os_attach(name); + } + dprintf("%s complete\n", __func__); + return (error); +} + + +static void zvol_os_rename_device_cb(void *param) +{ + zvol_state_t *zv = (zvol_state_t *)param; + int locks; + if ((locks = zvol_os_verify_and_lock(zv, zv->zv_open_count == 0)) == 0) + return; + + zvol_os_verify_lock_exit(zv, locks); +// zvolRenameDevice(zv); +} + +static void +zvol_os_rename_minor(zvol_state_t *zv, const char *newname) +{ + // int readonly = get_disk_ro(zv->zv_zso->zvo_disk); + + ASSERT(RW_LOCK_HELD(&zvol_state_lock)); + ASSERT(MUTEX_HELD(&zv->zv_state_lock)); + + strlcpy(zv->zv_name, newname, sizeof (zv->zv_name)); + + /* move to new hashtable entry */ + zv->zv_hash = zvol_name_hash(zv->zv_name); + hlist_del(&zv->zv_hlink); + hlist_add_head(&zv->zv_hlink, ZVOL_HT_HEAD(zv->zv_hash)); + + zvol_os_spawn(zvol_os_rename_device_cb, zv); + + /* + * The block device's read-only state is briefly changed causing + * a KOBJ_CHANGE uevent to be issued. This ensures udev detects + * the name change and fixes the symlinks. This does not change + * ZVOL_RDONLY in zv->zv_flags so the actual read-only state never + * changes. This would normally be done using kobject_uevent() but + * that is a GPL-only symbol which is why we need this workaround. + */ + // set_disk_ro(zv->zv_zso->zvo_disk, !readonly); + // set_disk_ro(zv->zv_zso->zvo_disk, readonly); +} + +static void +zvol_os_set_disk_ro(zvol_state_t *zv, int flags) +{ + // set_disk_ro(zv->zv_zso->zvo_disk, flags); +} + +static void +zvol_os_set_capacity(zvol_state_t *zv, uint64_t capacity) +{ + // set_capacity(zv->zv_zso->zvo_disk, capacity); +} + +int +zvol_os_open_zv(zvol_state_t *zv, int flag, int otyp, struct proc *p) +{ + int error = 0; + int locks; + + dprintf("%s\n", __func__); + + /* + * make sure zvol is not suspended during first open + * (hold zv_suspend_lock) and respect proper lock acquisition + * ordering - zv_suspend_lock before zv_state_lock + */ + if ((locks = zvol_os_verify_and_lock(zv, zv->zv_open_count == 0)) + == 0) + return (SET_ERROR(ENOENT)); + + ASSERT(MUTEX_HELD(&zv->zv_state_lock)); + ASSERT(zv->zv_open_count != 0 || RW_READ_HELD(&zv->zv_suspend_lock)); + + if (zv->zv_open_count == 0) { + error = zvol_first_open(zv, !(flag & FWRITE)); + if (error) + goto out_mutex; + } + + if ((flag & FWRITE) && (zv->zv_flags & ZVOL_RDONLY)) { + error = EROFS; + goto out_open_count; + } + + zv->zv_open_count++; + + zvol_os_verify_lock_exit(zv, locks); + + return (0); + +out_open_count: + if (zv->zv_open_count == 0) + zvol_last_close(zv); + +out_mutex: + zvol_os_verify_lock_exit(zv, locks); + + if (error == EINTR) { + error = ERESTART; + schedule(); + } + return (SET_ERROR(error)); +} + +int +zvol_os_open(dev_t devp, int flag, int otyp, struct proc *p) +{ + zvol_state_t *zv; + int error = 0; + + dprintf("%s\n", __func__); + + if (!getminor(devp)) + return (0); + + zv = zvol_os_find_by_dev(devp); + if (zv == NULL) { + return (SET_ERROR(ENXIO)); + } + + error = zvol_os_open_zv(zv, flag, otyp, p); + + mutex_exit(&zv->zv_state_lock); + return (SET_ERROR(error)); +} + +int +zvol_os_close_zv(zvol_state_t *zv, int flag, int otyp, struct proc *p) +{ + int locks; + dprintf("%s\n", __func__); + + if ((locks = zvol_os_verify_and_lock(zv, TRUE)) == 0) + return (SET_ERROR(ENOENT)); + + ASSERT(MUTEX_HELD(&zv->zv_state_lock)); + ASSERT(zv->zv_open_count != 1 || RW_READ_HELD(&zv->zv_suspend_lock)); + + zv->zv_open_count--; + + if (zv->zv_open_count == 0) + zvol_last_close(zv); + + zvol_os_verify_lock_exit(zv, locks); + + return (0); +} + +int +zvol_os_close(dev_t dev, int flag, int otyp, struct proc *p) +{ + zvol_state_t *zv; + int error = 0; + + dprintf("%s\n", __func__); + + if (!getminor(dev)) + return (0); + + zv = zvol_os_find_by_dev(dev); + if (zv == NULL) { + return (SET_ERROR(-ENXIO)); + } + + error = zvol_os_close_zv(zv, flag, otyp, p); + + mutex_exit(&zv->zv_state_lock); + return (0); +} + +void +zvol_os_strategy(struct buf *bp) +{ + +} + +int +zvol_os_get_volume_blocksize(dev_t dev) +{ + /* XNU can only handle two sizes. */ + return (DEV_BSIZE); +} + +int +zvol_os_ioctl(dev_t dev, unsigned long cmd, caddr_t data, int isblk, + cred_t *cr, int *rvalp) +{ + int error = 0; + zvol_state_t *zv = NULL; + + dprintf("%s\n", __func__); + + if (!getminor(dev)) + return (ENXIO); + + zv = zvol_os_find_by_dev(dev); + + if (zv == NULL) { + dprintf("zv is NULL\n"); + return (ENXIO); + } + + // + + mutex_exit(&zv->zv_state_lock); + + return (SET_ERROR(error)); +} + +int +zvol_init(void) +{ + int threads = MIN(MAX(zvol_threads, 1), 1024); + + zvol_taskq = taskq_create(ZVOL_DRIVER, threads, maxclsyspri, + threads * 2, INT_MAX, TASKQ_PREPOPULATE | TASKQ_DYNAMIC); + if (zvol_taskq == NULL) { + return (-ENOMEM); + } + + zvol_init_impl(); + return (0); +} + +void +zvol_fini(void) +{ + zvol_fini_impl(); + taskq_destroy(zvol_taskq); +} + +/* ZFS ZVOLDI */ + +_Function_class_(PINTERFACE_REFERENCE) void +IncZvolRef(PVOID Context) +{ + zvol_state_t *zv = (zvol_state_t *)Context; + atomic_inc_32(&zv->zv_open_count); +} + +_Function_class_(PINTERFACE_REFERENCE) void +DecZvolRef(PVOID Context) +{ + zvol_state_t *zv = (zvol_state_t *)Context; + atomic_dec_32(&zv->zv_open_count); +} + +zvol_state_t * +zvol_name2zvolState(const char *name, uint32_t *openCount) +{ + zvol_state_t *zv; + + zv = zvol_find_by_name(name, RW_NONE); + if (zv == NULL) + return (zv); + + if (openCount) + *openCount = zv->zv_open_count; + + mutex_exit(&zv->zv_state_lock); + return (zv); +} diff --git a/module/unicode/CMakeLists.txt b/module/unicode/CMakeLists.txt new file mode 100644 index 000000000000..13f8023b8299 --- /dev/null +++ b/module/unicode/CMakeLists.txt @@ -0,0 +1,9 @@ + +use_clang() + +wdk_add_library(unicodekern + u8_textprep.c + uconv.c +) + +target_link_libraries(unicodekern PRIVATE splkern) diff --git a/module/zcommon/CMakeLists.txt b/module/zcommon/CMakeLists.txt new file mode 100644 index 000000000000..270a65cca45f --- /dev/null +++ b/module/zcommon/CMakeLists.txt @@ -0,0 +1,21 @@ + +use_clang() + +wdk_add_library(zcommonkern + cityhash.c + zfeature_common.c + zfs_comutil.c + zfs_deleg.c + zfs_fletcher.c + zfs_fletcher_avx512.c + zfs_fletcher_intel.c + zfs_fletcher_sse.c + zfs_fletcher_superscalar.c + zfs_fletcher_superscalar4.c + zfs_namecheck.c + zfs_prop.c + zpool_prop.c + zprop_common.c +) + +target_link_libraries(zcommonkern PRIVATE splkern) diff --git a/module/zcommon/zfs_fletcher.c b/module/zcommon/zfs_fletcher.c index 619ddef0243a..3c23ddd99d88 100644 --- a/module/zcommon/zfs_fletcher.c +++ b/module/zcommon/zfs_fletcher.c @@ -457,12 +457,14 @@ fletcher_4_native_impl(const void *buf, uint64_t size, zio_cksum_t *zcp) if (ops->uses_fpu == B_TRUE) { kfpu_begin(); - } - ops->init_native(&ctx); - ops->compute_native(&ctx, buf, size); - ops->fini_native(&ctx, zcp); - if (ops->uses_fpu == B_TRUE) { + ops->init_native(&ctx); + ops->compute_native(&ctx, buf, size); + ops->fini_native(&ctx, zcp); kfpu_end(); + } else { + ops->init_native(&ctx); + ops->compute_native(&ctx, buf, size); + ops->fini_native(&ctx, zcp); } } @@ -505,12 +507,14 @@ fletcher_4_byteswap_impl(const void *buf, uint64_t size, zio_cksum_t *zcp) if (ops->uses_fpu == B_TRUE) { kfpu_begin(); - } - ops->init_byteswap(&ctx); - ops->compute_byteswap(&ctx, buf, size); - ops->fini_byteswap(&ctx, zcp); - if (ops->uses_fpu == B_TRUE) { + ops->init_byteswap(&ctx); + ops->compute_byteswap(&ctx, buf, size); + ops->fini_byteswap(&ctx, zcp); kfpu_end(); + } else { + ops->init_byteswap(&ctx); + ops->compute_byteswap(&ctx, buf, size); + ops->fini_byteswap(&ctx, zcp); } } @@ -827,7 +831,11 @@ abd_fletcher_4_init(zio_abd_checksum_data_t *cdp) cdp->acd_private = (void *) ops; if (ops->uses_fpu == B_TRUE) { +#if defined(_WIN32) && defined(_KERNEL) + kfpu_begin_ctx(cdp); +#else kfpu_begin(); +#endif } if (cdp->acd_byteorder == ZIO_CHECKSUM_NATIVE) ops->init_native(cdp->acd_ctx); @@ -849,7 +857,11 @@ abd_fletcher_4_fini(zio_abd_checksum_data_t *cdp) ops->fini_byteswap(cdp->acd_ctx, cdp->acd_zcp); if (ops->uses_fpu == B_TRUE) { +#if defined(_WIN32) && defined(_KERNEL) + kfpu_end_ctx(cdp); +#else kfpu_end(); +#endif } } @@ -911,7 +923,7 @@ zio_abd_checksum_func_t fletcher_4_abd_ops = { #define IMPL_FMT(impl, i) (((impl) == (i)) ? "[%s] " : "%s ") -#if defined(__linux__) +#if defined(__linux__) || defined(_WIN32) static int fletcher_4_param_get(char *buffer, zfs_kernel_param_t *unused) @@ -940,14 +952,43 @@ fletcher_4_param_set(const char *val, zfs_kernel_param_t *unused) return (fletcher_4_impl_set(val)); } -#else +#endif /* Linux || Windows */ + +#ifdef _WIN32 + +int +win32_fletcher_4_param_set(ZFS_MODULE_PARAM_ARGS) +{ + uint32_t val; + static unsigned char str[1024] = ""; + + *type = ZT_TYPE_STRING; + + if (set == B_FALSE) { + if (fletcher_4_initialized) + fletcher_4_param_get(str, NULL); + *ptr = str; + *len = strlen(str); + return (0); + } + + ASSERT3P(ptr, !=, NULL); + + fletcher_4_impl_set(*ptr); + + return (0); +} + +#endif /* WIN32 */ + +#ifdef __FreeBSD__ #include static int fletcher_4_param(ZFS_MODULE_PARAM_ARGS) { - int err; + int err = 0; if (req->newptr == NULL) { const uint32_t impl = IMPL_READ(fletcher_4_impl_chosen); diff --git a/module/zcommon/zfs_prop.c b/module/zcommon/zfs_prop.c index 3db6fd13f4ae..282f180eade3 100644 --- a/module/zcommon/zfs_prop.c +++ b/module/zcommon/zfs_prop.c @@ -388,6 +388,14 @@ zfs_prop_init(void) { NULL } }; + static zprop_index_t mimic_table[] = { + { "off", ZFS_MIMIC_OFF }, + { "hfs", ZFS_MIMIC_HFS }, + { "apfs", ZFS_MIMIC_APFS }, + { "ntfs", ZFS_MIMIC_NTFS }, + { NULL } + }; + struct zfs_mod_supported_features *sfeatures = zfs_mod_list_supported(ZFS_SYSFS_DATASET_PROPERTIES); @@ -727,6 +735,13 @@ zfs_prop_init(void) zprop_register_hidden(ZFS_PROP_REDACTED, "redacted", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, "REDACTED", B_FALSE, sfeatures); + zprop_register_string(ZFS_PROP_DRIVELETTER, "driveletter", "-", + 0, ZFS_TYPE_FILESYSTEM, "off | on | A-Z ", + "DRIVELETTER", sfeatures); + zprop_register_index(ZFS_PROP_MIMIC, "com.apple.mimic", 0, + PROP_INHERIT, ZFS_TYPE_FILESYSTEM, "off | ntfs", + "COM.APPLE.MIMIC_HFS", mimic_table, sfeatures); + /* * Properties that are obsolete and not used. These are retained so * that we don't have to change the values of the zfs_prop_t enum, or diff --git a/module/zfs/CMakeLists.txt b/module/zfs/CMakeLists.txt new file mode 100644 index 000000000000..c994db41daee --- /dev/null +++ b/module/zfs/CMakeLists.txt @@ -0,0 +1,198 @@ + +use_clang() + +wdk_add_library(zfskern + KMDF 1.9 + abd.c + aggsum.c + arc.c + blake3_zfs.c + ../avl/avl.c + blkptr.c + bplist.c + bpobj.c + bptree.c + bqueue.c + brt.c + btree.c + dataset_kstats.c + dbuf.c + dbuf_stats.c + ddt.c + ddt_zap.c + dmu.c + dmu_diff.c + dmu_object.c + dmu_objset.c + dmu_recv.c + dmu_redact.c + dmu_send.c + dmu_traverse.c + dmu_tx.c + dmu_zfetch.c + dnode.c + dnode_sync.c + dsl_bookmark.c + dsl_crypt.c + dsl_dataset.c + dsl_deadlist.c + dsl_deleg.c + dsl_destroy.c + dsl_dir.c + dsl_pool.c + dsl_prop.c + dsl_scan.c + dsl_synctask.c + dsl_userhold.c + edonr_zfs.c + fm.c + gzip.c + hkdf.c + lz4.c + lz4_zfs.c + lzjb.c + metaslab.c + mmp.c + multilist.c + objlist.c + pathname.c + range_tree.c + refcount.c + rrwlock.c + sa.c + sha2_zfs.c + skein_zfs.c + spa.c + spa_checkpoint.c + spa_config.c + spa_errlog.c + spa_history.c + spa_log_spacemap.c + spa_misc.c + spa_stats.c + space_map.c + space_reftree.c + txg.c + uberblock.c + unique.c + vdev.c + vdev_draid.c + vdev_draid_rand.c + vdev_indirect.c + vdev_indirect_births.c + vdev_indirect_mapping.c + vdev_initialize.c + vdev_label.c + vdev_mirror.c + vdev_missing.c + vdev_queue.c + vdev_raidz.c + vdev_raidz_math.c + vdev_raidz_math_scalar.c + vdev_raidz_math_avx2.c + vdev_raidz_math_avx512f.c + vdev_raidz_math_avx512bw.c + vdev_raidz_math_sse2.c + vdev_raidz_math_ssse3.c + vdev_rebuild.c + vdev_removal.c + vdev_root.c + vdev_trim.c + zap.c + zap_leaf.c + zap_micro.c + zcp.c + zcp_get.c + zcp_global.c + zcp_iter.c + zcp_set.c + zcp_synctask.c + zfeature.c + zfs_byteswap.c + zfs_chksum.c + zfs_fm.c + zfs_fuid.c + zfs_impl.c + zfs_ioctl.c + zfs_log.c + zfs_onexit.c + zfs_quota.c + zfs_ratelimit.c + zfs_replay.c + zfs_rlock.c + zfs_sa.c + zfs_vnops.c + zil.c + zio.c + zio_checksum.c + zio_compress.c + zio_inject.c + zle.c + zrlock.c + zthr.c + zvol.c +) + +target_link_libraries(zfskern PRIVATE splkern icpkern zstdkern) + +file(GLOB CSOURCES CONFIGURE_DEPENDS "*.c") + +list(REMOVE_ITEM CSOURCES "${CMAKE_CURRENT_SOURCE_DIR}/txg.c") + +function(add_macro_property) + foreach (I ${CSOURCES}) + get_filename_component(OUTPUT_FILE_WE ${I} NAME_WE) + set_source_files_properties(${I} PROPERTIES COMPILE_FLAGS -DWPPFILE=${CMAKE_SOURCE_DIR}/out/build/${OUTPUT_FILE_WE}.tmh) + message(STATUS "FILES_IN ======: ${CMAKE_SOURCE_DIR}/out/build/${OUTPUT_FILE_WE}.tmh") + endforeach() +endfunction() + +function(tracewpp OUTPUT_DIR SOURCE) + # [mlr] list of .tmh files to be generated -> TMH + set(WPP_DIR "${WDK_ROOT}/bin/${WDK_VERSION}") + + get_filename_component(FILEN ${SOURCE} NAME) + set(TMH_FILEN ${FILEN}.tmh) + set(TMH ${OUTPUT_DIR}/${TMH_FILEN}) + set(EXTENSIONS ".c") + + # [mlr] cmake only converts the command name to the native path format. the + # path names to be used in arguments must be converted manually. + + file(TO_NATIVE_PATH ${SOURCE} NATIVE_SOURCE) + file(TO_NATIVE_PATH ${WPP_DIR} NATIVE_WPP_DIR) + file(TO_NATIVE_PATH ${OUTPUT_DIR} NATIVE_OUTPUT_DIR) + + # [mlr] note that if -preserveext: occurs after the source file specification, it has + # no effect. + + set(TRACE "TraceEvent{FLAGS=MYDRIVER_ALL_INFO}(LEVEL, MSG, ...)") + set(DPRINT "dprintf{FLAGS=MYDRIVER_ALL_INFO, LEVEL=TRACE_INFO}(MSG, ...)") + set(CFGDIR "${WPP_DIR}/wppconfig/rev1") + set(SCAN "${WDK_ROOT}/Include/wdf/kmdf/1.9/WdfTraceEnums.h") + set(WPP_MACRO "WPP_INLINE __inline") + + execute_process(COMMAND "${NATIVE_WPP_DIR}/${WDK_PLATFORM}/tracewpp.exe" + -scan:${SCAN} /D${WPP_MACRO} + -cfgdir:${CFGDIR} -I${CMAKE_CURRENT_BINARY_DIR} -odir:${NATIVE_OUTPUT_DIR} -km -func:${TRACE} + -func:${DPRINT} -gen:{km-default.tpl}*.tmh ${NATIVE_SOURCE}) + +endfunction() + + +function(wpp OUTPUT_DIR) + + add_macro_property() + # [mlr] invoke tracewpp() for each source file, adding the resulting file to a list + # named TMH. + message(STATUS "OUTPUT_DIR ======: ${OUTPUT_DIR}") + + foreach ( I ${CSOURCES} ) + tracewpp(${OUTPUT_DIR} ${I}) + endforeach() + +endfunction() + +if(NOT CMAKE_BUILD_TYPE STREQUAL "Debug") + wpp("${CMAKE_SOURCE_DIR}/out/build") +endif() diff --git a/module/zfs/dsl_scan.c b/module/zfs/dsl_scan.c index 1dd44171c10e..df4be0f01d11 100644 --- a/module/zfs/dsl_scan.c +++ b/module/zfs/dsl_scan.c @@ -3032,8 +3032,43 @@ dsl_scan_ds_maxtxg(dsl_dataset_t *ds) return (smt); } +#if defined(_WIN32) && defined(_KERNEL) +/* + * Windows lets us callout to a function with a + * larger stack size, which is needed in scrub and resilver. + * but since it takes only one argument, we have to double + * wrap it here. + */ +struct _win32_larger_stack { + dsl_scan_t *scn; + dmu_tx_t *tx; +}; + +static void +dsl_scan_visit_impl(dsl_scan_t *scn, dmu_tx_t *tx); + +void +dsl_scan_visit_wrap(void *arg1) +{ + struct _win32_larger_stack *arg = (struct _win32_larger_stack *)arg1; + dsl_scan_visit_impl(arg->scn, arg->tx); +} + static void dsl_scan_visit(dsl_scan_t *scn, dmu_tx_t *tx) +{ + struct _win32_larger_stack wls = { scn, tx }; + if (KeExpandKernelStackAndCallout(dsl_scan_visit_wrap, &wls, + MAXIMUM_EXPANSION_SIZE) != 0) + dsl_scan_visit_impl(scn, tx); +} + +static void +dsl_scan_visit_impl(dsl_scan_t *scn, dmu_tx_t *tx) +#else +static void +dsl_scan_visit(dsl_scan_t *scn, dmu_tx_t *tx) +#endif { scan_ds_t *sds; dsl_pool_t *dp = scn->scn_dp; diff --git a/module/zfs/sa.c b/module/zfs/sa.c index f9daaabbed3e..426252fddb86 100644 --- a/module/zfs/sa.c +++ b/module/zfs/sa.c @@ -1978,8 +1978,10 @@ sa_modify_attrs(sa_handle_t *hdl, sa_attr_type_t newattr, SA_ADD_BULK_ATTR(attr_desc, j, newattr, locator, datastart, buflen); } - ASSERT3U(j, ==, attr_count); + if (action != SA_REMOVE) { + ASSERT3U(j, ==, attr_count); + } error = sa_build_layouts(hdl, attr_desc, attr_count, tx); if (old_data[0]) diff --git a/module/zfs/spa.c b/module/zfs/spa.c index 88ee4ea9f458..bec72cd82657 100644 --- a/module/zfs/spa.c +++ b/module/zfs/spa.c @@ -1128,6 +1128,8 @@ spa_taskqs_init(spa_t *spa, zio_type_t t, zio_taskq_type_t q) pri++; #elif defined(__FreeBSD__) pri += 4; +#elif defined(_WIN32) + pri++; #else #error "unknown OS" #endif @@ -9436,6 +9438,7 @@ spa_sync(spa_t *spa, uint64_t txg) spa->spa_sync_starttime = gethrtime(); taskq_cancel_id(system_delay_taskq, spa->spa_deadman_tqid); + spa->spa_deadman_tqid = taskq_dispatch_delay(system_delay_taskq, spa_deadman, spa, TQ_SLEEP, ddi_get_lbolt() + NSEC_TO_TICK(spa->spa_deadman_synctime)); diff --git a/module/zfs/vdev.c b/module/zfs/vdev.c index 30551feb6322..811918f2d6a0 100644 --- a/module/zfs/vdev.c +++ b/module/zfs/vdev.c @@ -2470,6 +2470,24 @@ vdev_copy_path_impl(vdev_t *svd, vdev_t *dvd) (u_longlong_t)dvd->vdev_guid, dvd->vdev_path); } +#ifdef _WIN32 + /* Win uses physpath more as the visible ones are so ugly */ + if (svd->vdev_physpath != NULL && dvd->vdev_physpath != NULL) { + if (strcmp(svd->vdev_physpath, dvd->vdev_physpath) != 0) { + zfs_dbgmsg("vdev_copy_path: vdev %llu: " + "physpath changed from '%s' to '%s'", + (u_longlong_t)dvd->vdev_guid, + dvd->vdev_physpath, svd->vdev_physpath); + spa_strfree(dvd->vdev_physpath); + dvd->vdev_physpath = spa_strdup(svd->vdev_physpath); + } + } else if (svd->vdev_physpath != NULL) { + dvd->vdev_physpath = spa_strdup(svd->vdev_physpath); + zfs_dbgmsg("vdev_copy_path: vdev %llu: physpath set to '%s'", + (u_longlong_t)dvd->vdev_guid, dvd->vdev_physpath); + } +#endif + /* * Our enclosure sysfs path may have changed between imports */ diff --git a/module/zfs/vdev_raidz_math.c b/module/zfs/vdev_raidz_math.c index e12b96170f55..fea93aa8b255 100644 --- a/module/zfs/vdev_raidz_math.c +++ b/module/zfs/vdev_raidz_math.c @@ -633,7 +633,8 @@ vdev_raidz_impl_set(const char *val) return (err); } -#if defined(_KERNEL) && defined(__linux__) +#if defined(_KERNEL) +#if defined(__linux__) || defined(_WIN32) static int zfs_vdev_raidz_impl_set(const char *val, zfs_kernel_param_t *kp) @@ -666,6 +667,32 @@ zfs_vdev_raidz_impl_get(char *buffer, zfs_kernel_param_t *kp) return (cnt); } +#endif /* Linux || Windows */ + +#ifdef _WIN32 +int +win32_zfs_vdev_raidz_impl_set(ZFS_MODULE_PARAM_ARGS) +{ + uint32_t val; + static unsigned char str[1024] = ""; + + *type = ZT_TYPE_STRING; + + if (set == B_FALSE) { + if (raidz_math_initialized) + zfs_vdev_raidz_impl_get(str, NULL); + *ptr = str; + *len = strlen(str); + return (0); + } + + ASSERT3P(ptr, !=, NULL); + + vdev_raidz_impl_set(*ptr); + + return (0); +} +#endif module_param_call(zfs_vdev_raidz_impl, zfs_vdev_raidz_impl_set, zfs_vdev_raidz_impl_get, NULL, 0644); diff --git a/module/zfs/zap_micro.c b/module/zfs/zap_micro.c index 085d9cd8b4b6..5e781fb18aa1 100644 --- a/module/zfs/zap_micro.c +++ b/module/zfs/zap_micro.c @@ -628,12 +628,15 @@ zap_lockdir(objset_t *os, uint64_t obj, dmu_tx_t *tx, int err = dmu_buf_hold(os, obj, 0, tag, &db, DMU_READ_NO_PREFETCH); if (err != 0) return (err); +#ifndef _WIN32 + // Triggers under Windows, so until we can figure it out... #ifdef ZFS_DEBUG { dmu_object_info_t doi; dmu_object_info_from_db(db, &doi); ASSERT3U(DMU_OT_BYTESWAP(doi.doi_type), ==, DMU_BSWAP_ZAP); } +#endif #endif err = zap_lockdir_impl(db, tag, tx, lti, fatreader, adding, zapp); if (err != 0) diff --git a/module/zfs/zfs_fuid.c b/module/zfs/zfs_fuid.c index add4241dcc99..e7dbfcac9b93 100644 --- a/module/zfs/zfs_fuid.c +++ b/module/zfs/zfs_fuid.c @@ -399,6 +399,24 @@ zfs_fuid_map_id(zfsvfs_t *zfsvfs, uint64_t fuid, return (UID_NOBODY); } +#elif defined(_WIN32) +uid_t +zfs_fuid_map_id(zfsvfs_t *zfsvfs, uint64_t fuid, + cred_t *cr, zfs_fuid_type_t type) +{ + uint32_t index = FUID_INDEX(fuid); + const char *domain; + uid_t id; + + if (index == 0) + return (fuid); + + domain = zfs_fuid_find_by_idx(zfsvfs, index); + ASSERT(domain != NULL); + + id = UID_NOBODY; + return (id); +} #elif defined(__linux__) uid_t zfs_fuid_map_id(zfsvfs_t *zfsvfs, uint64_t fuid, diff --git a/module/zfs/zfs_ioctl.c b/module/zfs/zfs_ioctl.c index f91a2f3bbca5..25c57fecbd91 100644 --- a/module/zfs/zfs_ioctl.c +++ b/module/zfs/zfs_ioctl.c @@ -4037,6 +4037,11 @@ zfs_ioc_destroy(zfs_cmd_t *zc) if (ost == DMU_OST_ZFS) zfs_unmount_snap(zc->zc_name); +#if defined(_WIN32) && defined(_KERNEL) + if (ost == DMU_OST_ZVOL) + zvol_os_detach(zc->zc_name); +#endif + if (strchr(zc->zc_name, '@')) { err = dsl_destroy_snapshot(zc->zc_name, zc->zc_defer_destroy); } else { @@ -4071,6 +4076,11 @@ zfs_ioc_destroy(zfs_cmd_t *zc) } } +#if defined(_WIN32) && defined(_KERNEL) + if (ost == DMU_OST_ZVOL && err != 0) + zvol_os_attach(zc->zc_name); +#endif + return (err); } @@ -6976,7 +6986,7 @@ zfs_ioc_change_key(const char *dsname, nvlist_t *innvl, nvlist_t *outnvl) static zfs_ioc_vec_t zfs_ioc_vec[ZFS_IOC_LAST - ZFS_IOC_FIRST]; -static void +void zfs_ioctl_register_legacy(zfs_ioc_t ioc, zfs_ioc_legacy_func_t *func, zfs_secpolicy_func_t *secpolicy, zfs_ioc_namecheck_t namecheck, boolean_t log_history, zfs_ioc_poolcheck_t pool_check) @@ -7588,7 +7598,12 @@ zfsdev_state_init(void *priv) ASSERT(MUTEX_HELD(&zfsdev_state_lock)); +#if defined(_WIN32) && defined(_KERNEL) + minor = minor((dev_t)priv); +#else minor = zfsdev_minor_alloc(); +#endif + if (minor == 0) return (SET_ERROR(ENXIO)); diff --git a/module/zfs/zfs_log.c b/module/zfs/zfs_log.c index 50325907b0d1..ee973d40f701 100644 --- a/module/zfs/zfs_log.c +++ b/module/zfs/zfs_log.c @@ -253,6 +253,24 @@ zfs_xattr_owner_unlinked(znode_t *zp) } if (tzp != zp) zrele(tzp); +#elif defined(_WIN32) + zhold(zp); + /* + * if zp is XATTR node, keep walking up via z_xattr_parent until we + * get the owner + */ + while (zp->z_pflags & ZFS_XATTR && zp->z_xattr_parent != NULL) { + ASSERT3U(zp->z_xattr_parent, !=, 0); + if (zfs_zget(ZTOZSB(zp), zp->z_xattr_parent, &dzp) != 0) { + unlinked = 1; + break; + } + + zrele(zp); + zp = dzp; + unlinked = zp->z_unlinked; + } + zrele(zp); #else zhold(zp); /* diff --git a/module/zfs/zvol.c b/module/zfs/zvol.c index cd4e6f0c7558..4860540e3771 100644 --- a/module/zfs/zvol.c +++ b/module/zfs/zvol.c @@ -37,6 +37,7 @@ * Copyright 2014 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2016 Actifio, Inc. All rights reserved. * Copyright (c) 2012, 2019 by Delphix. All rights reserved. + * Portions Copyright 2022 Andrew Innes */ /* @@ -90,7 +91,7 @@ unsigned int zvol_inhibit_dev = 0; unsigned int zvol_volmode = ZFS_VOLMODE_GEOM; struct hlist_head *zvol_htable; -static list_t zvol_state_list; +list_t zvol_state_list; krwlock_t zvol_state_lock; typedef enum { @@ -174,7 +175,7 @@ zvol_find_by_name_hash(const char *name, uint64_t hash, int mode) * before zv_state_lock. The mode argument indicates the mode (including none) * for zv_suspend_lock to be taken. */ -static zvol_state_t * +zvol_state_t * zvol_find_by_name(const char *name, int mode) { return (zvol_find_by_name_hash(name, zvol_name_hash(name), mode)); @@ -1279,6 +1280,9 @@ zvol_remove_minors_impl(const char *name) * By holding zv_state_lock here, we guarantee that no * one is currently using this zv */ +#ifdef _WIN32 + zvol_os_detach_zv(zv); +#endif /* If in use, leave alone */ if (zv->zv_open_count > 0 || diff --git a/module/zstd/CMakeLists.txt b/module/zstd/CMakeLists.txt new file mode 100644 index 000000000000..9f4c4b03d577 --- /dev/null +++ b/module/zstd/CMakeLists.txt @@ -0,0 +1,30 @@ + +use_clang() + +wdk_add_library(zstdkern + lib/common/entropy_common.c + lib/common/error_private.c + lib/common/fse_decompress.c + lib/common/pool.c + lib/common/zstd_common.c + lib/common/xxhash.c + lib/compress/fse_compress.c + lib/compress/hist.c + lib/compress/huf_compress.c + lib/compress/zstd_compress_literals.c + lib/compress/zstd_compress_sequences.c + lib/compress/zstd_compress_superblock.c + lib/compress/zstd_compress.c + lib/compress/zstd_double_fast.c + lib/compress/zstd_fast.c + lib/compress/zstd_lazy.c + lib/compress/zstd_ldm.c + lib/compress/zstd_opt.c + lib/decompress/huf_decompress.c + lib/decompress/zstd_ddict.c + lib/decompress/zstd_decompress.c + lib/decompress/zstd_decompress_block.c + zfs_zstd.c +) + +target_include_directories(zstdkern BEFORE PRIVATE "include") diff --git a/module/zstd/include/limits.h b/module/zstd/include/limits.h index 3bf5b67765ae..8ed2eb27de40 100644 --- a/module/zstd/include/limits.h +++ b/module/zstd/include/limits.h @@ -48,6 +48,8 @@ extern "C" { #elif defined(__linux__) #include #include +#elif defined(_WIN32) +#include_next #else #error "Unsupported platform" #endif diff --git a/module/zstd/include/stddef.h b/module/zstd/include/stddef.h index 3f46fb8b033e..8de3c2957273 100644 --- a/module/zstd/include/stddef.h +++ b/module/zstd/include/stddef.h @@ -47,6 +47,8 @@ extern "C" { #include #elif defined(__linux__) #include +#elif defined(_WIN32) +#include_next #else #error "Unsupported platform" #endif diff --git a/module/zstd/include/stdint.h b/module/zstd/include/stdint.h index 2d98a556c23e..3227286ad4c5 100644 --- a/module/zstd/include/stdint.h +++ b/module/zstd/include/stdint.h @@ -47,6 +47,8 @@ extern "C" { #include #elif defined(__linux__) #include +#elif defined(_WIN32) +#include_next #else #error "Unsupported platform" #endif diff --git a/module/zstd/include/string.h b/module/zstd/include/string.h index 7474e7f1af0f..696e8acafb78 100644 --- a/module/zstd/include/string.h +++ b/module/zstd/include/string.h @@ -48,6 +48,8 @@ extern "C" { #include /* memcpy, memset */ #elif defined(__linux__) #include /* memcpy, memset */ +#elif defined(_WIN32) +#include_next #else #error "Unsupported platform" #endif diff --git a/module/zstd/lib/common/bitstream.h b/module/zstd/lib/common/bitstream.h index 37b99c01eed3..624de3d6f976 100644 --- a/module/zstd/lib/common/bitstream.h +++ b/module/zstd/lib/common/bitstream.h @@ -140,7 +140,7 @@ MEM_STATIC unsigned BIT_highbit32 (U32 val) { assert(val != 0); { -# if defined(_MSC_VER) /* Visual */ +# if defined(_MSC_VER) && !defined(__clang__) /* Visual */ unsigned long r=0; return _BitScanReverse ( &r, val ) ? (unsigned)r : 0; # elif defined(__GNUC__) && (__GNUC__ >= 3) /* Use GCC Intrinsic */ diff --git a/module/zstd/lib/common/compiler.h b/module/zstd/lib/common/compiler.h index 95e9483521d4..c816d70eb539 100644 --- a/module/zstd/lib/common/compiler.h +++ b/module/zstd/lib/common/compiler.h @@ -110,7 +110,7 @@ # define PREFETCH_L1(ptr) (void)(ptr) /* disabled */ # define PREFETCH_L2(ptr) (void)(ptr) /* disabled */ #else -# if defined(_MSC_VER) && (defined(_M_X64) || defined(_M_I86)) /* _mm_prefetch() is not defined outside of x86/x64 */ +# if defined(_MSC_VER) && (defined(_M_X64) || defined(_M_I86)) && !defined(__clang__) /* _mm_prefetch() is not defined outside of x86/x64 */ # include /* https://msdn.microsoft.com/fr-fr/library/84szxsww(v=vs.90).aspx */ # define PREFETCH_L1(ptr) _mm_prefetch((const char*)(ptr), _MM_HINT_T0) # define PREFETCH_L2(ptr) _mm_prefetch((const char*)(ptr), _MM_HINT_T1) @@ -163,7 +163,7 @@ #endif /* disable warnings */ -#ifdef _MSC_VER /* Visual Studio */ +#if defined(_MSC_VER) && !defined(__clang__) /* Visual Studio */ # include /* For Visual 2005 */ # pragma warning(disable : 4100) /* disable: C4100: unreferenced formal parameter */ # pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ diff --git a/module/zstd/lib/common/cpu.h b/module/zstd/lib/common/cpu.h index 6e8a974f62d7..43123856a044 100644 --- a/module/zstd/lib/common/cpu.h +++ b/module/zstd/lib/common/cpu.h @@ -20,7 +20,7 @@ #include "mem.h" -#ifdef _MSC_VER +#if defined(_MSC_VER) && !defined(__clang__) #include #endif @@ -36,7 +36,7 @@ MEM_STATIC ZSTD_cpuid_t ZSTD_cpuid(void) { U32 f1d = 0; U32 f7b = 0; U32 f7c = 0; -#if defined(_MSC_VER) && (defined(_M_X64) || defined(_M_IX86)) +#if defined(_MSC_VER) && !defined(__clang__) && (defined(_M_X64) || defined(_M_IX86)) int reg[4]; __cpuid((int*)reg, 0); { diff --git a/module/zstd/lib/common/mem.h b/module/zstd/lib/common/mem.h index 7fbb2e98215e..253d252cdf93 100644 --- a/module/zstd/lib/common/mem.h +++ b/module/zstd/lib/common/mem.h @@ -25,7 +25,7 @@ extern "C" { /*-**************************************** * Compiler specifics ******************************************/ -#if defined(_MSC_VER) /* Visual Studio */ +#if defined(_MSC_VER) && !defined(__clang__) /* Visual Studio */ # include /* _byteswap_ulong */ # include /* _byteswap_* */ #endif @@ -270,7 +270,7 @@ MEM_STATIC void MEM_write64(void* memPtr, U64 value) MEM_STATIC U32 MEM_swap32(U32 in) { -#if defined(_MSC_VER) /* Visual Studio */ +#if defined(_MSC_VER) && !defined(__clang__) /* Visual Studio */ return _byteswap_ulong(in); #elif (defined (__GNUC__) && (__GNUC__ * 100 + __GNUC_MINOR__ >= 403)) \ || (defined(__clang__) && __has_builtin(__builtin_bswap32)) @@ -285,7 +285,7 @@ MEM_STATIC U32 MEM_swap32(U32 in) MEM_STATIC U64 MEM_swap64(U64 in) { -#if defined(_MSC_VER) /* Visual Studio */ +#if defined(_MSC_VER) && !defined(__clang__) /* Visual Studio */ return _byteswap_uint64(in); #elif (defined (__GNUC__) && (__GNUC__ * 100 + __GNUC_MINOR__ >= 403)) \ || (defined(__clang__) && __has_builtin(__builtin_bswap64)) diff --git a/module/zstd/lib/common/xxhash.c b/module/zstd/lib/common/xxhash.c index 597de18fc895..27fcbf743c53 100644 --- a/module/zstd/lib/common/xxhash.c +++ b/module/zstd/lib/common/xxhash.c @@ -5,7 +5,7 @@ * You can contact the author at : * - xxHash homepage: http://www.xxhash.com * - xxHash source repository : https://github.com/Cyan4973/xxHash - * + * * This source code is licensed under both the BSD-style license (found in the * LICENSE file in the root directory of this source tree) and the GPLv2 (found * in the COPYING file in the root directory of this source tree). @@ -196,7 +196,7 @@ static U64 XXH_read64(const void* memPtr) # define XXH_rotl64(x,r) ((x << r) | (x >> (64 - r))) #endif -#if defined(_MSC_VER) /* Visual Studio */ +#if defined(_MSC_VER) && !defined(__clang__) /* Visual Studio */ # define XXH_swap32 _byteswap_ulong # define XXH_swap64 _byteswap_uint64 #elif GCC_VERSION >= 403 diff --git a/module/zstd/lib/common/zstd_internal.h b/module/zstd/lib/common/zstd_internal.h index 4a86d186a967..2f5010661a61 100644 --- a/module/zstd/lib/common/zstd_internal.h +++ b/module/zstd/lib/common/zstd_internal.h @@ -260,7 +260,7 @@ typedef enum { * - ZSTD_overlap_src_before_dst: The src and dst may overlap, but they MUST be at least 8 bytes apart. * The src buffer must be before the dst buffer. */ -MEM_STATIC FORCE_INLINE_ATTR +MEM_STATIC FORCE_INLINE_ATTR void ZSTD_wildcopy(void* dst, const void* src, ptrdiff_t length, ZSTD_overlap_e const ovtype) { ptrdiff_t diff = (BYTE*)dst - (const BYTE*)src; @@ -392,7 +392,7 @@ MEM_STATIC U32 ZSTD_highbit32(U32 val) /* compress, dictBuilder, decodeCorpus { assert(val != 0); { -# if defined(_MSC_VER) /* Visual */ +# if defined(_MSC_VER) && !defined(__clang__) unsigned long r=0; return _BitScanReverse(&r, val) ? (unsigned)r : 0; # elif defined(__GNUC__) && (__GNUC__ >= 3) /* GCC Intrinsic */ diff --git a/module/zstd/lib/compress/zstd_compress_internal.h b/module/zstd/lib/compress/zstd_compress_internal.h index db73f6ce21f2..fb15cbc03747 100644 --- a/module/zstd/lib/compress/zstd_compress_internal.h +++ b/module/zstd/lib/compress/zstd_compress_internal.h @@ -497,7 +497,7 @@ static unsigned ZSTD_NbCommonBytes (size_t val) { if (MEM_isLittleEndian()) { if (MEM_64bits()) { -# if defined(_MSC_VER) && defined(_WIN64) +# if defined(_MSC_VER) && defined(_WIN64) && !defined(__clang__) unsigned long r = 0; return _BitScanForward64( &r, (U64)val ) ? (unsigned)(r >> 3) : 0; # elif defined(__GNUC__) && (__GNUC__ >= 4) @@ -514,7 +514,7 @@ static unsigned ZSTD_NbCommonBytes (size_t val) return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58]; # endif } else { /* 32 bits */ -# if defined(_MSC_VER) +# if defined(_MSC_VER) && !defined(__clang__) unsigned long r=0; return _BitScanForward( &r, (U32)val ) ? (unsigned)(r >> 3) : 0; # elif defined(__GNUC__) && (__GNUC__ >= 3) @@ -529,7 +529,7 @@ static unsigned ZSTD_NbCommonBytes (size_t val) } } else { /* Big Endian CPU */ if (MEM_64bits()) { -# if defined(_MSC_VER) && defined(_WIN64) +# if defined(_MSC_VER) && defined(_WIN64) && !defined(__clang__) unsigned long r = 0; return _BitScanReverse64( &r, val ) ? (unsigned)(r >> 3) : 0; # elif defined(__GNUC__) && (__GNUC__ >= 4) @@ -543,7 +543,7 @@ static unsigned ZSTD_NbCommonBytes (size_t val) return r; # endif } else { /* 32 bits */ -# if defined(_MSC_VER) +# if defined(_MSC_VER) && !defined(__clang__) unsigned long r = 0; return _BitScanReverse( &r, (unsigned long)val ) ? (unsigned)(r >> 3) : 0; # elif defined(__GNUC__) && (__GNUC__ >= 3) diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh index 0e0879823619..0cd3d916f1e4 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_mount/zfs_mount_011_neg.ksh @@ -69,8 +69,10 @@ for opt in "-o abc" "-O"; do done #verify that zfs mount fails with volume and snapshot -log_must zfs snapshot $TESTPOOL/$TESTFS@$TESTSNAP -log_mustnot eval "zfs mount $TESTPOOL/$TESTFS@$TESTSNAP >/dev/null 2>&1" +# Update: Windows and macOS now require snapshots to be +# mountable. +# log_must zfs snapshot $TESTPOOL/$TESTFS@$TESTSNAP +# log_mustnot eval "zfs mount $TESTPOOL/$TESTFS@$TESTSNAP >/dev/null 2>&1" if is_global_zone; then log_must zfs create -V 10m $TESTPOOL/$TESTVOL diff --git a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh index 3524efcd076d..6efbd323374b 100755 --- a/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh +++ b/tests/zfs-tests/tests/functional/cli_root/zfs_unmount/zfs_unmount_008_neg.ksh @@ -92,7 +92,8 @@ for arg in ${badargs[@]}; do done # Testing invalid datasets -for ds in $snap $vol "blah"; do +# Windows / macOS allow snapshot now; removing $snap +for ds in $vol "blah"; do for opt in "" "-f"; do log_mustnot eval "zfs unmount $opt $ds >/dev/null 2>&1" done