Skip to content

Commit

Permalink
update vignettes
Browse files Browse the repository at this point in the history
  • Loading branch information
@shikokuchuo
shikokuchuo committed Nov 15, 2024
1 parent b870e11 commit 3d48dc8
Show file tree
Hide file tree
Showing 2 changed files with 58 additions and 59 deletions.
103 changes: 51 additions & 52 deletions vignettes/mirai.Rmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,15 +62,15 @@ To wait for and collect the return value, use the mirai's `[]` method:

``` r
m[]
#> [1] 4.021118 4.931766 5.243967 3.375100 5.667178
#> [1] 3.495676 3.816495 4.628665 3.491260 5.778629
```
As a mirai represents an async operation, it is never necessary to wait for it. Other code can continue to be run. Once it completes, the return value automatically becomes available at `$data`.

``` r
m
#> < mirai [$data] >
m$data
#> [1] 4.021118 4.931766 5.243967 3.375100 5.667178
#> [1] 3.495676 3.816495 4.628665 3.491260 5.778629
```
For easy programmatic use of `mirai()`, '.expr' accepts a pre-constructed language object, and also a list of named arguments passed via '.args'. So, the following would be equivalent to the above:

Expand All @@ -82,7 +82,7 @@ args <- list(time = x$time, mean = x$mean)

m <- mirai(.expr = expr, .args = args)
m[]
#> [1] 2.344214 5.151972 2.745938 5.707968 3.244619
#> [1] 3.425921 4.885346 4.717019 2.501762 5.443241
```

[&laquo; Back to ToC](#table-of-contents)
Expand Down Expand Up @@ -158,15 +158,14 @@ for (i in 1:10) {
}
#> iteration 1 successful
#> iteration 2 successful
#> Error: random error
#> iteration 3 successful
#> iteration 4 successful
#> iteration 5 successful
#> iteration 6 successful
#> iteration 7 successful
#> Error: random error
#> iteration 8 successful
#> iteration 9 successful
#> Error: random error
#> iteration 10 successful
```
Further, by testing the return value of each 'mirai' for errors, error-handling code is then able to automate recovery and re-attempts, as in the above example. Further details on [error handling](#errors-interrupts-and-timeouts) can be found in the section below.
Expand Down Expand Up @@ -201,12 +200,12 @@ status()
#>
#> $daemons
#> i online instance assigned complete
#> abstract://f52b35b9a832a2152cb68451 1 1 1 0 0
#> abstract://033141983db5a787d41c6b62 2 1 1 0 0
#> abstract://f5b1e54c4524b0ad70f97cb1 3 1 1 0 0
#> abstract://0c2dde598cb4db7faea10fa5 4 1 1 0 0
#> abstract://4c233fb0f8673194b1f4e043 5 1 1 0 0
#> abstract://32c59496251de9055106a772 6 1 1 0 0
#> abstract://bb7da8970444e783a3f63281 1 1 1 0 0
#> abstract://faf804d9f786b1ccc4f76628 2 1 1 0 0
#> abstract://e8e180caa09bc41f4067298a 3 1 1 0 0
#> abstract://3faa8cb0dadbe3c24fc35133 4 1 1 0 0
#> abstract://bdca3dd925f7416391b86f12 5 1 1 0 0
#> abstract://2e4db69b3b9c198610e51062 6 1 1 0 0
```
The default `dispatcher = "process"` creates a `dispatcher()` background process that connects to individual daemon processes on the local machine. This ensures that tasks are dispatched efficiently on a first-in first-out (FIFO) basis to daemons for processing. Tasks are queued at the dispatcher and sent to a daemon as soon as it can accept the task for immediate execution.

Expand Down Expand Up @@ -238,7 +237,7 @@ status()
#> [1] 6
#>
#> $daemons
#> [1] "abstract://d47e6c20df78276654d7893e"
#> [1] "abstract://1ebf9fad5f9a05dfb8def571"
```
This implementation sends tasks immediately, and ensures that tasks are evenly-distributed amongst daemons. This means that optimal scheduling is not guaranteed as the duration of tasks cannot be known *a priori*. As an example, tasks could be queued at a daemon behind a long-running task, whilst other daemons are idle having already completed their tasks.

Expand Down Expand Up @@ -266,11 +265,11 @@ m <- mirai(capture.output(str(con)))
m[]
#> [1] "Formal class 'SQLiteConnection' [package \"RSQLite\"] with 8 slots"
#> [2] " ..@ ptr :<externalptr> "
#> [3] " ..@ dbname : chr \"/tmp/Rtmp8RYD0H/file38fb134e0316\""
#> [3] " ..@ dbname : chr \"/tmp/RtmpgPsUPA/file4e907e57a6b6\""
#> [4] " ..@ loadable.extensions: logi TRUE"
#> [5] " ..@ flags : int 70"
#> [6] " ..@ vfs : chr \"\""
#> [7] " ..@ ref :<environment: 0x56e93a2ecdf8> "
#> [7] " ..@ ref :<environment: 0x5fb884ba8ae0> "
#> [8] " ..@ bigint : chr \"integer64\""
#> [9] " ..@ extended_types : logi FALSE"
```
Expand Down Expand Up @@ -371,7 +370,7 @@ status()
#> [1] 0
#>
#> $daemons
#> [1] "tcp://hostname:42771"
#> [1] "tcp://hostname:42761"
```
The number of daemons connecting to the host URL is not limited and network resources may be added or removed at any time, with tasks automatically distributed to all connected daemons.

Expand Down Expand Up @@ -461,10 +460,10 @@ daemons(n = 2, url = host_url())
#> [1] 2
launch_remote(1:2)
#> [1]
#> Rscript -e 'mirai::daemon("tcp://hostname:45023",rs=c(10407,-1977357818,1537914063,-944747324,685206773,301969714,384986891))'
#> Rscript -e 'mirai::daemon("tcp://hostname:34471",rs=c(10407,-1464045629,542467624,-1598368247,142924342,1806531391,-2073238412))'
#>
#> [2]
#> Rscript -e 'mirai::daemon("tcp://hostname:35177",rs=c(10407,-2118558994,-1471211395,-214113251,1708896166,-1342731093,-35397167))'
#> Rscript -e 'mirai::daemon("tcp://hostname:45693",rs=c(10407,-1513314584,1587671761,1092626800,-185579079,11250700,-1550799561))'
daemons(0)
#> [1] 0
```
Expand All @@ -491,37 +490,37 @@ The generated self-signed certificate is available via `launch_remote()`. This f
``` r
launch_remote(1)
#> [1]
#> Rscript -e 'mirai::daemon("wss://hostname:33667/1",tls=c("-----BEGIN CERTIFICATE-----
#> Rscript -e 'mirai::daemon("wss://hostname:35957/1",tls=c("-----BEGIN CERTIFICATE-----
#> MIIFNzCCAx+gAwIBAgIBATANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQDDAhrdW1h
#> bW90bzERMA8GA1UECgwITmFub25leHQxCzAJBgNVBAYTAkpQMB4XDTAxMDEwMTAw
#> MDAwMFoXDTMwMTIzMTIzNTk1OVowMzERMA8GA1UEAwwIa3VtYW1vdG8xETAPBgNV
#> BAoMCE5hbm9uZXh0MQswCQYDVQQGEwJKUDCCAiIwDQYJKoZIhvcNAQEBBQADggIP
#> ADCCAgoCggIBAMwra+feTABrzKCo8WrHmZg/rVUSz1oGNS8V1xNyrGMtdks7+PSu
#> Wo8Zl4BYBUHKEWKMePsa6Xs5fjmxxoBj4GMbGP7y+YLWmZ5GEkjKCj4XqSX3vcgM
#> Iw5iO+KsPnZAFfvYmXt1vXDwklcoHiFYO3A1YKRNrKP0RpIHCnsn2aLlinBw5bpf
#> ZbnaJUcLvg3YM+K5KYvH5JTpE5dnrFMD00por0OGwhT/nUGng5ZAj9Cc/dzFjkDd
#> s+UrHxwFbUfjaeiw4VV3vHr6mJEDWNYCdBrBr0q6kaiZUOJN+5DcUriXx059WfEw
#> oDtJgY+9urVpCMsLlKWg4A14jnAVDGLexp/LxxsgEHIfG+Co95QBKCFLslQgS9PO
#> HVE9bpEtXOOwAHnYzImfu3lR4uroiMWDSN/9XkLG01iIRYeAgSch7k2jDTvR/L6v
#> AywWigfifHX3qGZrqr9uhpPukhXQ2lhN4PG+vLkmowovscIY0l7vDXOsVW+B2APH
#> nZUjmd1tyKI7vvqzMb4gqjLIcETfV8a32ATB9zihRnwq4dzv4PTbv8X3ChYAaVvQ
#> PL3CKXw4FIQGctwbTiSopsp5CfoJ9Dy6vregoe4+33KsMwYV3nGATpGZ+YMb4uf2
#> xEqN/0jrSJmEQqrIfhUxjwBQDuKbEsnmE/jQJTbJTdPneMhkf4wcOBerAgMBAAGj
#> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFHduHiXzvsvbh82mVQTA
#> m3mgEwAsMB8GA1UdIwQYMBaAFHduHiXzvsvbh82mVQTAm3mgEwAsMA0GCSqGSIb3
#> DQEBCwUAA4ICAQA9ONCt0Ms6T9pz8jOiS++aEfYd9rOxAPHjqV0WtEIt5XqgbLmy
#> UBqEho3gW6gPXT99AOzAAF93QHIoSHeaLLskbRsO5AtvhaFUx0D9tkcZCk46uT4i
#> E4vk+H64r/EAaedkowA7c54vwHP+0XnDp8FKWDQaj9UWFC7WPnoYiAqu274YHpP2
#> HDneTf2qb0+ALI52/8EL7YKbsAOpi42x35die9qS2zYbA8CNThDvWsMFCviC8bgM
#> YAnH/5bqrzh9VyFn6EasDk7NXt8Gyadn7avZRWDjbKvEwrBbvmoWUJh0YBShMZSy
#> I+88xHid9WJWJYGNL7fBvVxzfnew8cVU0D20YWK2WN/JMrNKg570fhIGSXW2rNRw
#> x/hZ7a3JdADmUSMV8YRZmLIx48P/X7CgEyhZZVWWxPez0vz6srOOxLvxTod84CDB
#> ApyWcm5GylpkRtMZdX4fkvZtc22Hi21cjZTjL1OU8olRgjycRuxqJI+zBMlepUk+
#> jYGcMp4mczoK+KCqZI9BR5M4hs4nurE2QDaeEW8gC6tM4KlP5O+dI+CjXb8QV6IZ
#> VgfxXVSch+9LpCynaQf+JdomygH35d+XYsps7KjkASJeig9poT7QdDqpWSAyID21
#> RAWLnGaioVrEqp5SDiVEGyh0bX4fmSFWXv1TorfLDNbB1GAm6pGXn8yLjQ==
#> ADCCAgoCggIBAKgWqagLiM5FC5EuvEWA3drnGLa9E5oPUq9ITajT7AYtSE8gW9WZ
#> tIWK631uyeJyxZ8k+D/REkV7dJJONDKZ4csyNgoVSoLlswW7OuqoOxT7wA6laa5b
#> ld8fRKDQkRnhV/qE4YQvPvH1TXUBOqGfg8EdcusN3p17vB2j17RtlnlNSiTGavkn
#> fz8xsNsowJjWLYz05ZMzkiL37cjccDgP0/gG97w4aRSZtaLJSWNs/EKjzKhIe2iI
#> kMFafiG9FABJF5qhh5L0vWdvfncwcWo2wQhvDTXFhzWXPYhwZU0o8CDAfkESIuF7
#> VGNLeQsL9lnFbiJiOc4c1IPovxExl04bLvxQQ++iadgNQYwabXNkYJSNdIm/Txdq
#> SFIhmEvfJf5sQva3md22mDB40zxEo/STaOeLxxDhlqjbMFzN3xLgxCb7AhyIG12e
#> BGWzPuGZBxvzCFqYUd9x/GLmdsA99tGPrEBrNhASmb39X4MjG/cThFyhJfRrr1Ez
#> Aifs5XZjamPXB3JR0kTIt42OuMmxytx3eifjZEalgUpNUlyHIl0PZoTtDSHX/p8X
#> DkG/c0/3faFujkJbPitljqzZyoEAdVHbmEzcDAUkqtAZ9NSXEIs1qrMji8+z9+85
#> t0atsUTk8JpYNUbahg2m6/Nmr0xE/ZRVAzjBw2EMQssCA+5bXZBguxB9AgMBAAGj
#> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFIjQkLX0q5tObwAcmh45
#> +MaTpFpkMB8GA1UdIwQYMBaAFIjQkLX0q5tObwAcmh45+MaTpFpkMA0GCSqGSIb3
#> DQEBCwUAA4ICAQCdXAXqpdnhoPZ6mTjk2ecuqzoZ/+xSRj9D2BAGWsBVHYOUxw6v
#> 5XnosewJop8Jd0GLAs1DjjVkxuBEF1tJiN6qEaoqVOTBoO8t3GL4iu6jb/LmC759
#> BsygcuPQZCTQW70/Fhjp6/Vnklrokfu5FW5oYPdq3EKk4y2CEON0Jkc18i91MH7R
#> yFE/yALuDcNMi411UeB9mZt47mZZ12AyX/m/ftRjCONWwvofnDJq3kQFqF2oHwW/
#> 0KklUQauvpCk4phJuxSNrxFg35dAqhHccFSBFNcqK1HGPLK2lZVQqepa39ls7Zhb
#> fbZqxN0jaJtQ2MzPgOUogejGS0+dWdUZUTdp8h63rthtCM7kQ9FzEf/TmQcbZwIO
#> MVcGQrTPtoP+8VHuClyizHE+Ct7wdqTjMJEmR+dvNQMx1EAWr2kuYZ1XGHsAfDlM
#> ZhIy04rDO8ghT7TImEoi8Z36prDvulETbN0t2RNNXqE4nc/fNEl/WNAx61PeZdZ7
#> 0uktJVR7ekSr6rQILCLYagEuCOhlc7Mvrxf0LIXQCuGmzDxIpaKuC3UbXNBwAGDl
#> THfQzCmb9+tO9EO5nbs9C7CHNhnWtw2sgh+uRlqgg2ps10/QHvL9rQAgcThwmkAT
#> gBVWwO072B2m5H+2BIyi/05hBfW0r0z3b49ay3QspwSLktY6X472jV0GPg==
#> -----END CERTIFICATE-----
#> ",""),rs=c(10407,-190365515,-1460862478,1356191947,1879828880,1039554641,2031211998))'
#> ",""),rs=c(10407,-455422690,750700295,1273478044,1219766253,-2043149878,1421521091))'
```
The printed value may be deployed directly on a remote machine.

Expand Down Expand Up @@ -754,10 +753,10 @@ daemons(4)
vec <- c(1, 1, 4, 4, 1, 1, 1, 1)
system.time(mirai_map(vec, Sys.sleep)[])
#> user system elapsed
#> 0.003 0.003 4.007
#> 0.004 0.002 4.006
system.time(parLapply(cl, vec, Sys.sleep))
#> user system elapsed
#> 0.010 0.004 8.013
#> 0.005 0.007 8.010
```
`.args` is used to specify further constant arguments to `.f` - the 'mean' and 'sd' in the example below:

Expand All @@ -767,13 +766,13 @@ with(
mirai_map(1:3, rnorm, .args = list(mean = 20, sd = 2))[]
)
#> [[1]]
#> [1] 20.27145
#> [1] 21.88944
#>
#> [[2]]
#> [1] 20.55287 22.25358
#> [1] 17.87407 20.08740
#>
#> [[3]]
#> [1] 20.18246 20.24844 19.74873
#> [1] 21.64744 18.48347 16.43075
```
Use `...` to further specify objects referenced but not defined in `.f` - the 'do' in the anonymous function below:

Expand All @@ -785,16 +784,16 @@ ml <- mirai_map(
)
#> Warning: mirai is launching one local daemon for a map operation as none previously set
ml
#> < mirai map [3/3] >
#> < mirai map [1/3] >
ml[]
#> $a
#> [1] "c8"
#> [1] "88"
#>
#> $b
#> [1] 96 66
#> [1] 39 3b
#>
#> $c
#> [1] "6b20bb"
#> [1] "1b5f6c"
```
Use of `mirai_map()` assumes that `daemons()` have previously been set. If not then one (non-dispatcher) daemon is set to allow the function to proceed. This ensures safe behaviour, but is unlikely to be optimal, so please ensure daemons are set beforehand.

Expand Down
14 changes: 7 additions & 7 deletions vignettes/parallel.Rmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ status(cl)
#> [1] 4
#>
#> $daemons
#> [1] "abstract://e755083adb3ac002d81d87c7"
#> [1] "abstract://4fc130b92777dbce706aaaa8"
stop_cluster(cl)
```
Making a cluster specifying 'url' without 'remote' causes the shell commands for manual deployment of nodes to be printed to the console.
Expand All @@ -60,10 +60,10 @@ cl <- make_cluster(n = 2, url = host_url())
#> Shell commands for deployment on nodes:
#>
#> [1]
#> Rscript -e 'mirai::daemon("tcp://hostname:41567",rs=c(10407,-1229812554,-355079233,-704283916,613800293,-1049741598,-1259508485))'
#> Rscript -e 'mirai::daemon("tcp://hostname:38547",rs=c(10407,987529368,-709709383,-455625178,-350041489,-644694556,1812636565))'
#>
#> [2]
#> Rscript -e 'mirai::daemon("tcp://hostname:41567",rs=c(10407,-936602309,482053993,1829342693,1529156475,2009204106,1430761999))'
#> Rscript -e 'mirai::daemon("tcp://hostname:38547",rs=c(10407,2025237564,31519412,1895592412,1818446374,-452272375,1687922752))'
stop_cluster(cl)
```

Expand All @@ -85,10 +85,10 @@ doParallel::registerDoParallel(cl)
m <- matrix(rnorm(9), 3, 3)
foreach(i = 1:nrow(m), .combine = rbind) %dopar%
(m[i, ] / mean(m[i, ]))
#> [,1] [,2] [,3]
#> result.1 0.6168828 1.1209518 1.2621654
#> result.2 1.9631338 0.3970546 0.6398116
#> result.3 1.8112756 8.5974163 -7.4086918
#> [,1] [,2] [,3]
#> result.1 0.5860066 1.0688031 1.3451903
#> result.2 0.6856291 -0.1415699 2.4559407
#> result.3 0.8224936 1.6700118 0.5074946
# simple parallel matrix multiply
a <- matrix(1:16, 4, 4)
b <- t(a)
Expand Down

0 comments on commit 3d48dc8

Please sign in to comment.