Skip to content

Commit

Permalink
simplifies status()
Browse files Browse the repository at this point in the history
  • Loading branch information
shikokuchuo committed Aug 4, 2023
1 parent 9d6ac53 commit c1dd3ff
Show file tree
Hide file tree
Showing 7 changed files with 76 additions and 71 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ BugReports: https://github.com/shikokuchuo/mirai/issues
URL: https://shikokuchuo.net/mirai/, https://github.com/shikokuchuo/mirai/
Encoding: UTF-8
Depends:
R (>= 2.12)
R (>= 3.5)
Imports:
nanonext (>= 0.9.1)
RoxygenNote: 7.2.3
7 changes: 5 additions & 2 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,11 @@
+ Straight pass through without serialization/unserialization allows higher performance and lower memory utilisation.
+ Fixes edge cases of `status()` occasionally failing to communicate with dispatcher.
* `saisei()` with argument `force = TRUE` now immediately regenerates the socket and returns any ongoing mirai as an 'errorValue'. This allows tasks that consistently hang or crash to be cancelled rather than being repeated when a new daemon connects.
* `status()` now returns the daemons status matrix with the column 'i' for ease of use with other functions such as `saisei()` or `launch_local()` etc. that require this.
* Error messages tweaked to be more accurate and informative.
* Tweaks to `status():`
+ The daemons status matrix adds a column 'i' for ease of use with other functions such as `saisei()` or `launch_local()` etc.
+ `$daemons` now returns NULL if daemons have not been set.
* Certain error messages are more accurate and informative.
* Requires nanonext >= [0.9.1], and consequently R >= 3.5.

# mirai 0.9.1

Expand Down
21 changes: 11 additions & 10 deletions R/mirai.R
Original file line number Diff line number Diff line change
Expand Up @@ -866,13 +866,14 @@ saisei <- function(i = 1L, force = FALSE, .compute = "default") {
#'
#' @return A named list comprising:
#' \itemize{
#' \item{\strong{connections}} {- number of active connections. Always 1L
#' when using dispatcher as there is a single connection to the dispatcher,
#' which in turn connects to the daemons.}
#' \item{\strong{daemons}} {- if using dispatcher: a status matrix (see
#' Status Matrix section below), or else an integer 'errorValue' if
#' communication with dispatcher was unsuccessful. If not using
#' dispatcher: the character host URL. If daemons are not set: 0L.}
#' \item{\strong{connections}} {- integer number of active connections.
#' \cr Using dispatcher: Always 1L as there is a single connection to
#' dispatcher, which connects to the daemons in turn.}
#' \item{\strong{daemons}} {- of variable type.
#' \cr Using dispatcher: a status matrix (see Status Matrix section below),
#' or else an integer 'errorValue' if communication with dispatcher failed.
#' \cr Not using dispatcher: the character host URL.
#' \cr Not set: NULL.}
#' }
#'
#' @section Status Matrix:
Expand Down Expand Up @@ -911,9 +912,9 @@ saisei <- function(i = 1L, force = FALSE, .compute = "default") {
status <- function(.compute = "default") {

envir <- ..[[.compute]]
active <- length(envir[["sock"]])
list(connections = if (active) stat(envir[["sock"]], "pipes") else 0L,
daemons = if (length(envir[["sockc"]])) query_status(envir) else if (active) envir[["urls"]] else 0L)
sock <- envir[["sock"]]
list(connections = if (is.null(sock)) 0L else as.integer(stat(sock, "pipes")),
daemons = if (is.null(envir[["sockc"]])) envir[["urls"]] else query_status(envir))

}

Expand Down
26 changes: 13 additions & 13 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ The daemons interface may also be used to send tasks for computation to remote d

Call `daemons()` specifying 'url' as a character value the host network address and a port that is able to accept incoming connections.

The examples below use an illustrative local network IP address of '10.216.62.86'.
The examples below use an illustrative local network IP address of '10.75.34.138'.

A port on the host machine also needs to be open and available for inbound connections from the local network, illustratively '5555' in the examples below.

Expand All @@ -293,15 +293,15 @@ The default `dispatcher = TRUE` creates a background `dispatcher()` process on t
It is recommended to use a websocket URL starting `ws://` instead of TCP in this scenario (used interchangeably with `tcp://`). A websocket URL supports a path after the port number, which can be made unique for each daemon. In this way a dispatcher can connect to an arbitrary number of daemons over a single port.

```{r localqueue}
daemons(n = 4, url = "ws://10.216.62.86:5555")
daemons(n = 4, url = "ws://10.75.34.138:5555")
```

Above, a single URL was supplied, along with `n = 4` to specify that the dispatcher should listen at 4 URLs. In such a case, an integer sequence is automatically appended to the path `/1` through `/4` to produce these URLs.

Alternatively, supplying a vector of URLs allows the use of arbitrary port numbers / paths, e.g.:

```{r vectorqueue, eval=FALSE}
daemons(url = c("ws://10.216.62.86:5566/cpu", "ws://10.216.62.86:5566/gpu", "ws://10.216.62.86:7788/1"))
daemons(url = c("ws://10.75.34.138:5566/cpu", "ws://10.75.34.138:5566/gpu", "ws://10.75.34.138:7788/1"))
```

Above, 'n' is not specified, in which case its value is inferred from the length of the 'url' vector supplied.
Expand All @@ -311,10 +311,10 @@ Above, 'n' is not specified, in which case its value is inferred from the length
On the remote resource, `daemon()` may be called from an R session, or directly from a shell using Rscript. Each daemon instance should dial into one of the unique URLs that the dispatcher is listening at:

```
Rscript -e 'mirai::daemon("ws://10.216.62.86:5555/1")'
Rscript -e 'mirai::daemon("ws://10.216.62.86:5555/2")'
Rscript -e 'mirai::daemon("ws://10.216.62.86:5555/3")'
Rscript -e 'mirai::daemon("ws://10.216.62.86:5555/4")'
Rscript -e 'mirai::daemon("ws://10.75.34.138:5555/1")'
Rscript -e 'mirai::daemon("ws://10.75.34.138:5555/2")'
Rscript -e 'mirai::daemon("ws://10.75.34.138:5555/3")'
Rscript -e 'mirai::daemon("ws://10.75.34.138:5555/4")'
```

Expand All @@ -323,7 +323,7 @@ Note that `daemons()` should be set up on the host machine before launching `dae
`launch_remote()` may also be used to launch daemons directly on a remote machine. For example, if the remote machine at 10.216.62.100 accepts SSH connections over port 22:

```{r launchremote, eval=FALSE}
launch_remote(1:4, command = "ssh", args = c("-p 22 10.216.62.100", .))
launch_remote(1:4, command = "ssh", args = c("-p 22 10.75.34.138", .))
```
```{r launchremotereal, echo=FALSE}
launch_remote(1:4)
Expand Down Expand Up @@ -362,7 +362,7 @@ Closing the connection causes the dispatcher to exit automatically, and in turn
By specifying `dispatcher = FALSE`, remote daemons connect directly to the host process. The host listens at a single URL, and distributes tasks to all connected daemons.

```{r remote, eval=FALSE}
daemons(url = "tcp://10.216.62.86:0", dispatcher = FALSE)
daemons(url = "tcp://10.75.34.138:0", dispatcher = FALSE)
```

Alternatively, simply supply a colon followed by the port number to listen on all interfaces on the local host, for example:
Expand All @@ -378,17 +378,17 @@ Note that above, the port number is specified as zero. This is a wildcard value
On the network resource, `daemon()` may be called from an R session, or an Rscript invocation from a shell. This sets up a remote daemon process that connects to the host URL and receives tasks:

```
Rscript -e 'mirai::daemon("tcp://10.216.62.86:0")'
Rscript -e 'mirai::daemon("tcp://10.75.34.138:0")'
```
Note that `daemons()` should be set up on the host machine before launching `daemon()` on remote resources, otherwise the daemon instances will exit if a connection is not immediately available. Alternatively, specifying `daemon(asyncdial = TRUE)` will allow daemons to wait (indefinitely) for a connection to become available.

`launch_remote()` may also be used to launch daemons directly on a remote machine. For example, if the remote machine at 10.216.62.100 accepts SSH connections over port 22:
`launch_remote()` may also be used to launch daemons directly on a remote machine. For example, if the remote machine at 10.75.34.100 accepts SSH connections over port 22:

```{r launchremoteu, eval=FALSE}
launch_remote("tcp://10.216.62.86:0", command = "ssh", args = c("-p 22 10.216.62.100", .))
launch_remote("tcp://10.75.34.138:0", command = "ssh", args = c("-p 22 10.75.34.100", .))
```
```{r launchremoteureal, echo=FALSE}
launch_remote("tcp://10.216.62.86:0")
launch_remote("tcp://10.75.34.138:0")
```

The returned vector comprises the shell commands executed on the remote machine.
Expand Down
Loading

0 comments on commit c1dd3ff

Please sign in to comment.