Can't access VM with a network that has subnetworks

[ozanstats]

I can't access an existing VM as gce_vm() ignores the network argument and uses its default:

vm = gce_vm(
  name = "myvm",
  network = "mynetwork",
  username = "<username>",
  password = "<password>",
)
# YYYY-MM-DD HH:MM:SS> Creating http firewall rule
# YYYY-MM-DD HH:MM:SS> Request Status Code: 404
# Error: API returned: The resource 'projects/<project>/global/networks/default' was not found

This could be related to #110 as "mynetwork" has subnetworks:

gce_list_networks()
$id
[1] "projects/<project>/global/networks/"

$items
   id creationTimestamp      name                                                                           selfLink autoCreateSubnetworks
1 xxx               xxx mynetwork https://www.googleapis.com/compute/v1/projects/<project>/global/networks/mynetwork                 FALSE

                                                                                       subnetworks
1 https://www.googleapis.com/compute/v1/projects/<project>/regions/<region>/subnetworks/<subnetwork>,
https://www.googleapis.com/compute/v1/projects/<project>/regions/<region>/subnetworks/<subnetwork>,
https://www.googleapis.com/compute/v1/projects/<project>/regions/<region>/subnetworks/<subnetwork>

  routingMode            kind
1      GLOBAL compute#network

$selfLink
[1] "https://www.googleapis.com/compute/v1/projects/<project>/global/networks/"

$kind
[1] "compute#networkList"

[ozanstats]

Following up this issue, I can't create a new instance either:

vm = gce_vm(
  name = "<newinstance>",
  username = "<username>",
  password = "<password>",
)
# Error in value[[3L]](cond) : argument is missing, with no default

vm = gce_vm(
  name = "<newinstance>",
  network = "mynetwork",
  username = "<username>",
  password = "<password>",
)
# Error in value[[3L]](cond) : argument is missing, with no default

[MarkEdmondson1234]

This should be supported now by a new function gce_make_network() which you can pass to the VM creation functions. The default is gce_make_network("default")

Could you download the latest version from GitHub and see if it works with you?

[MarkEdmondson1234]

Note that username and password are only relevant if you are launching an RStudio Server template.

[ozanstats]

Thank you for taking the time to work on this. Here is my attempt to use gce_make_network():

## works fine when starting a VM that was not running
vm = gce_vm(
  name = "my-existing-vm",
  network = gce_make_network(
    network = "my-network",
    subnetwork = "my-subnetwork"
  )
)
# YYYY-MM-DD HH:MM:SS>  VM previously created but not running, starting VM
# YYYY-MM-DD HH:MM:SS>  Operation running...
# YYYY-MM-DD HH:MM:SS>  Operation running...
# YYYY-MM-DD HH:MM:SS>  Operation complete in 4 secs

## errors when trying to get a running VM
vm = gce_vm(
  name = "my-existing-vm",
  # excluding network doesn't change the outcome
  network = gce_make_network(
    network = "my-network",
    subnetwork = "my-subnetwork"
  )
)
# YYYY-MM-DD HH:MM:SS>  Creating http firewall rule
# YYYY-MM-DD HH:MM:SS>  Request Status Code: 404
# Error: API returned:  The resource 'projects/<project>/global/networks/default' was not found

## errors when trying to create a new VM with subnetwork
vm = gce_vm(
  name = "my-new-vm",
  network = gce_make_network(
    network = "my-network",
    subnetwork = "my-subnetwork"
  ),
  predefined_type = "n1-standard-8"
)
# YYYY-MM-DD HH:MM:SS>  Creating standard VM
# YYYY-MM-DD HH:MM:SS>  Request Status Code: 400
# Error: API returned:  Invalid value for field 'resource.networkInterfaces[0].subnetwork': 'my-subnetwork'. The URL is malformed.

[MarkEdmondson1234]

Could you rerun with options(googleAuthR.verbose = 2) to get some more logging then I can examine where the new JSON should go.

…

________________________________ From: Ozan Adıgüzel <[email protected]> Sent: Monday, February 3, 2020 7:17 PM To: cloudyr/googleComputeEngineR Cc: Mark; Assign Subject: Re: [cloudyr/googleComputeEngineR] Can't access VM with a network that has subnetworks (#154) Thank you for taking the time to work on this. Here is my attempt to use gce_make_network(): ## works fine when starting a VM that was not running vm = gce_vm( name = "my-existing-vm", network = gce_make_network( network = "my-network", subnetwork = "my-subnetwork" ) ) # YYYY-MM-DD HH:MM:SS> VM previously created but not running, starting VM # YYYY-MM-DD HH:MM:SS> Operation running... # YYYY-MM-DD HH:MM:SS> Operation running... # YYYY-MM-DD HH:MM:SS> Operation complete in 4 secs ## errors when trying to get a running VM vm = gce_vm( name = "my-existing-vm", # excluding network doesn't change the outcome network = gce_make_network( network = "my-network", subnetwork = "my-subnetwork" ) ) # YYYY-MM-DD HH:MM:SS> Creating http firewall rule # YYYY-MM-DD HH:MM:SS> Request Status Code: 404 # Error: API returned: The resource 'projects/<project>/global/networks/default' was not found ## errors when trying to create a new VM with subnetwork vm = gce_vm( name = "my-new-vm", network = gce_make_network( network = "my-network", subnetwork = "my-subnetwork" ), predefined_type = "n1-standard-8" ) # YYYY-MM-DD HH:MM:SS> Creating standard VM # YYYY-MM-DD HH:MM:SS> Request Status Code: 400 # Error: API returned: Invalid value for field 'resource.networkInterfaces[0].subnetwork': 'my-subnetwork'. The URL is malformed. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#154?email_source=notifications&email_token=AAYCPLB4M3HVNMXTK2LC3ITRBBNUFA5CNFSM4KOME4P2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKU3FBA#issuecomment-581546628>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAYCPLHS4JBZULC6VKQX7DTRBBNUFANCNFSM4KOME4PQ>.

[ozanstats]

options(googleAuthR.verbose = 2)

## works fine when starting a VM that was not running
vm = gce_vm(
  name = "my-existing-vm",
  network = gce_make_network(
    network = "my-network",
    subnetwork = "my-subnetwork"
  )
)
# 2020-02-03 12:42:33> Token exists.
# 2020-02-03 12:42:33> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/instances/?filter=status%20eq%20TERMINATED
# 2020-02-03 12:42:33> VM previously created but not running, starting VM
# 2020-02-03 12:42:33> Token exists.
# 2020-02-03 12:42:33> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/instances/<instance>/start/
# 2020-02-03 12:42:34> Starting operation...
# 2020-02-03 12:42:34> Token exists.
# 2020-02-03 12:42:34> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/operations/<operation>/
# 2020-02-03 12:42:34> Operation running...
# 2020-02-03 12:42:37> Token exists.
# 2020-02-03 12:42:37> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/operations/<operation>/
# 2020-02-03 12:42:38> Operation running...
# 2020-02-03 12:42:41> Token exists.
# 2020-02-03 12:42:41> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/operations/<operation>/
# 2020-02-03 12:42:44> Operation complete in 6 secs
# 2020-02-03 12:42:44> Token exists.
# 2020-02-03 12:42:44> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/instances/<instance>/

## errors when trying to get a running VM
vm = gce_vm(
  name = "my-existing-vm",
  # excluding network doesn't change the outcome
  network = gce_make_network(
    network = "my-network",
    subnetwork = "my-subnetwork"
  )
)
# 2020-02-03 12:47:36> Token exists.
# 2020-02-03 12:47:36> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/instances/?filter=status%20eq%20TERMINATED
# 2020-02-03 12:47:37> Token exists.
# 2020-02-03 12:47:37> Request: https://www.googleapis.com/compute/v1/projects/<project>/global/firewalls/?
# 2020-02-03 12:47:37> Creating http firewall rule
# 2020-02-03 12:47:37> Token exists.
# 2020-02-03 12:47:37> Request: https://www.googleapis.com/compute/v1/projects/<project>/global/firewalls/
# 2020-02-03 12:47:37> Body JSON parsed to: {"name":"allow-http","allowed":[{"IPProtocol":"tcp","ports":[80]}],"sourceRanges":"0.0.0.0/0","sourceTags":{}}
# 2020-02-03 12:47:37> Request Status Code: 404
# 2020-02-03 12:47:37> API returned error: The resource 'projects/<project>/global/networks/default' was not found
# 2020-02-03 12:47:37> No retry attempted: The resource 'projects/<project>/global/networks/default' was not found
# Error: API returned: The resource 'projects/<project>/global/networks/default' was not found

## errors when trying to create a new VM with subnetwork
vm = gce_vm(
  name = "my-new-vm",
  network = gce_make_network(
    network = "my-network",
    subnetwork = "my-subnetwork"
  ),
  predefined_type = "n1-standard-8"
)
# 2020-02-03 12:49:49> Token exists.
# 2020-02-03 12:49:49> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/instances/?filter=status%20eq%20TERMINATED
# 2020-02-03 12:49:49> Token exists.
# 2020-02-03 12:49:49> Request: https://www.googleapis.com/compute/v1/projects/<project>/global/networks/my-network/
# 2020-02-03 12:49:49> Creating standard VM
# 2020-02-03 12:49:49> Token exists.
# 2020-02-03 12:49:49> Request: https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/family/debian-9/
# 2020-02-03 12:49:50> Token exists.
# 2020-02-03 12:49:50> Request: https://www.googleapis.com/compute/v1/projects/<project>/zones/<zone>/instances/
# 2020-02-03 12:49:50> Body JSON parsed to: {"machineType":"zones/<zone>/machineTypes/n1-standard-8","name":"my-new-vm","disks":[{"initializeParams":{"sourceImage":"projects/debian-cloud/global/images/debian-9-stretch-v20191210"},"autoDelete":true,"boot":true,"type":"PERSISTENT","deviceName":"my-new-vm-boot-disk"}],"networkInterfaces":[{"network":"https://www.googleapis.com/compute/v1/projects/<project>/global/networks/my-network","subnetwork":"my-subnetwork","accessConfigs":[{"type":"ONE_TO_ONE_NAT"}]}],"serviceAccounts":[{"email":"project-service-account@<project>.iam.gserviceaccount.com","scopes":["https://www.googleapis.com/auth/cloud-platform"]}]}
# 2020-02-03 12:49:50> Request Status Code: 400
# 2020-02-03 12:49:50> API returned error: Invalid value for field 'resource.networkInterfaces[0].subnetwork': 'my-subnetwork'. The URL is malformed.
# 2020-02-03 12:49:50> No retry attempted: Invalid value for field 'resource.networkInterfaces[0].subnetwork': 'my-subnetwork'. The URL is malformed.
# Error: API returned: Invalid value for field 'resource.networkInterfaces[0].subnetwork': 'my-subnetwork'. The URL is malformed.

[mc51]

I had the same issue. Setting network = gce_make_network("mynetwork", subnetwork = "mysubnet") didn't resolve the issue at first. This is because per default gce_vm(...) has open_webports = T. This leads to this error message:

2020-10-12 10:46:06> Creating http firewall rule
i 2020-10-12 10:46:06 > Request Status Code:  404
Error: API returned: The resource 'projects/aaa-my-project/global/networks/default' was not found

It seems, that when creating the firewall rules, the correct network is not taken into account. Instead, it tries to create it for the "default" network, which does not exist in my case.