Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add examples for BackupService and RestoreService #617

Open
trevorbonas opened this issue Nov 27, 2023 · 3 comments
Open

Add examples for BackupService and RestoreService #617

trevorbonas opened this issue Nov 27, 2023 · 3 comments
Assignees
Labels
good first issue Good for newcomers

Comments

@trevorbonas
Copy link

Proposal:
Add examples demonstrating how to use the BackupService and RestoreService as they are both currently undocumented.

Current behavior:
Examples are missing apart from brief comments in the source code.

Desired behavior:
It would be nice to have examples demonstrating how to use both services, possibly in the examples directory or in the README. Personally, I would like to use BackupService and RestoreService to perform migrations but am unsure how to use these services.

Alternatives considered:
Other alternatives to the BackupService and RestoreService is using the Influx CLI with Python subprocess calls, but this is less testable and not preferred.

Use case:
Performing backups or restores using the InfluxDB Python client would be much easier with documentation.

@bednar
Copy link
Contributor

bednar commented Nov 28, 2023

Hi @trevorbonas,

Thanks for using our client.

Is this something you would be willing to help with? All PR is welcome and we will be happy to review your submission.

Regards

@bednar bednar added the good first issue Good for newcomers label Nov 28, 2023
@trevorbonas
Copy link
Author

@bednar I would love to but:

  • I don't know how to use these services correctly.

  • I get errors with BackupService:
    Platform: macOS Sonoma 14.1.1 Apple M1 Max.
    Python version: 3.11.6.
    influxdb-client-python version: 1.38.0.
    InfluxDB version: OSS 2.7.4.
    Reproducible code:

    from influxdb_client import InfluxDBClient
    from influxdb_client.service import BackupService
    
    # Using the Docker container setup with scripts/influxdb-restart.sh
    # and imported data from the bird migration sample set
    client = InfluxDBClient(url="http://localhost:8086", token="my-token",
        org="my-org")
    metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()

    Error:

    Traceback (most recent call last):
    File "/Users/trevor.bonas/reproduce.py", line 6, in <module>
    metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/service/backup_service.py", line 182, in get_backup_metadata_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 365, in request
    return self.rest_client.GET(url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 268, in GET
    return self.request("GET", url,
           ^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 250, in request
    r.data = r.data.decode('utf8')
             ^^^^^^^^^^^^^^^^^^^^^
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 166: invalid continuation byte

    When the hardcoded value in influxdb_client/_sync/rest.py is changed from utf8 to latin1 then this error occurs:

    Traceback (most recent call last):
    File "/Users/trevor.bonas/reproduce.py", line 6, in <module>
    metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/service/backup_service.py", line 182, in get_backup_metadata_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 343, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 185, in __call_api
    return_data = self.deserialize(response_data, response_type)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 257, in deserialize
    return self.__deserialize(data, response_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 296, in __deserialize
    return self.__deserialize_model(data, klass)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 647, in __deserialize_model
    instance = klass(**kwargs)
               ^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/domain/metadata_backup.py", line 51, in __init__
    self.kv = kv
    ^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/domain/metadata_backup.py", line 72, in kv
    raise ValueError("Invalid value for `kv`, must not be `None`")  # noqa: E501
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ValueError: Invalid value for `kv`, must not be `None`

    Argument values for get_backup_metadata_with_http_info have no effect re: the errors.

Let me know whether my usage is correct, whether you have any tips for getting BackupService working, or whether I should make a separate issue for these errors.

@lenoqt
Copy link

lenoqt commented Mar 13, 2024

@bednar I would love to but:

  • I don't know how to use these services correctly.

  • I get errors with BackupService:
    Platform: macOS Sonoma 14.1.1 Apple M1 Max.
    Python version: 3.11.6.
    influxdb-client-python version: 1.38.0.
    InfluxDB version: OSS 2.7.4.
    Reproducible code:

    from influxdb_client import InfluxDBClient
    from influxdb_client.service import BackupService
    
    # Using the Docker container setup with scripts/influxdb-restart.sh
    # and imported data from the bird migration sample set
    client = InfluxDBClient(url="http://localhost:8086", token="my-token",
        org="my-org")
    metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()

    Error:

    Traceback (most recent call last):
    File "/Users/trevor.bonas/reproduce.py", line 6, in <module>
    metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/service/backup_service.py", line 182, in get_backup_metadata_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 365, in request
    return self.rest_client.GET(url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 268, in GET
    return self.request("GET", url,
           ^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 250, in request
    r.data = r.data.decode('utf8')
             ^^^^^^^^^^^^^^^^^^^^^
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 166: invalid continuation byte

    When the hardcoded value in influxdb_client/_sync/rest.py is changed from utf8 to latin1 then this error occurs:

    Traceback (most recent call last):
    File "/Users/trevor.bonas/reproduce.py", line 6, in <module>
    metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/service/backup_service.py", line 182, in get_backup_metadata_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 343, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 185, in __call_api
    return_data = self.deserialize(response_data, response_type)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 257, in deserialize
    return self.__deserialize(data, response_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 296, in __deserialize
    return self.__deserialize_model(data, klass)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 647, in __deserialize_model
    instance = klass(**kwargs)
               ^^^^^^^^^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/domain/metadata_backup.py", line 51, in __init__
    self.kv = kv
    ^^^^^^^
    File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/domain/metadata_backup.py", line 72, in kv
    raise ValueError("Invalid value for `kv`, must not be `None`")  # noqa: E501
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ValueError: Invalid value for `kv`, must not be `None`

    Argument values for get_backup_metadata_with_http_info have no effect re: the errors.

Let me know whether my usage is correct, whether you have any tips for getting BackupService working, or whether I should make a separate issue for these errors.

The library is badly outdated it seems, I am trying to restore a bucket and using RestoreService.post_restore_bucket_id returns the following error, and then further checking on my server, the endpoint that is pointing to this method, doesn't exists. The other methods for restoring or taking backups also don't work, I switched to the golang version and it looks that one is up to date.

---------------------------------------------------------------------------
ApiException                              Traceback (most recent call last)
Cell In[6], line 1
----> 1 restore_srv.post_restore_bucket_id("927b185ee3412511", "../../backup-bucket/")

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/service/restore_service.py:53, in RestoreService.post_restore_bucket_id(self, bucket_id, body, **kwargs)
     51     return self.post_restore_bucket_id_with_http_info(bucket_id, body, **kwargs)  # noqa: E501
     52 else:
---> 53     (data) = self.post_restore_bucket_id_with_http_info(bucket_id, body, **kwargs)  # noqa: E501
     54     return data

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/service/restore_service.py:76, in RestoreService.post_restore_bucket_id_with_http_info(self, bucket_id, body, **kwargs)
     57 """Overwrite storage metadata for a bucket with shard info from a backup..
     58 
     59 This method makes a synchronous HTTP request by default. To make an
   (...)
     71          returns the request thread.
     72 """  # noqa: E501
     73 local_var_params, path_params, query_params, header_params, body_params = \
     74     self._post_restore_bucket_id_prepare(bucket_id, body, **kwargs)  # noqa: E501
---> 76 return self.api_client.call_api(
     77     '/api/v2/restore/bucket/{bucketID}', 'POST',
     78     path_params,
     79     query_params,
     80     header_params,
     81     body=body_params,
     82     post_params=[],
     83     files={},
     84     response_type='str',  # noqa: E501
     85     auth_settings=[],
     86     async_req=local_var_params.get('async_req'),
     87     _return_http_data_only=local_var_params.get('_return_http_data_only'),  # noqa: E501
     88     _preload_content=local_var_params.get('_preload_content', True),
     89     _request_timeout=local_var_params.get('_request_timeout'),
     90     collection_formats={},
     91     urlopen_kw=kwargs.get('urlopen_kw', None))

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py:343, in ApiClient.call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, async_req, _return_http_data_only, collection_formats, _preload_content, _request_timeout, urlopen_kw)
    304 """Make the HTTP request (synchronous) and Return deserialized data.
    305 
    306 To make an async_req request, set the async_req parameter.
   (...)
    340     then the method will return the response directly.
    341 """
    342 if not async_req:
--> 343     return self.__call_api(resource_path, method,
    344                            path_params, query_params, header_params,
    345                            body, post_params, files,
    346                            response_type, auth_settings,
    347                            _return_http_data_only, collection_formats,
    348                            _preload_content, _request_timeout, urlopen_kw)
    349 else:
    350     thread = self.pool.apply_async(self.__call_api, (resource_path,
    351                                    method, path_params, query_params,
    352                                    header_params, body,
   (...)
    356                                    collection_formats,
    357                                    _preload_content, _request_timeout, urlopen_kw))

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py:173, in ApiClient.__call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, _return_http_data_only, collection_formats, _preload_content, _request_timeout, urlopen_kw)
    170 urlopen_kw = urlopen_kw or {}
    172 # perform request and return response
--> 173 response_data = self.request(
    174     method, url, query_params=query_params, headers=header_params,
    175     post_params=post_params, body=body,
    176     _preload_content=_preload_content,
    177     _request_timeout=_request_timeout, **urlopen_kw)
    179 self.last_response = response_data
    181 return_data = response_data

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py:388, in ApiClient.request(self, method, url, query_params, headers, post_params, body, _preload_content, _request_timeout, **urlopen_kw)
    379     return self.rest_client.OPTIONS(url,
    380                                     query_params=query_params,
    381                                     headers=headers,
   (...)
    385                                     body=body,
    386                                     **urlopen_kw)
    387 elif method == "POST":
--> 388     return self.rest_client.POST(url,
    389                                  query_params=query_params,
    390                                  headers=headers,
    391                                  post_params=post_params,
    392                                  _preload_content=_preload_content,
    393                                  _request_timeout=_request_timeout,
    394                                  body=body,
    395                                  **urlopen_kw)
    396 elif method == "PUT":
    397     return self.rest_client.PUT(url,
    398                                 query_params=query_params,
    399                                 headers=headers,
   (...)
    403                                 body=body,
    404                                 **urlopen_kw)

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/rest.py:311, in RESTClientObject.POST(self, url, headers, query_params, post_params, body, _preload_content, _request_timeout, **urlopen_kw)
    308 def POST(self, url, headers=None, query_params=None, post_params=None,
    309          body=None, _preload_content=True, _request_timeout=None, **urlopen_kw):
    310     """Perform POST HTTP request."""
--> 311     return self.request("POST", url,
    312                         headers=headers,
    313                         query_params=query_params,
    314                         post_params=post_params,
    315                         _preload_content=_preload_content,
    316                         _request_timeout=_request_timeout,
    317                         body=body,
    318                         **urlopen_kw)

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/rest.py:261, in RESTClientObject.request(self, method, url, query_params, headers, body, post_params, _preload_content, _request_timeout, **urlopen_kw)
    258     _BaseRESTClient.log_body(r.data, '<<<')
    260 if not 200 <= r.status <= 299:
--> 261     raise ApiException(http_resp=r)
    263 return r

ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json; charset=utf-8', 'X-Influxdb-Build': 'OSS', 'X-Influxdb-Version': 'v2.7.5', 'X-Platform-Error-Code': 'not found', 'Date': 'Wed, 13 Mar 2024 12:35:42 GMT', 'Content-Length': '47'})
HTTP response body: {"code":"not found","message":"path not found"}

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

3 participants