single API call to receive all deployed systems info

AirVantage M2M Cloud APIs usage: build your server application or integrate into your ERP

single API call to receive all deployed systems info

Postby aramezani » Fri Sep 29, 2017 7:53 am

We have many devices connected to Airvantage cloud, and at the moment the following API call returns MAX 100 items at a time which forces us to do a multiple API calls ( ~ 1000 times) with offsets to obtain all systems info:

/api/v1/systems?

I was wondering if the same can be done by just using a single API call and return all systems info with a single shot?

Thanks!
aramezani
 
Posts: 5
Joined: Mon Sep 04, 2017 7:03 am
Has thanked: 2 times
Have thanks: 0 time

Re: single API call to receive all deployed systems info

Postby rjacolin » Fri Sep 29, 2017 9:39 am

Hello,

we don't supply an API which returns all systems info with a single call in json. The closest one is the /v1/operations/systems/export API which creates an operation in order to export systems information in a CSV file.

Why a pagination system?
The pagination mechanism is required because:
  • a UI can not handle 10 000 results in a single call. Moreover showing 10 000 results doesn't make sense for a user. The user will filter the result or needs to aggregate the information to be readable and understable.
  • Security/scalability: Most of the time, the users just read the first page before sorting, filtering the results. It will consume a lot of resources in order to return the information and will impact other users (or blocks the platform by doing a single request to get information about 10 000 000 systems).
  • If your application has to process the results, use the export API. As it is asynchronous, the impact on the platform is lower.

The pagination is a best practice when building a REST APIs. For example, you can have a look to this page which describes the best practices for REST API: https://hackernoon.com/restful-api-desi ... e1d954e7c9

Can you give us more details about your processing in order to be able to give you more information? What is the purpose of your API call (UI, processing, ...)? What is the call frequency?

Regards.

For this post the author rjacolin
has been thanked by :
aramezani (Fri Sep 29, 2017 3:14 pm)
rjacolin
 
Posts: 51
Joined: Tue Jun 19, 2012 6:29 am
Location: Toulouse
Has thanked: 0 time
Have thanks: 17 times

Re: single API call to receive all deployed systems info

Postby aramezani » Fri Sep 29, 2017 12:31 pm

rjacolin wrote:
Can you give us more details about your processing in order to be able to give you more information? What is the purpose of your API call (UI, processing, ...)? What is the call frequency?

Regards.


Thanks.

In fact we would like to perform the automatic fleet managment and avoid interacting with GUI and make direct API calls using a Python client.

Hence, we would like to have information such as the current firmware version of all of our "nodes" in a single shot, avoiding repetetive calls, and if necessary upgrade a subset of the nodes with e.g. the current firmware. Note that the firmware upgarde is just a use-case.
aramezani
 
Posts: 5
Joined: Mon Sep 04, 2017 7:03 am
Has thanked: 2 times
Have thanks: 0 time

Re: single API call to receive all deployed systems info

Postby aramezani » Fri Sep 29, 2017 3:08 pm

rjacolin wrote:
we don't supply an API which returns all systems info with a single call in json. The closest one is the /v1/operations/systems/export API which creates an operation in order to export systems information in a CSV file.

[*] If your application has to process the results, use the export API. As it is asynchronous, the impact on the platform is lower.[/list]


Also I have noticed that you need to provide the label parameter to /v1/operations/systems/export, which makes its general use somehow limited to a subset of nodes specified by a label.
aramezani
 
Posts: 5
Joined: Mon Sep 04, 2017 7:03 am
Has thanked: 2 times
Have thanks: 0 time

Re: single API call to receive all deployed systems info

Postby rjacolin » Fri Oct 06, 2017 12:35 pm

Hello,

the best way is to use the export API with a label which has been applied on all the fleet (and which can be removed right after the operation has been done).

I will create a ticket improvement in our internal ticketing system in order to improve how an application can filter (or not) in order to create such operation (for example, export systems without a label).

Regards,
Robert
rjacolin
 
Posts: 51
Joined: Tue Jun 19, 2012 6:29 am
Location: Toulouse
Has thanked: 0 time
Have thanks: 17 times

Re: single API call to receive all deployed systems info

Postby aramezani » Fri Oct 06, 2017 2:21 pm

Thanks.

Now, if I want to retrieve certain data points from all of our systems (e.g. the firmware version of our systems, etc..), I assume that I have to use the following API call:

/v1/systems/data/fleet

However, in our case which have more than 200k systems out of which we are interested just in the active systems (~100k).

Hence, firstly how can I filter just the active systems and then how the pagination is being handled in this case (I assume data points from MAX 500 systems will be retuned at a time but the offsets are not defined)?
aramezani
 
Posts: 5
Joined: Mon Sep 04, 2017 7:03 am
Has thanked: 2 times
Have thanks: 0 time

Re: single API call to receive all deployed systems info

Postby rjacolin » Fri Oct 06, 2017 3:00 pm

You can get the last values from the export API (/v1/operations/systems/export)
You can:
  • "fields" : ["Name", "State", "Metadata", ... ] in order to get system information (FW version). See the full list in the top System lleft tab, "Fields" tab: https://doc.airvantage.net/av/reference ... iew-Fields
  • "dataset" : "dataset_uid": define your list of data in a dataset (Configure > Report) and use the report uuid (report and data set are different names for the same thing). Report uuid can be get from the url when you display the report
  • "data" : ["data_id", ... ]: list the data uuid you want to be exported

In my opinion, you may use 1 or 2 dataset it will be more convenient.

By the way, /v1/systems/data/fleet gives you the list of aggregated data. Aggregated data are data computed with a mathematical function (minimum, maximum, mean, count, sum, ...) on a specific period (hour, day, month, year). For example, you can get the mean value of a temperature of the current month. I don't think it is what you want.

Regards,
Robert
rjacolin
 
Posts: 51
Joined: Tue Jun 19, 2012 6:29 am
Location: Toulouse
Has thanked: 0 time
Have thanks: 17 times

Re: single API call to receive all deployed systems info

Postby aramezani » Fri Oct 06, 2017 3:19 pm

Notes taken!

How can I get the exported CSV output for post-processing?

I noticed that when I POST, the success of operation just will be returned (sorry I am not an expert).

Thanks!
aramezani
 
Posts: 5
Joined: Mon Sep 04, 2017 7:03 am
Has thanked: 2 times
Have thanks: 0 time

Re: single API call to receive all deployed systems info

Postby rjacolin » Fri Oct 06, 2017 3:25 pm

Indeed, this API creates an operation. The operation will be executed. You can use two ways: in the export API you can define a callback to be called when the operation state changed and so get operation uuid in order to get the file URL. See Operations section, for Operations API: https://doc.airvantage.net/av/reference ... ration-v1/
The API you have to call is /v1/operations/{uid}/result in order to download the file.

You can poll the operation detail until the operation is success.

Regards,
Robert

For this post the author rjacolin
has been thanked by :
aramezani (Mon Oct 09, 2017 12:51 pm)
rjacolin
 
Posts: 51
Joined: Tue Jun 19, 2012 6:29 am
Location: Toulouse
Has thanked: 0 time
Have thanks: 17 times


Return to AirVantage Enterprise Platform

Who is online

Users browsing this forum: No registered users and 1 guest