Optimized API call, query timing for the table aws_ec2_instance_type and added support to accept the wildcard pattern(t2*) for instance type. Closes #2292

[ParthaI]

Integration test logs

Logs

N/A

Example query results

Results

> select instance_type, instance_type_pattern from aws_ec2_instance_type where instance_type_pattern = 'c7i.*'
+----------------+-----------------------+
| instance_type  | instance_type_pattern |
+----------------+-----------------------+
| c7i.16xlarge   | c7i.*                 |
| c7i.xlarge     | c7i.*                 |
| c7i.large      | c7i.*                 |
| c7i.metal-48xl | c7i.*                 |
| c7i.4xlarge    | c7i.*                 |
| c7i.2xlarge    | c7i.*                 |

[ParthaI]

Changes Overview:

API Call Optimization:

• The table now makes 2 API calls per query, compared to earlier where it made API calls for each row.
• Query execution time has been reduced by ~75%.

---

Performance Comparison:

Earlier (73.8s):

> select count(*) from aws_ec2_instance_type;
+-------+
| count |
+-------+
| 11403 |
+-------+

Time: 73.8s. Rows returned: 1. Rows fetched: 11,380. Hydrate calls: 11,403.

Now (12.2s):

> select count(*) from aws_ec2_instance_type;
+-------+
| count |
+-------+
| 11403 |
+-------+

Time: 12.2s. Rows returned: 1. Rows fetched: 11,403. Hydrate calls: 0.

---

Wildcard Search Support:

• Added a new column: instance_type_pattern to enable wildcard pattern searches.
• In the query select * from aws_ec2_instance_type where instance_type = 't2*', the API returned empty rows due to PostgreSQL-level filtering.
  • The instance_type column contains exact values like 't2.small', not 't2*', causing a mismatch between the column value and the wildcard pattern.
  • To address this, the instance_type_pattern column was introduced to allow proper filtering using wildcard patterns.

Example Query:

> select instance_type, instance_type_pattern from aws_ec2_instance_type where instance_type_pattern = 'c7i.*';
+----------------+-----------------------+
| instance_type  | instance_type_pattern |
+----------------+-----------------------+
| c7i.16xlarge   | c7i.*                 |
| c7i.xlarge     | c7i.*                 |
| c7i.large      | c7i.*                 |
| c7i.metal-48xl | c7i.*                 |
| c7i.4xlarge    | c7i.*                 |
| c7i.2xlarge    | c7i.*                 |
| c7i.metal-24xl | c7i.*                 |
| c7i.8xlarge    | c7i.*                 |
| c7i.12xlarge   | c7i.*                 |


Time: 1.3s. Rows returned: 176. Rows fetched: 176. Hydrate calls: 0.

Scans:
  1) aws_ec2_instance_type.aws: Time: 1.2s. Fetched: 176. Hydrates: 0. Quals: instance_type_pattern = 'c7i.*'.

The above query fetched and displayed 176 rows in just 1.3 seconds.

---

@e-gineer / @misraved, your feedback on the table design and suggestions regarding the column naming for instance_type_pattern would be greatly appreciated.

Thank you!

[github-actions]

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[misraved]

@ParthaI what happens if we pass limit 15 to the query? In an ideal case scenario, I wouldn't want to make a batch API call, that passes in 100 instance types.

Please take a look at the minor review comments, thanks!!

[misraved]

Suggested change

plugin.Logger(ctx).Error("Pattern =>>>", pattern)

[misraved]

Suggested change

plugin.Logger(ctx).Error("Pattern matched with =>>>", instanceType)

[ParthaI]

@misraved, thank you for the suggestion! I’ve updated the logic to optimize the API call using the limit query parameter.