Missing incidents with display_type=9 on map

[kamieniarz]

There's a case when showcases with no details are not displayed on map. So basically pokestop has in db display_type=9, expiration is in future but map displays nothing on that pokestop proly cause it holds expired showcase (showcase_expiry) from the past. In fact there should be a logic that displays incident instead of showcase when incident.expiration>pokestop.showcase_expiry and incident.display_type=9

[TurtIeSocks]

This seems like an issue for Golbat.

[Fabio1988]

Seems to be an edge case, as golbat cleans incidents when expired every few minutes/hours?
And we know that showcase is hiding other incidents at the same time

[Fabio1988]

Now as I followed your discussion on the server I know what you mean:

• showcase was looked up in the past with details
• scout sent to that location today (it did not check for details)
• actual incident (type=9= showcase) wasn't shown in RM, as it was filtered because of old expiration date, not updated because details were no asked for

@kamieniarz can you pls post db row? Imo thr GMO contains the expiry date, without lookup

[kamieniarz]

There is expiration date without lookup and it's been like that proly since forever

Here's pokestop table

+-------------------------------------+-------------------+-------------------+-------------------------+------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------+------------+---------+------------+-----------------+--------------+------------------+-------------------------------------------------------------------------------------------------+-------------------------------------+------------------------------+---------------------+---------+---------+----------------------+------------+------------+------------------+----------------+-----------------+------------------------+------------------------+-----------------------------+--------------------------+------------------------------+----------------------------------------------+----------------------------+-----------------------------+--------------+--------------------------+-------------+-------------------+---------------+---------------------+------------------+------------------------------+-------------------------------+---------------------------+---------------------------------+---------------------+--------------------------+--------------------------+---------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                  | lat               | lon               | name                    | url                                                                                                                          | lure_expire_timestamp | last_modified_timestamp | updated    | enabled | quest_type | quest_timestamp | quest_target | quest_conditions | quest_rewards                                                                                   | quest_template                      | quest_title                  | cell_id             | deleted | lure_id | first_seen_timestamp | sponsor_id | partner_id | ar_scan_eligible | power_up_level | power_up_points | power_up_end_timestamp | alternative_quest_type | alternative_quest_timestamp | alternative_quest_target | alternative_quest_conditions | alternative_quest_rewards                    | alternative_quest_template | alternative_quest_title     | quest_expiry | alternative_quest_expiry | description | quest_reward_type | quest_item_id | quest_reward_amount | quest_pokemon_id | alternative_quest_pokemon_id | alternative_quest_reward_type | alternative_quest_item_id | alternative_quest_reward_amount | showcase_pokemon_id | showcase_pokemon_form_id | showcase_pokemon_type_id | showcase_ranking_standard | showcase_expiry | showcase_rankings                                                                                                                                                                                                                                                                                                                  |
+-------------------------------------+-------------------+-------------------+-------------------------+------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------+------------+---------+------------+-----------------+--------------+------------------+-------------------------------------------------------------------------------------------------+-------------------------------------+------------------------------+---------------------+---------+---------+----------------------+------------+------------+------------------+----------------+-----------------+------------------------+------------------------+-----------------------------+--------------------------+------------------------------+----------------------------------------------+----------------------------+-----------------------------+--------------+--------------------------+-------------+-------------------+---------------+---------------------+------------------+------------------------------+-------------------------------+---------------------------+---------------------------------+---------------------+--------------------------+--------------------------+---------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 00000000000000000000000000000000.16 | 00.00000000000000 | 00.00000000000000 | xxxxxxxxxxxxxxxxxxxxxxx | https://lh3.googleusercontent.com/7I5-WlHsau36QbI_cJJASEZSZDlGZQ7uiWhWo9mgyw_LQOc76PA7wN26WyXF6DFYGtjXhjH6BhKMvcjIxxxxxxxxxx |            1701155413 |              1713653163 | 1713693574 |       1 |          5 |      1713657694 |            5 | []               | [{"info":{"costume_id":0,"form_id":2792,"gender_id":0,"pokemon_id":58,"shiny":false},"type":7}] | challenge_base_spin_moderate_june23 | quest_visit_pokestops_plural | 5117288628988411904 |       0 |     501 |           1690632111 |          0 | NULL       |                1 |              0 |              50 |                   NULL |                     46 |                  1713662227 |                        1 | [{"info":{},"type":42}]      | [{"info":{"amount":5,"item_id":2},"type":2}] | geotargeted_ar_scan        | geotarget_quest_description |   1713736800 |               1713736800 | NULL        |                 7 |          NULL |                NULL |               58 |                         NULL |                             2 |                         2 |                               5 |                 214 |                     NULL |                     NULL |                         2 |      1713117600 | {"total_entries":4,"last_update":1713079031,"contest_entries":[{"rank":1,"score":723.8112692858176,"pokemon_id":214,"form":0,"costume":0,"gender":2},{"rank":2,"score":713.8497916731253,"pokemon_id":214,"form":0,"costume":0,"gender":1},{"rank":3,"score":669.2578001207355,"pokemon_id":214,"form":0,"costume":0,"gender":1}]} |
+-------------------------------------+-------------------+-------------------+-------------------------+------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------+------------+---------+------------+-----------------+--------------+------------------+-------------------------------------------------------------------------------------------------+-------------------------------------+------------------------------+---------------------+---------+---------+----------------------+------------+------------+------------------+----------------+-----------------+------------------------+------------------------+-----------------------------+--------------------------+------------------------------+----------------------------------------------+----------------------------+-----------------------------+--------------+--------------------------+-------------+-------------------+---------------+---------------------+------------------+------------------------------+-------------------------------+---------------------------+---------------------------------+---------------------+--------------------------+--------------------------+---------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

And incident table

+---------------------+-------------------------------------+------------+------------+--------------+-------+-----------+------------+-----------+-------------------+-------------+-------------------+-------------+-------------------+-------------+
| id                  | pokestop_id                         | start      | expiration | display_type | style | character | updated    | confirmed | slot_1_pokemon_id | slot_1_form | slot_2_pokemon_id | slot_2_form | slot_3_pokemon_id | slot_3_form |
+---------------------+-------------------------------------+------------+------------+--------------+-------+-----------+------------+-----------+-------------------+-------------+-------------------+-------------+-------------------+-------------+
| 0000000000000000000 | 00000000000000000000000000000000.16 | 1713600000 | 1713722400 |            9 |     0 |         0 | 1713600493 |         0 |              NULL |        NULL |              NULL |        NULL |              NULL |        NULL |
+---------------------+-------------------------------------+------------+------------+--------------+-------+-----------+------------+-----------+-------------------+-------------+-------------------+-------------+-------------------+-------------+

It's not an edge case. Simply new showcase without lookup

[Fabio1988]

It's not an edge case. Simply new showcase without lookup

That's why I added second comment ;)
Your Export is quite useless without headers on mobile

[TurtIeSocks]

Does this accurately returns the incidents in question?

SELECT * FROM golbat.incident WHERE pokestop_id IN (SELECT id FROM golbat.pokestop WHERE showcase_expiry < unix_timestamp());

The sub query, the one selecting the IDs with stops that have expired showcases returns 46 on its own from my very large pokestop table. The entire query doesn't return anything though...

[kbtbc]

...to add some data, I got 233 results w above sub query, and 14 results in full query.

[kamieniarz]

@Fabio1988 I was on mobile ;) Edited message

[kamieniarz]

@TurtIeSocks 331 results that expire in 18 min

[TurtIeSocks]

Can you describe what filters are on (or off) to replicate this behavior? As well as what you see exactly (screenshots please) and what you would expect to see? showcase_expirary is not used much in ReactMap so I'm still not 100% sure what the issue is.

[kamieniarz]

The issue is what I explained in first post. Db entries I provided above show that that specific pokestop has showcase but map won't display it in any scneario (filters don't matter, I couldn't make them display like other showcases display). Expired showcases shouldn't be taken under consideration if their expiry date is before expiration date of pokestop with display_type=9. Here's an example of pokestop that should have showcase on it but didn't cause it had showcase data from the past in pokestop table (exact situation from my db entries)

[TurtIeSocks]

Please try showcase-expirary branch and report back, since I don't seem to have any of these edge cases, I haven't tested it.

[kamieniarz]

Too late. I'll have to wait for another event cause showcases are gone in 1min. And it's not an edge case unless you lookup every single showcase - then it simply won't be a case. Imagine you lookup all showcases possible, in whole country. And on next event you don't do that but instead you scan whole country with fort mode only. You won't see any showcase on map

[Fabio1988]

Then go test in nyc with scannext and be nice ;)

[Fabio1988]

There is expiration date without lookup and it's been like that proly since forever

Here's pokestop table

+-------------------------------------+-------------------+-------------------+-------------------------+------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------+------------+---------+------------+-----------------+--------------+------------------+-------------------------------------------------------------------------------------------------+-------------------------------------+------------------------------+---------------------+---------+---------+----------------------+------------+------------+------------------+----------------+-----------------+------------------------+------------------------+-----------------------------+--------------------------+------------------------------+----------------------------------------------+----------------------------+-----------------------------+--------------+--------------------------+-------------+-------------------+---------------+---------------------+------------------+------------------------------+-------------------------------+---------------------------+---------------------------------+---------------------+--------------------------+--------------------------+---------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                  | lat               | lon               | name                    | url                                                                                                                          | lure_expire_timestamp | last_modified_timestamp | updated    | enabled | quest_type | quest_timestamp | quest_target | quest_conditions | quest_rewards                                                                                   | quest_template                      | quest_title                  | cell_id             | deleted | lure_id | first_seen_timestamp | sponsor_id | partner_id | ar_scan_eligible | power_up_level | power_up_points | power_up_end_timestamp | alternative_quest_type | alternative_quest_timestamp | alternative_quest_target | alternative_quest_conditions | alternative_quest_rewards                    | alternative_quest_template | alternative_quest_title     | quest_expiry | alternative_quest_expiry | description | quest_reward_type | quest_item_id | quest_reward_amount | quest_pokemon_id | alternative_quest_pokemon_id | alternative_quest_reward_type | alternative_quest_item_id | alternative_quest_reward_amount | showcase_pokemon_id | showcase_pokemon_form_id | showcase_pokemon_type_id | showcase_ranking_standard | showcase_expiry | showcase_rankings                                                                                                                                                                                                                                                                                                                  |
+-------------------------------------+-------------------+-------------------+-------------------------+------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------+------------+---------+------------+-----------------+--------------+------------------+-------------------------------------------------------------------------------------------------+-------------------------------------+------------------------------+---------------------+---------+---------+----------------------+------------+------------+------------------+----------------+-----------------+------------------------+------------------------+-----------------------------+--------------------------+------------------------------+----------------------------------------------+----------------------------+-----------------------------+--------------+--------------------------+-------------+-------------------+---------------+---------------------+------------------+------------------------------+-------------------------------+---------------------------+---------------------------------+---------------------+--------------------------+--------------------------+---------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 00000000000000000000000000000000.16 | 00.00000000000000 | 00.00000000000000 | xxxxxxxxxxxxxxxxxxxxxxx | https://lh3.googleusercontent.com/7I5-WlHsau36QbI_cJJASEZSZDlGZQ7uiWhWo9mgyw_LQOc76PA7wN26WyXF6DFYGtjXhjH6BhKMvcjIxxxxxxxxxx |            1701155413 |              1713653163 | 1713693574 |       1 |          5 |      1713657694 |            5 | []               | [{"info":{"costume_id":0,"form_id":2792,"gender_id":0,"pokemon_id":58,"shiny":false},"type":7}] | challenge_base_spin_moderate_june23 | quest_visit_pokestops_plural | 5117288628988411904 |       0 |     501 |           1690632111 |          0 | NULL       |                1 |              0 |              50 |                   NULL |                     46 |                  1713662227 |                        1 | [{"info":{},"type":42}]      | [{"info":{"amount":5,"item_id":2},"type":2}] | geotargeted_ar_scan        | geotarget_quest_description |   1713736800 |               1713736800 | NULL        |                 7 |          NULL |                NULL |               58 |                         NULL |                             2 |                         2 |                               5 |                 214 |                     NULL |                     NULL |                         2 |      1713117600 | {"total_entries":4,"last_update":1713079031,"contest_entries":[{"rank":1,"score":723.8112692858176,"pokemon_id":214,"form":0,"costume":0,"gender":2},{"rank":2,"score":713.8497916731253,"pokemon_id":214,"form":0,"costume":0,"gender":1},{"rank":3,"score":669.2578001207355,"pokemon_id":214,"form":0,"costume":0,"gender":1}]} |
+-------------------------------------+-------------------+-------------------+-------------------------+------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------+------------+---------+------------+-----------------+--------------+------------------+-------------------------------------------------------------------------------------------------+-------------------------------------+------------------------------+---------------------+---------+---------+----------------------+------------+------------+------------------+----------------+-----------------+------------------------+------------------------+-----------------------------+--------------------------+------------------------------+----------------------------------------------+----------------------------+-----------------------------+--------------+--------------------------+-------------+-------------------+---------------+---------------------+------------------+------------------------------+-------------------------------+---------------------------+---------------------------------+---------------------+--------------------------+--------------------------+---------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

And incident table

+---------------------+-------------------------------------+------------+------------+--------------+-------+-----------+------------+-----------+-------------------+-------------+-------------------+-------------+-------------------+-------------+
| id                  | pokestop_id                         | start      | expiration | display_type | style | character | updated    | confirmed | slot_1_pokemon_id | slot_1_form | slot_2_pokemon_id | slot_2_form | slot_3_pokemon_id | slot_3_form |
+---------------------+-------------------------------------+------------+------------+--------------+-------+-----------+------------+-----------+-------------------+-------------+-------------------+-------------+-------------------+-------------+
| 0000000000000000000 | 00000000000000000000000000000000.16 | 1713600000 | 1713722400 |            9 |     0 |         0 | 1713600493 |         0 |              NULL |        NULL |              NULL |        NULL |              NULL |        NULL |
+---------------------+-------------------------------------+------------+------------+--------------+-------+-----------+------------+-----------+-------------------+-------------+-------------------+-------------+-------------------+-------------+

It's not an edge case. Simply new showcase without lookup

I wonder if Golbat should also update showcase_expiry column of pokestop to the value from gmo incident... Imo this column should only be in incident table :/

[kamieniarz]

@Fabio1988 I can't just go to nyc. I'd have to mess around in db to recreate

[Fabio1988]

@Fabio1988 I can't just go to nyc. I'd have to mess around in db to recreate

Already scanned nyc? Then Santa monica pier :)

[kamieniarz]

@Fabio1988 I can't just go to nyc. I'd have to mess around in db to recreate

Already scanned nyc? Then monica beach :)

I think you don't understand the issue... To scan nyc and expect to recreate the issue there I'd have to have in db showcases from nyc that already expired. I don't have such cause I never scanned nyc. All I can do is modify my db to recreate or wait for another showcase event. I prefer the latter

[Fabio1988]

@Fabio1988 I can't just go to nyc. I'd have to mess around in db to recreate

Already scanned nyc? Then monica beach :)

I think you don't understand the issue... To scan nyc and expect to recreate the issue there I'd have to have in db showcases from nyc that already expired. I don't have such cause I never scanned nyc. All I can do is modify my db to recreate or wait for another showcase event. I prefer the latter

So with a fresh gmo scan this doesn't happen and RM showing the showcase? Even with no showcase details

[kamieniarz]

@Fabio1988 I can't just go to nyc. I'd have to mess around in db to recreate

Already scanned nyc? Then monica beach :)

I think you don't understand the issue... To scan nyc and expect to recreate the issue there I'd have to have in db showcases from nyc that already expired. I don't have such cause I never scanned nyc. All I can do is modify my db to recreate or wait for another showcase event. I prefer the latter

So with a fresh gmo scan this doesn't happen and RM showing the showcase? Even with no showcase details

Issue won't occur if you have no data in showcase related fields in pokestop table. So yeah, if you use only GMO scan without showcase lookup and never did showcase lookup in that specific area - issue won't happen. But as said earlier, if you scan whole city/country/whatever using showcase lookup, event ends and on next showcase event you use GMO scan only - you won't see any showcase on map.

[kbtbc]

I checked out branch showcase-expirary and the results are the same as above, showcase data is still missing from React Map.

[kamieniarz]

@TurtIeSocks I can also confirm that showcase-expirary didn't fix that issue (so far)

[kamieniarz]

I've also noticed an interesting state, when showcase wasn't fully scanned. It was displayed with correct end date, but challenge type and mond from ranging were taken from previous showcase that ended 😬 But I guess this specific state is nothing rm could handle. Anyway main issue still occurs unless it was meant to be fixed in latest commits - I haven't tried these yet

[TurtIeSocks]

Haven't touched this issue because I still can't replicate. Someone who's experiencing it will need to be the one to address it.

[kamieniarz]

It's easy to replicate. Scan area with mons and on new showcase event scan same area with fort mode only