View doesn't work at the very beginning

[ChopperLee2011]

preset

1. one view create the first time
2. dataset is small, like < 10 records

query data by view method, getConnector().view( viewName, modelName), it always returns [], unless I trigger that view query on the admin UI or add { stale: 1 } in the method, then I can get multiple records as expected.
My local couchbase version is Community Edition 5.1.1 build 5723 and the cluster is working as normal, only > 80% RAM using.

the issue cannot be produced once the view is triggered, which means you can delete and rebuild view can work as usual.

[ChopperLee2011]

one case: if you calling model.destoryAll(), then create data. the view cannot get the updated data unless you input stale:1 option

[xavierchow]

@ChopperLee2011 This is the designed behavior of couchbase: https://docs.couchbase.com/server/4.1/developer-guide/views-operation.html.
What do you want to change?

[ChopperLee2011]

@xavierchow I want to get correct data when I use the find model method, it can have a few sec delays.
currently the lib is setting the stale: NONE by default which means you can't get new records from view query. I can pass stale: 1 options for each find method to get real-time data, but it will have performance issues that you guys already have discuss here.
and @haishanh give me one solution from other project, they set a cronjob to query with stale:1 per min. that could work but seems not perfect, like you can't write test case easily, and you need to set many timer for each view, and you can't cover up when the query cronjob failed.

Since some project is using stale: before before, and we don't have that kind of performance reported, so I gonna still using {stale: before} for each find method for now, waiting for a better idea.

cc @superbogy

[xavierchow]

It depends on your use case,

1. Do you really need a find/query with a fresh result? What's the impact if the API returns stale record?
2. How often does the document change?
3. Having no perf issue reported doesn't mean it's safe to set stale: before everywhere, eventually, you need a performance test or profiling to give you some ideas on how to set the view query option.

Surely, we can talk about details tmr if you want.

[superbogy]

As Couchbase’s official description of view index update strategy, it is hard to trigger view index refresh for most projects.

# default viewUpdateDaemon
{
  "updateInterval": 5000,
  "updateMinChanges": 5000,
  "replicaUpdateMinChanges": 5000
}

About updateInterval:
the time interval in milliseconds, the default is 5000 milliseconds. At every updateInterval the views engine checks if the number of document mutations on disk is greater than updateMinChanges. If true, it triggers the view update. The documents stored on disk potentially lag documents that are in-memory for tens of seconds.

Here is my solution:
As we know the write action is far less than the read action, so I tend to add a query with stale to force refresh view index after an update or create happened.

fake code for the function postWithId of couchbaseX:

return this.connection
      .call('insertAsync', id, record, options)
      .then((res) => {
        const opts = {
          key: this.modelName,
          id_range: [id, id], // limit id start and id end
          stale: 1,
        };
        return this.connector
          .view('connector', 'byModelName', opts)
          .then(records => {
            if (!records.length) {
              return null;
            }

            return res;
          });
      })
      .then(updated => {
        return updated ? [id, updated.cas] : [];
      });

It looks good compatible with the early version couchbase3, it doesn't need to pass a stale in query everywhere. We don't need to refresh the view index if there is none data update.

@xavierchow
@ChopperLee2011
@CCharlieLi
@makara

[superbogy]

For some special projects we can deliver an option like refresh=false to tell postWithId function we don't need to update the index and skip the query.

[xavierchow]

As we know the write action is far less than the read action,

This is a false assumption, sometimes we might have massive write operations in a short time,
but your idea is good as long as we have a toggle.
Let's hear the comment from @ChopperLee2011 @makara @CCharlieLi

[xavierchow]

@ChopperLee2011 could you grab this?
There is another option @superbogy mentioned before, that is a global flag in the datasource.json to control the default stale.

[ChopperLee2011]

@xavierchow sure