[enhancement](cloud) refine block file cache evict policy (apache#42451)

- Enhance cache performance and reduce cache misses by:
    - Prevent starvation of any particular cache.
    - Improve disk space utilization.
    - Maintain strict and systematic eviction priorities.
- Enhance the observability of caching strategies, allowing cache
behavior to be intuitively self-explanatory through monitoring.
- Improve code comprehensibility: unify the cache framework to avoid
ad-hoc handling of TTL, increasing the modularity of the code.
- Reduce operational complexity by eliminating and standardizing
configuration items.

---------

Signed-off-by: freemandealer <[email protected]>

be/src/common/config.cpp

@@ -1011,7 +1011,7 @@ DEFINE_Bool(enable_file_cache_query_limit, "false");
 DEFINE_mInt32(file_cache_enter_disk_resource_limit_mode_percent, "90");
 DEFINE_mInt32(file_cache_exit_disk_resource_limit_mode_percent, "80");
 DEFINE_mBool(enable_read_cache_file_directly, "false");
-DEFINE_mBool(file_cache_enable_evict_from_other_queue_by_size, "false");
+DEFINE_mBool(file_cache_enable_evict_from_other_queue_by_size, "true");
 DEFINE_mInt64(file_cache_ttl_valid_check_interval_second, "0"); // zero for not checking
 // If true, evict the ttl cache using LRU when full.
 // Otherwise, only expiration can evict ttl and new data won't add to cache when full.
@@ -1292,7 +1292,7 @@ DEFINE_Int64(num_s3_file_upload_thread_pool_min_thread, "16");
 // The max thread num for S3FileUploadThreadPool
 DEFINE_Int64(num_s3_file_upload_thread_pool_max_thread, "64");
 // The max ratio for ttl cache's size
-DEFINE_mInt64(max_ttl_cache_ratio, "90");
+DEFINE_mInt64(max_ttl_cache_ratio, "50");
 // The maximum jvm heap usage ratio for hdfs write workload
 DEFINE_mDouble(max_hdfs_wirter_jni_heap_usage_ratio, "0.5");
 // The sleep milliseconds duration when hdfs write exceeds the maximum usage