Cache Configuration

Caching only improves performance if the cached results remain correct and reflect the current state of the data. To achieve this, cached entries must be invalidated (removed or refreshed).

1. Manual Invalidation

Description: Cache is cleared explicitly by an administrator or user. This can be done through the BI platform’s interface, API, or administrative tools.

Use cases:

• Troubleshooting scenarios when cached results may appear inconsistent.
• Forcing immediate refresh of dashboards after data updates.

Advantages:

• Complete control over when data is refreshed.
• Useful in environments where data changes are unpredictable.

Limitations:

• If not managed proactively, users may unknowingly see stale data.
• Requires manual intervention, which may not scale well.

2. Time-based Invalidation

Description: Cached results are automatically invalidated after a predefined time interval (time-to-live, or TTL).

Use cases:

• Periodic refresh cycles (e.g., reports updated every hour).
• Datasets that change slowly or at predictable intervals.

Configuration:

• TTL can be defined per dashboard. Read about Dashboard-level invalidation
• Example: cache expiry every 15 minutes, 1 hour, or once per day.

Advantages:

• Easy to configure and predictable.
• Ensures data is refreshed without user intervention.

Limitations:

• Stale data can be served until expiration, which may not be acceptable for real-time use cases.
• Too short a TTL reduces caching benefits; too long increases the risk of outdated data.

3. Dynamic Invalidation (Event-driven)

Description: Cached results are invalidated automatically whenever the underlying data changes. This is managed in the background without user intervention.

How it works:

• Grafieks monitors data sources, pipelines, or triggers.
• When new data is ingested or updated, the system identifies which cached queries depend on the modified data.
• Those cache entries are invalidated automatically.
• Optionally, the platform may recompute popular queries in the background so that fresh results are ready before users request them.

Use cases:

• Analytics environments with frequent incremental data loads.

Advantages:

• Minimizes risk of showing stale results.
• Provides freshness guarantees for critical decision-making dashboards.

Limitations:

• Adds overhead to track and detect data changes.
• Requires integration with data pipelines or event notifications.

Choosing the Right Policy

Selecting the correct invalidation policy depends on the balance between performance (fewer cache refreshes) and freshness (how up-to-date the data must be).

• Manual: Best for ad-hoc updates or when administrators want full control.
• Time-based: Suitable for predictable refresh cycles where some delay is acceptable.
• Dynamic: Ideal for real-time analytics and dashboards where users expect data accuracy at all times.

In practice, organizations may combine these approaches: for example, use time-based invalidation as a baseline but enable dynamic invalidation for mission-critical data sources.

Troubleshooting and Maintenance

Caching systems may occasionally face issues. Common situations include:

Stale data observed:

• Verify invalidation policies.
• Manually clear cache to ensure a fresh query run.

Low cache hit ratio:

• Queries may be too unique or dynamic to benefit from caching.
• Increase TTL.

Excessive cache invalidations:

• Dynamic triggers may be too sensitive.
• Adjust thresholds or reduce scope of monitored changes.

Maintenance tasks:

• Administrators can reset cache globally if systemic data changes occur.