Asset Inventory Alignment With Entity Store V2

by Editorial Team 47 views
Iklan Headers

Let's dive into the crucial task of aligning Asset Inventory with Entity Store V2. This transition involves several key areas that need careful attention to ensure a smooth and functional experience. We'll explore the motivations behind this alignment, the specific tasks required, and what's considered out of scope.

Motivation

The primary motivation for this alignment stems from the architectural changes introduced by Entity Store V2. While the underlying data migration at the Elasticsearch layer is expected to be seamless, Asset Inventory has certain integration points that are likely to be affected. These include API versioning, onboarding assumptions, and dataview/index targeting. Therefore, it's essential to update Asset Inventory to accommodate these changes, ensuring it remains fully functional throughout the V2 transition. This may involve implementing interim UI states to handle asynchronous migration processes gracefully.

  • API Versioning: With Entity Store V2, the API endpoints may change, requiring Asset Inventory to update its calls to the new endpoints. This ensures that Asset Inventory can communicate effectively with the updated Entity Store.
  • Onboarding Assumptions: The onboarding process in Asset Inventory currently relies on certain assumptions about the state of the data and the availability of transforms and index names in V1. These assumptions need to be revisited and adjusted to align with the V2 environment.
  • Dataview/Index Targeting: Asset Inventory dataviews need to be updated to point to the new indices created by Entity Store V2, rather than the old V1 indices. This ensures that Asset Inventory is displaying the correct and up-to-date data.

To address these challenges, a series of concrete changes are required to ensure Asset Inventory remains functional during and after the V2 transition. This includes updating API endpoints, adjusting onboarding flows, and handling migration timing effectively. By addressing these points, we can ensure a seamless transition and maintain the functionality of Asset Inventory.

Definition of Done

To ensure that the alignment of Asset Inventory with Entity Store V2 is successful, we need to define clear and measurable criteria for completion. The following list outlines the specific tasks that must be completed to consider this alignment as "done".

  • Update API Endpoints: The Asset Inventory must be updated to use the Entity Store V2 API endpoints (or updated routes) once available. This ensures that all communications between Asset Inventory and Entity Store are using the correct and current API versions.
  • Update Dataviews: The Asset Inventory dataview(s) must be updated to point to Entity Store V2 indices instead of V1 indices. This guarantees that the Asset Inventory is displaying data from the correct data source.
  • Review and Adapt Onboarding Flow: The onboarding flow must be reviewed and adapted for V2 behavior. This includes several sub-tasks:
    • Ensure enablement and readiness checks align with V2 capabilities: The checks that determine whether Asset Inventory can be enabled must be updated to reflect the capabilities of Entity Store V2.
    • Remove any remaining assumptions about V1 transforms/index names: Any assumptions in the onboarding process about the naming or structure of V1 transforms and indices must be removed.
    • Confirm post-enable "verifying data" logic remains correct with V2: The logic that verifies data after enablement must be confirmed to work correctly with Entity Store V2.
  • Clarify and Implement Handling for Migration Timing: The timing of the Entity Store migration must be clarified, and handling implemented accordingly:
    • Confirm whether Entity Store migration runs during Kibana upgrade vs. via background job after upgrade: Determine whether the migration occurs during the Kibana upgrade process or as a background job afterward.
    • If background, define UI strategy for the intermediate state (e.g., "upgrade in progress / maintenance mode" instead of "no data"): If the migration occurs in the background, define a UI strategy for the intermediate state, such as displaying an "upgrade in progress" or "maintenance mode" message instead of indicating "no data".
    • Align with Entity Store entity-store/status API if it will expose a maintenance state: Align with the Entity Store entity-store/status API if it will expose a maintenance state, allowing the UI to accurately reflect the status of the migration.
  • Confirm Behavior Around Generic Entity Initialization: Confirm the behavior around Generic Entity initialization:
    • Today it is initialized only when the Asset Inventory Tech Preview feature flag is enabled: Understand that currently, it is initialized only when the Asset Inventory Tech Preview feature flag is enabled.
    • Validate whether this remains intended given Graph also needs Generic Entity: Validate if this initialization method remains intended, especially considering that Graph also requires Generic Entity.
    • Align with Product/Graph/Entity Store owners and update gating accordingly: Align with Product, Graph, and Entity Store owners and update gating accordingly, ensuring that the initialization method is appropriate for all components.
  • Validate End-to-End Behavior in a V2 Environment: The end-to-end behavior must be validated in a V2 environment. This includes:
    • Fresh enablement flow: Test the enablement flow from scratch in a V2 environment.
    • Existing installation upgraded from V1 to V2 (with/without data): Test the upgrade process from V1 to V2, both with and without existing data.
    • No-data scenario after the timeout window: Test the scenario where no data is available after the timeout window.
  • Document Known Breaking Changes and Any Tech Preview Caveats: Any known breaking changes and tech preview caveats must be documented. This provides users with clear expectations and guidance on potential issues.

By completing these tasks, we can ensure that Asset Inventory is fully aligned with Entity Store V2 and that users have a seamless and reliable experience.

Onboarding Flow Adaptation

The onboarding flow is a critical part of ensuring users can seamlessly start using Asset Inventory with Entity Store V2. Adapting this flow requires careful attention to detail to address the changes introduced by V2. This includes ensuring enablement and readiness checks are aligned with V2 capabilities, removing assumptions about V1 transforms and index names, and confirming the post-enable data verification logic remains correct.

  • Enablement and Readiness Checks: The enablement and readiness checks are the first line of defense in ensuring a smooth onboarding experience. These checks must accurately reflect the capabilities of Entity Store V2. This means updating the checks to verify the presence of V2-specific features, ensuring that the necessary V2 indices are available, and confirming that the system meets the minimum requirements for V2 compatibility. By aligning these checks with V2 capabilities, we can prevent users from attempting to enable Asset Inventory in environments that are not yet ready, reducing potential errors and frustration.
  • Removing V1 Assumptions: The current onboarding flow may contain assumptions about V1 transforms and index names. These assumptions need to be identified and removed. This involves reviewing the code to identify any hardcoded references to V1-specific names or structures and replacing them with dynamic references that adapt to the V2 environment. This ensures that the onboarding process is flexible and can handle the changes introduced by V2 without breaking.
  • Post-Enable Data Verification: After enabling Asset Inventory, it's essential to verify that data is being correctly ingested and displayed. The post-enable data verification logic must be updated to work with Entity Store V2. This includes updating the queries used to verify data, ensuring that the correct V2 indices are being targeted, and confirming that the data being displayed is accurate and up-to-date. By verifying the data after enablement, we can catch any potential issues early and ensure that users have a reliable and accurate view of their assets.

By carefully adapting the onboarding flow, we can ensure that users have a seamless and trouble-free experience when starting with Asset Inventory in a V2 environment. This involves updating enablement checks, removing V1 assumptions, and verifying data post-enablement.

Handling Migration Timing

The timing of the Entity Store migration is a critical factor that impacts how Asset Inventory functions during the transition. Understanding whether the migration runs during the Kibana upgrade or as a background job after the upgrade is essential for defining the appropriate UI strategy. If the migration runs in the background, a UI strategy for the intermediate state, such as displaying an "upgrade in progress" or "maintenance mode" message, is crucial to avoid confusion and ensure a smooth user experience.

  • Migration Timing Confirmation: The first step is to confirm exactly when the Entity Store migration occurs. Does it happen as part of the Kibana upgrade process, or does it run as a separate background job after the upgrade is complete? Understanding this timing is essential for determining how to handle the transition in the UI.
  • UI Strategy for Background Migration: If the migration runs in the background, it's crucial to define a UI strategy for the intermediate state. Displaying a message such as "upgrade in progress" or "maintenance mode" can inform users that the system is undergoing maintenance and that data may not be immediately available. This is preferable to displaying a "no data" message, which could be misinterpreted as an error or a loss of data. The UI strategy should also include a mechanism for notifying users when the migration is complete and the data is available.
  • Aligning with Entity Store Status API: To accurately reflect the status of the migration in the UI, it's important to align with the Entity Store entity-store/status API, if it exposes a maintenance state. This API can provide real-time information about the progress of the migration, allowing the UI to display accurate and informative messages to users. By aligning with this API, we can ensure that users are always aware of the current status of the system and can plan their activities accordingly.

By carefully considering the timing of the Entity Store migration and implementing an appropriate UI strategy, we can minimize disruption to users and ensure a smooth transition to V2.

Generic Entity Initialization

The initialization of Generic Entity is an important aspect to consider, especially given its dependency on the Asset Inventory Tech Preview feature flag. It's crucial to validate whether this remains the intended behavior, considering that Graph also needs Generic Entity. Aligning with Product, Graph, and Entity Store owners and updating the gating accordingly is essential to ensure consistency and avoid potential conflicts.

  • Current Initialization Behavior: Currently, Generic Entity is initialized only when the Asset Inventory Tech Preview feature flag is enabled. This means that if the feature flag is disabled, Generic Entity is not initialized, which can impact other components that rely on it.
  • Validation with Graph Dependency: Given that Graph also needs Generic Entity, it's important to validate whether the current initialization behavior remains appropriate. If Graph requires Generic Entity regardless of the Asset Inventory Tech Preview feature flag, then the initialization logic needs to be adjusted to ensure that Generic Entity is always initialized when Graph is enabled.
  • Alignment with Stakeholders: To ensure consistency and avoid potential conflicts, it's essential to align with Product, Graph, and Entity Store owners. This involves discussing the initialization behavior of Generic Entity and determining the appropriate gating mechanism. The goal is to ensure that Generic Entity is initialized in a way that meets the needs of all components that rely on it, without causing any unexpected side effects.

By carefully considering the initialization behavior of Generic Entity and aligning with stakeholders, we can ensure that it is initialized in a way that supports all components that rely on it, without causing any conflicts or unexpected behavior.

Validation in a V2 Environment

Validating end-to-end behavior in a V2 environment is a critical step in ensuring a smooth transition. This includes testing fresh enablement flows, existing installations upgraded from V1 to V2 (with and without data), and no-data scenarios after the timeout window. Thorough validation helps identify potential issues and ensures that Asset Inventory functions as expected in the new environment.

  • Fresh Enablement Flow: Testing the fresh enablement flow in a V2 environment ensures that new users can successfully set up and start using Asset Inventory without any issues. This involves verifying that all the necessary steps in the onboarding process are functioning correctly and that users can access and view their assets as expected.
  • Upgraded Installations: Testing existing installations upgraded from V1 to V2, both with and without data, is crucial for ensuring a seamless transition for existing users. This involves verifying that the upgrade process is smooth and that all existing data is correctly migrated to the new environment. It also involves testing scenarios where data may be missing or incomplete to ensure that Asset Inventory handles these situations gracefully.
  • No-Data Scenarios: Testing no-data scenarios after the timeout window ensures that Asset Inventory handles situations where data is not available. This involves verifying that the UI displays appropriate messages to users and that the system does not crash or become unresponsive.

By thoroughly validating end-to-end behavior in a V2 environment, we can identify and address any potential issues before they impact users, ensuring a smooth and reliable experience.

Documentation

Documenting known breaking changes and any tech preview caveats is essential for providing users with clear expectations and guidance on potential issues. This documentation should be easily accessible and should provide detailed information about any changes that may impact users, as well as any limitations or known issues with the tech preview.

By providing clear and comprehensive documentation, we can help users understand the changes introduced by Entity Store V2 and how they may impact their use of Asset Inventory. This will help reduce confusion and ensure a smoother transition to the new environment.

Out of Scope

It's important to define what is considered out of scope for this alignment effort to manage expectations and maintain focus. The following items are explicitly excluded from the scope:

  • Re-designing the entire Asset Inventory onboarding UX beyond what’s required for V2 compatibility.
  • Guaranteeing backward compatibility/data guarantees beyond tech preview expectations (unless Product explicitly requires it).

Related Tasks/Epics

Several related tasks and epics are essential for the successful alignment of Asset Inventory with Entity Store V2. These include:

  • Entity Store V2 migration implementation (Entity Store team ownership)
  • Entity Store status API implementations (install/migration/maintenance state)
  • Graph dependency on Generic Entity (alignment task)

By addressing these related tasks and epics, we can ensure a comprehensive and coordinated approach to aligning Asset Inventory with Entity Store V2.