Skip to content

Add OCI infrastructure content#14612

Open
KanenasCS wants to merge 4 commits into
Azure:masterfrom
KanenasCS:oci-infra
Open

Add OCI infrastructure content#14612
KanenasCS wants to merge 4 commits into
Azure:masterfrom
KanenasCS:oci-infra

Conversation

@KanenasCS

@KanenasCS KanenasCS commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Required items, please complete

Change(s):

  • Solution: Oracle Cloud Infrastructure — version 3.0.10 → 3.1.0
  • Data Connectors/Oracle_Cloud_Infrastructure_CCP/OCI_DataConnector_DCR.json: rebuilt the DCR ingestion-time transform. The output no longer stores the raw data and oracle dynamic payloads alongside their flattened columns, and drops HTTP request/response header noise, CloudEvents boilerplate (dataschema_s, specversion_s), duplicated fields (time_t, id_s, flow-log data_source*/destination* copies of the normalized SrcIpAddr/DstIpAddr set), credential-bearing request headers (authorization, Cookie, opc_obo_token, auth_info, opc_principal), and the stateChange.current.keyValue public-key blob. Fixed SrcIpAddr mapping typo (identity.ipAddresidentity.ipAddress) with fallback for audit events. Input stream declaration unchanged.
  • Data Connectors/Oracle_Cloud_Infrastructure_CCP/OCI_DataConnector_Table.json: schema regenerated to match the transform output — 201 → 86 columns.
  • Data Connectors/Oracle_Cloud_Infrastructure_CCP/azuredeploy_OCI_DataConnector_poller_connector.json: embedded DCR/table synced to the above.
  • Removed the deprecated Azure Function connector and its assets (AzureFunctionOCILogs/, OCILogsConn.zip, OCI_logs_API_FunctionApp.json, azuredeploy_OCI_logs_API_FunctionApp.json, host.json, proxies.json, requirements.txt). The V3 package already shipped only the CCF connector.
  • Parsers/OCILogs.yaml (v2.0.0): now reads OCI_LogsV2_CL only (removed the OCI_Logs_CL union); derives EventStartTime/EventEndTime from flow-log epoch fields via unixtime_seconds_todatetime() with TimeGenerated fallback.
  • All 10 Analytic Rules and 10 Hunting Queries: requiredDataConnectors retargeted from the deprecated OracleCloudInfrastructureLogsConnector/OCILogs to the CCF connector OracleCloudInfraConnector/OCI_LogsV2_CL. Queries unchanged.
  • Workbooks/OracleCloudInfrastructureOCI.json: replaced the outdated manual parser-deployment note (parser installs with the V3 solution). Queries unchanged.

Reason for Change(s):

  • After migrating from the Function App connector to CCF, daily ingestion into OCI_LogsV2_CL increased ~3x with no increase in OCI event count. Root cause: the DCR transform stored each event up to three times — the full raw data/oracle payloads, ~190 flattened columns extracted from those same payloads, and normalized aliases duplicating several fields again. Since Log Analytics bills post-transform bytes, this directly tripled Sentinel/Log Analytics cost for all users of this connector.
  • The slimmed transform keeps every column referenced by the solution's Parser, Analytic Rules, Hunting Queries, and Workbook (verified per file), preserving all shipped detections while removing duplicated and security-irrelevant content. Removing raw authorization/Cookie/OBO-token headers and key material from a 180-day table is also a security improvement.
  • The deprecated Function App connector and the parser's OCI_Logs_CL dependency were removed as the legacy pipeline is superseded by CCF; content still pointed at the deprecated connector id, so rule tiles showed "not connected" for CCF-only workspaces.
  • SrcIpAddr typo fix restores source-IP population for audit events (previously empty), improving existing detections and workbook panels.

Version Updated:

  • Yes — all 10 modified Analytic Rule templates version-bumped (e.g. OCIDiscoveryActivity 1.0.0 → 1.0.1, OCIEventRuleDeleted 1.0.1 → 1.0.2, etc.); Parser bumped to 2.0.0; Solution version 3.1.0.

Testing Completed:

  • Yes
  • Solution rebuilt with createSolutionV3.ps1 (V3) and validated with arm-ttk; only the two known CCF-specific exceptions remain (IDs Should Be Derived From ResourceIDs on CCF resource ids and [second known exception]).
  • Deployed the packaged solution to a clean Microsoft Sentinel test workspace (no custom parsers/functions/tables), connected via Content Hub, and confirmed end-to-end ingestion into OCI_LogsV2_CL for audit and VCN flow log categories.
  • Ingestion volume evidence: avg row size via estimate_data_size(*) reduced from [X] KB to [Y] KB (~[Z]% reduction); daily GB via Usage reduced accordingly with unchanged event counts.
  • Verified all 10 Analytic Rules, 10 Hunting Queries, and the Workbook execute without syntax errors against the new schema and return expected results; parser OCILogs resolves in a workspace without the legacy OCI_Logs_CL table.

Checked that the validations are passing and have addressed any issues that are present:

  • Yes — KQL and YAML validation run locally per repo guidance; JSON/YAML of all changed files validated; no references to OCI_Logs_CL or the deprecated connector id remain in the solution.

@KanenasCS KanenasCS requested review from a team as code owners July 2, 2026 12:27
@v-atulyadav v-atulyadav requested a review from Copilot July 2, 2026 13:22
@v-atulyadav v-atulyadav added the Solution Solution specialty review needed label Jul 2, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates the Oracle Cloud Infrastructure solution package to v3.1.0 by optimizing CCF ingestion (smaller post-transform footprint), removing legacy Function App connector assets, and retargeting shipped content to OCI_LogsV2_CL / OracleCloudInfraConnector.

Changes:

  • Rebuilt the CCF DCR transform and regenerated the OCI_LogsV2_CL table schema to remove duplicated/raw payload content and header noise.
  • Removed deprecated Azure Function connector templates/assets; updated parser, analytic rules, hunting queries, and workbook notes to align with the CCF pipeline/table.
  • Bumped solution + content versions and updated packaging UI/test parameters.

Reviewed changes

Copilot reviewed 39 out of 56 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
Solutions/Oracle Cloud Infrastructure/Workbooks/OracleCloudInfrastructureOCI.json Updates workbook note about parser installation and normalizes query string content.
Solutions/Oracle Cloud Infrastructure/SolutionMetadata.json Updates solution metadata (providers/support/author).
Solutions/Oracle Cloud Infrastructure/ReleaseNotes.md Adds 3.1.0 release notes entry describing ingestion optimization and cleanup.
Solutions/Oracle Cloud Infrastructure/Parsers/OCILogs.yaml Updates parser to v2-only against OCI_LogsV2_CL and derives event times from epoch fields.
Solutions/Oracle Cloud Infrastructure/Package/testParameters.json Adds workbook name test parameter.
Solutions/Oracle Cloud Infrastructure/Package/createUiDefinition.json Updates installer description counts; adds workbook section and updates connector references in text.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIUserUpdatedInstances.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIUserTerminatedInstances.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIUserSources.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIUserNewUsers.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIUserDeletedUsers.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIUserDeleteActions.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIUpdateActivities.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCILaunchedInstances.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIDestinationsOut.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Hunting Queries/OCIDestinationsIn.yaml Retargets required connector + datatype to OracleCloudInfraConnector / OCI_LogsV2_CL.
Solutions/Oracle Cloud Infrastructure/Data/Solution_OCILogs.json Updates solution version and metadata, including BasePath/Author.
Solutions/Oracle Cloud Infrastructure/Data Connectors/requirements.txt Removes legacy Function App requirements.
Solutions/Oracle Cloud Infrastructure/Data Connectors/proxies.json Removes legacy Function App proxies config.
Solutions/Oracle Cloud Infrastructure/Data Connectors/host.json Removes legacy Function App host configuration.
Solutions/Oracle Cloud Infrastructure/Data Connectors/azuredeploy_OCI_logs_API_FunctionApp.json Removes legacy Function App ARM deployment template.
Solutions/Oracle Cloud Infrastructure/Data Connectors/Oracle_Cloud_Infrastructure_CCP/azuredeploy_OCI_DataConnector_poller_connector.json Updates embedded DCR/table and connector strings; aligns embedded artifacts with optimized schema.
Solutions/Oracle Cloud Infrastructure/Data Connectors/Oracle_Cloud_Infrastructure_CCP/OCI_DataConnector_Table.json Regenerates table schema to match optimized transform output (fewer columns).
Solutions/Oracle Cloud Infrastructure/Data Connectors/Oracle_Cloud_Infrastructure_CCP/OCI_DataConnector_DataConnectorDefinition.json Updates connector definition metadata (publisher).
Solutions/Oracle Cloud Infrastructure/Data Connectors/Oracle_Cloud_Infrastructure_CCP/OCI_DataConnector_DCR.json Rebuilds ingestion-time transform and output stream mapping.
Solutions/Oracle Cloud Infrastructure/Data Connectors/OCI_logs_API_FunctionApp.json Removes deprecated connector definition JSON.
Solutions/Oracle Cloud Infrastructure/Data Connectors/AzureFunctionOCILogs/sentinel_connector.py Removes legacy Function App implementation.
Solutions/Oracle Cloud Infrastructure/Data Connectors/AzureFunctionOCILogs/main.py Removes legacy Function App implementation.
Solutions/Oracle Cloud Infrastructure/Data Connectors/AzureFunctionOCILogs/function.json Removes legacy Function App binding definition.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIUnexpectedUserAgent.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCISSHScan.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIMultipleRejects.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIMultipleInstancesTerminated.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIMultipleInstancesLaunched.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIMetadataEndpointIpAccess.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIInsecureMetadataEndpoint.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIInboundSSHConnection.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIEventRuleDeleted.yaml Retargets required connector+datatype and bumps rule version.
Solutions/Oracle Cloud Infrastructure/Analytic Rules/OCIDiscoveryActivity.yaml Retargets required connector+datatype and bumps rule version.

Comment thread Solutions/Oracle Cloud Infrastructure/Data/Solution_OCILogs.json Outdated
Comment thread Solutions/Oracle Cloud Infrastructure/SolutionMetadata.json
Comment thread Solutions/Oracle Cloud Infrastructure/SolutionMetadata.json
k.lianos added 3 commits July 2, 2026 17:20
…ema validation test and update the OCI Data Connector DCR and poller connector templates to reflect the correct configuration. Also, update the main solution package and template to include the latest changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Solution Solution specialty review needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants