Temporary Log Analytics Logging

Version 0.4.2

February 2025

Introduction

This guide describes how to get information about offloads and onloads that have been processed by ShArc. It's a temporary feature that allows to gather some information about the processed files, but it will be replaced with the history feature later.

KQL Queries to Analyze Job Results

1. Job Statistics

For ShArc v1.1 and earlier:
Use the original Job Statistics query for jobs run with versions prior to 1.2:

AppServiceConsoleLogs
| where ResultDescription has "'Job' started with"
    or ResultDescription has "'Job' finished with"
    or ResultDescription endswith "(JobId: "
    or ResultDescription has "Offload finished successfully"
    or ResultDescription has "Onload finished successfully"
    or ResultDescription has "Onload finished with error"
    or ResultDescription has "Skipped item"
| extend JobId = extract(@"\(JobId: ([0-9a-fA-F-]+)\)", 1, ResultDescription)
| extend EventType =
    case(
        ResultDescription has "'Job' started with", "Start",
        ResultDescription has "'Job' finished with", "End",
        ResultDescription has "Offload finished successfully" or ResultDescription has "Onload finished successfully", "Success",
        ResultDescription has "Onload finished with error", "OnloadError",
        ResultDescription has "Skipped item", "SkippedItem",
        "Other"
    )
| extend JobType = case(
        ResultDescription has "Onload", "Onload",
        ResultDescription has "Offload", "Offload",
        "Unknown"
    )
| summarize
    StartTime = arg_min(TimeGenerated, TimeGenerated),  // Correctly gets the earliest TimeGenerated
    EndTime = arg_max(TimeGenerated, TimeGenerated),    // Correctly gets the latest TimeGenerated
    JobType = any(JobType), // Picks the JobType from any Start message
    SuccessCount = countif(EventType == "Success"),
    OnloadErrorCount = countif(EventType == "OnloadError"),
    SkippedItemCount = countif(EventType == "SkippedItem")
    by JobId
| extend JobType = iff(isnotempty(JobType), JobType, "Unknown") // Ensure JobType is never empty
| extend Duration = EndTime - StartTime
| project JobId, StartTime, EndTime, Duration, JobType, SuccessCount, OnloadErrorCount, SkippedItemCount
| order by StartTime desc

For ShArc v1.2 and later:
The Job Statistics query has been updated to support the new Policy feature. Use this enhanced query for statistics for jobs run with version 1.2 and later:

AppServiceConsoleLogs
| where ResultDescription has "'Restore' started with"
    or ResultDescription has "'Offload' started with"
    or ResultDescription has "'Policy Offload' started with"
    or ResultDescription has "'Job' finished"
    or ResultDescription has "(JobId: "
    or ResultDescription has "Skipped file"
| extend JobId = extract(@"\(JobId: ([0-9a-fA-F-]+)\)", 1, ResultDescription)
| where isnotempty(JobId) 
| extend EventType =
    case(
        ResultDescription has "' started with", "Start",
        ResultDescription has "Job finished", "End",
        ResultDescription has "finished successfully for file", "Success",
        ResultDescription has "Job finished with error", "OnloadError",
        ResultDescription has "Skipped file", "SkippedFile",
        "Other"
    )
| extend JobType = case(
        ResultDescription has "Restore", "Restore",
        ResultDescription has "Policy Offload", "PolicyOffload",
        ResultDescription has "Offload", "Offload",
        "Unknown"
    )
| summarize
    StartTime = arg_min(TimeGenerated, TimeGenerated),
    EndTime = arg_max(TimeGenerated, TimeGenerated),
    AllJobTypes = make_set(JobType),
    SuccessCount = countif(EventType == "Success"),
    OnloadErrorCount = countif(EventType == "OnloadError"),
    SkippedItemCount = countif(EventType == "SkippedFile")
    by JobId
| extend JobType = case(
    set_has_element(AllJobTypes, "Restore"), "Restore",
    set_has_element(AllJobTypes, "PolicyOffload"), "PolicyOffload",
    set_has_element(AllJobTypes, "Offload"), "Offload",
    "UserRestore"
)
| extend Duration = EndTime - StartTime
| project JobId, StartTime, EndTime, Duration, JobType, SuccessCount, OnloadErrorCount, SkippedItemCount
| order by StartTime desc

2. Job Details

Please note, that the first line requires to specify the JobId. This id is provided in the Job Statistics query results and can be copied from there.

let jobIdToCheck = "af6a32de-0a93-49a0-80c9-71c022d3bd9b"; // Replace with the actual JobId
AppServiceConsoleLogs
| where ResultDescription has strcat("(JobId: ", jobIdToCheck, ")")
| extend EventType =
    case(
        ResultDescription has "Offload finished successfully" or ResultDescription has "Onload finished successfully", "Success",
        ResultDescription has "Onload finished with error", "Error",
        ResultDescription has "Skipped item", "Skipped",
        "Other"
    )
| where EventType in ("Success", "Error", "Skipped")
| project TimeGenerated, jobIdToCheck, EventType, ResultDescription
| order by TimeGenerated asc

How to save KQL Queries in Log Analytics

To save the queries in Log Analytics for later use, follow these steps:

Step 1: Navigate to Log Analytics in your ShArc App Service.

Step 2: Copy the KQL query code into the query pane.

Copy the KQL query code that is provided above into the New Query pane.

Step 3: Save the KQL query with a descriptive name.

Click on Save => Save as query and specify a name for the query.

How to Run the Saved KQL Queries in Log Analytics

Navigate to Log Analytics in your ShArc App Service
Search for and open the saved KQL query
Configure the query parameters

For the Job Statistics Query, specify the time range according to your needs:
For the Job Details Query, specify the job id in the query as well as time range and number of results that should be shown:

Run the query and review the results.

Known Issues

The maximum number of results that you can retrieve in the Log Analytics portal experience is 30,000. However, when you share a Log Analytics query with Excel or PowerBI, the maximum limit is 500,000.

For the Job Details Query, specify the job id in the query as well as time range and number of results that should be shown:

Run the query and review the results.