Share via

how to properly parse data in custom logs for log analytcis(i have no response from query actually) ?

ALGOURDIN Erwan 1 Reputation point
2021-06-24T13:58:30.197+00:00

I have some data, in a txt file. i create my txt with a powershell script who encode in utf8, but anything work when i import the log in log analytics.
the data is parse like this in the txt:
Time=Date Data1=data Data2=data2
Time=Date Data1=data Data2=data2

need to know how to parse my data properly, to have response with my query

Cordially

Microsoft Security | Microsoft Sentinel
0 comments No comments

1 answer

Sort by: Most helpful
  1. Konstantinos Lianos 225 Reputation points Student Ambassador
    2026-03-18T09:07:02.49+00:00

    Hello Erwan,

    Your file format is valid for Custom Text Logs via AMA, but Azure Monitor won’t automatically split Time=... Data1=... Data2=... into separate fields. For custom text logs, each log entry is typically ingested as a single string in the RawData column unless you parse it in a query or with a DCR transformation. Also, the file must be ASCII or UTF-8 (not UTF-16), and each new record should be appended on a new line.

    start by querying it like this:

    CustomTable_CL

    | parse RawData with "Time=" EventTime " Data1=" Data1 " Data2=" Data2

    | extend EventTime = todatetime(EventTime)

    | project TimeGenerated, EventTime, Data1, Data2, RawData

    Make sure the file is really saved as UTF-8 or ASCII. Azure Monitor text collection does not support UTF-16.

    Make sure each event is on one line. For custom text logs, if Azure Monitor doesn’t detect a supported timestamp format for record splitting, it falls back to end-of-line as the delimiter.

    Make sure new data is appended to the file, not overwritten.

    If you want the fields to be stored as separate columns permanently, use a DCR transformation. Transformation output must match the destination table schema and must include TimeGenerated.

    0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.