Microsoft Azure MVP

Where should I Store my Logs? – Table / Queue / Blob

Andrew Gravell
Where should I Store my Logs? – Table / Queue / Blob
5 (100%) 1 vote

A common question that I get is, where should I store my logs?  Most people have chosen Tables, but is this always a best practice?

Verbose logs in block blobs

Logs are usually accessed in a group so you always access in a time range kind of limited fashion. I want to know what happened in my service from time A to B. And so storing this in blobs actually gives you a better throughput. What you could do is all your role instances of your process can start writing to its own log. It can buffer these log messages and then with a single writer can keep uploading these blocks whenever it reaches some amount of time or size and when you upload a hundred blocks or something like that so you can commit it. So you can use these blobs to start storing a raw log.

Store actionable errors into queues

However, there are critical messages sometimes in your system that you want to act upon quickly and that is where Tables will be used and Tables are being scanned to see if these critical messages (alerts) have appeared. You can store alerts in a queue and have a worker that processes it. In addition, you may also store alerts in table for reporting.

 


Leave a Comment

Your email address will not be published. Required fields are marked *

Loading...
1Code.Blog - Your #1 Code Blog