SqlClient Instrumentation for OpenTelemetry

This is an Instrumentation Library, which instruments Microsoft.Data.SqlClient and System.Data.SqlClient and collects traces about database operations.

[!WARNING] Instrumentation is not working with Microsoft.Data.SqlClient v3.* due to the issue. It was fixed in 4.0 and later.

[!CAUTION] This component is based on the OpenTelemetry semantic conventions for traces. These conventions are in Development, and hence, this package is a pre-release. Until a stable version is released, there can be breaking changes.

Steps to enable OpenTelemetry.Instrumentation.SqlClient

Step 1: Install Package

Add a reference to the OpenTelemetry.Instrumentation.SqlClient package. Also, add any other instrumentations & exporters you will need.

dotnet add package --prerelease OpenTelemetry.Instrumentation.SqlClient

Step 2: Enable SqlClient Instrumentation at application startup

SqlClient instrumentation must be enabled at application startup.

Traces

The following example demonstrates adding SqlClient traces instrumentation to a console application. This example also sets up the OpenTelemetry Console exporter, which requires adding the package OpenTelemetry.Exporter.Console to the application.

using OpenTelemetry.Trace;

public class Program
{
    public static void Main(string[] args)
    {
        using var tracerProvider = Sdk.CreateTracerProviderBuilder()
            .AddSqlClientInstrumentation()
            .AddConsoleExporter()
            .Build();
    }
}

Metrics

The following example demonstrates adding SqlClient metrics instrumentation to a console application. This example also sets up the OpenTelemetry Console exporter, which requires adding the package OpenTelemetry.Exporter.Console to the application.

using OpenTelemetry.Metrics;

public class Program
{
    public static void Main(string[] args)
    {
        using var tracerProvider = Sdk.CreateMeterProviderBuilder()
            .AddSqlClientInstrumentation()
            .AddConsoleExporter()
            .Build();
    }
}

List of metrics produced

The instrumentation is implemented based on metrics semantic conventions. Currently, the instrumentation supports the following metric.

ASP.NET Core

For an ASP.NET Core application, adding instrumentation is typically done in the ConfigureServices of your Startup class. Refer to documentation for OpenTelemetry.Instrumentation.AspNetCore.

ASP.NET

For an ASP.NET application, adding instrumentation is typically done in the Global.asax.cs. Refer to the documentation for OpenTelemetry.Instrumentation.AspNet.

Advanced configuration

This instrumentation can be configured to change the default behavior by using SqlClientTraceInstrumentationOptions.

Enrich

[!NOTE] Enrich is available on .NET runtimes only.

This option can be used to enrich the activity with additional information from the raw SqlCommand object. The Enrich action is called only when activity.IsAllDataRequested is true. It contains the activity itself (which can be enriched), the name of the event, and the actual raw object.

Currently there is only one event name reported, "OnCustom". The actual object is Microsoft.Data.SqlClient.SqlCommand for Microsoft.Data.SqlClient and System.Data.SqlClient.SqlCommand for System.Data.SqlClient.

The following code snippet shows how to add additional tags using Enrich.

using var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddSqlClientInstrumentation(opt => opt.Enrich
        = (activity, eventName, rawObject) =>
    {
        if (eventName.Equals("OnCustom"))
        {
            if (rawObject is SqlCommand cmd)
            {
                activity.SetTag("db.commandTimeout", cmd.CommandTimeout);
            }
        };
    })
    .Build();

Processor, is the general extensibility point to add additional properties to any activity. The Enrich option is specific to this instrumentation, and is provided to get access to SqlCommand object.

RecordException

[!NOTE] RecordException is available on .NET runtimes only.

This option can be set to instruct the instrumentation to record SqlExceptions as Activity events.

The default value is false and can be changed by the code like below.

using var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddSqlClientInstrumentation(
        options => options.RecordException = true)
    .AddConsoleExporter()
    .Build();

Filter

[!NOTE] Filter is available on .NET runtimes only.

This option can be used to filter out activities based on the properties of the SqlCommand object being instrumented using a Func<object, bool>. The function receives an instance of the raw SqlCommand and should return true if the telemetry is to be collected, and false if it should not. The parameter of the Func delegate is of type object and needs to be cast to the appropriate type of SqlCommand, either Microsoft.Data.SqlClient.SqlCommand or System.Data.SqlClient.SqlCommand. The example below filters out all commands that are not stored procedures.

using var traceProvider = Sdk.CreateTracerProviderBuilder()
   .AddSqlClientInstrumentation(
       opt =>
       {
           opt.Filter = cmd =>
           {
               if (cmd is SqlCommand command)
               {
                   return command.CommandType == CommandType.StoredProcedure;
               }

               return false;
           };
       })
   .AddConsoleExporter()
   .Build();

Trace Context Propagation

[!NOTE] Only CommandType.Text commands are supported for trace context propagation. Only .NET runtimes are supported.

Database trace context propagation can be enabled by setting OTEL_DOTNET_EXPERIMENTAL_SQLCLIENT_ENABLE_TRACE_CONTEXT_PROPAGATION environment variable to true. This uses the SET CONTEXT_INFO command to set traceparent information for the current connection, which results in an additional round-trip to the database.

Experimental features

[!NOTE] Experimental features are not enabled by default and can only be activated with environment variables. They are subject to change or removal in future releases.

DB query parameters

[!NOTE] This feature is available on .NET runtimes only.

The OTEL_DOTNET_EXPERIMENTAL_SQLCLIENT_ENABLE_TRACE_DB_QUERY_PARAMETERS environment variable controls whether db.query.parameter.<key> attributes are emitted.

Query parameters may contain sensitive data, so only enable this experimental feature if your queries and/or environment are appropriate for enabling this option.

OTEL_DOTNET_EXPERIMENTAL_SQLCLIENT_ENABLE_TRACE_DB_QUERY_PARAMETERS is implicitly false by default. When set to true, the instrumentation will set db.query.parameter.<key> attributes for each of the query parameters associated with a database command.

Activity Duration calculation

Activity.Duration represents the time the underlying connection takes to execute the command/query. Completing the operation includes the time up to determining that the request was successful. It doesn't include the time spent reading the results from a query set (for example enumerating all the rows returned by a data reader).

This is illustrated by the code snippet below:

using var connection = new SqlConnection("...");
connection.Open();

using var command = connection.CreateCommand();
command.CommandText = "select top 100000 * from Users";

// Activity duration starts
using var reader = command.ExecuteReader();
// Activity duration ends

// Not included in the Activity duration
while (reader.Read())
{
}

References

• OpenTelemetry Project

• OpenTelemetry semantic conventions for database calls