JSON to DataSet SQL CLR

I have been working with SQL CLR functions and procedure recently, and have come up with a crazy solution involving JSON, Analysis Services, and Visual Studio Database projects. Before we get into the crazy solution, we are going to take another step into SQL CLR Procedures which will help with understanding the pain points with using an external library. In a previous post we looked at creating JSON objects and in another post creating SQL CLR functions and procedures. In this blog post we are going to deserialize a JSON object and return the data as dataset to the results window.

The idea behind this blog post is to pass a JSON object to a SQL CLR procedure, deserialize the JSON object into a .Net object, load the .Net object into dataset, and return to dataset to the client. Now before you start thinking, “with SQL2016 we have JSON support, why do we need this?” One, an important part of this blog post is to show how to include external libraries into a CLR and how to allow SQL Server to use them. Two, we will need JSON in the crazy solution later, so showing how to work with JSON in .NET will help in the next blog post. Three, it will make for an “interesting” blog post and I could not think of anything better.

Data Setup

Here is the sample JSON object that we will pass to the SQL CLR procedure, which happens to be the grades from a student in a previous JSON blog post.

{
  "Grades": [
    {
      "CourseCode": "ICS140",
      "CourseName": "Introduction to Programming",
      "Grade": "A"
    },
    {
      "CourseCode": "ICS225",
      "CourseName": "Web Programming",
      "Grade": "B+"
    },
    {
      "CourseCode": "ICS240",
      "CourseName": "Advanced Programming",
      "Grade": "B-"
    },
    {
      "CourseCode": "ICS310",
      "CourseName": "Database Management",
      "Grade": "B"
    }
  ]
}

What we expect to return is a data set that looks like the following table.

CourseCode CourseName Grade
ICS140Introduction to ProgrammingA
ICS225Web ProgrammingB+
ICS240Advanced ProgrammingB-
ICS310Database ManagementB

CLR Object List

JsonToDataSet will accept a JSON object in the form of a NVARCHAR(MAX) column and will return a full dataset where the grade array will be converted into the dataset. The element names will become the column headers in the dataset.

Project Setup

Even though we are going to deploy this SQL CLR procedure to the same database, we are going to create a new project called JsonSQLCLR, and we will follow the same process as we did for our FirstSQLCLR project. From the Visual Studio start page, click on the “Create a new Project”, from the new project page, find and click on the “SQL Server Database Project”, and click “Next”. In the “Configure your new project” we are going to add JsonSQLCLR for the project name and select our path again.

SQL CLR

Once Visual Studio finishes configuring the SQL Server Database Project type, right click on the “JsonSQLCLR” project and select the “Add” option, then the “New Item…” option. In the Add New Item window we will select the “SQL CLR C#” option in the left options list. In the center window we will select the “SQL CLR C# Stored Procedure” option, and name it “JsonToDataSet.cs”.

SQL CLR

JsonToDataSet

In the JsonToDataSet method we will add code to send data back to the client using a SQLPipe object, and code to allow the JSON object to be passed into the procedure as a SqlString parameter.

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
 
public partial class StoredProcedures
{ 
  public static void JsonToDataSet(SqlString studentGrades)
  {
    SqlPipe sqlPipe = SqlContext.Pipe; 
  }
}

Now that we have the JSON object as a string in the method, we need to deserialize and put it into a .NET object that we can access and work with. Before we can deserialize the JSON object we need to define a class that we can use to reference and populate data into. We could write our own class, however when there are free tools available to do the work, we will use the free tools. BTW, I like free tools. This Json Utils site will generate C#.NET classes for us by passing in our JSON string. The output will look like the following, which we will add below our StoredProcedures class. At a high level, the following structures state we can have a student with a list of grades, and each grade can have a Corse Code, Course Name and Grade.

public class Grades
{
    public string CourseCode { get; set; }
    public string CourseName { get; set; }
    public string Grade { get; set; }
}
 
public class Student
{
    public IList<Grades> grades { get; set; }
}

The Student class implementation is using an IList type, which will cause the build to fail, because the project does not currently understand what an IList type is. The library for an IList type is found in the System  Collections  Generic library, so we will need to import the library. Add the following line near the top of the coding file, you will see other using statements there as well.

public class Gradesusing System.Collections.Generic;

After the line is added we should be able to compile the project.

Now that we have the Student and Grades classes implemented, we will need to desearlize the JSON object, in a SQL Server Database Project we do not have libraries loaded by default for working with JSON objects, so we will need to import a JSON library. There are a few libraries that can do this, however, some are not available to SQL Server project types. The ideal library is System.Text.Json which is available through NuGet, but currently it is not an option for SQL Server database projects or integration services projects. Instead, we are going to use Json.NET from Newtonsoft, which is also a great utility for working with JSON in C#.NET. We can download the libraries from https://www.newtonsoft.com/json. Add the Newtonsoft.Json.dll to a location that is accessible to your project and the SQL Server instance.

To add the JSON libraries into our project we will right click on “References” and select “Add References…” In the Add Reference window we will select Browse and navigate to the location where you stored Newtonsoft.Json.dll.

SQL CLR SQL CLR

After we have added a reference to the library in the project we need to import the Newtonsoft.Json library in our code. Add the following line near the top of the coding file, you will see other using statements there as well.

using Newtonsoft.Json;

Now we are ready to deserialize the JSON object. Within the Newtonsoft.JSON library, we are going to use a generic method called DeserializeOjbect to convert the JSON object into a Student object

JsonConvert.DeserializeObject<ClassType>([Json Object]);

The DeserializeObject method accepts a Student class in the method call and uses the JSON object passed in as a parameter to create a student object. At a high level the Student class gives a mapping to the DeserializeObject method, and the DeserializeObject method will try to populate as much as it can based on that mapping. If the JSON object passed in does not match, the mapping no data will be mapped and you will be left with an empty Student object. Here is the full call which will instantiate the student object and allow us to start working with it.

Student student = JsonConvert.DeserializeObject<Student>(studentGrades.ToString());

Now we have our JSON object deserialized, we can define out output. In our previous post we create a metadata object to store our formatted date. This worked for a single output, however this time we need to output three columns. So we will need to define a list of metadata objects. We will accomplish this by creating a generic list of type SqlMetaData. We will also use a short cut syntax for creating the SqlMetaData objects. Instead of creating variables and separate objects like this:

SqlMetaData courseCodeMetatData = new SqlMetaData("CourseCode", SqlDbType.NVarChar, 20)
sqlMetaData.Add(sqlMetaData);

We are going to use a shortcut syntax by skipping the variable creation and creating the objects directly in the sqlMetaData list. The full syntax will look like this:

List<SqlMetaData> sqlMetaData = new List<SqlMetaData>();
 
sqlMetaData.Add(new SqlMetaData("CourseCode", SqlDbType.NVarChar, 20));
sqlMetaData.Add(new SqlMetaData("CourseName", SqlDbType.NVarChar, 250));
sqlMetaData.Add(new SqlMetaData("Grade", SqlDbType.NVarChar, 5));
 
SqlDataRecord sqlDataRecord = new SqlDataRecord(sqlMetaData.ToArray());

Now we are ready to start sending data back to the client. Before we can start sending data we need to mark the beginning of the dataset that we are going to start sending to the client. To do this we will call the SendResultsStart method, this method will mark the beginning of the dataset sending process.

sqlPipe.SendResultsStart(sqlDataRecord);

Now that the pipe has been opened to the client we send data from the Student class back to the client. We will accomplish this by traversing the student object with a foreach loop, capturing and adding to the SQL Data Record list, and finally send each record back to the client.

foreach (Grades grade in student.grades)
{
    sqlDataRecord.SetString(0, grade.CourseCode);
    sqlDataRecord.SetString(1, grade.CourseName);
    sqlDataRecord.SetString(2, grade.Grade);
    sqlPipe.SendResultsRow(sqlDataRecord);
}

All that is left is to mark the end of the dataset in the SQL Pipe.

sqlPipe.SendResultsEnd();

Full Code

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using Newtonsoft.Json;
public partial class StoredProcedures
{
    [Microsoft.SqlServer.Server.SqlProcedure]
    public static void JsonToDataSet(SqlString studentGrades)
    {
        SqlPipe sqlPipe = SqlContext.Pipe;
 
        Student student = JsonConvert.DeserializeObject<Student>(studentGrades.ToString());
 
        List<SqlMetaData> sqlMetaData = new List<SqlMetaData>();
 
        sqlMetaData.Add(new SqlMetaData("CourseCode", SqlDbType.NVarChar, 20));
        sqlMetaData.Add(new SqlMetaData("CourseName", SqlDbType.NVarChar, 250));
        sqlMetaData.Add(new SqlMetaData("Grade", SqlDbType.NVarChar, 5));
 
        SqlDataRecord sqlDataRecord = new SqlDataRecord(sqlMetaData.ToArray());
 
        sqlPipe.SendResultsStart(sqlDataRecord);
        foreach (Grades grade in student.grades)
        {
            sqlDataRecord.SetString(0, grade.CourseCode);
            sqlDataRecord.SetString(1, grade.CourseName);
            sqlDataRecord.SetString(2, grade.Grade);
            sqlPipe.SendResultsRow(sqlDataRecord);
        }
        sqlPipe.SendResultsEnd();
    }
}
 
public class Grades
{
    public string CourseCode { get; set; }
    public string CourseName { get; set; }
    public string Grade { get; set; }
}
 
public class Student
{
    public IList<Grades> grades { get; set; }
}

SQL CLR Security

Before we deploy the JSON SQL CLR procedure we need to implement the same security features with an asymmetric key to allow the SQL CLR procedure to execute in the database. Here are the steps for creating an asymmetric key, signing the CLR project, and creating a login that binds the asymmetric key and allows the CLR objects to execute. More details from the previous post can be found here.

  1. Create an asymmetric key using the strong name utility command line tool installed with Visual Studio 2019.
  2. Create an ASYMMETRIC KEY in SQL Server
  3. Create a SQL Login based on the ASYMMETRIC KEY
  4. Grant Permissions to Assemblies using the SQL Login
  5. Sign the SQL CLR Project with the asymmetric key
SQL CLR

Deploy the CLR Objects

After the solution successfully compiles, the security has been configured both on the Database Project and SQL Instance, we are ready to deploy the solution to the SQL Server. To deploy the SQL CLR objects we will right click on the “JsonSQLCLR” project and select “Publish…” We are going to deploy to the same database as we did in the FirstSQLCLR blog post. We are doing this for two reasons, first to show that we can and second, we are being lazy and not creating another database.

SQL CLR

With any luck the deployment should have failed, which means that the assemblies and permissions have not been deployed and the database environment is not set to have Trustworthy=On. We should see an error like below. We will need to click on the “View Results” link to see the error message.

SQL CLR

As the error message states, we are seeing this error message because to the Newtsonsoft.Json library is not found in the database. So we will need to add the Newtsonsoft.Json library to the FirstSQLCLR database. We have a few options to add the library into the database. We can add the assembly as a trusted assembly in SQL server, we can create logins for the assemblies, or we can create certificates for the assemblies. Here are the sql scripts for each option.

  1. Using Trusted assemblies
    USE FirstSQLCLR
    GO
     
    DECLARE @clrName NVARCHAR(4000) = N'Newtonsoft.Json'
    DECLARE @clrBin VARBINARY(max) = 0x4D5A90000300000004000000FFFF0 – this is simplified for the blog post, this string is usually very large
    DECLARE @hash VARBINARY(64)
    SET @hash = hashbytes('SHA2_512', @clrBin);
     
    EXEC sys.sp__trusted_assembly @hash, @clrName;
     
    -- Now that we have a trusted the assembly we can add it into the database.
    CREATE ASSEMBLY [JsonSQLCLR]
        AUTHORIZATION [dbo]
        FROM 0x4D5A90000300000004000000FFFF0; – this is simplified for the blog post, this string is usually very large
  2. Using Asymmetric Keys
    USE master
    GO
     
    CREATE ASYMMETRIC KEY NewtonsoftJsonKey
    FROM EXECUTABLE FILE = 'C:\Development\SQLCLR\Newtonsoft.Json.dll';
     
    CREATE LOGIN NewtonsoftJsonLogin FROM ASYMMETRIC KEY NewtonsoftJsonKey;
     
    GRANT UNSAFE ASSEMBLY TO NewtonsoftJsonLogin;
     
    --Now that we have a login that has permissions to the assembly we can add it into the database.
    USE FirstSQLCLR
    GO
     
    CREATE ASSEMBLY [Newtonsoft.Json]
    FROM 'C:\Development\SQLCLR\Newtonsoft.Json.dll'
    WITH PERMISSION_SET = UNSAFE
  3. Using Certificates
    USE master
    GO
     
    CREATE CERTIFICATE NewtonsoftCert
    FROM EXECUTABLE FILE = 'C:\Development\SQLCLR\Newtonsoft.Json.dll';
    GO
     
    CREATE LOGIN NewtonsoftJsonLogin 
    FROM CERTIFICATE NewtonsoftJsonCert;
    GO
     
    GRANT UNSAFE ASSEMBLY TO NewtonsoftJsonLogin;
    GO
     
    --Now that we have a login that has permissions to the assembly we can add it into the database.
    USE FirstSQLCLR
    GO
     
    CREATE ASSEMBLY [Newtonsoft.Json]
    FROM 'C:\Development\SQLCLR\Newtonsoft.Json.dll'
    WITH PERMISSION_SET = UNSAFE
  4. In this blog post we are going to use option three the certificates. We will start out by creating the certificate in the master database using the following script. Being that we have placed the Newtonsoft.Json.dll in the proper directory these statements should execute successfully.

    USE master
    GO
     
    -- CERTIFICATE [Newtonsoft.Json]
    CREATE CERTIFICATE NewtonsoftCert
    FROM EXECUTABLE FILE = 'C:\Development\SQLCLR\Newtonsoft.Json.dll';
    GO
     
    CREATE LOGIN NewtonsoftJsonLogin 
    FROM CERTIFICATE NewtonsoftJsonCert;
    GO
     
    GRANT UNSAFE ASSEMBLY TO NewtonsoftJsonLogin;
    GO

    Now we are going to try adding the missing Newtonsoft.Json assembly into the database using the following script.

    USE FirstSQLCLR
    GO
     
    CREATE ASSEMBLY [Newtonsoft.Json]
    FROM 'C:\Development\SQLCLR\Newtonsoft.Json.dll'
    WITH PERMISSION_SET = UNSAFE

    Notice that now we got an error in SQL Server Management Studio? As the error states, we are missing another assembly. The reason here is that the Newtonsoft.Json assembly has a dependency on other assemblies. If the assembly that you are importing has multiple references you may have to import all of the other dependent references. This can be a tedious repetitive task, however, for this Json SQL CLR project here are the certificate, login, and assembly create statements. You may need to adjust the paths to the dll’s depending on the version Visual Studio and .Net installed.

    USE Master
    GO
    -- CERTIFICATE MicrosoftNetCert
    CREATE CERTIFICATE MicrosoftNetCert
    FROM EXECUTABLE FILE = 'C:\Windows\Microsoft.NET\assembly\GAC_MSIL\System.Runtime.Serialization\v4.0_4.0.0.0__b77a5c561934e089\System.Runtime.Serialization.dll'
    GO
     
    CREATE LOGIN MicrosoftNetLogin 
    FROM CERTIFICATE MicrosoftNetCert
    GO
     
    GRANT UNSAFE ASSEMBLY TO MicrosoftNetLogin
    GO
     
    -- CERTIFICATE [Newtonsoft.Json]
    CREATE CERTIFICATE NewtonsoftCert
    FROM EXECUTABLE FILE = 'C:\Development\SQLCLR\Newtonsoft.Json.dll';
    GO
     
    CREATE LOGIN NewtonsoftLogin 
    FROM CERTIFICATE NewtonsoftCert;
    GO
     
    GRANT UNSAFE ASSEMBLY TO NewtonsoftLogin;
    GO
     
     
    USE FirstSQLCLR
    GO
    -- ASSEMBLY [System.ServiceModel.Internals]
    CREATE ASSEMBLY [System.ServiceModel.Internals]
    FROM 'C:\Windows\Microsoft.NET\assembly\GAC_MSIL\System.ServiceModel.Internals\v4.0_4.0.0.0__31bf3856ad364e35\System.ServiceModel.Internals.dll'
    WITH PERMISSION_SET = UNSAFE
    GO
     
    --ASSEMBLY [SMDiagnostics]
    CREATE ASSEMBLY [SMDiagnostics]
    FROM 'C:\Windows\Microsoft.NET\assembly\GAC_MSIL\SMDiagnostics\v4.0_4.0.0.0__b77a5c561934e089\SMDiagnostics.dll'
    WITH PERMISSION_SET = UNSAFE
    GO
     
    -- ASSEMBLY [System.Runtime.Serialization]
    CREATE ASSEMBLY [System.Runtime.Serialization]
    FROM 'C:\Windows\Microsoft.NET\assembly\GAC_MSIL\System.Runtime.Serialization\v4.0_4.0.0.0__b77a5c561934e089\System.Runtime.Serialization.dll'
    WITH PERMISSION_SET = UNSAFE
    GO
     
    -- ASSEMBLY [Newtonsoft.Json]
    CREATE ASSEMBLY [Newtonsoft.Json]
    FROM 'C:\Development\SQLCLR\Newtonsoft.Json.dll'
    WITH PERMISSION_SET = UNSAFE;
    GO

    After executing scripts, providing everything was successful, you should see the assemblies in the FirstSQLCLR database under the Programmability –> Assemblies folder. Now that we have the assemblies created we can try to deploy our JSON SQL CLR procedure to the FirstSQLCLR database again. SQL CLR

    Execute the JSON CLR Procedure

    Now that we have successfully deployed the JSON SQL CLR procedure, we can try to execute it. We will need to create a variable to store the JSON object, and pass that to the procedure. We can execute the SQL CLR procedure with the following SQL Code. If everything goes well, we should see a result set with course and grade information populated based on the JSON object that was passed in.

    USE [FirstSQLCLR]
    GO
     
    DECLARE @studentGrades nvarchar(max) = '{
      "Grades": [
        {
          "CourseCode": "ICS140",
          "CourseName": "Introduction to Programming",
          "Grade": "A"
        },
        {
          "CourseCode": "ICS225",
          "CourseName": "Web Programming",
          "Grade": "B+"
        },
        {
          "CourseCode": "ICS240",
          "CourseName": "Advanced Programming",
          "Grade": "B-"
        },
        {
          "CourseCode": "ICS310",
          "CourseName": "Database Management",
          "Grade": "B"
        }
      ]
    }'
     
    EXECUTE [dbo].[JsonToDataSet]  @studentGrades
    GO
    SQL CLR

    Summary

    This was step two “partial craziness” for creating SQL CLR procedures in Visual Studio database projects. This post was focused more on importing libraries and assemblies into SQL Server than working with JSON. However, it also showed how we can build a bigger and more dynamic result set that can be returned from the CLR to the client. We have looked at a “slightly” more complex version of a CLR. Previously we looked at simple CLR objects and the intricacies of security and deployment. Now we are ready to look at the real craziness that I get myself into.

Simple SQL CLR Objects

I have been working with SQL CLR functions and procedures recently, and have come up with a crazy solution involving JSON, Analysis Services, and Visual Studio Database Projects. However, Before we get into the crazy solutions, we are going to take a look at a simple SQL CLR implementation. This simple implementation will include procedures and functions that return formatted dates. Using a simple solution will help with the intricacies of Visual Studio projects, CLR security, project deployment, and CLR execution. In this blog post we will take a look at creating SQL CLR functions in Visual Studio 2019. We will return data as a message as well as a dataset to the results window. We will implement proper SQL CLR security with asymmetric keys and signed assemblies.

The idea behind this blog post is to enhance the current CONVERT function that accepts formats for dates. There are plenty of predefined formats, however this SQL CLR function will allow you to build any date format that C#.net allows, here is a reference for building custom date formats. We will build three different SQL CLR objects, two procedure and one function.

CLR Object List

DateFormatMessage will accept a timestamp and a format, and will return the formatted date to the Message window.

DateFormatResult will accept a timestamp and a format, and will return the formatted date to the Result window.

DateFormatFunction will accept a timestamp and a format, and will return the formatted date as a result from a function call within a query call.

Project Setup

We will create our SQL CLR objects by creating a database project within Visual Studio 2019. After opening Visual Studio 2019 we will select the “Create a new Project” option

Visual Studio 2019 Landing Page

In the “Create new project” window we will select the “SQL Server Database Project” type. We can either scroll through all of the project types or you can do a key word search for the type.

Visual Studio 2019 New Project

In the “Configure your new project” window we are going to name our project “FirstSQLCLR”, specify the location where our source code will be stored on disk, and leave the solution name the same as the project.

Visual Studio 2019 Configure Project

Once Visual Studio finishes configuring the SQL Server Database Project type, we will have an empty database project. There are a couple ways to add the SQL CLR object to the project, we are going to use the Solution Explorer route. In the Solution Explorer windows we will right click on the “FirstSQLCLR” project and select the “Add” option, then the “New Item…” option.

Visual Studio 2019 Add New

In the Add New Item window we will select the “SQL CLR C#” option in the left options list. In the center window we will select the “SQL CLR C# Stored Procedure” option, and name it “DateForamtMessage.cs”.

ADD SQL CLR Procedure

In the DateFormatMessage.cs code window we will have an empty C# method body similar to the following code snippet.

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
 
public partial class StoredProcedures
{
    [Microsoft.SqlServer.Server.SqlProcedure]
    public static void DateFormatMessage ()
    {
        // Put your code here
    }
}

DateFormatMessage

In the DateFormatMessage method we will add code to send data back to the caller using a SQLPipe object, and code to format the passed in date with the passed in format. One thing to note here is that we are passing a string value back to the SqlPipe object, which will be returned to the message window in management studio.

Here is the completed code.

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
 
public partial class StoredProcedures
{
    [Microsoft.SqlServer.Server.SqlProcedure]
    public static void DateFormatMessage(SqlDateTime sqlDateTime, SqlString CSharpDateFormat)
    {
        SqlPipe sqlPipe = SqlContext.Pipe;
        DateTime dateTime = (DateTime)sqlDateTime;
 
        // Apply the date format to the ToString() function
        string formattedDate = dateTime.ToString(CSharpDateFormat.ToString());
 
        // send the results back
        sqlPipe.Send(formattedDate);
    }
}

DateFormatResult

We will repeat the previous steps for the DateFormatResult.cs SQL CLR procedure, however the code will be slightly modified from the message version. The additional code will allow us to return a record set instead of a single string. The following lines will define a record set object and populate the single value in the object.

// using a meta data object, setup the resultset configuration
SqlMetaData sqlMetaData = new SqlMetaData("formattedDate", SqlDbType.NVarChar, 25);
SqlDataRecord sqlDataRecord = new SqlDataRecord(sqlMetaData);
 
// store the data into the dataset
sqlDataRecord.SetString(0, formattedDate);

Here is the completed code.

using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
 
public partial class StoredProcedures
{
    [Microsoft.SqlServer.Server.SqlProcedure]
    public static void DateFormatResult(SqlDateTime sqlDateTime, SqlString CSharpDateFormat)
    {
        SqlPipe sqlPipe = SqlContext.Pipe; 
        DateTime dateTime = (DateTime)sqlDateTime;
 
        // Apply the date format to the ToString() function
        string formattedDate = dateTime.ToString(CSharpDateFormat.ToString());
 
        // using a meta data object, setup the resultset configuration
        SqlMetaData sqlMetaData = new SqlMetaData("formattedDate", SqlDbType.NVarChar, 25);
        SqlDataRecord sqlDataRecord = new SqlDataRecord(sqlMetaData);
 
        // store the data into the dataset
        sqlDataRecord.SetString(0, formattedDate);
 
        // send the results back
        sqlPipe.Send(sqlDataRecord);
    }
}

DateFormatFunction

To add the DateFormatFunction we will repeat most of the same steps, however we will need to choose a “SQL CLR C# User Defined Function” instead of the “SQL CLR C# Stored Procedure” option in the Add New Item window.

SQL CLR Add UDFunction

The code for a user defined function is simplified, as we will not need a SqlPipe to return the results. We are also only returning a single string value, so we will not need the record set object either. Being we are returning a single value from the function, we only need to return the formatted string.

Here is the completed code.

using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
 
public partial class UserDefinedFunctions
{
    [Microsoft.SqlServer.Server.SqlFunction]
    public static SqlString DateFormatFunction(SqlDateTime sqlDateTime, SqlString CSharpDateFormat)
    {
        DateTime dateTime = (DateTime)sqlDateTime;
 
        // Apply the date format to the ToString() function
        string formattedDate = dateTime.ToString(CSharpDateFormat.ToString());
 
        // return the formatted string 
        return new SqlString(formattedDate);
    }
}

We are ready to try to compile our code. Again there are multiple ways to compile our code, we will use the Solution Explorer route. In the Solution Explorer windows we will right click on the “FirstSQLCLR” project and select the “Rebuild” option.

SQL CLR Add UDFunction

Providing everything was typed in correctly and we have no build errors we should see that the build succeeded in the output window.

SQL CLR Add UDFunction

SQL CLR Security

Before we deploy the SQL CLR objects we need to implement the security features needed to allow custom code to run within SQL Server. There are a couple ways to do this, however, we are only going to focus on a secure option implementing asymmetric keys. We will not set the database’s TRUSTWORTHY option on ON, nor will we update the CLR Strict Security option. Here are the steps for creating an asymmetric key, signing the CLR project, and creating a login that binds the asymmetric key and allows the CLR objects to execute.

  1. Create an asymmetric key using the strong name utility command line tool installed with Visual Studio 2019.
  2. Create an ASYMMETRIC KEY in SQL Server
  3. Create a SQL Login based on the ASYMMETRIC KEY
  4. Grant Permissions to Assemblies using the SQL Login
  5. Sign the SQL CLR Project with the asymmetric key

Step 1: Create the Asymmetric Key

Sn.exe is a free utility shipped with Visual Studio that will create asymmetric key. We will need to open the command window with Administrator privileges to generate the key. We will run this command with just the –k switch and a path to store the key.

Strong Name Utility

Step 2: Create an ASYMMETRIC KEY in SQL Server

In the master database we need to create the asymmetric key from the file that was generated in Step 1. If you have not already created a master key for your SQL instance you will need to do that in this step.

USE MASTER
GO
 
-- Create the Master  Key for the SQL Instance
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '[Some Really Strong Password]';
 
-- Create the asymmetric key from the strong named key used to sign the CLR project
CREATE ASYMMETRIC KEY FirstSQLCLR FROM FILE = 'C:\Development\SQLCLR\FirstSQLCLR\FirstSQLCLR.snk'

Step 3: Create a SQL Login based on the ASYMMETRIC KEY

In the master database we will need to create a SQL Login that we be generated from the asymmetric key from Step 2. This login will allow SQL Server to access the assembly and execute based on the permissions in step 4.

USE MASTER
GO
 
-- Create a login from the asymmetric key to access the CLR
CREATE LOGIN FirstSQLCLR FROM ASYMMETRIC KEY FirstSQLCLR;

Step 4: Grant Permissions to Assemblies using the SQL Login

In the master database we need to assign assembly permissions to the Login from Step 3. We are choosing UNSAFE because it give us the most flexibility, however depending on your needs you may want to choose SAFE or EXTERNAL ACCESS. See the CLR Integration Code Access Security for more details

USE MASTER
GO
 
-- Set the assembly permission level for the login
GRANT UNSAFE ASSEMBLY TO FirstSQLCLR;

Step 5: Sign the SQL CLR Project with the asymmetric key

Back in Visual Studio, we need to access the project properties. We will right click on the “FirstSQLCLR” project and select “Properties”. When the properties window opens, we will select the “SQL CLR” option. We will select “UNSAFE” as the Permission Level and then click on the “Signing” button. This will open the Signing dialog box. In this dialog box we will select the “Sign the assembly” checkbox, then using the drop down list we will browse for the asymmetric key created in Step 1.

Visual Studio Project Settings

Deploy the CLR Objects

After the solution successfully compiles, the security has been configured both on the Database Project and SQL Instance, we are ready to deploy the solution to the SQL Server. To deploy the SQL CLR objects we will right click on the “FirstSQLCLR” project and select “Publish…”

Visual Studio Deploy SQL CLR

The Target Database Setting window will open. Here we will need to select the SQL instance and Database we want to deploy the SQL CLR object to. To configure the SQL Instance, select the “Edit…” button to complete the instance settings, specify the database name, and click on the “Publish” Button. We could script this out, however, that is a discussion for another day.

Deploy Database Settings

If the deployment was successful you will see a window like below.

Deployment Status

Moving over to the SQL Instance, if we refresh the database, we should now see the SQL CLR procedures and CLR function.

Management Studio CLR View

Execute the CLR Objects

Now that we have successfully deployed the SQL CLR Objects, we can try to execute them. Call a SQL CLR procedure looks very similar to a standard procedure in SQL Server. We will create variables for the date and format parameters and pass them to the procedure.

USE [FirstSQLCLR]
GO
 
DECLARE @sqlDateTime DATETIME = '2020-05-13 21:25:02.833'
      , @CSharpDateFormat VARCHAR(50) = 'yyyyMMdd'
 
EXEC [dbo].[DateFormatMessage] @sqlDateTime, @CSharpDateFormat

After we execute the procedure, we may see a message that looks like this: Msg 6263, Level 16, State 1, Line 7 Execution of user code in the .NET Framework is disabled. Enable “clr enabled” configuration option.

Here is a sample call.

SQL CLR Error

This error is because the current instance is not configured to allow SQL CLR objects from executing. We can turn this on with the following query to reconfigure the instance with the clr enabled flag.

EXEC sp_configure 'clr enabled', 1;  
RECONFIGURE;  
GO

After we apply the configuration setting and try to execute the procedure again we should have a successful query call and see the following result. Here you can see the formatted date in the Message window.

SQL CLR Message

Let’s try running the other procedure DateFormatResults, notice that this time instead of the procedure returning a string value to the message window we see the data returned in the Results window. Here is the code to call the procedure and the results.

USE [FirstSQLCLR]
GO
 
DECLARE @sqlDateTime DATETIME = '2020-05-13 21:25:02.833'
      , @CSharpDateFormat VARCHAR(50) = 'yyyyMMdd'
 
EXEC [dbo].[DateFormatResult] @sqlDateTime, @CSharpDateFormat
SQL CLR Result

Finally let’s take a look at executing the SQL CLR Function DateFormatFunction. Again we call this the same as we would call a user defined function. This time we will call the function in a select statement and the result is returned in the Results window.

USE [FirstSQLCLR]
GO
 
DECLARE @sqlDateTime DATETIME = '2020-05-13 21:25:02.833'
      , @CSharpDateFormat VARCHAR(50) = 'yyyyMMdd'
 
SELECT [dbo].[DateFormatFunction] (@sqlDateTime, @CSharpDateFormat)
SQL CLR Function

Summary

This was a “brief” introduction into creating SQL CLR objects from Visual Studio database projects. We took a look at the permission configuration with signing assemblies and creating asymmetric keys. In a future post we will look at creating another SQL CLR procedure where we read JSON objects. This will help show how to incorporate additional binaries into SQL Server. After that, we will finally get to the craziness with SQL Server, SSAS, CLR, JSON, and Visual Studio Database Projects.

Data Sets with JSON (SQL 2016)

With the release of SQL Server 2016 JSON objects support is now available. With the ever growing popularity and versatility of JSON, including JSON support into SQL Server is good for both database and application developers. In a previous post we looked at how to output a JSON object in a dataset using the FOR XML PATH clause and some string concatenation. This process can be rather difficult to read and updating can be a challenge. In this blog post we are going to use the FOR JSON PATH clause to simplify the JSON building process.

A brief background regarding JSON (JavaScript Object Notation). JSON is a lightweight data interchange format similar to XML, however it is based on specific programing languages such as C and Java. It is a data format that is easily understandable for both humans and machines. JSON is machine and language independent, which makes it a great language for data interchange between different systems. For more information regarding JSON please see the following URL https://www.json.org/json-en.html..

In the previous post our goal was to output the student information with a JSON object that described the courses and grades the student received.

{
  "Grades": [{
    "CourseCode": "[Course Code]",
    "CourseName": "[Course Name]",
    "Grade": "[Course Grade]"
  }]
}

In the previous blog post we used some crazy string concatenation to build out the JSON Object. Here is that query, with the string concatenation and using the SUBSTRING function to clean up the excess commas and whitespace.

SELECT s.StudentID
	 , s.FirstName
	 , s.LastName
	 , GradesJSON = '{"Grades":[' + SUBSTRING((
			SELECT ', {"CourseCode":"'+ c.CourseCode + '", "CourseName":"' + c.CourseName + '", "Grade":"' + sc.Grade + '"}' 
			FROM #StudentCourse sc JOIN #Courses c ON c.CourseID = sc.CourseID
			WHERE s.StudentID = sc.StudentID
			FOR XML PATH('')),3,1000) + ']}'
FROM #Students s;

Using the FOR JSON PATH clause instead of FOR XML PATH clause will simplify our syntax and make our code easier to read and understand. We will still need to output the same columns and use the correlated sub query idea to map the student course/grade data to the proper student in the outer query. The names of the columns will become the data elements in JSON object.

SELECT s.StudentID
     , s.FirstName
     , s.LastName
     , GradesJSON = (
           SELECT CourseCode = c.CourseCode
                , CourseName = c.CourseName
                , Grade = sc.Grade
           FROM #StudentCourse sc
           JOIN #Courses c ON c.CourseID = sc.CourseID
           WHERE s.StudentID = sc.StudentID
           FOR JSON PATH
       )
FROM #Students s
WHERE StudentID = 10005;
StudentID FirstName LastName Courses
10005WillWilliams[{“CourseCode”:”ICS140″, “CourseName”:”Introduction to Programming”, “Grade”:”A”}]

The only thing that is missing in the root grades element, so lets add that in.

SELECT s.StudentID
     , s.FirstName
     , s.LastName
     , GradesJSON = (
           SELECT CourseCode = c.CourseCode
                , CourseName = c.CourseName
                , Grade = sc.Grade
           FROM #StudentCourse sc
           JOIN #Courses c ON c.CourseID = sc.CourseID
           WHERE s.StudentID = sc.StudentID
           FOR JSON PATH, ROOT('Grades')
       )
FROM #Students s
WHERE StudentID = 10005;
StudentID FirstName LastName Courses
10005WillWilliams{“Grades”:[{“CourseCode”:”ICS140″, “CourseName”:”Introduction to Programming”, “Grade”:”A”}]}

There we have it, the same dataset structure including the nested JSON objects without all of the crazy string concatenation and SUBSTRING Function usage. Here is the full script:

DROP TABLE IF EXISTS #Students
GO
 
DROP TABLE IF EXISTS #Courses
GO
 
DROP TABLE IF EXISTS #StudentCourse
GO
 
CREATE TABLE #Students (
   StudentID INT PRIMARY KEY NOT NULL
 , FirstName NVARCHAR(50) NOT NULL
 , LastName NVARCHAR(50) NOT NULL
 )
GO
 
CREATE TABLE #Courses (
	CourseID INT PRIMARY KEY NOT NULL
  , CourseCode NVARCHAR(50) NOT NULL
  , CourseName NVARCHAR(50) NOT NULL
)
 
CREATE TABLE #StudentCourse(
	StudentID INT NOT NULL
  , CourseID INT NOT NULL
  , Grade VARCHAR(5) NULL
)
 
INSERT   INTO #Students
         (StudentID, FirstName, LastName)
VALUES   (10001, N'Bob',N'Roberts')
	   , (10002, N'John',N'Johnson')
	   , (10003, N'Tom',N'Tompson')
	   , (10004, N'Rich',N'Richardson')
	   , (10005, N'Will',N'Williams');
GO
 
INSERT INTO #Courses (
    CourseID
  , CourseCode
  , CourseName
)
VALUES (140, N'ICS140', N'Introduction to Programming')
     , (225, N'ICS225', N'Web Programming')
     , (240, N'ICS240', N'Advanced Programming')
     , (310, N'ICS310', N'Database Management');
 
INSERT INTO #StudentCourse (StudentID, CourseID, Grade)
VALUES (10001, 140, 'A')
     , (10001, 225, 'B+')
     , (10001, 240, 'B-')
     , (10001, 310, 'B')
     , (10002, 240, 'C+')
     , (10002, 310, 'A-')
     , (10004, 140, 'A')
     , (10004, 225, 'B')
     , (10005, 140, 'A');
 
 
SELECT s.StudentID
     , s.FirstName
     , s.LastName
     , GradesJSON = (
           SELECT CourseCode = c.CourseCode
                , CourseName = c.CourseName
                , Grade = sc.Grade
           FROM #StudentCourse sc
           JOIN #Courses c ON c.CourseID = sc.CourseID
           WHERE s.StudentID = sc.StudentID
           FOR JSON PATH, ROOT('Grades')
       )
FROM #Students s;

Data Sets with JSON

Prior to SQL Server 2016, SQL Server did not provide many options to work with JSON documents. However, JSON has become a favored format for developers to pass data between applications and application layers. This blog post we will take a look at how to build a JSON document and return it in a result set. We are going to build on a previous post where we built a comma separated list of values and returned them in a data set. Here is the result of the previous blog post

StudentID FirstName LastName Courses
10001BobRobertsICS140, ICS225, ICS240, ICS310
10002JohnJohnsonICS240, ICS310
10003TomTompson
10004RichRichardsonICS140, ICS225
10005WillWilliamsICS140

This time we are going to change the output to return a JSON object that will contain the course code, course name and grade the student received. The JSON document structure should look like this:

{
  "Grades": [{
    "CourseCode": "[Course Code]",
    "CourseName": "[Course Name]",
    "Grade": "[Course Grade]"
  }]
}

Before we dig into this blog a brief background regarding JSON (JavaScript Object Notation) may help. JSON is a lightweight data interchange format similar to XML, however it is based on programing languages constructs derived from languages such as C and Java. It is a data format that is easily understandable for both humans and machines, making it ideal for transferring data between applications. JSON is machine and language independent, which makes it a great language for data interchange between different systems. For more information regarding JSON please see the following URL https://www.json.org/json-en.html.

We have one minor tweak to the table structure from the previous blog post where we add the grade to the Student Course weak entity table

CREATE TABLE #StudentCourse(
    StudentID INT NOT NULL
  , CourseID INT NOT NULL
  , Grade VARCHAR(5) NULL
);
GO
 
INSERT INTO #StudentCourse (StudentID, CourseID, Grade)
VALUES (10001, 140, 'A')
     , (10001, 225, 'B+')
     , (10001, 240, 'B-')
     , (10001, 310, 'B')
     , (10002, 240, 'C+')
     , (10002, 310, 'A-')
     , (10004, 140, 'A')
     , (10004, 225, 'B')
     , (10005, 140, 'A');

What we want the script to return is a result set with student details including a JSON document with course and grade information which will look like this:

StudentID FirstName LastName Courses
10001BobRoberts{“Grades”:[{“CourseCode”:”ICS140″, “CourseName”:”Introduction to Programming”, “Grade”:”A”}, {“CourseCode”:”ICS225″, “CourseName”:”Web Programming”, “Grade”:”B+”}, {“CourseCode”:”ICS240″, “CourseName”:”Advanced Programming”, “Grade”:”B-“}, {“CourseCode”:”ICS310″, “CourseName”:”Database Management”, “Grade”:”B”}]}
10002JohnJohnson{“Grades”:[{“CourseCode”:”ICS240″, “CourseName”:”Advanced Programming”, “Grade”:”C+”}, {“CourseCode”:”ICS310″, “CourseName”:”Database Management”, “Grade”:”A-“}]}
10003TomTompson
10004RichRichardson{“Grades”:[{“CourseCode”:”ICS140″, “CourseName”:”Introduction to Programming”, “Grade”:”A”}, {“CourseCode”:”ICS225″, “CourseName”:”Web Programming”, “Grade”:”B”}]}
10005WillWilliams{“Grades”:[{“CourseCode”:”ICS140″, “CourseName”:”Introduction to Programming”, “Grade”:”A”}]}

We will reuse most of the query and simply add some additional string concatenation. The previous sub query just concatenated a comma to the course code, this time we are going to build repeating JSON objects within the array.

{"CourseCode": "[Course Code]","CourseName": "[Course Name]","Grade": "[Course Grade]"}
-- CSV Version
SELECT ', ' + sc.Grade 
FROM #StudentCourse sc JOIN #Courses c ON c.CourseID = sc.CourseID
WHERE s.StudentID = sc.StudentID
FOR XML PATH('')
 
-- New JSON Version
SELECT ', {"CourseCode":"'+ c.CourseCode + '", "CourseName":"' + c.CourseName + '", "Grade":"' + sc.Grade + '"}' 
FROM #StudentCourse sc JOIN #Courses c ON c.CourseID = sc.CourseID
WHERE s.StudentID = sc.StudentID
FOR XML PATH('')

Now that we have the repeating JSON objects within the array, we need to build out the top object in the document and the array structure. We can accomplish this by adding additional string concatenation in the outer portion of the query.

SELECT s.StudentID
	 , s.FirstName
	 , s.LastName
	 , GradesJSON = '{"Grades":[' + SUBSTRING((
			SELECT ', {"CourseCode":"'+ c.CourseCode + '", "CourseName":"' + c.CourseName + '", "Grade":"' + sc.Grade + '"}' 
			FROM #StudentCourse sc JOIN #Courses c ON c.CourseID = sc.CourseID
			WHERE s.StudentID = sc.StudentID
			FOR XML PATH('')),3,1000) + ']}'
FROM #Students s;

Here is the full script:

DROP TABLE IF EXISTS #Students
GO
 
DROP TABLE IF EXISTS #Courses
GO
 
DROP TABLE IF EXISTS #StudentCourse
GO
 
CREATE TABLE #Students (
   StudentID INT PRIMARY KEY NOT NULL
 , FirstName NVARCHAR(50) NOT NULL
 , LastName NVARCHAR(50) NOT NULL
 )
GO
 
CREATE TABLE #Courses (
	CourseID INT PRIMARY KEY NOT NULL
  , CourseCode NVARCHAR(50) NOT NULL
  , CourseName NVARCHAR(50) NOT NULL
)
 
CREATE TABLE #StudentCourse(
	StudentID INT NOT NULL
  , CourseID INT NOT NULL
  , Grade VARCHAR(5) NULL
)
 
INSERT   INTO #Students
         (StudentID, FirstName, LastName)
VALUES   (10001, N'Bob',N'Roberts')
	   , (10002, N'John',N'Johnson')
	   , (10003, N'Tom',N'Tompson')
	   , (10004, N'Rich',N'Richardson')
	   , (10005, N'Will',N'Williams');
GO
 
INSERT INTO #Courses (
    CourseID
  , CourseCode
  , CourseName
)
VALUES (140, N'ICS140', N'Introduction to Programming')
     , (225, N'ICS225', N'Web Programming')
     , (240, N'ICS240', N'Advanced Programming')
     , (310, N'ICS310', N'Database Management');
 
INSERT INTO #StudentCourse (StudentID, CourseID, Grade)
VALUES (10001, 140, 'A')
     , (10001, 225, 'B+')
     , (10001, 240, 'B-')
     , (10001, 310, 'B')
     , (10002, 240, 'C+')
     , (10002, 310, 'A-')
     , (10004, 140, 'A')
     , (10004, 225, 'B')
     , (10005, 140, 'A');
 
 
SELECT s.StudentID
	 , s.FirstName
	 , s.LastName
	 , GradesJSON = '{"Grades":[' + SUBSTRING((
			SELECT ', {"CourseCode":"'+ c.CourseCode + '", "CourseName":"' + c.CourseName + '", "Grade":"' + sc.Grade + '"}' 
			FROM #StudentCourse sc JOIN #Courses c ON c.CourseID = sc.CourseID
			WHERE s.StudentID = sc.StudentID
			FOR XML PATH('')),3,1000) + ']}'
FROM #Students s;

In a future post I will show how to build this Data Set using JSON functions in SQL 2016.

Comma Separated Values using the FOR XML PATH clause

In a previous post we looked at how we can build out a delimited list using the ISNULL and COALESCE functions. This functionality is great if you need to return the delimited list through a function or persist into a variable, however what if you need to return a comma separated list in a result set? This blog post will look at, and describe, how to build out a delimited or comma separated list using FOR XML PATH clause. Using tricks with the FOR XML PATH clause, a correlated subqueries, and un-named columns will give us all the tools needed to return a delimited list within a result set.

So the idea for this blog post is that we take a list of student’s details and include a comma separated list of the course codes that they have registered for. Here is a data diagram that represents our sample dataset.

CSV FOR XML PATH

Here are the populated tables.

StudentID FirstName LastName
10001 Bob Roberts
10002 John Johnson
10003 Tom Thompson
10004 Rich Richardson
10005 Will Williams

310ICS310Database Management

CourseID CourseCode CourseName
140 ICS140 Introduction to Programming
225 ICS225 Web Programming
240 ICS240 Advanced Programming
StudentID CourseID
10001 140
10001 225
10001 240
10001 310
10002 240
10002 310
10004 140
10004 225
10005 140

What we would expect the script to return is a result set with student details including a comma separated list of courses that looks like this:

StudentID FirstName LastName Courses
10001 Bob Roberts ICS140, ICS225, ICS240, ICS310
10002 John Johnson ICS240, ICS310
10003 Tom Tompson  
10004 Rich Richardson ICS140, ICS225
10005 Will Williams ICS140

Using the following temp table structure and sampling of student and course data gives us the data that we will need for this example.

DROP TABLE IF EXISTS #Students
GO
DROP TABLE IF EXISTS #Courses
GO
DROP TABLE IF EXISTS #StudentCourse
GO
 
CREATE TABLE #Students (
   StudentID INT PRIMARY KEY NOT NULL
 , FirstName NVARCHAR(50) NOT NULL
 , LastName NVARCHAR(50) NOT NULL
 )
GO
 
CREATE TABLE #Courses (
    CourseID INT PRIMARY KEY NOT NULL
  , CourseCode NVARCHAR(50) NOT NULL
  , CourseName NVARCHAR(50) NOT NULL
)
 
CREATE TABLE #StudentCourse(
    StudentID INT NOT NULL
  , CourseID INT NOT NULL
)
 
INSERT   INTO #Students (StudentID, FirstName, LastName)
VALUES   (10001, N'Bob',N'Roberts')
       , (10002, N'John',N'Johnson')
       , (10003, N'Tom',N'Tompson')
       , (10004, N'Rich',N'Richardson')
       , (10005, N'Will',N'Williams');
GO
 
INSERT INTO #Courses (
    CourseID
  , CourseCode
  , CourseName
)
VALUES (140, N'ICS140', N'Introduction to Programming')
     , (225, N'ICS225', N'Web Programming')
     , (240, N'ICS240', N'Advanced Programming')
     , (310, N'ICS310', N'Database Management');
 
INSERT INTO #StudentCourse (StudentID, CourseID)
VALUES (10001, 140)
     , (10001, 225)
     , (10001, 240)
     , (10001, 310)
     , (10002, 240)
     , (10002, 310)
     , (10004, 140)
     , (10004, 225)
     , (10005, 140);

We are going to start out by creating a simple query that returns all of the data we need, to generate the output. This will include all data from the student table, all data from the student course weak entity table, and only course code data from the course table.

SELECT s.StudentID, s.FirstName, s.LastName, c.CourseCode
FROM #Students s
LEFT JOIN #StudentCourse sc ON sc.StudentID = s.StudentID
LEFT JOIN #Courses c ON c.CourseID = sc.CourseID
StudentID FirstName LastName CourseCode
10001 Bob Roberts ICS140
10001 Bob Roberts ICS225
10001 Bob Roberts ICS240
10001 Bob Roberts ICS310
10002 John Johnson ICS240
10002 John Johnson ICS310
10003 Tom Tompson  
10004 Rich Richardson ICS140
10004 Rich Richardson ICS225
10005 Will Williams ICS140

Now that we have our dataset, let’s turn our focus on creating the comma separated list of course codes. Here we will implement our firsts bit of trickery. We are going to use the FOR XML clause, however we are going to pass an empty string to the PATH function. This will rename the XML root node as an empty string, thus not returning a root node. You can test this by running both queries, the first query will have a root node of <row> whereas the second query only returns the <CourseCode> nodes.

SELECT c.CourseCode
FROM #Students s
LEFT JOIN #StudentCourse sc ON sc.StudentID = s.StudentID
LEFT JOIN #Courses c ON c.CourseID = sc.CourseID
FOR XML PATH;
 
SELECT c.CourseCode
FROM #Students s
LEFT JOIN #StudentCourse sc ON sc.StudentID = s.StudentID
LEFT JOIN #Courses c ON c.CourseID = sc.CourseID
FOR XML PATH('');

Being that we were able to remove the root node by simply renaming the node, we need to do the same for the column name CourseCode. Here comes some more trickery, if we concatenate a string and do not provide an alias, the column name is undefined and thus is empty. In Management Studio you should see something like this “(No column name)”. This trickery actually works out to have an advantage because we need to concatenate our strings with a comma to create a comma separated list of values. So running the following query will give us all of the values in a comma separated list.

SELECT ', ' + c.CourseCode
FROM #Students s
LEFT JOIN #StudentCourse sc ON sc.StudentID = s.StudentID
LEFT JOIN #Courses c ON c.CourseID = sc.CourseID
FOR XML PATH('');
, ICS140, ICS225, ICS240, ICS310, ICS240, ICS310, ICS140, ICS225, ICS140

So now we have all of the course codes in one big comma separated list, we need to get them into sub lists for each student. If we nest the query with the FOR XML clause in a sub query and reference the student id from the outer query, this will limit the comma separated lists to only the course codes that are linked to the students.

SELECT s.StudentID
	 , s.FirstName
	 , s.LastName
	 , Courses = (
			SELECT ', ' + c.CourseCode
			FROM #StudentCourse sc JOIN #Courses c ON c.CourseID = sc.CourseID
			WHERE s.StudentID = sc.StudentID
			FOR XML PATH(''))
FROM #Students s;

The last piece is a bit of data clean up, using a STUFF or SUBSTRING we can remove the extra comma and white space from the comma separated list. Here is the final query that will return the result with a comma separated list in the result set.

SELECT s.StudentID
	 , s.FirstName
	 , s.LastName
	 , Courses = SUBSTRING((
			SELECT ', ' + c.CourseCode
			FROM #StudentCourse sc JOIN #Courses c ON c.CourseID = sc.CourseID
			WHERE s.StudentID = sc.StudentID
			FOR XML PATH('')),3,1000)
FROM #Students s;