Populate SQL tables with test data

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;







up vote
2
down vote

favorite












I have some unit-tests that must be executed against a real database. This requires populating the database with test-data.



In order to simplify this process I created the SqlTableSeeder. It uses the SqlBulkCopy internally to insert the data from the specified DataTable. Optionally it truncates the target table first.



The reason why I picked the SqlBulkcopy is because certain queries need to be tested for performance and I'll be creating data-tables with a couple of millions of rows.



public static class SqlTableSeeder

public static async Task SeedAsync(this SqlConnection connection, string schema, string table, DataTable data, bool truncate = true)

if (Transaction.Current == null)

throw new InvalidOperationException($"nameof(SeedAsync) can be executed only within a transaction scope.");


var identifier = connection.CreateIdentifier(schema, table);

if (truncate)

// Using "truncate" because some databases/tables do not allow "delete".
await connection.ExecuteQueryAsync($"truncate table identifier", command => command.ExecuteNonQueryAsync());


using (var bulkCopy = new SqlBulkCopy(connection, SqlBulkCopyOptions.KeepIdentity, null))

bulkCopy.DestinationTableName = identifier;

foreach (var column in data.Columns.Cast<DataColumn>().Select(c => c.ColumnName))

bulkCopy.ColumnMappings.Add(column, column);


await bulkCopy.WriteToServerAsync(data);



public static void Seed(this SqlConnection connection, string schema, string table, DataTable data, bool truncate = true)

SeedAsync(connection, schema, table, data, truncate).GetAwaiter().GetResult();




where CreateIdentifier is an extension implemented as follows:



 public static string CreateIdentifier(this SqlConnection connection, params string names)

if (connection == null) throw new ArgumentNullException(nameof(connection));

using (var commandBuilder = DbProviderFactories.GetFactory(connection).CreateCommandBuilder())

// ReSharper disable once PossibleNullReferenceException - commandBuilder is never null for SqlConnection.
return names.Select(commandBuilder.QuoteIdentifier).Join(".");





Example



As an example of how I'm using it, I post one of the unit-tests I created for it and the schema of the test table.



I import the test-data from a CSV, I turn into a DataTable and pass this to the seeder which takes just 4 lines of code (not counting the constants).



[TestClass]
public class SqlTableSeederTest

private const string ConnectonString = "Data Source=(local);Initial Catalog=TestDb;Integrated Security=SSPI;";
private const string Schema = "dbo";
private const string Table = "SqlTableSeederTest";
private static readonly ITypeConverter Converter =
TypeConverter
.Empty
.Add<StringToInt32Converter>()
.Add<StringToDateTimeConverter>();

[TestMethod]
public void Seed_WithoutId_Seeded()

var csvReader = CsvReader.FromFile(@"testdataSqlTableSeederTest-without-id.csv");
var sqlColumns = SqlHelper.Execute(ConnectonString, connection => connection.GetColumnFrameworkTypes(Schema, Table));
var csv = csvReader.AsEnumerable().ToDataTable(sqlColumns, Converter);

SqlHelper.Execute(ConnectonString, connection => connection.Seed(Schema, Table, csv));

Assert.AreEqual(3, SqlHelper.Execute(ConnectonString, connection =>

return connection.ExecuteQuery($"select count(*) from [Schema].[Table]", command => command.ExecuteScalar());
));




CREATE TABLE [dbo].[SqlTableSeederTest](
[_id] [int] IDENTITY(1,1) NOT NULL,
[_nvarchar] [nvarchar](50) NULL,
[_datetime] [datetime2](7) NULL,
CONSTRAINT [PK_SqlTableSeederTest] PRIMARY KEY CLUSTERED
(
[_id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]



So what do you think of this helper? Could it be even shorter and still SOLID and easy to use? Did I miss anything important? I'll be needing it in several unit-test projects so convenience is an important factor too.







share|improve this question



























    up vote
    2
    down vote

    favorite












    I have some unit-tests that must be executed against a real database. This requires populating the database with test-data.



    In order to simplify this process I created the SqlTableSeeder. It uses the SqlBulkCopy internally to insert the data from the specified DataTable. Optionally it truncates the target table first.



    The reason why I picked the SqlBulkcopy is because certain queries need to be tested for performance and I'll be creating data-tables with a couple of millions of rows.



    public static class SqlTableSeeder

    public static async Task SeedAsync(this SqlConnection connection, string schema, string table, DataTable data, bool truncate = true)

    if (Transaction.Current == null)

    throw new InvalidOperationException($"nameof(SeedAsync) can be executed only within a transaction scope.");


    var identifier = connection.CreateIdentifier(schema, table);

    if (truncate)

    // Using "truncate" because some databases/tables do not allow "delete".
    await connection.ExecuteQueryAsync($"truncate table identifier", command => command.ExecuteNonQueryAsync());


    using (var bulkCopy = new SqlBulkCopy(connection, SqlBulkCopyOptions.KeepIdentity, null))

    bulkCopy.DestinationTableName = identifier;

    foreach (var column in data.Columns.Cast<DataColumn>().Select(c => c.ColumnName))

    bulkCopy.ColumnMappings.Add(column, column);


    await bulkCopy.WriteToServerAsync(data);



    public static void Seed(this SqlConnection connection, string schema, string table, DataTable data, bool truncate = true)

    SeedAsync(connection, schema, table, data, truncate).GetAwaiter().GetResult();




    where CreateIdentifier is an extension implemented as follows:



     public static string CreateIdentifier(this SqlConnection connection, params string names)

    if (connection == null) throw new ArgumentNullException(nameof(connection));

    using (var commandBuilder = DbProviderFactories.GetFactory(connection).CreateCommandBuilder())

    // ReSharper disable once PossibleNullReferenceException - commandBuilder is never null for SqlConnection.
    return names.Select(commandBuilder.QuoteIdentifier).Join(".");





    Example



    As an example of how I'm using it, I post one of the unit-tests I created for it and the schema of the test table.



    I import the test-data from a CSV, I turn into a DataTable and pass this to the seeder which takes just 4 lines of code (not counting the constants).



    [TestClass]
    public class SqlTableSeederTest

    private const string ConnectonString = "Data Source=(local);Initial Catalog=TestDb;Integrated Security=SSPI;";
    private const string Schema = "dbo";
    private const string Table = "SqlTableSeederTest";
    private static readonly ITypeConverter Converter =
    TypeConverter
    .Empty
    .Add<StringToInt32Converter>()
    .Add<StringToDateTimeConverter>();

    [TestMethod]
    public void Seed_WithoutId_Seeded()

    var csvReader = CsvReader.FromFile(@"testdataSqlTableSeederTest-without-id.csv");
    var sqlColumns = SqlHelper.Execute(ConnectonString, connection => connection.GetColumnFrameworkTypes(Schema, Table));
    var csv = csvReader.AsEnumerable().ToDataTable(sqlColumns, Converter);

    SqlHelper.Execute(ConnectonString, connection => connection.Seed(Schema, Table, csv));

    Assert.AreEqual(3, SqlHelper.Execute(ConnectonString, connection =>

    return connection.ExecuteQuery($"select count(*) from [Schema].[Table]", command => command.ExecuteScalar());
    ));




    CREATE TABLE [dbo].[SqlTableSeederTest](
    [_id] [int] IDENTITY(1,1) NOT NULL,
    [_nvarchar] [nvarchar](50) NULL,
    [_datetime] [datetime2](7) NULL,
    CONSTRAINT [PK_SqlTableSeederTest] PRIMARY KEY CLUSTERED
    (
    [_id] ASC
    )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
    ) ON [PRIMARY]



    So what do you think of this helper? Could it be even shorter and still SOLID and easy to use? Did I miss anything important? I'll be needing it in several unit-test projects so convenience is an important factor too.







    share|improve this question























      up vote
      2
      down vote

      favorite









      up vote
      2
      down vote

      favorite











      I have some unit-tests that must be executed against a real database. This requires populating the database with test-data.



      In order to simplify this process I created the SqlTableSeeder. It uses the SqlBulkCopy internally to insert the data from the specified DataTable. Optionally it truncates the target table first.



      The reason why I picked the SqlBulkcopy is because certain queries need to be tested for performance and I'll be creating data-tables with a couple of millions of rows.



      public static class SqlTableSeeder

      public static async Task SeedAsync(this SqlConnection connection, string schema, string table, DataTable data, bool truncate = true)

      if (Transaction.Current == null)

      throw new InvalidOperationException($"nameof(SeedAsync) can be executed only within a transaction scope.");


      var identifier = connection.CreateIdentifier(schema, table);

      if (truncate)

      // Using "truncate" because some databases/tables do not allow "delete".
      await connection.ExecuteQueryAsync($"truncate table identifier", command => command.ExecuteNonQueryAsync());


      using (var bulkCopy = new SqlBulkCopy(connection, SqlBulkCopyOptions.KeepIdentity, null))

      bulkCopy.DestinationTableName = identifier;

      foreach (var column in data.Columns.Cast<DataColumn>().Select(c => c.ColumnName))

      bulkCopy.ColumnMappings.Add(column, column);


      await bulkCopy.WriteToServerAsync(data);



      public static void Seed(this SqlConnection connection, string schema, string table, DataTable data, bool truncate = true)

      SeedAsync(connection, schema, table, data, truncate).GetAwaiter().GetResult();




      where CreateIdentifier is an extension implemented as follows:



       public static string CreateIdentifier(this SqlConnection connection, params string names)

      if (connection == null) throw new ArgumentNullException(nameof(connection));

      using (var commandBuilder = DbProviderFactories.GetFactory(connection).CreateCommandBuilder())

      // ReSharper disable once PossibleNullReferenceException - commandBuilder is never null for SqlConnection.
      return names.Select(commandBuilder.QuoteIdentifier).Join(".");





      Example



      As an example of how I'm using it, I post one of the unit-tests I created for it and the schema of the test table.



      I import the test-data from a CSV, I turn into a DataTable and pass this to the seeder which takes just 4 lines of code (not counting the constants).



      [TestClass]
      public class SqlTableSeederTest

      private const string ConnectonString = "Data Source=(local);Initial Catalog=TestDb;Integrated Security=SSPI;";
      private const string Schema = "dbo";
      private const string Table = "SqlTableSeederTest";
      private static readonly ITypeConverter Converter =
      TypeConverter
      .Empty
      .Add<StringToInt32Converter>()
      .Add<StringToDateTimeConverter>();

      [TestMethod]
      public void Seed_WithoutId_Seeded()

      var csvReader = CsvReader.FromFile(@"testdataSqlTableSeederTest-without-id.csv");
      var sqlColumns = SqlHelper.Execute(ConnectonString, connection => connection.GetColumnFrameworkTypes(Schema, Table));
      var csv = csvReader.AsEnumerable().ToDataTable(sqlColumns, Converter);

      SqlHelper.Execute(ConnectonString, connection => connection.Seed(Schema, Table, csv));

      Assert.AreEqual(3, SqlHelper.Execute(ConnectonString, connection =>

      return connection.ExecuteQuery($"select count(*) from [Schema].[Table]", command => command.ExecuteScalar());
      ));




      CREATE TABLE [dbo].[SqlTableSeederTest](
      [_id] [int] IDENTITY(1,1) NOT NULL,
      [_nvarchar] [nvarchar](50) NULL,
      [_datetime] [datetime2](7) NULL,
      CONSTRAINT [PK_SqlTableSeederTest] PRIMARY KEY CLUSTERED
      (
      [_id] ASC
      )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
      ) ON [PRIMARY]



      So what do you think of this helper? Could it be even shorter and still SOLID and easy to use? Did I miss anything important? I'll be needing it in several unit-test projects so convenience is an important factor too.







      share|improve this question













      I have some unit-tests that must be executed against a real database. This requires populating the database with test-data.



      In order to simplify this process I created the SqlTableSeeder. It uses the SqlBulkCopy internally to insert the data from the specified DataTable. Optionally it truncates the target table first.



      The reason why I picked the SqlBulkcopy is because certain queries need to be tested for performance and I'll be creating data-tables with a couple of millions of rows.



      public static class SqlTableSeeder

      public static async Task SeedAsync(this SqlConnection connection, string schema, string table, DataTable data, bool truncate = true)

      if (Transaction.Current == null)

      throw new InvalidOperationException($"nameof(SeedAsync) can be executed only within a transaction scope.");


      var identifier = connection.CreateIdentifier(schema, table);

      if (truncate)

      // Using "truncate" because some databases/tables do not allow "delete".
      await connection.ExecuteQueryAsync($"truncate table identifier", command => command.ExecuteNonQueryAsync());


      using (var bulkCopy = new SqlBulkCopy(connection, SqlBulkCopyOptions.KeepIdentity, null))

      bulkCopy.DestinationTableName = identifier;

      foreach (var column in data.Columns.Cast<DataColumn>().Select(c => c.ColumnName))

      bulkCopy.ColumnMappings.Add(column, column);


      await bulkCopy.WriteToServerAsync(data);



      public static void Seed(this SqlConnection connection, string schema, string table, DataTable data, bool truncate = true)

      SeedAsync(connection, schema, table, data, truncate).GetAwaiter().GetResult();




      where CreateIdentifier is an extension implemented as follows:



       public static string CreateIdentifier(this SqlConnection connection, params string names)

      if (connection == null) throw new ArgumentNullException(nameof(connection));

      using (var commandBuilder = DbProviderFactories.GetFactory(connection).CreateCommandBuilder())

      // ReSharper disable once PossibleNullReferenceException - commandBuilder is never null for SqlConnection.
      return names.Select(commandBuilder.QuoteIdentifier).Join(".");





      Example



      As an example of how I'm using it, I post one of the unit-tests I created for it and the schema of the test table.



      I import the test-data from a CSV, I turn into a DataTable and pass this to the seeder which takes just 4 lines of code (not counting the constants).



      [TestClass]
      public class SqlTableSeederTest

      private const string ConnectonString = "Data Source=(local);Initial Catalog=TestDb;Integrated Security=SSPI;";
      private const string Schema = "dbo";
      private const string Table = "SqlTableSeederTest";
      private static readonly ITypeConverter Converter =
      TypeConverter
      .Empty
      .Add<StringToInt32Converter>()
      .Add<StringToDateTimeConverter>();

      [TestMethod]
      public void Seed_WithoutId_Seeded()

      var csvReader = CsvReader.FromFile(@"testdataSqlTableSeederTest-without-id.csv");
      var sqlColumns = SqlHelper.Execute(ConnectonString, connection => connection.GetColumnFrameworkTypes(Schema, Table));
      var csv = csvReader.AsEnumerable().ToDataTable(sqlColumns, Converter);

      SqlHelper.Execute(ConnectonString, connection => connection.Seed(Schema, Table, csv));

      Assert.AreEqual(3, SqlHelper.Execute(ConnectonString, connection =>

      return connection.ExecuteQuery($"select count(*) from [Schema].[Table]", command => command.ExecuteScalar());
      ));




      CREATE TABLE [dbo].[SqlTableSeederTest](
      [_id] [int] IDENTITY(1,1) NOT NULL,
      [_nvarchar] [nvarchar](50) NULL,
      [_datetime] [datetime2](7) NULL,
      CONSTRAINT [PK_SqlTableSeederTest] PRIMARY KEY CLUSTERED
      (
      [_id] ASC
      )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
      ) ON [PRIMARY]



      So what do you think of this helper? Could it be even shorter and still SOLID and easy to use? Did I miss anything important? I'll be needing it in several unit-test projects so convenience is an important factor too.









      share|improve this question












      share|improve this question




      share|improve this question








      edited Jan 16 at 9:12
























      asked Jan 12 at 19:46









      t3chb0t

      32.1k54195




      32.1k54195




















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          2
          down vote



          accepted











          So what do you think of this helper?




          It looks good, but I am missing the parameter validation for a public method




          • DataTable data -> blows very late if it is null.


          • string table -> blows at least if truncate == true

          Seeing




          // Using "truncate" because some databases/tables do not allow "delete".




          just makes me happy because too often you see only senseless comments. This is a valid and strong comment explaining why you did this.






          share|improve this answer





















            Your Answer




            StackExchange.ifUsing("editor", function ()
            return StackExchange.using("mathjaxEditing", function ()
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
            );
            );
            , "mathjax-editing");

            StackExchange.ifUsing("editor", function ()
            StackExchange.using("externalEditor", function ()
            StackExchange.using("snippets", function ()
            StackExchange.snippets.init();
            );
            );
            , "code-snippets");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "196"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            convertImagesToLinks: false,
            noModals: false,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );








             

            draft saved


            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f184983%2fpopulate-sql-tables-with-test-data%23new-answer', 'question_page');

            );

            Post as a guest






























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            2
            down vote



            accepted











            So what do you think of this helper?




            It looks good, but I am missing the parameter validation for a public method




            • DataTable data -> blows very late if it is null.


            • string table -> blows at least if truncate == true

            Seeing




            // Using "truncate" because some databases/tables do not allow "delete".




            just makes me happy because too often you see only senseless comments. This is a valid and strong comment explaining why you did this.






            share|improve this answer

























              up vote
              2
              down vote



              accepted











              So what do you think of this helper?




              It looks good, but I am missing the parameter validation for a public method




              • DataTable data -> blows very late if it is null.


              • string table -> blows at least if truncate == true

              Seeing




              // Using "truncate" because some databases/tables do not allow "delete".




              just makes me happy because too often you see only senseless comments. This is a valid and strong comment explaining why you did this.






              share|improve this answer























                up vote
                2
                down vote



                accepted







                up vote
                2
                down vote



                accepted







                So what do you think of this helper?




                It looks good, but I am missing the parameter validation for a public method




                • DataTable data -> blows very late if it is null.


                • string table -> blows at least if truncate == true

                Seeing




                // Using "truncate" because some databases/tables do not allow "delete".




                just makes me happy because too often you see only senseless comments. This is a valid and strong comment explaining why you did this.






                share|improve this answer














                So what do you think of this helper?




                It looks good, but I am missing the parameter validation for a public method




                • DataTable data -> blows very late if it is null.


                • string table -> blows at least if truncate == true

                Seeing




                // Using "truncate" because some databases/tables do not allow "delete".




                just makes me happy because too often you see only senseless comments. This is a valid and strong comment explaining why you did this.







                share|improve this answer













                share|improve this answer



                share|improve this answer











                answered Jan 16 at 10:36









                Heslacher

                43.9k359152




                43.9k359152






















                     

                    draft saved


                    draft discarded


























                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f184983%2fpopulate-sql-tables-with-test-data%23new-answer', 'question_page');

                    );

                    Post as a guest













































































                    Popular posts from this blog

                    Chat program with C++ and SFML

                    Function to Return a JSON Like Objects Using VBA Collections and Arrays

                    Will my employers contract hold up in court?