F#: Disabling SSL Certificate validation

Yesterday I wanted to download some content off a website with F#, however unfortunately the certificate of the website was expired.

    let result = 
        try
            let request = 
                "https://somewebsite/with/expired/ssl/certificate/data.json?paramx=1&paramy=2"
                |> WebRequest.Create

            let response = 
                request.GetResponse ()

            // parse data
            let parsed = "..." 

            Ok parsed
        with
        | ex ->      
            Error ex

If we execute this, then result would be of Error with the following exception:

SSL certificate validation exception
SSL certificate validation exception
ex.Message
"The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel."
ex.InnerException.Message
"The remote certificate is invalid according to the validation procedure."

So how do we fix this?

The solution is to set the following code at startup of the application (or at least before the first call):

ServicePointManager.ServerCertificateValidationCallback <- 
    new RemoteCertificateValidationCallback(fun _ _ _ _ -> true)

Notice that you should not do this, because this does not validate the certificate at ALL!
Also, this is for ALL calls, if you want to do it on a specific call you need to do make some changes.

First of all, it doesn’t work with WebRequest.Create, you need to use WebRequest.CreateHttp, or cast the WebRequest to HttpWebRequest, as the property we need, ServerCertificateValidationCallback is not available on WebRequest, only on HttpWebRequest. The resulting code looks like this:

            let request = 
                "https://somewebsite/with/expired/ssl/certificate/data.json?paramx=1&paramy=2"
                |> WebRequest.CreateHttp

            request.ServerCertificateValidationCallback <- new RemoteCertificateValidationCallback(fun _ _ _ _ -> true)

            let response = 
                request.GetResponse ()

Again, don’t do this in production!

If need be, do it on a single HttpWebRequest, like the last example, and write some code so that you ignore the expiration part, but leave in place the validation part.

Code on Github!

The impact of SqlDataReader.GetOrdinal on performance

I recently had a discussion about the impact of SqlDataReader.GetOrdinal on execution of a SqlClient.SqlCommand. I then decided to run some code to measure the difference, because I think that’s the only way to get a decent opinion. This is the code that I’ve used to run a certain query 1000 times:

private void InvokeQuery(Action mapObject)
{
    Stopwatch stopwatch = Stopwatch.StartNew();

    for (int i = 0; i < Iterations; i++)
    {
        using (var sqlCommand = new SqlCommand(this._query, this._sqlConnection))
        {
            using (SqlDataReader sqlDataReader = sqlCommand.ExecuteReader())
            {
                while (sqlDataReader.NextResult())
                {
                    mapObject(sqlDataReader);
                }
            }
        }
    }

    stopwatch.Stop();

    Debug.WriteLine("Running {0} queries took {1} milliseconds!", Iterations, stopwatch.ElapsedMilliseconds);
}

mapObject uses either directly the ordinal, or fetches the ordinal based on the column name. Also, I moved everything inside of the for loop to ensure nothing could be reused between queries. Here are the mapObject Actions, with GetOrdinal:

Action<SqlDataReader> = sqlDataReader =>
{
    int salesOrderID = sqlDataReader.GetOrdinal("SalesOrderID");
    int revisionNumber = sqlDataReader.GetOrdinal("RevisionNumber");
    int orderDate = sqlDataReader.GetOrdinal("OrderDate");
    int dueDate = sqlDataReader.GetOrdinal("DueDate");
    int shipDate = sqlDataReader.GetOrdinal("ShipDate");
    int status = sqlDataReader.GetOrdinal("Status");
    int onlineOrderFlag = sqlDataReader.GetOrdinal("OnlineOrderFlag");
    int salesOrderNumber = sqlDataReader.GetOrdinal("SalesOrderNumber");
    int purchaseOrderNumber = sqlDataReader.GetOrdinal("PurchaseOrderNumber");
    int accountNumber = sqlDataReader.GetOrdinal("AccountNumber");
    int customerID = sqlDataReader.GetOrdinal("CustomerID");
    int salesPersonID = sqlDataReader.GetOrdinal("SalesPersonID");
    int territoryID = sqlDataReader.GetOrdinal("TerritoryID");
    int billToAddressID = sqlDataReader.GetOrdinal("BillToAddressID");
    int shipToAddressID = sqlDataReader.GetOrdinal("ShipToAddressID");
    int shipMethodID = sqlDataReader.GetOrdinal("ShipMethodID");
    int creditCardID = sqlDataReader.GetOrdinal("CreditCardID");
    int creditCardApprovalCode = sqlDataReader.GetOrdinal("CreditCardApprovalCode");
    int currencyRateID = sqlDataReader.GetOrdinal("CurrencyRateID");
    int subTotal = sqlDataReader.GetOrdinal("SubTotal");
    int taxAmt = sqlDataReader.GetOrdinal("TaxAmt");
    int freight = sqlDataReader.GetOrdinal("Freight");
    int totalDue = sqlDataReader.GetOrdinal("TotalDue");
    int comment = sqlDataReader.GetOrdinal("Comment");
    int rowguid = sqlDataReader.GetOrdinal("rowguid");
    int modifiedDate = sqlDataReader.GetOrdinal("ModifiedDate");

    var temp = new SalesOrderHeader(
        salesOrderID: sqlDataReader.GetInt32(salesOrderID),
        revisionNumber: sqlDataReader.GetInt16(revisionNumber),
        orderDate: sqlDataReader.GetDateTime(orderDate),
        dueDate: sqlDataReader.GetDateTime(dueDate),
        shipDate: sqlDataReader.GetDateTime(shipDate),
        status: sqlDataReader.GetInt16(status),
        onlineOrderFlag: sqlDataReader.GetBoolean(onlineOrderFlag),
        salesOrderNumber: sqlDataReader.GetString(salesOrderNumber),
        purchaseOrderNumber: sqlDataReader.GetString(purchaseOrderNumber),
        accountNumber: sqlDataReader.GetString(accountNumber),
        customerID: sqlDataReader.GetInt32(customerID),
        salesPersonID: sqlDataReader.GetInt32(salesPersonID),
        territoryID: sqlDataReader.GetInt32(territoryID),
        billToAddressID: sqlDataReader.GetInt32(billToAddressID),
        shipToAddressID: sqlDataReader.GetInt32(shipToAddressID),
        shipMethodID: sqlDataReader.GetInt32(shipMethodID),
        creditCardID: sqlDataReader.GetInt32(creditCardID),
        creditCardApprovalCode: sqlDataReader.GetString(creditCardApprovalCode),
        currencyRateID: sqlDataReader.GetInt32(currencyRateID),
        subTotal: sqlDataReader.GetDecimal(subTotal),
        taxAmt: sqlDataReader.GetDecimal(taxAmt),
        freight: sqlDataReader.GetDecimal(freight),
        totalDue: sqlDataReader.GetDecimal(totalDue),
        comment: sqlDataReader.GetString(comment),
        rowguid: sqlDataReader.GetGuid(rowguid),
        modifiedDate: sqlDataReader.GetDateTime(modifiedDate)
        );
};

And without GetOrdinal:

Action<SqlDataReader> mapSalesOrderHeader = sqlDataReader =>
{
    new SalesOrderHeader(
        salesOrderID: sqlDataReader.GetInt32(0),
        revisionNumber: sqlDataReader.GetInt16(1),
        orderDate: sqlDataReader.GetDateTime(2),
        dueDate: sqlDataReader.GetDateTime(3),
        shipDate: sqlDataReader.GetDateTime(4),
        status: sqlDataReader.GetInt16(5),
        onlineOrderFlag: sqlDataReader.GetBoolean(6),
        salesOrderNumber: sqlDataReader.GetString(7),
        purchaseOrderNumber: sqlDataReader.GetString(8),
        accountNumber: sqlDataReader.GetString(9),
        customerID: sqlDataReader.GetInt32(10),
        salesPersonID: sqlDataReader.GetInt32(11),
        territoryID: sqlDataReader.GetInt32(12),
        billToAddressID: sqlDataReader.GetInt32(13),
        shipToAddressID: sqlDataReader.GetInt32(14),
        shipMethodID: sqlDataReader.GetInt32(15),
        creditCardID: sqlDataReader.GetInt32(16),
        creditCardApprovalCode: sqlDataReader.GetString(17),
        currencyRateID: sqlDataReader.GetInt32(18),
        subTotal: sqlDataReader.GetDecimal(19),
        taxAmt: sqlDataReader.GetDecimal(20),
        freight: sqlDataReader.GetDecimal(21),
        totalDue: sqlDataReader.GetDecimal(22),
        comment: sqlDataReader.GetString(23),
        rowguid: sqlDataReader.GetGuid(24),
        modifiedDate: sqlDataReader.GetDateTime(25));
};

With GetOrdinal the results are:

CreateWithGetOrdinal
CreateWithGetOrdinal

And without:

CreateWithoutGetOrdinal
CreateWithoutGetOrdinal

As you can see the performance difference is so low that I honestly don’t think you should sacrifice the readability and maintainability of your code vs a mere 82 milliseconds on a 1000 queries. Readability speaks for itself, you don’t talk with ints anymore, and for maintainability, consider the following: If your query column(s) change and you forget to update your code, GetOrdinal will throw an IndexOutOfRangeException, instead of maybe get an InvalidCastException or, if you’re really unlucky, another column and then broken code behavior… One sidenote to add:

GetOrdinal performs a case-sensitive lookup first. If it fails, a second, case-insensitive search occurs (a case-insensitive comparison is done using the database collation). Unexpected results can occur when comparisons are affected by culture-specific casing rules. For example, in Turkish, the following example yields the wrong results because the file system in Turkish does not use linguistic casing rules for the letter ‘i’ in “file”. The method throws an IndexOutOfRange exception if the zero-based column ordinal is not found.

GetOrdinal is kana-width insensitive.

So do watch out with cases, and your culture rules. Good luck, and let me know your opinion!

PS: the project itself is hosted on GitHub, you can find it here!

TransactionScope & SqlConnection not rolling back? Here’s why…

A while back we ran into an issue with one of our projects where we executed a erroneous query (missing DELETE statement), and then left the database in an inconsistent state.

Which is weird, considering the fact that we use a TransactionScope.

After some digging around I found the behavior I wanted, and how to write it in correct C#.

Allow me to elaborate.

Consider a database with 3 tables:

T2 --> T1 <-- T3

Where both T2 and T3 link to an entity in T1, thus we cannot delete lines from T1 that are still referenced in T2 or T3.

I jumped to C# and started playing with some code, and discovered the following (mind you, each piece of code is actually supposed to throw an exception and abort):

This doesn’t use a TransactionScope, thus leaving the database in an inconsistent state:

using (var sqlConnection = new SqlConnection(ConnectionString))
{
    sqlConnection.Open();

    using (SqlCommand sqlCommand = sqlConnection.CreateCommand())
    {
        sqlCommand.CommandText = "USE [TransactionScopeTests]; DELETE FROM T3; DELETE FROM T1;"; 
        // DELETE FROM T1 will cause violation of integrity, because rows from T2 are still using rows from T1.

        sqlCommand.ExecuteNonQuery();
    } 
}

Now I wanted to wrap this in a TransactionScope, so I tried this:

using (var sqlConnection = new SqlConnection(ConnectionString))
{
    sqlConnection.Open();

    using (var transactionScope = new TransactionScope())
    {
        using (SqlCommand sqlCommand = sqlConnection.CreateCommand())
        {
            sqlCommand.CommandText = "USE [TransactionScopeTests]; DELETE FROM T3; DELETE FROM T1;"; 

            sqlCommand.ExecuteNonQuery();
        }

        transactionScope.Complete();
    }
}

Well guess what, this essentially fixes nothing. The database, upon completion of the ExecuteNonQuery() is left in the same inconsistent state. T3 was empty, which shouldn’t happen since the delete from T1 failed.

So what is the correct behavior?

Well, it doesn’t matter whether you create the TransactionScope or the SqlConnection first, as long as you Open() the SqlConnection inside of the TransactionScope:

using (var transactionScope = new TransactionScope())
{
    using (var sqlConnection = new SqlConnection(ConnectionString))
    {
        sqlConnection.Open();

        using (SqlCommand sqlCommand = sqlConnection.CreateCommand())
        {
            sqlCommand.CommandText = "USE [TransactionScopeTests]; DELETE FROM T3; DELETE FROM T1;"; 

            sqlCommand.ExecuteNonQuery();
        }

        transactionScope.Complete();
    }
}                                                                                                                           

Or the inverse (swapping the declaration of the TransactionScope and SqlConnection):

using (var sqlConnection = new SqlConnection(ConnectionString))
{
    using (var transactionScope = new TransactionScope())
    {
        sqlConnection.Open();

        using (SqlCommand sqlCommand = sqlConnection.CreateCommand())
        {
            sqlCommand.CommandText = "USE [TransactionScopeTests]; DELETE FROM T3; DELETE FROM T1;"; 

            sqlCommand.ExecuteNonQuery();
        }

        transactionScope.Complete();
    }
}

I wrote the test cases on a project on GitHub which you can download, compile and run as Tests for yourself!

https://github.com/CSharpFan/transaction-scope

Have a good one,

-Kristof

About a dictionary, removing and adding items, and their order.

I had a weird problem today using a Dictionary. The process involved removing and adding data, and then printing the data. I assumed that it was ordered. I was wrong! Let me show you:

var dictionary = new Dictionary<int, string>();

dictionary.Add(5, "The");
dictionary.Add(7, "quick");
dictionary.Add(31, "brown");
dictionary.Add(145, "fox");

dictionary.Remove(7); // remove the "quick" entry

After a while I added another line to the dictionary:

dictionary.Add(423, "jumps");

While printing this data I discovered an oddity.

dictionary
    .ToList()
    .ForEach(e => Console.WriteLine("{0} => {1}", e.Key, e.Value));

What do you expect the output of this to be?

5 => The
31 => brown
145 => fox
423 => jumps

However the actual result was this:

5 => The
423 => jumps
31 => brown
145 => fox

The documentation tells us the following:

For purposes of enumeration, each item in the dictionary is treated as a KeyValuePair<TKey, TValue> structure representing a value and its key. The order in which the items are returned is undefined.

Interested in the actual behavior I looked at the source code of Dictionary here.

If you look closely, first at Remove and then to Add (and subsequently Insert) you can see that when you remove an item it holds a reference (in freelist) to the free ‘entry’.

What’s more weird is the behavior when you delete 2 entries, and then add 2 others:

var dictionary = new Dictionary<int, string>();

dictionary.Add(5, "The");
dictionary.Add(7, "quick");
dictionary.Add(31, "brown");
dictionary.Add(145, "fox");

dictionary.Remove(7); // remove the "quick" entry
dictionary.Remove(31); // also remove the "brown" entry

dictionary.Add(423, "jumps");
dictionary.Add(534, "high");

dictionary
    .ToList()
    .ForEach(e => Console.WriteLine("{0} => {1}", e.Key, e.Value));

Which yields:

5 => The
534 => high
423 => jumps
145 => fox

But for that you’ll need to look at line 340 and further!

So what have we learned? It’s not ordered until MSDN tells you!

Have a good one!

Default values and overloads are not the same!

Consider the following class of the Awesome(r) library, using default parameters.

public class Foo
{
    public void DoCall(int timeout = 10)
    {
        /* awesome implementation goes here */
    }
}

You get the dll and that class & function in your code, like this:

Foo foo = new Foo();

foo.DoCall();

Can’t get much easier than this right?

Then the Awesome(r) library gets updated:

public class Foo
{
    public void DoCall(int timeout = 20)
    {
        /* awesome implementation goes here */
    }
}

Notice that the default value has changed. You assume that when you just overwrite the dll in production, you will adopt the new behavior.

Nop. You need to recompile. Let me show you: the problem with default values is that the developer of Awesome(r) library is no longer in control of it.

Let’s take a look at an excerpt of the IL where we create a new Foo and call DoCall without specifying timeout:

  IL_0000:  newobj     instance void AwesomeLibrary.Foo::.ctor()
  IL_0005:  stloc.0
  IL_0006:  ldloc.0
  IL_0007:  ldc.i4.s   10
  IL_0009:  callvirt   instance void AwesomeLibrary.Foo::DoCall(int32)
  IL_000e:  ret

This is a release build.

Notice how on line 4 value 10 gets pushed on the the stack, and the next line calls the DoCall.

This is a big danger in public APIs, and this is why the developer of Awesome(r) library should have used an overload instead of a default parameter:

public class Foo
{
    public void DoCall()
    { 
        this.DoCall(20); 
    }

    public void DoCall(int timeout)
    {
        /* awesome implementation goes here */
    }
}

This ensures that when a new version of Awesome(r) library is released AND that if that release is backwards API compatible, it can just be dropped in, without you having to recompile your whole codebase (but you should still test it 😛 )

The behavior of FlagsAttribute is probably not what you suspect

Let’s create another enum:

enum Foo
{
    A,
    B,
    C,
    D
}

You add the FlagsAttribute:

[FlagsAttribute]
enum Foo
{
    A,
    B,
    C,
    D
}

Meaning you want to use the Enum as a Flag, so you can combine them. For example:

Foo foo = Foo.B | Foo.C | Foo.D;

Later, you pass this value on, and you want to test for the presence of Foo.A:

// foo is the same foo as previous 
var hasA = (foo & Foo.A) == Foo.A;

Console.WriteLine("hasA: {0}", hasA);

You think that hasA is false. Is it? It’s not:

Does foo include Foo.A?
Does foo include Foo.A?

How come? Applying the FlagsAttribute doesn’t DO anything with the generated constants for your enum members.

As per the documentation you still need to do it yourself:

Define enumeration constants in powers of two, that is, 1, 2, 4, 8, and so on. This means the individual flags in combined enumeration constants do not overlap.

So we update our enum:

[FlagsAttribute]
enum Foo
{
    A = 1,
    B = 2,
    C = 4,
    D = 8
}

and then we test our code again:

Foo foo = Foo.B | Foo.C | Foo.D;
var hasA = (foo & Foo.A) == Foo.A;

Console.WriteLine("hasA: {0}", hasA);

And the result is:

Does foo include Foo.A? It does!
Does foo include Foo.A? It does!

Success!

Hope you have a good one,

-Kristof

PS: please not that I should have added a None enum member, as per the documentation:

Use None as the name of the flag enumerated constant whose value is zero. You cannot use the None enumerated constant in a bitwise AND operation to test for a flag because the result is always zero.

Foreach now captures variables! (Access to modified closure)

Foreach has changed in C# 5.0!

Consider the following piece of code in C# < 5.0:

public class Test
{
    public static void Main()
    {
        var words = new[] { "foo", "bar", "baz", "beer" };
        var actions = new List<Action>();
        foreach (string word in words)
        {
            actions.Add(() => Console.WriteLine(word));
        }

        actions.ForEach(e => e());
    }
}

What will this print?

Some of you will see the warning that ReSharper will print on line 9.

Access to foreach variable in closure. May have different behaviour when compiled with different versions of compiler

Notice the second sentence, and remember this warning, we’ll get back to it!

Now go ahead, try and run this in Visual Studio 2010. This will be your result:

beer beer beer beer

While I do love beer, this is not what I expect.

So how do we fix it? Well, either let ReSharper fix it (Alt+Enter -> Enter), or manual, capture the current word in a different variable:

public class Test
{
    public static void Main()
    {
        var words = new[] { "foo", "bar", "baz", "beer" };
        var actions = new List<Action>();
        foreach (string word in words)
        {
            string temp = word;
            actions.Add(() => Console.WriteLine(temp));
        }

        actions.ForEach(e => e());
    }
}

Problem solved. The code above has identical results in Visual Studio 2012.

However…

Using the first piece of code (without our temp variable) in Visual Studio 2012 the result is as follows:

foo bar baz beer

Wait what?

The compiler has changed (note that even for .NET 3.5, 4, and 4.5 in Visual Studio 2012 the 4.5 compiler is used!).

Meaning that our variable word is now declared inside of the foreach loop, and not outside.

This change can be found in the C# 5.0 spec, page 247-248, found on your machine when you’ve installed VS2012 (not Express) in: C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC#\Specifications\1033

If v was declared outside of the while loop, it would be shared among all iterations, and its value after the for loop would be the final value, 13, which is what the invocation of f would print. Instead, because each iteration has its own variable v, the one captured by f in the first iteration will continue to hold the value 7, which is what will be printed. (Note: earlier versions of C# declared v outside of the while loop.)

Note 1: read the file to get the meaning of ‘v’ and those values (like 13 and 7).
Note 2: I’ve tweeted to some guys to get the spec online.

While this is not necessarily a problem for projects coming from 2010 and upgrading to 2012, it can be an issue when you are doing round-tripping, for example in mixed teams. Developers using 2012 need to use the old behavior.

Worse, your build system is not at 2012 yet, so your result are different!

Watch out for this!

Have a good one,

-Kristof

When using an enum in PowerShell, use the member’s name, not the member’s value

Consider the following enum in C#:

enum State
{
    Started,
    Stopped,
    Unknown
}

Note that I have not added an explicit value for the enum members. They will be generated by the compiler. As stated in the C# spec:

… its associated value is set implicitly, as follows:

  • If the enum member is the first enum member declared in the enum type, its associated value is zero.
  • Otherwise, the associated value of the enum member is obtained by increasing the associated value of the textually preceding enum member by one. This increased value must be within the range of values that can be represented by the underlying type, otherwise a compile-time error occurs.

Found at http://www.microsoft.com/en-us/download/details.aspx?id=7029, page 400-401 (I can’t find the version for 4.5 though…).

Now what are the consequences of this? Consider the following piece of PowerShell:

$result = $serviceController.GetServiceStatus()
if($result -eq 1)
{
    MyLib.StartService()
}

This will work, because PowerShell implicitly converts the int to the actual enum member.

However since we are assuming the value can go wrong. In the next version you add extra values, say for example to represent a starting/stopping service:

enum State
{
    Starting,
    Started,
    Stopping,
    Stopped,
    Unknown
}

Since now all the values are shifted when you run your PowerShell again you start the service when it’s already started 😉 .

Solution?

First of all (as a consumer), use the enum’s member name instead of its value:

$result = $serviceController.GetServiceStatus()
if($result -eq [MyLib.State]::Stopped)
{
    MyLib.StartService()
}

This will ensure that you get the value for Started, not for anything else.

As a developer of a library you should ensure that you never mess up the order of an enum, by adding new values as last, or (prefered) set the value yourself:

enum State
{
    Started = 0,
    Stopped = 1,
    Unknown = 2,
}

Becomes:

enum State
{
    Starting = 3,
    Started = 0,
    Stopping = 4,
    Stopped = 1,
    Unknown = 2,
}

And now you can also perfectly reorder them so the numbers are sequential:

enum State
{
    Started = 0,
    Stopped = 1,
    Unknown = 2,
    Starting = 3,
    Stopping = 4,
}

Hope you have a good one,

-Kristof

ASP.NET MVC Mobile: where is my style?

I was playing with ASP.NET MVC Mobile to start something quickly for my phone.

I then updated the NuGet packages, and my style was GONE.

Seems that jQuery 1.9.* is incompatible with jQuery.Mobile 1.2.

According to the website 1.3 is out, so the package should soon be there, so until then you can revert from jQuery 1.9.* to 1.8.3 with the following code in your Package Manager Console:

uninstall-package jQuery -Force # ignore that some have dependencies on jQuery
install-package jQuery -Version 1.8.3

jQuery downgrade

Have a good one,

-Kristof