Jacobian

The Jacobian matrix is used when performing a change of coordinates, often to simplify an integration. This is because the Jacobian tells us how much the area around an N-dimensional space is skewed when changing bases. Intuitively we would think that for some change of bases function 
$$T(u,v)\ =\ (x(u,v), y(u,v))$$ 
that the integral 
$$\int\int_D\ f(x,y) dx\ dy = \int\int_{D^*} f(x(u,v), y(u,v)) du\ dv$$
However, it is easy to find cases where this does not hold. For example, take
$$T(u, v) = (-u^2 + 4u, v)$$ 
Over the unit square 

The area of the unit square is obviously 1, but after applying the change of basis function T, we arrive at the area below

Please note that in this graph the x intercept is 3, making the area captured under the new basis 3. You can prove this to yourself by seeing that
$$T(1,1) = (-1 + 4, 1) = (3, 1)$$
And similarly for the other coordinates in the unit square. Showing that 
$$\int\int_D f(x,y) dx dy \neq \int\int_{D^*} f(x(u,v), y(u,v)) du dv$$
What this really shows is that our intuition was wrong, we need to multiply the result of our coordinate change by some value indicating the sensitivity of the points to the change. This is known as the Jacobian. More specifically, we multiply by the determinant of the Jacobian matrix. Please take a look at the bottom of this post to learn about the man himself, Carl Gustav Jacob Jacobi.
The Jacobian is defined as 
$$\frac{\delta\ x_i}{\delta\ y_i}\ \forall\ i$$
That is, the derivative over all your change of basis functions with respect to each new basis. The result of this is a matrix, but we are looking for a single value to multiply our integral by so we take the determinant. The determinant of a matrix of values 
$$a_{(0,0)}, a_{(1,0)}, a_{(0,1)}, a_{(1,1)}$$ 
Where subscripts indicate the (row, col) position in the matrix is
$$a_{(0,0)} *a_{(1,1)} - a_{(1,0)} * a_{(0,1)}$$
Above is the most well known application to the Jacobian matrix. However, the Jacobian is really just a set of derivatives that indicate the sensitivity of functions to changes in their input values. In fact, the Jacobian matrix comes up in Machine Learning when using backpropagation to calculate the loss of a particular component in a modular neural network.

In the modular network above, note that minimizing the error with respect to w requires finding the intermediary derivative of each y_i with respect to the inputs z_i. Since the Jacobian of all y_i with respect to all x_i tells us how sensitive y is to changes in inputs x to our network, we can use it to say that any errors are approximately the sum of the Jacobian over all y_i with respect to x_i evaluated at x_i. We can think of this as a measure of the sensitivity of the error with respect to the inputs. For a more detailed explanation please see Pattern Recognition And Machine Learning by Christopher M. Bishop, page 247, section 5.3.4., from which I took this example and the above image. 
The Jacobian matrix itself is interesting in the information it provides and its applications, but before looking at it in more detail I knew nothing about its creator, Carl Gustav Jacob Jacobi. A quick look at the wikipedia page shows he was an impressive Mathematician and part of a family of distinguished people. His older brother Moritz Von Jacobi contributed to physics and engineering and Carl was home schooled by his uncle until the age of 12. At 12 he was moved to a private school, and in half a year was put into senior level courses. The only reason he didn't go to college immediately is that the universities were not accepting applicants under 16 years old. Before going to college, by now bored by his education, he tried to solve the quintic equation by radicals (whatever that is). At 21 years old he was lecturing on the theory of curved surfaces at the University of Berlin. On the other hand, when I was 21 I was either learning or forgetting about the Jacobian matrix in my multivariate calculus course.

graph 1 (unit square) - source
graph 2 - wolfram alpha
example - Vector Calculus 5th edition by Marsden and Tromba 
modular network image - Pattern Recognition and Machine Learning - Bishop pg. 247
Jacobian application - Pattern Recognition and Machine Learning - Bishop pg. 247

Lucky 13

The following was taken from Nautilus article "How Designers Engineer Luck Into Video Games" by Simon Parkin which can be found here.

"Olaf Haraldsson, an 11th-century Norwegian king, once wagered a kingdom in a faith-testing game of dice. Olaf was locked in a territorial dispute with the king of Sweden over the island of Hising; eventually the two agreed to settle the matter with a dice throw. The Swedish king rolled two sixes, the highest possible score, and said there was no point in continuing the game. Olaf insisted on taking his throw; a recent convert to Christianity, he was certain that God would steer the dice in his favour. His faith was vindicated with double sixes. The men continued to take turns throwing their dice, twelve after twelve. The matter was finally settled when, during Olaf’s final throw, one of the dice split in two, to show both a six and a one, winning him the kingdom on an unprecedentedly lucky 13."

Python Swap Function

In the coming semester it is likely that I will T.A. a course in Python. Lately I've been researching the language and using it for my pet projects that never get finished, interviews, etc...

One thing I found interesting is writing a swap in function. For some sorting algorithms we often have to swap the values at two indices in an array, so this does come up fairly often. What I found especially interesting was that you could do this in one line using Python. Where in most programming languages you have to write something like...

In Python, this can be written as... 

This is because Python evaluates assignments right to left. When the right hand side gets evaluated, Python creates a tuple (val_i, val_j) then assigns the variables in the left their corresponding tuple values. 

Most experienced Python programmers already knew this, but I found it pretty cool. All credit for the knowledge goes to this stack overflow post.

Researching Encryption in Windows and C# .NET

I was recently tasked with researching different methods of encryption for a license management module within my company and thought it might be useful to write down some of my findings. 

Cryptography has been around for a long time. Some of the more famous forms of old cryptography are in the Roman army, which used the "Caesar Cypher" to shift all letters to the left by three. "A" would become "X", "B" would become "Y", etc.

These days there are many more forms of data, and different ways to intercept it. Specifically we have data in-transit, and data at-rest.

For data-in-transit we use a public key private key method. The actual encryption of this method involves factoring extremely large numbers into their primes. If you're interested, this is a great video to watch.

With data-at-rest we use more complicated algorithms. In the Microsoft .NET library, these encryption methods are very well documented and held within the System.Security.Cryptography namespace. This includes methods for both in-transit and at-rest cryptography. 

Within the namespace there are two main encryption methods, Rijndael and Aes. The Microsoft Recommended method of encrypting data-at-rest is the AES method because Rijndael will not work when the FIPS-compliant security setting is enabled on a windows operating system. 

In conclusion, if you are encrypting a file, database, or some data on disk in a windows environment you should be using either the AesManaged, or AesCryptoServiceProvider classes from the System.Security.Cryptography namespaces. 

Fermat's Near Miss

A lot of people have heard about Fermat's last theorem. Originally proposed in the 1600s, the theorem was finally solved in the 20th century by a mathematician named Andrew WIles. 

Fermat's theorem says that there are no solutions (x,y,z) to the equation x^n + y^n = z^n for n > 2. If you've studied math or physics at a university or in some cases high school level you have likely heard of it. Today I was reminded of an example from the Simpsons which is purported to show a solution where x^n + y^n = z^n. Specifically...

There we see 3987^12 + 4365^12 = 4472^12. And if you were to plug this into a handheld calculator, it would be correct!

So what is going on? How could Homer Simpson have a counter-example to Andrew Wiles proof? In fact, these numbers are so large that most calculators round the error off and they appear to be the same! X, y, z combinations like this that "solve" Fermat's last theorem are known as near misses. I was reminded of this interesting example by this video, but have also read it in this book.

Also of note, in the holy doughnut to filled in doughnut progression along the bottom of the chalkboard, Homer is "solving" a question in topology! Unfortunately, in topology, a doughnut and a sphere do not equate... 

C# .Net Utility in WPF

I very recently had the opportunity to develop a simple tool in wpf and C# .Net to execute some procedural actions necessary to quickly make copies of our application. Surely there is an "easier" way to do this with windows system scripting, but for the sake of education and usability (we don't want our clients running windows scripts) we chose to write an executable. 

Several things surprised me during the development. First, wpf has become much easier. The most useful resource I found was WPF MVVM In Depth by Brian Noyes. In his tutorial he builds WPF up from code-behinds and simple applications that take inputs and generate outputs. The most essential parts for me were the building blocks of implementing INotifyPropertyChanged and ICommand. This was good enough for my current application, but more complex applications meant to display and manipulate data will probably go into using ObservableCollections and defining sql server data sources. 

This project, along with other personal things kept me very busy the last few weeks, though I hope to return to posting more frequently. I've now changed the blog name from "Daily Blog" to simply "Blog".

C# .NET and WPF

I recently had a terribly boring project in which I had to go through 27 pdf documents and compare copies word for word. That means reading a total of 54 documents and comparing for exact word for word copies. And no, they could not be done with pdf comparison tools, it was required that human eyes search through each document at some point.

In order not to burn out, I started a project in C#.NET and WPF on the side. What I found was that C# is one of the easiest languages there is. I was able to create classes and structures using the .NET toolset to model my data in about two hours. Then I tried to connect it to a WPF interface.

WPF is ridiculously difficult to "jump into". I found it really hard to get resources online, then half the resources said you need to use the prism library to do WPF correctly and encountered similar difficulty in finding prism examples. All the examples I found used code-behinds to bind data, which, from the Microsoft examples, are an antipattern. I really felt like there were more contradictory and dated examples of WPF and C#.NET code than there are of javascript (though I'm sure that isn't true).