Category: Miscellaneous

Since the summer of last year I’ve been an online coding instructor for children and young adults for theCoderSchool. I’ve taught students 1-on-1, 2-on-1, and sometimes groups of five or more. In this blog post, I will be share a few takeaways I’ve gained from my teaching experiences over the past year. 
 
Be aware of the curse of knowledge
 
One of the first takeaways I gained when I started teaching was how experience was how my software engineering experience  impacts the quality of my instruction. While it has provided me a depth of topics and interesting insights to infuse into my lessons, at times it has also caused me to assume that people know things the same way that I do. I’ve been coding professionally for well over a decade. Coding has practically become a second language to me. I can think and reason in code so quickly that it’s unconscious. Because of this, I at times forget that my students are learning code for the first time and don’t grasp the concepts as deeply as I do.  
 
There were many moments when I first started out that I would quickly zip through an explanation that in my mind was really simple, only to look up and meet blank look on my student’s face. 
 

Here’s some things I do to avoid this:

1. Put myself in my student’s shoes – I try to remember what it was like when I was learning how to code. I think about what questions I might have about certain concepts and how I might understand them. I try to get inside their heads and think about how I can adjust my instruction.
2. I slow down – I take my time to explain coding concepts and make sure my students understand them before moving on. I avoid moving on to very abstract, complex concepts such as classes and objects too soon; this is especially the case for students under the age of 10, most of whom tend to lack the cognition to fully grasp the concept. 

Patience is a Must

Becoming aware of the curse of knowledge has also made me realize the importance of patience in teaching. The greatest gift a teacher can give a student is lots of support and approval of where they are in their journey. It takes a lot of time and persistence to learn how to code. When a student feels like they can learn at their own pace without any expectation of being at a particular skill level they are more likely to continue engaging with the material. I’ve seen students who lost interest in learning to code because either the instructor moved too quickly through the material or the instructor got frustrated with the student’s lack of progress. 

As an instructor it can be frustrating to see that after spending an exorbitant amount of time teaching and answering questions about a coding concept, a student still fail to grasp it. But when I pause to reflect on my own learning journey, I remember moments when I struggled to understand many computer science concepts, including the very ones my students struggle with. What I immediately realize is that learning is a process. 

We often view the act of learning to be something akin to downloading a data file from the internet. I think a better analogy for learning is that of building an extension to a house. I’m not a builder so please excuse the bad analogies in advance. Learning is about finding the right building blocks that connect with the existing structure (i.e. a student’s current knowledge and understanding). Sometimes it might be necessary to knock down existing walls (i.e. false ideas and misconceptions) to make room for the new structures you want to build. And sometimes you might need to add new structural support to the house (i.e. giving students lots of practice with the concepts) to keep everything in place. Depending on the house and on the expansion, this can be easy or it can be a challenge. But that challenge is not a bad thing. It’s an essential part of the process. Because without facing that challenge, nothing will ever get built.

So now I have a lot more patience and understanding for anyone learning how to code. I don’t see their struggle as a problem. I see it as a sign that they are on the path to building something great!

Growing creative problem solvers

Probably the biggest realization I’ve had since I started teaching is that the value that I provide my students is more than merely teaching them how to code. I’m teaching them how to solve problems.

Software developers are probably some of the most creative problem solvers out there. Everyday they are tasked to solve technical problems that are complex enough to make the head of the average spin a thousand times over. The key is to their success is a structured approach that they use when tacking each problem. This same approach is highly applicable to areas outside of coding and can provide kids benefits for future career opportunities as well as help them find solutions to problems they encounter on a regular basis.

Something I’ve been taking on since making this realization is not just giving my students the answers to problems when they get stuck, but also showing them strategies that can be used to solve any problem and coaching them to use it as often as possible.  

Are you a coding instructor? What are some of your takeaways from your experience? Feel free to share them in the comments below.

This article is part of the multi part series on binary data. If you’re knowledge of positive and negative integers and binary numbers is a bit flakey, I’d recommend that you read the part 1 and part 2 before you continue.

In this article we will discuss how fractions are represented in binary and some limitations you must keep in mind when developing software that require very precise calculations.

To start things off consider how we represent floating points in the decimal system.

Thousands	Hundreds	Tens	Ones	.	Tenth	Hundredth	Thousandth	Ten-Thousanth	Hundred-Thousanth
10^3	10^2	10^1	10^0	.	10^-1	10^-2	10^-3	10^-4	10^-5
1	9	8	1	.	0	8	4	2	7

Each position to the right of the decimal point represents some fraction of 10. Each position to the right is 10 times smaller than the previous position.

This is very much how floats in binary is represented.

Eight	Four	Two	One	.	One-Half	One-Fourth	One-Eighth	One- Sixteenth	One-Thirty Second
2^3	2^2	2^1	2^0	.	2^-1	2^-2	2^-3	2^-4	2^-5
1	9	8	1	.	0	8	4	2	7

Converting from binary floats to decimal fractions is the same as converting integers. Multiply each bit by it’s positional value and add them up. Observe:

 101.011

• The first digit on the left is 1. It’s is in the 2^2 position, making 1 * 2^2 = 4 
• The next digit is 0. It’s in the 2^1 position, meaning 0 * 2^1 = 0 
• The next digit is 1. It’s in the 2^0 position, meaning 1 * 2^0 = 1 
• The next digit is 0. It’s in the 2^-1 position, meaning 0 * 2^-1 = 0
• The next digit is 1. It’s in the 2^-2 position, meaning 1 * 2^-2 = 0.25
• The next digit it 1. It’s in the 2^-3 position, meaning 1 * 2^-3 = 0.125

Adding these numbers together we get 4 + 0 + 1 + 0 + 0.25 + 0.125 = 5.375

Now suppose we have a 12.25 and we want to convert this to binary. Here’s how it’s done:

First convert the integer part to binary:

12 therefore becomes 1100

Next we deal with the fractional part of the number. For this we repeatedly multiply the fractional part of number by 2 until we get 0, each time adding a binary 1 digit when the product is greater than or equal to 1 and a binary 0 when it is less than 1:

Multiplication by 2	Product	Digit
0.25 * 2	0.5	0
0.5 * 2	1.0	1

The fractional part of the number is .01. Combine this with the integer part and we get 1100.01

Great, now that you we’ve gone over how to convert floating point from decimal to binary, let’s talk about how these are stored in a computer.

The IEEE Floating Point Standard

Most computer systems today follow the IEEE 754 standard for representing real numbers. In the technical standard, floating point numbers are represented using 3 components. These are:

1. The sign bit
2. The exponent
3. The mantissa

I’ll break each of these down and explain what they mean.

Do you remember learning about scientific notation in primary school? In case you haven’t heard of it before or need a refresher, scientific notation is a method for writing very large or small numbers. The numbers 890,000,000 and 0.00000051 or example is written in scientific notation as 8.9 x 10⁸ and 5.1 x 10^-7.  A number in scientific notation is expressed as a number between 1 and 10 multiplied by a power of 10.

Binary real numbers can also be expressed in scientific notation. A binary number in scientific notation is expressed as a number 0 or 1 multiplied by a power of two. Let’s look at the decimal number 85.125 and express that in binary scientific notation:

First we convert 85.125 to binary. Following the steps outlined above, we will get 1010101.001. Now to express this in scientific notation we will move the decimal point six bits to the left and then multiply by 2⁶. The result will be 1.010101001 x 2^6.

If this number was stored as a single precision 32 bit float, here’s what it will look like:

Sign bit	Exponent	Mantissa
0	10000101	01010100100000000000000

Let’s unpack things a bit further…

The sign bit represents the sign of the number. If the float is positive than this bit will be 0 otherwise it is 1.

The exponent is 8 bits in length and represents the exponent part of the expression. You may have noticed that notice that exponent is not 00000110, which is 6 in binary. This is because 127 was added to the number. The 127 serves as a bias term and allows the computer to represent both positive and negative exponents.

The mantissa is 23 bits in length and represents the part of the number in scientific notation that is to the right of the decimal point.

This number can also be represented in a double precision 64 bit floating point. Here’s what it would look like:

Sign bit	Exponent	Mantissa
0	10000000101	0101010010000000000000000000000000000000000000000000

As you can see the double precision floating point type uses 1 bit to represent the sign of the number, 11 bits for the exponent with a bias of 1023, and 52 bits for the mantissa. Just as with the different types of integer types covered in the last article, single precision and double precision floating point can store different ranges of real numbers.

The range a single precision float can represent approximately +/- 10^38.25 while the range of a double precision float is +/- 10^308.25

With that large of a range you would think that a computer can represent any type of number you provide it accurately. But you would be mistaken. Allow me to explain.

The Limits of Floating Point Numbers

I’ll begin my explanation by first demonstrating the conversion of the real number 0.4 into binary.

Multiplication by 2	Product	Digit
0.4 * 2	0.8	0
0.8 * 2	1.6	1
0.6 * 2	1.2	1
0.2 * 2	0.4	0
0.4 * 2	0.8	0
0.8 * 2	1.6	1
0.6 * 2	1.2	1
…	…	…

As you can see the number 0.4 in binary repeats infinitely. 0.011001100110011001100110011…

The number 0.3 and 0.1 are other examples of numbers where you will see this behavior. I’ll leave it up to you to convert them to binary and see for yourself.

There are many decimal floating points like these that cannot be represented exactly in binary. As a consequence, what you often get when you type floating point numbers into a computer is not the exact number, but an approximation of that number.

That is why when you run certain calculations on a computer like 0.1 + 0.2, you get 0.30000000000000004 instead of exactly 0.3.

In many cases, the approximation isn’t an issue. We lob off the trailing digits we don’t need and make it 0.3. But where it really becomes an issue is when we are dealing with applications that must make very precise calculations. Financial applications one case where precise calculations are crucial.

So what can be done for cases like these? In these situations, developers turn to Decimal data types, data types that are designed to represent decimal numbers precisely in a computer system. In the case of finance applications, developers may also choose to represent money in cents rather than in dollars, allowing them to use an integer data type instead of a floating point type to store the money.

That’s all, folks!

I’ve hoped you enjoyed the binary data series. Feel free to leave a comment if you have any questions.

Let’s continue where we left off. In the last article I covered what are binary numbers and how to convert from binary to decimal. If you had a chance to read it, you’re now probably wondering how positive and negative numbers are represented in a computer. This article will cover this.

Adding binary numbers

Before I cover positive and negative numbers, I must first touch on a few things that will make some of the concepts I will bring up later easier to follow. The first thing I will touch on is how addition works in binary.

Binary addition follows the same rules as addition in the decimal system.

In the decimal system, when the result of an addition equals 10, we carry 1 over to the next digit on the left.

For example:

12 + 9 = 21

In binary addition, we carry 1 over when the sum of the values equal 2:

0 + 0 = 0

0 + 1 = 1

1 + 0 = 1

1 + 1 = 10, carry over 1, i.e. 10

Example:

      1  0  ¹0  1  ¹0  1

  +      1    0  1   0  1

  =  1  1    1  0   1  0

The value 2 in the binary system is equivalent to 10 in the decimal system.

Bits, bits, and more bits

The next topic I will touch on is how numbers are  stored and organized in a computer system. A single binary digit is called a bit. Bits are the most basic units of information in a computer system. A bit by itself can only provide two values: 0 and 1. But as you seen in the last article, a computer can represent larger numeric values with more bits.

Programming languages typically provide a variety of numeric data types, each of which allow for a different range of numbers to be stored. Here are the types you will come across:

Data Type	# Bits	Number Range
Byte	8	0 – 256 (2^8)
Short/Word	16 (2 Bytes)	0 – 65536 (2^16)
Double Word	32 (4 Bytes)	0 – 4,294,967,296 (2^32)
Long	64 (8 Bytes)	0 – 18,446,744,073,709,551,616

Positive and Negative Integers

To represent negative numbers, one bit in the number is used to indicate whether or not the number is positive or negative. If this bit is 0 than the number is positive. If the bit is 1 than the number is negative. This bit is called the sign bit and is the leftmost bit in the binary number

Let’s say that we using 8 bits to represent positive and negative numbers. This is what the number 105 will look like:

Sign (High order bit)							Low Order Bit
0	1	1	0	1	0	0	1

And this what -105 looks like:

Sign (High order bit)							Low Order Bit
1	0	0	1	0	1	1	1

If you compare the two binary numbers you will notice that for every bit in the number 105 except for the right most bit, the 1s was exchanged for 0s and the 0s were exchanged for 1s. We have a special name for these type of numbers. It’s called two’s complement.

The two’s complement for any binary number can be computed by simply inverting the each bit (turning 0s into 1s and 1s into 0s) and adding 1 to the result.

As another example here’s the number 78 and its two’s complement:

78

Sign (High order bit)							Low Order Bit
0	1	0	0	1	1	1	0

And -78

Sign (High order bit)							Low Order Bit
1	0	1	1	0	0	1	0

As mentioned before most programming languages provide a number of numeric data types that allow the user to store numbers of varying ranges. Since 1 bit is used to represent negative numbers, the range of numbers that can be represented will be different

Data Type	# Bits	Number Range
Byte	8	-127 to 127
Short/Word	16 (2 Bytes)	-32767 to 32767
Double Word	32 (4 Bytes)	-2,147,483,648 – 2,147,483,647
Long	64 (8 Bytes)	(-2^63)+1 – (2^63)-1

Some languages like C/C++ provide signed and unsigned integers. Signed integers which use one bit to represent negative numbers and unsigned integers which do not.

So now that you know about signed and unsigned integers, what do you think will happen if you perform a calculation like 2,147,483,647 + 1 and store the result in a signed integer? What do you think will happen if you store the result of 4,294,967,296 + 1 in an unsigned integer?

In the case of unsigned integers, what happens depends on the programming language and the compiler. Some languages will raise an error when this happens. Most however will either perform a modulo/wrap around operation or have some undefined behavior.

In the example above the let’s let UINT = 4,294,967,296 + 1. Then

UINT + 1 = 0

UINT + 2 = 1

UINT + 3 = 2

And so on..

In the  case of signed integers the result might be even more surprising. Let’s let SINT = 2,147,483,647. Here’s what you can mostly expect:

SINT + 1 = -2,147,483,648

SINT + 2 = -2,147,483,647

SINT + 3 = -2,147,483,646

And so on..

The result becomes a negative number.

These cases, where the result of an integer operation does not fit within the allocated memory space, are called integer overflow errors. As you can imagine these errors are very dangerous because they often lead to unexpected behaviors.

Avoiding integer overflow errors usually comes down to checking the result of the addition. If the sum of two numbers is less than either of the operands, then you know for sure an integer overflow occurred.

What’s next…

In this article, you learned how positive and negative integers are represented in binary and special cases you should be aware of when adding integers. Next article we will discuss floating point numbers.

If you’ve ever…

• Wondered by 0.1 + 0.2 equals 0.30000000000000004
• Were curious why you may get an integer errors when adding up large numbers, or
• Why you might get a small number when adding two big numbers

Then read on. In this multi part article series, you will learn the answer to all of these questions. I guarantee you that if you take the time to read these articles, you will be a much better coder. Knowledge of how a computer represents numbers will help you design more precise and stable applications.

Let’s Begin!

Base 2 Number System

The base 10 number system, aka the decimal system, is the number system we use in our everyday life. As you all know in the decimal system each digit of a number can have a value ranging from 0 to 9. The places and positions of the numbers are based on powers of 10.

etc.	Hundred Thousands	Ten Thousands	Thousands	Hundreds	Tens	Ones
…	105	104	103	102	101	100
…	4	7	6	9	4	5

Each number position is 10 times the value to the right of it.

A computer on the other hand uses a different number system for representing numbers. Every piece of data stored in a computer system is represented in base 2. This is also known as a binary number. Each position in a binary number are based on powers of 2.

etc.	256	128	64	32	16	8	4	2	1
…	28	27	26	25	24	23	22	21	20
…	1	0	0	1	1	1	0	0	1

Each position in a binary number can have a value of either 0 or 1.

Here’s an example of a binary number:

100011

The binary number above represents the decimal number 35. We’ll get to how to convert from binary to decimal  in a bit, but let’s quickly cover counting in binary.

Counting in Binary

Below is a table of numbers from 0 to 20 and their representation in binary. Can you see the pattern?

Number	20	19	18	17	16	15	14	13	12	11	10	9	8	7	6	5	4	3	2	1	0
Binary	10100	10011	10010	10001	10000	01111	01110	01101	01100	01011	01010	01001	01000	00111	00110	00101	00100	00011	00010	00001	00000

Counting in binary is very similar to how we count in decimal.  In decimal we start counting at 0 then count from 1 all the up to 9, then start at 0 again but add 1 to the digit on the left, resulting in the number 10.  Counting in binary is the exact same process, except we reach 10 much sooner.

Converting from Binary to Decimal

To convert a binary number to a decimal number, multiply each digit by its binary positional value and then all together. Here’s an example to make things clearer:

100011

• The first digit on the left is 1. It’s is in the 2⁵ position, making 1 * 2⁵ = 32
• The next digit is 0. It’s in the 2⁴ position, meaning 0 * 2⁴ = 0
• The next digit is 0. It’s in the 2³ position, meaning 0 * 2³ = 0
• The next digit is 0. It’s in the 2² position, meaning 0 * 2² = 0
• The next digit is 1. It’s in the 2¹ position, meaning 1 * 2¹ = 2
• The next digit it 1. It’s in the 2⁰ position, meaning 1 * 2⁰ = 1

Adding these products together, we get 35.

Converting from Decimal to Binary

Converting from decimal to binary is accomplished by repeatedly dividing the decimal number by 2 until the quotient becomes 0. The remainder of the division at each iteration becomes a digit in the binary number.

Here’s an example:

Converting 13 to binary

Division by 2	Quotient	Remainder
13/2	6	1
6/2	3	0
3/2	1	1
1/2	0	1

13 is 1101 in binary

That’s it for now. Next article in the series, we will talk about positive and negative numbers in binary.

There are a number of techniques for testing the performance of newly developed machine learning models. Have you ever wondered which technique you should use for your data science project and when to use it? The goal of this post to provide some helpful guidelines.

First things first, how big is your dataset?

If you’re working with a very small dataset, consider using leave one out cross validation. In this method each observation is held out from the dataset for testing. The rest of the observations are used to train the model. This is repeated N times  for each observation in the dataset and the test error is computed as the average of all N errors.

If you have a decently sized to large dataset then turn your attention to the next set of methods.

Say you’re just kicking off a data science project, are in a bit of a time crunch, and just need a quick and dirty model to serve as an starting point for your project. The holdout method is a good go to technique to use. This technique involves splitting the dataset into a “train” and “test” set. The training set is used to train the model and the test set is used to see how well the model performs on unseen data. A common approach is to use 80% of the data for training and 20% of the data for testing.

If you have more time on your hands K fold cross validation is the preferred method over hold out as it allows a better indication of how the model will perform on unseen data. In this approach the data is randomly split into k groups. One of the groups is used as the test set and the rest of the groups is used for training. This process is repeated until each group is used as the test set. The test error is computed as the average test error from all groups.

But regular K fold cross validation and hold is not ideal for datasets containing class imbalance. For cases like those, stratified cross validation is the way to go. In this approach the train-test splits are chosen to ensure that each set has the same percentage of examples of each target as the complete dataset.

If you choose K-fold cross validation and you intend to use it tune the hyperparameters for your model and also get an error estimate, there is much better approach my friend. Using nested cross validation for this task is suitable as it avoids overfitting. This article provides a detailed breakdown of the technique https://machinelearningmastery.com/nested-cross-validation-for-machine-learning-with-python/.

If you’re working with time series data, where the goal is to produce of forecast of future events, none of the approaches discussed earlier will work. Time series cross validation is what you’ll need. In this approach a single observation is used as the test set and the training set is composed of observations that occurred prior to the test observation. This is repeated multiple observations used as test points. The forecast error is computed by averaging the errors for each test point. This approach can be extended to evaluate multi-step forecasts.

Well, that’s it. Feel free to share your thoughts in the comment section.

Take care

One of the biggest killers of any Data Analysis project is Bias. Bias in data can appear in many forms. Today I will briefly describe three types of bias and how to avoid them.

Algorithm Bias

This is the kind of bias that results from selecting a machine learning model that that is too simple for the problem at hand. A classic example of this is using a linear regression model on data where no linear relationship exists between the predictors and the attribute being predicted. The model is unable to capture all of the signal in the dataset. You’ll know you’re dealing with this kind of bias when you see a very high inaccuracies in both your training and test set. 

To avoid this error:

Experiment with models that have more complexity. Use cross validation techniques to measure the performance of the your models and select a model that has an optimal balance between bias and variance.

Measurement Bias

This kind of bias happens as a result of errors in the data collection process. A few examples of this kind of bias is:

• Image classification system where data is collected from a camera that has a much higher image quality than what will be used in production
• Incorrectly labeled audio files for a project where the goal is to build a model that identifies male voices from female voices
• A survey containing leading questions which influences answers in a particular direction.

To avoid this error:

• Compare the outputs of the measurement tools used for data collection to make sure they are consistent with the tools used in production.
• Properly train labelers and annotation workers before putting them to work on data.
• Comply with survey best practices

Sample Bias

This kind of bias is also a result of faults in the data collection process. Whereas measurement bias is a result of measurement errors in the data collection, this kind of bias is caused when the data used comes from a sample of individuals that is not representative of the population of interest.

Amazon’s failed AI Recruiting Tool, which I wrote about in my last post, is one great example of this type of bias. Because the dataset used for the product came from resumes submitted from mostly male applicants, the AI tool had a strong preference for male candidates over female candidates.

To avoid this error:

• Clearly identify the goals of the project the data will be collected for and the intended audience the project is meant for.
• Use random sampling techniques  to make sure that every member of the population has an equal chance of being selected.
• Double check work to make sure no mistakes were made.

Employee staffing and recruiting is serious business. According to this source, staffing and recruitment in the United States was valued over $150 billion in 2019. Many companies (especially organizations like Facebook and Google) spend millions scouring the country for talented professionals and interviewing them opportunities. Given the amount of time, energy, and stress involved, it makes sense that some firms would want to automate some (or all) of the recruitment process.

Many believe that AI can be a super effective means to automatically identifying qualified candidates from the sea of resumes that often hit a company’s applicant tracking system. Some may also believe that unlike a human who is often fallible and have biases that can cause them to reject an otherwise qualified candidate, an AI has no such biases and can evaluate a candidate with lightning quick, analytical precision.

The actual truth is that AI, if not properly designed and managed, can magnify the biases of its creators, often in ways that were unintended.

In late 2018, Amazon terminated an AI product they’ve been using internally to automatically vet candidates for a number of roles. The reason — their AI models had a strong preference towards female candidates.

What went wrong?

AI is neither sexist nor does it have any particular opinion about people or society. It merely finds patterns in data that best correlate to the target specified by the creator.

There’s been a number of articles already written on the subject, a lot of which rightfully point out that the data used to develop the tool was resumes submitted over a 10 year period, a majority of which were submitted by males.

Biggest takeaway for in this story is to avoid solely relying on AI to make business decisions.