Brought to you by Ethegra Technologies

Written by Matthew Simon

One of the most relevant technologies of this day and age is Digital Data.

Digital Data is everywhere and everyone is trying to use it! When see an event and record what happens we are creating new data or information. With digital data, we are recording information on our computers. Because computers (more or less) speak with data wherever we find computers we can find digital data. This data can be used to understand events, understand people or predict what will happen next.

We can use data for a lot of convenient applications in our day to day lives as well. I just finished watching Netflix on my friend’s account      (As we all do from time to time) and I was recommended to watch the Blacklist which just so happens to be my exact kinda show. I was able to find the show because Netflix looked at my viewing choices and knew I was about that badass drama multilayered FBI themed show with convoluted plots and capers. The service was able to recommend to me what to watch through a technology called Machine Learning.

For a while, Machine Learning has been the darling child of Silicon Valley. If you have heard of Artificial intelligence, Machine learning is to full-blown artificial intelligence what Charmander is to Charizard. It breathes fire, but it doesn’t quite have its wings yet. Skipping over nerdy pokemon references, Machine Learning is basically a baby step to computers being able to think for themselves.

With a mix of math and computer wizardry, Machine Learning lets us train computers to do work on its own without much human input. In concept, this is an amazing technology.

What can Machine Learning/Ai do (Big Picture)?

Machine learning allows computers to see things that we as people cannot. Humans are really good at finding patterns in the world that we see but humans do not see the world in numbers as a computer does. Due to the rise of the digital age (note you are reading this on a computer or it was printed from one) a good chunk of our modern world is conducted in numbers. Those images you share on the gram, the music you listen to, this article all of those are real numbers rearranged to represent something that people can understand. Machine Learning lets computers find some kind of pattern in all of these numbers and group the numbers or predict the numbers that will happen next.

We have used this technology to predict drugs that might work in pharmaceuticals or in marketing to group together what customers want, what products and how we can identify them. This tool is incredibly powerful in fields like marketing where the entire point is to understand customers and predict behavior. It is also useful in predicting the patterns of money (finance) or what people might like to see (fashion). Industries have only scratched the surface of what this technology can do. Even outside of business profits, we can use machine learning to predict people’s nutrition habits, how often they should visit a doctor, what routines might improve their physical and mental health the most. In short, this technology has the potential to be amazing, not just in a profit making way.

How does it do this?

Machine learning is good at two things, classifying data and predicting data. How can these two things lead to such powerful results??? That does not make sense.

“I fear not the man who has practiced 10,000 kicks once, but I fear the man who has practiced one kick 10,000 times.”- Bruce Lee

What this means is that it doesn’t matter how many things someone can do, it matters how many things someone can do well. And Machine learning can do these things really, really well

Classification

Classification is essentially answering the question of what group does this belong to. It’s like how they used to train kids in preschool — is this a square or a circle? And what peg does this belong in? This is one of the key parts of a recommendation system. Given a set of shows/movies find the pattern that connects all of them. Figure out how to define that group and if a new movie comes in that belongs to the group of ‘Movies Matthew Watches’ send that movie as a recommendation.

This can be used in medical diagnosis ‘ Is this a Tumor or not a tumor?’ This can be used for spam detection ‘ Is this spam or not spam’ and this can be used in computer vision (very hard to do) ‘Is this a dog or not a dog’

Prediction/Regression

On the other hand we have predictions. Machine learning cannot tell us the future exactly, but it can tell us ‘How much of this can we expect’. ‘How much rain might we get this year?’ ‘How much might a 3 bedroom in this neighborhood cost’. This is really useful in places where we collect data that can be turned into numbers that we can predict. This comes useful in management or in finance where we spend a lot of time looking at things in terms of numbers or in health data (important to me) and biometrics ‘How do we make this community healthier, how much vegetable intake can we expect in this area?’

If you want to know how much? Prediction/Regression can help us.

These tools combined create interesting applications and allow us to apply our knowledge of data to the world around us. As I said before this technology is amazing

Like most amazing technologies Black and Brown people are often the last to get access and these amazing technologies are often used against us. Personally, my goal is for our community to understand how these technologies can hurt us and can help us and for us to work proactively in that realm. This is why I will share with you how machine learning works and some of the dangers that lurk around the corner.

How does Machine Learning work?

A Machine learning algorithm is a lot like training a new employee to do a task. The coder is the manager and the computer is the employee. It is the job of the manager to make sure their employee has everything they need to succeed and get the job done. Machine Learning details the steps we take to make sure our computer can get its job done.

When we are trying to train our employees, first, we give them a set of rules and a goal to reach, with the hope that they get to the goal. At first, it takes some adjusting and a lot of meetings to make sure they are getting the job right. Eventually, the employee gets the hang of doing the job and learns the nuances more and more. Bit by bit we know we are making progress when they can do most or all of the tasks without our help. We make sure the employee’s training does not affect the employee proving themselves out in the field/ the actual job,

Machine Learning is just like this! First, we have a task that we want the computer to do. We give the computer a set of rules, a goal to reach and some data to apply the rules on. The computer uses our rules to look for a pattern in the data that we give it. At first, the computer is a bit awkward, fumbles around, and finds some strange patterns that may or may not make sense. We measure the computer’s progress based on how well the computer finds the patterns that are useful to us. Then we adjust our rules and parameters and give the computer more and more data and based on the rules we gave it and our measurement of its progress it starts to learn/figure out how best to do the task. Once the computer can find the patterns that we want really well we send the Machine Learning algorithm into the world for it to do real tasks for us. As it does these tasks it will continue to learn. Like a freshly trained employee, the algorithm will use this real-world experience to become better and better until it is really really good at what it does.

If our problem is clearly defined and our employee is up to the task, it is an easy training process once we get the rules right. In this case, it is usually easy to train other employees to do this task.

If our rules are badly created/ the task is beyond the employee’s ability/ the employee doesn’t have the proper resources (very common), we might get some result, but definitely not the result we wanted.

A couple of notes here: When I say we give the computer rules, what we are doing is giving it formulas that translate what we want it to do in the human world to math speech. These formulas have usually been worked on for decades and centuries and people are still working on them to make them do the tasks that we want them to do. If you want to learn more I can direct you to resources on the topic or even write how machine learning formulas work in a later post.

What makes a good machine learning algorithm or a good employee (take notes, quiz you later):

First and foremost, we need a goal that our computer/employee can reasonably accomplish. There are some problems a computer can machine learn with our current understanding of math and technology and some that it can’t. As I said before, machine learning can do two things really well: predict or classify, if you ask it to do something else it might spit in your face (hence the llama).

Once we have a goal for the algorithm, we express the goal that we want the computer to accomplish in terms of math. We call this expression a Hypothesis function (statistics). The hypothesis function is how we translate our language to computer language and say reach this goal, please. The answer to the hypothesis function is the answer we are looking for. Depending on what the problem is the Hypothesis function varies and is different. 

As a note If you still have nightmares from high school statistics, please do not worry. Much like how a calculator comes pre-programmed to do long division without you having to get out a pen and pencil, many machine learning and AI algorithms come ready baked to deploy in apps or programming tools. This added context is nice to have but not necessary knowledge in deploying machine learning or AI algorithms, so there is no need to worry! (now back to our regularly scheduled programming) 

Once we are sure we have a problem our computer/employee can accomplish we need to have A set of tasks to complete in order to get the goal. Without a way of reaching the goal, our intern/employee will be lost when figuring out what to do. When programming machine learning employees ‘Cost Functions’ are the tools we use to guide the computer, these are our tasks. Usually, when we give the computer a cost function it is like playing a game of hot and cold. The closer to the right pattern the computer is the hotter the computer is and the less it costs, the further away from the right pattern the colder the computer is and the more it costs. The computer’s task is then to find the hottest place and record what it finds there which gives us the answer to the hypothesis. However, the computer still doesn’t know how to do that

Once we have defined the goal and the tasks to reach that goal we now want to have A set of Rules and Guidelines to complete the tasks. These rules and guidelines are what we call…. Drumroll please The Learning Algorithm (hence the guy reading — by the way i imagine Donald Glover/Miles Morales in that outfit). The learning algorithm is how the computer figures out what direction to go to get warmer. It is a formula that tells the computer how to get closer and closer to solving the cost function and getting an answer to the hypothesis. What makes these formulas/Rules/Guidelines powerful is:

• We do not need to know the exact path we just need to show the computer how to find it and
• The learning algorithm not only tells the computer how to think about the task, but it also tells us when the computer/employee is messing up so we know when we need to drag them to a meeting and debug.

Also, even though there are a bunch of different hypothesis functions and many different cost functions there are only a few learning algorithms in machine learning that are powerfully and consistently used. It is in the learning algorithms strength that we base all the other work on. Without it we would have a bunch of equations and a bunch of headaches.

Our machine learning algorithm/employee has everything they need to succeed or at least all the parts we can design ourselves. Now once we know we have the right problem and the right rules we need to give our employees The right data. Data is equivalent to the customers that come into our store or business or the resources we give to our employees to complete the tasks. We can do our best to set goals for our employees, give them tasks and give them a way to complete those tasks but if we do not have the proper resources (enough data) our employees cannot complete their tasks. Just as scarily, we cannot control our customers or fully control the quality of the resources. Whoever our customers are, our employees will adapt to and we cannot control how they adapt to them.

Data is not nor will ever be perfect. This means that machine learning is not nor will ever be perfect. Even if we get every mathematical step correct (which is also not easy to do), we still have to deal with imperfect data that comes from an imperfect world. When people forget this, things can go horribly wrong What is an example where machine learning predictions can go wrong or lead us to an incorrect result?

Example of how Machine Learning data can lead to incorrect assumptions and how do we deal with that 

Imagine you were an ice cream truck dispatcher ( Just imagine this please). It is your job to send out the ice cream trucks to cover the most kids and get the most profit.

You send 25 trucks to neighborhood A and 12 trucks to neighborhood B. You do this because of the hunch that neighborhood A has more kids that want ice cream. When the trucks came back, Neighborhood A sold 25 units of ice cream and neighborhood B sold 12 units of ice cream. Which neighborhood wants ice cream the most?

It might be easy to say that neighborhood A wants more ice cream because they sold more units and that confirms what you originally believed. However, in reality, every truck in Neighborhood A sold the same amount as in Neighborhood B. Each truck sold 1 unit of ice cream. There is strong evidence that the kids in both neighborhoods like ice cream the same. However, since we sent more trucks to neighborhood A we sold more ice cream there plain and simple.

Now, let’s go back to our original example of training an employee. What if we took this information we collected about Neighborhood A and Neighborhood B and gave it to a new employee that knew nothing about the trucks and logistics? Then we tell them to predict where to send ice cream trucks. We tell this new employee, only base the prediction on the information we just gave you and focus on where we sell the most units of ice cream. We don’t care how many each truck sells, we just tell our employees to get us the most ice cream sold. What do you think it’s going to say? Spoiler Alert: It’s not B.

No matter how good the employee is, it will take the data we give it and spit out the answer that matches it. In this case, always predict that we will sell more ice cream in A than B because the way we collected the data says we sold more ice cream in Neighborhood A than in Neighborhood B. This result has far reaching effects on all kinds of machine learning and AI outputs in data. It means that just as important is the algorithm, it is important where we get the data from and what assumptions we make as we created the machine learning system. 

If you would like to follow up more on this topic, this is what we call bias in data. There are all kinds of biases in data and it is the job of a statistician or critical data user to detect and account for the various kinds of bias. No matter how good we are at collecting data or thinking about it, it is almost impossible to get rid of bias whether it comes from our personal beliefs, from the way we collected the data or for what we are using the data for. The best we can do as citizens of a data society is be aware of bias and minimize it where we can see it. Why does this matter? 

Now let’s put this into context. Due to the structure of our society, we will never be able to collect data equally across all places and have perfect data in every situation. This means that no matter how sophisticated our Machine Learning / AI predictive model is, it will only be able to give us a prediction with as much information or consideration as we input into it. What this means is that it is critical that when we look at data we put it into context and make sure that the information we are getting from our Machine Learning and AI systems is both reasonable and equitable. This means we have to use various resources, double check our information and connect with members of the communities that our data touches on to make sure we are building systems that will improve the lives of people that use them. What I argue is that while it is important to build sound technical systems it is also important to educate people about data and build sound people systems. When thinking about doing data science research or building machine learning algorithms we have to center people and empathy as much as we center the predictive capabilities of the work. Once we do this only then can we get the true power of our Machine learning and AI systems.

What does this have to do with AI?

Well the parent of Generative Ai as we know Chatgpt/Claude and any of the other Language learning models is Machine Learning. Essentially 60% of AI is a computer learning to classify and prediction mountains and mountains of data until it learns how to mimic and respond to human speach. In addition to the machine learning algortithms the computer also stores the information that it picks up from this data and uses what is called a learning algorithm to create connections and a map of all this information. Then when you ask an LLM a question it refers to this map to give you the most cohesive answer that it can. It does this so many times until it sounds like a human. We will be going into more how this works, what is AI and how to actually use it so stay tuned!

Summary 

To recap, we discussed: What is Machine Learning, Why it is important, How do machines think about problems and what are the steps to building machine learning and AI systems. We also took it a step further and discussed what makes a good and successful machine learning system and began to include how to examine and unpack bias in data science systems as well as how we can integrate people and empathy in the path to learning more about data and our world. 

I hope this piece helped to demystify a bit about the world of data science and machine learning and encourages you to ask more questions about it, particularly about how we can use data science to make a safer, healthier and more equitable world. 

If  this was interesting to you please stay tuned for more content on data science and social good from Ethegra Technologies LLC.