The story

In January 2023 we started this newsletter with no big expectations. We just wanted to work together on something since we love each other’s content.

We created 50+ posts on Substack, and had the chance to interview and work with awesome data folks. We learned a lot during this time and hope you did as well. That was our goal. To give back to this awesome audience that grew to 5K. Crazy numbers we reached during this short time.

It was a great ride, but life hit hard on us in the last few months, so it became clear that we couldn’t continue to work on this. Both of us work full time and we have our own projects. We don’t have 30 hours a day (unfortunately) so we need to cut down.

This time DSBoost got the hit sadly.

What’s next?

Leaving DSBoost doesn’t mean that we are exiting. Again, both of us have our plans and you all will benefit from it.

Here is how this will happen:

• The posts we published so far will remain on this site for a few months.

• You will receive posts from MLPills, since David stays active on that platform.

• You will also receive content from Levi, who will publish from Data Ground Up.

Technically this means more content from us, but working separately.

To make it effortless for all our readers, all the subscribers from DSBoost will be transformed into the new platforms.

Thank you!

Thank you for supporting us under the name of DSBoost.

Please keep this up for the future, since as mentioned: We will see you soon!

Best,

David & Levi

Should Kids Still Learn to Code? (Hunter Kempf) - KNN Ep. 182

In a recent statement, Nvidia CEO Jensen Huang has predicted the death of traditional coding.

In this episode, Ken and Hunter reacted to this news:

Key takeaways

With the latest advancements in LLM, it’s easy to see how programming will change dramatically, but the core principles will not.

In the future you will not write code with “computer language”, but with your native language. Instead of writing traditional code, you will create prompts that instruct AI to generate the necessary code. The so-called prompt engineering skill will evolve.

However, the argument that children should no longer learn programming is misleading!

Like mathematics, programming teaches logical thinking, creativity, and problem-solving skills. Learning to program can enhance one's ability to approach problems systematically. That is an invaluable skill not just in tech.

Programming is an awesome learning tool generally:

• The feedback loop is super fast. You build things and can test them immediately if they run and work or not. You get an error and fix the error within minutes. You fail quickly but learn quickly as well.

• Coding is open-ended. You can and must think outside the box, be creative, and use your imagination. You can build whatever you want.

Google’s new AI and it’s struggles

Google recently released its new AI model, Gemini AI, but the launch was anything but smooth.

The model generated historically inaccurate images and outputs that raised racial concerns, so Google paused the image generation tool.

Why did it happen?

A primary issue was the rush to release the product. Reports suggest that bad Gemini responses slipped through testing because there was a hurry to ship the app.

AI competition is the highest priority of top tech companies since this is the hottest topic on the market.

It turns out that the Gemini model itself does not power the Gemini app. Instead, an older text-to-photo model was used to supplement Gemini.

Google co-founder Sergey Brin admitted that the problems are from a lack of detailed testing and said that the model is still a work in progress.

Now that’s a really dangerous territory. Even the creators of these AI models cannot fully understand what happens in the background!

Is it only Gemini?

All AI models have similar challenges, though they may not be as visually striking as those seen with Gemini.

ChatGPT was also criticised for it’s “political views”.

An AI model is, after all, a creation of humans and will only be as effective as the programmers and data behind it.

Poor performance could be the result of bad coding, inadequate training data, or a combination of both. AI is never wrong!

Tweets to think about

People don’t enter the field anymore, because they think AI will take away DS jobs since it writes relatively good code.

• As Santiago points out coding is just one part (smaller part) of the job. If you want to be the one who codes 24/7 DS is not your field.

• DS is about the analysis and the takeaways. You need some statistics, mathematics, communication, presentation, data ethics, domain, and so much more skills.

• AI is good at the basics, but when domain knowledge enters the chat AI leaves. Try to niche down early in your career and you will leave behind AI soon.

  A related tweet:

The more you know the more you realize you don't know. That’s so true about Machine Learning and Data Science in general. It’s a broad field with all the mathematics, statistics, and coding.

As a beginner, it’s tempting to learn a lot of new things and try to be as fast as the technology around us. It’s impossible, so the sooner you realize, the better.

Again, try to niche down asap!

source

How can I compete with this?

Reddit post:

I graduated 2 years ago, and have 2 years of experience. Just started applying to new roles.

I keep finding that nearly every role has a disproportionate number of applicants with master degrees, whereas I only have a BS.

I feel like experience matters more, but hiring managers will just decide to give the people with the masters more attention. Is this true?

Our view

While having a master’s degree can be beneficial, experience is also highly valued in the field of data science.

Practical experience, especially when it can be directly applied to the job at hand, can often outweigh the theoretical knowledge gained from a degree.

Some advice

1. If you’re currently unemployed, the first step is to get a job, even if it’s not directly related to Data Science. If you’re already employed, try to find ways to apply data science in your current role and then include it in your resume.

2. It’s highly beneficial if you can attribute a dollar value to your contribution in your current role.

3. There are data scientist resume templates available that you can edit and use depending on where you’re at in your data science career (entry-level, senior, or looking for a manager role). This will allow you to stand out by having a tailored resume.

4. A strong portfolio can be built for any type of role, including non-technical roles like product and marketing. The best portfolio projects share a few themes that can impress any hiring manager, no matter the field. Don’t forget the deployment part! This can make you stand out with respect to other candidates.

5. It could also be beneficial to learn multiple programming languages. For instance, learning both R and Python can be advantageous as both have their strengths. Also acquiring complementary skills such as marketing or financial knowledge, could be highly beneficial.

6. It’s important to understand the data science interview process and make sure you get prepared for each step.

7. Also learn soft skills, especially communication! The interview and partly a DS role is about communication. You need to sell yourself in a short amount of time in an interview. You cannot do that without good communication skills. Communication is also an important part of the job. Presentation, code explanation, educating others, etc. All requires good soft skills.

(Cover image created with Dall-E)

💭Some Thoughts on Apple’s New Tech in DS

The internet is full with the Vision Pro these days. How it can affect our beloved industry?

Interactive Visuals

The biggest advantage of Apple VP is the AR experience. You can interact with things in the 3D world. This will enable interactive data visualizations, but not in the context we experienced before. You will be able to walk between the bars in a bar chart.

Collaborative Work

The above can help collaborative data exploration and decision-making processes. Your team will be able to sit around the same chart in the shared virtual space.

Enhanced Data Collection

For fields that rely on visual data such as medical research, and environmental science, the Apple Vision Pro could offer new methods for gathering high-quality datasets.

Productivity

In VP you can use as many windows/screens as you want. Some will find this distracting, but for many, it will be a productivity boost. You can also block the world around you, so no distractions at work. You can ‘literally’ work from the Moon.

Interactive Learning

The best way to learn something is to interact with it. With AR and VR interactivity is on the highest level so far. High-tech teaching will never be the same after VP.

Spatial Data

Devices like the VP need to understand the environment and world around them to work safely and effectively. For this reason, there is a big need for Spatial Data and Computer Vision insights.

Simulation and Testing Environments

Developers could use the device to create simulated environments for testing AI algorithms and models.

😂Tweets to Laugh and Learn

If we talk about Apple VP and DS… 😂

At this moment AI is just so random… It is the dumbest thing but changes the world at the same time.

After some laughs it’s time to learn.

These are all awesome sites but don’t forget to practice what you learn with some projects! Go to Kaggle, download a dataset, and do some cool analysis to begin your portfolio of projects.

🎙️Podcast Notes

SDS 752: AI is Disadvantaging Job Applicants, But You Can Fight Back — with Hilke Schellmann

Key takeaways:

AI in hiring processes

• AI is increasingly used in the hiring process, but it is not always fair, accurate, or transparent.

• Many AI hiring systems use biased data, flawed logic, and opaque methods to screen out applicants, especially women and minorities.

• Job seekers can fight back against unfair AI hiring practices by researching the company's hiring process, preparing for video interviews, and challenging the results if they feel discriminated against.

AI ethics

• AI ethics is a crucial topic for data scientists, as they have the responsibility to design, develop, and deploy AI systems that are aligned with human values and social good.

• Data scientists should be aware of the potential harms and risks of AI, such as privacy violations, discrimination, manipulation, and deception, and take steps to mitigate them.

• They should also be aware of the potential benefits and opportunities of AI, such as innovation, efficiency, and empowerment, and seek to maximize them.

• Data scientists should follow best practices and standards for AI, such as fairness, accountability, and transparency, and adhere to ethical principles and codes of conduct.

• Data scientists should communicate and collaborate with other stakeholders, such as employers, regulators, users, and society, to ensure that AI is used in a trustworthy and beneficial way.

👉 So remember, as a Data Scientist you have a big responsibility: you need to be aware of both harms and risks, and benefits and opportunities.

From Divinity Student to Data Scientist (Immanuel Tacky) - KNN Ep. 178

In this episode, Ken talked to Immanuel who had a hard journey to Data Science. Everyone can learn from his story, so here are our notes:

Here is Immanuel’s road:

1. He did a Bootcamp where they sent over the CVs to employees but had no success.

2. He did some courses online and added the takeaways to his resume.

3. He applied to companies for almost 2 years (sent 600 applications) and only got 3 responses.

4. Feeling like he had some gaps in his knowledge, Immanuel decided to hit the books again. He went for a tech MBA. University isn’t just about what you learn; it’s about who you meet. Networking is gold in the tech world.

5. He did a tech MBA program.

6. He got an internship and then a full-time role.

Note: Everyone’s journey is different and Bootcamps work for some while it’s a waste of energy and money for others. The biggest takeaway here is to never give up and hard work will pay off.

The University Debate: Worth it or Not?

You will see a lot of people online discrediting universities. Immanuel’s story shows that listening to those people is not always the best strategy.

Here is a crazy story Ken shares: Sometimes, just being enrolled in a master's program can make your resume shine. Some folks even landed jobs and then left the program. (It's not Ken’s recommended strategy, but it worked for some.)

The other big advantage is that you have access to a great network and you can develop connections that will be beneficial in the future.

They can also give you real-life experience through internships! In some cases, companies directly go to the universities and interview some students for a role and hire maybe 4-5 of them.

The other side is the prices of these programs. The cost can be higher than the reward in some cases, but with scholarships, it can be easier.

💼What kind of Data Scientist is in demand for 2024?

Here are some skills that could be beneficial for a Data Scientist in 2024 according to the previous Reddit post:

• Python Skills: Python is one of the most popular programming languages for data science. It’s used for data analysis tasks and has powerful libraries such as pandas, NumPy, and matplotlib for data manipulation, statistical analysis, and data visualization. Python is also dominant in advanced data science subdomains, including machine learning and deep learning.

• SQL: Knowing SQL allows you to retrieve, manipulate, and analyze data stored in relational databases.

• Data Analysis: Data scientists often need to collect, analyze, evaluate, review, organize, and visualize data. They organize the data and perform statistical calculations to find trends that can solve problems for a client or their employer and inform important business decisions.

• Versatility: Data scientists often work on tasks typically done by data engineers, data analysts, or software engineers. Embrace these opportunities to diversify your skill set and increase your value in the job market.

• Leadership Skills: Given the nature of their responsibilities, data scientists require a balanced set of technical skills and leadership skills. This includes the ability to communicate effectively, manage projects, and lead teams.

• MLOps: 5 years ago, there was more focus on training Machine Learning models and obtaining insights. This is not enough nowadays, companies want the models to be automated and deployed in a pipeline and want Data Scientists to retrain models when a data drift occurs.

Now some additional advice taken from comments based on work experience in the same post:

• Career Progression: The demand for data scientists is high. It might seem daunting if you're just starting, but with perseverance and continuous learning, you can aim for these roles.

• Entry-Level Competition: Be prepared for stiff competition at the entry-level. Make your application stand out by gaining practical experience wherever possible and showcasing your unique skills and projects.

• Versatility: As a data scientist, you might find yourself working on tasks typically done by data engineers, data analysts, or software engineers. Embrace these opportunities to diversify your skill set and increase your value in the job market.

• Job Market: Navigating the job market can be challenging, even for those already in the field. Networking, staying updated with industry trends, and continuous learning can be beneficial.

Remember, these are general observations and experiences shared by users on Reddit and may not reflect the entire industry's situation. For the most accurate information, it's recommended to conduct further research or consult with industry professionals.

Good luck on your data science journey!

Data Storytelling and Visualization with Lea Pica from Present Beyond Measure

by Datacamp

Key takeaways:

The audience

Knowing your audience and tailoring your approach accordingly can make a significant difference in the impact of your presentation.

You should always adjust your presentations to the audience! Here are some tips:

Targeting the Decision Makers

In most business meetings, you're primarily addressing the decision makers. These people are the ones calling the shots and to whom you're essentially 'selling' your insights. They're typically overwhelmed so your data presentation needs to be concise and persuasive, like a sales pitch.

Targeting the 'System Enablers'

They are not primary decision-makers but valuable allies. They prefer straightforward presentations without the complexity of deep statistical analysis.

Targeting a technical group

Your analysis should be robust, detailed and be prepared for challenges. This group demands precision, and being well-prepared to navigate their critiques is essential.

Hollywood aspects of storytelling

Take the audience on an emotional journey of transformation!

Organize your information not just as a series of charts and tables but as a cohesive story.

Tips for implementing storytelling in data presentations:

- Identify your 'Villain': Every story has a conflict or a 'villain'. In the context of data, the 'villain' could be a challenge or obstacle your organization faces.

- Amplify the Stakes: It’s crucial to emphasize what your organization stands to lose if no action is taken.

- Use drama: While over-dramatization is not advisable, adding a degree of drama helps in emphasizing the potential loss or impact.

- Communicate the significance: Your presentation should answer why the audience should care and what could happen if they ignore the data.

- Inspire action: Finally, motivate your audience to act based on the data presented.

Other quick tips

- Limit Key Ideas: The human brain's working memory can only handle about three to five main ideas effectively. You don't want to overwhelm your audience.

- Avoid presenting spreadsheets and screenshots directly: Instead of overwhelming the audience with a wall of numbers, it's better to focus on key data points. Extract the most relevant information and present it in a clear, visual format like a bar or line chart.

- Line plots and bar plots are powerful tools: despite the variety of chart types available, these two are effective and versatile. Understanding the context of information consumption is crucial when choosing the appropriate chart type.

- Environment Matters: The environment in which information is presented influences the choice of visualizations. Dashboards are suited for monitoring and trends, with line graphs being essential for tracking performance metrics. In contrast, presentations may involve aggregate information summarized in bars with deeper dives into specific points.

- Attention Management: Leveraging visual elements, animations, and intentional pacing can help maintain audience engagement. You can also use keyboard shortcuts to turn slides black or white to refocus attention and strategically insert blank slides for pauses.

- Incorporating Real Imagery: Beyond traditional charts, real-life images, avatars, and photographs can enhance engagement and create emotional connections with the audience. This approach humanizes data, making it more relatable and memorable. Decisions are often emotionally driven, the use of imagery, particularly emotionally resonant images, is encouraged to complement data-driven arguments.

Create Impactful Narratives

- Presentation Throughline: The concept of a "Throughline" is a crucial tool for creating focused and impactful presentations. It involves summarizing the presentation in a single sentence, helping maintain clarity and coherence throughout the presentation.

- Adaptable Storytelling: While it's beneficial to start with a planned throughline, the storytelling process can be fluid. As you delve deeper into the data, you may discover new insights that add twists to the narrative. The key is to ensure that the final presentation aligns with the throughline.

- Recording and Self-Reflection: Utilize self-recording as a tool for improving speaking skills. Listen to your recordings critically, evaluating vocal patterns, cadence, and engagement levels. This practice helps identify areas for improvement.

- Reading Children's Bedtime Stories: Reading children's stories can be a useful vocal warm-up exercise. The exaggerated and animated style of reading helps bring variation to cadence, improving diction, and adding more excitement to the delivery.

- Using Humor: Humor can also be an effective tool for keeping the audience engaged.

- Educational Resources: Reading books on visual communication, storytelling mechanics, and speaking skills tailored for data and marketing practitioners is recommended.

- Audience Focus: It is critical to shift the focus from oneself to the audience. Understanding the audience's needs and delivering content that aligns with their interests is key to successful presentations. Think about who the audience is and what they care about when preparing presentations.

Interested in learning how to select the best chart for your presentation? This post can help you master the basics:

I hope this was useful!

Let us know in the comments what type of chart you normally prefer and why.

SDS 745: 2024 Data Science Trend Predictions - with Sadie St. Lawrence

Jon and Sadie St. Lawrence talk about their predictions and what 2024 will bring in Data Science. 

Here are the key takeaways:

Sadie’s predictions are:

1. Hardware evolution: Demand will increase for data and complex models. ➡️ We need better hardware;

2. LLMOS: LLMs will kill Operating Systems.

3. Slow-thinking model: LLMs with more computational and mathematical knowledge will arise;

4. Tool consolidation: Full stack data analysis apps will appear.

5. Workforce Upheaval: Data roles are changing. Old jobs will disappear and new ones appear.

Jon’s predictions for 2024 are: 

1. AI bubble bursting: In 2023 there were too many AI startups. Only the companies with solid foundations will remain.

2. Breakthroughs in Edge AI: Being able to run LLMs on a phone will be common.

Let’s dive a little deeper into these topics:

1. Hardware evolution

   In the upcoming year, we are going to see an increased demand for data and hardware. The industry will be “data and hardware hungry”.

   We have to add more data to existing models to perform better. Thus we need some innovations in the field. But not only by the most important companies, like Nvidia. We will probably see new companies who recognize the opportunity.

   Jon mentions Qstar, a new method for training our LLMs with less data. With tools like Qstar, we can be more efficient with our data. It is important in industries like healthcare where data is gold.

2. LLMOS

   What if large language models will replace operating systems?

   The way we are working today is going to be disrupted by LLMs like it was by mobile phones. We are going to switch from click and point workflow to communicating with our LLM assistant, who will do what we ask.

   This level of technology will not be available this year, but it's an interesting concept.

3. Slow-thinking model

   What exactly is a slow-thinking model?

   LLM models today are fast-thinking models. They are good at the creative side, but they are just throwing words together unconsciously. These models are helpful no doubt, but they are still not at a good level in topics like math.

   This is where slow-thinking models come into play.

   These models are good at solving complex problems at a high level. They could help in fields, like physics, and healthcare where they could be used for discoveries. We will see some advancements in this niche, but there are also some concerns. They may be super good at mathematics and could encrypt existing security walls.

4. Tool consolidation

   Organizations are still going to have budget constraints this year. Because of that, the tools that can do all the processes in one will be more searched and valued.

   A great example is Microsoft Fabric. It connects to any data cloud and does your data governance, ETL, and visualization. It is a full data tech stack all-in-one tool. Similar solutions will be much cheaper for a company with budget constraints.

5. Workforce Upheaval - Old jobs will disappear and new ones will appear.

    

   Sadie shares a story when she got into data science over 10 years ago.

   At that time companies employed full-stack data scientists. They had to clean and manipulate data, build visualizations, and communicate the results. These processes are now split up into multiple job functions.

   Those range from data engineers to data analysts and data project managers.

   Now, these roles will also transform into something different with LLMs.

6. AI bubble bursting

   The AI bubble is going to burst in 2024!

   In 2023 a new AI startup appeared every minute. This trend cannot continue. Investors will want to see some returns on their investments. A lot of AI startups will disappear and only the companies with a good foundation will remain in this field.
   

7. Breakthroughs in Edge AI
   

   In the next few years, we will be able to run large language models on a phone!
   

   Now we are limited. We need to send a query along an API to some central server that does heavy-lifting computing and send the response back.
   

   But can you imagine when a smartphone can run its own LLM?
   

   This would open up a lot of possibilities in many industries like agriculture, mining, and healthcare.

Stay ahead of the AI wave! 🌊

New in the field? Before starting to learn Data Science and Machine Learning you need to build a solid foundation. Check this thread to learn more about it:

Do you already have a solid foundation? Time to learn Data Science:

• Python

• SQL

• Power BI

• Machine Learning

You can learn all of these for free thanks to these 6 YouTube videos:

Enjoy and good learning!

1. Python & Pandas

Python is the most popular language in Data Science. You cannot go wrong with it. While R is fantastic for statistics, Python is more versatile, making it invaluable if you ever decide to pivot from Data Science.

After getting comfortable with Python, introduce yourself to Pandas. This powerful library is the go-to tool for data manipulation and analysis.

For a comprehensive introduction to Python, check out this tutorial.

And for Pandas use this playlist.

2. Data Visualization

It's not only about crafting visually appealing charts but also about understanding data cleaning, unleashing creativity, and getting used to coding best practices.

I usually suggest starting in your niche or with a topic you like. If you like football, you should recreate some scoring charts.

You will not find clean datasets for all charts, so it’s a good activity to practice data cleaning as well.

Matplotlib is the biggest data visualization library in Python, start with this tutorial.

3. SQL: The Language of Data

It’s a must for any serious data professional. SQL it the backbone for storing and retrieving data in most applications around us.

Dive into SQL with this comprehensive guide.

4. BI Tools

Business Intelligence (BI) tools are essential in translating complex data into actionable insights. This skill set is not just about creating visuals; it encompasses presentation, communication, and effective dashboarding techniques.

After mastering Power BI or Tableau, you're close to the Junior Data Analyst level. Start with this tutorial.

5. Machine Learning (ML)

The next level is to add some ML skills. Familiarize yourself with basic ML algorithms, and grasp the concepts of supervised and unsupervised learning.

You don’t need to do everything from scratch. Libraries like scikit-learn are awesome to start with.

Begin your ML journey with this tutorial.

6. The Cloud

The cloud is revolutionizing data storage and computation. Understanding platforms like AWS is crucial for handling big data, deploying ML models, and ensuring scalable data solutions.

Check out this playlist to get started with cloud computing.

7. Niche down

After the basics, you can go deeper into some niche topics.

For example in Business / Finance some time-series knowledge is a must.

Here is a great roadmap:

Another fun topic is Natural Language Processing.

I am not sure if you have heard about ChatGPT in 2023 😊.

Learn about language models with this playlist.

Best wishes for 2024 and happy learning to all our readers!

SDS 737: scikit-learn's Past, Present and Future - with scikit-learn co-founder Dr. Gaël Varoquaux

Scikit-learn has 1.4 M downloads a day - even though most of them are from bots - it's still a crazy number.

But how did it start?

Around 2005 basic ML codes were created in the SciPy package. At that time there was no efficient ML solution available, so around 2008, the team reached out to SciPy to outmove the ML codes. It was only a small community of developers.

The vision

They wanted to create an ML textbook that is actually runnable. It was about making ML understandable and executable. This vision attracted many contributors, expanding Scikit-learn's capabilities.

Scikit-learn is built on simplicity. This approach makes it accessible even for beginners. The focus is on valid data science practices, like model evaluation and data preparation.

For interpretation, documentation is so important. This makes the tool easy for people to use it well!

If you use Scikit-learn, you don't need to understand deeply what's behind the model. That's why the team set a lot of default values, so it is easy to apply a model with limited knowledge.

Of course, it doesn't mean that the default value is optimal, but you will not get errors when run the code.

Here is a related post from Santiago (click to see the whole post):

How to contribute to open source?

Coding background is not necessarily the most important skill.

Explaining concepts, and making structural changes requires other skills. Scikit-learn has an individual 'documentation team' to review documentation changes.

The community encourages starting with smaller projects and gradually taking on more complex tasks. First, you should talk in the community. Open source is a social thing.

The future

Scikit-learn plans to introduce features for better user experience, like reports providing feedback on models.

For example, if you have imbalanced data it will flag that you shouldn't look at accuracy. It will also evaluate the model, list the important parameters, etc.

They also want to make it easier for users to join Scikit-learn with other libraries.

Improve your skills

We are providing a 15% discount on Train In Data courses when you use the code DSBOOSTNL at checkout. Not only will you benefit from the valuable insights and practical knowledge shared in these courses, but you will also be supporting DSBoost for each course purchased through this offer. Don't miss out on this opportunity to enhance your data science skills. Enroll now and take the next step in your data science journey!

Explore the courses!

Featured threads

And now, because we are in a festive period, here you have two Christmas-themed threads. Enjoy!

This week, we're thrilled to present a special issue that embraces the festive spirit of the season! As numerous countries across the globe gear up for Christmas celebrations, we've curated six exciting project ideas designed to help you hone your Data Science skills with a touch of holiday cheer. Dive into the joy of learning and exploration with these festive-themed projects.

🎅6 Christmas Data Science Projects

1. 🎄Christmas Song Prediction with Neural Networks: Utilize neural networks to analyze radio play patterns and predict the minutes until the next airing of iconic Christmas songs like "Last Christmas" or "All I Want For Christmas Is You." Also, you could forecast the number of times these songs will be searched in Google (check this link). By training on historical data, the model can capture the temporal trends associated with these festive tunes, providing a fun and festive way to anticipate holiday music.

   

2. 🦌Reindeer Detection Image Classifier: Create an image classifier capable of identifying reindeer in pictures, employing deep learning techniques. This project could find applications in monitoring wildlife during the holiday season or enhancing the magical elements of Christmas-themed apps and games.

   

3. 🎅Facial Recognition for Santa Claus: Develop a facial recognition system specifically tailored for identifying Santa Claus in various settings. This could add a whimsical touch to holiday photo albums or even be used as a playful feature in virtual experiences, allowing users to "spot Santa" in their surroundings.

   

4. 🎁Christmas Present Weight Prediction with Machine Learning: Use regression algorithms such as linear regression or decision Christmas-trees to predict the weight of presents on Santa's sleigh based on factors like size, shape, and wrapping. This project could provide insights into optimizing gift distribution and help ensure Santa's sleigh is well-balanced for a smooth journey.

   

5. 🛷Optimizing Santa's Path in Your Region/Country: Apply optimization algorithms to determine the most efficient path for Santa Claus to take in delivering presents within a specific region or country. Consider factors such as geographic layout, population density, and historical gift distribution patterns to create an optimized route, ensuring a timely and joyful delivery experience for everyone.

   

6. 📜Christmas Story Generation and Illustration: Combine natural language processing using ChatGPT for story generation and image synthesis with models like DALL-E2, Midjourney, or Stable Diffusion to illustrate the narrative. Create a unique Christmas story with AI-generated characters and scenes, enhancing the holiday spirit through a blend of imaginative storytelling and visually appealing illustrations. A perfect gift for Christmas!

   

This post was inspired by the article: How to Survive Christmas as a Data Scientist by Isabelle Flückiger.

I hope you’ve enjoyed it! We want to hear about any other Christmas-related project ideas you may have!

From the DSBoost team, would like to wish you a very Merry Christmas!

K-means clustering is a widely used technique in data science. Now we bring you some content so you can master this algorithm:

Intro

An introduction by David here:

• How the model is trained

• How the prediction works

• Advantages

• Disadvantages

Advanced Level

Matt explains K-Means on an advanced level. He lists 10 points about the algorithm you must know!

Code

How to implement K-Means in Python:

Some productivity tips

The Ivy Lee Method is a simple, five-step time management system. Here are the steps:

1. End of Day Task List: At the end of each work day, write down the six most important tasks to accomplish the next day. It's important not to exceed six tasks.

2. Prioritize the Tasks: Organize these tasks in order of importance.

3. Focus on the First Task: Start the next day with the most important task. Work on it until it's completed before moving on to the next task.

4. Move to the Next Task: Proceed to the next task only after finishing the previous one. If a task is too long, break it down into smaller parts and follow the same process.

5. Repeat: At the end of the day, spend 10 minutes planning the next day's tasks.

This method, created by Ivy Lee in 1918, emphasizes prioritization and focus to increase productivity. It was so effective that Charles M. Schwab, the president of Bethlehem Steel Corporation, paid Lee a significant sum for it. James Clear, in his book "Atomic Habits," advocates for this method due to its simplicity and effectiveness.

Get Ready for the interview with these:

(Thread is here or click the image)

These posts from Nick are awesome.

You will find Python, Machine Learning, and general Data Science interview questions.

Here is another one:

The simple path to success in ML:

That’s indeed a really simple path. It involves only 7 ‘baby’ steps. If you walk through these you will have a great intro to the field.

But people usually misunderstand ‘simple’. It doesn’t mean easy.

Now we make it a little easier for you by providing some links to use.

1. Learn Python

2. Google collab intro

3. Pandas

4. Seaborn

The Seaborn official documentation is awesome, check it out here.

Some tips to handle mistakes

Check the full discussion

This Reddit post discusses concerns about making mistakes at work, specifically from the perspective of a Data Analyst/Scientist. The original poster (u/JLane1996) expresses worry about their lack of formal statistical or data scientist training, fearing flaws in coding or misinterpretation of results. Here we summarise the top responses from various users who offer advice and reassurance:

1. Emphasis on Learning from Mistakes: Users point out that all data scientists make mistakes, but good ones learn from them. They suggest staying organized with learning, perhaps by maintaining a document for notes and finding a mentor for guidance.

2. Importance of Process and Review: Several comments underline the importance of a structured process and regular review with peers or management. Process-focused approaches are recommended over focusing solely on individual skills, as processes can help identify and improve weak points.

3. Suggestions for Further Education: Some responses advise working through statistics textbooks or courses to fill knowledge gaps, highlighting the importance of continuous education in fields like econometrics and graduate-level statistics.

4. Acknowledging and Addressing Imposter Syndrome: Many users identify the OP's feelings as imposter syndrome, noting that it's common among professionals who are new to their field. They advise accepting and learning from mistakes, and emphasize that no one is expected to be perfect, especially in a complex field like data science.

5. Practical Tips and Reassurances: Practical tips like writing out steps for analyses, implementing peer reviews, and focusing on the final results of work are also given. The consensus is that mistakes are a natural part of the learning process and can be mitigated through careful procedures and continuous learning.

Check the full discussion

This Reddit post started by u/dhaitz on common mistakes by junior data scientists sparked an insightful discussion. Here we have highlighted the top 10 errors:

1. Data Leakage in Training and Evaluation: A significant mistake is allowing data leakage from training to evaluation sets. This can give a false impression of model performance, especially in cases involving temporal elements.

2. Inadequate Handling of Temporal Data: Junior data scientists often struggle with handling temporal data, leading to data leakage. Suggestions like participating in ML competitions (e.g., Kaggle) were given for better learning.

3. Preprocessing Data Before Splitting: A common error is preprocessing data (like normalization) before splitting into training and test sets, which can bias the test data.

4. Incorrect Application of Categorical Encoding: Using methods like target encoding on the entire dataset rather than just the training set can lead to leakage.

5. Overestimating Model Performance: There's a tendency to overestimate a model's performance, especially when initial results seem exceptionally good, without thorough debugging and validation.

6. Lack of Domain Knowledge and Data Intuition: Junior data scientists often focus too much on modeling without gaining a sufficient understanding of the data and its domain context.

7. Not Starting with a Simple Baseline: There's a tendency to skip establishing a simple baseline model to understand its limitations before moving to complex models.

8. Overreliance on Complex Models: New data scientists often prefer complex or 'fancy' algorithms without assessing their necessity or effectiveness for the specific data type or problem.

9. Failure to Recognize Business Impact: Many junior data scientists fail to understand how their models can practically impact business decisions or actions.

10. Mistaking Features for Causes: There is confusion between identifying important features in a model and interpreting them as the actual causes of outcomes.

These points underscore the importance of a balanced approach in data science that combines technical skills with a deep understanding of data context, careful preprocessing, and an awareness of the business implications of models.

We are excited to announce a collaboration between DSBoost and Train In Data! As part of our launch offer, we are providing a 15% discount on Train In Data courses when you use the code DSBOOSTNL at checkout. Not only will you benefit from the valuable insights and practical knowledge shared in these courses, but you will also be supporting DSBoost for each course purchased through this offer. Don't miss out on this opportunity to enhance your data science skills. Enroll now and take the next step in your data science journey!

Explore the courses!

Podcast of the week 🎙️

Data & AI for Good, with Marga Hoek, Founder & CEO, Business for Good

We hear a lot about the bad, destructive consequences of technology, especially AI.

‘It will take my job.‘

‘AI will kill humanity.‘ It may sound extreme, but it’s definitely in the conversation.

But reality is not that horrible. We should focus more on the good things technology brings to our lives. The mentioned podcast does exactly this.

Here are some takeaways:

• Achieving a sustainable society is not possible without technological advancements. There is no more powerful tool to tackle global challenges than ethical tech developments.

• Trust is inevitable. Distrust is the result of negative incidents, thanks to the lack of governance usually.

• Yes, AI can take away jobs, but at the same time, it will create some. ‘According to the World Economic Forum, 25% of jobs will be disrupted by AI and emerging technologies, with 75 million jobs disappearing. However, 133 new job titles will appear.’

No-code tools are all around us. Hundreds of new GPT applications and models are coming out every week. Are these powerful enough, to swipe out coding from the Data jobs?

BI Tools are becoming drag-and-drop applications, where a high schooler can create basic dashboards. Combining that with GPT models and some well-formed prompts, the output can be on the Junior Data Analyst level easily.

GPT can write simple SQL or Python code in seconds. If you tried writing code with AI, then I’m sure that you felt that it would easily take over coding soon.

Here are some reasons why:

• Interpretability: Writing code or creating dashboards are just a small part of Data jobs. Communicating the results, placing them into a business context, and making decisions based on those results are other. That’s something that we don’t want to give into AI’s hands in the near future. Even if coding will change, Analysts will stay at least for this reason.

• Maintenance: No code tools are great in the first 5 minutes. But as soon as something changes they will crash. With code, you can make them foolproof and easier to maintain.

• Flexibility: No-code tools are restricted. They cannot think outside the box. If you want to be creative, you need the flexibility that code can provide. To further strengthen this part, AI lives in the same box. It’s hard to step out using only AI.

• Data pipeline: The above-mentioned tools only provide solutions to some small parts of the data pipeline. They work really well with cleaned data that are ready to publish. But what about the previous steps? The data collection, cleaning, and preparation are still out there to solve and these are the harder tasks usually.

Conclusion

Even though coding is a big part of a Data job today, your future career will not depend on how writing code will change in the future!

Lifestyle tips 🧠

You're reading this newsletter because you aim to succeed, build a career in a promising field, and better yourself. Achieving these goals starts with taking care of your brain, ensuring it stays healthy and operates at full capacity. That's what we're focusing on in this issue. We'll cover practical tips on how to safeguard your brain health and keep your mental performance at its best.

Here are the 11 things Dr. Amen mentions in The Diary of a CEO podcast that can prematurely age your brain or damage it:

1. Drop out of school and stop engaging in new learning - This can cause your brain to disconnect itself due to a lack of new connections being formed.

2. Being in a job that does not require new learning - It's a risk factor for dementia due to a lack of mental stimulation.

3. Being lonely - Loneliness is a risk factor for dementia and is terrible for brain function.

4. Eating a lot of red meat if your iron and ferritin levels are high - Ferritin stores iron, which can damage the brain.

5. Increasing inflammation by never flossing and not caring for your teeth - This can lead to gum disease, which is associated with an increased risk for heart disease, depression, and dementia.

6. Avoiding fish in the diet - People who don’t eat fish tend to have less grey matter in their brains and may have lower levels of omega-3 fatty acids, which is associated with smaller brain size.

7. Eating a diet high in processed foods - This type of diet is linked to various health problems, including brain health issues.

8. Blaming everything on genes without adopting a prevention program - Even if you have a genetic predisposition to certain health issues, lifestyle choices play a significant role in their actual development.

9. Using substances such as marijuana, vaping, and alcohol - These can negatively impact the brain's development and increase the risk of various mental health issues.

10. Experiencing head trauma from activities like football, soccer, rugby, boxing, and not paying attention while walking (e.g., texting) - This can lead to brain injuries.

11. Feeding your brain "poison" during its development - This includes bad nutrition and substance use, which can damage the brain during a critical period of development.

Follow this advice to take care of your brain by avoiding these detrimental habits and engaging in healthier lifestyle choices.

This week we interviewed Marco Franzon. He is a Data Engineer and DevOps.

• How did your background in biochemistry and nanobiotechnology influence your approach to data science and DevOps in your current role at eXact lab?

I don’t have a direct benefit from the biochemistry knowledge field, as long as I am mostly a software developer at the moment. But my background has helped for sure to reach customers in bio-fields. The opportunity to share a common lingo gives me the possibility to identify the problem more quickly and tailor the best solution for them. (BTW, I had also an MD in Data Science, which is more related to my daily job)

• Transitioning from an academic focus in the life sciences to a career in data engineering and software development is quite a leap. What motivated this transition and how did you prepare for it?

I took an MD in Data Science and Scientific Computing in order to acquire all the necessary skills to fill the gap. During the courses, I had different job opportunities but I chose eXact lab because it looked at me as the perfect place to grow, learn and have fun building great stuff with super experienced people. The motivation of the transition is a growing interest but also an affinity to the programming/coding world. At a certain point, I worked in the lab during the day and followed tutorials on how to code during the night, so I decided that it was time to change.

• You've worked with a range of technologies, from Docker and Kubernetes to Python and R. For data scientists looking to broaden their technical skill set, which technologies would you recommend focusing on? What resources do you use or suggest to use to learn these skills?

In general, I love reading (well done) documentation. I strongly believe that the best way to learn something is to read the guides/tutorials/docs made by the authors of those technologies. This not only gives you the basics of the language/technology but also the way to think. These days we see the usage of generative AI to learn something new, in general, it is a good thing, but if you can choose between a good doc or a generated answer I strongly suggest the docs. On the other hand, it is fundamental to practise, 50% is theory, and 50% is practice. I used to learn theoretical and practical concepts in parallel to be sure to have a clear in mind what I was doing. 

As a data scientist, I strongly suggest learning the most used languages in this field, Python and R, then Docker to be sure that your results will be reproducible for everybody.  Databases is another fundamental set of skills, SQL and NoSQL, both are important and used in many different fields. I suggest understanding the infrastructure and not only how to make a query on them. In this way, you will be capable of making good design choices for your data when you have to store/organise it.

• DevOps is a key part of modern software development, especially in data-intensive applications. How do you see DevOps evolving in the context of AI and machine learning workflows?

DevOps is very important in the AI/ML world because it covers a wide range of aspects. The reproducibility problem, for example, DevOps technologies gives a data scientist the right environment to develop its models/algorithms, which can be reproduced by any other data scientist. Another aspect is the deployment in production. Thanks to DevOps, for example, the MLOps specialisation, it is possible to put in production the ML model without rewriting it from scratch, to be compliant with the development environment.

• As someone who has written for a scientific dissemination website, how important do you think communication skills are in the data science field, and what tips can you give for effectively communicating complex technical concepts?

A data scientist has to understand data and then explain them to others. So, communication skills are very important. I think that one of the most important skills is to make well-done graphics like diagrams, plots and so on. Let data tell the story. You have just to add some “glue” between what you are showing to tell a reasonable story but I think that the results should be clear to everybody without any long explanation.

A common exercise is to tell a complex concept to someone who is completely new to that topic, if it gets the point you are on the right track. 

• With your varied experience in both software development and scientific research, what advice would you give to data scientists aspiring to develop robust, scalable infrastructure for their data workflows?

1. Think simple. Don’t overcomplicate your code.

2. Use well-supported libraries or the core libraries.

3. Don’t look for “shortcuts”, be sure to understand the problem.

4. Test VERY WELL your solution.

5. Be sure to be reproducible, in terms of scientific results, but also in terms of environment and compatibility. 

• How do you approach the challenge of balancing rapid development and deployment with the need to ensure robust and secure operations in your DevOps role?

Having a good team is one of the most important things to be sure that the development and the operations work fine without losing time. At eXact lab, fortunately, we have a lot of experienced developers, not only in terms of technical skills, but also in team work, and this is crucial to move fast. So, the answer is having a good team gives you the opportunity to balance the development time and the DevOps activities. 

• In the realm of machine learning and AI, data is king. From your experience at eXact lab, what are some best practices for ensuring the quality and integrity of data in machine learning workflows?

Metadata. Data is just an amount of numbers or objects without metadata. Handling, saving in the proper way and in a proper format the metadata gives you the possibility to optimise the data usage and gain more value from them.

• The integration of machine learning models into web services, as you've done with FastAPI, is becoming more common. Could you share your insights on the importance of this integration and any potential pitfalls to avoid?

FastAPI is one of the most used frameworks to build REST API services. It is quite ready to use in the ML field with a lot of work done to make its usage simple for serving ML models. It is a good choice because it is easy to configure and scale. Obviously, it does not solve all the problems, for example, you have to be sure to monitor your model with a dedicated service (in terms of accuracy but also performance).

• Given your experience with Docker-compose and Kubernetes in creating scalable infrastructures, what would you say are the top considerations data scientists should keep in mind when they transition to a DevOps-oriented role?

Install Docker, try to wrap your model and make it a service. When you are familiar with this flow, move to something bigger like Kubernetes. Try to do the same but in a K8S cluster. Another time, when you have done you have seen enough to be considered a junior DevOps I think ;).

During the process, in my opinion, it is important to keep in mind that you should be able to reproduce what you are doing in every machine or environment, cloud or on-prem, it does not matter. This forces you to learn the best practices of DevOps and forget bad practices which bring you related to a certain Operating System or language/library version.

• In today’s digital age, platforms like Twitter are invaluable for professional networking and learning. Can you describe how your engagement on Twitter has influenced your career or provided opportunities for growth?

I really like to talk with people on Twitter to share thoughts and ideas about tech topics. Every day I learn something new, sometimes it is a new tool to simplify a task, and sometimes it is a tip or trick to make things faster in the code. At the moment it does not change my career, maybe one day, who knows ;)

• Open-source contributions on GitHub are a testament to a developer's work and collaboration in the community. Could you highlight one of your open-source projects on GitHub that you're particularly proud of and the impact it has had on the community or a specific audience?

I love open source, I started my journey thanks to open-source projects and tech communities and now it is time to give something back. I have some different open source projects on GitHub and I will be more than happy if other devs want to join me in the development. One of the most appreciated at the moment is the mojo-is-awesome repository which is a collection of public resources to start with Mojo programming language, a very promising language for AI, but also benchmarks to evaluate it with respect to Python. I also want to mention other two projects which are pytorch-in-public and docker-in-public in which I want to share my knowledge and experience in these fields with ML algorithms written from scratch and recipe for docker containers. 

• Finally, in the ever-evolving landscape of AI and machine learning, continuous learning is essential. How do you keep your skills up-to-date and what resources do you often turn to?

Continuous learning is one of the most important aspects for a developer. I am trying to stay up to date by reading news and Twitter is a good source of news. Once I find something interesting I have a look at the publication and/or the repository, if it is public or some good blog post. It is not easy to find good resources today due to the abnormal amount of “flash” posts which do not give you any value. In general, HackerNews is a good starting point.

Follow Marco on 𝕏

1. Create a God's Eye View for Learning: Establish a learning dashboard where you track your progress on various skills you're acquiring. This might include your study schedule, resources, progress indicators, and any deadlines for completing certain courses or certifications. This overview helps you see whether you're allocating enough time to your learning goals and allows you to adjust as needed.

2. Learning Checklists: Craft two succinct checklists; one for daily work/study duties, and another for your ongoing learning and skill enhancement. Begin your day by ticking off learning tasks such as doing some course lessons, reviewing a new research paper or practicing a programming exercise, then move to your work-related tasks. You could also schedule some learning tasks after finishing your responsibilities. This ensures a balanced progression in your current role and your personal development.

3. Strategic Learning Breaks: Recognize that intensive learning requires mental breaks. Use short breaks in your day for lighter educational tasks, such as listening to a podcast in your field or reviewing flashcards. Reserve longer, uninterrupted time blocks for deep focus sessions where you can engage in more complex study, like working through a challenging coding problem or learning a new language.

4. Inbox Zero for Learning: If you use email or online platforms to receive learning materials, apply the Inbox Zero method. Archive completed lessons, mark assignments or projects that need more time, and keep your inbox clean so you can quickly find what you need for your next study session.

5. Atomic 80/20 Rule for Skill Acquisition: Identify the 20% of learning activities that contribute to 80% of your skill advancement. Focus on those activities that yield the most learning. For instance, if practising speaking a new language is more beneficial than passive listening, allocate more time to conversation practice.

6. Intentional AI Learning Prompts: Use AI tools to enhance your learning. For example, you could create prompts to summarize complex material, generate practice questions, or even explain difficult concepts in simpler terms. Tailor the AI prompts to address your particular learning needs, thereby creating a personalized study assistant.

7. Recovery Forecast for Learning: Plan recovery time after intense learning sessions, workshops, or exams. This allows your brain to consolidate information and prevents burnout. Just as you would after a night out, schedule downtime after a heavy study day or learning event.

8. Capture and Organise Learning Tasks: Whenever a new learning opportunity or resource comes to your attention, immediately capture it in a digital tool or app. Later, organize these tasks into your learning schedule, deciding when you'll explore each one and how it fits into your overall learning plan. You can also use a physical media.

By adapting these time management tips to your learning and working schedule, you can create a balanced approach that accommodates continuous personal and professional development alongside your regular duties.

Do you like this content? Would you like to see more like this?

With your feedback, we can improve DSBoost. Click on a link to vote:

• 👍 Yes, I would love more like this

• 😐 Yes, I like it, but I prefer the previous content

• 👎 No, this doesn't appeal to me

Here you can see the original video that served as inspiration for these 8 tips:

Learning new skills is a key ability in your life. That’s why finding good courses is essential. There is a lot of noise on the internet, so we want to make it easy for you. As part of our launch offer, we are providing a 15% discount on Train In Data courses when you use the code DSBOOSTNL at checkout.*

• Explore feature engineering and feature selection methods

• Discover tactics for optimizing hyperparameters and addressing imbalanced data

• Master fundamental machine learning methods and their Python application

Enroll today and take the next step in mastering the world of data science!

Unlock the learning!

*Sponsored: by purchasing any of their courses you would also be supporting DSBoost.

OpenAI Devday

OpenAI held its first developer event on Monday.

Here are the 5 most important announcements:

1. GPT-4 Turbo: An improved version of the GPT-4 model was introduced, with a larger context window and the ability to understand both text and images.

2. Custom GPT Creation: Users can now build their own versions of GPT for various use cases without needing to code, and developers can connect these GPTs to external databases or knowledge bases.

3. GPT Store: There are plans to launch a store where users can publish their GPT creations, which will initially feature creations from verified builders.

4. Assistants API: This new API allows developers to build their own agent-like experiences that can retrieve outside knowledge or perform specific actions through programming functions.

5. DALL-E 3 API: The text-to-image model is now available through an API, which includes built-in moderation tools and allows for various output formats.

This week we interviewed Soledad Galli. She is a Data Scientist, Best-selling instructor and Top Data Science Voice on LinkedIn!

What inspired you to enter the field of data science, and how did you navigate your early learning journey?

One of the things I enjoyed the most while working as a research scientist in academia, were statistics and data analysis. Carrying out the experiments was daunting. But analysing the data… that I did love. That passion was what initially drew me to data science.

My journey into data science began when I noticed its increasing popularity on platforms like LinkedIn. I found some former colleagues who were now data scientists, so I reached out to them to ask how I could make the switch as well.

They recommended starting with online courses. I didn’t believe it was possible at first, but I enrolled in the courses anyways. I mean, what else can you do, right? You have to begin somewhere.

So I enrolled in a couple of online courses, including the Data Science Specialization on Coursera and the Analytics Edge on edX. These courses helped me take the first steps in data science. I then complemented my learning with hands-on data analysis projects, often sourced from the courses or from platforms like Kaggle. That gave me the practical experience to successfully navigate interview questions.

And, probably because of my academic background, I read relevant literature as well, including the works of Leo Breiman, a pioneer in the development of random forests.

Eventually, I secured my first job in the field.

How has your experience in finance and insurance shaped you as a data scientist?

My job at the finance company was also my first job as a data scientist. There was a steep learning curve for me. I needed to learn not just data science topics, but also, how to work in a corporate environment.

I learned that people consider you an expert, even when I did not feel like an expert myself. My colleagues were seeking my input and insight, while I was wondering why they should trust me. After all, I was a biologist, and my knowledge of credit risk was borderline zero.

So what I learned from that is that teamwork is what matters. Perhaps none of us were experts in the sense that academics consider us to be. But all of us had something to bring to the table. Including myself. And that was what mattered.

I also learned that data science in real life does not look like the projects you do in the online courses. The data comes from various sources, it's huge, and you need to tidy it yourself. And, discovering how things get done, professionally, it’s really hard.

Now, the online content has grown, people are sharing more. A few years back, the information was scarcer. It took me an incredible amount of time and search on the internet to find out how to preprocess data beyond “mean imputation” and “one hot encoding” for example. And I thought, wait a minute, all of us are spending an incredible amount of time on researching exactly the same things. Why not simply make available the product of the research that I had already done? So others can find it quicker.

That, and my gratitude to online education, that allowed me to step into data science, motivated me to create my first course on feature engineering. And since then, I came a long way.

Can you share a particular challenge you faced while developing and implementing machine learning models in a business setting, and how you overcame it?

I think the biggest challenge, or perhaps the biggest error many of us make, particularly on our first projects, is not understanding how the model will be used once it is developed. For a credit risk model, or an insurance model, that means not understanding which customers it will serve, what will be done with the outputs of the models, what data is available at the time the model is called, or who is going to interpret the results of those models.

When you start developing a model detached from that reality, you could end up using variables that are not available in production, training the model on data that is not representative of the customers it will serve, or creating models that are so complex that people can’t understand them and or use them.

The funny thing is, that you could have spent weeks or months developing the “perfect” model, but once you are deploying it, and you face these issues, you need to implement fixes quickly, and that affects the quality of the final product.

So, what I try to do, before even starting to think about what data will I need, is take time to sit with the various stakeholders, and understand how they will interact with the models. I talk to the users of the models, say the fraud investigators and risk analysis, to understand what they need from the model. I discuss with the software engineers to understand how the model will be integrated in production and which data will be available for the model to consume. In short, I try to anticipate as many pain points as possible, to minimise the quick fixes that we need to do while deploying. It won’t prevent all uncertainties. But some are better than none.

Your books and courses cover a range of intermediate topics in machine learning. What advice do you have for beginners in the field who are looking to advance their skills and knowledge?

I think the key is to never stop learning. Never stop questioning. You don’t need to know everything to land your first job. But as you go along, the more you know, the more resourceful you will be to solve problems and troubleshoot things when the results are not what you expect.

Concretely, that means, understanding what the machine learning models do, which model is suitable for which scenario, what are the trade-offs of each model, say interpretability vs performance, or speed vs complexity.

The other thing that was key for me to learn is, for everything that I want to code, someone has coded it already and it is available in an open source library. I spent hours writing code that you could do in one line of pandas.

The open-source ecosystem for machine learning is growing at incredible speed. So, I think it’s worth taking some time to look for what it’s out there because that will save you a lot of time.

Can you tell us a bit about “Train In Data”?

Train in Data is an online education school that I created at the back of the popularity of my courses on feature engineering and feature selection. With our courses, we try to breach the gap between beginner data science courses and what you need as a data science practitioner.

We teach the theory of the subject at hand, how to implement the methods or algorithms in Python, and we provide real-life examples and advice regarding when you should choose one methodology over the other.

The idea is to save data scientists the time they would spend if they were to do the research themselves, and at the same time show simple, elegant and efficient Python implementations that they can adapt and re-use in their projects.

We are excited to announce a collaboration between DSBoost and Train In Data! As part of our launch offer, we are providing a 15% discount on Train In Data courses when you use the code DSBOOSTNL at checkout. Not only will you benefit from the valuable insights and practical knowledge shared in these courses, but you will also be supporting DSBoost for each course purchased through this offer. Don't miss out on this opportunity to enhance your data science skills. Enroll now and take the next step in your data science journey!

Explore the courses!

Feature-engine is a highly successful open-source project with a large community of contributors. What have been some of the key lessons learned from developing and maintaining this project?

The first and most important lesson is that creating and maintaining Feature-engine was and continues to be a lot of fun and super rewarding.

I can’t explain what it feels like to see a project grow and learn how people are using it, or recommending it to others.

The second lesson is, you don’t need to be a top notch programmer to create, maintain or contribute to open source. I started Feature-engine with the motivation to become a better developer myself. That tells you a lot about what I thought of my Python skills when I started the project. It has worked wonders for me.

So in summary, I’d say, contributing to open-source can be fun, rewarding and help you become a better programmer. So I’d recommend not shying away from making that first contribution to any library you like. The developers will be more than happy.

How do you balance your roles as an instructor, author, and developer, and what drives you to continue contributing to the field in such a multifaceted way?

To be honest I just go with the flow. There are times when Feature-engine gets a lot of contributions, so I need to put more time into reviewing the pull requests. When Scikit-learn or pandas make a new release that breaks backward compatibility, and that happens more often than you’d think, I need to leave anything else that I am doing to make Feature-engine compatible with the latest versions. Otherwise, I focus mostly on creating courses and books.

The motivation comes from the continuous reward that I receive from the people who take my courses or read my books. We get amazing reviews on Trustpilot, or they reach out over LinkedIn or Twitter to say that the courses are wonderful, or they changed their lives, or have helped them to succeed at an interview. And many reach out as well with suggestions for new courses.

Can you discuss a particular moment or achievement in your career that has been especially meaningful or rewarding for you?

One of the moments marked my career as a data scientist were those 15 minutes after my first talk at a tech meeting. It was at Pydata, in London in 2017. I had barely over a year as a data scientist, hardly seeing myself as an expert in the field. I was encouraged to present our credit risk model, and so I did. And the result was amazing. I got so many questions at the end of the talk, and people kept coming throughout the meeting to ask more and more. They were genuinely interested, which reinforced my idea that I should get out more and spread the knowledge.

What trends or developments in data science and artificial intelligence are you most excited about, and how do you see them shaping the future of the field?

I am looking forward to the new and coming regulations on how corporations can use machine learning and what are their responsibilities to the users. Artificial intelligence has, without question, a lot of potential to solve complex and important problems. And we know as well that it has caused tremendous damage, through for example, the spread of misinformation, or through biases that more severely affect the most vulnerable sectors of society. If you haven’t heard about this, I invite you to read “Weapons of Math Destruction” by Cathy O-Neil, which provides a nice yet thorough summary.

I hope that this will force corporations to put the interest of users ahead of the interest of profit, and make them create products that are useful for society and not just for their pockets.

Follow Sole on 𝕏

This week we interviewed Patrick and Janik Tinz, also known as the Tinz Twins.

Studies & Choice

Can you describe your academic background and what led you to choose that particular field of study?

Yes, of course. But first, we'd like to welcome the audience. Hey everyone, we're the Tinz Twins, and our academic journey has been the same. For this reason, please don't be surprised that we always speak of "we". At school, we were very interested in mathematics and physics. In addition, we were also excited about the technological progress of the big tech companies. So we decided to do a Bachelor in Computer Science. During our studies, we learned a lot about software engineering and databases. But very little about machine learning and AI. During our Bachelor's studies, we worked as an assistant in the Mathematics Department. During this time, we had contact with numerous Master's students from the Data Science Master's program. After some discussions, we decided that a Master's in Data Science was the right step for us. We finally completed our master's degree on a part-time basis with the company Accso. Accso is a software consulting company. At Accso, we were able to work on some exciting data science and software projects with a focus on industrial clients. It was a great time. Through our master's degree, we are officially allowed to use the title "Data Scientist". After a year of working full-time at Accso, we decided to start our own business, "Tinz Twins". At Tinz Twins, we focus on Data Science in the financial sector. In addition to our self-employment, we are still studying for an MBA in Finance at the University of Applied Sciences - Burgenland (Austria).

Reflection & Regret

Looking back on your educational journey, is there anything you would change or do differently if given the chance?

No, definitely not. We are very grateful for all the experiences we have had. In our bachelor studies, and at Accso we learned basic software engineering know-how. Finally, in our Master's, we learned analysis skills so that today we can design AI-based applications from scratch. We're just thankful for all the great people we've met along the way so far.

Future Orientation

Where do you see yourself in the next 5 years career-wise, and what motivates that vision?

That's a great question. We want to make Tinz Twins a strong brand in the Data Science for Finance field over the next five years. We are currently working on several projects for this. On the one hand, we have our international business line. We teach data science and finance skills in particular. Our platforms for this are X (formerly Twitter) and Tinz Twins Hub. On Tinz Twins Hub, we publish every week exciting articles about Data Science topics. On X, we post educational content about Data Science in an accessible way. Furthermore, we run the Medium publication Towards Finance. The publication aims to build a global investment research community. We want to promote the knowledge exchange between software engineers, data scientists, financial analysts and financial experts. In our opinion, Towards Finance is the place to learn all about Finance, Investment Research, and Artificial Intelligence in the financial sector. On the other hand, we have our German-speaking business. In Germany, we launched a YouTube channel on finance. We also run a weekly e-mail newsletter about finance news. We call this project Tinz Twins Invest. Tinz Twins Invest should be a platform where the community can exchange information about investment topics.

Data Science & Its Trajectory

How do you envision the future of data science, especially with the rapid advancements in AI and machine learning?

AI should improve existing products and workflows. We don't believe that we need an Artificial General Intelligence (AGI). We need specific AIs that add value in specific domains. Yes, ChatGPT is a great tool to solve rudimentary tasks. But no more than that. It doesn't work well when we want to solve specific tasks. It doesn't help to use retrieval-augmented generation (RAG) either. At some point general models reach their limits, so we think we should rather use fine-tuned models for specific tasks. Apple and Meta go this way. Both are trying to make their products better through AI. That's the right way to go, in our opinion.

Career Alignment with Data Science

Given the current trends in data science, how do you plan to align your career with these developments?

The performance of today's large language models (LLM) is impressive. In the future, we'd like to focus more on LLMs in the financial sector. We think that LLMs have the potential to change many industries from the ground up. For this reason, we have been planning to develop an LLM-powered investment research app for some time. The app is currently under development. If you want to stay informed about this project, follow us on our channels.

Data Science Impact on Finance

In your opinion, how might the evolution of data science affect the finance sector in the coming years?

In our view, LLMs will play a big role in the financial industry, the so called FinLLMs. Examples are BloombergGPT (a proprietary LLM for finance developed by Bloomberg) or FinGPT (developed by the open-source community AI4Finance Foundation). We think the future of FinLLMs and financial platforms should be open-source. A great project in our eyes is OpenBB. OpenBB is an open-source investment research software platform. The platform gives you access to high-quality financial market data and analytical tools. OpenBB's goal is to make investment research accessible to everyone. In addition, OpenBB is already beginning to integrate LLM-powered features into its platform. OpenBB Terminal is free. A license for a Bloomberg Terminal costs $24,000 per year.

Personal Adaptation

Considering potential shifts in data science that could impact finance, how do you plan to adapt or evolve in your role?

In the future, we want to focus primarily on FinLLMs. We also want to educate ourselves further in the area of finance. Our vision is to make Data Science for Finance accessible to everyone. That's also the reason why we decided to do an MBA in Finance. We also love to learn new things and invite everyone to become a Towards Finance writer. On Towards Finance, we want to share our knowledge about finance and data science with everyone in the future. Let’s bring together the know-how of all the experts in these fields because we can achieve great things together.

Skills & Knowledge

What new skills or areas of knowledge do you believe will be crucial for professionals in the intersection of data science and finance?

In the financial industry, many products are proprietary. In the future, it is essential to take a look at open-source products. It happens a lot in the field of open finance. Soon, Bloomberg's high prices will no longer be justified as there will be cheaper and open-source alternatives. The use of LLMs in the financial industry will keep many professionals busy. How can we use LLMs efficiently in finance apps? How can we fine-tune LLMs for financial use cases? It is an interesting and exciting topic for the coming years. The democratization of AI and finance must go hand in hand!

Career Advice

For someone looking to enter the world of data science with an interest in finance, what advice would you offer?

It is essential to have an understanding of time series analysis. In addition, you should learn how time series algorithms (RNNs, LSTMs, GRUs, ARIMA, ...) work. If you need data for exploration, we recommend using the OpenBB SDK. We have learned one thing in the last few years. It is super important to understand the specific domain. A good data scientist has domain-specific knowledge, which allows him to generate added value from data. A deep understanding of the task is essential, and you should keep the task as small as possible. And you'll learn the most by working with real-world data. Also, you don't necessarily need the largest ML model to solve tasks. Simple models are often sufficient to achieve good results.

Interdisciplinary Impact

How do you think data science could bring about interdisciplinary collaborations, especially in areas like finance, and what benefits do you foresee from such collaborations?

Thanks for this interesting question! An exchange with financial experts is essential to use data science effectively in the financial sector. We think it's important to exchange ideas with others from different domains. For this reason, we want to promote this exchange through our Medium publication Towards Finance. Only together can we achieve great things. Data science, financial knowledge, and technological progress must go hand in hand. That's how we can make the financial industry more transparent, accessible, and efficient.

Follow them on 𝕏

Hi, this is Pawel Jastrzebski. In my day job I’m a data analyst but my online presence  (@pawjast on X & Medium) mostly revolves around data visualization. 

Today I’ll show you one of the graphs I’ve created recently and explain some of the design choices I’ve made. The data for the visualization you’ll see in a moment is a very simple one and comes from the following tweet:

https://x.com/stats_feed/status/1709139000257393085?s=20

The World of Statistics often plots very simple stats like the one above. And as much as I enjoy them, every time I see a wall of numbers, I’m thinking:

And this one time I decided to plot the data and see it’s really better looking when visualized. This is what I got:

This plot was created in Python and Matplotlib only. At the bottom of this post, you’ll find a link to a full code.

It started off as a simple plot with dots along the x-axis denoting a founding year and a label next to it. But it was ugly. The labels were jam-packed and staggered on top of each other. 

So I moved the labels to the y-axis. This gave me a similar effect to the horizontal bar chart. Except there are dots instead of bars. I’ve also added a mild grey shading underneath.

I think it’s a nice touch as it looks a bit like a mountain slope. Every manufacturer (every new dot) is like another milestone that the automotive business has collectively achieved (by growing bigger and having more players in the game). 

I was quite happy with the result as it was but I wanted to add more depth to the data. Without spending too much time, it was easy to expand the original data by adding the continent the manufacturer was founded in.

At this point, it could have been any other feature: current market value, current market share, annual volume of cars, etc. But I’ve very generically limited myself to the continent of origin.

Once I’ve added the continent, I color-coded the dots accordingly and added the legend. The graph had more information now but it became cluttered and difficult to read again. 

Therefore, I’ve decided to add the bottom part where we can see a clear split between the continents. Also, this is the plot where I provide one more bit of information by using the country code to indicate the origin place for a given manufacturer.

At this point, I felt the graph design was complete and all that was left to do was visual touches (choosing fonts/colors/hiding spines/grids, etc.). 

From the graph you can clearly see that:

• The majority of the automotive industry was developed in the first half of the XX century.

• It started in Europe.

• Asia started developing their own cars well after Europe and the US.

After publication, I had another idea of how you can add some more context to this modest dataset.

I’ve decided to add a bar chart next to the main graph. The bar chart shows the time difference between the current manufacturer (one per bar) and the first established company. Alternatively, this could have been replaced by the time difference between the current manufacturer’s and the previous manufacturer’s founding year. 

From the programmatic side:

• The data was stored as a Pandas DataFrame

• After I’ve added the continent information, I used Pandas’ groupby() function to split the data. This helped me with assigning respective colors to each continent

• To plot the dots, I’ve used Matplotlib’s ax.plot() function as it does the job well and there was no need to resort to the ax.scatter()

• The sidebar chart in the extended version of the plot was created using ax.barh() function 

• The last element of the plot was lines connecting the dots with the labels. They were plotted using ax.vlines() and ax.hlines()

All this was bundled together in a subplot using Matplotlib’s fairly recent function:

• plt.subplot_mosaic()

This function makes it very easy to create a grid layout.

The raw structure for my plot was coded with the following lines:

And it produced the following grid:

And that’s it for today.

You can explore the full code here: 

https://github.com/pawjast/vizarts/blob/main/src/dataviz_automotive_manufacturers_founding_year%20v1.ipynb

📑 Recommended Content

Pawel has a list on X with people who create amazing visuals.

You can follow it here:

https://twitter.com/i/lists/1702600998861013394?s=20

If we talk about visuals why don’t you check this roadmap that includes how to create such charts as well?

https://x.com/levikul09/status/1714207652761260463?s=20

Here are some takeaways:

• ML Engineers > ML Ops > Data Scientist. That is the ranking in salaries for all seniority levels.

• ML positions have a higher demand for Ph.D. holders. (27% vs 23%)

• In some ML positions you will find C and Java while in DS positions SQL. However, Python is still king:

Skills

• DS positions focus on statistics and data visualization, while ML jobs are more towards deep learning skills, especially in tools like PyTorch and TensorFlow.

• Communication is a requirement in 3/4 of the positions. This fact also shows that these positions are usually cross-functional and interaction with others is part of the everyday tasks.

The goal of data roles is to get insights and make better decisions.

Effective communication ensures that insights and findings are clearly understood by coworkers or stakeholders.

Educating others about the data process and eli5 methodologies can also be a task. With efficient communication, you can make others feel that they understand the data as well.

If you are a good Data Scientist or ML engineer but bad in communication you will be lost in this field.

This thread may help you to be better at it:

https://twitter.com/levikul09/status/1624713693270540294

• If you are into MLOps focus on data processing technologies like Spark, Databricks, and Snowflake!

• Regarding cloud AWS dominates. But it is more important for ML jobs than DS positions.

• Regarding visualization tools Tableau and Power BI are the leading technologies. Nothing surprising here.

Bad news for juniors

Again, this was an extract from this interesting post by jobs-in-data.com

Make sure to check out the original article! It is an awesome read!

Featured Content 🧵

https://x.com/freest_man/status/1709573626297831844?s=20

Learn ML without libraries: