Rich Sutton

March 13, 2019

The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation. These two need not run counter to each other, but in practice they tend to. Time spent on one is time not spent on the other. There are psychological commitments to investment in one approach or the other. And the human-knowledge approach tends to complicate methods in ways that make them less suited to taking advantage of general methods leveraging computation. There were many examples of AI researchers' belated learning of this bitter lesson, and it is instructive to review some of the most prominent.

In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess. When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers. They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. These researchers wanted methods based on human input to win and were disappointed when they did not.

A similar pattern of research progress was seen in computer Go, only delayed by a further 20 years. Enormous initial efforts went into avoiding search by taking advantage of human knowledge, or of the special features of the game, but all those efforts proved irrelevant, or worse, once search was applied effectively at scale. Also important was the use of learning by self play to learn a value function (as it was in many other games and even in chess, although learning did not play a big role in the 1997 program that first beat a world champion). Learning by self play, and learning in general, is like search in that it enables massive computation to be brought to bear. Search and learning are the two most important classes of techniques for utilizing massive amounts of computation in AI research. In computer Go, as in computer chess, researchers' initial effort was directed towards utilizing human understanding (so that less search was needed) and only much later was much greater success had by embracing search and learning.

In speech recognition, there was an early competition, sponsored by DARPA, in the 1970s. Entrants included a host of special methods that took advantage of human knowledge---knowledge of words, of phonemes, of the human vocal tract, etc. On the other side were newer methods that were more statistical in nature and did much more computation, based on hidden Markov models (HMMs). Again, the statistical methods won out over the human-knowledge-based methods. This led to a major change in all of natural language processing, gradually over decades, where statistics and computation came to dominate the field. The recent rise of deep learning in speech recognition is the most recent step in this consistent direction. Deep learning methods rely even less on human knowledge, and use even more computation, together with learning on huge training sets, to produce dramatically better speech recognition systems. As in the games, researchers always tried to make systems that worked the way the researchers thought their own minds worked---they tried to put that knowledge in their systems---but it proved ultimately counterproductive, and a colossal waste of researcher's time, when, through Moore's law, massive computation became available and a means was found to put it to good use.

In computer vision, there has been a similar pattern. Early methods conceived of vision as searching for edges, or generalized cylinders, or in terms of SIFT features. But today all this is discarded. Modern deep-learning neural networks use only the notions of convolution and certain kinds of invariances, and perform much better.

This is a big lesson. As a field, we still have not thoroughly learned it, as we are continuing to make the same kind of mistakes. To see this, and to effectively resist it, we have to understand the appeal of these mistakes. We have to learn the bitter lesson that building in how we think we think does not work in the long run. The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.

One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries. All these are part of the arbitrary, intrinsically-complex, outside world. They are not what should be built in, as their complexity is endless; instead we should build in only the meta-methods that can find and capture this arbitrary complexity. Essential to these methods is that they can find good approximations, but the search for them should be by our methods, not by us. We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done.

References

• The Bitter Lesson Blog Post

Below 13 key points to understand:

1. Exemptions, reversals, deals to come. It’s already happened with semi-conductors (exempted), Canada (USMCA) and Mexico (USMCA). Moreover, Trump has invited revised terms with his Rose Garden speech predicting “foreign presidents, prime ministers, kings, queens, ambassadors and everyone else who will soon be calling to ask for exemptions from these tariffs."

2. Trump’s tariff calculation not linked to actual tariff rates. Previously, most US trade partners had similar tariff rates as what the US charged them. Before the announcement, the US previously charged an average trade-weighted 2.2% tariff on (most favoured nation) imports, the European Union charged 2.7% and Vietnam charged 5.1%. Yet in its revised tariff rates, the US has used a formula linked to bringing bilateral trade surpluses with the US down to zero rather than using other countries’ tariff rates. Leaving aside the accuracy of the formula, it used incorrect elasticities, it has resulted in a 20% tariff on the EU and 46% tariff on Vietnam.

3. Ending US trade deficit will be painful. If the aim, then, is to bring the US trade balance from deficit to zero, then other countries cannot just offer to lower tariffs, but they have to strengthen currencies, stimulate domestic demand to enable them to buy more US goods and export much less. The flip side is that the US will need to import less and have a weaker currency. Historically, the fastest way to do this is a US recession. This begs the question of whether a trade partner needs to get rid of tariffs or the much harder task of bringing their trade surplus with the US to zero. Indeed, the last major US current account reversals were the GFC crisis of 2008 and the US recession of 1990-91.

4. New personality-led rather than rules-based trading system. The invitation to negotiate and poor methodology invites an emerging market style system of economic policy. Decisions could be made more on short-term political grounds rather than economic factors. For example, domestic sectors could get exemptions if they make political donations, promise to freeze prices (price controls!) or exit their DEI policies. Trading partners may get a deal if they shift their political biases. This invites a spoils system like the 1800s or modern corporate capture.

5. Trade uncertainty rather than tariffs leads to recession. While the scale of the tariffs is unprecedented in modern times, the US economy could likely manage it, rather it’s the unpredictability of future changes that could paralyse investment and consumer purchases. This uncertainty could lead to a recession, at the very least in manufacturing.

6. Trading arbitrage galore. Ireland has a 20% tariff, while Northern Ireland has a 10% tariff. China has a fully loaded additional 54% tariff, while trading ally Brazil has 10%. France has a 20% tariff, while its Caribbean outpost Guadeloupe has a 10% tariff. There is much scope of trade arbs.

7. Incentive for other countries to escalate. The aggressive tactics of Canada in response to earlier tariff threats and subsequent softening of US tariffs suggest tougher retaliation works. China has already fired that salvo with its aggressive actions on Friday. Trump has already showed signs of compromise by delaying its TikTok decision. Germany and France want to do the same and have discussed accelerating their investigation into X (possible $1bn fine) and imposing tariffs on US services.

8. US was winning in previous trading system. Despite the latest rhetoric, the US had created the world trading system to its advantage. The US had forced other countries to open up to US capital and trade. And despite, not running current account surpluses like Germany or Japan, the US has had higher productivity and higher living standards than both. Indeed, in 2000, US GDP per capita (PPP) was $55,000, Germany’s was $51,000 and Japan’s was $39,000. Today, US GDP per capita is $77,500, Germany’s is $63,000 and Japan’s is $47,500. The US has been the winner. If there have been weaknesses in the US order, it has been US income equality and the dramatic rise China and its associated over-production.

9. Tariffs are big tax hike on the poor. On income equality, tariffs hit the poor the most, they consume more of their income with tariffed products. In effect, the US has hiked taxes on the poor.

10. China could still gain. Most countries would have joined the US in trying to contain China’s insatiable production machine. In fact, Trump 1.0 re-framed US’s relationship with China and led to rare bipartisan support for containing China. Back then, Trump initiated China and sector tariffs, and then Biden continued them and added a domestic industrialisation component (CHIPs Act). Now, Trump has ripped this up. The administration has ended the bipartisan policy and by targeting all countries, the US has thrown a lifeline to China in it being able to form new trade alliances against the US.

11. Reindustrialisation won’t happen. One of the biggest supposed reasons for the tariffs is rebuilding US manufacturing. This is loosely modelled on the Asia experience where countries from China to Korea to Japan have used tariffs to build up their manufacturing bases. But these came with a long-term vision, strong bureaucracies (Japan’s MITI), a subordination of the banking system to manufacturing and a focus on competitive export markets. The US has not laid a vision with any of these. Instead, it is following the South American model of populist import substitution much like Peron (rather than Milei) in Argentina.

12. Everyone missing the long-term costs of AI boom. Much has been made about how previous administrations were too beholden to free trade that cost US workers their jobs. The driving force of this in the 1990s were US businesses that wanted to out-source to lower costs. Then we had the financial sector lobbying for de-regulation which helped spurred the housing bubble. In both cases, it was left to the government to deal with the costs (lost jobs and banking crises). Today the new shock is AI, and while the US administration is focusing on fall-out from earlier shocks, it is missing the potential societal costs of the AI shock. Indeed, recent submissions by tech companies demanded the US help invest in energy and data centres. There was no mention on how to internalise the costs of future job losses.

13. The real culprit is free capital. All the attention has been on trade flows, yet financial flows are massive and since the 1970s have dominated. The post-war boom of the US was an environment of constrained financial and capital flows. Today, unfettered capital flows create as many imbalances as trade flows do. Indeed, the track record of corporate tax cuts shows as much with corporates more incentivised to do stock buybacks rather than capex. The real disruptive policy move would be to constrain financial and capital flows and align them with long-term investment.

A good night’s sleep can feel life-changing, so imagine if sleep science could improve your slumber on a long-term, fundamental level. At Six Senses Douro Valley (and branches in Ibiza and the Maldives) you can opt into a Sleep Wellness Programme designed by renowned sleep expert Dr. Michael J Breus, which uses sleep-tracking analysis, as well as charting movement and oxygen levels to hack your way to a better snooze. See more at Six Senses.

2. Get fit in Dubai (UAE)

Styled as the ultimate hotel for an active lifestyle, SIRO One Za’abeel has been designed in collaboration with world-class athletes including British-Somali boxer Ramla Ali and the AC Milan football club. That means alongside the expected Dubai-level glamour you’ll also find a 2,000m² fitness and recovery centre complete with scientifically backed treatments. These include cryotherapy—exposing the body to sub-zero temperatures to activate collagen production and boost endorphins—and the MLX i3Dome, a detoxifying treatment that can help improve your immune function, mood, and blood PH levels. See more at Siro Hotels Dubai.

3. Be soothed by the sea in Benicàssim (Spain)

Simply being near the sea can feel therapeutic—just ask the Victorians who prescribed 'taking the sea air' for convalescent therapy—but the water itself also has medicinal qualities. Thalassotherapy means the use of sea water and marine products as a therapeutic treatment, and at Palasiet Thalasso Clinic & Hotel in Benicàssim, it’s used to improve rheumatic conditions such as back pain, as well as baths to help you de-stress, ease fatigue, and improve circulation. See more at Palasiet.

4. Follow the science in Santorini (Greece)

Alongside picture-postcard views of blue-domed churches and soaring, sea-kissed cliffs, Santorini is also no stranger to chic hotels. At Katikies Kirini in Oia, alongside caldera views from balcony plunge pools, you’ll find a branch of the ZOE Bio Regenerative Wellness Clinic. It’s a science-led space offering one- and three-day programmes that aim to help you replenish essential vitamins, detoxify your body, and improve circulation through a series of treatments including blood analysis, IV therapy, and diagnostic scans. See more at Katakies.

5. Let Nature Lead in Transylvania (Romania)

If the concept of biohacking is all about getting back in tune with our bodies to supercharge natural processes, where better to do it than in nature itself? Nestled in the leafy Transylvanian countryside and amid towering trees and sun-dappled fields, Akasha Wellness Retreat near Brasov offers an organic paleo and biohacking programme that merges science, nature, and diet to activate dormant potential in the body. Conceived in collaboration with doctors and nutritionists, the experience uses techniques including guided meditations, intermittent fasting, vagus nerve stimulation, and sound healing to promote longevity. See more at Akasha Retreat.

6. Be led by biomarkers in Bay of Kotor (Montenegro)

Located in the glamorous One&Only Portonovi resort in Montenegro’s near Podgorica UNESCO-protected Bay of Kotor, the Chenot Espace describes itself as a medical spa in a holiday setting. Don’t worry—this doesn’t mean scary invasive procedures. Instead, guests can opt for a number of wellness concepts that use diagnostic tests and biomarkers alongside Chinese Medicine principles to create bespoke programmes. The two-day deluxe spa weekend includes a medical consultation, energetic massage, hydro-aromatherapy treatment, a phyto-mud body wrap enriched with microalgae, and a carefully curated plant-based diet—all designed to detoxify, revitalise, and eliminate tension. See more One and Only Bay of Kator.

7. Go Supersonic in Baden-Baden (Germany)

While the term biohacking is relatively new, the theory is well-established—the Romans, after all, harnessed the medicinal qualities of thermal waters across their empire to help wounded soldiers recuperate. At Maison Messmer in the German spa town of Baden-Baden, bathing in those mineral-rich spring waters can also be combined with more modern practices. Try an anti-ageing ultrasound treatment that exposes facial skin to sonic waves to encourage active ingredients to penetrate into deeper dermal layers, which allows the skin to absorb and store more moisture. See more at Maison Messmer Baden Baden.

8. Seek out the scent of happiness in Bari (Italy)

We all know that scents are powerful triggers, with certain fragrances transporting you to a particular place or time in an instant. But did you know that scientific research has also shown synergies between our olfactory systems and neurophysiological effects? At the Vair Spa at the gorgeous Borgo Egnazia hotel in Puglia, you can embark on a scent-filled adventure into happiness designed by aromatherapy expert Dr. Luca Fortuna. It uses essential oils as a sensorial key to understand your own wellbeing and happiness. Once you've completed your initial olfactory journey you can even enjoy a massage using your customised blend of oils. See more at Vair Spa.

9. Channel your inner supermodel in Cotswolds (UK)

   

While biohacking focuses more on promoting longevity and health than appearances, most things that improve your inner health will also have a positive impact on your external wellbeing. With that in mind, The Lakes By Yoo in the leafy Cotswolds, 90 minutes from London, has teamed up with Kate Moss—fashion icon and founder of the COSMOS Skincare brand—to curate a spa experience designed to help you switch off and reconnect with yourself. In true supermodel fashion, though, it’s the pampering you’ll gain; the experience starts with an invigorating body brush and bracing ice bath before being cocooned in an infrared bubble to aid detoxification and muscle recovery, and a facial massage to restore balance. See more at The Lakes by Yoo.

10. (Bio) Hack like Vikings in Stockholm (Sweden)

HEAT was founded by Sophia Lie in 2018 as Scandinavia´s first full spectrum infrared sauna lounge. Located in the centre of downtown Stockholm, HEAT offers private, full spectrum infrared sauna, compression therapy, LED light therapy and LPG massage. Here at HEAT, they have developed a transformative wellness experience grounded in three vital pillars: sweat, sculpt, recover. Working together, biohacking treatments help your body detoxify, optimize, and heal at the cellular level – to leave you glowing from the inside-out.  See more at HEAT by Sophia Lie.

Creating simple Retrieval-Augmented Generation (RAG) chatbots using LangChain, Promptflow, and Semantic Kernel involves different approaches and tools. Here's a comparison and overview of how to create RAG chatbots with each framework.

LangChain

LangChain is a framework that helps build applications with large language models. It provides tools to manage prompts, chain LLMs together, and integrate with external data sources for retrieval.

Figure - LangChain Overview

Steps to Create a RAG Chatbot with LangChain:

1. Setup Environment: Install the necessary libraries

   pip install langchain openai faiss

2. Initialize LangChain Components:

   • Define a retriever using FAISS (or another vector store).

   • Create a language model instance (e.g., OpenAI's GPT-3).

   • Chain them together.

3. Example Code:

   from langchain import OpenAI, VectorDBQA
   from langchain.vectorstores import FAISS
   from langchain.embeddings.openai import OpenAIEmbeddings
   
   # Initialize the retriever
   embeddings = OpenAIEmbeddings()
   vector_store = FAISS.from_documents(documents, embeddings)
   retriever = VectorDBQA(vector_store)
   
   # Initialize the language model
   llm = OpenAI(api_key="your_openai_api_key")
   
   # Create the chain
   chain = retriever | llm
   
   # Define a query
   query = "What is the capital of France?"
   
   # Get the response
   response = chain.run(query)
   print(response)

Promptflow

Promptflow is a platform for managing, deploying, and iterating on LLM prompts. It allows for easy integration with various models and APIs.

Figure - Promptflow Overview

Steps to Create a RAG Chatbot with Promptflow:

1. Setup Environment: Install Promptflow SDK.

pip install promptflow

1. Define Prompts and Workflows:

   • Create a retrieval component.

   • Define the generation prompt.

   • Set up the workflow to chain them together.

2. Example Code:

import requests
import promptflow as pf
from promptflow.components import OpenAI, VectorSearch
from promptflow.text_splitter import RecursiveCharacterTextSplitter

# Fetch the document content from the URL
url = "https://example.com/document"
response = requests.get(url)
document_text = response.text

# Split the document text into chunks
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = text_splitter.split_text(document_text)

# Initialize vector search with chunks
vector_search = VectorSearch.from_texts(chunks, model_name="openai_embedding_model")

# Initialize text generator
generator = OpenAI(api_key="your_openai_api_key")

# Define workflow
workflow = pf.Workflow(
    steps=[
        vector_search,
        generator
    ]
)

# Define a query
query = "What is the capital of France?"

# Run the workflow
response = workflow.run(query)
print(response)

Semantic Kernel

Semantic Kernel is a lightweight framework designed to handle semantic search and text generation. It emphasizes modularity and ease of use.

Figure - Semantic Kernel

Steps to Create a RAG Chatbot with Semantic Kernel:

1. Setup Environment: Install Semantic Kernel.

   pip install semantic-kernel

2. Initialize Semantic Kernel Components:

   • Set up a semantic search engine.

   • Integrate with a text generation model.

3. Example Code:

   import requests
   from semantic_kernel import SemanticKernel, SearchEngine, TextGenerator
   from semantic_kernel.text_splitter import RecursiveCharacterTextSplitter
   
   # Fetch the document content from the URL
   url = "https://example.com/document"
   response = requests.get(url)
   document_text = response.text
   
   # Split the document text into chunks
   text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
   chunks = text_splitter.split_text(document_text)
   
   # Initialize search engine with chunks
   search_engine = SearchEngine.from_texts(chunks, model_name="openai_embedding_model")
   
   # Initialize text generator
   text_generator = TextGenerator(api_key="your_api_key")
   
   # Create semantic kernel
   kernel = SemanticKernel(search_engine, text_generator)
   
   # Define a query
   query = "What is the capital of France?"
   
   # Get the response
   response = kernel.query(query)
   print(response)

Comparison

1. LangChain:

   • Strengths: Highly modular, easy to integrate with various LLMs and vector stores, extensive chaining capabilities.

   • Use Cases: Complex applications requiring multiple LLMs, custom chains, and flexible integrations.

2. Promptflow:

   • Strengths: Simplified prompt management, easy iteration on prompts, integrated workflow management.

   • Use Cases: Applications needing rapid prototyping and iteration on prompts, deployment of prompt workflows.

3. Semantic Kernel:

   • Strengths: Lightweight, focused on semantic search and generation, easy to set up.

   • Use Cases: Simple applications needing semantic search and text generation without extensive customization.

Summary

Each framework has its strengths and is suited to different types of RAG chatbot applications. LangChain offers the most flexibility and power for complex integrations, Promptflow excels in prompt management and workflow iteration, and Semantic Kernel provides a straightforward approach for simpler use cases. Choose the one that best fits the complexity and requirements of your project.

Machine learning and data science models need to be continuously improved as the input data to the model changes, models may become suboptimal. Two powerful ensemble learning techniques, boosting and bagging, have emerged as effective methods for achieving this. These techniques combine multiple models to produce a more accurate and stable prediction than any individual model could. This article explores what boosting and bagging are, how they work, and provides examples from various domains to illustrate their application.

What are Boosting and Bagging?

Bagging (Bootstrap Aggregating) is designed to reduce the variance of a model by training multiple models on different subsets of the training data and then averaging their predictions. This technique is particularly useful when dealing with high-variance models, such as decision trees.

Boosting aims to reduce both bias and variance by sequentially training models. Each model in the sequence focuses on correcting the errors made by the previous models. Common boosting algorithms include AdaBoost, Gradient Boosting, and XGBoost.

Figure - Illustrating Bagging vs Boosting

Credits: Flaticon.com

Key Parameters in Bagging and Boosting

Both bagging and boosting involve several parameters that control their behavior:

• base_estimator: The base model used to create multiple instances (e.g., DecisionTreeClassifier or DecisionTreeRegressor).

• n_estimators: The number of base models or boosting stages to be trained. Increasing this number generally improves performance but also increases computational cost.

• random_state: A seed for the random number generator to ensure reproducibility of results.

Bagging Examples

1. Finance: Predicting Stock Prices

Bagging can be used to predict stock prices by aggregating predictions from multiple models trained on different subsets of historical stock data.

from sklearn.ensemble import BaggingRegressor
from sklearn.tree import DecisionTreeRegressor
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

# Assume we have a DataFrame df with stock features and target price
# df = pd.read_csv('stock_data.csv')

X = df.drop('price', axis=1)
y = df['price']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

base_model = DecisionTreeRegressor()
bagging_model = BaggingRegressor(base_estimator=base_model, n_estimators=50, random_state=42)
bagging_model.fit(X_train, y_train)
y_pred = bagging_model.predict(X_test)
print(f'Mean Squared Error: {mean_squared_error(y_test, y_pred)}')

2. Healthcare: Disease Diagnosis

Bagging can improve the accuracy of disease diagnosis by aggregating predictions from multiple models trained on different subsets of patient data.

from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load dataset
data = load_breast_cancer()
X, y = data.data, data.target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

base_model = DecisionTreeClassifier()
bagging_model = BaggingClassifier(base_estimator=base_model, n_estimators=50, random_state=42)
bagging_model.fit(X_train, y_train)
y_pred = bagging_model.predict(X_test)
print(f'Accuracy: {accuracy_score(y_test, y_pred)}')

Boosting Examples

1. Marketing: Customer Churn Prediction

Boosting can predict customer churn by sequentially training models to focus on customers most likely to churn based on historical data.

from sklearn.ensemble import GradientBoostingClassifier
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Assume we have a DataFrame df with customer features and churn target
df = pd.read_csv('customer_churn.csv')

X = df.drop('churn', axis=1)
y = df['churn']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

boosting_model = GradientBoostingClassifier(n_estimators=100, random_state=42)
boosting_model.fit(X_train, y_train)
y_pred = boosting_model.predict(X_test)
print(f'Accuracy: {accuracy_score(y_test, y_pred)}')

2. E-commerce: Product Recommendation

Boosting can improve the accuracy of product recommendations by focusing on patterns in customer behavior that previous models missed.

from sklearn.ensemble import AdaBoostClassifier
from sklearn.tree import DecisionTreeClassifier
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Assume we have a DataFrame df with user features and product purchase target
# df = pd.read_csv('product_recommendation.csv')

X = df.drop('purchased', axis=1)
y = df['purchased']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

base_model = DecisionTreeClassifier()
boosting_model = AdaBoostClassifier(base_estimator=base_model, n_estimators=100, random_state=42)
boosting_model.fit(X_train, y_train)
y_pred = boosting_model.predict(X_test)
print(f'Accuracy: {accuracy_score(y_test, y_pred)}')

Summary

Boosting and bagging are powerful techniques in the data scientist’s toolkit, capable of enhancing model performance across various domains. By understanding and leveraging these methods, one can build more accurate and robust models, whether predicting stock prices, diagnosing diseases, forecasting customer churn, or recommending products. These techniques demonstrate the power of ensemble learning in tackling complex predictive tasks.

Note: This was originally published August 2022 on my Medium blog before the ChatGPT-driven AI craze started.

Given the recent advances in Artificial Intelligence (AI) and its two subcategories known as Natural Language Processing (NLP) and generative adversarial networks (GAN), a plethora of new platforms have emerged on the Internet combining the create a new art form or a category of art: text-to-image art.

The latest text-to-image tools such as DALL*E 2, Midjourney and NightCafe are so powerful and simple to use that today everyone with a access to internet can become an artist in a matter of minutes. Having painted a bit in my teens and having always had an interest in poetry, I decided to find out if there is a budding poetic artist (or an artistic poet for that matter) in me after all. So I took one of the platform NightCafe (which I found easiest to use) out for a spin.

Les Fleurs du Mal

Les Fleurs du Mal is a poetry collection by the great French poet Charles Baudelaire from the 19th century. Given that his poems are either very depressing, or very beautiful, or sometimes both, I picked one from the latter category titled Albatross, which talks about the beauty and existential suffering of being a poet. The English translation is below, but you can also find the French original on FlreursDuMal.org.

The Albatross

Often, to amuse themselves, the men of a crew
Catch albatrosses, those vast sea birds
That indolently follow a ship
As it glides over the deep, briny sea.
Scarcely have they placed them on the deck
Than these kings of the sky, clumsy, ashamed,
Pathetically let their great white wings
Drag beside them like oars.
That winged voyager, how weak and gauche he is,
So beautiful before, now comic and ugly!
One man worries his beak with a stubby clay pipe;
Another limps, mimics the cripple who once flew!
The poet resembles this prince of cloud and sky
Who frequents the tempest and laughs at the bowman;
When exiled on the earth, the butt of hoots and jeers,
His giant wings prevent him from walking.

• Charles Baudelaire (translated by William Aggeler, The Flowers of Evil)

Instead of feeding the entire poem into AI, thought let us first start with a few lines so chose the following passage:

The poet resembles this prince of cloud and sky
Who frequents the tempest and laughs at the bowman

Results

After a few clicks and a few minutes of processing, NightCafe generated my come-back art piece shown below.

Figure — Passage of Albatross as Read and Seen by Text-to-Image AI

If you liked the image above, please give me a star, or whatever your favourite emoji to this post to support my budding international internet artist career. If you did not like the image above, stop reading now, or simply visit TikTok instead as they recently implemented their own text-to-image algorithm.

Applications

One of the most obvious applications of this new technology would be book and magazine story illustrations, but there certainly will be many others — a picture is more than thousand words, as the old saying goes.

References

• NightCafe

• Natural Language Processing

• Generative Adversarial Networks

• Robert A Consalves on Medium

Why this, why now!

Everyone should have a personal journal, and Substack provides an opportunity for a longer form content than most of the social media platforms out there on this planet.

After having tried multiple publishing platforms over the years, for the moment, this feels like a better home for my diverse interests such as biology, AI, software, art, finance and longevity, to mention a few.

Do not forget to press like and subscribe buttons! :)

Subscribe now