Skip to main content
Home

Main navigation

  • Home
User account menu
  • Log in
By Skander, 15 November, 2024

The Exploration-Exploitation Balance: The Epsilon-Greedy Approach in Multi-Armed Bandits

Exploration versus exploitation

In this article, I will explore the balance between exploration and exploitation, a key concept in reinforcement learning and optimization problems. To illustrate this, I will use the multi-armed bandit problem as an example. I will also explain how the epsilon-greedy strategy effectively manages this balance.

By Skander, 12 November, 2024

Comparison of Three Multi-armed Bandit Strategies

Comparison of multi-armed bandit strategies
In a previous article, I introduced the design and implementation of a multi-armed bandit (MAB) framework. This framework was built to simplify the implementation of new MAB strategies and provide a structured approach for their analysis. Three strategies have already been integrated into the framework: RandomSelector, MaxAverageRewardSelector, and UpperConfidenceBoundSelector. The goal of this article is to compare these three strategies.
By Skander, 8 November, 2024

Design and Implementation of A Unifying Framework For Multi-armed Bandit Solvers

Multi-armed bandit framework

In previous blog posts, we explored the multi-armed bandit (MAB) problem and discussed the Upper Confidence Bound (UCB) algorithm as one approach to solving it. Research literature has introduced multiple algorithms for tackling this problem, and there is always room for experimenting with new ideas. To facilitate the implementation and comparison of different algorithms, we introduce a framework for MAB solvers.

By Skander, 3 November, 2024

Analyzing the Upper Confidence Bound Algorithm

Number of impressions for each ad over time.

This article focuses on evaluating the implementation of the Upper Confidence Bound (UCB) algorithm discussed herein. The evaluation is conducted using a single dataset provided by Super Data Science.

By Skander, 1 November, 2024

A Python Implementation of The Upper Confidence Bound Reinforcement Learning Algorithm

multi-armed bandit

This article explores the implementation of a reinforcement learning algorithm called the Upper Confidence Bound (UCB) algorithm. Reinforcement learning, a subset of artificial intelligence, involves an agent interacting with an environment through a series of episodes or rounds. In each round, the agent makes a decision that may yield a reward. The agent's ultimate objective is to learn a strategy that maximizes its cumulative reward over time.

By Skander, 14 August, 2024

Clean and Reusable Property Validation Using TypeScript Decorators

typescript logo

In this blog post, we’ll explore how to implement two custom object property validators using TypeScript decorators. While popular libraries like class-validator already provide a rich set of decorator-based validators, our goal here is to demonstrate how to build your own—specifically, a @Positive validator and a @NotEmpty validator—in a clean and reusable way.

By Skander, 15 June, 2013

Inferno By Dan Brown

Dan Brown Inferno Book Cover

I am an avid reader of Dan Brown books. I loved reading "Angels and Demons". His book "The Davinci Code" motivated me to learn more about the three major monotheist religions from a historical point of view.

I was anticipating the publication of "The Lost Symbol" book. The book came after a three-years delay and it was such a disappointment. I had the impression that Daniel Brown was writing his book for Holywood and not for his readers. I said to myself, Brown is dead as an author and he will never dare to publish a book again. A passing fashion.

Novels
By Skander, 11 March, 2013

What Is Your Erdös Number?

Paul erdos

I have purchased, last weekend, Paul Hoffman's book The Man Who Only Loved Numbers. This is a biography of the great and equally eccentric number theorist of all times Paul Érdös.

The book has been published in 1998, two years after the death of Érdös in Warsaw. The first 25% of Hoffman's book is interesting and entertaining at the same time. It tell the story of a man who devoted his whole life to Mathematics, who travelled from country to country carrying all his belongings in a briefcase and a bag.

Science books
  • More From Skander

My Apps

  • One-dimensional Cellular Automata Simulator
  • Collatz (Syracuse) Sequence Calculator / Visualizer
  • Erdős–Rényi Random Graph Generator / Analyzer
  • KMeans Animator
  • Language Family Explorer

New Articles

Agent-Based Modeling: Exploring Complexity Through Simulation
Pedalytics, Part 1 — Scoping, Architecting, and Generating an MVP with AI
Building a Web Application with AI: My Experience with Vibe Coding and AI-Assisted Software Engineering
Beyond Coding by Addy Osmani - A Book Review
When Free LLMs Turned Restrictive: Lessons from Building a YouTube Recommender with Gemini

Skander Kort