Maximizing Revenue Menggunakan Thompson Sampling – Part 3

Pada modul ini kita lakukan modifikasi dari kode sebelumnya dengan menambahkan perhitungan regret.

Regret curve dari (Strategi Random atau Thompson Sampling) adalah plot selisih antara
strategi terbaik dan model yang digunakan terhadap round(state).

#import library
import numpy as np
import matplotlib.pyplot as plt
import random

#setting parameter
N = 10000
d = 9

#create simulation
conversion_rates = [0.05,0.13,0.09,0.16,0.11,0.04,0.20,0.08,0.01]
X = np.array(np.zeros([N,d]))
for i in range(N):
  for j in range(d):
    if np.random.rand() <= conversion_rates[j]:
      X[i,j] = 1


# Implementing a Random Strategy and Thompson Sampling with Regret Curve
strategies_selected_rs = []
strategies_selected_ts = []
total_reward_rs = 0
total_reward_ts = 0
total_reward_bs = 0
numbers_of_rewards_1 = [0] * d
numbers_of_rewards_0 = [0] * d
rewards_strategies = [0] * d
regret = []
for n in range(0, N):
  # Random Strategy
  strategy_rs = random.randrange(d)
  strategies_selected_rs.append(strategy_rs)
  reward_rs = X[n, strategy_rs]
  total_reward_rs = total_reward_rs + reward_rs
  # Thompson Sampling
  strategy_ts = 0
  max_random = 0
  for i in range(0, d):
    random_beta = random.betavariate(numbers_of_rewards_1[i] + 1,
    numbers_of_rewards_0[i] + 1)
    if random_beta > max_random:
      max_random = random_beta
      strategy_ts = i
  reward_ts = X[n, strategy_ts]
  if reward_ts == 1:
    numbers_of_rewards_1[strategy_ts] = numbers_of_rewards_1[strategy_ts] + 1
  else:
    numbers_of_rewards_0[strategy_ts] = numbers_of_rewards_0[strategy_ts] + 1
  strategies_selected_ts.append(strategy_ts)
  total_reward_ts = total_reward_ts + reward_ts
  # Best Strategy
  for i in range(0, d):
    rewards_strategies[i] = rewards_strategies[i] + X[n, i]
  total_reward_bs = max(rewards_strategies)
  # Regret
  regret.append(total_reward_bs - total_reward_ts)


# Computing the Absolute and Relative Return
absolute_return = total_reward_ts - total_reward_rs
relative_return = (total_reward_ts - total_reward_rs) / total_reward_rs * 100
print("Absolute Return: {:.0f} $".format(absolute_return))
print("Relative Return: {:.0f} %".format(relative_return))

# Plotting the Histogram of Selections
plt.hist(strategies_selected_ts)
plt.title('Histogram of Selections')
plt.xlabel('Strategy')
plt.ylabel('Number of times the strategy was selected')
plt.show()
plt.close()

# Plotting the Regret Curve
plt.plot(regret)
plt.title('Regret Curve')
plt.xlabel('Round')
plt.ylabel('Regret')
plt.show()

Absolute Return: 886 $
Relative Return: 89 %

File Google Colabs bisa diakses di https://colab.research.google.com/drive/1ukCbCnftawcEBmr6piF62JP-HEsKGRC7?usp=sharing

Dengan berakhirnya modul ini, tutorial Studi Kasus Penerapan AI Pada Bisnis telah selesai.

Semoga bermanfaat.