Eksperimen Trading Saham Menggunakan Deep Reinforcement Learning – 4

by

Melanjutkan dari modul sebelumnya, pada modul ini akan dibahas mengenai data loading dan training.

Pada tutorial kIta akan menggunakan data saham Apple.

stock_name = "AAPL"
data = dataset_loader(stock_name)

Setelah data berhasil diload, berikutnya adalah mendefinisikan model dengan mengatur hyperparameter.

window_size = 10
episodes = 1000

batch_size = 32
data_samples = len(data) - 1

Setelah parameter diatur, kita create model dengan memanggil class AI_Trader

trader = AI_Trader(window_size)

Anda bisa periksa apakah model berhasil dibuat dengan melihat model summary.

trader.model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 32)                352       
_________________________________________________________________
dense_1 (Dense)              (None, 64)                2112      
_________________________________________________________________
dense_2 (Dense)              (None, 128)               8320      
_________________________________________________________________
dense_3 (Dense)              (None, 3)                 387       
=================================================================
Total params: 11,171
Trainable params: 11,171
Non-trainable params: 0

Proses terakhir adalah melakukan training

for episode in range(1, episodes + 1):
  
  print("Episode: {}/{}".format(episode, episodes))
  
  state = state_creator(data, 0, window_size + 1)
  
  total_profit = 0
  trader.inventory = []
  
  for t in tqdm(range(data_samples)):
    
    action = trader.trade(state)
    
    next_state = state_creator(data, t+1, window_size + 1)
    reward = 0
    
    if action == 1: #Buying
      trader.inventory.append(data[t])
      print("AI Trader bought: ", stocks_price_format(data[t]))
      
    elif action == 2 and len(trader.inventory) > 0: #Selling
      buy_price = trader.inventory.pop(0)
      
      reward = max(data[t] - buy_price, 0)
      total_profit += data[t] - buy_price
      print("AI Trader sold: ", stocks_price_format(data[t]), " Profit: " + stocks_price_format(data[t] - buy_price) )
      
    if t == data_samples - 1:
      done = True
    else:
      done = False
      
    trader.memory.append((state, action, reward, next_state, done))
    
    state = next_state
    
    if done:
      print("########################")
      print("TOTAL PROFIT: {}".format(total_profit))
      print("########################")
    
    if len(trader.memory) > batch_size:
      trader.batch_train(batch_size)
      
  if episode % 10 == 0:
    trader.model.save("ai_trader_{}.h5".format(episode))
0%|          | 0/1256 [00:00<?, ?it/s]Episode: 1/1000
AI Trader bought:  $ 24.657499
AI Trader sold:  $ 24.757500  Profit: $ 0.100000
AI Trader bought:  $ 24.285000
AI Trader bought:  $ 24.387501
AI Trader sold:  $ 23.832500  Profit: - $ 0.452499
AI Trader sold:  $ 23.775000  Profit: - $ 0.612501
AI Trader bought:  $ 24.025000
AI Trader sold:  $ 23.350000  Profit: - $ 0.674999
AI Trader bought:  $ 23.900000
AI Trader bought:  $ 23.985001
AI Trader bought:  $ 24.245001
AI Trader sold:  $ 24.697500  Profit: $ 0.797501
AI Trader bought:  $ 24.695000
AI Trader bought:  $ 24.957500
AI Trader sold:  $ 24.967501  Profit: $ 0.982500
  3%|▎         | 33/1256 [00:06<03:45,  5.43it/s]AI Trader sold:  $ 24.990000  Profit: $ 0.744999
  3%|▎         | 34/1256 [00:09<20:31,  1.01s/it]AI Trader sold:  $ 24.857500  Profit: $ 0.162500
  3%|▎         | 35/1256 [00:12<32:57,  1.62s/it]AI Trader sold:  $ 24.665001  Profit: - $ 0.292500
  3%|▎         | 36/1256 [00:15<41:01,  2.02s/it]AI Trader bought:  $ 24.334999
  3%|▎         | 37/1256 [00:17<46:34,  2.29s/it]AI Trader bought:  $ 24.167500
  3%|▎         | 38/1256 [00:20<51:07,  2.52s/it]AI Trader bought:  $ 25.737499
  3%|▎         | 39/1256 [00:24<54:40,  2.70s/it]AI Trader sold:  $ 26.084999  Profit: $ 1.750000
  3%|▎         | 40/1256 [00:27<56:36,  2.79s/it]AI Trader sold:  $ 26.052500  Profit: $ 1.885000
  3%|▎         | 41/1256 [00:30<58:38,  2.90s/it]AI Trader bought:  $ 26.512501
  3%|▎         | 42/1256 [00:33<59:34,  2.94s/it]AI Trader sold:  $ 26.120001  Profit: $ 0.382502
  4%|▎         | 45/1256 [00:42<1:00:18,  2.99s/it]AI Trader sold:  $ 26.870001  Profit: $ 0.357500
  4%|▍         | 48/1256 [00:51<1:01:15,  3.04s/it]AI Trader bought:  $ 27.000000
  4%|▍         | 49/1256 [00:54<1:01:35,  3.06s/it]AI Trader bought:  $ 26.982500
  4%|▍         | 51/1256 [01:01<1:02:51,  3.13s/it]AI Trader sold:  $ 27.370001  Profit: $ 0.370001
  4%|▍         | 52/1256 [01:04<1:02:06,  3.10s/it]AI Trader bought:  $ 27.344999
  4%|▍         | 53/1256 [01:07<1:01:44,  3.08s/it]AI Trader bought:  $ 27.305000
  4%|▍         | 54/1256 [01:10<1:00:47,  3.03s/it]AI Trader bought:  $ 27.270000
  4%|▍         | 55/1256 [01:13<1:00:30,  3.02s/it]AI Trader sold:  $ 27.340000  Profit: $ 0.357500
  5%|▍         | 57/1256 [01:19<1:00:52,  3.05s/it]AI Trader sold:  $ 27.212500  Profit: - $ 0.132500
  5%|▍         | 59/1256 [01:25<1:00:23,  3.03s/it]AI Trader sold:  $ 26.892500  Profit: - $ 0.412500
  5%|▍         | 60/1256 [01:28<1:00:09,  3.02s/it]AI Trader bought:  $ 26.735001
  5%|▍         | 62/1256 [01:34<1:00:38,  3.05s/it]AI Trader bought:  $ 26.500000
  5%|▌         | 63/1256 [01:37<1:00:30,  3.04s/it]AI Trader sold:  $ 26.525000  Profit: - $ 0.745001
  5%|▌         | 65/1256 [01:43<1:00:41,  3.06s/it]AI Trader bought:  $ 26.932501
  5%|▌         | 66/1256 [01:46<1:01:10,  3.08s/it]AI Trader sold:  $ 26.924999  Profit: $ 0.189999

Model ini masih sangat sederhana untuk real melakukan trading. Mungkin untuk meningkatkan akurasi dari model, bisa digunakan layer LSTM atau menambahkan sentimen analisis.

Semoga artikel ini bisa bermanfaat dan menambah wawasan.

File colab bisa akses di https://colab.research.google.com/drive/1r0SyVacV534AsmkBfcMCM9kOlq3dxSG3?usp=sharing

Sharing is caring:

Leave a Comment