• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • Rob Spoor
  • Bear Bibeault
Saloon Keepers:
  • Jesse Silverman
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • Al Hobbs
  • salvin francis

Reinforcement Learning in small applications

 
Greenhorn
Posts: 6
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I've been fascinated by what RL can do and the successes it has had accomplishing really complicated tasks like Go or Robotic hand manipulations. However, I've often wondered if there are reasons why more practical and small scale applications couldn't be done using RL. For instance, I'm often finding myself copying and pasting each item in a list from one application another. I've wondered why RL could 't be trained to do these kinds of practical applications. I understand training would need to happen in some kind of simulated environment, and it seemed to me OpenAI was headed in that direction with the Universe product (which has since been largely abandoned). So is there any work being done in this direction?
Looking forward to reading the book.
Randy
 
Author
Posts: 3
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Randy,

Very interesting question! Thanks for asking it.

RL can certainly be applied to practical and small scale applications. For example RL can be used to replace heuristics with trained policies in resource management (e.g. Resource Management with Deep RL) or to improve the performance of algorithmic problems such as binary search, sorting, and caching (e.g. Predicted Variables in Programming). RL has also been applied to recommender systems (e.g. Top-K Off-Policy Correction for a REINFORCE Recommender System) and chip placement (e.g. Chip Placement with Deep RL). These last two examples are practical but admittedly not small scale.

Paper links:

Resource Management with Deep Reinforcement Learning
https://dl.acm.org/doi/10.1145/3005745.3005750

Predicted Variables in Programming
https://openreview.net/forum?id=B1epooR5FX

Top-K Off-Policy Correction for a REINFORCE Recommender System
https://arxiv.org/abs/1812.02353

Chip Placement with Deep Reinforcement Learning
https://arxiv.org/abs/2004.10746

More generally, there are a number of different aspects to consider when thinking about whether RL is well suited to a problem.

1. Can the problem be framed as an RL problem? I.e. as an agent that takes actions that change the state of an environment?

2. How difficult is it to create a training environment? This includes the states, actions, rewards, and transition function. Part IV of the book is dedicated to environment design and discusses this in more detail.

3. How can the agent be evaluated safely in a realistic setting? How to ensure an agent behaves safely and appropriately when deployed?

Depending on the answers to questions, other machine learning approaches such as supervised learning may be more suitable. Especially since deep RL models are challenging to train when compared with supervised deep learning.

Hope this helps,
Laura
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic