Win a copy of Terraform in Action this week in the Cloud forum!

Brandon Brown

Author
+ Follow
since Sep 17, 2020
Cows and Likes
Cows
Total received
5
In last 30 days
0
Total given
0
Likes
Total received
2
Received in last 30 days
0
Total given
0
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Brandon Brown

Don Horrell wrote:Thank you for your interesting comment.

Could you explain a bit more about what you mean by a "dynamic environment" please?

So, if Reinforcement Learning can select an advertisement for a user, that sounds similar to classification (of images?). A person walks past a smart advertising board which somehow identifies that person and the "label" is the type of advert that will be displayed for that person.
Or have I got the wrong end of the stick? Or perhaps even the wrong stick?


Cheers
Don.



Yes the label would be one of some finite number of possible advertisements. The objective here, however, would be for the RL algorithm to optimize the clickthrough rate. So rather than an image classification where the algorithm is trained based on whether or not the classification is correct or not, the decision here is not binary. There is no one correct ad, some ads will result in more clicks (if these are ads on a website) than others, and the goal is to learn which ad will cause a potential customer to be most likely to respond to the ad by buying.

A dynamic environment just to mean that again, the decision isn't a correct/incorrect labeling, but a set of actions that lead to more or less of some outcome (points in a game, clicks for ads, money if trading stocks, etc)

Don Horrell wrote:Hi Alexander and Brandon.
How does reinforcement learning differ from normal "gradient descent" learning?

Thanks
Don.



So gradient descent is actually a particular kind of optimization strategy, just one of many ways of tuning a set of parameters of a function to be optimal according to some objective. All parametric machine learning and statistical models need to be optimized (or "fit" to data) and gradient descent is the most popular as it is scalable, iterative, and works well with big data. So we use gradient descent in RL since RL generally uses neural networks or other complex machine learning models underneath the hood, so to speak.

Don Horrell wrote:Hi.

What are the main features of reinforcement learning, in a nutshell?
Is reinforcement learning likely to be better in general than other ML techniques e.g. faster learning, more accurate results?

Thanks
Don.



RL is a framework for decision-making in a dynamic environment. It is not a specific machine learning model like a neural network or support vector machine. RL often uses these machine learning models but is a framework, not a model. RL is useful for decision-making problems where you do not know the "right" answer (there are no labels), you just know how to quantify better and worse answers/decisions. If you speak abstract math, RL is like a mathematical structure like a group that may contain a function (being akin to a specific machine learning model.)

Don Horrell wrote:Hi Alexander and Brandon.

Can Reinforcement Learning be used for Multi-Label classification?
We have a potential application where the labels run to tens or perhaps a few hundred thousand different labels. Is Reinforcement Learning something we should be investigating for this?

Thanks
Don.



If it's a static sort of classification problem like image classification, then RL wouldn't be the right way to conceptualize the problem. However, if your classification problem exists in a dynamic environment, such as deciding which advertisement to display for a particular user, which may change depending on dynamic user and sitewide data, then RL could be useful (see multi-armed bandits). RL is a framework for decision-making in a dynamic environment.

Don Horrell wrote:Hi Alexander and Brandon.

Is Reinforcement Learning good for Natural Language Processing problems?
How would you apply it to NLP?

Thanks
Don.



Hi Don, thanks for the question!

In terms of actually parsing and representing natural language, the focus would be on particular kinds of learning algorithms such as transformer deep learning models. I'm not aware of much practical or research use of RL for ordinary NLP problems, however, people do study how multiple interacting RL agents can generate simple language which is very neat, but the focus there is on understanding the emergence of language from multiple interacting agents.

Jack Donahey wrote: Is it mainly about neuron nets? or can you use enforced learning with other types of AI ?



Hello! Reinforcement learning is a general framework that is not tied to neural networks (any function approximator will due, or even a database for low complexity problems), however, modern applied reinforcement learning almost always uses neural networks as the "engine" within the RL framework due to their representational power and scaling properties.
Hello - Looking forward to discussing our book Deep Reinforcement Learning in Action here.
1 year ago