Don Horrell

Greenhorn
+ Follow
since Oct 29, 2004
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
3
Received in last 30 days
0
Total given
0
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Don Horrell

Hi Alexander and Brandon.

There has been some discussion that ML algorithms are generally black boxes, with little or no insight into why certain inputs produce certain labels or outputs.
Does Reinforcement Learning produce algorithms that are any more understandable that the often opaque "computer says no" neural networks produced using other ML techniques?


Thanks
Don.
Hi Alexander and Brandon.

You have mentioned neural networks when answering one of the other questions.
I have used TensorFlow a little bit (from Python) - are there any bits of TensorFlow2 or extra library code to make implementing RL easier?
Is TensorFlow2 covered in your book?


Thanks
Don.
Hi Alexander and Brandon.

Is RL a supervised learning technique, or unsupervised? Or is it somewhere between?
It seems to be supervised in the sense that (I think) you need to reward good outcomes and penalise bad outcomes, so there is some goal to aim for.
However, the supervision seems to be much less precise than other supervised techniques, where you label training data with a "correct answer".
It seems to be more than unsupervised, as you are giving the algorithm some guidance.


Thanks
Don.
Thank you for your interesting comment.

Could you explain a bit more about what you mean by a "dynamic environment" please?

So, if Reinforcement Learning can select an advertisement for a user, that sounds similar to classification (of images?). A person walks past a smart advertising board which somehow identifies that person and the "label" is the type of advert that will be displayed for that person.
Or have I got the wrong end of the stick? Or perhaps even the wrong stick?


Cheers
Don.
Hi Alexander and Brandon.

Thanks for your previous answers to my questions on Classification and NLP.
I think I should have started with this much broader question:
What sort of applications is Reinforcement Learning best at?


Thanks
Don.
Hi Alexander and Brandon.

Is Reinforcement Learning good for Natural Language Processing problems?
How would you apply it to NLP?

Thanks
Don.
Hi Alexander and Brandon.

Can Reinforcement Learning be used for Multi-Label classification?
We have a potential application where the labels run to tens or perhaps a few hundred thousand different labels. Is Reinforcement Learning something we should be investigating for this?

Thanks
Don.
Hi.

What are the main features of reinforcement learning, in a nutshell?
Is reinforcement learning likely to be better in general than other ML techniques e.g. faster learning, more accurate results?

Thanks
Don.
Hi Alexander and Brandon.
How does reinforcement learning differ from normal "gradient descent" learning?

Thanks
Don.
Just to clarify, this is a multi-LABEL problem, not multi-class.
Apologies for my mistake.


Don.
My other ML interest is topic modelling, using document vectors.
There do not seem to be pre-trained sets of document vectors available yet, but when there are, how could we use transfer learning to take a pre-trained set of document vectors and adapt it to domain-specific documents e.g. medical documents, documents about programming, patents etc?
Thanks for your reply, Paul. I'm a little confused though.
The pre-trained FastText word embeddings I have downloaded map words to vectors, so in my case (using TensorFlow to do some NLP classification), I can only train my classifier on the words in the embedding list.
That is the crux of my original question - how can I add domain-specific vocabulary to pre-trained word embeddings. Will your book cover this?


Thanks
Don.
Hi all.

I am trying to train a simple CNN on this dataset, which is multi-class and natural language:
https://www.kaggle.com/badalgupta/stack-overflow-tag-prediction/data

I am using word embeddings from FastText.
I have converted the words to index numbers in my vocab (from FastText), then used a (non-trainable) TensorFlow Embedding layer to convert the index numbers to word vectors using the pre-trained FastText embeddings.
The labels are multi-hot encoded (there are 100 labels).
The output activation is sigmoid and the loss is binary crossentropy, as that is what many websites recommend.
I have just split the train/validation/test sets randomly for now, so they do not take into account of the labels.

When I train the CNN, the "accuracy" gets to 0.99 very quickly and the loss is low.
At the end of each epoch the precision, recall and F1 scores gradually improve, then plateau at around 0.35.

The predictions are poor, with the maximum probability from the sigmoid output often as low as 6%, so the network does not seem to be properly trained. With a high accuracy and low loss, any training will be very slow anyway.

As there is a fairly large skew in the number of times each label has been allocated, I have used the class_weight parameter when fitting, to try to assist the training.

Does anyone have the experience to point to where I should start my investigation? There are so many things to twiddle!
Perhaps the transfer-learning expert will have some ideas.


Thanks
Don.
What are the strengths and weaknesses of Gensim and TensorFlow for NLP?
Which is best for the different types of project?