Start-of-semester thoughts:
Teaching NLP is quite depressing, and I don't know how to do it well. I am torn between the two perspectives:

1) Teach the interesting problems. Why language is interesting. Why language is hard. How is language structured. What should we look at.
The problem here is that this will be mostly presenting a set of open questions, without good solutions to any of them. We suck at everything. And things we do quite well on, like say tagging and parsing, we don't really know what to do with these structures once we have them.
And for things which we do sort-of-ok on, like coreference, we don't really have a good solution, and also no real handle on how to improve things further. This feels stuck.
And for things like pragmatics, dialog... we really don't have anything. Not even proper training data.
On the other hand, we can take the deep learning perspective, and:
2) teach embeddings, BERT, fine-tuning, squad, etc. This results in a set of techniques that work, but:
(a) we don't really know why
(b) they also fail a lot.
(c) and again we don't know how to improve.
This results in a potentially very "useful" course, but which feels very "empty" from an academic perspective, and also not really about "NLP" (in my view), because it really is reducing many things to sequence-processing. It works remarkably well --- which is good ---,
but is also does not work on any on the interesting questions I said we don't have answers for.

So, stuck between what we can do now (much more than 5 years ago, still very little), what we understand about our current tools (very very little) and where we want to be (very far).
Am I the only person bothered by this?
How do you handle this when teaching / thinking?
You can follow @yoavgo.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled: