• Coroutine: like a subroutine but can stop in the middle and back to the point where it “yields”. When it yields, it yields to some other cooperative functions. It is popular in multithreading environment. Since there is no need for synchronization.
  • Credit assignment:
  • Embedding: represent an object with a vector. Essentially it is the same as one-hot encoding followed by product of an embedding matrix.
  • Goal preservation: a sufficiently sophisticated enough agent will prevent its goal from being modified.
    • Terminal vs instrumental goal: terminal goal is what is given. Instrumental goal can be derived by an intelligent agent to try achieve the terminal goal.
  • Hume’s Guilotine: there is an unavoidable separation of coherent logic flow from descriptive (is) statement and prescriptive (ought) statement
  • Pascal’s wager: it is better to believe in God. If one doesn’t, one will be tortured for infinity afterlife if God exist.
    • Problem: payoff is very large but probability is very small as well. The expected payoff can be anything
    • Counter-argument: there is another God and required me to believe in him only as well. If I don’t, I will also be penalized.
  • Privacy: a dataset is private if combinations of operation (mechanism) will not disclose sensitive characteristic of any one record in the dataset
    • Differential private: adding/deleting a single record will have no effective change of the dataset
  • Reasoning:
  • Reward hacking: an agent tries to exploit the slight difference between the reward function meant to be and the reward function that was actually written
  • Reward modeling: rather than manually defining a reward function, construct a reward function through training
  • Side effect: fail to include specify in the reward function everything that we care about. The agent may assume that everything not mentioned in the reward function has zero value. This may lead to large negative side effect
  • Transformer:
Copyright OU-Tulsa Lab of Image and Information Processing 2020
Tech Nerd theme designed by Siteturner