Wednesday, September 21, 2011

Matlab MLPR

Hello People,

our group is busy with the matlab assignement for about two days and we dont get any further. So could someone maybe help us...

Here is what we did until know:
-we found some features
-we computed mu(greek letter) for all features, but the values of it are very low (<1%)
-so if we compute the p(xC_spam) for one email it also will have only very small values
-appearently this is not correct, we should do something with Maximum likelihood, but we using it in the wrong way.

Does anybody know how we could do this?


Greetings,
Hoda, Christos and Kristin

10 comments:

  1. Could you post the code that you are using to compute mu_IK and p(x | C_k)?

    Also, keep in mind that you need to compute p(C_k | x) to check whether your e-mail is spam or not. First compute p(x | C_k), p(x) = p(x | C1) p(C1) + p(x | C2) p(C2) and p(C_k) and use Bayes' rule to determine p(C_k | x).

    I hope this helps.

    Mihai

    ReplyDelete
  2. What mihai said is correct.
    It also does not matter if p(xC_spam) becomes small, as p(xC_ham) will also get very small if you dont regularize. Since the regularization terms for both are the same you can ignore them and just see if

    p(xC_spam) > p(xC_ham) without regularization. Do compute both!!!!

    ReplyDelete
  3. I meant ofcourse p(C_spam| X) and p (C_ham | X) in all cases ;)

    ReplyDelete
  4. Hey guys thank you all for the reactions. I'm working on it to see what's gonna happen...

    Tijmen I didn't really get the concept of regularization. When it says to regularize it what does it means?

    Hoda

    ReplyDelete
  5. I meant like: normal base rule is:

    p(C | x) = p(x | C) p(C) / p(x)

    You throw out the p(x) term because you will only compare the chances, you dont need to know them exactly. Regularization basically means that you make your probability distribution behave like one (be in the range from 0 to 1).

    ReplyDelete
  6. Do we even need to use regularisation if the regularisation terms are the same and thus ignored as you mentioned earlier Tijmen?

    ReplyDelete
  7. Well for general classification the normalization terms P(x) can be ignored. However, on the last page of the questions Gwenn asks us to swap around the decision boundary from 0 to 1. Then you have to normalize with P(x) for this question to make sense. It's not hard though, you will just have
    P(C1|x) = p(x|C1) P(C1)/(p(x|C1) P(C1) + p(x|C2) P(C2))
    Then you will get [0,1] chances for your classes.

    ReplyDelete
  8. If anyone still has problems with this, send me an e-mail or post here. I'll be up most of the night to finish the report anyhow :)

    T

    ReplyDelete
  9. I'm getting this question a lot, so I'll post it here. Do not use for loops in Matlab unless you really have to. Feature selection should take 5 seconds max. You can get all the data you need with the following three lines of code

    [S1, numSpamFiles] = countwords(spamTrainDir);
    [S2, numHamFiles] = countwords(hamTrainDir);
    [WORDS, COUNTS] = wordtable(S1,S2);

    The WORDS array contains all words, and COUNTS contains all frequencies of spam in column 1, and all frequencies in ham in column 2. Just calculate the difference of class-conditional probabilities in a seperate array, and use that array to sort WORDS. Then you can select the first n features :)

    ReplyDelete
  10. Here's what Vasilis and me did: http://dl.dropbox.com/u/6166376/MLPR%20lab%202.7z

    Hope it helps some people. Also, if you find any mistakes, we'd appreciate it if you let us know.

    ReplyDelete