get posterior distribution of predicted probabilities

[bakaburg1]

Hello!

I'm trying your great tool on a classification problem, specifically to classify the famous Iris dataset as a test.

The package works great, but I get just one predicted value per row x class when using the predict_proba method.

What am I doing wrong? I expected to be able to get a full posterior distribution per row x class.

All the best

[iivalchev]

The posterior is exactly what you get from predict_proba. Could you share the actual code you are using?

[bakaburg1]

Here's the code (in R using reticulate).

I think I understood where my misunderstanding lies. TabPFN outputs the posterior predictive distr of the predicted classes, which then you compress in a probability, not the posterior of the class probabilities, like a binomial model would do isn't it?

And I guess there's no way to get the second...

tp <- reticulate::import("tabpfn")

model <- tp$TabPFNClassifier()

model$fit(as.matrix(iris[,1:4]), iris$Species)

model$predict_proba(as.matrix(iris[, 1:4])) |> as_tibble()

# A tibble: 150 × 3
      V1          V2          V3
   <dbl>       <dbl>       <dbl>
 1  1.00 0.000000741 0.000000632
 2  1.00 0.00000114  0.000000504
 3  1.00 0.000000987 0.000000706
 4  1.00 0.00000219  0.000000822
 5  1.00 0.000000837 0.000000785
 6  1.00 0.00000140  0.000000474
 7  1.00 0.00000203  0.000000761
 8  1.00 0.000000904 0.000000695
 9  1.00 0.00000299  0.000000725
10  1.00 0.00000180  0.000000666
# ℹ 140 more rows

[iivalchev]

This sounds right, TabPFN approximates directly P(y_new | X_new, X_train, y_train)

[iivalchev]

Sorry I missed your last question :)

not the posterior of the class probabilities, like a binomial model would do isn't it?

In TabPFN you get just a single probability prediction per class per test sample, not a distribution. You might be able to get per ensemble member predictions before being averaged, but is not supported out of the box.

[bakaburg1]

ok, then consider it as a feature request! it would be great to have posterior uncertainty around estimated probabilities.