I would like to improve my forecasting skills. For that, I prepared the app to track predictions. Each prediction can also include extra comments or related commitments, to help me understand why it matters. Also, the app calculates Brier score and shows how many predictions will resolve soon.
Specifically I want to track three types of thoughts:
- Predictions - I think USA will invade Iran by Saturday, 60% chance
- Commitments - I want to do X until May, because I think it will… - is also a prediction if you frame it correctly
- Decisions - I did X, because I expected Y by June.
I don’t think I have to explain why I want to improve my forecasting and decision making skills, it seems quite obvious. The idea is to push a lot of predictions and thoughts over time. P value is not editable, so I can make a different prediction over time for the same topic and see what influenced my train of thought (via comments I can make in the app).
I am 90% sure something like this or better already exists. Prediction markets are not private. I don’t need a diary app too, I need math and scores.
I am open to your feedback regarding the idea of the app, data model and other thoughts related to improving one’s forecasting skills.
3 Likes
I use Fatebook for a lot of this! And have a Beeminder goal to make regular predictions, since Beeminder has an integration with it.
For me I mostly predict personal stuff, often related to time management and anxiety, so I can see how good my predictions are about how long it takes me to do stuff or how likely my anxiety thoughts are to be true, and those are private (I think all predictions are private by default). There are public predictions people shared and it has been interesting to add my own predictions for some.
2 Likes
oh fatebook is excellent. Can you tell more about what changes have you noticed, how is your behaviour and decision making skills affected by Fatebook?
I’d say I’ve noticed that the process of actually making a prediction and taking a few minutes to reflect on anxiety stuff instantly makes me downgrade how high my prediction is of a bad thing actually happening. If you could record my instant reaction to the likelihood of something bad happening, it’d be 90%… by the time I’m logging it in Fatebook it might be 50%.
There was one where I would have predicted something bad happening (which if pushed I’d define as “any dental work other than fillings and cleaning being required”) at the dentist at near 100% before making a prediction, predicted at 50% on Fatebook initially, downgraded it to 30% a little later, and then of course nothing terrible happened.
However, I also tried to make myself make a more specific prediction: I will need at least three of my fillings replaced, basing that on knowing one was broken and that some of them are quite old. Not a great outcome for someone with a dentist phobia, but not a vague scary terrifying one either: something I can definitely handle. I predicted that at 95% (and indeed I need six fillings replaced, some due to damage, two due to the dentist choosing the wrong material previously). I think forcing my vague fears into a solid shape and marshalling the actual facts to make a prediction about was really helpful, too.
I think in both directions it was useful to see the outcome: that I was wrong to catastrophise, and that by looking at actual data and past patterns and so on, I did come up with an accurate prediction.
Re: time management, mostly I have discovered that I am over-optimistic to a fault. I’ve been trying to shift towards some realism there but mostly what happens is that making a prediction with high confidence pushes me to make it correct. 
Which is useful, but not quite I had intended.
4 Likes
Thank you. I’m carefully listening and learning. How many predictions have you made over last year, or overall? Alternatively; what’s your beeminder commitment please?
I want to understand the behavioural aspect; how do I grow to live with forecasting as the first class citizen of my life.
I found it; 3 a day, that seems like a lot, I don’t know if there’s enough things happening to me 
Three a day indeed! A lot of them are about reading, which I would normally do every day, so there’s always something.
1 Like
Interesting that you predicted that US will invade Iran by Sat., which did happen.
Check my prediction tool: skorytnicki's Predictions | Forecaster - I built it to keep friends accountable, to get my own stats and to make actually decisions. Each prediction can have policy atttached to it (what to do if false/true).
On Iran, I also predicted it will happen a week earlier (false 
).
I saw news about huge build up, I thought it’s too much army to just sit there. Now I saw Hegseth saying it’s not a long operation to teach them democracy, so I think it’s just 60 percent chance of regime change. Not because US will not succeed. What we will see is intensive bombing, chaos in top level command, depleting 
military potential, but no Afghanistan or Iraq style occupation. It’s a new type of war, this is my bet.
I have zero skill in geopolitics, just monitoring situation.
You can click Challenge button on my profile to get more analysis and play together.
So on top of betting for fun, you can pin predictions that will guide your life. Eg make a bet about your beeminder goal and then take action based on the result. Pin it, let it guide your life.
I think I’m onto something as natural next step is to close the loop:
- Resolve automatically based on internet (LLM doing search)
- Execute actions that are in policy via an integration, eg an HTTP call to call uncle button or whatever.
I use beeminder analogies to pass the use cases, but the gist is also beemindery. Commit to something. Attach a consequence to it. Take an action. Improve and repeat.
The irony of starting this side project is that my main objective is to be able to… close the side projects. I start some stuff, never end it, generate obligations and suffer. I think this tells us something about human nature, I mean the structure of the commitments that we take. What commitments to take to feel free and happy. There’s something Christian to it, also something existential - you’re bound to carry your cross, but the burden of choosing a cross might be even a harder problem.
I started with adding beeminder integration and telegram notifications. I would like to follow with resolutions as workflows (if false trigger this) but I feel it makes sense to wait and improve this idea.
1 Like