WhiteICE version information: Difference between revisions
No edit summary |
No edit summary |
||
| (14 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
<strong>WhiteICE v1. |
<strong>WhiteICE v1.40b (in development)</strong> |
||
Added variances of target terms to RL model. This makes RL to learn actually something when it can know how accurate EEG-signal estimates of targets are. Still needs more tuning and implementation of variance terms everywhere in the code. However, thought exploration is now better, results are worse???? |
|||
| ⚫ | |||
Increased forgetting time from one hour to three hours so more data is available for optimization. |
|||
TODO: really fix "--save-video" to be faster (worse optimization) and not to run out of memory for buffering video frames for encoding. |
|||
<strong>WhiteICE v1.39b (most recent)</strong> |
|||
| ⚫ | |||
Program learns very slowly, 3h 20minutes (300 models) didn’t show big improvements. Try to increase tau from 0.02 to 0.1 and see if it still increases performance a bit. => tau increase seem to give worse results??? |
|||
Added forgetting to reinforcemen learning, now 30 minutes older measurements are not used. TODO: increase tau from 0.02 to 0.1 (for testing) so model can learn temporal dynamics better. |
|||
DONE: Added additional measurements to RL's state vector. |
|||
NOW: 5000 msecs stimulation time seems to give much better results than 1000 ms (2-3% effect). However, forgetting time should be maybe increased from 30minutes to 60minutes? (TODO: test it) AND test 10 sec long stimulus again to see if there is better effect (+3.1% effect vs random and about zero effect to distance). |
|||
<strong>FINAL</strong>: 5000 msecs stimulation time which often gives 3% result. Note that this is worse than in 1.38b which seem to give 5% result but RL algorithms in 1.39 are more sophisticated. |
|||
<strong>WhiteICE v1.38b</strong> |
<strong>WhiteICE v1.38b</strong> |
||
Latest revision as of 00:46, 7 August 2026
WhiteICE v1.40b (in development)
Added variances of target terms to RL model. This makes RL to learn actually something when it can know how accurate EEG-signal estimates of targets are. Still needs more tuning and implementation of variance terms everywhere in the code. However, thought exploration is now better, results are worse????
Increased forgetting time from one hour to three hours so more data is available for optimization.
TODO: really fix "--save-video" to be faster (worse optimization) and not to run out of memory for buffering video frames for encoding.
WhiteICE v1.39b (most recent)
New version with improved RL algorithm that SHOULD give more often positive results (in many hours long stimulation). Stimulation interval is 1000ms. Effect seem to be only about 1% with this new code.
Program learns very slowly, 3h 20minutes (300 models) didn’t show big improvements. Try to increase tau from 0.02 to 0.1 and see if it still increases performance a bit. => tau increase seem to give worse results???
Added forgetting to reinforcemen learning, now 30 minutes older measurements are not used. TODO: increase tau from 0.02 to 0.1 (for testing) so model can learn temporal dynamics better.
DONE: Added additional measurements to RL's state vector.
NOW: 5000 msecs stimulation time seems to give much better results than 1000 ms (2-3% effect). However, forgetting time should be maybe increased from 30minutes to 60minutes? (TODO: test it) AND test 10 sec long stimulus again to see if there is better effect (+3.1% effect vs random and about zero effect to distance).
FINAL: 5000 msecs stimulation time which often gives 3% result. Note that this is worse than in 1.38b which seem to give 5% result but RL algorithms in 1.39 are more sophisticated.
WhiteICE v1.38b
This version re-enabled longer stimulation interval 5000ms which seem to have positive effect to "episodes-result.txt" average reinforcement value time-series when stimulatio is run for many hours. Typical effect seem to be 5% (versus random stimulation)
WhiteICE v1.37b
This experimental version used very short stimulation interval 500ms.