Happy new year from the stockwell team! We have been busy triaging bugs an=
d getting the test-verify job to be fine tuned for all platforms and test s=
If you want to read about what we plan to do in the near future, you can fo=
llow along in this tracking bug:
One change I wanted to make everyone aware of is the threshold we use to se=
t the whiteboard tag: [stockwell disable-recommended]. For the last 4 mont=
hs that has been 200 failures in 30 days. We look at this whiteboard tag t=
wice a week and focus on those bugs- basically if there isn't signs of an u=
pcoming patch we disable the test to reduce the pain on the trees.
What I would like to change on January 15th, is the threshold so that it is=
150 failures over 21 days. In analyzing data over the last 4 months there=
would have been 2 bugs which we would have disabled which ended up getting=
fixed a week later- the rest of the bugs had active development taking pla=
ce and would have been fixed/resolved as they were normally.
The advantage of doing this is we would disable tests 9 days faster, reduci=
ng failures on the tree and more importantly fewer failures on your try pus=
hes with almost the exact same end result.
Please reply if you have concerns or other ideas we should consider.