Expedition Navigation Software 
1 - Model Accuracy
- Expert
Offline - Registered: 3/01/2019
- Posts: 26
How does Model Accuracy actually work?
Model Accuracy allows a navigator and crew to instantly understand the strengths and weaknesses of particular GRIB sources (ex: GFS, EC, COAMPS, Predict Wind, ect…) and make a more educated and confident decision on which GRIB source to trust the most. The utility of tracking macro and micro shifts in TWD and watching the shifts inside frontal systems is invaluable in understanding the power of each GRIB source. The ability to have absolute confidence in the primary GRIB which you are routing your boat off of is in fact the best GRIB available is truly intangible. It allows the navigator to spend less time in the nav station and more time resting or on deck.
It allows the crew to know the heading and tactics are based off the best information available and nothing was left to just a “gut feeling” or chance.Inside the graphical plots and results pages the user can identify the “tendencies” of each GRIB (forecasting under/over speed for TWS or TWD left or right of actual captured data). Understanding the EXACT tendencies over a selected time period will allow the navigator to correctly twist TWD and raise/lower TWS inside the optimal routing functions of other sailing software. This way the projected optimal course can always be based on not only the most accurate weather file, but the most accurate weather file that is correctly “calibrated” for the current and past forecast errors.
There are a plethora of GRIB sources, all claiming they are the best source for you and your team to trust as you cross an ocean. Finally there is an application which provides a metric for the consumer, the absolute end user, which ranks the accuracy of different GRIB sources and exposes their successes and shortcomings. This software was made strictly for you, the sailor.
Professional sailing teams spend massive amounts of time and money constantly refining the VPPs (polars) of the boat to ensure the optimal routing software has the most accurate projected performance. Yet every “optimal course” has two variables in the solution, the boat’s VPPs and a weather GRIB. Until now there has been no way to formally analyze the accuracy while on the water of the second 50% of the optimal solution, the GRIB. Without reservation a team is now able to know they are pairing their best VPPs with the best GRIB possible for the optimized route. Every sailing team always wants to know they have the absolute best VPPs loaded into their routing software, now they can know they have the absolute best GRIB as well.
The Model Accuracy software provides levels of confidence in GRIB predictions by using statistical and other analytic techniques to evaluate GRIBs in the context of TWS & TWD forecast performance. The analyses are time and location sensitive, and take into account variations in GRIB data granularity. Mathematical techniques are used to clean the "noisy" data that is often encountered in ocean deployment, so as to provide consistent predictions that can be relied on for tactical decision making. The predictions are delivered to the user in numerical and graphical format, thus providing both very precise prediction information as well as easy-to-comprehend overviews that raise the user's understanding of the prediction.
- JL
- Contributor
Offline
- Registered: 6/30/2019
- Posts: 8
Re: How does Model Accuracy actually work?
Hi,
Interesting development I have been trying to get from other developers... without success! Which makes me very curious about your software I just discovered via the Expedition forum. Do you know whether you are (still!) the only ones to do this so far?
A few technical question:
1. Why use TWS and TWD and not Ground Wind data, which is what the forecasts provide?
2. What corrections do you apply to TWD and TWS before comparing to forecasts?
3. Is your correlation "static", simply cross-correlating (adjusted) boat measurements and forecasts along the boat course, at the time she was there? Or do you allow for time and space shifts as forecast may be accurate wind-wise but just offset in time by a few hours or in space by a few (tens of ) NM?
4. Do you have the ability to work with Ensemble data sets?
JL
Last edited by JL (6/30/2019 3:12 am)
- ZeeZee
- Expert
Offline
- Registered: 3/30/2016
- Posts: 321
Re: How does Model Accuracy actually work?
Good questions JL!
Looking forward to the answers.
- Model Accuracy
- Expert
Offline
- Registered: 3/01/2019
- Posts: 26
Re: How does Model Accuracy actually work?
We just made an explainer video above:
Please checkout the link.
I will respond in detail.
- •
Thread Starter- Model Accuracy
- Expert
Offline
- Registered: 3/01/2019
- Posts: 26
Re: How does Model Accuracy actually work?
So sorry in my delay here! Here are your answers:
1. Why use TWS and TWD and not Ground Wind data, which is what the forecasts provide?
On the GRIB file side of the equation, TWS and TWD are taken directly from the GRIB file at 10 meter height.
On the wind instrument side of the equation, TWS and TWD are moved from the height of the mast to 10 meter height using the formula available in Expedition user manual.
The analysis is of the forecast files to what was captured by the sailboat instruments.
2. What corrections do you apply to TWD and TWS before comparing to forecasts?
Just height correction, so knowing your mast height is critical. And we apply Magnetic variation as necessary to your data if it is not captured raw as True. This way it is apples to apples (ex: true to true) for forecast file to logged data for the comparison.
3. Is your correlation "static", simply cross-correlating (adjusted) boat measurements and forecasts along the boat course, at the time she was there?
Correct.
Or do you allow for time and space shifts as forecast may be accurate wind-wise but just offset in time by a few hours or in space by a few (tens of ) NM?
We are working on this. Currently you can easily see how the forecast may lead/lag a big wind shift (ex: frontal passage), but there is no time recommendation in the analysis. Our next version will most likely have that recommendation, just like it recommends increase/decrease TWS and left/right TWD.
4. Do you have the ability to work with Ensemble data sets?
Yea buddy! Just put all your gribs in the grib folder and let Model Accuracy sort them out. HOWEVER!, if the file names are the same for the GRIB file, you may need to put each file in its own seperate folder to make sure it is havin a fair fight in its evaluation and isnt overlooked. I recommend using this method of that folder structure for really similar named grib files.
So sorry in the delay! email is at support@modelaccuracy.com any time for a free screen share demo as well!
- •
Thread Starter- JL
- Contributor
Offline
- Registered: 6/30/2019
- Posts: 8
Re: How does Model Accuracy actually work?
Thanks for the answers.
Regarding the first point, it's a matter of definition and consistency. TWS and TWD, although named "True", are usually the wind components experienced by a boat that would be static on water, i.e., moving with any current that prevails. So TW includes the wind resulting from the movement of the boat due to the current.
Ground Wind is the data provided by weather forecast: this is the wind that would be felt at 10-m height by instruments that would be static on Earth, so without any effect of current.
Since we can calculate Ground Wind on a boat equipped with a GPS, I was wondering why you chose TW instead of GW.
This is really of interest in high current and/or low wind situations.
As for the rest, I really like the concept but find it difficult to work without the ability to shift in time and space. I sail in the Med, where forecasts notoriously challenging. They are becoming better but still tend to suffer a time lag and a spatial shift of some extent. Without accounting for this, it would be dicy to bet on a forecast because it seems to do a better job "here and now" over a given time frame on the boat's course. It will be interesting to see how you handle these shifts.
Looking forward to these developments.
- Model Accuracy
- Expert
Offline
- Registered: 3/01/2019
- Posts: 26
Re: How does Model Accuracy actually work?
Regarding the first point, it's a matter of definition and consistency. TWS and TWD, although named "True", are usually the wind components experienced by a boat that would be static on water, i.e., moving with any current that prevails. So TW includes the wind resulting from the movement of the boat due to the current.
Ground Wind is the data provided by weather forecast: this is the wind that would be felt at 10-m height by instruments that would be static on Earth, so without any effect of current.
Since we can calculate Ground Wind on a boat equipped with a GPS, I was wondering why you chose TW instead of GW.
This is really of interest in high current and/or low wind situations.
I am tracking. Makes sense. This is actually the first I have ever heard of ground wind terminology. We operate with the TWS and TWD that is provided within the Expedition log files, that is it. We are reliant on the math that produces that number within the log file.
As for the rest, I really like the concept but find it difficult to work without the ability to shift in time and space. I sail in the Med, where forecasts notoriously challenging. They are becoming better but still tend to suffer a time lag and a spatial shift of some extent. Without accounting for this, it would be dicy to bet on a forecast because it seems to do a better job "here and now" over a given time frame on the boat's course. It will be interesting to see how you handle these shifts.
So the next versions will provide that time shift recommendation. But, currently, with the graphical analysis for TWS and TWD you can see very clearly the lag behind on a macro trend, and then adjust within Expedition routing software accordingly. All you yould have to do is interpret the time scale on the x axis of the graphs. We are working on a forml fwd/backward recommendation time value.
To better understand this functionality I encourage you to download the software and play around with the test data that is installed in the free version. If you decide to buy the software there are more test set files on the website for this very purpose. Also, give some of our testimonials a review, you can see a top level of navigators are using the software right now.
Looking forward to these developments.
Awesome! We are by navigators for navigators, and hope to create value for your and your team!
- •
Thread Starter- ZeeZee
- Expert
Offline
- Registered: 3/30/2016
- Posts: 321
Re: How does Model Accuracy actually work?
Thanks for the response.
I'm with JL: using GWD and GWS instead of TWD and TWS is essential for Model Accuracy to be useful in areas with currents.
GWD/GWS are not recorded in the Expedition log files, but all the data you need to calculate them is there: Set, Drift (I think Leeway is already taken care for in the TWS/TWD of Expedition?). I'm sure Nick can help you which formulas to use in order to make sure they match up with how Expedition makes these calculations.
Great animation video to show the benefits of MA!!
- Nick
- Moderator
Offline
- Registered: 9/16/2015
- Posts: 2,980
Re: How does Model Accuracy actually work?
I need to redo the log files and Expedition boat and weather boat channels as it has all expanded beyond the original Expedition usage. Some Expedition applications have over 100 boats mapped. Ground wind, course etc not being logged as Exp can easily calculate is just one aspect.
The trick is doing it without breaking things for everyone.