Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSFS 2024 #1198

Open
albar965 opened this issue Oct 17, 2024 · 17 comments
Open

MSFS 2024 #1198

albar965 opened this issue Oct 17, 2024 · 17 comments

Comments

@albar965
Copy link
Owner

albar965 commented Oct 17, 2024

The petition was removed and the discussion now continues in the developer forum: https://devsupport.flightsimulator.com/t/provide-readable-airport-data-for-little-navmap/11926

Petition for readable information and documentation is here https://forums.flightsimulator.com/t/provide-readable-airport-data-for-little-navmap/672891

The Store path appears to be: %localappdata%Packages\Microsoft.Limitless_8wekyb3d8bbwe\LocalCache\UserCfg.opt
The Steam path appears to be: %appdata%\Microsoft Flight Simulator 2024\UserCfg.opt

SDK Documentation

https://docs.google.com/document/d/e/2PACX-1vRGZ2lfaCLbtruuUi03QnHTsNHp1BUFYCCNKYHxaQTi6ducqGG1Hh6dIZ0KiRxrZN459GKsZMr3-Hp5/pub

Probably APX files (containing the airports) in
C:\Users\<username>\AppData\Local\Packages\Microsoft.Limitless_8wekyb3d8bbwe\LocalState\VFSProjection\scenery

  • Steam: %APPDATA%\Microsoft Flight Simulator 2024\StreamedPackages
  • MS Store: %LOCALAPPDATA%\Packages\Microsoft.Limitless_8wekyb3d8bbwe\LocalState\StreamedPackages
@albar965 albar965 added this to the Release 3.0.12 milestone Oct 17, 2024
@albar965 albar965 self-assigned this Oct 17, 2024
@albar965
Copy link
Owner Author

Discussion and more information here: https://www.avsim.com/forums/topic/655517-little-navmap-support-for-msfs-2024/

@tracernz
Copy link

Both MSFS2020 and MSFS2024 provide a SimConnect facilities API. This will give information even from marketplace packages.
MSFS2020: https://docs.flightsimulator.com/html/Programming_Tools/SimConnect/API_Reference/Facilities/SimConnect_AddToFacilityDefinition.htm
MSFS2024: https://docs.flightsimulator.com/msfs2024/html/6_Programming_APIs/SimConnect/API_Reference/Facilities/SimConnect_AddToFacilityDefinition.htm

You can lookup facilities on demand (e.g. when the user types an ident into a search field) https://docs.flightsimulator.com/html/Programming_Tools/SimConnect/API_Reference/Facilities/SimConnect_RequestFacilityData_EX1.htm, or subscribe to discover all of the facilities in the reality bubble around the user https://docs.flightsimulator.com/html/Programming_Tools/SimConnect/API_Reference/Facilities/SimConnect_SubscribeToFacilities_EX1.htm.

The downside is that it doesn't fit with your current architecture.

@albar965
Copy link
Owner Author

Thanks for the links. Did not run into this so far. At least one can get the most important information through it.
Lookup on demand does not help since it would be a complete rewrite which will definitely not happen.

So I'd have to call SimConnect_RequestFacilitesList to get all airports from the cache. I suppose there is no limitation of the number of airports.
Then for each airport SimConnect_AddToFacilityDefinition and SimConnect_RequestFacilityData.
Get information and add to the LNM SQLite database. Adapt all references to airports from procedures and navaids.

Thinking how fast you can get 40000 airports through this interface.

Still have to read the navdata BGLs to get navaids outside the bubble.

The downside is that it doesn't fit with your current architecture.

Yeh. I'm not sure if I'd like to continue to jump through hoops for such ... design decisions. Only one programmer here and this can take months to implement. In the meantime everybody uses SimBrief and the internal planner. 🤷‍♂️

Look at X-Plane, which uses a text format for all data that is normally slow, but still loads faster than any MSFS. And their format is even officially documented.

Alex

@albar965
Copy link
Owner Author

@tracernz Sorry for frustration. Not aimed at you.
The links you posted look like the best API so far to do this albeit quite some effort to implement.

I'd read navaids and stuff as usual from the BGL files since these are limited by the bubble and only fetch airports using this interface. I have to do a prototype to test if performance is ok.

Good thing is that this (hopefully) allows to access all the encrypted add-on airports and does not need any merging of airports with delete records. Anyway big effort to add this.

@tracernz
Copy link

No worries, I fully understand. 👍

@nabeelio
Copy link

nabeelio commented Dec 3, 2024

@albar965 matching my findings too, I read the scenery for my acars application. Good luck to us all. Requesting it directly from the sim is untenable, doesn't look like there's any geometry functionality either

@albar965
Copy link
Owner Author

albar965 commented Dec 3, 2024

I read the scenery for my acars application. Good luck to us all. Requesting it directly from the sim is untenable, doesn't look like there's any geometry functionality either

Reading airports including all procedures from SimConnect is doable and fast but does not provide aprons.

I already have half the implementation and can read all the airports, runways, parking, taxi, SID, STAR and approaches but I'm not ready yet to commit this to atools. I'm using the suggested functions by @tracernz above (HUGE thank you for the hint!).

This is a big effort since I practically have to write a new data compiler.

See here for all you can fetch from SimConnect: https://docs.flightsimulator.com/msfs2024/html/6_Programming_APIs/SimConnect/API_Reference/Facilities/SimConnect_AddToFacilityDefinition.htm#airport

And now I'm fighting with the BGL changes for navaid objects like waypoints, VOR and NDB. It seems that idents and names are stored using a different encoding. As usual all undocumented. ☹️

It's bizarre that I can read worldwide airports from SimConnect but no navaids. Navaids are limited by a 200 NM radius around the user aircraft. 🤦‍♂️

Alex

@nabeelio
Copy link

nabeelio commented Dec 3, 2024

I already have half the implementation and can read all the airports, runways, parking, taxi, SID, STAR and approaches but I'm not ready yet to commit this to atools. I'm using the suggested functions by @tracernz above (HUGE thank you for the hint!).

Nice! I'm planning to use those same functions, thanks @tracernz! My task is much simpler than yours, luckily. I think I can get away with using SimConnect almost entirely

And now I'm fighting with the BGL changes for navaid objects like waypoints, VOR and NDB. It seems that idents and names are stored using a different encoding. As usual all undocumented. ☹️

Oof, I was never good at decoding that type of stuff anyway, so good luck. It was so painful for me figuring out the 2020 formats.

It's bizarre that I can read worldwide airports from SimConnect but no navaids. Navaids are limited by a 200 NM radius around the user aircraft. 🤦‍♂️

This is possibily because of some licensing or other restrictions because of their data source, can't extract it all and use it for some other application(s)

@albar965
Copy link
Owner Author

albar965 commented Dec 3, 2024

Oof, I was never good at decoding that type of stuff anyway, so good luck. It was so painful for me figuring out the 2020 formats.

Neither am I. Very time consuming, therefore.

Do not hesitate to ask. I can always answer a question about BGL or point you to the right place in my code if needed.

Alex

@nabeelio
Copy link

nabeelio commented Dec 3, 2024

Thanks! Same goes for you.
Your code has been a great reference in the past. Very clean and easy to read.

@leppie
Copy link

leppie commented Dec 5, 2024

It seems that idents and names are stored using a different encoding.

Maybe similar to how they do it in SPB's ?

See https://github.com/leppie/spb2xml/blob/master/TextDecode.cs (and the .Data.cs file).

IIRC I just threw all the ANSI chars into a field (ie ...a-zA-Z0-9...), then ran the SPB compiler, got the mapping, then rotate by 1, and repeat 256 times. Viola, perfect lookup table for decoding.

I would not be surprised if it was the same encoding. I doubt they would use encryption.

Neither am I.

You can use the same approach to bruteforce the lookups out.

@albar965
Copy link
Owner Author

albar965 commented Dec 5, 2024

I found out that the names in the BGL files are encoded like described here: ICAO Identifiers and region codes.
But using a datatype long long (64 bit) and a bit shift of 6 instead of 5 to get space for eight characters.

I'm making good progress here. 🙂

@cowreth
Copy link

cowreth commented Dec 17, 2024

Hi Friends,

Alex, the original petition on the forums has been removed, which is a great news (coupled with the mention of this in the dev roadmap). I originally went to the forums to see if any communication was made on the point, but had to fallback here to ask : do you feel like the solution being built will be accessible to other devs ? Or will it be a private endpoint ?

You may be under NDA. If so, it's okay to just ignore the question ;)

Cheers,
Seb

@Maeyanie
Copy link

Maeyanie commented Dec 17, 2024

Given this is a GPL3-licensed project, if code using the endpoint weren't permitted to be shared, it wouldn't be allowed for LittleNavMap to use it.

@albar965
Copy link
Owner Author

Asobo won't develop a private endpoint. I wonder if they'll even add the missing features to the current SimConnect. I hope they do.

About progress in LNM for now:
I can now get all airport information via SimConnect. This includes airports, runways, taxiways, parking, start positions, helipads, SID, STAR and approaches. You have to start the simulator to allow LNM to load the information but the process looks stable and sufficiently fast here.

I have to read navaids from the BGL files where I also could decode all changes now. This is waypoints, VOR, NDB, ILS and airways. Hope these files do not disappear in the future. This would be a deal breaker.

The code is not ready yet to release. Still have to fix bugs.

Missing features so far: Runway lighting status (needed for night operations), airport closed status, add-on status, aprons and airspace boundaries. Airport boundaries disappeared with the last MSFS 2024 update. I asked in the dev forum about it (post here). I do not see these as show stoppers.

Also have to adapt the PLN export to the new MSFS24 format. Meh.

@Maeyanie
Copy link

Hoping for the best here too.

If they do remove the BGL entirely, would it be workable to do a hybrid approach of loading from mostly from SimConnect, and Navigraph files for navaids? That's definitely less optimal since not everyone has them, but it seems better than nothing.

@albar965
Copy link
Owner Author

If they do remove the BGL entirely, would it be workable to do a hybrid approach of loading from mostly from SimConnect, and Navigraph files for navaids?

Currently there is a limitation to fetch navaids only 200 NM around the user aircraft through SimConnect. It's simply not doable if this limitation is not lifted.

You can always switch navaids to the Navigraph database and use the MSFS airports that were loaded through SimConnect. But this needs a Navigraph subscription.

This is how it looks now using MSFS 2024:

SID, STAR, ILS and approaches:
Screenshot 2024-12-17 214159

Airways and navaids:
Screenshot 2024-12-17 214446

Gates and taxiways but no aprons at EDDF:
Screenshot 2024-12-17 223542

EDDF overview:
Screenshot 2024-12-17 223645

Alex

albar965 added a commit to albar965/atools that referenced this issue Dec 25, 2024
Location to read navaid BGL files are
...\AppData\Local\Packages\Microsoft.Limitless_8wekyb3d8bbwe\LocalState\StreamedPackages
...\AppData\Roaming\Microsoft Flight Simulator 2024\LocalState\StreamedPackages
albar965/littlenavmap#1198
albar965 added a commit to albar965/atools that referenced this issue Dec 25, 2024
albar965 added a commit to albar965/atools that referenced this issue Dec 25, 2024
This commit shows all required changes to read MSFS 2024 navdata from BGL files. It does not include
refactoring, SimConnect changes and changes to the database writing code.

It does not cover airports and procedures which are read through SimConnect using SimConnectLoader
and SimConnectWriter classes.

Added new types and records and adapted code to changed structures.
Fixed issue where extra ramps and gates were not recognized as gates or ramps.
albar965/littlenavmap#1198
albar965 added a commit to albar965/atools that referenced this issue Dec 25, 2024
albar965 added a commit to albar965/atools that referenced this issue Dec 25, 2024
albar965 added a commit to albar965/atools that referenced this issue Dec 25, 2024
…2024 SimConnect

This fetches airports, runways, starts, parking taxiways and all procedures via SimConnect and
writes them to the database.

albar965/littlenavmap#1198
albar965 added a commit that referenced this issue Dec 25, 2024
Now using MSFS 2024 SimConnect to load airport data.
Added MSFS 2024 in general.
Using different display name for MSFS 2024 and 2020
Adjusted database mode checking for better hints and error messages.

#1198
albar965 added a commit to albar965/navdatareader that referenced this issue Dec 25, 2024
Now loading airports using SimConnect
albar965/littlenavmap#1198
albar965 added a commit to albar965/atools that referenced this issue Dec 25, 2024
albar965 added a commit that referenced this issue Dec 25, 2024
albar965 added a commit to albar965/littlenavconnect that referenced this issue Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants