Script to initialize an SD card for MiST, SiDi and SiDi128 FPGA retro board

This script initializes or updates the SD cards for either a MiST, SiDi or SiDi128 FPGA retro system.
It tries to create a complete as possible collection of cores and their required ROM files to get an out-of-the-box working SD card.
There are seperate scripts for Linux and Windows hosts.
With PowerShell installed on a Linux system the genSD.ps1 script is executable on a Linux system too.

This script collects the work of many other contributors (FPGA, Hardware, Software, ...) to generate a 'distribution' for the systems mentioned above.
Thanks to all these contributors for their work (see links to repositories).

After executing this script appr.

	MiST	SiDi	SiDi128
.rbf	~260	~220	~200
.arc	~1340	~1220	~1250
.rom	~1380	~1260	~1290
.ram	~390	~330	~350

files are collected/generated.

Preconditions

Get the scripts by

downloading as .zip
downloading as .tar

clone this repo

git clone https://github.com/gcopoix/makeSD_mist.git

The SD card must be

at least 32GB
formatted with FAT, FAT32 or exFAT (preferred!)

or use a folder on HDD with the same available disk space.

Note: As the .mra parsing and download of the MAME ROMs has changed significantly, please delete your repos/mame cache folder before executing this script to make use of the enhanced MRA ROM download.

Usage

simply call the script with minor parameters:

genSD [-s <mist|sidi|sidi128>] [-d <destination SD drive or folder>]

with

option	description	example
-s	Target system (mist, sidi or sidi128). This parameter is mandatory	-s mist -s sidi -s sidi128
-d	Destination folder the target folders and files are generated. If this option isn't specified, `SD\sidi` will be used by default.	-d /media/SD-Card -d E:
-h	Show some help text.	-h

Examples

Initialize SD card for MiST, SD card in drive E: (Windows)
```
genSD -s mist -d E:
```
Initialize SD card for MiST, SD card mounted to /media/SD-Card (Linux):
```
<path-to-script/>genSD.sh -s mist -d /media/SD-Card
```
Initialize SD card for SiDi, SD card in drive E: (Windows)
```
genSD -s sidi -d E:
```
Initialize SD card for SiDi128, SD card in drive E: (Windows)
```
genSD -s sidi128 -d E:
```
Initialize SD card for MiST, create in subfolder (Windows)
```
genSD -s mist
```
Initialize SD card for MiST, create in subfolder (Linux)
```
<path-to-script>/genSD.sh -s mist
```

What is created

This script fills/updates an SD card or folder with

the Computer/Console/Arcade Cores (organized in subfolders)
the required ROMs and some optional sample Game/HDD files
ARM Firmware update file
default mist.ini

to get an SD card which can be directly plugged into MiST, SiDi or SiDi128.

The generated folder structure:

├── Arcade
│   ├── Gehstock
│   │   ├── ...
│   │   └── ...
│   ├── Jotego
│   │   ├── ...
│   │   └── ...
│   ├── ...
│   └── ...
│
├── Computer
│   ├── Amiga
│   ├── Amstrad CPC
│   ├── Atari ST
│   ├── C64
│   ├── ...
│   └── ...
│
├── Console
│   ├── Atari 2600
│   ├── Astrocade
│   ├── Nintendo NES
│   ├── ...
│   └── ...
│
├── <subfolders unfortunately required by some cores in root folder>
├── ...
│
├── <additional files unfortunately required by some cores in root folder>
├── ...
│
├── core.rbf      # <- menu core (named sidi128.rbf on SiDi128)
└── firmware.upg  # latest ARM firmware update file

The Arcade cores are installed with their required ROM files.
The information about the Arcade cores/roms is parsed from the .mra files (e.g. for jotego's cores) and converted to .arc files together with the created .rom (and .ram) file.

Cache folders

During installation, several temporary folders are created in the current folder of the genSD script:

├── repos
│   ├── jotego           # Jotego repository
│   ├── mame             # cache folder for MAME Arcade ROM files
│   ├── MiST
│   │   ├── binaries     # MiST core repository
│   │   ├── gehstock     # Gehstock MiST core repository
│   │   └── sorgelig     # Sorgelig MiST core repositories
│   ├── SiDi
│   │   ├── ManuFerHi    # ManuFerHi SiDi core repository
│   │   └── eubrunosilva # eubrunosilva SiDi core repository
│   └── misc             # miscellaneous core support files, e.g. game archives, ...
│       ├── Atari 2600
│       ├── Next186
│       └── ...
└── tools                # required tools for executing this script

It is a good idea to keep these folders as the next run will be much faster (git only needs to update and not clone, the download of the MAME ROMs and the other miscellaneous files can be skipped, ...).
This saves more than 10GB of data not being fetched again.

Developing

The project contains a workspace file for Visual Studio Code with pre-configured

plugin recommendations
- Bash Debugger
- PowerShell Debugger
script debug/launch settings

Best practice here is to simply open the project by double-click on the genSD.code-workspace file.
VisualStudio Code will open the project and install/configure the required plugins.

For testing/debugging specific cores, please refer to some test code left disabled in the scripts.

A configuration file for PowerShell ScriptAnalyzer is provided and used by both Visual Studio Code and the Windows test.bat script

Testing

There are test scripts to be executed with Linux and Windows.
These scripts will

do some script checking
generate a destination set for
- mist
- sidi
- sidi128
with each 2 runs:
- 1st: run with empty cache (=initial run)
- 2nd: run with cache and SD (=update run)

Both folders should contain the same content at the end (assuming no git or source repo update in between).
Additionally, a full log for each set is created in the 'SD#' folder with some summary at the end:

Missing core .rbf files
Missing MAME ROMs
MAME ROMs with wrong checksum
MAME ROMs with missing parts
some statistics about the number of .rbf, .arc and .rom files

If executing the test.sh script with Linux on an exFAT formatted destination drive, please execute this script with sudo privileges (sudo ./test.sh).
See Known Issues for details (the test execution would break and ask for the sudo password during test run otherwise).

Additionally it is always a good idea to compare the Linux and Windows generated sets and logs.
Please keep in mind that each test execution will consume about 50GB of HDD space (the Linux version even twice as it will test the PowerShell script too) - not talking about the test run time (several hours)

Known issues

exFAT and DOS attributes (Linux version only)
MiST, SiDi and SiDi128 can read FAT, FAT32 or exFAT (since firmware_210525) fomatted SD cards.
The menu core uses the
- System attribute (to show subfolders) and
- Hidden attribute (to hide cores from menu).
Linux can write these attributes with fatattr tool on FAT or FAT32 drives and SD cards, but for exFAT formatted SD cards root privileges are required.
To support the DOS attributes correctly on exFAT formated target devices the script
- will ask for the sudo password or
- needs to be called with sudo, e.g.
```
sudo genSD.sh [options]
```
The windows Powershell script executed on a Windows system doesn't have this issue.
exFAT SD card, but MiST/SiDi firmware version too old
If the firmware version of your system is older than 210525 (without support for exFAT drives) means you can't read an exFAT formated SD card.
This can be solved by
1. format an SD card with FAT or FAT32 filesystem
2. copy core.rbf and firmware.upg (>= 210525) to the root of this SD card
3. boot your system with this card
4. update the firmware from the (menu) core
Now your system is ready for exFAT formatted SD cards.
executing the script directly from SD card
Executing the script directly from an SD card (script located on the SD card) only works if the SD card is formated using exFAT file system.
With an FAT/FAT32 formated drive the script will fail (the size of the jtbin repository (appr. 7GB) exceeds the max. file size for an FAT/FAT32 formatted card (4GB)).
And (if executed on a Linux host) the executable flag of the script and the downloaded mra tool would be missing (Permission denied" error).
.mra parsing of ROM files
The ROM file names parsed from the .mra files refer a MAME version. But unfortunetly many ROMS, if fetched from their referred MAME version, don't match.
I tried to find a best matching set of download URLs incl. some extra handling and special downloads, but for some ROM archives mra still complains: \
- ROMs not found:
  - kchamp2p.zip
  - timescan3.zip
- ROMs found, but with MD5 mismatch or missing parts:
  - btime.zip
  - journey.zip
  - xevious.zip
  - clubpacm.rom
  - combh.zip
  - wbml.zip
  - lottofun.rom
  - spdball.zip
I would appreciate any ideas to improve/fix this.
Missing Jotego Cores
Some .mra files in the Jotego repository refer missing .rbf files in the mist, sidi or sidi128 folders. The reason for the missing cores are
- insufficient target board FPGA ressources and
- different development status

Ideas for improvements

more ROMs
As this is an early version, there are lots of ROMs and Games to add/fix - please give me a hint or pull request.
I think here we have a structured base to improve in the future.
files and folders required in root folder
Many cores require special files and folders in the root of the SD-Card for their ROM/Game/... files. This makes in my opinion the folder structure a bit messy, especially if we want to have a full core distribution.
I would recommend the default root folder of a running core is by default the folder of the core, what would make a modular setup of the SD card much easier.
May be somebody (or I myself) will find the time to introduce this feature in the ARM firmware.
mist.ini
Currently the script simply uses the default mist.ini from the main repository.
Generating a configuration with optimal settings for each core would be a nice additional feature for this scripts.
Jotego provides an extended mist.ini file for his cores.
MiSTer support
Need to check the typical MiSTer setup and align with this script.
Target systems of this script are MiST, SiDi and SiDi128 (cheaper than MiSTer).

It would be nice if all cores would be built for both MiST, SiDi and SiDi128 (as the hardware features are nearly identical).
SiDi128 has even more resources so is be able to run all cores a MiST or SiDi does (and even more).

Thanks here to Jotego for his jtbin Arcade repository providing releases for multiple FPGA platforms (depending on FPGA ressources).

Name		Name	Last commit message	Last commit date
.vscode		.vscode
Linux		Linux
Windows		Windows
test		test
.gitignore		.gitignore
LICENSE.txt		LICENSE.txt
README.md		README.md
genSD.code-workspace		genSD.code-workspace

Navigation Menu

Repository files navigation

Script to initialize an SD card for MiST, SiDi and SiDi128 FPGA retro board

Preconditions

Usage

Examples

What is created

Cache folders

Developing

Testing

Known issues

Ideas for improvements

Links

MiST repositories

SiDi/SiDi128 repositories

General repositories

Alternatives to this repository

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Script to initialize an SD card for MiST, SiDi and SiDi128 FPGA retro board

Preconditions

Usage

Examples

What is created

Cache folders

Developing

Testing

Known issues

Ideas for improvements

Links

MiST repositories

SiDi/SiDi128 repositories

General repositories

Alternatives to this repository

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages