Basics of DMI and the scripting

The Language/Interpreter/Test and Simulation

Moderators: Steve Cranage, Robert Plaster

Post Reply
User avatar
Steve Cranage
Posts: 11
Joined: Wed Jul 21, 2021 10:56 pm
Location: Colorado Springs, CO
Contact:

Basics of DMI and the scripting

Post by Steve Cranage »

Last January I had the task of creating a test environment for SMS. I wanted it to be flexible enough to perform both synthetic benchmarks (the type that simulate real work) as well as profiling benchmarks (these separate workload by process for optimization purposes).

I also wanted a framework for performing data management tasks, so for example things could be crontab'd. Later Kimberly came to me and said she had a problem with remote execution and progress bars, salt just didn't support unbuffered returns, so progress indications weren't doable.

So I ended up wrapping all 3 requirements into a new subsystem I call dmi, for data management interpreter. It's a bit of perl code that uses salt for remote execution. It also creates its own socket back to the calling process for progress bar purposes.
dmi.PNG
dmi.PNG (16.93 KiB) Viewed 85750 times
The way this now works is the UI presents a form for the function to be performed, like doing a baseline operation on a file system. So the form is filled out by the user indicating what to run baseline on, then the UI saves a copy of the baseline command options in XML to a public directory we maintain. Then dmi is called locally with a remote execution flag -r hostname telling it to bind a port and print everything received on the port to stdout. The dmi interpreter running on the master issues the salt run.cmd to the remote minion with -c flag telling it to direct it's output to the master port.

So take my simple baseline example, the form looks like this:
baseline_form.PNG
baseline_form.PNG (101.96 KiB) Viewed 85750 times
And the xml file generated looks like this:
baseline_xml.PNG
baseline_xml.PNG (21.22 KiB) Viewed 85750 times
This is now the framework for all sorts of data management tasks, like transcribing a volumeset from disk to tape, creating and running simulation workloads, and all sorts of other stuff that has yet to be done.

I could really use help on maintenance and future features for dmi, if any perl programmers have some spare time and want to help, please LMK.

BTW, due to resource constraints, the version of dmi distributed is the minimum functions for a couple of features where it is really needed in the UI for real time progress updates, the code base I'm holding back for now is much larger.
Steve Cranage
Principal Architect, Co-Founder
DeepSpace Storage

Post Reply