Don't use these yet

Comments/Suggestions/Feature Requests/Questions about general usage

Moderators: Steve Cranage, Robert Plaster

Post Reply
User avatar
Steve Cranage
Posts: 11
Joined: Wed Jul 21, 2021 10:56 pm
Location: Colorado Springs, CO
Contact:

Don't use these yet

Post by Steve Cranage »

There are a number of things you will see in the community release that lay somewhere between 'not yet implemented' to 'not yet adequately tested' here is the list as it stands today:
  • Stay away from freespace policy. We did a very simple implementation, and it worked when we tested it, but we barely tested it. All it does is search the file system tables for candidates to purge based on size and access time and then runs sms_purge -f filename. The rewrite is in my corner, the new version will be a new dmi macro and will include in the form a 'test' button so the user can see how much space will be freed by the policy. We promise to get back to this one soon, but feel free to keep reminding us. We really need external input to know how things need to be prioritized.
  • Site exit policy doesn't yet exist. Our plan was that if a file changed or was created that matched some criteria, we do some action. We were thinking along the lines of a Business Process Management environment as an example, but we never did anything beyond that. If you have requirements, let us know and we'll discuss
  • Stay away from anything to do with exports. We built this a couple years ago now, and it was really cool, we showed it off at SC19 in Denver. But 'it's been neglected since then and we need to revisit it before anyone tries to use it again. Basically you can take any volumeset, like a ceph resident one for example and copy it to an 'export' volumeset, that goes to tape or any other removable media types, and prints a QR label to stick to the tape(s). You can then take the media to another system running the CDS, and read the QR code to get the file listing, or auto import to the local CDS. The tapes are standard label, so they support versions, and can be read with our standalone tape readers if there isn't a DS instance running where they are needed.

    Our plan is to move this over to dmi, and change the metaphor for loading into a dropbox so you can do the file system synthesis into the export drop box, and chose to share by S3 or export to removable. The list of files can span up to 10000 volumes.
UPDATE on exports, Kim is removing these menus from the UI that will be uploaded next week, That will be the version with test and simulation support. We'll get it back up soon when it's in a better place.
  • Metadata replication and cache policy were both created over 10 years ago and have been dormant since. They both need testing before being promoted back into a production release, and there is another federation policy that also was working, but fragile so it's not in the UI. They will all be addressed, but not till we figure out how to add staffing.
  • S3 policy is also very basic in its current form, it is just a proof of concept really. We need to add full lifecycle management, but that is another task awaiting additional staffing.
  • Erasure coding is all done, we think, but has not been tested beyond what Dan did in writing this layer. The way we implemented EC is as a volumeset attribute, so it is highly granular, you define your EC profile (it can be anything supported by the Mellanox ConnectX-4 and above cards), then add the EC profile to the volumset definition. This is a pretty high priority for us right now, but try to be patient.
UPDATE on EC, Kim is taking this out of the UI as well, it will be back soon as we can finish testing.
Steve Cranage
Principal Architect, Co-Founder
DeepSpace Storage

Post Reply