You find it here:
https://www.bkwine.com/our-books/biodynamic-organic-and-natural-winemaking-sustainable-viticulture-and-viniculture/buy-the-book/

I am interested in getting all your publications regarding organic vineyards. Are you emailing back these books?

No need to apologise Alan. It is another of my pet subjects. 🙂

I have for a long time been thinking of writing up how I do it.

And yes, I agree with you, it is closely coupled.

Data volume is a daunting challenge. At one place I worked we had a reciprocal arrangement wit a nearby institution and we each hosted the second head of each other’s backup systems, joined by a dedicated fibre. We also investigated point-to-point wireless and that would have worked too. The initial backup was done locally and subsequent deltas were quite modest.
I recall the first time I heard about a LAN and was blown away that they communicated at “ten million bits” per second, i.e. 10mbs. It seemed like a lot.
I’m a bit out of the game but am curious why you are using a NAS instead of a fibre-channel SAN? I know SANs were crazy expensive and maybe the performance benefits no longer exist.
Good luck with managing all this. Better you than me!
Anyhow, apologies for discussing backups on a DAM blog but the two things seem increasingly intertwined.

Thanks for the nice feedback and the very comprehensive info!

In order to optimize working with NAS devices, we moved part of the cache files in Daminion on a server-side.
You don’t need to download large files from NAS just to preview them. Do it only if you need to get an original file.

Daminion can generate and store medium res JPEGs (for images), full-res JPEGs for PSD/TIFF/BMP/PSB, and medium res MPGs (for videos) on a server-side disk.

You need 20x times less disk space for preview storage than for storing originals on NAS disks. And the overall preview performance is much faster!

But its no longer stills which are the primary issue, the big issue data wise for us is the video we now produce, and TBH this make the issues with stills look like kids stuff. Whilst we are still managing in-house around 300,000 still images, we are also producing 2-3 corporate video projects a week, which are averaging around 1.5tb of data per project.

We have a in-house NAS system now which can accommodate about 60tb of data, and we then back this up to HD and remove copies off site for safe keeping.

So the big issues we are now dealing with, and which folks might want to think about when dealing with DAM going forward, as the volumes of data increase, are for us…

1) Network speeds; This has gone from being a non-issue to a major issue… moving data and projects around the network is a major barrier to productivity. 10gb network capability is now absolutely essential… and this needs to be complemented by NAS capabilities to write at good sustained speeds as well…. so going forward folks really should be investing in 10gb switches, cabling, NAS and network cards for their machines… costs are falling so this isn’t so unrealistic.

One quick tip… DO NOT buy a NAS which is not 10gb capable.

2) Back up speeds… writing data to external drives for removal can take an enormous amount of time with a USB connection, so again ensure you are investing in machines that have fast external connections like thunderbolt three… and then if your getting drives to back up to, ensure they can use that speed. (Desktop drives with USB-c etc in a housing are cheaper per TB than bare 3.5 inch HD’s… )

3) External internet connection speed… In reality, the whole backing up to physical drives and taking them off site is becoming an impossibility, its costly, with having to buy in 4tb of external drives now per week, and then taking the time to deal with that…. and its taking up a not inconsequential amount of space off site as well… so the only real answer we feel is to have automated backup from the NAS to a high quality cloud facility… so upload speeds are crucial… residential or SME connections based on FTTC tech are not fast enough, With our current load of data I calculated out it would take a year to back up with our existing link… Folks need to be looking for now a lease line connection, and we are commissioning a 1gb up and down dedicated line next month.

4) Data expiration; In reality for most small businesses such as ours… being responsible for the long term archival storage of all of this data is non-sensical. As such we are now expiring large quantities of data on a monthly basis… we only guarantee a 1 year archival solution for customers now. We are all far to beholden to large quantities of in reality probably useless data with zero value, and unless we as photographers and video guys start to cull this stuff, its just going to eat us alive. So we need to cull more. Granny did this when she had her photos mounted in albums…. we need to be less precious about the data.

We have gone from needing a simple and robust method of organising our data in 2013, to in reality a far more complex set of needs that stems from the shear volume of data now being generated by modern cameras, and modern photography (as we move into video)…. and to keep ahead of that we need to expand our thinking way beyond just the app we use to manage the images, to how we deal with that body of data in the round…

In reality we need much much faster network speeds both in house and out, combined with a robust cloud based system which includes DAM functions working off a file based storage facility and this backed up by sound data expiration policies as well as the passing of the risk of archiving to customers.

Adios!

I exaggerated, Per. It would only take about two solid weeks to upload those files if I got the fastest available fibre connection. But I don’t plan to pay for that or for an adobe subscription or for four TB of storage so it’s off the table for me. It may be a good solution for others but not me.
I agree that it would make a good, if costly, offsite backup but think it’s a mistake to rely on the cloud as primary storage.

Agree with you Roy. We have four local copies (of which one is off-site) and two cloud based.

Local:
1. Nas file server
2. Nas Raid 1 copy (automatic)
3. USB attached HD copy of #1 (automatic)
4. Periodically swapped out #3 and put in bank vault (manual)

Cloud:
1. Amazon (manual)
2. Photoshelter (manual)

Traveling, updating 5 gb worth of important cloud resident files. The folder containing the files was actually called “IMPORTANT”. One day could not open the folder. Called, provider could not open the folder. No backup at the provider of a functioning folder. No most current local backup at my level as I was too busy with changing locations almost daily.

The cloud is only for convenience. It is the very last data to update. All data should originate locally, be backed up locally and the cloud can be updated whenever. Never works fine.

I was at a Microsoft Developer’s conference about 5 years ago and I was astonished at how many had lost cloud stored data. Mine was Apple (probably hosted by Google), I would imagine many, if not most, of the developers had MS cloud storage.

YMMV