Netscraps

A windsurfing, CSS-grudging, IE-hating, web-developing, gigantic-machine-puzzling blog

Month: March 2016

MediaTemple TrueSpeed CDN Control Panel Is Not What You Think

This is MediaTemple’s sales page for their TrueSpeed™ CDN, which is a rebranded product operated by SiteLock.com:

MediaTemple TrueSpeed CDN Sales Page

Sounds great & what a deal, right? I thought so. Note the “integrated control panel”. Here’s what you actually get for the control panel:

Only having on/off & purge-all controls is not the right way to manage a CDN, in the same way that driving using only the ignition switch & yelling “get the fuck out!” to all passengers is not the right way to drive a bus.

Hopefully this guy from the TrueSpeed CDN sales page is designing a new control panel.

This guy on MediaTemple’s CDN sales page is no doubt hard at work designing a real control panel.

With MediaTemple’s TrueSpeed CDN there are no traffic stats or reporting, & no asset purge controls except for all-or-nothing.

I had been using Edgecast ProCDN (through MediaTemple) which has a real control panel with nice purge controls & very detailed traffic stats/reporting.

If you sign up for the same CDN through SiteLock.com & not via MediaTemple, you get a much more capable control panel with all the features you’d expect for managing a CDN, but you’ll probably pay more than $30/month.

I think MediaTemple’s TrueSpeed CDN is still a good deal & no doubt they have plenty of customers who don’t need CDN stats or any real controls.

But MediaTemple should be far more upfront about what customers actually get for the “integrated control panel”. Maybe they should call it what it is: “integrated on/off/purge buttons”.

In lieu of MediaTemple being upfront about it, now you know!

Migrating MediaTemple’s GridServer (GS) to Dedicated Virtual (DV) VPS

I recently moved lots of websites off MediaTemple’s GridServer (gs) platform to their Dedicated Virtual (dv) platform. I’ve kind of abused Grid Server for the past 12 years, but finally the overage fees caught up.

I went with the Plesk 12.5/CentOS6 hosting option one year later, I upgraded DV accounts & did this all over again for Plesk Onyx 17/CentOS7. The standalone DV server was tempting but I don’t quite know enough about Linux admin to go that route.

Here were some of the bigger migration issues going from MediaTemple’s GridServer (gs) to Dedicated Virtual/VPS (dv) service.

Plesk doesn’t come with root enabled.

Chances are you’ll need to enable root, which is done through AccountCenter. At first I tried not to enable root but there’s just too many fixes/workarounds that you can’t do without root access.

Plesk creates websites with the web directory set to httpdocs/

…Whereas GridServer uses html/ for the top-level web directory. I prefer the shorter “html”, & I really don’t like changing things like this in Git which we use for development. Luckily it’s easy to change how Plesk works here.

THE FIX: In Plesk, click on your domain & click Hosting Settings. Change Document Root to html/. Plesk will create html/ but leaves httpdocs/ so you’ll have to delete httpdocs/ manually.

Note: for the rest of this post I’ll continue to reference “httpdocs/” for consistency.

Plesk puts cgi-bin/ as a subdirectory of httpdocs/

In other words, Plesk uses httpdocs/cgi-bin/. GridServer had cgi-bin/ at the same level as html/. So basically if you’ve used Git for years like we have, you can either move the folder in Git & hope the history stays or change how Plesk works. Moving the folder & keeping the Git history is possible, but messing with Git gives me the creeps.

THE FIX: Create cgi-bin/ where you want it & set permissions using chmod 755 cgi-bin/. It’s probably good to follow Plesk convention where top-level web directories are assigned to the psacserv group so chgrp psaserv cgi-bin/ too. This worked with Plesk 12 under CentOS6, but Plesk Onyx under CentOS7 requires cgi-bin/ to stay assigned to the psacln group or else Perl scripts running under Apache mod_cgi will return the 500 error “End of script output before headers” (thank you MediaTemple CloudTech Supervisor Gary R for figuring that out).

Then in Plesk under the domain, click Apache & nginx Settings. Scroll to the “Additional Apache Directives” section and add:

ScriptAlias "/cgi-bin/" "/var/www/vhosts/domain.com/cgi-bin/"

I found out Plesk directly supports moving cgi-bin to the www-root level & doesn’t need a ScriptAlias added manually. Run these commands:

/usr/local/psa/bin/domain -u domain.com -cgi-mode www-root
/usr/local/psa/admin/bin/httpdmng --reconfigure-domain domain.com

In Plesk under Hosting Settings, now you’ll see a select box:

Finally click Hosting Settings & disable “Perl”. It’s not what you think. This Plesk option actually disables mod_perl, & does not disable “regular” mod_cgi Perl.

Mod_perl is very efficient but typically requires porting over your Perl scripts, or else it can wreak havoc, as described in pretty much the entire porting guide.

Fun fact: The Plesk “Perl” option isn’t tied to the cgi-bin/ location. It only updates <Directory> options for the httpdocs/ folder — you can watch Plesk change the setting in vhosts/system/domain.com/conf/httpd_ip_default.conf — so if you’ve moved cgi-bin/ out of httpdocs/ to the www-root level, it won’t actually matter whether or not you disable the “Perl” option in Plesk.

Last step is to delete the now-defunct httpdocs/cgi-bin/ directory.

Plesk creates websites directories with a bunch of default & testing files.

THE FIX: SSH to your account, change to the website directory & rm -rf html/*. I’m assuming you know enough about Linux to realize this deletes everything in html/ so make sure you haven’t uploaded the website files you want to keep, yet.

Plesk sets up subdomains as subdirectories of the parent website.

In other words, Plesk creates domain.com/httpdocs/subdomain.com/. This sucks. Luckily, easy fix. When adding the subdomain site in Plesk, pick the primary domain option rather than the subdomain option. Ignore the “www” prefix. The subdomain site will work fine & you’ll just have an extra domain alias for www.subdomain.domain.com in nginx & Apache that won’t be used.

Plesk forces you to create a different user for each web site.

The files all get assigned to that user, & group psacln. The httpdocs/ directory is also assigned to the same user, & group psaserv. So, users can’t browse each others’ web folders. By default, with Plesk you can’t have an FTP account that has access to multiple web sites, which is what I needed since we use one Git repository deployed via BeanStalk to manage ~30 very similar websites with shared resources.

THE FIX: First su root & grant the psacserv group to ssh/ftp users that you want to have access to the full range of web directories (but not root privileges): usermod -a -G psaserv username — this just adds psacserv as a secondary group & the primary group stays psacln — so any new files created via these accounts will still fit the Plesk convention.

I set up one user for SSH & FTP with access to all website directories. If you go that one-user route rather than the Plesk-created users for each website, reset directory & file ownership to your super-web-user with: chown -R username:psacln * That chown operates recursively starting from the current directory, so run that only from within html/ and cgi-bin/ because otherwise it will try to reset ownership on your system files, log files & other non-public stuff that should probably stay assigned to root.

Similarly depending on how you upload/migrate your web files, you may need to set correct permissions on files & directories within html/:

find . -type f -exec chmod 644 {} +
find . -type d -exec chmod 755 {} +

(Execute these commands only from within the directories you want affected.)

For cgi-bin/ you’ll probably want to chmod 755 script files rather than 644, so your scripts (Perl in the example below) have world-read/execute permissions:

find . -type f -name '*.pl' -exec chmod 755 {} +

Final step is have each web property execute CGI scripts as your one user. For each domain’s Apache & nginx settings, in “Additional directives for HTTP “, add:

SuexecUserGroup "username" "psacln"

Plesk sets each FTP user’s home directory to within a web directory.

Again we use Git/BeanStalk with several repos that manage groups of similar websites, so I needed an FTP login for BeanStalk to have access to the vhosts/ directory where all the website directories are located.

THE FIX: You can change the home directory for your FTP user in bash through the normal way, & Plesk doesn’t care:

usermod -d /var/www/vhosts/ username

Plesk runs all cron jobs as root.

Any files that your cron job creates get root user permissions & are not available to the web server users. Plesk shows the cron user as root, but it’s not anything you can change. Yes, this is lame.

THE FIX: Have each cron run a shell script that uses su, sudo or runuser to switch to the web-level user first. For example, have the cron run a shell script with:

/sbin/runuser username -s /var/www/vhosts/domain.com/cgi-bin/somescript.pl

Or you could have each cron job command (each entry in Plesk) start with one of the user-switching methods. But since cron commands become the email subject for status notifications, the subject line would start with “runuser” etc all the time.

Plesk doesn’t serve web font files correctly by default.

We serve css & web fonts from a different domain than the main website, so we need to have an access control header to allow that. The standard code is:

<FilesMatch "\.(ttf|otf|eot|woff)$">
<IfModule mod_headers.c>
    Header set Access-Control-Allow-Origin "*"
</IfModule>
</FilesMatch>

This worked great on GridServer in .htdocs, but didn’t have any effect after we migrated to Plesk. As far as I can tell, mod_headers is enabled by default so that’s not the problem. Eventually I noticed the response header for the web font files was nginx & not Apache. I got it working by going into Apache & nginx Settings for the domain & disabling “Smart static files processing”. I think what’s happening is without the second box checked that defines specific extensions, nginx serves font files by default so the request never makes it to Apache.

Even if it works for you to have nginx serve your font files, I found nginx serves them as text/plain so under the MIME types section at the top of the same screen, it might help to add:

font/ttf .ttf
font/opentype .otf
application/font-woff .woff
application/vnd.ms-fontobject .eot

MediaTemple warns that you’ll lose all your (gs) IMAP email as soon as you click Point At Another Server in AccountCenter.

FIX: True, but you can still switch your site over to (dv) without losing old email. Instead of clicking Point At Another Server, go about adding/migrating your site & ignore Plesk’s DNS warning. Then when you’re ready to switch web traffic over, in AccountCenter, edit DNS & change the A records from your (gs) IP to your (dv) IP.

Make sure you have email set up in Plesk before switching A records, because although you won’t lose your old email, new email will start going to your (dv) accounts.

This way although your domain gets switched over to (dv), you won’t lose your old (gs) email because you can still access it through your gridserver domain (sXXXXXX.gridserver.com) or xxxx-xxxx.accessdomain.com. If you’re not sure what these are, in AccountCenter, click on Server Guide for your grid server & look at the Email section.

Incidentally migrating IMAP email is really easy with Thunderbird. Add your new IMAP account, select all the folders in your old account, drag over to new & you’re done. Then do the Point At Another Server thing.

Plesk doesn’t come with cpan or any Perl modules (or gcc).

I’m a dinosaur, I guess. The “yum” installer does have a bunch of Perl modules available but is missing lots of the common modules I use. Cpan works fine for me. Luckily you can use yum install install CPAN & gcc:

yum install perl-CPAN
yum install gcc

BEFORE YOU RUN CPAN: the yum-installed CPAN adds some environment variables to ~root/.bashrc which sets cpan to install Perl modules under root rather than in one of the @INC locations, so fix that by deleting the added lines in .bashrc & also delete ~root/perl5/ and ~root/.cpan/. Then run cpan setup & picking “lib::local” should put Perl modules into a web-accessible lib/ directory.

Using cpan to install GD doesn’t work.

Cpan aborts with an error message: Could not find gdlib-config in the search path. Please install libgd 2.0.28 or higher. I’m not a sysadmin & didn’t want to mess with this. Easy fix — use “yum” to install GD instead:

yum install gd

Mod_perl under CentOS7 behaves inconsistently compared to the CentOS6 Perl CGI environment.

With CentOS7 the only way I could get Perl scripts working via Apache was to use mod_perl, & with mod_perl the output/success of Perl scripts is inconsistent due to the way mod_perl compiles our Perl scripts that weren’t specifically coded for mod_perl. Generally the first request is successful while subsequent requests fail.

The big problem I ran into was when global subroutines defined in our custom modules are loaded via require("somemodule.pm"), the error log shows “undefined subroutine”. I believe this is why (in my case, it’s scenario 3).

Added bonus, under mod_perl, relative paths don’t work. I could work around that if I had to, but the custom module problems are a deal-killer.

Still troubleshooting. This sucks.

The MT sales rep set up the (dv) account with a made-up domain.

The issue here is my rep felt they needed to put down something temporary while I got started with the site migration. I don’t think the fake domain was a good idea because it caused problems. The second time I went through this DV account switch, the rep used my primary domain for setup & everything worked just fine. The fake domain isn’t just for a name in AccountCenter, which is what tech support first told me.

The fake domain gets set as Reverse DNS for your new (dv) service IP address, which can cause your IP to get blacklisted for email. Minor detail, yep.

THE FIX: As soon as you get your primary site migrated over, fix the Reverse DNS (AccoutCenter, DNS section). Then change the primary AccountCenter domain for your (dv) account to your real domain – that’s hidden in AccountCenter under Server Guide.

Replace Your Noisy Drobo5N (or 5D) Fan

PLEASE NOTE: Although I initially wrote this post about the Drobo 5N, several commenters have pointed out that the Drobo 5N & 5D use the same chassis & so this is equally helpful for fixing noisy Drobo 5D fans too.

Drobo5N fan replacementDrobo makes a nice backup system, but the cooling fans are crap. The fan in my Drobo 5N was no exception & failed after 3 years, most of which it spent on standby with the drives spun down. As far as I know, the Drobo cooling fan runs constantly even when the drives are spun down.

For the cost of a $15 fan & maybe an hour of your time, you can replace the fan in your $500 Drobo5N backup system.

You’ll need a replacement 120x25mm 12VDC 1.9W 2-wire fan, soldering iron & solder (or very small wire nuts), phillips screwdriver, 1/16″ shrink tubing (and lighter) or electrical tape, & a paperclip.

Also see the very helpful comments on these steps from Tad Harrison — the 3rd comment at the end of this post.

  1. Shut down & unplug the Drobo. Remove the hard drives & remember the drive order in the bays. It might not matter, but why tempt fate?
  2. Remove the magnetic faceplace & pull out the rubber gasket from the groove behind the faceplate.
  3. Unscrew the 4 rubber feet on the bottom of the unit. Also remove the small hatch on the bottom that covers the mSATA bay.
  4. Slide the metal case off the chassis. I don’t think it matters which end you slide it off. The case is split in half on the bottom. You’ll need to lift up one side just a bit to get the case to slide past a few items in the mSATA bay.
  5. Push in the plastic tabs to remove the fan backplate on one end & the drive bays on the other end. I think there’s 5 tabs for each. This part was easy.
  6. Unscrew 4 screws total, on the sides at the fan end: 1 upper & 1 lower screw. Leave all the other screws in place. Trust me on this.
  7. Slide the top of the chassis back & up to separate it from the bottom/sides. This can be tricky – use a flat screwdriver to pry the lower slot closest to the back (see photo) if it’s stuck. Unplug the white plastic fan & power switch connectors when you can get to them – they just pull, no clips.
  8. Drobo5N fan anchor

    Don’t pry off from the head end!

    Finally, easy access to the fan! Not so fast, Batman. The fan is held in place by 4 plastic anchors that work like drywall anchors – there’s a center pin which spreads out the tip of the anchor when it’s pushed in all the way. Best method for removal is push the center pin from the fan side with a paperclip or small nail until the anchor pops out. Don’t try prying the anchors from the cap end or you’ll likely end up breaking them off.

  9. Cut the fan wire close to the old fan so you have decently long leads on the connector plug end.
  10. Solder (or use wire nuts, or butt splice connectors) the plug onto the new fan wires. If you solder, use shrink tubing! It’s awesome. Remember to slide the tubing on, before you solder.

Now you’re ready to put everything back together. Really this is just an excuse to start the instruction numbering over at #1. Also now that you’re in this deep, let’s make putting your Drobo back together look like 4 steps:

  1. Reattach the fan to the chassis with the plastic anchors you didn’t break. Make sure the the fan is oriented correctly so it blows air out the rear vent. The side with the hub struts is the “exhaust” side, so you would want that facing the rear. Stare at the fan blade shape for a bit & you’ll figure it out. Some fans have arrows on the cowling that indicate airflow direction.
  2. Slide the chassis back together, reconnect the fan & power switch connectors, & screw the 4 screws back in. PRO TIP: make sure the fan wires aren’t in the way of the fan blades.
  3. Snap the fan backplate & the drive bays back in. It can be a little tricky to get the drive bays seated all the way. Make sure you line up the many clear plastic nubs along the bottom edge with the all holes (these are the blue lights that indicate capacity).
  4. Slide the chassis back into the case. Reinstall the mSATA hatch cover, rubber feet, rubber gasket, drives, & magnetic faceplate. Plug in & turn on.

Nice work. You saved $500 on a new Drobo.

Hope the new fan lasts longer than the old one.

Workaround For MediaTemple’s Lame Gridserver Cron Job Limitation

MediaTemple limits their GridServer (GS) customers to only 5 cron jobs.

Some restrictions make sense … some don’t!

This makes absolutely no sense.

MediaTemple allows cron jobs to run as often as every 5 minutes … I needed more than 5 weekly processes, which put no strain on the server compared to someone who sets up 5 jobs to repeat every 5 minutes.

Limits on cron jobs by run frequency would be far too logical.

As a workaround, combine all your weekly crons into one daily job, your daily crons into an hourly job, etc. Then create a shell script that uses date logic tests to branch out to different jobs, based on the day of the week or hour of the day.

For example, here’s a daily cron script that branches out into weekly jobs:

if [ $(date +%u) -eq 1 ]; then ./monday.pl
elif [ $(date +%u) -eq 2 ]; then ./tuesday.pl
elif [ $(date +%u) -eq 3 ]; then ./wednesday.pl
....
elif [ $(date +%u) -eq 7 ]; then ./sunday.pl
fi

If you don’t have a job for every day of the week, just leave out the logic test for that day & the cron script will exit without running anything further.

Similarly with a cron script set to run hourly, you can test the hour to run up to 24 different daily jobs (one for each hour):

if [ $(date +%k) -eq 0 ]; then ./twelve-oclock-am.pl
elif [ $(date +%k) -eq 1 ]; then ./one-oclock-am.pl
elif [ $(date +%k) -eq 2 ]; then ./two-oclock-am.pl
....
elif [ $(date +%k) -eq 23 ]; then ./eleven-oclock-pm.pl
fi

Taking this to the extreme, in theory you could use an every-5-minutes cron job to test $(date +%M) and run up to 12 different jobs per hour. And it goes without saying you can combine these day/hour/minute logic tests to create 2,016 possible combinations, or throw in some $(date +%d) day-of-the-month tests for 62,496 possible cron jobs. Take that, MediaTemple-cron-limitation-type people!

Obviously the script names in the examples above (monday.pl, twelve-oclock-am.pl, etc) can be changed to anything you want.

I’m not on MediaTemple’s GridServer platform anymore, but hope this helps someone. I switched to a their DV/VPS platform & promptly discovered they assigned a blacklisted IP to my account. So far they are refusing to fix it. Good thing MediaTemple doesn’t sell cars.

UPDATE: To their credit, MediaTemple finally saw the logic in providing new accounts with non-blacklisted IPs, so that’s good.

Powered by WordPress & Theme by Anders Norén