Forum Moderators: rogerd & travelin cat

Message Too Old, No Replies

What's your WordPress development process?

WAMP, sandbox, Git, or?

         

lorax

2:12 pm on Aug 17, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've been reading up on how people build with WordPress and/or make ongoing upgrades, edits, customizations - in particular where they do this work.

I've long done the work live on the server (cowboy coding) - making my edits to local files then uploading them to the server and checking them. If something breaks, I make a change and try again. Since I use child themes almost exclusively now, there's less risk and with good backups in place, I'm relatively confident that if something really goes sideways, I'd be able to get back to where I was (hasn't happened in the 8+ years I've been working with WordPress). I always keep a copy of the original files as I work too - just in case and it's quicker to restore.

I've worked with sandbox sites in the past too. Where I created a duplicate site with a duplicate database and used a sub-domain to point to it (dev.thesite.com for example). Then I restrict access to this by denying every IP but my own. From here I can build & break things until I'm ready to go live then simply remove the restrictions, change DNS to use this as the live site and make sure WordPress is set to the live URL instead of the development version.

As I work for a University now, I've been preparing to move to a host that offers a staging environment. The staging environment does almost the same thing as sandboxing but automatically. This affords me the same space to play in. When I'm ready, it's a simple matter to click a button and the staging environment is pushed to the live environment. We just need to keep track of what phase of development each of our sites are in so we don't overwrite our changes/progress for a staging instance. It comes down to having a very clear documented process so each member of my team knows exactly what to do if/when they need to make an update.

A few months ago I attended a WordPress meetup on using Git with WordPress. And our own ergophobe posted about an issue using Git and the fact that WordPress puts the site root and home URLs in the database: [webmasterworld.com...] I'd like to explore this more sometime but probably not anytime soon as the staging environment seems like it will work fine for our needs.

I think it would be nice to hear how others do their development work. What process do you use and how do you push the site live? What sort of headaches do you run into and does it work well for you or do you wish you could use something different?

Narsto

7:18 am on Aug 19, 2014 (gmt 0)

10+ Year Member



We have a pretty similar process. I always keep freshest backup whenever I will test something or perform a major update. I also do it first on my local machine for testing before uploading it on the live server. Haven't really encountered any major issue so far and I hope it stays that way.

graeme_p

1:48 pm on Aug 19, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I always use version control, but not usually git: I prefer Fossil as I do not work in large teams. This goes for whatever I am working in (its not usually Wordpress, but I do the same for the occasional WP site as well).

You definitely should use some version control system. It is very useful to know who changed what and when, it is often useful to look at old versions of code, and it means you can revert any file, or directory, of the whole site to an old version.

I work primarily on local copies of sites. I sync with a dev site as and when I need to show a client what is happening.

I do sometimes make changes directly to live sites, but I use an editor that directly opens files over SFTP (or fakes doing so). Most Linux text editor can do it, and Komodo Edit does it cross platform.

not2easy

4:08 pm on Aug 20, 2014 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I once set up a WordPress site on a local MAMP install. It didn't dawn on me until I uploaded it that all links would need to be changed because they are in the .sql file. That was several years ago. Now I may take a copy and tweak the css on my local server and it is handy for that. I have since learned that this problem could have been avoided. A good text editor can find/replace and fix it all up.

I use an app to design my own themes, can't say I love using it, it was made for Windows and the Mac version has been in beta for oh, 8 years now. I switch over to Windows to get full functionality.

Hoople

12:15 am on Aug 21, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have a test site I use to verify plug-ins working correctly with various themes in use. For prototyping new stuff I feel more comfortable on my server rather than a customers.

A secondary purpose is to show my customers a live site that they comment on to finalize a design/redesign. Once payment is in hand the files are copied to their site.

Kendo

12:38 am on Aug 21, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I usually set up a new alias and install the latest WordPress just to be sure of compatibilty with the latest.

But I found that one needs to be careful when doing this, because although that site may not be referenced from anywhere on the web and that you might be the only person who knows the URL, it can still be found by others, especially those who specialise in exploiting WordPress sites with SEO hacks.

Any ideas about where they get the site info from?

lorax

1:52 am on Aug 21, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



OT
Any ideas about where they get the site info from?

I don't suppose you have Google toolbar installed?

graeme_p

9:26 am on Aug 21, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



If the test site has its own IP or is the default site for that IP they can find it by large scale scanning.

One thing I sometimes do is require HTTP auth to access the site: simple, and enough to block automated attacks.

ergophobe

3:41 pm on Aug 21, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



As you mentioned, I use

- a test server (WAMP or LAMP local or remote, depending - you pretty much have to have a live server if using Jetpack for example).

- git hosted on Bitbucket

- master, dev and live branches, with other branches created and destroyed as needed.

- develop on feature branch on dev machine until it's okay.
- pull down your test data
- run your automated tests (Selenium IDE in my case)
- manual sanity check
- merge into dev branch
- merge into live branch
- push live branch to repo
- pull live branch to live site
- run automated tests
- cross fingers

Kendo

8:18 pm on Aug 21, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I don't suppose you have Google toolbar installed?


No, nothing like that. No add-ons that aren't ours. I may have checked the site using Chrome but usually Firefox check suffices. If WP phones home when installed that could be another possibility.

We see a lot of crappy requests on most of our sites, looking for things like FCkeditor, wp-admin, admin folders. So if we are being targeted they may getting our aliases from DNS and trying each one.

not2easy

10:38 pm on Aug 21, 2014 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



We see a lot of crappy requests on most of our sites, looking for things like FCkeditor, wp-admin, admin folders. So if we are being targeted they may getting our aliases from DNS and trying each one.

I have several non-WordPress sites and the bots request the same files there. I have seen no evidence that encountering a 404 makes them stop asking. I have set up bot traps with those names: "wp-login.php" so any request is an automatic and a permanent 403.

ergophobe

7:01 pm on Sep 14, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Wordpress is driving me crazy... I don't understand how to have a consistent development process on it given the absolute URL and file path problem I mentioned earlier. - [webmasterworld.com...]

Now as I get back into Wordpress, I find that

- there is no theme/plugin uninstall. You can disable, but there is nothing built in to themes and plugins to clean up after themselves if you want to uninstall. So the database fills up with settings from plugins you were trying. Ridiculous frankly.

- because of absolute filepaths and URLs littered through the database, it remains exceedingly hard to run a test server and a live server and migrate data. The solution I mentioned in the other thread doesn't actually work. If you got to test.example.com to the login screen, WP will submit the form to example.com if that's what your site_url is set to in the database. Again, terribly aggravating.

You can't even log in to your test server without manually going into the database and changing the site_url. What kind of crazy is that?

So I have yet to come up with a dev workflow that actually works.

I think the only possibility that I can think of is to create a virtual machine where, if the site is to be deployed to example.com, on the VM example.com resolves to localhost. And the VM file strucutre would have to be the same as the target machine. That might work.

Still kind of at a loss though.

not2easy

8:33 pm on Sep 14, 2014 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



You may need to get more familiar with using MyPHP to delete unwanted/unused tables. Some plugins are far worse about being difficult to uninstall like JetPack.

What you are looking for is done in the wp-config.php file where you set the db name, username, and db login. You would need two separate configurations, but unless you have different table prefixes in your sql file, you should be able to export from one and import to the other. I say that because to move the whole WP site all you need to do is export the sql, install a new, empty WP install on the new site and import the sql file to the new domain. (and then mess with plugins, settings, etc.) The hard part is not having the same path settings for the db name, username and logins, so that gets changed in your wp-config.php file.

ergophobe

3:14 am on Sep 15, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




(and then mess with plugins, settings, etc.) The hard part is not having the same path settings for the db name, username and logins, so that gets changed in your wp-config.php file.


Right. That's my point. That is completely jacked as a process. I finally sorted out my issue and some of the things I ran into

- plugins that were storing absolute paths in the DB. This is a huge no-no because it means you can't deal with this in version control or with local settings files

- plugins and themes that leave cruft behind.

The problem with the siteurl and home setting can be dealt with in the way I mentioned in the other thread. So I have this code to handle that


// We let test sites override basic connection settings with a local settings file
// with a path based on the current user so that we don't accidentally find other
// files on the server.
// If we don't find a file, we fall back on the settings that work on the live site. So
// that we don't accidentally bring down the live site if there's a problem
//
// Config file must be in a directory one level up from server's DOCUMENT_ROOT

$config_file = dirname($_SERVER['DOCUMENT_ROOT']) . '/wpconfig/wp-config.example.php';
if (file_exists($config_file)) {
include($config_file);
}
else {
define('WP_HOME','http://example.com');
define('WP_SITEURL','http://example.com');
define('DB_NAME', 'dbname');
define('DB_USER', 'dbuser');
define('DB_PASSWORD', 'password');
define('DB_HOST', 'localhost');
}


Then in the included file wp-config.example.php you have something like



define('WP_HOME','http://test1.example.com');
define('WP_SITEURL','http://test1.example.com');
define('DB_NAME', 'testdb');
define('DB_USER', 'testuser');
define('DB_PASSWORD', 'testpassword');
define('DB_HOST', 'localhost');


That generally handles issues in WP core and lets you have a reasonably intelligent process where you can push and pull from a git repo and upload a DB and and not have your site die. But the second you start adding plugins and themes all bets are off. You have to manually mess with things.

So you're pretty much stuck with making untestable changes on a live site unless you do as I suggested and develop in a virtual machine that has the same file structure as the target machine and has the DNS set up so that the target domain resolves to localhost on the VM.

That's a fair hassle overall.

ergophobe

6:18 am on Sep 15, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



By the way, the problem with plugins is particularly acute because you can't do search and replace fixes. The problem is that lots of plugins save their settings as serialized arrays. So if you change the text, the entire thing breaks in PHP.

In other words, you might have

a:1{s:51:"http://test1.example.com/path/to/uploaded/image.jpg"}

If I do a simple search and replace for the domain name, I end up with

a:1{s:51:"http://example.com/path/to/uploaded/image.jpg"}

That of course will render ALL the settings null and void because the PHP parser can no longer unserialize that array because the strength length is wrong. That is, in fact what was happening to me, but it was happening more or less transparently (no PHP errors, notices or warnings). I couldn't figure out why the DB upload was resulting in a completely different look - it's because theme settings were being ignored universally because one of them included a file upload with an absolute path. I can't for the life of me understand why the WP developers think this is a good idea.

graeme_p

10:58 am on Sep 15, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



What an insane serialisation format!

Like a lot of what you complain about, it just reflects the fact Wordpress is a blog CMS that is flexible enough to be used as a general CMS, but it is being used as a web framework.

ergophobe

3:28 pm on Sep 15, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



That's the standard serialization format used with the PHP serialize() unserialize() functions, but it's not a very robust way to store data.

So it's not the serialisation format that's odd, it's that WP is not really set up to have a robust pluggable architecture, so developers can fall back on just serializing data and throwing it into the wp_options table as a single DB value.

This is to some extent related to the comment I made above about not having any in-built cleanup mechanism either. The plugins and theme options do not track the values they save and they spread them about willy-nilly in the wp_options table. So there's no proper uninstall for anything and it's next to impossible to hook into an existing plugin or create a script that would handle some of the tasks I'm talking about. The best you could do, for example, when scanning the DB for file paths and URLs would be to try every value, either with a regex or simply try to unserialize it and see if you succeed, and then if you succeed iterate through the arrays and re-serialize them and then write them back to the DB.

blog CMS that is flexible enough to be used as a general CMS, but it is being used as a web framework


In the case in question, though, I'm basically using it as a simple blog. The problems with the plugin architecture come to light long before you try to do anything complex with Wordpress. It's a product of a desire to not break existing plugins. So in some way, Wordpress is becoming the Windows of web platforms, in that the desire to maintain backward compatibility drives architectural decisions that ultimately make the system unstable.

So at this point, Wordpress would have to add things like the *ability* to have an uninstall routine without the *requirement* to have it and hope that over time that issue is more or less handled. Similarly for the file paths and so forth.

I can accept a database setting that sets the site base URL and the base file path, that then gets used elsewhere, but aside from a handful of special cases (backup scripts that need to save files outside of web root, for example), absolute paths should always be built dynamically based on global settings. Then an API to set a path would default to disallowing absolute paths and URLs, but would allow developers to override it (for example).

Maybe in Wordpress 5. It is open source, so it's just a matter of pushing hard enough and long enough and with good code and you might effect those kinds of changes.

lorax

12:06 pm on Sep 16, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@ergophobe - Where did you get the idea the WordPress team is trying to maintain backward compatibility?

graeme_p

12:48 pm on Sep 16, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



That's the standard serialization format used with the PHP serialize() unserialize() functions, but it's not a very robust way to store data.


I guessed. If its really insane, its probably PHP!

So it's not the serialisation format that's odd, it's that WP is not really set up to have a robust pluggable architecture, so developers can fall back on just serializing data and throwing it into the wp_options table as a single DB value.


I think that is just one of a number of ways in which the design of Wordpress puts too much of a burden on developers because it is *not* a framework.

It reminds me somewhat of people writing what are, in effect, fairly complex database driven apps in Excel - the code is VBA and each Excel sheet acts as either a table or is used for display.

Now spreadsheets are wonderful for certain things, but not as a platform for serious database apps. Wordpress is a great CMS in many ways - except for a somewhat old fashioned architecture and being written in PHP ;) However it is just not well suited as a platform for writing web apps.

ergophobe

4:22 pm on Sep 18, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



@lorax - any system that's auto-updating can't afford to make API changes. Wordpress has tended to try to avoid plugins except on major releases and, if possible, not even then. But now that it can be set to auto-update, it really can't change APIs.

I'm not saying they can't and won't break APIs all over the place with WP5.