Forum Moderators: coopster
I have a comlex issue and need to design a php script to do certain functions for my business..
first off I don't even know, if this is all possible with php... im just basically asking, if this is all possible, or which languages you would go about this project..
first let me explain a little bit of my current process..
I have an account on a site for my business.. And, they have a datafeed of .txt file in delimited tab for cvs copy paste input..
Then, I copy paste their .txt file as my own .txt file. Upload my .txt file into my web host.. And, then rename the .txt file to .cvs file..
After that, I call a sql query to upload the tons of data into the database within seconds, using the data load in php admin..
Once, the database load is complete.. I created a .php page which parses this data in the database.. Which restructs and outputs the data into a delimited tab format for copy pasting into a .csv, with all of that data constructed the way I need it..
Finally, I transfer this .csv file into another hosted site, where all of the information belongs..
-as you can see this can be a time consuming process. Especially, when the first .txt file is updated several times a day.. And, my business needs the most frequent updated information..
So, back to my question.. Is there a way to automate this whole process, using php, or a combination of languages..
I would like to have a script that could login, and download the first .txt file automatically, and insert it into a database.. if at all possible, though this part I could do manually, if a must- this is the fastest step manually anyways..
Ok, then, I already have a script that parses the database data to the way I want..
Though, after parsing, is there a way I can save this as a csv, or .txt file, and then upload it into another website, or import it, as this is the final step..
The last step where I would import the data.. I have ftp access to the final site where the data should be imported..
Also, it would be nice, if I could set this automated system to do this every hour, or the time I set through out the day.. and, SSL would be needed probably for the last site.. Since, you must have SSL to upload through ftp..
just a quick summary, I would want an automated system to,
collect data behind password protected pages > insert data into a database > parse database data through php > upload parsed data to another login proected hosted site w/ ftp access available.
anyways, sorry for such a long post -again, but I really would like to create this project,, and need some guidance on what languages to use, if it is even possible, and any other input you all want to give me..
I assume that the txt file you get and the one you send have a standar name/pattern. Something like: [file_name]_[date].txt perhpas or something similar to this?
- get file from anotehr server/site: You can check fopen() and curl() as a start.
- put data to your database: easily done since your fetched data will be either a string or an array. Depending on that type you break/explode/seperate the data and store them to your db
- create your own cvs/txt file: easily done also.. countless scripts out there that do this (or you can write your own script)
- final step of sending these data to a third server/site: this is also easy since you have already done that in the first step. Now, instead of getting data, you are posting data or a file. You will need to use ftp upload method probably but thats as easy as connecting to a local database (take a look into the ftp function i the php manual).
My idea: start by creatig seperate files for each step. Use them manualy and slowly put them together into one large file/script. Now, when you have one script that does all that, all you have to do is to setup a crontab job to run this script at your desired hours/days/weeks. Crontab is esily accesible from your server's cp (if you are using apache - don't know if this exists in windows machines).
haha, figured it would be easy for someone else, and just complex to me..
thanks again
as a general rule.. no project is ever too complex as long as you could break it into smaller projects
later you merge them together creating somethig complex (and quite possible that something only you could understand :) )
If you ever get stuck, post here.. we will help you
And in the last part (where yoiu merge all small scripts together) you gonna need help optimizing your code to avoid script time outs, server running out of memory and other such misbehaviours.. again.. post here
I have learning php for about 1.5 years now.. And, I am always amazed at what it can do.. I think aftering knowing about the curl function.. I will be trying to make everything I do on the net automated, lol..
thanks again..
