Forum Moderators: open

Message Too Old, No Replies

How to get website thumbnails?

         

tntpower

7:10 am on Nov 22, 2004 (gmt 0)

10+ Year Member



Alexa provides website thumbnails.

I am interested how they get these thumbshots. Do they write a custom browser that can generate image from parsing HTML?

kaled

10:51 am on Nov 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It's a guess, but I imagine they use a standard browser but with some modifications to library code (so that rendering takes place in memory) or possibly an automated bolt-on script of some sort.

One tricky part of this method would be determining when rendering was complete. However, you could simply monitor the status bar looking for the word 'Done'. You could also monitor traffic.

I doubt that a unique rendering engine is used to take snapshots.

Kaled.

Mark_A

2:10 pm on Nov 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



You could try reverse engineering windows explorer, on one of my machines it gives me a thumbnail of an html page when I highlight the file.

I assume at least that these guys have done something similar.

willis1480

4:12 pm on Nov 22, 2004 (gmt 0)

10+ Year Member



"snag it" is a program that runs on ie and other browsers I am sure. It will take a snapshot of a page and you can create thumb easily. Still a very manual process, but not bad. I use it for my web design company. Free trial from download.com, but i am sure you can find it for free if you look hard enough :)

tntpower

8:38 am on Nov 23, 2004 (gmt 0)

10+ Year Member



It is plausible to fulfil this feature by (virtual) printing with browser's built-in print fuction. But I think it is very slow.

TheDoctor

8:41 pm on Nov 23, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If you're going to do this manually, with not too many pages, the easy way is to take a screen snapshot (Alt-Printscreen in windows), paste that into your favourite image editor and work from there.

tntpower

1:12 am on Nov 24, 2004 (gmt 0)

10+ Year Member



I have approximately 3000 websites, most of them are not listed in DMOZ. So I am unable to use thumbshots.org or alexa.

tedster

2:34 am on Nov 24, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Check out this thread in our Perl/CGI forum for ideas:

Script to generate thumbnail [webmasterworld.com]

#4
mattx17:

One way you can do it is you have a browser such as mozilla installed on the server, and you also have X running (and a window manager), you can get a screenshot this way:

Leave the browser window open, fully maximized, in your server's X session. Run the command (from your script):

mozilla -remote openURL('http://www.mysite.com')

This should open the URL up in the servers X session. Once that is done, take a screen shot with any of the available tools for X. We'll use xwd as an example:

xwd -frame -root -out myscreenshot.xwd

This will take a full screenshot of the X root window, which should have the browser window open at that URL.

After that, you may have to use ImageMagick or NetPBM to convert the file into something usable for the web.

Hope this helps, even if it just points you in the right direction.

tntpower

5:48 am on Nov 24, 2004 (gmt 0)

10+ Year Member



tedster, Thanks. This solutions sounds good, it is supposed to work,I will try.

A custom browser may be the best solution, but that is far far beyond me. :)

jez_kewler

8:41 am on Dec 4, 2004 (gmt 0)

10+ Year Member



Hi folks,

bumping this up, because I still just wonder if there is just a simple perl/php solution available, without installing/configuring an x-server on my leased server.

thanks,j

keyplyr

9:20 am on Dec 4, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I still just wonder if there is just a simple perl/php solution available

Try this:


<?
// this script creates a thumbnail image from an image file - can be a .jpg .gif or .png file
// where $thumbsize is the maximum width or height of the resized thumbnail image
// where this script is named resize.php
// call this script with an image tag
// <img src="/php/resize.php?path=imagepath"> where path is a relative path such as subdirectory/image.jpg
$thumbsize = 200;
$imagesource = $_GET['path'];
$filetype = substr($imagesource,strlen($imagesource)-4,4);
$filetype = strtolower($filetype);
if($filetype == ".gif") $image = @imagecreatefromgif($imagesource);
if($filetype == ".jpg") $image = @imagecreatefromjpeg($imagesource);
if($filetype == ".png") $image = @imagecreatefrompng($imagesource);
if (!$image) die();
$imagewidth = imagesx($image);
$imageheight = imagesy($image);
if ($imagewidth >= $imageheight) {
$thumbwidth = $thumbsize;
$factor = $thumbsize / $imagewidth;
$thumbheight = $imageheight * $factor;
}
if ($imageheight >= $imagewidth) {
$thumbheight = $thumbsize;
$factor = $thumbsize / $imageheight;
$thumbwidth = $imagewidth * $factor;
}
$thumb = @imagecreatetruecolor($thumbwidth,$thumbheight);
imagecopyresized($thumb, $image, 0, 0, 0, 0, $thumbwidth, $thumbheight, $imagewidth, $imageheight);
imagejpeg($thumb);
imagedestroy($image);
imagedestroy($thumb);
?>

tntpower

10:14 am on Dec 5, 2004 (gmt 0)

10+ Year Member



Thanks, I tried but does not work. I will try again tomorrow.

tedster

7:31 pm on Dec 5, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



We have a second active thread here about the same question:

[webmasterworld.com...]

jollymcfats shares another solution in Message #4 of that thread: using Xvfb - an optional package in most Linux distributions, usually with names like xorg-x11-Xvfb or XFree86-Xvfb.

keyplyr

9:27 am on Dec 8, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




Thanks, I tried but does not work...

You know of course your server must have PHP installed and you either must be parsing HTML pages for PHP or name your pages with the .php extension.

jino

11:34 am on Dec 8, 2004 (gmt 0)

10+ Year Member



I am using a web research tool that does this really well.. Not sure if u can use it for your purpose.
Its called surfulater.

Whenever I come across a site/page that I'm interested in, I highlight the part of the page Im interested in and send it to surfulater. It then creates a thumbnail image of that page and paste the highlighted html into the app.

Great tool.

Lorel

4:56 pm on Dec 8, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



For those without their own server:

View the page at a very high resolution so you can get most of the page on one screen then use the screenshot command. Someone else posted the Window's command above; on the Mac it's (Command-shift-3 or Command-control-shift-3 puts it into the clipboard so you can paste it into your image program).

Iguana

5:57 pm on Dec 8, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I suspect the problem is to do it programatically for thousands of pages and to directly convert an HTML file into a jpg or png. Copy/Paste solutions may lead to repetitive strain injury.

Windows 2000 had the facility built in to view webpages as thumbnails. It is possible to hook into that facility using Visual Basic.

You need to have the HTML rendered and not just the 'View Source' HTML - that way any Javascript/css can be applied. Again Visual Basic provides an interface to Internet Explorer as well as reading a database of the thousands of pages you want to generate thumbnails for. You would need to be a decent VB programmer to go this route (or C++ or Delphi).

I have done this in 2000 but I haven't tried it with Windows XP - XP doesn't display thumbnails of webpages (but the underlying calls MAY still exist)

sammydafish

6:14 pm on Dec 8, 2004 (gmt 0)

10+ Year Member



from msg#17
I have done this in 2000 but I haven't tried it with Windows XP - XP doesn't display thumbnails of webpages (but the underlying calls MAY still exist)

yes it does, view a folder with html files as thumbnails and they will be displayed. It just doesn't render them in the side bar like 2000 did. I assume whatever solution you came up with on win2k would also work on XP.

tntpower

6:16 am on Dec 10, 2004 (gmt 0)

10+ Year Member



Thanks. You are right. It works on my remote server but for somehow, it does not work on my local machine.

But the key issue is I want to generate screen shot by reading HTML, instead of from an existed image.

You know of course your server must have PHP installed and you either must be parsing HTML pages for PHP or name your pages with the .php extension.

jez_kewler

7:22 pm on Jan 3, 2005 (gmt 0)

10+ Year Member



Thanks keyplyr,

but how can you convert the image on any homepage )an URL as input) into a rendered JPG? like thumbshots?

keyplyr

9:00 pm on Jan 3, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



...but how can you convert the image on any homepage )an URL as input) into a rendered JPG? like thumbshots? - jez_kewler

Well, the instructions are there. As far as image paths, those can be adjusted.