homepage Welcome to WebmasterWorld Guest from 54.166.110.222
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Microsoft / Microsoft IIS Web Server and ASP.NET
Forum Library, Charter, Moderators: ocean10000

Microsoft IIS Web Server and ASP.NET Forum

    
Extremely long compile time
150K+ aspx files, takes 10+ minutes to compile
Big_Gig




msg:948297
 6:36 pm on Aug 8, 2005 (gmt 0)

Hey all,

I have a web app that has 150,000+ aspx files, without a code-behind. Every time I transfer an updated dll to the server, it takes 10+ minutes to load the first time.

I understand that .net must compile the app the first time it's run, but this is excessive! Are there any ways to speed this process up, or do it offline?

Thanks!

 

tomasz




msg:948298
 8:00 pm on Aug 8, 2005 (gmt 0)

Wow, 150k pages. What is your dll file size?

TheNige




msg:948299
 8:05 pm on Aug 8, 2005 (gmt 0)

That is the JIT compilation that happens everytime you modify/add something in the bin folder. Using your numbers it is compiling about 250 aspx files per second when restarting the application.

Why do you have 150,000 aspx pages? Sounds like you should use a database and some URL rewriting if you are concerned about having unique file names.

Big_Gig




msg:948300
 8:55 pm on Aug 8, 2005 (gmt 0)

I understand that the JIT compilation happens every time I update something in the bin folder - it's just the time that it takes that I'm trying to reduce...

I have a TON of content in my databases... The vast majority of aspx files on my site do not have a code-behind. They're created dynamically from a database on another machine, and FTP'd to the site. So, my dll is not very big - 230KB.

I am concerned about unique file names, for spidering purposes. Not sure what you mean by URL rewriting, but each of my files is named uniquely.

Is there a way to precomile the aspx pages when all the pages don't exist on my development machine?

TheNige




msg:948301
 10:59 pm on Aug 8, 2005 (gmt 0)

It doesn't matter if you use a code behind or not. The ASPX page gets compiled as a class too when the application is started. So you have 150,000 pages getting compiled.

If you just want to use them as static content pages it would probably have been better to use standard htm extensions for them.

URL rewriting is taking something like

www.example.com/MyPage.aspx?PageID=295678 and turning into
www.example.com/pages/MyPageTitle/295678/Page.aspx

That way you could have 1 page getting the data from the database...for each page in your site and then rewrite the url so it appears as a different physical page to visitors and spiders.

Big_Gig




msg:948302
 11:21 pm on Aug 8, 2005 (gmt 0)

Thanks TheNige,

I guess I am sort of using url rewriting in a way, but all of the files are loaded onto the server with their own unique file name like:

www.example.com/articles/20050601/3847_some_file_name.aspx

The reason I decided to use aspx files in the first place was for the dynamic header and footer, to allow easy changes to the look and feel of the pages... well, that seems to have backfired. I'd rather have static htm files now... but thousands of my aspx files are already spidered by the search engines.

So, every single aspx file must have JIT compilation, regardless of whether there is a code-behind or not? Is there any way around this - to compile at load time like asp? If not, I wonder if the best thing to do would be to republish everything as an htm file, and 301 permanent redirect from the current aspx files to the htm version - and then remove the aspx versions after a couple of months...

Have any thoughts on that?

f00sion




msg:948303
 12:40 am on Aug 9, 2005 (gmt 0)

holy crap, if they are automatically generated from a different program why not have that also include your header and footer and give them .htm extensions? Use rewriting to redirect any request to .aspx files to the .htm file of the same name/directory.

TheNige




msg:948304
 1:06 am on Aug 9, 2005 (gmt 0)

If your pages are already being generated from a database, could you design 1 page to accept a querystring paramter and then display the correct info from the database.

If you can, then do some research on ASP.Net url rewriting and you could then keep the same file names and paths that you already have...except you wouldn't have 150,000 individual files. The URL re-writing is done in an HTTP handler that you can add to your application.

This method would help you avoid messing up any search engine rankings by avoiding chaning your paths and file extensions.

I'm not an expert on it but there are some tutorials out there.

sharbel




msg:948305
 1:45 am on Aug 9, 2005 (gmt 0)

I just want to echo everyone else's suggestion of either creating standard .htm pages with some header/footers or creating 1 .aspx page dynamically pulling the content from the database, with some good URL re-writing.

Big_Gig




msg:948306
 4:12 am on Aug 9, 2005 (gmt 0)

Thanks for the help guys. I'm going to re-publish all the aspx files as htm files and write some logic into my 404 page to do a 301 redirect to find the proper file.

dutch_dude




msg:948307
 3:23 pm on Aug 11, 2005 (gmt 0)

Remember to turn off debugging in a deployed website, look in web.config to do this. Debugging turned on has every page compiled seperately. This is a lot slower.

If you also turn on caching in the pages there really should not be any performance problem over static html.

write some logic into my 404 page to do a 301 redirect

I would not do that, will confuse search engines very much. If you want to rank well that is :)

Big_Gig




msg:948308
 3:49 pm on Aug 11, 2005 (gmt 0)

I definitely have debugging turned off - no problem there. I have no problem with the performance of the pages once the app is compiled the first time. Although, my site is highly unreliable at the moment, and I have questions about the stability of so many aspx files on IIS...

I've just finished re-publishing my entire content base in html and am preparing to do the 301 redirect switch. It works like this:

1. In web.config, I have the following code:

<customErrors mode="On">
<error statusCode="404" redirect="/404.aspx" />
</customErrors>

This code redirects any 404 errors to a nice looking 404 page, which notifies me of the missing file.

2. In the 404.aspx.vb file I have the following code:

If InStr(LCase(Request("aspxerrorpath")), "/articles/") > 0 Then
'if this user is looking for an article, redirect them to the html version
Dim sReDir = Request("aspxerrorpath")
sReDir = Replace(sReDir, ".aspx", ".html")
Response.Status = "301 Moved Permanently"
Response.AddHeader("Location", sReDir)
Response.End()
End If

This code looks for anybody going to an aspx version of the file, and redirects them to the html version. It is a 301 redirect, so the search engines should not be confused, and should index the new page. All of my directories and site pages have been updated to point to the new html versions, so future spiders should index the new pages.

Why do you say that this is bad for the search engines?

dutch_dude




msg:948309
 4:21 pm on Aug 11, 2005 (gmt 0)

If you redirect the client with a 301 before a 404 is send then there should not be a problem.

If the requested page really does not exist, also not as static html make sure the client does receive a 404. Otherwise you may get duplicate content issues. Google seems to request lots of gibberish URLs lately to detect generated spam.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Microsoft / Microsoft IIS Web Server and ASP.NET
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved