Loading in the same XML doc multiple times

Is it a good idea / bad idea / doesn't matter

[katebp]

Hello,

I have a site which generates the top menu, left menu, breadcrumb links etc all using XslCompiledTransform and .sitemap file.

In order to make it easy for people to re-use my code I've done them all as user-controls, so they can be dragged and dropped into the relevant place without having to manually copy out (break) code.

(User-controls - that's something I love about .NET!)

For instance:

leftMenuNav.ascx

<%@ Control Language="C#" AutoEventWireup="true" CodeFile="LeftMenuNav.ascx.cs" Inherits="managed_content_usercontrols_LeftMenu" %>
<asp:Literal ID="litSubNavMenu" runat="server"></asp:Literal>

leftMenuNav.ascx.cs

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Xml;
using System.Xml.Xsl;
using System.Xml.XPath;
//
public partial class managed_content_usercontrols_LeftMenu : System.Web.UI.UserControl
{
 protected void Page_Load(object sender, EventArgs e)
 {
 // load site xml
 XmlTextReader siteXML = new XmlTextReader(Server.MapPath("~/xml/dotnettemplate.sitemap"));
 XPathDocument xpathDoc = new XPathDocument(siteXML);
 //Section menu
 XmlTextReader sectionXSL = new XmlTextReader(Server.MapPath("~/xslt/InternalMenu.xslt"));
 XslCompiledTransform SubNavMenu = new XslCompiledTransform();
 // Create the XsltArgumentList
 XsltArgumentList xslArg = new XsltArgumentList();
 // Create a parameter which represents the current date and time.
 DateTime d = DateTime.Now;
 xslArg.AddParam("pageid", "", GlobalVariables.PageID);
 SubNavMenu.Load(sectionXSL);
 StringBuilder sb = new StringBuilder();
 using (StringWriter sw = new StringWriter(sb))
 {
 SubNavMenu.Transform(xpathDoc, xslArg, sw);
 litSubNavMenu.Text = sw.ToString();
 }
 sectionXSL.Close();
 siteXML.Close();
 xslArg.Clear();
 sb = null;
 }
}

will load in the left menu when it's added to a page / master.

My question is - all my user controls load in the same site xml. Is this inefficient or does it not make too much difference?

And if it's a worrying waste of memory - how should I start fixing it?

Thanks all.

[marcel]

It's not so much a waste of memory, but it definitely is an IO bottleneck (constantly reloading the same files from disk)

I would advise using caching, there are a number of possibilities in your case. If you do a quick search for ASP.Net caching you should find lots of examples.

I will post some possibilities for you this evening, unfortunately I do not have the time at the moment.

[marcel]

Looks like my lunch break has been extended :)

Here is the easiest, but in my opinion worst option, add the following line to you UserControl:

<%@ OutputCache Duration="30" VaryByParam="*"%>

This is my preferred option, caching the objects themselves, ie:

XmlTextReader siteXML;
// try loading from cache
siteXML = (XmlTextReader)HttpContext.Current.Cache.Get("dotnettemplate");
if (siteXML == null)
{
// not found in cache, retrieve from file
siteXML = new XmlTextReader(Server.MapPath("~/xml/dotnettemplate.sitemap"));
// save to cache
HttpContext.Current.Cache.Insert("dotnettemplate", siteXML);
}

This is a very simple example, have a look into the Cache.Insert method, there are a number of overloads, for Sliding expiration, absolute expiration, file dependancy etc. etc.

[katebp]

That's great Marcel - plenty to get my teeth into there!

[katebp]

I've been playing around with the code above for a while now and while saving and retrieving the siteXML to/from the cache isn't a problem, when the value of siteXML is set by the cache the XslCompiledTransform doesn't work. It doesn't show an error - it just doesn't display anything.

After reading this forum question: [forums.asp.net...] I think the problem is with the XmlTextReader - but other forums have stressed that XmlTextReader is preferable to XmlDocument.

So now I am even more confused!

Breaking code that was working is such a pain, even if I know it's for the best in the long-run.

My code (while I'm here):

  // site xml
  XmlTextReader siteXML;
  // loading xml from cache 
  siteXML = (XmlTextReader)HttpContext.Current.Cache.Get("dotnettemplate");
  Response.Write("// loading xml from cache ");
  if (siteXML == null)
  {
   // not found in cache, retrieve from file 
   siteXML = new XmlTextReader(Server.MapPath("~/xml/dotnettemplate.sitemap"));
   Response.Write("// not found in cache, retrieve from file");
   // save to cache 
   HttpContext.Current.Cache.Insert("dotnettemplate", siteXML, new CacheDependency(Server.MapPath("~/xml/dotnettemplate.sitemap")), Cache.NoAbsoluteExpiration, Cache.NoSlidingExpiration, CacheItemPriority.NotRemovable, null);
  }
  XPathDocument xpathDoc = new XPathDocument(siteXML);
  //Section menu
  XmlTextReader sectionXSL = new XmlTextReader(Server.MapPath("~/xslt/SectionMenu.xslt"));
  XslCompiledTransform SectionNavMenu = new XslCompiledTransform();
  // Create the XsltArgumentList
  XsltArgumentList xslArg = new XsltArgumentList();
  // Create a parameter which represents the current date and time.
  DateTime d = DateTime.Now;
  xslArg.AddParam("pageid", "", GlobalVariables.PageID);
  SectionNavMenu.Load(sectionXSL);
  StringBuilder sb = new StringBuilder();
  using (StringWriter sw = new StringWriter(sb))
  {
   SectionNavMenu.Transform(xpathDoc, xslArg, sw);
   litSectionMenu.Text = sw.ToString();
  }
  sectionXSL.Close();
  siteXML.Close();
  xslArg.Clear();
  sb = null;

[marcel]

You might be better of caching the output of the Literal, something like this:

String leftMenuNav = (string)HttpContext.Current.Cache.Get("leftMenuNav");
if (String.IsNullOrEmpty(leftMenuNav))
{
// leftMenuNav not found in cache, Create it

// Load files process etc.

using (StringWriter sw = new StringWriter())
{
SubNavMenu.Transform(xpathDoc, xslArg, sw);
leftMenuNav = sw.ToString();
}
// save to cache
HttpContext.Current.Cache.Insert("leftMenuNav", leftMenuNav);
}
litSubNavMenu.Text = leftMenuNav;
}

I've left out the loading and processing code here for clarity

[marcel]

Hi Katebp,

you mention using the XMLTextreader as the preferred method, but if I have read the code snippet correctly it looks like the XMLTextReader is only used as a temporary container before being loaded into the XPathDocument and XSLCompiledTransform objects. (and the StringBuilder doesn't seem to do anything at all)

Did the code I posted here [webmasterworld.com] not work? (in a later test it appeared to work fine for me)

[katebp]

Ahem. I missed the final post of that thread. Will go and play with that now.

[marcel]

I was a little too hasty with my idea of caching the content of the literal as I now see that it is dependant on a PageId. (with my last example you would *always* get the same output, irrespective of the PageId value)

You will be better off caching the xpathDoc and SubNavMenu objects instead. (the cached xpathDoc can then also be used in other areas of the site, ie. UserControls, Global.asax etc.)

[katebp]

Excellent, I've now cached the xpathDoc and it's all working splendidly.

My code is:

  //loading site xml from cache
  XPathDocument xpathSiteXML;
  xpathSiteXML = (XPathDocument)HttpContext.Current.Cache.Get("dotnettemplate");
  if (xpathSiteXML == null)
  {
   // not found in cache, retrieve from file 
   xpathSiteXML = new XPathDocument(Server.MapPath("~/xml/dotnettemplate.sitemap"));
   //
   // save to cache 
   Cache.Insert("dotnettemplate", xpathSiteXML, new CacheDependency(Server.MapPath("~/xml/dotnettemplate.sitemap")), Cache.NoAbsoluteExpiration, Cache.NoSlidingExpiration, CacheItemPriority.NotRemovable, null);
  }
  //
  //Sub nav menu 
  XslCompiledTransform SubNavMenu = new XslCompiledTransform();
  SubNavMenu.Load(Server.MapPath("~/xslt/InternalMenu.xslt"));
  //
  // Create the XsltArgumentList
  XsltArgumentList xslArg = new XsltArgumentList();
  //
  // Create a parameter which represents the pageid
  xslArg.AddParam("pageid", "", GlobalVariables.PageID);
  //
  using (StringWriter sw = new StringWriter())
  {
   SubNavMenu.Transform(xpathSiteXML, xslArg, sw);
   litSubNavMenu.Text = sw.ToString();
  }
  xslArg.Clear();

Hurrah! I am just looking now at tidying up / closing everything I need to, but am chuffed that it's working.

Thanks Marcel!

[marcel]

Great! I'm glad it works. Just one more thing, I would consider also caching the SubNavMenu object. Anything that will avoid disk access is good for performance.

Also, when caching, you can set a file dependancy, so that the cache will automatically be flushed if the file changes, this might come in handy if your CMS system changes the sitemap.

it's quite easy, you can cache it like this:

CacheDependency dependancy = new CacheDependency(Server.MapPath("~/xml/dotnettemplate.sitemap"));
HttpContext.Current.Cache.Insert("dotnettemplate", xpathSiteXML , dependancy);

[katebp]

Aha - I've already done that above, albeit in a slightly different way:

// save to cache 
Cache.Insert("dotnettemplate", xpathSiteXML, new CacheDependency(Server.MapPath("~/xml/dotnettemplate.sitemap")), Cache.NoAbsoluteExpiration, Cache.NoSlidingExpiration, CacheItemPriority.NotRemovable, null); 

Now it's just the xslt caching to sort out...

[katebp]

Right, I'm now caching the XSLT (in case anyone gets here via google)

XslCompiledTransform getXslFile = new XslCompiledTransform();
    XslCompiledTransform SubNavMenu;
    SubNavMenu = (XslCompiledTransform)HttpContext.Current.Cache.Get("SubNavMenu");
    if (SubNavMenu == null)
    {
      getXslFile.Load(Server.MapPath("~/xslt/InternalMenu.xslt"));
      Response.Write("loading side menu from cache");
      // save to cache
      Cache.Insert("SubNavMenu", getXslFile, new CacheDependency(Server.MapPath("~/xslt/InternalMenu.xslt")), Cache.NoAbsoluteExpiration, Cache.NoSlidingExpiration, CacheItemPriority.NotRemovable, null);
      SubNavMenu = (XslCompiledTransform)HttpContext.Current.Cache.Get("SubNavMenu");
    }

For some reason I have to introduce a new object: getXslFile rather than just use SubNavMenu, but it works so it'll do.

[marcel]

Aha - I've already done that above, albeit in a slightly different way

oh yes, I see it now... should get me some reading glasses ;)