Testing Testing, 1, 2, 3, 45678...

As one of the first consultants to work with a client, I'll sometimes get questions about testing approaches with SDL Tridion. I've served as a business analyst before and have done light testing, but the Tridion-related answer for testing really depends on what you mean by testing.

For example, I once had a research prompt to find performance monitoring and testing tools for my previous .NET teams and found out the following challenges.

• IT Operations "testing" focuses on bandwidth, packets sniffing, and a more appliance approach. They want to get insight on the stability of systems.
• Developers are focused on personal toolkits of sorts. They can purchase such tools almost out of pocket and are easily approved through a PO, signed by a manager. Think "shrinkwrap" like the options from RedGate.

But this is only a performance subset of "testing" and it's doesn't necessarily focus on stress or load testing.


Rehman Zafar describes at least ten types of testing and I'm not an expert at them, but we did cover them in college (University for my European readers):

• Unit Testing
• Integration Testing
• Functional Testing
• System Testing
• Stress Testing
• Performance Testing
• Usability Testing
• Acceptance Testing
• Regression Testing
• Beta Testing  

Architecturally, SDL Tridion separates content from presentation in a fractal-like manner from Content Management versus Content Delivery, down to field names and descriptions. There are two sides to test: Content Management (Tridion) and Content Delivery (your presentation layer).

Content Delivery Testing is "Easy"

The good news is that all of your Content Delivery testing has the same "footprint" of what you've tested before. You'll still test markup, script, and application code after introducing Tridion as your CMS. This covers delivery-side Functional, System, Stress, Performance, Usability, Acceptance, Regression, and visitor Beta Testing under the Nunoism, "How would you do it without Tridion?" The only thing somewhat unique to Tridion is Acceptance Testing might be on a specific server that Tridion publishes to and that any issues surfaced might relate to Tridion delivery-side features.

What's left includes testing things like the Content Manager, its Editors, and "Publishing."

Templating Testing Examples

Before I get into all the things you might consider on the Content Manager (CM) side, here are some unit and integration test ideas followed some recommendations for Functional Testing.

• Do Unit Testing with Template Builder, optionally using the examples from T-Cubed
• An automated integration testing post and though not quite Integration Testing, but Building Blocks describes a Continuous Integration approach for C# Template Building Blocks

Aside from template code, which is probably the best quick win you might start with, you can break CM-side testing down along different types of testing as well as by interface.

CM-Side Testing

On the Content Manager side, your focus for testing should be on the implementation or the specific Tridion content model, extensions, workflow, and automation developed for an organization. Testing Tridion itself is really up to the vendor, though you should keep notes and report things that feel like a product issue rather than a implementation problem.

• Functional Testing should be based on your functional requirements (learn more about an SDL Tridion Functional Design)
• Usability Testing falls a lot on the product itself, but whether the entry forms (based on schema) are "usable" depends on how you've created your Tridion content model (100 fields for a promo isn't Tridion's fault, but that fact that you added 100 fields to the promo schema)
• Beta Testing and Acceptance Testing for new CM-side functionality would happen in a lower Tridion DTAP environment
• Regression Testing would also be based on your functional requirements to an extent

Within the Content Manager editors (the Content Manager Explorer and Experience Manager), this includes:

• Organization and Authorization: Blueprint, Folders, Structure Groups (Pages), and Category/Keywords
• Creation in Content Manager Explorer: schemas, pages (add components with templates), and field options
• Creation in Experience Manager: page types, content types, and regions, as well as publishing to just Staging and "Update Preview

In terms of Content Delivery (CD), also look at:

• Publishing and Integrations: publishing pages to the site(s) and confirming the results, you can check this down to separate phases (rendering, transport, deployment, and commitment) if needed
• Testing across each publication targets and CMS environments

Check these manually or can consider using scripts and code. The Content Manager offers the Core Service API to let you programmatically inspect Tridion items, whereas CD testing would be through the CD Webservice (if used), the CD API, and possibly broker database queries (read only, if at all!). You're looking to confirm pages, component presentations, and metadata in CD.

For Experience Manager, it really depends on your implementation. But you'll want to confirm functionality in terms of:

• Page Types and Content/Component Types
• Items saved to correct publication
• Visibility and permissions if set up
• Confirm Component Presentation combinations work as expected (available schemas work with different templates), prioritizing important combinations
• Test regions work as expected for allowed content types (pairs of components and component templates)

Final (Initial) Thoughts

Here are some other considerations, recommendations and gotchas. Focus on the parts that are important to you and/or combining scenarios into your test criteria.

• Scope and tools
• Break down tests down by important variables (browser, schema, field, template, extension, steps in the process), list out options (in Excel or a similar list), and test manually.
• Optionally consider a tool like Selenium for screen automation and/or log analysis / reporting tools (BareTail, Splunk, etc). I haven't had a chance at using these specifically with the new UI, but it can they can help you validate the tags and rendered pages in general.
• Implementation option: for ease-of-testing, add template and component information via TBB, only rendered to Staging. These could let you manually inspect and validate tcm ids and other information.
• Consider using validation tools or sites to confirm markup
• Test with at least the browser your authors will use ("it works in <my browser>" discussions occasionally apply to the CME)
• Test rich text fields
• See if empty fields affect your presentation / editability
• Confirm required fields match your design
• Verify any schema facets (xsd filters)
• Copy and paste text, use shortcut combinations, and otherwise replicate what your authors would expect from the editor
• Character combinations and special characters (&, -, -, (c), (r))-do templates and presentation escape or encode these as expected
• Confirm UTF-8 character set (or evaluate why you'd choose something different!)
• Verify XSLT rich text filter behavior on fields (if any)
• Evaluate rendering
• Confirm pages with heavy JavaScript, especially if you manipulate the DOM (HTML node tree) in ways that may affect the UI tags
• Check for issues with any code-driven rendering functionality
• Validate the non-rendered source (from CD file system, if possible)
• Experience Manager:
• Tip: avoid block elements such as headings or <div> within <span>
• Browsers may force changes to bad markup, affecting Experience Manager

Consider creating and using a set of dedicated test content which could help with future (regression) testing. Capture any of the typical issues you've seen with CMS and Web technology in general. For future schema, consider collecting some test text that includes the above test scenarios.

But this is just an overview (that probably would read better as a video or checklist) and it isn't necessarily a new topic (read Dominic Cronin's Total Quality post circa 2006). Got thoughts on "Testing Tridion?" Tell us about them in a comment or a post. If you've been trying to keep up with the new new community, we're almost through Q1 and "testing" might be a great topic for your next post or two.