TST Discussions Rss Feed

New Post: Testing login permissions

JohnMayo — Thu, 02 May 2013 18:26:28 GMT

I need to write tests to validate the permissions for various application logins. I'm planning on using the EXECUTE AS LOGIN = 'ApplicationLogin'; and REVERT statements in the tests. Is that the best way to tackle this problem? Any advice on how best to test permissions?

New Post: Change TST.Bat to allow for UserName/Password instead of only supporting trusted connections

JohnMayo — Thu, 17 Jan 2013 23:22:02 GMT

In the next version, can you please change the TST.Bat file to allow a username/password combination to be used when a trusted connection can't be used.

New Post: Output XML in JUnit format for CI?

johnmayo — Tue, 26 Jun 2012 03:56:59 GMT

If you work on the JUnit XML approach, please post here how that worked out for you.

New Post: Output XML in JUnit format for CI?

larsthorup — Tue, 26 Jun 2012 03:07:36 GMT

Thank you for your input! I can see that outputting TeamCity tags might work for TeamCity. However, I would like to have a more generic solution, and still believe that outputting in the generic JUnit XML format might be useful.

New Post: code coverage

johnmayo — Mon, 25 Jun 2012 21:49:33 GMT

No, I haven't made these extensions available yet. I need to clean up the code and put tests around the functionality and haven't had the time to do so yet.

New Post: Output XML in JUnit format for CI?

johnmayo — Mon, 25 Jun 2012 20:52:24 GMT

I was able to to integrate TST tests with TeamCity by creating a version of Internal.RunOneSuiteInternal that prints out a few things around the test:

##teamcity[testStarted name=''TestNameGoesHere"]

And after the test either:

##teamcity[testFinished name=''TestNameGoesHere'' duration=''TestDurationGoesHere']

##teamcity[testFailed name=''TestNameGoesHere'']

depending if the test passed or failed.

I put that version of Internal.RunOneSuiteInternal in a different database along with modified versions of versions of the Runner.RunAll, Runner.RunSuite, Internal.RunTestSession and Internal.RunOneSuiteInternal that called it. The change was pretty easy. I've also added statement coverage statistics to that code. Clean up the code and getting some tests around it are one of my backburner projects.

New Post: Output XML in JUnit format for CI?

larsthorup — Mon, 25 Jun 2012 04:13:22 GMT

I would like to run my T.S.T tests on TeamCity or Jenkins for Continuous Integration. I believe the XML format currently produced by T.S.T is not consumable directly by TeamCity or Jenkins. Would I have to write an XSLT transformation to the ubiquitous JUnit format myself, or is there some such transformation available already? Or are there better way to reach my goal?

New Post: How does Assert.TableEquals work?

lmolnar — Sat, 02 Jun 2012 23:59:41 GMT

Yes, #ExpectedResult and #ActualResult are the one that are compared and this cannot be changed. Take a look at the documentation where this is explained in details. The TST QUICK start database has some examples as well.

New Post: How does Assert.TableEquals work?

AnjanaPatil — Thu, 31 May 2012 01:33:09 GMT

Hi!

I am new here, I was wondering how does Assert.TableEquals work?

I expected Assert.TableEquals ActualTableName ExpectedTableName

Or is that it only and always compares tables by following names #ActualResult and #ExpectedResult?

Thanks

Anjana

New Post: code coverage

whgibbo — Tue, 10 Apr 2012 11:22:42 GMT

Hi John,

Are these extensions available ?

Thanks

New Post: Assert.TableEquals does not support the DATE data type

lmolnar — Sun, 01 Apr 2012 03:04:02 GMT

Implemented in V1.9.

New Post: Add @ExpectedErrorLikeMessage to Assert.RegisterExpectedError API

lmolnar — Sun, 01 Apr 2012 03:03:43 GMT

Implemented in V1.9.

New Post: RegisterExpectedError but got different error text

lmolnar — Sun, 01 Apr 2012 03:03:21 GMT

Implemented in V1.9.

New Post: code coverage

johnmayo — Fri, 30 Mar 2012 12:11:16 GMT

It is possible to get statement coverage statistics and I've written some extensions around TST to do that. The approach I took was to add comments into the code being tested tagging each line of code, parse the stored procedures at the beginning of a test session for the tags and then ran a trace while the tests were running which could then be parsed for the tags. The end result was a list of tags for all of the statements that could be run and a list of tags for all of the statements that were run which made it easy to generate both statement coverage statistics and a list of which code coverage tags weren't run.

New Post: code coverage

whgibbo — Fri, 30 Mar 2012 07:31:25 GMT

Hi,

Does anybody know if it is possible to get any code coverage status from the tests?

If so, could you please explain how ?

Many thanks

New Post: TST for Data validations

lmolnar — Sun, 25 Mar 2012 16:33:27 GMT

Short answer:
TST is designed primarily for code validation and not for data validation. There are some downsides in using TST for data validation (see below).

Long answer:
The question is: What kind of ETL jobs are we talking about? Are those actual production mode ETL jobs with real data or test ETL jobs where you seed the data for the purpose of exercising test scenarios? From your description I assume we are talking about the first case: production mode ETL jobs.

The first case: If your goal is to make sure that production data is in good shape and you want to validate production data that results after ETL jobs are run. Using TST for this scenario has one important downside: One individual TST test procedure will stop at the first assert that fails. This means that if you have a test that has several asserts you will only be able to see the first failure and not be sure about the rest. Also I imagine that after you run the ETL you not only want to see if a certain relation is broken but you want to see in how many places and where are those places (as in how many rows and which rows). That may also be cumbersome to do with TST.

Keep in mind that in this case, what you do is to validate particular data and you will not necessarily be able to find certain bugs that you may have in your stored procedures. And this brings us to the second case:

The second case: If your goal is to make sure that you don’t have bugs in your stored procedures. Validating data after the production ETL is run, is not a reliable way to achieve this goal. There are several issues:
1. You find out about the bug only after the data is already impacted.
2. There are bugs that code validation will catch easily and data validation will not catch at all. This is probably the worst case. You end up running with bugs that may not be discovered or are discovered by chance long time after data corruption is introduced in production.
3. You may have bugs that will not be hit in the production for quite a while. Fixing a bug found 6 months after the system was released to production is most of the time way more difficult than fixing the bug 10 minutes after you introduced it.

If your goal is to do data validation (first case) then TST is not ideal. If your goal is to make sure that you don’t have bugs in your stored procedures (second case), TST is a good tool.

New Post: Assert.TableEquals does not support the DATE data type

lmolnar — Sun, 25 Mar 2012 03:28:39 GMT

ETA: April 7th or earlier

New Post: Add @ExpectedErrorLikeMessage to Assert.RegisterExpectedError API

lmolnar — Fri, 23 Mar 2012 11:33:04 GMT

ETA: April 7th or earlier.

New Post: RegisterExpectedError but got different error text

lmolnar — Fri, 23 Mar 2012 08:36:52 GMT

ETA: March 7th or earlier

New Post: TST for Data validations

archanagvnk — Mon, 19 Mar 2012 22:32:00 GMT

Hi, I am working as a SDET in Database Testing. I am tryitng to find out a tool for Testing stored procedures in my SQL server. We have some jobs for ETL which runs these stored procedures. I need to test the Databse for data validations after stored procedures. Most of the tables don't have foreign keys. Any ideas about creating automated test cases in T-SQL language ? Is TST best for it ? I am not good in C# or .NET so Iam looking for T-SQL tools. Please let me know.

Archana