Module talk:ScribuntoUnit

I think there is a need for an assertion that checks that a string contains expected text. Motivation is as follows: you should test behaviour, not test methods and each test method should only test one thing. Suppose you have a complicated module (pl:Moduł:Koordynaty) that 1) adds a formatted link 2) prints out a formatted text 3) adds the page to a calculated category (and maybe will do other things in the future we cannot predict, and we don't want people adding one new feature to fix assertions for all other features just because module output has grown). To write a proper unit test, you should write 3 separate test methods, each checking only one functionality and no other, so that when only 1 of the functionalities is broken, exactly 1 test fails and no other. To achieve this, you need a method like

assertThat(coordinates.invoke("53 45", "23 45"), containsString("geohack/53.45W_23.45E/type:city"))

where you check a part of the module output and don't care about the rest. Also, there are modules that create multi-line outputs. Tests need to be readable and check only what is relevant. Therefore an "contains string" assertion would be handy.

In JUnit, you can deal with the lack of "assertContains" by using assertTrue() with a custom message displayed when the assertion fails. Here, there is no way to pass the failure message. Hence, assertTrue from ScribuntoUnit is not suitable for implementing "contains string" assertions, because there will be no meaningful error message.

I suggest either adding a new method "assertContains" or extending "assertTrue" with a failure message - or perhaps performing both. --Derbeth ^talk 21:40, 1 January 2014 (UTC)[reply]

That sounded like a good idea to me, and it will be useful for Module:Documentation which I'm working on now, so I went ahead and added an assertContains method. Take a look and let me know if you have any suggestions for how it could be improved. — Mr. Stradivarius ^{♪ talk ♪} 16:32, 2 January 2014 (UTC)[reply]

It looks great, thank you :) Do you have a page where you check the assertions work correctly and that their diagnostic messages look good? I did not find Module:ScribuntoUnit/testcases. --Derbeth ^talk 20:01, 2 January 2014 (UTC)[reply]

Great. But perhaps we should name it assertStringContains in case we later implement a method that checks if a table contains a given table key? And yes, we should have tests for the assertions as well. I'm not really sure how to write good tests for a testing module, but I started experimenting on Module:ScribuntoUnit/testcases. – Danmichaelo (talk) 08:42, 3 January 2014 (UTC)[reply]

So renamed. Sounds like a good reason to me. — Mr. Stradivarius ^{♪ talk ♪} 14:05, 3 January 2014 (UTC)[reply]

Automated tests are needed, but I think there is also a need for a visual presentation of how the module works in practise. I have created such a page: Module:ScribuntoUnit/showcase (run on the talk page). If I did not invoke something wrong, it seems that assertWithinDelta ignores the custom message. Plus, test methods are run in no particular order - can we do something about it? --Derbeth ^talk 21:08, 3 January 2014 (UTC)[reply]

Agree, the showcase page is a good idea. Seems like the output is not so consistent between the different methods; on failure, some methods include "actual" and "expected", while others do not. If "actual" and "expected" are included, "message" is not shown. This leads me to thinking that we might be better of just replacing the two columns by a single column. I updated Module:ScribuntoUnit/sandbox and Module talk:ScribuntoUnit/showcase. What do you think? – Danmichaelo (talk) 18:21, 5 January 2014 (UTC)[reply]

I made small tweaks to the sandbox code. I like it now. --Derbeth ^talk 10:31, 6 January 2014 (UTC)[reply]

The docs for assertDeltaContains assertWithinDelta says " While 1/3 == 9/3 evaluates as false, ...". This is an obvious result, as they are completely different numbers. I assume this was a typo for "1/3 == 3/9 evaluates as false"; however, when I tried this in the debug console just now it evaluated as true. Is there a better example we can use to demonstrate floating point error here? Personally, I haven't seen it action in Scribunto yet, so I can't think of a good one. — Mr. Stradivarius ^{♪ talk ♪} 14:02, 3 January 2014 (UTC)[reply]

True, that was a completely wrong example :) I've added a new one, and tested it in the debug console. Btw., the reason I added the method was that I ran into the problem on no:Module:Coordinate/testtilfeller. – Danmichaelo (talk) 00:45, 5 January 2014 (UTC)[reply]

Thanks for clearing that up. :) It's interesting to see how the floating point error works in a real-world example. I've given the docs a copy edit and expanded them a little to note that integers can be represented exactly up to 2^53. Feel free to tweak that if you want. — Mr. Stradivarius ^{♪ talk ♪} 11:48, 5 January 2014 (UTC)[reply]

At the moment, if two different tables are tested for equality, the wikitable generated by the module simply lists them both as "table". It would be nice to actually show the contents of the table in a reasonably nice format. Perhaps we can do something with Module:User:Anomie/deepToString and/or mw.logObject? deepToString fails when trying to display mw.title objects, and while mw.logObject handles them ok, it only outputs to the log buffer. So we would need to do some tweaking to get something that suits our purposes, but it probably wouldn't be too hard. — Mr. Stradivarius ^{♪ talk ♪} 12:00, 5 January 2014 (UTC)[reply]

Hm, how does it fail? I tried using mw.title at Module_talk:ScribuntoUnit/showcase, and it seems to work, but I'm not familiar with it. – Danmichaelo (talk) 18:47, 5 January 2014 (UTC)[reply]

You used the mw.title library, rather than a title object. To get a title object you need to use code like mw.title.new('Wikipedia:Sandbox'). I tried doing just that in the showcase module, and the whole script fails with a "not enough memory" error. The problem is that some of the title object fields are self-references. For example:

local sandbox = mw.title.new('Wikipedia:Sandbox') -- title object for [[Wikipedia:Sandbox]]
sandbox.basePageTitle -- Also the title object for [[Wikipedia:Sandbox]]
sandbox.basePageTitle.basePageTitle -- Again, the same title object.

So if you try and iterate recursively through any given title object, while respecting metatables, you will just keep on going forever. Scribunto then dies when it runs into the memory limit. This problem has been resolved somehow in mw.logObject, but I haven't looked deeply enough at the code to know exactly what was done. — Mr. Stradivarius ^{♪ talk ♪} 11:06, 6 January 2014 (UTC)[reply]

@Luis150902: thanks for your recent module config changes. A while ago I adapted this module to be used at the English Wiktionary. One thing I found lacking was a category the test page automatically gets assigned to upon failure. Maybe this could now be added as a configurable setting? I would also suggest to move the configuration into Module:ScribuntoUnit/configuration or similar, so that the module can be copied as is. – Jberkel (talk) 05:22, 26 February 2018 (UTC)[reply]

@Jberkel:  Done Luis150902 (talk | contribs) 18:35, 26 February 2018 (UTC)[reply]

Hi, would anyone be interested in collaborating on making this module multilingual using Module:TNT? With this approach, we will get these benefits:

• identical code on all wikis, using global translation table
• no need for config submodule - can store than in the translation table too
• Once in sync, keeping it in sync will be easier with multilingual distribution system.

--Yurik (talk) 22:25, 5 May 2019 (UTC)[reply]