1. Before reporting, make sure that you are using the latest version of the software.
2. Before reporting something as a bug, make sure it is in fact a bug.
3. Before reporting, play around with the bug to identify it with at least a bit of accuracy.
4. Report through the proper channel, usually an issue tracking system.
5. Make sure that the bug has not already been reported.
6. Invest some time to find a proper title for the bug.
7. State the name of the product in which you found the bug.
8. State the version of the product in which you found the bug.
9. Describe your environment (Operating System name and version, etc)
10. List the steps to reproduce the problem.
11. State the "observed result" of following the steps.
12. State the "expected result" of following the steps.
13. State the reproducibility of the bug. (Or your best estimate of it.)
14. Stick to the facts.
15. Address only one bug per bug report.
16. Include any useful attachments.
Here is a slightly more detailed discussion of each one of the above items:
1. Before reporting, make sure that you are using the latest version of the software.
If not, then your bug report has a high probability of being irrelevant. Nobody wants to waste time with irrelevant stuff.
2. Before reporting something as a bug, make sure that it is in fact a bug.
Just because the software is not doing what you expected it to do, it does not mean that the software is not working correctly. It is quite possible that the software is working precisely as intended, so what you think of as a bug is not really a bug, it is a change request. The main difference between bug reports and change requests is that in the case of bug reports, the title of the issue describes something which is unwanted, e.g. "my newly created item does not appear at the top of the list" while in the case of change requests, the title of the issue describes something which is wanted, e.g. "sort item list by creation date instead of name". If you mistakenly report a change request as a bug you are going to be sending the programmers on a wild goose change looking for recent changes in the code that might have caused the observed behavior, while there are none to be found.
3. Before reporting, play around with the bug to identify it with at least a bit of accuracy.
Programmers tend to not appreciate it very much when users report just the first superficial manifestation of anything that appears wrong. A guy filed a bug report once about an application failing to open a document which he had just created, without first bothering to check whether the document file actually existed on his disk. As it turned out after a lot of wasted time and frustration, it was perfectly normal for the application to not be able to open the file, because the file did not exist, and the bug report should instead have been about the application failing to create the file in the first place. Another guy filed a report which included a complicated sequence of steps for arriving at a particular rarely used screen which contained a text edit box whose cursor was not blinking. He did so without noticing that no cursor was blinking on any text edit box in the entire application, not even in the search field of the main screen. So, the number of steps to reproduce could have been 2 instead of 12. Do not be the guy who sends the programmers on wild goose chases: try to investigate the bug before reporting it, so as to file a meaningful report.
4. Report through the proper channel, usually an issue tracking system.
If you have been explicitly informed that proper channels for reporting bugs in your organization include sending direct emails to the programmers, sticking post-it notes on their office doors, or striking up conversations with them by the water cooler, then fine, so be it. But with most software development teams, the one and only acceptable channel for reporting bugs is a specialized issue tracking system. Yes, you will need to first learn how to use it, there is no way around that.
5. Make sure that the bug has not already been reported.
Reporting the same bug multiple times wastes the programmers time and makes them less likely to pay attention to future bug reports submitted by the same person. Decent issue tracking systems tend to scan the text of your bug report and try to find similar bug reports for you, in case one of them is about the same issue. Of course, in order for this feature to work, you need to be verbose while describing the bug, so that the issue tracking system has enough keywords to search for. If your bug has already been reported, amend the original report with whatever new information you might have.
6. Invest some time to find a proper title for the bug.
The title should contain as much information as possible in an as concise form as possible, so as to quickly give an accurate idea of what it is about without delving into unnecessary details. Also, the title should uniquely identify the bug among all existing similar bugs and all future similar bugs. Ideally, if a bug pops up in the future whose title would be exactly the same as the title of the bug you are currently filing, then that future bug should be nothing but a re-appearance of the bug you are reporting. So, for example, "problem with the profile screen" is a very bad title for a bug, because there is a huge number of entirely unrelated things that can go wrong with a profile screen. A better bug title would be "profile picture remains unchanged when a new one is uploaded".
7. State the name of the product in which you found the bug.
The development team may have produced many different applications, so they need to know precisely which one you are talking about, but even if they have produced only one, chances are that they would still like to see that you are capable of calling it with its proper name before they accept a bug report from you. So, be sure to state the name of the product in which you found the bug, and be sure to use the precise product name as reported by the product itself, (e.g. "Acme Point of Sales For Windows") instead of the name that you use in every day language, (e.g. "the store app") or the name that you think is the name of the product even though you have never really looked it up. (e.g. "acme". Which, in this case, happens to be the name of the company, not the name of the product. Not only programmers, but even technical support personnel hate it when users do that.)
8. State the version of the product in which you found the bug.
It goes without saying that without the version number, the report is useless. Also keep in mind that the bug might persist for several versions or internal releases, so every time you come across the same bug in a later version, amend the bug report to indicate that it is still there.
9. Describe your environment (Operating System name and version, etc)
The programmers will probably need to know at least a little bit, possibly even a lot, about the software and hardware environment that you were using at the time that the bug manifested itself. This will include at the very least your operating system name and version. If the programmers need more than that, then generally, they will have (or they ought to have) produced some guidelines explaining precisely what additional information is required for bug reports on their product. Be thorough and meticulous: give the precise names and version numbers as reported by the systems themselves (e.g. "Windows 7 Ultimate 64 bit, Version 6.1, Build 7601, Service Pack 1") instead of the names that you would use in every day language (e.g. "64 bit Seven.")
10. List the steps to reproduce the problem.
Even if the programmers opt to believe you when you say that you "saw a bug", there is not much they can do about it unless you tell them precisely what you did that caused the bug to manifest itself. Begin with the very first step, no matter how obvious, (e.g. "launch Acme Point of Sales For Windows") and continue listing steps until the problem occurs. Be sure to list not only what you perceive to be the intended results of your actions (e.g. "save the file") but also your actual actions (e.g. "save the file by hitting Ctrl+S") not only because these two may, unbeknownst to you, differ, (Ctrl+S might do something other than saving the file,) but also because different ways of achieving the same thing might have different side-effects, so the programmers need to know about those, too. (Ctrl+S might just save the file, while other ways to save the file might involve opening up a file-save dialog, which in turn involves loading resources needed to display the dialog, instantiating file preview plugins, transitioning the application GUI in and out of a modal state, etc.) Be very clear and precise, and make sure that what you wrote can be understood unambiguously and effortlessly by someone who is not you. If you have found multiple significantly different scenarios to reproduce the same bug, list them all. If you have found some sequence of steps which gets the job done without triggering the bug, be sure to also include it.
11. State the "observed result" of following the steps.
Explain precisely what it is that you consider to be the bug, no matter how obvious it may seem to you. Simply saying "open the properties screen and then boom- there it is!" will not do. The programmers may have stared at the properties screen many times in the past without noticing that which you have noticed, or they may be aware of multiple different issues within the properties screen, which you know nothing about. They may even arrive at a perfectly fine properties screen after following your steps, so they need to have some quick and definitive way of knowing whether they are seeing the bug that you have seen. No programmer likes having to guess something which the bug reporter could have easily stated. Also, a great percentage of bug reports are false reports about things that are the way they are by design and not by accident, so the programmers need to be able to tell right up front whether you are reporting an actual bug, and not some feature which you do not quite like, or not quite understand.
12. State the "expected result" of following the steps.
Explain precisely what you expected to happen in place of that which actually happened and which you consider to be a bug. In many cases it is just a negated form of the statement made in the previous item, (e.g. "Observed result: the application crashes. Expected result: the application should not have crashed.") so it is not very useful, but in many other cases it can be quite a bit more descriptive than that, and that's where it is really useful. (e.g. "Actual result: nothing happens. Expected result: the text should have started dancing on the screen.") Diligently filling this in, no matter how obvious it might seem, shows that you are not just any random person out there, but an insider of the art of software testing and bug reporting. If there happen to be any specifications documents describing what the software should do, this is the right place to include a reference to the section of the specification document which states what should have happened.
13. State the reproducibility of the bug. (Or your best estimate of it.)
On most systems, most bugs manifest themselves each time someone follows the steps to reproduce them. Such bugs are said to have a reproducibility of 100%. Unfortunately, some bugs only manifest themselves part of the time, and some particularly nasty ones manifest themselves very rarely. Be sure to state the reproducibility of the bug that you are reporting. If you do not, then the person reviewing your bug report will assume 100% reproducibility, so if the bug does not manifest itself on the first try, they will close your bug report with a "works for me" comment. Nobody will waste time repeating the same steps over and over just in case this is a less-than-100%-reproducible bug which was simply not reported as such. Of course, in order to estimate the reproducibility of a bug, you will need to trigger it a few times. Triggering it only once is generally unacceptable, and besides, if you trigger it only once then clearly you have not played around with it enough in order to identify it with accuracy, as a previous item in this checklist requires.
14. Stick to the facts.
The programmers are primarily interested in the facts, not in your interpretations of the facts, nor in your assumptions about what the facts are. If you would like to include your thoughts besides the facts, that is fine, but be sure to clearly indicate where the facts end and your own opinions begin, and be sure that in your eagerness to start writing your opinions you have not left out any of the facts.
15. Address only one bug per bug report.
A different bug should receive a separate bug report. When in doubt, filing two different bug reports is the way to go, because a bug report can easily be marked as a duplicate of another, whereas multiple bugs rolled into one report are not as easy to split into separate bug reports.
16. Include any useful attachments.
A screenshot showing the problem can be very useful to the people who will be troubleshooting it. If the bug only happens when you use the application with some specific external file, be sure to attach a copy of that file to your bug report. For example, if "the profile picture remains unchanged after uploading a new one", then please do attach the picture which you tried uploading. Also, if the application you are using has undergone any configuration from your part, then you need to attach your configuration files with your bug report, or else your bug report might not make sense to the programmers. For example, if you have assigned a key to perform a certain action, then stating in your bug report that you pressed that key is meaningless, because the programmers do not know what action you have assigned to that key. Your configuration file will tell them. Also, many applications log everything they do in special "debug logs". If you have a debug log, be sure to attach it to your bug report. Note that attachments are useful only in addition to the previous items in this checklist, and never as a substitute to any of them, so the fact that you included an attachment does not in any way exempt you from having to fill in all of the above. "Actual result: see attachment" is unacceptable. Please describe the actual result with words, (hint: searchable keywords,) and follow your description with "also see attachment."
--michael.gr
No comments:
Post a Comment