Archive for the ‘Ramblings’ Category

The 80% Rule for ML Results

Sunday, May 26th, 2019

One remarkable thing that easily deceives us with machine learning is that how quickly one can see good, encouraging results.

When developing a new algorithm during my time in grad school I often had some bug in the implementation. The results despite the bug were still good, but not as good as the final, bug free version (as far as we know). Despite bugs, the system was learning and doing something.

When training a new model using data from a system that has a bug we still get a working model. Removing the bugs will often result in sizable performance improvements of the model, more than adding a new feature would have.

When starting new projects in my experience we quickly see great results. Often the simplest thing gets us about 80% to where we would have a new product or a launch-able new model. Demos often look interesting, the system doing something right and a few funny cases wrong. Often people become excited and expectations rise for when this new thing may ship.

I feel it’s important to test carefully, have a metric and specific criteria for launch that measure performance against a ground truth. This may sound obvious, but many new techniques (GANs etc.) there’s often no ground truth and performance is eyeballed. Again, the first 80% are easy. The remaining percentage points will take a while…

Deferring Decisions to AI

Wednesday, February 14th, 2018

Would you let an AI make decisions for you? With all the buzz about AI and calls for its regulation (apparently AI will kill us all some day), I’ve been thinking about under what circumstances we might delegate decision making in certain areas to machines (call it AI). That’s not as crazy as it sounds, because mankind has been doing that for years already.

(more…)

Big Data, Predictive Policing and the Tyranny/Anarchy Trade-Off

Saturday, August 17th, 2013

Bloomberg had an interesting article today about a talk on the implications and privacy trade-offs of predictive policing and profiling. Jim Adler’s “felon classifier” is also described in his blog.

Basically, he built a classifier predicting from some innocuous (but possibly correlated variables) the likelihood of somebody having a felony offense. The classifier isn’t meant to be used in practice (from eye-balling the Precision/Recall curve in the talk slides, I estimate an AUC of about 0.6-ish; not too great), but it was built to start a discussion. It turns out that courts have upheld the use of profiling in some cases as “reasonable suspicion,” a legal standard for the police to stop somebody and investigate. This could lead to “predictive policing” being taken even further in the future. Due to the model outputting a score Jim also discusses the trade-off of where the prediction of such a model may be actionable – he calls it the Tyranny/Anarchy Trade-Off (a catchy name 🙂

Having done statistical work in criminal justice before, I think predictive analysis can be helpful in many areas of policing and criminal justice in general (e.g., parole supervision). On the other hand, I find profiling and supporting a “reasonable suspicion” from statistical models unconvincing. I think the courts will have to figure out a minimum reliability standard for such predictors, and hopefully they’ll set the threshold far higher than what the ‘felony classifier’ is producing. There’s just too many ways using a statistical model for “reasonable suspicion” to go wrong. Even if variables of protected classes (gender, ethnicity, etc.) are not used directly, there may be correlated variables (hair-color, income, geographic area) as discussed in the talk Jim gave. Even more problematic in my mind would be variables that do not or hardly ever change, as they would lead to the same people being hassled over and over again. Also the training data from which these models are built is biased since everybody in it by definition has been arrested before. It’s beyond me how one can correct for this sample bias in a reliable way. Frankly, I don’t think policing by profiling (statistical or otherwise) can be done well, and hopefully courts will recognize that eventually.

 

Programs stealing the input focus

Sunday, June 7th, 2009

Ok, this post is more of a rant. I’m one of those people that are a bit impatient when starting a program on my desktop. When I start up my Windows machine I click on several buttons in the “quicklaunch” bar to fire up what I’ll need to use – Outlook, R / SPSS /SAS, Winamp etc. So why do all sorts of dialogs pop up in my face while I am typing? Why does winamp have to pop up while I’m typing my email password? And why do they have to switch the input focus so that whatever I’ve happen to type now ends up in the wrong window? This is so annoying. Stealing the input focus is a known problem that has been written about countless times. It’s even against the GUI programming guidelines. “Do not steal the input focus” – what’s so difficult about that?

As a first consequence Norton Internet Security is now gone from my machine forever after it kept reminding me constantly – specifically with an uncanny accuracy when I was busy playing computer games – that I need to renew my anti-virus subscription or bad things will happen to my computer. And bad things did happen to my video game. But not anymore…

On the upside, there’s a carefully hidden option in the Windows XP Powertoys (TweakUI) that is supposed to prevent programs from stealing the input focus. It made things better, but doesn’t seem to work all the time.

Famous bugs in AI game engine caught on tape

Saturday, May 2nd, 2009

Found this on aigamedev and some of them are really hilarious: AI game bugs caught on tape

Vundo?

Thursday, April 16th, 2009

My girlfriend caught a new (?) version of some malware on her machine; what a nuisance and scanners don’t seem to recognize this thing… Some think it’s Vundo others just complain that it’s packed. It doesn’t quite fit the Vundo description,though. MD5 8e06f428178cbfbf12a8372fa6b16d0d size 50688 bytes. It registers some CLSID 721ee819 – b263 – 42e0 – a594 – b82fd0f24bdf , a browser-helper object and various things for notifications by the LSA service plus AppInit_Dll. It constantly restores these keys and it seems that even stomping out all the threads that this DLL-thing spawned everywhere won’t help. I overlooked something and it just comes back as soon as the next GUI app is started. As soon as I know how to get rid of it, I’ll update this post.

Update 1:

It hooks AppInit, the run key using rundll32 to start itself and the LSA notification (something Hijackthis doesn’t check). I can kill all the threads that this thing generates in each executable with ProcessExplorer and regmon will show that the constant checking of the appinit-key stops. However, as soon as the next GUI application is started it is back. So I deleted all the events and mutex objects that things created (I found some clues in the strings in memory) in each executable, again making sure that I didn’t miss anything, and it took a few seconds this time for it to come back. There’s “something” that will load the DLL with OpenProcess to load the DLL into the process space. Since the strings in the DLL show that it opens and writes to process memory this wouldn’t be surprising; question is how I find the threads that do this. Other odd things include that svchost starts a window-less iexplore.exe presumably to upload some stuff to a server or something. It might have some sloppy rootkit (RootkitRevealer went nuts with file-system discrepancies), because I can’t find the DLL (using “dir”) referenced in the keys, yet the tab-extension finds it and overwriting the non-existant file gets an access denied. Some interesting strings from the decrypted memory image of the DLL:

wscntfy.exe wscntfy_mtx mrt.exe explorer.exe iexplore.exe opera.exe firefox.exe Global\ mrt.exe explorer.exe iexplore.exe opera.exe firefox.exe dll .tmp exe rdl InprocServer32 \Internet Explorer\PhishingFilter Enabled Rundll32.exe ” ThreadingModel Both \Internet Explorer\ieuser.exe -Embedding tmp MS Juan cpm las SHELL32.dll ole32.dll OLEAUT32.dll vector<T> too long unknown ntoskrnl.exe ntkrnlmp.exe ntkrnlpa.exe ntkrpamp.exe Mozilla/4.0 (compatible; MSIE 6.0) WinNT 5.1 LoadLibraryW Kernel32 SeDebugPrivilege http://82.98.235.208/form/index.html exficale.com pancolp.com /frame.html url suid dnsapi.dll DnsQuery_A DnsRecordListFree Global\ wuauserv SYSTEM CURRENT_USER Advapi32.dll ConvertStringSidToSidA IsWow64Process kernel32 shell32.dll SHGetKnownFolderPath wininet.dll InternetOpenUrlA HttpOpenRequestA InternetCloseHandle InternetConnectA InternetOpenA InternetSetOptionA InternetQueryOptionA HttpQueryInfoA HttpSendRequestA InternetReadFile HttpAddRequestHeadersA HTTP/1.1 POST Content-Length ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/ InprocServer32 setupapi.dll IsUserAdmin BITS b’kJ SHGetFolderPathW CoCreateInstance CoTaskMemFree CoInitialize CoUninitialize CoCreateGuid __dllonexit _onexit _XcptFilter _initterm _amsg_exit _adjust_fdiv WriteFile FlushFileBuffers LocalFree CreateFileW GetFileSize VirtualAlloc ReadFile VirtualFree GetModuleFileNameW lstrcpyW CreateMutexW GetLastError WaitForMultipleObjects GetExitCodeThread lstrlenW OpenMutexW WaitForSingleObject GetProcAddress GetModuleHandleA OpenProcess VirtualAllocEx WriteProcessMemory CreateRemoteThread VirtualFreeEx CreateToolhelp32Snapshot Process32FirstW lstrcmpiW Process32NextW GetCurrentProcess OpenEventW SetEvent Sleep ResetEvent lstrcatW MoveFileW MoveFileExW SetFilePointer SetEndOfFile ReleaseMutex GetModuleFileNameA DisableThreadLibraryCalls ExitProcess LoadLibraryW InitializeCriticalSection DeleteCriticalSection EnterCriticalSection LeaveCriticalSection GetSystemTimeAsFileTime FreeLibrary LoadLibraryA GetLogicalDriveStringsW GetDriveTypeW DeleteFileW GetTickCount GetCurrentThreadId CreateDirectoryW GetSystemTime SystemTimeToFileTime SetFileTime GetWindowsDirectoryA GetVolumeInformationA CreateProcessW OpenMutexA OpenEventA GetCurrentThread GetCurrentProcessId TerminateProcess TerminateThread CreateEventW WideCharToMultiByte HeapAlloc GetProcessHeap HeapFree SetFileAttributesW InterlockedIncrement InterlockedDecrement GetVersion lstrcmpiA lstrcpynW InterlockedExchange InterlockedCompareExchange RtlUnwind QueryPerformanceCounter UnhandledExceptionFilter SetUnhandledExceptionFilter KERNEL32.dll CallNextHookEx SetWindowsHookExA PostMessageA UnhookWindowsHookEx GetSystemMetrics USER32.dll OpenProcessToken LookupPrivilegeValueA AdjustTokenPrivileges RegCreateKeyExW RegDeleteValueW RegFlushKey RegCloseKey RegDeleteKeyW RegQueryValueExW RegSetValueExW RegOpenKeyExW SetSecurityInfo RegEnumValueW GetTokenInformation IsValidSid ConvertSidToStringSidW OpenSCManagerA OpenServiceA ControlService ChangeServiceConfigA AllocateAndInitializeSid CheckTokenMembership FreeSid InitializeSecurityDescriptor SetSecurityDescriptorDacl ConvertStringSidToSidA SetEntriesInAclA DuplicateTokenEx SetTokenInformation GetLengthSid SetThreadToken RegQueryInfoKeyA RegEnumKeyExA RegOpenKeyExA RegQueryValueExA CloseServiceHandle QueryServiceConfigA QueryServiceStatusEx StartServiceA ADVAPI32.dll LocalAlloc RaiseException _except_handler3 222.dll DllCanUnloadNow DllGetClassObject Software\Microsoft\Windows\CurrentVersion\Run Software\Microsoft\Windows\CurrentVersion\Explorer\Browser Helper Objects CLSID SYSTEM\CurrentControlSet\Control\Lsa Notification Packages Software\Microsoft\Windows NT\CurrentVersion\Windows AppInit_DLLs LoadAppInit_DLLs Software\Microsoft\Internet Explorer\Main Check_Associations Software\Microsoft\Windows\CurrentVersion\Ext\Settings Software\Microsoft SYSTEM\CurrentControlSet\Control\Session Manager PendingFileRenameOperations PendingFileRenameOperations2 Software\Microsoft\Windows\CurrentVersion\Explorer\ShellExecuteHooks Software\Microsoft\Security Center UpdatesDisableNotify Software\Microsoft\Security Center\Svc EnableNotifications EnableNotifications\Ref Software\Microsoft\Windows NT\CurrentVersion DigitalProductId RegisteredOrganization RegisteredOwner C:\WINDOWS\system32\renobuda C:\WINDOWS\system32\calc.exe C:\WINDOWS\system32\defariha.dll C:\WINDOWS\system32\defariha.dll C:\WINDOWS\system32\dadeyisi.dll C:\WINDOWS\system32\vofehafi.dll {721ee819-b263-42e0-a594-b82fd0f24bdf} Global\vimegolatiturew Global\nifuseguji C:\WINDOWS\system32\mrt.exe own1 hdn_dsk .uroledup.com .uroledup.com .?AVCDownloader@@ .?AVCUrlStorage@@ .?AUIObjectWithSite@@ .?AVCConBHO@@ .?AUIUnknown@@ .?AUIClassFactory@@ .?AVCFactory@@

Update 2: Ok, I got rid of it. Turns out there’s no root-kit; the DLL was simply marked as hidden (I feel stupid…). Killing all the threads off, preventing it from re-loading and then re-installing the Service-Pack seems to have gotten rid of it for good.

Wealth does not pass three generations – A Note on the Stimulus Plan

Friday, February 20th, 2009

Lately I’ve gotten interested in the how the financial markets work, how we got into the current crisis and all that. And I learned about an interesting Chinese proverb: 富不过三代 (fu bu guo san dai) Literally: Wealth does not pass three generations. I’ve read articles that hypothesize that the economy is in a phase of “resetting”, i.e. we are at the end of the the Kondratieff Cycle. If this is true, then there’s probably not much we can do about it. In fact, it looks like the next week will be very interesting in the capital markets since we are close to a crash. Let’s hope for the best…
Just a quick comment about the stimulus plan the US government is trying to implement. The problem I see with the idea is that they will need to raise the money with new debt. Quite a lot of new debt. This could prove more difficult than they think (quotes from a treasury press release; emphasis mine):

But the ramp up in debt issuance remains in its early stages. As the US government and also foreign governments continue their efforts to stabilize their respective economies, the supply of government and quasi-government paper will grow rapidly. The sheer magnitude of paper set to be issued raises the possibility that investors at some point will demand a concession of some sort, lifting yields in parts of the term structure beyond those justified by macro fundamentals. As a country with a current account deficit and a majority of Treasury debt held abroad, the US is more at risk of such a development than a country such as Japan where the government bond market is primarily domestically held.

The expansion in quasi-government paper contributes to the risk of market saturation. Banks have issued nearly $150 billion in FDIC-backed paper since the programs introduction. Spreads on this paper have been narrowing over time with the latest deal, paper offered by Citi, pricing just 30 basis points over Libor. Real money investors have purchased the bulk of this paper in an attempt to pick up yield over Treasurys while not taking on additional credit risk. In some respects, this paper has replaced GSE debt as the instrument of choice for real money investors looking for modestly higher yielding, quasi-government debt.

China, on the other hand, could slow its accumulation of dollar-denominated debt. Such a trend already has begun to develop with respect to its accumulation of overall dollar assets as the flow of private capital into China has cooled alongside the global downturn, alleviating the need to offset capital inflows.

What that means is that the interest rate that the government might have to pay on the money could rise to levels making the whole idea prohibitively expensive. Or it could be simply impossible to find that much money on the sidelines that they can borrow (there’s a limited amount of money out there available for investing). Also they are going to remove money from the regular capital markets – what is more save right now than treasuries? I hope the government knows what they are doing…

Valentine’s day tango

Sunday, February 15th, 2009

EN TUS BRAZOS from et su on Vimeo.

Kids, Games and Sociopaths

Tuesday, December 2nd, 2008

In an interview of Professor Marc Bekoff on the radio he described some of the research on sociopathy and playing games. Research indicates that sociopaths don’t play and never learned how to play with other people. A lot of learning is done by playing, boundaries and rules are established and kids learn how to get along with other kids. And then you have a school ban the game of tag and other chase games and even a Virginia School banning all touching between kids. I wonder if the school thought about the consequences of these policies. The kids might turn out like the one in this story illustrating the negative effects of child fear mongering and overprotective parenting. What are kids in Virginia supposed to do? I guess we should have more video games in school then 🙂 (sidenode: see player quits World of Warcraft (WoW) as an extreme example for how excessive video games can mess up lives). Interesting times indeed…

Back from Conference of the American Society of Criminology (ASC 2008)

Friday, November 21st, 2008

I just got back from ASC 2008 (Conference of the American Society of Criminology). It’s the main conference for everything in criminology and has a wide international attendance. This was the first conference of this kind I attended and it was quite different from what I’m used to. There were more than 20 tracks – yep,20 talks going on at the same time. It’s impossible to pick and choose; the program was a book with a few hundred pages containing only titles and names (no abstracts) of the sessions and talks. Wow… But still way too many talks. I think the conference would be better if there would be a review process of the abstracts as some of the talks didn’t quite match the advertised title.

However, from the sessions I attended about two thirds of the presenters fail to show up. In one particular case I was interested in seeing a talk critical about an psychometric instrument I have worked with and the presenters bailed despite that we saw them in the morning in the conference hotel. That’s something I haven’t seen happen in computer science conferences at all. Some of the studies presented were a bit funny (small sample, no hold-out set etc.). Overall I got one new idea out of it that could turn out to be interesting: a diversity measure for static recidivism risk models.

Unfortunately St. Louis was a bit boring. It has pretty parks, but e.g. Tango dancing ends at 11pm (2am in Denver – at the earliest). Oh well…