[Idea/Request] all-in-onel monitoring framework / plugin with plugeable modules $$

[znuffie]

[Idea/Request] all-in-one monitoring framework / plugin with plugeable modules $$

Hi,

I've been brainstorming this for a few days now, and I think this is just what HB needs.

The issue:

• We have many monitoring plugins that work just... half
• Most of current monitoring plugins have a single specific event to monitor and a single specific output event (IRC, sms, email, sound playing)

My solution consists of a Monitoring Framework, if I could say that, basically acts like a host for multiple monitoring modules and output modules.

So, what's a monitoring module? Consider it a plugin for a plugin, the basic ones I can think about are:

• chat module - /say, /guild monitoring (think: EspChatLog)
• whisper module - /whisper monitoring (think: EspWhispAlert)
• leveling module - when you reach X level
• pvp - you win a bg, enter a bg, whatever pvp related
• loot module - watch out for X item to drop *OR* monitor all looted (optionally above X quality)
• following module - detect when someone follows you more than X seconds/minutes (think: eSafe)
• [more ideas are welcomed]

Now, this module should watch for all of those events I described above.

The 2nd part of this plugin consists of output modules (or, if you'd like, action modules - do something when a monitoring module detects something)

This should/could include the following:

• IRC module - connect to an IRC server and joins specified X channel and outputs all the data there (something along the lines of jIRC)
• WebServer - something along the lines of CowStats, but most likely simpler to implement
• Jabber/MSN/ICQ/Yahoo Messenger - same as the IRC module, but different protocols, preferably each protocol a single module, less work that way
• Exit - a simple exit module that closes your game when something happens (exit when followed, when you get a whisper etc.)
• Sound - same as the exit module, but instead play a sound
• SMS - Think KingSms
• E-Mail - Get emails when something happens! (this could replace KingSms in some cases where email2sms is available for different carriers)
• WebApi - Something along the lines how eStat works - requesting a certain URL each time an event occurs
• Database - write stuff to an Sqlite
• ...and more

Tried to use my flowchart skills, but I suck at it:


The combined power of such a plugin is amazing! Think about that you could pretty much do all sorts of combinations with these options.

Most monitoring plugins that exist already could be ported to this framework and it would turn out that it's even easier to work on them when you don't have to worry about input and output data when writing them!

As I know that this is a pretty complex stuff to do, I'm gonna set a bounty of $50 to the person who builds the main plugin plus 2 input modules (whisper & loot) and one output module (either jabber or irc). (I'm gonna pay by PayPal, preferably)

TLDR Version: If you're not a coder and you don't understand what I'm talking about and you wouldn't be willing to add more $ to the bounty, this stuff is not for you.

 

[cowdude]

Well, I'll add a few lines to your explanations.

1. HB2 chat listeners are halfway broken right now. You need to rely a lot on LUA code, which is NOT stable at all. In my case, that's why CowStats is crashing: it prevents HB to peacefully read WoW's memory.

2. You're talking about databases, which sounds rather nice as these are nice toys for any dev. I still disagree with you about centralizing data on a single/few DB servers. Looks like trouble are coming unless the dev team let you use a server for this.

3. The most of what you're asking is already done in CowStats. Edit it yourself or simply use its JSON output to feed a DB or anything you like.

4. I hope you're not looking for a real dev to code you this for 50usd.This represents a lot of work (even if you seal code from this forum.......) and you better love C# to mess up with such a project.

5. Keep in mind that HB evolves a lot each month, including its plugin API (and bugs). Hence you should, and I mean you must keep your code clean and adaptable for each new upcoming version.

And well, a small 6th point: I'm still somewhat coding CowStats until some methods and events get fixed. The actual "nightly build" already includes socket connections with a simple protocol to send stats and stuff to another remote server, which could dispatch it as you wish (HTTP or anything else).

Good night. : o)

 

[znuffie]

Thanks for your input, cowdude

About point #2: that's for each person to use, I guess, it's not a requirement, was just an idea thrown there, there could be 3rd party providers of services for this and it doesn't sound too bad

#4: I realise $50 is not too much, but look at it another way, you're doing a hell lot of work on CowStats for free And it's an amazing plugin!

#5: I realise and I've considered HB's quirks - that's why this could actually serve as an "abstraction" model. Output plugins (irc, database, web services, email etc.) wouldn't have to worry about interacting directly with HB2, but with the "host-plugin" I'm talking about.

 

[cowdude]

Thanks for your input, cowdude

About point #2: that's for each person to use, I guess, it's not a requirement, was just an idea thrown there, there could be 3rd party providers of services for this and it doesn't sound too bad

#4: I realise $50 is not too much, but look at it another way, you're doing a hell lot of work on CowStats for free And it's an amazing plugin!

#5: I realise and I've considered HB's quirks - that's why this could actually serve as an "abstraction" model. Output plugins (irc, database, web services, email etc.) wouldn't have to worry about interacting directly with HB2, but with the "host-plugin" I'm talking about.

Yep, using a standalone app would be nice. That would require a lot of work with other monitoring plugins' devs.

If someone want to help a bit, I'm still looking for a nice-and-easy-working-out-of-the-box-"awesomesauce" protocol to transmit data.

I personally HATE xml sockets (thank you MSN). C# object serializing is too much closed and heavy to fit this task either. I thought about keeping a JSON serializer as you can find APIs to read this on many, many languages. The last alt solution I see would be a plain text protocol like IRC.

 

[znuffie]

Was just discussing this with someone else in the chat, and we kind of agree that IRC would be the simplest way (and less painful) to do it. It's easy to parse, and and's a really simple protocol (altough, it would be nice to actually use ssl, and that kind of makes it a bit harder)

 

[king]

It certainly is do-able with a stable release. I haven't played around with the chat handlers lately but I had a plugin that let you reply with a text. Also all other things can be done writing out sets to a text and have a separate program running the show.

 

[babyxor]

some kind of an api like glider had would be awesome. it was really easy to use and coding applications like webinterface's and so on was very easy ...

 

[SkyHigh]

Would love to see this realized.
The webapi and the msn support seem particularly interesting.
I also think communication between the modules should be a two way thing.
Throwing stuff in the chatbox should certainly be a requirement.
Some basic stuff like "hearthAndLogof" would be interesting too.
Just throwing ideas out there.

 

[cowdude]

Alright, I tweaked cowstats a bit, deleted html-related stuff. It now opens a server socket on given port, password protected (no, you won't find it on the forums tonight).

Allows you to get the same stats and stuff than the previous CowStats, including the screenshot.

Current implementation uses regular ASCII (unix) input and outputs either JSON or base64 strings for binary data (like the screenshot).

I personally prefer working with serialized objects than parsing awful and meaningless raw text strings.

About security: SSL is a must, I need to know how to enable it with tcp sockets in C#. But still, you'll need to send your password as a SHA1 hash.

 

[z0mg]

cowdude,

I don't really use the buddies so I never toke a look at your plugin myself, but can't you just run some (My)SQL statements on a X-timed-event and FTP for any images/screenshots and then just have it online?

 

[cowdude]

cowdude,

I don't really use the buddies so I never toke a look at your plugin myself, but can't you just run some (My)SQL statements on a X-timed-event and FTP for any images/screenshots and then just have it online?

No. You can still write an app that populates a DB and saves screenshots while reading json output. I assume you're a php guy. In that case, remember you can open sockets and easily parse json in php. Znuffie did it really easily with his wowhead quest/items reader. See www.json.org

 

[znuffie]

z0mg: as far as I know, you can't do sql inside hb, if that's what you're thinking about. Apoc requested the posibility of using external dlls in plugins, but we'll wait and see

 

[cowdude]

        private struct Charts
        {
            public string xpHour;
            public string honorHour;
            public string goldHour;
        }
        private struct Counters
        {
            public uint kills;
            public uint loots;
            public uint deaths;
            public float killsHour;
            public float lootsHour;
            public float deathsHour;

            public uint honor;
            public float honorHour;
            public double gold; //in gold, fraction
            public float goldHour; //same
        }
        private struct Map
        {
            public uint zoneId;
            public string zone;
            public double u, v; //ingame coords, bound between 0 and 1
            public float x, y, z; //abs world coords
        }
        private struct XP
        {
            public int level;
            public uint xp; //current xp done for this lvl
            public uint xpLevel; //total xp for this lvl
            public float xpHour;
            public uint timeToLevel; //seconds
        }
        private struct Stats
        {
            public string name;
            public int level;
            public string playerClass;
            public float durability;
        }
        private struct Bot
        {
            public string state;
            public uint uptime; //seconds
        }

Also includes screenshot, bot log, chat log, loots table and stats, charts URL (google charts)

Logs are not real-time at the moment, which means the server won't notify the client whenever HB catches a chat message or logs something.
Why? I want flow control on the socket server output. Therefore a client can ask "give me the latest log entries since 3 minutes ago". If there are way too much entries to submit, he'll only receive at most Nmax entries.
This also allow you to know what happened before you got connected to the server, as it logs everything instead of stupidly dispatching log entries in real time.
I know that's more greedy than using events but let's keep things simple.

Anything else needed?

 

[moujoohoo]

hope cowdude will update plugin soon loved it

 

[znuffie]

Actually I do think that real-time is better, with an UDP socket listener (more easy to do) or a TCP listener (more reliable, slower).

Then in a webui you could fetch the latest updates with ajax.

This doesn't really fit the initial idea I had, though, or at least I can't see how all this fits together

 

[cowdude]

Actually I do think that real-time is better, with an UDP socket listener (more easy to do) or a TCP listener (more reliable, slower).

Then in a webui you could fetch the latest updates with ajax.

This doesn't really fit the initial idea I had, though, or at least I can't see how all this fits together

I'm glad to know you have basic knowledge of network protocols. Try not to mix everything together about it, might be better.

Last point: this intend to be useful in monitoring, which means you don't especially need fastest possible speed. Real-time monitoring crashes HB and I don't intend to add video streaming nor WoW remote control.

 

[Apoc]

I'm glad to know you have basic knowledge of network protocols. Try not to mix everything together about it, might be better.

Last point: this intend to be useful in monitoring, which means you don't especially need fastest possible speed. Real-time monitoring crashes HB and I don't intend to add video streaming nor WoW remote control.

Real-time monitoring doesn't crash HB at all. Your code most likely does. (Easiest way to fix it; is to queue things up, and handle them in a separate thread. Only doing WoW related things in the Pulse() func that HB calls itself.)

Also; a generic framework like the one the OP wants is fairly simple. I may lay down some POC code later to show how it could be done.

 

[cowdude]

Real-time monitoring doesn't crash HB at all. Your code most likely does. (Easiest way to fix it; is to queue things up, and handle them in a separate thread. Only doing WoW related things in the Pulse() func that HB calls itself.)

Also; a generic framework like the one the OP wants is fairly simple. I may lay down some POC code later to show how it could be done.

I know what I'm talking about Apoc. Chances to crash HB are near to 100% if you spam LUA calls from HB. I guess that's a try/catch missing on HB. The best workaround is to use a cache.

You can try it for yourself: spam 10000 times a dummy LUA call inside Pulse. You'll get an exception quite quickly (readuintXX at 0x8 failed). Even 5-10 times is fine but would require more time before the exception appears.

 

[Apoc]

I know what I'm talking about Apoc. Chances to crash HB are near to 100% if you spam LUA calls from HB. I guess that's a try/catch missing on HB. The best workaround is to use a cache.

You can try it for yourself: spam 10000 times a dummy LUA call inside Pulse. You'll get an exception quite quickly (readuintXX at 0x8 failed). Even 5-10 times is fine but would require more time before the exception appears.

I don't know what you're possibly monitoring that requires that amount of Lua calls. Also; it won't crash HB, it will simply throw exceptions due to their DirectX hook going out of synch. As for the ReadUInt16/32/64, that's due to *someone* not making sure they're using valid objects. (You can't read values from address 0x000000)

 

[cowdude]

I don't know what you're possibly monitoring that requires that amount of Lua calls. Also; it won't crash HB, it will simply throw exceptions due to their DirectX hook going out of synch. As for the ReadUInt16/32/64, that's due to *someone* not making sure they're using valid objects. (You can't read values from address 0x000000)

Yeah, that's what I call a "crash". It throws the error reporting box and bybye if you close it. That exception cannot be caught inside a plugin.

What need such amount of lua calls? I'll give you two awful cases:
- loot tracking, as objectmanager returns all kind of wow items. You need to check which ones are in your bags
- get ingame map coordinates

Both could easily be done without LUA injection, but it's not currently implemented. Too bad.