“The future ain’t what it used to be.”

-Yogi Berra

  • 2 Posts
  • 398 Comments
Joined 2 years ago
Cake day: July 29th, 2023
  • TropicalDingdong@lemmy.worldtoFediverse@lemmy.world•Determining the reason no one replied to your Lemmy post.English
    4·
    13 days ago

    Well I appreciate the effort regardless. If you want any support in getting towards a more “proper” network analysis, I’ve dm’d you a link you can use to get started. If nothing else it might allow you to expand your scope or take your investigations into different directions. The script gets more into sentiment analysis for individual users, but since Lemmy lacks a basic API, the components could be retooled for anything.

    Also, you might consider that all a scientific paper is, at the end of the day, is a series of things like what you’ve started here, with perhaps a little more narrative glue, and the repetitive critique of a scientific inquiry. All scientific investigations start with exactly the kind of work you are presenting here. Then you PI comes in and says “No you’ve done this wrong and that wrong and cant say this or that. But this bit or that bit is interesting”, and you revise and repeat.

  • TropicalDingdong@lemmy.worldtoFediverse@lemmy.world•Determining the reason no one replied to your Lemmy post.English
    91·
    13 days ago

    So lets just cover a few things…

    Hypothesis testing:

    The phrase “if your post got less than 4 comments, that was statistically significant” can be misleading if we don’t clearly define what is being tested. When you perform a hypothesis test, you need to start by stating:

    Null hypothesis (H₀): For example, “the average number of comments per post is λ = 8.2.”

Alternative hypothesis (H₁): For example, “the average number of comments per post is different from 8.2” (or you could have a directional alternative if you have prior reasoning).

    Without a clearly defined H₀ and H₁, the statement about significance becomes ambiguous. The p-value (or “significance” level) tells you how unusual an observation is under the assumption that the null hypothesis is true. It doesn’t automatically imply that an external factor caused that observation. Plugging in numbers doesn’t supplant the interpretability issue.

    “Statistical significance”

    The interpretation that “there is a 95% probability that something else caused it not to get more comments” is a common misinterpretation of statistical significance. What the 5% significance level really means is that, under the null hypothesis, there is only a 5% chance of observing an outcome as extreme as (or more extreme than) the one you obtained. It is not a direct statement about the probability of an alternative cause. Saying “something else caused” can be confusing. It’s better to say, “if the observed comment count falls in the critical region, the observation would be very unlikely under the null hypothesis.”

    Critical regions

    Using critical regions based on the Poisson distribution can be useful to flag unusual observations. However, you need to be careful that the interpretation of those regions aligns with the hypothesis test framework. For instance, simply saying that fewer than 4 comments falls in the “critical region” implies that you reject the null when observing such counts, but it doesn’t explain what alternative hypothesis you’re leaning toward—high engagement versus low engagement isn’t inherently “good” or “bad” without further context. There are many, many reasons why a post might end up with a low count. Use the script I sent you previously and look at what happens after 5PM on a Friday in this place. A magnificent post at a wrong time versus a well timed adequate post? What is engagement actually telling us?

    Model Parameters and Hypothesis Testing

    It appears that you may have been focusing more on calculating the Poisson probabilities (i.e., the parameters of the Poisson distribution) rather than setting up and executing a complete hypothesis test. While the calculations help you understand the distribution, hypothesis testing requires you to formally test whether the data observed is consistent with the null hypothesis. Calculating “less than 4 comments” as a cutoff is a good start, but you might add a step that actually calculates the p-value for an observed comment count. This would give you a clearer measure of how “unusual” your observation is under your model.

  • TropicalDingdong@lemmy.worldtoFediverse@lemmy.world•Determining the reason no one replied to your Lemmy post.English
    141·
    13 days ago

    So I modeled that with a Poisson distribution, and I learnt that to a 5% significance level, if your post got less than 4 comments, that was statistically significant. Or in other words – there is a 95% probability that something else caused it not to get more comments. Now that could be because it is an AMAZING post – it covered all the points and no one has anything left to say. Or it’s because it’s a crappy post and you should be ashamed in yourself. Similarly a “good post”, one that gets lots of comments, would be any post that gets more than 13 comments. Anything in-between 4 and 13 is just an average post.

    So, like, I do have a background in stats and network analysis, and I’m not sure what you are trying to say here.

    if your post got less than 4 comments, that was statistically significant.

    Statistically significant what? What hypothesis are you testing? Like, how are you setting this question up? What is your null?

    Because I don’t believe your interpretation of that conclusion. It sounds like mostly you calculated the parameters of a poisson and then are interpreting them? Because to be clear, thats not the same as doing hypothesis testing and isn’t interpretable in that manner. Its still fine, and interesting, and especially useful when you are doing network analysis, but on its on, its not interpretable in this manner. It needs context and we need to understand what test you are running, and how you are setting that test up.

    I’m asking these questions not to dissuade you, but to give you the opportunity to bring rigor to your work.

    Should you like, to further your work, I have set up this notebook you can maybe use parts of to continue your investigations or do different investigations.